From: Tony Hain [mailto:alh-ietf@tndh.net] Sent: Thursday, April 26, 2001 11:47 AM
April 26, 2001 9:29 AM John Fraizer wrote:
We only have jumbo frames enabled on router<->router links. The GigE ports facing the aggregation switches runs standard 1500 MTU.
Hence my original question. Packets across the GE will be 1500 unless you are packing them.
April 25, 2001 8:10 PM John Fraizer wrote:
Partially because I can. Partially because there seems to be a performance increase when you start stuffing the pipe.
Assuming you are just passing the packets as received from the aggregation switch, this would only happen if your router hardware was better at managing jumbo buffer allocations than 1500B ones. Clearly it will waste large chunks of memory, so do you have measurements to show the actual performance increase?
This depends on the type of traffic. We use it to enhance performance of the data tier. We've jiggered the TCP/IP stacks for ~4500 byte packets and have knee-capped the slow-start algorithm (which doesn't make sense in a pure switched environment anyway). What we then wind up with, is a dedicated data LAN between our application servers and the Oracle database servers. It comes out to about an order of magnitude increase in performance and SQL query responsiveness. At first we went to jumbo packets. We saw such a radical improvement that we started investigating and found the slow-start issue. Jumbo packets are one way around the slow-start problem if you can't jigger the stack itself. Most of the queries are reasonably short so we saw some serious improvements by killing the slow-start. If you can modify your IP stack then it still pays to use jumbo packets because you reduce the overhead on the wire. We got sufficient performance improvement that we were able to defer GigE implementation, at some sites. Those sites are using switched FDX 100baseTX.
Roeland you are talking about jumbo frames from the end system lan, while John is talking about only using the jumbo frames between the routers. My point was that in John's environment the packets will all be 1500 since the packets are restricted to that size just to get to the router with the GE interface. I understand that there are perf gains as long as the entire path supports the larger packets, but I don't understand the claim that having a bigger pipe in the middle helps. Tony -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Roeland Meyer Sent: Thursday, April 26, 2001 12:13 PM To: 'alh-ietf@tndh.net'; John Fraizer; Paul Lantinga Cc: nanog@merit.edu Subject: RE: jumbo frames
From: Tony Hain [mailto:alh-ietf@tndh.net] Sent: Thursday, April 26, 2001 11:47 AM
April 26, 2001 9:29 AM John Fraizer wrote:
We only have jumbo frames enabled on router<->router links. The GigE ports facing the aggregation switches runs standard 1500 MTU.
Hence my original question. Packets across the GE will be 1500 unless you are packing them.
April 25, 2001 8:10 PM John Fraizer wrote:
Partially because I can. Partially because there seems to be a performance increase when you start stuffing the pipe.
Assuming you are just passing the packets as received from the aggregation switch, this would only happen if your router hardware was better at managing jumbo buffer allocations than 1500B ones. Clearly it will waste large chunks of memory, so do you have measurements to show the actual performance increase?
This depends on the type of traffic. We use it to enhance performance of the data tier. We've jiggered the TCP/IP stacks for ~4500 byte packets and have knee-capped the slow-start algorithm (which doesn't make sense in a pure switched environment anyway). What we then wind up with, is a dedicated data LAN between our application servers and the Oracle database servers. It comes out to about an order of magnitude increase in performance and SQL query responsiveness. At first we went to jumbo packets. We saw such a radical improvement that we started investigating and found the slow-start issue. Jumbo packets are one way around the slow-start problem if you can't jigger the stack itself. Most of the queries are reasonably short so we saw some serious improvements by killing the slow-start. If you can modify your IP stack then it still pays to use jumbo packets because you reduce the overhead on the wire. We got sufficient performance improvement that we were able to defer GigE implementation, at some sites. Those sites are using switched FDX 100baseTX.
Hi, Isn't it a lot more cpu-intensive to 'collect' some 1500-byte frames into some larger bucket, reassemble it into a jumbo-frame when the next box has to chop it in order to send it out on a Sonet/PPP/etc interface which might have a smaller MTU again? Doesn't make too much sense to me. I guess that was Tony's aim as well.. Kurt
Roeland you are talking about jumbo frames from the end system lan, while John is talking about only using the jumbo frames between the routers. My point was that in John's environment the packets will all be 1500 since the packets are restricted to that size just to get to the router with the GE interface. I understand that there are perf gains as long as the entire path supports the larger packets, but I don't understand the claim that having a bigger pipe in the middle helps.
Tony
-- noris network AG * tel +49 911 93 52-0 * internet Kilianstraße 142 * fax +49 911 93 52-100 * solution 90425 Nürnberg * http://www.noris.net * provider
On Fri, 27 Apr 2001, Kurt Kayser wrote:
Hi,
Isn't it a lot more cpu-intensive to 'collect' some 1500-byte frames into some larger bucket, reassemble it into a jumbo-frame when the next box has to chop it in order to send it out on a Sonet/PPP/etc interface which might have a smaller MTU again?
Doesn't make too much sense to me. I guess that was Tony's aim as well..
Kurt
Roeland you are talking about jumbo frames from the end system lan, while John is talking about only using the jumbo frames between the routers. My point was that in John's environment the packets will all be 1500 since the packets are restricted to that size just to get to the router with the GE interface. I understand that there are perf gains as long as the entire path supports the larger packets, but I don't understand the claim that having a bigger pipe in the middle helps.
I dont think that anyone discussed doing that... What was being said was that it makes sence to use jumbo frames between routers when they are encapsulating packets from links with a 1500b mtu, so you don't have to reduce your MTU to 1450 or fragment, i.e. endnode-ether-router>tunnel-jumbo_ether-router-jumbo-ether-tunnel>router-eth-end
Greg Maxwell wrote:
that it makes sence to use jumbo frames between routers when they are encapsulating packets from links with a 1500b mtu, so you don't have to reduce your MTU to 1450 or fragment, i.e. endnode-ether-router>tunnel-jumbo_ether-router-jumbo-ether-tunnel>router-et h-end
Thank you. This explains why jumbo frames make sense. Without the context that there are tunnels, statements about performance gains appeared to be subjective. Tony
participants (4)
-
Greg Maxwell -
Kurt Kayser -
Roeland Meyer -
Tony Hain