Thus spake Tony Hain:
Hence my original question. Packets across the GE will be 1500 unless you are packing them.
Or unless you actually have >1500 MTU to the hosts, which is quite possible. A traffic study from MCI's backbone (obviously years ago) showed nearly 40% (byte-wise) of their traffic was in packets >1500 bytes. With the death of FDDI, this has probably come down, but GE-attached servers in colos should push it back up.
Assuming you are just passing the packets as received from the aggregation switch, this would only happen if your router hardware was better at managing jumbo buffer allocations than 1500B ones. Clearly it will waste large chunks of memory, so do you have measurements to show the actual performance increase?
Routers usually have separate buffer pools for common packet sizes (or use buffer vectors), so the MTU of the interface does not noticeably affect memory usage. Router performance is, however, directly related to packet size, since forwarding overhead is per-packet and not per-byte. It is much easier to fill big pipes with 9000 byte packets than 1500 byte packets.
Tony
S -- Stephen Sprunk "So long as they don't get violent, I want to CCIE #3723 let everyone say what they wish, for I myself have K5SSS always said exactly what pleased me." --Albert Einstein