On Tue, Sep 15, 2009 at 05:39:33PM -0300, Brian Dickson wrote:
And more specifically, possibly an interface MTU (or ip mtu, I forget which).
If there is a mismatch between ends of a link, in one direction, MTU-sized packets get sent, and the other end sees those as "giants".
Well if the interface or ip mtu was smaller on one end, this would result in a lower mss negotiation and you would just have smaller but working packets. The bad situation is when there is a layer 2 device in the middle which eats the big packets and doesn't generate an ICMP needfrag. For example, if there was a 1500-byte only ethernet switch in the middle of this link, it would drop anything > 1500 bytes and prevent path mtu discovery from working, resulting in silent blackholing. I was assuming that wasn't the case here based on the 4474 mtu (was assuming sonet links or something), but looking at the original message he doesn't say what media or what might be in the middle, so its possible 4474 is a manually configured mtu causing blackholing.
I've seen situations where the MTU is calculated incorrectly, when using some technology that adds a few bytes (e.g. VLAN tags, MPLS tags, etc.).
Even when things are working as intended, different vendors mean different things when they talk about MTU. For example, Juniper and Cisco disagree as to whether the mtu should include layer 2 or .1q tag overhead, resuling in inconsistent MTU numbers which are not only different between the vendors, but which can change depending on what type of trunk you're running between the devices. Enabling > 1500 byte MTUs is a dangerous game if you don't know what you're doing, or if you're connected to other people who are sloppy and don't fully verify their MTU settings on every link.
War stories make for great bar BOFs at NANOG meetings. :-)
Never ending supply of those things. :) -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)