On Wed, 01 Sep 2010 23:18:55 +0200 Simon Leinen <simon.leinen@switch.ch> wrote:
Jack Bates writes:
1) Your originating host may be breaking PMTU (so the packet you send is too large and doesn't make it, you never resend a smaller packet, but it works when tracerouting from the other side due to PMTU working in that direction and you are responding with the same size packet).
Your mentioning PMTU discovery issues in connection with 6to4 prompts me to confess how our open 6to4 relay has probably contributed to the perception of brokenness of 6to4 for quite a while *blush*.
The relay runs on a Cisco 7600 with PFC3 - btw. this is an excellent platform to run an 6to4 relay on, because it can do the encap/decap in hardware if configured correctly.
At some point of the relay becoming popular (load currently fluctuates between 80 Mb/s and 200 Mb/s), I noticed that our router very often failed to send ICMPv6 messages such as "packet too big".
That potentially starts to explain why I haven't noticed PMTUD issues on my 6to4 tunnel that I've been running for a number of years, and have been quite surprised be and somewhat doubtful of people to say there are issues. I've pumped the MTU up to 1472 on it to suit my PPPoE 1492 MTU, instead of leaving it at 1280, and haven't had any issues that have made me suspect PMTUD. I'm in Asia Pacific so I've probably either been using 6to4 gateways that are lightly used, or ones that have had this parameter changed.
First I suspected our control-plane rate-limit (CoPP) configuration, but couldn't find anything there.
Finally I found that I had to configure a generous "ipv6 icmp error-interval"[1], because the (invisible) default configuration will only permit one such ICMPv6 message to be generated every 100 milliseconds, and that's WAY insufficient for a popular router. We currently use
ipv6 icmp error-interval 2 100
(max. steady state rate 500 ICMPv6s/second - one every 2 milliseonds - with bursts up to 100) with no ill effects.
Note that the same rate-limit will also cause stars in IPv6 traceroutes through popular routers if the default setting is used.
The issue is probably not restricted to Cisco, as the ICMPv6 standard (RFC 4443) mandates that ICMPv6 error messages be rate limited. It even has good (if hand-wavy) guidance on how to arrive at defaults - the values used on our Cisco 7600 (and possibly all other IOS devices?) correspond to the RFC's suggestion for "a small/mid-size device" *hrmpf* (yes Randy, I know I should get real routers :-).
Anybody knows which defaults are used by other devices/vendors?
In general, rate limits are very useful for protecting routers' notoriously underpowered control planes, but (1) it's hard to come up with reasonable defaults, and (2) I suspect that not most people don't monitor them (because that's often hard), and thus won't notice when normal traffic levels trip these limits. -- Simon. [1] See http://www.cisco.com/en/US/docs/ios/ipv6/command/reference/ipv6_06.html#wp21...