On Fri, 30 Sep 2011, Christopher Morrow wrote:

If you do nothing the default behavior is to send the packet to the RP... why? (why would you want this packet sent to the RP? it's got a valid destination, no? so deliver it out the egress interface?)

I was told it's because PFC3B can't look into the packet far enough to determine what the payload is (TCP/UDP etc) and port, that's only the RP that can do ACL handling of the packet. So if you configure "forward", people can put a fragmentation header on the packet and skip past your ACL. -- Mikael Abrahamsson email: swmike@swm.pp.se

On Fri, Sep 30, 2011 at 6:02 AM, Saku Ytti <saku@ytti.fi> wrote:

On (2011-09-30 01:55 -0400), Christopher Morrow wrote:

...

when will vendors learn that punting to the RE/RP/smarts for packets in the fastpath is ... not just 'unwise' but wholesale stupid? :(

What to do with IP options or IPv6 hop-by-hop options? What to do with IPv6 packets which contain options which push TCP/UDP past your lookup view?

a switch to be used that stops processing this sort of thing, in an internet core (and honestly most enterprise core) routers, all I want is packet-in/packet-out. there's no need for anything else, stop trying to send line-rate packets to the cpu.

Punting transit is not only not stupid but also necessary in hardware routers which cannot handle every case in hardware (which is all routers).

no. all you need is a default 'do not process these, just fwd them' switch. (or, a switch at any rate that the operator can select one way or the other, they SHOULD know what is the best for their deployment).

There should just be adequate way to limit these and there should exist default limitation.

I really think zero limit is the right limit... (for a large number of deployments)

On Fri, Sep 30, 2011 at 10:26 AM, Saku Ytti <saku@ytti.fi> wrote:

explained. And probably issues I'm not aware of. Unsure if blind forwarding is best option. But I'm all for giving operator options, but calling it stupid that vendors punt something is misguided.

after this long, yes... this is just dumb, there's no reason that the default should be punt. There are cases (you've brought up a few) where it's required today because of design limitations, there really shouldn't be cases like this anymore. this isn't our first rodeo, 'lessons learned' and all that...

...

I really think zero limit is the right limit... (for a large number of deployments)

Traceroute would also break. Unpoliced punting certainly is extremely unwise,

traceroute could certainly be handled in the fastpath.

but punting to a level that does not introduce significant CPU load, should be safest default.

what is that limit? from a single port? from a single linecard? from a chassis? how about we remove complexity here and just deal with this in the fastpath? My point in calling this all 'stupid' is that by now we all have been burned by this sort of behavior, vendors have heard from all of us that 'this is really not a good answer', enough is enough please stop doing this. -chris

On 30/09/2011 15:45, Christopher Morrow wrote:

traceroute could certainly be handled in the fastpath.

which traceroute? icmp? udp? tcp? Traceroute is not a single protocol.

what is that limit? from a single port? from a single linecard? from a chassis? how about we remove complexity here and just deal with this in the fastpath?

on a pfc3, the mls rate limiters deal with handling all punts from the chassis to the RP. It's difficult to handle this in any other way.

My point in calling this all 'stupid' is that by now we all have been burned by this sort of behavior, vendors have heard from all of us that 'this is really not a good answer', enough is enough please stop doing this.

"This is a Hard Problem". There is a balance to be drawn between hardware complexity, cost and lifecycle. In the case of the PFC3, we're talking about hardware which was released in 2000 - 11 years ago. The ipv6 fragment punting problem was fixed in the pfc3c, which was released in 2003. I'm aware that cisco is still selling the pfc3b, but they really only push the rsp720 for internet stuff (if they're pushing the 6500/7600 line at all). Nick

On 30/09/2011 16:38, Mohacsi Janos wrote:

They are pushing sup2T - however more for enterprise ip layer (6500 series).

they are now, yes. But until the sup2t started becoming available a couple of weeks ago the only option for the 6500 was a sup720. You're right that this was only pushed on the enterprise market. Of course, if you wanted a 10g capable service provider router and didn't want an asr9k, they were pushing the 7600 because the 6500 is a switch and the 7600 is a router and the two are totally different, no really you've gotta believe it. But at least the rsp720 could handle ipv6 fragments better. Nick

On Fri, Sep 30, 2011 at 11:24 AM, Nick Hilliard <nick@foobar.org> wrote:

On 30/09/2011 15:45, Christopher Morrow wrote:

...

traceroute could certainly be handled in the fastpath.

which traceroute?  icmp?  udp?  tcp?  Traceroute is not a single protocol.

traceroute is really an example of 'packet expired, send unreachable'... that, today is basically: o grab 64bytes of header (or something similar) o shove that in a payload o use the src as the dst o stick my src on o set icmp o crc and fire there's not really any need to do this in the slow path, is there? -chris

On Fri, Sep 30, 2011 at 12:38 PM, Nick Hilliard <nick@foobar.org> wrote:

On 30/09/2011 17:30, Christopher Morrow wrote:

...

traceroute is really an example of 'packet expired, send unreachable'... that, today is basically:   o grab 64bytes of header (or something similar)   o shove that in a payload   o use the src as the dst   o stick my src on   o set icmp   o crc and fire

there's not really any need to do this in the slow path, is there?

there are unconfirmed rumours that icmp ping and traceroute are handled by hardware on the asr1k.  I don't know if they are true.   But you're right -

some platforms do some/all of this in hardware, yes. (I forget the matrix)

it would be good to support this without resorting to hammering the routing engine.  I don't really like the idea of punters running traceroutes reducing my bgp convergence time.

this is exactly why punting anything NOT management and/or routing-protocols should be banned. Thanks for making that point explicitly. -chris

On Fri, Sep 30, 2011 at 9:32 PM, Dobbins, Roland <rdobbins@arbor.net> wrote:

On Sep 30, 2011, at 11:44 PM, Christopher Morrow wrote:

...

this is exactly why punting anything NOT management and/or routing-protocols should be banned. Thanks for making that point explicitly.

And this is the requirement which should be placed in RFPs, along with other specific requirements for ACL handling, flow telemetry functionality, uRPF, et. al.

If folks want to influence vendors to do the Right Thing, they have to expend the time and effort to quantify and qualify said Right Thing(s), and then put it into RFP requirements.  Otherwise, complaining post-procurement isn't generally going to accomplish much.

yes, my bitchfest was also a 'could we all start asking for this, now?' ... :)

which traceroute? icmp? udp? tcp? Traceroute is not a single protocol.

Router processing is only dependent on noticing that TTL is expiring, and being able to return an ICMP message (including a "quote" of part of the original packet) to the sender.

...

what is that limit? from a single port? from a single linecard? from a chassis? how about we remove complexity here and just deal with this in the fastpath?

on a pfc3, the mls rate limiters deal with handling all punts from the chassis to the RP. It's difficult to handle this in any other way.

If the rate limit is done "in hardware" (which one should hope), then it would be more natural to do it on a per-PFC/DFC basis. So on a box with DFCs on all linecards, it would be per linecard, not per chassis. Maybe someone who knows for sure can decide.

...

My point in calling this all 'stupid' is that by now we all have been burned by this sort of behavior, vendors have heard from all of us that 'this is really not a good answer', enough is enough please stop doing this.

"This is a Hard Problem". There is a balance to be drawn between hardware complexity, cost and lifecycle. In the case of the PFC3, we're talking about hardware which was released in 2000 - 11 years ago.

Um, no, in 2000 there was no PFC3. That came out (on the Supervisor 720) in March 2003.

The ipv6 fragment punting problem was fixed in the pfc3c, which was released in 2003.

The PFC 3C was announced (with the RSP720) in December 2006.

I'm aware that cisco is still selling the pfc3b, but they really only push the rsp720 for internet stuff (if they're pushing the 6500/7600 line at all).

See Janos' reply, the Catalyst 6500 seems alive and kicking with the Supervisor 2T. The 7600 is a somewhat different story. As far as I see, all development is going into feature-rich ES+ cards and a few relatively narrow applications such as mobile backhaul and FTTH aggregation(?). We have been using the 7600 as a cheap fast IPv4/IPv6 (and later also MPLS) backbone router. According to Cisco we should probably move "up" to the ASR9000 or CRS-3, but I'm tempted to "downgrade" to Catalyst 6500 with Sup-2T (until we need 100G :-). -- Simon.

Path MTU discovery would also break... oh wait, that's usually broken anyway. -Vinny -----Original Message----- From: Saku Ytti [mailto:saku@ytti.fi] Sent: Friday, September 30, 2011 10:27 AM To: nanog@nanog.org Subject: Re: Cisco 7600 PFC3B(XL) and IPv6 packets with fragmentation header On (2011-09-30 10:09 -0400), Christopher Morrow wrote:

a switch to be used that stops processing this sort of thing, in an internet core (and honestly most enterprise core) routers, all I want is packet-in/packet-out. there's no need for anything else, stop trying to send line-rate packets to the cpu.

This would break e.g. RSVP. For some instances dropping all of them in hardware is an option, for other instances ignoring and forwarding without understanding is ok but some situation you simply must punt.

no. all you need is a default 'do not process these, just fwd them' switch. (or, a switch at any rate that the operator can select one way or the other, they SHOULD know what is the best for their deployment).

It would also break L4 ACL under certain situations, as well as RSVP as already explained. And probably issues I'm not aware of. Unsure if blind forwarding is best option. But I'm all for giving operator options, but calling it stupid that vendors punt something is misguided.

I really think zero limit is the right limit... (for a large number of deployments)

Traceroute would also break. Unpoliced punting certainly is extremely unwise, but punting to a level that does not introduce significant CPU load, should be safest default. -- ++ytti