On Jul 18, 2009, at 2:37 PM, Saku Ytti wrote:

I'm guessing point Roland was making (which he likely would have not made couple moons ago:)

I've made this point for years, quite publicly, actually - even when it was unpopular for me to do so in certain quarters. ;> uRPF for 7600/6500 can only be in one mode for the whole box, all interfaces. This is a major problem in many cases. The NetFlow issues render flow telemetry unusable in production situations. The ACLs work very differently on this platform due to LOU issues, as you say. Most folks don't know this, and many end up overflowing their TCAMs and not realizing it until their boxes fall over, heh. If one has fairly complex ACLs covering various ranges of ports, ACLs on 7600/6500 quickly become very difficult to manage.

EARL8 (Nexus7k) fixes the IPv6/uRPF and IPv6/ACL issue.

And the NetFlow issues. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Unfortunately, inefficiency scales really well. -- Kevin Lawton

On Jul 18, 2009, at 5:05 PM, Darren Bolding wrote:

Can someone provide a link, or more detail, on the netflow issues. Particularly as they relate to 6509's and sup720's.

mls table can hold 256K entries at 93% efficiency, so you end up with about 239K flows total. No packet-sampled control of flow creation, so the table is likely to be overflowed in production edge situations, leading to non-deterministically skewed stats. No logical OR of TCP flags throughout a TCP flow - can't classify SYN- floods, RST-floods, et. al. No stats on dropped traffic. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Unfortunately, inefficiency scales really well. -- Kevin Lawton

On Sat, Jul 18, 2009 at 03:05:32AM -0700, Darren Bolding wrote:

Can someone provide a link, or more detail, on the netflow issues. Particularly as they relate to 6509's and sup720's.

The long and short of it is the current hardware (EARL7) is incapable of doing sampling (i.e. looking at 1 out of every Nth packets). It gathers all of the flow data into tcam and THEN does sampling in software, but by that point its already too late because the tcam is exhausted. Turning on sampling actually makes it worse, because it forces a flowmask which fills the tcam even faster. In my experience, even with extremely aggressive aging and a dest only flowmask (discarding all src and L4 port information to make it fit better), it tops out at around 2Gbps of "generic wholesale IP" traffic you can sample. Obviously when it runs out of steam is dependent on the number of flows in your network, you could be much better or much worse depending on your traffic, but the point is it usually doesn't work for most people. Adding DFC daughterboards makes this capacity scale linearly, i.e. you go from 2Gbps system-wide capacity to 2Gbps per slot capacity, but this typically doesn't make any difference. The only recent improvement is that in SXH+ and SRB+ software you can now enable netflow on a per-interface basis rather than a global basis (before this, all traffic was sampled globally regardless of what you configured on the interfaces). This can let you exclude interfaces you don't care about (such as core links) use your limited resources only on interfaces you do care about (such as edge links). Until they come out with the EARL8 SUPs (what have they pushed that back to now, 2011? :P) you are basically SOL in the netflow dept. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)