Have a look at http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note0918... // Olof On Sun, Mar 15, 2009 at 3:20 AM, Andy Bierlair <globichen@gmail.com> wrote:

yes ip cef, this is enabled:

 IP fast switching is enabled  IP fast switching on the same interface is disabled  IP Flow switching is enabled  IP CEF switching is enabled  IP Flow switching turbo vector  IP Flow CEF switching turbo vector

and so on...

- Andy

On Sun, Mar 15, 2009 at 3:08 AM, Bill Blackford <BBlackford@nwresd.k12.or.us> wrote:

...

just a shot in the dark. Do you have 'ip cef' in global config?

-b ________________________________________ From: Andy Bierlair [globichen@gmail.com] Sent: Saturday, March 14, 2009 6:55 PM To: nanog@nanog.org Subject: Netflow on SUP720-3BXL

I’m trying to run netflow on one of our Cisco core routers (SUP720-3BXL), but I think I am hitting some limitations because of this:

%EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold exceeded, TCAM Utilization [99%]

The setup of netflow looks like this:

 ip flow-cache entries 524288

 mls aging fast time 5 threshold 32

 mls aging long 300

 mls aging normal 60

 mls netflow usage notify 80 300

 mls flow ip full

 no mls flow ipv6

 mls nde sender version 5

 no mls verify ip checksum

 no mls acl tcam share-global

 ip flow-export source Loopback0

 ip flow-export version 5 origin-as

 ip flow-export destination <ip> <port>

Then I have this enabled on all border interfaces/vlans (peering / transit / other core routers) that are of interest for my stats:

 ip route-cache flow

Some more details about the problem:

#sh mls netflow table-contention detailed Earl in Module 5 Detailed Netflow CAM (TCAM and ICAM) Utilization ================================================

TCAM Utilization             :   100%

ICAM Utilization             :   13%

Netflow TCAM count           :   262033

Netflow ICAM count           :   17

Netflow Creation Failures    :   4822220

Netflow CAM aliases          :   1

#sh mls netflow table-contention aggregate Earl in Module 5 Aggregate Netflow CAM Contention Information =============================================

Netflow Creation Failures    :   130003616

Netflow Hash Aliases         :   4

I understand that the TCAM is full, but what can I do against it? This is a busy core router:

Aggregated traffic: 7-8 GBIT/s

Packets per Second: 1.0 - 1.2 Million

I wouldn't mind analyzing only every 10th or 100th flow, which seems to be a common practice.

Any good piece of advice is welcome.

Thanks!

- Andy