Collecting flows at an IXP
Hi All I'm busy doing some digging to find a solution for collecting layer-2 flows data on a medium sized IXP. All we have at the moment is some MRTG graphs and we're trying to get a better view into IPv4 vs IPv6, src and dst MACs, packet sizes and also perhaps port & protocol trends. I found Richard A. Steenbergen's NANOG 39 presentation and not much since then. Is it still correct that Cisco does not support sFlow? Are you able to get the same kind of useful data using Netflow v9? Which FOSS flow collectors do an decent/adequate job at crunching about 10Gbps worth of flows and presenting it in a useful way? Thanks -- Graham Beneke
On 26/06/2012 07:45, Graham Beneke wrote:
Which FOSS flow collectors do an decent/adequate job at crunching about 10Gbps worth of flows and presenting it in a useful way?
Just to clarify - there are 3 switch fabrics involved here. One from vendor C, one from vendor J and a third new fabric from an unchosen vendor. So ideally something that can accept the flows from various vendors. I'm also hoping for some insight on flows support and caveats with the various vendors and platforms since the this third vendor still must be chosen and it would be handy to quantify the flows support of the proposed platform. -- Graham Beneke
On 26/06/2012 07:06, Graham Beneke wrote:
Just to clarify - there are 3 switch fabrics involved here. One from vendor C, one from vendor J and a third new fabric from an unchosen vendor.
So ideally something that can accept the flows from various vendors.
I'm also hoping for some insight on flows support and caveats with the various vendors and platforms since the this third vendor still must be chosen and it would be handy to quantify the flows support of the proposed platform.
Graham, INEX has open-sourced its IXP provisioning system (github.com/inex), and part of that, is an sflow stats munging system which we use to present member to member data flows, split out by ipv4 / ipv6 traffic, pps and bits/sec. It would be easy enough to put in more graphing options. We haven't released the sflow stuff yet, but it's certainly there and in production, and shoudn't be too much trouble to get into git. Your difficulty is going to be that you're mixing and matching two different systems with completely different characteristics. sflow is easy because you can multiply the sample rate by the packet size (and possibly by an offset constant) and get a statistically representative sample of what's happening on the switch. But netflow is much more messy to handle. Mixing them together will be a lot of fun. Further to this, if you're using Cisco PFC3 based system (sup720 / rsp720 / etc), then this isn't going to work because Cisco WS-X67xx cards don't export mac addresses in netflow data. This is a hardware limitation; nothing you can do about it. This breaks point to point traffic analysis. Nick
Hi Graham, Have you had a look at Argus? http://www.qosient.com/argus/ It works well for us and they have very active support community to boot! Cheers, Harry On 06/26/2012 01:45 AM, Graham Beneke wrote:
Hi All
I'm busy doing some digging to find a solution for collecting layer-2 flows data on a medium sized IXP. All we have at the moment is some MRTG graphs and we're trying to get a better view into IPv4 vs IPv6, src and dst MACs, packet sizes and also perhaps port & protocol trends.
I found Richard A. Steenbergen's NANOG 39 presentation and not much since then.
Is it still correct that Cisco does not support sFlow?
Are you able to get the same kind of useful data using Netflow v9?
Which FOSS flow collectors do an decent/adequate job at crunching about 10Gbps worth of flows and presenting it in a useful way?
Thanks
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, On 06/25/2012 10:45 PM, Graham Beneke wrote:
Hi All
I'm busy doing some digging to find a solution for collecting layer-2 flows data on a medium sized IXP. All we have at the moment is some MRTG graphs and we're trying to get a better view into IPv4 vs IPv6, src and dst MACs, packet sizes and also perhaps port & protocol trends.
I found Richard A. Steenbergen's NANOG 39 presentation and not much since then.
Is it still correct that Cisco does not support sFlow?
Are you able to get the same kind of useful data using Netflow v9?
Which FOSS flow collectors do an decent/adequate job at crunching about 10Gbps worth of flows and presenting it in a useful way?
Thanks
Another option to consider would be nfsen/nfdump..running nicely as a plugin under cacti so we get a central view. regards, /virendra -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iF4EAREIAAYFAk/qHC0ACgkQ3HuimOHfh+FioQD/Qs7/fje8hziGEym2Wh0sIDWI 16p7ZC+6cJnUUGHzPJsA/jzn0/iCwDCFO8UKSjkXuEpwysRo8U/WeZpTKcbzvhHN =k2E2 -----END PGP SIGNATURE-----
participants (4)
-
Graham Beneke -
Harry Hoffman -
Nick Hilliard -
virendra rode