On 2014-11-20 23:59, Roland Dobbins wrote:
On 21 Nov 2014, at 4:36, Pavel Odintsov wrote:
I tried to use netflow many years ago but it's not accurate enough and not so fast enough and produce big overhead on middle class network routers.
These statements are not supported by the facts. NetFlow (and other varieties of flow telemetry) has been used for many years for traffic engineering-related analysis, capacity planning, and security purposes. I've never seen the CPU utilization on even a modest mid-range router rise above single-digits, except once due to a bug (which was fixed quickly).
Flow telemetry scales and provides invaluable edge-to-edge traceback information. NetFlow telemetry is accurate enough to be used for all the purposes noted above by network operators across the world, from the smallest to the largest networks in the world.
There are several excellent open-source NetFlow analysis tools which allow folks to benefit from NetFlow analysis without spending a lot of money. Some of these projects have been maintained and enhanced for many years; their authors would not do that if NetFlow analytics weren't sufficient to needs.
Packet-based analysis is certainly useful, but does not scale and does not provide traceback information.
FastNetMon can handle 2-3 million of packets per second and ~20Gbps on standard i7 2600 Linux box with Intel 82599 NIC.
See the comments above with regards to scale. This is inadequate for a network of any size, it does not provide traceback information, and it does not lend itself to broad deployment across a network of any size.
I'm sure FastNetMon is a fine tool, and it's very good of you to spend the time and effort to develop it and to make it available. However, making demonstrably-inaccurate statements about other technologies which are in wide use by network operators and which have a proven track record in the field is probably not the best way to encourage folks to try FastNetMon.
Netflow is stateful stuff, and just to run it on wirespeed, on hardware, you need to utilise significant part of TCAM, i am not talking that on some hardware it is just impossible to run it. So everything about netflow are built on assumption that hosting or ISP can run it. And based on some observations, majority of small/middle hosting providers are using minimal,just BGP capable L3 switch as core, and cheapest but reliable L2/L3 on aggregation, and both are capable in best case to run sampled sFlow. And last thing, from one of public papers, netflow delaying factors: 1. Flow record expiration 2. Exporting process • Typical delay: 15-60 sec. So for a small hosting(up to 10G), i believe, FastNetMon is best solution. Faster, and no significant investments to equipment. Bigger hosting providers might reuse their existing servers, segment the network, and implement inexpensive monitoring on aggregation switches without any additional cost again. Ah, and there is one more huge problem with netflow vs FastNetMon - netflow just by design cannot be adapted to run pattern matching, while it is trivial to patch FastNetMon for that, turning it to mini-IDS for free. --- Best regards, Denys