On 2014-11-22 18:00, freedman@freedman.net wrote:
Cisco ASRs and MXs with inline jflow can do hundreds of K flows/second without affecting packet forwarding.
Yes, i agree,those are good for netflow, but when they already exist in network.
Does it worth to buy ASR, if L3 switch already doing the job (BGP/ACL/rate-limit/routing)?
Not suggesting that anyone should change out their gear though per my other message, I've seen SPAN make things go wonky on almost every vendor that ISPs use for switching. Well, i always try to stay on safe side. Additionally, sure, i do mirror for RX only, RX+TX often can exceed interface rate too :)
Well, if it is available, except hardware limitations, there is second obstacle, software licensing cost. On latest JunOS, for example on EX2200, you need to purchase license (EFL), and if am not wrong it is $3000 for 48port units.
So if only sFlow feature is on stake, it worth to think, to purchase license, or to purchase server. Prices for JFlow license on MX, just for 5/10G is way above cost of very decent server.
I believe that smaller MXs can run it for free. Larger providers we've worked with often have magic cookies they can call in to get it enabled, but I understand you're talking about the smaller-provider (or at least ~ 10gig per POP across multiple POPs) case.
We see a lot of Brocade for switching in hosting providers, which makes sFlow easy, of course.
Oh, Brocade, recent experience with ServerIron taught me new lesson, that i can't do bonding on ports as i want, it has limitations about even/odd port numbers and etc. Most amazing part i just forgot, that i have this ServerIron, and it is a place where i run DDoS protection (but it works perfectly over "tap" way). Thanks for reminding about this vendor :)
And with the right setup you can run FastNetMon or other tools in addition to generating flow that can be of use for other purposes as well...
Technically there is ipt_NETFLOW, that can generate netflow on same box, for statistical/telemetry purposes. But i am not sure it is possible to run them together.
At frac 10gig you can just open pcap on a 10gig interface on a Linux box getting a tap, of course.
What we did was use myricom cards and the myri_snf drivers and take from the single-consumer ring buffers into large in-RAM ring buffers, and make those ring buffers available via LD_PRELOAD or cli tools to allow flow, snort, p0f, tcpdump, etc to all be run at the same time at 10gig.
The key for that is not going through the kernel IP stack, though.
Ntop's pf_ring, which is basically same idea, but can run on Intel cards. Just maybe because never had myricom in hands, and it is difficult to obtain them here.
But taps can be difficult or at least time consuming for people to put in at scale. Even, we've seen, for folks with 10G networks. Often because they can get 90% of what they need for 4 different business purposes from just flow :)
About scaling, i guess it depends on proper deployment strategy and sysadmins/developers capabilities. For example to deploy new ruleset for my pcap-based "homemade" analyser to 150 probes across the country - is just one click.
Sounds cool. You should write up that use case. Hopefully you've secured the metadata/command push channel well enough :)
For servers it is ssh with key authentication, and push system doesn't contain private key, it is forwarded over ssh agent from developer pc. Sure, it is better also to sign by assymmetric crypto update also, keep keys on smartcard, but in this case it is not necessary. --- Best regards, Denys