ipfix/netflow/sflow generator for Linux
At my current place of work, we use all Linux routers. I need to do some IP accounting/reporting and am currently trying to use Scrutinizer. Scrutinizer can use netstream, jstream, ipfix, netflow, and sflow data without qualms. My only issue is that I can't seem to find any good software for Linux that works with multiple interfaces to generate the flow information. I've tried ndsad, nprobe, softflowd, host sflow, and ipcad without much luck. Most of the software only works on one interface (which is useless as I need to do accounting for numerous interfaces). I've had the best luck with ipcad. The only thing that seems to not work with it is that it doesn't correctly give the interface number in the flow information. It refers to all interfaces as interface 65535. I've tried the config option for ipcad to map an interface directly to an SNMP interface ID, but that option of the config file seems to be ignored. Ntop functionally does exactly what I need, but it's extremely buggy. It segfaults after a few minutes, regardless of Linux distro or Ntop version. So..any ideas on what I can do to get good flow information from our Linux routers?
IPtraf can be setup to look at flows per-block, per interface, per vlan, etc and export the data every minute / 5 minutes. Back in the day I had it scripted to dump data into rrdtool and give pretty graphs. See the man page, it's well written. Cheers, -Jack Carrozzo On Mon, Dec 6, 2010 at 2:15 PM, Thomas York <straterra@fuhell.com> wrote:
At my current place of work, we use all Linux routers. I need to do some IP accounting/reporting and am currently trying to use Scrutinizer. Scrutinizer can use netstream, jstream, ipfix, netflow, and sflow data without qualms. My only issue is that I can't seem to find any good software for Linux that works with multiple interfaces to generate the flow information. I've tried ndsad, nprobe, softflowd, host sflow, and ipcad without much luck. Most of the software only works on one interface (which is useless as I need to do accounting for numerous interfaces).
I've had the best luck with ipcad. The only thing that seems to not work with it is that it doesn't correctly give the interface number in the flow information. It refers to all interfaces as interface 65535. I've tried the config option for ipcad to map an interface directly to an SNMP interface ID, but that option of the config file seems to be ignored.
Ntop functionally does exactly what I need, but it's extremely buggy. It segfaults after a few minutes, regardless of Linux distro or Ntop version. So..any ideas on what I can do to get good flow information from our Linux routers?
On Mon, Dec 06, 2010 at 02:15:10PM -0500, Thomas York wrote:
I've had the best luck with ipcad. The only thing that seems to not work with it is that it doesn't correctly give the interface number in the flow information. It refers to all interfaces as interface 65535. I've tried the config option for ipcad to map an interface directly to an SNMP interface ID, but that option of the config file seems to be ignored.
Ntop functionally does exactly what I need, but it's extremely buggy. It segfaults after a few minutes, regardless of Linux distro or Ntop version. So..any ideas on what I can do to get good flow information from our Linux routers?
Fix ipcad to send the interface number. - Matt -- Just because we work at a University doesn't mean we're surrounded by smart people. -- Brian Kantor, in the monastery
-----Original Message----- From: Thomas York [mailto:straterra@fuhell.com] Sent: Monday, December 06, 2010 2:15 PM To: nanog@nanog.org Subject: ipfix/netflow/sflow generator for Linux
At my current place of work, we use all Linux routers. I need to do some IP accounting/reporting and am currently trying to use Scrutinizer. Scrutinizer can use netstream, jstream, ipfix, netflow, and sflow data without qualms. My only issue is that I can't seem to find any good software for Linux
works with multiple interfaces to generate the flow information. I've
ndsad, nprobe, softflowd, host sflow, and ipcad without much luck. Most of the software only works on one interface (which is useless as I need to do accounting for numerous interfaces).
I've had the best luck with ipcad. The only thing that seems to not work with it is that it doesn't correctly give the interface number in the flow information. It refers to all interfaces as interface 65535. I've tried
I've used fprobe with great success. You can run multiple instances of fprobe for the different interfaces. --Samuel fprobe: a NetFlow probe - libpcap-based tool that collects network traffic data and emit it as NetFlow flows towards the specified collector. WWW: http://sourceforge.net/projects/fprobe -- Samuel Petreski Sr. Security Analyst Georgetown University that tried the config
option for ipcad to map an interface directly to an SNMP interface ID, but that option of the config file seems to be ignored.
Ntop functionally does exactly what I need, but it's extremely buggy. It segfaults after a few minutes, regardless of Linux distro or Ntop version. So..any ideas on what I can do to get good flow information from our Linux routers?
-----Original Message----- From: Thomas York [mailto:straterra@fuhell.com] Sent: Monday, December 06, 2010 2:15 PM To: nanog@nanog.org Subject: ipfix/netflow/sflow generator for Linux
At my current place of work, we use all Linux routers. I need to do some IP accounting/reporting and am currently trying to use Scrutinizer. Scrutinizer can use netstream, jstream, ipfix, netflow, and sflow data without qualms. My only issue is that I can't seem to find any good software for Linux
works with multiple interfaces to generate the flow information. I've
ndsad, nprobe, softflowd, host sflow, and ipcad without much luck. Most of the software only works on one interface (which is useless as I need to do accounting for numerous interfaces).
I've had the best luck with ipcad. The only thing that seems to not work with it is that it doesn't correctly give the interface number in the flow information. It refers to all interfaces as interface 65535. I've tried
fprobe doesn't work properly because it has the input and output interface IDs as both 0. In Scrutinizer, this makes the flow look like all the data came in the interface and immediately left via the same interface. Also, this causes problems when running multiple instances of fprobe. This seems to be the issue with most of the flow software I've tried. -----Original Message----- From: Samuel Petreski [mailto:sp446@georgetown.edu] Sent: Monday, December 06, 2010 3:38 PM To: 'Thomas York'; nanog@nanog.org Subject: RE: ipfix/netflow/sflow generator for Linux I've used fprobe with great success. You can run multiple instances of fprobe for the different interfaces. --Samuel fprobe: a NetFlow probe - libpcap-based tool that collects network traffic data and emit it as NetFlow flows towards the specified collector. WWW: http://sourceforge.net/projects/fprobe -- Samuel Petreski Sr. Security Analyst Georgetown University that tried the config
option for ipcad to map an interface directly to an SNMP interface ID, but that option of the config file seems to be ignored.
Ntop functionally does exactly what I need, but it's extremely buggy. It segfaults after a few minutes, regardless of Linux distro or Ntop version. So..any ideas on what I can do to get good flow information from our Linux routers?
Have you considered argus? It can deliver "argus flows" from multiple interfaces. From http://www.qosient.com/argus/ :
Argus can be considered an implementation of the architecture described in the IETF IPFIX Working Group. Argus pre-dates IPFIX, and the project has actively contributed to the IPFIX effort, however, Argus technology should be considered a superset of the IPFIX architecture, providing "proof of concept" implementations for most aspects of the IPFIX applicability statement. Argus technology can read and process Cisco Netflow data, and many sites develop audits using a mixture of Argus and Netflow records.
Ken On 12/6/2010 2:44 PM, Thomas York wrote:
fprobe doesn't work properly because it has the input and output interface IDs as both 0. In Scrutinizer, this makes the flow look like all the data came in the interface and immediately left via the same interface. Also, this causes problems when running multiple instances of fprobe.
This seems to be the issue with most of the flow software I've tried.
-----Original Message----- From: Samuel Petreski [mailto:sp446@georgetown.edu] Sent: Monday, December 06, 2010 3:38 PM To: 'Thomas York'; nanog@nanog.org Subject: RE: ipfix/netflow/sflow generator for Linux
I've used fprobe with great success. You can run multiple instances of fprobe for the different interfaces.
--Samuel
fprobe: a NetFlow probe - libpcap-based tool that collects network traffic data and emit it as NetFlow flows towards the specified collector.
WWW: http://sourceforge.net/projects/fprobe
-- Samuel Petreski Sr. Security Analyst Georgetown University
-----Original Message----- From: Thomas York [mailto:straterra@fuhell.com] Sent: Monday, December 06, 2010 2:15 PM To: nanog@nanog.org Subject: ipfix/netflow/sflow generator for Linux
At my current place of work, we use all Linux routers. I need to do some IP accounting/reporting and am currently trying to use Scrutinizer. Scrutinizer can use netstream, jstream, ipfix, netflow, and sflow data without qualms. My only issue is that I can't seem to find any good software for Linux that works with multiple interfaces to generate the flow information. I've tried ndsad, nprobe, softflowd, host sflow, and ipcad without much luck. Most of the software only works on one interface (which is useless as I need to do accounting for numerous interfaces).
I've had the best luck with ipcad. The only thing that seems to not work with it is that it doesn't correctly give the interface number in the flow information. It refers to all interfaces as interface 65535. I've tried the config option for ipcad to map an interface directly to an SNMP interface ID, but that option of the config file seems to be ignored.
Ntop functionally does exactly what I need, but it's extremely buggy. It segfaults after a few minutes, regardless of Linux distro or Ntop version. So..any ideas on what I can do to get good flow information from our Linux routers?
-- Ken Anderson Pacific Internet - http://www.pacific.net
Never heard of it. I'll give it a shot. Another project that uses argus also looks interesting.. http://nautilus.oshean.org/wiki/Periscope -----Original Message----- From: Ken A [mailto:ka@pacific.net] Sent: Monday, December 06, 2010 4:04 PM To: nanog@nanog.org Subject: Re: ipfix/netflow/sflow generator for Linux Have you considered argus? It can deliver "argus flows" from multiple interfaces. From http://www.qosient.com/argus/ :
Argus can be considered an implementation of the architecture described in the IETF IPFIX Working Group. Argus pre-dates IPFIX, and the project has actively contributed to the IPFIX effort, however, Argus technology should be considered a superset of the IPFIX architecture, providing "proof of concept" implementations for most aspects of the IPFIX applicability statement. Argus technology can read and process Cisco Netflow data, and many sites develop audits using a mixture of Argus and Netflow records.
Ken On 12/6/2010 2:44 PM, Thomas York wrote:
fprobe doesn't work properly because it has the input and output interface IDs as both 0. In Scrutinizer, this makes the flow look like all the data came in the interface and immediately left via the same interface. Also, this causes problems when running multiple instances of fprobe.
This seems to be the issue with most of the flow software I've tried.
-----Original Message----- From: Samuel Petreski [mailto:sp446@georgetown.edu] Sent: Monday, December 06, 2010 3:38 PM To: 'Thomas York'; nanog@nanog.org Subject: RE: ipfix/netflow/sflow generator for Linux
I've used fprobe with great success. You can run multiple instances of fprobe for the different interfaces.
--Samuel
fprobe: a NetFlow probe - libpcap-based tool that collects network traffic data and emit it as NetFlow flows towards the specified collector.
WWW: http://sourceforge.net/projects/fprobe
-- Samuel Petreski Sr. Security Analyst Georgetown University
-----Original Message----- From: Thomas York [mailto:straterra@fuhell.com] Sent: Monday, December 06, 2010 2:15 PM To: nanog@nanog.org Subject: ipfix/netflow/sflow generator for Linux
At my current place of work, we use all Linux routers. I need to do some IP accounting/reporting and am currently trying to use Scrutinizer. Scrutinizer can use netstream, jstream, ipfix, netflow, and sflow data without qualms. My only issue is that I can't seem to find any good software for Linux that works with multiple interfaces to generate the flow information. I've tried ndsad, nprobe, softflowd, host sflow, and ipcad without much luck. Most of the software only works on one interface (which is useless as I need to do accounting for numerous interfaces).
I've had the best luck with ipcad. The only thing that seems to not work with it is that it doesn't correctly give the interface number in the flow information. It refers to all interfaces as interface 65535. I've tried the config option for ipcad to map an interface directly to an SNMP interface ID, but that option of the config file seems to be ignored.
Ntop functionally does exactly what I need, but it's extremely buggy. It segfaults after a few minutes, regardless of Linux distro or Ntop version. So..any ideas on what I can do to get good flow information from our Linux routers?
-- Ken Anderson Pacific Internet - http://www.pacific.net
On Dec 7, 2010, at 3:44 AM, Thomas York wrote:
fprobe doesn't work properly because it has the input and output interface IDs as both 0.
IIRC, this can be altered via a config change. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Sell your computer and buy a guitar.
It can, but then you are setting the input/output IDs statically. That would work fine if your router only had 2 interfaces. We currently have routers with a single (or few) WAN interfaces and multiple internal interfaces and there isn't any way to statically categorize the data. -----Original Message----- From: Dobbins, Roland [mailto:rdobbins@arbor.net] Sent: Monday, December 06, 2010 4:20 PM To: North American Network Operators Group Subject: Re: ipfix/netflow/sflow generator for Linux On Dec 7, 2010, at 3:44 AM, Thomas York wrote:
fprobe doesn't work properly because it has the input and output interface IDs as both 0.
IIRC, this can be altered via a config change. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Sell your computer and buy a guitar.
On Dec 7, 2010, at 4:24 AM, Thomas York wrote:
It can, but then you are setting the input/output IDs statically. That would work fine if your router only had 2 interfaces.
With a probe of this type, northbound/southbound tagging is generally sufficient, in my experience (i.e., let's not make the perfect the enemy of the merely good). ;> ----------------------------------------------------------------------- Roland Dobbins <rdobbins@arbor.net> // <http://www.arbornetworks.com> Sell your computer and buy a guitar.
fprobe doesn't work properly because it has the input and output interface IDs as both 0.
fprobe-ulog fixes this. From the http://fprobe.sourceforge.net/ front page: fprobe-ulog - libipulog-based fork of fprobe. It obtains packets through linux netfilter code (iptables ULOG target). The main advantages of this version are native input/output interface SNMP-index support and significant performance benefit. Of course, this version work on linux only. We have used it here for a few years and have been quite happy with it. E
Try PMACCT, it is pretty handy. Yiming On 12/06/2010 01:15 PM, Thomas York wrote:
At my current place of work, we use all Linux routers. I need to do some IP accounting/reporting and am currently trying to use Scrutinizer. Scrutinizer can use netstream, jstream, ipfix, netflow, and sflow data without qualms. My only issue is that I can't seem to find any good software for Linux that works with multiple interfaces to generate the flow information. I've tried ndsad, nprobe, softflowd, host sflow, and ipcad without much luck. Most of the software only works on one interface (which is useless as I need to do accounting for numerous interfaces).
I've had the best luck with ipcad. The only thing that seems to not work with it is that it doesn't correctly give the interface number in the flow information. It refers to all interfaces as interface 65535. I've tried the config option for ipcad to map an interface directly to an SNMP interface ID, but that option of the config file seems to be ignored.
Ntop functionally does exactly what I need, but it's extremely buggy. It segfaults after a few minutes, regardless of Linux distro or Ntop version. So..any ideas on what I can do to get good flow information from our Linux routers?
participants (8)
-
Dobbins, Roland -
Eric S. Johnson -
Jack Carrozzo -
Ken A -
Matthew Palmer -
Samuel Petreski -
Thomas York -
Yiming Gong