Hardware capture platforms
We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware? -- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
Check out packet forensics depending on what your ultimate requirements are. Jared Mauch On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <john@hypergeek.net> wrote:
We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware?
-- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
solera makes some nice boxes also On Tue, Jul 29, 2008 at 7:35 PM, Jared Mauch <jared@puck.nether.net> wrote:
Check out packet forensics depending on what your ultimate requirements are.
Jared Mauch
On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <john@hypergeek.net> wrote:
We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware?
-- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <jared@puck.nether.net> wrote:
Check out packet forensics depending on what your ultimate requirements are.
I would also add a 'see packet forensics'...
On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <john@hypergeek.net> wrote:
We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware?
-- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and especially his books (Tao of Network Security Monitoring and Extrusion Detection) are the best sources I have ever found, concerning [not only] taps and[/but] so much more on the subject - proper usage and best methodologies and practices for network monitoring (and not only for security!!!) Stefan On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <morrowc.lists@gmail.com
wrote:
On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <jared@puck.nether.net> wrote:
Check out packet forensics depending on what your ultimate requirements are.
I would also add a 'see packet forensics'...
On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <john@hypergeek.net> wrote:
We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but
I've
got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware?
-- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
There are several things that you can do with open source solutions, however looking at the data may be a bit more difficult than something like Network Generals or Solera Networks capture appliances. It is still doable and is definitely much much cheaper... Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea. You can use regular old tcpdump with the -C option to rotate logs tcpdump -i blah -s0 -C <filesize to rotate>, etc. or you can use Daemonlogger which does pretty much the same thing... http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html On Tue, Jul 29, 2008 at 6:45 PM, Network Fortius <netfortius@gmail.com> wrote:
Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and especially his books (Tao of Network Security Monitoring and Extrusion Detection) are the best sources I have ever found, concerning [not only] taps and[/but] so much more on the subject - proper usage and best methodologies and practices for network monitoring (and not only for security!!!)
Stefan
On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <morrowc.lists@gmail.com
wrote:
On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <jared@puck.nether.net> wrote:
Check out packet forensics depending on what your ultimate requirements are.
I would also add a 'see packet forensics'...
On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <john@hypergeek.net> wrote:
We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but
I've
got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware?
-- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and especially his books (Tao of Network Security Monitoring and Extrusion Detection) are the best sources I have ever found, concerning [not only] taps and[/but] so much more on the subject - proper usage and best methodologies and practices for network monitoring (and not only for security!!!)
Stefan
On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <morrowc.lists@gmail.com
wrote:
On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <jared@puck.nether.net> wrote:
Check out packet forensics depending on what your ultimate requirements are.
I would also add a 'see packet forensics'...
On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <john@hypergeek.net> wrote:
We've deployed a bunch taps in our network and now we need a
Hubs sure are fun... I would trunk the ports you are monitoring, and run the port monitor on the trunk port instead (one trunk port, one port per VLAN, plus one span) which will help with your density. This is assuming the analysis software you have can read the dot1q tags, but means you do not need to burn two ports per monitor. -----Original Message----- From: James Pleger [mailto:jpleger@gmail.com] Sent: Tuesday, July 29, 2008 19:26 To: nanog@merit.edu Subject: Re: Hardware capture platforms There are several things that you can do with open source solutions, however looking at the data may be a bit more difficult than something like Network Generals or Solera Networks capture appliances. It is still doable and is definitely much much cheaper... Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea. You can use regular old tcpdump with the -C option to rotate logs tcpdump -i blah -s0 -C <filesize to rotate>, etc. or you can use Daemonlogger which does pretty much the same thing... http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html On Tue, Jul 29, 2008 at 6:45 PM, Network Fortius <netfortius@gmail.com> wrote: platform on
which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware?
-- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote:
Hubs sure are fun...
This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches. What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch. W --- In the past I have bought some cheap 4 port commodity switches (form Circuit City or somewhere similar), found the datasheet for the chipset (it was a Broadcom something or other) and tied the pin to ground that disables the learning mode (actually, I think that the pin just set the size of the learning table to be 0 entries). While this works, doing it once was more than enough :-)
I would trunk the ports you are monitoring, and run the port monitor on the trunk port instead (one trunk port, one port per VLAN, plus one span) which will help with your density. This is assuming the analysis software you have can read the dot1q tags, but means you do not need to burn two ports per monitor.
-----Original Message----- From: James Pleger [mailto:jpleger@gmail.com] Sent: Tuesday, July 29, 2008 19:26 To: nanog@merit.edu Subject: Re: Hardware capture platforms
There are several things that you can do with open source solutions, however looking at the data may be a bit more difficult than something like Network Generals or Solera Networks capture appliances. It is still doable and is definitely much much cheaper...
Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea.
You can use regular old tcpdump with the -C option to rotate logs
tcpdump -i blah -s0 -C <filesize to rotate>, etc.
or you can use Daemonlogger which does pretty much the same thing...
http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html
Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and especially his books (Tao of Network Security Monitoring and Extrusion Detection) are the best sources I have ever found, concerning [not only] taps and[/but] so much more on the subject - proper usage and best methodologies and practices for network monitoring (and not only for security!!!)
Stefan
On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <morrowc.lists@gmail.com
wrote:
On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <jared@puck.nether.net> wrote:
Check out packet forensics depending on what your ultimate requirements are.
I would also add a 'see packet forensics'...
On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <john@hypergeek.net> wrote:
We've deployed a bunch taps in our network and now we need a
On Tue, Jul 29, 2008 at 6:45 PM, Network Fortius <netfortius@gmail.com> wrote: platform on
which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware?
-- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
-- "Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life." -- Terry Pratchett
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Warren Kumari wrote:
On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote:
Hubs sure are fun...
This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches.
What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps
While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch.
Hubs are still available that are REAL hubs. I got 4 netgears about a year ago and they are still available. However, there is a problem with your specification: No hub (that I am aware of) can do 1Gbps. All hubs are 10/100 AFAIK. Jon Kibler - -- Jon R. Kibler Chief Technical Officer Advanced Systems Engineering Technology, Inc. Charleston, SC USA o: 843-849-8214 c: 843-224-2494 s: 843-564-4224 My PGP Fingerprint is: BAA2 1F2C 5543 5D25 4636 A392 515C 5045 CF39 4253 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkiQty8ACgkQUVxQRc85QlOA1ACfWWGa6FcwzcKT1PN+0pBRky46 bUQAnAxgqV4hfGEZBSgPoMXP8+3/PS+k =ynxx -----END PGP SIGNATURE----- ================================================== Filtered by: TRUSTEM.COM's Email Filtering Service http://www.trustem.com/ No Spam. No Viruses. Just Good Clean Email.
On Wed, Jul 30, 2008 at 02:47:11PM -0400, Jon Kibler wrote:
Hubs are still available that are REAL hubs. I got 4 netgears about a year ago and they are still available.
However, there is a problem with your specification: No hub (that I am aware of) can do 1Gbps. All hubs are 10/100 AFAIK.
And, note carefully: some "dual-speed hubs" are actually a 10BT hub and a 100BT hub *with a switch between them*. I forget which brand I caught this on, but it bit me a couple of years back. Which speed cable you plug in determines which hub you're talking to. Yes, it's weird. Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274 Those who cast the vote decide nothing. Those who count the vote decide everything. -- (Josef Stalin)
On Jul 31, 2008, at 12:31 PM, Jay R. Ashworth wrote:
On Wed, Jul 30, 2008 at 02:47:11PM -0400, Jon Kibler wrote:
Hubs are still available that are REAL hubs. I got 4 netgears about a year ago and they are still available.
However, there is a problem with your specification: No hub (that I am aware of) can do 1Gbps. All hubs are 10/100 AFAIK.
Ok, so I guess what I am speaking is not strictly a hub, it is a non- learning bridge (single collision domain per port, full duplex, etc). There used to be a bunch of devices sold like this -- there were a few really cheap chipsets (AFAIR, Vitesse SparX VSCsomething was one of them -- basically a standard switch chipset that they shaved a few cents off because there was no learning logic / memory) that many people used in cheap "hubs"... I still have some of these somewhere and will rip the lid off to figure out exactly what it was so I can get some more...
And, note carefully: some "dual-speed hubs" are actually a 10BT hub and a 100BT hub *with a switch between them*. I forget which brand I caught this on, but it bit me a couple of years back.
Which speed cable you plug in determines which hub you're talking to.
I see your weird hub story and raise you one: I went along to one of my wife's clients to help lug a printer up the stairs... We get it on the desk and I go to plug in the Ethernet port -- I follow some cables and find this small white switch jammed behind a photocopier -- I pull it out and it has, emblazoned in large red letters on the front, "10/100 Hub with Switch" -- this was back in the day when switches were still cool... I turn it around, and on the back there is... a switch, one side is marked "10M" and the other is marked "100M"... After I stopped laughing I tested it, and sure enough, its a standard hub, and you can make the ports either run at 10Mbps or 100Mbps by flipping the switch... I *really* wish I had replaced and kept it... W
Yes, it's weird.
Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http:// baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
Those who cast the vote decide nothing. Those who count the vote decide everything. -- (Josef Stalin)
-- Do not meddle in the affairs of wizards, for they are subtle and quick to anger. -- J.R.R. Tolkien
I have had the same problem and solved it with a rare (even then) 100BT Only hub. I still have at least one stashed away. For years though, I have been using bonding on Linux to combine multiple tap streams. We also use hardware aggregators for the higher volume applications. Jon On Thu, Jul 31, 2008 at 12:31 PM, Jay R. Ashworth <jra@baylink.com> wrote:
And, note carefully: some "dual-speed hubs" are actually a 10BT hub and a 100BT hub *with a switch between them*. I forget which brand I caught this on, but it bit me a couple of years back.
Which speed cable you plug in determines which hub you're talking to.
Yes, it's weird.
Cheers, -- jra -- Jay R. Ashworth Baylink jra@baylink.com Designer The Things I Think RFC 2100 Ashworth & Associates http://baylink.pitas.com '87 e24 St Petersburg FL USA http://photo.imageinc.us +1 727 647 1274
Those who cast the vote decide nothing. Those who count the vote decide everything. -- (Josef Stalin)
Jay R. Ashworth wrote:
And, note carefully: some "dual-speed hubs" are actually a 10BT hub and a 100BT hub *with a switch between them*. I forget which brand I caught this on, but it bit me a couple of years back.
3COM Dual-Speed 10/100 hubs were this way. Got bit by that too back in the day. Technically I think all hubs supporting both 10 and 100 would have to do this. I can't think of any technical way of getting around the problem without doing this. Justin
On Wed, 30 Jul 2008, Jon Kibler wrote:
However, there is a problem with your specification: No hub (that I am aware of) can do 1Gbps. All hubs are 10/100 AFAIK.
GigE is PtP at the physical-layer by the IEEE 802.3ad specification. It's just not possible to have a dumb, GigE hub. You have to have a switch that can be told to L2-forward everything to one or more ports (e.g. through a port-mirroring feature, or by disabling MAC learning). Also, though probably not terribly relevant, various switches have various bugs/malfeatures that cause them to consume certain kinds of frames rather than forward them (e.g. consuming all or certain kinds of ISO frames). regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: lisp, v.: To call a spade a thpade.
On Fri, 1 Aug 2008, Paul Jakma wrote:
GigE is PtP at the physical-layer by the IEEE 802.3ad specification. It's
Gah, I meant 802.3ab, of course.
just not possible to have a dumb, GigE hub. You have to have a switch that can be told to L2-forward everything to one or more ports (e.g. through a port-mirroring feature, or by disabling MAC learning).
Also, though probably not terribly relevant, various switches have various bugs/malfeatures that cause them to consume certain kinds of frames rather than forward them (e.g. consuming all or certain kinds of ISO frames).
regards, -- Paul Jakma paul@clubi.ie paul@jakma.org Key ID: 64A2FF6A Fortune: Anything is possible, unless it's not.
Warren Kumari wrote:
On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote:
Hubs sure are fun...
This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches.
True enough. For those of us who need and want something non-switched, eBay and other used hardware places are the only real option.
What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps
I don't believe that such a thing ever existed. Hubs that did 10/100, certainly, but I've never ever seen a hub that did gig speeds. When I realized hubs were about to be an endangered species, I started purchasing new and used. I have at least two that (other than testing) have never been used.
While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch.
The original poster needed to deploy a tap, and a hub (for him) would defeat the purpose entirely. If you really really need a hub (or two), your best bet is to start looking at various resellers. Pity you're not closer; I'm retired, and no longer really need the six or eight that I still have. -- In April 1951, Galaxy published C.M. Kornbluth's "The Marching Morons". The intervening years have proven Kornbluth right. --Valdis Kletnieks
The Cisco 8 port 10/100/1000 switch (WS-C2960G-8TC-L) supports RSPAN which would allow you to tap all the ports even though it's a switch. It's about $750, so it's not a cheap option, but it's not outrageous either. It's the right size also. ---- Matthew Huff | One Manhattanville Rd OTA Management LLC | Purchase, NY 10577 www.otaotr.com | Phone: 914-460-4039 aim: matthewbhuff | Fax: 914-460-4139 -----Original Message----- From: Lynda [mailto:shrdlu@deaddrop.org] Sent: Wednesday, July 30, 2008 2:52 PM To: Nanog Subject: Re: Hardware capture platforms Warren Kumari wrote:
On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote:
Hubs sure are fun...
This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches.
True enough. For those of us who need and want something non-switched, eBay and other used hardware places are the only real option.
What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps
I don't believe that such a thing ever existed. Hubs that did 10/100, certainly, but I've never ever seen a hub that did gig speeds. When I realized hubs were about to be an endangered species, I started purchasing new and used. I have at least two that (other than testing) have never been used.
While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch.
The original poster needed to deploy a tap, and a hub (for him) would defeat the purpose entirely. If you really really need a hub (or two), your best bet is to start looking at various resellers. Pity you're not closer; I'm retired, and no longer really need the six or eight that I still have. -- In April 1951, Galaxy published C.M. Kornbluth's "The Marching Morons". The intervening years have proven Kornbluth right. --Valdis Kletnieks
Lynda wrote:
Warren Kumari wrote:
What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps
I don't believe that such a thing ever existed. Hubs that did 10/100, certainly, but I've never ever seen a hub that did gig speeds.
Depends what you mean by 'hub' I guess. I thought the term referred to a device that was half-duplex only, and had no address learning. GE has never supported half-duplex. Sam
All, On the subject of turning off mac learning on a switch, I've just discovered this - an unusual way of using RSPAN to force the MAC learning off on Cisco switches: http://blog.internetworkexpert.com/2008/02/05/turning-switch-into-hub/ # Turn MAC learning on ports Fa0/1 - 3 vtp mode transparent ! vlan 555 remote-span ! interface range Fa 0/1 - 3 switchport trunk encapsulation dot1q switchport mode trunk switchport trunk allowed vlan 555 switchport trunk native vlan 555 Sam Sam Stickland wrote:
Lynda wrote:
Warren Kumari wrote:
What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps
I don't believe that such a thing ever existed. Hubs that did 10/100, certainly, but I've never ever seen a hub that did gig speeds.
Depends what you mean by 'hub' I guess. I thought the term referred to a device that was half-duplex only, and had no address learning. GE has never supported half-duplex.
Sam
Warren Kumari wrote:
On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote:
Hubs sure are fun...
This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches.
What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps
While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch.
D-Link sells a smallish 8-port managed Gigabit switch that allows you to disable learning on the ports -- DGS-3200-10 -- http://www.dlink.com/products/?sec=0&pid=674 I don't know where they hide the manuals on the D-Link US site, but Google turned them up on their Russian ftp server ?? While not incredibly cheap, it seems reasonable at about $300. As a bonus, it seems to have pretty complete IPv6 support. We wanted to do something similar with a 10G switch (SMC8708L2). It let's you set the size of the MAC table, but not to zero. However, we found that setting the size of the table to 1 entry effectively disabled learning.
W ---
In the past I have bought some cheap 4 port commodity switches (form Circuit City or somewhere similar), found the datasheet for the chipset (it was a Broadcom something or other) and tied the pin to ground that disables the learning mode (actually, I think that the pin just set the size of the learning table to be 0 entries). While this works, doing it once was more than enough :-)
Nice hack!
Warren Kumari wrote:
On Jul 29, 2008, at 10:43 PM, Darryl Dunkin wrote:
Hubs sure are fun...
This might be a stupid question, but where can one get small hubs these days? All of the common commodity (eg: 4 port Netgear) "hubs" these days are actually switches.
What I am looking for is: Small enough to live in my notebook bag (e.g.: 4 port with a wall wart.) Cheap Simple 10/100/1000Mbps
You won't find the gig-e hub out there for sale despite some ieee 802.3 participants staunch defense of 1/2 duplex gig-e support and the resulting complications that caused/s... Perversely when traveling I actually use the Ethernet ports on my soekris configured as a bridge for this application. A device with 4 Ethernet ports plus a wifi radio which can be configured as bridges, routed, nated etc if that's what's desired. the soekris is not gig-e capable and it's forwarding capacity is a bit closer to the low hundreds of megs, but it travels in my bag, has disk, wifi etc. MSI industrial makes a mini-itx mainboard that will take an intel core2 has 3 embedded gig-e ports and a 16x pci-e slot that you can put a multiport gig or 2 x 10Gbe interface in... I have a utility 10" deep rackmount that I drag around with that in it when I need more power than the soekris can deliver... http://www.logicsupply.com/products/ms_9642
While a tap would work, I'd prefer a hub because I can then use it to connect machines together in a pinch.
W ---
In the past I have bought some cheap 4 port commodity switches (form Circuit City or somewhere similar), found the datasheet for the chipset (it was a Broadcom something or other) and tied the pin to ground that disables the learning mode (actually, I think that the pin just set the size of the learning table to be 0 entries). While this works, doing it once was more than enough :-)
I would trunk the ports you are monitoring, and run the port monitor on the trunk port instead (one trunk port, one port per VLAN, plus one span) which will help with your density. This is assuming the analysis software you have can read the dot1q tags, but means you do not need to burn two ports per monitor.
-----Original Message----- From: James Pleger [mailto:jpleger@gmail.com] Sent: Tuesday, July 29, 2008 19:26 To: nanog@merit.edu Subject: Re: Hardware capture platforms
There are several things that you can do with open source solutions, however looking at the data may be a bit more difficult than something like Network Generals or Solera Networks capture appliances. It is still doable and is definitely much much cheaper...
Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea.
You can use regular old tcpdump with the -C option to rotate logs
tcpdump -i blah -s0 -C <filesize to rotate>, etc.
or you can use Daemonlogger which does pretty much the same thing...
http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html
Richard's blog @ http://taosecurity.blogspot.com/search?q=taps and especially his books (Tao of Network Security Monitoring and Extrusion Detection) are the best sources I have ever found, concerning [not only] taps and[/but] so much more on the subject - proper usage and best methodologies and practices for network monitoring (and not only for security!!!)
Stefan
On Tue, Jul 29, 2008 at 7:12 PM, Christopher Morrow <morrowc.lists@gmail.com
wrote:
On Wed, Jul 30, 2008 at 12:35 AM, Jared Mauch <jared@puck.nether.net> wrote:
Check out packet forensics depending on what your ultimate requirements are.
I would also add a 'see packet forensics'...
On Jul 29, 2008, at 7:10 PM, "John A. Kilpatrick" <john@hypergeek.net> wrote:
We've deployed a bunch taps in our network and now we need a
On Tue, Jul 29, 2008 at 6:45 PM, Network Fortius <netfortius@gmail.com> wrote: platform on
which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware?
-- John A. Kilpatrick john@hypergeek.net Email| http://www.hypergeek.net/ john-page@hypergeek.net Text pages| ICQ: 19147504 remember: no obstacles/only challenges
-- "Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life." -- Terry Pratchett
On 30 Jul 2008, at 03:26, James Pleger wrote:
Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea.
Never try to aggregate multiple TAPs with a hub. You will just create a bucket load of collisions and end up with a useless data feed presented to your monitoring tool. If you want to aggregate multiple TAP feeds into a smaller number of devices(s), most of the TAP vendors make some form of link aggregation device. Or, depending on the OS and sniffer you use, you may be able to bond the interfaces on the capture device. -Leon
You can use regular old tcpdump with the -C option to rotate logs
tcpdump -i blah -s0 -C <filesize to rotate>, etc.
or you can use Daemonlogger which does pretty much the same thing...
http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html
Second that. Using hub to tap into a single link is also risky. I used to monitor single FE link with 100M hub. After link had moderate utilization >20%, collision led was lit all the time. I've had good experience with VSS Monitoring Ethernet Aggregator taps. Also Catalyst 2960 SPAN seems to work OK. As for capture PC, we've been using regular PC with Wireshark. That's good for single FE link, but has problem with GE and multiple links. BR, Juuso On Wed, Jul 30, 2008 at 4:26 PM, Leon Ward <seclists@rm-rf.co.uk> wrote:
On 30 Jul 2008, at 03:26, James Pleger wrote:
Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea.
Never try to aggregate multiple TAPs with a hub. You will just create a bucket load of collisions and end up with a useless data feed presented to your monitoring tool. If you want to aggregate multiple TAP feeds into a smaller number of devices(s), most of the TAP vendors make some form of link aggregation device.
Or, depending on the OS and sniffer you use, you may be able to bond the interfaces on the capture device.
-Leon
You can use regular old tcpdump with the -C option to rotate logs
tcpdump -i blah -s0 -C <filesize to rotate>, etc.
or you can use Daemonlogger which does pretty much the same thing...
http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html
On 31 Jul 2008, at 14:16, Juuso Lehtinen wrote:
Second that.
Using hub to tap into a single link is also risky. I used to monitor single FE link with 100M hub. After link had moderate utilization
20%, collision led was lit all the time.
I've had good experience with VSS Monitoring Ethernet Aggregator taps. Also Catalyst 2960 SPAN seems to work OK.
As for capture PC, we've been using regular PC with Wireshark. That's good for single FE link, but has problem with GE and multiple links.
If you need to increase the speed of your capture tool, maybe this [1] link may be of use. It is an implementation of a libpcap that implements a shared memory ring buffer which can result in some capture performance gains. [1] http://public.lanl.gov/cpw/ -Leon
BR, Juuso
On Wed, Jul 30, 2008 at 4:26 PM, Leon Ward <seclists@rm-rf.co.uk> wrote:
On 30 Jul 2008, at 03:26, James Pleger wrote:
Something you might want to look into is traffic aggregation with a switch or hub. You can buy an Allied Telesyn switch and basically turn it into a hub by disabling switchport learning. Just an idea.
Never try to aggregate multiple TAPs with a hub. You will just create a bucket load of collisions and end up with a useless data feed presented to your monitoring tool. If you want to aggregate multiple TAP feeds into a smaller number of devices(s), most of the TAP vendors make some form of link aggregation device.
Or, depending on the OS and sniffer you use, you may be able to bond the interfaces on the capture device.
-Leon
You can use regular old tcpdump with the -C option to rotate logs
tcpdump -i blah -s0 -C <filesize to rotate>, etc.
or you can use Daemonlogger which does pretty much the same thing...
http://www.snort.org/users/roesch/Site/Daemonlogger/Daemonlogger.html
Hey, On Thu, 31 Jul 2008 16:00:36 +0100 Leon Ward <seclists@rm-rf.co.uk> wrote:
On 31 Jul 2008, at 14:16, Juuso Lehtinen wrote:
Second that.
Using hub to tap into a single link is also risky. I used to monitor single FE link with 100M hub. After link had moderate utilization
20%, collision led was lit all the time.
I've had good experience with VSS Monitoring Ethernet Aggregator taps. Also Catalyst 2960 SPAN seems to work OK.
As for capture PC, we've been using regular PC with Wireshark. That's good for single FE link, but has problem with GE and multiple links.
If you need to increase the speed of your capture tool, maybe this [1] link may be of use. It is an implementation of a libpcap that implements a shared memory ring buffer which can result in some capture performance gains.
Better off - http://www.ntop.org/PF_RING.html I've seen tenfold decrease in CPU usage using PF_RING.
-Leon
[ cut ] -- Best regards, Nickola Kolev
On Tue, 29 Jul 2008, John A. Kilpatrick wrote:
We've deployed a bunch taps in our network and now we need a platform on which to capture the data. Our bandwidth is currently pretty low but I've got 8 links to tap, which means I need 16 ports. Has anyone done any research on doing accurate packet capture with commodity hardware?
A hardware based capture card is the only way to get to any real throughput. Check out Endace cards, that will let you do line rate gig e or better and has native libpcap interface. You also may want to check out WildPackets cards.
<> Nathan Stratton CTO, BlinkMind, Inc. nathan at robotics.net nathan at blinkmind.com http://www.robotics.net http://www.blinkmind.com
On Jul 31, 2008, at 5:44 AM, nathan@robotics.net wrote:
Check out Endace cards, that will let you do line rate gig e or better and has native libpcap interface.
I believe Endace also have a productized box containing their capture cards (NinjaProbe); it can be used to capture packets, and can also export NetFlow telemetry based upon the captured traffic. Arbor, Narus, and Lancope have similar NetFlow-via-packet-capture capabilities. ----------------------------------------------------------------------- Roland Dobbins <rdobbins@cisco.com> // +66.83.266.6344 mobile History is a great teacher, but it also lies with impunity. -- John Robb
participants (23)
-
Christian Koch -
Christopher Morrow -
Darryl Dunkin -
James Pleger -
Jared Mauch -
Jay R. Ashworth -
Joel Jaeggli -
John A. Kilpatrick -
Jon Kibler -
Jon Meek -
Justin Shore -
Juuso Lehtinen -
Larry J. Blunk -
Leon Ward -
Lynda -
Matthew Huff -
nathan＠robotics.net -
Network Fortius -
Nickola Kolev -
Paul Jakma -
Roland Dobbins -
Sam Stickland -
Warren Kumari