Johnathan is correct about not using perl for this. There are some iptables modules, but they're all out of date or incomplete (I mention this because if you get around to making them work decent, I'll love you for it). Otherwise, perl -> IPC::Run -> ipt isn't going to gain you anything. And I'd be amazed if you could even keep up with a gbit. Per signature detection, see Bro. Though, it seems the ipt state module might fit the bill just fine. And you could log that and then have an ETL that scraped your log file and created a new ACL based on that (so that hardware could do the majority of the work). I'm sure an ipt -> acl isn't a new idea and you can probably find something that handles most edge cases. On Jun 13, 2013 7:12 PM, "Phil Fagan" <philfagan@gmail.com> wrote:
Yeah, I only thought of perl cause I'm used to running through 'while true' loops and someone showed me Perl was about 400x faster....good thing I'm not running through 10gb/s worth of data :-D
Figured getting closer to hardware was the way to go.....I'll have to check out PF_RING.
On Thu, Jun 13, 2013 at 4:49 PM, Jonathan Lassoff <jof@thejof.com> wrote:
On Thu, Jun 13, 2013 at 3:38 PM, Phil Fagan <philfagan@gmail.com> wrote:
I would assume something FreeBSD based might be best....
Meh... personal choice. I prefer Linux, mostly because I know it best and most network application development is taking place there.
On Thu, Jun 13, 2013 at 4:37 PM, Phil Fagan <philfagan@gmail.com> wrote:
I really like the idea of a stripe of linux boxes doing the heavy
lifting.
Any suggestions on platforms, card types, and chip types that might be better purposed at processing this type of data?
Personally, I'd use modern-ish Intel Ethernet NICs. They seem to have the best support in the kernel.
I assume you could write some fast Perl to ingest and manage the tables? What would the package of choice be for something like this?
Heh... "fast" Perl. As for programming the processing, I would do as much as possible in the kernel, as passing packets off to userland really slows everything down. If you really need to, I'd do something with Go and/or C these days.
Using iptables and the "string" module to match patterns, you can chew through packets pretty efficiently. This comes with the caveat that this can only match against strings contained within a single packet; this doesn't do L4 stream reconstruction.
You can do some incredibly-parallel stuff with ntop's PF_RING code, if you blow more traffic through a single core than it can chew through.
It all depends on what you're trying to do.
On Thu, Jun 13, 2013 at 3:11 PM, Jonathan Lassoff <jof@thejof.com>
wrote:
Are you trying to block flows from becoming established, knowing what you're looking for ahead of time, or are you looking to examine a stream of flow establishments, and will snipe off some flows once you've determined that they should be blocked?
If you know a 5-tuple (src/dst IP, IP protocol, src/dst L4 ports) you want to block ahead of time, just place an ACL. It depends on the platform, but those that implement them in hardware can filter a lot of traffic very quickly. However, they're not a great tool when you want to dynamically reconfigure the rules.
For high-touch inspection, I'd recommend a stripe of Linux boxes,
with
traffic being ECMP-balanced across all of them, sitting in-line on
traffic path. It adds a tiny bit of latency, but can scale up to process large traffic paths and apply complex inspections on the traffic.
Cheers, jof
On Thu, Jun 13, 2013 at 12:32 PM, Eric Wustrow <ewust@umich.edu> wrote:
Hi all,
I'm looking for a way to block individual TCP flows (5-tuple) on a 1-10 gbps link, with new blocked flows being dropped within a millisecond or so of being added. I've been looking into using OpenFlow on an HP Procurve, but I don't know much in this area, so I'm looking for better alternatives.
Ideally, such a device would add minimal latency (many/expandable CAM entries?), can handle many programatically added flows (hundreds
--j the per
second), and would be deployable in a production network (fails in bypass mode). Are there any COTS devices I should be looking at? Or is the market for this all under the table to pro-censorship governments?
Thanks,
-Eric
-- Phil Fagan Denver, CO 970-480-7618
-- Phil Fagan Denver, CO 970-480-7618
-- Phil Fagan Denver, CO 970-480-7618