Is current DDoS detecting method effective?
Hi, I use flow-tools to monitor the link bandwidth utilization on three backbone interfaces. The total bandwidth utilized is about 11Gbps, and netflow data is analyzed to show statistics on some special port (e.g. port 0, port 445 etc.). I think this could give us some indication of possible DoS attach, but it's hard to monitor DoS attack on all hosts or all ports. In fact, I'm not sure whether traffic monitoring could REALLY help to identify some DoS attack, esp. in ISP networks. My questions include: 1) what should be protected in ISP networks? the ISP's own network or both ISP's network and its customers? I think the answer is, ISP should only care about the safety of its own network, which should be overprovisioned ( not only link bandwidth but also CPU/MEM etc.); we could use some technique like reverse route checking and ACL to immunize those core router/switch from DoS. 2) What's the cost should we take to identify any possible DoS in ISP network? I think it will cost a lot if we keep monitoring traffic on all edge routers ( both to backbone network and to customers), because we have to set up traffic monitoring on all interfaces and we have to set up analysis hosts whose ability have to be increased time to time. While the gainback is not obivious ( at least Botnet could not be crashed easily). 3) Is those technique use in current days really effective ? Where can I find some theretical analysis on the method Arbor used to identify DoS? To my experience, network attack is continuous. I do a experiment in our network, I put a Win2003 server on access layer. After 24 hours, the software firewall on it recorded about 10,0000 scan&attack attemps. Arbor says its product build up traffic model before identify DoS, while DoS may have been on its peak point when Arbor's box is building up its traffic model!! So, how can we do with DoS in ISP network? --- "David J. Hughes" <bambi@hughes.com.au> wrote:
I know you said not Arbor, but I'd second this opinion. I used Arbor at a medium-sized European ISP and it was fantastic at
trial period found a lot of smaller DoS attacks on our network that we didn't even know were there, and this was without a particular
the development time you'd spend building something
On 04/03/2005, at 5:17 AM, Chris Roberts wrote: the job. Just in the baseline. I think like (we tried building
similar with cflowd et al) would outweigh the costs... This is always a moot point if you don't have the cash though I guess :-)
Another option on the commercial front is from Esphion in New Zealand (www.esphion.com). I've been involved with deploying their products at a large hosting provider in Australia and I've been very impressed with the performance and reliability. It's now an integral part (if not the corner stone) of our DOS mitigation procedure. Good bit of kit.
David ...
__________________________________________________ Do You Yahoo!? Log on to Messenger with your mobile phone! http://sg.messenger.yahoo.com
On Mon, 7 Mar 2005, Joe Shen wrote:
To my experience, network attack is continuous. I do a experiment in our network, I put a Win2003 server on access layer. After 24 hours, the software firewall on it recorded about 10,0000 scan&attack attemps. Arbor says its product build up traffic model before identify DoS, while DoS may have been on its peak point when Arbor's box is building up its traffic model!!
you aren't distinguishing between 'dos attack' and 'scan' or 'probe' or 'welcome to the Internet!' traffic. The Arbor systems may see 'scan' traffic (depending upon sample rates and traffic loads) and they may not... They aren't designed to see that, they are designed to: (speaking of peakflow SP, peakflow Traffic, peakflow DoS only... peakflow X isn't really a 'provider' solution as much as a 'enterprise' tool) 1) to watch traffic and alarm against thresholds 2) track traffic trends over time 3) report traffic trends over time (possibly some other things out of scope of this discussion... someone from Arbor could/should clarify) Some of your cflowd gathering should also see these things, but they will need data correlation, something Arbor already went to the trouble of doing for you... So, define: "attack" and then see if your tool fits that definition.
Hi,
you aren't distinguishing between 'dos attack' and 'scan' or 'probe' or 'welcome to the Internet!' traffic. The Arbor systems may see 'scan' traffic (depending upon sample rates and traffic loads) and they may not... They aren't designed to see that, they are designed to: (speaking of peakflow SP, peakflow Traffic, peakflow DoS only... peakflow X isn't really a 'provider' solution as much as a 'enterprise' tool)
That's what I think current tool not enough, because we can not think ongoing traffic is not malicious when tools are building up 'normal' traffic model in ISP networks. But, in enterprise network this could be achived because traffic pattern for a enterprise could be estimated, and load on special server could be controled by threshhold (but, think about CNN website on 911 )
1) to watch traffic and alarm against thresholds 2) track traffic trends over time 3) report traffic trends over time
So, it need to define what should be monitored ( port, protocol, application data set ...) ?
(possibly some other things out of scope of this discussion... someone from Arbor could/should clarify)
Some of your cflowd gathering should also see these things, but they will need data correlation, something Arbor already went to the trouble of doing for you... So, define: "attack" and then see if your tool fits that definition.
So, I think current tool is just for enterprise , or for ISPs who want to provide anti-DoS services. regards Joe __________________________________________________ Do You Yahoo!? Log on to Messenger with your mobile phone! http://sg.messenger.yahoo.com
On Mon, 07 Mar 2005 06:11:35 +0000 (GMT), Christopher L. Morrow <christopher.morrow@mci.com> wrote:
Some of your cflowd gathering should also see these things, but they will need data correlation, something Arbor already went to the trouble of doing for you... So, define: "attack" and then see if your tool fits that definition.
So I can safely say that Detecting DDoS attacks is mostly done using Netflow data, now the only tool(known) on the market to analyze for attacks is Arbor, now besides being expensive, which is a problem for Mid-sizes ISPs, doing that with open-source tools(cflowd,...) isnt quite easy for a network engineer, who rarely has programming experience, thats my problem now, we either need to outsource or buy Arbor, I've seen open-source Netflow DDoS specific apps. anyone tried them (Zazu and Panoptis) -With the small experience i've gained to work out these tools, - Zazu is still under devel. but some times reports nice results - couldnt compile panoptis Any luck with (stager, Silktools, ntop,...)? I wish there could be a documented ISPs experience for using open-source tools to detect DDoS, or a homegrown script that uses flow-tools to report anomalies. Any news of undergoing projects or papers for the above, there are too many on Blackholing, but not how to get the IP to blackhole) Regards
On Mon, Mar 07, 2005 at 01:43:29PM +0200, Kim Onnel wrote:
On Mon, 07 Mar 2005 06:11:35 +0000 (GMT), Christopher L. Morrow <christopher.morrow@mci.com> wrote:
Some of your cflowd gathering should also see these things, but they will need data correlation, something Arbor already went to the trouble of doing for you... So, define: "attack" and then see if your tool fits that definition.
So I can safely say that Detecting DDoS attacks is mostly done using Netflow data, now the only tool(known) on the market to analyze for attacks is Arbor, now besides being expensive, which is a problem for Mid-sizes ISPs, doing that with open-source tools(cflowd,...) isnt quite easy for a network engineer, who rarely has programming experience, thats my problem now, we either need to outsource or buy Arbor,
I've seen open-source Netflow DDoS specific apps. anyone tried them (Zazu and Panoptis)
-With the small experience i've gained to work out these tools, - Zazu is still under devel. but some times reports nice results - couldnt compile panoptis
Any luck with (stager, Silktools, ntop,...)?
I wish there could be a documented ISPs experience for using open-source tools to detect DDoS, or a homegrown script that uses flow-tools to report anomalies.
I once took some time to write a netflow processing system. It's not that hard.. If you want some "basic" detection, I recommend doing something like this: sort by the top "proto+dstip+dstport+tcpflags" combination. The more of these you see, the more it may look weird. As Chris mentioned before, what defines the "bad" threshold depends on your customerbase. Maybe 1Kpps is bad. Maybe 100Kpps is normal. Cisco publishes the netflow datagram specification, so you may be able to write an optimized netflow daemon that doesn't take up too much cpu/disk/whatnot if you discard the lower levels of the "noise". Once you define your 'signal' and 'noise' you can then determine what is valuable to you. http://www.cisco.com/warp/public/cc/pd/iosw/ioft/neflct/tech/napps_wp.htm - jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
* Jared Mauch:
If you want some "basic" detection, I recommend doing something like this:
sort by the top "proto+dstip+dstport+tcpflags" combination. The more of these you see, the more it may look weird.
You should also run a similar query for source IPs in your netblocks, particularly one restricted to 25/TCP. 8->
Cisco publishes the netflow datagram specification, so you may be able to write an optimized netflow daemon that doesn't take up too much cpu/disk/whatnot if you discard the lower levels of the "noise".
I wouldn't optimize prematurely. I was surprised how far you can get with simple Perl script, a slightly increased socket buffer size for the receiving UDP socket, and rotating ASCII log files.
* Kim Onnel:
So I can safely say that Detecting DDoS attacks is mostly done using Netflow data, now the only tool(known) on the market to analyze for attacks is Arbor, now besides being expensive, which is a problem for Mid-sizes ISPs,
Who qualifies as a mid-sized ISP? What equipment is typical? Even the most simple approach, based on sampled Netflow, an off-the-shelf SQL database (PostgreSQL preferred) and a couple of Perl scripts can work wonders. You won't get reliable automated alerts, but you can run ad-hoc queries to find out what's going on on your network when something or somebody else has detected a problem. The people already doing this probably consider this trivial, so it's not well documented. I tried to write something down, but never found the time to really polish it: <http://cert.uni-stuttgart.de/projects/flows/> DoS detection can be quite hard, especially if you have many compromised Windows boxes and you can't force the owners to clean them (because it's too expensive to contact them, for example). This results in a lot of background noise and useless flow data, too. If there's little background noise, you can use rather straightforward SQL query that you run periodically.
participants (5)
-
Christopher L. Morrow -
Florian Weimer -
Jared Mauch -
Joe Shen -
Kim Onnel