Real-Time Mitigation of Denial of Service Attacks Now Available With AT&T
Hello folks, Here is a great move from one of the biggest NSPs, I'm sure we will see L3, Sprint, UUNet and others will do the same soon to gain more customers since DDoS is the nightmare of the internet now. http://biz.yahoo.com/prnews/040601/nytu051a_1.html Thanks, -J __________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/
Major providers such as Sprint and UUNet have had null route communities available for quite some time... Unless I am mistaken? John Obi wrote:
Hello folks,
Here is a great move from one of the biggest NSPs, I'm sure we will see L3, Sprint, UUNet and others will do the same soon to gain more customers since DDoS is the nightmare of the internet now.
http://biz.yahoo.com/prnews/040601/nytu051a_1.html
Thanks,
-J
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/
On Wed Jun 02, 2004 at 12:29:25AM -0700, Eric Kuhnke wrote:
Major providers such as Sprint and UUNet have had null route communities available for quite some time... Unless I am mistaken?
Indeed. However, the AT&T thing looks like a combination of Arbor PeakFlow:DoS for automated DoS detection on the network, and what used to be Riverhead (and now acquired by Cisco) for "traffic scrubbing" to allow normal traffic to continue to be passed to nodes under attack. COLT have been doing this exact same thing in the UK for a while now. Simon -- Simon Lockhart | Tel: +44 (0)1628 407720 (x(01)37720) | Si fractum Technology Manager | Fax: +44 (0)1628 407701 (x(01)37701) | non sit, noli BBC Internet Ops | Email: Simon.Lockhart@bbc.co.uk | id reficere BBC Technology, Maiden House, Vanwall Road, Maidenhead. SL6 4UB. UK
Simon,
However, the AT&T thing looks like a combination of Arbor PeakFlow:DoS for automated DoS detection on the network, and what used to be Riverhead (and now acquired by Cisco) for "traffic scrubbing" to allow normal traffic to continue to be passed to nodes under attack.
COLT have been doing this exact same thing in the UK for a while now.
We have been doing it *globally* for over a year now, using Arbor Peakflow DDOS and Riverhead Guard, [both of which are excellent products from excellent vendors [take note Cisco!]]. http://www.nanog.org/mtg-0306/afek.html is the presentation Nico gave at SLC. http://www.techweb.com/wire/story/TWB20040428S0006 Guess which site was on the COLT network. We've also protected a large number of our customers against the blackmailing pay us 10K or we'll DDOS you type situations, although few want to press release that type of situation :-) Regards, Neil.
On Wed, 2 Jun 2004, Eric Kuhnke wrote: :: Major providers such as Sprint and UUNet have had null route communities :: available for quite some time... Unless I am mistaken? :: quoting the press release: "The mitigation option uses the principles of analysis, filtering, scrubbing and diversion to protect against such attacks." So i would guess they are using riverhead and hoping they can scale their mitigation clusters faster than DOS kids can scale their botnets. That said, *golf clap* for doing something. Hope their system works as well as their marketing. :-) -jba :: John Obi wrote: :: :: > Hello folks, :: > :: > Here is a great move from one of the biggest NSPs, I'm :: > sure we will see L3, Sprint, UUNet and others will do :: > the same soon to gain more customers since DDoS is the :: > nightmare of the internet now. :: > :: > http://biz.yahoo.com/prnews/040601/nytu051a_1.html :: > :: > Thanks, :: > :: > -J :: > :: > :: > :: > :: > __________________________________ :: > Do you Yahoo!? :: > Friends. Fun. Try the all-new Yahoo! Messenger. :: > http://messenger.yahoo.com/ :: > :: > :: __ [jba@analogue.net] :: analogue.networks.nyc :: http://analogue.net
On 2-jun-04, at 9:39, jeffrey.arnold wrote:
quoting the press release: "The mitigation option uses the principles of analysis, filtering, scrubbing and diversion to protect against such attacks."
Great. So now we're going to see all the spam filtering issues (false positives...) for IP in general? Am I just being cynical or is the port-80-only-internet coming closer and closer? -- "Every computer sold in the US is safe by default. It is powered off, disconnected, in a factory sealed box" - Sean Donelan, on NANOG
John Obi wrote:
... since DDoS is the nightmare of the internet now.
The sad fact is that simple ingress and egress filtering would eliminate the majority of bogus traffic on the Internet -- including (D)DoS attacks. If all ISPs would simply drop all outbound packets whose source address is not a valid IP for the subnet of origin, and all inbound packets that do not have valid source IP addresses, the DDoS problem would be (for all intents and purposes) fixed. If proper filtering was done, then any DoS attacks would have to have either valid source IP addresses, or IP addresses that spoofed IPs within their network of origin. In either case, identifying and shutting down the attackers would become a greatly simplified task compared to the mess it is today. Why no filtering by ISPs? "Because it takes resources and only benefits the other guy" -- unless your network is the one under attack. Maintenance of the ACLs should not be the issue. A single ACL for each subnet would be all that would be required for egress filtering. About 30 ACLs on an inbound border router would be required for ingress filtering. Keeping the ingress ACLs current is a brain-dead task -- just subscribe to the bogon mailing list at cymru.com. ACLs have had a bad reputation for greatly slowing down routers. That may have been true in the past, but properly written ACLs do not seem to have a significant impact on most new routers. Yes, they may cut peak through-put a few percent -- but if you are running that close to the edge, it is time to upgrade anyway. IMHO, there is absolutely no excuse for not doing ingress and egress filtering. In fact, if you are an ISP, I would argue that you are negligent in your fiduciary responsibilities to your customers and shareholders if you are not filtering source IP addresses. Fancy solutions may make great marketing, but simple proper router filtering is a very workable lower-cost solution. (Step down from soap box.) At least, that's my $0.02 worth. Jon Kibler -- Jon R. Kibler Chief Technical Officer A.S.E.T., Inc. Charleston, SC USA (843) 849-8214 ================================================== Filtered by: TRUSTEM.COM's Email Filtering Service http://www.trustem.com/ No Spam. No Viruses. Just Good Clean Email.
Jon R. Kibler wrote:
Why no filtering by ISPs? "Because it takes resources and only benefits the other guy" -- unless your network is the one under attack.
There you have the "operational" issue in a nutshell. No dime, no do. -- Requiescas in pace o email Ex turpi causa non oritur actio http://members.cox.net/larrysheldon/
"Jon" == Jon R Kibler <Jon.Kibler@aset.com> writes:
Jon> The sad fact is that simple ingress and egress filtering would Jon> eliminate the majority of bogus traffic on the Internet -- Jon> including (D)DoS attacks. If all ISPs would simply drop all Jon> outbound packets whose source address is not a valid IP for the Jon> subnet of origin, and all inbound packets that do not have valid Jon> source IP addresses, the DDoS problem would be (for all intents Jon> and purposes) fixed. The majority of the DDoS traffic that's been received here over the past year has had 100% valid and accurate source IP addresses. -- Andrew, Supernews http://www.supernews.com
We see approximately 60-70 spoofed DDOS attacks per day at our Network Telescope: http://noc.ilan.net.il/research/riverhead/ The volume of backscatter we see on our "dark space" /16 is about 50kb/sec: http://noc.ilan.net.il/stats/TAU-GIGAPOP/riverbsc-gp1.ilan.net.il.html I have no way of proving it but I assume we see only 10% of the daily DDOS attacks that take place on the Internet since as you state - most of the attacks these days are from "100% valid and accurate source IP addresses". -Hank
The majority of the DDoS traffic that's been received here over the past year has had 100% valid and accurate source IP addresses.
-- Andrew, Supernews http://www.supernews.com
On Wed, 2004-06-02 at 17:25, Jon R. Kibler wrote:
The sad fact is that simple ingress and egress filtering would eliminate the majority of bogus traffic on the Internet -- including (D)DoS attacks.
Couldn't agree more. It would probably cut hacked zombies (and that way spam) by at least as much as DDoS traffic, in general we'd all have far less problems if ISP's would stick to simple solutions where they're needed. Although there are DoS's coming from valid IP's, 99 out of a 100 of these valid IP's are zombies hacked by using spoofed IP's so the hacker isn't traceable. Good filtering will make this a lot harder to pull off.
Why no filtering by ISPs? "Because it takes resources and only benefits the other guy" -- unless your network is the one under attack.
And this is exactly the kind of ignorant thinking that prevents us from solving the spam and DoS problems, while the exact same people can't stop complaining about the spammers and script-kiddies ruining their lunch.
Maintenance of the ACLs should not be the issue. A single ACL for each subnet would be all that would be required for egress filtering. About 30 ACLs on an inbound border router would be required for ingress filtering. Keeping the ingress ACLs current is a brain-dead task -- just subscribe to the bogon mailing list at cymru.com.
If maintenance of ACLs was a problem for large ISPs, they'd be out of business since that would imply they don't have the staff to keep their networks running, let alone well enough to actually have customers on it. I've probably heard the argument about the money it would cost and the staff it would take a million times, but the fact is that if every ISP did it's filtering, you'll see the need for troubleshooting, spamfiltering, recovering from hackers, and mitigating DoS attacks drop enormously. I'm 100% sure this would lead to lower maintenance costs, not the other way around.
ACLs have had a bad reputation for greatly slowing down routers. That may have been true in the past, but properly written ACLs do not seem to have a significant impact on most new routers. Yes, they may cut peak through-put a few percent -- but if you are running that close to the edge, it is time to upgrade anyway.
Only very small ISPs relying on 36xx's or multilayer switching instead of larger, more powerful might be still valid cases where ACL's are a problem. But those aren't the ISPs generating 80% of all useless traffic, it's the big boys that have plenty of hardware to burn that refuse to do anything about it.
IMHO, there is absolutely no excuse for not doing ingress and egress filtering.
Hear hear -- --- Erik Haagsman Network Architect We Dare BV tel: +31(0)10 7507008 fax:+31(0)10 7507005 http://www.we-dare.nl
On Wed, Jun 02, 2004 at 06:00:38PM +0200, Erik Haagsman wrote:
Only very small ISPs relying on 36xx's or multilayer switching instead of larger, more powerful might be still valid cases where ACL's are a problem.
Interesting assertion. Care to support it? --Jeff
On Wed, 2004-06-02 at 19:32, Jeff Aitken wrote:
On Wed, Jun 02, 2004 at 06:00:38PM +0200, Erik Haagsman wrote:
Only very small ISPs relying on 36xx's or multilayer switching instead of larger, more powerful might be still valid cases where ACL's are a problem.
Interesting assertion. Care to support it?
It's not unusual for smaller ISP's and small hosting companies to rely on low-spec equipment that can just deal with normal traffic flows, but start falling apart when a traffic spike hits and access lists are present. As an example, take a lower end IronCore Foudry switch with a management II or III and make a comparison between the impact a DoS has with and without access lists present. Altough it's still depending on exact network topology and the type of traffic, it's usually a difference of night and day performance wise, and the absence or presence of access-lists can mean the difference between keeping the network running while under attack and having it fall over, especially since all access list handling is taken care of by the CPU. This isn't the case for anyone anywhere that uses this type of equipment, but I can understad smaller networks with smaller budgets and equipment running close to their max hesitance to put access lists and filtering polcies in place. On the other hand, the smaller the network, the smaller the amount of actual filters needed, so you might wonder if that's even a reason not to filter. Cheers, -- --- Erik Haagsman Network Architect We Dare BV tel: +31(0)10 7507008 fax:+31(0)10 7507005 http://www.we-dare.nl
On Thu, 3 Jun 2004, Erik Haagsman wrote:
On Wed, 2004-06-02 at 19:32, Jeff Aitken wrote:
On Wed, Jun 02, 2004 at 06:00:38PM +0200, Erik Haagsman wrote:
Only very small ISPs relying on 36xx's or multilayer switching instead of larger, more powerful might be still valid cases where ACL's are a problem.
Interesting assertion. Care to support it?
It's not unusual for smaller ISP's and small hosting companies to rely on low-spec equipment that can just deal with normal traffic flows, but start falling apart when a traffic spike hits and access lists are present. As an example, take a lower end IronCore Foudry switch with a
Or, look at some examples in the 6500 family even, not really a 'low end' switch, but still able to fail spectacularly under abnormal conditions. (provided you don't have super new Sup720 and other wizz-bang-foo hot off the presses) -Chris
On Thu, Jun 03, 2004 at 10:55:14AM +0200, Erik Haagsman wrote:
Only very small ISPs relying on 36xx's or multilayer switching instead of larger, more powerful might be still valid cases where ACL's are a problem.
Interesting assertion. Care to support it?
It's not unusual for smaller ISP's and small hosting companies [...]
You missed what I was getting at. You asserted that only very small ISPs (i.e., those using 36xx-class hardware) are subject to ACL problems. There are many large-ish ISPs still stuck with some amount of obsolete hardware. My point was that while it's easy for someone whose network consists of 10 routers to say "well gee, upgrade already" it's not that easy when your network includes hundreds or thousands of components that need to be upgraded or replaced, to the tune of several million dollars. This is especially true if you're simply upgrading old hardware; in addition to pouring money into an obsolete platform (is that a wise business decision?), the investment of new capital dollars doesn't directly generate additional revenue, which makes it harder to sell to the folks who hold the purse strings. --Jeff
On Thu, 2004-06-03 at 21:10, Jeff Aitken wrote:
You missed what I was getting at. You asserted that only very small ISPs (i.e., those using 36xx-class hardware) are subject to ACL problems. There are many large-ish ISPs still stuck with some amount of obsolete hardware.
OK, sorry about the confusion...I see where your going now.
My point was that while it's easy for someone whose network consists of 10 routers to say "well gee, upgrade already" it's not that easy when your network includes hundreds or thousands of components that need to be upgraded or replaced, to the tune of several million dollars.
True, but no-one is saying the entire network should be done in one fell swoop. Eventually, larger companies WILL have to replace outdated components and when they do they can replace them and at the same time make sure ACL's or uBRF or whatever you use is in place. And before that, you could at least make sure your newer equipment that CAN easily take ACLs is properly configured. Currently most larger companies do neither, always pointing out the cost of doing a huge network wide upgrade that in actuality no-one is expecting them to do. Even if only a percentage of a large ISP's network (especially xDSL and HFC services) is properly configured, it'll save a lot of grief, cutting maintenance cost for the ISP itself as well as causing less headaches for other companies. And over time you just gradually update parts where you're replacing equipment that's at the end of it's lifecycle anyway. Cheers, -- --- Erik Haagsman Network Architect We Dare BV tel: +31(0)10 7507008 fax:+31(0)10 7507005 http://www.we-dare.nl
True, but no-one is saying the entire network should be done in one fell swoop. Eventually, larger companies WILL have to replace outdated components and when they do they can replace them and at the same time make sure ACL's or uBRF
uRPF even..weird typo
or whatever you use is in place. And before that, you could at least make sure your newer equipment that CAN easily take ACLs is properly configured. Currently most larger companies do neither, always pointing out the cost of doing a huge network wide upgrade that in actuality no-one is expecting them to do. Even if only a percentage of a large ISP's network (especially xDSL and HFC services) is properly configured, it'll save a lot of grief, cutting maintenance cost for the ISP itself as well as causing less headaches for other companies. And over time you just gradually update parts where you're replacing equipment that's at the end of it's lifecycle anyway.
Cheers, --
Erik Haagsman Network Architect We Dare BV tel: +31(0)10 7507008 fax:+31(0)10 7507005 http://www.we-dare.nl
On Jun 2, 2004, at 9:25 AM, Jon R. Kibler wrote:
The sad fact is that simple ingress and egress filtering would eliminate the majority of bogus traffic on the Internet -- including (D)DoS attacks. If all ISPs would simply drop all outbound packets whose source address is not a valid IP for the subnet of origin, and all inbound packets that do not have valid source IP addresses, the DDoS problem would be (for all intents and purposes) fixed. If proper filtering was done, then any DoS attacks would have to have either valid source IP addresses, or IP addresses that spoofed IPs within their network of origin. In either case, identifying and shutting down the attackers would become a greatly simplified task compared to the mess it is today.
Why no filtering by ISPs? "Because it takes resources and only benefits the other guy" -- unless your network is the one under attack.
Maintenance of the ACLs should not be the issue. A single ACL for each subnet would be all that would be required for egress filtering. About 30 ACLs on an inbound border router would be required for ingress filtering. Keeping the ingress ACLs current is a brain-dead task -- just subscribe to the bogon mailing list at cymru.com.
ACLs have had a bad reputation for greatly slowing down routers. That may have been true in the past, but properly written ACLs do not seem to have a significant impact on most new routers. Yes, they may cut peak through-put a few percent -- but if you are running that close to the edge, it is time to upgrade anyway.
IMHO, there is absolutely no excuse for not doing ingress and egress filtering. In fact, if you are an ISP, I would argue that you are negligent in your fiduciary responsibilities to your customers and shareholders if you are not filtering source IP addresses.
Fancy solutions may make great marketing, but simple proper router filtering is a very workable lower-cost solution.
(Step down from soap box.) At least, that's my $0.02 worth.
While I mostly agree with your sentiment, one minor detail.. Based on recent observations of many folks, "spoofing is out of vogue". So much so that some recent discussions I've had with several folks lead me to believe that less than 1% of DDOS attacks today employ source address spoofing. As such, the value of techniques such as backscatter analysis and traceback decrease as well. I suspect that [at least] the perception of wide-scale BCP 38/uRPF and the sheer size and firepower of botnets today has resulted in a very significant decline in source-spoofed attacks. Clever folks actually spoof within the local (sometimes classful) subnet, making it slightly more difficult to identify the concerned host (IF your traceback functions ever make it to the "true Internet ingress" segment where a host resides, which is more often than not unlikely). I suspect this is largely because we do such a poor job fixing compromised hosts that miscreants needn't worry much about losing significant portions of their botnets to traceback and cleanup - as Rob suggests, they're more concerned with losing them to other miscreants. This is also representative of the inversion in attack methods over the past several years (i.e., the inversion from TCP-SYN type stuff to raw UDP-fill-the-pipe style attacks). Nonetheless, ingress filtering certainly helps significantly. -danny
On Wed, Jun 02, 2004 at 10:19:08AM -0600, Danny McPherson wrote:
While I mostly agree with your sentiment, one minor detail..
Based on recent observations of many folks, "spoofing is out of vogue". So much so that some recent discussions I've had with several folks lead me to believe that less than 1% of DDOS attacks today employ source address spoofing. As such, the value of techniques such as backscatter analysis and traceback decrease as well.
Nonsense... While many more attacks are non-forged (see: hacked windows machines in giant DDoS bot-nets that doesn't care about hiding the origin because there are too many hosts to do anything about anyways) than they were in the past, forged source attacks still make up huge portions of the packets being thrown around. What people may being seeing is that poorly randomized source attacks are being automatically filtered by uRPF loose or other means before they ever reach the target. I keep track of my network border filter counters, and believe me spoofed attacks are not going out of style, especially from foreign and certain smaller networks. As a customer of someone who does this kind of filtering and maintains sufficient border capacity, you may never see the gigabits of src bogons, protocol 0 or 255, port 0, 40 byte syns w/no MSS option, etc, and assume that these attacks are out of style because the only ones that get through are the WinXP MSS+SACK unforged drone SYNs. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
On Jun 2, 2004, at 10:56 AM, Richard A Steenbergen wrote:
What people may being seeing is that poorly randomized source attacks are being automatically filtered by uRPF loose or other means before they ever reach the target. I keep track of my network border filter counters, and believe me spoofed attacks are not going out of style,
How do you discriminate *DDOS attacks employing source address spoofing* from broken NATs, rampant worms, PMTU and other related misconfiguration resulting in backscatter and similar garbage - with filter counters? Given, tactically deployed filters in order to mitigate a specific attack to a particular destination would likely glean some value WRT the validity of the source distribution for a given attack, but not generally deployed filters for any destination. And exactly what represents "spoofed" by your definition? Note again that I explicitly called out **DDOS attacks employing source address spoofing**, which is non-inclusive of spoofing in general employed by worms and the like, or common misconfigurations and brokenness that results in the slew of random garbage floating about.
especially from foreign and certain smaller networks.
I'd be extremely interested in any empirical evidence you have to support this, and in better understanding exactly how you determined "foreign and certain smaller networks" were indeed the source of many of these spoofed packets.
As a customer of someone who does this kind of filtering and maintains sufficient border capacity, you may never see the gigabits of src bogons, protocol 0 or 255, port 0, 40 byte syns w/no MSS option, etc, and assume that these attacks are out of style because the only ones that get through are the WinXP MSS+SACK unforged drone SYNs.
I agree, if it's filtered before someone observes it, it won't be observed :-) However, distinguishing between coordinated DDOS attacks that employ source address spoofing and "run of the mill" spoofing (by worms and the like) or simple misconfiguration of some sort resulting in "backscatter" is key. -danny
On Wed, Jun 02, 2004 at 11:39:39AM -0600, Danny McPherson wrote:
On Jun 2, 2004, at 10:56 AM, Richard A Steenbergen wrote:
What people may being seeing is that poorly randomized source attacks are being automatically filtered by uRPF loose or other means before they ever reach the target. I keep track of my network border filter counters, and believe me spoofed attacks are not going out of style,
How do you discriminate *DDOS attacks employing source address spoofing* from broken NATs, rampant worms, PMTU and other related misconfiguration resulting in backscatter and similar garbage - with filter counters? Given, tactically deployed filters in order to mitigate a specific attack to a particular destination would likely glean some value WRT the validity of the source distribution for a given attack, but not generally deployed filters for any destination.
If it walks like a duck, and it sounds like a duck, it is probably a duck. RFC1918 sourced space, most likely from misconfigured NATs and such, account for only a very small amount of the bogon-source packets which go splat. Most of the DoS attempts by volume don't fall into the category of questionable. When you see a 100Mbps stream (from a single ingress interface, with consistant TTL's) of IP proto 0 or 255, or tcp port 0, or classic SYN flooders (SYN w/no MSS) or stream (randomized seq# and fixed ack# on a packet w/TH_ACK flag only) targetting a specific IP/port with a source address of iph.ip_src.s_addr = random(), it is pretty easy to tell those apart from the usual background noise of a worm.
especially from foreign and certain smaller networks.
I'd be extremely interested in any empirical evidence you have to support this, and in better understanding exactly how you determined "foreign and certain smaller networks" were indeed the source of many of these spoofed packets.
Some days it helps to actually have an operational network, instead of being a researcher. Even without interesting tools it isn't terribly hard to look at your PNI graphs, match up the hundreds-of-meg spikes with specific DoS incidents, and go from there. Not to point fingers at anyone in particular, but it seems to be the same foreign networks who tend to have little control over their spammers. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)
On Jun 2, 2004, at 12:36 PM, Richard A Steenbergen wrote:
If it walks like a duck, and it sounds like a duck, it is probably a duck. RFC1918 sourced space, most likely from misconfigured NATs and such, account for only a very small amount of the bogon-source packets which go splat.
But worms, OTOH, seems to be much more persistent.
Most of the DoS attempts by volume don't fall into the category of questionable. When you see a 100Mbps stream (from a single ingress interface, with consistant TTL's) of IP proto 0 or 255, or tcp port 0, or classic SYN flooders (SYN w/no MSS) or stream (randomized seq# and fixed ack# on a packet w/TH_ACK flag only) targetting a specific IP/port with a source address of iph.ip_src.s_addr = random(), it is pretty easy to tell those apart from the usual background noise of a worm.
Sure..
Some days it helps to actually have an operational network, instead of being a researcher. Even without interesting tools it isn't terribly hard to look at your PNI graphs, match up the hundreds-of-meg spikes with specific DoS incidents, and go from there. Not to point fingers at anyone in particular, but it seems to be the same foreign networks who tend to have little control over their spammers.
Heh.. I certainly don't consider myself a researcher, or an operator (any longer) for that matter (though I do have access to a significant amount of both research and operational data and tend not to call a duck a goose simply because I heard a quack :-) -danny
On Wed, 02 Jun 2004 11:39:39 MDT, Danny McPherson <danny@tcb.net> said:
How do you discriminate *DDOS attacks employing source address spoofing* from broken NATs, rampant worms, PMTU and other related misconfiguration resulting in backscatter and similar garbage - with filter counters?
A bogon packet is a bogon packet Filter them all and let the appropriate deity sort them out (unless you bill by traffic volume ;)
Based on recent observations of many folks, "spoofing is out of vogue". So much so that some recent discussions I've had with several folks lead me to believe that less than 1% of DDOS attacks today employ source address spoofing. As such, the value of techniques such as backscatter analysis and traceback decrease as well.
You should be right. If hacker use distributed network of zombie to set up massive attack, he do not bother about revealing back address of the packets (you can find a zombied machine, so what - he have a lot of them); on the other hand, it is much simpler to program such attack without frauding src address. SRC spoofing does not work thru firewalls, and makes zombie detection very simple on the originating side (for example, we log all packets with wrong SRC addresses, originated from our network ports in the INTRANET network).
I suspect that [at least] the perception of wide-scale BCP 38/uRPF and the sheer size and firepower of botnets today has resulted in a very significant decline in source-spoofed attacks. Clever folks actually spoof within the local (sometimes classful) subnet, making it slightly more difficult to identify the concerned host (IF your traceback functions ever make it to the "true Internet ingress" segment where a host resides, which is more often than not unlikely).
I suspect this is largely because we do such a poor job fixing compromised hosts that miscreants needn't worry much about losing significant portions of their botnets to traceback and cleanup - as Rob suggests, they're more concerned with losing them to other miscreants.
This is also representative of the inversion in attack methods over the past several years (i.e., the inversion from TCP-SYN type stuff to raw UDP-fill-the-pipe style attacks).
Nonetheless, ingress filtering certainly helps significantly.
-danny
On Wed, Jun 02, 2004 at 11:25:24AM -0400, Jon R. Kibler wrote:
John Obi wrote:
... since DDoS is the nightmare of the internet now.
The sad fact is that simple ingress and egress filtering would eliminate the majority of bogus traffic on the Internet -- including (D)DoS attacks. If all ISPs would simply drop all outbound packets whose source address is not a valid IP for the subnet of origin, and all inbound packets that do not have valid source IP addresses, the DDoS problem would be (for all intents and purposes) fixed. If proper filtering was done, then any DoS attacks would have to have either valid source IP addresses, or IP addresses that spoofed IPs within their network of origin. In either case, identifying and shutting down the attackers would become a greatly simplified task compared to the mess it is today.
Sorry to say this, but IMHO this is a naive view. It would only marginally lessen the severity of attacks. The bulk of machines being used for DOS attacks are compromised hosts and largely intercontinental (from observations made from attacks against my clients.) There are already machines sequentially opening HTTP sockets, retrieving a particular URL, and repeating that process thousands of times. These sorts of attacks can't be spoofed. And yet when I attempt to contact the administrators of those machines (even when I find them in the US under the auspices of major service providers with "good" abuse departments), I get zero response to the problem. So then if if the people writing this DOS software don't care about hiding the addresses for this type of attack, why hide the addresses from others? The same sort of damage will be done wether the addresses are spoofed or not. Filtering traffic isn't the principle issue (though it will help.) The real problem is administrators who either don't care or flat refuse to do anything about it. (Yes, the word "NO" has been said many times when I've asked someone to investigate a possibly compromised host even when supported by many hundreds of kilobytes of filter logs.) And then of course, even if they DO respond, the end user is the one who ultimately has to solve the problem and good luck getting THAT to happen. (Yes, I know I'm a bit cynical about this but thats the result of long and hard experience fending off such events.)
Why no filtering by ISPs? "Because it takes resources and only benefits the other guy" -- unless your network is the one under attack.
Every one of my connections has rpf enabled unless there is a very valid reson not to. (and thats done case by case.) Recent improvements (I say recent, meaning over the last 5 years or so) have made such efforts markedly more effective. The problem, as you state, is getting the world at large to utilize these mechanisms.
Maintenance of the ACLs should not be the issue. A single ACL for each subnet would be all that would be required for egress filtering. About 30 ACLs on an inbound border router would be required for ingress filtering. Keeping the ingress ACLs current is a brain-dead task -- just subscribe to the bogon mailing list at cymru.com.
For smaller networks, yes. For larger networks, they can have 2 or 3 hundred connections to a single border router with alloted IP space varying daily. Meaning there would have to be frequent updates to an upstream ACL (which may well be across an OC48) and lead to many human caused outages. Simply not practicle for all networks. -Wayne
You even do not need to maintain ACL - many routers have 'back-path verification' feature. I wonder, why DSL and other 'consumer level' providers are not doing it for 100% of their customers. ----- Original Message ----- From: "Jon R. Kibler" <Jon.Kibler@aset.com> To: <nanog@merit.edu> Sent: Wednesday, June 02, 2004 8:25 AM Subject: Re: Real-Time Mitigation of Denial of Service Attacks Now Available With AT&T
John Obi wrote:
... since DDoS is the nightmare of the internet now.
The sad fact is that simple ingress and egress filtering would eliminate the majority of bogus traffic on the Internet -- including (D)DoS attacks. If all ISPs would simply drop all outbound packets whose source address is not a valid IP for the subnet of origin, and all inbound packets that do not have valid source IP addresses, the DDoS problem would be (for all intents and purposes) fixed. If proper filtering was done, then any DoS attacks would have to have either valid source IP addresses, or IP addresses that spoofed IPs within their network of origin. In either case, identifying and shutting down the attackers would become a greatly simplified task compared to the mess it is today.
Why no filtering by ISPs? "Because it takes resources and only benefits the other guy" -- unless your network is the one under attack.
Maintenance of the ACLs should not be the issue. A single ACL for each subnet would be all that would be required for egress filtering. About 30 ACLs on an inbound border router would be required for ingress filtering. Keeping the ingress ACLs current is a brain-dead task -- just subscribe to the bogon mailing list at cymru.com.
ACLs have had a bad reputation for greatly slowing down routers. That may have been true in the past, but properly written ACLs do not seem to have a significant impact on most new routers. Yes, they may cut peak through-put a few percent -- but if you are running that close to the edge, it is time to upgrade anyway.
IMHO, there is absolutely no excuse for not doing ingress and egress filtering. In fact, if you are an ISP, I would argue that you are negligent in your fiduciary responsibilities to your customers and shareholders if you are not filtering source IP addresses.
Fancy solutions may make great marketing, but simple proper router filtering is a very workable lower-cost solution.
(Step down from soap box.) At least, that's my $0.02 worth.
Jon Kibler -- Jon R. Kibler Chief Technical Officer A.S.E.T., Inc. Charleston, SC USA (843) 849-8214
================================================== Filtered by: TRUSTEM.COM's Email Filtering Service http://www.trustem.com/ No Spam. No Viruses. Just Good Clean Email.
Here is a great move from one of the biggest NSPs, I'm sure we will see L3, Sprint, UUNet and others will do the same soon to gain more customers since DDoS is the nightmare of the internet now.
http://biz.yahoo.com/prnews/040601/nytu051a_1.html
Thanks,
Forgive me for being cynical, but couldn't this just AT&T putting an IDS in their POP (or virtually, in their POP) in front of the Customer connections? We are all talking about huge, all encompassing solutions to DDOS that we'd like to see... But what are the chances of that? And is AT&T going to ignore customers that aren't subscribers of AT&T Protect -- even though their network "is monitoring & filtering malicious" traffic at its edges? Deepak Jain AiNET
"[att] which handles 1.3 petabytes of data per day" woooo "For example, in the case of the most recent Sasser worm, AT&T Internet Protect identified precursors to the worm several weeks before it was fully launched. AT&T immediately notified AT&T Internet Protect customers of the malicious activity and provided recommendations for remediation through a secure Web portal to help them proactively protect their networks." oh? how did they do that then.. any at&t protect customers want to fwd the notification... ? Steve On Wed, 2 Jun 2004, John Obi wrote:
Hello folks,
Here is a great move from one of the biggest NSPs, I'm sure we will see L3, Sprint, UUNet and others will do the same soon to gain more customers since DDoS is the nightmare of the internet now.
http://biz.yahoo.com/prnews/040601/nytu051a_1.html
Thanks,
-J
__________________________________ Do you Yahoo!? Friends. Fun. Try the all-new Yahoo! Messenger. http://messenger.yahoo.com/
On Thu, 3 Jun 2004, Stephen J. Wilcox wrote: : "[att] which handles 1.3 petabytes of data per day" : woooo : : > http://biz.yahoo.com/prnews/040601/nytu051a_1.html Let's see... 1.3*10^15 Bytes/Day / 86400 Sec/Day = 15.04B/s 120.4Mbps. Hmm, think they might upgrade to an OC-12 someday? >:-) scott
On Thu, 3 Jun 2004, Petri Helenius wrote: : Scott Weeks wrote: : > : >Let's see... 1.3*10^15 Bytes/Day / 86400 Sec/Day = 15.04B/s : >120.4Mbps. Hmm, think they might upgrade to an OC-12 someday? >:-) : > : : You probably noticed already but your calculation is off by three orders : of magnitude. That'll teach me to send to nanog in pre-coffee mode. 775 OC-12s full all day long. Now we're getting somewhere... :-) scott
participants (22)
-
Alexei Roudnev -
Andrew - Supernews -
Christopher L. Morrow -
Danny McPherson -
Deepak Jain -
Eric Kuhnke -
Erik Haagsman -
Hank Nussbacher -
Iljitsch van Beijnum -
Jeff Aitken -
jeffrey.arnold -
John Obi -
Jon R. Kibler -
Laurence F. Sheldon, Jr. -
Neil J. McRae -
Petri Helenius -
Richard A Steenbergen -
Scott Weeks -
Simon Lockhart -
Stephen J. Wilcox -
Valdis.Kletnieks＠vt.edu -
Wayne E. Bouchard