Re: Global Blackhole Service
Hi Suresh, But in the meanwhile, a decade later, it does not longer exist. At least, i can't reach that host, and i was unable to find working documentation on google of how about this project works, today. In fact, the first link that google gave out, says that this project is dead at least 2 years ago. http://www.dnsbl.com/2007/02/status-of-rblmapsvixcom-invalid-domain.html I think that we all have a real opportunity here for make something that can be useful to all. And, we are not talk of spam here, but, to mitigate time, money and patience consuming DDoS attacks, which often are easier to mitigate only at the Source and at the Destination, while at Destinatation, sink is the only viable solution that is out there today. regards, --- Nuno Vieira nfsi telecom, lda. nuno.vieira@nfsi.pt Tel. (+351) 21 949 2300 - Fax (+351) 21 949 2301 http://www.nfsi.pt/ ----- "Suresh Ramasubramanian" <ops.lists@gmail.com> wrote:
On Fri, Feb 13, 2009 at 8:27 PM, Jens Ott - PlusServer AG <j.ott@plusserver.de> wrote:
- - What do you think about such service? - - Would you/your ASN participate in such a service? - - Do you see some kind of usefull feature in such a service? - - Do you have any comments?
Ah. rbl.maps.vix.com from about a decade back when it was available as a bgp feed. But only for ddos sources.
srs
DDoS drones - especially with botnets - can produce a really large zone To start with google "spamhaus drop list". Then look at the cbl and see if you think its worth using as a bgp feed On Fri, Feb 13, 2009 at 9:20 PM, Nuno Vieira - nfsi telecom <nuno.vieira@nfsi.pt> wrote:
Hi Suresh,
But in the meanwhile, a decade later, it does not longer exist.
At least, i can't reach that host, and i was unable to find working documentation on google of how about this project works, today.
In fact, the first link that google gave out, says that this project is dead at least 2 years ago.
http://www.dnsbl.com/2007/02/status-of-rblmapsvixcom-invalid-domain.html
I think that we all have a real opportunity here for make something that can be useful to all.
And, we are not talk of spam here, but, to mitigate time, money and patience consuming DDoS attacks, which often are easier to mitigate only at the Source and at the Destination, while at Destinatation, sink is the only viable solution that is out there today.
regards, --- Nuno Vieira nfsi telecom, lda.
nuno.vieira@nfsi.pt Tel. (+351) 21 949 2300 - Fax (+351) 21 949 2301 http://www.nfsi.pt/
----- "Suresh Ramasubramanian" <ops.lists@gmail.com> wrote:
On Fri, Feb 13, 2009 at 8:27 PM, Jens Ott - PlusServer AG <j.ott@plusserver.de> wrote:
- - What do you think about such service? - - Would you/your ASN participate in such a service? - - Do you see some kind of usefull feature in such a service? - - Do you have any comments?
Ah. rbl.maps.vix.com from about a decade back when it was available as a bgp feed. But only for ddos sources.
srs
-- Suresh Ramasubramanian (ops.lists@gmail.com)
<j.ott@plusserver.de> wrote:
- - What do you think about such service? - - Would you/your ASN participate in such a service? - - Do you see some kind of usefull feature in such a service? - - Do you have any comments?
----- "Suresh Ramasubramanian" <ops.lists@gmail.com> wrote:
Ah. rbl.maps.vix.com from about a decade back when it was available as a bgp feed. But only for ddos sources.
Nuno Vieira - nfsi telecom <nuno.vieira@nfsi.pt> writes:
But in the meanwhile, a decade later, it does not longer exist.
it still exists (same ASN, different bgp peer address) as a commercial service now operated by Trend Micro. noncommercial alternatives exist, considering here the Spamhaus and Cymru offerings. (i regret that i was unable to continue the service noncommercially, but lawyers are expensive, and volunteers burn out faster than employees, and so on.)
In fact, the first link that google gave out, says that this project is dead at least 2 years ago.
http://www.dnsbl.com/2007/02/status-of-rblmapsvixcom-invalid-domain.html
fun. perhaps i'll stop getting 100+ queries per second to the nameservers of rbl.maps.vix.com some day before i die, now that google is on my side.
I think that we all have a real opportunity here for make something that can be useful to all.
i think Spamhaus and Cymru are way ahead of you in implementing such a thing, and it's likely that there are even commercial alternatives to Trend Micro although i have not kept up on those details. -- Paul Vixie
Paul Vixie wrote:
i think Spamhaus and Cymru are way ahead of you in implementing such a thing, and it's likely that there are even commercial alternatives to Trend Micro although i have not kept up on those details.
I think there's a misunderstanding from what I've read about what is being blackholed. We are not talking about blackholing the senders, but a massive scale method of blackholing the victims at the victim's request to protect infrastructure. Currently this type of service usually doesn't extend beyond one or two ASs and depending on traffic flows can still cause damage, especially through exchange points. With enough support and use, this would allow a larger portion of bad traffic to be null routed closer to the sender origination points. Since the null routing BGP servers would expect a larger routing table from these /32 networks, they would be placed at key points capable of handling the larger tables; compared to just allowing the /32's out into the wild and possibly exceeding route/memory constraints. It can also be used as authoritative information that an IP is undergoing a DOS attack, and large volumes of connections to that IP should be considered suspect. I consider this a much more useful method of detecting DOS traffic leaving your infected users than the emails which are usually sent out by those being hit by DOS. Jack
blackholing victims is an interesting economics proposition. you're saying the attacker must always win but that they must not be allowed to affect the infrastructure. and you're saying victims will request this, since they know they can't withstand the attack and don't want to be held responsible for damage to the infrastructure. where you lose me is where "the attacker must always win".
Listen online to my favorite hip hop radio station http://www.Jellyradio.com On Feb 13, 2009, at 9:35 AM, Paul Vixie <vixie@isc.org> wrote:
blackholing victims is an interesting economics proposition. you're saying the attacker must always win but that they must not be allowed to affect the infrastructure. and you're saying victims will request this, since they know they can't withstand the attack and don't want to be held responsible for damage to the infrastructure.
where you lose me is where "the attacker must always win".
Perhaps removing the challenge from the attacker will bore them and they lose interest? However if an attackers goal is to put someone out of business, they will keep it up until the deed is done. Identifying the attacker is important. They must be the one who is in trouble, not the victim. We have seen attackers extorting customers for money with things like "100k wired to Nevis bank account or attack continues". In any case I do not believe a victim should be responsible for infrastructure damage caused by some random criminal attacking them. While I understand that it's that customer receiving the attack; the providers must work with the customer to trace it back to the source. A hacker who thinks the customer is on a security weak provider will return seeking your other customers. However if the hacker feels you are security savvy then he may choose another target. Everyone wins. Also, rather than penalize the victim for damage, you could always unplug them to interdict the damage. By going after the hacker, you could prosecute and perhaps gain some nice press/media about the strength of your orginization as a side dish to the satisfying meal of eating your enemy?
Paul Vixie wrote:
blackholing victims is an interesting economics proposition. you're saying the attacker must always win but that they must not be allowed to affect the infrastructure. and you're saying victims will request this, since they know they can't withstand the attack and don't want to be held responsible for damage to the infrastructure.
Blackholing victims is what is current practice. For each stage of affected infrastructure, the business/provider will make requests to their peers to blackhole the victim IP to protect the bandwidth caps or router throughput caps. Most providers, I imagine, don't ask the victim. The victim is unintentionally in violation of a TOS or AUP in many cases, but just as importantly, the provider can point out that the service to the customer was useless to begin with, and so the provider protected the rest of the customers who were not directly attacked. Sometimes the attack is to something simple, like the IP of a modem bank or router just upstream of the intended victim. Such cases are no-brainers. We didn't need public access to that IP anyways. It'll break a few traceroutes, but otherwise, business goes on. In a few cases, it has been the end target IP of a customer which was dynamic in nature. The IP was blackholed for 3-5 days and the customer was transfered to a new IP and warned not to piss off the attacker.
where you lose me is where "the attacker must always win".
Do you have a miraculous way to stop DDOS? Is there now a way to quickly and efficiently track down forged packets? Is there a remedy to shutting down the *known* botnets, not to mention the unknown ones? The attacker will always win if he has a large enough attack platform/botnet. Attacks aren't random in nature. Someone pissed someone else off that was, or knew someone who was, self proclaimed l33t. How many threads are in nanog archives on using prefix lists, uRPF, etc? Most of the problems that allow DDOS traffic are not technical problems, as much as they are economic and political problems. While all this is worked out, we have one solution we know works. If we null route the victim IP, the traffic stops at the null route. Since most attackers don't care to DOS the ISP, but just to take care of that end point, they usually don't start shifting targets to try and keep the ISP itself out. Jack
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jack Bates schrieb:
Paul Vixie wrote:
Do you have a miraculous way to stop DDOS? Is there now a way to quickly and efficiently track down forged packets? Is there a remedy to shutting down the *known* botnets, not to mention the unknown ones?
This is another issue, and _all_ of us are in charge to keep their net clean from outgoing DoS. Most outgoing DoS inside our network are mitigated - ok most of the time the dos'ing server is being disconnected - in less than 10 minutes, as we do not only check what's coming in, but also check what our customers are sending out. And as soon as someone forges IPs, he's disconnected unless we know what was happening (mostly hacked servers) and the issue was fixed. As it is the nature of DoS that there are lots of packets send, they can easily be identified in (s|c|net)flows ... unfortunately there are _lots_ of ISP not having automated mechanism for misuse-detection and mitigation, or if they have some, they don't care about alarms. Therefore I agree, the only practicable way to protect the majority of customers is to blackhole the IP under attack. Even if the DoS is not DDoS, but coming from one single source... 99,9% of any emails to any NOC worldwide is not being answered in less than one hour (especially in "out-shift-hours") and from the 0.1% left I bet 99,9% of the DoS are also not stopped during this hour. And one hour of DoS may make some small ISP loose more money then they earn per month!
While all this is worked out, we have one solution we know works. If we null route the victim IP, the traffic stops at the null route. Since most attackers don't care to DOS the ISP, but just to take care of that end point, they usually don't start shifting targets to try and keep the ISP itself out.
ACK!
Jack
- -- =================================================================== Jens Ott Leiter Network Management Tel: +49 22 33 - 612 - 3501 Fax: +49 22 33 - 612 - 53501 E-Mail: j.ott@plusserver.de GPG-Fingerprint: 808A EADF C476 FABE 2366 8402 31FD 328C C2CA 7D7A PlusServer AG Daimlerstraße 9-11 50354 Hürth Germany HRB 58428 / Amtsgericht Köln, USt-ID DE216 740 823 Vorstand: Jochen Berger, Frank Gross, Jan Osthues, Thomas Strohe Aufsichtsratsvorsitz: Claudius Schmalschläger =================================================================== -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkmVv5EACgkQMf0yjMLKfXptpQCeNNgDOxXWoTBHA5W5yCwifcG2 IasAnAh06DE3qry/puXzBs05pBfIMSS/ =boMf -----END PGP SIGNATURE-----
a minor editorial comment: Jens Ott - PlusServer AG <j.ott@plusserver.de> writes:
Jack Bates schrieb:
Paul Vixie wrote:
Do you have a miraculous way to stop DDOS? Is there now a way to quickly and efficiently track down forged packets? Is there a remedy to shutting down the *known* botnets, not to mention the unknown ones?
the quoted text was written by jack bates, not paul vixie. -- Paul Vixie
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Paul Vixie schrieb:
a minor editorial comment:
Jens Ott - PlusServer AG <j.ott@plusserver.de> writes:
Jack Bates schrieb:
Paul Vixie wrote:
Do you have a miraculous way to stop DDOS? Is there now a way to quickly and efficiently track down forged packets? Is there a remedy to shutting down the *known* botnets, not to mention the unknown ones?
the quoted text was written by jack bates, not paul vixie.
Sorry ... must have deleted a little to much from context .... Didn'r want to move someones word into the otherones mouth ... Have a nice sunday - -- =================================================================== Jens Ott Leiter Network Management Tel: +49 22 33 - 612 - 3501 Fax: +49 22 33 - 612 - 53501 E-Mail: j.ott@plusserver.de GPG-Fingerprint: 808A EADF C476 FABE 2366 8402 31FD 328C C2CA 7D7A PlusServer AG Daimlerstraße 9-11 50354 Hürth Germany HRB 58428 / Amtsgericht Köln, USt-ID DE216 740 823 Vorstand: Jochen Berger, Frank Gross, Jan Osthues, Thomas Strohe Aufsichtsratsvorsitz: Claudius Schmalschläger =================================================================== -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkmXz+UACgkQMf0yjMLKfXqC+ACfbj1PcMQknt6R3G5or5iqHD5f 5awAniuOjy+Eoxq4TLd0x7ekQqaeIX9r =oNog -----END PGP SIGNATURE-----
On Fri, Feb 13, 2009 at 1:04 PM, Jack Bates <jbates@brightok.net> wrote:
Paul Vixie wrote:
blackholing victims is an interesting economics proposition. you're saying the attacker must always win but that they must not be allowed to affect the infrastructure. and you're saying victims will request this, since they know they can't withstand the attack and don't want to be held responsible for damage to the infrastructure.
Blackholing victims is what is current practice. For each stage of affected
it is A current practice.. so is filtering, so is scrubbing... there is no one answer for this.
infrastructure, the business/provider will make requests to their peers to blackhole the victim IP to protect the bandwidth caps or router throughput caps.
or cause no one really cares about: your.mama.wears.combat.boots.tobed.com ... or other silly 95%-of attacked, things.
where you lose me is where "the attacker must always win".
Do you have a miraculous way to stop DDOS? Is there now a way to quickly and
There are purchasable answers to this problem... 3 (at least) providers in the US (and at least one now offers it globally) offer traffic scrubbing services. I know that one offers it at a very reasonable price even...
efficiently track down forged packets? Is there a remedy to shutting down
you can track streams of forged packets, but that's not super important here. Forged packets actually make this part of the problem (stopping the dos) easier, not harder.
the *known* botnets, not to mention the unknown ones?
there are lots of folks tracking and shutting down botnets, it's not horribly effective in stopping this sort of thing. I can vividly recall tracking down 4 nights in a row the same 'botnet' (same controller person, different C&C and mostly different bots) as they were being used to attack a customer of mine at the time. This with the cooperation of 2 other very large ISP's in the US and one vendor security team even. In the end though a simple scrubbing solution was deemed the simplest answer for all involved.
The attacker will always win if he has a large enough attack
For extreme cases this is true, but there are quite a lot of things on the spectrum which don't require super human efforts, and don't even require intervention from the ISP if proper precautions are taken at the outset. -chris
I think this solution addresses a number of issues that the current blackhole process lacks. Generally when a blackhole is sent to your provider, they in turn pass that on to the rest of their routers, dropping the traffic as soon as it hits their network. The traffic is still taking up just as much capacity up to that point. Were a system implemented as discussed, providers are able to prevent traffic that is known to be malicious from even exiting their network, which in the end works out better for everyone. -- Regards, Jake Mertel Nobis Technology Group, L.L.C. Web: http://www.nobistech.net/ Phone: (312) 281-5101 ext. 401 Fax: (808) 356-0417 Mail: 201 West Olive Street Second Floor, Suite 2B Bloomington, IL 61701 -----Original Message----- From: Christopher Morrow [mailto:morrowc.lists@gmail.com] Sent: Friday, February 13, 2009 1:59 PM To: NANOG list Subject: Re: Global Blackhole Service On Fri, Feb 13, 2009 at 1:04 PM, Jack Bates <jbates@brightok.net> wrote:
Paul Vixie wrote:
blackholing victims is an interesting economics proposition. you're saying the attacker must always win but that they must not be allowed to affect the infrastructure. and you're saying victims will request this, since they know they can't withstand the attack and don't want to be held responsible for damage to the infrastructure.
Blackholing victims is what is current practice. For each stage of affected
it is A current practice.. so is filtering, so is scrubbing... there is no one answer for this.
infrastructure, the business/provider will make requests to their peers to blackhole the victim IP to protect the bandwidth caps or router throughput caps.
or cause no one really cares about: your.mama.wears.combat.boots.tobed.com ... or other silly 95%-of attacked, things.
where you lose me is where "the attacker must always win".
Do you have a miraculous way to stop DDOS? Is there now a way to quickly and
There are purchasable answers to this problem... 3 (at least) providers in the US (and at least one now offers it globally) offer traffic scrubbing services. I know that one offers it at a very reasonable price even...
efficiently track down forged packets? Is there a remedy to shutting down
you can track streams of forged packets, but that's not super important here. Forged packets actually make this part of the problem (stopping the dos) easier, not harder.
the *known* botnets, not to mention the unknown ones?
there are lots of folks tracking and shutting down botnets, it's not horribly effective in stopping this sort of thing. I can vividly recall tracking down 4 nights in a row the same 'botnet' (same controller person, different C&C and mostly different bots) as they were being used to attack a customer of mine at the time. This with the cooperation of 2 other very large ISP's in the US and one vendor security team even. In the end though a simple scrubbing solution was deemed the simplest answer for all involved.
The attacker will always win if he has a large enough attack
For extreme cases this is true, but there are quite a lot of things on the spectrum which don't require super human efforts, and don't even require intervention from the ISP if proper precautions are taken at the outset. -chris
where you lose me is where "the attacker must always win".
Do you have a miraculous way to stop DDOS? Is there now a way to quickly and efficiently track down forged packets? Is there a remedy to shutting down the *known* botnets, not to mention the unknown ones?
there are no silver bullets. anyone who says otherwise is selling something.
The attacker will always win if he has a large enough attack platform/...
While all this is worked out, we have one solution we know works.
"we had to destroy the village in order to save it."
If we null route the victim IP, the traffic stops at the null route. Since most attackers don't care to DOS the ISP, but just to take care of that end point, they usually don't start shifting targets to try and keep the ISP itself out.
if you null route the victim IP, the victim is off the air, so the DDoS is a success even though it mostly does not reach its target. you're proposing that we lower an attacker's costs. in a war of economics that's bad juju, and all wars are about economics. there are no silver bullets. isp's who permit random source addresses on packets leaving their networks are creating a global hazard, and since they are defending their practices on the basis of thin profit margins it's right to call this "the chemical polluter business model." as long as the rest of us continue to peer with these chemical polluters, then anyone on the internet can be the victim of a devastating DDoS at any time and at low cost. that's not a silver bullet however. if most ISP's controlled their source addresses there would still be DDoS's and then the new problem would be lack of real-time cooperation along the lines of "hi i'm in the XYZ NOC and we're tracking a DDoS against one of our customers and 14% of it is coming from your address space, here's the summary of timestamp-ip-volume and here's a pointer to your share of the netflows, can you remediate?" the answer will start out just like today's BCP38 answer, no we can't afford the staff or technology to do that, and then lawyers would worry about liability, and we'd all have to worry about monopolies, censorship, social engineering, and so on. in all of these cases the problem is the margins themselves. just as the full cost of a fast food cheeseburger is probably about $20 if you count all the costs that the corporations are shifting onto society, so it is that the full cost of a 3MBit/sec DSL line is probably $300/month if you count all the costs that ISPs shift onto digital society. the usual argument goes (and i'm just putting it out here to save time, though i'm betting several respondants will not read closely and so will just spew this out as though it's their original idea and as though i had not dismissed it many times over the decades): "we cannot build a digital economy without cost shifting since noone would pay what it really costs during the rampup". i don't dignify that with a reply, either here in effigy, or if anyone happens to trot it out again.
FYI - I think Paul knows exactly what you are talking about. Hint - review the seminar: http://www.nanog.org/meetings/nanog36/abstracts.php?pt=Mzk5Jm5hbm9nMzY=&nm=n anog36
-----Original Message----- From: Jack Bates [mailto:jbates@brightok.net] Sent: Friday, February 13, 2009 9:23 AM To: Paul Vixie Cc: nanog@merit.edu Subject: Re: Global Blackhole Service
Paul Vixie wrote:
i think Spamhaus and Cymru are way ahead of you in implementing such a thing, and it's likely that there are even commercial alternatives to Trend Micro although i have not kept up on those details.
I think there's a misunderstanding from what I've read about what is being blackholed. We are not talking about blackholing the senders, but a massive scale method of blackholing the victims at the victim's request to protect infrastructure. Currently this type of service usually doesn't extend beyond one or two ASs and depending on traffic flows can still cause damage, especially through exchange points.
With enough support and use, this would allow a larger portion of bad traffic to be null routed closer to the sender origination points. Since the null routing BGP servers would expect a larger routing table from these /32 networks, they would be placed at key points capable of handling the larger tables; compared to just allowing the /32's out into the wild and possibly exceeding route/memory constraints.
It can also be used as authoritative information that an IP is undergoing a DOS attack, and large volumes of connections to that IP should be considered suspect. I consider this a much more useful method of detecting DOS traffic leaving your infected users than the emails which are usually sent out by those being hit by DOS.
Jack
participants (10)
-
Barry Raveendran Greene
-
Chris Jester
-
Christopher Morrow
-
Jack Bates
-
Jake Mertel
-
Jens Ott - PlusServer AG
-
Nuno Vieira - nfsi telecom
-
Paul Vixie
-
Randy Bush
-
Suresh Ramasubramanian