Re: BCP38 thread 93,871,738,435 (was Re: register.com down sev0?)
No. I think that is indicative of the problem. Don't you? - ferg -- Sean Donelan <sean@donelan.com> wrote: On Thu, 26 Oct 2006, Fergie wrote:
I don't want to detract from the heat of this discussion, as important as it is, but it (the discussion) illustrates a point that RIPE has recognized -- and is actively perusing -- yet, ISPs on this continent seem consistently to ignore: The consistent implementation of BCP 38.
It is nothing less than irresponsible, IMO...
Why _is_ that?
Do you have any data concerning the actual consistent deployment of BCP38++ in different parts of the world? -- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/
The only data I have is from the MIT anti-spoofing test project which has been pretty consistent for a long time. About 75%-80% of the nets, addressses, ASNs tests couldn't spoof, and about 20%-25% could. The geo-location maps don't show much difference between parts of the world. RIPE countries don't seem to be better or worse than ARIN countries or APNIC countries or so on. ISPs on every continent seem to be about the same. http://spoofer.csail.mit.edu/summary.php If someone finds the silver bullet that will change the remaining 25% or so of networks, I think ISPs on every continent would be interested. On Thu, 26 Oct 2006, Fergie wrote:
No.
I think that is indicative of the problem.
Don't you?
-- Sean Donelan <sean@donelan.com> wrote: On Thu, 26 Oct 2006, Fergie wrote:
I don't want to detract from the heat of this discussion, as important as it is, but it (the discussion) illustrates a point that RIPE has recognized -- and is actively perusing -- yet, ISPs on this continent seem consistently to ignore: The consistent implementation of BCP 38.
It is nothing less than irresponsible, IMO...
Why _is_ that?
Do you have any data concerning the actual consistent deployment of BCP38++ in different parts of the world?
On Thu, 2006-10-26 at 02:20 -0400, Sean Donelan wrote:
http://spoofer.csail.mit.edu/summary.php
If someone finds the silver bullet that will change the remaining 25% or so of networks, I think ISPs on every continent would be interested.
Financial incentive is the key. If there is none, those with the most to gain (backbone operators) also have to power to create such incentive. It wouldn't be fundamentally different from the basic network policing that happened on the academic networks which formed the Internet backbone in the 80s and early 90s. Keywords: ========= Work with OS and CPE vendors to include probes with equipment/software. Create lists of badly behaved prefixes. Drop offending prefixes from the DFZ. -- Result: BPC compliance or go the "scenic route" (bust). Problem solved .. move on. Problems: ========= Politics. Ill-informed politicians can come up with the most incredible excuses to protect offenders. Decide who define the criteria used to identify "offending networks", and administer the filtering recommendation. Has to tolerate some "collateral damage". Widespread misconception of an untouchable "public internet". Such a thing doesn't exist. The net still consist of interconnected privately owned networks within which the owner/operator is free to implement and enforce whatever policies they want. Some countries may require that customers/users are informed about the existence and consequences of such restrictions, but that shouldn't be much of a problem either. I'd be more than happy to tell anyone who object to BCP38 to look elsewhere for network connectivity. -- Per Heldal - http://heldal.eml.cc/
On Thu, 26 Oct 2006 02:20:48 -0400 (EDT), Sean Donelan <sean@donelan.com> wrote:
The only data I have is from the MIT anti-spoofing test project which has been pretty consistent for a long time. About 75%-80% of the nets, addressses, ASNs tests couldn't spoof, and about 20%-25% could.
The geo-location maps don't show much difference between parts of the world. RIPE countries don't seem to be better or worse than ARIN countries or APNIC countries or so on. ISPs on every continent seem to be about the same.
http://spoofer.csail.mit.edu/summary.php
If someone finds the silver bullet that will change the remaining 25% or so of networks, I think ISPs on every continent would be interested.
That would be nice -- but I wonder how much operational impact that would have. As you note, the 20-25% figure (of addresses) has been pretty constant for quite a while. Assuming that subverted machines are uniformly distributed (a big assumption) and assuming that their methodology is valid (another big assumption), that means we've already knocked out the 75-80% of the sources of spoofed IP address attacks. Has anyone seen a commensurate reduction in DDoS attacks? I sure haven't heard of that. Are people saying that the problem would be several times worse if anti-spoofing weren't in place? As best I can tell, the limiting factor on attack rates isn't the lack of sources but the lack of a profit motive for launching the attacks. Put another way, anti-spoofing does three things: it makes reflector attacks harder, it makes it easier to use ACLs to block sources, and it helps people track down the bot and notify the admin. Are people actually successfully doing either of the latter two? I'd be surprised if there were much of either. That leaves reflector attacks. Are those that large a portion of the attacks people are seeing? I agree that anti-spoofing is a good idea, and I've said so for a long time. I was one of the people who insisted that AT&T do it, way back when. But I'm not convinced it's a major factor here. --Steven M. Bellovin, http://www.cs.columbia.edu/~smb
* Steven M. Bellovin:
As you note, the 20-25% figure (of addresses) has been pretty constant for quite a while. Assuming that subverted machines are uniformly distributed (a big assumption)
I doubt this assumption about distribution is valid. At least over here, consumer-grade ISPs (think DSL with dynamic IP addresses) apply ingress filters, while real ISPs don't. If you're lucky, you get egress filters at some border routers, but it's not standard at all. Customer-facing interfaces are generally unfiltered. (But I have to admit that we recently ran into filters at an upstream's upstream, so there's at least some BCP 38 adoption.)
On Thu, 26 Oct 2006 17:07:32 +0200, Florian Weimer <fw@deneb.enyo.de> wrote:
* Steven M. Bellovin:
As you note, the 20-25% figure (of addresses) has been pretty constant for quite a while. Assuming that subverted machines are uniformly distributed (a big assumption)
I doubt this assumption about distribution is valid. At least over here, consumer-grade ISPs (think DSL with dynamic IP addresses) apply ingress filters, while real ISPs don't. If you're lucky, you get egress filters at some border routers, but it's not standard at all. Customer-facing interfaces are generally unfiltered.
Those are good points. It would be interesting to look at the raw AS data and see what classes of organizations were represented. Unfortunately, that data is not publicly available, according to the FAQ for that project. --Steven M. Bellovin, http://www.cs.columbia.edu/~smb
On Thu, 2006-10-26 at 13:03 -0400, Steven M. Bellovin wrote:
On Thu, 26 Oct 2006 17:07:32 +0200, Florian Weimer <fw@deneb.enyo.de> wrote:
* Steven M. Bellovin:
As you note, the 20-25% figure (of addresses) has been pretty constant for quite a while. Assuming that subverted machines are uniformly distributed (a big assumption)
I doubt this assumption about distribution is valid. At least over here, consumer-grade ISPs (think DSL with dynamic IP addresses) apply ingress filters, while real ISPs don't. If you're lucky, you get egress filters at some border routers, but it's not standard at all. Customer-facing interfaces are generally unfiltered.
Those are good points. It would be interesting to look at the raw AS data and see what classes of organizations were represented. Unfortunately, that data is not publicly available, according to the FAQ for that project.
Microsoft no longer demands licensing for Sender-ID. Executing SPF scripts may become increasingly popular for validating SMTP clients against MAILFROM, PRA, or perhaps DKIM domains. SPF scripts can be far more malicious than even spoofed source reflective DNS attacks. Dozens of TXT or hundreds of A record transactions might be invoked by the recipients of each message. In addition to MTAs processing SPF script, popular applications like SpamAssassin executes a different script after an elapsed timeout of 5 seconds. Such short timeouts eliminate congestion avoidance. SPF is analogous to a complex lock where there is no obvious place for a key. One must trust a stranger to operate their own unique DNS based mechanism. The macro capability of this script also allows any element of an email-address to randomize DNS queries against any unseen victim domain. SMTP logs may not indicate who initiated an attack, nor indicate when an attack was in progress. BCP38 filtering or ACLs on recursive DNS will not offer protection from this exploit, which can achieve gains much higher than spoofed source reflective DNS attacks. Spam being sent through Bot farms has already set the stage for untraceable DNS attacks based upon SPF. In addition to taking out major interconnects, these attacks can: a) inundate authoritative DNS; b) requests A records from anywhere; c) probe IP address, port, and the transaction IDs of resolvers; While not as bad as eavesdropping, it still places the network and the integrity of DNS at risk. All of this while the spam is still being delivered. What a productivity tool! How is this attack detected? How is this attack avoided? -Doug
How is this attack avoided?
Sounds like the attack is inherent in SPF. In that case, avoiding it is simple. Discourage the use of SPF, perhaps by putting any SPF using domain into a blacklist. Eventually, people will stop using SPF and the attack vector goes away. --Michael Dillon
On Fri, 27 Oct 2006 Michael.Dillon@btradianz.com wrote:
How is this attack avoided?
Sounds like the attack is inherent in SPF. In that case,
how did the thread about dns providers and rfc compliance morph into SPF and spam discussions?
How is this attack avoided?
Sounds like the attack is inherent in SPF. In that case,
how did the thread about dns providers and rfc compliance morph into SPF and spam discussions?
Ask Doug Otis. He stated that SPF sets the stage for DDoS attacks against DNS servers. Presumably he said this because it points to another *COST* of DDoS that could be used as a business justification to implement BCP38. Or you could look at it as a weakness of SPF that should be used as a justification for discouraging its use. After all if we discourage botnets because they are DDoS enablers, shouldn't we discourage other DDoS enablers like SPF? --Michael Dillon
On Fri, 27 Oct 2006 Michael.Dillon@btradianz.com wrote:
Or you could look at it as a weakness of SPF that should be used as a justification for discouraging its use. After all if we discourage botnets because they are DDoS enablers, shouldn't we discourage other DDoS enablers like SPF?
under this assumption we should discourage user use of the internet... :( anyway, please let's get back to the original discussion :)
On Oct 27, 2006, at 10:03 AM, Chris L. Morrow wrote:
On Fri, 27 Oct 2006 Michael.Dillon@btradianz.com wrote:
Or you could look at it as a weakness of SPF that should be used as a justification for discouraging its use. After all if we discourage botnets because they are DDoS enablers, shouldn't we discourage other DDoS enablers like SPF?
under this assumption we should discourage user use of the internet... :( anyway, please let's get back to the original discussion :)
As Steve already pointed out, BCP38 is not a complete solution. Not only does SPF prevent the source of a Botnet attack from being detected, it also enables significantly greater amplification than might be achieved with a spoofed source DNS reflective attack. In addition, the Botnet resources are not wasted, as their spam is still being delivered. This aspect alone dangerously changes the costs related to such attacks. It seems wholly imprudent not to consider SPF in the same discussion. -Doug
On Fri, 27 Oct 2006, Douglas Otis wrote:
As Steve already pointed out, BCP38 is not a complete solution. Not only does SPF prevent the source of a Botnet attack from being detected, it also enables significantly greater amplification than might be achieved with a spoofed source DNS reflective attack. In addition, the Botnet resources are not wasted, as their spam is still being delivered. This aspect alone dangerously changes the costs related to such attacks. It seems wholly imprudent not to consider SPF in the same discussion.
-Doug
Doug, I wonder, HOW do you intend / do track down the source of a botnet attack? I know how I and others do it. There are three approaches which fork everywhere on an expression tree. If you believe SPF prevents you from doing it, can you elaborate how?
On Sat, 2006-10-28 at 00:52 -0500, Gadi Evron wrote:
If you believe SPF prevents you from doing it, can you elaborate how?
From a victim's perspective, there could be tens or hundreds of
Spam referencing malicious SPF scripts can result in PASS or NEUTRAL, where the message and message rates may be normal. Recipients will not notice the role they are playing in an ongoing attack. There would be few clues, and BCP38 or ACLs will not prevent an SPF attack. thousands of attack sources. No source represents an address of a Botnet. An attack could be composed of A-record transactions for hosts not seen in any message, or related to the domains of any SPF script. These SPF scripts might also later morph to frustrate forensic analysis or real-time blocking. SPF scripts add indirection from what is within a message. An attacking transaction would pass through DNS from one of the hundred thousand recipients. Finding a recipient will not link a DNS transaction to a message. The source of the message may also be a reputable provider. The recipient would need to trace the targets of all associated SPF scripts. A particular SPF script might be one of a hundred other scripts targeting the same victim, however. Analysis designed not to add to an attack can also be seen by the attacker. Nothing in the experimental SPF or Sender-ID RFCs explain how such catastrophic attacks are avoided. Their recommended premature termination of SPF scripts ensures there is no congestion avoidance as well. How would you identify and quell an SPF attack in progress? -Doug
On Sun, 29 Oct 2006, Douglas Otis wrote:
On Sat, 2006-10-28 at 00:52 -0500, Gadi Evron wrote:
If you believe SPF prevents you from doing it, can you elaborate how?
Spam referencing malicious SPF scripts can result in PASS or NEUTRAL, where the message and message rates may be normal. Recipients will not notice the role they are playing in an ongoing attack. There would be few clues, and BCP38 or ACLs will not prevent an SPF attack.
From a victim's perspective, there could be tens or hundreds of thousands of attack sources. No source represents an address of a Botnet. An attack could be composed of A-record transactions for hosts not seen in any message, or related to the domains of any SPF script. These SPF scripts might also later morph to frustrate forensic analysis or real-time blocking.
SPF scripts add indirection from what is within a message. An attacking transaction would pass through DNS from one of the hundred thousand recipients. Finding a recipient will not link a DNS transaction to a message. The source of the message may also be a reputable provider. The recipient would need to trace the targets of all associated SPF scripts. A particular SPF script might be one of a hundred other scripts targeting the same victim, however. Analysis designed not to add to an attack can also be seen by the attacker.
Nothing in the experimental SPF or Sender-ID RFCs explain how such catastrophic attacks are avoided. Their recommended premature termination of SPF scripts ensures there is no congestion avoidance as well.
How would you identify and quell an SPF attack in progress?
Okay, now I understand. You speak of an attack specfically utilizing SPF, not of how SPF relates to botnets or attack traceback. The same could be said for web servers, databases behind them, DNS-SEC crypto calculations, etc.
-Doug
Gadi.
On Sun, 2006-10-29 at 09:40 -0600, Gadi Evron wrote:
On Sun, 29 Oct 2006, Douglas Otis wrote:
How would you identify and quell an SPF attack in progress?
Okay, now I understand.
You speak of an attack specifically utilizing SPF, not of how SPF relates to botnets or attack traceback.
The same could be said for web servers, databases behind them, DNS-SEC crypto calculations, etc.
The described indirect SPF attack does not utilize packet source spoofing, and yet may achieve amplifications greater than 1000:1. The resources to stage an SPF attack would be the ever present spam, where about 70% this is coming from Botnets. In the case of spam related SPF, the attack itself can be virtually free. While also consuming an attacker's resources, a DNS reflective attack with spoofed source packets represents a far lower impact when compared to the SPF attack. SPF represents a grave danger without means for mitigation. The same can not be said for these other protocols. -Doug
On Sun, 29 Oct 2006, Douglas Otis wrote:
On Sun, 2006-10-29 at 09:40 -0600, Gadi Evron wrote:
On Sun, 29 Oct 2006, Douglas Otis wrote:
How would you identify and quell an SPF attack in progress?
Okay, now I understand.
You speak of an attack specifically utilizing SPF, not of how SPF relates to botnets or attack traceback.
The same could be said for web servers, databases behind them, DNS-SEC crypto calculations, etc.
The described indirect SPF attack does not utilize packet source spoofing, and yet may achieve amplifications greater than 1000:1. The resources to stage an SPF attack would be the ever present spam, where about 70% this is coming from Botnets. In the case of spam related SPF, the attack itself can be virtually free.
While also consuming an attacker's resources, a DNS reflective attack with spoofed source packets represents a far lower impact when compared to the SPF attack. SPF represents a grave danger without means for mitigation. The same can not be said for these other protocols.
There's a lot that can be done with DDoS techonology and amplification that has not yet been done. You are 100% right. There is even more that can be done with current technology. If it takes 200 or so bots to generate ~10Gbps traffic using DNS amplification... 'New' ideas should remain quiet, thing is, they remain quiet and the bad guys are all over them, long after this silence is harmful.
-Doug
Gadi.
how did the thread about dns providers and rfc compliance morph into SPF and spam discussions?
for the spf hammerers, everything looks like a nail? :) personally, i think it is overloading of mpls, dns, and bgp. :) randy
* Douglas Otis:
Spam being sent through Bot farms has already set the stage for untraceable DNS attacks based upon SPF. In addition to taking out major interconnects, these attacks can:
a) inundate authoritative DNS;
b) requests A records from anywhere;
c) probe IP address, port, and the transaction IDs of resolvers;
(b) and (c) are not new developments because lots of MTAs already perform A lookups on HELO arguments, and MX lookups on sender domains.
While not as bad as eavesdropping, it still places the network and the integrity of DNS at risk. All of this while the spam is still being delivered. What a productivity tool!
The purpose of SPF, as it is deployed, is to facilitate routing solicited bulk email around spam filters. Look at email.bn.com/IN/TXT to get the idea. This application requires some of the indirection features offered by SPF.
On Fri, 2006-10-27 at 14:11 +0200, Florian Weimer wrote:
* Douglas Otis:
Spam being sent through Bot farms has already set the stage for untraceable DNS attacks based upon SPF. In addition to taking out major interconnects, these attacks can:
a) inundate authoritative DNS;
b) requests A records from anywhere;
c) probe IP address, port, and the transaction IDs of resolvers;
(b) and (c) are not new developments because lots of MTAs already perform A lookups on HELO arguments, and MX lookups on sender domains.
Each message's SPF script can prompt for web-site addresses while also inundating the web-site's DNS with other randomized requests. Network gains achieved by each script can reach 70:1, and when instances of execution (MTA/MUA, MAILFROM/PRA/DKIM, and recipient) are considered, gains per message may exceed 1000:1 while still permitting delivery and while not exposing who their victim was.
While not as bad as eavesdropping, it still places the network and the integrity of DNS at risk. All of this while the spam is still being delivered. What a productivity tool!
The purpose of SPF, as it is deployed, is to facilitate routing solicited bulk email around spam filters. Look at email.bn.com/IN/TXT to get the idea. This application requires some of the indirection features offered by SPF.
The risk is from an amplification achieved by SPF scripts while still hiding which messages are attacking. Bulk senders can use APL RRs (42) (see rfc3123) to list the CIDRs of their SMTP clients without imposing these risks. Standardized prefixes such as _smtp-c0 and _smtp-c1 permits chaining signaled with a "continuation" address-family, as example. Executing powerful SPF scripts from strangers is a heavily promoted bad idea that truly needs to be discouraged. -Doug
On Oct 26, 2006, at 9:33 AM, Steven M. Bellovin wrote:
Put another way, anti-spoofing does three things: it makes reflector attacks harder, it makes it easier to use ACLs to block sources, and it helps people track down the bot and notify the admin. Are people actually successfully doing either of the latter two? I'd be surprised if there were much of either. That leaves reflector attacks. Are those that large a portion of the attacks people are seeing?
I disagree. As someone who has been attacked by spoof-source packets, and not-spoof-source packed, I can say, from personal experience, that the former is much, much easier to mitigate. And, as I posted before, even if all universal adoption of BCP38 means is that DDoS attacks move to botnets with 100% real source IP addresses, that would still be a Very Good Thing, IMHO. But perhaps others feel differently. Or perhaps they just haven't been attacked enough. :) -- TTFN, patrick
Put another way, anti-spoofing does three things: it makes reflector attacks harder, it makes it easier to use ACLs to block sources, and it helps people track down the bot and notify the admin. Are people actually successfully doing either of the latter two? I think it's a time constraint- looking up, sorting and notifying admins about 10,000 attack sources isn't practical. I'd love to do it- but I don't have time. That said- if someone notifies me of a compromised host I immediately investigate- and I suspect so would everyone else on this
list. Has anyone put together a centralized system where you can send in a list of attacking bots, let it automatically sort by allocation, and then let it notify the appropriate admin with a list of [potentially] compromised hosts? Then again: Considering how many admins don't care, how many end users don't care/know, and how quickly many of thee systems would get re-infected maybe it's all a bit pointless.
I'd be surprised if there were much of either. That leaves reflector attacks. Are those that large a portion of the attacks people are seeing? Everything I have seen of late has been legitimate traffic originating from across the globe. With tens of thousands of compromised hosts that's all it takes.
-Don
On Thu, 26 Oct 2006, Don wrote:
Has anyone put together a centralized system where you can send in a list of attacking bots, let it automatically sort by allocation, and then let it notify the appropriate admin with a list of [potentially] compromised hosts?
mynetwatchman [1] comes to mind and so does dshield [2] [1] http://www.mynetwatchman.com [2] http://www.dshield.org -- William Leibzon Elan Networks william@elan.net
----- Original Message ----- From: "william(at)elan.net" <william@elan.net> To: "Don" <don@calis.blacksun.org> Cc: <nanog@merit.edu> Sent: Thursday, October 26, 2006 8:17 AM Subject: Re: BCP38 thread 93,871,738,435 (was Re: register.com down sev0?)
On Thu, 26 Oct 2006, Don wrote:
Has anyone put together a centralized system where you can send in a list of attacking bots, let it automatically sort by allocation, and then let it notify the appropriate admin with a list of [potentially] compromised hosts?
mynetwatchman [1] comes to mind and so does dshield [2]
[1] http://www.mynetwatchman.com [2] http://www.dshield.org
-- William Leibzon Elan Networks william@elan.net
Anyone familiar with these folks? http://www.simplicita.com/Simplicita_Research_Data_Partner_Program.html --Michael
participants (14)
-
Chris L. Morrow -
Don -
Douglas Otis -
Fergie -
Florian Weimer -
Gadi Evron -
Michael Painter -
Michael.Dillon＠btradianz.com -
Patrick W. Gilmore -
Per Heldal -
Randy Bush -
Sean Donelan -
Steven M. Bellovin -
william(at)elan.net