Randy, I don't think I implied anything of the sort. I did, however, pipe up when a BCP is mentioned that I endorse, and co-authored -- and likewise, cannot figure out for life of me, why there is such push-back from the Ops community on doing The Right Thing. Having said that, botnets don't need to spoof addresses -- the sheer dispersion of geographic and AS infection base renders the whole point of spoofing almost moot. And having said that, it doesn't make BCP 38 any less valid. - ferg -- Randy Bush <randy@psg.com> wrote:
I don't want to detract from the heat of this discussion, as important as it is, but it (the discussion) illustrates a point that RIPE has recognized -- and is actively perusing -- yet, ISPs on this continent seem consistently to ignore: The consistent implementation of BCP 38.
oh? you have knowledge that this botnet attack used spoofed source addresses? randy -- "Fergie", a.k.a. Paul Ferguson Engineering Architecture for the Internet fergdawg(at)netzero.net ferg's tech blog: http://fergdawg.blogspot.com/
On Thu, Oct 26, 2006 at 06:03:54AM +0000, Fergie wrote:
Randy,
I don't think I implied anything of the sort.
I did, however, pipe up when a BCP is mentioned that I endorse, and co-authored -- and likewise, cannot figure out for life of me, why there is such push-back from the Ops community on doing The Right Thing.
The challenge is that the router vendors still haven't done "The Right Thing". I have one device that 1) halves its forwarding table space by enabling u-rpf 2) can only do either strict or loose mode rpf *GLOBALLY* so I can not strict rpf-check a static customer AND loose rpf someone larger for unrouted space. because of the above (#1 isn't that bad, but #2 is) I can't enable u-rpf on the device as a policy. Changing one interface from loose -> strict silently changes all other u-rpf interfaces and then customers gripe about dropped packets. obviously moving these checks closer to the edge is ideal, such as always doing rpf on the ethernet lan interface for your customer CPE.
Having said that, botnets don't need to spoof addresses -- the sheer dispersion of geographic and AS infection base renders the whole point of spoofing almost moot.
yup, it's an evolving threat, even if some solution to the botnet problem is discovered, it will take years to fix. Think of the smurf amplifiers that are still out there[1]. - jared 1 - http://www.powertech.no/smurf/ -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.
At 07:25 AM 10/26/2006, Jared Mauch wrote:
On Thu, Oct 26, 2006 at 06:03:54AM +0000, Fergie wrote:
Randy,
I don't think I implied anything of the sort.
I did, however, pipe up when a BCP is mentioned that I endorse, and co-authored -- and likewise, cannot figure out for life of me, why there is such push-back from the Ops community on doing The Right Thing.
The challenge is that the router vendors still haven't done "The Right Thing".
I have one device that
1) halves its forwarding table space by enabling u-rpf 2) can only do either strict or loose mode rpf *GLOBALLY* so I can not strict rpf-check a static customer AND loose rpf someone larger for unrouted space.
It was possible to implement BCP38 before the router vendors came up with uRPF.
because of the above (#1 isn't that bad, but #2 is) I can't enable u-rpf on the device as a policy. Changing one interface from loose -> strict silently changes all other u-rpf interfaces and then customers gripe about dropped packets.
obviously moving these checks closer to the edge is ideal, such as always doing rpf on the ethernet lan interface for your customer CPE.
Yes, it is. And does not require uRPF. I know you're looking to do the right thing. It's important though that this not be put entirely on the router vendors. How many "managed T1" services out there have routers controlled by the ISP providing them? How many of those routers are configured with a single line ACL that would implement BCP38 sufficiently? How many aggregation routers for incoming T1s are not configured with a single line ACL per T-1 to ensure the packets coming in are from assigned, not-multihomed space? If scripts are being used to auto-configure routers to ship out to T-1 customers, then appropriate ACLs should be written by such scripts at the same time. Scripts that configure aggregation switches should similarly be reviewed for ACL inclusion. It's certainly helpful to have implementations such as uRPF to help make it easier to deploy BCP38, but deployment of BCP38 is not dependent on the existence of uRPF.
Having said that, botnets don't need to spoof addresses -- the sheer dispersion of geographic and AS infection base renders the whole point of spoofing almost moot.
yup, it's an evolving threat, even if some solution to the botnet problem is discovered, it will take years to fix. Think of the smurf amplifiers that are still out there[1].
Dan (the other co-author of the BCP in question)
It was possible to implement BCP38 before the router vendors came up with uRPF.
Further, uRPF is frequently a very inefficient means of implementing BCP 38. Consider that you're going to either compare the source address against a table of 200,000 routes or against a handful of prefixes that you've statically configured in an ACL. Yes, I realize that the latter approach is more of a managerial hassle, but for those of you who feel that your silicon is running a tad too warm, you may wish to consider this as a possible performance improvement technique. YMMV. Your former router vendor, Tony
On Thu, 26 Oct 2006, Tony Li wrote:
It was possible to implement BCP38 before the router vendors came up with uRPF.
Further, uRPF is frequently a very inefficient means of implementing BCP 38. Consider that you're going to either compare the source address against a table of 200,000 routes or against a handful of prefixes that you've statically configured in an ACL.
Isn't that only a problem if you want to run a loose mode uRPF? Given that loose mode uRPF isn't very useful in most places where you'd like to do ingress filtering, this doesn't seem like a big issue.. BTW, I still keep wondering why Cisco hasn't implemented something like Juniper's feasible-path strict uRPF. Works quite well with multihomed and asymmetric routing as well -- no need to fiddle with communities, BGP weights etc. to ensure symmetry. -- Pekka Savola "You each name yourselves king, yet the Netcore Oy kingdom bleeds." Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
Pekka Savola wrote: > On Thu, 26 Oct 2006, Tony Li wrote: >>> It was possible to implement BCP38 before the router vendors >>> came up with uRPF. >> Further, uRPF is frequently a very inefficient means of implementing BCP >> 38. Consider that you're going to either compare the source address >> against a table of 200,000 routes or against a handful of prefixes that >> you've statically configured in an ACL. > > Isn't that only a problem if you want to run a loose mode uRPF? > Given that loose mode uRPF isn't very useful in most places where > you'd like to do ingress filtering, this doesn't seem like a big > issue.. Strict mode uRPF is likely to be implemented by performing a full forwarding table lookup and then comparing the packet's incoming interface to the interface from the forwarding table result. Tony
On Fri, 27 Oct 2006, Tony Li wrote: > Pekka Savola wrote: > > On Thu, 26 Oct 2006, Tony Li wrote: > >>> It was possible to implement BCP38 before the router vendors > >>> came up with uRPF. > >> Further, uRPF is frequently a very inefficient means of implementing BCP > >> 38. Consider that you're going to either compare the source address > >> against a table of 200,000 routes or against a handful of prefixes that > >> you've statically configured in an ACL. > > > > Isn't that only a problem if you want to run a loose mode uRPF? > > Given that loose mode uRPF isn't very useful in most places where > > you'd like to do ingress filtering, this doesn't seem like a big > > issue.. > > Strict mode uRPF is likely to be implemented by performing a full > forwarding table lookup and then comparing the packet's incoming > interface to the interface from the forwarding table result. Pekka might have meant wouldn't you build a seperate 'urpf table' per interface perhaps? (just guessing at his intent) though there is only one 'urpf table' which is the fib, right?
On Thu, 26 Oct 2006, Tony Li wrote:
It was possible to implement BCP38 before the router vendors came up with uRPF.
Further, uRPF is frequently a very inefficient means of implementing BCP 38. Consider that you're going to either compare the source address against a table of 200,000 routes or against a handful of prefixes that you've statically configured in an ACL.
Yes, I realize that the latter approach is more of a managerial hassle, but for those of you who feel that your silicon is running a tad too warm, you may wish to consider this as a possible performance improvement technique. YMMV.
Your former router vendor, Tony
Erm, most ISP's I talk to (since I became aware of this not too long ago) believe this is a perfect replacement for BCP38. And yet, spoofing is possible from their space. Gadi.
On Thu, 2006-10-26 at 06:03 +0000, Fergie wrote:
Having said that, botnets don't need to spoof addresses -- the sheer dispersion of geographic and AS infection base renders the whole point of spoofing almost moot.
A lot of new possibilities arise if spoofing can be eliminated with near 100% certainty. Some examples: Automated filtering. Automated notification to providers. "Cut off host X or..." Expose compromised systems and hold their owners financially responsible for damages. Severe punishment of large number of users may cause outrage, basis for regress, class-action lawsuits, and maybe finally turn the attention to the real source of the problem; software vendors whose products are of such a dismal quality that they'd be banned worldwide from just about any market other than that for computer software. -- Per Heldal - http://heldal.eml.cc/
I don't think I implied anything of the sort.
ahhh, but you did.
I don't want to detract from the heat of this discussion, as important as it is, but it (the discussion) illustrates a point that RIPE has recognized -- and is actively perusing -- yet, ISPs on this continent seem consistently to ignore: The consistent implementation of BCP 38.
oh? you have knowledge that this botnet attack used spoofed source addresses?
if the register.com botnet attack was not from spoofed addresses, then bcp 38 would not have helped. the case for which we know bcp 38 is useful, is the dns reflector attack. so far, botnets seem to have no need to spoof, they just overwhelm you with zombies from real space. randy
On Thu, 26 Oct 2006, Randy Bush wrote:
I don't think I implied anything of the sort.
ahhh, but you did.
I don't want to detract from the heat of this discussion, as important as it is, but it (the discussion) illustrates a point that RIPE has recognized -- and is actively perusing -- yet, ISPs on this continent seem consistently to ignore: The consistent implementation of BCP 38.
oh? you have knowledge that this botnet attack used spoofed source addresses?
if the register.com botnet attack was not from spoofed addresses, then bcp 38 would not have helped.
the case for which we know bcp 38 is useful, is the dns reflector attack. so far, botnets seem to have no need to spoof, they just overwhelm you with zombies from real space.
And yet they do anyway. Before the "reflector attacks" run at the beginning of this year, you stated you do not see the need to deal with spoofing, as it is not something being exploited. It is being exploited, let's deal with it. Gadi.
randy
On Oct 26, 2006, at 11:24 AM, Randy Bush wrote:
the case for which we know bcp 38 is useful, is the dns reflector attack. so far, botnets seem to have no need to spoof, they just overwhelm you with zombies from real space.
Incorrect. While that is one mode of attack from a botnet, it is not the only mode. And there are reasons for even botnets to spoof source addresses. And reasons that the attack-ee would prefer they did not. Randy, are you REALLY arguing -against- BCP38? Or just yanking Fergie's chain 'cause it wouldn't have helped in this particular instance? -- TTFN, patrick
the case for which we know bcp 38 is useful, is the dns reflector attack. so far, botnets seem to have no need to spoof, they just overwhelm you with zombies from real space.
Incorrect.
While that is one mode of attack from a botnet, it is not the only mode. And there are reasons for even botnets to spoof source addresses. And reasons that the attack-ee would prefer they did not.
Randy, are you REALLY arguing -against- BCP38? Or just yanking Fergie's chain 'cause it wouldn't have helped in this particular instance?
i merely said that using this particular attack to launch yet another bcp38 religious dos against the nanog list was bogus. have we learned one new thing from the last day's oratory? personally, i long ago implemented spoofing blocking in all places i have been able to do so. but i am not foolish enough to believe that religious ranting on mailing lists is gonna change anyone from doing what makes business sense for their network. and, as spoofed attacks other than the dns reflector seem to have been rare, that perceived interest in anti-spoofing blocks is low when compared to other priorities in these hard times. i think we have converted those who were convertable and the rest watch the religious zealotry and scratch their heads. randy
but i am not foolish enough to believe that religious ranting on mailing lists is gonna change anyone from doing what makes business sense for their network.
Indeed! And it is not going to change the minds of the majority of network operations folks who are not on the NANOG list nor the majority of telecoms executives who are also not on the NANOG list. Back in the old days, the NANOG list did hold the majority of Internet operations folks so new ideas like flap dampening were able to spread quickly. But those days are long gone. NANOG still has an important educational role but it is no longer based on being part of the old boys club and knowing the secret handshake. In other words, there is no cohesive society of network operators which can be swayed by attempts at social engineering like shaming or cajoling. BCP 38 has had its day. Nowadays, it is more important to look at how to mitigate current DDoS techniques and to describe the larger problem and look for larger solutions. However, any attempt at larger solutions require a large amount of humility because nobody can say for sure, what will work and what won't. The fact remains that there is not a good technical method for mitigating large scale distributed DDoS that results in LARGE TRAFFIC FLOWS ENTERING A NETWORK FROM ALL PEERED ASES SIMULTANEOUSLY. Perhaps if we could find a way to allow the attacked AS to set ACLs automatically in all the source AS networks, that would help mitigate these attacks. For instance, consider a set of ASes which all install an ACL-setter box. These boxes all trust each other to send-receive ACL setting requests through a trusted channel. The owner of a box sets some limits on the ACLs that can be set, for instance n ACLs per AS, max ACL lifetime, etc. And the box owner also decides the subset of their routers which will accept an ACL for a given address range. Then when an attack comes in, the victim AS uses some tool to identify large sources, i.e. a CIDR block that covers some significant percentage of the source addresses in one AS. They then issue an ACL request to that AS to block the flow and the ACL takes effect almost instantaneously with no human intervention. Yes, this can result in some IP addresses being blocked unfairly, but the DDoS traffic levels often have the same impact. In any case, the AS holding the destination address is the one doing the blocking even though the mechanism is an ACL inside the source AS network. On the technical side, it is not a complex problem to put such a system in place. The complexity is largely in getting network operators to come to an agreement on the terms under which operator A will allow operator B to set ACLs in operator A's network. Until network operators see DDoS as a significant business problem, this will not happen. Note that a "business problem" does not refer solely to the direct costs of mitigating a DDoS attack. It also includes the indirect fallout which is harder to measure such as loss of goodwill, missed opportunities, etc. --Michael Dillon
On Thu, 26 Oct 2006, Fergie wrote:
and co-authored -- and likewise, cannot figure out for life of me, why there is such push-back from the Ops community on doing The Right Thing.
you could google answers from other folks but in shor: 1) it doesn't always work as advertised 2) people don't always tell you the routes the hold 3) equipment vendors don't alway splan properly for 'features' Not everyone is as smart as you (both) and can manage that problem as they scale...
participants (12)
-
Chris L. Morrow -
Daniel Senie -
Fergie -
Gadi Evron -
Jared Mauch -
Michael.Dillonļ¼ btradianz.com -
Patrick W. Gilmore -
Pekka Savola -
Per Heldal -
Randy Bush -
Tony Li -
Tony Li