I guess the 'filter box' can be a number of different things, based on your needs and preferences. At first glance CloudShield seems pretty beefy, am reading more as we speak. Yes, Juniper can be convinced to add things, we've asked for a few. ;) Part of the problem with asking for new things on an ASIC, takes time. Anything they add in their code to help filter will likely not be done in hardware, meaning potential impact. I know some people need to filter on their routers for various reasons, but my thoughts are to minimize this. A router that is working hard at just forwarding packets doesn't need to extra overhead of looking deep into packet headers to figure out what to do with packets. Juniper is better at this, as are some Cisco products, but the GSR is a crappy packet filter if you put enough traffic through it. Yes certain linecards are better than others, but the newer they are the more buggy they are, and we're talking HW here, so bug fixes will be awhile. The caveat to all this is obvious, these are big links and big routers, only applies to high traffic networks..as that is the only place the expense is justified. uRPF, however, works on (almost) any CEF capable Cisco router. Some may have already seen this, worth a look if not. http://www.cisco.com/warp/public/707/newsflash.html There are some limitations as to where uRPF works, SONET only on GSRs for example (thanks Cisco). I believe it will work on 65xx (SUP1A and SUP2 I think) regardless of interface type. Impact should be minimal, as it simply does a lookup in the CEF table, if the route isn't there it discards. Keep in mind this is NOT a filter, so the impact is much less, it is simply a CEF lookup, much more efficient than a filter. This will get rid of a HUGE percentage of spoofed packets that hit your network, and would also work pretty well if you are the source of an attack. There is some debate as to whether you must not have ANY RFC1918 space for this to work. We're trying to find this out (not a priority), if I get info I'll post. -----Original Message----- From: Richard A Steenbergen [mailto:ras@e-gerbil.net] Sent: Thursday, May 02, 2002 9:23 AM To: LeBlanc, Jason Cc: 'Pete Kruckenberg'; nanog@merit.edu Subject: Re: Effective ways to deal with DDoS attacks? On Thu, May 02, 2002 at 08:54:05AM -0700, LeBlanc, Jason wrote:
uRPF and Radware DoShield, one DoShield per link btw edge router and core router. Use IDS (yes there is a way to capture all your traffic and anaylyze it, regardless of bandwidth, no it isn't one box) to identify a signature, build a filter, config filter on DoShield, up to ~200Mb/s per DoShield of filtering with zero impact to legit traffic. Scale
horizontally
(add more links each with a DoShield on it) based on your ingress traffic.
I've seen many suggestions on this list, this is the only thing that works for huge (100Mb/s+) attacks. eBay is likely the biggest target on the net, this works for us 90% of the time. Is the HW expensive? Yes. (~$35k per DoShield I think) It works, it scales.
You might want to take a look at CloudShield (www.cloudshield.com), they have what can only be described as a pretty impressive looking box for this kind of stuff.
There is no way a Cisco router can handle filtering attacks past a certain point, nor is it capable of even filtering on some patterns in attack packets. Juniper is better, but still lacks some filtering capabilities. Your router is a router, not a packet filter, give up trying to make it do this until someone builds this into an ASIC on the router.
Thats what the IP2 does, match bytes in the headers and come back with a thumbs down or a thumbs up and a destination interface. It's really not that much harder to match the bytes for a dest port against a compiled ruleset and decide yes or no then it is to match the dest address against a forwarding table and decide which nexthop. They CAN filter on anything in the headers, it's just a matter of convincing them that the specific filter you want is something they should add to their software language and microcode. I'm sure as a core router vendor they must hear every feature request imaginable and not know which ones to follow up on. If anyone from Juniper is listening, I can tell you 4 things to add which will stop all existing packet kiddie tools in their tracks. But then again, I'd rather just have a language for bitmatching at any offset. :) -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
On Thu, May 02, 2002 at 09:41:33AM -0700, LeBlanc, Jason wrote:
Yes, Juniper can be convinced to add things, we've asked for a few. ;) Part of the problem with asking for new things on an ASIC, takes time. Anything they add in their code to help filter will likely not be done in hardware, meaning potential impact. I know some people need to filter on their routers for various reasons, but my thoughts are to minimize this. A router that is working hard at just forwarding packets doesn't need to extra overhead of looking deep into packet headers to figure out what to do with packets. Juniper is better at this, as are some Cisco products, but the GSR is a crappy packet filter if you put enough traffic through it. Yes certain linecards are better than others, but the newer they are the more buggy they are, and we're talking HW here, so bug fixes will be awhile.
I think you're misunderstanding how this works. http://www.juniper.net/news/features/ipii/faq_ip2.html http://www.juniper.net/techcenter/techpapers/200015-03.html 3. How does the Internet Processor II ASIC enable service providers to upgrade functionality without upgrading hardware? Essentially, the Internet Processor II ASIC contains logic that implements a number of lookup algorithms, including trees, tables, firewall programs, and a way to chain those individual lookups together in an arbitrary sequence. The final answer to an entire lookup, then, is the result of all the matches that were run. By implementing complex lookups as a series of fundamental primitives, the ASIC can support almost anything for which an application can be described. Since the ASIC implementation is so general, new functionality can be enabled in JUNOS software upgrades without having to swap hardware. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
On Thu, 2 May 2002, LeBlanc, Jason wrote:
There are some limitations as to where uRPF works, SONET only on GSRs for example (thanks Cisco). I believe it will work on 65xx (SUP1A and SUP2 I think) regardless of interface type. Impact should be minimal, as it simply does a lookup in the CEF table, if the route isn't there it discards.
It can do much more. You can use it to get rid spoofed source addresses from customers and peers without the need to maintain large access lists.
lookup, much more efficient than a filter. This will get rid of a HUGE percentage of spoofed packets that hit your network,
If you just filter out anything that's not in the routing table, that's about half the address space and it only works if the spoofers are stupid. When you're looking at pure bandwidth that's still helpful, but it doesn't really solve anything. However, You can use unicast RPF as a very efficient source address filter, by routing addresses to the null interface. This way you can get rid of huge amounts of unwanted sources in a very clean way. As long as we're asking for features: what I would like is a unicast RPF check that allows everything that isn't routed to the null interface. And of course unicast RPF period for all vendors who aren't Cisco.
On Thu, May 02, 2002 at 09:41:33AM -0700, LeBlanc, Jason wrote something like this: <snip>
There are some limitations as to where uRPF works, SONET only on GSRs for example (thanks Cisco). I believe it will work on 65xx (SUP1A and SUP2 I think) regardless of interface type. Impact should be minimal, as it simply does a lookup in the CEF table, if the route isn't there it discards. Keep in mind this is NOT a filter, so the impact is much less, it is simply a CEF lookup, much more efficient than a filter. This will get rid of a HUGE percentage of spoofed packets that hit your network, and would also work pretty well if you are the source of an attack. There is some debate as to whether you must not have ANY RFC1918 space for this to work. We're trying to find this out (not a priority), if I get info I'll post.
hmm... either you're being extremely vague, or you misunderstand how RPF works. http://www.cisco.com/univercd/cc/td/doc/product/software/ios121/121cgcr/secu... Its not checking cef to see if a route is there.... its making sure that a packet received on an interface came in on an interface that is the best return path to reach that packet. thereby explaining why multihomed customers will get borked in the event of using rpf. enjoy, -mark -- Support your local medical examiner--die strangely.
On Thu, May 02, 2002 at 12:04:35PM -0500, Mark Turpin wrote:
On Thu, May 02, 2002 at 09:41:33AM -0700, LeBlanc, Jason wrote something like this: <snip>
There are some limitations as to where uRPF works, SONET only on GSRs for example (thanks Cisco). I believe it will work on 65xx (SUP1A and SUP2 I think) regardless of interface type. Impact should be minimal, as it simply does a lookup in the CEF table, if the route isn't there it discards. Keep in mind this is NOT a filter, so the impact is much less, it is simply a CEF lookup, much more efficient than a filter. This will get rid of a HUGE percentage of spoofed packets that hit your network, and would also work pretty well if you are the source of an attack. There is some debate as to whether you must not have ANY RFC1918 space for this to work. We're trying to find this out (not a priority), if I get info I'll post.
hmm... either you're being extremely vague, or you misunderstand how RPF works. http://www.cisco.com/univercd/cc/td/doc/product/software/ios121/121cgcr/secu...
Its not checking cef to see if a route is there.... its making sure that a packet received on an interface came in on an interface that is the best return path to reach that packet.
thereby explaining why multihomed customers will get borked in the event of using rpf.
RPF works by matching the source address of the packet against the CEF table, in addition to the normal match against the destination address. There are multiple modes of operation, ranging from "is there a route for the source address to the specific interface it come in on" to "is there a route to the source address for ANY interface on the box" The former is used to stop your single homed customers from spoofing wildly into the internet. The latter is usually used as a stopgap measure to limit the number of spoofed packets coming into your network via transits. The number you'd expect to filter is 50%, assuming the attacker in question is using an evenly distributing random() function. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
On Thu, 2 May 2002, Richard A Steenbergen wrote:
RPF works by matching the source address of the packet against the CEF table, in addition to the normal match against the destination address. There are multiple modes of operation, ranging from "is there a route for the source address to the specific interface it come in on" to "is there a route to the source address for ANY interface on the box" The former is used to stop your single homed customers from spoofing wildly into the internet.
You can do this for multihomed customers to: it's just that multihomed customers can't use it for traffic coming from their transits (= you), because uRPF breaks asymmetric routing.
Jason described uRPF in Loose Check mode. This check to see if the source exist in the FIB. It cuts out some of the garbage while providing you a tool to do a remote-triggered (via BGP ) drop tool. Think of uRPF as a tool to do source based black hole filtering. uRPF Strict Mode is the original tool to help scale BCP38 filtering. This checks the FIB and the adjacency - insuring the source address of the packet coming into the interface has a patch to get back (hence checking the validity of the packet). This is a ISP-Customer edge tool. It _does_ work with multihomed customers for the most common multihoming configs. Just set that BGP weight on the customer's peering session. It is getting a little old, but check out the following: http://www.cisco.com/public/cons/isp/documents/uRPF_Enhancement.pdf http://www.cisco.com/public/cons/isp/security/
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Mark Turpin Sent: Thursday, May 02, 2002 10:05 AM To: LeBlanc, Jason Cc: nanog@merit.edu Subject: Re: Effective ways to deal with DDoS attacks?
On Thu, May 02, 2002 at 09:41:33AM -0700, LeBlanc, Jason wrote something like this: <snip>
There are some limitations as to where uRPF works, SONET only
on GSRs for
example (thanks Cisco). I believe it will work on 65xx (SUP1A and SUP2 I think) regardless of interface type. Impact should be minimal, as it simply does a lookup in the CEF table, if the route isn't there it discards. Keep in mind this is NOT a filter, so the impact is much less, it is simply a CEF lookup, much more efficient than a filter. This will get rid of a HUGE percentage of spoofed packets that hit your network, and would also work pretty well if you are the source of an attack. There is some debate as to whether you must not have ANY RFC1918 space for this to work. We're trying to find this out (not a priority), if I get info I'll post.
hmm... either you're being extremely vague, or you misunderstand how RPF works. http://www.cisco.com/univercd/cc/td/doc/product/software/ios121/12 1cgcr/secur_c/scprt5/scdrpf.htm
Its not checking cef to see if a route is there.... its making sure that a packet received on an interface came in on an interface that is the best return path to reach that packet. thereby explaining why multihomed customers will get borked in the event of using rpf. enjoy, -mark -- Support your local medical examiner--die strangely.
http://www.cisco.com/warp/public/707/newsflash.html There are some limitations as to where uRPF works, SONET only on GSRs for example (thanks Cisco). I believe it will work on 65xx (SUP1A and SUP2 I think) regardless of interface type. Impact should be minimal, as it simply does a lookup in the CEF table, if the route isn't there it discards.
We're running 6509's - both Sup1a and Sup2 - with 10, 100, and GigE links in a large campus environment. We did have some problems with the Sup2's running hybrid code, but the Sup1a's were fine. When we switched over to native IOS about six months ago, both the Sup1a's and Sup2's handled it without a problem or performance hit, even on some of our campus Gigabit links. Its a nice feature but, as someone already pointed out, its based on routing table entries so there is NO PROTECTION if someone on a subnet is spoofing the IP of another system on the same subnet. Having said that, we use it more so that we can quickly track the source of an attack if its originating on our network rather than as a means to protect ourselves from the big, bad Internet. Once we know the source, we know for sure what router interface its originating from, so we just start snooping traffic from that interface to find the offending MAC and go from there... Another limitation that we've found with uRPF is that it doesn't live well with multihomed systems (i.e. a host with two NIC's - each on a different subnet) because of the way most OS'es handle their default gateways. For anyone who is interested in our experience, drop me a note off list. If you have a solution for this multihoming problem, PLEASE email me off-list. Eric :)
In the referenced message, Eric Gauthier said: <snip>
Another limitation that we've found with uRPF is that it doesn't live well with multihomed systems (i.e. a host with two NIC's - each on a different subnet) because of the way most OS'es handle their default gateways. For anyone who is interested in our experience, drop me a note off list. If you have a solution for this multihoming problem, PLEASE email me off-list.
Eric :)
Most Cisco boxes have 3 related modes of uRPF: 1) pure RPF, if forwarding path back to source doesn't go via interface packet received from, then dump. I believe, but am not positive, that it will handle equal-cost-multipath ok in situations where that exists. 2) acl exceptions, if source matches the acl, allow the packet 3) not-so-pure RPF, if there is _any_ forwarding path back to the source via _any_ interface, then accept. for single-homed customers, simple uRPF for multihomed customers, acl exceptions based upon their registered IRR-policy, since the customer should already be registering in the IRR you have a list of all networks reachable via the customer, regardless of the actual real-time announcements or policy applications (prepending, localpref tweaks, etc) for peers that are clued-in, again acl exceptions based upon the peers registered policy for non-clued peers, accept based upon any available forwarding path [hopefully, by the 100th time you beat on the peer about spoofed crud coming from them, they'll get religion and register, since you'll have less overall spoofing to track down, you can devote it to slapping them around more] you should also in/egress filter known bogons at all borders, like src/dst in rfc1918 src/dst in class e src in class d (not dest, however) etc
On Fri, 3 May 2002, Stephen Griffin wrote:
for single-homed customers, simple uRPF for multihomed customers, acl exceptions based upon their registered IRR-policy, since the customer should already be registering in the IRR you have a list of all networks reachable via the customer, regardless of the actual real-time announcements or policy applications (prepending, localpref tweaks, etc)
This doesn't make any sense. If you use uRPF on a customer interface, it will check the packets coming in from the customer to see if they match the prefixes you route to that customer. So as long as what you route to them and what they use as source addresses are identical, you don't have a problem. For multihomed customers, these sets of prefixes should be identical, just like with single homed customers. The only time when those sets of prefixes is NOT the same is for a backup connection. But if a connection is a pure backup for incoming traffic, it's reasonable to assume it's a pure backup for outgoing traffic as well, so as long as the backup is dormant, you don't see any traffic so no uRPF problems. Using an exception access list is pretty silly: if you're going through the trouble of listing all a cutomer's prefixes in a list, you might as well just apply this acl to the interface rather than uRPF with the acl as exceptions. There is another way to handle backups: you can also set the weight to a higher value for customer routes. This way, the router connecting to the customer will always use the routes announced by the customer, even if the rest of the network deems them inferior to another route to this customer because of a lower local pref, a higher MED or a longer AS path. This mechanism is also useful for peering: because of the higher weight, the border router will always prefer the exchange (or private peering link) for all traffic to the customer, so uRPF works. The rest of the network can still decide to send the traffic to another exchange point.
for non-clued peers, accept based upon any available forwarding path [hopefully, by the 100th time you beat on the peer about spoofed crud coming from them, they'll get religion and register, since you'll have less overall spoofing to track down, you can devote it to slapping them around more]
If people stop peering with those network polluters they'll clean up their act soon enough.
you should also in/egress filter known bogons at all borders, like src/dst in rfc1918 src/dst in class e
Why? That doesn't buy you much except job security because someone spending so much time on those impressive filters can't be missed of course.
src in class d (not dest, however)
Some multicast apps set the source to the group address as well. Iljitsch van Beijnum
Has anyone used /31 mask addresses on their network? Toan
On Fri May 03, 2002 at 04:24:16PM -0400, Toan Do wrote:
Has anyone used /31 mask addresses on their network?
Yes, works fine (on an all Cisco network). We're starting to use /31's on internal links. Links to third parties are still /30's, as most other people are still wary. Simon -- Simon Lockhart | Tel: +44 (0)1737 839676 Internet Engineering Manager | Fax: +44 (0)1737 839516 BBC Internet Services | Email: Simon.Lockhart@bbc.co.uk Kingswood Warren,Tadworth,Surrey,UK | URL: http://support.bbc.co.uk/
Hi Simon, What IOS are you using /31s with ? André ----- Original Message ----- From: "Simon Lockhart" <simonl@rd.bbc.co.uk> To: "Toan Do" <toando9@aol.com> Cc: <nanog@merit.edu> Sent: Friday, May 03, 2002 10:26 PM Subject: Re: /31 mask address On Fri May 03, 2002 at 04:24:16PM -0400, Toan Do wrote:
Has anyone used /31 mask addresses on their network?
Yes, works fine (on an all Cisco network). We're starting to use /31's on internal links. Links to third parties are still /30's, as most other people are still wary. Simon -- Simon Lockhart | Tel: +44 (0)1737 839676 Internet Engineering Manager | Fax: +44 (0)1737 839516 BBC Internet Services | Email: Simon.Lockhart@bbc.co.uk Kingswood Warren,Tadworth,Surrey,UK | URL: http://support.bbc.co.uk/
On Fri May 03, 2002 at 10:30:05PM +0200, Andre Chapuis wrote:
What IOS are you using /31s with ?
Typically 12.0(x)S on GSR and VXR (where x is 10ish upwards) Simon -- Simon Lockhart | Tel: +44 (0)1737 839676 Internet Engineering Manager | Fax: +44 (0)1737 839516 BBC Internet Services | Email: Simon.Lockhart@bbc.co.uk Kingswood Warren,Tadworth,Surrey,UK | URL: http://support.bbc.co.uk/
Andre Chapuis wrote: For what purpose can this be used? Can a point to point link function with this subnet mask? It would be ok if the requirement for a network and broadcast IP were removed. Regards, Mannu
Hi Simon, What IOS are you using /31s with ? André
----- Original Message ----- From: "Simon Lockhart" <simonl@rd.bbc.co.uk> To: "Toan Do" <toando9@aol.com> Cc: <nanog@merit.edu> Sent: Friday, May 03, 2002 10:26 PM Subject: Re: /31 mask address
On Fri May 03, 2002 at 04:24:16PM -0400, Toan Do wrote:
Has anyone used /31 mask addresses on their network?
Yes, works fine (on an all Cisco network).
We're starting to use /31's on internal links. Links to third parties are still /30's, as most other people are still wary.
Simon -- Simon Lockhart | Tel: +44 (0)1737 839676 Internet Engineering Manager | Fax: +44 (0)1737 839516 BBC Internet Services | Email: Simon.Lockhart@bbc.co.uk Kingswood Warren,Tadworth,Surrey,UK | URL: http://support.bbc.co.uk/
Do you really need broadcasts on your p2p links ? I do not.. André ----- Original Message ----- From: "Manolo Hernandez" <mhernand@ustdata.com> To: "Andre Chapuis" <chapuis@ip-plus.net> Cc: <nanog@merit.edu> Sent: Friday, May 03, 2002 10:46 PM Subject: Re: /31 mask address Andre Chapuis wrote: For what purpose can this be used? Can a point to point link function with this subnet mask? It would be ok if the requirement for a network and broadcast IP were removed. Regards, Mannu
Hi Simon, What IOS are you using /31s with ? André
----- Original Message ----- From: "Simon Lockhart" <simonl@rd.bbc.co.uk> To: "Toan Do" <toando9@aol.com> Cc: <nanog@merit.edu> Sent: Friday, May 03, 2002 10:26 PM Subject: Re: /31 mask address
On Fri May 03, 2002 at 04:24:16PM -0400, Toan Do wrote:
Has anyone used /31 mask addresses on their network?
Yes, works fine (on an all Cisco network).
We're starting to use /31's on internal links. Links to third parties are still /30's, as most other people are still wary.
Simon -- Simon Lockhart | Tel: +44 (0)1737 839676 Internet Engineering Manager | Fax: +44 (0)1737 839516 BBC Internet Services | Email: Simon.Lockhart@bbc.co.uk Kingswood Warren,Tadworth,Surrey,UK | URL: http://support.bbc.co.uk/
Has anyone used /31 mask addresses on their network?
Yes, works fine (on an all Cisco network).
Maybe not interesting for an ISP, but I'm using it on a vlan interface on a 6500/7600. It works fine with IOS 12.1.8-EX5, but 12.1.11-E1 refused the configuration because it's not a p2p interface. I know, documentation only talks about p2p, but sometimes I don't like when a "feature" gets fixed... Andras
On Mon May 06, 2002 at 01:35:34PM +0200, JAKO Andras wrote:
Yes, works fine (on an all Cisco network).
Maybe not interesting for an ISP, but I'm using it on a vlan interface on a 6500/7600. It works fine with IOS 12.1.8-EX5, but 12.1.11-E1 refused the configuration because it's not a p2p interface. I know, documentation only talks about p2p, but sometimes I don't like when a "feature" gets fixed...
Indeed, I found this on Saturday! GSR would let me put a /31 on the ethernet port, but the 6509 at the other end of the ethernet wouldn't. Simon -- Simon Lockhart | Tel: +44 (0)1737 839676 Internet Engineering Manager | Fax: +44 (0)1737 839516 BBC Internet Services | Email: Simon.Lockhart@bbc.co.uk Kingswood Warren,Tadworth,Surrey,UK | URL: http://support.bbc.co.uk/
hmmm, as long as you allow directed broadcast on the interfaces... it should work... On 3 May 2002 at 16:24, Toan Do wrote:
Has anyone used /31 mask addresses on their network?
Toan
-- Miguel Mata-Cardona Intercom El Salvador mmata@sv.intercomnet.net
In the referenced message, Iljitsch van Beijnum said:
On Fri, 3 May 2002, Stephen Griffin wrote:
for single-homed customers, simple uRPF for multihomed customers, acl exceptions based upon their registered IRR-policy, since the customer should already be registering in the IRR you have a list of all networks reachable via the customer, regardless of the actual real-time announcements or policy applications (prepending, localpref tweaks, etc)
This doesn't make any sense. If you use uRPF on a customer interface, it will check the packets coming in from the customer to see if they match the prefixes you route to that customer. So as long as what you route to them and what they use as source addresses are identical, you don't have a problem.
think MEDs and localpref twiddles., identical prefixes do not necessarily relate to best paths.
For multihomed customers, these sets of prefixes should be identical, just like with single homed customers. The only time when those sets of prefixes is NOT the same is for a backup connection. But if a connection is a pure backup for incoming traffic, it's reasonable to assume it's a pure backup for outgoing traffic as well, so as long as the backup is dormant, you don't see any traffic so no uRPF problems.
Not always the case, customer behaviour can not be accurately modeled.
Using an exception access list is pretty silly: if you're going through the trouble of listing all a cutomer's prefixes in a list, you might as well just apply this acl to the interface rather than uRPF with the acl as exceptions.
the acl will only be used on packets failing the rpf check. interface acls get applied to all traffic.
There is another way to handle backups: you can also set the weight to a higher value for customer routes. This way, the router connecting to the customer will always use the routes announced by the customer, even if the rest of the network deems them inferior to another route to this customer because of a lower local pref, a higher MED or a longer AS path.
This mechanism is also useful for peering: because of the higher weight, the border router will always prefer the exchange (or private peering link) for all traffic to the customer, so uRPF works. The rest of the network can still decide to send the traffic to another exchange point.
I'm quite leery of having islands of widely divergent policy, and I wouldn't think it would help if you have multiple non-equal-cost-paths on the same router with which to accept traffic on.
for non-clued peers, accept based upon any available forwarding path [hopefully, by the 100th time you beat on the peer about spoofed crud coming from them, they'll get religion and register, since you'll have less overall spoofing to track down, you can devote it to slapping them around more]
If people stop peering with those network polluters they'll clean up their act soon enough.
This is unlikely to occur, unfortunately, so merely making it as annoying as possible for polluters and less annoying as possible for non-polluters, is probably the way to go.
you should also in/egress filter known bogons at all borders, like src/dst in rfc1918 src/dst in class e
Why? That doesn't buy you much except job security because someone spending so much time on those impressive filters can't be missed of course.
Because it is polite to not send crap to your neighbors, and advantageous to not have crap coming into your network.
src in class d (not dest, however)
Some multicast apps set the source to the group address as well.
How... odd...
Iljitsch van Beijnum
On Sat, 4 May 2002, Stephen Griffin wrote:
In the referenced message, Iljitsch van Beijnum said:
On Fri, 3 May 2002, Stephen Griffin wrote: For multihomed customers, these sets of prefixes should be identical, just like with single homed customers. The only time when those sets of prefixes is NOT the same is for a backup connection. But if a connection is a pure backup for incoming traffic, it's reasonable to assume it's a pure backup for outgoing traffic as well, so as long as the backup is dormant, you don't see any traffic so no uRPF problems.
Not always the case, customer behaviour can not be accurately modeled.
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound traffic for their customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever... uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :( -Chris
On Sun, 5 May 2002, Christopher L. Morrow wrote:
like with single homed customers. The only time when those sets of prefixes is NOT the same is for a backup connection. But if a connection
Not always the case, customer behaviour can not be accurately modeled.
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound traffic for their customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever...
Is there a compelling reason you should allow this? If yes, you can't use uRPF and you have to install an acl to do sanity checking on the customer's source addresses. If no, they'll have to announce those routes to you. If they set the no export community they still won't get any inbound traffic to speak of.
uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :(
Well if these are your worst abusers, it seems to me uRPF is exactly what those customers need. ;-)
Be mindful that uRPF Strict Mode was created to help scale BCP 38 filtering. If you have 1000 lease line customers and can use uRFP Strict Mode on 80% of those customers, that is 80% fewer BCP38 ACLs that you need to manage. For the other 20% you have uRFP + BGP tweaks or plain old ACLs. But as Chris inferred, that 20% where you cannot use simple uRPF is also the 20% most difficult customers.
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Iljitsch van Beijnum Sent: Sunday, May 05, 2002 12:44 AM To: Christopher L. Morrow Cc: nanog@merit.edu Subject: Re: Effective ways to deal with DDoS attacks?
On Sun, 5 May 2002, Christopher L. Morrow wrote:
like with single homed customers. The only time when those sets of prefixes is NOT the same is for a backup connection. But if a connection
Not always the case, customer behaviour can not be accurately modeled.
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound traffic for their customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever...
Is there a compelling reason you should allow this? If yes, you can't use uRPF and you have to install an acl to do sanity checking on the customer's source addresses. If no, they'll have to announce those routes to you. If they set the no export community they still won't get any inbound traffic to speak of.
uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :(
Well if these are your worst abusers, it seems to me uRPF is exactly what those customers need. ;-)
On Sun, 5 May 2002, Christopher L. Morrow wrote:
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound traffic for their customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever... uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :(
This dilemma has far reaching repercussions: If _you_ allow this and forego the unicast RPF check for these customers, this means your peers can't do uRPF towards you without breaking connectivity for these customers. In a perfect world, there would be no need to do uRPF on peering interfaces, because peers make sure they don't send packets with spoofed source addresses. Unfortunately, in the real world many networks STILL don't bother and thereby allow hard to trace and filter DDoS attacks to be launched from inside their networks. So what is the collective wisdom on the NANOG list? Is uRPF on peering interfaces a viable option and if it breaks esoteric customer configurations, too bad; or is it something that should be discouraged because it breaks legitimate customer needs? If you feel strongly one way or the other, but don't want to join the discussion, please reply with a "yes to peering uRPF" or "no to peering uRPF" in private email, and I'll summerize to the list. Iljitsch van Beijnum
On Sun, May 05, 2002 at 12:10:36PM +0200, Iljitsch van Beijnum wrote:
In a perfect world, there would be no need to do uRPF on peering interfaces, because peers make sure they don't send packets with spoofed source addresses. Unfortunately, in the real world many networks STILL don't bother and thereby allow hard to trace and filter DDoS attacks to be launched from inside their networks.
So what is the collective wisdom on the NANOG list? Is uRPF on peering interfaces a viable option and if it breaks esoteric customer configurations, too bad; or is it something that should be discouraged because it breaks legitimate customer needs?
This is not an all or nothing proposition. Some people argue that because they can't filter everyone perfectly, they shouldn't filter anyone at all. This is nonsense, if you can take the *30 SECONDS* of your time to take care of the 95% of your customers who aren't doing anything complex and who aren't going to have problems, both you and the entire internet are better off than if you spent the next 3 years trying to figure out how to do it for everyone. I personally would NOT try to filter peers this way, there are just too many legitimate chances for asymmetric routing at that level. Consider the simplest one of all: You buy transit from provider A and peer with provider B, destination C buys transit from provider B and peers with provider A. Obviously by localpref dest C is going to be sending traffic to you via provider A and you are going to be sending to them via provider B. Asymmetric routing is not evil, it is simply how the internet works. What we should be doing is encouraging the people with the 95% easy customers (or maybe even 100% depending on thier business) to start filtering. I've had a transit provider *coughinternapcough* refuse to apply RPF to my single homed port when I ASKED for it (how many of your customers will do this? :P), because they thought it would place too much load on their routers. And then we have Foundry, who in their infinite wisdom decided that an access-list should be sufficient for statically routed customers, so the RPF they finally came up with is only accessable as a BGP neighbor statement. Lots of people have their boxes for customer aggregation (because it'll be a cold day in hell before they're doing L3 in a core :P), and they decided to make it apply only to BPF neighbors, the one place you probably DON'T need it. These people need to be beaten with the clue stick on a regular basis until something finally sinks in, because obviously the first round didn't take. This is a situation similar to smurf, a mass education of people who don't read nanog (though I doubt most of the people here do) is required. The problem is, you can't make it a default configuration because of asymmetric routing (so someone actually has to think, not just type), and you can't run probes and tell if someone has RPF setup on a remote network (the only people with the resources to tell are the people with DDoS nets, and I don't think they're going to help) so you can publicly shame them. Until someone solves one of the above two, I don't think much work is going to be done. Making RPF where reasonable a requirement for peering is a place to start, but I don't see that as being enforcable. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
As one of the key people pushing uRPF .... uRPF 'Strict Mode' was never ever designed to put on the ISP peering links. It was created to help ISPs scale BCP 38 filtering on the ISP-Customer edge. Knock out the easy 80% of the customers who are simple single homed customers - leaving the other 20% for special BCP38 uRPF/BGP or ACL configs. uRPF 'Loose Check' was designed for any part of the edge - ISP to ISP (could be customer or peer); the ISP to Customer edge; or the ISP to Multihomed edge. The objective was to provide a quick way to trigger a network wide source address based black hole. It also provides an effective way to filter source addresses with martian and bogons (i.e. addresses not in the FIB). So consider uRPF Loose Check as a source address based "noise reduction" filter and a network wide DDOS counter measure. So people need to get the two straight. The are completely different: uRPF Strict Mode for BCP 38 (not for ISP-ISP Peering links) Cisco: ip verify unicast source reachable-via rx Juniper (as of 5.3): unicast-reverse-path active-paths uRPF Loose Check Mode (suitable for ISP-ISP Peering Links) Cisco: ip verify unicast source reachable-via any Juniper (as of 5.3): unicast-reverse-path feasible-paths So you need to frame you response as to the appropriate uRPF mode. One key reminder - uRPF is just another security tool for an ISP's security tool kit. There is no such thing as a perfect security tool. A craftsman is known not for his/her tools, but how well they use the tools they got to perform their art. Barry
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Iljitsch van Beijnum Sent: Sunday, May 05, 2002 3:11 AM To: Christopher L. Morrow Cc: nanog@merit.edu Subject: unicast RPF for peers viable?
On Sun, 5 May 2002, Christopher L. Morrow wrote:
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound traffic for their customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever... uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :(
This dilemma has far reaching repercussions:
If _you_ allow this and forego the unicast RPF check for these customers, this means your peers can't do uRPF towards you without breaking connectivity for these customers.
In a perfect world, there would be no need to do uRPF on peering interfaces, because peers make sure they don't send packets with spoofed source addresses. Unfortunately, in the real world many networks STILL don't bother and thereby allow hard to trace and filter DDoS attacks to be launched from inside their networks.
So what is the collective wisdom on the NANOG list? Is uRPF on peering interfaces a viable option and if it breaks esoteric customer configurations, too bad; or is it something that should be discouraged because it breaks legitimate customer needs?
If you feel strongly one way or the other, but don't want to join the discussion, please reply with a "yes to peering uRPF" or "no to peering uRPF" in private email, and I'll summerize to the list.
Iljitsch van Beijnum
First of all, this list is great! I feel this is one of the best lists I have ever seen... I thank you all for the great technical depth and openness. We have a product that offers DDoS mitigation with ACLs but I am interested in learning more about uRPF Loose Check Mode to see if we can integrate this idea into our product. In particular, I am interested in the ability of eliminating specific routes from the FIB under uRPF Loose Check Mode to effectively filter specific source addresses that are flooding. As I understand the concept, eliminating an address from the FIB such as x.y.0.0/24 would have the equivalent effect of installing a network-wide ACL rule: deny ip x.y.0.0/24 any Is this right? If this is right, is there any way to use uRPF Loose Check Mode to have an equivalent network-wide ACL rule of the form: deny <proto> x.y.0.0/24 <destination> where: <proto> is NOT ip and <destination> is NOT any? The reason why I ask is that we would like to keep control of these two important aspects of the traffic to avoid filtering out too much and therefore possibly affecting legitimate traffic. Think of the case where a flood targets one of multiple downstream customers and the spoofed addresses correspond to a popular address range (such as Yahoo). Doing a "deny ip x.y.0.0/24 any" would effectively shut down Yahoo's traffic for all downstream customers thus amplifying the attacker's effect. Livio.
As one of the key people pushing uRPF .... uRPF 'Strict Mode' was never ever designed to put on the ISP peering links. It was created to help ISPs scale BCP 38 filtering on the ISP-Customer edge. Knock out the easy 80% of the customers who are simple single homed customers - leaving the other 20% for special BCP38 uRPF/BGP or ACL configs.
uRPF 'Loose Check' was designed for any part of the edge - ISP to ISP (could be customer or peer); the ISP to Customer edge; or the ISP to Multihomed edge. The objective was to provide a quick way to trigger a network wide source address based black hole. It also provides an effective way to filter source addresses with martian and bogons (i.e. addresses not in the FIB). So consider uRPF Loose Check as a source address based "noise reduction" filter and a network wide DDOS counter measure.
So people need to get the two straight. The are completely different:
uRPF Strict Mode for BCP 38 (not for ISP-ISP Peering links)
Cisco: ip verify unicast source reachable-via rx
Juniper (as of 5.3): unicast-reverse-path active-paths
uRPF Loose Check Mode (suitable for ISP-ISP Peering Links)
Cisco: ip verify unicast source reachable-via any
Juniper (as of 5.3): unicast-reverse-path feasible-paths
So you need to frame you response as to the appropriate uRPF mode.
One key reminder - uRPF is just another security tool for an ISP's security tool kit. There is no such thing as a perfect security tool. A craftsman is known not for his/her tools, but how well they use the tools they got to perform their art.
Barry
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Iljitsch van Beijnum Sent: Sunday, May 05, 2002 3:11 AM To: Christopher L. Morrow Cc: nanog@merit.edu Subject: unicast RPF for peers viable?
On Sun, 5 May 2002, Christopher L. Morrow wrote:
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound
traffic for their
customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever... uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :(
This dilemma has far reaching repercussions:
If _you_ allow this and forego the unicast RPF check for these customers, this means your peers can't do uRPF towards you without breaking connectivity for these customers.
In a perfect world, there would be no need to do uRPF on peering interfaces, because peers make sure they don't send packets with spoofed source addresses. Unfortunately, in the real world many networks STILL don't bother and thereby allow hard to trace and filter DDoS attacks to be launched from inside their networks.
So what is the collective wisdom on the NANOG list? Is uRPF on peering interfaces a viable option and if it breaks esoteric customer configurations, too bad; or is it something that should be discouraged because it breaks legitimate customer needs?
If you feel strongly one way or the other, but don't want to join the discussion, please reply with a "yes to peering uRPF" or "no to peering uRPF" in private email, and I'll summerize to the list.
Iljitsch van Beijnum
On Sun, May 05, 2002 at 11:55:21AM -0700, Livio Ricciulli wrote:
In particular, I am interested in the ability of eliminating specific routes from the FIB under uRPF Loose Check Mode to effectively filter specific source addresses that are flooding.
As I understand the concept, eliminating an address from the FIB such as x.y.0.0/24 would have the equivalent effect of installing a network-wide ACL rule:
deny ip x.y.0.0/24 any
Not quite. First, lets be specific by what you mean by "remove from the FIB", as there are a number of different methods you could use. You could simply block it from the RIB when generating the FIB, you could go back after FIB generation and try to make it unresolved, or you could change the nexthop to "discard". If you're trying to replicate traditional firewall behavior (filter no matter what) you would have to do it post FIB generation, but if you are trying to replicate normal routing behavior (ex: a null route) you would have to do it during FIB generation, so that you can potentially have more specific routes which escape the "filter". Secondly, when you remove something from your FIB, you also block destination routing as well as source.
The reason why I ask is that we would like to keep control of these two important aspects of the traffic to avoid filtering out too much and therefore possibly affecting legitimate traffic. Think of the case where a flood targets one of multiple downstream customers and the spoofed addresses correspond to a popular address range (such as Yahoo). Doing a "deny ip x.y.0.0/24 any" would effectively shut down Yahoo's traffic for all downstream customers thus amplifying the attacker's effect.
It sounds like what you are looking for has nothing to do with the RPF or the FIB, but rather simply manual source address filtering. However, if the reason you're interested in RPF is because you want to source match filtering more efficiently, you may be interested in the data structure. Rather then walking a straight access-list rule set doing a comparison for every rule, you can make a "Filtering" Information Base mtrie for source address rules. This is the entire point of standard access-lists, and more recently compiled access-lists. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
Richard A Steenbergen wrote:
On Sun, May 05, 2002 at 11:55:21AM -0700, Livio Ricciulli wrote:
In particular, I am interested in the ability of eliminating specific routes from the FIB under uRPF Loose Check Mode to effectively filter specific source addresses that are flooding.
As I understand the concept, eliminating an address from the FIB such as x.y.0.0/24 would have the equivalent effect of installing a network-wide ACL rule:
deny ip x.y.0.0/24 any
Not quite.
First, lets be specific by what you mean by "remove from the FIB", as there are a number of different methods you could use. You could simply block it from the RIB when generating the FIB, you could go back after FIB generation and try to make it unresolved, or you could change the nexthop to "discard". If you're trying to replicate traditional firewall behavior (filter no matter what) you would have to do it post FIB generation, but if you are trying to replicate normal routing behavior (ex: a null route) you would have to do it during FIB generation, so that you can potentially have more specific routes which escape the "filter".
escaping the "filter" with more specific routes would be absolutely necessary.
Secondly, when you remove something from your FIB, you also block destination routing as well as source.
Good point; so in ACL equivalent language you are saying that taking out a FIB entry in uRPF Loose Check Mode is equivalent to a network-wide insertion of: deny ip x.y.0.0/24 any (from the uRPF Loose Check Mode) deny ip any x.y.0.0/24 (from the absence of a route) (modulo the more specific routes to escape the "filter" which could be expressed as prepended permits in the ACL equivalent world)
The reason why I ask is that we would like to keep control of these two important aspects of the traffic to avoid filtering out too much and therefore possibly affecting legitimate traffic. Think of the case where a flood targets one of multiple downstream customers and the spoofed addresses correspond to a popular address range (such as Yahoo). Doing a "deny ip x.y.0.0/24 any" would effectively shut down Yahoo's traffic for all downstream customers thus amplifying the attacker's effect.
It sounds like what you are looking for has nothing to do with the RPF or the FIB, but rather simply manual source address filtering.
Well, I am investigating if it is possible today to use uRPF Loose Check Mode to achieve network-wide source/destination address filtering functionality (it seems not from what you write). I immagine that it would be useful to use route advertisements to enforce network-wide access control policies. These policies, however, to be generally interesting for DDoS would have to be at least as expressive as "<deny|permit> <proto> <source> <destination>" (hence my questions). Livio.
On Sun, May 05, 2002 at 04:20:43PM -0700, Livio Ricciulli wrote:
Well, I am investigating if it is possible today to use uRPF Loose Check Mode to achieve network-wide source/destination address filtering functionality (it seems not from what you write). I immagine that it would be useful to use route advertisements to enforce network-wide access control policies. These policies, however, to be generally interesting for DDoS would have to be at least as expressive as "<deny|permit> <proto> <source> <destination>" (hence my questions).
Network wide? Ok that was my first guess actually, that you were trying to create an "acl protocol". Well, like I said, there isn't really such a thing as "removing" something from a FIB. A FIB is generated from a RIB, so something special would have to happen to make it "unresolved" (as if there was no route at all). You can make a route not installed in a RIB, but not remove others of the same or longer length from the FIB. :) Now on the other hand, it's interesting to consider what would (or could) happen if you had a special route for example to null0. Obviously with strict mode, you're not going to match because the nexthop is different. In loose mode on the other hand, there is some room to consider a route to null0 as not a valid route at all, and make it not match. I have no idea if that is the current behavior of anyone's implementation, but it potentially could be. Of course then you'd need protocol extensions to carry around actual null0 routes instead of a nexthop just reserved for null routes... So this entire conversation is pretty pointless. :) What we all really need is a protocol which can distribute filtering information network-wide. Go make one. :) -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
On Sun, 05 May 2002 22:11:12 EDT, Richard A Steenbergen said:
What we all really need is a protocol which can distribute filtering information network-wide. Go make one. :)
No, what we need is a protocol that can do *secured* distribution of filtering info net-wide. Otherwise, some bozo is going to accidentally inject a flter for 127/8, causing as much fun as the announcement of same a few years ago. And I'm *sure* there's at least a few people on this list that would be *very* tempted to inject filters for RFC1918 space for the benefit of those providers that don't egress filter it currently ;)
On Mon, May 06, 2002 at 12:50:53AM -0400, Valdis.Kletnieks@vt.edu wrote:
On Sun, 05 May 2002 22:11:12 EDT, Richard A Steenbergen said:
What we all really need is a protocol which can distribute filtering information network-wide. Go make one. :)
No, what we need is a protocol that can do *secured* distribution of filtering info net-wide. Otherwise, some bozo is going to accidentally inject a flter for 127/8, causing as much fun as the announcement of same a few years ago. And I'm *sure* there's at least a few people on this list that would be *very* tempted to inject filters for RFC1918 space for the benefit of those providers that don't egress filter it currently ;)
Nononono, by network-wide I ment *MY* network not the Internet. :) Though I really wouldn't mind seeing a well known community for "nexthop null0". How can people sit around pontificating on useless features for useless protocols all day long, and yet not do this? BTW, I don't know what announcing 127/8 would break since that should never leave or enter any systems, and I still take issue with the need to filter 1918 packets. -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
In the referenced message, Iljitsch van Beijnum said:
So what is the collective wisdom on the NANOG list? Is uRPF on peering interfaces a viable option and if it breaks esoteric customer configurations, too bad; or is it something that should be discouraged because it breaks legitimate customer needs?
I believe that every little bit helps. If some amount of collateral damage for odd configs is too much for you, you can still do the below. This should only break the most egregiously broken setups (sources in space which is entirely unreachable.) The most permissive configuration: loose-check RPF (allow if any path available) combined with: interface acls (in and outbound) deny src or dst in rfc1918 deny src or dst in class e !supposedly, some mcast apps set both src and dst to group !so permit permit src _and_ dst in class d !nothing else should have source in class d deny src in class d the interface acls aren't needed assuming you have no active routes for RFC1918, class d, or class e. IMHO, they are still a good idea anyways, esp. on _outbound_ to reduce crap sent to others. As with all things, every little bit helps. Filter what you can, contribute to the overall improvement of the net. Become a White Hat respected by all, or do nothing and become a Black Hat reviled by millions of small children.
In the referenced message, Christopher L. Morrow said:
On Sat, 4 May 2002, Stephen Griffin wrote:
In the referenced message, Iljitsch van Beijnum said:
On Fri, 3 May 2002, Stephen Griffin wrote: For multihomed customers, these sets of prefixes should be identical, just like with single homed customers. The only time when those sets of prefixes is NOT the same is for a backup connection. But if a connection is a pure backup for incoming traffic, it's reasonable to assume it's a pure backup for outgoing traffic as well, so as long as the backup is dormant, you don't see any traffic so no uRPF problems.
Not always the case, customer behaviour can not be accurately modeled.
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound traffic for their customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever... uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :(
-Chris
Tell them they will need to register their routes in the IRR, even if they don't necessarily advertise all or any of them. Build your exceptions based upon the irr, as for all bgp-speaking customers. If they don't do BGP in the above scenario, work out some other method to communicate legitimate traffic sources. I know this is fairly common pratice for UUNet insofar as routing filters with per-customer filters, due to UUNet's opposition to the IRR. I wish UU actually did IRR-based filters, to cut out the UU-specific annoyance I presently deal with, since the remainder of the folks I deal with handle the IRR. If there was some particular situation where neither IRR-based exceptions, or customer-specific exceptions just couldn't work, then do what you _can_ do. loose RPF checks based upon matching _any_ prefix, and interface filter inbound and outbound on known bogons. this, at least, constrains the area of ipv4 which can be used for spoofing, which is better than where you started.
On Sun, 5 May 2002, Stephen Griffin wrote:
In the referenced message, Christopher L. Morrow said:
On Sat, 4 May 2002, Stephen Griffin wrote:
In the referenced message, Iljitsch van Beijnum said:
On Fri, 3 May 2002, Stephen Griffin wrote: For multihomed customers, these sets of prefixes should be identical, just like with single homed customers. The only time when those sets of prefixes is NOT the same is for a backup connection. But if a connection is a pure backup for incoming traffic, it's reasonable to assume it's a pure backup for outgoing traffic as well, so as long as the backup is dormant, you don't see any traffic so no uRPF problems.
Not always the case, customer behaviour can not be accurately modeled.
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound traffic for their customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever... uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :(
-Chris
Tell them they will need to register their routes in the IRR, even if they don't necessarily advertise all or any of them. Build your exceptions based upon the irr, as for all bgp-speaking customers.
I'm not really sure how one customer IRR filtered (one example customer) is going to matter here, this is the equivalent of a customer connected via 2 t1's one to AT&T and one to UUNET, not announcing EVER his ATT routes to UU nor his UU routes to ATT. How would you know what traffic to expect from this customer at any point in time? He could just push all traffic over his ATT link outbound and only pull in UU on UU and ATT on ATT. Route filters aren't really a solution to this problem.
---rant removed---
If there was some particular situation where neither IRR-based exceptions, or customer-specific exceptions just couldn't work, then do what you _can_ do. loose RPF checks based upon matching _any_ prefix, and interface filter inbound and outbound on known bogons. this, at least, constrains the area of ipv4 which can be used for spoofing, which is better than where you started.
Access-lists on interfaces?? This does not scale, puts my network at risk and will certainly break some 'legitimate' traffic. Additionally, as I've said before, spoofed attacks don't really bother me, they are more easily stopped and tracked.
In the referenced message, Christopher L. Morrow said:
On Sun, 5 May 2002, Stephen Griffin wrote:
In the referenced message, Christopher L. Morrow said:
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound traffic for their customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever... uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :(
-Chris
Tell them they will need to register their routes in the IRR, even if they don't necessarily advertise all or any of them. Build your exceptions based upon the irr, as for all bgp-speaking customers.
I'm not really sure how one customer IRR filtered (one example customer) is going to matter here, this is the equivalent of a customer connected via 2 t1's one to AT&T and one to UUNET, not announcing EVER his ATT routes to UU nor his UU routes to ATT. How would you know what traffic to expect from this customer at any point in time? He could just push all traffic over his ATT link outbound and only pull in UU on UU and ATT on ATT. Route filters aren't really a solution to this problem.
not route-filtering. You use the irr-data to populate the exceptions to strict-mode rpf. The irr is more of a flight-plan of possibility. If the customer registers both sets of routes, and you use that data to build the acl, then it doesn't matter what the customer announces to you. Anything which fails the actual rpf check, will then be passed through the acl to selectively override the rpf check.
---rant removed---
If there was some particular situation where neither IRR-based exceptions, or customer-specific exceptions just couldn't work, then do what you _can_ do. loose RPF checks based upon matching _any_ prefix, and interface filter inbound and outbound on known bogons. this, at least, constrains the area of ipv4 which can be used for spoofing, which is better than where you started.
Access-lists on interfaces?? This does not scale, puts my network at risk and will certainly break some 'legitimate' traffic. Additionally, as I've said before, spoofed attacks don't really bother me, they are more easily stopped and tracked.
The object is to keep the interface acls short, and use the rpf logic to get rid of as much as you can. Rather than using interface acls themselves to emulate a strict rpf check. If you filter only known bogons (RFC1918, as an obvious example) I don't see what will be broken, except that which was broken to begin with. Along with things like compiled access-lists, the impact can be kept quite small. You can even add in performance circuit-breakers to speed things up. For example, allowing established TCP sessions, will (without even compiling the acls) allow a first-match circuit breaker on a large percentage of traffic. While spoofed traffic may not be a concern for as701, I know I, at least, would prefer not to get spoofed traffic via that network. I'm sure other paying customers probably feel similarly. I don't see how access-lists on edge interfaces don't scale, if they are all the same short anti-bogon acl. Junipers should be able to handle this ok, Cisco Engine3 (Edge) should be able to handle fine for GSR. I thought AS701 was moving towards vendor-J on the edge, as it was, but maybe I'm misremembering. Even if it isn't possible for as701 to rpf check/filter in every possible scenario, whatever _is_ possible would be appreciated (by me at least). "Progress, not perfection" as the saying goes...
On Sun, 5 May 2002, Stephen Griffin wrote:
In the referenced message, Christopher L. Morrow said:
On Sun, 5 May 2002, Stephen Griffin wrote:
In the referenced message, Christopher L. Morrow said:
I was hoping someone else might mention this, BUT what about the case of customers providing transit for outbound but not inbound traffic for their customers? We have many, many cases of customers that are 'default routing' for their customers that get inbound traffic down alternate customers or peers or wherever... uRPF seems like a not so good solution for these instances :( especially since some of these are our worst abusers :(
-Chris
Tell them they will need to register their routes in the IRR, even if they don't necessarily advertise all or any of them. Build your exceptions based upon the irr, as for all bgp-speaking customers.
I'm not really sure how one customer IRR filtered (one example customer) is going to matter here, this is the equivalent of a customer connected via 2 t1's one to AT&T and one to UUNET, not announcing EVER his ATT routes to UU nor his UU routes to ATT. How would you know what traffic to expect from this customer at any point in time? He could just push all traffic over his ATT link outbound and only pull in UU on UU and ATT on ATT. Route filters aren't really a solution to this problem.
not route-filtering. You use the irr-data to populate the exceptions to strict-mode rpf. The irr is more of a flight-plan of possibility. If the customer registers both sets of routes, and you use that data to build the acl, then it doesn't matter what the customer announces to you. Anything which fails the actual rpf check, will then be passed through the acl to selectively override the rpf check.
Perhaps I'm confused (which is likely in this case) but if the traffic is being transitted by 2 or 3 as's before it gets to me through 'default' routing how am I to know it was coming?
---rant removed---
If there was some particular situation where neither IRR-based exceptions, or customer-specific exceptions just couldn't work, then do what you _can_ do. loose RPF checks based upon matching _any_ prefix, and interface filter inbound and outbound on known bogons. this, at least, constrains the area of ipv4 which can be used for spoofing, which is better than where you started.
Access-lists on interfaces?? This does not scale, puts my network at risk and will certainly break some 'legitimate' traffic. Additionally, as I've said before, spoofed attacks don't really bother me, they are more easily stopped and tracked.
The object is to keep the interface acls short, and use the rpf logic to get rid of as much as you can. Rather than using interface acls themselves to emulate a strict rpf check. If you filter only known bogons (RFC1918, as an obvious example) I don't see what will be broken, except that which was broken to begin with.
Any access-list of any length severly impacts edge performance, if it works at all, and puts the network at risk. This is not dogma, this is proven time and again on a large operational network. They are never placed for 'permanent' reasons. It is expected that customers will properly handle their traffic... yes they don't always do it, but it is expected.
Along with things like compiled access-lists, the impact can be kept quite small. You can even add in performance circuit-breakers to speed things up. For example, allowing established TCP sessions, will (without even compiling the acls) allow a first-match circuit breaker on a large percentage of traffic.
Compiled access lists? Wow, you are a braver man than I. My experience with them has been 'sub optimal' to say the least. Where known traffic flows and known patterns, with reasonable route table sizes, are available compiled acls work fine. The internet is none of these :(
While spoofed traffic may not be a concern for as701, I know I, at least, would prefer not to get spoofed traffic via that network. I'm sure other paying customers probably feel similarly.
I don't see how access-lists on edge interfaces don't scale, if they are all the same short anti-bogon acl. Junipers should be able to handle this ok, Cisco Engine3 (Edge) should be able to handle fine for GSR. I thought AS701 was moving towards vendor-J on the edge, as it was, but maybe I'm misremembering.
How large is your edge? Do you have routers with +900 interfaces? Management of acls on interfaces, even if the gear were to support it, isn't feasible, nor is just dropping in an E3 card a solution, acls don't work reliably on E3 cards :( E2 cards are just as fun :( the really fun part comes with the 'limited' route table incurred with PSA acls on E2 cards! Anyway, the point here is acls on interfaces are not a solution at the backbone level. At the individual provider level its sure a nice thing, and likely it should be done, but not on the backbone. As to randomly filtering 1918 address space, at your edge that's fine, I'm not sure breaking things for those places that use 1918 internally and thus originate 'legitimate' 1918 address sources for 'error conditions' is a great plan. (please see last 16 flame-fests on martian filtering) -Chris
On Mon, May 06, 2002 at 05:39:05AM +0000, Christopher L. Morrow wrote:
Perhaps I'm confused (which is likely in this case) but if the traffic is being transitted by 2 or 3 as's before it gets to me through 'default' routing how am I to know it was coming?
You're talking about packets received from the internet, he's talking about packets received from your customers.
Any access-list of any length severly impacts edge performance, if it works at all, and puts the network at risk. This is not dogma, this is proven time and again on a large operational network. They are never placed for 'permanent' reasons. It is expected that customers will properly handle their traffic... yes they don't always do it, but it is expected.
It all depends on a) whats your equipment, and b) what do you define as an edge. If your edge is a T1 things are a lot different than if your edge is GigE and you have to use "core" (for the definition of core which means not providing features to compete on performance, and explaining it by telling you that you shouldn't need those features) equipment to provide it.
Compiled access lists? Wow, you are a braver man than I. My experience with them has been 'sub optimal' to say the least. Where known traffic flows and known patterns, with reasonable route table sizes, are available compiled acls work fine. The internet is none of these :(
If everyone who had been burnt by a Crisco bug in a certain feature never used that feature again, there would be no features. That said, compiled access-lists work fine for me. :)
How large is your edge? Do you have routers with +900 interfaces? Management of acls on interfaces, even if the gear were to support it, isn't feasible, nor is just dropping in an E3 card a solution, acls don't work reliably on E3 cards :( E2 cards are just as fun :( the really fun part comes with the 'limited' route table incurred with PSA acls on E2 cards!
If your vendor isn't providing you with working products, find a new vendor. I'm not going to touch that config with a 10ft cattle prod though, it better be automatically generated. That brings it down to the same level of distasteful tolerance for the good of the internet as script generated prefix lists. :) -- Richard A Steenbergen <ras@e-gerbil.net> http://www.e-gerbil.net/ras PGP Key ID: 0x138EA177 (67 29 D7 BC E8 18 3E DA B2 46 B3 D8 14 36 FE B6)
Stephen Griffin wrote:
Tell them they will need to register their routes in the IRR, even if they don't necessarily advertise all or any of them. Build your exceptions based upon the irr, as for all bgp-speaking customers.
not route-filtering. You use the irr-data to populate the exceptions to strict-mode rpf. The irr is more of a flight-plan of possibility. If the customer registers both sets of routes, and you use that data to build the acl, then it doesn't matter what the customer announces to you. Anything which fails the actual rpf check, will then be passed through the acl to selectively override the rpf check.
What about existing customers that don't yet use the IRR? Say you filter some BGP customers' route announcements using manually-built prefix-lists. Have found that by using distribute-list in (instead of prefix-list), one can simply refer the distribute-list # in the strict uRPF configuration and accomplish both functions (route filtering + uRPF) easily with one ACL. e.g.: ip verify unicast source reachable-via rx 49 access-list 49 permit x.x.x.x 0.0.0.255 access-list 49 permit y.y.y.y 0.0.0.252 access-list 49 deny any log Prefix-lists are preferable over ACL-based distribute-lists. Hey Cisco, please make uRPF configuration accept either distribute-lists or prefix-lists for the exception branching. I realize that to IOS ACLs and prefix-lists are not the same, but the benefits of prefix-lists vs. distribute-lists are many. It sounds that a lot of networks rely on IRRs for building BGP customer route filters. What method then is used for the cases where a customer is not already using the IRR? Forced IRR registration before BGP turnup? Or do you fallback on filtering by using prefix- or distribute-lists? What's NANOG's opinion: assuming that uRPF is implemented on all customer interfaces, are there any legitimate purposes for a customer to forward packets with source IP addresses not currently routed by the transit provider towards the customer (either static or BGP)?
What's NANOG's opinion: assuming that uRPF is implemented on all customer interfaces, are there any legitimate purposes for a customer to forward packets with source IP addresses not currently routed by the transit provider towards the customer (either static or BGP)?
IP Tunneling - it often makes more sense to send packets out that have a source address reachable only through the tunnel.
On Mon, 06 May 2002 19:04:11 EDT, Ralph Doncaster said:
IP Tunneling - it often makes more sense to send packets out that have a source address reachable only through the tunnel.
But aren't those source addresses hidden *inside* the encapsulation, and what's visible to routers are the source/dest IPs of the tunnel itself?
On Mon, 6 May 2002, Valdis.Kletnieks@vt.edu wrote:
On Mon, 06 May 2002 19:04:11 EDT, Ralph Doncaster said:
IP Tunneling - it often makes more sense to send packets out that have a source address reachable only through the tunnel.
But aren't those source addresses hidden *inside* the encapsulation, and what's visible to routers are the source/dest IPs of the tunnel itself?
What I'm saying is that if something comes in through the tunnel, the shortest route to the destination is often not to go back out through the tunnel.
In the referenced message, Steven W. Raymond said:
Stephen Griffin wrote:
Tell them they will need to register their routes in the IRR, even if they don't necessarily advertise all or any of them. Build your exceptions based upon the irr, as for all bgp-speaking customers.
not route-filtering. You use the irr-data to populate the exceptions to strict-mode rpf. The irr is more of a flight-plan of possibility. If the customer registers both sets of routes, and you use that data to build the acl, then it doesn't matter what the customer announces to you. Anything which fails the actual rpf check, will then be passed through the acl to selectively override the rpf check.
What about existing customers that don't yet use the IRR? Say you filter some BGP customers' route announcements using manually-built prefix-lists. Have found that by using distribute-list in (instead of prefix-list), one can simply refer the distribute-list # in the strict uRPF configuration and accomplish both functions (route filtering + uRPF) easily with one ACL.
the IRR is merely an input vector. an alternate input vector is manual entry. the output would be an acl or prefix-list. I don't believe the format of a routing-use acl and an RPF-use acl is the same. My recollection is that when used for route filtering you have: access-list foo {permit|deny} ip network wildbits netmask wildbits where for RPF, or traditional traffic filter is access-list foo {permit|deny} ip source wildbits dest wildbits I guess you could use a "standard acl" however I wouldn't recommend it for filtering routes. Even if you could use prefix-lists for uRPF, you would want to match more-specifics, whereas generally you don't want to match (unbounded) more-specifics on route filters. RtConfig can generate either style from IRR data. It isn't too hard to generate either style from a manual list either.
e.g.: ip verify unicast source reachable-via rx 49 access-list 49 permit x.x.x.x 0.0.0.255 access-list 49 permit y.y.y.y 0.0.0.252 access-list 49 deny any log
Prefix-lists are preferable over ACL-based distribute-lists. Hey Cisco, please make uRPF configuration accept either distribute-lists or prefix-lists for the exception branching. I realize that to IOS ACLs and prefix-lists are not the same, but the benefits of prefix-lists vs. distribute-lists are many.
How would uRPF respond to the following prefix-list? ip prefix-list foo deny 0.0.0.0/0 ge 25 ip prefix-list foo permit 1.2.3.0/24 ip prefix-list foo permit 0.0.0.0/0 le 16 Would it accept all sources within 1.2.3.0/24? What about 10.0.0.0/8? I guess it could ignore "ge" and "le". Although how it would resolve conflicts is an unknown. It might try to correspond to actual prefixes, but that seems unlikely.
It sounds that a lot of networks rely on IRRs for building BGP customer route filters. What method then is used for the cases where a customer is not already using the IRR? Forced IRR registration before BGP turnup? Or do you fallback on filtering by using prefix- or distribute-lists?
In my experience, providers that require IRR registration often allow the customer to register their own objects, or offer to proxy-register their customers objects. The preference generally being on the customer registering their own objects, since it gives the customer the greatest degree of control (especially should they change providers.)
What's NANOG's opinion: assuming that uRPF is implemented on all customer interfaces, are there any legitimate purposes for a customer to forward packets with source IP addresses not currently routed by the transit provider towards the customer (either static or BGP)?
Yes, I think there are definitely legitimate reasons why a customer would source traffic from prefixes where the actively selected route does not point back at the interface. This is why acl exceptions and the loose match came to be. With customers, the acl exception is probably appropriate. If the customer exhibits sufficient clue, and demonstrates that they are doing RPF checks, I could definitely see relaxing restrictions against them. If they are providing transit to other BGP-speakers, this is probably the case. As in all things, you know your customer best, so you know how loose you are willing to make things, with the potential that it may make you look bad.
Stephen Griffin wrote:
where for RPF, or traditional traffic filter is access-list foo {permit|deny} ip source wildbits dest wildbits
Hrrmm, since uRPF checks only the source address, the "standard ACL" seems most appropriate to me.
I guess you could use a "standard acl" however I wouldn't recommend it for filtering routes. Even if you could use prefix-lists for uRPF, you would want to match more-specifics, whereas generally you don't want to match (unbounded) more-specifics on route filters.
RtConfig can generate either style from IRR data. It isn't too hard to generate either style from a manual list either.
It certainly wouldn't hurt to have both a prefix-list for route filtering and ACL for the uRPF exceptions. It's just that I am lazy and thought it would be "neat" for one list to fulfill both requirements, since it is essentially the same input data in two different formats.
How would uRPF respond to the following prefix-list? ip prefix-list foo deny 0.0.0.0/0 ge 25
The implicit deny @ the end of the prefix-list seems a better way to accomplish the same result as above (deny anything longer than /24). In other words, instead use a prefix-list containing an explicit list of the permitted networks, rather than pattern matching to deny what bad stuff might be announced.
ip prefix-list foo permit 1.2.3.0/24 ip prefix-list foo permit 0.0.0.0/0 le 16
Would it accept all sources within 1.2.3.0/24? What about 10.0.0.0/8? I guess it could ignore "ge" and "le". Although how it would resolve conflicts is an unknown. It might try to correspond to actual prefixes, but that seems unlikely.
To restate above, just permit explicit networks customer plans to announce & source traffic from. Don't wildcard in customer prefix-lists inbound. Every source packet address received "should" be covered by his prefix-list (even if not the FIB entry best path choice). Every other source IP address packet is dropped. In fantasy land, uRPF "could" confirm that each packet source address matches at least one of the networks in the prefix-list.
Yes, I think there are definitely legitimate reasons why a customer would source traffic from prefixes where the actively selected route does not point back at the interface. This is why acl exceptions and the loose match came to be. With customers, the acl exception is probably appropriate.
Would you agree it is indeed necessary for every BGP customer-facing interface to implement exception checking with strict uRPF? Customer-set communities can change local pref easily enough to break strict uRPF lacking exception checking. But with the ACL permitting exceptions based upon every possible network customer may be sourcing from, the entry doesn't even have to be best path in the FIB to permit the packet. Customer needed only to have gotten the ISP to include it in his prefix-list at some point.
In the referenced message, Steven W. Raymond said:
Stephen Griffin wrote:
where for RPF, or traditional traffic filter is access-list foo {permit|deny} ip source wildbits dest wildbits
Hrrmm, since uRPF checks only the source address, the "standard ACL" seems most appropriate to me.
true, although I'ld still use extended acl or prefix-list for route-filtering.
I guess you could use a "standard acl" however I wouldn't recommend it for filtering routes. Even if you could use prefix-lists for uRPF, you would want to match more-specifics, whereas generally you don't want to match (unbounded) more-specifics on route filters.
RtConfig can generate either style from IRR data. It isn't too hard to generate either style from a manual list either.
It certainly wouldn't hurt to have both a prefix-list for route filtering and ACL for the uRPF exceptions. It's just that I am lazy and thought it would be "neat" for one list to fulfill both requirements, since it is essentially the same input data in two different formats.
I prefer to think of it as optimal, rather than lazy. :)
How would uRPF respond to the following prefix-list? ip prefix-list foo deny 0.0.0.0/0 ge 25
The implicit deny @ the end of the prefix-list seems a better way to accomplish the same result as above (deny anything longer than /24). In other words, instead use a prefix-list containing an explicit list of the permitted networks, rather than pattern matching to deny what bad stuff might be announced.
I agree that explicit matches are good. My concern is when you build in safeguards for a customer-input stream (any automated system not requiring clued review).
ip prefix-list foo permit 1.2.3.0/24 ip prefix-list foo permit 0.0.0.0/0 le 16
Would it accept all sources within 1.2.3.0/24? What about 10.0.0.0/8? I guess it could ignore "ge" and "le". Although how it would resolve conflicts is an unknown. It might try to correspond to actual prefixes, but that seems unlikely.
To restate above, just permit explicit networks customer plans to announce & source traffic from. Don't wildcard in customer prefix-lists inbound. Every source packet address received "should" be covered by his prefix-list (even if not the FIB entry best path choice). Every other source IP address packet is dropped. In fantasy land, uRPF "could" confirm that each packet source address matches at least one of the networks in the prefix-list.
Well, I could see having a safety-net of denies at the top built-in, since you wouldn't necessarily want your customer to register things like rfc1918, your aggregates, or long prefixes. This would lead to optimzing with deny 0.0.0.0/0 ge 25 (or whatever you consider too long). I agree that it would be nice, but I don't see it happening.
Yes, I think there are definitely legitimate reasons why a customer would source traffic from prefixes where the actively selected route does not point back at the interface. This is why acl exceptions and the loose match came to be. With customers, the acl exception is probably appropriate.
Would you agree it is indeed necessary for every BGP customer-facing interface to implement exception checking with strict uRPF? Customer-set communities can change local pref easily enough to break strict uRPF lacking exception checking. But with the ACL permitting exceptions based upon every possible network customer may be sourcing from, the entry doesn't even have to be best path in the FIB to permit the packet. Customer needed only to have gotten the ISP to include it in his prefix-list at some point.
It may not be necessary, but it is (imho) advisable, for the reasons you mention. Other, general cases would be multi-homed static customers, where you may statically route an aggregate to all locations, and route smaller chunks to individual locations. Others would be specific cases, based upon combined customer/provider needs. If a BGP customer was providing transit to others, and actively doing RPF checks themselves (and you trust them) I could see just doing loose checks against them.
We have been working on this problem for the last two years and have productized a practical way to deal with DDoS. Given what I have read today, I think most people on this list would like it. If you send me email, I will send you a white paper that is much more detailed than what you can find on http://www.reactivenetwork.com. Livio.
participants (18)
-
Andre Chapuis -
Barry Raveendran Greene -
Christopher L. Morrow -
Eric Gauthier -
Iljitsch van Beijnum -
JAKO Andras -
LeBlanc, Jason -
Livio Ricciulli -
Manolo Hernandez -
Mark Turpin -
Miguel Mata-Cardona -
Ralph Doncaster -
Richard A Steenbergen -
Simon Lockhart -
Stephen Griffin -
Steven W. Raymond -
Toan Do -
Valdis.Kletnieks＠vt.edu