RE: dsl providers that will route /24
Are we talking business grade DSL or Resi grade DSL?
-----Original Message----- From: Roeland Meyer [mailto:rmeyer@mhsc.com] Sent: Sunday, March 25, 2001 3:04 PM To: 'michael'; nanog@merit.edu Subject: RE: dsl providers that will route /24
If you find such, I want to know too.
-----Original Message----- From: michael [mailto:michael@aplatform.com] Sent: Sunday, March 25, 2001 8:08 AM To: nanog@merit.edu Subject: dsl providers that will route /24
Hello,
Looking for contact information on dsl providers that route IP space not within their assigned block. I am also curious on high speed wireless doing the same.
Thanks,
Michael...
-------------------------------------------------------------- ----------- This email server is running an evaluation copy of the MailShield anti- spam software. Please contact your email administrator if you have any questions about this message. MailShield product info: www.mailshield.com .
Hello, Business grade. And it was suggested that I follow up with more specific location. California, San Francisco bay area. Thanks, Michael... On Sun, 25 Mar 2001, Jason Lixfeld wrote:
Are we talking business grade DSL or Resi grade DSL?
-----Original Message----- From: Roeland Meyer [mailto:rmeyer@mhsc.com] Sent: Sunday, March 25, 2001 3:04 PM To: 'michael'; nanog@merit.edu Subject: RE: dsl providers that will route /24
If you find such, I want to know too.
-----Original Message----- From: michael [mailto:michael@aplatform.com] Sent: Sunday, March 25, 2001 8:08 AM To: nanog@merit.edu Subject: dsl providers that will route /24
Hello,
Looking for contact information on dsl providers that route IP space not within their assigned block. I am also curious on high speed wireless doing the same.
Thanks,
Michael...
-------------------------------------------------------------- ----------- This email server is running an evaluation copy of the MailShield anti- spam software. Please contact your email administrator if you have any questions about this message. MailShield product info: www.mailshield.com .
Try www.rhythms.com. michael wrote:
Hello,
Business grade. And it was suggested that I follow up with more specific location.
California, San Francisco bay area.
Thanks,
Michael...
On Sun, 25 Mar 2001, Jason Lixfeld wrote:
Are we talking business grade DSL or Resi grade DSL?
-----Original Message----- From: Roeland Meyer [mailto:rmeyer@mhsc.com] Sent: Sunday, March 25, 2001 3:04 PM To: 'michael'; nanog@merit.edu Subject: RE: dsl providers that will route /24
If you find such, I want to know too.
-----Original Message----- From: michael [mailto:michael@aplatform.com] Sent: Sunday, March 25, 2001 8:08 AM To: nanog@merit.edu Subject: dsl providers that will route /24
Hello,
Looking for contact information on dsl providers that route IP space not within their assigned block. I am also curious on high speed wireless doing the same.
Thanks,
Michael...
-------------------------------------------------------------- ----------- This email server is running an evaluation copy of the MailShield anti- spam software. Please contact your email administrator if you have any questions about this message. MailShield product info: www.mailshield.com .
-- ------------------------------------------------------------ Roland Dobbins <rdobbins@netmore.net> // 408.859.4137 voice
Jason Lixfeld wrote:
Are we talking business grade DSL or Resi grade DSL?
Residential DSL is typically a bridged solution, meaning you don't even get to use a router... which renders the answer to this question a moot point... -- Steven J. Sobol/CTO/JustThe.net LLC | sjsobol@NorthShoreTechnologies.net SAY IT LOUD: I'M GEEK AND I'M PROUD! | 888.480.4NET (4638) 216.619.2NET (2638) http://NorthShoreTechnologies.net | http://ClevelandProductions.com http://JustThe.net | Powered by Linux, pizza, Coke, Cuervo, and cheap beer.
Why? You can announce the space on the clients' behalf... On Mon, 26 Mar 2001, Steve Sobol wrote:
Jason Lixfeld wrote:
Are we talking business grade DSL or Resi grade DSL?
Residential DSL is typically a bridged solution, meaning you don't even get to use a router... which renders the answer to this question a moot point...
-- Steven J. Sobol/CTO/JustThe.net LLC | sjsobol@NorthShoreTechnologies.net SAY IT LOUD: I'M GEEK AND I'M PROUD! | 888.480.4NET (4638) 216.619.2NET (2638) http://NorthShoreTechnologies.net | http://ClevelandProductions.com http://JustThe.net | Powered by Linux, pizza, Coke, Cuervo, and cheap beer.
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben -- -- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
I said
Residential DSL is typically a bridged solution, meaning you don't even get to use a router... which renders the answer to this question a moot point...
Alex Rubenstein questioned:
Why? You can announce the space on the clients' behalf...
Ok, and what will they do with it, since I don't know of any residential DSL services that offer routing? **SJS (who is apparently missing something obvious) -- Steven J. Sobol/CTO/JustThe.net LLC | sjsobol@NorthShoreTechnologies.net SAY IT LOUD: I'M GEEK AND I'M PROUD! | 888.480.4NET (4638) 216.619.2NET (2638) http://NorthShoreTechnologies.net | http://ClevelandProductions.com http://JustThe.net | Powered by Linux, pizza, Coke, Cuervo, and cheap beer.
Just becuase the medium is a layer 2 bridge between you doesn't mean they couldn't stick a cisco xxxx and run BGP on the other end, ethernet to 'ethernet (really BVI). ISP|-atm0------|DSL Cloud|-------|DSL bridge|-----e0-|cisco something| On Mon, 26 Mar 2001, Steve Sobol wrote:
I said
Residential DSL is typically a bridged solution, meaning you don't even get to use a router... which renders the answer to this question a moot point...
Alex Rubenstein questioned:
Why? You can announce the space on the clients' behalf...
Ok, and what will they do with it, since I don't know of any residential DSL services that offer routing?
**SJS (who is apparently missing something obvious)
-- Steven J. Sobol/CTO/JustThe.net LLC | sjsobol@NorthShoreTechnologies.net SAY IT LOUD: I'M GEEK AND I'M PROUD! | 888.480.4NET (4638) 216.619.2NET (2638) http://NorthShoreTechnologies.net | http://ClevelandProductions.com http://JustThe.net | Powered by Linux, pizza, Coke, Cuervo, and cheap beer.
-- Alex Rubenstein, AR97, K2AHR, alex@nac.net, latency, Al Reuben -- -- Net Access Corporation, 800-NET-ME-36, http://www.nac.net --
Just becuase the medium is a layer 2 bridge between you doesn't mean they couldn't stick a cisco xxxx and run BGP on the other end, ethernet to 'ethernet (really BVI).
ISP|-atm0------|DSL Cloud|-------|DSL bridge|-----e0-|cisco something|
Why not just find someone else willing to advertise your /24 and tunnel your packets to you? DS
> Why not just find someone else willing to advertise your /24 and tunnel > your packets to you? Because he'd be using three times as much bandwidth, perhaps? -Bill
Why not just find someone else willing to advertise your /24 and tunnel your packets to you?
Because he'd be using three times as much bandwidth, perhaps?
-Bill
Three times as much is absolute worst case. In reality, it's more like twice as much for just his incoming traffic. In any event, an entry in the global routing table for a DSL line is not exactly polite networking anyway, so what's a bit more excess? DS
> Three times as much is absolute worst case. In reality, it's more like > twice as much for just his incoming traffic. Uh, how do you figure? Each inbound packet comes into the tunnel-host site, out of the tunnel-host site, and into the DSL host site. Each outbound packet takes the reverse path. Three times as much bandwidth. -Bill
[ On Monday, March 26, 2001 at 08:53:08 (-0800), Bill Woodcock wrote: ]
Subject: RE: dsl providers that will route /24
> Three times as much is absolute worst case. In reality, it's more like > twice as much for just his incoming traffic.
Uh, how do you figure? Each inbound packet comes into the tunnel-host site, out of the tunnel-host site, and into the DSL host site. Each outbound packet takes the reverse path. Three times as much bandwidth.
Well, yes, but only if the tunnel host has only one network connection. If the tunnel host is also connected to the same DSL network then things are just peachy for both parties and the DSL provider sees almost none (just that which is NAT'ed) of the DSL user's bandwidth on their upstream(s) (though the DSL user's bandwidth in either direction will be limited to the lower of either the tunnel host's primary upstream link, or the uplink bandwidth of the tunnel host's DSL connection). This can still be a great benefit to a small network, especialy if the DSL provider has a nice big HTTP cache and maybe an NNTP server too, etc. I ran my home network with basically this setup for over a year on a cable modem and other than the fact that the cable provider has an extrememly broken internal network (7% *minimum* loss!), I had great success. -- Greg A. Woods +1 416 218-0098 VE3TCP <gwoods@acm.org> <robohack!woods> Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>
Three times as much is absolute worst case. In reality, it's more like twice as much for just his incoming traffic.
Uh, how do you figure? Each inbound packet comes into the tunnel-host site, out of the tunnel-host site, and into the DSL host site. Each outbound packet takes the reverse path. Three times as much bandwidth.
-Bill
Each inbound packet goes from its source to the tunnel-host, then from the tunnel-host to its destination. That's two transits instead of one. If the tunnel-host is very close to the destination, the added leg will tend to be less than the first leg, so even a doubling may be an overestimate. As for outbound packets, why do they need to take the reverse path? There's no reason the tunnel can't be unidirectional. Even if the ISP is stupid and filters its customers' legitimate traffic, forcing them to encapsulate the outbound packets, the same argument still applies. Obviously, you have to choose a tunnel-host wisely. Ideally, you would pick one that meets your DSL provider very closely. In any event, the argument that VPNs "waste Internet bandwidth" rings pretty hollow. People buy Internet access to have Internet bandwidth to use for whatever applications they have. Heck, I would argue that USENET is the biggest waste of Internet bandwidth there is. That doesn't mean it should go away. DS
I think you're getting caught up in semantics here. You're arguing about total bandwidth used on the Internet (let's say amount of data times distance traveled, for lack of a better metric), which will vary highly depending on the individual setup. Bill is talking about the amount of data sent over customer local loops, which in many (but of course not all) Internet access billing arrangements is what costs the customer money. Anyhow, quibbling over exact usage measurements doesn't really detract from Bill's main point, that tunneling in and out of one network to get to another uses more resources than not doing so. Hopefully we can all agree that tunnels/VPNs/whatever you want to call them use some resources, are extremely useful in some situations, but aren't the best solution for every possible problem, without having to get into yet another huge flame war. -Steve On Mon, 26 Mar 2001, David Schwartz wrote:
Three times as much is absolute worst case. In reality, it's more like twice as much for just his incoming traffic.
Uh, how do you figure? Each inbound packet comes into the tunnel-host site, out of the tunnel-host site, and into the DSL host site. Each outbound packet takes the reverse path. Three times as much bandwidth.
-Bill
Each inbound packet goes from its source to the tunnel-host, then from the tunnel-host to its destination. That's two transits instead of one. If the tunnel-host is very close to the destination, the added leg will tend to be less than the first leg, so even a doubling may be an overestimate.
As for outbound packets, why do they need to take the reverse path? There's no reason the tunnel can't be unidirectional. Even if the ISP is stupid and filters its customers' legitimate traffic, forcing them to encapsulate the outbound packets, the same argument still applies.
Obviously, you have to choose a tunnel-host wisely. Ideally, you would pick one that meets your DSL provider very closely.
In any event, the argument that VPNs "waste Internet bandwidth" rings pretty hollow. People buy Internet access to have Internet bandwidth to use for whatever applications they have. Heck, I would argue that USENET is the biggest waste of Internet bandwidth there is. That doesn't mean it should go away.
DS
-------------------------------------------------------------------------------- Steve Gibbard scg@gibbard.org
On Mon, Mar 26, 2001 at 12:40:45PM -0800, David Schwartz wrote:
As for outbound packets, why do they need to take the reverse path? There's no reason the tunnel can't be unidirectional. Even if the ISP is stupid and filters its customers' legitimate traffic, forcing them to encapsulate the outbound packets, the same argument still applies.
Ummm, s/if the ISP is stupid/if the ISP is doing the right thing/ You do filter what source addresses your customers can use, don't you? -- John Payne http://www.sackheads.org/jpayne/ john@sackheads.org http://www.sackheads.org/uce/ Fax: +44 870 0547954 To send me mail, use the address in the From: header
On Mon, Mar 26, 2001 at 12:40:45PM -0800, David Schwartz wrote:
As for outbound packets, why do they need to take the reverse path? There's no reason the tunnel can't be unidirectional. Even if the ISP is stupid and filters its customers' legitimate traffic, forcing them to encapsulate the outbound packets, the same argument still applies.
Ummm, s/if the ISP is stupid/if the ISP is doing the right thing/
You do filter what source addresses your customers can use, don't you?
No, I don't. If I see illegitimate traffic, I block it. If I see suspicious traffic, I investigate it. But I give my customers the benefit of the doubt. They pay me for Internet access. That means they can do whatever they want with the Internet provided it's legal and doesn't impose an undue burden on anyone else using the Internet. A one-way VPN is a legitimate use and shouldn't be subject to prior restraint. On the other hand, if I saw a customer abusing this privilege, I would definitely *NOT* respond with a filter (except maybe as a stopgap until I could contact the relvant administrators). The fact is, silently covering over a problem doesn't help anyone. In specific, it doesn't help my customer find the problem, which is most likely a root compromise on one of their machines. It is, IMO, stupid to hide a serious problem with a filter. That won't make the problem go away. In this instance, the problem is a compromised machine, a misconfiguration, or a customer who is trying to launch network attacks. I'm sure we've all heard stories of major network disruptions being caused by this type of filtering policy. ISP1 filters routes it hears from CUSTOMER1. So the fact the CUSTOMER1's filters are broken is never noticed. Then one day, ISP1 accidentally breaks its filters. Boom! Filtering should be a last resort if there is no other way to accomplish the desired goal or where small misconfigurations on the other end have the ability to cause massive damage in a very small amount of time. Filtering should _never_ be used to hide a real problem unless there is absolutely no other option. In this case, there are *many* other options. DS
On Tue, Mar 27, 2001 at 01:22:26PM -0800, David Schwartz had this to say:
They pay me for Internet access. That means they can do whatever they want
In the interests of preserving bandwidth (and Susan's sanity :) ) can we please head off this religious war before it starts again? We have flamed each other about this at least 3 times in the past 6 months already (e.g. the "they pay me, they can do what they like" vs. "enforce the Right Thing" argument) -- Scott Francis scott@ [work:] v i r t u a l i s . c o m Systems Analyst darkuncle@ [home:] d a r k u n c l e . n e t PGP fingerprint 7ABF E2E9 CD54 A1A8 804D 179A 8802 0FBA CB33 CCA7 illum oportet crescere me autem minui
On Tue, 27 Mar 2001, David Schwartz wrote:
I'm sure we've all heard stories of major network disruptions being caused by this type of filtering policy. ISP1 filters routes it hears from CUSTOMER1. So the fact the CUSTOMER1's filters are broken is never noticed. Then one day, ISP1 accidentally breaks its filters. Boom!
Um, look at what you wrote. Filter breaks, Boom. We egress filter on our customers routers and ingress filter on our routers. That way, in the event of either breaking down, there is (hopefully) still an appropriate filter in place to prevent a Boom!
Filtering should be a last resort if there is no other way to accomplish the desired goal or where small misconfigurations on the other end have the ability to cause massive damage in a very small amount of time. Filtering should _never_ be used to hide a real problem unless there is absolutely no other option. In this case, there are *many* other options.
DS
Forgive me if I (and the vast majority of the network ops I know) don't subscribe to this point of view. Filter, Filter, Filter. If you want to know about broken customer filters, filter on their ingress to your network with logging. Flat out not filtering just so you know when "there is a problem" is, in my humble opinion, irresponsible network administration. --- John Fraizer EnterZone, Inc
On Tue, 27 Mar 2001, David Schwartz wrote:
I'm sure we've all heard stories of major network disruptions being caused by this type of filtering policy. ISP1 filters routes it hears from CUSTOMER1. So the fact the CUSTOMER1's filters are broken is never noticed. Then one day, ISP1 accidentally breaks its filters. Boom!
Um, look at what you wrote. Filter breaks, Boom.
Exactly, because the presence of the filter allowed the actual problem to be ignored. In this case, the error isn't in applying the filter --- you have to in this case because too much damage could be done too quickly without it. The error was in pretending that the filter solved the problem.
We egress filter on our customers routers and ingress filter on our routers. That way, in the event of either breaking down, there is (hopefully) still an appropriate filter in place to prevent a Boom!
Do you confirm that the egree filters are working? Or do you assume that the presence of the ingress filters allows you to not worry about it? Filters are a necessary tool in cases where large amounts of damage can be done in very small amounts of time. But they don't solve any actual problems, they just minimize the damage.
Filtering should be a last resort if there is no other way to accomplish the desired goal or where small misconfigurations on the other end have the ability to cause massive damage in a very small amount of time. Filtering should _never_ be used to hide a real problem unless there is absolutely no other option. In this case, there are *many* other options.
Forgive me if I (and the vast majority of the network ops I know) don't subscribe to this point of view. Filter, Filter, Filter. If you want to know about broken customer filters, filter on their ingress to your network with logging.
The problem is, the filter will block legitimate traffic. IP does not provide any sure way to tell a spoofed packet from an unspoofed packet. Do an informal survey. Ask network operators who ingress filter whether they log and investigate packets that hit the filter. I will bet you that more than 2/3 say they don't. In other words, the filter substitutes for fixing the problem, and the problem could be as serious as a root compromise.
Flat out not filtering just so you know when "there is a problem" is, in my humble opinion, irresponsible network administration.
That's not the reason I don't filter. I don't filter because the filter will stop legitimate traffic and isn't necessary if the network is competently administered. So long as you can quickly resolve the problem when there is actual abuse, the filter is not the right solution. DS
On Tue, 27 Mar 2001 15:18:08 PST, David Schwartz said:
The problem is, the filter will block legitimate traffic. IP does not provide any sure way to tell a spoofed packet from an unspoofed packet.
Hmm.. if I *know* that my customer has a single-homed /24, and I see a packet come in from his /24 that has a source address outside that /24, there's a *pretty* *good* chance that something squirrely is going on. But we *know* that this crowd is a "tough room" - we just *had* a flame fest regarding filtering RFC1918 addresses. So I won't go there again. ;)
Do an informal survey. Ask network operators who ingress filter whether they log and investigate packets that hit the filter. I will bet you that more than 2/3 say they don't. In other words, the filter substitutes for
And a survey of DNS servers quite recently showed that 16% still haven't upgraded to non-hackable versions of BIND. A lot of people drive without seat belts too. Just because 2/3 of a group do something doesn't mean it's a good idea. Valdis Kletnieks Operating Systems Analyst Virginia Tech
On Tue, 27 Mar 2001 15:18:08 PST, David Schwartz said:
The problem is, the filter will block legitimate traffic. IP does not provide any sure way to tell a spoofed packet from an unspoofed packet.
Hmm.. if I *know* that my customer has a single-homed /24, and I see a packet come in from his /24 that has a source address outside that /24, there's a *pretty* *good* chance that something squirrely is going on.
Right. However, it's also entirely possible that the traffic is legitimate. Consider, for example, a sub-customer migrating between two ISPs each with static IPs who currently has them both up. The optimum approach is to investigate it, determine if it's legitimate or not, and act appropriately. The lazy approach is to filter it and if it's legitimate, wait for the customer to complain. The worst possible approach is to ignore it (either filtering it or not) and hope that if it is a serious problem, the customer will fix it themself. The filtering advocates don't seem to particularly care whether the problem is fixed or not. What they're missing is that filtering is simply a 'level of service' issue. What's a security and community issue is that root compromises and misconfigurations that threaten others be detected and repaired. Filters can't do that. DS
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 28 Mar 2001, David Schwartz wrote:
On Tue, 27 Mar 2001 15:18:08 PST, David Schwartz said:
The problem is, the filter will block legitimate traffic. IP does not provide any sure way to tell a spoofed packet from an unspoofed packet.
Hmm.. if I *know* that my customer has a single-homed /24, and I see a packet come in from his /24 that has a source address outside that /24, there's a *pretty* *good* chance that something squirrely is going on.
Right. However, it's also entirely possible that the traffic is legitimate. Consider, for example, a sub-customer migrating between two ISPs each with static IPs who currently has them both up.
The optimum approach is to investigate it, determine if it's legitimate or not, and act appropriately. The lazy approach is to filter it and if it's legitimate, wait for the customer to complain. The worst possible approach is to ignore it (either filtering it or not) and hope that if it is a serious problem, the customer will fix it themself.
The filtering advocates don't seem to particularly care whether the problem is fixed or not. What they're missing is that filtering is simply a 'level of service' issue. What's a security and community issue is that root compromises and misconfigurations that threaten others be detected and repaired. Filters can't do that.
No, no, no. You are erring on the side of openness, rather than on the side of security. If you do have a root compromise and someone is able to send out spoofed packets from that system on your network, how are you supposed to know? If you are not filtering spoofed packets, you have no way to know there there is anything going wrong on your network. As far as traffic outside of my address space being legitimate on my network. No. The only traffic outside my address space allowed on my network (if I allowed it at all) would be pre-arranged addresses. If this is the case, your scenario above would be included. This is a policy statment that should be clearly defined by each ISPs routing policy. "I only route my packets unless otherwise arranged". If another ISP connects to me, I will give them an address range, or, if they have their own address range, I will route that. Routing of non-portable address space will only be made under special circumstances. yada yada ya." You get the idea. If you err on the side of security, you keep everything closed until you have to open it up. Just like a firewall. This way you KNOW what is being passed and what will be allowed to be passed when you setup the connection to the customer. If you have and have always had egress filtering, then there is no problem when setting up a new customer. If you add it after the fact, you have to be very careful and explain to the customer why it needs to be there. If everyone edge network on the Internet did egress filtering, we wouldn't have the problem of spoofed packets today. Period. It's that simple. === Tim Flames are welcome. :-) ********************************************** Tim Winders, MCSE, CNE, CCNA Associate Dean of Information Technology South Plains College Levelland, TX 79336 Phone: 806-894-9611 x 2369 FAX: 806-894-1549 Email: TWinders@SPC.cc.tx.us ********************************************** -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (OSF1) Comment: Made with pgp4pine 1.75-6 iEYEARECAAYFAjrCXLsACgkQTPuHnIooYbyPIACgoqTCfd0oEkeZkmct7PmYxBt0 BjgAoK8QMTR2+MR8gm+f4a4EZFpW9vT2 =PqMy -----END PGP SIGNATURE-----
No, no, no. You are erring on the side of openness, rather than on the side of security.
Exactly! And that's the crux of the issue here. We are not talking about a firewall. We are not talking about a military installation. We are talking about our customers, and we should be taking an 'innocent until proven guilty' approach with them whenever it is reasonably possible to do so. There are some cases where it certainly isn't possible to do so. BGP route filtering is a great example. An unfiltered connection could allow a misconfigured customer to do massive amounts of damage very quickly. That's not tolerable.
If you do have a root compromise and someone is able to send out spoofed packets from that system on your network, how are you supposed to know? If you are not filtering spoofed packets, you have no way to know there there is anything going wrong on your network.
Perhaps youa re using the term "filtering" differently from the way I am. When I say "filtering", I'm referring to blocking. Logging and analyzing is wonderful. Filtering is neutral (can be good or bad depending upon many factors).
As far as traffic outside of my address space being legitimate on my network. No. The only traffic outside my address space allowed on my network (if I allowed it at all) would be pre-arranged addresses. If this is the case, your scenario above would be included.
This is a level of service issue. If you want, you can coerce your customers to pre-arrange what IPs they can use on your service. This may make things harder for their customers, but you can do it if you want to. Fine with me, I don't care. (But think long and hard before coercing your customers into an arrangement you yourself couldn't live with.) I think you misunderstand me. I don't oppose source address filtering. Heck, you can shut off your customers from 9 AM to 12 PM if you want to and if they agree to it. That's a level of service issue between you and your customers. I'll go one further -- if you're not going to investigate suspicious traffic (because it's too expensive or you're too lazy or whatever), it's probably better that you filter than not. At least that way you might minimize the damage done to others, and that's certainly a good thing. I don't have a problem with filtering traffic that can't possibly be legitimate. If you're one of those people who agrees that packets with RFC1918 source IPs have no place on the Internet, then filter that. You can even advocate that others filter it, because it has no possibility of blocking legitimate traffic. What I do oppose is militant filtering advocacy where those filters will filter out legitimate traffic. ISP's should not feel coerced into "erring on the side of security" by filtering their customer's possibly legitimate traffic when there are reasonable alternatives. In this case, there is -- allow, analyze, follow up, filter if and where neccessary. What I also oppose is advocacy of filtering that claims that filtering fixes the problem. It doesn't, it just minimizes the damage. Hiding the fact that a misconfigured firewall is leaking packets with inside IPs or the fact that a machine has been root compromised (or worse, that the actual admin likes to launch DoS attacks) ultimately harms everyone. Another problem with the belief that ingress source address filtering is the ultimate solution to the problem of spoofed packets is that it makes it too easy to ignore the fact that there really is a problem. After all, if filtering solves the problem perfectly, there's no need to work on a solution, all you have to do is militantly insist that everyone filter. On the other hand, if there's a general understanding that filtering is only one possible solution that has problems of its own, perhaps they'll continue to work on much better solutions. DS
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 David - I will start off by saying this was the most intelligent reply I have read from you. Thank you. Now...
I'll go one further -- if you're not going to investigate suspicious traffic (because it's too expensive or you're too lazy or whatever), it's probably better that you filter than not. At least that way you might minimize the damage done to others, and that's certainly a good thing.
Yes, I agree with that. From what I have seen, this is the problem. Many ISPs/Corporations/Whoever that do not do egress blocking, also do not do any type of log analysis or even logging of suspicious traffic to be analyzed. I am willing to back down on my militant stance and say that if you are willing to take the time and energy to log, analyze, and track down questionable traffic, then you could be exempt from egress blocking.
I don't have a problem with filtering traffic that can't possibly be legitimate. If you're one of those people who agrees that packets with RFC1918 source IPs have no place on the Internet, then filter that. You can even advocate that others filter it, because it has no possibility of blocking legitimate traffic.
I'm glad you agree with that. There are many others who do not. I agree that core routers have no business filtering/blocking. That function belongs on the edge. But, I do believe edge routers should have ingress and egress filtering to help minimize security threats. (Unless you are willing to track it down as above).
What I also oppose is advocacy of filtering that claims that filtering fixes the problem. It doesn't, it just minimizes the damage. Hiding the fact that a misconfigured firewall is leaking packets with inside IPs or the fact that a machine has been root compromised (or worse, that the actual admin likes to launch DoS attacks) ultimately harms everyone.
This is true. However, I err on the side of caution, blocking that traffic and following up why it was there in the first place. As I have the policies in place that prohibit such traffic, there is nothing legitimate in the first place. Again, as long as you followup on the problem, I can see where not having an egress filter would be OK.
Another problem with the belief that ingress source address filtering is the ultimate solution to the problem of spoofed packets is that it makes it too easy to ignore the fact that there really is a problem. After all, if filtering solves the problem perfectly, there's no need to work on a solution, all you have to do is militantly insist that everyone filter. On the other hand, if there's a general understanding that filtering is only one possible solution that has problems of its own, perhaps they'll continue to work on much better solutions.
The solution is for everyone to log/analyze/inspect the traffic on their network. Unfortunately, that's just not done. I do have ingress/egress filtering. I used to log all the RFC1918 crap coming into my network. Unfortunately, when talking with upstream providers who are "leaking" these, I would always get: "not from us", or "can't track it, sorry", or "you are filtering it, why do you care?". So, I gave up logging and tracking it down. I also have ingress filtering to block my own addresses from coming into my network. I rarely see these type of packets coming into my network, but when I do, I try to track them down. Unfortunately, I usually get the same type responses as above. No one seems to care. Because of my experience in trying to track down problems, I have come to be militant about egress filtering. === Tim ********************************************** Tim Winders, MCSE, CNE, CCNA Associate Dean of Information Technology South Plains College Levelland, TX 79336 Phone: 806-894-9611 x 2369 FAX: 806-894-1549 Email: TWinders@SPC.cc.tx.us ********************************************** -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (OSF1) Comment: Made with pgp4pine 1.75-6 iEYEARECAAYFAjrCgxUACgkQTPuHnIooYbxsCgCggoLqZJUASl9fsV3LTsaQKbYQ 0wsAmQFtRrzH9DNGjW7Z1l/nu9RmehGy =4EHh -----END PGP SIGNATURE-----
I think we're in 99.9% percent agreement, which is probably about the best you can expect between two human beings. Let me respond to one thing:
I do have ingress/egress filtering. I used to log all the RFC1918 crap coming into my network. Unfortunately, when talking with upstream providers who are "leaking" these, I would always get: "not from us", or "can't track it, sorry", or "you are filtering it, why do you care?". So, I gave up logging and tracking it down.
I also have ingress filtering to block my own addresses from coming into my network. I rarely see these type of packets coming into my network, but when I do, I try to track them down. Unfortunately, I usually get the same type responses as above. No one seems to care.
Because of my experience in trying to track down problems, I have come to be militant about egress filtering.
I do agree with you that most large ISPs don't seem to care about what comes out of their pipes and will just tell you that if you don't like what you're getting from them, you should filter it. However, that doesn't mean that you should take that same attitude with your customers. I do have ingress filtering to block packets with origin IP addresses assigned to my own machines and LANs. I don't have ingress filtering on transit or peer connections for IPs subassigned to my customers. Some customers might see such filtering as a service, some might see it as a detriment that limits their own topological flexibility. What happens if one of your customers is multihomed, loses his link to you, and tries to reach another one of your customers through his other ISP? Or do you make exceptions to this filter for multihomed customers? (The problem is, with VPNs and mobile IP schemes, every customer is potentially multihomed.) IMO, this is something best done on the customer's routers. Obviously, for your own 'local' IPs, you are the customer. DS
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I think we're in 99.9% percent agreement, which is probably about the best you can expect between two human beings. Let me respond to one thing:
Agreed!
What happens if one of your customers is multihomed, loses his link to you, and tries to reach another one of your customers through his other ISP? Or do you make exceptions to this filter for multihomed customers? (The problem is, with VPNs and mobile IP schemes, every customer is potentially multihomed.)
IMO, this is something best done on the customer's routers. Obviously, for your own 'local' IPs, you are the customer.
Again, agreed. I only ingress filter on my own addresses, RFC918 and a long list of other addresses which are "bad". I wouldn't filter out a customer's addresses. I would give them instructions on how to do so and why it's a good thing. :-) === Tim ********************************************** Tim Winders, MCSE, CNE, CCNA Associate Dean of Information Technology South Plains College Levelland, TX 79336 Phone: 806-894-9611 x 2369 FAX: 806-894-1549 Email: TWinders@SPC.cc.tx.us ********************************************** -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (OSF1) Comment: Made with pgp4pine 1.75-6 iEYEARECAAYFAjrCjwkACgkQTPuHnIooYbzSFACghjFsDa0n0bMQKjxBw9/Z9W9S 3+UAoM3t0xUcffHTmnliriGLneGwJALV =KQ24 -----END PGP SIGNATURE-----
On Wed, 28 Mar 2001, David Schwartz wrote:
We are not talking about a firewall. We are not talking about a military installation. We are talking about our customers, and we should be taking an 'innocent until proven guilty' approach with them whenever it is reasonably possible to do so.
What's wrong with writing it into the contract that you only let packets pass out of their network with source addresses that you've assigned them? You could also state that you will let other networks out as they see fit.
There are some cases where it certainly isn't possible to do so. BGP route filtering is a great example.
Yes, so is filtering packets with forged source addresses.
An unfiltered connection could allow a misconfigured customer to do massive amounts of damage very quickly. That's not tolerable.
Note how this applies to filtering source IPs.
Perhaps youa re using the term "filtering" differently from the way I am. When I say "filtering", I'm referring to blocking. Logging and analyzing is wonderful. Filtering is neutral (can be good or bad depending upon many factors).
OK, so one of your customers, who is being 'watched' but not filtered has all 30 of his Linux boxes rooted. He then proceeds to launch a massive DOS attack against me. I guess you'd notice this when it's convenient? Or do you have an intricate log-watching utility that will page you out of bed. I can't call you, because I don't know where the traffic is really coming from.
This is a level of service issue. If you want, you can coerce your customers to pre-arrange what IPs they can use on your service. This may make things harder for their customers, but you can do it if you want to. Fine with me, I don't care. (But think long and hard before coercing your customers into an arrangement you yourself couldn't live with.)
If they know enough to be talking BGP with two providers, they likely know enough to tell you what the IPs they are announcing are. Why is it such a big deal to simply put "sanity filters" on? This argument seems to be drawn between 'those who've been attacked from a non-filtered connection' and 'those who haven't been attacked by same'. Charles [...]
DS
On Wed, Mar 28, 2001, David Schwartz wrote:
No, no, no. You are erring on the side of openness, rather than on the side of security.
Exactly! And that's the crux of the issue here.
We are not talking about a firewall. We are not talking about a military installation. We are talking about our customers, and we should be taking an 'innocent until proven guilty' approach with them whenever it is reasonably possible to do so.
.. on todays internet? Right. You've never been hit by a DoS. That you can't trace. My cable modem provider filters. Good on them. It means that when I set up my tunnel for my "portable /24", I had to think hard for 5 minutes to convince FreeBSD's ipfw that it wanted to stuff all packets from my /24 out the tunnel, rather than just defaulting. <irony> It was really hard. Really. </irony>
What I also oppose is advocacy of filtering that claims that filtering fixes the problem. It doesn't, it just minimizes the damage. Hiding the fact that a misconfigured firewall is leaking packets with inside IPs or the fact that a machine has been root compromised (or worse, that the actual admin likes to launch DoS attacks) ultimately harms everyone.
The internet is misconfigured. The internet is designed around a trusted non-authenticated layer 3, and this layer can be used for good *AND* evil. Now if only responsible people were allowed on said internet, then the lack of protection wouldn't be an issue. Do you remember the Great SMTP Relay Closing a few years ago? The relays were open, until people started using them for increasing amounts of spam. Now, us Responsible People(tm) knew that open relays were great for when you were roaming or your outgoing relay was broken, but now ..?
Another problem with the belief that ingress source address filtering is the ultimate solution to the problem of spoofed packets is that it makes it too easy to ignore the fact that there really is a problem. After all, if filtering solves the problem perfectly, there's no need to work on a solution, all you have to do is militantly insist that everyone filter. On the other hand, if there's a general understanding that filtering is only one possible solution that has problems of its own, perhaps they'll continue to work on much better solutions.
Enforce filtering, or replace the internet infrastructure with something better? Guess whats a better choice technically, guess whats got a better choice of happening[0] ? Adrian [0] With the attitude people have for filtering, I guess neither. ;-) -- Adrian Chadd "The fact you can download a 100 megabyte file <adrian@creative.net.au> from half way around the world should be viewed as an accident and not a right." -- Adrian Chadd and Bill Fumerola
Adrian Chadd wrote:
Do you remember the Great SMTP Relay Closing a few years ago? The relays were open, until people started using them for increasing amounts of spam. Now, us Responsible People(tm) knew that open relays were great for when you were roaming or your outgoing relay was broken, but now ..
...people have come up with ways to allow people to roam that aren't wide open to abuse. :) -- Steven J. Sobol/CTO/JustThe.net LLC | sjsobol@NorthShoreTechnologies.net SAY IT LOUD: I'M GEEK AND I'M PROUD! | 888.480.4NET (4638) 216.619.2NET (2638) http://NorthShoreTechnologies.net | http://ClevelandProductions.com http://JustThe.net | Powered by Linux, pizza, Coke, Cuervo, and cheap beer.
-- Jason Slagle - CCNP - CCDA Network Administrator - Toledo Internet Access - Toledo Ohio - raistlin@tacorp.net - jslagle@toledolink.com - WHOIS JS10172 /"\ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . \ / ASCII Ribbon Campaign . If dreams are like movies then memories X - NO HTML/RTF in e-mail . are films about ghosts.. / \ - NO Word docs in e-mail . - Adam Duritz - Counting Crows On Wed, 28 Mar 2001, David Schwartz wrote:
I'll go one further -- if you're not going to investigate suspicious traffic (because it's too expensive or you're too lazy or whatever), it's probably better that you filter than not. At least that way you might minimize the damage done to others, and that's certainly a good thing.
I don't have a problem with filtering traffic that can't possibly be legitimate. If you're one of those people who agrees that packets with RFC1918 source IPs have no place on the Internet, then filter that. You can even advocate that others filter it, because it has no possibility of blocking legitimate traffic.
What I do oppose is militant filtering advocacy where those filters will filter out legitimate traffic. ISP's should not feel coerced into "erring on the side of security" by filtering their customer's possibly legitimate traffic when there are reasonable alternatives. In this case, there is -- allow, analyze, follow up, filter if and where neccessary.
Thats all well and good if you are going to have someone monitor the logs of these packets 24x7, but if you have a customer get hacked and start spewing shitloads of spoofed sourced packets at various networks (Insert your favorite DDOS Drone here), then the damage is high, immediate, and done by the time you notice it in most cases. Jason
Thats all well and good if you are going to have someone monitor the logs of these packets 24x7, but if you have a customer get hacked and start spewing shitloads of spoofed sourced packets at various networks (Insert your favorite DDOS Drone here), then the damage is high, immediate, and done by the time you notice it in most cases.
Jason
They could do almost exactly the same amount of damage with an unspoofed UDP flood and it would still take a human action to stop it. The attack can still hop from victim to victim until the problem is stopped at its source. The problem still won't get stopped at its source until someone with the ability to stop it is summoned and alterted to the problem. Odds are, an attacker will used spoofed packets if he can. potentially spoofed packets will trigger an investigation on my network. An unspoofed UDP flood probably won't (especially if it hops from victim to victim). So if the attacker uses spoofed packets, he may get cut off at the source (and the problem actually solved) sooner. On the other hand, unspoofed packets will probably trigger a call to the administration of the source network faster. Of course, you don't know that attack is unspoofed, so you really can't be sure what the source is. The important thing to realize is that neither of these situations is ideal. That is, filters don't solve the problem. We need to acknowledge that we have a problem and don't have a solution to it. Only then will the problem be analyzed, solutions proposed, and implemented. One possibility is a hop-by-hop reverse tracing protocol. Another possibility is some form of source authentication. For unspoofed floods, the solution may be a way to 'push' a filter up a chain of routers. I don't know, I'm not smart enough to solve the problem by myself. All I can do is keep yelling as loudly as I can that there is a problem and that we do need a really good solution. DS
They could do almost exactly the same amount of damage with an unspoofed UDP flood and it would still take a human action to stop it.
This is a false premise. I get hit with one-off attacks pretty often (oversized pings against my NT boxes, etc.), which are impossible to trace because of invalid source addresses. Source filters would mean that those attacks would be identifiable period, which they are not now. -- Eric A. Hall http://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
They could do almost exactly the same amount of damage with an unspoofed UDP flood and it would still take a human action to stop it.
This is a false premise. I get hit with one-off attacks pretty often (oversized pings against my NT boxes, etc.), which are impossible to trace because of invalid source addresses.
Source filters would mean that those attacks would be identifiable period, which they are not now.
Not so. You could still never be sure whether the attack was spoofed or not. That the address the attacks appear to come from employ source filters doesn't help you. At least if they're spoofed and the origin network logs packets that appear spoofed, the one off attack will be investigated and whatever caused it to happen will be actually fixed. If it's not spoofed, it won't trigger anything at its origin, and odds are the origin site will be unable to do anything because the attack may have been spoofed and there will be no local logs. So long as spoofing is possible, you cannot be sure where an attack came from unless you can either log it at its source or trace the stream to its source. That's the problem, and filters don't fix that. DS
On Thu, Mar 29, 2001 at 03:08:24PM -0800, David Schwartz wrote:
They could do almost exactly the same amount of damage with an unspoofed UDP flood and it would still take a human action to stop it.
This is a false premise. I get hit with one-off attacks pretty often (oversized pings against my NT boxes, etc.), which are impossible to trace because of invalid source addresses.
Source filters would mean that those attacks would be identifiable period, which they are not now.
Not so. You could still never be sure whether the attack was spoofed or not. That the address the attacks appear to come from employ source filters doesn't help you.
At least if they're spoofed and the origin network logs packets that appear spoofed, the one off attack will be investigated and whatever caused it to happen will be actually fixed. If it's not spoofed, it won't trigger anything at its origin, and odds are the origin site will be unable to do anything because the attack may have been spoofed and there will be no local logs.
So long as spoofing is possible, you cannot be sure where an attack came from unless you can either log it at its source or trace the stream to its source. That's the problem, and filters don't fix that.
"I don't filter spoofed packets because there are others that don't filter them" aiming to be so good at following the crowd that you're the last person there? -- John Payne http://www.sackheads.org/jpayne/ john@sackheads.org http://www.sackheads.org/uce/ Fax: +44 870 0547954 To send me mail, use the address in the From: header
Not so. You could still never be sure whether the attack was spoofed or not. That the address the attacks appear to come from employ source filters doesn't help you.
This is your excuse not to filter? That there is one other network which doesn't, so the network is already insecure? Would the network be more secure if you also filtered or less secure if you didn't? -- Eric A. Hall http://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/
Not so. You could still never be sure whether the attack was spoofed or not. That the address the attacks appear to come from employ source filters doesn't help you.
This is your excuse not to filter? That there is one other network which doesn't, so the network is already insecure?
Would the network be more secure if you also filtered or less secure if you didn't?
It's less secure if peoplee can spoof packets without detection. Filtering is one means of solving this problem. There are others. Filtering is not a perfect solution, there are others that are even better than filtering, they just take more work. DS
[ On Thursday, March 29, 2001 at 18:40:09 (-0800), David Schwartz wrote: ]
Subject: RE: dsl providers that will route /24
It's less secure if peoplee can spoof packets without detection. Filtering is one means of solving this problem. There are others. Filtering is not a perfect solution, there are others that are even better than filtering, they just take more work.
In an operations context anything that takes more work literally can not be better. Filters are automatic, tireless, and usually flawless too (at least once they're correctly deployed). -- Greg A. Woods +1 416 218-0098 VE3TCP <gwoods@acm.org> <robohack!woods> Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>
On Thu, Mar 29, 2001 at 06:40:09PM -0800, David Schwartz had this to say:
It's less secure if peoplee can spoof packets without detection. Filtering is one means of solving this problem. There are others. Filtering is not a perfect solution, there are others that are even better than filtering, they just take more work.
I personally prefer omniscience as a solution; however, it seems to be difficult to come by and is unsupported by most major vendors.
DS
-- Scott Francis scott@ [work:] v i r t u a l i s . c o m Systems Analyst darkuncle@ [home:] d a r k u n c l e . n e t PGP fingerprint 7ABF E2E9 CD54 A1A8 804D 179A 8802 0FBA CB33 CCA7 illum oportet crescere me autem minui
On Thu, 29 Mar 2001 15:08:24 PST, David Schwartz said:
So long as spoofing is possible, you cannot be sure where an attack came
And spoofing is possible because people don't filter.
from unless you can either log it at its source or trace the stream to its source. That's the problem, and filters don't fix that.
So we shouldn't filter at the source because we can't fix the problem unless we're filtering at the source? Or are you saying that because seat belts fail 1% of the time, we shouldn't use them to help in the other 99% of the crashes?
On Thu, 29 Mar 2001 15:08:24 PST, David Schwartz said:
So long as spoofing is possible, you cannot be sure where an attack came
And spoofing is possible because people don't filter.
No, spoofing is possible because the protocol has no source authentication capability.
from unless you can either log it at its source or trace the stream to its source. That's the problem, and filters don't fix that.
So we shouldn't filter at the source because we can't fix the problem unless we're filtering at the source?
I never said people shouldn't filter. I've always maintained that filters are a useful tool. Filters just don't solve this particular problem.
Or are you saying that because seat belts fail 1% of the time, we shouldn't use them to help in the other 99% of the crashes?
No. I'm saying that so long as spoofing is possible without detection, you can never be sure where an attack is really coming from without cooperation from the source network. These are real problems, and filtering doesn't solve them. DS
On Thu, 29 Mar 2001, David Schwartz rambled on as if he had a CLUE about:
Source filters would mean that those attacks would be identifiable period, which they are not now.
Not so. You could still never be sure whether the attack was spoofed or not. That the address the attacks appear to come from employ source filters doesn't help you.
David, you're only showing the WHOLE WORLD that you DON'T KNOW WHAT THE ^#&* YOU'RE TALKING ABOUT!!!!!! If someone tries to source an address we don't allow, it DIES INSIDE OUR NETWORK *AND LOGS AN ATTEMPT*!!! Lets look at this. It *DID NOT* make it out to the global internet and it *DID* catch our attention. *WIN WIN SITUATION*!!!!!!!!! Tell me where I'm wrong. PLEASE!
At least if they're spoofed and the origin network logs packets that appear spoofed, the one off attack will be investigated and whatever caused it to happen will be actually fixed. If it's not
You can NOT be this uneducated, can you? How can ALLOWING the attack to take place by not filtering be any better than BLOCKING it and seeing in the logs that it was attempted and thumping the appropraite customer? I've watched this go on for a week and I've come to the (hopefully mistaken) conclusion that you're just a lazy ass who refuses to do PREVENTATIVE filtering in hopes that there won't be a problem. The ONLY reason you could have for NOT filtering is that you hope that the NOC of the network being DoS'd will be able to track YOUR network down as a source and THUMP you their self! Either that or your customers are such dumb %&cks that they can't manage to tell you what source IP's they'll have.... In which case, THEY SHOULD BE FILTERED 100x over to begin with!
spoofed, it won't trigger anything at its origin, and odds are the origin site will be unable to do anything because the attack may have been spoofed and there will be no local logs.
What are you talking about? LOG AT INGRESS!!!! Investigate the logs. It's that simple. You just seem too %&*#^%#* lazy to do so.
So long as spoofing is possible, you cannot be sure where an attack came from unless you can either log it at its source or trace the stream to its source. That's the problem, and filters don't fix that.
Son, spoofing is possible AS LONG AS INGRESS CONNECTIONS ARE NOT FILTERED BY SOURCE ADDRESS! I'm tired and bored of people like you. Plain and simple. Consider yourself filtered as a preventative measure.
DS
My sentiments EXACTLY. --- John Fraizer EnterZone, Inc
On Thu, Mar 29, 2001 at 02:42:29PM -0800, David Schwartz wrote:
Thats all well and good if you are going to have someone monitor the logs of these packets 24x7, but if you have a customer get hacked and start spewing shitloads of spoofed sourced packets at various networks (Insert your favorite DDOS Drone here), then the damage is high, immediate, and done by the time you notice it in most cases.
Jason
They could do almost exactly the same amount of damage with an unspoofed UDP flood and it would still take a human action to stop it. The attack can still hop from victim to victim until the problem is stopped at its source. The problem still won't get stopped at its source until someone with the ability to stop it is summoned and alterted to the problem.
But the *unspoofed* packets are traceable. The victim can pick up the phone and call your operations and alert them.
Odds are, an attacker will used spoofed packets if he can. potentially spoofed packets will trigger an investigation on my network. An unspoofed UDP flood probably won't (especially if it hops from victim to victim).
Some of us that have been flooded don't appreciate playing the odds that the provider of the flooder will notice.
So if the attacker uses spoofed packets, he may get cut off at the source (and the problem actually solved) sooner. On the other hand, unspoofed packets will probably trigger a call to the administration of the source network faster. Of course, you don't know that attack is unspoofed, so you really can't be sure what the source is.
No, but it gives a good indication. And your NOC can find out if the packets are actually coming from your customer (unspoofed) or not (spoofed). If its unspoofed then we're on the phone to the right people. If its spoofed, we're SOL.
The important thing to realize is that neither of these situations is ideal. That is, filters don't solve the problem. We need to acknowledge that we have a problem and don't have a solution to it. Only then will the problem be analyzed, solutions proposed, and implemented.
Filters mean "least damage".
One possibility is a hop-by-hop reverse tracing protocol. Another possibility is some form of source authentication. For unspoofed floods, the solution may be a way to 'push' a filter up a chain of routers.
I don't know, I'm not smart enough to solve the problem by myself. All I can do is keep yelling as loudly as I can that there is a problem and that we do need a really good solution.
And until we get a really good solution, a really good workaround is not letting spoofed packets into your network from your customers. -- John Payne http://www.sackheads.org/jpayne/ john@sackheads.org http://www.sackheads.org/uce/ Fax: +44 870 0547954 To send me mail, use the address in the From: header
But the *unspoofed* packets are traceable. The victim can pick up the phone and call your operations and alert them.
If they were spoofed, they wouldn't have to because we'd already be investigating. And even if they're not spoofed, you can't know they're not spoofed, so there's no way to know you got the right person.
Odds are, an attacker will used spoofed packets if he can. potentially spoofed packets will trigger an investigation on my network. An unspoofed UDP flood probably won't (especially if it hops from victim to victim).
Some of us that have been flooded don't appreciate playing the odds that the provider of the flooder will notice.
Right, that's why every provider has to come up with some reasonable way to deal with this problem. Filtering is one, but it doesn't solve the whole problem. Monitoring is one, but it doesn't solve the whole problem either.
So if the attacker uses spoofed packets, he may get cut off at the source (and the problem actually solved) sooner. On the other hand, unspoofed packets will probably trigger a call to the administration of the source network faster. Of course, you don't know that attack is unspoofed, so you really can't be sure what the source is.
No, but it gives a good indication. And your NOC can find out if the packets are actually coming from your customer (unspoofed) or not (spoofed). If its unspoofed then we're on the phone to the right people. If its spoofed, we're SOL.
Well that's the real problem. Every attack is potentially spoofed and there are no good tools for dealing with spoofed attacks. Filtering doesn't solve either of those two problems.
The important thing to realize is that neither of these situations is ideal. That is, filters don't solve the problem. We need to acknowledge that we have a problem and don't have a solution to it. Only then will the problem be analyzed, solutions proposed, and implemented.
Filters mean "least damage".
Again, no. A unicast UDP flood can do just as much damage. So filters do not reduce the damage.
I don't know, I'm not smart enough to solve the problem by myself. All I can do is keep yelling as loudly as I can that there is a problem and that we do need a really good solution.
And until we get a really good solution, a really good workaround is not letting spoofed packets into your network from your customers.
Exactly -- the problem is there's no good way to tell a spoofed packet from an unspoofed packet. Some form of source authentication would solve that. DS
On Thu, Mar 29, 2001 at 06:40:03PM -0800, David Schwartz wrote:
But the *unspoofed* packets are traceable. The victim can pick up the phone and call your operations and alert them.
If they were spoofed, they wouldn't have to because we'd already be investigating. And even if they're not spoofed, you can't know they're not spoofed, so there's no way to know you got the right person.
So you automatically know about every single spoofed packet your customers send? If they're not spoofed, then the operations folk that I would be speaking to could verify that they're not spoofed by looking at their customer's traffic.
Odds are, an attacker will used spoofed packets if he can. potentially spoofed packets will trigger an investigation on my network. An unspoofed UDP flood probably won't (especially if it hops from victim to victim).
Some of us that have been flooded don't appreciate playing the odds that the provider of the flooder will notice.
Right, that's why every provider has to come up with some reasonable way to deal with this problem. Filtering is one, but it doesn't solve the whole problem. Monitoring is one, but it doesn't solve the whole problem either.
Filtering removes the "is this spoofed or not?" problem. We wouldn't be flooded with packets with sources in unallocated space. Oh, and smurf would disappear completely. Monitoring is reactive. Filtering is proactive. Filtering means I won't be flooded by spoofed packets coming from your customers. Filtering means I won't be smurfed by your customers. (Sure, they could still act as smurf amps, but they couldn't originate a smurf attack). Monitoring means I could still be flooded with spoofed packets, and you might notice and switch it off.
So if the attacker uses spoofed packets, he may get cut off at the source (and the problem actually solved) sooner. On the other hand, unspoofed packets will probably trigger a call to the administration of the source network faster. Of course, you don't know that attack is unspoofed, so you really can't be sure what the source is.
No, but it gives a good indication. And your NOC can find out if the packets are actually coming from your customer (unspoofed) or not (spoofed). If its unspoofed then we're on the phone to the right people. If its spoofed, we're SOL.
Well that's the real problem. Every attack is potentially spoofed and there are no good tools for dealing with spoofed attacks. Filtering doesn't solve either of those two problems.
If everyone filtered, it would solve the problem. If most people filter, it reduces the problem. If nobody filters it does nothing to address the problem. Do you want to help, or do you want to do as little as possible?
The important thing to realize is that neither of these situations is ideal. That is, filters don't solve the problem. We need to acknowledge that we have a problem and don't have a solution to it. Only then will the problem be analyzed, solutions proposed, and implemented.
Filters mean "least damage".
Again, no. A unicast UDP flood can do just as much damage. So filters do not reduce the damage.
If you're ingress filtering, when I pick up the phone I can call your operations folk and they can verify that your customer is flooding me, and stop the flood *quickly*. Speed means a reduction in damage. Sure, someone else could be spoofing your customers addresses, but your operations folk would be able to verify that your customer is not flooding me... so I'm back to square one. *but* even in this case, I'm no worse off than I am today... so putting filters in either means that attacks get shut down more quickly, or they get treated the same as today. It does not make things worse.
I don't know, I'm not smart enough to solve the problem by myself. All I can do is keep yelling as loudly as I can that there is a problem and that we do need a really good solution.
And until we get a really good solution, a really good workaround is not letting spoofed packets into your network from your customers.
Exactly -- the problem is there's no good way to tell a spoofed packet from an unspoofed packet. Some form of source authentication would solve that.
Or you could make sure you're not part of the problem by putting source address filters on your ingress. -- John Payne http://www.sackheads.org/jpayne/ john@sackheads.org http://www.sackheads.org/uce/ Fax: +44 870 0547954 To send me mail, use the address in the From: header
On Thu, Mar 29, 2001 at 06:40:03PM -0800, David Schwartz wrote:
But the *unspoofed* packets are traceable. The victim can pick up the phone and call your operations and alert them.
If they were spoofed, they wouldn't have to because we'd already be investigating. And even if they're not spoofed, you can't know they're not spoofed, so there's no way to know you got the right person.
So you automatically know about every single spoofed packet your customers send?
We know every pair of source and destination IP addresses and the number of packets and number of bytes. We also know (approximately) the start and stop times.
If they're not spoofed, then the operations folk that I would be speaking to could verify that they're not spoofed by looking at their customer's traffic.
Exactly. So you would need cooperation from the source network to establish the source of the attack.
Monitoring is reactive. Filtering is proactive. Filtering means I won't be flooded by spoofed packets coming from your customers. Filtering means I won't be smurfed by your customers. (Sure, they could still act as smurf amps, but they couldn't originate a smurf attack).
You will instead get flooded by unspoofed packets. And you'll have to start the process of tracking and fixing the problem.
Monitoring means I could still be flooded with spoofed packets, and you might notice and switch it off.
This is a classic example of a straw man. I'm arguing for the superiority of a log, monitor, follow up policy compared to a filtering policy. You're arguing against a policy that doesn't including monitoring and following up. As I've already said, every provider must come up with a strategy for dealing with spoofed packets, unspoofed floods, compromised customers, and so on. Ideally, though, the actual structural problems would be fixed.
Well that's the real problem. Every attack is potentially spoofed and there are no good tools for dealing with spoofed attacks. Filtering doesn't solve either of those two problems.
If everyone filtered, it would solve the problem. If most people filter, it reduces the problem. If nobody filters it does nothing to address the problem.
The problem of spoofed packets. But this is just one of many, many problems.
Do you want to help, or do you want to do as little as possible?
Actually, I want to do the most possible. And the first thing to do is to realize that there aren't any really good solutions out there. The worst possible thing to do is to go around claiming that if people just ingressed filtered, the problems would go away. The fact is that there are very real problems for which there aren't good solutions. For example, if you own the IP 1.2.3.4, you should have a way of shutting off packets from 4.3.2.1 to 1.2.3.4 at their source. But you don't. We also need a very good general understanding that you shouldn't send 'a lot' of traffic to a destination unless you can confirm that the real destination wants that traffic. There are still new protocols coming out that break this rule very badly. So if filtering works well for you, great, filter. However, if you want to claim that filtering will solve the Internet's DoS problems, then you are part of the problem.
Again, no. A unicast UDP flood can do just as much damage. So filters do not reduce the damage.
If you're ingress filtering, when I pick up the phone I can call your operations folk and they can verify that your customer is flooding me, and stop the flood *quickly*. Speed means a reduction in damage.
On the other hand, if it were coming from my network, odds are someone here would be calling you. And I'd probably be blocking packets to your machine before you even noticed you were being flooded.
Sure, someone else could be spoofing your customers addresses, but your operations folk would be able to verify that your customer is not flooding me... so I'm back to square one. *but* even in this case, I'm no worse off than I am today... so putting filters in either means that attacks get shut down more quickly, or they get treated the same as today. It does not make things worse.
You are *much* worse of if the reason you are getting flooded is because: 1) Someone isn't monitoring his network because he thought filtering solved the problem, or 2) You don't have tools to trace/block the packets because they weren't developed because people believed that filtering would solve the problem. For clueless administrators, they'll probably botch their filters, think they've solved the problems, and not monitor their network. For clueful administrators, they'll catch the problem early and won't pose a threat to you, filters or not. What you really want are useful *tools*. Tools to allow you to control the traffic that is heading towards you. Or perhaps source authentication. Or perhaps something else.
Exactly -- the problem is there's no good way to tell a spoofed packet from an unspoofed packet. Some form of source authentication would solve that.
Or you could make sure you're not part of the problem by putting source address filters on your ingress.
Yes, that's one way to make sure you're not part of the problem. There are others. None are perfect. The problem really needs to be solved at its root. DS
On Thu, Mar 29, 2001 at 07:19:54PM -0800, David Schwartz wrote:
On Thu, Mar 29, 2001 at 06:40:03PM -0800, David Schwartz wrote: So you automatically know about every single spoofed packet your customers send?
We know every pair of source and destination IP addresses and the number of packets and number of bytes. We also know (approximately) the start and stop times.
And you have someone watching the logs in realtime 24 hours a day? Ready to start investigation within seconds of your customer flooding someone, or kicking off a smurf attack?
If they're not spoofed, then the operations folk that I would be speaking to could verify that they're not spoofed by looking at their customer's traffic.
Exactly. So you would need cooperation from the source network to establish the source of the attack.
Duh
Monitoring is reactive. Filtering is proactive. Filtering means I won't be flooded by spoofed packets coming from your customers. Filtering means I won't be smurfed by your customers. (Sure, they could still act as smurf amps, but they couldn't originate a smurf attack).
You will instead get flooded by unspoofed packets. And you'll have to start the process of tracking and fixing the problem.
Yes, but with unspoofed packets, 99% of the tracking work is done.
Monitoring means I could still be flooded with spoofed packets, and you might notice and switch it off.
This is a classic example of a straw man. I'm arguing for the superiority of a log, monitor, follow up policy compared to a filtering policy. You're arguing against a policy that doesn't including monitoring and following up.
No, I'm arguing proactive against reactive. Do you install all your servers with open mail relays and wait for them to be abused before closing them down? Do you install your servers with an empty root password and wait for someone to abuse that before you do something?
As I've already said, every provider must come up with a strategy for dealing with spoofed packets, unspoofed floods, compromised customers, and so on. Ideally, though, the actual structural problems would be fixed.
OK, so go rewrite IP, or at least a means of unspoofable packet tracing.
If everyone filtered, it would solve the problem. If most people filter, it reduces the problem. If nobody filters it does nothing to address the problem.
The problem of spoofed packets. But this is just one of many, many problems.
and the one we're talking about.
Do you want to help, or do you want to do as little as possible?
Actually, I want to do the most possible. And the first thing to do is to realize that there aren't any really good solutions out there. The worst possible thing to do is to go around claiming that if people just ingressed filtered, the problems would go away.
The fact is that there are very real problems for which there aren't good solutions. For example, if you own the IP 1.2.3.4, you should have a way of shutting off packets from 4.3.2.1 to 1.2.3.4 at their source. But you don't.
We also need a very good general understanding that you shouldn't send 'a lot' of traffic to a destination unless you can confirm that the real destination wants that traffic. There are still new protocols coming out that break this rule very badly.
So if filtering works well for you, great, filter. However, if you want to claim that filtering will solve the Internet's DoS problems, then you are part of the problem.
No, it will make a big difference though. Same as closing open mail relays and blocking direct-to-MX connections won't solve the spam problem.
If you're ingress filtering, when I pick up the phone I can call your operations folk and they can verify that your customer is flooding me, and stop the flood *quickly*. Speed means a reduction in damage.
On the other hand, if it were coming from my network, odds are someone here would be calling you. And I'd probably be blocking packets to your machine before you even noticed you were being flooded.
I'm glad you're so confident. Personally, I hope never to find out.
Sure, someone else could be spoofing your customers addresses, but your operations folk would be able to verify that your customer is not flooding me... so I'm back to square one. *but* even in this case, I'm no worse off than I am today... so putting filters in either means that attacks get shut down more quickly, or they get treated the same as today. It does not make things worse.
You are *much* worse of if the reason you are getting flooded is because:
1) Someone isn't monitoring his network because he thought filtering solved the problem, or
No, because if its filtered, I'm gonna be calling the right NOC first time.
2) You don't have tools to trace/block the packets because they weren't developed because people believed that filtering would solve the problem.
That doesn't stop them being developed though. -- John Payne http://www.sackheads.org/jpayne/ john@sackheads.org http://www.sackheads.org/uce/ Fax: +44 870 0547954 To send me mail, use the address in the From: header
On Thu, Mar 29, 2001 at 07:19:54PM -0800, David Schwartz wrote:
On Thu, Mar 29, 2001 at 06:40:03PM -0800, David Schwartz wrote: So you automatically know about every single spoofed packet your customers send?
We know every pair of source and destination IP addresses and the number of packets and number of bytes. We also know (approximately) the start and stop times.
And you have someone watching the logs in realtime 24 hours a day? Ready to start investigation within seconds of your customer flooding someone, or kicking off a smurf attack?
Log analysis is automatic. If a traffic flow is sufficiently suspicious, it will summon someone to investigate it. The definition of 'sufficiently suspicious' is quite complex. I don't want to go into too much detail about how my network works because I really don't want people to know what they can get away with and what's likely to get them caught. But this really isn't supposed to be about about how I run my network.
No, I'm arguing proactive against reactive. Do you install all your servers with open mail relays and wait for them to be abused before closing them down? Do you install your servers with an empty root password and wait for someone to abuse that before you do something?
I don't give the world as a whole the benefit of the doubt -- I give my customers the benefit of the doubt. I take reasonable steps to minimize the damage they can do, but I can't drop it to zero.
As I've already said, every provider must come up with a strategy for dealing with spoofed packets, unspoofed floods, compromised customers, and so on. Ideally, though, the actual structural problems would be fixed.
OK, so go rewrite IP, or at least a means of unspoofable packet tracing.
Whatever needs to happen to solve this at its root won't happen so long as people continue to insist that all it takes is proper filtering.
The problem of spoofed packets. But this is just one of many, many problems.
and the one we're talking about.
No. Unspoofed floods will hurt you just as badly as spoofed ones. And knowing the source won't help you much if it's 5:30 on a Friday and nobody who can fix the source network is around.
No, it will make a big difference though. Same as closing open mail relays and blocking direct-to-MX connections won't solve the spam problem.
Sure, if every ISP created and adopted a good policy for dealing with abuse, DoS attacks, and the rest, that would go a long way. But just like with mail, closing open relays and blocking direct connections is not the real solution, it's just a bunch of bandaids. Yes, the bandaids are better than nothing, but they're hardly the right solution.
On the other hand, if it were coming from my network, odds are someone here would be calling you. And I'd probably be blocking packets to your machine before you even noticed you were being flooded.
I'm glad you're so confident. Personally, I hope never to find out.
You probably never will. We get attacked far more often than we are the source of attacks. (We are lucky in the fact that we've been able to be highly selective in our customers, so they are probably generally more clueful than most people's.)
1) Someone isn't monitoring his network because he thought filtering solved the problem, or
No, because if its filtered, I'm gonna be calling the right NOC first time.
Again, doesn't help you 5:30 on a Friday if the only person who can log into the router isn't there. If they're an OC3 government site and you're a small site whose T1 is being saturated, I don't think their provider is going to shut them down.
2) You don't have tools to trace/block the packets because they weren't developed because people believed that filtering would solve the problem.
That doesn't stop them being developed though.
It's been my experience that it has. No demand equals no supply. Read this list, and you'll see it over and over, "if everyone would just ingress filter ...". In many cases, the 'edge' at which you're supposed to filter is mythical. DS
On Fri, Mar 30, 2001 at 12:20:22AM -0800, David Schwartz wrote:
Again, doesn't help you 5:30 on a Friday if the only person who can log into the router isn't there. If they're an OC3 government site and you're a small site whose T1 is being saturated, I don't think their provider is going to shut them down.
No, but they can modify their ingress filters to block the flood from transiting their network and hitting me. (And still get paid by that government site for the priviledge ;) -- John Payne http://www.sackheads.org/jpayne/ john@sackheads.org http://www.sackheads.org/uce/ Fax: +44 870 0547954 To send me mail, use the address in the From: header
On Fri, Mar 30, 2001 at 12:20:22AM -0800, David Schwartz wrote:
Again, doesn't help you 5:30 on a Friday if the only person who can log into the router isn't there. If they're an OC3 government site and you're a small site whose T1 is being saturated, I don't think their provider is going to shut them down.
No, but they can modify their ingress filters to block the flood from transiting their network and hitting me. (And still get paid by that government site for the priviledge ;)
The time this actually happened, the ISP wouldn't do anything without talking to the site contact. And, after all, I couldn't prove the packets really came from their customer. (And who was I to tell them what to do? And who was my customer representative? And what was my account number?) In fact, I couldn't even be positive the packets transited their network. The origin site had more than one uplink at more than one point on their network. There was no easy way for me to determine which of their possible paths the traffic was taking and thus knowing who to talk to, other than the origin site. The network admin there had just left for the weekend, and they would page him. Perhaps things would be different now. This was circa 1997. The point is, an unspoofed attack can do as much damage as a spoofed attack and can be almost as frustrating. I will admit, howover, that it is nicer to know almost exactly who to blame (and, for that matter, who to sue if it comes to that.) If the person who launches the attack has enough compromised hosts, you can go crazy tracking them down one by one. One day more recently we got hit with a DDoS attack that wasn't spoofed (or at least was mostly unspoofed). We sent out over 100 emails and followed up by telephone for the highest traffic sources. Fortunately, the traffic level was only about 20Mbps, but it took it about a week to die down as we tracked down the administration of each origin machine in turn. I'd love to see IP redesigned from the ground up with a view towards the lessons learned to date. Of course that's just not going to happen. But I think smaller structural or protocol changes might make a large dent. As an example, imagine if you could construct a filter 'block all traffic with source IP X and destination IP Y' and your router could bubble the filter close to the origin. For security, you would have to receive the filter from the interface you would normally route traffic to Y to, and you would 'push the filter up' any interfaces you received a matching packet on. How good an idea is that? Could it be implemented? What are the security implications of it? Are there better ways? Or what if you could make a rule that said 'track all traffic with source mask X and destination IP Y and report to IP Z'. This request would also bubble up the same way and in a few minutes, you'd have a report of a list of routers going closer and closer to the source. To automatically stop the attack, you could make a filter (that also bubbled up) that blocked all traffic to a particular destination IP but only as they transit the selected routers. You would select one or two routers as high up on that chain as possible (to block as little legitimate traffic as possible). And before you know it, the spoofed attack would be both traced and filtered near its source. That's the kind of effort that's going to have to take place if this problem is ever going to be actually solved. All the ingress filtering in the world won't solve the actual problems. Claiming it will discredits the effort to find real solutions. DS
On Thu, 29 Mar 2001, David Schwartz wrote:
So you automatically know about every single spoofed packet your customers send?
We know every pair of source and destination IP addresses and the number of packets and number of bytes. We also know (approximately) the start and stop times.
So, uh, how about dropping AND logging? Wouldn't that make everyone happy? And as for those "special" customers that comprise a subset of your entire customer base, what's wrong with talking to them when you bring the connection up? Charles
DS
[ On Thursday, March 29, 2001 at 18:40:03 (-0800), David Schwartz wrote: ]
Subject: RE: dsl providers that will route /24
Right, that's why every provider has to come up with some reasonable way to deal with this problem. Filtering is one, but it doesn't solve the whole problem. Monitoring is one, but it doesn't solve the whole problem either.
Filtering illegal source addresses, and monitoring your filters, will eliminate *all* possibility of being the source of a spoofed DoS against someone else. Absolutely, positively, guaranteed. No ifs, ands, or buts. There really is no valid excuse any more for not doing it.
Well that's the real problem. Every attack is potentially spoofed and there are no good tools for dealing with spoofed attacks. Filtering doesn't solve either of those two problems.
Yes, exactly, every attack is potentially using spoofed source addresses, which is why monitoring your filters *and* your netflow stats together, will give you a very good idea of who might be trying to perform an unspoofed DoS, or even if a significant enough number of your customers have been hacked and are being used to perform a DDoS. Filtering absolutely blocks all spoofed attacks, leaving you with only the easy ones to deal with. It also absolutely works 100% of the time, unlike NOC operators who might not always be watching similar passive monitoring, or who might be distracted by some other apparently more important event. Passive monitoring is not a technical control and cannot in any way compete against hard technical controls.
Again, no. A unicast UDP flood can do just as much damage. So filters do not reduce the damage.
No, filters won't block a non-spoofed UDP flood, but they're very likely to point the finger at someone who's trying to perform such an attack *before* they can successfully pull it off! (at least they will until attackers get smart enough not to tip their hat by trying a spoof first)
Exactly -- the problem is there's no good way to tell a spoofed packet from an unspoofed packet. Some form of source authentication would solve that.
Every packet with a source address that's not assigned to the customer who it is arriving from *IS* a spoofed packet, regardless of *why* it has an errant address. They must all be filtered regardless of content or purpose! The sooner your customers realise their configuration errors, the better (and the happier they'll be!). Yes customers should do anti-spoofing filtering on both source and destination addresses too, but that does not in any way excuse any provider from doing likewise on *all* edge connections. -- Greg A. Woods +1 416 218-0098 VE3TCP <gwoods@acm.org> <robohack!woods> Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>
On Thu, Mar 29, 2001 at 10:14:54PM -0500, Greg A. Woods wrote:
Filtering illegal source addresses, and monitoring your filters, will eliminate *all* possibility of being the source of a spoofed DoS against someone else. Absolutely, positively, guaranteed. No ifs, ands, or buts. There really is no valid excuse any more for not doing it.
Other then software limitations, routers and switches which can't handle this kind of load, the inability to always know what packets are spoofed.
Exactly -- the problem is there's no good way to tell a spoofed packet from an unspoofed packet. Some form of source authentication would solve that.
Every packet with a source address that's not assigned to the customer who it is arriving from *IS* a spoofed packet, regardless of *why* it has an errant address. They must all be filtered regardless of content or purpose! The sooner your customers realise their configuration errors, the better (and the happier they'll be!).
Now that's a very broad statment that's just not true. There are reasons that packets with a source address not assigned to an ISP may come across the link and be valid, look at DirectPC. Past that if the customer has customers who have blocks assigned from other providers, this becomes a huge and almost impossible to manage real-time list. Big filter lists hit router cpu's, and cost human time. And remember this isn't like filtering BGP customers where if the route doesn't get through it's not always a big deal, you are _dropping_ packets that may be valid.
Yes customers should do anti-spoofing filtering on both source and destination addresses too, but that does not in any way excuse any provider from doing likewise on *all* edge connections.
I'm guessing you talk to a lot of router vendors and listen to their half-truths about their filtering abilities. It's one thing to filter one customer, it's another to filter hundreds of customers utilizing hundreds or thousands of blocks on a single device, just the looking at the configuration becomes a nightmare. Also there's a big difference between an edge device pushing a few megs and one pushing many gigs when it comes to any type of packet filtering. -- ------------------------------------------------------------------------------- : Steven Noble / Network Janitor / Be free my soul and leave this world alone : : My views = My views != The views of any of my past or present employers : -------------------------------------------------------------------------------
On Thu, Mar 29, 2001 at 07:46:44PM -0800, Steve Noble wrote:
On Thu, Mar 29, 2001 at 10:14:54PM -0500, Greg A. Woods wrote:
Filtering illegal source addresses, and monitoring your filters, will eliminate *all* possibility of being the source of a spoofed DoS against someone else. Absolutely, positively, guaranteed. No ifs, ands, or buts. There really is no valid excuse any more for not doing it.
Other then software limitations, routers and switches which can't handle this kind of load, the inability to always know what packets are spoofed.
If a global transit free network can ingress filter all of their customers, without CPU or other logistic problems, I'd be surprised if the majority of ISPs on this list can't do otherwise. OK, if you're UUNET and providing connectivity to a load of ISPs, you might not be able to filter those customers, but you can require that they filter their customers.
Exactly -- the problem is there's no good way to tell a spoofed packet from an unspoofed packet. Some form of source authentication would solve that.
Every packet with a source address that's not assigned to the customer who it is arriving from *IS* a spoofed packet, regardless of *why* it has an errant address. They must all be filtered regardless of content or purpose! The sooner your customers realise their configuration errors, the better (and the happier they'll be!).
Now that's a very broad statment that's just not true. There are reasons that packets with a source address not assigned to an ISP may come across the link and be valid, look at DirectPC.
"Apart from the address block we've assigned you, will you be using addresses in netblocks of other providers? For example, you might have a connection to another ISP, or you might be using DirectPC"
Past that if the customer has customers who have blocks assigned from other providers, this becomes a huge and almost impossible to manage real-time list. Big filter lists hit router cpu's, and cost human time. And remember this isn't like filtering BGP customers where if the route doesn't get through it's not always a big deal, you are _dropping_ packets that may be valid.
And the CPU cost is tiny. Netflow switching reduces it even more. -- John Payne http://www.sackheads.org/jpayne/ john@sackheads.org http://www.sackheads.org/uce/ Fax: +44 870 0547954 To send me mail, use the address in the From: header
On Thu, Mar 29, 2001 at 09:31:31PM -0800, John Payne wrote:
If a global transit free network can ingress filter all of their customers, without CPU or other logistic problems, I'd be surprised if the majority of ISPs on this list can't do otherwise. OK, if you're UUNET and providing connectivity to a load of ISPs, you might not be able to filter those customers, but you can require that they filter their customers.
I'm not saying that some or most ISP's can't do it, I'm saying that not _ALL_ can, so the global statements that there is no reason not to do not apply. Many people have older hardware that works just fine for customer traffic but would not stand up to filters. If I'm pressed to choose between a router/switch that does a better job of providing connectivity to my customers and one that can do line speed ACL's.. You know which one I'll choose. I'm not going to chose my hardware just because it can filter. Even Cisco is releasing hardware that can't do what you are saying, go look at the Engine 4 card, the latest, greatest from Cisco. Should I stop my network deployment just to be able to filter? Should I take the depreciation hit just so I can filter customers in the future and dump these cards, losing my investment? I can't see it, sorry.
Now that's a very broad statment that's just not true. There are reasons that packets with a source address not assigned to an ISP may come across the link and be valid, look at DirectPC.
"Apart from the address block we've assigned you, will you be using addresses in netblocks of other providers? For example, you might have a connection to another ISP, or you might be using DirectPC"
That's fine, but do you do it with everyone? For example I have a T1 and DSL in my house, my DSL provider could care less that I have another connection, but if I feel like it, is there any reason I shouldn't send traffic out the DSL link that is source from IP's only routed over my T1?
Past that if the customer has customers who have blocks assigned from other providers, this becomes a huge and almost impossible to manage real-time list. Big filter lists hit router cpu's, and cost human time. And remember this isn't like filtering BGP customers where if the route doesn't get through it's not always a big deal, you are _dropping_ packets that may be valid.
And the CPU cost is tiny. Netflow switching reduces it even more.
That's wholy dependent on the hardware fire up some filters on a Engine 4 card and tell me this :) -- ------------------------------------------------------------------------------- : Steven Noble / Network Janitor / Be free my soul and leave this world alone : : My views = My views != The views of any of my past or present employers : -------------------------------------------------------------------------------
[ On Thursday, March 29, 2001 at 19:46:44 (-0800), Steve Noble wrote: ]
Subject: Re: dsl providers that will route /24
Other then software limitations, routers and switches which can't handle this kind of load, the inability to always know what packets are spoofed.
Well, some can, actually. There are even machines designed almost specifically for this kind of thing, though I'm not sure if you could deploy them cost effectively. The Juniper folks laughed at me for being so pessimistic to even question their ability to do such filtering at wire speed (though I guess I'll see in real life real soon :-). Of course connecting a Juniper at the edge is a rare thing, and even the one I'll get to play with isn't going to stay there long....
Now that's a very broad statment that's just not true. There are reasons that packets with a source address not assigned to an ISP may come across the link and be valid, look at DirectPC.
From what I know of DirectPC the end user must dial into a provider that's made arrangements for their access to DirectPC ergo the user's packets are not spoofed in that case.
Any other bad/broken/incorrect examples to show us?
Past that if the customer has customers who have blocks assigned from other providers, this becomes a huge and almost impossible to manage real-time list.
If the customer is another ISP who has blocks assigned from one of their other upstream providers and they're sending the wrong packets down the wrong pipes then they've got their routing all messed up and they need to fix it anyway. The sooner you block their packets and make them see the error in their ways the sooner their performance and reliability will return to the levels it should be at and the happier they'll be.
Big filter lists hit router cpu's, and cost human time.
Big filter lists? How many connections do you have on any given edge router? How many network blocks are assigned to those customers? I'll bet when you do the math it's not that big of a list (relatively speaking). How many packets per second do your edge routers handle anyway? As for CPU time, well demand better from your hardware vendors. Given what's available now (and been available for some time now), there's no excuse for a router doing such basic filtering in it's main CPU any more. Like I said too you might even be able to justify separate wire-speed filter boxes to sit just in front of edge routers too (I don't know -- I've no idea how the costs work out vs. the costs of managing spoof-based DoS attacks).
And remember this isn't like filtering BGP customers where if the route doesn't get through it's not always a big deal, you are _dropping_ packets that may be valid.
No packet with an invalid source address is valid. Period. That's the definition!
I'm guessing you talk to a lot of router vendors and listen to their half-truths about their filtering abilities. It's one thing to filter one customer, it's another to filter hundreds of customers utilizing hundreds or thousands of blocks on a single device,
Given the numbers you spout I think you're talking core here. I'm talking edge. Any edge provider that's got a customer with hundreds or thousands of netblocks has that customer connected at the wrong place.
just the looking at the configuration becomes a nightmare.
What, don't you have a provisioning system that would automate this kind of thing? If not and if it's a nightmare then you're well beyond being in dire need of having a real provisioning system!
Also there's a big difference between an edge device pushing a few megs and one pushing many gigs when it comes to any type of packet filtering.
Ah ha! You are talking core. I never suggested trying to do any filtering in the core. The filtering must be done at the edge! Why do "you people" keep jumping to the wrong assumptions?!?!?!?!? -- Greg A. Woods +1 416 218-0098 VE3TCP <gwoods@acm.org> <robohack!woods> Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>
Every packet with a source address that's not assigned to the customer who it is arriving from *IS* a spoofed packet, regardless of *why* it has an errant address. They must all be filtered regardless of content or purpose! The sooner your customers realise their configuration errors, the better (and the happier they'll be!).
Greg A. Woods
That definition, if you really mean it, would make nearly every packet on the Internet spoofed. Sooner or later, pretty much every packet winds up coming into a router with a source not assigned to the customer on the other end of that link. I prefer a much more useful definition of "spoofed". A packet is said to be spoofed if it is introduced onto the Internet and originated on a machine whose administration has not been assigned that IP address for use on the Internet. I can cite you several sources that support my definition. But I don't think you really believed what you said anyway. I'd love to hear your explanation of why a unidirectional VPN is a configuration error. DS
[ On Thursday, March 29, 2001 at 19:55:05 (-0800), David Schwartz wrote: ]
Subject: RE: dsl providers that will route /24
That definition, if you really mean it, would make nearly every packet on the Internet spoofed. Sooner or later, pretty much every packet winds up coming into a router with a source not assigned to the customer on the other end of that link.
think edge man, EDGE!
I prefer a much more useful definition of "spoofed". A packet is said to be spoofed if it is introduced onto the Internet and originated on a machine whose administration has not been assigned that IP address for use on the Internet.
And that's different from my definition, how? You say "machine", I say "link". Which part of that picture does the average ISP have control over?
I'd love to hear your explanation of why a unidirectional VPN is a configuration error.
Your VPN is tunnelled and encrypted, no? (BTW, "unidirectional VPN" is an oxymoron -- a net does not go one way) -- Greg A. Woods +1 416 218-0098 VE3TCP <gwoods@acm.org> <robohack!woods> Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>
Subject: RE: dsl providers that will route /24
That definition, if you really mean it, would make nearly every packet on the Internet spoofed. Sooner or later, pretty much every packet winds up coming into a router with a source not assigned to the customer on the other end of that link.
think edge man, EDGE!
This 'edge' is potentially mythical. Most circuts go to both machines and customers. Ultimately, the edge is the source machine.
I prefer a much more useful definition of "spoofed". A packet is said to be spoofed if it is introduced onto the Internet and originated on a machine whose administration has not been assigned that IP address for use on the Internet.
And that's different from my definition, how? You say "machine", I say "link". Which part of that picture does the average ISP have control over?
Well that's the problem. Further up the line, you can't tell where a packet 'really' originated.
I'd love to hear your explanation of why a unidirectional VPN is a configuration error.
Your VPN is tunnelled and encrypted, no? (BTW, "unidirectional VPN" is an oxymoron -- a net does not go one way)
'Unidirectional VPN' is not an oxymoron. A VPN emulates a private pipe by using a public network. A unidirectional VPN emulates a unidirectional private pipe using a public network. Sometimes, that's all you need. For example, suppose you have two offices that each have a /24 from different ISPs. You have no private link between them. For some reason, you need to have a machine at one location with an IP address from the 'wrong' /24. What you'd like to have is a private network between them. Since you don't have one, you use a virtual private network. Obviously, inbound packets to this IP will arrive at the 'wrong' place, so you need to tunnel them to the right place. However, outbound packets have both source and destination addresses that are valid on the public Internet. You could tunnel them, but that would result in increased bandwidth consumption and gain you basically nothing. DS
On Fri, Mar 30, 2001, David Schwartz wrote:
'Unidirectional VPN' is not an oxymoron. A VPN emulates a private pipe by using a public network. A unidirectional VPN emulates a unidirectional private pipe using a public network. Sometimes, that's all you need.
For example, suppose you have two offices that each have a /24 from different ISPs. You have no private link between them. For some reason, you need to have a machine at one location with an IP address from the 'wrong' /24. What you'd like to have is a private network between them. Since you don't have one, you use a virtual private network.
Obviously, inbound packets to this IP will arrive at the 'wrong' place, so you need to tunnel them to the right place. However, outbound packets have both source and destination addresses that are valid on the public Internet. You could tunnel them, but that would result in increased bandwidth consumption and gain you basically nothing.
Having to setup and use your own servers for your customer outbound mail must be hard. I mean, wouldn't it be much easier just to point smtp.yourisp at some other large ISP who already have spent the money? Or news? Heaven forbid if your NNTP server went down, couldn't you quickly point nntp.yourisp at a large / close ISP so your customers still had NNTP access? Wouldn't it be nice for ISPs to do that? AHAHAHAHAHAHAHAHA. Not in todays Internet. Why isn't it the same with IP? Why does IP have to be unfiltered? So you have to run a bi-directional VPN in order to get the traffic *properly encapsulated*. Jesus, if the internet was built by people like you, we'd have a haphazard, chaotic routing core continuously flapping and changing topology.. .. oh wait. We do. See what happens when you don't assume everyone is evil[0] ? Adrian [0] This wasn't a poke at the internet pioneers. I just don't think they saw the internet being overrun by script kiddies, thats all. -- Adrian Chadd "The fact you can download a 100 megabyte file <adrian@creative.net.au> from half way around the world should be viewed as an accident and not a right." -- Adrian Chadd and Bill Fumerola
On Thu, 29 Mar 2001, David Schwartz spewed:
If they were spoofed, they wouldn't have to because we'd already be investigating. And even if they're not spoofed, you can't know they're not spoofed, so there's no way to know you got the right person.
WTF?: Yes we CAN tell if they're spoofed. It's easy. If EVERYONE stops peering with and listening to announcements from dumb@$$ operators who refuse to implement PREVENTATIVE measures. It's just that simple. Modify peering and transit agreements to include a "If you're a dumbass, we shut your silly ass off!" clause. David, don't bank on peering or obtaining transit from ANYONE that I have ANY influence on.
Well that's the real problem. Every attack is potentially spoofed and there are no good tools for dealing with spoofed attacks. Filtering doesn't solve either of those two problems.
They are ONLY potentially spoofed because there are STILL lame operators like yourself out there that refuse to implement PREVENTATIVE MEASURES!
Again, no. A unicast UDP flood can do just as much damage. So filters do not reduce the damage.
How's that? The last time I checked, my "are you a customer" filters worked against both TCP and UDP. It sounds like you're just LAZY. Do you mind if I quote you to the next reporter who calls?
And until we get a really good solution, a really good workaround is not letting spoofed packets into your network from your customers.
Exactly -- the problem is there's no good way to tell a spoofed packet from an unspoofed packet. Some form of source authentication would solve that.
Um, David... Do you actually READ the list or do you just randomly reply? Here's a clue for you. 1) Require that your customers notify you of any source addresses that they'll be using *PRIOR* to allowing them through. Tunneling is MUCH more rare than spoofing. 2) Require that your customers (BGP speakers) register their networks in RADB or whichever database you choose. (Don't worry. From the sounds of it, NONE of us want your customers...the spoofing b@$tard$...and as such, we're not really interested in who they are beyond filtering them.)
DS
..still not going there. --- John Fraizer EnterZone, Inc
I don't know if you're just not reading what I'm writing or if we're speaking different languages or what. We're totally talking past each other. Here's one misunderstanding. I wrote:
... the problem is there's no good way to tell a spoofed packet from an unspoofed packet.
You wrote:
Um, David... Do you actually READ the list or do you just randomly reply? Here's a clue for you.
1) Require that your customers notify you of any source addresses that they'll be using *PRIOR* to allowing them through. Tunneling is MUCH more rare than spoofing.
2) Require that your customers (BGP speakers) register their networks in RADB or whichever database you choose. (Don't worry. From the sounds of it, NONE of us want your customers...the spoofing b@$tard$...and as such, we're not really interested in who they are beyond filtering them.)
Now, I think you considered what I quoted from you above as replying to what I quoted from me above. However, neither of the two things you stated makes it any easier for me to tell if a packet is spoofed or not. The statement of mine that you were attempting to reply to was, "there's no good way to tell a spoofed packet from an unspoofed packet". And I've already made it clear that by "spoofed", I mean a packet that didn't originate on a machine administered by someone authorized to use that IP address on the public Internet. I get a packet with the origin IP address '206.4.2.1'. How do I tell whether the packet is spoofed or whether it actually originated at the machine authorized to use the IP address '206.4.2.1'? Perhaps a dialup machine on my customer's LAN was assigned that IP address from another provider. Regardless of whether I filter or not, the fact remains that a filter can't tell a spoofed packet from an unspoofed packet. Here's another misunderstanding:
Again, no. A unicast UDP flood can do just as much damage. So filters do not reduce the damage.
How's that? The last time I checked, my "are you a customer" filters worked against both TCP and UDP.
Obviously, I was talking about an unspoofed flood. I mention UDP because most unspoofed floods (at least, that I've seen) are UDP. It's the easiest flood to launch if you haven't root compromised a box. This is rapidly degenerating from a discussion of network operations into a series of personal attacks. DS
On Fri, 30 Mar 2001, David Schwartz wrote:
I don't know if you're just not reading what I'm writing or if we're speaking different languages or what. We're totally talking past each other.
I'm not misreading anything. I'm reading what you have written which is (reading between the lines) "I'm too lazy to keeo track of what IP space my customers use or will be originating traffic from..."
Here's one misunderstanding. I wrote:
... the problem is there's no good way to tell a spoofed packet from an unspoofed packet.
You wrote:
Um, David... Do you actually READ the list or do you just randomly reply? Here's a clue for you.
1) Require that your customers notify you of any source addresses that they'll be using *PRIOR* to allowing them through. Tunneling is MUCH more rare than spoofing.
2) Require that your customers (BGP speakers) register their networks in RADB or whichever database you choose. (Don't worry. From the sounds of it, NONE of us want your customers...the spoofing b@$tard$...and as such, we're not really interested in who they are beyond filtering them.)
Now, I think you considered what I quoted from you above as replying to what I quoted from me above. However, neither of the two things you stated makes it any easier for me to tell if a packet is spoofed or not.
WHAT? If something comes from a customer that doesn't match your filters for that customer you DROP IT! IT'S THAT SIMPLE! Inconvenience for ONE customer or ONE customer's customers is MUCH less or a problem than your customer (or customer's customer) sending traffic from unauthorized address space!
The statement of mine that you were attempting to reply to was, "there's no good way to tell a spoofed packet from an unspoofed packet". And I've already made it clear that by "spoofed", I mean a packet that didn't originate on a machine administered by someone authorized to use that IP address on the public Internet.
I get a packet with the origin IP address '206.4.2.1'. How do I tell whether the packet is spoofed or whether it actually originated at the machine authorized to use the IP address '206.4.2.1'? Perhaps a dialup machine on my customer's LAN was assigned that IP address from another provider.
OK. It's apparent from your arguement that you are not a transit
AS. Here's the simple answer for you: (1)You use address space provided by (your company) or: ( 2)You use address space that you have PROOF (whois.arin.net, etc) that you are authorized to use. (3)Any packets sourced from address space not fitting the above two critiria will be DROPPED! It's that simple. Believe it or not, MOST _REAL_ providers use police very close to what I have outlined.
Regardless of whether I filter or not, the fact remains that a filter can't tell a spoofed packet from an unspoofed packet.
Yes you can. You know what the customer is assigned and what they have told you they will be sourcing from. It's EASY to build a filter to match.
Here's another misunderstanding:
Again, no. A unicast UDP flood can do just as much damage. So filters do not reduce the damage.
How's that? The last time I checked, my "are you a customer" filters worked against both TCP and UDP.
Obviously, I was talking about an unspoofed flood. I mention UDP because most unspoofed floods (at least, that I've seen) are UDP. It's the easiest flood to launch if you haven't root compromised a box.
David, the only obvious thing here is that you're either clueless or criminally negligent. (Both may be true.) The above example has no bearing on this discussion. We're talking (at least those of us with clue) about filtering packets sourced from address space that is not assigned to or OK'd to be used by "customer X".
This is rapidly degenerating from a discussion of network operations into a series of personal attacks.
I agree. I've already wasted too much of my time on you. You're obviously too arrogant/clueless to effect the changes necessary to remove your network from the ever growing list of connected networks that make up the Internet. I reply only in hope that others will see the error of your ways and not be pointed out a myself and many others have done to you. 14649 eh? Why doesn't the high number surprise me? 161.211.0.0/16 How did you fall into a /16 being so clueless?
DS
Coming closer but, still not going there... --- John Fraizer EnterZone, Inc
I won't play your name calling game.
WHAT? If something comes from a customer that doesn't match your filters for that customer you DROP IT! IT'S THAT SIMPLE! Inconvenience for ONE customer or ONE customer's customers is MUCH less or a problem than your customer (or customer's customer) sending traffic from unauthorized address space!
What the heck does that have to do with anything we're discussing? We weren't talking about what you do with it, we were talking about whether you could tell if it was spoofed or not.
(1)You use address space provided by (your company) or: (
2)You use address space that you have PROOF (whois.arin.net, etc) that you are authorized to use.
(3)Any packets sourced from address space not fitting the above two critiria will be DROPPED!
Okay, so you have a filter that checks with ARIN? Or does your filter, *surprise* have no way to tell whether a packet is spoofed or not.
It's that simple. Believe it or not, MOST _REAL_ providers use police very close to what I have outlined.
I'm not talking about that. I'm talking about whether it's possible for a filter to tell whether a packet is spoofed or not.
Regardless of whether I filter or not, the fact remains that a filter can't tell a spoofed packet from an unspoofed packet.
Yes you can. You know what the customer is assigned and what they have told you they will be sourcing from. It's EASY to build a filter to match.
Yes, it's easy to build a filter to match. I never said it wasn't. However, the filter will simply determine whether a packet matches the list of IP address you know the customer is authorized to use. It can't actually tell if a packet is spoofed or not. Get it? A spoofed packet is one that's introduced to the Internet and didn't originate from a machine whose administration is authorized to source that IP on the global Internet. No filter can establish where a packet actually originated, so no filter can tell that. Okay? Get it? A filter cannot determine whether or not a packet is spoofed. Believe me, if a filter really could tell whether a packet was spoofed or not, the problem of ending spoofing would be much, much simpler.
Here's another misunderstanding:
Again, no. A unicast UDP flood can do just as much damage. So filters do not reduce the damage.
How's that? The last time I checked, my "are you a customer" filters worked against both TCP and UDP.
Obviously, I was talking about an unspoofed flood. I mention UDP because most unspoofed floods (at least, that I've seen) are UDP. It's the easiest flood to launch if you haven't root compromised a box.
David, the only obvious thing here is that you're either clueless or criminally negligent. (Both may be true.) The above example has no bearing on this discussion. We're talking (at least those of us with clue) about filtering packets sourced from address space that is not assigned to or OK'd to be used by "customer X".
And the point I'm trying to make is that an unspoofed attack can do just as much damage as a spoofed attack. So stopping spoofing may or may not reduce the damage an attack does. It all depends upon how long it takes you to stop the attack at its source.
14649 eh? Why doesn't the high number surprise me?
161.211.0.0/16
How did you fall into a /16 being so clueless?
That's the provider for the particular machine I'm sitting at at 2 in the morning. I have no connection to them (except that I'm a customer). DS
On Fri, 30 Mar 2001, David Schwartz wrote:
I won't play your name calling game.
Name calling? I don't play games. I label as I see fit however. If it fits, which it obviously did, deal with it or FIX YOUR PROBLEM.
WHAT? If something comes from a customer that doesn't match your filters for that customer you DROP IT! IT'S THAT SIMPLE! Inconvenience for ONE customer or ONE customer's customers is MUCH less or a problem than your customer (or customer's customer) sending traffic from unauthorized address space!
What the heck does that have to do with anything we're discussing? We weren't talking about what you do with it, we were talking about whether you could tell if it was spoofed or not.
Um, the last time I checked, we were talking about spoofed packets. It has EVERYTHINH to do with that.
(1)You use address space provided by (your company) or: (
2)You use address space that you have PROOF (whois.arin.net, etc) that you are authorized to use.
(3)Any packets sourced from address space not fitting the above two critiria will be DROPPED!
Okay, so you have a filter that checks with ARIN? Or does your filter, *surprise* have no way to tell whether a packet is spoofed or not.
No. I've got better. Check this out. http://www.isi.edu/ra/RAToolSet/RtConfig.html Wow! It will build filter sets based on registered objects! What a ^&(#*( concept!
It's that simple. Believe it or not, MOST _REAL_ providers use police very close to what I have outlined.
I'm not talking about that. I'm talking about whether it's possible for a filter to tell whether a packet is spoofed or not.
Yes! It is! If you build your filters based on what IP addresses you know should be sourced from Customer-X, you *CAN* know and filter spoofed packets. It all comes down to this. Do you announce it? yes? You should probably accept traffic to/from it. You DON'T announce it? Well.. In that case, you've got one of two things going on: (1) Customer-X is spoofing (2) Customer-X isn't announcing address space to you that they're going to be originating traffic from. In EITHER case, if CUSTOMER-X hasn't made prior arrangement (and you with your upstreams and them with theirs) the traffic is going to be dropped SOMEWHERE anyway because you won't have a route object for it in the irrd.
Regardless of whether I filter or not, the fact remains that a filter can't tell a spoofed packet from an unspoofed packet.
Yes you can. You know what the customer is assigned and what they have told you they will be sourcing from. It's EASY to build a filter to match.
Yes, it's easy to build a filter to match. I never said it wasn't. However, the filter will simply determine whether a packet matches the list of IP address you know the customer is authorized to use. It can't actually tell if a packet is spoofed or not.
OK. Check this concept out. If the customer isn't authorized to use the address space, AND THEY ARE...... IT'S SPOOFED -- BOGUS -- BAD -- BAD -- BAD!!!!!
Get it?
I got it years ago, Son. Do YOU get it or are we all wasting our time trying to educate the terminally ignorant?
A spoofed packet is one that's introduced to the Internet and didn't originate from a machine whose administration is authorized to source that IP on the global Internet. No filter can establish where a packet actually originated, so no filter can tell that. Okay? Get it? A filter cannot determine whether or not a packet is spoofed.
Huh? If they're authorized to originate the packet, they TELL YOU and you VERIFY THAT and you open your filters up for them to DO SO! If NOT, you NIX IT! Check this out. YOUR INGRESS FILTER, if YOU'RE their connection to the global internet, or ONE of their connections to the global internet, can tell if where the packet originated from! It's filtering THEIR CONNECTION! Do we have to take up a collection to buy you some clue or what?
Believe me, if a filter really could tell whether a packet was spoofed or not, the problem of ending spoofing would be much, much simpler.
The problem is SIMPLE. It's just two simple steps. INGRESS filter your customers. Everyone does that, we have a FEW rogues (you fit that mold) to filter/blackhole/disconnect.
Obviously, I was talking about an unspoofed flood. I mention UDP because most unspoofed floods (at least, that I've seen) are UDP. It's the easiest flood to launch if you haven't root compromised a box.
David, the only obvious thing here is that you're either clueless or criminally negligent. (Both may be true.) The above example has no bearing on this discussion. We're talking (at least those of us with clue) about filtering packets sourced from address space that is not assigned to or OK'd to be used by "customer X".
And the point I'm trying to make is that an unspoofed attack can do just as much damage as a spoofed attack. So stopping spoofing may or may not reduce the damage an attack does. It all depends upon how long it takes you to stop the attack at its source.
Stopping spoofing brings ready accountability to the networks that are actually ORIGINATING the attacks and reduces the time involved in tracking them down. When we KNOW it's not spoofed, we can track the REAL SOURCE down REAL QUICK! How can you be so ignorant to this fact and still possess the mental faculties required to type?
14649 eh? Why doesn't the high number surprise me?
161.211.0.0/16
How did you fall into a /16 being so clueless?
That's the provider for the particular machine I'm sitting at at 2 in the morning. I have no connection to them (except that I'm a customer).
Oh. OK. So you don't ever run a network and I've wrongfully malligned someone who most likely has a ^&*# clue? Everyone who is associated with AS14649 and 161.211.0.0/16, I'm sorry. Once again, I've been duped into thinking I was replying to someone who actually RAN A ^%#*& NETWORK just because they managed to subscribe to NANOG and NANOG-POST. We _REALLY_ need to get more stringent subscription requirements for NANOG-POST.
DS
Wouldn't be prudent and you probably wouldn't understand it if I did. --- John Fraizer EnterZone, Inc
I'm going to keep this really simple and go really slow so there's no chance of a misunderstanding. You have a customer A. He has two customers, B and C. Your filter allows A, B, and C's assigned addresses as source addressees on the link to/from customer A. Your customer A, receives a packet from customer B with a source address assigned to customer C. Your filter allows it even though it's spoofed. You know why that is? Because your filter can't tell a spoofed packet from an unspoofed packet. Customer B dials up to another ISP. He gets an IP address. He sends a packet sourced with that IP address to your customer A who forwards it to you. It's not spoofed, but your filter blocks it. Do you know why that is? Because your filter can't tell a spoofed packet from an unspoofed packet. You may be entirely happy with your filter, and it may be doing exactly what you want it to do. I won't dispute that. But the fact remains that your filter cannot tell a spoofed packet from an unspoofed packet. And there's a simple reason for this -- your filter can't tell where a packet actually originated, and that's what you need to know to tell whether it's spoofed or not. Do you understand my point yet? A filter cannot tell a spoofed packet from an unspoofed packet. We've gone back and forth about four times and this simple point still seems to elude you. I wish I liked to play the name calling game as much as you do. DS PS: Am I the only one who was actually a little happy the day some big name sites got hit with DDoS attacks thinking this would finally bring some attention and real solutions to the problem of DoS attacks? Am I the only one disappointed with the fact that things have not gotten significantly better since then?
On Fri, 30 Mar 2001, David Schwartz wrote:
I'm going to keep this really simple and go really slow so there's no chance of a misunderstanding.
You have a customer A. He has two customers, B and C. Your filter allows A, B, and C's assigned addresses as source addressees on the link to/from customer A.
Your customer A, receives a packet from customer B with a source address assigned to customer C. Your filter allows it even though it's spoofed. You know why that is? Because your filter can't tell a spoofed packet from an unspoofed packet.
Um... Check this out. Customer B should be filtering TOO! They know what addresses they're assigned!!!!!!!
Customer B dials up to another ISP. He gets an IP address. He sends a packet sourced with that IP address to your customer A who forwards it to you. It's not spoofed, but your filter blocks it. Do you know why that is? Because your filter can't tell a spoofed packet from an unspoofed packet.
Yes it is spoofed. If we're not announcing a route to it and and it's originating from our network (customer or not) it's SPOOFED! I don't care what you say. No inbound route equals no outbound route. Sure, in the pre-script-kiddie world, this would have been a quasi-legit packet but NOW, it's NOT! Technology exists to allow all LEGIT traffic to be properly tunneled. If they want to VPN, do it two-way. Otherwise, NIX IT!
You may be entirely happy with your filter, and it may be doing exactly what you want it to do. I won't dispute that. But the fact remains that your filter cannot tell a spoofed packet from an unspoofed packet. And there's a simple reason for this -- your filter can't tell where a packet actually originated, and that's what you need to know to tell whether it's spoofed or not.
My filter knows if we're ANNOUNCING a route to the originating address. If we're NOT, we NIX the packet. No way back -- No way from. It's that simple.
Do you understand my point yet? A filter cannot tell a spoofed packet from an unspoofed packet. We've gone back and forth about four times and this simple point still seems to elude you. I wish I liked to play the name calling game as much as you do.
I've understood all along. And you're mistaken. Filters know what you tell them. If you're announcing a route, you should accept traffic to/from it. If not, you nix it. It's that simple. As for name calling, I've exercised my weekly allotment of restraing in the past 2 hours. Get a clue, PLEASE.
DS
PS: Am I the only one who was actually a little happy the day some big name sites got hit with DDoS attacks thinking this would finally bring some attention and real solutions to the problem of DoS attacks? Am I the only one disappointed with the fact that things have not gotten significantly better since then?
Yes. You are amount an elite STUPID few who found gratification in an event that cause the REAL networks on the wire tons of grief. Your statement above has caused me to disgard any remaining respect I might have had for you because it is DUMB %^Ks like YOU that allowed it to happen in the first place. Had the attacks come from source addresses that were known to only originate from network-X, it would be easy to NIX. Because of the fact that there are so manu clueless individuals like yourself our there (and providers who are more than happy to sell you connectivity), the attack was MUCH MORE EFFECTIVE. As for things getting better, they won't until the clueless take a hint from: (1) History (2) Those of us who already have a clue. --- John Fraizer EnterZone, Inc
-- Jason Slagle - CCNP - CCDA Network Administrator - Toledo Internet Access - Toledo Ohio - raistlin@tacorp.net - jslagle@toledolink.com - WHOIS JS10172 /"\ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . \ / ASCII Ribbon Campaign . If dreams are like movies then memories X - NO HTML/RTF in e-mail . are films about ghosts.. / \ - NO Word docs in e-mail . - Adam Duritz - Counting Crows On Thu, 29 Mar 2001, David Schwartz wrote:
They could do almost exactly the same amount of damage with an unspoofed UDP flood and it would still take a human action to stop it. The attack can still hop from victim to victim until the problem is stopped at its source. The problem still won't get stopped at its source until someone with the ability to stop it is summoned and alterted to the problem.
Odds are, an attacker will used spoofed packets if he can. potentially spoofed packets will trigger an investigation on my network. An unspoofed UDP flood probably won't (especially if it hops from victim to victim).
So if the attacker uses spoofed packets, he may get cut off at the source (and the problem actually solved) sooner. On the other hand, unspoofed packets will probably trigger a call to the administration of the source network faster. Of course, you don't know that attack is unspoofed, so you really can't be sure what the source is.
I can argue the converse of this. Unless the attacker is spoofing a static source, I can usually spot a potentially unspoofed attack. Even if he IS using a static spoofed source, it only costs me a little bit to call and see if the packets are indeed coming from the machine in question. If I'm being attacked hard, chances are, I will notice it before you examine your logs, unless like I said you have someone monitoring then 24 hours a day. I will then try to wake up a live body on your end to investigate. If the packets are spoofed, I have to wait for you to examine your logs to potentially stop it, or attempt to get an upstream to do a traceback, which is a long drawn out process. Personally, I prefer to leave the ability to determine the likely source of a non random attack in my hands, not waiting for you to view your logs. And nothing says I CAN'T log if I deny spoofed packets, therefor catching them when they try spoofed packets before realizing they won't work. Jason
David Schwartz <davids@webmaster.com> spewed:
The important thing to realize is that neither of these situations is ideal. That is, filters don't solve the problem. We need to acknowledge that
Filters don't solve the problem. That I'll agree to. Filters prevent it from being MY problem when your dumbass customer has a dumbass customer who has a shell server that is r00ted and becomes a DoS source. How can you be so blind?
we have a problem and don't have a solution to it. Only then will the problem be analyzed, solutions proposed, and implemented.
The problem is people like you who refuse to filter. If you filter, you can only be the problem if the SOURCE ip addresses belong to YOU! If your customers want to do mobile IP, no problem. Have them register with your NOC what IP addresses they'll be sourcing. Then, YOU register as an origin for those IP addresses in (pick your database that is mirrored in RADB). *THEN* if your customer causes a problem, we know who to contact by looking at not only ARIN but also RADB.
I don't know, I'm not smart enough to solve the problem by myself. All I
...And it appears you're too stubborn to fix what portion of the problem you have control over.
can do is keep yelling as loudly as I can that there is a problem and that we do need a really good solution.
And all I can say is that if I find out that you're network is the source of a DoS towards mine, I'll do whatever necessarry, no matter the consequences to your network, to minimize the damage to my network. This includes ANYTHING I can think of to make the problem (read YOUR NETWORK) go away.
DS
I won't even go there. --- John Fraizer EnterZone, Inc
Filters don't solve the problem. That I'll agree to. Filters prevent it from being MY problem when your dumbass customer has a dumbass customer who has a shell server that is r00ted and becomes a DoS source. How can you be so blind?
Please explain to me how a flood with real origin IPs hurts your pipes any less than a flood with spoofed origin IPs. Okay, you know who to call. How much does that help you if you have a T1 and the flood source is an OC48 government site, it's 6:30 PM on a Friday, the only person on-site who can access the router is just left and their ISP is not going to shutdown an OC48 government contract just to protect your T1 and your ISP doesn't want to mess with their router configuration until their Sunday morning maintenance window. I've been there. Knowing where it was coming from didn't do me a damn bit of good. So, now, how is it any less your problem if my dumbass customer has a dumbass customer who has a shell server that is r00ted and becomes a DoS source? With or without filters, the traffic has to be monitored. Suspicious flows have to be investigated. Staff has to be there to deal with the problem and the staff has to be competent. There might be real solutions to these problems. Automated hop-by-hop reverse tracing -- true source authentication -- reverse filter propogation. But none of these things will be developed or deployed if the party line is that ingress filtering is the solution to the DoS problem. How can you be so blind? DS
David. We're seeing a flood of RFC1918 sourced traffic bounce off our ingress filters. This is causing unwarranted load on our edge routers. I feel that we should be compensated by whomever is originating or allowing the origination of said packets. As a result of your strance on the matter, we're going to be naming you as a defendant in a civil lawsuit. Please be sure to have available packet logs from _all_ of your customers at _every_ _ingress_ and _egress_ point on your network for the past 24 and next 48 hours. See you in court. --- John Fraizer EnterZone, Inc
David.
We're seeing a flood of RFC1918 sourced traffic bounce off our ingress filters.
This is causing unwarranted load on our edge routers. I feel that we should be compensated by whomever is originating or allowing the origination of said packets.
As a result of your strance on the matter, we're going to be naming you as a defendant in a civil lawsuit. Please be sure to have available packet logs from _all_ of your customers at _every_ _ingress_ and _egress_ point on your network for the past 24 and next 48 hours.
See you in court.
We don't log egress points. Obviously, you're so deep into personal attack mode that you're not capable of rational argument. By the way, we haven't had a customer hit us with a single packet with an RFC1918 source address in about 4 months. The last time, one of our customers had a misconfigured firewall and was quite glad we alterted them to the problem. We do get hit with a lot of them, and we pass them on to our customers unless our customers request that we filter them. I don't think there's a clear consensus that this is wrong. If someone numbers a gateway inside RFC1918 space (which *is* wrong, IMO) blocking the packets could cause problems. Of course, one can argue that it's the bad numbering of the gateway that cause the problem, and I won't disagree with that argument. There's also something to be said for interoperating with broken configurations. My current position is that it's definitely wrong to introduce packets with RFC1918 source addresses to the global Internet, but that it's not necessarily a good idea to filter them if you happen to see them. Others may disagree, but I *definitely* don't want to start up this debate again. DS
participants (19)
-
Adrian Chadd
-
Alex Rubenstein
-
Bill Woodcock
-
Charles Sprickman
-
David Schwartz
-
Eric A. Hall
-
Jason Lixfeld
-
Jason Slagle
-
John Fraizer
-
John Payne
-
michael
-
Roland Dobbins
-
Scott Francis
-
Steve Gibbard
-
Steve Noble
-
Steve Sobol
-
Tim Winders
-
Valdis.Kletnieks@vt.edu
-
woods@weird.com