sean@donelan.com (Sean Donelan) writes:

How many exploits does uRPF block?

that's hard to measure since we end up not receiving those. but one can assume that spoofed-source attacks aren't tried, either because (1) it's easier to just use a high number of windows-xp drones, or because of (2) uRPF deployment.

Does uRPF solve more problems than it causes, and saves more than it costs?

until you know what percentage of the attacks you don't see is due to (1) vs (2) above, you can't really pose that question meaningfully. anytime there's a way to protect against a whole class of attack weapons, we have to deploy it. this is war, information warfare. let's deprive the enemy of options until we can force them to meet us on our own chosen terms. -- Paul Vixie

Sean Donelan wrote:

Would you rather ISPs spend money to 1. Deploying S-BGP? 2. Deploying uRPF? 3. Respond to incident reports?

Why are we limited to that set?

SD> Date: Sat, 6 Mar 2004 22:04:58 -0500 (EST) SD> From: Sean Donelan SD> Would you rather ISPs spend money to SD> 1. Deploying S-BGP? SD> 2. Deploying uRPF? SD> 3. Respond to incident reports? Let's look at the big picture instead of a taking a shallow mutex approach. If SAV were universal (ha ha ha!), one could discount spoofed traffic when analyzing flows. But, hey, why bother playing nice and helping other networks, eh? Am I the only one who's had IWFs -- even legitimate entities -- complain about packets "from your network" that weren't? It certainly would have been nice if $other_networks had used SAV. SAV doesn't take long to implement. Considering the time spent discounting spoofing when responding to incidents, I think there would be a _net_ savings (no pun intended) in time spent responding to incidents. Alas, that requires cooperation and doesn't provide instantaneous gratification. If it doesn't make/save a quick buck, why bother? Detection of sarcasm is left as an exercise to the reader. Eddy -- EverQuick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _________________________________________________________________ DO NOT send mail to the following addresses : blacklist@brics.com -or- alfra@intc.net -or- curbjmp@intc.net Sending mail to spambait addresses is a great way to get blocked.

sean@donelan.com (Sean Donelan) writes:

SAV doesn't tell you where the packets came from. At best SAV tells you where the packets didn't come from.

...which is incredibly more valuable than not knowing anything at all.

You would be wrong. There are networks that have deployed SAV/uRPF.

They saw no _net_ savings.

In the real world, it costs more to deploy and maintain SAV/uRPF.

in the therefore-unreal world i live in, the ability to tell a GWF ("goober with firewall") that the incident report they sent our noc could not possibly have come from here, is a net cost savings over having to prove it every time.

Have you noticed this thread is full of people who don't run large networks saying other people who do run networks should deploy SAV/uRPF.

distinguishingly, i do help run a network, and i'm not limiting my accusation ("you guys are slackers") to uPRF-free networks of any particular size ("big"). -- Paul Vixie

SD> Date: Sun, 7 Mar 2004 17:47:09 -0500 (EST) SD> From: Sean Donelan SD> In practice, GWF's ... send reports about packets which have SD> our IP addresses, but didn't originate here. The last thing Probably because someone else failed to implement SAV. If $origin_net prevented spoofing your IP space, you'd not have had the problem. If other networks prevented spoofed sources, nobody else could source a packet from your address space. In this case, a packet apparently sourced from you network definitely would have come from your network. Therefore you'd no longer need to check to see if a packet was spoofed. Notice how AS_PATHs and netblock announcements tend to get filter. Why? SD> you want to admit is you do SAV because GWF think SAV means SD> every packet with that source address must have originated SD> here. Uh, no... a spoofed packet from someone else's network means you had no control over it. That's pretty obvious. Eddy -- EverQuick Internet - http://www.everquick.net/ A division of Brotsman & Dreger, Inc. - http://www.brotsman.com/ Bandwidth, consulting, e-commerce, hosting, and network building Phone: +1 785 865 5885 Lawrence and [inter]national Phone: +1 316 794 8922 Wichita _________________________________________________________________ DO NOT send mail to the following addresses : blacklist@brics.com -or- alfra@intc.net -or- curbjmp@intc.net Sending mail to spambait addresses is a great way to get blocked.

On Mon, 8 Mar 2004, E.B. Dreger wrote:

SD> Date: Sun, 7 Mar 2004 16:17:50 -0500 (EST) SD> From: Sean Donelan

SD> SAV doesn't tell you where the packets came from. At best SD> SAV tells you where the packets didn't come from.

If SAV were universal, source addresses could not be spoofed. If source addresses could not be spoofed...

in a perfect world yes, for today we still have LOTS of folks that firewall in one direction only. A great example of this is the great firewall of China :( How, if they are filtering every packet that leaves their country, can I still get attacked from them? :( Until this is a default behaviour and you can't screw it up (ala directed-broadcast) this will be something we all have to deal with.

SD> Have you noticed this thread is full of people who don't run SD> large networks saying other people who do run networks should SD> deploy SAV/uRPF.

1. SAV is most effective at the edge, which often implies the smaller networks should be doing it

excellent, the original point of the conversation has been satisfied... uRPF for the core is not a good plan, edge networks are a great place for this. Doing this on single homed customers is a great step in the right direction. However, as you say, the best place for this is on the edge of the network. So this implies that each edge LAN router will/should have uRPF or atleast an acl permitting only local LAN traffic to source from it, right? I have a question, I wonder if uRPF works on low end platforms without running CEF? Do all low-end platforms gracefully support CEF along with the other things enterprises typically do on routers? (just a question really...)

2. I've not seen large networks talking about their awful experiences with SAV.

it melts routers, good enough for you? Specifically it melts linecards :( my experience is only on Cisco equipment though, so the linecard/ios/rev games must be played. If you upgrade, or initially install, E3 cards a large portion of this care is not necessary though. This is a problem that could be migrated out as new equipment/capabilities hit everyone's networks. I suspect that market pressure will push things in this direction anyway over time.

Christopher L. Morrow wrote:

...

2. I've not seen large networks talking about their awful experiences with SAV.

it melts routers, good enough for you? Specifically it melts linecards :( my experience is only on Cisco equipment though, so the linecard/ios/rev games must be played. If you upgrade, or initially install, E3 cards a large portion of this care is not necessary though. This is a problem that could be migrated out as new equipment/capabilities hit everyone's networks. I suspect that market pressure will push things in this direction anyway over time.

That was exactly what I was doing by saying I will only get service from ISPs that run loose-uRPF in cores. (or all edges, including peering links.) I will not take service from ISP X, who is cheaper than ISP Y, if ISP X cannot assure me that I will not get bogon sourced traffic on my link. What you are saying above is not a technical argument against uRPF (as you grant that there is equipment that will do uRPF at core speeds.) - its a business one. So I am giving you a business incentive to take to your managers. "Customers want this service which we cannot deliver w/o upgrades. Customers will not give us money unless we spend this money, and they will go to our competitors who have infrastructure that can do it." If your vendors cannot deliver equipment that meets your requirements to meet your customers' needs, you need to say the same thing to your vendors, and vote with dollars for those that can.

Sean Donelan wrote:

On Mon, 8 Mar 2004, Steve Francis wrote:

...

That was exactly what I was doing by saying I will only get service from ISPs that run loose-uRPF in cores. (or all edges, including peering links.)

I will not take service from ISP X, who is cheaper than ISP Y, if ISP X cannot assure me that I will not get bogon sourced traffic on my link.

Why do you care how a provider does X?

Your requirement doesn't seem to be run loose-uRPF in cores, although that may be one way a provider chooses to solve the problem. You requirement is "not get bogon sourced traffic on your link."

I know its tempting to tell other people how to run their networks. But specifying the solution sometimes cuts out alternative solutions which work just as well or maybe better.

Correct. I was overstating my requirement. What I really want is as you described: I want assurance that any packet I receive on my proposed circuit is NOT sourced from a patently false IP address. (i.e. no packets sourced from reserved IP addresses, RFC 1918 IP addresses; addresses from blocks not yet allocated by routing registries, or addresses from blocks that are not currently being announced via BGP to the Internet.) I would also prefer that such packets be dropped as far as possible from the POP I am connected to, to minimise the chance of such packets overloading the carriers circuits into that POP. I know of no way to do this other than loose-uRPF in the core, or at least loose-uRPF on all edges, including peering connections. Can any of the operators that are arguing against loose-uRPF in the core state if they run loose uRPF on all peering connections, regardless of speed, as well as on all their edges? Or propose another way to achieve the same thing?

Sean Donelan wrote:

On Mon, 8 Mar 2004, E.B. Dreger wrote:

...

SD> They saw no _net_ savings. SD> SD> In the real world, it costs more to deploy and maintain SD> SAV/uRPF.

The benefit is to other networks. When other networks make your life easier, you benefit.

This confirms my statement.

How much do you save by putting handrails on your stairways? Restrooms in you lobby? Precipitators on your smoke stacks?

goemon@anime.net (Dan Hollis) writes:

... This isnt the point. The point is, why should others suffer the burden of your clients spewing bogon/spoofed/nonsense garbage at them?

when i found out that two e-mail based service companies who had been acquired by yahoo had stopped doing verification of e-mail addresses as a result of that acquisition, i asked why. it turns out that yahoo's "properties" are designed for margin rather than for sustainability, and that the reason we all have to bear the burden of the resulting torrent of acquaintant-spam and unverified swill is simply that yahoo knows that most isp's are unwilling to boycott their e-mail. (though i do it here.) earlier that same week, i complained to amazon about lack of verification and heard this same story except this time it was from their abuse-bot. the only amazing thing about it is how bald-faced both companies are about shifting their costs onto the rest of the community. so i think the non-rhetorical answer to your perhaps-rhetorical question above is simply "because they must" (or "because we can", as you please.)

The effect is cumulative. If everyone takes this lazy apathetic approach to network administration, it hurts everyone.

when we put these CEO's on a quarterly schedule for answering the question, "what have you done for us lately?" this result was pretty much preordained.

Its the difference between being a good neighbor and being the fat beerbelly neighbor with dogs barking all night and rusting camaro with no tires up on cinderblocks on his beercan littered lawn.

just for reference, the short form of the latter is "chickenboner" (because of the pile of same found under their airstream's kitchen window.)

Just because everyone else doesnt maintain a good network doesnt mean you shouldnt.

yea, verily. -- Paul Vixie

On Sun, Mar 07, 2004 at 09:24:44PM -0500, Sean Donelan wrote:

On Mon, 8 Mar 2004, E.B. Dreger wrote:

...

SD> They saw no _net_ savings. SD> SD> In the real world, it costs more to deploy and maintain SD> SAV/uRPF. [snip]

In the real word, there are different networks with different tools and different gear. In some networks, it is a flip of the switch, you are done, and can move on. The direct benefit to my network is eliminating a category of crap from it. I save having to deal with that category. Yes there is other crap, but reducing the workload... reduces the workload. [snip]

has correctly deployed SAV. Even if everyone deploys SAV/uRPF you never know when someone may misconfigure something, so you still have to keep doing everything you were doing.

You mean internally to the network? Config management must exist for a huge number of reasons. Drop the right knob in your standards and move on. I don't follow 'having to keep doing everything' when I have one less things to do.

In the mean time, you get to pay for the extra costs for deploying SAV/uRPF in addition to doing everything you were already doing.

I'm sorry your network has such huge costs for trivial changes that follow simple logic. Actually, I've lost track of how many tiers of soapboxes are involved here, so I'm not sure what level of hypothetical-vs-real this [sub]thread is tackling. I'll encourage my competators to let more crap on their networks. I'll take out the trash at the points where I can. -- RSUC / GweepNet / Spunk / FnB / Usenix / SAGE

On Sun, Mar 07, 2004 at 09:24:44PM -0500, Sean Donelan wrote:

...

If you want others to help you, help them.

I've already done my part. I'm still waiting for others to help me. Should I be expecting a check in the mail?

No. The work you've done is expected of you, as a good Internetwork neighbour. If you're not a good neighbour, next time you need my help, or the help of anyone else I know, please expect the finger. Yes, I suppose this does sound somewhat like a cross between an old-school network, and rule by bullying. But we don't have a better way (yet). -- Avleen Vig Systems Administrator Personal: www.silverwraith.com EFnet: irc.mindspring.com (Earthlink user access only)

sean@donelan.com (Sean Donelan) writes:

...

If you're not a good neighbour, next time you need my help, or the help of anyone else I know, please expect the finger.

But I keep trying to do good work; and you keep giving me the finger. Why should I keep trying to do good work? Remember it works both ways.

i guess i don't understand your comments. you have argued for relaxation of vigilance in the practice of uRPF, on the basis of high cost and assymetric benefit. you have attempted to justify your employer's lack of effort in this area on the basis that the benefit is not only assymetric but also negligible. this sure sounds like a copout. did you actually do something good but you aren't allowed to say so in public? -- Paul Vixie

Sean Donelan wrote:

On Sun, 7 Mar 2004, E.B. Dreger wrote:

...

SAV doesn't take long to implement. Considering the time spent discounting spoofing when responding to incidents, I think there would be a _net_ savings (no pun intended) in time spent responding to incidents.

You would be wrong. There are networks that have deployed SAV/uRPF.

They saw no _net_ savings.

In the real world, it costs more to deploy and maintain SAV/uRPF.

Have you noticed this thread is full of people who don't run large networks saying other people who do run networks should deploy SAV/uRPF.

But there hasn't been anyone who does run large networks saying they deployed SAV/uRPF and it saved them money, made their network run better or improved the world?

Where do you draw the line between large and not large? Does a university with a /16 count as large? We do both SAV and a version of uRPF. It makes our network run better, saves us money (reduces the amount of time we spend on support and makes troubled/distressed/evil/mean/nasty boxes easier to track down) and reduces backbone congestion making the network run better. Another benefit is it improves the world (betcha' were wondering if I'd squeeze all that in). We're now blocking all SMTP traffic leaving the campus from non-blessed sources (read mail servers). The first day doing this we had comments about less junk mail traffic. We block traffic we consider harmful that shouldn't leave the campus. We're trying to do our part. Any suggestions how we can do better? Ken