Hello, To help quench the effects of smurf attacks on our network, we CEF-CAR all ICMP on our egress points to about 200% of normal ICMP flows. However, when a upstream becomes full of ICMP (even though we dump most of it), it still affects our external connectivity. My question is, why don't larger upstream providers use CEF-CAR (assuming that most use this) do the same to limit the effect of smurf attacks on thier (and subsequently, thier customers') networks? The floor is open for flames. -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Atheism is a non-prophet organization. I route, therefore I am. Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member Father of the Network and Head Bottle-Washer Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834 Don't choose a spineless ISP; we have more backbone! http://www.nac.net -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Alex, I've asked our transit providers to do this, and one out of three is CARing ICMP. One said, sorry, can't do it on our router for "technical reasons" (think very large national provider). Another said, since we have lots and lots of customers (implying that there is no "normal ICMP flows" level), and we're carrying it over our network to you, your router might as well do the work of discarding the packets (think very savvy colocation provider). To attack the problem in a different way, why aren't more providers (esp. the colocation providers) using RPF on the edges? There seems to be a general feeling that RPF is broken (bugids please? operational experiences with routing/network diagrams) -- yes, it can't be used everywhere (ie. not on core/backbone routers), but then again, it shouldn't. Yet, it has very good use at the edge. Adi In message <Pine.BSF.4.05.9905010211070.5195-100000@iago.nac.net>, alex@nac.net writes:
Hello,
To help quench the effects of smurf attacks on our network, we CEF-CAR all ICMP on our egress points to about 200% of normal ICMP flows.
However, when a upstream becomes full of ICMP (even though we dump most of it), it still affects our external connectivity.
My question is, why don't larger upstream providers use CEF-CAR (assuming that most use this) do the same to limit the effect of smurf attacks on thier (and subsequently, thier customers') networks?
The floor is open for flames.
-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Atheism is a non-prophet organization. I route, therefore I am. Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member Father of the Network and Head Bottle-Washer Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834 Don't choose a spineless ISP; we have more backbone! http://www.nac.net -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
On Fri, 30 Apr 1999, R.P. Aditya wrote:
One said, sorry, can't do it on our router for "technical reasons" (think very large national provider).
Yes, I was told this by one upstream as well.
Another said, since we have lots and lots of customers (implying that there is no "normal ICMP flows" level), and we're carrying it over our network to you, your router might as well do the work of discarding the packets (think very savvy colocation provider).
The point is missed here; don't they realise there is benefit to them? -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Atheism is a non-prophet organization. I route, therefore I am. Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member Father of the Network and Head Bottle-Washer Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834 Don't choose a spineless ISP; we have more backbone! http://www.nac.net -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
After dealing with UUNet security regarding several smurf incidents I asked them this same question. Their response (and I'm sure it would be the same response of others) was that a lot of the routers on their network couldn't handle the load of using CEF-CAR to limit smurf attacks. I'm not sure how true that statement was since I'm not familiar with any part of UUNet's backbone equipment other than what I used to get my DS3 from at Insync and now with my MAE Houston connection, but from what I've heard the backbones of a lot of NSP's aren't all made up of Cisco 12000's or even 7500's, and I'd guess a fair amount of the existing routers out there are borderline overloaded since it's next to impossible to get most backbone providers to filter traffic when you're under attack. UUNet certainly wouldn't for us because of "router CPU overhead" last time I was under attack. Just my $.02... -- Joseph W. Shaw - jshaw@insync.net Freelance Computer Security Consultant and Perl Programmer Free UNIX advocate - "I hack, therefore I am." On Sat, 1 May 1999 alex@nac.net wrote:
To help quench the effects of smurf attacks on our network, we CEF-CAR all ICMP on our egress points to about 200% of normal ICMP flows.
However, when a upstream becomes full of ICMP (even though we dump most of it), it still affects our external connectivity.
My question is, why don't larger upstream providers use CEF-CAR (assuming that most use this) do the same to limit the effect of smurf attacks on thier (and subsequently, thier customers') networks?
On Sat, 1 May 1999, Joe Shaw wrote:
After dealing with UUNet security regarding several smurf incidents I asked them this same question. Their response (and I'm sure it would be the same response of others) was that a lot of the routers on their network couldn't handle the load of using CEF-CAR to limit smurf attacks.
The explanation I got from uunet regarding smurf attacks and why they dont shut down their smurf amplifiers when notified repeatedly about them, is that their ascend tnt's dont support icmp filtering. -Dan
* Dan Hollis (goemon@sasami.anime.net) [990501 08:35]:
On Sat, 1 May 1999, Joe Shaw wrote:
After dealing with UUNet security regarding several smurf incidents I asked them this same question. Their response (and I'm sure it would be the same response of others) was that a lot of the routers on their network couldn't handle the load of using CEF-CAR to limit smurf attacks.
The explanation I got from uunet regarding smurf attacks and why they dont shut down their smurf amplifiers when notified repeatedly about them, is that their ascend tnt's dont support icmp filtering.
-Dan
I had a different view for the worldcom pops. as they got customers with sub-t1 and t1 they connect it to smaller devices and digger ones to cisco 7513 / b - stdx and fore switches. Nevertheless some URL pointing the uunet structure of a gigapop: http://info.uu.net/tv/unite/low/hubs.html Jan -- Jan Czmok Senior Network Engineer GIGABELL AG
On Sat, 1 May 1999, Joe Shaw wrote:
After dealing with UUNet security regarding several smurf incidents I asked them this same question. Their response (and I'm sure it would be the same response of others) was that a lot of the routers on their network couldn't handle the load of using CEF-CAR to limit smurf attacks.
"the load" ? The point of CAR is that is happens in the CEF path, with no/negligible (1 to 2%) additional load. Are UUNet's routers running that close to the edge? I'd doubt it.
I'm not sure how true that statement was since I'm not familiar with any part of UUNet's backbone equipment other than what I used to get my DS3 from at Insync and now with my MAE Houston connection, but from what I've heard the backbones of a lot of NSP's aren't all made up of Cisco 12000's or even 7500's, and I'd guess a fair amount of the existing routers out there are borderline overloaded since it's next to impossible to get most backbone providers to filter traffic when you're under attack. UUNet certainly wouldn't for us because of "router CPU overhead" last time I was under attack.
What does a 'sho cdp nei' show on your uu-net connecting router? -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Atheism is a non-prophet organization. I route, therefore I am. Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member Father of the Network and Head Bottle-Washer Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834 Don't choose a spineless ISP; we have more backbone! http://www.nac.net -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
In a previous e-mail, alex@nac.net said:
My question is, why don't larger upstream providers use CEF-CAR (assuming that most use this) do the same to limit the effect of smurf attacks on thier (and subsequently, thier customers') networks?
There are several issues with doing this, any one of which might prevent a provider from using it. 1) Can't run CEF. There are some situations under which CEF causes problems. The good news is these are getting to be fewer and fewer every day, but as recently as 6 months ago it would regularly crash routers with some line cards under heavy loads. I expect this reason to disappear completely within another 6 months. Also, in the can't run catagory there are some (usually smaller) providers still using 7000's, 4000's, and other (dare I say even 2501's?) for customer attach. 2) Can't spare the CPU. Sometimes this has to do with the load of CAR, although generally I expect this is due to other things. If you have 150-200 T1 customers on a 7513 (easy to get with CT3 cards) and you run BGP to even just 25% of them, and you still have RSP2's then you probably don't have CPU to even think about giving to CAR, no matter how little it uses. 3) Can't manage it. Providers are understaffed with clueful people. That's a universal truth. If you have 1000 customers, that's 1000 CAR entries to make, 1000 people who may ask why packets get dropped when they do some ICMP thing, 1000 people who might bug you to change to access list parameters. When you have a lot of customers it's probably best to make an all or nothing decision, one off's in large networks tend to make junior engineers make mistakes when they don't understand what's really going on. 4) Don't care. I don't mean this in shallow "screw the customer" way. Rather, if you're a large provider and you provide service to a small provider who's being smurfed you might assume the small provider did something to prevoke the attack, and as such the burden should be on them to track down the sources and report them so they can be perminantly shut off. If it doesn't saturate your links and your routers it's not your problem. 5) It's none of their business. This one works people up. The logic goes like this. If my provider CAR's ICMP automatically, why don't they also CAR porn automatically, so it's only a little traffic. Oh, and SPAM, that should be CAR'ed to help reduce it. All e-mail to and from a competitor, that should be CAR'ed really low.... It's a dangerous road to go down. My $0.02 is I would be very upset if my provider automatically put any sort of "filter" (including CAR) on my links. I do think it is reasonable for them to make whatever effort they can to help me if I get smurfed though. The effort may be CAR, it may be simple filtering, it may be a legitimate "our routers can't take it". -- Leo Bicknell - bicknell@ufp.org Systems Engineer - Internetworking Engineer - CCIE 3440 Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org
1) Can't run CEF. There are some situations under which CEF causes problems. The good news is these are getting to be fewer and fewer every day, but as recently as 6 months ago it would regularly crash routers with some line cards under heavy loads. I expect this reason to disappear completely within another 6 months.
Good arguement. But it seems that no one is doing it.
Also, in the can't run catagory there are some (usually smaller) providers still using 7000's, 4000's, and other (dare I say even 2501's?) for customer attach.
Au contrair, monfrair (sp?!); CEF & CAR is available on many platforms now; we've got it running on 3600's, 4700's, and 7200's. My understanding is that is will also work on 2500's (I was told anything but PowerPC based systems).
2) Can't spare the CPU. Sometimes this has to do with the load of CAR, although generally I expect this is due to other things. If you have 150-200 T1 customers on a 7513 (easy to get with CT3 cards) and you run BGP to even just 25% of them, and you still have RSP2's then you probably don't have CPU to even think about giving to CAR, no matter how little it uses.
As said before, the demonstrable increase in load using CAR is abot 0-2%.
3) Can't manage it. Providers are understaffed with clueful people.
Is this really that hard? access-list 175 permit icmp any any int bleh/bleh rate-limit input access-group 175 128000 8000 8000 conform-action transmit exceed-action drop rate-limit output access-group 175 128000 8000 8000 conform-action transmit exceed-action drop
4) Don't care. I don't mean this in shallow "screw the customer" way. them so they can be perminantly shut off. If it doesn't saturate your links and your routers it's not your problem.
But it could/might. I've seen repeatedly when other downstreams off the same upstream router as us be attacked, the upstreams router usually is unhappy.
5) It's none of their business. This one works people up. The logic goes like this. If my provider CAR's ICMP automatically, why don't they also CAR porn automatically, so it's only a little traffic. Oh, and SPAM, that should be CAR'ed to help reduce it. All e-mail to and from a competitor, that should be CAR'ed really low....
It's a dangerous road to go down.
I don't subscribe to this. Your talking about two different levels of the ISO model :-) -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Atheism is a non-prophet organization. I route, therefore I am. Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member Father of the Network and Head Bottle-Washer Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834 Don't choose a spineless ISP; we have more backbone! http://www.nac.net -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Also, in the can't run catagory there are some (usually smaller) providers still using 7000's, 4000's, and other (dare I say even 2501's?) for customer attach.
Au contrair, monfrair (sp?!); CEF & CAR is available on many platforms now; we've got it running on 3600's, 4700's, and 7200's. My understanding is that is will also work on 2500's (I was told anything but PowerPC based systems).
According to Cisco's documentation CAR is supported on these platforms: * Cisco 2600 series * Cisco 3600 series * Cisco 4500 series * Cisco 4700 series * Cisco 7200 series This is the listing in the IOS 12.0 documentation. I am not sure if CAR is even supported in previous versions of their IOS. It is unlikely, but it is also possible the upstream provider is running non CISCO hardware which doesn't support CAR... === Tim ------------------------------------------------------------------------- | Tim Winders, CNE, MCSE | Email: Tim.Winders@SPC.cc.tx.us | | Network Administrator | Phone: 806-894-9611 x 2369 | | South Plains College | Fax: 806-897-4711 | | Levelland, TX 79336 | | -------------------------------------------------------------------------
This is the listing in the IOS 12.0 documentation. I am not sure if CAR is even supported in previous versions of their IOS.
It's also avail in 11.1.2x(CC) for the 7xxx's. -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- Atheism is a non-prophet organization. I route, therefore I am. Alex Rubenstein, alex@nac.net, KC2BUO, ISP/C Charter Member Father of the Network and Head Bottle-Washer Net Access Corporation, 9 Mt. Pleasant Tpk., Denville, NJ 07834 Don't choose a spineless ISP; we have more backbone! http://www.nac.net -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
In a previous e-mail, alex@nac.net said:
This is the listing in the IOS 12.0 documentation. I am not sure if CAR is even supported in previous versions of their IOS.
It's also avail in 11.1.2x(CC) for the 7xxx's.
7[25].. There are no new CC images for 7000's. -- Leo Bicknell - bicknell@ufp.org Systems Engineer - Internetworking Engineer - CCIE 3440 Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org
In a previous e-mail, Stephen Stuart said:
It's also avail in 11.1.2x(CC) for the 7xxx's.
7[25]..
There are no new CC images for 7000's.
But the rsp images work if you have an RSP7000, yes?
Correct. Cisco makes this confusing. RP No CC RP+SP No CC RP+SSP No CC RSP7000 CC -- Leo Bicknell - bicknell@ufp.org Systems Engineer - Internetworking Engineer - CCIE 3440 Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org
It's also avail in 11.1.2x(CC) for the 7xxx's.
7[25]..
There are no new CC images for 7000's.
But the rsp images work if you have an RSP7000, yes?
Stephen
Yes... Cisco Internetwork Operating System Software IOS (tm) RSP Software (RSP-PV-M), Experimental Version 12.0(3.7)S4 [soma-v120_4_s_throttle.build 100] Copyright (c) 1986-1999 by cisco Systems, Inc. Compiled Tue 20-Apr-99 12:21 by soma Image text-base: 0x60010908, data-base: 0x60AB2000 ... sandbox uptime is 8 hours, 11 minutes System returned to ROM by reload System restarted at 14:45:39 UTC Sat May 1 1999 System image file is "slot0:rsp-pv-mz.120-3.7.S4" cisco RSP7000 (R4600) processor with 131072K/2072K bytes of memory. R4600 CPU at 100Mhz, Implementation 32, Rev 2.0
participants (9)
-
alex＠nac.net -
bmanning＠vacation.karoshi.com -
Dan Hollis -
Jan Ahrent Czmok -
Joe Shaw -
Leo Bicknell -
R.P. Aditya -
Stephen Stuart -
Tim Winders