- NANOG - lists.nanog.org

RE: MPLS VPNs or not?
by Irwin Lazar 07 Aug '01

07 Aug '01

IMHO, this was a poorly written article. From a security standpoint, every argument against MPLS-VPNs could also apply to ATM or F/R PVCs in that none of the three provide any encryption. They try to paint MPLS as something its not, then criticize it for not being what they want it to be. It's akin to criticizing an automobile for not being able to fly. I thought it most ludicrous that Tom Nolle would criticize MPLS-VPNs because they will likely only be used on carrier's private IP networks. Based on that logic, we should also criticize ATM and Frame Relay since carriers only offer services across their own backbones. I also found it interesting that the critics of MPLS as "bad for the 'net" work for a company that currently offers MPLS-based VPN services in the U.S. (IP Enabled Frame Relay) and is rolling those services out to Europe while also expanding them to support ATM speeds. I'm betting that there are quite a few very upset product managers at AT&T this morning. Irwin > -----Original Message----- > From: Hank Nussbacher [mailto:hank@att.net.il] > Sent: Tuesday, August 07, 2001 1:37 AM > To: nanog(a)merit.edu > Subject: MPLS VPNs or not? > > > > Experts call MPLS bad for 'Net > > http://www.nwfusion.com/news/2001/0806mpls.html > > Intersting read. This should start a nice long thread :-) > > -Hank >

1 0

Followup on Cleveland Keith Building Fire
by Sean Donelan 07 Aug '01

07 Aug '01

The Keith Building is your typical multi-tenent, multi-purpose office building, which also has become a defacto carrier hotel. Over 20 different carriers have equipment, and circuits in the building. Although the property damage was limited, and no one was severely injured, during the fire department response several providers had service interruptions. In a multi-tenent building, it is often difficult to do pre-fire planning. Even though we are all competitors, in such situations, we all share the same fate. On fire-life-safety issues it would make sense for those of us in each of organizations to ensure the overall safety of the buildings we are all in. The National Fire Protection Assocation is developing guidelines for telecommunication facilities. Are there any lessons we can learn from the Keith Building fire which should be included in the recommended guidelines? Can we work with the local fire departments on more effective pre-fire planning?

1 0

RE: TCP session disconnection caused by Code Red?
by James Smith 07 Aug '01

07 Aug '01

I can see "connection refused" being caused by lack of resources (memory, not CPU) caused by ARP requests not being resolved and waiting to time out. What happens is that all the outstanding ARP requests use up all available memory, so no buffers can be allocated for new incoming connections. Send enough requests fast enough, looking for enough different IP's, memory gets exhausted. Been there, seen it happen, tweaked the WellFleet/Bay/Nortel knob to limit the amount of space used for ARP resolution, and solved the problem caused by too many outstanding ARPs. Of course, I could have stuffed in more memory, but limiting the space used for the process was easier. James H. Smith II NNCDS NNCSE Systems Engineer The Presidio Corporation -----Original Message----- From: George William Herbert [mailto:gherbert@retro.com] Sent: Monday, August 06, 2001 2:57 PM To: mike harrison; nanog(a)merit.edu Subject: Re: TCP session disconnection caused by Code Red? mike harrison <meuon(a)highertech.net> wrote >Blaz Zupan <blaz(a)amis.net> wrote: >> For the last few days, our network seems to be basically unreachable from the >> outside. Most incoming TCP sessions (web requests, incoming mail, telnet >> sessions, etc.) often fail with a simple "Connection refused" like nobody is > >Your routers are brain dead from the load.. routers that are used to >handling a few thousand connections are being asked to handle 10's of >thousands. 1 good 1000+ address scan from an ISDN user kills my >Lucent/Ascend TNT unless we filter for it. I've been told (but not given permission to forward details of who/how/what) that some major sites with a single router and relatively flat network topology are dying due to the ARP request flood that is being generated by Code Red scans on the inside of their border router choking the router. Check the rate of ARP requests coming off your border router and see if it seems excessive; if so, that may be it. -george william herbert gherbert(a)retro.com

2 1

TCP session disconnection caused by Code Red?
by Blaz Zupan 07 Aug '01

07 Aug '01

I'm seeking the collective wisdom of this list to try to prove that I'm not halucinating. For the last few days, our network seems to be basically unreachable from the outside. Most incoming TCP sessions (web requests, incoming mail, telnet sessions, etc.) often fail with a simple "Connection refused" like nobody is listening on that port (but of course there is a server running on the port). It does not matter which service I'm trying to connect to, it also does not matter which server I'm connecting to. It happens randomly, i.e. three successive connections fail, then 5 connections succeed, then 2 connections fail, etc. Outgoing connections (i.e. our customers surfing the web) work just fine - except active FTP, which of course contains an implicit "incoming" TCP session. I've been fighting with this since friday and have done some packet traces. By comparing successful and unsuccesful packet traces I came to the conclusion that my problems are being caused by incoming TCP packets with the RST bit set. So I applied the following access list on the link to our upstream: access-list 170 deny tcp any any rst access-list 170 permit ip any any After doing this, our incoming TCP sessions magically started working. Looking at the packet counters, I see about 20% of our incoming packets are TCP RST packets. Putting the same filter on an internal link, I see about 1% of TCP RST packets. Turning on access list logging, I see that most of the packets are destined for port 80 on unused IP addresses (which are nullrouted) - which I guess is Code Red searching for victims. So now tell me, am I dreaming or has Code Red found a bug in Cisco IOS? :) Or is this some kind of new (or old?) denial of service attack disguising as Code Red? I've reported this to psirt(a)cisco.com and I'm waiting to see what they come up with. Anyone seen anything like this? I see the same thing with 12.0(14)S2, 12.0(17)S, 12.0(18)S and 12.2(1). Machine is a 7206 VXR, NPE-400, PA-2E3, 2FE I/O controller. Blaz Zupan, Medinet d.o.o, Trzaska 85, SI-2000 Maribor, Slovenia E-mail: blaz(a)amis.net, Tel: +386-2-320-6320, Fax: +386-2-320-6325

3 5

Re: The Death of TCP/IP
by Michael 07 Aug '01

07 Aug '01

This is the article Cringely should have written: http://www.dismal.com/thoughts/article.asp?aid=1316 Michael > -----Original Message----- > From: Dan Hollis [mailto:goemon@anime.net] > Sent: Monday, August 06, 2001 3:11 PM > To: Mike Batchelor > Cc: nanog(a)merit.edu > Subject: RE: The Death of TCP/IP > > > > On Mon, 6 Aug 2001, Mike Batchelor wrote: > > > http://www.pbs.org/cringely/pulpit/pulpit20010802.html > > Cringley was amusing as the host for "Pirates of Silicon > Valley", but his > > expertise in technology clearly stops there. > > Is this the same cringley who falsely claimed a Ph.D from Stanford? > > -Dan > > -- > [-] Omae no subete no kichi wa ore no mono da. [-] >

1 0

Re: TCP session disconnection caused by Code Red?
by Jim Warner 07 Aug '01

07 Aug '01

George Herbert (gherbert(a)retro.com) said: I've been told (but not given permission to forward details of who/how/what) that some major sites with a single router and relatively flat network topology are dying due to the ARP request flood that is being generated by Code Red scans on the inside of their border router choking the router. Check the rate of ARP requests coming off your border router and see if it seems excessive; if so, that may be it. My campus is now seeing around 500 packets per second of CodeRed connection requests into blocks that total 130,000 addresses. The work of ARPing for all that stuff is distributed across 6 routers (mostly Cisco RSM) and they're doing OK. The experience at Cal State campuses is variable. They do tend to have flat topologies behind a single entrance router. Our neighbors at CSU Monterey Bay melted under the load _before_ CR-II emerged. They had their upstream install an access list permitting an explicit list of campus web servers and blocking port 80 to everything else. Their router is mfg by Alcatel, a PowerRail 7652 OmniCore 5052. I don't have a lot of details about their setup but I do know how big their assigned address pool is -- about 6000 addresses. That leaves me wondering about the importance of quality of the ARP implementation in the router code. I have heard that CSU Long Beach (Cabletron router) had similar problems. I'm looking for anyone that knows some details of Alcatel and SSR routers that might help us understand this. I don't think this is a case where the ingenuity of ASIC designers is the important parameter. Have any of the router benchmarks dealt with the capabilities and robustness of what I'm imagining must be process-level parts of the a router implementation? -jim warner, University of California Santa Cruz

1 0

Re: TCP session disconnection caused by Code Red?
by George William Herbert 06 Aug '01

06 Aug '01

mike harrison <meuon(a)highertech.net> wrote >Blaz Zupan <blaz(a)amis.net> wrote: >> For the last few days, our network seems to be basically unreachable from the >> outside. Most incoming TCP sessions (web requests, incoming mail, telnet >> sessions, etc.) often fail with a simple "Connection refused" like nobody is > >Your routers are brain dead from the load.. routers that are used to >handling a few thousand connections are being asked to handle 10's of >thousands. 1 good 1000+ address scan from an ISDN user kills my >Lucent/Ascend TNT unless we filter for it. I've been told (but not given permission to forward details of who/how/what) that some major sites with a single router and relatively flat network topology are dying due to the ARP request flood that is being generated by Code Red scans on the inside of their border router choking the router. Check the rate of ARP requests coming off your border router and see if it seems excessive; if so, that may be it. -george william herbert gherbert(a)retro.com

7 6

The Death of TCP/IP
by Robert Hough 06 Aug '01

06 Aug '01

Felt like sharing this most amusing article that I discovered in my Inbox this morning: http://www.pbs.org/cringely/pulpit/pulpit20010802.html -- Robert Hough (rch(a)acidpit.org)

6 5

Re: CodeRedII worm..
by Larry Sheldon 06 Aug '01

06 Aug '01

> > worm creates a known backdoor. I'm certain that both the CodeRedII author > > and other black hats would love for us to compile a list of afflicted hosts > > for them to use. > > They have a few 'friendly' webservers collecting addresses > just like we do. Everyone on the 'net with a sniffer or web log now > has such a list. It's a good thought though. If we are pretty sure that is the case, how about posting a list somewhere for the good guys to see--or somebody send email to the ARIN-listed contact for the IP addresses detected. I'm trying to build a detector here, but it is hard, given the resources I can bring to bear. Mostly me, which means we are in really bad shape, resource-wise.

4 3

Negative ARP caching [was Re: TCP session disconnection caused by Code Red? ]
by Alex Bligh 06 Aug '01

06 Aug '01

>> Probably >> a bad idea. Rate limiting, as RFC 1122 suggests, would seem to be much >> better. > > Agree that rate-limiting is a good idea (indeed that's INCOMPLETE[n] and > [t1] in my proposal) but I don't see how it helps here. Or to put it another way: So if you are storing sufficient state in the OS to do rate-limiting (i.e. keeping state on incomplete / nonexistant entries too), then put it to some use, and (say) halve the rate-limit every time one proves non-existant (and you drop the queued packet(s)), (i.e. twice as many seconds between ARP packets), down to some minimum, like one every 5 mins, and reset the rate limit on reception of any IP packet from that machine and/or successful ARP. This is almost the same thing as I suggested, but looks more like rate limiting, with some intelligence as to the rate. -- Alex Bligh Personal Capacity

1 0