- Test - lists.nanog.org

RE: TNT issues "workaround"
by Ejay Hire 25 Aug '03

25 Aug '03

In response to this, I'd like to comment on Lucent's excellent response to this issue. Never before have I seen such an effective application of hear-no-evil, see-no-evil, and speak-no-evil. While other vendors were wasting our time sending out notifications of bugs / vulnerabilities with workarounds / fixes, Lucent's response was much more efficient and eloquent. Here is a summary of the responses I've received over several days phone calls, calling for my current employer who does not have a service contract, and one for one of my consulting customers who does have a service contract. If you have a service contract, I'm allowed to tell you we aren't aware of any issues. If you don't have a service contract, I'm allowed to sell you a service contract. On that note, I might have some TNT's for sale pretty soon. -Ejay -----Original Message----- From: Brian Wallingford [mailto:brian@meganet.net] Sent: Saturday, August 23, 2003 12:41 AM To: nanog(a)merit.edu Subject: TNT issues "workaround" I haven't seen specific details posted here, so: Like many others, we've had a few TNTs online for years without hiccups or reboots until this week. Beginning late Sunday, we saw seemingly random blade reboots, and total system crashes. Errors ranged from memory leaks to infinite loops on the controller blade, but all blades were susceptible. HDLC2 blades seemed to be particularly vulnerable. We saw boxes that had been rock-solid for very long periods suddenly rebooting at periods ranging from 20 minutes to 4 hours, with no obvious cause (i.e., nothing more specific than the above). Border and core filtering of icmp echo * did little good. On the suggestion of some folks on another list, and against my better judgment, we disabled route caching in order to free up additional memory (though memory did not appear fragmented). This stabilized all involved boxes, and surprisingly, did not result in significant degradation of end user performance. Granted, it's not a true fix, but it may get you a few extra Z's at night. hth, brian

3 2

Virus
by Wesley Vaux 25 Aug '03

25 Aug '03

Has anyone noticed that after patching and cleaning machines that they become infected again? Is this just me? What is your process for remove Nachi.worm. Sorry for bringing this back up. Wes Vaux, CCNA, CCDA Network Security Engineer, 9000 Regency Pkwy Ste 500 Cary, NC 27511 t 919.463.6782 f 919.463.1290 Global Knowledge Experts Teaching Experts http://www.globalknowledge.com

1 0

Oh the shame of being lame!
by Rob Thomas 25 Aug '03

25 Aug '03

Hi, NANOGers. Are your DNS zones coming up lame? Why not check the DNS lamer report as you sip your Monday morning coffee? There are 19621 entries in the lamer report for the week ending 24 AUG 2003. These entries are often indicators of greater problems with name server configurations. <http://www.cymru.com/DNS/lame.html> We update this report each week, Sunday evening US CT. On a related note: We want your data! Please consider donating to Team Cymru the lame delegation log entries from your name server logs. We are happy to credit you and your employer, or keep the source anonymous, as you prefer. Such donations help the community at large. Be the first in your ASN to donate data! :) If you are interested in donating data, please contact us at team-cymru at cymru.com. Thanks! Rob, for Team Cymru. -- Rob Thomas http://www.cymru.com ASSERT(coffee != empty);

1 0

RE: FW: TNT issues "workaround"
by John Lord 24 Aug '03

24 Aug '03

Ive been watching mine and finally see this error *Mar 19 14:48:53.951 UTC: %SYS-2-MALLOCFAIL: Memory allocation of 2708 bytes failed from 0x603FE6C0, alignment 0 Pool: Processor Free: 6402796 Cause: Memory fragmentation Alternate Pool: None Free: 0 Cause: No Alternate pool -Process= "ISDN L2 Process", ipl= 0, pid= 94 -Traceback= 603FC690 603FDC90 603FE6C8 60102990 600A84A8 600A8D34 *Mar 19 14:48:54.635 UTC: %SYS-2-CFORKMEM: Process creation of Async tty Reset failed (no memory). -Process= "Serial Background", ipl= 0, pid= 8 -Traceback= 6040D2A4 60414AEC 606B03C4 606B06A0 600704C4 Im doing b) blocking all echo/echo-reply coming in from dial-up users (i.e. apply an input acl to your virtual-template and/or group-async interfaces). But it doesn't seem to stop it from locking up. Ive been killing users as I see users with the worm , This is on a cisco as5300 by the way. Whats the command to disable route cache? John Lord(lord(a)allturbo.com) It Manager AllTurbo Internet Services Inc 410-213-9388 Office www.allturbo.com -----Original Message----- From: jlewis(a)lewis.org [mailto:jlewis@lewis.org] Sent: Saturday, August 23, 2003 6:43 PM To: Ross Chandler Cc: John Lord; nanog(a)merit.edu Subject: Re: FW: TNT issues "workaround" On Sat, 23 Aug 2003, Ross Chandler wrote: > > I seem to be having the same or similar problems with my Cisco boxes > > also , they either reboot or the pris hang , users get busy's but no > > one is logged in at all , when I do a show isdn status it shows b > > channels in use but no one on, the only way to fix is reboot the box > > , and it seems to be timed , everyday at 1400 and 2200 hours , since > > Monday anybody body heard of ciscos acting funny this week? > > Perhaps your fast switching route cache is filling up memory. If > you're willing to risk it enable CEF on all interfaces. Some of the older cisco access-servers don't even support CEF. The cisco failures seem to be memory starvation/fragmentation issues caused by out of control route-cache growth caused by the nachi worm's attempt to ping so many different hosts so quickly while looking for systems to spread to. You can work around the issue by: a) using policy routing to pass all dialup traffic through a route-map that sends 92 byte echo/echo-reply packets to null0. b) blocking all echo/echo-reply coming in from dial-up users (i.e. apply an input acl to your virtual-template and/or group-async interfaces). c) disabling route caching on the egress interface of the access server. I'm doing a mix of a (on the access-servers that this works on) and b where a doesn't work...and tested c this morning and found it appears to work. ---------------------------------------------------------------------- Jon Lewis *jlewis(a)lewis.org*| I route System Administrator | therefore you are Atlantic Net | _________ http://www.lewis.org/~jlewis/pgp for PGP public key_________

2 1

Fortune: Peter Lewis reports from the front lines of the worm wars
by Sean Donelan 24 Aug '03

24 Aug '03

Peter Lewis, a writer for Fortune.com (and I believe previously for the New York Times) reviews his week in his Weblog. The East Coast blackout, two disk crashes, a backhoe cuts his phone lines, worm attacks on AOL Time Warner's corporate network, and then the worm gets him (or rather his computer). http://www.fortune.com/fortune/ontech/0,15918,477882,00.html The Worm Turns - August 22 2003 > > Did this week suck, or what? > > Perhaps as retribution for my smug comments on the radio last week that > MY house, MY office and MY computer networks were unaffected by the Big > Blackout of 2003, the technology banshees came visiting Monday morning. [...]

1 0

XO as Backbone provider - try again
by Bil Herd 24 Aug '03

24 Aug '03

Sorry for the HTML post (boo, hiss) Anyone have positive or negative experiences with XO as a 'tier1' provider? We are re-evaluating orur backbone connections. Bil Herd - INS

3 2

FW: TNT issues "workaround"
by John Lord 23 Aug '03

23 Aug '03

I seem to be having the same or similar problems with my Cisco boxes also , they either reboot or the pris hang , users get busy's but no one is logged in at all , when I do a show isdn status it shows b channels in use but no one on, the only way to fix is reboot the box , and it seems to be timed , everyday at 1400 and 2200 hours , since Monday anybody body heard of ciscos acting funny this week? John Lord(lord(a)allturbo.com) It Manager AllTurbo Internet Services Inc 410-213-9388 Office www.allturbo.com -----Original Message----- From: Brian Wallingford [mailto:brian@meganet.net] Sent: Saturday, August 23, 2003 1:41 AM To: nanog(a)merit.edu Subject: TNT issues "workaround" I haven't seen specific details posted here, so: Like many others, we've had a few TNTs online for years without hiccups or reboots until this week. Beginning late Sunday, we saw seemingly random blade reboots, and total system crashes. Errors ranged from memory leaks to infinite loops on the controller blade, but all blades were susceptible. HDLC2 blades seemed to be particularly vulnerable. We saw boxes that had been rock-solid for very long periods suddenly rebooting at periods ranging from 20 minutes to 4 hours, with no obvious cause (i.e., nothing more specific than the above). Border and core filtering of icmp echo * did little good. On the suggestion of some folks on another list, and against my better judgment, we disabled route caching in order to free up additional memory (though memory did not appear fragmented). This stabilized all involved boxes, and surprisingly, did not result in significant degradation of end user performance. Granted, it's not a true fix, but it may get you a few extra Z's at night. hth, brian

3 2

Moving quickly in network design
by Sean Donelan 23 Aug '03

23 Aug '03

http://www.washingtonpost.com/wp-dyn/articles/A34422-2003Aug22_2.html >Jonathan Zittrain, a Harvard Law assistant professor. "Now one person >really can change the world. But that's also what's terrifying." > >When hackers three decades ago found they could get free calls on pay >phones using a toy whistle that mimicked the phone's network signals, >they exploited the system's vulnerabilities in much the same manner as >today's viruses, Zittrain said. Phone firms, though, were able to >quickly change their network design. But the Internet is fundamentally a >different type of technology. Actually it took the telephone system over 30 years to convert from multi-frequency (per-trunk) singaling to common channel interoffice signaling. The signaling standard has gone through a few versions and is now known as SS7. SS7 security isn't all its cracked up to be. The Bell System lucked out because they had started development in the 1960's of new signaling methods not because of security, but because of money. Busy signals were tying up half the long distance lines in the US, and in the old days Ma Bell didn't get paid for busy signals. The business case for developing CCIS was primarly driven by more efficient (ie. more "paid" calls) use of the network, not security. By the time Capt'n Crunch found his 2600Hz whistle in the early 1970's, Ma Bell could "move quickly" (i.e. it still took several more years) because it had spent a decade already working on CCIS.

1 0

RTT/delay presentation from Defcon
by Anton L. Kapela 23 Aug '03

23 Aug '03

All, I put a small talk on at this years Defcon, discussing some of the rtt work I've been doing. For those interested in the topic, I've placed an mp3 of the presentation and my slides here: http://144.92.40.150/~xam/misc/dc-11/dc11-talk.htm http://144.92.40.150/~xam/misc/dc-11/020.MP3 Enjoy, --Tk

1 0

Sobigf + BGP
by J. Oquendo 23 Aug '03

23 Aug '03

I was reading some PDF files on BGP along with Routing TCP/IP v2, and I found myself pondering what a nasty damn worm it would be if someone were to do something using winpcap in conjucting with the worm/virus, and I was a bit confused, disturbed, lost. So I drew up a quick question complete with ascii which can be viewed at politrix.org/segment/brat.txt for those who get a distorted diagram... Apologies beforehand if this post seems a bit odd, but I did not see anything similar to a networking 'vuln'dev', and besides I wouldn't think that any one here would do something malicious with any idea that actually worked for the worse. ------------------------------------------------------------- brat.txt I was thinking about the recent polymorphic Sobigf worm/virus and wondered about the following hypothetical scenario... Sorry about this ASCIIgram, I didn't want to look for Visio nor any other graphic program to do this in, strictly terms to keep it gritty... So here goes. Attacker scripts Sobigf variant with a virii/worm generator, and uses pcap (packet capture) under Windows to have his worm send out predefined packets. Let's say he created what I call a 'BRAT' BGP Router Attack Tool. Now this tool isn't something major it simply sends out two types of packets aimed at routers running BGP. They're both Notification Messages: ========================================= Packet 1 = BGP NM ERROR CODE 2 SUBCODE 2 | Packer 2 = BGP NM ERROR CODE 6 | ========================================= Now we have the hosts' information: www.targetednap.net (4 if's) 192.168.1.1 192.168.4.1 10.10.1.1 10.10.5.1 VIC's nap.maefi.com Link 1 nsp.maefee.com Link 2 nap.maefo.com Link 3 nsp.maefum.com Link 4 Link 1 Link 2 \ / ----------------- | | | Targetednap.net | | | ----------------- / \ Link 3 Link 4 Script kiddiot sets up his worm/virii to send packets as Targetednap to all VIC's as Targetednap via spoofing using WinPCAP. Given the rate of connections that were mentioned for SoBigf, what could happen say if route dampening were used between the routers. Would penalties keep adding up making the connection intolerable because of latency, would it ignore it. Or what could happen say if worm was smart enough to send NLRI's of something like $targetvalue=0 Wouldn't this knock off connections between BR's/ABR's, etc. Are there any flags one can take to prevent this from occurring. Keep in mind that packet creation is not difficult. My guess would be, even if someone didn't get all fancy with the packets being sent, a couple of million packets sent with say a: ping -l 25000 $VIC as $TARGETEDNAP would be enough to cause some massive latency, maybe even disconnect a backbone perhaps? Anyone care to share links on security on this level if any are available ================================================= J. Oquendo rsvp: segment ... antioffline . com PGP Fingerprint 39A7 24C6 A9A0 6C67 96CA 0302 F1D3 2420 851E E3D0 http://www.politrix.org http://www.antioffline.com ================================================= -- __________________________________________________________ Sign-up for your own personalized E-mail at Mail.com http://www.mail.com/?sr=signup CareerBuilder.com has over 400,000 jobs. Be smarter about your job search http://corp.mail.com/careers

2 1