- Test - lists.nanog.org

Process management
by Sean Donelan 08 Jul '98

08 Jul '98

>Customer Support and process management seems to be the Achilles' heel of >the Internet industry. This industry has seemed to build up a terrible >reputation for poor customer support. Coming from the Nuclear industry, I >am truly jaded when it comes to process management, but the Internet >industry seems to have taken a track as far as possible on the other side. Ok, so I'm wierd. I enjoy reading the incident reports nuclear plant operators have to file. One nuclear power plant operator discovered the diesel fuel for their standby emergency generators was actually syrup. How many ISPs actually test what's in that tanker truck before filling up their generators? Or how about the time all the uninterruptiable power supplies at a plant turned off in unison because of a bug in the UPS firmware. How many ISPs would even know that other people had a problem with a particular brand of UPS, and they should check theirs? Has anyone, anywhere, ever really solved these problems? I don't know, but I suspect not. The nuclear industry generates a LOT of paperwork, but its just a bunch of dead trees if no one reads and learns from it. The nuclear industry seems to suffer from the 'Its not my fault because I followed approved procedures.' The internet industry seems to suffer from the 'Its not my fault because no one controls the Internet.' The airline industry suffers from 'Its always the captin's fault.' The telephone industry suffers from 'Its not my fault, we're the telephone company, we can't be wrong.' And so forth. No one just 'knows' this stuff. The cliches do ring true. You either learn it the hard way from your own mistakes. Or you learn it the easy way from other people's mistakes. I'm not saying mistakes can be eliminated. Too many useful discoveries have resulted from mistakes. AS7007 was pretty nasty, but if we had learned our lesson, it might have been worth it. The problem is we didn't learn our lesson. We really need to work on communicating the lessons learned. The U.S. Military has a wonderful web site on 'lessons learned' which they update almost every day. If you are cynical, you might say that just shows the military makes mistakes almost every day. On the other hand, if the lessons are really learned, it means every day there is one less mistake which will occur in the future. I suspect one of the academic types would love to write a grant proposal to operate a web site for Internet operations lessons learned. The question is would anyone contribute anything to it. Doubtful after the first month. And who gets the book deal with O'Reilly. >There apparently have been several organizations developed recently that >claim to be attempting to address this problem. However, I have not seen >an industry-wide affect by the operations of these organizations. Then let's get together and form one. The standard operating procedure for forming a new organization is everyone puts in $25,000 and the organization puts out a press release announcing the start of a search for an executive director. We can name W.C. Fields as an ex-officio board member so we can say to tiny tykes, "Go away son, you're bothering me." As you point out there are a number of organizations which have popped up. However it seems to be a lot easier to get mega-corporations to contribute funding in exchange for having their name on a 'worthy cause' than it is to do something concrete. With the millions of dollars ISP salespeople have told me they are spending on their customer service center phone switches, why do I continue to have such a problem getting an answer the question "What phone number should I call if I want to report a problem?" -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

2 1

Re: small vent
by dlr＠bungi.com 08 Jul '98

08 Jul '98

[In the message entitled "Re: small vent" on Jun 28, 14:54, Jon Lewis writes:] > On Sun, 28 Jun 1998, Mark Milhollan wrote: > > > "David R. Conrad" writes: > > >b) someone has dummied up a prefix in a mail header or something (typical) > > >c) there is a bug in the database software (happens) > > > > I doubt B since the IP address queried was that of the TCP peer. > > Granted it could have been a spoofed session, but that would have > > required a level of expertise that I've never seen used merely to > > attempt delivery of a few thousand pieces of e-mail. > > What about d). Perhaps someone temporarily announced routes for > unallocated space, setup a mail/spam server in that IP space, sent out > their mail, and stopped exporting the route. > > Is there a place one can search to see a history of BGP announcements for > a given route? This has, in fact, happened before. One of the reasons that the unallocated spaces are listed on the RBL. -- Dave Rand dlr(a)bungi.com http://www.bungi.com

3 2

MTU problems with GRE tunnels (fwd)
by Jens Schweikhardt 08 Jul '98

08 Jul '98

To whom it may concern, here is some email I received in the last months followed by some of my observations which might be related to the problems discussed. I have posted my obervations to comp.sys.dcom.cisco and opened a trouble ticket with cisco's technical assistance center. # Forwarded message: # > From merit.edu!errors-nohumans Fri Jun 5 23:44:49 1998 # > Message-Id: <3.0.3.32.19980605095358.006ebd4c(a)mailhost.ip-plus.net> # > X-Sender: bridge(a)mailhost.ip-plus.net # > X-Mailer: QUALCOMM Windows Eudora Pro Version 3.0.3 (32) # > Date: Fri, 05 Jun 1998 09:53:58 +0100 # > To: nanog(a)merit.edu # > From: philip bridge <bridge(a)ip-plus.net> # > Subject: MTU problems with GRE tunnels # > Mime-Version: 1.0 # > Content-Type: text/plain; charset="us-ascii" # > Sender: owner-nanog(a)merit.edu # > Content-Length: 1881 # > # > I'm experiencing problems with fragmentation due to Cisco GRE tunnel # > overhead: the way I understand it, the MTU if a GRE tunnel will always be # > less than the MTU of the underlying IP cloud (in our case 1500 bytes) due # > to the IP encapsulation overhead. So 1500 byte packets attempting to # > traverse the tunnel will be fragmented, or dropped if the DF bit is set, in # > which case an ICMP message is send back to the originating host # > # > We're trying to use GRE tunnels extensivly in some fancy added-value # > Internet services, and it seems that there is a small but significant # > amount of application traffic out there that has problems when traversing a # > GRE tunnel with MTU < 1500. We've seen two problems: # > # > - 1500 byte packets with DF set. This is either application traffic, or MTU # > path discovery is broken, because the same packets get sent repeatedly # > - 1500 byte packets get fragmented, but the destination host cannot cope # > with the fragmentation (firewall issues?) # > # > We see this on a variety of platforms (from 2500, 7507) and a variety of # > IOS releases (11.1(18)CC, 11.1(2), 11.2(5). Talking to another provider # > indicates that the same problem exists with other vendors, and is having # > the same severe impact. # > # > Thinking about it, this is a problem is to be expected with IP tunnels of # > all types, but I am surprised at the extent it's influence on our # > customer's applications (such as large emails). I do not want to overstate # > the proportion of traffic we see with this problem - but it does seem to be # > enough to render GRE tunnels very problematic - to say the least. But I # > know lots of people are using GRE for this or similar applications...so # > what am I missing here. # > # > thanks in advance for help/tips # > # > Phil # > # > # > # > ______________________________________________________________ # > Philip Bridge # > ++41 31 688 8262 bridge(a)ip-plus.net www.ip-plus.ch # > PGP: DE78 06B7 ACDB CB56 CE88 6165 A73F B703 # > # # # -- # Bernhard Kroenung, Bahnhofstr 8, 36157 Ebersburg/Rhoen, Germany +49 6656 910101 # @work : bernhard(a)kroenung.de Work: +49 661 9011777 # @home : horke(a)Rhoen.De @school : Bernhard.Kroenung(a)Informatik.FH-Fulda.De # hello, world\n Here's something very strange I observe with GRE tunnels (the default tunnel mode). It looks like cisco routers send IP datagrams violating RFC 791 [Internet Protocol] over GRE tunnels. In particular, the length field of the IP header is computed incorrectly to *not* include the size of the IP header. RFC 791 says about the length field: <quote> Total Length: 16 bits Total Length is the length of the datagram, measured in octets, including internet header and data. This field allows the length of a datagram to be up to 65,535 octets. ... </quote> I have an application on my workstation that serves as one endpoint of a GRE tunnel. In fact, it's such a tiny perl program that I have appended it at the end of this mail. Here's the tunnel config on my cisco, which is a IOS (tm) 4500 Software (C4500-P-M), Version 11.2(9), RELEASE SOFTWARE (fc1): interface Tunnel2 description GRE Test Tunnel ip address 10.0.0.1 255.255.255.252 tunnel source 193.174.247.254 !another iface of this cisco tunnel destination 193.174.247.193 !my workstation's address tunnel key 42 !optional Let's ping the other end of the tunnel: io#ping 10.0.0.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds: ..... Success rate is 0 percent (0/5) Here's what the perl tunnel endpoint outputs: Length of received packet: 128 <<<<<<<<< Note this version: 4 header len: 5 tos: 0 length: 108 <<<<<<<<< Note this id: 1586 flags: 0 offset: 0 ttl: 255 protocol: 47 chksum: 16895 source: 193.174.247.254 destination: 193.174.247.193 20 00 08 00 00 00 00 2a 45 00 00 64 01 39 00 00 ff 01 a6 5d 0a 00 00 01 0a 00 00 02 08 00 51 68 00 00 23 a5 00 00 00 01 9a 8b 6e b0 ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd ab cd Or let's try a telnet session: io#telnet 10.0.0.2 Trying 10.0.0.2 ... Length of received packet: 72 <<<<<<<<< Note this version: 4 header len: 5 tos: 0 length: 52 <<<<<<<<< Note this id: 1591 flags: 0 offset: 0 ttl: 255 protocol: 47 chksum: 16946 source: 193.174.247.254 destination: 193.174.247.193 20 00 08 00 00 00 00 2a 45 00 00 2c 00 00 00 00 ff 06 a7 c9 0a 00 00 01 0a 00 00 02 52 02 00 17 52 c8 26 04 00 00 00 00 60 02 10 c0 a8 9a 00 00 02 04 05 98 We note that the length as reported in the IP header is always 20 octets less than what we receive on the socket. This leads me to the question Do you cisco guys read RFCs? :-) Regards, Jens Schweikhardt -- ## Network Operation Center, DFN-Verein Geschäftsstelle Stuttgart ## ## http://www.noc.dfn.de/ finger trouble(a)noc.dfn.de wartung(a)noc.dfn.de ## ## >>>>>> mailto: noc(a)noc.dfn.de <<<<<< ## Here's my perl script: #!/usr/local/bin/perl5 -w # # GRE Tunnel Endpoint; prints all GRE packets received. # # Author: Jens Schweikhardt <schweikh(a)noc.dfn.de> # # >>> You probably need root permission to open the raw socket. <<< use Socket qw (SOCK_RAW PF_INET); use strict; my $gre = 47; # Generic Routing Encapsulation my $rbits; # bitmask with read file descriptors for select my $out; # writable copy of rbits for select to clobber my $nready; # return value from select unless (socket (SOCKET, &PF_INET(), &SOCK_RAW(), $gre)) { print STDERR "gre socket: $!\n"; exit 1; } $rbits = ''; vec ($rbits, fileno SOCKET, 1) = 1; for (;;) { $nready = select ($out = $rbits, undef, undef, undef); last unless defined $nready; # Should not happen... &receive_packet () if $nready; # A packet is waiting } close SOCKET; exit 0; sub receive_packet { my $from_msg = ''; my $from_saddr = recv (SOCKET, $from_msg, 1500, 0); unless (defined $from_saddr) { print STDERR "recv: $!\n"; return 0; } print "\nLength of received packet: ", length ($from_msg), "\n"; my ($delivery_ip_version, $delivery_ip_ihl, $delivery_ip_tos, $delivery_ip_length, $delivery_ip_id, $delivery_ip_flags, $delivery_ip_offset, $delivery_ip_ttl, $delivery_ip_proto, $delivery_ip_chksum, $delivery_ip_src, $delivery_ip_dst, $delivery_ip_options, $delivery_ip_data ) = &ip_unpack ($from_msg); print "version: $delivery_ip_version\n"; print "header len: $delivery_ip_ihl\n"; print "tos: $delivery_ip_tos\n"; print "length: $delivery_ip_length\n"; print "id: $delivery_ip_id\n"; print "flags: $delivery_ip_flags\n"; print "offset: $delivery_ip_offset\n"; print "ttl: $delivery_ip_ttl\n"; print "protocol: $delivery_ip_proto\n"; print "chksum: $delivery_ip_chksum\n"; printf "source: %u.%u.%u.%u\n", unpack ('C4', pack ('L', $delivery_ip_src)); printf "destination: %u.%u.%u.%u\n", unpack ('C4', pack ('L', $delivery_ip_dst)); &dump ($delivery_ip_data); } sub dump { my $len = length ($_[0]); if ($len > 0) { my @octet = split //, $_[0]; my $i; for ($i = 1; $i <= $len; ++$i) { printf " %02x", unpack ('C', $octet[$i-1]); print "\n" unless $i % 16; } print "\n" if $i % 16; } else { print " [NO DATA]\n"; } } # Format of an IP packet, RFC 791. # sub ip_unpack { my $packet = shift; if (length ($packet) < 20) { print STDERR "ip packet too short: ", length ($packet), " bytes\n"; exit 1; } my ( $version, $tos, $length, $id, $flags, $ttl, $proto, $chksum, $src, $dst ) = unpack ('CCnnnCCnNN', $packet); my $ihl = $version & 017; $version >>= 4; if ($version != 4) { print STDERR "ip version mismatch, expected 4, got $version\n"; exit 1; } my $offset = $flags & 017777; $flags >>=13; my $options = substr ($packet, 20, $ihl * 4 - 20); my $data = substr ($packet, $ihl * 4); return ( $version, $ihl, $tos, $length, $id, $flags, $offset, $ttl, $proto, $chksum, $src, $dst, $options, $data ); }

5 5

RADIUS Server
by Chin Yuan 08 Jul '98

08 Jul '98

Can someone gave me a hint that who is the most popular RADIUS server in the current Internet community? Any information would be appreciated. Please response to my private email address, Thanks. -------------------- Chin Yuan Shasta Networks cyuan(a)shastanets.com

1 0

Re: Cisco scripts??
by Arvind Ramarajan 07 Jul '98

07 Jul '98

Take a look at "rtrmon", perl based scripts to gather information from routers. Last i heard about it, it could also do router config uploads/downloads. Check out: http://www.vix.com/rtrmon Hope it helps.. Arvind. At 11:43 AM 7/7/98 -0700, you wrote: >You would need to have a MIB for them. The best way to find >out is to use a tool like Scotty to walk the MIB. I don't >think that cisco stores the ASCII config as a MIB entry. > >What you probably want is to grab the config via a rcp from >the box. I would recommend looking at the looking glass tools >from DIGEX (nitrous.digex.net) Should give you some ideas on code > >hope this helps.. > >jmbrown(a)ihighway.net > >At 11:47 AM 7/7/98 -0500, you wrote: >> Does anyone know where I can get ahold of some sort of script that querys >>the routers via SNMP that shows the running configs on them? I want to >>have a web based interface that ONLY shows the running configs. I am aware >>of the built in http server that is on the routers, but I would rather do >>it via SNMP . Any information would be appreciated. >> >>Thanks, >> >>Tim Kempka >> >> >>____________________________________________________ >>Tim Kempka E-Mail: tim(a)flash.net >>Network Engineer Pager: page_tim(a)flash.net >>FlashNet Communications Phone: 817-589-2390 ext. 378 >> > > Name: Arvind Ramarajan Org : AT&T Labs Loc : 307 MiddleTown Lincroft Rd Lincroft NJ 07738 Rm#: 1M-339 E-mail:arvi7@qsun.ho.att.com Phone: (732)-576-5542

1 0

Re: Tool for automatically educating smurf amplifiers ...
by Richard Thomas 07 Jul '98

07 Jul '98

-----Original Message----- From: Doug McLaren <dougmc(a)feeding.frenzy.com> To: nanog(a)merit.edu <nanog(a)merit.edu> Date: Monday, July 06, 1998 3:06 PM Subject: Tool for automatically educating smurf amplifiers ... >Lately one of our machines has been the target of several smurf >attacks (no idea why, probably some user kicked off an IRCer from >their channel or something equally silly) and so I set out to email >each of the sites used as smurf amplifiers ... > >I couldn't find any sort of tool to do this for me, so I wrote one. > >It's certainly still needs some work, but I think it'll be useful in >it's current condition to anybody else who's tried to do this. > >If we can notify the smurf amplifiers that they're being abused and >let them know what they need to do to fix it, maybe we can make smurf >attacks a thing of the past (or at least less effective, as the >smurfers will have to look harder to find good amplifiers.) > >In any event, you can get my program at : > > http://www.frenzy.com/~dougmc/smurf-complain.pl > >There's lots of room for improvements, so if you have some changes, by >all means send them to me. > >It uses `ipw' to get contact information. If you don't have `ipw', >get it from : > > http://www.e-scrub.com/ipw > >Also, while you may wish to use `tcpdump' or look at your router's >logs to see where the ICMP echo reply packets were coming from, I was >using icmpinfo, which you can get from : > > http://hplyot.obspm.fr/~dl/icmpinfo.html > >So far, after running the program once and sending out about 50 >emails, I've gotten about 17 bounces and about 15 emails saying >they'll fix or have fixed their routers, and two or three emails >asking for details or a more clear explanation ... fairly promising. Not to toot my own horn but you might wanna try using a little proggy I wrote called SmurfLog, available at http://www.sy.net/security. It only records echo replies from unique /24's, preventing the few gig logfiles that you can get from icmpinfo.

1 0

Tool for automatically educating smurf amplifiers ...
by Doug McLaren 06 Jul '98

06 Jul '98

Lately one of our machines has been the target of several smurf attacks (no idea why, probably some user kicked off an IRCer from their channel or something equally silly) and so I set out to email each of the sites used as smurf amplifiers ... I couldn't find any sort of tool to do this for me, so I wrote one. It's certainly still needs some work, but I think it'll be useful in it's current condition to anybody else who's tried to do this. If we can notify the smurf amplifiers that they're being abused and let them know what they need to do to fix it, maybe we can make smurf attacks a thing of the past (or at least less effective, as the smurfers will have to look harder to find good amplifiers.) In any event, you can get my program at : http://www.frenzy.com/~dougmc/smurf-complain.pl There's lots of room for improvements, so if you have some changes, by all means send them to me. It uses `ipw' to get contact information. If you don't have `ipw', get it from : http://www.e-scrub.com/ipw Also, while you may wish to use `tcpdump' or look at your router's logs to see where the ICMP echo reply packets were coming from, I was using icmpinfo, which you can get from : http://hplyot.obspm.fr/~dl/icmpinfo.html So far, after running the program once and sending out about 50 emails, I've gotten about 17 bounces and about 15 emails saying they'll fix or have fixed their routers, and two or three emails asking for details or a more clear explanation ... fairly promising. -- Doug McLaren, dougmc(a)frenzy.com Unsolicited email of a commercial or advertising nature is not welcomed.

1 0

Re: MTU problems with GRE tunnels (fwd)
by brandon＠rd.bbc.co.uk 06 Jul '98

06 Jul '98

> # it seems that there is a small but significant > # amount of application traffic out there that has problems when traversing a > # GRE tunnel with MTU < 1500. We've seen two problems: > # > # - 1500 byte packets with DF set. This is either application traffic, or MTU > # path discovery is broken, because the same packets get sent repeatedly We see the same effect, if anyone wants a site to test against try one of ours - news.bbc.co.uk (193.130.149.130) This is a set of NT servers behind an RND Director, and we've found you never get past MTU discovery (though oddly if you go straight to an individual machine address, e.g. 193.130.149.131 it does work. The only client verified is IE4 on Win95. We were working on the NT stack as the probable cause (MTU black hole discovery defaults to off) Brandon

1 0

Thanks to all re: cflowd and IOS stuff
by John M. Brown 05 Jul '98

05 Jul '98

Looks like I got it to finally work! :) Just for the record. 11.1(19) CC1 Enterprise is what is needed if you want ISL, BGP, IP, and working flow-export stuff. lets hope that cisco will get this into a IP only release someday. I really hate wasting memory on SNA kaka I don't need. jmbrown

1 0

Follow-on ?? to Cisco IOS ver and clfowd
by John M. Brown 05 Jul '98

05 Jul '98

Ok, I am blue in the face now... Is there a cisco release naming / feature expert in the house ??:) What version of IOS properly supports IP, BGP, NAT, ISL and NetFlow where ALL of this stuff works. I tried 11.1(18)CC it has the Netflow stuff but ISL is broken arggggg jmbrown(a)ihighway.net

3 2