January 2002 - Test - lists.nanog.org

RE: router startup behavior
by Borchers, Mark 16 Jan '02

16 Jan '02

> > Just guessing - you're seeing these events between midnight > > and 5 am? > > Hm, couldn't reist this one: "which time zone"? > > Just hinting that even though it's that time interval in the US, local > time is different in other places around the world, so if this is > causing disturbance, others are probably being hit in their working > hours. Heh, the "perpetual global maintenance window syndrome", eh? A very useful concept... > Besides, I was under the impression that to activate a new outbound > roting policy on a Cisco, you could just modify / replace it, but that > you would still have to do > > router#clear ip bgp xxx soft out > > to activate it. This means that the policy for an existing peer can > be modified without having to remove the peering and reenable it > shortly thereafter (something which would cause needless route > flapping). Somewhat true. The new policy would not be applied to routes that were already in the table, but would be applied to any adds/withdrawals that occur once the policy changes are placed in the configuration. This fact has a synergistic effect when you're making changes that affect lots of sessions. So a policy change made a significant time before a clear or soft clear could in fact result in flaps. Also consider: there are a lot of routers using traditional ACL's in their policies (as opposed to things like prefix-lists which have more granular editing features) which would necessitate removing the ACL completely and rewriting it with updated lines. Due to the above, a potential for leaks exists unless the session is either shutdown or deleted while acl's are being modified. If the sequence of events in a configuration script is not well thought out, the result could be what Ratul has observed in his study. > > Regards, > > - Håvard >

1 0

RE: router startup behavior
by Borchers, Mark 16 Jan '02

16 Jan '02

> the most likely cause would be one of: <items deleted for brevity> > (c) script used to configure router(s) adds a 'network' > statement prior to trimming route-filters Yeah, (c) seems most likely to me. Ratul, a script like this or some variant could cause what you are seeing: config-router# no neighbor <a> config-router# no neighbor <b> config-router# no neighbor <c> (script to rewrite filters executes) config-router# neighbor <a> remote-as <x> config-router# neighbor <a> remote-as <y> config-router# neighbor <a> remote-as <z> (sessions start coming up) config-router# neighbor <a> route-map <A> out config-router# neighbor <b> route-map <B> out config-router# neighbor <c> route-map <C> out config-router# Ctrl-Z # clear ip bgp external soft out Just guessing - you're seeing these events between midnight and 5 am? > At 01:10 PM 14/01/2002 -0800, Ratul Mahajan wrote: > > > >to the best of my knowledge, here is what is happening. > > > >1. router starts rebooting > >2. there are routes in the routing table, some of which are not to > >be announce according to filters > >3. bgp sessions comes up; the filters have not yet taken effect > >4. start announcing routes > >5. filters come up > >6. the router realizes that it made a mistake and withdraws > the routes not > >meant to be announced. > > > >i should also point out that all such incident are not 1000 > router. most > >of them are 20-50, but i have seen non-trivial number of > ~100 prefixes, > >and a couple more than that. > > > > -- ratul > > > >On Mon, 14 Jan 2002, Ratul Mahajan wrote: > > > > > > > > > > > at university of washington, we are doing a measurement > study of bgp > > > misconfiguration > > > (http://www.cs.washington.edu/homes/ratul/bgp/index.html) > > > > > > one of the things we found is that there are a lot of > announcements of > > > more-specifics that come and go within a matter of 2-5 minutes. > > > > > > by talking to the operators involved in these incidents, > we found that > > > most of these are caused when the router is rebooted > (intentionally or > > > not). while some operators were aware of this side > effect, most were not, > > > and were taken by surprise that they just injected > anywhere from 1-1000 > > > routes into BGP only to withdraw them a couple of minutes later. > > > > > > i would like to understand this behavior better. is this behavior > > > vendor-specific (cisco?) or pervasive? is there a > configuration style that > > > causes or avoids this "spill-over"? > > > > > > my understanding is limited to this happens when the bgp > session comes up > > > too soon, before the filters have taken effect. could > someone familiar > > > with router internals shed some light on it? > > > > > > the problem is limited to route origination only, or also > propagation? > > > in other words, can a router propagate a route it should not while > > > starting up because export filters are not yet in place? > > > > > > never ever gotten my hands dirty into router > configuration; your input > > > would be invaluable. > > > > > > thanks, > > > -- ratul > > > > > > > > > >

2 1

The Cidr Report
by Tony Bates 16 Jan '02

16 Jan '02

This is an auto-generated mail on Fri May 25 23:00:00 PDT 2001 It is not checked before it leaves my workstation. However, hopefully you will find this report interesting and will take the time to look through this to see if you can improve the amount of aggregation you perform. The report is split into sections: 0) General Status List the route table history for the last week, list any possibly bogus routes seen and give some status on ASes. 1) Gains by aggregating at the origin AS level This lists the "Top 30" players who if they decided to aggregate their announced classful prefixes at the origin AS level could make a significant difference in the reduction of the current size of the Internet routing table. This calculation does not take into account the inclusion of holes when forming an aggregate so it is possible even larger reduction should be possible. 2) Weekly Delta A summary of the last weeks changes in terms of withdrawn and added routes. Please note that this is only a snapshot but does give some indication of ASes participating in CIDR. Clearly, it is generally a good thing to see a large amont of withdrawls. 3) Interesting aggregates Interesting here means not an aggregate made as a set of classful routes. Thanks to xara.net for giving me access to their routing tables once a day. Please send any comments about this report directly to me. Check http://www.employees.org/~tbates/cidr-report.html for a daily update of this report. ------------------------------------------------------------------------------ CIDR REPORT for 25May01 0) General Status Table History ------------- Date Prefixes 040501 104512 050501 104646 060501 104816 070501 106340 080501 105979 230501 101875 240501 101755 250501 101869 Check http://www.employees.org/~tbates/cidr.plot.html for a plot of the table history. Possible Bogus Routes --------------------- *** Bogus 80.9.0.0/18 from AS3215 *** Bogus 80.9.128.0/18 from AS3215 *** Bogus 80.64.0.0/20 from AS3246 *** Bogus 80.64.128.0/20 from AS20751 *** Bogus 80.64.224.0/20 from AS3259 *** Bogus 80.65.128.0/20 from AS9057 *** Bogus 80.65.224.0/20 from AS13193 *** Bogus 80.66.32.0/20 from AS702 *** Bogus 80.66.128.0/20 from AS1239 *** Bogus 80.67.0.0/20 from AS20750 *** Bogus 80.67.32.0/20 from AS20752 *** Bogus 218.2.0.0/15 from AS4134 *** Bogus 218.4.0.0/16 from AS4134 *** Bogus 218.6.128.0/17 from AS4134 AS Summary ---------- Number of ASes in routing system: 10851 Number of ASes announcing only one prefix: 6431 (3657 cidr, 2774 classful) Largest number of cidr routes: 975 announced by AS701 Largest number of classful routes: 1409 announced by AS701 1) Gains by aggregating at the origin AS level --- 25May01 --- ASnum NetsNow NetsCIDR NetGain % Gain Description AS1221 1391 1031 360 25.9% TELSTRA-AS AS13999 103 11 92 89.3% autonomous system number assigned AS11252 91 33 58 63.7% autonomous system number assigned AS226 150 93 57 38.0% Los Nettos AS16758 63 6 57 90.5% autonomous system number assigned AS2149 320 265 55 17.2% PSINET-2 AS1 592 539 53 9.0% GTE Internetworking AS11686 143 91 52 36.4% Education Networks of America AS10692 66 14 52 78.8% DLS Internet Services AS10692 AS17561 111 60 51 45.9% UNKNOWN AS13544 69 22 47 68.1% autonomous system number assigned AS11774 74 28 46 62.2% AS 11774 Object AS1267 67 25 42 62.7% Infostrada S.p.A. AS1237 94 52 42 44.7% KISTI AS1239 441 400 41 9.3% SprintLink Backbone AS12235 48 7 41 85.4% Cove Software Systems Inc AS2652 113 75 38 33.6% Government of Canada (GENet) back AS209 263 226 37 14.1% Qwest Communications AS19834 49 12 37 75.5% autonomous system number assigned AS18447 46 9 37 80.4% autonomous system number assigned AS13345 67 30 37 55.2% autonomous system number assigned AS14 51 15 36 70.6% Columbia University in the City o AS12302 63 28 35 55.6% MobiFon S.A. AS2007 153 119 34 22.2% Infonet Services Corporation AS11507 87 53 34 39.1% TouchAmerica Montana AS10620 52 18 34 65.4% autonomous system number assigned AS11311 42 9 33 78.6% autonomous system number assigned AS10479 40 7 33 82.5% ITESM-Rectoria Zona Norte AS1913 115 83 32 27.8% autonomous system number assigned AS11859 45 13 32 71.1% autonomous system number assigned For the rest of the previous weeks gain information please see http://www.employees.org:80/~tbates/cidr-report.html 2) Weekly Delta Please see http://www.employees.org:80/~tbates/cidr-report.html for this part of the report 3) Interesting aggregates Please see http://www.employees.org:80/~tbates/cidr-report.html for this part of the report

4 3

Re: router startup behavior
by Danny McPherson 16 Jan '02

16 Jan '02

> > The routers in question run a BGP redistribution process every > 60 seconds and it just happened to run (quite often) just as > filters were being rewritten, resulting in "the gunk" not being > tagged as "keep internal" and therefore being advertised > externally. A subsequent run of the BGP process realized a > policy was in place withdrew the routes. I should clarify here: A subsequent run of the BGP resdistribution process with the newly rewritten policy in place resulted in the routes being tagged with the appropriate "keep internal" community and "re- advertised". Upon receipt of the now tagged routes, routers that had previously announced the untagged routes withdrew them (per egress policy specification). -danny

1 0

Re: router startup behavior
by Danny McPherson 16 Jan '02

16 Jan '02

I've actually seen this behavior (more specifics being leaked and then promptly withdrawn), though the trigger wasn't quite so obvious. It turned out what was happening is that a bunch of gunk was being redistributed into BGP and tagged with "keep internal" communities to avoid advertisement to EBGP peers. Implicit policy was being applied at the domain egress such that if the "keep internal" community values weren't present the routes were by default advertised externally. In addition, BGP policies (i.e., prefix filters) that identified "keep internal" address space (and pretty much everything else, for that matter) were being rewritten every night on every BGP router in the network (regardless of change). The routers in question run a BGP redistribution process every 60 seconds and it just happened to run (quite often) just as filters were being rewritten, resulting in "the gunk" not being tagged as "keep internal" and therefore being advertised externally. A subsequent run of the BGP process realized a policy was in place withdrew the routes. The symptoms were somewhat random as the leaked routes varied from day to day, depending on which routers throughout the domain happened to have a BGP redistribution process collide with a policy rewrite -- but all in all, in a large network it seemed to occur on at least a couple of routers every night. My conclusions: o Redistributing into BGP can get ugly and should be avoided o Blindly rewriting policies (or any configuration elements) that haven't changed is bad o Implicit route advertisement policies are bad -danny

1 0

ipsrvtrace
by Steve Bellovin 15 Jan '02

15 Jan '02

Quite a number of people have been asking me about ipsrvtrace. It's a tool I wrote a few months ago, to investigate a service-specific black hole. (Packets to one net to port 80 didn't get through; packets to other ports or other nets did, and the network manager insisted that there were no firewalls or transparent proxies in the way.) Anyway -- the code is not (yet) released. It runs on NetBSD, FreeBSD, and OpenBSD; a variant runs on Linux, but I haven't folded the Linux changes back into the base version yet. If you need that sort of functionality today, use tcptraceroute, which you can find at http://michael.toren.net/tcptraceroute --Steve Bellovin, http://www.research.att.com/~smb Full text of "Firewalls" book now at http://www.wilyhacker.com

2 1

Re: huh
by Steven M. Bellovin 15 Jan '02

15 Jan '02

In message <200201151959.g0FJxFv03307(a)nms.lcs.mit.edu>, "David G. Andersen" wri tes: > >Ian A Finlay just mooed: >> >> I wonder what's up? >> >> bash-2.04$ traceroute windowsupdate.microsoft.com >> traceroute to windowsupdate.microsoft.com (207.68.131.27), 30 hops max, 40 >> [...] >> 8 POS6-0.GW4.DCA8.ALTER.NET (152.63.35.197) 14.747 ms 13.515 ms >> 12.878 ms >> 9 65.195.34.226 (65.195.34.226) 653.529 ms 709.526 ms 702.782 ms >> 10 * * * >> 11 * * * >> 12 * * * > > Um, it's firewalled? Most of microsoft isn't traceroutable or >pingable. Yup: b129$ ipsrvtrace -p 80 windowsupdate.microsoft.com 1 oden.research.att.com 135.207.31.1 0.474 0.338 0.304 2 janus.research.att.com 135.207.1.2 1.360 1.951 2.577 3 argus.research.att.com 192.20.225.225 2.973 3.505 4.063 4 12.119.155.157 12.119.155.157 3.543 4.035 4.603 5 gbr5-p52.n54ny.ip.att.net 12.123.192.10 4.897 5.433 6.188 6 tbr2-p013301.n54ny.ip.att.net 12.122.11.25 6.190 7.795 8.417 7 ggr1-p320.n54ny.ip.att.net 12.122.12.22 4.518 5.232 6.299 8 POS5-1.BR1.NYC9.ALTER.NET 204.255.169.93 6.384 7.661 8.211 9 0.so-6-0-0.XL2.NYC9.ALTER.NET 152.63.18.222 5.242 6.388 6.952 10 0.so-0-0-0.XR2.NYC9.ALTER.NET 152.63.9.89 5.377 5.897 6.586 11 0.so-3-0-0.TR2.NYC9.ALTER.NET 152.63.22.94 4.873 5.388 5.949 12 125.at-7-0-0.TL2.DCA8.ALTER.NET 146.188.141.197 11.861 12.356 12.932 13 0.so-4-3-0.XL2.DCA8.ALTER.NET 152.63.144.30 11.740 13.227 13.792 14 POS7-0.GW4.DCA8.ALTER.NET 152.63.35.201 10.737 11.228 11.778 15 65.195.34.226 65.195.34.226 155.934 156.415 156.973 16 iusbsecurc1202-ge-6-0.msft.net 207.68.128.66 13.109 13.598 14.142 17 - - * * * 18 207.68.131.27 207.68.131.27 13.988 14.373 * b130$ traceroute windowsupdate.microsoft.com traceroute to windowsupdate.microsoft.com (207.68.131.27), 30 hops max, 40 byte packets 1 oden (135.207.31.1) 0.424 ms 0.270 ms 0.245 ms 2 janus (135.207.1.2) 1.156 ms 2.943 ms 1.346 ms 3 argus (192.20.225.225) 2.345 ms 1.875 ms 1.749 ms 4 12.119.155.157 (12.119.155.157) 3.412 ms 3.288 ms 3.567 ms 5 gbr5-p52.n54ny.ip.att.net (12.123.192.10) 4.277 ms 4.860 ms 4.038 ms 6 tbr2-p013301.n54ny.ip.att.net (12.122.11.25) 5.238 ms 5.344 ms 4.821 ms 7 ggr1-p320.n54ny.ip.att.net (12.122.12.22) 4.360 ms 5.456 ms 4.098 ms 8 POS5-1.BR1.NYC9.ALTER.NET (204.255.169.93) 4.823 ms 4.466 ms 4.360 ms 9 0.so-6-0-0.XL2.NYC9.ALTER.NET (152.63.18.222) 4.753 ms 5.054 ms 6.305 ms 10 0.so-0-0-0.XR2.NYC9.ALTER.NET (152.63.9.89) 5.017 ms 4.816 ms 4.572 ms 11 0.so-3-0-0.TR2.NYC9.ALTER.NET (152.63.22.94) 6.842 ms 9.812 ms 4.747 ms 12 125.at-7-0-0.TL2.DCA8.ALTER.NET (146.188.141.197) 11.163 ms 11.162 ms 11.214 ms 13 0.so-4-3-0.XL2.DCA8.ALTER.NET (152.63.144.30) 11.414 ms 11.665 ms 11.232 ms 14 POS7-0.GW4.DCA8.ALTER.NET (152.63.35.201) 10.842 ms 11.577 ms 10.759 ms 15 65.195.34.226 (65.195.34.226) 170.249 ms 123.845 ms 135.542 ms 16 * * * 17 * * * 18 * * * 19 * *^C --Steve Bellovin, http://www.research.att.com/~smb Full text of "Firewalls" book now at http://www.wilyhacker.com

4 5

huh
by Ian A Finlay 15 Jan '02

15 Jan '02

I wonder what's up? -Ian bash-2.04$ traceroute windowsupdate.microsoft.com traceroute to windowsupdate.microsoft.com (207.68.131.27), 30 hops max, 40 byte packets 1 CYH-A100-VL10.GW.CMU.NET (128.2.10.254) 0.871 ms 0.454 ms 0.404 ms 2 HYPER-VL255.GW.CMU.NET (128.2.255.7) 0.484 ms 0.460 ms 0.449 ms 3 bar-cmu-g4-0-0-0.psc.net (192.88.115.181) 0.335 ms 0.312 ms 0.308 ms 4 killi-foo.psc.net (192.88.115.4) 2.569 ms 6.004 ms 5.461 ms 5 POS2-3.GW3.PIT1.ALTER.NET (65.194.72.113) 8.867 ms 6.489 ms 4.980 ms 6 518.at-2-0-0.XR1.DCA1.ALTER.NET (152.63.36.250) 13.141 ms 14.134 ms 12.994 ms 7 195.at-7-3-0.XL1.DCA8.ALTER.NET (146.188.163.6) 34.551 ms 13.301 ms 11.845 ms 8 POS6-0.GW4.DCA8.ALTER.NET (152.63.35.197) 14.747 ms 13.515 ms 12.878 ms 9 65.195.34.226 (65.195.34.226) 653.529 ms 709.526 ms 702.782 ms 10 * * * 11 * * * 12 * * * 13 * * * 14 * * * 15 * * * 16 * * * 17 * * * 18 * * * 19 * * * 20 * * * 21 * * * 22 * * * 23 * * * 24 * * * 25 * * * 26 * * *

3 3

RE: huh
by Hunter, Jonathan 15 Jan '02

15 Jan '02

> I wonder what's up? > They're having some fundamental DNS and/or network problems, it would seem (and have been having them for quite a while now). http://www.theregister.co.uk/content/4/23685.html Cheers, Jonathan

1 0

RE: Request for info: DSL test gear
by Bender, Andrew 15 Jan '02

15 Jan '02

Since you know of the spirent stuff, have a look at the adtech 4000 (http://adtech.spirentcom.com/) this system is more capable at the ATM layer, if that matters for you. Also, try ixia (http://www.ixiacom.com/) they are the winner on price / performance with IP datagrams. Regards, Andrew Bender taqua.com > -----Original Message----- > From: Tim Irwin [mailto:tim@eng.bellsouth.net] > Sent: Tuesday, January 15, 2002 9:53 AM > To: NANOG > Subject: Request for info: DSL test gear > > DSL Test System requirements: > > -Interfaces: ATM, Ethernet, DS3, OC3, OC12, 10/100 > -Generate approximately 32,000 PPP sessions > -Bi-directional traffic > -PPP throughput tests > -L2TP emulation (both LAC and LNS) > -Ability to perform L2TP conformance test as per RFC 2661 > -Simulate user traffic and session turn up rates > -Full negotiation of the IP and PPP parameters > -Ability to vary the rate of the PPP session startup > -Ability to see the sessions and associated users or IP addresses > -Take measurements while generating the L2TP tunnels and PPP sessions > -Generate PPPoE sessions through the L2TP tunnels > -Generate PPPoA sessions using AAL5 and RFC1483 > -Generate PPPoEoA sessions using AAL5 and RFC1483

2 1