July 1999 - NANOG - lists.nanog.org

Re: Toronto bell canada central office fire
by Sean Donelan 19 Jul '99

19 Jul '99

nathan(a)robotics.NET (Nathan Stratton) writes: >As a volunteer fire fighter I will agree with you on that one. I often >wonder why more people don't sue for damage, I guess there is >little they can do. Some people in fire fighters actually have fun >cutting holes where they don't need cut or flooding places with water >they know don't need it. Our department is very good, but many of them are >not concerned with the property at all. One of the lessons from the Los Angeles CO fire was the danger to firefighters with fires in telecommunication facilities. Fortunately such fires are rare. But that rarity means few departments have any experience with such fires. Hinsdale was a wake up call for the telecommunications industry, L.A. was a wake up call for the fire service. Five years later, NFPA, AT&T and Telcordia (bellcore) are still debating what should be done. For the most part, the experience has not yet reached down to either the local fire departments or the people building data and telecommunication facilities. http://www.nfpa.org/Research/summaries/los_angeles.html -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

6 5

Toronto bell canada central office fire
by Sean Donelan 18 Jul '99

18 Jul '99

I don't know if things are worse when I stay in the country, or when I leave. And, no, my airplane had not entered Canadian airspace at the time of the Bell Canada fire. I believe I was still over Iceland when it started. Here's how the Toronto newspapers covered the story: http://www.globalandmail.ca/gam/National/19990717/UMAINN1.html http://www.thestar.com/editorial/news/990717NEW04_CI-MAIN17.html And here is what a the phone company had to say: http://forums.sympatico.ca/WebX/WebX.cgi?13@190.0iYha9hEaX5^0@.ef9843d http://www.bell.ca/en/corp/aboutbell/newsroom/archives/nr99/nr99e0716.asp While Bell Canada is doing its best at PR damage control, it does point out the importance of not assuming the telephone company is infallible. Even if you believe the phone company claim of 99.999% reliability, stuff will still happen. This is not the first, nor I suspect the last, time a single central office failure had an out-of-proportion impact on the national network. All phone companies seem especially surprised by the affect of office failures on data circuits. They just don't seem to put as high a priority on data as they do on voice. Although Bell (and indeed any carrier) hates giving out information about their network, and how your circuits are actually routed. You need that information to make sure Bell (or some other carrier) doesn't route all your circuits through the same facility. Just because you paid for diversity, doesn't mean you got it or six months later it is still diverse. Even virtual circuits on ATM or Frame-relay pass through a physical switch somewhere. Sometimes I think the carriers don't want to give you the information, because they don't know themselves. After one carrier had a 2-day outage, including several delays due to mistakes in the carrier's records, one of my government customers made the carrier physically walk them through the entire physical circuit route, including every patch board and mux, to prove the carrier now knew where the circuit went. It was interesting :-) But the goal isn't just to point out what was wrong. But also to add any lessons to the knowledge base so we can avoid the same mistakes. The major C.O. fires in New York city, Chicago(Hinsdale), Los Angeles all have reports looking at the causes and effects. And I'm sure the Toronto fire will also have a report. In most C.O. fires, even Hinsdale, the fire itself does very little damage. It is all the byproducts of a fire and fire-fighting which cause the disruption and damage. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

3 2

Corrected urls (Re: Toronto bell canada central office fire)
by Sean Donelan 17 Jul '99

17 Jul '99

>http://www.globeandmail.ca/gam/National/19990717/UMAINN1.html >http://www.thestar.ca/editorial/news/990717NEW01_CI-MAIN17.html -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

1 0

TCP tools.
by Forrest W. Christian 17 Jul '99

17 Jul '99

I really can't think of a better place to ask this, so If anyone has any ideas of a place better suited please let me know (no flames please). I'm looking for a (free or quite inexpensive) tool to help diagnose some bizzare TCP problems we're having with some TCP flows stalling exactly 32k into the transfer. This isn't the same file and it's almost exacly 32k each time... but I digress. Is there a tool out there that can sniff a TCP flow and do some basic analysis such as missed acks, duplicate packets, Window Sizes, etc. etc. Mainly, I'm looking at something which can compare what it sees with some semblance of a normal flow and pick out "errors". Other tools to track down stalls would be helpful also. No, I'm not looking for ping tools or flood tools, etc., but instead something which can help me determine why this particular TCP implementation is stalling consistently in serveral different environments so I can go yell at the vendor. I realize that most if not all packet sniffers will provide the information neccessary to do this analysis by hand, but picking apart multiple flows bit by bit can be tedious and error prone, and not something I particularly want to do if there is an easier way. Thanks for all pointers. - Forrest W. Christian (forrestc(a)imach.com) KD7EHZ ---------------------------------------------------------------------- iMach, Ltd., P.O. Box 5749, Helena, MT 59604 http://www.imach.com Solutions for your high-tech problems. (406)-442-6648 ----------------------------------------------------------------------

2 1

clarification of my last message about 'MRTG.author'
by k claffy 17 Jul '99

17 Jul '99

dave rand (above.net) was also an author of the original MRTG. didn't mean to omit credit to any knights... [sorry dave] k

1 0

Canada Fiber Cut?
by Majdi Abbas 16 Jul '99

16 Jul '99

So far I've got a downed PVC to Toronto, and a total inability to call many numbers in the Toronto area (I get a reorder)...this appears to be a widepsread outage...has anyone heard anything? Started at 10:23 EDT. --msa

6 6

The Cidr Report
by Tony Bates 16 Jul '99

16 Jul '99

This is an auto-generated mail on Fri Jul 16 12:00:00 PDT 1999 It is not checked before it leaves my workstation. However, hopefully you will find this report interesting and will take the time to look through this to see if you can improve the amount of aggregation you perform. The report is split into sections: 0) General Status List the route table history for the last week, list any possibly bogus routes seen and give some status on ASes. 1) Gains by aggregating at the origin AS level This lists the "Top 30" players who if they decided to aggregate their announced classful prefixes at the origin AS level could make a significant difference in the reduction of the current size of the Internet routing table. This calculation does not take into account the inclusion of holes when forming an aggregate so it is possible even larger reduction should be possible. 2) Weekly Delta A summary of the last weeks changes in terms of withdrawn and added routes. Please note that this is only a snapshot but does give some indication of ASes participating in CIDR. Clearly, it is generally a good thing to see a large amont of withdrawls. 3) Interesting aggregates Interesting here means not an aggregate made as a set of classful routes. Thanks to xara.net for giving me access to their routing tables once a day. Please send any comments about this report directly to me. Check http://www.employees.org/~tbates/cidr-report.html for a daily update of this report. ------------------------------------------------------------------------------ CIDR REPORT for 16Jul99 0) General Status Table History ------------- Date Prefixes 090799 61684 100799 60957 110799 60942 120799 61016 130799 61055 140799 61223 150799 61253 160799 61126 Check http://www.employees.org/~tbates/cidr.plot.html for a plot of the table history. Possible Bogus Routes --------------------- AS Summary ---------- Number of ASes in routing system: 5294 Number of ASes announcing only one prefix: 2768 (1530 cidr, 1238 classful) Largest number of cidr routes: 437 announced by AS701 Largest number of classful routes: 838 announced by AS701 1) Gains by aggregating at the origin AS level --- 16Jul99 --- ASnum NetsNow NetsCIDR NetGain % Gain Description AS271 348 145 203 58.3% BCnet Backbone AS4293 270 161 109 40.4% IMCI AS2609 123 25 98 79.7% EUnet-TN AS577 278 186 92 33.1% Bell Canada Backbone AS174 579 490 89 15.4% Performance Systems International AS3749 146 58 88 60.3% TECNET AS7046 323 241 82 25.4% UUNET-CUSTOMER AS4200 184 103 81 44.0% AGIS (Apex Global Information Ser AS7545 139 64 75 54.0% TPG Internet Pty Ltd AS1221 495 422 73 14.7% TELSTRA-AS AS10928 118 51 67 56.8% UNKNOWN AS9269 79 13 66 83.5% Hong Kong CTI AS7496 99 33 66 66.7% Power Up AS7657 215 151 64 29.8% The Internet Group Limited AS6944 63 6 57 90.5% RapidSite, Inc. AS4755 132 76 56 42.4% Videsh Sanchar Nigam Ltd. India AS4740 312 259 53 17.0% Ozemail Pty Ltd (ASN-OZEMAIL) AS719 459 408 51 11.1% LANLINK autonomous system AS2493 163 112 51 31.3% iSTAR Internet, Inc. AS684 88 43 45 51.1% Manitoba Regional Network Backbon AS803 78 34 44 56.4% SaskNet Backbone AS549 220 176 44 20.0% ONet Backbone AS3737 110 69 41 37.3% PenTeleData Inc. (ASN-PTD) AS1 495 459 36 7.3% BBNPLANET AS1785 246 213 33 13.4% NYSERNet Backbone AS6762 33 2 31 93.9% Telecom Italia international high AS4307 32 1 31 96.9% SVINET-1 AS11515 32 1 31 96.9% UNKNOWN AS8023 43 13 30 69.8% CASTLES AS7745 38 8 30 78.9% BIGSKY-AS For the rest of the previous weeks gain information please see http://www.employees.org:80/~tbates/cidr-report.html 2) Weekly Delta Please see http://www.employees.org:80/~tbates/cidr-report.html for this part of the report 3) Interesting aggregates Please see http://www.employees.org:80/~tbates/cidr-report.html for this part of the report

1 0

Re: TCP tools.
by Paul G. Donner 16 Jul '99

16 Jul '99

tcpdump/unix At 03:36 AM 7/16/1999 -0600, Forrest W. Christian wrote: > >I really can't think of a better place to ask this, so If anyone has any >ideas of a place better suited please let me know (no flames please). > >I'm looking for a (free or quite inexpensive) tool to help diagnose some >bizzare TCP problems we're having with some TCP flows stalling exactly 32k >into the transfer. This isn't the same file and it's almost exacly 32k >each time... but I digress. > >Is there a tool out there that can sniff a TCP flow and do some basic >analysis such as missed acks, duplicate packets, Window Sizes, etc. etc. >Mainly, I'm looking at something which can compare what it sees with some >semblance of a normal flow and pick out "errors". Other tools to track >down stalls would be helpful also. No, I'm not looking for ping tools or >flood tools, etc., but instead something which can help me determine why >this particular TCP implementation is stalling consistently in serveral >different environments so I can go yell at the vendor. > >I realize that most if not all packet sniffers will provide the >information neccessary to do this analysis by hand, but picking apart >multiple flows bit by bit can be tedious and error prone, and not >something I particularly want to do if there is an easier way. > >Thanks for all pointers. > >- Forrest W. Christian (forrestc(a)imach.com) KD7EHZ >---------------------------------------------------------------------- >iMach, Ltd., P.O. Box 5749, Helena, MT 59604 http://www.imach.com >Solutions for your high-tech problems. (406)-442-6648 >---------------------------------------------------------------------- >

1 0

MRTG.author karma++'s (tobi's yummy new ISP stat graphing tool)
by k claffy 16 Jul '99

16 Jul '99

folks new tool of potential operational ISP interest from Tobi (author of MRTG, visiting CAIDA on sabbatical this summer): http://www.caida.org/Tools/RRDtool/ if you use/like MRTG, check out RRDtool (Round Robin Database), which is basically the time-series graphing and data storage/management component of MRTG, broken out and `done right'. (--tobi) `done right' == much more flexible/configurable, (but still supports lots of auto-config stuff for folks w better things to do...) perhaps more importantly: orders of magnitude faster/scalable, stores data compactly so as not to expand over time so RRDtool provides a powerful building block for other frontends, couple of which you may have seen: http://www.caida.org/Tools/RRDtool/frontends/ & can also be used via simple wrapper scripts (e.g., sh/perl) also: RRD tool can now produce png images as well as gif (due to the increasing copyright sensitivity surrounding gif format. gif still default output format, but no more lzw compression. png better technical solution anyway, if legaljunk isn't compelling enough argument to switch. discussing this issue further on nanog@ would be a good way to get me killed for even mentioning it (for those w that goal)) portability: should run on most reasonable unices (and actually runs on NT.. ), but we're grateful for feedback/experiences to tobi(a)caida.org. licensing: http://www.caida.org/Tools/RRDtool/license.html (i always suspected karma was measureable in music. tho i reckon CDnow should send tobi some royalties): note Tobi wrote RRDtool mostly on his own time (True Knight), been refining it this summer while at CAIDA & releasing incremental pre-1.0 versions to dev list for a while but reckons it's finally nanog-strength his next windmill.caida is re-working MRTG itself, which makes now an optimal time for ISPs to provide input on what additions to MRTG would enhance its utility for them tobi(a)caida.org (so far, he's got requests/plans for: running as daemon as well as from cron; simpler interface for polling data of various datasources; template-based webpage generation both static and on demand; better config paradigm (perl-module-based) 95th percentile plotting ) tnx, k and tobi @caida.org

1 0

TCG Fiber Cut: San Diego <-> Orange & Anaheim
by Kit Knox 15 Jul '99

15 Jul '99

There is currently a TCG fiber cut affecting at least 45 DS3's between San Diego <-> Orange & Anaheim. We lost circuits at 9:30AM PDT. Anyone have any details? -- Kit Knox <kit(a)connectnet.com>

1 0