- Test - lists.nanog.org

NRIC: 4th quarter 1998 report
by David Lesher 14 Apr '99

14 Apr '99

Unnamed Administration sources reported that Sean Donelan said: > - 4Q98 had more outages than any prior fourth quarter ......... Clearly all these problems have been caused by allowing CLEC co-lo access; likely by allowing Those People to use the restrooms. [WSJ had an article recently on the ILEC's petty behaviour re: co-lo space...] -- A host is a host from coast to coast.................wb8foz(a)nrk.com & no one will talk to a host that's close........[v].(301) 56-LINUX Unless the host (that isn't close).........................pob 1433 is busy, hung or dead....................................20915-1433

1 0

NRIC: 4th quarter 1998 report
by Sean Donelan 14 Apr '99

14 Apr '99

The Network Reliability and Interoperability Council met today at the FCC. I thought I would review a few highlights from the 4th Quarter 1998 Network Reliability Steering Committee report. http://www.atis.org/ No it wasn't just your imagination, the last three months of 1998 were worse than normal. - 4Q98 had more outages than any prior fourth quarter - The aggregate outage index was significantly higher, and second highest on record - The number of switch outages was significantly higher - More CO power outages were reported than any other fourth quater - Procedural Error outages were in the "red" region of the NRSC control chart - There is a significant increasing trened in both the number of facility outages, and the number of CO power outages - There is an increasing trend in DACS outages, close to being statistically significant - There is an significant increasing trend in both the frequency and impact of Procedural Errors as the root cause of outages Good news, the number of SS7 CCS outages continues to decline. The mean time between outages was 2.0 days, and the median outage duration was 3.3 hours. This is only based on those outages reported to the FCC. As most of you know I have a continuing problem with the definition of a reportable outage, particular as they affect digital/data services. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

1 0

S.F. Major Outage.
by Gregory A. Carter 14 Apr '99

14 Apr '99

We get experienced about a 30 minute cut on our DS1 and was informed that there was a major outage in the San Francisco area. Anyone have any more info on this? Greg +(Omni@Dynmc.Net)------------------------------------------------------+ | Dynamic Networking Solutions InterX Technologies | | Senior Network Administrator bits/keyID 1024/7DF9C285 | | omni(a)interx.net omni(a)itstudio.net omni(a)undernet.org omni(a)webpop3.com | +--------[ DC 50 57 59 C3 76 46 E8 EB 75 A8 94 FE 96 9E D3 ]----------+

1 0

Major fiber cut in Tyson's Corner area
by Paul Ferguson 14 Apr '99

14 Apr '99

For what it's worth: I just heard on the local news that 3 fiber optic cables, as well as one large copper bundle was cut in the Tyson's Corner area (Northern Virginia). No idea what impact it may have on net traffic, although one fiber strand was reported as belonging to MCI. For those who may not know, MAE-East is in the Tysons Corner vicinity. - paul

2 2

Re: Major fiber cut in Tyson's Corner area
by David Lesher 14 Apr '99

14 Apr '99

TV Gnews is reporting that the villain is NOT (!!) a backhoe. Rather, it's a boring machine, the kind that rams a 10' length of rod into the ground, then backs up to accept another section. (MFS contractors used same here last summer. The nifty part is how the steer same 140' out and 20' down....) The strike is ~12' deep; Hell Atlantic & friends have 2 backhoes and considerable shoring on site at present. Servint.com was named as being knocked down by this. I am worried. Does this mean that OTHER kinds of machinery are now subject to Creutzfiber-Jakhoe disease? Is it being spread through Diesel fuel? -- A host is a host from coast to coast.................wb8foz(a)nrk.com & no one will talk to a host that's close........[v].(301) 56-LINUX Unless the host (that isn't close).........................pob 1433 is busy, hung or dead....................................20915-1433

1 0

Hrrrmmmmm
by Richard Irving 14 Apr '99

14 Apr '99

> (OTOH, perhaps one should not use a >provider that is so big the provider's engineering staff cannot even >manage their own network.) :\

1 0

Re: FYI ONLY >> Non-Operational (at least directly)
by I Am Not A Lawyer Simpson 14 Apr '99

14 Apr '99

> From: I Am Not An Isp <patrick(a)ianai.net> > Naturally - they make a lot more money lying to you and then paying the > penalty when they get caught. Like 100% SLAs - a provider who offers a > 100% SLA *knows* they can't actually meet their guarantee, so they budget > in $X for payment of the penalties. And sometimes they even make the > process ridiculous to get your refund. > If they get caught, the penalty can be pretty high. When you sell a service that you don't actually provide, you can be changed with fraud and treble damages. Under the uniform commercial code, selling a product that works half as well as specified, you still have to refund the entire price, no half discount. As I understand it, after the Milan Michigan cut a few years ago, MCI had to refund (to a major auto company) the _entire_ amount of the contract from its beginning, as they had never provided the physically redundant circuits specified in the contract. No matter that they provided _some_ service that worked up until the cut -- they didn't provide the service specified in the contract. And to retain the rest of the contract, MCI had to put in a redundant fiber -- right down the center of Ann Arbor, at huge expense, including millions resurfacing some roads that had just been resurfaced a couple of years before, and (I've heard) paying 40 dollars per foot per year for the easements. NB: Too bad Merit doesn't have as good lawyers.... They were hit by the same cut, and I'm not sure they even got a refund. I guess the moral is, the contract is only as good as the amount of money that you are willing to spend on lawyers. WSimpson(a)UMich.edu Key fingerprint = 17 40 5E 67 15 6F 31 26 DD 0D B9 9B 6A 15 2C 32

1 0

Re: FYI ONLY >> Non-Operational (at least directly)
by Sean Donelan 13 Apr '99

13 Apr '99

>It does not ease the situation WHEN your bits transit one provider's >backbone (whom you chose to assure redundancy) does ultimately receive >transit (at times such transit is not for long distances..) from a yet >another whose infrastructure is now effected. I am not suggesting however, >that is the case here.. AT&T is investigating. :) Yeah, that's one reason why I stopped including the name of the carrier in most fiber cuts. It was getting too hard to tell who was who with all the swapping, leasing, sub-leasing, etc. Cable&Wireless is paying Level 3 $670 million for pieces of its fiber. GTE/BBN paid Qwest a ton for pieces of its fiber. AT&T, Sprint, MCI/Worldcom fiber overlaps so much its hard to tell them apart in some places. Qwest and IXC keep digging up other people's fiber, which would indicate their new fiber is being placed very close to other fiber. The non-facilities based carriers sometimes do better if they can guess which carriers have non-overlapping facilities, and buy from multiple physical facilities. But its not uncommon to see a dozen large ISPs have problems when someone trashes an interesting SONET ring somewhere, like Herndon/Mclean this morning. All the good right-of-ways are already taken. Heck, last year I was on some beautiful Caribbean islands with populations less than 100 people. We found both a backhoe and a cut telephone cable on them. In ten years (I ordered my first circuit in 1989), I've never gotten a good, coherent answer why any of my diversely, redundent circuits were both out-of-service at the same time due to the same event. Lots of credits, lots of apologies, but never a straight answer why something that should never happen, modulo the end of the world, happened. Good luck on AT&T's investigation.... -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

3 2

Re: Major fiber cut in Tyson's Corner area
by Sean Donelan 13 Apr '99

13 Apr '99

ferguson(a)cisco.COM (Paul Ferguson) writes: >I just heard on the local news that 3 fiber optic cables, as well as >one large copper bundle was cut in the Tyson's Corner area (Northern >Virginia). No idea what impact it may have on net traffic, although one >fiber strand was reported as belonging to MCI. > >For those who may not know, MAE-East is in the Tysons Corner vicinity. Things that make you go Hmmm! Yesterday I received a very strange phone call from a person claiming to be with MCI/Worldcom wanting to talk to me about verifying Y2K readiness of our equipment at MAE-East. He wanted to verify the address, and wanted to send a tech over to check serial numbers of equipment. Our MCI/Worldcom sales rep says he knows nothing about MCI/Worldcom doing this, nor why MCI/Worldcom wouldn't already know the address of their own building. I wonder if he just was fishing, to find out where to send the backhoe :-) -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation

2 1

Re: Major fiber cut in Tyson's Corner area
by Avi Freedman 13 Apr '99

13 Apr '99

Paul Ferguson (ferguson(a)cisco.com) wrote: : For what it's worth: : I just heard on the local news that 3 fiber optic cables, as well as : one large copper bundle was cut in the Tyson's Corner area (Northern : Virginia). No idea what impact it may have on net traffic, although one : fiber strand was reported as belonging to MCI. : For those who may not know, MAE-East is in the Tysons Corner vicinity. : - paul In one area, in particular (which affects CAIS, Servint, and one of netaxs's POPs) - >From MFS - "The YF system between 81b and q37 is still down; their techs are still working on it." The last update was at 4pm, which said that due to fact that Bell Atlantic owns the duct in which the accident occurred, this could be an extended outage. The outage was caused by a company called Fishell. They were boring next to the BA duct and hit the fiber line multiple times and in multiple locations. There are currently 41 DS3's and 1 OC3 down. He didn't know who the OC3 belonged to. Earlier today, an OC12 belonging to L3 was down, but it has now been restored. Then from MFS - The cut is much more involved than was thought previously. MFS fiber guys went to the site to find the cables covered in concrete, and mixed in with the whole BA damaged duct. It looks like right now MFS is going to have to wait until BA fixes the duct before they can even attempt to fix the fiber. At 12:30 this afternoon, there was an estimated ETA of 2 to 3 hours downtime. That's now been changed to read an "extended downtime". The master ticket number is 412 573. So they managed to bore into the duct then fill it with concrete. Now, Bell is fine (and has dual-entrance around the problem), so as you can imagine, they do NOT want to let MFS un-conrete the duct and repair. Fun fun. Still, apparently 6861 Elm St, Vienna, VA, with at least TWO full-up OC48s of capacity, appears not to have dual-entrance... If anyone is willing to home a customer onto their SMDS circuit for a day, in exchange for various tricks and favors, the noc(a)netaxs.com folks would like to know. SMDS is primarily a backup for them, and 6861 is a backup/2ndary POP for netaxs, but there is one customer so far-out that they had to use SMDS as a primary point of connection. Avi

1 0