I just spent 80 hours over four days trying to get a DS3 to uu.net fixed - turns out the circuit goes from their POP to WorldCom, from WorldCom to MFS, and -then- to PacBell. Only problem is, they didn't tell me that, and so we started troubleshooting from the far side of the PacBell end, thinking that it was PacBell all the way through, and only finding out after 2 days what the toplogy really looked like. Of course, they'd just assumed that there was no way that -their- portion of the circuit could have problems. PacBell even replaced a module in a DACS 3, thinking -that- was the problem. So it was a bad card or something in an MFS OC-48 mux . . . fine, MFS replaced it. Then they started doing intrusive testing on some OC-3 bundles they were having problems with, of which bundles my DS3 was apparently caught up in, so the circuit kept pogoing for another 8 hours. Since uu.net bought WorldCom and MFS, you'd think they'd have better coordination, no? All that, combined with the fact that MFS and WorldCom apparently have zero remote diags capability and will only dispatch techs when you threaten to sue them (literally), made for a great weekend. ----------------------------------------------------------- Roland Dobbins <rdobbins@netmore.net> // 818.535.5024 voice -----Original Message----- From: Jim Mercer [mailto:jim@reptiles.org] Sent: Monday, September 18, 2000 10:53 PM To: nanog@merit.edu Subject: problems with uunet/worldcom/whoever-they-bought-this-week ok, this is getting ludicrous. [ cut to the chase, if you are a non-US uunet/worldcom/mci customer, you can skip my diatribe and jump down to the bottom ] this is the third monday night when we have had a major BGP/connectivity outage with uunet.ca. two mondays ago, (ie 3 weeks), we had an 11 hour combined outage of either connectivity and/or BGP routing. this was topped off by them restoring the config from an old backup, including a slew of old static routes. last monday we had a 4 hour outage, similar issues. tonight, we got: 18:30 me -> uunet.ca: so, what's the problem uunet.ca: not sure, US-NOC is looking into it 19:15 me -> uunet.ca: any ETA? uunet.ca: problem solved, although the CPU on the router is 100% due to BGP recalc. should be OK soon. 21:30 me -> uunet.ca: so, what's the problem? uunet.ca: bad ATM card, been replaced, back up, although the CPU is 100% due to BGP recalc. should be OK soon. 01:30 me -> uunet.ca: so, what's the problem? uunet.ca: US-NOC is saving the configs customer-by-customer and will be rebooting the router RSN. no ETA. i'm not sure, but a lot of major problems seem to have cropped up since uunet.ca dumped their local technicians and went dependent on the US-NOC. are other non-US uunet customers getting the same treatment? -- [ Jim Mercer jim@reptiles.org +1 416 410-5633 ] [ Reptilian Research -- Longer Life through Colder Blood ] [ Don't be fooled by cheap Finnish imitations; BSD is the One True Code. ]