Re: MCI WorldCom fiber cut in White Plains, NY
scottd@cloud9.net (Scott Drassinower) writes:
MCI WorldCom has a fiber cut in White Plains, NY, as of 0902 EDT this morning. MFS service seems to be decimated, although the Brooks network is totally intact.
i guess fibre is just not being laid in rings any more? why on earth should a single backhoe cut be able to take ANY circuit out? for the "exchange level peering" PAIX and Ames are setting up, PAIX is using NextLink as the dark fibre provider. but PAIX is not buying a path, we are buying a strand. it's ~250 miles long and goes all the way around the Bay. we're going to interrupt it in two places (PAIX and Ames) with Sonet-capable gear. when the inevitable backhoe comes, only one of the two ways around this ring will be sliced. my hope is that we can restore that cut before the other backhoe comes and slices at the other radian. but i've got to say, i didn't invent this topology -- it's what i've been told is the only way to do something like this, so i'm doing it like i was told to. apparently noone told MFS that this was the only way to do something like this? why pay the complexity and cost overhead of SONET if you're not building rings? -- Paul Vixie <vixie@mibh.net> >> But what *IS* the internet? > It's the largest equivalence class in the reflexive transitive > symmetric closure of the relationship "can be reached by an IP > packet from". --Seth Breidbart
(Chuckle) When I first started looking into SONet back in'92, and later with the Milan (Michigan) cut that took out so many "redundant" paths, I learned that the "redundant" "ring" part of SONet was "intended" to be preventative against an electronics failure, so that a single transceiver failure won't kill the circuit. This is the same reason that they provision "Automatic Protection Switching" (APS). But the two paths on the ring (and the APS circuits) are usually both in the same fiber bundle. For a educational network I was consulting on, they actually ran the pair of bundles between buildings in two liner sleeves within a single duct. Saved on right of way. So, even when you have a firm contract and circuits specified, and it looks like a ring on paper, you have no way of knowning that they actually gave you geographically diverse paths, without walking the couple of hundred miles and inspecting them yourself. APS is really a useless feature. Even when you get the contract to say there are diverse paths, they will later groom the circuits to put them in the same bundle. Paul A Vixie wrote:
why pay the complexity and cost overhead of SONET if you're not building rings?
WSimpson@UMich.edu Key fingerprint = 17 40 5E 67 15 6F 31 26 DD 0D B9 9B 6A 15 2C 32
At 11:49 PM -0400 10/9/99, William Allen Simpson wrote: [snip]
APS is really a useless feature. Even when you get the contract to say there are diverse paths, they will later groom the circuits to put them in the same bundle.
Quite right that grooming is a constant threat. Depending on the carrier, one safeguard: when you write a contract in which you pay for facility diversity, write in a provision that lets you audit the Data Circuit Layout Record (or whatever the specific carrier calls it) periodically -- perhaps every 30 to 60 days. While they may groom both paths into a single cable, they will document this. By Murphy's Law, of course, they will tend to groom just after you've audited the documentation, but it's some help. I'm trying to remember the details, but I believe it was MCI in the Chicago or Florida area (not to single them out, just an example) that was sued successfully for not diversifying media for which facility diversity was specified, even though there was no actual outage.
Paul A Vixie wrote:
why pay the complexity and cost overhead of SONET if you're not building rings?
vixie@mibh.net said:
why pay the complexity and cost overhead of SONET if you're not building rings?
Because on a long path, collapsed SONET (i.e. running on a single path) is about half the cost than diverse SONET. Sometimes less than half as the second leg is less direct. You get what you pay for. -- Alex Bligh GX Networks (formerly Xara Networks)
On Sat, Oct 09, 1999 at 11:42:28AM -0700, Paul A Vixie wrote:
scottd@cloud9.net (Scott Drassinower) writes:
MCI WorldCom has a fiber cut in White Plains, NY, as of 0902 EDT this morning. MFS service seems to be decimated, although the Brooks network is totally intact.
why pay the complexity and cost overhead of SONET if you're not building rings?
In a IP backbone, we prefer to have 2 diverse path's to IP use, than one with protection, this gives a better overall utilisation, and more control of what happends. /Jesper -- Jesper Skriver (JS4261-RIPE), Network manager Tele Danmark DataNet, IP section (AS3292) One Unix to rule them all, One Resolver to find them, One IP to bring them all and in the zone to bind them.
why pay the complexity and cost overhead of SONET if you're not building rings?
In a IP backbone, we prefer to have 2 diverse path's to IP use, than one with protection, this gives a better overall utilisation, and more control of what happends.
i don't entirely agree. two reasons. one: you have to build headroom into your network (whether sonet or IP) to handle at least 2X nominal load -- and this is harder, not easier, in the case of IP level diversity (since it is easier to squeeze past the 2X requirement when the beancounters aren't giving you everything you say you need, and they'll learn this.) two: i prefer to have redundancy at the physical level AND at the IP level -- because back when all my network was carrying was spam and netnews, i actually welcomed the peace and quiet of an outage, but now that there are customers involved, i want peace of mind instead. i take it that it's mostly the bypass carriers and the long haul carriers who cheese out on ringing their sonet on diverse paths? i know pac bell rings just about everything. how are the other regionals?
On Mon, Oct 11, 1999 at 01:33:23AM -0700, Paul A Vixie wrote:
why pay the complexity and cost overhead of SONET if you're not building rings?
In a IP backbone, we prefer to have 2 diverse path's to IP use, than one with protection, this gives a better overall utilisation, and more control of what happends.
i don't entirely agree. two reasons. one: you have to build headroom into your network (whether sonet or IP) to handle at least 2X nominal load -- and this is harder, not easier, in the case of IP level diversity (since it is easier to squeeze past the 2X requirement when the beancounters aren't giving you everything you say you need, and they'll learn this.) two: i prefer to have redundancy at the physical level AND at the IP level -- because back when all my network was carrying was spam and netnews, i actually welcomed the peace and quiet of an outage, but now that there are customers involved, i want peace of mind instead.
We're so lucky to get what we ask for ;-) It's all down to having enough spare capacity, and when you have half of your capacity stuck in SONET/SDH protection, you cannot use that for short peaks of traffic, and you dont have the option of saying, in case of a outage, we will suffer 20% performance drop, which on expensive circuits (like trans atlantic ones) could be what one wanted. One of our trans atlantic STM1/OC3's are with protection (the only way we could get it, as the protection is on STM16/OC48 level), meaning we have 310 Mbps of capacity but can only use 155 Mbps both in the normal and outage senario, where if we had 2 STM1/OC3's with diverse routing, we could choose to use 200 Mbps in the normal senario, and live with 155 Mbps in the backup senario.
i take it that it's mostly the bypass carriers and the long haul carriers who cheese out on ringing their sonet on diverse paths? i know pac bell rings just about everything. how are the other regionals?
Here the most normal is to get circuits without protection, but with the option of manual rerouting in case of a longer outage. /Jesper -- Jesper Skriver (JS4261-RIPE), Network manager Tele Danmark DataNet, IP section (AS3292) One Unix to rule them all, One Resolver to find them, One IP to bring them all and in the zone to bind them.
It's all down to having enough spare capacity, and when you have half of your capacity stuck in SONET/SDH protection, you cannot use that for short peaks of traffic, and you dont have the option of saying, in case of a outage, we will suffer 20% performance drop, which on expensive circuits (like trans atlantic ones) could be what one wanted.
it's all down to the definition of "acceptable loss", then.
participants (5)
-
Alex Bligh -
Howard C. Berkowitz -
Jesper Skriver -
Paul A Vixie -
William Allen Simpson