So... is mail not getting in/out from Nanog right now, or is the fairly major fiber cut in San Jose not newsworthy on the operational list anymore....?
I'm gonna guess that people were too distracted with "Oh crap, where'd the internet go?"
I couldn't get to outside email from inside for a while.
So since there's the question, for those not in the know, the word is that there was a cut through half of a 1000 strand cable owned by Level3, affecting 502 fibers.
It's my understanding that they've been splicing since 23:45 GMT and that they're moving through the mass of fibers at an impressive rate.
I have it on good authority (fiber provider, around 5:30 pm pst) that the outage happened in two phases. First, an electrical contractor backhoed a large fiber link in downtown San Jose (address deleted due to security concerns) this morning, causing moderate damage. Initial repair efforts apparently went nowhere; there wasn't enough cable and trench space to work with. Around 2pm PST the repair lead onsite had the remaining cable sliced at manhole access points a couple of hundred meters each direction from the original cut, to give them clean surfaces and workspace for the splice. Which of course took out the other hundreds of pairs which had survived the initial backhoe. We certainly didn't get any warning that was going to happen, and neither did anyone else we were able to talk to over the course of the afternoon. I also understand from a couple of fiber providers that an unusually large amount of other fiber had been groomed onto that one cable over the last few years of consolidations. Eight or nine people I had talked to thought they had geographically distinct ring loops that turned out to be on that one cable when the second cut took it down hard. -george william herbert gherbert@retro.com
On 8/5/05 8:12 PM, "George William Herbert" <gherbert@retro.com> wrote:
First, an electrical contractor backhoed a large fiber link in downtown San Jose (address deleted due to security concerns) this morning, causing moderate damage.
That's just plain silly. As if we (or even your imagined 'terrorist') don't know where the fiber runs around here. Mindlessly classifying everything as 'secret' is a tactic I'd expect of DHS, not NANOG. 'Need to know' does not appear anywhere in the constitution. -- Joe McGuckin ViaNet Communications (address deleted due to security concerns)
On Sat, 6 Aug 2005, Joe McGuckin wrote:
On 8/5/05 8:12 PM, "George William Herbert" <gherbert@retro.com> wrote:
First, an electrical contractor backhoed a large fiber link in downtown San Jose (address deleted due to security concerns) this morning, causing moderate damage.
That's just plain silly. As if we (or even your imagined 'terrorist') don't know where the fiber runs around here.
well.. theres lots of ducting going down streets but not that many folks know which of them are the major cable routes, i think keeping specific detail discrete is reasonable in a fire near where i am a couple years ago: http://www.theregister.co.uk/2002/10/23/arson_suspected_in_manchester_cable/ it seemed a bit of a coincidence that both the active and protect paths of a major sdh route got hit in this attack and it took out a lot of long distance circuits Steve
Stephen, The point I'm trying to make is that over classifying everything as 'secret' or 'confidential' at this late date is useless. The horse is already out of the barn. You can omit the site of a fiber backhoe accident from an email and say it's due to security concerns, but I can call any telecom vendor who sells SONET or metro ethernet services and get them to fax me a map of their network. At the very minimum all I have to do is keep an eye out for USA markings on the street. Or I could call USA and the next day people with paint cans would be marking up the street, showing me exactly where to dig. If someone wants to cause trouble, the information they need is freely available. The so-called security provisions most telecom companies use are just enough to deter curious teen-agers. On 8/7/05 8:15 AM, "Stephen J. Wilcox" <steve@telecomplete.co.uk> wrote:
On Sat, 6 Aug 2005, Joe McGuckin wrote:
On 8/5/05 8:12 PM, "George William Herbert" <gherbert@retro.com> wrote:
First, an electrical contractor backhoed a large fiber link in downtown San Jose (address deleted due to security concerns) this morning, causing moderate damage.
That's just plain silly. As if we (or even your imagined 'terrorist') don't know where the fiber runs around here.
well.. theres lots of ducting going down streets but not that many folks know which of them are the major cable routes, i think keeping specific detail discrete is reasonable
in a fire near where i am a couple years ago: http://www.theregister.co.uk/2002/10/23/arson_suspected_in_manchester_cable/
it seemed a bit of a coincidence that both the active and protect paths of a major sdh route got hit in this attack and it took out a lot of long distance circuits
Steve
-- Joe McGuckin ViaNet Communications 994 San Antonio Road Palo Alto, CA 94303 Phone: 650-213-1302 Cell: 650-207-0372 Fax: 650-969-2124
Maybe a market difference.. most maps I've obtained in the UK have been under NDA with established relationships already. Altho I suspect they're more concerned at showing me who's duct and fiber they're actually on.. Steve On Sun, 7 Aug 2005, Joe McGuckin wrote:
Stephen,
The point I'm trying to make is that over classifying everything as 'secret' or 'confidential' at this late date is useless. The horse is already out of the barn.
You can omit the site of a fiber backhoe accident from an email and say it's due to security concerns, but I can call any telecom vendor who sells SONET or metro ethernet services and get them to fax me a map of their network. At the very minimum all I have to do is keep an eye out for USA markings on the street. Or I could call USA and the next day people with paint cans would be marking up the street, showing me exactly where to dig.
If someone wants to cause trouble, the information they need is freely available. The so-called security provisions most telecom companies use are just enough to deter curious teen-agers.
On 8/7/05 8:15 AM, "Stephen J. Wilcox" <steve@telecomplete.co.uk> wrote:
On Sat, 6 Aug 2005, Joe McGuckin wrote:
On 8/5/05 8:12 PM, "George William Herbert" <gherbert@retro.com> wrote:
First, an electrical contractor backhoed a large fiber link in downtown San Jose (address deleted due to security concerns) this morning, causing moderate damage.
That's just plain silly. As if we (or even your imagined 'terrorist') don't know where the fiber runs around here.
well.. theres lots of ducting going down streets but not that many folks know which of them are the major cable routes, i think keeping specific detail discrete is reasonable
in a fire near where i am a couple years ago: http://www.theregister.co.uk/2002/10/23/arson_suspected_in_manchester_cable/
it seemed a bit of a coincidence that both the active and protect paths of a major sdh route got hit in this attack and it took out a lot of long distance circuits
Steve
Eight or nine people I had talked to thought they had geographically distinct ring loops that turned out to be on that one cable when the second cut took it down hard.
Perhaps now people will begin to take physical separacy seriously and write grooming protocols and SLAs into their contracts? Or was this type of service "good enough"? --Michael Dillon
On 8/8/05, Michael.Dillon@btradianz.com <Michael.Dillon@btradianz.com> wrote:
Eight or nine people I had talked to thought they had geographically distinct ring loops that turned out to be on that one cable when the second cut took it down hard.
Perhaps now people will begin to take physical separacy seriously and write grooming protocols and SLAs into their contracts?
Or was this type of service "good enough"?
What was the actual cost of this outage to operators in SLA credits? Perhaps it's just a function of economics: it's cheaper to plan for the 3 fiber cuts per 1000 route miles per year (or whatever the silly rate was I came across somewhere) and hand out credits than actually engineer physical redundancy. But seriously, has anyone quantified what this or similar cuts *actually cost*? aaron.glenn
participants (5)
-
Aaron Glenn -
George William Herbert -
Joe McGuckin -
Michael.Dillonļ¼ btradianz.com -
Stephen J. Wilcox