http://www.chron.com/disp/story.mpl/ap/tx/5148125.html (AUSTIN) Telephone service was out for seven hours in rural Central Texas after bees attacked a construction worker, causing him to jump off his tractor and hit a lever that lowered an auger that sliced a fiber-optic line.
Sean Donelan wrote:
http://www.chron.com/disp/story.mpl/ap/tx/5148125.html
(AUSTIN) Telephone service was out for seven hours in rural Central Texas after bees attacked a construction worker, causing him to jump off his tractor and hit a lever that lowered an auger that sliced a fiber-optic line.
Is this a 7 hour outage a comment on rural Central Texas availability of fiber splicers or novel ways fiber gets cut? Anytime you talk about "rural" I'm impressed with 7 hours, however -- isn't SONET supposed to make this better? Deepak
On Sep 21, 2007, at 2:38 PM, Deepak Jain wrote:
Anytime you talk about "rural" I'm impressed with 7 hours, however -- isn't SONET supposed to make this better?
We had a customer hit by this, and actually saw services restored for a few minutes in just four hours, but then they went back down. --Chris
On Fri, 21 Sep 2007 15:38:30 EDT, Deepak Jain said:
Anytime you talk about "rural" I'm impressed with 7 hours, however -- isn't SONET supposed to make this better?
I'm not in Texas, but I am rural - there's plenty of places around here where it's just not economically feasible to run 2 diverse fiber paths to a town. Heck, a lot of these places didn't get their *first* fiber until fairly recently. http://maps.google.com/maps?f=q&hl=en&geocode=&q=grundy,+virgina&ie=UTF8&ll=37.279107,-82.099457&spn=0.414686,0.782776&t=h&z=11&om=1 Not a place you'll find a redundant SONET ring. ;)
Deepak Jain wrote:
Sean Donelan wrote:
http://www.chron.com/disp/story.mpl/ap/tx/5148125.html
(AUSTIN) Telephone service was out for seven hours in rural Central Texas after bees attacked a construction worker, causing him to jump off his tractor and hit a lever that lowered an auger that sliced a fiber-optic line.
Is this a 7 hour outage a comment on rural Central Texas availability of fiber splicers or novel ways fiber gets cut?
I'm thinking that getting hit by an auger might put the fiber more into the "mangled" category rather than simply cut.
Anytime you talk about "rural" I'm impressed with 7 hours, however -- isn't SONET supposed to make this better?
SONET... yeah, right. We had a fiber-seeking backhoe take out some fiber in the mid-Willamette Valley recently - it took out long distance for several smaller local phone companies for about that long as well. 911 service for at least some people too. I'm interestedly awaiting the final word on what happened there... Jeff
On Fri, 21 Sep 2007, Deepak Jain wrote:
Is this a 7 hour outage a comment on rural Central Texas availability of fiber splicers or novel ways fiber gets cut?
Anytime you talk about "rural" I'm impressed with 7 hours, however -- isn't SONET supposed to make this better?
Sure, if: 1. the protect path is configured and enabled 2. both the working and protect paths don't run through the same conduit/duct/buffer jms
I'm forking this thread to complain about vendor L's international long haul network. Protected Sonet service (T3). DC to UK. I see more outage notifications than you'd *believe* since the service was established for a customer a few weeks ago. Whether its mandatory fiber relocation or some work in France... or all of the above. Now, getting a notification about a 50ms "switch hit" for protected service is great. No worries or concerns -- even superlative. However, when I see "Location of Maintenance: France" and a 5 minute outage for a protected SONET service on a supposedly redundant, high quality International voice/data network... well, let's just say I'm not impressed -- on 36 hrs notice, no less. I can't do anything with respect to an SLA since there is advanced notice, but isn't it reasonable to assume that in this day-and-age running a properly protected T3 isn't *that hard* anymore???? Especially in advance -- you know, shunt the traffic to one your other circuits because, you know, you are supposed to have this massive network. I think I just put my naive hat on today and I need to go drink a little clue. I'm sure I'll be applying some clue with a manual re-route over another Vendor's network shortly. ;) Deepak
I'm forking this thread to complain about vendor L's international long haul network. Protected Sonet service (T3). DC to UK.
I wonder if anyone is using PWE3 for this kind of service, perhaps in an academic/research environment? It would be interesting to compare notes on outages, latency/jitter, etc. --Michael Dillon
On 9/21/07, Deepak Jain <deepak@ai.net> wrote:
However, when I see "Location of Maintenance: France" and a 5 minute outage for a protected SONET service on a supposedly redundant, high quality International voice/data network... well, let's just say I'm not impressed -- on 36 hrs notice, no less.
I can't do anything with respect to an SLA since there is advanced notice, but isn't it reasonable to assume that in this day-and-age running a properly protected T3 isn't *that hard* anymore???? Especially in advance -- you know, shunt the traffic to one your other circuits because, you know, you are supposed to have this massive network.
Typically for a service like that, the carrier would have one or more long-haul rings, either SONET or DWDM with the SONET on top of it or something, interconnected to access rings from a local access carrier at each end, either close to the customer endpoint or possibly back at regional interconnects. The long-haul carrier might not control the local access carrier except through an SLA, so it could be that the local carrier messed up. One hopes that the interconnections between the networks are also at diverse POPs (at least for big countries like France), but it is possible for the interconnections to fail clumsily. The "shunt the traffic to other circuits" approach is naively correct, but I've seen more than one case of "discover that one side of an access ring has failed by shunting the traffic from the other side to do maintenance and having a customer call you a couple of minutes later to report an outage at their headquarters", which is absolutely never supposed to happen, of course :-) What a 5-minute outage sounds like to me is "The early-90s T3-based restoration system recovered the circuit on an end-to-end basis using DACSs instead of SONET restoring it", but I'd be surprised to see than happening at a newer carrier like Vendor L who built most of their network after SONET technology became affordable. ---- Thanks; Bill Note that this isn't my regular email account - It's still experimental so far. And Google probably logs and indexes everything you send it.
Speaking on Deep Background, the Press Secretary whispered:
Anytime you talk about "rural" I'm impressed with 7 hours, however -- isn't SONET supposed to make this better?
Sure, if: 1. the protect path is configured and enabled 2. both the working and protect paths don't run through the same conduit/duct/buffer
Look up zip 95428. There is one route in, period. Power is on one side of the road, phone copper the other. My customer gave up and moved out after PacBell wanted 3-4 years to get glass in... -- A host is a host from coast to coast.................wb8foz@nrk.com & no one will talk to a host that's close........[v].(301) 56-LINUX Unless the host (that isn't close).........................pob 1433 is busy, hung or dead....................................20915-1433
On Fri, Sep 21, 2007 at 04:49:22PM -0400, Justin M. Streiner wrote:
Anytime you talk about "rural" I'm impressed with 7 hours, however -- isn't SONET supposed to make this better?
Sure, if: 1. the protect path is configured and enabled 2. both the working and protect paths don't run through the same conduit/duct/buffer
I am continually amazed at how often this is the case. I realize that it's expensive to run these lines but when you put your working and protect in the same cable or different cables in the same trench (not even a trench a few feet apart, but the same trench and same innerduct), you have to EXPECT that you're gonna have angry customers. And yet when telco folks learn that this has occured, they often fein being as surprised as the customers. Truely amazing. --- Wayne Bouchard web@typo.org Network Dude http://www.typo.org/~web/
I realize that it's expensive to run these lines but when you put your working and protect in the same cable or different cables in the same trench (not even a trench a few feet apart, but the same trench and same innerduct), you have to EXPECT that you're gonna have angry customers. And yet when telco folks learn that this has occured, they often fein being as surprised as the customers.
Has anyone calculated what the cost of doing this correctly once vs the ongoing support/SLA/etc issues of repairing it when it goes boom is? I've gotta believe that for >90% of the situations where diverse routes exist, just being used as dual linear paths, its cheaper in the long term to do it "right" and cut the size of your outside plant crew (assigned to break/fix) by 90%. :) Deepak
Has anyone calculated what the cost of doing this correctly once vs the ongoing support/SLA/etc issues of repairing it when it goes boom is? I've gotta believe that for >90% of the situations where diverse routes exist, just being used as dual linear paths, its cheaper in the long term to do it "right" and cut the size of your outside plant crew (assigned to break/fix) by 90%. :)
It is definitely cheaper to "do it right" the first time, but to "keep it right", the operator needs to pay very close attention to each and every circuit groom that they perform, less they end up with a degenerate loop somewhere. Depending on the state of their circuit routing database(s), the exercise of checking for overlap against "the other half" of the ring can anywhere from trivial to impossible. I don't think operators intentionally foul this up, but it's real easy to get wrong, particular in the fallout after accumulating a bunch of different companies fiber plants and circuit route systems and trying to consolidate everything for savings. I'm just providing this in answer to "why can't telcos get this right", as there's no reason to think that grooming activity was at all involved in why this particular carrier got stung... ;-) /John
On 9/22/07, Wayne E. Bouchard <web@typo.org> wrote:
I realize that it's expensive to run these lines but when you put your working and protect in the same cable or different cables in the same trench (not even a trench a few feet apart, but the same trench and same innerduct), you have to EXPECT that you're gonna have angry customers. And yet when telco folks learn that this has occured, they often fein being as surprised as the customers.
.. and as long as they are the only telco with copper in the area, they could care less, I guess?
jump off his tractor and hit a lever that lowered an auger that sliced a fiber-optic line.
ps: That story had a kind of "... that killed the rat that ate the malt that made the house that jack built" feel to it. srs
There's a difference between folding a ring or pushing out a spoke to feed a few customers and providing connectivity to a town. I think building a SONET ring, or any kind of redundancy, has more to do with a rural telco's commitment to it's customers than the bottom line. Remember, the building of plant contributes to the cost study, so it may end up having zero cost in the end. Frank -----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu] On Behalf Of Wayne E. Bouchard Sent: Friday, September 21, 2007 7:00 PM To: Justin M. Streiner Cc: nanog@merit.edu Subject: Re: Bee attack, fiber cut, 7-hour outage On Fri, Sep 21, 2007 at 04:49:22PM -0400, Justin M. Streiner wrote:
Anytime you talk about "rural" I'm impressed with 7 hours, however -- isn't SONET supposed to make this better?
Sure, if: 1. the protect path is configured and enabled 2. both the working and protect paths don't run through the same conduit/duct/buffer
I am continually amazed at how often this is the case. I realize that it's expensive to run these lines but when you put your working and protect in the same cable or different cables in the same trench (not even a trench a few feet apart, but the same trench and same innerduct), you have to EXPECT that you're gonna have angry customers. And yet when telco folks learn that this has occured, they often fein being as surprised as the customers. Truely amazing. --- Wayne Bouchard web@typo.org Network Dude http://www.typo.org/~web/
participants (13)
-
Bill Stewart -
Chris Boyd -
David Lesher -
Deepak Jain -
Frank Bulk -
Jeff Shultz -
John Curran -
Justin M. Streiner -
michael.dillon＠bt.com -
Sean Donelan -
Suresh Ramasubramanian -
Valdis.Kletnieks＠vt.edu -
Wayne E. Bouchard