Ameritech Service Quality Report
The Michigan Public Service Commission has launched an investigation into Ameritech's service quality. According to Ameritech, in August the estimated average repair time will be 115 hours (almost 5 days). Significantly up over previous years' average repair times of about a day (25-28 hours). http://www.cis.state.mi.us/mpsc/comm/ameritech.htm What I find interesting isn't Ameritech's long repair times, but how do providers promise 4 hour repair times when the dominant local loop provider takes over a day to fix something on a good day.
Sean, as you know, typical SLAs provide very limited compensation for missing a deadline. It seems to me that most SLAs are crafted from the faith that normally circuits will be up a very high percentage of the time, and of the time that its down, it might not be down due to a circuit failure, and of those times it is a circuit failure, a customer may not notice/request an SLA-based compensation. Further, in my limited experience, SLAs are rarely actually paid for by providers (i.e. it never clears accounting). Good marketing though. Deepak Jain AiNET (This is not meant to refer to any specific provider). On 3 Sep 2000, Sean Donelan wrote:
The Michigan Public Service Commission has launched an investigation into Ameritech's service quality. According to Ameritech, in August the estimated average repair time will be 115 hours (almost 5 days). Significantly up over previous years' average repair times of about a day (25-28 hours).
http://www.cis.state.mi.us/mpsc/comm/ameritech.htm
What I find interesting isn't Ameritech's long repair times, but how do providers promise 4 hour repair times when the dominant local loop provider takes over a day to fix something on a good day.
: Sean, as you know, typical SLAs provide very limited compensation for : missing a deadline. : : It seems to me that most SLAs are crafted from the faith that normally : circuits will be up a very high percentage of the time, and of the time : that its down, it might not be down due to a circuit failure, and of those : times it is a circuit failure, a customer may not notice/request an : SLA-based compensation. : : Further, in my limited experience, SLAs are rarely actually paid for by : providers (i.e. it never clears accounting). They are often enough; however, many times labor expenses for babysitting to expedite payment nearly equal compensation. So yes, it's often simply another marketing gotcha. -brian
On Sun, Sep 03, 2000 at 01:30:18AM -0700, Sean Donelan wrote:
The Michigan Public Service Commission has launched an investigation into Ameritech's service quality. According to Ameritech, in August the estimated average repair time will be 115 hours (almost 5 days). Significantly up over previous years' average repair times of about a day (25-28 hours).
http://www.cis.state.mi.us/mpsc/comm/ameritech.htm
What I find interesting isn't Ameritech's long repair times, but how do providers promise 4 hour repair times when the dominant local loop provider takes over a day to fix something on a good day.
You're presuming they answer the phone. This has been the case for the past few years: Two weeks ago, I called Ameritech over a hicap repair issue. I spent 75 minutes on hold, only to get disconnected. After calling back, I spent 90 minutes on hold, only to get to an individual who transferred me. I was disconnected during the transfer. Indeed, we had to wake up the VP of another Ameritech division, in the middle of the night, to get his people to open a ticket on this circuit. That was the only way we got it looked at at all. Ameritech repair is useless. While living and working in Michigan, I frequently had to go to the PSC over stupid issues Ameritech should have been able to resolve quickly. --msa
On Sun, Sep 03, 2000 at 02:09:09AM -0700, Majdi S. Abbas wrote:
What I find interesting isn't Ameritech's long repair times, but how do providers promise 4 hour repair times when the dominant local loop provider takes over a day to fix something on a good day.
You're presuming they answer the phone.
[... all too familiar story of ameritech sucking deleted ...]
Ameritech repair is useless. While living and working in Michigan, I frequently had to go to the PSC over stupid issues Ameritech should have been able to resolve quickly.
Not to turn this thread into a Ameritech old war stories (I've got 'em though...), but... I plan either attending the one public commission meeting nearest to me eor at least writing the commission at the address provided from Sean's URL, and I'd encourage other Michiganders[1] who have been screwed over to do so as well. -- Bill Fumerola - Network Architect, BOFH / Chimes, Inc. billf@chimesnet.com / billf@FreeBSD.org 1. Yes, that's how we reference ourselves. Don't ask.
What I find interesting isn't Ameritech's long repair times, but how do providers promise 4 hour repair times when the dominant local loop provider takes over a day to fix something on a good day.
You're presuming they answer the phone.
[... all too familiar story of ameritech sucking deleted ...]
Ameritech repair is useless. While living and working in Michigan, I frequently had to go to the PSC over stupid issues Ameritech should have been able to resolve quickly.
Not to turn this thread into a Ameritech old war stories (I've got 'em though...), but...
I plan either attending the one public commission meeting nearest to me eor at least writing the commission at the address provided from Sean's URL, and I'd encourage other Michiganders[1] who have been screwed over to do so as well.
If anyone here in Illinois is interested in doing something like this for Ameritech, I'd be interested in joining up. I've had T1's down for WEEKS, and am currently down because of a missed install date after we moved. They told me the lines were up, we packed up, moved, then found out "Oops, we didn't actually finish the install.". -- Kevin
What I find interesting isn't Ameritech's long repair times, but how do providers promise 4 hour repair times when the dominant local loop provider takes over a day to fix something on a good day.
Repair time promises are marketing tools, just like X% availablity SLAs. They don't really think all repairs will happen in the stated time, nor do they think all customers will have X% availability. What they do thinl is that in most months, most customers won't go down. And they'll pay the penalty to the rest, because those that do go down might well be down longer than the stated repair time and it might well drive availability below the guaranteed level. They don't have to *ever* fix something in under 4 hours just because they offer such a guarantee, because they will still live up to the guarantee to most customers (i.e. customers that never go down). Ask for an SLA that pays you, say, $100,000 if they miss the mark. You won't get it, because know they're going to miss the mark. They can afford to do so, as long as they don't miss it often, and they keep the costs of missing it low enough. -- Brett
Brett Frankenberger wrote:
Ask for an SLA that pays you, say, $100,000 if they miss the mark. You won't get it, because know they're going to miss the mark. They can afford to do so, as long as they don't miss it often, and they keep the costs of missing it low enough.
Irrelevant. Ameritech is getting spanked for not crediting customers who have had outages; the cost of the SLA doesn't matter. -- North Shore Technologies, Cleveland, OH http://NorthShoreTechnologies.net Steve Sobol, BOFH - President, Chief Website Architect and Janitor Linux Instructor, PC/LAN Program, Natl. Institute of Technology, Akron, OH sjsobol@NorthShoreTechnologies.net - 888.480.4NET - 216.619.2NET
On Mon, 4 Sep 2000, Steve Sobol wrote:
Irrelevant. Ameritech is getting spanked for not crediting customers who have had outages; the cost of the SLA doesn't matter.
Yes, but you need to have systems in place to get your credit. I.E. Covad is outside their SLA with us everyday, but they don't have systems in place to credit us. It cost us more time and money to fight with them over the money then it is worth.
<> Check out the new VoIP list: http://lists.robotics.net/listinfo/voiplist -- Nathan Stratton CTO, Exario Networks, Inc. nathan@robotics.net nathan@exario.net http://www.robotics.net http://www.exario.net
Nathan Stratton wrote:
On Mon, 4 Sep 2000, Steve Sobol wrote:
Irrelevant. Ameritech is getting spanked for not crediting customers who have had outages; the cost of the SLA doesn't matter.
Yes, but you need to have systems in place to get your credit. I.E. Covad is outside their SLA with us everyday, but they don't have systems in place to credit us. It cost us more time and money to fight with them over the money then it is worth.
That's not the issue with Ameritech, or at least they aren't saying in public that that's the problem. Ameritech is whining that PUCO (the Public Utilities Commission of Ohio) is telling them that they owe a lot more in refunds then what they actually owe. It doesn't seem to be a technical or policy issue in this case; they're arguing details. Whether this involves consumers, corporate end-users, ISP's and resellers, or any combination of the three groups, is information I don't have. Obviously, if the fines are being levied as a result of consumer complaints, there is no SLA involved. -- North Shore Technologies, Cleveland, OH http://NorthShoreTechnologies.net Steve Sobol, BOFH - President, Chief Website Architect and Janitor Linux Instructor, PC/LAN Program, Natl. Institute of Technology, Akron, OH sjsobol@NorthShoreTechnologies.net - 888.480.4NET - 216.619.2NET
It's been about five months since I last dealt with Ameritech in Michigan, but prior to that I was dealing with them on a pretty much daily basis for a few years. I may be forgetting something, but I don't remember ever having a five day repair time on a previously working HiCap circuit. I had one that took two or three days once, involving what was apparrently a particularly nasty wet cable problem on some underground copper, but according to our customer Ameritech did have people out there working on it for much of the outage period. More often I'd wait on hold for an hour before finally talking to somebody who would open a ticket and have a tester call me back an hour or so later. Then, if it was an internal problem rather than something requiring a dispatch, they'd generally get it fixed pretty quickly. If it required a dispatch, they'd still generally be able to get somebody out to the site within a few hours. The hold times were never what I considered acceptable, and several circuits had the same problems multiple times, but it was nothing like what Sean's summary, or the first couple pages of that Ameritech report (I haven't read the rest yet), would seem to imply. Residential or small business repair is another story, and week lead times, before escallation, seemed pretty typical. With escallation, it was generally possible to get that down to two or three days, which could still be pretty bad. My impression is that most carriers promising four hour repair times are selling HiCap type stuff, where those lead times may be almost possible. -Steve On 3 Sep 2000, Sean Donelan wrote:
The Michigan Public Service Commission has launched an investigation into Ameritech's service quality. According to Ameritech, in August the estimated average repair time will be 115 hours (almost 5 days). Significantly up over previous years' average repair times of about a day (25-28 hours).
http://www.cis.state.mi.us/mpsc/comm/ameritech.htm
What I find interesting isn't Ameritech's long repair times, but how do providers promise 4 hour repair times when the dominant local loop provider takes over a day to fix something on a good day.
-------------------------------------------------------------------------------- Steve Gibbard scg@gibbard.org
(hi steve!) I have found that most carriers abilities (Including Ameritech) to test ds3 and greater circuits is very poor also. Most require a CO dispatch, and none of their sites are manned, nor do they have remote testing access to put up a loop at a sonet element (very frustrating) for testing. T1's (or other circuits that can be looped up) tend to get fixed fairly frequently, but I have found that some locations have chronic problems, either because there are problems with the smartjack shelves that they use, or because there are very intermittent problems with the pairs into a location. On resold circuits (such as Covad dsl loops to customers), trouble is typically cleared within 24 hours of them isolating the problem with Ameritech, but this typically becomes a 3+ day ETR on services I have experienced. Friends that have recently moved I have heard have waited 45+ days for POTS installs of residental telephone service. I do think that it has been very shortsighted of Ameritech to not have started to roll out more fiber huts to bring services on fiber to the CO. I know there is a hut less than a mile from my home, but most people in my area run directly to the CO (4+ miles away) instead of going to this hut, which would free up more services. I have also found it very tiring that Ameritech closes tickets without contacting us, or "because that's our policy", without reporting back if they found trouble. Hiring a few more people to answer the phones for the hicap services would be nice too.. I have had problems getting into some of their queues, as once they are full, you can't get in any further.. /sigh Their 4 hour times for hicap services are also misleading, as that gives them 4 hours to find a tech before escalation, not 4 hours to fix the service. - Jared On Tue, Sep 05, 2000 at 03:31:31PM -0700, Steve Gibbard wrote:
It's been about five months since I last dealt with Ameritech in Michigan, but prior to that I was dealing with them on a pretty much daily basis for a few years.
I may be forgetting something, but I don't remember ever having a five day repair time on a previously working HiCap circuit. I had one that took two or three days once, involving what was apparrently a particularly nasty wet cable problem on some underground copper, but according to our customer Ameritech did have people out there working on it for much of the outage period. More often I'd wait on hold for an hour before finally talking to somebody who would open a ticket and have a tester call me back an hour or so later. Then, if it was an internal problem rather than something requiring a dispatch, they'd generally get it fixed pretty quickly. If it required a dispatch, they'd still generally be able to get somebody out to the site within a few hours. The hold times were never what I considered acceptable, and several circuits had the same problems multiple times, but it was nothing like what Sean's summary, or the first couple pages of that Ameritech report (I haven't read the rest yet), would seem to imply.
Residential or small business repair is another story, and week lead times, before escallation, seemed pretty typical. With escallation, it was generally possible to get that down to two or three days, which could still be pretty bad.
My impression is that most carriers promising four hour repair times are selling HiCap type stuff, where those lead times may be almost possible.
-Steve
On 3 Sep 2000, Sean Donelan wrote:
The Michigan Public Service Commission has launched an investigation into Ameritech's service quality. According to Ameritech, in August the estimated average repair time will be 115 hours (almost 5 days). Significantly up over previous years' average repair times of about a day (25-28 hours).
http://www.cis.state.mi.us/mpsc/comm/ameritech.htm
What I find interesting isn't Ameritech's long repair times, but how do providers promise 4 hour repair times when the dominant local loop provider takes over a day to fix something on a good day.
-------------------------------------------------------------------------------- Steve Gibbard scg@gibbard.org
-- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine. END OF LINE |
I agree that most of the noise in Michigan is about residential repair times. Repair times for leased data circuits could always be better, but don't seem to be nearly as bad as what people are experiencing for residential repair. For leased data circuits repair of previously installed and working circuits could be faster, but doesn't usually take weeks. However, installs of new leased data circuits is a different story. Delivery dates for new installs are more or less meaningless and it can take weeks or months longer than promised to get a new circuit installed. I agree that things get worse as you start to move up to higher speed circuits. I agree that waiting on hold to talk to someone about repairs is a big problem. Coordination between IXCs and Ameritech also seems very poor at times. As best we can tell the problems are not specific to Michigan and seem to be occurring in all Ameritech states. These problems aren't new, but have been building over a long period of time. My real question is what can be done about this other than complaining on this or similar lists? Are the PSCs in the states served by Ameritech likely to take effective action? The Michigan PSC is pretty weak and I am not too sure that they will be able to make much of a difference. Is Ameritech likely to fix the problems on their own given some time? Since Ameritech is now SBC, what is SBC's track record in this area in other states? I've heard rumors from other areas where SBC has purchased an ILEC that this sort of poor service is pretty common, but this is only rumors. One of the claims is that a large part of the problem is due to too few staff and that in turn is due to the very tight national labor market. If this is true, it would seem that these sorts of problems would be occurring throughout the country and wouldn't be limited to one region or one company. How does Ameritech compare to other LECs? -Jeff Ogden Merit At 3:31 PM -0700 9/5/00, Steve Gibbard wrote:
It's been about five months since I last dealt with Ameritech in Michigan, but prior to that I was dealing with them on a pretty much daily basis for a few years.
I may be forgetting something, but I don't remember ever having a five day repair time on a previously working HiCap circuit. I had one that took two or three days once, involving what was apparrently a particularly nasty wet cable problem on some underground copper, but according to our customer Ameritech did have people out there working on it for much of the outage period. More often I'd wait on hold for an hour before finally talking to somebody who would open a ticket and have a tester call me back an hour or so later. Then, if it was an internal problem rather than something requiring a dispatch, they'd generally get it fixed pretty quickly. If it required a dispatch, they'd still generally be able to get somebody out to the site within a few hours. The hold times were never what I considered acceptable, and several circuits had the same problems multiple times, but it was nothing like what Sean's summary, or the first couple pages of that Ameritech report (I haven't read the rest yet), would seem to imply.
Residential or small business repair is another story, and week lead times, before escallation, seemed pretty typical. With escallation, it was generally possible to get that down to two or three days, which could still be pretty bad.
My impression is that most carriers promising four hour repair times are selling HiCap type stuff, where those lead times may be almost possible.
-Steve
On 3 Sep 2000, Sean Donelan wrote:
The Michigan Public Service Commission has launched an investigation into Ameritech's service quality. According to Ameritech, in August the estimated average repair time will be 115 hours (almost 5 days). Significantly up over previous years' average repair times of about a day (25-28 hours).
http://www.cis.state.mi.us/mpsc/comm/ameritech.htm
What I find interesting isn't Ameritech's long repair times, but how do providers promise 4 hour repair times when the dominant local loop provider takes over a day to fix something on a good day.
--------------------------------------------------------------------- ----------- Steve Gibbard scg@gibbard.org
On Wed, Sep 06, 2000 at 08:15:52AM -0400, Jeff Ogden wrote:
track record in this area in other states? I've heard rumors from other areas where SBC has purchased an ILEC that this sort of poor service is pretty common, but this is only rumors.
I have found that while some individual technicians are very good and give good service, as a whole SBC is unresponsive to customer's wishes, and prone to promising one thing then delivering another. In particular, they kept reprovisioning my phone lines with SLCs, causing a good line to suddenly not work worth a crap for 28.8k and up customers. They also were notorious for telling people "we've tested your T1, and the problem is on your end" when they hadn't tested at all, then when you pester them into testing again they suddenly find a problem on their end. Happened to me more than once. Others have complained about them missing install dates by weeks or months, but that never happened to me. However, we were a much larger customer than the complainers.
On Wed, 6 Sep 2000, Shawn McMahon wrote:
In particular, they kept reprovisioning my phone lines with SLCs, causing a good line to suddenly not work worth a crap for 28.8k and up customers.
Shawn: What? Are they not sticking clock cards into those SLC's? I find that hard to believe. :) Have someone order ISDN. I have to ask though. What's better, marginal 28.8 for a number of customers or good 33.6 and higher for a few. Don't you want more customers? Chuck
On Wed, Sep 06, 2000 at 09:20:11AM -0400, Charles Scott wrote:
What? Are they not sticking clock cards into those SLC's? I find that hard to believe. :) Have someone order ISDN.
Unlimited-use BRI in that town is $150 a month. Another way SBC is serving the public.
I have to ask though. What's better, marginal 28.8 for a number of customers or good 33.6 and higher for a few. Don't you want more customers?
Keep in mind that this was 1997 and 1998, and the phone lines were so bad all over town that nobody was getting more than 28.8 anyway. The compression was resulting in people getting 19.2k and 16.8k connections, and they were rightfully considering this to be unacceptable. They of course always blamed us, and indeed often it was our lines with the problem so they had a point. The phone company would test them for a while, "find nothing wrong", and things would mysteriously clear up afterwards. Then the Wal-mart across the street would need new lines, and suddenly we'd have 3 or 4 go bad for a couple days until we'd notice the pattern.
On Wed, 6 Sep 2000, Jeff Ogden wrote:
One of the claims is that a large part of the problem is due to too few staff and that in turn is due to the very tight national labor market. If this is true, it would seem that these sorts of problems would be occurring throughout the country and wouldn't be limited to one region or one company. How does Ameritech compare to other LECs?
Bell Atlantic (verizon?) in Massachusetts is no better. Repair times for hi-cap circuits are about the same (2-4 hours to get someone who can begin to test / troubleshoot). Residential installs aren't as bad as the Ameritech reports seem to indicate. But BA has its own problems: they are suffering massive interoffice capacity shortages in several key Boston-area central offices and lead times on DS3 and above are as bad as 6 months, depending on where you want the circuit. BA is blaming their capacity problems on Y2K preparations rather than the labor market, but it is obvious that they have personnel shortages, especially on the DSL side of the house. It has been a long time since I had to deal with US West or PacBell, but BA North seems to be in no better shape than Ameritech. I'd like to know if this is the state of LECs across the country or a couple of isolated regional problems. travis pugh shore.net
-Jeff Ogden Merit
Jeff: You should certainly know about these things. There are scant few others who have had more experience with Ameritech. However, I think while coming close to the real problem, you didn't quite hit it. Also having some experience with Ameritech, I know that there are some problems that are long term and others that are more receint. One, for example is problems trying to talk to someone in the HiCap center. For as long as I can remember, that's always been a problem. If you got someone in 15 or 20 minutes, it was a stoke of luck. Historically however, once you did, you got pretty good service and things generally got fixed in short order. It also used to be that personel were generally well trained and equipped. I think this is where things have slipped with Ameritech over receint years. There's no question that downsizing Ameritech has taken a toll on their ability to respond to customer needs. I think it's also taken a toll on the people who work there, and that is clearly part of the problem. In addition, Ameritech's physical resources are being strained to the limit. Unfortunately, downsizing came at just the time the Internet was taking off. With a large expansion in data circuits and many thousands of residential customers ordering second and third lines, there was a trainwreck waiting to happen. A symptom of this is the rumored (not so much rumor) rotating line troubles, where supposedly there isn't enough local capacity to service local customers so a certain number of them need to be out of service at any one time. I don't think it's any surprise that in many areas residential repair is going to be slow. Another point I'd like to make is the way that many people have treated Ameritech. I've seen numerous small ISP's, and even some larger ones, publicly blame Ameritech for delays, outages and disruptions. Sometime in blatant detail. I've also seen the effect on the Ameritech personel in the field. Not only are they dealing with more and more limited resources, but then are getting slammed by the very people they're trying to help. Not a good combination. Clearly it all comes down to dollars. Ameritech has done a lot of cost cutting. There's no question that it shows. They have also built and rebuilt infrastructure to try to keep up with demand. At some point, there needs to be a balance of quality of service and cost. Speaking of money, we all know what the solution to that is--charge more. There's no question that the PSC will approve rate increases to permit Ameritech to solve these problems. In other words, be carefull about what you wish for. Chuck Scott Gaslight Media On Wed, 6 Sep 2000, Jeff Ogden wrote:
I agree that most of the noise in Michigan is about residential repair times. Repair times for leased data circuits could always be better, but don't seem to be nearly as bad as what people are experiencing for residential repair. For leased data circuits repair of previously installed and working circuits could be faster, but doesn't usually take weeks. However, installs of new leased data circuits is a different story. Delivery dates for new installs are more or less meaningless and it can take weeks or months longer than promised to get a new circuit installed. I agree that things get worse as you start to move up to higher speed circuits. I agree that waiting on hold to talk to someone about repairs is a big problem. Coordination between IXCs and Ameritech also seems very poor at times. As best we can tell the problems are not specific to Michigan and seem to be occurring in all Ameritech states. These problems aren't new, but have been building over a long period of time.
My real question is what can be done about this other than complaining on this or similar lists? Are the PSCs in the states served by Ameritech likely to take effective action? The Michigan PSC is pretty weak and I am not too sure that they will be able to make much of a difference. Is Ameritech likely to fix the problems on their own given some time? Since Ameritech is now SBC, what is SBC's track record in this area in other states? I've heard rumors from other areas where SBC has purchased an ILEC that this sort of poor service is pretty common, but this is only rumors.
One of the claims is that a large part of the problem is due to too few staff and that in turn is due to the very tight national labor market. If this is true, it would seem that these sorts of problems would be occurring throughout the country and wouldn't be limited to one region or one company. How does Ameritech compare to other LECs?
-Jeff Ogden Merit
participants (15)
-
Bill Fumerola -
Brett Frankenberger -
Brian Wallingford -
Charles Scott -
Deepak Jain -
Jared Mauch -
Jeff Ogden -
Kevin Day -
Majdi S. Abbas -
Nathan Stratton -
Sean Donelan -
Shawn McMahon -
Steve Gibbard -
Steve Sobol -
Travis Pugh