Customer Support and process management seems to be the Achilles' heel of the Internet industry. This industry has seemed to build up a terrible reputation for poor customer support. Coming from the Nuclear industry, I am truly jaded when it comes to process management, but the Internet industry seems to have taken a track as far as possible on the other side.
Ok, so I'm wierd. I enjoy reading the incident reports nuclear plant operators have to file. One nuclear power plant operator discovered the diesel fuel for their standby emergency generators was actually syrup. How many ISPs actually test what's in that tanker truck before filling up their generators? Or how about the time all the uninterruptiable power supplies at a plant turned off in unison because of a bug in the UPS firmware. How many ISPs would even know that other people had a problem with a particular brand of UPS, and they should check theirs? Has anyone, anywhere, ever really solved these problems? I don't know, but I suspect not. The nuclear industry generates a LOT of paperwork, but its just a bunch of dead trees if no one reads and learns from it. The nuclear industry seems to suffer from the 'Its not my fault because I followed approved procedures.' The internet industry seems to suffer from the 'Its not my fault because no one controls the Internet.' The airline industry suffers from 'Its always the captin's fault.' The telephone industry suffers from 'Its not my fault, we're the telephone company, we can't be wrong.' And so forth. No one just 'knows' this stuff. The cliches do ring true. You either learn it the hard way from your own mistakes. Or you learn it the easy way from other people's mistakes. I'm not saying mistakes can be eliminated. Too many useful discoveries have resulted from mistakes. AS7007 was pretty nasty, but if we had learned our lesson, it might have been worth it. The problem is we didn't learn our lesson. We really need to work on communicating the lessons learned. The U.S. Military has a wonderful web site on 'lessons learned' which they update almost every day. If you are cynical, you might say that just shows the military makes mistakes almost every day. On the other hand, if the lessons are really learned, it means every day there is one less mistake which will occur in the future. I suspect one of the academic types would love to write a grant proposal to operate a web site for Internet operations lessons learned. The question is would anyone contribute anything to it. Doubtful after the first month. And who gets the book deal with O'Reilly.
There apparently have been several organizations developed recently that claim to be attempting to address this problem. However, I have not seen an industry-wide affect by the operations of these organizations.
Then let's get together and form one. The standard operating procedure for forming a new organization is everyone puts in $25,000 and the organization puts out a press release announcing the start of a search for an executive director. We can name W.C. Fields as an ex-officio board member so we can say to tiny tykes, "Go away son, you're bothering me." As you point out there are a number of organizations which have popped up. However it seems to be a lot easier to get mega-corporations to contribute funding in exchange for having their name on a 'worthy cause' than it is to do something concrete. With the millions of dollars ISP salespeople have told me they are spending on their customer service center phone switches, why do I continue to have such a problem getting an answer the question "What phone number should I call if I want to report a problem?" -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation
Sean Donelan wrote:
We really need to work on communicating the lessons learned. The U.S. Military has a wonderful web site on 'lessons learned' which they update almost every day. If you are cynical, you might say that just shows the military makes mistakes almost every day. On the other hand, if the lessons are really learned, it means every day there is one less mistake which will occur in the future.
Having been there, I can state that military lessons-learned efforts are partially effective. Many of the lessons-learned get a cursory reading, are quickly forgotten, and mistakes are repeated. However, the lessons learned that are incorporated into operational checklists (another pet tool in the military) really do help prevent future mistakes.
There apparently have been several organizations developed recently that claim to be attempting to address this problem. However, I have not seen an industry-wide affect by the operations of these organizations.
As a heirarchical organization, the military has a huge machinery in place to review and revise operating procedures all the time and disseminate them in the form of checklist revisions. An example would be NATOPS, which provides naval aviation with procedures for everything from dealing with the loss of an engine, to making a normal landing, to refueling. I can't imagine this practice being replicated on an industry-basis. But operating procedures at a company level, in a checklist format, could benefit from an industry-wide clearinghouse for lessons learned. Following checklists for everything can take the fun out of a job in a hurry, because they have a natural tendency to evolve into excruciatingly detailed documents and turn the operator into an automaton. However, they can be indispensable in times of various types of predictable crises and emergencies.
participants (2)
-
Mark Borchers -
Sean Donelan