Sorry, another long one. Relevance to NANOG, well, NOCs are customers too. And "remote" NOCs often report problems that effect your paying customers too. Customer service should be of interest to operations folks, at least to the extent the problems are getting reported to the right people to fix. I doubt I can change anyone's mind that providing explanations to customers and non-customers when the network has problems is good for business. In the future I will simply recommend to customers to buy services from NSPs which do provide explanations when their networks fail. Since I haven't found a perfect network yet, I suspect it includes everyone on this list.
Why should they talk to you? Do you pay them a service fee? That's my base issue, there is a hierarchy, and you can't skip rope to the other guy. It just doesn't work, there's nothing in the system to encourage it.
The hierarchy is dead. None of the old NSFnet regional have a monopoly on service in their regions any more. Outside of the US, there are still a few monopoly providers, but they are a rare breed. If you aren't providing the level of service I need, I'll go to someone who can. If XYZ's NOC gives me better service than ABC's NOC, I'll recommend XYZ to my customers.
Sean Donelan has a terribly good point, he's my customer, and his words mean alot, but I can't agree w/ him that he should/could demand the same thing from another ex-NSFnet regional, or from Sprint. I certainly see no reason why I should do this work for you.
Because it is in their self-interest? You are correct I can't make anyone run their network how I would like it run, not even MIDNET (GI). But I can point out long-term problems and code of silence is costing such providers money, and has already cost them customers. For example, I really wish my direct providers would stop munging BGP announcements, or explain why they are doing it. If I have made a mistake, I would like to fix it. Otherwise I will come to the conclusion those provider's NOCs are not up to the job and find a different provider that can do the job. When someone (anyone) reports a problem effecting connectivity with your network, more than likely the reverse is also true for your paying customers. DRA has a bunch of customers connected through just about every major NSP in North America and a couple of other continents. The only time "I" call another NSP is when the process has become totally FUBARed. When I call another NSP, it is usually that NSP's last chance to keep a paying customer on their network. I might call BARRNET because the University of California-Davis has reported problems reaching DRA to DRA's help desk, and the problem hasn't been resolved. No, BARNET doesn't *have* to talk to me. And I will report the same back to the customer. However, I suspect it is in BARRNET's self-interest to work with me in resolving the problem to ensure UC-Davis has end-to-end reliability. I track network reliability by dollars (not packet loss, not latency). I measure network providers, good and bad, by how many of our customers have used their own dollars to buy private lines to St. Louis because they couldn't get the reliability they needed from the network provider. It is not a pretty picture. <http://dranet.dra.com/dranet.html> has a picture where our private line customers are located. If you are an NSP, every one of those green boxes (some boxes represent many paying customers) is an arrow through the heart of your (former) customers view of your network reliability. If you are an NSP in one of those areas, not dealing with these problems or providing coherent explanations has cost you cold-hard cash. Money is something I expect most upper managers to understand. DRA makes its profits elsewhere. DRAnet is simply a vertical market VPN used to sell access to other things. I'm happy to use the Internet and other NSP's to provide that VPN, when the quality exists. On the other hand, if I have to manage a not-so virtual VPN with private lines to achieve the required level of quality, I do. Maybe Adam Smith's invisible hand will correct this eventually.
There seems to be this large obsession with linking information to action. If you get an update you think something's happening. Perhaps it's needed, but stuff will happen whether your hand is held or not.
As I said before: Ideally I want a reliable network. If you can't provide a perfectly reliability network I want an explanation when I can't get through. And I want the problem fixed. The better the explanation, the longer I'm willing to give you to fix the problem. If I get no explanation, I expect the problem to already be fixed. The current situation is the customer gets neither the explanation nor action solving the problem. My proof is the DRAnet map. DRA's customers take a very, very long time to budget money. Those green boxes represent customers whose problems went unanswered, and unsolved for a long time before they gave up on their NSP and expended their own dollars for a private line to St. Louis. Since the technicians seem to be having a very difficult time fixing the network, I thought upper management could meet my other goal. Give the customer an explanation. I'm not pointing fingers at any particular NSP, because frankly I don't have enough fingers to point. Everyone had problems. Yes, even DRA's NOC has fallen down a few times. I'm not asking for perfection, but an explanation when things don't work, while you fix the problem. The Internet is a global cooperative network. If people don't cooperate, the global nature of the network fails. Since your customers may in fact want to use the Internet to communicate globally, problems effect customers globally. When I go to the US Post Office, sometimes there is a sign on the wall that postal service to Timbukto may be delayed because Timbukto's main post office was blown up. I have no idea how many postal customers in Olivette, Missouri send mail to Timbukto. Even though the US Post Service has no control over rebuilding Timbukto's main post office, the US Post Service has discovered it is good customer service to inform their customers why their mail to Timbukto may be delayed. Can't NSPs provide their customers an explanation at least as well as the US Post Office? -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation