Re: NOC communications (was Re: Process management)
You bring up an interesting (if unspoken) point in your prior statement, and that is: "Communications between entities is worthwhile if the communications facility has a high signal-to-noise rate." While I am not certain, I have strong suspicions that the SS7 NOCs pushed their emergency directory numbers to the general number due to useless, misdirected calls.
Most of the SS7 providers involved reported no problems with misuse of their contact information. A couple of providers said they were changing the contact information out of fear the information might be misused. One provider stated they had problems, but didn't give any specifics. As some anecdotal evidence I've found our WHOIS e-mail contact information tends to get spammed by the small time bulk mailers, usually whose own contact information is bogus. While our phone information tends to get junk calls by large corporations, usually one particular major ISP's telemarketing operation trying to sell Internet service. Handling such calls through the general purpose number can work well, IF and I repeat IF, the staff at the general purpose number is trained how to handle the calls and direct the information expeditiously. Unfortunately, it seems the person changing the emergency contact information forgets to inform the customer service operation they will now be getting those calls. Some companies seem to do this well, some seem to do this poorly. As I found out, you never know when it will be your newest, greenest tech who happens to get the call at the exact same time the senior engineer is in the bathroom. Even though the calls are fairly rare, it is an ongoing training process. Of course, if the general purpose contact number was working well, there wouldn't be a need for another contact method.
Being a secretary is not something that a NOC can afford to do.
Being a secretary is also something most NOCs don't do well. It is a different set of skills. Dealing with a good receptionist is a pleasure. On the other hand, a bad receptionist is really bad. How many folks remember the recurring receptionist character sketch on Saturday Night Live. I doubt most NOC engineers would last 30 minutes on the front desk of a large law firm.
Therefore, to avoid this problem, one must limit the ubiquity of the contact mechanism and increase the value of each message. Using a medium like a phone number is of course the standard method for contact in any emergency situation (email is great, but it lacks a rapid question-answer-experiment ability) but phones also carry with them the ease of use that works against them, as well as for them. A phone number gets handed out on web sites, "emergency call" sheets, etc. and soon people who do not have anything directly relating to operations are calling the operations hot line. Either more staff is required to start answering these questions, or (more likely) the "hot line" becomes not-so-hot and it will go unanswered or not taken seriously, or simply nobody will care about it and it will get forwarded to the Void.
Yes, I'm aware of the 9-1-1 problem. If you make something that works well, everyone is afraid it will be overwhelmed with all sorts of calls not relevant to its primary purpose. On the other hand, if you make a something that is so restricted, it will never be abused, but it will also likely never get used or important pieces of information won't get through. How do you set things up to create a 'shared fate' among all the participants? Do you need a strong moderator, editor or dispatcher to make such a system work? To avoid the 'whose in charge' problem do we draw lots each week for the moderator position? If each of the participants don't feel some pressure from their management, shareholders, customers to make a good faith effort; will any and every system fail of active neglect? There have been several attempts in the past, and have not had a sucessfull track record. I'm going to keep throwing ideas against the wall to see what sticks. Are any of these ideas even close to being useful? Or do people think there is not a problem and I should just shut up. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation
At 02:39 AM 7/13/98 -0500, Sean Donelan wrote: [snip]
As some anecdotal evidence I've found our WHOIS e-mail contact information tends to get spammed by the small time bulk mailers, usually whose own contact information is bogus. While our phone information tends to get junk calls by large corporations, usually one particular major ISP's telemarketing operation trying to sell Internet service.
Having a different number/email address on your AS than what's on your domain name might solve some of these problems- it might not. Being much more aggressive and outwardly unfriendly to people who call the number on your AS records becomes realistic if you have distinctly declared it for NOC communications only. A brief note to your counterpart at the offending organization (in your major ISP example) might be enough to clear things up, or perhaps an ARIN "page of shame" in some public place would do the trick. (ha! Try running that one up the flagpole in front of ARIN's lawyers.)
Handling such calls through the general purpose number can work well, IF and I repeat IF, the staff at the general purpose number is trained how to handle the calls and direct the information expeditiously. Unfortunately, it seems the person changing the emergency contact information forgets to inform the customer service operation they will now be getting those calls. Some companies seem to do this well, some seem to do this poorly. As I found out, you never know when it will be your newest, greenest tech who happens to get the call at the exact same time the senior engineer is in the bathroom. Even though the calls are fairly rare, it is an ongoing training process.
I disagree with your point that NOC-to-NOC communications should ever be forwarded to a general number, even with adequate training. General purpose numbers are not staffed 24x7, and the good intent of a well-informed general purpose number gets sidetracked swiftly during call-center grooming and re-organization of departments and other diversions that occur during the life of a company. How many times have you called an ISP's "hotline" to talk with someone who has no idea what a NOC is? Or how many times have you been on hold for more than 15 minutes only to be transferred to another queue? The best intentions are no match for entropy if the people in charge of the system do not have direct control over the fate of where the call lands.
Of course, if the general purpose contact number was working well, there wouldn't be a need for another contact method.
My point exactly - we all have proof that the current system doesn't work very well. [snip]
Therefore, to avoid this problem, one must limit the ubiquity of the contact mechanism and increase the value of each message.
[snip]
Yes, I'm aware of the 9-1-1 problem. If you make something that works well, everyone is afraid it will be overwhelmed with all sorts of calls not relevant to its primary purpose. On the other hand, if you make a something that is so restricted, it will never be abused, but it will also likely never get used or important pieces of information won't get through.
How do you set things up to create a 'shared fate' among all the
In the "Fantasy" proposal (low-baud satellite bi-directional group communication), this would be solved by forcing everyone to have a terminal. A line-printer (or thermal, if you want it to be a little more quiet ;) based box is easy to understand, easy to use, and can be placed in the lights-camera-action NOC for easy access. It's slow enough that it's not going to be used for social interaction (like an IRC channel or MUSH would) but even a 2400 baud stream would certainly be able to print out enough information on each provider to give an adequate exchange rate. Of course, this scenario is overkill in my mind, and extremely difficult to implement in the anarchy (classic use of the term) that exists in the Internet today, even among just we members under ARIN's umbrella. participants?
Do you need a strong moderator, editor or dispatcher to make such a system work? To avoid the 'whose in charge' problem do we draw lots each week for the moderator position? If each of the participants don't feel some pressure from their management, shareholders, customers to make a good faith effort; will any and every system fail of active neglect? There have been several attempts in the past, and have not had a sucessfull track record.
We already have a central authority that collects two perfectly good points of information about each AS in North America (supposedly.) Between the email addresses and phone numbers, there should be ONE method that gets through to someone. The problem is information rot - phone numbers that ring to the boiler room and email addresses for people who have not worked at said companies for 2 years. No additional system is needed - just fix the one we have. Fixing may take more than just good will by the participants - it will probably take fear of policy application. (it's not a DOS - it's "policy application" if the majority agrees to the terms.) The only argument against competent repair of the information database is "fear of spam". This can be handled by swift and public flogging of offenders, or merely excessive anti-response at the particular abuser. Perhaps making AS records only available to members would cut down on "farming" of such databases by un-involved third parties. I think a username/password combination would be a good idea, provided that it was a reliable system.
I'm going to keep throwing ideas against the wall to see what sticks. Are any of these ideas even close to being useful? Or do people think there is not a problem and I should just shut up. -- Sean Donelan, Data Research Associates, Inc, St. Louis, MO Affiliation given for identification not representation
It's a problem that we've all hit at one point or another. It's not crisis-level all the time, though, so it's not at the forefront of people's minds. It's only when you're looking for the right phone number of the AS that's advertising the more specific route for your biggest customer do you realize that something has to be done to keep the data up-to-date. JT
On Mon, Jul 13, 1998 at 11:23:08AM -0400, John Todd wrote:
I disagree with your point that NOC-to-NOC communications should ever be forwarded to a general number, even with adequate training. General purpose numbers are not staffed 24x7, and the good intent of a well-informed general purpose number gets sidetracked swiftly during call-center grooming and re-organization of departments and other diversions that occur during the life of a company. How many times have you called an ISP's "hotline" to talk with someone who has no idea what a NOC is? Or how many times have you been on hold for more than 15 minutes only to be transferred to another queue? The best intentions are no match for entropy if the people in charge of the system do not have direct control over the fate of where the call lands.
Of course, if the general purpose contact number was working well, there wouldn't be a need for another contact method.
My point exactly - we all have proof that the current system doesn't work very well.
This gives me a chance to plug my noc list again, and get everyone else to contribute to it. http://puck.nether.net/netops/nocs.html Questions or comments about the list? Direct them to me. As for most people being clueless when you talk to them, education is the only path. There's not much else that can be done, there's not anything that prohibits people from starting a clueless company. It's all marketing, if you can market yourselves well, and service your market in a not-unreasonable fashion, they're not likeley to listen to someone else anyways. But we don't want to do something like require people to be certified before becoming an ISP do we? that would be worse. Going on a quest and attempting to educate people as best possible is currently the only path for dealing with cluelesness. As for bad contact information in internet,whois databases, etc.. arin, internic, etc.. should attempt to be proactive at going after these folks. Taking a look at one (you can see it's false) entry in the internic database is: --- Holy Sh1t, INC I gotta take a #2 ASS, PA 56765 --- Not trying to make an example out of these folks, it's just one where any snail mail would never make it there. - Jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net Nether Net | http://puck.nether.net/~jared/ | "You Go To Hell! You Go To Hell and You Die!"
participants (3)
-
Jared Mauch -
John Todd -
Sean Donelan