Re: Outage (planned and unplanned) notification
David Carmean <dlc@avtel.net> wrote:
Engineers which are too vocal do not seem to survive in telco environment for too long.
Even those who don't employ hammer and nail to make their point; i.e. are simply relating facts about real-time operational issues?
The corporate establishment generally frowns at amateurish attempts to do "damage control" public relations. That's the job for P.R. people, not engineers, you see? Now, the only question is where to find P.R. people who can tell cidr from cider. --vadim PS The word play above is not a joke. I've seen a real person writing "cider" in memos. PPS It is mandatory to pass THC urine test to qualify for a network engineer position in a certain ISP. Having pulse is not required. I hope that clarifies the situation somehow.
The corporate establishment generally frowns at amateurish attempts to do "damage control" public relations. That's the job for P.R. people, not engineers, you see? Now, the only question is where to find P.R. people who can tell cidr from cider.
It's not only peers, it's customers as well. I take full BGP from 3 people. I normally do my best to ensure when we have a routing related problem we get some qualified personnel to ring the provider concerned. Attached is an example phone call (provider = P, customer = C). I've hidden the identities concerned as I'm sure it wasn't actually the fault of the personnel concerned, and composited 2 calls. But this seems to be becoming the norm. Why "yes we have a major problem, we're looking at it, and we'll keep you up to date by email and by phone if necessary" doesn't work is beyond me. Then I can switch them off and switch them back on when they've fixed it. Alex Bligh Xara Networks -- P: Network Operations, X speaking, how can I help? C: We seem to be having problems with backtraffic to us through your network. Can you check you see us right at (say) MAE-East? P: Um, no, I'm not near a terminal at the moment. C: Ur, could you move to a terminal, or transfer me to someone who is near a terminal please? P: All the terminals are busy at the moment. C: Oh well, could you tell me if you are running BGP Dampening on your MAE-East router? P: Well I think we run BGP at MAE-East. (so on for five minutes until transfered) P: Y speaking, how can I help? C: We seem to be having problems with backtraffic to us through your network. Can you check you see us right at (say) MAE-East? P: Your BGP session has been up for a while. C: OK, well [NSP] (for instance) can't reach us. Traceroutes star out when it hits an [NSP] router. P: I can ping your router. C: Yes, and I can get to your NOC fine. But I can't get to [NSP]. P: We can get to [NSP] fine. It must be a problem in your network. C: Can you check you are sending [NSP] are routes? They claim they aren't hearing them from you. P: There are no problems between us and [NSP]. < repeat for 10 minutes > C: Are you sure you have no other network problems at the moment? P: Not really. C: None? [NSP] said you might have had some. P: Well we do have a few problems at the moment. But everything is still working. C: Can you tell me what the problems are? P: They don't affect you. < repeat until bored> Eventually it transpires an East/West fibre cut prevented all East coast customers of C reaching all West coast [NSP] destinations. ... sigh...
participants (2)
-
Alex.Bligh -
Vadim Antonov