[ On Wednesday, January 24, 2001 at 20:30:12 (-0500), Henry Yen wrote: ]
Subject: Re: Microsoft spokesperson blames ICANN
On Wed, Jan 24, 2001 at 06:01:29AM -0500, Greg A. Woods wrote:
[ On Wednesday, January 24, 2001 at 13:09:45 (-0800), Roeland Meyer wrote: ]
{OBofftopic: hmm, look at the two timestamps, above. did greg reply to roeland's e-mail before it was written?}
As far as I can tell all my system clocks are close enough to true network time that NTP hasn't been complaining! :-) Note though that my logs show my reply being sent at 18:01 -0500 and the message came back to me with the date header intact and reading: Date: Wed, 24 Jan 2001 18:01:29 -0500 (EST) so perhaps the error is actually in your MUA (i.e. in its formation of the "On ... wrote:" line when preparing the quoted message). Apparently it gets the "AM" and "PM" wrong when converting from a 24-hour clock to a 12-hour clock. It should have written "06:01:29PM -0500".
whoa, slow down... microsoft apparently hasn't quite figured out what hit them (and in these later hours there's implications that there is more than one issue happening here).
In this particular case it's totally irrelevant what hit them. They needed to get at least one solid reliable replacement nameserver up and running and answering on one of those IP addresses as soon as humanly possible if they were to try an mitigate the damage. If it were me and running the show and if I had even a hint that there were malicious agents responsible I'd have grabbed as many raw packets off the network as I could conveniently and quickly store, then I'd have literally pulled the plug on at least two of the machines and sent the works off for forensic analysis while a new, slightly different, and far more secure, machine was brought in to provide this most critical service. Given the hindsight gained from reading their announcment (and guessing what really happened), perhaps they even did that, but it shouldn't have taken them so many more hours to figure out that the world still wasn't seeing their DNS no matter what they did to those servers. Of course it wouldn't have been nearly so critical an issue requiring such quick and dirty action if they would have had more diverse DNS servers. Part of the problem of course is that they may not have percieved the full extent of their problem as quickly as some of us from the outside were imagining it to be, though that's somewhat difficult to understand, especially given the nature of the discussion in open forums such as this one.... I wouldn't have been "kicking" them while they were still down if it wasn't that they'd clearly and obviously tied their own noose and stepped into it and then pulled the lever on their own trap-door themselves! The comedy of errors in their recovery attempts and the enormous delay in returning their DNS to operational status points out several grave operational errors, but none of those errors should ever have caused any visible problems in the first place -- the root cause of their problems remains in the fact that they did not follow the best common practices already well documented by other's who have learned these lessons from the school of hard knocks. -- Greg A. Woods +1 416 218-0098 VE3TCP <gwoods@acm.org> <robohack!woods> Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>