In a message written on Mon, Nov 19, 2012 at 04:21:55PM -0700, Van Wolfe wrote:
Did anyone else experience issues with NTP today? We had our server times update to the year 2000 at around 3:30 MT, then revert back to 2012.
I'm surprised the various time geeks aren't all posting their logs, so I'll kick off: /tmp/parse-peerstats.pl peerstats.20121119 56250 76367.354 192.5.41.41 91b4 -378691200.312258363 0.088274002 0.014835425 0.263515353 56250 77391.354 192.5.41.41 91b4 -378691200.312258363 0.088274002 0.018668790 0.263749719 56250 78204.354 192.5.41.40 90b4 -378691200.785377324 0.088179350 0.014812585 0.263668835 56250 78416.355 192.5.41.41 91b4 -378691200.785974681 0.088312507 0.014832943 0.209966600 56250 79229.355 192.5.41.40 90b4 -378691200.785377324 0.088179350 0.018668723 378691200.785523713 56250 79442.355 192.5.41.41 91b4 -378691200.785974681 0.088312507 0.018689918 378691200.786114931 Or in more human readable form: /tmp/parse-peerstats.pl peerstats.20121119 192.5.41.41 off by -378691200.312258363 192.5.41.41 off by -378691200.312258363 192.5.41.40 off by -378691200.785377324 192.5.41.41 off by -378691200.785974681 192.5.41.40 off by -378691200.785377324 192.5.41.41 off by -378691200.785974681 The script, if you want to run against your own stats: #!/usr/bin/perl while (<>) { chomp; ($day, $second, $addr, $status, $offset, $delay, $disp, $skew) = split; if (($offset > 10) || ($offset < -10)) { # print "$addr off by $offset\n"; # More human friendly print "$_\n"; # Full details } } It just looks for servers off by more than 10 econds and then prints the line. 378691200 seconds is ~12 years, which lines up with the year 2000 dates some are reporting. The IP's are tick.usno.navy.mil and tock.usno.navy.mil. I can confirm from my vantage point that tick and tock both went about 12 years wrong on Nov 19th for a bit, I can also report that my NTP server with sufficient sources correctly determined they were haywire and ignored them. If your machines switched dates yesterday it probably means you're NTP infrastructure is insufficiently peered and diversified. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/