On Sat, Dec 31, 2005 at 05:06:59PM -0800, Roy wrote:
Kevin Day wrote:
Last NTP spam:
I'm by no means an NTP expert, if anyone else is, please pipe up.
About 30 minutes before the leap second should have occurred, several of our systems reported "xntpd[13742]: time reset 0.958385 s", which was really strange. They moved the wrong direction, and they did it early. Shortly after, those systems lost ntp association and began drifting. About 10 minutes after midnight all have regained sync. I wasn't checking things that early to see why, it's possible some of our NTP sources started disagreeing on what the correct time was, and would also match what other people have reported off-list, going back as far as 18 hours before midnight.
Several public NTP sources are now indicating a "leap second alarm" (setting the leap bits to 11), which will cause most NTP clients to rule them out as a source. ntp-2.gw.uiuc.edu is an example:
130.126.24.44: Server dropped: Leap not in sync server 130.126.24.44, port 123 stratum 2, precision -19, leap 11, trust 000 refid [128.174.38.133], delay 0.03357, dispersion 0.00049
According to ntpdate, its clock seems to have stopped about 5 minutes before midnight, and hasn't yet recovered.
Other NTP servers haven't cleared their "today is a leap second day" bit, which they should have by now. Some NTP implementations rule out servers that don't agree with what their "master" server thinks the leap second bits should be. My reading of the NTP spec says that at 00:00:00 the leap bits should have been returned to zero. Attempting to sync from one of these servers will produce a "Next leap second occurs at 00:00:00.000 UTC Sun Jan 01 2006" message, but that should be harmless as long as they correct themselves a while before midnight.
Still others have their clocks off by a significant amount(10+ minutes) and think they're still in sync, but since I started typing this email, they all have corrected themselves.
While I can't say anything broke on our network as a result of the leap second, a good percentage of our gear lost NTP sync or had some kind of NTP problem around midnight UTC. You may want to check your NTP status at some point, in case something drifted quite a way off and won't step itself back now because the difference is too great.
-- Kevin
There is at least one stratum-1 server here on the West coast that my NTP says is now off by 1 second. Several stratum-2 are synced to it and are now off also. So checking servers might be a good idea
Are they a GPS sync (or do you know?) The GPS system doesn't really handle the leap second situation the same as others, there is a little blurb here that talks about it: http://gpsinformation.net/main/gpstime.htm I know that I saw my GPS output the following: @051231235958 @051231235959 @060101000000 @060101000000 @060101000001 @060101000002 @060101000003 Which is mostly correct, it should have really read 5960 instead. I saw my el-cheapo clocks drift by a second at midnight utc. I suggest changing your clock sources to something more reliable if you're seeing folks that are not diligent stratum-1 sources. I suggest this as a source: http://www.stupi.se/Time/ I'm kinda curious what CDMA network clocks said around this time or if they just drifted, i'm sure someone here put their cmda phone in debug and watched it. - jared -- Jared Mauch | pgp key available via finger from jared@puck.nether.net clue++; | http://puck.nether.net/~jared/ My statements are only mine.