In a message written on Tue, Nov 20, 2012 at 02:28:19PM -0500, Jay Ashworth wrote:
I'm curious, Leo, what your internal setup looks like. Do you have an internal pair of masters, all slaved to those externals and one another, with your machines homed to them? Full mesh? Or something else?
My particular internal setup is a tad weird, and so rather than answer your question, I'm going to answer with some generalities. The right answer of course depends a lot on how important it is that boxes have the right time. If you have 4 or more physical sites, I believe the right answer is to have on the order of 8 NTP servers. 2 each in 4 sites reaches the minimum nicely with redundancy. These boxes can have GPS, CDMA or other technologies if you want, but MUST peer with at least 10 stratum-1 sources outside of your network. Of course if you have more sites, one server in each of 8 sites is peachy. Those on a budget could probably get by with 4 servers total, but never less! All "critical" devices should then be synced to the full set of internal servers. 4 boxes minimum, 8-10 preferred. NTP will only use the 10 best servers in it's calculations, so there is a steep dropoff of diminishing returns beyond 10. For most ISP's I would include all routers in this list. For the "non-critical" devices? Well, there it gets more complex. For most I would only configure one server, their default gateway router. Of course, pushing out a set of 4+ to themm if that is easy is a great thing to do. The interesting thing here is that no devices except for your NTP servers should ever peer with anything outside of your network. Why? Let's say your NTP servers all go crazy together. The outside world is cut off, GPS is spoofed, the world is ending. All that you have left is that all of your devices are in time to each other....so at least your logs still coorelate and such. So having every device under your master set of NTP servers is important. One guy with an external peer may choose to use that, and leave the hive mind, so to speak. For small players, less than 4 sites, typically just use the NTP pool servers, configuring 4 per box minimum. If you want the same protection I just outlined in the paragraph before, make 4 of your servers talk to the outside world, and make everything else talk to those. Want to give back to the community? Get a GPS/CDMA/Whatever box and make it part of the NTP pool. Want to step up your game (which is what I do), reach out to various Stratum-1's on the net (or find free, open ones) and peer up 8-20 of them.
In my last big gig, it was recommended to me that I have all the machines which had to speak to my DBMS NTP *to it*, and have only it connect to the rest of my NTP infrastructure. It coming unstuck was of less operational impact than *pieces of it* going out of sync with one another...
Yep, a prime example of the scenario I described above. Depending on your level of network redundancy, number of NTP servers, and so on, this is a fine solution. With one NTP server (the DBMS) the downstream will always use it, and stay in sync. It's a valid and good config in many situations. -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/