On Thu, 24 Jul 2008, Paul Ferguson wrote:
If your nameservers have not been upgraded or you did not enable the proper flags, eg: dnssec-enable and/or dnssec-validation as applicable, I hope you will take another look.
Let's hope some very large service providers get their act together real soon now.
There is always a tension between discovery, changing, testing and finally deployment. DNS vendors learned about the vulnerability on March 31 (or possibly earlier). DNS vendors waited over 3 months to publically release their patches, even though they knew their customers and users were vulnerable. It probably took the vendors some time to change their code, test their changes, work on beta releases in various deployments because programmers are human and sometimes patches have bugs too. Then they announced their patches to the world, and the world (and ISPs, etc) has much less time to regression test and verify the systems still work. Vendors have released bugging patches in the past. Patching a large ISP infrastructure under ordinary circumstances can be challanging. If it takes software vendors 90+ days to fix something, is it a surpise it may take a large ISP more than 14 days? If they move to quickly and crash the resolvers because of a bug the human programmers may have not forseen in the ISPs DNS architecture, the Internet is effectively "down" for a large number of users. Result: Bad press, angry customers, lawsuits, etc. If they don't move quickly enough and the vulnerability is exploited by a human bad guy, the Internet is effectively "corrupted" for a large number of users. Result: Bad press, angry customers, lawsuits, etc. Damned if they do, damned if they don't. Or in this case: Damned if they are too fast, damned if they are too slow. I don't think there really is a correct answer. People are going to say they suck no matter what. Anyone who has ever been in the position of scheduling security patches across a large ISP knows they aren't going to get much thanks. Although I didn't know the right answer, I did try to always patch production network first and the corporate network last; so if we didn't get everything finished before the exploit hit I could tell customers we did try to put the customer first. Although internal MIS folks would sometimes get mad at me for waiting to tell them. Some people think you should patch the corporate network first, and the production network later. So it brings up the ancient question about the schedule of vulnerability announcements and whether some providers of some core infrastructure should have an early start to patch their systems; because everyone else will be depending on them functioning to obtain the patches when the vulnerability is widely disclosed. How do you decide how early, who, what, how, ... Or do not play favorites, and announce everything to everyone at the exact same time; and its off to the races. Or something in between.