Hi Brad, Paul, I received the below note from ARIN yesterday evening regarding their NS changes on Thursday. This should explain what we're seeing. It appears some older versions of BIND and some os-bundled resolver libraries may be affected. Trent -------- Original Message -------- Subject: Re: [ARIN-20031003.584] response to reverse PTR queries Date: Mon, 06 Oct 2003 17:14:29 -0400 (EDT) From: Cathy Murphy <cathym@arin.net> To: Trent Arsenault We've isolated the problem you've been experiencing to how the earlier BIND resolvers work. It appears that they do not properly follow referrals. The upshot is to get newer versions of your resolver software. Here is what is happening: dig -x 129.159.39.15 * Assuming no cache, a query goes to the root servers, which return no answer but give the name servers for 129.in-addr.arpa (referral) to [chia,dill,...indigo].arin.net * A query then goes to the .net name servers [a,b...m].gtld-servers.net for [chia,dill...indigo].arin.net This is where the problem arises. There is no A record (glue) for [chia...indigo].arin.net on the .net nameservers. Previously, our zones were also served by arrowroot.arin.net and buchu.arin.net, which had (inappropriate) glue records in .net. (You can still see them by doing "dig @a.gtld-servers.net arrowroot.arin.net".) We are phasing out these servers. On Thursday, we removed arrowroot and buchu from the delegations for our in-addr.arpa zones. Although the change was done correctly it uncovered a hidden misconfiguration which enabled older BIND resolvers to work. The return of arrowroot's A record in this way violated DNS rules on authority and credibility of data, facets better enforced in newer versions of BIND. Apparently older versions of BIND resolvers rely on this A record to descend the tree towards the answer and are unable to handle the referral that should have been returned. So now, instead of responding with an A resource record, the .net name servers are correctly responding with a referral to the arin.net name servers: [a3,b3...].nstld.com. What the resolver should do is follow this referral to get the A record of the [chia,dill,etc].arin.net name servers. Instead, it appears to time out. We haven't figured out what versions of BIND do and do not work with our changes. We also haven't dug deeply enough to know why the older resolvers fail to pursue the referral messages. I apologize if this seems like it's another case of "we're making a change and you have to live with it." All of the options we've identified for avoiding this were kludges and hacks. We couldn't come up with a way to help older resolvers without otherwise risking some other misconfiguration. Regards, Cathy Murphy ARIN Trent Arsenault trent@trent.us At 04:58 PM 10/6/2003, Brad Knowles wrote:
At 4:09 PM -0700 2003/10/06, Trent Arsenault wrote:
I've been in touch with ARIN on the same issue noticed at a different site.
According to ARIN, some older BIND resolvers aren't handling the referrals that they get back from the gtld-servers for some of ARIN's name servers. The problem started Thursday when ARIN changed the list of NS's for the ARIN in-addr.arpa zones.
ARIN is still investigating and I'm waiting to hear back.
From what I can tell, the responses are being handed back authoritatively. From my reading of RFC 2821, for a delegation this shouldn't happen. The root nameservers don't do this for .com, and they should do the same for in-addr.arpa (they delegate .arpa to themselves, so that would be okay).
Note that k.root-servers.net does not return any glue. I'm guessing they might now be running on nsd instead of BIND -- yup, looks like they're running nsd 1.2.2, at least according to the following:
% dig @k.root-servers.net. chaos txt version.server
; <<>> DiG 9.2.2 <<>> @k.root-servers.net. chaos txt version.server ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64350 ;; flags: qr rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION: ;version.server. CH TXT
;; ANSWER SECTION: version.server. 0 CH TXT "NSD 1.2.2"
;; Query time: 35 msec ;; SERVER: 193.0.14.129#53(k.root-servers.net.) ;; WHEN: Tue Oct 7 01:32:54 2003 ;; MSG SIZE rcvd: 54
This looks to me like it is related to the same problem that we're having at the moment with UltraDNS and how they hand out answers marked as authoritative for zones where there is also host registration for that name (see hark.org and usenix.org).
I've done some zone transfers from c.root-servers.net, and from what I can tell, the zones appear to be okay in and of themselves. I'm wondering if this problem isn't somehow related to the version of BIND that may be running on most of these systems -- perhaps a bug with the new "delegation-only" and "root-delegation-only" code?
I don't know. This looks pretty strange to me. Paul -- do you have any ideas?
-- Brad Knowles, <brad.knowles@skynet.be>
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania.
GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)