Re: Just got on this thing (perhaps very belatedly) - root server trouble?
You hear that right folks. About 30% of the nameservers which supposedly are authoritative for .COM domains are either: 1) Non-existant (they don't resolve to an IP address) 2) Unreachable or 3) Don't know what "." is (!)
Now, if it turns out that the number of so-called delegations which aren't really backed by authority records is also 30% of the listing, then that means that of the 790,000+ domains in the COM zone, only about 265,000 are "real", in that they have both a nameserver online AND a proper authority record on that nameserver.
This is a direct result of NSI accepting applications for domains, and listing them, without checking for authoritative SOA records before issuing the records in the COM zone!
I'm apalled at these numbers.
For once we agree. NSI should have stopped this practice long ago. You'll be pleased to hear that there are other name registries (for instance the one serving the "no" (Norway) TLD that actually perform this check. Note that checking when an application is received isn't really enough. In Norway we run regular (monthly) checks of all the second-level domains under "no", and we always find a number of name servers which have ceased being authoritative in the time since last check. Steinar Haug, Nethelp consulting, sthaug@nethelp.no
You hear that right folks. About 30% of the nameservers which supposedly are authoritative for .COM domains are either: 1) Non-existant (they don't resolve to an IP address) 2) Unreachable or 3) Don't know what "." is (!)
Now, if it turns out that the number of so-called delegations which aren't really backed by authority records is also 30% of the listing, then that means that of the 790,000+ domains in the COM zone, only about 265,000 are "real", in that they have both a nameserver online AND a proper authority record on that nameserver.
This is a direct result of NSI accepting applications for domains, and listing them, without checking for authoritative SOA records before issuing the records in the COM zone!
I'm apalled at these numbers.
For once we agree. NSI should have stopped this practice long ago. You'll be pleased to hear that there are other name registries (for instance the one serving the "no" (Norway) TLD that actually perform this check.
Note that checking when an application is received isn't really enough. In Norway we run regular (monthly) checks of all the second-level domains under "no", and we always find a number of name servers which have ceased being authoritative in the time since last check.
Steinar Haug, Nethelp consulting, sthaug@nethelp.no
We haven't even gotten that far yet. I'm just looking at the following things right now: 1) Does the hostname listed in the NS line resolve. 2) Does the resolved hostname actually GO anywhere. 3) Is there something listening on UDP port 53 at that location. 4) Does it know what "." is. We're now well into the "C"s, and so far 32% of the NS lines in the TLD list for COM file fail one of these four tests! This is pretty clearly unacceptable, and far worse than I had ever imagined it was. -- -- Karl Denninger (karl@MCS.Net)| MCSNet - The Finest Internet Connectivity http://www.mcs.net/~karl | T1's from $600 monthly to FULL DS-3 Service | 99 Analog numbers, 77 ISDN, Web servers $75/mo Voice: [+1 312 803-MCS1 x219]| Email to "info@mcs.net" WWW: http://www.mcs.net/ Fax: [+1 312 803-4929] | 2 FULL DS-3 Internet links; 400Mbps B/W Internal
On Tue, 18 Feb 1997, Karl Denninger wrote: [...]
We're now well into the "C"s, and so far 32% of the NS lines in the TLD list for COM file fail one of these four tests!
This is pretty clearly unacceptable, and far worse than I had ever imagined it was.
So does the fact that there are lots of unused-yet-allocated domains in .com have any negative impact on you or anyone else? -- Matt Ranney - mjr@ranney.com This is how I sign all my messages.
On Tue, 18 Feb 1997, Karl Denninger wrote:
[...]
We're now well into the "C"s, and so far 32% of the NS lines in the TLD list for COM file fail one of these four tests!
This is pretty clearly unacceptable, and far worse than I had ever imagined it was.
So does the fact that there are lots of unused-yet-allocated domains in .com have any negative impact on you or anyone else? -- Matt Ranney - mjr@ranney.com
This is how I sign all my messages.
What do you think happens to the nameservers on the net when they're asked for a domain that doesn't have functional servers, and they sit and churn trying to resolve the names? BTW, churn is the right word. Its taking anywhere from 5-10 *seconds* to come back as NXDOMAIN on each request for those that fail to resolve, and this is from the IANA roots. This IS a functional problem - and worse, all those non-existant zones and the VM churn they generate on the COM TLD servers is probably the REASON that we're looking at this kind of horrid performance! -- -- Karl Denninger (karl@MCS.Net)| MCSNet - The Finest Internet Connectivity http://www.mcs.net/~karl | T1's from $600 monthly to FULL DS-3 Service | 99 Analog numbers, 77 ISDN, Web servers $75/mo Voice: [+1 312 803-MCS1 x219]| Email to "info@mcs.net" WWW: http://www.mcs.net/ Fax: [+1 312 803-4929] | 2 FULL DS-3 Internet links; 400Mbps B/W Internal
On Tue, 18 Feb 1997, Karl Denninger wrote: [...]
What do you think happens to the nameservers on the net when they're asked for a domain that doesn't have functional servers, and they sit and churn trying to resolve the names?
BTW, churn is the right word. Its taking anywhere from 5-10 *seconds* to come back as NXDOMAIN on each request for those that fail to resolve, and this is from the IANA roots.
So aside from programs like yours, who ever asks for domains that aren't in use?
This IS a functional problem - and worse, all those non-existant zones and the VM churn they generate on the COM TLD servers is probably the REASON that we're looking at this kind of horrid performance!
Thats fair, I guess. Perhaps some of those $100 in fees for unused domains could be used to buy more RAM for root server operators. -- Matt Ranney - mjr@ranney.com This is how I sign all my messages.
On Tue, 18 Feb 1997, Karl Denninger wrote:
[...]
What do you think happens to the nameservers on the net when they're asked for a domain that doesn't have functional servers, and they sit and churn trying to resolve the names?
BTW, churn is the right word. Its taking anywhere from 5-10 *seconds* to come back as NXDOMAIN on each request for those that fail to resolve, and this is from the IANA roots.
So aside from programs like yours, who ever asks for domains that aren't in use?
How much bounced mail to postmaster do you see due to unresolvable domain names? I see lots, and every one of those churns and does the "wait for the NXDOMAIN response" thing -- in Sendmail, or some other tool, but the point is that this DOES impact service.
This IS a functional problem - and worse, all those non-existant zones and the VM churn they generate on the COM TLD servers is probably the REASON that we're looking at this kind of horrid performance!
Thats fair, I guess. Perhaps some of those $100 in fees for unused domains could be used to buy more RAM for root server operators. -- Matt Ranney - mjr@ranney.com
This is how I sign all my messages.
VM churn is a serious problem; its not just RAM space that causes (or fixes) it. There are a fixed number of contexts in a processor, and those contexts and VM page groupings are also of fixed size. At some point you thrash the VM system even if you can fit it all into RAM (God help you if you have to go to page space in a production system!) You're also assuming that NSI is getting the $100 fees. Why would you pay if you never intended to actually use the domain? You wouldn't. This correlates rather well with the crying that NSI has been doing in the trade press recently about how horrible their financials REALLY are, despite appearances to the contrary. Hell, if they only took real registrations as opposed to shams, by my best guess right now they'd drop the load by anywhere from 30 to 60 percent, and in ADDITION to that the other root servers, all but one of which they DO NOT pay for, would be more efficient for everyone. Has nobody looked at this before? Speaking of which, you folks *do* know that with the NSI TLDs on the root servers that they basically get 90% of their nameserver infrastructure, distributed at that, for free, and 4 out of the 10 "normal" servers are actually paid for with government funds (directly or indirectly; two are DOD, and two are public universities), right? -- -- Karl Denninger (karl@MCS.Net)| MCSNet - The Finest Internet Connectivity http://www.mcs.net/~karl | T1's from $600 monthly to FULL DS-3 Service | 99 Analog numbers, 77 ISDN, Web servers $75/mo Voice: [+1 312 803-MCS1 x219]| Email to "info@mcs.net" WWW: http://www.mcs.net/ Fax: [+1 312 803-4929] | 2 FULL DS-3 Internet links; 400Mbps B/W Internal
On Tue, 18 Feb 1997, Matt Ranney wrote:
BTW, churn is the right word. Its taking anywhere from 5-10 *seconds* to come back as NXDOMAIN on each request for those that fail to resolve, and this is from the IANA roots.
So aside from programs like yours, who ever asks for domains that aren't in use?
Plenty of folks do, some gomer registers a domain, doesn't tell his upstream about it, doesn't do DNS for it, but hands out an email address of joe@my.*&^ed.domain.com, they try and email him.. you get the picture. Day after day we see dozens of lame delegation errors. Very often for the same domains time and time again. I don't usually agree with Karl but in this case he's right.
This IS a functional problem - and worse, all those non-existant zones and the VM churn they generate on the COM TLD servers is probably the REASON that we're looking at this kind of horrid performance!
Thats fair, I guess. Perhaps some of those $100 in fees for unused domains could be used to buy more RAM for root server operators.
Or better yet, to teach the idiots at NSI how to maintain a database. [-] Brett L. Hawn (blh @ nol dot net) [-] [-] Networks On-Line - Houston, Texas [-] [-] 713-467-7100 [-]
On Tue, 18 Feb 1997, Brett L. Hawn wrote: [...]
joe@my.*&^ed.domain.com, they try and email him.. you get the picture. Day after day we see dozens of lame delegation errors. Very often for the same domains time and time again. I don't usually agree with Karl but in this case he's right.
So the rate at which these servers serve queries is measured in queries/second, and we are worried about dozens of failed queries per day? Don't get me wrong, if NSI isn't getting paid to keep all those domains in the database, then they shouldn't be there, but it doesn't seem like as big a problem as Karl is making it out to be. -- Matt Ranney - mjr@ranney.com This is how I sign all my messages.
At 4:44 PM -0800 2/18/97, Matt Ranney wrote:
Don't get me wrong, if NSI isn't getting paid to keep all those domains in the database, then they shouldn't be there, but it doesn't seem like as big a problem as Karl is making it out to be.
NSI's bookkeeping is so confused, they're really not sure who has paid and who hasn't. From what I can tell, they are only deleting domains that haven't paid in about 6 months AND do not have authoratative nameservice working. And it appears of that domain has a registered nameserver in its namespace, then even if the nameserver isn't responding it does not get removed. My theory is their database is so screwed up and in such a fragile state, they're scared to make any big changes to it. Removing a domain with a registered nameserver in it seems like it is a "big change". Add the the fact that they are showing non-payment for many domains that have actually made payment, they don't want to shut off the domain while trying to sort out the paperwork.
Matt Ranney put this into my mailbox:
On Tue, 18 Feb 1997, Karl Denninger wrote:
[...]
What do you think happens to the nameservers on the net when they're asked for a domain that doesn't have functional servers, and they sit and churn trying to resolve the names?
BTW, churn is the right word. Its taking anywhere from 5-10 *seconds* to come back as NXDOMAIN on each request for those that fail to resolve, and this is from the IANA roots.
So aside from programs like yours, who ever asks for domains that aren't in use?
See the logs below. These machines run simple TCP services such as httpd and ftpd. They're simply trying to extract domain names from IP addresses for purposes of logging. These logs also filter the "lame delegation" notices. I did this because otherwise the output below would be at least double in size. Day in and day out I encounter more and more and more and more domains whose administrators haven't a clue. Either their IP addresses don't properly map to hosts, or these hostnames don't map back to the same IPs, or there's a CNAME in an MX or NS record, or some violation of RFC. I'm half-tempted to write up a perl script that does a 'dig SOA' on each domain that shows up in these logs and send off 'learn something about DNS' type e-mail to the mail address that shows up. It's getting more and more annoying that people don't take the time to set up their DNS correctly. I can understand when people make mistakes or have something that'll be corrected by tomorrow. But most of these problems happen over and over and over and nobody knows and/or cares. Someone once wrote to me complaining that they couldn't connect to my FTP server. Turns out that the site had all their dialup IPs mapping to 'dialup.hisdomain.com'. And what did 'dialup.hisdomain.com' point to? The termserver. I sent the admin mail about it, explaining politely that this was contrary to RFC and standards and all that and was the reason why this poor user couldn't ftp to my site. He wrote back refusing to change it because "well, I haven't gotten any other complaints so it obviously works fine, and besides, your looking up both the IP address and hostname is just a needless waste of time." I realize we can't exactly go out and educate the masses forcefully. Internic, however, can. I would like to see them (somehow) set aside the resources to perform random audits of domains; even if only to ensure that all named DNSs are actually authoritative. If it turns out that one of them isn't, the domain holder has a week to correct the situation. If at the end of the week the problem still hasn't been fixed, the domain is put on hold until such time as the problem has been fixed. Lawsuits? Doubtful. What company would want to be caught arguing "Well, it's unfair that they turned us off just because we were doing it wrong, even though they told us how to do it right!" Not even Bob Allisat would want that. Anyhow, to end a ramble - we need to fix this crap, folks. It may not be 'nanog material', but where else do we post it? News of the Weird? -dalvenjah ---syslog excerpt--- Feb 18 13:30:49 dalnet.webmaster.com named[809]: ns_forw: query(221.3.159.192.in-addr.arpa) A RR negative cache entry (PARCPLACE.COM:) learnt (NODATA=192.153.56.3:NS=128.9.0.107) Feb 18 13:30:49 dalnet.webmaster.com named[809]: ns_forw: query(221.3.159.192.in-addr.arpa) No possible A RRs Feb 18 13:30:49 dalnet.webmaster.com named[809]: ns_resp: query(221.3.159.192.in-addr.arpa) No possible A RRs Feb 18 13:30:49 dalnet.webmaster.com named[809]: ns_resp: query(221.3.159.192.in-addr.arpa) A RR negative cache entry (PARCPLACE.COM:) learnt (NODATA=192.153.56.3:NS=128.9.0.107) Feb 18 13:40:57 dalnet.webmaster.com named[809]: ns_forw: query(220.104.204.207.in-addr.arpa) NS points to CNAME (ns.ionet.net:) learnt (CNAME=206.41.131.3:NS=206.41.128.10) Feb 18 13:42:18 dalnet.webmaster.com named[809]: ns_forw: query(23.16.242.207.in-addr.arpa) A RR negative cache entry (204.97.248.2:) learnt (NXDOMAIN=192.112.36.4:NS=128.167.1.100) Feb 18 13:42:18 dalnet.webmaster.com named[809]: ns_resp: query(23.16.242.207.in-addr.arpa) A RR negative cache entry (204.97.248.2:) learnt (NXDOMAIN=192.112.36.4:NS=128.167.1.100) Feb 18 13:42:18 dalnet.webmaster.com named[809]: ns_resp: query(23.16.242.207.in-addr.arpa) No possible A RRs Feb 18 13:42:56 dalnet.webmaster.com named[809]: ns_forw: query(23.16.242.207.in-addr.arpa) No possible A RRs Feb 18 19:23:18 ns1 syslog: gethostbyaddr: www.erinet.com. != 207.0.229.23 Feb 18 19:23:20 ns1 syslog: gethostbyaddr: www.erinet.com. != 207.0.229.23 Feb 18 19:23:27 ns1 syslog: gethostbyaddr: www.erinet.com. != 207.0.229.23 Feb 18 19:23:51 ns1 syslog: gethostbyaddr: www.erinet.com. != 207.0.229.23 Feb 18 19:23:53 ns1 syslog: gethostbyaddr: www.erinet.com. != 207.0.229.23 Feb 18 19:17:34 ns1 in.ftpd[16721]: warning: can't verify hostname: gethostbyname(ma29.axionet.com) failed -- Dalvenjah FoxFire (aka Sven Nielsen) "It brought me a Mr. Potato Head, Founder, the DALnet IRC Network Scully. It knew that I wanted a Mr. Potato Head!" e-mail: dalvenjah@dal.net WWW: http://www.dal.net/~dalvenjah/ whois: SN90 Try DALnet! http://www.dal.net/
Thats fair, I guess. Perhaps some of those $100 in fees for unused domains could be used to buy more RAM for root server operators.
This has in fact been done. NSI sent me a replacement for F.ROOT-SERVERS.NET: MESSAGE Alpha boot: available memory from _0xe46000 to 0x1fffe000 Digital UNIX V4.0B (Rev. 564); Mon _Dec 23 14:52:07 PST 1996 physical memory = 512.00 megabytes. available memory = 497.74 megabytes. using 1958 buffers containing 15.29 _megabytes of memory AlphaStation 255/300 system DECchip 21071
On Tue, 18 Feb 1997, Paul A Vixie wrote:
Thats fair, I guess. Perhaps some of those $100 in fees for unused domains could be used to buy more RAM for root server operators.
This has in fact been done. NSI sent me a replacement for F.ROOT-SERVERS.NET:
MESSAGE Alpha boot: available memory from _0xe46000 to 0x1fffe000 Digital UNIX V4.0B (Rev. 564); Mon _Dec 23 14:52:07 PST 1996 physical memory = 512.00 megabytes. available memory = 497.74 megabytes. using 1958 buffers containing 15.29 _megabytes of memory AlphaStation 255/300 system DECchip 21071
Hey, that's great. So are you seeing performance problems from all the bloat that they are allowing to creep into the tables? -- Matt Ranney - mjr@ranney.com This is how I sign all my messages.
This might actually be relevant to NANOG. Wow. Amazing.
physical memory = 512.00 megabytes. available memory = 497.74 megabytes. using 1958 buffers containing 15.29 _megabytes of memory AlphaStation 255/300 system DECchip 21071
Hey, that's great.
So are you seeing performance problems from all the bloat that they are allowing to creep into the tables?
None. Difficult though this is to say after working for DEC during the "dark times" of the MicroVAX II and DECstation 5000, and after being such a strong BSD/OS booster in the years since then: the Alpha is amazing, and Digital UNIX (used to be DEC OSF/1) is not nearly as horrible as Solaris or HP-UX (or SunOS or Ultrix for that matter). The above system runs like the wind. 512MB, 333MHz, 256bit memory bus, 4MB cache, wide SCSI. I wish I could get sources for Digital UNIX, but I can use NetBSD (or Linux, I guess) in applications where that's absolutely needed. For the most part I've been able to compile any 4.4BSD-ish (or POSIXish) application without any trouble, and Digital UNIX has a log structured file system that's hugely faster than stock BSD UFS/FFS. (My MH inbox has 2000 messages in it and I can "rescan" in mh-e in 12 seconds.) My internal net is about 50% Digital UNIX and 50% BSD/OS at this point. (Whenever someone else is buying, I ask for an Alpha with Digital UNIX.) If anyone on NANOG is evaluating architectures for high performance servers, this is the one to check out. (I have a hard time getting _anything_ to compile on Solaris or HP-UX.) Indeed, while I was at DEC (1988-1993), their hardware and software was just horrid. And now here I sit, typing this on a DEC HiNote II laptop running BSD/OS, using a DEC Roamabout wireless ethernet, running X emacs to a DEC Alpha that's my desktop workstation. After I left they started doing things right. (I used to cry in my beer about this, but now I just drink the beer.)
What do you think happens to the nameservers on the net when they're asked for a domain that doesn't have functional servers, and they sit and churn trying to resolve the names?
They use 30 bytes of memory per such query, during the time it takes to get back an NXDOMAIN, which is then cached for at least five minutes. Perhaps you had a point to make?
participants (7)
-
Brett L. Hawn -
Dalvenjah FoxFire -
Karl Denninger -
Matt Ranney -
Paul A Vixie -
Rusty H. Hodge -
sthaug＠nethelp.no