NSI Bulletin 098-010 | Update on Whois
Network Solutions has been seeing increased traffic on whois in the last few months. The number of queries has approximately doubled each month since June. At this time, 40% of our inbound http traffic is whois related. In an attempt to alleviate some of the load on the servers, we have taken the following steps: - Some of the worst offenders (single site, large number of accesses) will be blocked from access starting this week - Replication of the whois data will be started earlier in the evening to allow for more timely updates - Existing whois hosts have had processor and memory upgrades - Local file storage is now being used vice NFS - Medium-term and long-term development efforts are underway to redesign whois for optimal performance. This includes in-memory searches, better parsing and dynamic updates obviating the need for large daily data movements. _____________________________________ David H. Holtzman (dholtz@internic.net) Sr Vice President of Engineering Network Solutions, Inc.
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of David H. Holtzman Sent: Tuesday, September 01, 1998 5:04 PM To: NANOG Subject: NSI Bulletin 098-010 | Update on Whois
Network Solutions has been seeing increased traffic on whois in the last few months. The number of queries has approximately doubled each month since June. At this time, 40% of our inbound http traffic is whois related. In an attempt to alleviate some of the load on the servers, we have taken the following steps:
- Some of the worst offenders (single site, large number of accesses) will be blocked from access starting this week This seems totally inappropriate. - Replication of the whois data will be started earlier in the evening to allow for more timely updates - Existing whois hosts have had processor and memory upgrades - Local file storage is now being used vice NFS - Medium-term and long-term development efforts are underway to redesign whois for optimal performance. This includes in-memory searches, better parsing and dynamic updates obviating the need for large daily data movements.
Why don't you just allow ftp access to the whois database, and let others serve up the data?
_____________________________________ David H. Holtzman (dholtz@internic.net) Sr Vice President of Engineering Network Solutions, Inc.
Cheers, Alan
Why don't you just allow ftp access to the whois database, and let others serve up the data?
Because the bad people will be able to do arbitrary searches on the data to spam and figure out what domains have been removed. You know, generally make our lives more difficult! =) -- Michael L. Barrow * <mlbarrow@eni.net> * Network Engineer Epoch Internet * DSL Engineering * (949) 399-8413
At 05:03 PM 9/1/98 -0400, you wrote:
Network Solutions has been seeing increased traffic on whois in the last few months. The number of queries has approximately doubled each month since June. At this time, 40% of our inbound http traffic is whois related. In an attempt to alleviate some of the load on the servers, we have taken the following steps:
- Some of the worst offenders (single site, large number of accesses)
will
be blocked from access starting this week - Replication of the whois data will be started earlier in the evening to allow for more timely updates - Existing whois hosts have had processor and memory upgrades - Local file storage is now being used vice NFS - Medium-term and long-term development efforts are underway to redesign whois for optimal performance. This includes in-memory searches, better parsing and dynamic updates obviating the need for large daily data movements.
_____________________________________ David H. Holtzman (dholtz@internic.net) Sr Vice President of Engineering Network Solutions, Inc.
I may be wrong but aren't the fees that Network Solutions is charging for domain name registration "supposed" to cover the costs of providing WHOIS service for those domains? Is this just another attempt to nickel and dime everyone to death on some sort of new usage charge for WHOIS service? -------------------------------------------------------------------------- Christopher K. Phillips President - Vanion, Inc. V: 719-574-1826 F: 719-574-1837 C: 719-229-7890 -------------------------------------------------------------------------- The Enhanced Services Company --------------------------------------------------------------------------
I seemed to have missed the section he mentioned additional charges. I was thrilled to see they would be blocking domains as the high use ones are just spammers raping the database. Sincerely, Andy Walden System Administrator MTCO Communications 1-800-859-6826 On Tue, 1 Sep 1998, Christopher K. Phillips wrote:
At 05:03 PM 9/1/98 -0400, you wrote:
Network Solutions has been seeing increased traffic on whois in the last few months. The number of queries has approximately doubled each month since June. At this time, 40% of our inbound http traffic is whois related. In an attempt to alleviate some of the load on the servers, we have taken the following steps:
- Some of the worst offenders (single site, large number of accesses)
will
be blocked from access starting this week - Replication of the whois data will be started earlier in the evening to allow for more timely updates - Existing whois hosts have had processor and memory upgrades - Local file storage is now being used vice NFS - Medium-term and long-term development efforts are underway to redesign whois for optimal performance. This includes in-memory searches, better parsing and dynamic updates obviating the need for large daily data movements.
_____________________________________ David H. Holtzman (dholtz@internic.net) Sr Vice President of Engineering Network Solutions, Inc.
I may be wrong but aren't the fees that Network Solutions is charging for domain name registration "supposed" to cover the costs of providing WHOIS service for those domains? Is this just another attempt to nickel and dime everyone to death on some sort of new usage charge for WHOIS service?
-------------------------------------------------------------------------- Christopher K. Phillips President - Vanion, Inc. V: 719-574-1826 F: 719-574-1837 C: 719-229-7890 -------------------------------------------------------------------------- The Enhanced Services Company --------------------------------------------------------------------------
On Wed, Sep 02, 1998 at 09:58:16AM -0500, Andy Walden wrote:
I seemed to have missed the section he mentioned additional charges. I was thrilled to see they would be blocking domains as the high use ones are just spammers raping the database.
I think it's a good thing too, as long as they have a provision to reinstate providers who have dealt with abusers... -- "You know, I've decided lizards aren't too smart." --Me, to "Junior", one of my iguanas
On Tue, 1 Sep 1998, Christopher K. Phillips wrote:
I may be wrong but aren't the fees that Network Solutions is charging for domain name registration "supposed" to cover the costs of providing WHOIS service for those domains? Is this just another attempt to nickel and dime everyone to death on some sort of new usage charge for WHOIS service?
Could be me - but it kind of sounded like they were blocking abusers - like address harvesters - single sites with thousands of requests in a short period of time - that can't trully be labled a bad thing now can it? -- I am nothing if not net-Q! - ras@poppa.clubrich.tiac.net
Christopher K. Phillips wrote:
I may be wrong but aren't the fees that Network Solutions is charging for domain name registration "supposed" to cover the costs of providing WHOIS service for those domains? Is this just another attempt to nickel and dime everyone to death on some sort of new usage charge for WHOIS service?
Who said anything about usage charges? It looks like the proposed work mentioned by David Holtzman is to control _abuse_ of the whois service. Otherwise, the load on whois will grow without bounds and we'd eventually have to pay more in registration fees. Of course, that's just my figurin' and nothing official... -- Michael L. Barrow * <mlbarrow@eni.net> * Network Engineer Epoch Internet * DSL Engineering * (949) 399-8413
How many queries per day do you see? Given the simplicity of a typical query, I'd expect that a single server could handle many millions of requests/day.
mentioned by David Holtzman is to control _abuse_ of the whois service. Otherwise, the load on whois will grow without bounds and we'd eventually have to pay more in registration fees. Of course, that's just my figurin' and nothing official...
Who said anything about usage charges? It looks like the proposed work mentioned by David Holtzman is to control _abuse_ of the whois service. Otherwise, the load on whois will grow without bounds and we'd eventually have to pay more in registration fees. Of course, that's just my figurin' and nothing official...
True, that's what his statement appears to say, however, who is to define "abuse" of the whois service? As an example, a company I worked for a while back wanted to generate, on their statistics reports for their customers' web sites, who each domain was who was hitting their page, and who it belonged to (e.g. someone looking just at "ora.com" might not correlate that to "O'Reilly and Associates"). What we had then asked InterNIC for was a means of getting that data WITHOUT using whois. (We knew it was readily available, and publicly accessible, but wanted to avoid beating on the whois server to get it when it came time to generate reports). The people we talked to at InterNIC essentially told us to pound salt. My superior at the time had mentioned that we had two ways of going about this, the "net-friendly" way, and the "brute-force" way, and that InterNIC was forcing us to use the brute force way which could cause their servers undue load. Was this company "an abuser" because they wanted to do lookups of a useful nature? (And yes, they put in caching and such so they wouldn't be querying every domain every time.) Granted, I would define someone harvesting the whois database for email addresses an abuser, but since I feel the abuse use had "valid purpose", that it shouldn't be categorized as abuse. I bet InterNIC would claim it was though. It's a customer service issue... We are all (I assume) customers of InterNIC. We pay them money for domain name registrations, and they are in turn supposed to provide reliable whois service (among other things, of course). If they're finding that load on the whois server is higher than they expect, they might consider (a) finding WHY it is that high? are there people like my old employer out there doing whois requests to get a single field from a number of sites, (b) allow others to volunteer to host the data and serve up requests. Just my $0.02 worth, everyone else's mileage is sure to vary. Derek
Was this company "an abuser" because they wanted to do lookups of a useful nature? (And yes, they put in caching and such so they wouldn't be querying
Yes. Thats not the intent of the whois database in my opinion. Marketing and general greed causes the most problems on the Internet and generates the most instances of someone trying to exploit something for the almighty dollar. -andy
Derek Balling wrote:
True, that's what his statement appears to say, however, who is to define "abuse" of the whois service?
As an example, a company I worked for a while back wanted to generate, on their statistics reports for their customers' web sites, who each domain was who was hitting their page, and who it belonged to (e.g. someone
Interesting point and an interesting application. I never thought of doing that. I guess I just always think in terms of domain names. You mean, there's a world outside of the Internet? =) Since we're on the topic, what *is* the process for getting access to the whois DB? Is this even possible at this day and age? Isn't there already a process in place to get full copies of the root zone files? Couldn't this be extended to the whois data? Heck, it's just another contract.... -- Michael L. Barrow * <mlbarrow@eni.net> * Network Engineer Epoch Internet * DSL Engineering * (949) 399-8413
On Wed, 2 Sep 1998, Michael L. Barrow wrote:
Since we're on the topic, what *is* the process for getting access to the whois DB?
Talk to NSI.
Is this even possible at this day and age?
Yes. /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ Patrick Greenwell (800) 299-1288 v Systems Administrator (925) 377-1212 v NameSecure (925) 377-1414 f Coming to the ISPF? The Forum for ISPs by ISPs http://www.ispf.com \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
I tried it a few times just to see if I could see anything and was inspired by the p-u-s-s-y-c-a-m.com that was found in the earlier query and tried smokinpussy.com. It returned 'Sorry, you shouldn't see this record'. Guess I should have showed my ID first... Sincerely, Andy Walden System Administrator MTCO Communications 1-800-859-6826
At 11:20 AM 09/02/1998 -0500, Derek Balling wrote:
<<..SNIP..>> It's a customer service issue... We are all (I assume) customers of InterNIC. We pay them money for domain name registrations, and they are in turn supposed to provide reliable whois service (among other things, of course). If they're finding that load on the whois server is higher than they expect, they might consider (a) finding WHY it is that high? are there people like my old employer out there doing whois requests to get a single field from a number of sites, (b) allow others to volunteer to host the data and serve up requests.
Why not use rwhois and distribute the load like DNS is? Let registerants decide if they want to provide rwhois support for their own domain(s) or if they want to delegate the task to the InterNIC to handle it. Gee... imagine being able to update data about your DNS domains immediately instead of having to wait for the InterNIC to process the request. Just my $0.02. -- Dan Watts Vitts Networks dwatts@vitts.com
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Michael L. Barrow Sent: Wednesday, September 02, 1998 11:32 AM To: Christopher K. Phillips Cc: David H. Holtzman; nanog@merit.edu Subject: Re: NSI Bulletin 098-010 | Update on Whois
Christopher K. Phillips wrote:
I may be wrong but aren't the fees that Network Solutions is charging for domain name registration "supposed" to cover the costs of providing WHOIS service for those domains? Is this just another attempt to nickel and dime everyone to death on some sort of new usage charge for WHOIS service?
Who said anything about usage charges? It looks like the proposed work mentioned by David Holtzman is to control _abuse_ of the whois service. Otherwise, the load on whois will grow without bounds and we'd eventually have to pay more in registration fees. Of course, that's just my figurin' and nothing official...
Why is SAIC hiring (may have hired by now) a marketing manager to market the whois database? Cheers, Alan
-- Michael L. Barrow * <mlbarrow@eni.net> * Network Engineer Epoch Internet * DSL Engineering * (949) 399-8413
participants (11)
-
Alan Sullivan - VoTiV Systems -
Andy Walden -
Christopher K. Phillips -
Dan Watts -
David H. Holtzman -
Derek Balling -
jzeeff＠verio.net -
Michael L. Barrow -
Patrick Greenwell -
Rich Sena -
Steven J. Sobol