RE: High volume WHOIS queries
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Paul G Sent: Tuesday, March 01, 2005 5:03 AM To: nanog@merit.edu Subject: Re: High volume WHOIS queries
----- Original Message ----- From: "Stephen J. Wilcox" <steve@telecomplete.co.uk> To: "joe mcguckin" <joe@via.net> Cc: "Dan Lockwood" <dlockwood@shastacoe.org>; "NANOG" <nanog@merit.edu> Sent: Tuesday, March 01, 2005 4:53 AM Subject: Re: High volume WHOIS queries
altho arguably its not up to arin to provide processing
deployments.
if you can get a local copy why not have your clients resolve back to
power for all these that?
that is the point of his post actually - arin told him that he can't do that without pointing out where this is prohibited in the aup. i can see their point - they're trying to restrict the practicality of attempting to harvest the data and an open to the public whois server with no access restrictions would defeat that.
I don't know that this is the case, I suspect it's resource management. If the database is getting slaughtered by applications on uncontrolled auto pilot, it's unusable for the rest of us. -M<
----- Original Message ----- From: "Hannigan, Martin" <hannigan@verisign.com> To: "Paul G" <paul@rusko.us>; <nanog@merit.edu> Sent: Tuesday, March 01, 2005 9:17 AM Subject: RE: High volume WHOIS queries
-----Original Message----- From: owner-nanog@merit.edu [mailto:owner-nanog@merit.edu]On Behalf Of Paul G Sent: Tuesday, March 01, 2005 5:03 AM To: nanog@merit.edu Subject: Re: High volume WHOIS queries
--- snip ---
point - they're trying to restrict the practicality of attempting to harvest the data and an open to the public whois server with no access restrictions would defeat that.
I don't know that this is the case, I suspect it's resource management. If the database is getting slaughtered by applications on uncontrolled auto pilot, it's unusable for the rest of us.
well, the OP quoted a portion of the aup that requires bulk whois data recipients to take measures to prevent harvesting, so i presume that arin does care about that and, in fact, that consideration is likely the reason they declined to permit the OP to run *his own* whoisd off of his *local* copy of the data. -p --- paul galynin
On Tue, 1 Mar 2005, Paul G wrote:
point - they're trying to restrict the practicality of attempting to harvest the data and an open to the public whois server with no access restrictions would defeat that.
I don't know that this is the case, I suspect it's resource management. If the database is getting slaughtered by applications on uncontrolled auto pilot, it's unusable for the rest of us.
well, the OP quoted a portion of the aup that requires bulk whois data recipients to take measures to prevent harvesting, so i presume that arin does care about that and, in fact, that consideration is likely the reason they declined to permit the OP to run *his own* whoisd off of his *local* copy of the data.
If memory serves, that restriction didn't appear until spam became a problem. The verbiage in the AUP is there to give ARIN recourse in the event that some spammer, and it has happened, runs a harvest against domain names or serialized NIC handles to seed a spam source. - billn
On Tue, Mar 01, 2005 at 09:17:48AM -0500, Hannigan, Martin wrote:
I don't know that this is the case, I suspect it's resource management. If the database is getting slaughtered by applications on uncontrolled auto pilot, it's unusable for the rest of us.
Understood. So why not make it easy -- both for yourselves and for everyone else? Just publish all WHOIS data on static web pages -- not even marked up with HTML, just plain ASCII text -- whose URLs are easy to construct, a la www.verisign.com/foo/bar/blah/example1.com www.verisign.com/foo/bar/blah/example2.net and refresh them from backing store whenever the "real" data changes. (And yes, I realize I'm using an example based on domains, not networks, but I trust it's still applicable.) This makes the load on the servers about as small as it's going to get. (Heck, they could be served from a cut-down web server designed to serve static content only.) It also makes it trivially easy for people to look things up without worrying about rate-limiting. Heck, once the search engines indexed it, it'd be even easier. As to "...then the spammers will mass-harvest it...": they already HAVE. They're busy selling it to each other on CD/DVD and via other means. This has been going on for years, and however-they're-doing-it, they're doing it well enough to acquire recently-modified data. So that toothpaste is completely out of the tube and there's no way to put it back in. I don't think any substantive purpose is served by pretending/wishing that it's otherwise: there's a demand for this data, and plenty of money to be made by those who will supply it, therefore it's going to be acquired and sold. But the people who *can't* access the data -- not without taking measures to evade the rate-blocking that's in place -- are abuse victims who are trying to track down those responsible. So I view the problem of overload on WHOIS servers as self-inflicted damage, easily fixed by giving up the pretense that restricting access to the data has any real value for anyone. (Well, it *does* benefit those selling it, but I trust that ensuring their profits isn't a goal that anyone's particularly worried about. ;-) ) ---Rsk
participants (4)
-
Bill Nash -
Hannigan, Martin -
Paul G -
Rich Kulawiec