I'm in a disagreement with ARIN about my application for bulk whois data. I've got a software program that needs resolve AS numbers to the Company Name of the owner. The software app has need to do this on a very high volume. E.g. I run a report that returns the top 100 AS destinations for my network and I want to resolve the numbers to the names as part of the report generation. Since ARIN throttles the number of queries that you can execute against their servers I seems to "just make sense" that you would do the processing using local data. That is all fine and good, but the problem comes when I distribute the software to users. ARIN's AUP for bulk whois states: "Redistributing bulk ARIN WHOIS data is explicitly forbidden. It is permissible to publish the data on an individual query or small number of queries at a time basis, as long as reasonable precautions are taken to prevent automated querying by database harvesters." My original AUP application stated that I would transfer the data to the users using an XML file on a regular basis. Clearly in violation of the first point. Fine. But now after a phone conversation they are telling me that I can not operate a server to distribute the data on a "per query" basis too. Providing a server that answers whois queries just like ARIN seems to be clearly permissible based on the remaining AUP verbage. At this point the only thing I can get out of the guy/gal on the phone is "NO!". Does anyone have any experience doing something like this? How about a sanity check? Am I completely wrong in how I'm interpreting the AUP? Thanks, Dan
How about caching the data from previous ARIN whois lookups? I do agree that the bulk data and high volume limitations on whois servers are silly... -joe On 2/28/05 1:30 PM, "Dan Lockwood" <dlockwood@shastacoe.org> wrote:
I'm in a disagreement with ARIN about my application for bulk whois data. I've got a software program that needs resolve AS numbers to the Company Name of the owner. The software app has need to do this on a very high volume. E.g. I run a report that returns the top 100 AS destinations for my network and I want to resolve the numbers to the names as part of the report generation. Since ARIN throttles the number of queries that you can execute against their servers I seems to "just make sense" that you would do the processing using local data.
That is all fine and good, but the problem comes when I distribute the software to users. ARIN's AUP for bulk whois states:
"Redistributing bulk ARIN WHOIS data is explicitly forbidden. It is permissible to publish the data on an individual query or small number of queries at a time basis, as long as reasonable precautions are taken to prevent automated querying by database harvesters."
My original AUP application stated that I would transfer the data to the users using an XML file on a regular basis. Clearly in violation of the first point. Fine. But now after a phone conversation they are telling me that I can not operate a server to distribute the data on a "per query" basis too. Providing a server that answers whois queries just like ARIN seems to be clearly permissible based on the remaining AUP verbage. At this point the only thing I can get out of the guy/gal on the phone is "NO!".
Does anyone have any experience doing something like this? How about a sanity check? Am I completely wrong in how I'm interpreting the AUP?
Thanks, Dan
-- Joe McGuckin ViaNet Communications 994 San Antonio Road Palo Alto, CA 94303 Phone: 650-213-1302 Cell: 650-207-0372 Fax: 650-969-2124
altho arguably its not up to arin to provide processing power for all these deployments. if you can get a local copy why not have your clients resolve back to that? Steve On Tue, 1 Mar 2005, joe mcguckin wrote:
How about caching the data from previous ARIN whois lookups?
I do agree that the bulk data and high volume limitations on whois servers are silly...
-joe
On 2/28/05 1:30 PM, "Dan Lockwood" <dlockwood@shastacoe.org> wrote:
I'm in a disagreement with ARIN about my application for bulk whois data. I've got a software program that needs resolve AS numbers to the Company Name of the owner. The software app has need to do this on a very high volume. E.g. I run a report that returns the top 100 AS destinations for my network and I want to resolve the numbers to the names as part of the report generation. Since ARIN throttles the number of queries that you can execute against their servers I seems to "just make sense" that you would do the processing using local data.
That is all fine and good, but the problem comes when I distribute the software to users. ARIN's AUP for bulk whois states:
"Redistributing bulk ARIN WHOIS data is explicitly forbidden. It is permissible to publish the data on an individual query or small number of queries at a time basis, as long as reasonable precautions are taken to prevent automated querying by database harvesters."
My original AUP application stated that I would transfer the data to the users using an XML file on a regular basis. Clearly in violation of the first point. Fine. But now after a phone conversation they are telling me that I can not operate a server to distribute the data on a "per query" basis too. Providing a server that answers whois queries just like ARIN seems to be clearly permissible based on the remaining AUP verbage. At this point the only thing I can get out of the guy/gal on the phone is "NO!".
Does anyone have any experience doing something like this? How about a sanity check? Am I completely wrong in how I'm interpreting the AUP?
Thanks, Dan
----- Original Message ----- From: "Stephen J. Wilcox" <steve@telecomplete.co.uk> To: "joe mcguckin" <joe@via.net> Cc: "Dan Lockwood" <dlockwood@shastacoe.org>; "NANOG" <nanog@merit.edu> Sent: Tuesday, March 01, 2005 4:53 AM Subject: Re: High volume WHOIS queries
altho arguably its not up to arin to provide processing power for all
deployments.
if you can get a local copy why not have your clients resolve back to
these that? that is the point of his post actually - arin told him that he can't do that without pointing out where this is prohibited in the aup. i can see their point - they're trying to restrict the practicality of attempting to harvest the data and an open to the public whois server with no access restrictions would defeat that. perhaps asking arin if they would consent to you running a server open to registered users of your app behind authentication of some sort is worth a try? -p --- paul galynin
On 2/28/05 1:30 PM, "Dan Lockwood" <dlockwood@shastacoe.org> wrote:
I'm in a disagreement with ARIN about my application for bulk whois data. I've got a software program that needs resolve AS numbers to the Company Name of the owner. The software app has need to do this on a very high volume. E.g. I run a report that returns the top 100 AS destinations for my network and I want to resolve the numbers to the names as part of the report generation. Since ARIN throttles the number of queries that you can execute against their servers I seems to "just make sense" that you would do the processing using local data.
Not exactly what you are looking for, but how about ftp://ftp.arin.net/info/asn.txt Lists AS number, the whois AS name, and POC handle for each AS. Jeff -- Jeff Bartig | University of Wisconsin - Madison jeffb@doit.wisc.edu | Division of Information Technology (608) 262-8336 | Network Services/WAN Engineering
On Tue, 2005-03-01 at 16:40 -0600, Jeff Bartig wrote:
On 2/28/05 1:30 PM, "Dan Lockwood" <dlockwood@shastacoe.org> wrote:
I'm in a disagreement with ARIN about my application for bulk whois data. I've got a software program that needs resolve AS numbers to the Company Name of the owner. The software app has need to do this on a very high volume. E.g. I run a report that returns the top 100 AS destinations for my network and I want to resolve the numbers to the names as part of the report generation. Since ARIN throttles the number of queries that you can execute against their servers I seems to "just make sense" that you would do the processing using local data.
Not exactly what you are looking for, but how about
ftp://ftp.arin.net/info/asn.txt
Lists AS number, the whois AS name, and POC handle for each AS.
Jeff
If you are also interested in AS info outside the ARIN region, the following file may be of interest -- http://bgp.potaroo.net/as1221/asnames.txt -Larry
At 06:00 PM 01-03-05 -0500, Larry J. Blunk wrote:
ftp://ftp.arin.net/info/asn.txt
Lists AS number, the whois AS name, and POC handle for each AS.
Jeff
If you are also interested in AS info outside the ARIN region, the following file may be of interest --
Or this: http://netgeo.caida.org/aslatlong.txt -Hank
At 03:01 AM 3/2/2005, Hank Nussbacher wrote:
At 06:00 PM 01-03-05 -0500, Larry J. Blunk wrote:
ftp://ftp.arin.net/info/asn.txt
Lists AS number, the whois AS name, and POC handle for each AS.
Jeff
If you are also interested in AS info outside the ARIN region, the following file may be of interest --
Interestingly, the Caida data lacks AS numbers issued in the last several months, while the data is present in the arin.net and potaroo.net listings. Not sure where Caida gets its data, but the other sources would seem to be better for the project the original poster had in mind. It's also more than a bit odd, IMO, to assign a latitude and longitude to AS numbers. While some networks are well localized, many others span continents. As such, the value of that part of the caida data set seems nebulous.
participants (8)
-
Dan Lockwood -
Daniel Senie -
Hank Nussbacher -
Jeff Bartig -
joe mcguckin -
Larry J. Blunk -
Paul G -
Stephen J. Wilcox