On Wed, 7 Sep 2005, Albert Meyer wrote:
I've been talking to ARIN about the rwhois setup on our SWIPped blocks, and there appears to be a problem with the standard output from whois.arin.net.
Be carefull about using word "standard", there is no standard output for whois plus both for ARIN and other RIRs output depends on the options used when queries. In this case, I think you wanted to say that the problem you see is with their default output.
The two rwhois clients I've tried are rwhois and jwhois.
jwhois is not an rwhois client. It does whois query on port 4321 but does not actually interpret that data as real rwhois client would have.
The rwhois client behavior is something like this: 1. Query whois.arin.net. 2a. If the response contains the name of an rwhois server, query that server and return its output. 2b. If the response doesn't contain the name of an rwhois server, follow the links. Query every rwhois server you find and return all of the output.
I have not tried rwhois as a client for a while, but I believe you're not describing it properly. Rwhois would do rwhois (port 4321) query on root.rwhois.net and would follow rwhois system referrals from there (which on the technical level is not the same as whois listed referral line). As of about year ago ARIN finally (after 4 or 5 years!) fixed those rwhois referrals (previously it would only work for very few blocks) so that rwhois server would follow to your own rwhois system. I do want to note though that rwhois output at arin is not alwas insync with their whois data (I think they generate it and once/day or less, where as whois is updated twice/day) and does not list any contact data - only organization name.
The jwhois client behavior is something like this:
1. Query whois.arin.net. 2a. If the response contains the name of an rwhois server, query that server and return its output. 2b. If the response doesn't contain the name of an rwhois server, return the SWIP.
That seems reasonable for simple whois system which does not want to go into all these complexities and rare cases of mixed SWIP and rwhois usage. BTW - Completewhois behavior is to: 1. Output swip data including every swip for multiple levels 2. Look at if there is rwhois referral ReferralServer in the swip (AND also look at if rwhois server is listed by name in the comments and try to use that as well even if ReferralServer is not present) and if so do rwhois lookup (using rwhois library but without following referrals) to listed server.
On blocks which are SWIPped to CoreNAP by an upstream provider, the response from whois.arin.net does not include an rwhois record. For example, if I type:
whois -h whois.arin.net 65.59.252.0
You might want to try whois -h whois.arin.net "+ 65.59.252.0"
The whois server returns this:
Level 3 Communications, Inc. LC-ORG-ARIN-BLK2 (NET-65-56-0-0-1) 65.56.0.0 - 65.59.255.255 Core NAP, L.P. LVLT-CORENAP-NETBLOCK-03 (NET-65-59-252-0-1) 65.59.252.0 - 65.59.252.255 VC Sterling, Inc. NET-65-59-252-0-1 (NET-65-59-252-0-2) 65.59.252.0 - 65.59.252.255
Since there is no rwhois server listed here, rwhois clients don't necessarily manage to find the referral. rwhois apparently follows both links and returns results from every rwhois server it finds, but jwhois doesn't follow either link; it just returns the SWIP info. I believe that the correct response to this query would be:
That is a problem with jwhois and not with default arin output which is very simplistic and lists only list of swips without detail on each swip. Jwhois should query the detail or use extended query ("+") to find all the relevent information. BTW, compare this to the output given by completewhois: whois -h whois.completewhois.com 65.59.252.1 (saved in http://www.completewhois.com/cgi-bin/whois.cgi?query=R33857865) As you notice it tried to do query to both rwhois.level3.net and to rwhois.corenap.com as well as displayed SWIP data. While admitedly its too much information, that is how the system works - it provides as much data as possible and lets user decide on what is relevent to him/her. But by automated means this really is a good example of how to confuse the ip whois client, because you have 2 rwhois servers listed (for top-most ip block and 2nd level deligation block) as well as SWIPs all the way to the assignment to customer. If you want automated systems to use rwhois, then query to your own server would never have been made (the referral to follow would be rwhois.level3.net), actually let me correct myself - if rwhois.level3.net were to run rwhois server accessible to public, the correct rwhois-only setup would be for L3 to enter rwhois referral to rwhois.corenap.com on their server (like ARIN does on root.rwhois.net). In this case client would find top-most rwhois server as rwhois.level3.net and follow from there and get to the lowest rwhois server available and give the results. But if you don't want automated systems to use rwhois data as primary, then the correct is to output the lowest swip level data in detail, i.e. whois -h whois.arin.net NET-65-59-252-0-1
Level 3 Communications, Inc. LC-ORG-ARIN-BLK2 (NET-65-56-0-0-1) 65.56.0.0 - 65.59.255.255 ReferralServer: rwhois://rwhois.level3.net:4321 Core NAP, L.P. LVLT-CORENAP-NETBLOCK-03 (NET-65-59-252-0-1) 65.59.252.0 - 65.59.252.255 ReferralServer: rwhois://rwhois.corenap.com:4321/ VC Sterling, Inc. NET-65-59-252-0-1 (NET-65-59-252-0-2) 65.59.252.0 - 65.59.252.255
I'd be against any change like this because it will severely break a lot of currently whois client software that know about this ARIN list data and this would not be compatible with it.
I've read through the apparently relevant RFCs (812, 954, 1714, 1834, 1835, 1913, 1914, 2050, 2167, 3912) but did not find a clear specification of correct WHOIS server output.
You're correct. Whois protocol has no standards for output format. There were however several attempts to do that on top of whois with "whois+" and "rwhois" as well as RPSL.
The people I talked to at ARIN say that the configuration of whois.arin.net can be changed based on "significant community consensus" but they suggested that the problem could be fixed by rewriting the jwhois client (and any other client that doesn't follow links to search for an rwhois server).
I agree with ARIN's assessment. But if I were doing coding for jwhois I would not do it as you suggest and would insted write code that can output the lowest level ARIN deligation (follow the lowest swip in the list when its returned as above) and then look at that data and see if that lowest swip has listed ReferralServer and follow that. In your case it would not cause lookup to rwhois.corenap.com, but as I said that is very very rare case that causes a lot of confusion for pretty much every whois client except completewhois.
I spent a fair amount of time looking through the (apparently non-searchable) mailing list archive at http://lists.arin.net/pipermail/dbwg/ and saw some discussion of rwhois issues but I didn't manage to find information showing how the previous change was initiated.
ARIN formed subgroup within DBWG to discuss rwhois issues about 3 years ago.
Questions: 1. Does anyone agree that the present lack of rwhois server information in the initial WHOIS response for SWIPped blocks is a problem?
My general feeling is that ARIN is wrong in providing default list response rather then detailed data. But as far as rwhois not being listed in default list response, I do not believe it to be appropriate either.
2. Can anyone think of a compelling reason why rwhois server information should not be included in the initial response to a standard whois query for all IP blocks, including SWIPped blocks, besides the fact that it is not included now?
Yes, I can - see above.
3. Would this change (adding rwhois server information to the initial response to a standard whois query for SWIPped blocks) break your scripts that parse WHOIS output?
Yes, it would.
4. How disruptive was the change when rwhois server information was initially added to WHOIS output?
Not disruptive because it was done to detailed data output and everyone expected data there to be multiple "Field: data" lines and that new fields could be added in the future.
5. Was the issue fully thought through at that time, and the rwhois server information intentionally left out of the initial response for SWIPped blocks, or did this happen by accident?
I can't speak for ARIN, but I think they fully realized that they are adding ReferralServer line only for detailed output and not list/summary.
6. Does anyone know where that change process was documented?
Ask ARIN, I don't think it is documented though. -- William Leibzon Elan Networks william@elan.net