Correct inclusion of rwhois info in WHOIS server output?
I've been talking to ARIN about the rwhois setup on our SWIPped blocks, and there appears to be a problem with the standard output from whois.arin.net. The two rwhois clients I've tried are rwhois and jwhois. The rwhois client behavior is something like this: 1. Query whois.arin.net. 2a. If the response contains the name of an rwhois server, query that server and return its output. 2b. If the response doesn't contain the name of an rwhois server, follow the links. Query every rwhois server you find and return all of the output. The jwhois client behavior is something like this: 1. Query whois.arin.net. 2a. If the response contains the name of an rwhois server, query that server and return its output. 2b. If the response doesn't contain the name of an rwhois server, return the SWIP. On blocks which are owned by CoreNAP, that works fine. For example, if I type: whois -h whois.arin.net 66.219.44.0 The whois server returns our complete SWIP record including: ReferralServer: rwhois://rwhois.corenap.com:4321/ So this block works fine with both jwhois and rwhois: bash-2.05$ jwhois 66.219.44.0 [Querying whois.arin.net] [Redirected to rwhois.corenap.com:4321] [Querying rwhois.corenap.com] [rwhois.corenap.com] %rwhois V-1.5:003fff:00 cache02.ns.corenap.com (by Network Solutions, Inc. V-1.5.7.3) network:Auth-Area:66.219.32.0/19 ... On blocks which are SWIPped to CoreNAP by an upstream provider, the response from whois.arin.net does not include an rwhois record. For example, if I type: whois -h whois.arin.net 65.59.252.0 The whois server returns this: Level 3 Communications, Inc. LC-ORG-ARIN-BLK2 (NET-65-56-0-0-1) 65.56.0.0 - 65.59.255.255 Core NAP, L.P. LVLT-CORENAP-NETBLOCK-03 (NET-65-59-252-0-1) 65.59.252.0 - 65.59.252.255 VC Sterling, Inc. NET-65-59-252-0-1 (NET-65-59-252-0-2) 65.59.252.0 - 65.59.252.255 Since there is no rwhois server listed here, rwhois clients don't necessarily manage to find the referral. rwhois apparently follows both links and returns results from every rwhois server it finds, but jwhois doesn't follow either link; it just returns the SWIP info. I believe that the correct response to this query would be: Level 3 Communications, Inc. LC-ORG-ARIN-BLK2 (NET-65-56-0-0-1) 65.56.0.0 - 65.59.255.255 ReferralServer: rwhois://rwhois.level3.net:4321 Core NAP, L.P. LVLT-CORENAP-NETBLOCK-03 (NET-65-59-252-0-1) 65.59.252.0 - 65.59.252.255 ReferralServer: rwhois://rwhois.corenap.com:4321/ VC Sterling, Inc. NET-65-59-252-0-1 (NET-65-59-252-0-2) 65.59.252.0 - 65.59.252.255 I've read through the apparently relevant RFCs (812, 954, 1714, 1834, 1835, 1913, 1914, 2050, 2167, 3912) but did not find a clear specification of correct WHOIS server output. The people I talked to at ARIN say that the configuration of whois.arin.net can be changed based on "significant community consensus" but they suggested that the problem could be fixed by rewriting the jwhois client (and any other client that doesn't follow links to search for an rwhois server). I spent a fair amount of time looking through the (apparently non-searchable) mailing list archive at http://lists.arin.net/pipermail/dbwg/ and saw some discussion of rwhois issues but I didn't manage to find information showing how the previous change was initiated. Questions: 1. Does anyone agree that the present lack of rwhois server information in the initial WHOIS response for SWIPped blocks is a problem? 2. Can anyone think of a compelling reason why rwhois server information should not be included in the initial response to a standard whois query for all IP blocks, including SWIPped blocks, besides the fact that it is not included now? 3. Would this change (adding rwhois server information to the initial response to a standard whois query for SWIPped blocks) break your scripts that parse WHOIS output? 4. How disruptive was the change when rwhois server information was initially added to WHOIS output? 5. Was the issue fully thought through at that time, and the rwhois server information intentionally left out of the initial response for SWIPped blocks, or did this happen by accident? 6. Does anyone know where that change process was documented?
On Wed, 7 Sep 2005, Albert Meyer wrote:
1. Does anyone agree that the present lack of rwhois server information in the initial WHOIS response for SWIPped blocks is a problem?
I think the real problem here is that jwhois is preferring ARIN's whois service rather than their rwhois service, because the human-readable whois output looks nicer. Oh, and another possibility is that for a very long time, ARIN's rwhois server was badly borken and just redirected back to IANA (oops!). (It appears to be working now as of this writing.) If jwhois instead understood rwhois protocol (it does not currently), it would have no problem, because ARIN has referrals all the way down to you for the network you posted (65.59.252.x). That is, of course, the whole point of using the structured rwhois protocol in the first place. :) -- -- Todd Vierling <tv@duh.org> <tv@pobox.com> <todd@vierling.name>
On Wed, 7 Sep 2005, Albert Meyer wrote:
I've been talking to ARIN about the rwhois setup on our SWIPped blocks, and there appears to be a problem with the standard output from whois.arin.net.
Be carefull about using word "standard", there is no standard output for whois plus both for ARIN and other RIRs output depends on the options used when queries. In this case, I think you wanted to say that the problem you see is with their default output.
The two rwhois clients I've tried are rwhois and jwhois.
jwhois is not an rwhois client. It does whois query on port 4321 but does not actually interpret that data as real rwhois client would have.
The rwhois client behavior is something like this: 1. Query whois.arin.net. 2a. If the response contains the name of an rwhois server, query that server and return its output. 2b. If the response doesn't contain the name of an rwhois server, follow the links. Query every rwhois server you find and return all of the output.
I have not tried rwhois as a client for a while, but I believe you're not describing it properly. Rwhois would do rwhois (port 4321) query on root.rwhois.net and would follow rwhois system referrals from there (which on the technical level is not the same as whois listed referral line). As of about year ago ARIN finally (after 4 or 5 years!) fixed those rwhois referrals (previously it would only work for very few blocks) so that rwhois server would follow to your own rwhois system. I do want to note though that rwhois output at arin is not alwas insync with their whois data (I think they generate it and once/day or less, where as whois is updated twice/day) and does not list any contact data - only organization name.
The jwhois client behavior is something like this:
1. Query whois.arin.net. 2a. If the response contains the name of an rwhois server, query that server and return its output. 2b. If the response doesn't contain the name of an rwhois server, return the SWIP.
That seems reasonable for simple whois system which does not want to go into all these complexities and rare cases of mixed SWIP and rwhois usage. BTW - Completewhois behavior is to: 1. Output swip data including every swip for multiple levels 2. Look at if there is rwhois referral ReferralServer in the swip (AND also look at if rwhois server is listed by name in the comments and try to use that as well even if ReferralServer is not present) and if so do rwhois lookup (using rwhois library but without following referrals) to listed server.
On blocks which are SWIPped to CoreNAP by an upstream provider, the response from whois.arin.net does not include an rwhois record. For example, if I type:
whois -h whois.arin.net 65.59.252.0
You might want to try whois -h whois.arin.net "+ 65.59.252.0"
The whois server returns this:
Level 3 Communications, Inc. LC-ORG-ARIN-BLK2 (NET-65-56-0-0-1) 65.56.0.0 - 65.59.255.255 Core NAP, L.P. LVLT-CORENAP-NETBLOCK-03 (NET-65-59-252-0-1) 65.59.252.0 - 65.59.252.255 VC Sterling, Inc. NET-65-59-252-0-1 (NET-65-59-252-0-2) 65.59.252.0 - 65.59.252.255
Since there is no rwhois server listed here, rwhois clients don't necessarily manage to find the referral. rwhois apparently follows both links and returns results from every rwhois server it finds, but jwhois doesn't follow either link; it just returns the SWIP info. I believe that the correct response to this query would be:
That is a problem with jwhois and not with default arin output which is very simplistic and lists only list of swips without detail on each swip. Jwhois should query the detail or use extended query ("+") to find all the relevent information. BTW, compare this to the output given by completewhois: whois -h whois.completewhois.com 65.59.252.1 (saved in http://www.completewhois.com/cgi-bin/whois.cgi?query=R33857865) As you notice it tried to do query to both rwhois.level3.net and to rwhois.corenap.com as well as displayed SWIP data. While admitedly its too much information, that is how the system works - it provides as much data as possible and lets user decide on what is relevent to him/her. But by automated means this really is a good example of how to confuse the ip whois client, because you have 2 rwhois servers listed (for top-most ip block and 2nd level deligation block) as well as SWIPs all the way to the assignment to customer. If you want automated systems to use rwhois, then query to your own server would never have been made (the referral to follow would be rwhois.level3.net), actually let me correct myself - if rwhois.level3.net were to run rwhois server accessible to public, the correct rwhois-only setup would be for L3 to enter rwhois referral to rwhois.corenap.com on their server (like ARIN does on root.rwhois.net). In this case client would find top-most rwhois server as rwhois.level3.net and follow from there and get to the lowest rwhois server available and give the results. But if you don't want automated systems to use rwhois data as primary, then the correct is to output the lowest swip level data in detail, i.e. whois -h whois.arin.net NET-65-59-252-0-1
Level 3 Communications, Inc. LC-ORG-ARIN-BLK2 (NET-65-56-0-0-1) 65.56.0.0 - 65.59.255.255 ReferralServer: rwhois://rwhois.level3.net:4321 Core NAP, L.P. LVLT-CORENAP-NETBLOCK-03 (NET-65-59-252-0-1) 65.59.252.0 - 65.59.252.255 ReferralServer: rwhois://rwhois.corenap.com:4321/ VC Sterling, Inc. NET-65-59-252-0-1 (NET-65-59-252-0-2) 65.59.252.0 - 65.59.252.255
I'd be against any change like this because it will severely break a lot of currently whois client software that know about this ARIN list data and this would not be compatible with it.
I've read through the apparently relevant RFCs (812, 954, 1714, 1834, 1835, 1913, 1914, 2050, 2167, 3912) but did not find a clear specification of correct WHOIS server output.
You're correct. Whois protocol has no standards for output format. There were however several attempts to do that on top of whois with "whois+" and "rwhois" as well as RPSL.
The people I talked to at ARIN say that the configuration of whois.arin.net can be changed based on "significant community consensus" but they suggested that the problem could be fixed by rewriting the jwhois client (and any other client that doesn't follow links to search for an rwhois server).
I agree with ARIN's assessment. But if I were doing coding for jwhois I would not do it as you suggest and would insted write code that can output the lowest level ARIN deligation (follow the lowest swip in the list when its returned as above) and then look at that data and see if that lowest swip has listed ReferralServer and follow that. In your case it would not cause lookup to rwhois.corenap.com, but as I said that is very very rare case that causes a lot of confusion for pretty much every whois client except completewhois.
I spent a fair amount of time looking through the (apparently non-searchable) mailing list archive at http://lists.arin.net/pipermail/dbwg/ and saw some discussion of rwhois issues but I didn't manage to find information showing how the previous change was initiated.
ARIN formed subgroup within DBWG to discuss rwhois issues about 3 years ago.
Questions: 1. Does anyone agree that the present lack of rwhois server information in the initial WHOIS response for SWIPped blocks is a problem?
My general feeling is that ARIN is wrong in providing default list response rather then detailed data. But as far as rwhois not being listed in default list response, I do not believe it to be appropriate either.
2. Can anyone think of a compelling reason why rwhois server information should not be included in the initial response to a standard whois query for all IP blocks, including SWIPped blocks, besides the fact that it is not included now?
Yes, I can - see above.
3. Would this change (adding rwhois server information to the initial response to a standard whois query for SWIPped blocks) break your scripts that parse WHOIS output?
Yes, it would.
4. How disruptive was the change when rwhois server information was initially added to WHOIS output?
Not disruptive because it was done to detailed data output and everyone expected data there to be multiple "Field: data" lines and that new fields could be added in the future.
5. Was the issue fully thought through at that time, and the rwhois server information intentionally left out of the initial response for SWIPped blocks, or did this happen by accident?
I can't speak for ARIN, but I think they fully realized that they are adding ReferralServer line only for detailed output and not list/summary.
6. Does anyone know where that change process was documented?
Ask ARIN, I don't think it is documented though. -- William Leibzon Elan Networks william@elan.net
On 08/09/05, william(at)elan.net <william@elan.net> wrote:
Be carefull about using word "standard", there is no standard output for whois plus both for ARIN and other RIRs output depends on the options used when queries. In this case, I think you wanted to say that the problem you see is with their default output.
It would be SUCH a good thing if there was some move to standardize these There is, of course, CRISP ..
Be carefull about using word "standard", there is no standard output for whois plus both for ARIN and other RIRs output depends on the options used when queries. In this case, I think you wanted to say that the problem you see is with their default output.
It would be SUCH a good thing if there was some move to standardize
these
There is, of course, CRISP ..
CRISP is merely smoke and mirrors at present. There is the whois format used by all the other RIRs which is, I believe, based on RPSL. RIPE even makes their server freely available. Assuming that there is some kind of database behind SWIP and whois, it should not be that difficult for ARIN to switch to using the RIPE format for displaying answers to whois queries. --Michael Dillon
On Thu, 8 Sep 2005 Michael.Dillon@btradianz.com wrote:
There is the whois format used by all the other RIRs which is, I believe, based on RPSL. RIPE even makes their server freely available. Assuming that there is some kind of database behind SWIP and whois, it should not be that difficult for ARIN to switch to using the RIPE format for displaying answers to whois queries.
There is a long and occasionally sordid tale behind the history of various WHOIS server output formats. However, for ARIN's server specifically, assuming you want to change anything, you want to be making comments on ARIN's dbwg mailing list; see www.arin.net . But in brief, RIPE NCC started up, developed their own. APNIC started up, lacked resources, used the RIPE NCC-developed server. ARIN started up some years later, inherited NSI's server + database, and kept on using it. LACNIC started up, and modified the .BR registry's WHOIS server to provide RPSL-like output. AFRINIC started up, and used the RIPE NCC-developed server. No-one was really interested in rwhois for ages ;) -- Bruce Campbell
On Sep 8, 2005, at 6:05 AM, Michael.Dillon@btradianz.com wrote:
CRISP is merely smoke and mirrors at present.
Here's a repost of a message I sent recently to the ARIN ppml on this subject: ============================== Here's an update of what is going on in this space. From the standardization point of view: - the AREG draft went through a number of revisions and finally passed working group last call at revision -12. It is in the hands of the IESG now. - two transfer protocols have been in the works to replace BEEP. These are LWZ (UDP) and XPC (TCP). Both should be much, much easier to implement than BEEP. Of course, drafts and RFCs are not much use by themselves, so from the implementation point of view: - At RIPE 48 last year, I demoed IRISServ and Pimmit (a client) running AREG -06. I've been waiting for the AREG spec to settle down before updating them to the latest. I have been intending to demo this software at a NANOG meeting, but the timing never seems to work out. - At this past IETF, I had discussions with the RIPE NCC engineers regarding re-use of either IRISServ or Flowerbed (both are servers, they just have different architectures) for their systems. Their long-term strategy is to build something complete from the ground up, but they wanted something they could use in the near-term as a prototype. - Also at this past IETF, I spent a couple of hours with the LACNIC engineers doing some interoperability testing. I just spent the last week putting my code in a more useable state to further that work. =============================================== -andy
Thanks to everyone who replied on and off-list. I'm concluding that there is a problem with WHOIS server output, caused mostly by a lack of standards, but people with more influence than me are already working on fixing that. In the meantime I'll see if I talk to the gnu maintainer about making jwhois more rwhois-friendly.
Marco d'Itri's whois client does quite a lot of what you want, including automatically following an rwhois referral when it finds one Of course not much can be done when the rwhois server to which you're redirected just doesn't respond suresh@etherkiller 11:33:29 <~> $ telnet rwhois.level3.net 4321 Trying 209.244.1.179... srs On 09/09/05, Albert Meyer <from_nanog@corenap.com> wrote:
Thanks to everyone who replied on and off-list. I'm concluding that there is a problem with WHOIS server output, caused mostly by a lack of standards, but people with more influence than me are already working on fixing that. In the meantime I'll see if I talk to the gnu maintainer about making jwhois more rwhois-friendly.
-- Suresh Ramasubramanian (ops.lists@gmail.com)
participants (7)
-
Albert Meyer -
Andrew Newton -
Bruce Campbell -
Michael.Dillonļ¼ btradianz.com -
Suresh Ramasubramanian -
Todd Vierling -
william(at)elan.net