Completewhois New Features - RBL Lookup and Search Utilities
Hello everyone, Over the past month several new features and utilities have been added that are likely to be of interest here. In this post I'll focus on RBL lookup related utilities which have to do with rbl data from about 30 lists (with one or two exceptions, pretty much covers 25 most used free lists) that we're collecting and aggregating in the database (updates once per day) for analysis and allow users to check on based on individual queries and ip ranges. I. First of there is now direct whois lookup facility to check if ip address or domain is in one of those lists - you can simply do whois -h rbl.completewhois.com ip-address/domain OR (to show all lists that were checked) whois -h rbl.completewhois.com RBL_INCLUDENOMATCH=ON ip-address/domain OR (to include RBL data with normal whois lookup) whois -h whois.completewhois.com RBL ip-address/domain The queries are completed in average 1/2 second, so results are always fast. Note that no ip ranges are accepted (or going to be) on the whois interface. II. The web interface and light documentation for RBL Lookup (individual queries interface to our system) is available at http://www.completewhois.com/rbl_lookup.htm The website utility has two types of output display - one user-friendly (now default) table showing lists that matched and did not as red and green and including links to the list pages (good for less RBL-familiar users who want to know what to do) and simple format based on whois (can be easy to cut-paste from) and which is used when you want to also combine query with whois and dns data. The website lookup CGI can also be refernced directly (already is and used quite heavily) from other places and applications, do it as in this example: http://www.completewhois.com/cgi-bin/rbl_lookup.cgi?query=62.139.100.213 There is also real-time RBL check utility for 200 lists (not using our database and so quite slow) available on the bottom of the page and you even have a choice there to use several dns libraries (ADNS, FireDNS, BIND resolver) and compare how fast/slow they work... III. Another utility on the website allows to do IP range searches and is intended to be used primarily by ISPs and network operators to check on the listings that cover their own ip blocks (this is to help make operators aware of the extent of possible abuse coming from their network). The interface to this is available at: http://www.completewhois.com/rbl_search.htm The search utility is restricted to maximum /24 range as allowing more then that could in my opinion (and others I consulted) facilitate abuse rather then help stop it. To be able to do more then /24 lookup on your ip block(s), you will need to register and get username and password in our system (its still all free - registration is just making sure only ISP who is assigned the ip block can do query on entire block). Also note that use of this utility is covered by separate AUP. The results from ip range search also come in several formats, including simple list on the website, comprehensive webtable format as well as an option to produce CSV file for export to spreadsheet. The queries and searches are typically done in 1-2 seconds for /24 and about 4-8 seconds for /16 (with 1000 matches from various RBLs).Webtable adds additional couple seconds for large (500+) matches. In the future if there is an interest, further work will be done on the search interface ISP features to allow not only one-time lookups but reports that can be generated and sent automatically. Options will also include ability to do query based on specified time range (i.e. only new RBL entries that appeared in last 7 days). You will need to tell me what you want and how its to be presented, if you expect new features and note that any further work on this will be done end August or later when I come back from IETF conference. P.S. For those who like statistics, there are currently 1.8 million individual RBL entries which as an aggregate cover about 2 /8s (not all fair comparison because spews level2 covers large ip ranges where as many other lists are more specific). About 100 thousand (varies, very low on weekend but can be lot more some days) get updated every day and most active as far as updates is Spamhaus XBL. --- William Leibzon Elan Networks william@elan.net
participants (1)
-
william(at)elan.net