Whois lookups (was: 2010.10.04 NANOG50 day 1 morning notes posted)
http://kestrel3.netflight.com/2010.10.04-NANOG50-morning-notes.txt " Whois traffic has been going through the roof; they added more proxies in front to support it. Apparently, there's IP management packages that do whois queries. It would be good to find out who is doing it, and talk to ARIN engineering, to find a better way of handling it. We can't keep up if so many machines on the internet keep doing it like this. Source addresses are all over, they're all over, not sign of bots; could be a DLL or mac system startup that's doing it. Please, don't embed whois lookups in everyone's computers like this!! " The only thing I know of is that packages like fail2ban that perform WHOIS lookups when blocking IPs to generate abuse POC notification emails. So more SSH bruteforce attacks = more whois lookups. Nathan
For those who might care, I've put version 1.0 of my notes from the morning session up at http://kestrel3.netflight.com/2010.10.04-NANOG50-morning-notes.txt
On 10/4/2010 10:05, Nathan Eisenberg wrote:
http://kestrel3.netflight.com/2010.10.04-NANOG50-morning-notes.txt
" Whois traffic has been going through the roof; they added more proxies in front to support it. Apparently, there's IP management packages that do whois queries. It would be good to find out who is doing it, and talk to ARIN engineering, to find a better way of handling it. We can't keep up if so many machines on the internet keep doing it like this. Source addresses are all over, they're all over, not sign of bots; could be a DLL or mac system startup that's doing it. Please, don't embed whois lookups in everyone's computers like this!! "
The only thing I know of is that packages like fail2ban that perform WHOIS lookups when blocking IPs to generate abuse POC notification emails. So more SSH bruteforce attacks = more whois lookups.
Or the new whois doesn't scale as well as the old one. ~Seth
On Oct 4, 2010, at 1:25 PM, Seth Mattinen wrote:
Or the new whois doesn't scale as well as the old one.
Seth - New WHOIS scales much better than the old one; it would have extremely challenging to assemble enough equipment to handle the current query rate. Look at the NANOG presentation slide for the exact query rate graph, but we're handling orders of magnitude more queries at present. /John
On Oct 4, 2010, at 9:58 AM, John Curran wrote:
Or the new whois doesn't scale as well as the old one. New WHOIS scales much better than the old one; it would have extremely challenging to assemble enough equipment to handle
On Oct 4, 2010, at 1:25 PM, Seth Mattinen wrote: the current query rate. Look at the NANOG presentation slide for the exact query rate graph, but we're handling orders of magnitude more queries at present.
Looking at the graph on your 3 slide, it looks like ARIN is getting around 3200 whois queries per second. How much of that query load is a result of non-port 43 queries (that is, making use of the REST features in the new server)? It looks like the exponentiation in query load started around the same time the Whois-RWS was deployed... Regards, -drc
On 10/4/10 4:58 PM, "David Conrad" <drc@virtualized.org> wrote:
On Oct 4, 2010, at 9:58 AM, John Curran wrote:
Or the new whois doesn't scale as well as the old one. New WHOIS scales much better than the old one; it would have extremely challenging to assemble enough equipment to handle
On Oct 4, 2010, at 1:25 PM, Seth Mattinen wrote: the current query rate. Look at the NANOG presentation slide for the exact query rate graph, but we're handling orders of magnitude more queries at present.
Looking at the graph on your 3 slide, it looks like ARIN is getting around 3200 whois queries per second. How much of that query load is a result of non-port 43 queries (that is, making use of the REST features in the new server)? It looks like the exponentiation in query load started around the same time the Whois-RWS was deployed...
Traffic increases a lot over the course of a day and follows a diurnal pattern. Right now we are seeing close to 7,000 queries per second during the height of the day. The original Whois cluster that Whois-RWS replaced could not serve more than 800 queries per second. There were two spikes. The first was right after we deployed Whois-RWS. For two months, we saw a consistent load maxing at 2400 queries per second. The second spike happened on Sept 6. At that point, traffic jumped almost 3x to the current max of 7,000 queries per second and has been pretty consistent over the past month. The patterns that we see are interesting. Most interesting is the spike asking for ip addresses login servers for the likes of Facebook, AOL, and Yahoo. This pattern emerged on Sept 6. Various people have been looking at this but no good explanation has yet been found. Your guess is good as mine what the cause of this query growth. Regards, Mark
participants (5)
-
David Conrad -
John Curran -
Mark Kosters -
Nathan Eisenberg -
Seth Mattinen