On Thu, 20 Jan 2005, Suresh Ramasubramanian wrote:
Inktomi (now Yahoo!) sends it's spiders all over the Internet. Lately some of our systems are reporting that they open many HTTP connections to our web sites, without ever sending any data and immediately disconnecting. This is getting to a level where it disturbs us. I have heard previous stories of inktomi ignoring robots.txt (not seen
On Thu, 20 Jan 2005 14:30:04 +0200, Gadi Evron <gadi@tehila.gov.il> wrote: this for myself though). And there are threads like this - Quoting from http://www.webmasterworld.com/forum11/1968-1-15.htm
back in 1999 inktomi hammered our nameserver (which never has, and never will run http. ever.) After _weeks_ of complaining to them and to their upstream exodus (hah!) I finally got them to stop. Only to have them start up again a month later. not suprising to see them up to their old antics again. time to nullroute i guess? -Dan