-----Original Message----- Message: 3 Date: Wed, 8 Sep 2010 10:24:38 +0000 From: Nathan Eisenberg <nathan@atlasnetworks.us> Subject: RE: yahoo crawlers hammering us To: "nanog@nanog.org" <nanog@nanog.org> Message-ID: <8C26A4FDAE599041A13EB499117D3C28164AC3A7@ex-mb- 1.corp.atlasnetworks.us>

Customers don't want to deliver their content to search engines? That seems silly.

I have a private website; I don't want the site to be listed or content found via a search engine. I want to be able to give the URL out to friends etc. but I don't want all of the world hotlink or whatever - sure, I can password protect the site, but for now I have a ton of rewrite conditions in place. #m