Search Engine Accountability (was RE: Reality Check)
How about something like this (not suggesting as an DNS replacement!): Search Engine requires a key certificate style license embeded in the html or website (whether it's internet centralized or just for the search engine). A "rating" and "type" is associated with the license for the type of site it is. As the search engine crawls the site, if the license is gone... it doesn't get listed. If the type and rating doesn't match the information associated with the license data... it doesn't get listed. If the license certificate doesn't match the IP(s) associated with the certificate... it doesn't get listed. Has anybody tried something like this before? Karyn :: -----Original Message----- :: From: owen@dixon.delong.sj.ca.us [mailto:owen@dixon.delong.sj.ca.us] :: Sent: Friday, March 16, 2001 9:18 AM :: To: nanog@merit.edu; smcmahon@eiv.com :: Subject: Re: Reality Check :: :: :: :: :: > On Thu, Mar 15, 2001 at 03:07:30PM -0800, Scott Francis wrote: :: > > :: > > The same way people have learned to make sure that a :: search for "Anna :: > > Kournikova" (for instance) returns, say, 200 records :: that are sites/pages :: > > that have nothing whatever to do with Anna Kournikova, :: and a whole LOT to do :: > > with bringing in cash to the sites in question. :: > :: > This is self-defeating in the end; if your search site :: doesn't work, people :: > will stop using it. :: > :: Yes, but it's not the search engines that do it. It's the :: web sites that :: have learned how to put stuff in the view of the crawlers :: the search engines :: use that will show up when someone is looking for unrelated content. :: :: Since they do this in such a way that virtually EVERY search :: engine finds :: their bogus content, they don't care how many search engines :: go out of :: business, they'll just afflict the next one to come up. :: :: > If they stop using it, the advertising dollars will stop :: rolling in. :: > :: > Thus, it's in the best interest of the owner of the search :: site to fix :: > the problem. Hence why people are flocking to the latest :: best technology :: > they can find, such as Google. :: > :: Google is not immune to this, althoug it is better than some. :: :: :: Owen ::
On Fri, Mar 16, 2001 at 10:36:31AM -0800, Karyn Ulriksen had this to say:
How about something like this (not suggesting as an DNS replacement!):
Search Engine requires a key certificate style license embeded in the html or website (whether it's internet centralized or just for the search engine). A "rating" and "type" is associated with the license for the type of site it is. As the search engine crawls the site, if the license is gone... it doesn't get listed. If the type and rating doesn't match the information associated with the license data... it doesn't get listed. If the license certificate doesn't match the IP(s) associated with the certificate... it doesn't get listed. Has anybody tried something like this before?
good in theory, but in practice ... bleah. Let's just say the thought of having to go through every page of just my own site to add HTML in a certain spot makes me nauseous. I can't imagine doing this on sites with thousands of pages. Now, if you could mandate it in the robots.txt somehow ... but I've been noticing enough underhanded tricks in robots.txt using wget (like zero-sec redirects to pages that don't exist, or redirect to themselves) to not trust that option either. And say, didn't RSAC try this already? I seem to recall this being a configurable option in MSIE under the security settings ... -- Scott Francis scott@ [work:] v i r t u a l i s . c o m Systems Analyst darkuncle@ [home:] d a r k u n c l e . n e t PGP fingerprint 7ABF E2E9 CD54 A1A8 804D 179A 8802 0FBA CB33 CCA7 illum oportet crescere me autem minui
On Fri, 16 Mar 2001 10:36:31 PST, Karyn Ulriksen said:
Search Engine requires a key certificate style license embeded in the html or website (whether it's internet centralized or just for the
And the key came from *where*? And this would be different from the current hassles of getting a PKI certificate *how*?
search engine). A "rating" and "type" is associated with the license for the type of site it is. As the search engine crawls the site, if the license is gone... it doesn't get listed. If the type and rating doesn't
Woo woo.. So I have to fork over $$$ to a company to publish stuff on the web? Or is this license "for free" - in which case it's probably worth what you paid for it?
match the information associated with the license data... it doesn't get
OK.. A .JPG of a man and a giraffe would probably be acceptable for all ratings. A .JPG of a man and a giraffe having sex would probably NOT be acceptable. How did you tell the difference?
listed. If the license certificate doesn't match the IP(s) associated with the certificate... it doesn't get listed. Has anybody tried something like
Great. I relocate my server to a new co-lo and I drop off the search engines unless I fork out new money for a new license. And if you don't think this won't happen, I'll point you at all the companies that we currently license software from who are *always* ready at the drop of a hat to ship us a new license for a machine "because the CPU planar died and was replaced so the hostid is now different". Yeah. We *never* have a problem getting new licenses. ;)
this before?
Umm.. there's PICS and other web rating systems already extant. They've worked wonders, haven't they? Valdis Kletnieks Operating Systems Analyst Virginia Tech
participants (3)
-
Karyn Ulriksen -
Scott Francis -
Valdis.Kletnieks＠vt.edu