Whats a Yahoo! Slurp Spider????

Discussion in 'Getting Started' started by Illus, Dec 27, 2006.

  1. Illus

    Illus Member

    I was looking at the 'currently active users' list, and there are like 200 'Yahoo! Slurp Spider' users on there... What the heck is that?????
  2. FiatFan

    FiatFan Member

    A Yahoo slurp spider is simply a 'bot (short for robot) which travels the internet indexing pages for Yahoo's search engine. Not harmful but can use server resources. There are instructions on the Yahoo web site on how to restrict them.

  3. Illus

    Illus Member

    So basically, when someone searches for something on Yahoo, and gets a hit on The Gauge, it shows up as a Slurp Spider... Interesting. Thanks for the info.
  4. ezdays

    ezdays Out AZ way

    Well, basically they are here to calalog everything that goes on. Then when someone does a search on say, "detailing an HO scale BNSF GP-38", they will find a couple of links to some post here. There are frequently hundreds of them crawling around and as Tom says, they can eat up bandwidth and resources. I think they're like bats, coming out a night when most people can't see them...:D :D
  5. FiatFan

    FiatFan Member

    Just the opposite. If The Gauge shows as a hit on a Yahoo search, it was found (generally) by a slurp spider. All a spider does is travel the internet looking for new (or changed) pages and reports those pages back to Yahoo (or Google or any other search engine).

    If a spider is found on a server, it is simply cataloging the pages available and reporting those pages back to Yahoo.

  6. N Gauger

    N Gauger 1:20.3 Train Addict

    This is how "Cached" pages on Yahoo are updated as well as pages on "the wayback machine" are stored

    http://www.archive.org/web/web.php Go do a search for The gauge or any other web site.. :)
  7. ezdays

    ezdays Out AZ way

    Ahhhh, but where are the "Google spider bots", hmmmmm? Do these guys travel in the stealth mode or what????

    Think about that.sign1 sign1
  8. Cannonball

    Cannonball More Trains Than Brains

    I hate spiders.....

    Attached Files:

Share This Page