Whats a Yahoo! Slurp Spider????

Illus

Member
I was looking at the 'currently active users' list, and there are like 200 'Yahoo! Slurp Spider' users on there... What the heck is that?????
 

FiatFan

Member
A Yahoo slurp spider is simply a 'bot (short for robot) which travels the internet indexing pages for Yahoo's search engine. Not harmful but can use server resources. There are instructions on the Yahoo web site on how to restrict them.

Tom
 

Illus

Member
So basically, when someone searches for something on Yahoo, and gets a hit on The Gauge, it shows up as a Slurp Spider... Interesting. Thanks for the info.
 

ezdays

Out AZ way
Illus said:
So basically, when someone searches for something on Yahoo, and gets a hit on The Gauge, it shows up as a Slurp Spider... Interesting. Thanks for the info.
Well, basically they are here to calalog everything that goes on. Then when someone does a search on say, "detailing an HO scale BNSF GP-38", they will find a couple of links to some post here. There are frequently hundreds of them crawling around and as Tom says, they can eat up bandwidth and resources. I think they're like bats, coming out a night when most people can't see them...:D :D
 

FiatFan

Member
Illus said:
So basically, when someone searches for something on Yahoo, and gets a hit on The Gauge, it shows up as a Slurp Spider... Interesting. Thanks for the info.

Just the opposite. If The Gauge shows as a hit on a Yahoo search, it was found (generally) by a slurp spider. All a spider does is travel the internet looking for new (or changed) pages and reports those pages back to Yahoo (or Google or any other search engine).

If a spider is found on a server, it is simply cataloging the pages available and reporting those pages back to Yahoo.

Tom
 

N Gauger

1:20.3 Train Addict
FiatFan said:
Just the opposite. If The Gauge shows as a hit on a Yahoo search, it was found (generally) by a slurp spider. All a spider does is travel the internet looking for new (or changed) pages and reports those pages back to Yahoo (or Google or any other search engine).

If a spider is found on a server, it is simply cataloging the pages available and reporting those pages back to Yahoo.

Tom
This is how "Cached" pages on Yahoo are updated as well as pages on "the wayback machine" are stored

http://www.archive.org/web/web.php Go do a search for The gauge or any other web site.. :)
 

ezdays

Out AZ way
Ahhhh, but where are the "Google spider bots", hmmmmm? Do these guys travel in the stealth mode or what????

Think about that.sign1 sign1
 
Top