I was looking at the 'currently active users' list, and there are like 200 'Yahoo! Slurp Spider' users on there... What the heck is that?????
Well, basically they are here to calalog everything that goes on. Then when someone does a search on say, "detailing an HO scale BNSF GP-38", they will find a couple of links to some post here. There are frequently hundreds of them crawling around and as Tom says, they can eat up bandwidth and resources. I think they're like bats, coming out a night when most people can't see them...Illus said:So basically, when someone searches for something on Yahoo, and gets a hit on The Gauge, it shows up as a Slurp Spider... Interesting. Thanks for the info.
Illus said:So basically, when someone searches for something on Yahoo, and gets a hit on The Gauge, it shows up as a Slurp Spider... Interesting. Thanks for the info.
This is how "Cached" pages on Yahoo are updated as well as pages on "the wayback machine" are storedFiatFan said:Just the opposite. If The Gauge shows as a hit on a Yahoo search, it was found (generally) by a slurp spider. All a spider does is travel the internet looking for new (or changed) pages and reports those pages back to Yahoo (or Google or any other search engine).
If a spider is found on a server, it is simply cataloging the pages available and reporting those pages back to Yahoo.
Tom