Articles on web crawlers

RSS

Website Indexation Audit: How to Find & Remove Non-Essential Content

holding-out-trash-bags

Are you feeding search engines garbage? Taking a look at your entire index will allow you to gain an understanding of what Google and Bing see and index. Here's how to hone in on and assess your content, see if it's indexed, and clean it up. Are you feeding search engines garbage? Taking a look at your entire index will allow you to gain an understanding of what Google and Bing see and index. Here's how to hone in on and assess your content, see if it's indexed, and clean it up. 0 Comments

Behind the Scenes at Yahoo Labs, Part 3

The head of Yahoo Labs talks about the competition, the challenge of balancing searcher needs with webmaster desires, the future, and the next big thing in web search. The head of Yahoo Labs talks about the competition, the challenge of balancing searcher needs with webmaster desires, the future, and the next big thing in web search. 0 Comments

HotBot's Swiss Army Knife Deskbar

HotBot today is releasing its Quick-Search Deskbar, which not only provides instant access to the search engine, but is jam-packed with other useful tools and goodies as well. HotBot today is releasing its Quick-Search Deskbar, which not only provides instant access to the search engine, but is jam-packed with other useful tools and goodies as well. 0 Comments

Help LookSmart Crawl the Web

LookSmart is taking a new approach to discovering web content, offering a free downloadable screensaver program that also crawls the web when your computer is idle LookSmart is taking a new approach to discovering web content, offering a free downloadable screensaver program that also crawls the web when your computer is idle 0 Comments

Help LookSmart Crawl the Web

LookSmart is taking a new approach to discovering web content, offering a free downloadable screensaver program that also crawls the web when your computer is idle. LookSmart is taking a new approach to discovering web content, offering a free downloadable screensaver program that also crawls the web when your computer is idle. 0 Comments