Google has updated its suite of Webmaster Tools to increase reportage of which site resources are blocked from crawling. This new transparency could mean a boost in rankings.
Majestic SEO is expanding yet again. Fresh off of announcing the Majestic SEO Fresh Index increased to 90 days, the Majestic SEO Team has just introduced a new URL submission tool open to everyone, not just their paid subscribers.
Blekko has integrated more recent crawl data from their search engine crawler into their free SEO tools, giving marketers access to link data, duplicate content reports, and more, populated with information that may be just minutes old.
Search engines have evolved from primitive crawling technology and now have a very powerful combination of signals. This downloadable PDF “thought paper” shows how end user data, social, mobile, and personalization are shaping the future of search.
Due to an infrastructure change, Google News is now using the Googlebot instead of a dedicated crawler to examine web content. While most elements remain the same, webmasters should be aware of some minor differences in indexing and crawling.
As Yahoo and Bing progress on the Search Alliance, Yahoo said it is live testing results of Microsoft organic and paid search ... read more
The “Meet the Crawler” sessions at SES are always high on my personal “not to be missed” list. The session at San ... read more
How search engines work, including the difference between crawler-based search engines and human-powered directories. Explanation of what a spider is and how it crawls web pages.
Leigh Dodds provides a great rundown in his Google Scholar piece about one aggregator’s experience in getting content prepared for entry into ... read more