Ever wonder how spider/bots/crawlers behave? Well, if you did a new analysis "On Bots" was released at http://drunkmenworkhere.org/219. The article has an analysis and visualization of the behavior of search robots. The analysis covers Yahoo Slurp, Googlebot and MSNbot crawling 2 billion pages structured in a binary tree over 1 year. The study was conducted on a single site, so I am not sure how statistically valid it is over all sites on the Web. Just take a look at the overall results to see how much of a hog Yahoo is.
The Original Search Marketing Event is Back!
SES Denver (Oct 16) offers an intense day of learning all the critical aspects of search engine optimization (SEO) and paid search advertising (PPC). The mission of SES remains the same as it did from the start - to help you master being found on search engines. Register today!