SEO News


  1. Giving Links Away

    If you want the instructions to apply to all spiders, you can use an asterisk on the first line: User-agent: * You can also identify specific spiders to allow or disallow, such as googlebot, Yahoo's slurp, or Microsoft's MSNbot.

  2. Daily SearchCast, July 14, 2006: Judge Decides Google Can Rank Pages As It Pleases; Google Lets You Just Say No To ODP Descriptions; Microsoft An Enterprise Search Player? & More!

    If you want to just exclude MSN use msnbot" CONTENT="NOODP"> if you just want to exclude Google use . Today's search podcast covers KinderStart's loss in a lawsuit over search rankings; Google...

  3. Search Bot Behavior

    The analysis covers Yahoo Slurp, Googlebot and MSNbot crawling 2 billion pages structured in a binary tree over 1 year. Ever wonder how spider/bots/crawlers behave? Well, if you did a new analysis "On Bots" was released at http://drunkmenworkhere...