SEO News

Txt File Yahoo Slurp

RSS
  1. Giving Links Away

    If you want the instructions to apply to all spiders, you can use an asterisk on the first line: User-agent: * You can also identify specific spiders to allow or disallow, such as googlebot, Yahoo's slurp, or Microsoft's MSNbot.