Search Engine Watch
SEO News

Yahoo Slurp Adds Wildcard Support For Robots.txt

by ,  Comments

The Yahoo Search Blog announced that Yahoo's web crawler, aka Yahoo Slurp, now supports wildcards in the robots.txt file. The two parameters that Yahoo now supports include the "*" and the "$." The * will tell Yahoo to do a "wildcard match a sequence of characters in your URL." The & will tell Yahoo to do a "anchor the match to the end of the URL string." Many more details at the Yahoo Search Blog.


Know your Ambiguous Customer: Effective Multi-Channel Tracking
ClickZ Academy Wednesday, June 5 at 1pm ET - Learn why a move from the "batch and blast" email approach enables better conversations with your customers.
Register today - don't miss this free webinar!

Recommend this story

Commenting policy Add a Comment