SEO News
Search

MSN Says We'll Slow Down The News & Feed Crawling

author-default
by , Comments

Yahoo's Jeremy Zawodny noticed that MSN Search seemed to be overactively hitting his RSS feed, prompting MSN Search to respond via its blog that the news and blog crawler in testing may have been a tad hyperactive. MSN promises some tuning to get it to behave.

FYI, MSNBot does have instructions you can use to tell it to slow down, something put in place soon after the main web crawler prompted some complaints last year, when it was a bit hyperactive.

It's unclear whether the news/blog/feed crawler is using the same robot name and thus responding to the same commands. Jeremy doesn't have them on his site, nor did Greg Hughes who also blogged a complaint, but others who have complained might have.


SES LondonOptimising Digital Marketing Campaigns with Search, Social and Analytics
At SES London (9-11 Feb) you'll get an overview of the latest tools, tips, and tactics in Paid, Owned, Earned, Integrated Media and Business Intelligence to streamline your marketing campaigns in 2015. Register by 5 December to take advantage of Early Bird Rates.

Recommend this story

comments powered by Disqus