IndustryMSN Says We’ll Slow Down The News & Feed Crawling

MSN Says We'll Slow Down The News & Feed Crawling

Yahoo’s Jeremy Zawodny noticed that MSN Search seemed to be overactively hitting his RSS feed, prompting
MSN Search to respond via its blog that the news and blog crawler in testing may have been a tad
hyperactive. MSN promises some tuning to get it to behave.

FYI, MSNBot does have instructions you can use to tell it to
slow down, something put in place soon after the main web crawler prompted some complaints last year, when it was a bit hyperactive.

It’s unclear whether the news/blog/feed crawler is using the same robot name and thus responding to the same commands. Jeremy doesn’t have them on his site, nor did Greg
Hughes who also blogged a complaint, but others who have complained
might have.

Resources

The 2023 B2B Superpowers Index
whitepaper | Analytics

The 2023 B2B Superpowers Index

8m
Data Analytics in Marketing
whitepaper | Analytics

Data Analytics in Marketing

10m
The Third-Party Data Deprecation Playbook
whitepaper | Digital Marketing

The Third-Party Data Deprecation Playbook

1y
Utilizing Email To Stop Fraud-eCommerce Client Fraud Case Study
whitepaper | Digital Marketing

Utilizing Email To Stop Fraud-eCommerce Client Fraud Case Study

1y