MSN Says We'll Slow Down The News & Feed Crawling
Yahoo’s Jeremy Zawodny noticed that MSN Search seemed to be overactively hitting his RSS feed, prompting
MSN Search to respond via its blog that the news and blog crawler in testing may have been a tad
hyperactive. MSN promises some tuning to get it to behave.
FYI, MSNBot does have instructions you can use to tell it to
slow down, something put in place soon after the main web crawler prompted some complaints last year, when it was a bit hyperactive.
It’s unclear whether the news/blog/feed crawler is using the same robot name and thus responding to the same commands. Jeremy doesn’t have them on his site, nor did Greg
Hughes who also blogged a complaint, but others who have complained
might have.
More about:
The Merkle B2B 2023 Superpowers Index outlines what drives competitive advantage within the business culture and subcultures that are critical to success. It is the indispensable guide for B2B marketers to deliver world-class experiences and keep pace with the dynamic environment. Download Now
The ClicData survey found that various challenges exist that prevent organizations from achieving such gains. These challenges included inaccessible data formats and limited flexibility in displaying data in dashboards. Download Now
The need for fraud prevention in the digital world is critical now more than ever. Why? Thinking about your own behavior, consider how you complete transactions and how this has changed over the last 5 years. Download Now
The need for fraud prevention in the digital world is critical now more than ever. Why? Thinking about your own behavior, consider how you complete transactions and how this has changed over the last 5 years. Download Now