Yahoo Tips On Crawling Issues

The Yahoo Search Blog has some webmaster tips out today: Yahoo! Search Tips for Webmasters: Saving Bandwidth. Gzip your files, and they'll unzip and save you bandwidth when crawling. Respond with the right HTTP code, and it will only download pages if there's been a change to them. Getting slammed by the Slurp crawler? Make it sip rather than slurp through the crawl-delay mechanism.

About the author

Danny Sullivan was the founder and editor of Search Engine Watch from June 1997 until November 2006.

To contact current Search Engine Watch editorial staff, please click here.