When you have parts of your site that you don't want the search engine spiders to index, you let them know using a document called robots.txt. But for the coding-challenged, creating that document has not always been easy. Thankfully, Google has created a robots.txt generator as part of its Webmaster Tools.
Once your document is created, you can test it with the robots.txt analysis tool. Google points out that not every search engine recognizes robots.txt. They recommend securing truly sensitive material with password protection.
It's a big week for the Google Webmaster team. Today, they're hosting the first ever Google Webmaster chat, which begins at 12pm EST/9am PST. The chat will feature a site clinic and also discuss image optimization.
Upcoming Webinar: PPC Pause and Reflections for 2013
Thursday, December 12 - 2013 was a major turning point in search advertising. With Google's Enhanced Campaigns and Bing's innovative Smart Search capabilities in Windows 8.1, now is a great time to pause, reflect, and plan for the new year. Webinar attendance is free. Sign up today!