Google Webmaster Tools Now Finds Smartphone Crawl Errors


It can be bit complicated for websites with a high number of smartphone visitors to figure out issues like 404 errors when only smartphone visitors or desktop users may be affected. Often, users don’t realize there is a problem because the majority of the time, they’re doing troubleshooting and maintenance from a desktop computer; they just don’t notice the mobile issues unless someone specifically alerts them.

Google Webmaster Tools realizes this is an issue, especially with mobile traffic increasing at such a rapid rate. They’ve made some changes to their crawl errors page to include specific smartphone crawl errors that the Googlebot-Mobilebot discovers while crawling the web as a mobile useragent.

Pierre Far, a webmaster trends analysts, has announced that webmasters can now find a wide range of crawl information and errors for smartphones:

  • Server errors: A server error is when Googlebot got an HTTP error status code when it crawled the page.
  • Not found errors and soft 404s: A page can show a "not found" message to Googlebot, either by returning an HTTP 404 status code or when the page is detected as a soft error page.
  • Faulty redirects: A faulty redirect is a smartphone-specific error that occurs when a desktop page redirects smartphone users to a page that is not relevant to their query. A typical example is when all pages on the desktop site redirect smartphone users to the homepage of the smartphone-optimized site.
  • Blocked URLs: A blocked URL is when the site's robots.txt explicitly disallows crawling by Googlebot for smartphones. Typically, such smartphone-specific robots.txt disallow directives are erroneous. You should investigate your server configuration if you see blocked URLs reported in Webmaster Tools.

The mobile crawlers are already live in Webmaster Tools. Simply log into your account, click on “Crawl Errors” under the “Crawl” submenu, and select the smartphone tab to view any crawl errors from your website.

About the author

Jennifer Slegg began as a freelance writer, and turned to search engine optimization and writing content for the web in 1998. She has created numerous content-rich sites in niche markets and works with many clients on content creation, strategy, and monetization. She writes about many search industry and social media topics on her blog, and is a frequent speaker at search industry conferences on SEO, content marketing and content monetization. Acknowledged as the leading expert on the Google AdSense contextual advertising program, she runs JenSense, a blog dealing exclusively with contextual advertising. She is also the founder and editor of The SEM Post. She is known by many as her handle Jenstar on various webmaster forums.