Most SEOs face challenges with a single site in English, but the challenges compound exponentially when you add multiple countries and languages to the mix.
The most common of these problems is the use of language or location detection to dynamically show the visitor the language or country site of their preference or give them the option of selecting where they want to go. While this is a convenience to the visitor, it causes problems for the spiders to access the different versions of your site.
Dynamic Language Detection
Browser Language Detection is the most popular method. It looks at the visitor's language preference submitted to the server via the browser's request for the page.
For example, a person who uses the French-language version of Firefox will typically have their language preference set to "French." When Internet users visit a site with browser language detection, it will read the preference and automatically redirect the visitor to the French version of the site.
Another method is IP Location Detection. This serves up local content based on the IP location of the visitor. For example, if the IP location shows they accessed the site while in Munich, Germany, then the server would serve them a German language version of the site.
Both of these methods are problematic for search engines, because spiders often crawl from a specific location and don't signal language preference. For example, if Googlebot were crawling from Mountain View, Calif., then the Web server would detect an IP from the U.S. and the crawler would be routed to the U.S. version and potentially never see the German language version.
The most reliable test is asking employees in local markets to visit the site and see if they're redirected to anything other than the main global home page. If they're presented a local country version, then the site uses dynamic language detection.
Country/Location Site Maps & Selectors
If you have more than a single country or language version of your site, and your site hasn't deployed one of the dynamic means of redirection described above, then we would assume you've implemented a manual solution. The country selector will make it easy for a visitor to select the county or language of their choice.
A few multinational sites still make you select the version of the site you want to visit from a large map of the globe when you arrive at their home page. This is problematic, because search engines can't understand the image or know where to click, thereby preventing them from entering the site.
Using a pull-down menu at the top of the page to select your country is the most common option for presenting local content options of the global site. This approach allows the company to leverage the primary domain globally. If a visitor wants to select a different country or language, they can easily pull down the list and select the one they want.
Intel's global portal is an excellent example representing nearly 100 language/country options. They have effectively integrated a "noscript" tag with relative URLs that makes it easy for search spiders to detect all of these destination pages. This would have been a perfect example had they used fully-pathed URLs, but this often isn't possible due to staging and testing server requirements.
XML Sitemaps to the Rescue
XML sitemaps are a necessity for a global site, especially if you have overly engineered access into the local content. For detailed instructions on developing XML sitemaps, visit the standards Web site.
It's strongly recommended that you create a unique XML sitemap for each of your country or language versions of the site to ensure the key segments of your country sites are pushed to the search engines, despite any of the issues noted in this column. You may already have these sitemaps because they're required to use Google's Geographical Designator tool, which is critical to ensure you're showing up in local versions of Google.
Join us for Search Engine Strategies San Jose, August 10-14, 2009, at the McEnery Convention Center.