ComScore released figures in August showing a significant rise in worldwide search activity — a story that resulted in many headlines. Hidden in the release was the statement “…with Russian search engine Yandex growing at the fastest rate (94 percent) among the top 10.” Why is Yandex doing so well?
With Google’s growth rate at 58 percent, Microsoft’s at 41 percent, and others, including Baidu and Yahoo, showing much lower rates of growth (8 percent and 2 percent respectively), Yandex is doubling year over year and unbelievably beating the global search query shares of Yahoo, Microsoft, AOL, and Ask — even during these more difficult economic times.
A deeper analysis of comScore’s figures shows that Google gained most in comScore’s version of global search share moving up from 60.4 percent to 67.4 percent, but Yandex was the only other gainer with a modest move up of 0.3 percent (Microsoft and Ask were static, Yahoo and Baidu drifted down).
Everything points to Yandex being a remarkable success story. Is it economics? Or is it something to do with search technology?
Often described as a copy of Google, Yandex was actually launched in 1997, slightly before Google. Yandex’s long-held number one spot in Russia is attributed to it being built from the ground up to handle the complexities of the Russian language — and it was a first mover in the market.
The Yandex story also has many parallels with that of Google and Yahoo, with mathematicians Arkady Volozh and Arkady Borkovsky generating the idea for Russian language search through their then business Arkadia Inc. (later called CompTek). Originally, the focus was on document and intellectual property search. Then the parallels become less Googlesque because neither were students at the time, nor did they start in a garage.
More like eBay, Yandex was a project of CompTek. With its success, Volozh committed to the project, relinquishing his role as CEO at CompTek. Several followed Arkady from CompTek to found Yandex, including Elena Kolmanovskaya and Mikhail Fadeev — also mathematicians. Borkovsky eventually rejoined Yandex in 2008 after a spell with Inktomi and Yahoo in the U.S., where he is still based.
The focus on the Russian language is one key reason why Yandex is a fascinating case study in understanding how linguistics lead to success in multilingual SEO. Western search engines and algorithms are essentially built from English language building blocks. Sure, they can then be adapted, but the more extreme the language and the more complex the algorithm, the more the adaptation beings to creak under the strain.
Russian is a member of the Slavic family of languages, which includes a wide range of languages in Eastern Europe from Polish and Czech to Serbian and Croatian. Most of these languages don’t use the Cyrillic alphabet which is not, as often perceived, the cause of search engines’ difficulties in indexing Russian — rather, it’s the morphology, or shape and structure of the language.
There is no verb “to be” in the Russian present tense. Nor do “the” or “a” exist. Russian gets around this by changing the ending of the word. This is what’s known as an inflected language. Remember that “the” is treated as a stop word by search engines.
In order to know to treat it as a stop word in Russian, you’d have to know which bit of a word to chop off or stem. Unfortunately, stemming, which is routinely used in English to improve the relevancy of search by treating plurals and singulars as equivalents or reducing verbs to their component parts, is somewhat more complicated in Russian by the sheer variety of possibilities.
For example, the Russian word “Izvergatch” (which means to “erupt”) has 166 different grammar forms of the word. Added to that, all Russian verbs have an essential partner verb for imperfect or perfect meaning distinctions — and they aren’t necessarily spelled in a similar fashion. You can perhaps now begin to appreciate Yandex’s particular strength.
Meanwhile, the Russian economy hasn’t been unaffected by the global crisis. Its economy shrank by an annual rate of 9.8 percent during the first quarter of 2009 and only now is it showing the first green shoots. So nope, it’s not the economy this time, stupid.
Russian statistics organization Liveinternet.ru provides a snapshot of how search engines are doing in Russia, which gives further interesting insight. This shows just how closely fought the Google-Yandex battle is in Russia. Google moved into second place in the fall of 2007 and then reached is closest point to Yandex in late September 2008. But since then, Yandex has actually been romping ahead and moving further ahead of Google in share.
Below are the shares for September 2009, according to Liveinternet.ru data. They show that Yandex is still firmly ensconced as the number one and is gaining share.
While part of the reason for this is that Yandex holds a strong place in the Russian, Ukrainian, and Belorussian languages thanks to its search technology, it’s also true that some of Yandex’s key competition has been getting its wires well and truly crossed.
Rambler, which used to hold a strong second place in Russian search engines, has declared heavy losses this year — and analysts are blaming its lack of focus and letting its competition (Yandex and Google) run away with the market. It has even now been overtaken by Search.Mail.ru, which is powered by — guess who — Yandex.
Other facets give Yandex an iron grip on the Russian market. One is Yandex Money, which facilitates the Russian e-commerce market by acting like a bank for consumers’ money and enabling them to buy online when credit cards aren’t especially popular or suitable — using PayPal within Russia presents challenges!
On top of all that, Yandex is strong in contextual advertising, has a PPC system very similar to Google’s, and offers many of the most popular facilities you would expect from a search engine leader. It has also recently launched APIs, giving better access to its systems as well as offering keyword research tools.
In summary, Yandex was first, has a very deep connection with Russian language and culture, is financially sound, and is investing in its own future. Meanwhile, competitors are finding it very tough to keep up, which is why Google has gained 7 points globally, while Yandex has taken its 0.3 points in share at home — in Russia and Eastern Europe.