SEO News

New French Relevancy Study Puts Yahoo & Google Tied For First But Search Overall As Disappointing

by , Comments

Jean Véronis has published the results of a relevancy study he conducted with his students at the Université de Provence showing that, at least for French searches, Google and Yahoo are tied on the relevancy front but that search engines overall disappoint. However, he also takes a different spin on the stats that give the search engines higher marks and puts Yahoo slightly ahead of Google.

Jean summarizes the study here, with a more detailed PDF rundown here. Google, MSN and Yahoo along with Exalead, Voila and were evaluated. In summary, here's what was done:

  • Students were assigned topics and allowed to craft queries in French however they thought best to search for information on those topics.
  • The first page of results from the search engines involved for each query were retrieved. Only French-language results were pulled back, and the porn filter at each service was used (and US Department Of Justice, take note --- the study found these worked very well).
  • Students then reviewed the URLs retrieved to see if they were generally on the topic of the search. Each URL was scored to the relevancy of the topic, 0 being worst, 5 being perfect.

Scores were:

  • Google/Yahoo: 2.3
  • MSN: 2.0
  • Exalead: 1.8
  • Dir: 1.4
  • Voila: 1.2

As you can see, no service gained above the 2.5 middle ground on the 0 to 5 scale. But at the suggestion of Jakob Nielsen, Jean did a new comparison of scores, to take into account that users may be happy if they get at least one good result per search, rather than 10 of them. So -- if I understand right -- he calculated the average highest score for URLs on search results pages.

In other words (and again, if I understand right), he looked at all the search results pages from Yahoo, then looked at the highest scoring URL on each page, then figured the average. Relevancy shoots up with this much tighter examination, and Yahoo pushes slightly ahead of Google, while MSN and Exalead also close the gap:

  • Yahoo: 4.5
  • Google 4.3
  • MSN: 4.2
  • Exalead: 4.1
  • Voila: 3.6
  • Dir: 3.4

ClickZ Live New York What's New for 2015?
You spoke, we listened! ClickZ Live New York (Mar 30-Apr 1) is back with a brand new streamlined agenda. Don't miss the latest digital marketing tips, tricks and tools that will make you re-think your strategy and revolutionize your marketing campaigns. Super Saver Rates are available now. Register today!

Recommend this story

comments powered by Disqus