The battle of which search engine is the most relevant has been going on for years. Yesterday at SES Toronto's Searcher Behavior Research Update panel we had two presenters talk about studies they have conducted to determine which search engine is the most relevant. Google has won the relevancy battle in those studies, but was it due to Google's brand or due to Google's search results?
An InternetNews.com article also has a nice summary of the session. But let me explain them in a nutshell for you.
Gord Hotchkiss showed part two of his study, where he conducted an eye tracking test on users on both MSN, Yahoo and Google search results page. Google's results showed that people were more likely not to look down the page at results below the 2nd or 3rd listings, but MSN and Yahoo results showed that the user was much more likely to look down the results to results as low as 6 or 7. Does that mean Google's results were more relevant, sooner? Meaning people did not have to scroll to see more results sooner? He said, maybe or maybe not. He noted that the layout of Google's results at that time had bolded the keywords queried by the searcher, on the page - which may have prompted searchers to click on those results sooner with the bolded words than without the bolded words.
Lance Jones was the next speaker he conducted a study that took all the branding off the Google results and tested searchers to see if they like the results with and without knowing the results were from Google. The users scored the results from scores from 0 to 1,000. The users scored the results that were identical on both groups, 800 for Google branded results and 737 for unbranded Google results. This shows that knowing the results are from Google, adds a bias to how relevant the results are or are not.
So is Google more relevant? Well, my RustySearch live results show Google is now the leader in relevancy. When I first published the results, Yahoo was in the lead. At some point in the last 3-months or so, Google took over the lead.
The bottom-line is that relevancy is an incredibly hard factor to measure.