After taking a random sampling of 200,000 "Jeopardy" clues and performing searches on Google, Bing, Ask, Blekko, Wikipedia search, and Yandex, this search engine "found" the most correct answers.
If you responded, "What is Google?" you would be correct.
With all the hoopla surrounding the upcoming man vs. machine IBM challenge on the long-running trivia game show hosted by Alex Trebek, Stephen Wolfram decided to quiz the search engines. After inputting each clue into the search engines without quotes, he counted:
- How frequently the correct "Jeopardy" answer appeared somewhere in the title or text snippets.
- How frequently it appeared as the top document returned by the search engine.
Looking at the results, perhaps Ken Jennings should design a search engine?
Google bested Ask, Bing, and to a lesser extent Yandex on returning an answer anywhere on the search engine result page. Blekko was a somewhat distant fifth, while Wikipedia search was dead last.
In a close one, Google bettered Bing, with Yandex and Ask trailing for frequency returning an answer in the first document. Blekko and Wikipedia remained in fifth and sixth place respectively.
"In terms of 'Jeopardy,' what we see is that just using a plain old search engine gets surprisingly far," Wolfram wrote. "Of course, the approach here isn't really solving the complete 'Jeopardy' problem: it's only giving pages on which the answer should appear, not giving specific actual answers."
Wolfram also noted that the top search engines succeeded about 20 percent of the time in returning the correct answer in the title of the first result. Despite their successes, they're still no Ken Jennings, and Google and friends will need to do a lot more work before we get a Man vs. Search Engine episode, no doubt sponsored by AdWords.
Introducing SES Online
Want to view one of the sessions you missed or listen to an especially informative presenter a second time? SES New York sessions are available for purchase on ClickZ Academy's new e-Learning site. SES is now Online!