If Search Engines Could Read Your Mind

What if a search engine knew exactly what you were thinking, and unerringly provided perfect search results? The idea is not as farfetched as it sounds.

When people complain about “poor quality” or “irrelevant” search results, they almost never blame their own poorly formed request—yet bad queries are a huge part of the problem. It’s actually quite remarkable that search engines can take a sparse two or three word query and make sense of it. Lacking context, search engines are forced to virtually guess at your true intent.

If only we searchers would spell out our needs in greater detail, search engines wouldn’t need to rely so heavily on unreliable heuristics and other techniques that are imperfect indicators of meaning. Our laxity is one reason why most search engines offer search refinement tools, suggesting additional query terms, clustering results into conceptually related topics or otherwise trying to extract more clues about what a searcher is really looking for.

These refinement tools are relatively primitive, but for good reason—most of us are too lazy to take advantage of them. When we do, our results often improve. But more often than not we’re in too big of a hurry to get results. We’d much rather waste time scanning results and clicking back and forth among less-than-useful pages than craft a really good query or use search refinement tools.

But while we’re doing this, the search engines are observing our behavior, and learning from our fumbling activities. All of the major search engines employ artificial intelligence experts who are quietly laboring away at building common sense and worldly knowledge into our search tools.

Wait—wasn’t artificial intelligence widely discredited after promising miracles in the 1980s? Some of the grander claims made by AI proponents at the time have indeed fallen by the wayside. But AI is alive and well and there have been some serious breakthroughs—computerized voice recognition systems being a significant example of applied artificial intelligence.

Want to see an example of what artificial intelligence could do for search? Take a look at 20Q.net, which is an online version of the child’s game “twenty questions.” The premise behind the site is simple: Think of a common object, and then answer a series of questions. 20Q will then “guess” what you’re thinking about. Just about every time I’ve used it, 20Q has correctly identified the object I’ve imagined using fewer than 20 questions.

If 20Q.net can’t guess what you’re thinking, you “win” the game. The system then presents you with a list of other possible objects that you might have been thinking about. If the object is in the list, click on it, and 20Q will explain its logic to you, indicating contradictions between your answers and its own knowledge base. These aren’t necessarily “wrong,” but rather are an indicator of the learning 20Q.net has gained by interacting with other people.

20Q is a neural network that works much like a human brain. The software has been “trained” by thousands of users playing the game over the past decade. By interacting with users, the neural net has learned about objects in the real world, and continues to learn as it analyzes each game.

20Q.net has about ten million synaptic connections (by contrast, the human brain has about one hundred trillion). The game uses the neural-network to choose each question it asks you. After collecting enough clues, the neural-net also guesses what you’re thinking about.

To a certain degree, search engines already employ similar systems. Just as 20Q.net starts out with broad questions (is it animal, mineral, or vegetable) to “prune the tree” of possible branches, search engines do the same thing with the few clues offered by your search terms, eliminating thousands or millions of possibilities before even considering possible matches.

And these learning systems will improve, enabling the search engines to offer better results even if we searchers continue in our miserish search habits. In the words of Google’s Vice President of Engineering Wayne Rosing, “The day will come when Google won’t be a search engine anymore, because everything will be searchable. So, instead, we’ll have to algorithmically find you the good stuff. It will be an up-leveling of our ranking function, if you will, from what’s the best document to what’s the best, most well-formed knowledge on the subject.”

For more information about 20Q.net, see its frequently asked questions or take a quick tour of the game.

Search Headlines

NOTE: Article links often change. In case of a bad link, use the publication’s search facility, which most have, and search for the headline.

Medical researchers bankroll innovative online database
The Guardian May 11 2005 12:45PM GMT
Search engine specialists: Agencies seek higher returns
Revolution May 11 2005 12:44PM GMT
Google chairman’s seat sits gathering dust
San Francisco Chronicle May 11 2005 12:37PM GMT
Alta Vista hacker pleads guilty
San Jose Business Journal May 11 2005 6:21AM GMT
Interview with CEO: Yahoo is more than just a search
Israel Business Arena May 10 2005 9:07PM GMT
Yahoo to launch new flagship music service
CNET News.com May 10 2005 9:04PM GMT
New search tools help bargain hunters cut their shopping time
Los Angeles Times reg May 10 2005 8:13PM GMT
Will Clusty.com Be the Next Google?
Fortune May 10 2005 7:11PM GMT
Yahoo taps Amazon design guru as executive
Reuters (via Yahoo News) May 10 2005 7:06PM GMT
Answers.Com and Shopping.com Announce Partnership
Search Engine Watch May 10 2005 3:19PM GMT
Toronto Search Engine Strategies Conference Review
Search Engine Guide May 10 2005 2:28PM GMT
PubMed Now Offering RSS Feeds
Library Stuff May 10 2005 2:23PM GMT
Budgeting to Improve Web Conversions
Pandia May 10 2005 2:21PM GMT
Mental Models For Search Are Getting Firmer
Alertbox May 10 2005 2:20PM GMT
Google Scholar Expands Library Link Program
ResourceShelf May 10 2005 2:18PM GMT

Related reading

Simple Share Buttons