Where's Search Heading? Ask Yahoo's Chief Scientist

While Google has dominated the search space for the past few years, Yahoo has held its own as a respectable number two. But it's not content to remain there, and is in the midst of rolling out a whole new approach to search. While many of the details have not yet been discussed publicly, we do know that Yahoo is taking a task-based approach to its search strategy, improving results to focus on the user's task stage, according to Andrew Tomkins, chief scientist for search at Yahoo.

A search engine can make changes in two basic ways to today's search experience. It can take a top-down approach, starting with the user, and make changes to the way it understands and processes queries. Or it can take a bottom-up approach, improving the way the search engine extracts information and understands the content it has indexed.

At Yahoo, as at other search engines, researchers are approaching search from both directions. The common theme behind everything they're doing is to make it easier for users to accomplish their tasks and get things done, according to Tomkins.

Tomkins spent the last few years at Yahoo Research, working on projects that will guide the future of Yahoo Search. A few months ago, he was named chief scientist for web search. In his new role, he is putting some of those research projects into action.

Tomkins will deliver the morning keynote presentation today at Search Engine Strategies New York. He will share some ideas about search's future and how Yahoo has begun moving toward a new way of interacting with users.

"I like to think of search as the place people start for a lot of things they want to do. Sometimes, they get their answer and they're done. Other times, it's part of a process that may take months to complete, like buying a house or researching high-def televisions," Tomkins said. "The next generation of search will be about understanding the task a user has in mind and changing the way search operates to get those things done."

For example, users searching on general terms about HDTV might be shown broad research material. As they move closer to a purchase, they'd be shown more specific results, like reviews and price comparisons, he said.

"What's it really take to get the job done? We want to make task fulfillment seamless, both on and off the Web," Tomkins said.

Yahoo's refined approach to search

Yahoo began giving glimpses into where its search is headed last year when it added the Search Assist layer of query refinements in October. When a user begins typing a query, some suggestions are presented that finish or expand a query, or offer related concepts.

"The new style of search is much more of a conversation," Tomkins said. "It focuses on increasing a user's productivity, helping them complete complex tasks to get to their goal."

The engine has also been offering Shortcuts for some time, as Google has done with its OneBox results, Ask with its Smart Answers, and Microsoft with its Instant Answers.

In October, Yahoo made its general search results a blended search experience, with more photos, videos, and Shortcuts. It also expanded the number of common tasks it's created Shortcuts for, adding more results pulled from Yahoo properties like Flickr, Upcoming, and Yahoo Answers, as well as from third-party sites like YouTube.

For example, a search on a current movie title returns links to the trailer, reviews, show times, and ticket information. Yahoo decides which results to add these Shortcuts to by examining its query logs and identifying where users ended up for the most common queries.

Examining the logs and programming those results gives Yahoo a deeper understanding of what the user is trying to accomplish, Tomkins said.

Putting Users in Control

Rather than adding more behind-the-scenes attempts to discern a user's intent, Yahoo is instead focusing on exposing "levers" that will help users to refine their own search experience, Tomkins said.

"The user is the end arbiter," he said. "Instead of trying to control the experience, we'd rather give them different ways to indicate or confirm their intent. It's more like driving a car than getting on a train; they have the ability to control their destination."

Yahoo sees search as one of many tools that users come to the site for, alongside news, mail, and research on "life stuff," like real estate, food, and autos, Tomkins said.

With a large audience of engaged visitors, Yahoo is in the perfect position to understand the tasks people want to accomplish and how to create a search experience to match up with that, he said.

New Signals of Relevance

For the past 10 years, most search engines have relied heavily on analyzing anchor text, links, and content to determine relevance, he said.

Recently, research by the top search engines has looked at other signals, such as user clicks and engagement, that might indicate search result relevance.

"The search engine landscape is very much trying to take advantage of those signals," Tomkins said. "They're trying to squeeze what they can from what they see users engaging in."

A likely result of that kind of research might include new ways to rank content based on the search engine's understanding of a query and the task a user is engaged in, he said.

For search marketing practitioners, the best way to prepare for that kind of algorithm change would be to design a website along similar lines and build what would be most useful to users at various stages of their tasks, Tomkins said.

"The best strategy for ranking will be to find a sweet spot for user needs and customize pages to meet those needs – then rely on the engines to discern that value," Tomkins said. "We'll unlock the value of narrow, task-focused material."

About the author

Kevin Newcomb joined ClickZ in August 2004, covering search marketing and other online marketing topics. He has been reporting on web-based businesses since 2000.

Before the bubble burst, Kevin was a marketing manager for an online computer reseller, handling copywriting, e-mail marketing, search marketing and running the affiliate program.

With a combination of real-world marketing experience and years of business journalism, Kevin brings to ClickZ a unique ability to deliver news and training materials that help online marketers do their jobs better.