The Future of Search

Representatives from the world’s biggest search engine companies sat down with Danny Sullivan and several hundred of his closest friends to talk about the future of search and information retrieval. The imperative to avoid giving away deep company secrets meant that audience members had to listen extra closely and read between the lines at times.

A special report from the Search Engine Strategies 2004 Conference, March 1-4, New York City.

In this fast-changing field, some of these musings on the “future” feel dated already only a couple of months later. Nonetheless, the session provided an interesting overview of how these top portals plan to reach out to their audiences in the coming years. Presenters’ approaches to the core technological questions of the day varied significantly in both style and substance, giving knowledgeable attendees a study in contrasting corporate cultures at AOL, Yahoo, Google, and Ask Jeeves.

Few in the search industry today disagree that personalization is going to be a key driver of change. Assumptions about how to display search results pages to users are undergoing a steady transformation. Gerry Campbell of AOL search came at the issue from several different angles. First, the online experience in general should be structured to allow more recall of information that users need for basic workday and lifestyle management, from favorite stocks and birthdays to Internet connection speed. Search itself could be self-learning so that it would calculate the probability that a given user typing “eagles” meant a football team, and should offer much more by way of regional listings. Ordering a pizza from a local establishment should, Campbell argued, start with something as ambiguous a typed search query for “pizza.”

Google’s Craig Nevill-Manning focused his comments about personalization almost exclusively on the geographic targeting of keyword ads that is now offered as part of the AdWords program. He alluded to “targeting by IP address” and focused mainly on “targeting by country.” Given the current buzz around newer features of Adwords that allow sophisticated “radius targeting” in local areas, Nevill-Manning’s guarded delivery seemed to err far on the side of caution. He did emphasize that personalization research is part of the ongoing work of all search companies, and that there will “never be an ‘ah-ha! moment.'”

In the meantime, Google has released a taste of how users might interact with a personalized search tool in the future. Even this simple application of the technology illustrates what experts like Mike Grehan now argue: that the concept of a single set of rankings on a given phrase (what search marketers often call “the algorithm”) may soon be obsolete.

If one man’s trash is another man’s treasure, that must explain the comment by Yahoo’s Tim Cadogan that “integrated information sets” should be developed by editorial staff and product managers so that they could be presented automatically at the top of the search results page for the most common types of user query (such as weather). In some ways it seems as if the four largest portals are today working towards doing more of the “integrated answer set” work that Ask Jeeves itself has abandoned.

Moderator Danny Sullivan asked panelists if there is a drawback to consolidation in the industry. Is there a downside to the dwindling variety in ranking methodologies and index content? Ask Jeeves’ Paul Gardi was perhaps the most forthcoming of the group, offering a couple of “out-of-the-box” suggestions that seemed unrelated to his own company’s agenda. Gardi believes that we’ll see the emergence of more specialized indexes and more variations in the breadth and depth of content in the major indexes. This prediction has a ring of truth about it; it was made during the week that Yahoo announced an initiative to add more public-interest content (such as public radio archives) to their index. Not only does this give one hope that some of the romance of search will return – the idea that by using one tool as opposed to another, one might unearth some highly specialized information – but it could foreshadow the re-emergence of metasearch as an indispensable tool for users seeking to simplify the research process across multiple databases.

AOL’s Campbell said only that there needs to be “public discussion of the problem” of oligopolistic control over search results, whereas Nevill-Manning and Gardi made the important point that search tools should key on diversity by pointing users to a variety of different stores and reviews if they’re looking for information about, say, digital cameras. Most of these companies seem to understand that search engines lose their credibility when they turn into glorified referral services.

Some of the panelists’ most innovative comments were indeed focused on the advertising side of the equation. All pointed to a data-centric future, noting that few advertisers today make good use of the immense amount of data available to them. And there was nuanced discussion of return on investment, again implying that current advertisers are unsophisticated in their grasp of the ROI on search keyword buys. Gardi, for example, argued that advertisers need to better understand different buying cycles. Cadogan, in alluding to a “concept-based” approach (as opposed to keywords per se) seemed to be thrusting in the direction of the “media buy” model, but he also thought more advertisers would be understanding the “conversion connection” and doing more sophisticated “profit tracking.”

Foreshadowing comments that would be made three weeks later by Microsoft CEO Steve Ballmer, AOL’s Campbell predicted that the current trends in search advertising will usher in a “reformulation and reinvention of how other types of advertising work.”

The latter part of the session was devoted to considering the dystopic vision that Sullivan conjured for argument’s sake: a day when search becomes “all pay for play.” In keeping with the public relations spin that had been elaborately constructed that week for the announcement of Yahoo’s new paid inclusion program, Cadogan stated that Yahoo’s goal is to “discover all the content on the web for free,” and that 99% of the current iteration of the Yahoo index is pages discovered by their crawler or pages added through special arrangements with public agencies such as libraries. Using a phrase that was repeated by the company throughout the week, Cadogan added that Yahoo was committed to maintaining an “iron wall” between the process of paying to be included in the index and the algorithmic decision about how high to rank a particular page on any given search query.

But many attendees had already made up their minds on Yahoo’s new spin on paid inclusion, deciding that while not exactly untrue, it was at least misleading. Nevill-Manning’s concise statement of Google’s firm policy on the matter – “NO pay for inclusion, and a complete separation of the search index part from the money part,” – was greeted with loud, pointed applause.

Should AOL survive in its present form, it may be headed in a different direction. Campbell hedged on his vision for index results, making the good point that algorithmic results are “subject to spam,” and arguing for a need to deal with commercially-oriented queries “more systematically.” Might AOL’s diffidence towards Google’s philosophy on paid inclusion have some bearing on their current partnership? For what it’s worth, my Magic 8-Ball says “Better Not Tell You Now.”

Search Headlines

NOTE: Article links often change. In case of a bad link, use the publication’s search facility, which most have, and search for the headline.

For some beta testers, it’s about buzz, not bugs
CNET Jul 22 2004 5:28AM GMT
Google killers
APC Mag Jul 22 2004 4:51AM GMT
Yahoo attracted by depth of camera phone field
CNET Jul 22 2004 4:23AM GMT
The Strategy of Search
WebProNews Jul 22 2004 2:12AM GMT
Driving Traffic (Literally) to Brazilian Petrol Stations
ClickZ Today Jul 22 2004 1:49AM GMT
Mass Customization With a Personalized Experience
ClickZ Today Jul 22 2004 1:48AM GMT
Contextual and Behavioral Targeting: Worth a Try
ClickZ Today Jul 22 2004 1:48AM GMT
WSJ, Embrace RSS
ClickZ Today Jul 21 2004 11:06PM GMT
Internet needs brand ads to maintain spend growth
newmediazero Jul 21 2004 5:21PM GMT
Mitsui Launches Paid Search Network in Japan
ClickZ Today Jul 21 2004 4:51PM GMT
A Web of Ways to Bring Users to Your Site Jul 21 2004 2:56PM GMT
Google-envy and the Quest for Local Search
Media Post Jul 21 2004 2:54PM GMT
Net threat to other ad media can only get bigger
Sydney Morning Herald Jul 21 2004 2:34PM GMT
A Critical Mass of Advertisers Makes Contextual Ads More Relevant
Traffick Jul 21 2004 1:07PM GMT
Escher Staircases on the World Wide Web
First Monday Jun 8 2004 2:52PM GMT

Related reading

Simple Share Buttons