The State of the Search Engine Industry

Noted search engine experts and analysts explored the major themes and trends driving the current and future state of the industry in a lively, wide-ranging forum at the Search Engine Strategies conference.

The Search Industry Analyst Roundtable was led by moderator Safa Rashtschy, a financial analyst at U.S. Bancorp Piper Jaffray, who recently published an extensive report on search as a growing financial sector.

The controversial subject of paid inclusion started off the session, concerning the rising number of paid inclusion and XML-fed listings in a search engine’s supposedly non-paid listings. Panelist Greg Notess, editor of Search Engine Showdown said, “A lot of users have just assumed that people paid to get into the results. They’re largely unaware of the paid elements.”

Panelist Chris Sherman, editor of SearchDay, said, “I personally don’t think there’s any deception. The engines only see the abstract of the content [when viewing an XML feed”.” He continued, “The relevancy algorithms we have today may not work in the future when we have 500 billion pages, so maybe inclusion will be more necessary.”

“Google doesn’t need it,” said Danny Sullivan, the industry’s most well-known voice and editor of Search Engine Watch, also a panelist. “But, I do think that the paid versus unpaid issue is coming to a head.”

Brett Tabke of Webmaster World agreed. “The FTC or the government is going to have to look at it more closely,” he said.

Rashtschy switched subjects, asking the panelists if they thought there is an opportunity for specialized search and vertical search.

“Singingfish [multimedia search engine” came out of nowhere,” said Tabke. “So there could be other successes.”

Notess said, “Most of the vertical portals failed after the crash. Singingfish may be able to make it, along with healthcare and a few others. They need to balance between making money and having enough PR to educate the public about their resources. There is room for it – the general search engines might not work as well when the size of the Web grows.”

“Travel verticals are doing great,” Sherman concurred. “Expedia, Travelocity, for example. And the major commerce searches like Dealtime. Watch for more transaction oriented models like these to apply to other markets.”

Going back to paid inclusion issues, Rashtschy noted that there is a difference in the payment methods of the various programs availabe. Some site owners pay on a flat-fee basis for a fixed period of time per URL, whereas others pay on a cost-per-click (CPC) basis. The search engines have been accused of boosting the CPC listings above the others in an attempt to boost their profits, he said, asking the panelists for their opinions on whether the engines should separate the two types of inclusion products from one another other.

“There are issues with the ‘Trusted Feeds’ CPC program,” said Sullivan. “The flat fee program is okay, more like a paid submission program. I’d like to see the industry come up with a mechanism to separate the results without being ‘in your face’ as a consumer.”

“We haven’t had pure algorithmic search since the early days, before search engines found out they could make money,” said Notess.

The panel discussed the future of search as it relates to devices, such as phones, PDAs, and other non-PC hardware. All the panelists agreed that devices will continue to evolve as both an accessory and a necessity for users. Tabke said that he can run his entire operation from his pocket nowadays, reading his email en route from the airport, and updating his site via PDA from a wireless hot spot.

Sullivan suggested that most devices probably don’t really need a custom results set; rather, just a different format. He said that his experiences thus far in using his phone for search had been disappointing.

Rashtschy theorized that someday it will be possible to transmit a user’s location through a cell phone to Google automatically to take advantage of geo-search, to find relevant local information. Notess concurred, but said that for phones to really take off as a search device, other than to provide weather, directions and restaurant information, “We really need to change the habits of users. We need to build demand for geo-search, to teach users to search that way,” he said.

“Users need to get used to real-time transactions,” Rashtschy said. He gave a personal example of ordering books from Amazon on his phone while waiting in a doctor’s office.

Google and Overture have both recently launched contextual ad programs, wherein text ads from their respective pay-per-click (PPC) network advertising programs are also displayed alongside non-search content such as newsfeeds, discussion boards and editorial content (the ads are displayed based on keywords found on the page).

“Is it the same as search advertising?” asked Rashtschy.

Sullivan said, “Contextual ads have nothing to do with search, except that the search engines have extra inventory they need to sell. It’s not a ‘search mode’ and it’s not the same user behavior.”

Sherman also had three main concerns about the programs. “It’s not search,” he said. “I’m hearing from advertisers that the conversions are lower. Second, mismatches frequently occur.” He cited the example of a major airline ad displaying on a news story about an airline crash. Finally, he said, “Search engines are taking their eyes off of search. That’s why they failed in the late 90s.”

The panelists were asked to weigh in with their own personal “wish lists” for the search engines, and also to prognosticate one year from now into the future of search.

Notess said, “As a librarian, I’d like to see truncation and proximity locators built into search. One year from now, I think we’ll still see Google as the industry search leader.”

Sherman said he is a big fan of personalization, but would like the ability to override it in a search environment. He agrees that Google will still have the strongest mindshare and leadership next year. “But it’s Google’s game to lose now,” he said, “and it’s not unforeseeable.”

“I’d like to see a subscription service for an ad-free search environment,” said Tabke. “I’d also like to see personalization, saved searches, and more ‘related searches’ like Teoma does.” One year from now? “We’ll still be talking about, ‘What’s Microsoft going to do?’ We’ve been saying that for five years now. I think that IBM is the one to watch.”

Sullivan wants specialty database and search engines that “read my mind,” he said. Next year, he said he thinks it will be status quo, although “There is still room for a new player. Google got where it is today by being good, when all the others were so bad.”

Dana Todd is a founding partner of SiteLab International Inc. She obsesses on the flagrant misuse of the apostrophe because of her journalism and advertising background. She is a frequent speaker on Internet marketing topics, including search engine strategies and link-buying.