The Bumpy Road To Maximum Monetization

Last week, the mainstream press gobbled up the battle between Overture and Google for the hand of AOL as a search partner. Google won that engagement, causing a plunge in Overture's stock price and a renewed recognition of just what big business paid listings represent.

Overture alone expects to generate over 1/2 billion dollars in revenue this year. Add in revenue from all other types of search engine marketing activity, and the SEM industry figure probably approaches if not exceeds $1 billion.

Clearly, search engine marketing is no cottage industry, though it continues to be largely ignored beyond what Overture is doing. An example of this is how little coverage has been expended in the mainstream press on LookSmart's unprecedented move last month to a pure cost-per-click model for its commercial listings. The change could potentially cause LookSmart's revenue to eclipse what Overture earns.

Currently, most search engines come nowhere near monetizing all the links displayed on their search results pages. However, the quest for greater profits is likely to change this, especially if the search engines think it can be done without hurting the relevancy. LookSmart intends to be the pioneer in maximizing monetization, and the other search engines will be watching to see if the company is successful.

Giving Away Prime Link Real Estate

To understand how much more search engines could monetize, let's examine the major links presented on the search results pages of the three biggest portals, AOL, MSN and Yahoo. All three have areas where they make money directly by selling links through internal ad programs or where they indirectly benefit by promoting their own content, such as AOL's "Recommended Sites," MSN Search's "Featured Sites" (editorial results also appear here) and "Inside Yahoo."

Next, all three players display "Sponsored" links. Overture provides these to MSN and Yahoo while Google now powers them at AOL. Finally come the "editorial" links, where money is not supposed to influence how sites are ranked for particular queries. For the moment, think of these as "free links" that earn the portals no money, though they actually do earn some revenue, as we shall see.

Here's the link breakdown for a search on "book stores" at each portal. Bear in mind that for more unique queries, the number of "sponsored" links might drop or they might not appear at all. However, the figures below are fairly representative for any generally popular query.

Links AOL MSN Yahoo Average
Total Links 13 18 27 19
Internal Links 8% 22% 7% 12%
Sponsored Links 15% 17% 19% 17%
Free Links 77% 61% 74% 71%

As you can see, sponsored links only take up on average 17 percent of the available "link real estate" on the results pages for these portals. If this relatively small percentage of links can generate so much money for the portals, then imagine how much the portals might earn if they could convert the remaining 71 percent of "free links" into revenue generators.

Some of this does already happen. For example, Yahoo requires commercial sites to pay an annual fee for review and inclusion into the commercial portions of its directory. If accepted, then these sites have a chance of appearing in the editorial "Web Sites Matches" area of Yahoo's results. It's important to note that they aren't guaranteed to show up for any particular query terms, but by being included, they have a shot at getting traffic via Yahoo.

Similarly, both MSN and AOL (until summer) get some of their editorial results from Inktomi. In turn, Inktomi has "paid inclusion" programs that allow site owners to pay either an annual fee or use cost per click (CPC) pricing to get their pages included in its database. The portals that carry Inktomi's results share in some of the paid inclusion revenue.

Appeal Of Recurring Income

Annual or CPC pricing is attractive to search engines because it represents more value to them for the link real estate they provide. For example, Yahoo previously had charged a one-time review fee for commercial sites to be included in its listings. If accepted, the site never needed to pay again. The site might receive thousands if not hundreds of thousands of clicks over the period since it was accepted, all for having paid between $200 to $300 at some point in the past.

So much traffic for so little price meant that despite paying a submission fee, Yahoo still represented a huge source of essentially "free" traffic to site owners. However, Yahoo felt it deserved more for providing so much traffic, so it rolled out new annual pricing in December. The company wisely chose not to make this retroactive to sites already listed, but for new sites signing up, Yahoo has increased its potential income in one swoop.

LookSmart -- a major provider of listings to MSN -- was in a similar situation as Yahoo. Until last month, it charged a one-time fee for sites to be reviewed for inclusion into its commercial listings. If accepted, you received "all you could eat" traffic. LookSmart also had a different program, "LookListings," aimed at large businesses. This program gave you greater representation in the LookSmart database and thus greater potential traffic, but those participating in it had to pay continuing cost-per-click fees.

Now the one-time fee programs are completely gone at LookSmart. The company bypassing the annual fee choice that Yahoo made and went instead to cost-per-click pricing for all commercial listings. By doing this, LookSmart hopes to greatly decrease the amount of free traffic it has given to commercial sites in the past. In short, you "pay to play" in LookSmart.

First Real Test Of Pay-To-Play Editorial

What makes the LookSmart move unprecedented is that never before have we seen such a wide range of businesses be told that they need to pay a substantial amount of money to be included in a major search engine's editorial listings full time. If LookSmart is successful with this transition, then we'll probably see other search engines follow suit by taking a much harder line in monetizing their editorial results.

LookSmart would dispute the "substantial amount" part of my statement above, saying that sites are only required to spend $230 to be included with it in the first year, with that dropping (assuming no price increase) to $180 for subsequent years. In contrast, Yahoo charges a $300 annual fee.

However, the "full time" portion of my earlier statement is crucial. The minimum charges at LookSmart only get a site included in its listings until 100 clicks have been registered in a particular month, which many sites might find get used up in only a few days, if even that long. After that, sites have to either pay more money or risk disappearing until the following month. How much more? Many are reporting that LookSmart is telling them they need to pay in the hundreds of dollars per month to show up.

No other paid inclusion program feeding editorial results is so draconian as this. As said, Yahoo still gives "all you can eat" or unlimited traffic for a low price. The major crawler-based search engines, such as Inktomi, FAST and AltaVista, typically list the home pages of web sites continuously for free. On the off chance this doesn't happen, you could pay these crawlers in the range of $30 to $80 to be included per year in their respective databases and receive unlimited traffic, in return.

In short, where other search engines may charge small businesses $30 to $300 per year for full-time inclusion, LookSmart's change has shifted it to charging the same group $180 to $30,000 or more per year. LookSmart's attitude is that the traffic it has delivered to sites for free in the past is worth at least this much money if not more, so the company would be foolish not to maximize its revenue, especially if it wants to continue providing listings to MSN and other partners.

LookSmart is correct in that the traffic it has given away in its editorial listings has been undervalued, and that's true of the other search engines, as well. Commercial sites have indeed gotten a "free ride" on the search engines in years past (though arguably, those search engines needed to include a core of commercial sites in order to attract users). Now the fares are going up and will likely continue rising as long as sites see value in paying.

Confusion Caused By Greater Monetization

It remains to be seen whether the quality of search results is going to suffer, as the search engine industry inevitability moves toward greater monetization. What I call the "florist" argument gets employed as a defense that this won't happen. It goes something like, "If I list 100 online florists, does it really matter if another 25 decide not to be included, because they won't pay?" Probably not, but such answers really can only be measured on a query by query basis. We'll all have to watch and see.

One thing that is clear is that the move to greater monetize editorial results is fraught with potential for consumer confusion. "Am I searching the entire web or just a collection of Yellow Pages ads?," a user may wonder. "Do I have to pay for these programs or will search engines visit me anyway?," site owners may and do ask.

Those looking for answers will get mixed messages. Search engines with paid inclusion programs sell site owners on the idea that payment ensures that key pages are not omitted or allowed to grow out of date. The same companies will then proudly talk about how they have great, comprehensive and fresh results for users. The sidebar article below, "The Mixed Message Of Paid Inclusion," takes a closer look at how specific statements contrast against each other.

Such mixed messages don't create trust in search listings, which is a key reason why the Ralph Nader-backed group Commercial Alert filed a complaint over unclear listings with the US Federal Trade Commission last year. Now trust in search engines may take another hit, due to LookSmart's decision to make its cost-per-click pricing change retroactive to sites that previously already paid the company a one-time fee to be reviewed.

LookSmart emphasizes that it gave those who paid exactly what was promised, a yes or no answer about whether they would be included in its listings. Payment did not cover a lifetime supply of free clicks.

Many site owners and marketers disagree. Those posting on forums, who have contacted me via email or who I've talked with personally at our recent Search Engine Strategies conference in London believe that LookSmart essentially sold directory inclusion, and that to make cost-per-click pricing retroactive is unfair.

For from taking LookSmart for a "free ride," these people paid the required fare asked for when they got on the LookSmart bus and are now surprised, upset and disgusted to find the driver coming back to tell them to pay more after the fact. Many feel LookSmart is ripping them off, and the threat of a class action lawsuit gets sprinkled in their messages and conversations.

Establish Standards Or Have Them Imposed

So what has the effort to maximize monetization brought us? It's kept search engines in business, and that's an important point that cannot be forgotten. However, the haphazard way the search engine industry has lurched toward monetization has brought an FTC complaint and might bring a lawsuit over submission fees. In addition, we've also had lawsuits filed against search engines over ads involving trademarked terms.

These complaints impact every major search engine, regardless of who errs, because consumers and even experienced people may fail to understand the confusing nuances about how search engines sell listings. Google, the "pure" search engine learned this well earlier this year, when the launch of its new cost-per-click AdWords Select program got misinterpreted as Google selling its editorial results.

Alone among the major crawlers, Google has never had a paid inclusion program, feeling that would compromise the quality of its editorial results. It has also always clearly delineated the paid ads that it carries from its editorial results. Nevertheless, confusion associated with the AdWords Select launch -- and confusion not caused by Google -- still made it necessary for Google to post a special note on its home page reiterating that it does not sell editorial results.

Much talk is often made that the search engine marketing industry -- the third party "search engine optimization" firms that work to improve placement in search engines -- needs to have standards. However, the reality is that the search engine industry itself -- the Overtures, the Googles, the LookSmarts and so on -- need standards perhaps even more.

What do site owners get when they pay for submitting something? Is that price going to change down the line? What should they expect search engines to do for free? What should search engine users expect in their search results? Should paid placement listings, which are guaranteed to show up for particular keywords, be labeled using some standardized language? Should paid inclusion links, especially if they are going to be even more present in editorial listings, be flagged in some simple way such as with small icons, as a means of disclosure?

These are all issues that need answers. Either the search engines need to come up with those answers in a more unified voice or the inevitable move to greater monetization will be followed by the inevitable regulation of a profitable, growing and increasingly confusing industry by some governmental agency.

Pay For Placement?
Compendium of articles from Search Engine Watch and other sources about ways search engines are seeking to earn money, such as through paid listings or paid inclusion.

LookSmart Changes To Cost-Per-Click Listings
The Search Engine Report, May 6, 2002

Pay -- and keep paying -- or don't appear, LookSmart told existing and new listing customers last month, in a significant change to how the human-powered search engine lists web pages from commercial web sites.

The Mixed Message Of Paid Inclusion
The Search Engine Report, May 6, 2002

Pages pitching webmasters on the advantages of paid inclusion end up hurting search engines that run these programs, because they imply that the search engines have out-of-date, incomplete listings of the web. A look at the mixed messages that are being sent out by the web's major crawlers.

Overture & Inktomi Out, Google In At AOL, May 1, 2002

Google has been selected by AOL to provide editorial search results and paid listings to AOL's various search properties in the United States, including AOL Search, Netscape Search and CompuServe Search. The deal ousts Overture, which has provided AOL with paid listings since October 2000. Inktomi also loses out in the deal.

LookSmart shares slide after quarterly results
Reuters, April 30, 2002

LookSmart has narrowed its loss but has also seen revenue drop. Watch for next quarter's results to see if the new cost-per-click pricing of small business listings makes a substantial difference.

Overture Scores on Earnings, Yahoo, April 26, 2002,,12_1016411,00.html

Article from before the AOL deal about Overture's rising revenues. Since the AOL loss, Overture says things are even better, not worse, since it hadn't counted AOL into its projections and estimates it will gain revenue in other ways.

Overture beats Street, extends Yahoo deal
Reuters, April 25, 2002

Another article on Overture revenues.

Inktomi Loss Widens, Hurt by Soft Tech Spending
Reuters, April 17, 2002

Inktomi's overall revenue is down, but its search revenue is up slightly, to $13.2 million for the quarter.

Ask Jeeves Narrows Loss in First Quarter
Reuters, April 24, 2002

Short on Ask Jeeves losing $10 million last quarter -- but that's much better than the nearly $400 million it lost for the same period the previous year. Revenue dropped by $3 million to $16 million.

Yahoo Posts Loss, Sees Light at End of Tunnel, April 10, 2002,,12_1007521,00.html

Yahoo posted its sixth-straight quarterly loss, $54 million, but revenues of $193 million were above analysts' expectations.

The quest for perfect search, April 18, 2002

With search engines offering paid listings, is it the end for old-style "search engine optimization" of free listings. Perhaps, but companies that make the move to being search engine marketing experts should still survive, writes Per Koch.

Consumer Group Asks FTC To Investigate Search Ads, July 17, 2001

Is it deceptive advertising to include paid listings in your search results and not clearly label them as ads? A group backed by consumer advocate Ralph Nader believes so, and it's asking the US Federal Trade Commission to take action against seven major search companies.