Yahoo drew widespread criticism for new paid inclusion programs launched in March. What is Yahoo doing? How does it impact the advertiser and searcher? Are there changes Yahoo should be making? The first in a series of articles looking at paid inclusion.
In March, Yahoo rolled out its new Site Match paid inclusion program, which quickly drew wide-spread criticism. It was a terrible blow for Yahoo in its quest to win searchers away from Google. The move infuriated plenty of advertisers, as well.
Why the outcry? After all, paid inclusion is hardly new. It’s been offered for over two years. In this article, the first in a series on paid inclusion, we’ll examine why paid inclusion blew up in Yahoo’s face and will likely remain constantly mentioned by reviewers as a drawback to its search service.
What is Yahoo doing? How does it impact the advertiser and searcher? Are there changes Yahoo should be making? Let’s dive in.
It was in mid-February that Yahoo debuted its new “Yahoo Search Technology,” a brand new search engine to power searches on its web sites around the world. Previously, the company had used Google.
Just like Google, Yahoo’s new search engine gathers listings by crawling the web. These crawler-based listings are equivalent to a newspaper’s editorial columns. When you do a search, you’re supposed get back what the search engine thinks is most relevant to your query based on its ranking algorithm. That’s the search engine equivalent of editorial judgment.
Payment is not supposed to play a role in what gets ranked tops. If you want a guaranteed ranking, you instead need to buy paid placement listings. These are the search engine advertising results that run at the top and bottom of Yahoo’s search results page, under the Sponsor Results heading.
Just like Google’s crawler (and other crawlers), Yahoo’s crawler is not perfect. In particular:
As said, all the flaws above with Yahoo’s crawler are true of Google’s as well. The companies differ in how they seek solutions to the flaws.
Yahoo views payment as one of several ways to correct its problems. Yahoo’s new paid inclusion programs allow site owners to pay for guaranteed inclusion of their URLs and to get regular revisits. Can’t afford to pay? If you’re a non-profit organization, Yahoo may provide this solution for free.
Google believes payment has no role to play in fixing its crawler problems. It views any payment linked to its crawler-based results as potentially tainting those results. Instead, it seeks to solve its flaws purely through unpaid means.
Who’s right? It depends on your perspective.
As we’ll see later in this series, the lack of any type of paid solutions program to get help with crawler-based listings at Google has caused that company to come under fire from frustrated web site owners and marketers in the past, such as happened late last year. A paid support program, something’s that’s part of Yahoo’s paid inclusion program, might have spared Google these problems.
However, there’s no doubt that avoiding a full-blown paid inclusion program has given Google the upper hand, in terms of reputation. That’s because Yahoo’s use of paid inclusion has clearly caused some to view its results as tainted, despite its strong denials that paid inclusion content will be boosted.
Just look at what happened after Yahoo unveiled its new program in March, to see how paid inclusion hurt Yahoo:
“Questionable Results at Revamped Yahoo”
That was the headline of a Washington Post article following the program’s release. Only two weeks after debuting its brand new search technology, paid inclusion puts Yahoo’s search into ill repute.
“Do Internet searchers get good, accurate information? Or are the results of the search skewed to favor those who’ve paid to be in the index? The jury’s out on that one.”
That was from a Boston Globe story. Thanks to paid inclusion, readers get the impression that Yahoo’s search engine is on trial for wrong-doing. Will these same people remember to find out if the charges stuck, or will they stick with another search engine that doesn’t seem to be on trial?
“Yahoo executives say the payments won’t improve a site’s ranking on the list of results that appear after a search. But at the same time, Yahoo acknowledged that there will be no distinguishing marks to alert web surfers that a company had paid to be included.”
That’s from a Wall Street Journal story. If you understand the nuances of paid inclusion, then you might believe that paid and unpaid content can be mixed without hurting relevancy.
Unfortunately for Yahoo, most searchers don’t even know what paid inclusion is, much less its nuances. Instead, they’ll come away from reading an article like this assuming that Yahoo is simply selling its results and not disclosing the fact.
“New Search Engine Calls Up Resentment at Yahoo”
That was the headline of a Los Angeles Times article. Resentment? Wow. Many search engine marketers hoped Yahoo’s new search engine would be a lifeline to escape the Google domination they’ve felt. But in two short weeks, Yahoo transformed that potential goodwill into resentment for some.
Paid inclusion on Yahoo itself is new, but the concept of paid inclusion is not. Paid inclusion has been used by other search engines such as MSN, AltaVista and Inktomi (the last two now owned by Yahoo) for years. Why did Yahoo get such grief over it now?
From the advertiser perspective, it’s because Yahoo’s basic paid inclusion program Site Match seems to provide little value. If you believe the convoluted explanations that Yahoo provides, any reasonable person is going to say, “Thanks, but no thanks.” This is an issue I’ll explore more later in this series.
From the searcher perspective, it’s because Yahoo’s brand new search engine faces greater scrutiny in the new search engine wars to win users from Google.
Everything Yahoo does will be compared to Google. Google tells an easily understood story as to why paid inclusion should be wrong, wrong, wrong. Just look at this most recent example from Google’s IPO filing:
Some of our competitors charge web sites for inclusion in their indices or for more frequent updating of pages. Inclusion and frequent updating in our index are open to all sites free of charge. We apply these principles to each of our products and services. We believe it is important for users to have access to the best available information and research, not just the information that someone pays for them to see.
In contrast, Yahoo tells a complicated, hard-to-believe tale to justify the existence of paid inclusion.
To illustrate the difficulty in rationalizing paid inclusion, I wrote an article called The Mixed Message Of Paid Inclusion in 2002. I contrasted all the wonderful things that search engines said about their crawling capabilities to the dire tales they told when pitching paid inclusion to advertisers.
The difficulty with mixed messages hasn’t changed, as you can see using current material from Yahoo:
Yahoo…today announced that it has created a more comprehensive and relevant search experience for users through the deployment of its own algorithmic search technology —Yahoo Press Release, Feb. 18, 2004
Sounds like good news for searchers. But wait! What are advertisers being told about Yahoo-owned Overture Site Match, which feeds paid inclusion content into Yahoo’s search results?
Eliminate guesswork: Ensure that your pages are reviewed and included in the search index quickly and refreshed frequently. No waiting for search engines to find your site or guessing which content will be included. –Overture Site Match product page, May 18, 2004
On the one hand, searchers are told that Yahoo has a comprehensive and relevant search engine, which you’d also assume means it’s fresh. On the other hand, site owners are told that Yahoo’s search engine apparently just guesses about what to include and may not refresh that content frequently.
Is it any wonder that Yahoo’s gained bad press after unveiling its new programs? Either you have a great search engine or you don’t. Trying to play it both ways simply doesn’t fly.
Google certainly had a field day watching Yahoo try to justify its program. Google had just come off one of the worst periods of publicity it had ever known (this being before the Google Gmail announcement in April and major privacy concerns that have followed). Last December’s upset over Google ranking changes spawned a number of anti-Google forum discussions and articles.
What on earth could save Google? Yahoo’s complicated paid inclusion program came to the rescue. After weeks of having its own results questioned, Google got to sit back and watch Yahoo’s program get put under a microscope.
Google also brought out company cofounder Larry Page in unprecedented fashion for major publications. Gone were the typically cautious or limited comments about the competition Google’s normally given. It’s search war now, the gloves are off, and by not having paid inclusion, Google seemingly occupies high ground. Page confidently shot quotes like these not just across Yahoo’s bow but directly amidships:
“Any time you accept money to influence the results, even if it is just for inclusion, it is probably a bad thing.” — New York Times
“It’s really tricky when people start putting things in the search results,” — Wall Street Journal
If you need a last sign of Yahoo’s failure on the propaganda front, take note that Yahoo Watch has now been founded by Daniel Brandt.
Brandt created the anti-Google site of Google Watch back in 2002, when Google’s incredible rise in popularity was starting to raise concerns. Until now, Brandt has been content to hold Google solely responsible for the ills that often are applicable to its competitors as well.
Brandt did always say he might target other services, if he felt they warranted it. Now Yahoo’s paid inclusion programs managed to draw his ire. Google is no doubt pleased to find Yahoo is now part of the axis of search evil it formerly occupied alone.
With so much bad publicity over paid inclusion, you’d wonder why Yahoo would do it at all. As we’ll examine further in this series, the ability to earn more money is certainly a major factor. However, we’ll also look at Yahoo’s defense that paid inclusion can help search relevancy.
I’ll also cover the “level playing field” myth, which Google has employed so effectively. Google’s results are not necessarily more “pure” than Yahoo’s, just because Google lacks paid inclusion. There are plenty of commercial sites vying for good positions at Google just as Yahoo. It’s simply that Google itself doesn’t earn money directly off its editorial results.
I also expect Yahoo will likely begin fighting back against charges that it is less pure than Google by pointing out that Google’s AdSense program gives Google potentially as much incentive to skew results as does Yahoo’s paid inclusion program.
AdSense puts Google ads on pages across the web outside of Google. Obviously, if Google drives traffic to pages carrying its ads, the company may earn more money. It’s something Google strongly denied it would ever do when AdSense was launched, a denial it repeated again recently when I looked in December at allegations of Google favoring sites with AdSense content. Nevertheless, as with Yahoo, the incentive for favoritism is there.
I’d already planned to cover the AdSense issue as part of this series. However, I was especially surprised to have Yahoo raise the issue itself, when I was discussing this series with them last week. I don’t recall Yahoo employing the “Google has incentive to be bad” argument before. That’s why I suspect it may emerge as a new line of defense, if paid inclusion criticisms continue.
The series will also explore the future of paid inclusion. Will the other search engines turn toward or away from paid inclusion. Will Yahoo decide it needs to do inline disclosure of paid inclusion URLs, something I certainly hope will happen. All are areas I’ll be getting into as this series progresses.