Going Beyond FTC Paid Inclusion Disclosure Guidelines

A longer version of this story for Search Engine Watch members explains how to decipher listing URLs at Yahoo to spot paid inclusion content, how some results for popular queries may be hardcoded, finds paid inclusion showing up in only 2 percent of listings based on a small sample of 700 queries, and looks how sponsor labeling in Yahoo’s directory hasn’t appeared to hurt the sites shown there. Click here to learn more about becoming a member

In the second part of this series on paid inclusion, I wrote that while Yahoo currently meets US Federal Trade Commission recommendations on disclosing paid inclusion, I wanted to see them go beyond this.

To me, the FTC guidelines are dated. They were drafted in 2002 when paid inclusion was still fairly new, when people had more faith in search engine promises that rank boosting within paid inclusion wouldn’t happen, and when the main concern was on fixing the disclosure situation with paid placement.

Today, I’d argue there’s a lot less faith that paid inclusion content won’t be boosted. Because of this, I see disclosure beyond what the FTC requires as essential for any search engine that wants to use paid inclusion yet retain the trust of its searchers.

Lack Of Faith

Since paid inclusion was introduced, I’ve written several articles showing how paid inclusion content has managed to skyrocket to the top of the results at times on different search engines.

Explanations such as “blending problems” and “link flux” have been trotted out to account for these situations. I can believe and understand the explanations. Nevertheless, they’ve also been convenient problems for the paid inclusion programs to have, as they’ve helped generate revenue.

Toward the end of last year, a BusinessWeek investigation found that some advertisers were convinced paid inclusion helped them rank better, further hurting faith that rank boosting doesn’t happen.

No doubt, some advertisers may mistakenly assume they got a rank boost. For example, a person in the BusinessWeek story did paid inclusion with LookSmart and found an immediate gain with MSN.

This wasn’t because paid inclusion listings were boosted at MSN. Instead, it was because at that time, MSN placed ALL LookSmart listings (paid inclusion or not) before the crawler-based backup results from Inktomi. In other words, it wasn’t paid inclusion URLs that were boosted. It was LookSmart URLs that were boosted, and plenty of these did not come from paid inclusion.

Regardless of advertiser confusion, it’s impressions that count. If enough people believe paid inclusion provides a boost, then that slowly gets taken as fact. In addition, there are plenty of sophisticated advertisers who are not confused and fully believe paid inclusion helps them rank better. I’ve talked with some myself at our regular search conferences.

Show, Don’t Tell

The solution to the faith problem is better disclosure. Let the general public easily see what paid inclusion URLs show up in the results. That will let those who care about the issue make their own assessment, rather than having to just trust the word of a search engine.

In particular, I’d like to see “inline” disclosure provided. This means disclosure that happens right within the search results. Somehow, someway, make it easy to spot paid inclusion URLs.

It needn’t be blatant. A simple icon next to a paid inclusion URL would be enough to help those who care. In my keynote earlier this year at Search Engine Strategies in New York, I offered up a “blue dot” example. Here’s how it might look using Yahoo results for kameleon remote that I generated last week:


See the two blue dots after the second and fifth listings? They are flagging that these are paid inclusion URLs. How do I know? There are certain clues that can be found in the listing URLs, as explained more in the longer version of this article for Search Engine Watch members.

Problems With Deciphering

Certainly anyone could just go through and try to do a manual check using URL clues themselves. The Yahoo Watch search proxy makes this even easier, spotting suspected paid inclusion URL strings and putting a big $ symbols in front of them.

There remain problems with this, however. Daniel Brandt, who runs the Yahoo Watch search proxy, and I had a number of back and forth emails earlier this year as we tried to determine what was paid based on URLs strings. I’ve seen similar puzzlement on search forums, such as here on WebmasterWorld.com.

If it’s hard enough for those who live and breathe search to make these determinations, what hope is there for the average consumer? In addition, the exact strings used that identify paid inclusion URLs are subject to change and potentially could be entirely hidden.

No one should have to decipher such things. A few blue dots or something similar would easily let someone see the impact of paid inclusion on results. If you consistently see lots of dots, you might feel paid inclusion is getting weighted too heavily. If you see only a few or some scattered ones, you might have more faith that no boosting is happening.

Yahoo Response

So how about it, Yahoo? Can we have a few blue dots?

“Obviously, we’ve had this debate internally, and there are pros and cons,” said Tim Cadogan, vice president of search for Yahoo, when I spoke with him about the issue earlier this year.

One chief reason Yahoo says for why it is so far not doing inline disclosure is a fear of influencing searchers somehow against these results.

“The fact that someone paid for a value added service doesn’t have anything to do with how they are ranked. It’s not a meaningful part of the equation. Therefore, sharing that information with the user doesn’t give a lot of insight,” Cadogan said. “If we put a disclosure on each result, you can then argue about the relative ranking for eternity.”

I agree entirely. There are some people who will simply view any paid inclusion URL as somehow tainted. If even one shows up, some will argue that it may have been given an unfair boost.

Unfortunately, that’s also a consequence of mixing the paid inclusion URLs with the free listings. There’s no way I see around it. But at least with disclosure, people can’t argue that anything is being hidden or hard to discover.

Too Much Info? Make It An Option!

Yahoo also argues that inline disclosure might be too much information, something especially an issue given that the company is currently beta testing some cleaner presentations of its results.

“The page is already complex, and we don’t want to overload users. It’s not clear that it would helps the vast majority of users,” Cadogan said.

A solution I suggested was to make disclosure an option. For users who care, why not let them set their preferences to have blue dots or some type of equivalent inline disclosure be displayed.

To me, it’s a perfect solution to balancing too much information with providing the information some may care about. It’s certainly better than forcing people to do URL deciphering or to depend on third party sites to get disclosure. Cadogan said the idea was something that would be considered.

It’s also important to note that the disclosure could be done creatively. Yahoo’s content acquisition program is designed to bring in content from all over the web that might otherwise be missed. As I’ve said earlier in this series, why not be proud of this by perhaps a showing a special CAP logo or a “Quality Reviewed” mark to subtly flag any CAP content. That makes it a feature for users, though I’d still argue that paid CAP content would need to be distinguished from unpaid.

Segregate Paid Inclusion?

What about the idea of completely segregating paid inclusion results from those listed for free? Yahoo sees disadvantages to this.

“Introducing another section would be quite confusing, I think,” Cadogan said. “In the past, I can’t think the directory and web layers were optimal. I don’t think we’ll want to go back to something like that.”

Cadogan is referring to how Yahoo used to first show results from its human powered directory, then show results only after this that came from crawling the web. It stopped doing this late 2002.

I agree that adding a separate layer isn’t optimal. It is certainly in Yahoo’s financial interest to have paid inclusion content mixed in with the free content. However, pulling it out could also arguably hurt relevancy and in fact penalize some sites unfairly.

By the way, you might be wondering if the FTC has any new plans regarding paid inclusion disclosure given all the recent news. Last year, a News.com article found the FTC wanting the search engines to help consumers by providing as much disclosure as possible, but there was no suggestion of new guidelines.

I followed up with an FTC contact recently and was told the situation remains the same: there are no new actions or disclosure guidelines to speak of relating to paid inclusion.

Mixing Feeds Is The Future

Cadogan sees the ability to mix data from various feeds together into a single list as essential to search success. Right now, Yahoo is working to blend free feed content, paid feed content and crawler content. Additional data might also be mixed, such as local content, product listings, news headlines and so on.

The challenge in mixing multiple databases is that it’s hard to easily rank everything according to the same rules. Feed content may lack link analysis data or page formatting information that search engines use to rank ordinary HTML pages. How do you come up with a system that blends two completely different data types together yet somehow ranks everything properly?

For Ask Jeeves, it was a step too far. A day after Yahoo unveiled its paid inclusion program in March, Ask Jeeves made a timely announcement that it was dropping its own feed-based paid inclusion. The main reason was that Ask Jeeves found it too difficult to mix the different data types fairly.

It’s a challenge for Yahoo, as well, but one the company want to keep working on.

“We agree, feeds aren’t simple. That’s why we are making it our investment. We can understand why people might find it difficult. But if we don’t take on that challenge, we’re going to limit what search engines can do,” Cadogan said.

In the next part of this series, we’ll look more closely on how Yahoo’s CAP program works to take in feed content from paid and non-paid partners. We’ll also look at the efforts Google is doing to reach the same goals but without involving payment.

Want to discuss issues of disclosure raised in this article? You can do so in this special thread in our Search Engine Watch Forums and even vote on some ideas. Stop by and visit here: How (or Should) Paid Inclusion Be Disclosed?

In the second part of this series, I also talked about ideas of helping webmasters get support for their free listings outside of offering paid inclusion. I’ll be revisiting this issue more in the next installment of this series. You can suggest services you’d like to see in this forum thread: What Organic Search Support Services Would You Want?

Finally, we have a third thread inviting site owners using Yahoo’s paid inclusion programs to vote on and share what impact it’s had on their free listings, if any. You’ll find it here: Will Yahoo Search keep its promise?

A longer version of this story for Search Engine Watch members explains how to decipher listing URLs at Yahoo to spot paid inclusion content, how some results for popular queries may be hardcoded, finds paid inclusion showing up in only 2 percent of listings based on a small sample of 700 queries, and looks how sponsor labeling in Yahoo’s directory hasn’t appeared to hurt the sites shown there. Click here to learn more about becoming a member

Search Headlines

Due to technical difficulties, search headlines are unavailable today.

Related reading

Simple Share Buttons