FTC Recommends Disclosure To Search Engines

The US Federal Trade Commission has made a landmark recommendation to the search engine industry that it should improve disclosure of paid content within search results. The action came in response to a deceptive advertising complaint made last year by watchdog group Commercial Alert.

"As a general matter, clear and conspicuous disclosures would put consumers in a position to better determine the importance of these practices [use of paid content” in their choice of search engines to use," the FTC wrote, in a letter addressed both to search engines specifically named in the Commercial Alert complaint and the industry general.

Commercial Alert filed its complaint last July, claiming that a lack of disclosure about paid content integrated into search results constituted "deceptive advertising," something the FTC regulates against. The group, backed by consumer advocate Ralph Nader, sees the FTC's action as a victory, it said.

"We thank the FTC for their investigation and effort to protect citizens from sleazy search engines," said Gary Ruskin, executive director of Commercial Alert.

In the year since the complaint was filed, several search engines have made improvements that even Ruskin recognizes. However, there is still more that need to be done, he said.

"Practices have gotten somewhat better since I filed the complaint. But, some search engines have a long way to go," Ruskin said.

I've not yet queried the major search engines about their reaction to the FTC letter. Moreover, I'm going to wait a bit before doing so. I'd prefer to let the search engines have time to actually read what the FTC is recommending. This should provide more thoughtful answers about how they see themselves measuring up and any actions they plan to take. I'll then provide these responses in a follow-up to this article.

A Reuters article listed below does provide some preliminary reaction from search engines. Comments are as you might expect. Search engines cited say they aren't misleading with paid content but will still review and take seriously the FTC recommendations.

Claims about being in compliance with FTC's guidance don't hold up in a Search Engine Watch survey of paid content conducted last Friday, when the FTC letter was released. Only Google, among all the major search engines, was found to be providing the "clear and conspicuous" disclosure that the FTC recommends.

A sidebar article, Paid Content Disclosure Ratings, explains this in more detail. However, the main failing point for all the other major search engines revolves around the issue of "paid inclusion," where they accept money to list web sites but don't guarantee placement. We'll explore what the FTC thinks about this particular aspect of paid content further below, as well as "paid placement," where listings are guaranteed to rank highly for particular terms.

It's A Big Article -- Here's What's Coming!

Before going further, I thought it useful to provide a guide to the rest of this article. It's very long, and I've ordered the sections to best guide those who are new to the issues involved through an education process. Some people may prefer to jump directly to particular sections, and you can do this via the links below. Otherwise, just keep scrolling along. All links are to sections within this page.

Being Historically "Pure" Places Greater Burden To Disclose

I'll discuss the FTC's view about specific types of paid content below. However, it's important to understand that neither the original complaint nor the FTC's response to search engines suggests that the inclusion of paid content in search results is wrong. Rather, the concern is over disclosure. There is worry that the public may not realize that search engines have paid content, especially given that search engines have historically only provided "pure" editorial-style results.

The type of search engines we use today to scour the web first emerged in 1994. For the first six years of the industry's eight-year life span, the presence of paid listings in their search results was minimal to non-existent. That is to say, at least the presence of paid listings that were sold on behalf of the search engines themselves.

During this period, third-party "search engine optimization" firms were paid for listings obtained in search engines either through legitimate or illegitimate means, but the search engines did not benefit from this. Indeed, such third-party efforts continue today and can be a major problem for search engines, when done illegitimately. This is also in part why there is no such thing as truly "pure" search results.

GoTo (now called Overture) was the exception among the major search engines. It appeared in early 1998 and proudly boasted of selling placement to the highest bidder. In late 2000, this same company sparked a major change among all the other major search engines, in terms of paid content.

In September 2000, Overture signed what I'd consider its first major distribution deal, which placed the company's paid listings on Netscape Search and AOL Search. Other deals soon followed. By the end of 2001 -- in the space of a just year -- every major search engine had paid listings of some type prominently in its search results.

Overture was not the only provider of this paid content. For example, Inktomi had also unveiled a "paid inclusion" program in 2000 that distributed paid listings to its partners. Google, "officially" launched in September 1999, debuted its own paid listings just three months later.

Regardless of how paid listings were obtained, the point is that in a very short period of time, they began appearing alongside "editorial" search results. However, consumers might have no clue about this development. Paid listings might look the same as editorial results, or if they were labeled, titles such as "Partner Search Results" or "Featured Listings" didn't connote a paid arrangement.

Commercial Alert, among others, felt such a shift warranted greater disclosure of paid content. Users either had learned or believed that search engines represented a source of unbiased information, so if those search engines were going to change direction and be more influenced by payment, they needed to disclose this.

"Because of the earlier editorial integrity in search engine results, there is an implied representation to search engine users that listings are not skewed by marketing or commercialism. Consumers are accustomed to search engine protocols based on editorial integrity, and have not been told of the departure from these protocols. In effect, this is a high-tech case of 'bait and switch.'," the organization wrote in its complaint to the FTC.

The FTC has agreed with this view, stating in its response to Commercial Alert's complaint:

"Because search engines historically displayed search results based on relevancy to the search query, as determined by algorithms or other objective criteria, the staff believes that consumers may reasonably expect that the search results displayed by individual search engines are ranked in accordance with this standard industry practice - that is, based on a set of impartial factors. Thus, a departure from the standard practice, such as a search engine's insertion of paid-for placements in the search list, may need to be disclosed clearly and conspicuously to avoid the potential for deception," the letter said.

Issues With Paid Placement

So if payment is accepted in association with listings, this arrangement needs to be disclosed, the FTC has said. It further calls out two types of paid content arrangements, the first of which is "paid placement."

I've always described paid placement listings to be those where rankings for particular terms are guaranteed (see the Buying Your Way In page). The commission adopts a slightly broader definition, in its response to Commercial Alert:

"The staff considered 'paid placement' to be any program in which individual Web sites or URLs can pay for a higher ranking in a search results list, with the result that relevancy measures alone do not dictate their rank," the commission defined.

In its letter to search engines, the FTC advises them to consider distinguishing paid placement listings in some way from non-paid or editorial-style listings on the page:

"The staff recommends that if your search engine uses paid placement, you make any changes to the presentation of your paid-ranking search results that would be necessary to clearly delineate them as such, whether they are segregated from, or inserted into, non-paid listings. Factors to be considered in making such a disclosure clear and conspicuous are prominence, placement, presentation (i.e., it uses terms and a format that are easy for consumers to understand, and that do not contradict other statements made), and proximity to a claim that it explains or qualifies," the FTC wrote.

Getting Paid Placement Disclosure Right -- And Wrong

In talking with the principal staff members who authored the recommendations, FTC attorneys Beverly Thomas and Dean Forbes, they were extremely reluctant to say which search engines they think have the format for paid placement "right."

Indeed, as we shall see, the FTC is explicitly trying not to impose a "one size fits all" solution on the search engines, because they all have their own unique ways of presenting information.

Nevertheless, both indicated that personally -- not speaking with an official commission viewpoint -- they especially like the way Google has handled the situation with paid placement. Google's paid results are placed in visually distinct colored boxes, separate from the main editorial results. Moreover, they are listed in close proximity to the labels "Sponsored Link" or "Sponsored Links."

Of course, other search engines "delineate" or "segregate" their paid placement listings. At Yahoo, such listings look like "regular" listings but are separated from them, placed under a "Sponsor Matches" heading that also is associated with a hyperlink link that says, "What are Sponsor Matches?"

Selecting that link brings up a further explanation for consumers that the results are "paid for." This type of treatment was also seen favorably by Thomas and Forbes, from their personal perspectives.

AltaVista provides an example of what I think fails the recommendations. There, paid listings are segregated, as with Yahoo. However, the heading is "Products and Services," which I don't feel meets the "presentation" quality the FTC is looking for, which includes using terms that clearly denote the paid nature of content.

Indeed, the commission definitely did not like this type of uncertain wording, judging from the response it sent to Commercial Alert:

"Many of these sites appear to be headed in the right direction, using terms such as 'Sponsored Links' or 'Sponsored Search Listings' to denote payment for rankings....Other sites use much more ambiguous terms such as 'Products and Services,' 'News,' 'Resources,' 'Featured Listings,' 'Partner Search Results,' or 'Spotlight,' or no labels at all. To avoid deception, these sites should be labeled to better convey that paid placement is being used," the FTC wrote.

AltaVista might feel it is providing disclosure in another way, via the "Info" link that appears next to the label "Products and Services." Clicking on this link brings up a page that notes, "Sites advertising in this section have done so either directly through AltaVista, or through our distribution agreement with Overture." That's pretty clear, right?

Perhaps. However, the fact that the "Info" link is smaller than the Products and Services heading might be deemed as poor "presentation." In contrast, the corresponding link at Yahoo, while also smaller than the heading, is more verbose. It almost invites users who might be wondering about sponsored matches to discover, "What are Sponsor Matches?" More importantly, the label it is associated with unambiguously communicates the paid qualities of the listings below. This is not the case with AltaVista.

AltaVista is not alone in having problems. The sidebar article Paid Content Disclosure Ratings explains in more depth where search engines seem to be meeting the FTC recommendations or failing them, in my opinion.

Issues With Paid Inclusion

Paid inclusion is the other major type of paid listings specifically called out by the FTC. I've described paid inclusion as when someone is guaranteed to be included in a search engine's listings but given no promise to rank well for any particular terms. The FTC uses a similar definition:

"The staff considered 'paid inclusion' to be any program in which individual Web sites or URLs are included in a search engine's index, or pool, of sites available for display as search results, when that Web site or URL might not otherwise have been included, or might not have been included at a particular point in time, but for participation in the paid program," the commission wrote.

Some people, when they first learn about paid inclusion programs, believe that all these URLs should be segregated from "regular" listings, as commonly done with paid placement results. To me, that's not necessarily the best solution.

Even the best crawler-based search engines miss good content on the web. Paid inclusion programs can be used by site owners as a way to ensure that this content doesn't get overlooked. It's not fair to discriminate against this content, to assume that because it is paid, it isn't useful or relevant.

Similarly, the paid inclusion programs run by Yahoo and LookSmart now require all sites that wish to be listed in commercial areas to pay a fee. Should these sites be discriminated against while other commercial sites, those listed before fees were required, might not have to be segregated?

Handling paid inclusion is a tricky issue, which the FTC recognizes. This is why it has not suggested that paid inclusion listings be segregated, as it more strongly encourages in the case of paid placement. Instead, the emphasis is on better disclosure.

"The staff recommends that if your search engine uses paid inclusion programs that may distort rankings or placement criteria, you clearly describe how sites are selected for inclusion in your indices. Also, consumers should be able to easily locate your explanation of the paid inclusion program you use, and discern the impact of paid inclusion in search results lists," the FTC wrote to search engines.

You might take this to mean that paid inclusion need only be disclosed if it involves a ranking boost. If so, all of the major search engines offering such programs would claim to be off the hook. This is because all of them continue to assert that their paid inclusion programs provides no ranking preference.

However, Forbes clarified that commission wants paid inclusion to be disclosed even if no ranking boost is provided. The "placement criteria" portion of the clause above is a reference to being placed in the index in return for payment, not for being placed in a particular position.

Indeed, Forbes says that any type of paid inclusion program really should be disclosed. And to date, the FTC is not happy with what it sees, as shown by the response to Commercial Alert:

"Currently, although certain of the named search engines do, in fact, use paid inclusion, in the staff's view none of them adequately discloses its usage or offers clear and conspicuous explanations of its impact on search results," the FTC wrote.

How To Disclose Paid Inclusion

So what's the way forward with better disclosure of paid inclusion. The letter to Commercial Alert suggests that the FTC would like the labels associated with "regular" results, which may contain paid inclusion content, to both be clearer about this paid mix and lead to more detailed explanations:

"Through the use of clear and conspicuous disclosures, consumers should be able to easily locate a search engine's explanation of any paid inclusion program, and discern the impact of paid inclusion on search results lists....In the staff's view, labels such as 'Web Directory Sites,' 'Results,' 'Matching Sites,' and 'Reviewed Web Sites' may not clearly convey that certain sites or URLs in the search result list, or perhaps all of the sites or URLs in the search database, are participants in a paid inclusion program, rather than being included based on some other criteria that may not involve payment," the FTC wrote.

A tricky thing here will be to come up with labels that are different from those used for paid placement listings. For example, at MSN Search, the company calls its paid placement listings "Sponsored Sites." The next set of results often comes from LookSmart and may include a mix of non-paid, non-commercial content, non-paid commercial content and paid commercial content.

The current label for this information is "Web Directory Sites." As seen above, the FTC doesn't feel this label is satisfactory. What replaces it? Perhaps...

Paid & Unpaid Results
Commercial & Non-Commercial Listings
Paid & Unpaid Site Matches

None of these sounds particularly good. Given this, perhaps current headings will be deemed OK, as long as disclaimers are associated with them, such as with the example below:

Web Directory Sites
Includes both sites that have paid to be include or which are listed for free.
Learn more about how sites are selected.

In that example, the explanation might be deemed prominent enough and placed close enough to the results to make up for the more generic title. I'd also see it linking to a help page providing the consumer more detail.

My pet preference has also been for search engines using paid inclusion to consider a mechanism of flagging these URLs with a discrete icon. I think it is important for consumers to be able to see at a glance whether the results they get in response to a search is dominated by "paying" content. This isn't always bad, but if you are always getting commercial content coming up no matter what you search on, that can be an indicator that a search engine is not being as inclusive of all good content across the web, as it may claim.

In conclusion, the FTC wants disclosure of paid inclusion, but it is leaving it "very flexible" Thomas said about how the search engines achieve it.

"We do not say that you had to identify every paid inclusion URL, but you could. We do not say that you have to segregate those URLs, either," said Thomas. "We thought that consumers should be able to be informed that there is paid inclusion, how it operates and what impact it might be on the results listed."

Tricky Stuff: Content Promotion, Partnerships & Results From Others

The FTC did not address the issue of content promotion, where a portal site might list its own content over others in search results. For example, at Yahoo, a search usually brings up the "Inside Yahoo" matches at the top of the page. This is Yahoo's own content, being promoted over others.

Similarly, at MSN Search, the "Featured Listings" area may promote MSN's own content, or content of advertisers or sponsors, or even content of good web sites for free, if they editorially deserve to be there.

I asked Thomas and Forbes if there was any guidance on this issue, and the view that emerged seemed to be to err on the side of caution. Favoritism of any type should be disclosed, to avoid consumer confusion.

This would also apply to specialized search services gained from others. For example, several search engines use the news feed from Moreover.com. In turn, Moreover does have deals where it is paid to carry some news content. That would mean search engines providing access to Moreover's results should be looking at ways to disclose how inclusion is involved.

Similarly, shopping search engines that have deals to revenue share with merchants should be looking at how they disclose this information. And meta search engines -- those that get results from other search engines -- are definitely not excluded from the FTC's recommendations, as is covered next.

Who Does This Apply To?

The FTC sent the same letter of guidance to all the search engine companies named in the original complaint: AltaVista, AOL Time Warner, Ask Jeeves (which owns Teoma and Direct Hit), iWon, LookSmart, Microsoft and Terra Lycos.

However, the FTC's guidance isn't just for these companies but for the entire search engine industry. That's why letters have also gone out to Google, InfoSpace, Overture, Yahoo and the Walt Disney Internet Group (which runs Go.com).

The letters do not indicate that these search engines have done anything wrong. Rather, Forbes said these are simply services known to have a large audience of searchers. The commission wants to ensure they're made aware of its recommendations.

The FTC's letter has also been posted publicly on the FTC site, and it is applicable to anyone in the entire search engine industry, Forbes said. All search engines, of any type, large and small, should all be considering, reviewing and acting upon the recommendations, he confirmed.

Or, to quote the FTC from its letter to Commercial Alert, the guidelines "would include the named search engine companies, and other companies providing similar Internet search services to consumers, as well as meta search engines that submit simultaneous search queries to (and display results from) numerous third-party search engines."

Must They Follow The Guidelines? And If So, What Are Those?

Again, none of the search engines that received letters are being singled out for specific action by the FTC.

"This is not a threat letter but guidance and education," said Thomas.

Not being a threat doesn't mean the letter should be ignored. The FTC wants to educate search engines about what it feels they should correct. But should these issues not be addressed, the FTC could take enforcement action.

"While the commission hasn't taken action in this instance, we reserve the right to look at this issue in the future," said Forbes.

As mentioned earlier, the FTC has not provided a list of specific formats to follow for disclosure, preferring to give the search engines flexibility.

"We didn't want to impose specific rules, 'Do this or do that,' because we didnt think that provided enough leeway," Thomas said.

Instead, the FTC directs the search engines in its letter to look through an FTC guiding document published in 2000, "Dot Com Disclosures," which is designed to help online advertisers comply with consumer protection laws.

In addition, the search engines have been invited to get in touch with the FTC. A few had previously contacted the commission -- no names were provided -- but the FTC believes more will follow up.

"We expect the search engines might have questions. We put it into the letter that they can contact us," Forbes said.

Do Consumers Care?

Is the FTC involvement even necessary? Do consumers really care about whether payment is involved with search engine listings? There has been a real lack of studies on this topic, but the FTC response to Commercial Alert cited some interesting statistics from a new one.

"A Matter of Trust: What Users Want From Web Sites" was conducted for Consumer WebWatch -- a Consumers Union project -- by Princeton Survey Research Associates. The survey covered 1,500 online users in the United States, interviewed between December 20, 2001 and January 7, 2002.

Of the entire group, nearly everyone -- 87 percent -- said they'd used a search engine to find information on the web.

Those surveyed were then asked, "Have you heard or read about search engines being paid fees to list some sites more prominently than others in their search results?" Most had not:

No - 60%
Yes - 39%
Don't Know/Refused To Answer: 1%

The next question asked, "If a search engine is being paid to list other sites more prominently, how important is it to you that the search engine tells you about this policy in the search results or an easy-to-find page on the site?" Responses were:

Very important - 44%
Somewhat important - 36%
Not too important - 11%
Not important at all - 7%
Don't know/Refused - 2%

Clearly the vast majority wanted to know -- 80 percent expressed some desire. But if told, would that influence the choice to use a particular search engine? That was the last question:

"If a search engine clearly tells you in the search results that some sites are displayed prominently because they paid, would you be more likely to use that search engine, less likely to use it or would it not make any difference?"

No difference - 56%
More likely - 10%
Less likely - 30%
Don't know/Refused - 4%

For two-thirds of those asked, disclosure of paid listings wasn't going to prevent them from using a particular search engine. As for the remaining one-third, their opinions might have changed if they realized that every major search engine displays some sites prominently.

Unfortunately, the survey doesn't go into even more detail like this. Would opinions change if those surveyed were told that paid listings were common-place? Would the mixture of paid to non-paid content have an impact? More exploration of these types of questions would be useful.

But They Don't Read, They Don't Click

Anecdotally, the search engines will tell you that despite whatever labels they place in their results, search engine users tend to ignore these. They don't get read. So, while statistics from the survey may show that users say they want disclosure, in practice they seem not to notice, when it is provided.

For instance, MSN Search once had an "About" link that appeared next to the "Featured Sites" heading that brought up its disclosure statement. So few people clicked on this that it was dropped. (Concern about better disclosure has brought its return in a new way, the "About Results" link listed at the top of pages in MSN Search).

If no one is reading labels or clicking links to learn more about results, isn't that a sign they don't care? The FTC argues perhaps not -- instead, it may be a sign that the search engines need to do more investigation of how to reach out to their users.

"The company should probably do some consumer research to see whats the best way to set this up [disclosure” on the site," said Forbes.

I wouldn't be surprised if even after doing such research, most users still failed to read labels or click to learn more about results. However, this doesn't matter. Some users do care, do want to know, and information should be provided to them.

Not everyone reads the ingredients in the food they eat, but those who do tend to be very interested indeed in what's disclosed. In the search engine world, there are those who want to know more about the results they consume. Properly labeling those results will not only help these people, but it is something the FTC is now saying the search engines should do.

Resources & Links

FTC Response To Commercial Alert
FTC, June 27, 2002

FTC Letter To Search Engine Companies
FTC, June 27, 2002

FTC Dot Com Disclosures

This document provides guidelines from the FTC to companies about complying with consumer protection laws online. Search engines are being advised by the FTC to use it in creating effective labels and disclosures.

FTC Staff Issues Guidelines on Internet Advertising
FTC, May 3, 2002

Press release about the launch of the FTC's Dot Com Disclosures guidelines.

Commercial Alert

Search engines mull ad changes
Reuters, July 1, 2002

Reactions from several search engines to the FTC's letter.

Buying Your Way In To Search Engines
From Search Engine Watch, this chart explains where paid content can be found on major search engines and surveys whether that content appears to be disclosed in a way the FTC might approve of.

Paid Content Disclosure Ratings
Background about how the determination of a pass/fail grade for the disclosures chart above was made.

A Matter of Trust: What Users Want From Web Sites
Consumer WebWatch, January 2002

This is the consumer survey cited by the FTC. Please note it is in PDF format and a fairly large 425K file to download.

Consumer Group Asks FTC To Investigate Search Ads
SearchEngineWatch.com, July 17, 2001

Article from when the original complaint was filed.

GoTo Speaks Out On FTC Complaint
The Search Engine Report, Sept. 4, 2001

Comments from Overture's CEO Ted Meisel, after the complaint was filed. His comments were both right and wrong on many points.

Dot-com noir
Salon, July 1, 2002

Story of a not-so-savory interaction of a third-party search engine optimization firm and the "pure" unpaid results at search engines. Certainly not all search engine optimization companies are like this -- there are plenty of legitimate, helpful firms. However, it underscores the fact that even if paid listing weren't sold on behalf of search engines, people will go through great efforts to still make money off the unpaid ones.

Search Engine Advertising
Compilations of past articles by me and others on the myriad of issues involving paid content.

The Bumpy Road To Maximum Monetization
The Search Engine Report, May 6, 2002

I addressed the need for better labeling of search results among other issues in this article two months ago, concluding, "Either the search engines need to come up with those answers in a more unified voice or the inevitable move to greater monetization will be followed by the inevitable regulation of a profitable, growing and increasingly confusing industry by some governmental agency." Now that governmental agency has arrived. To prevent its suggested guidance from turning into mandatory actions, the search engines more than ever need to consider some standardized language and disclosure options.