Which search engines are adequately disclosing their paid placement and paid inclusion practices? Which ones are failing to do so? A new report offers some surprising insights.
In yesterday's SearchDay, I described a new report from advocacy watchdog Consumer WebWatch, which details the disclosure practices of the most heavily trafficked search sites. Here's a closer look at the findings for each of the engines considered in the report.
Consumer Reports WebWatch evaluated 15 search engines for this study, culling the sites from the list of the top-most trafficked search engines in December 2003, according to Nielsen//Net Ratings. The engines were 1stBlaze, AltaVista, AOL Search, Ask Jeeves, CNET's Search.com, Google, InfoSpace Web Search, Lycos Network, MSN Search, My Search, My Way Search, Netscape Search, Overture, Web Search and Yahoo Search.
Consumer WebWatch employed four librarians to look at each of the 15 sites and respond to a questionnaire designed to measure clear and conspicuous disclosure of paid placement and paid inclusion programs. The actual testing took place between April 28 and May 4, 2004, and as I noted yesterday, many search engines have altered result pages or changed disclosure information since then.
Notably, all search engines other than Yahoo that had been using paid inclusion programs have terminated their programs since the study.
Several search engines sometimes practiced "content promotion" used to highlight a site's related internal content, other worthwhile links, sponsored results or a combination of all three.
AOL Search received high marks for disclosure, both of Sponsored Links and content promotion. Labels were clear and easy to find, featured additional information ("Sponsored Links: Provided by a third party and not endorsed by AOL"), and all types of results had a link to a page with additional disclosure information.
Nonetheless, the disclosure information itself was considered to be merely adequate, at best. "While AOL does a good job at explaining the concept of paid placement in a concise manner, the explanation gets lost among the other information on the page," said one tester.
Bottom line: AOL Search exceeds FTC guidelines for paid placement disclosure. AOL natural search results are provided by Google, which does not use paid inclusion.
Ask Jeeves received good marks for good visibility of disclosure of "Sponsored Web Results," however, testers criticized the placement and faint gray color of the "About" link for additional information. This link, appearing on the opposite side of the page from the "Sponsored Web Results" heading, displays a page disclosing that paid placement links are provided by Google. Notably, one tester commented "In fact, the disclosure about Google-provided links is much clearer than the Google pages themselves."
The report noted that Ask Jeeves discontinued its paid inclusion programs in March. However, testers felt that the company failed to adequately disclose that Teoma, which powers natural search results for Ask Jeeves, is actually owned by the company, leading one tester to suspect deception. "This is a subtle use of paid inclusion where the search engine uses the fact they are using another companys search technology to obfuscate paid inclusion," said the tester.
Bottom line: Ask Jeeves exceeds FTC guidelines for paid placement disclosure. Although Ask Jeeves has discontinued its paid inclusion programs, the company is still honoring some existing contracts, a fact noted in its disclosure page. Contrary to the dubious conclusion drawn by the Consumer WebWatch tester, Ask Jeeves meets FTC disclosure guidelines for paid inclusion.
The report generally lauds Google's policies, but is critical in two respects. The testers roundly criticized the site for not providing accessible, more complete disclosure information. Google's "Sponsored Links" heading was deemed an adequate indicator of the site's use of paid placement, but it blended in with the page due to a thin, gray font, and there is no link to an explanation of disclosure practices.
Many of the sites that display Google AdWords Sponsored Listings, such as Ask Jeeves, Dogpile and others provide the disclosure notices that Google itself does not provide.
The report also said that "it was somewhat odd to discover most testers were unable to find information on the Google site itself regarding whether the engine used paid inclusion."
Bottom line: While Google meets FTC disclosure guidelines for its paid placement listings, it does not exceed them as do most of the other search engines surveyed. And although Google does not offer a paid inclusion program, it does not make this clear to users.
The report looked at Infospace.com, but the findings also apply to the company's other meta search properties, Dogpile, Metacrawler and Webcrawler as well. Testers found that InfoSpace did disclose paid placement—in fact, going beyond most other search engines by individually labeling each sponsored listing rather than grouping them under a single heading. But testers complained about layout, location of links and colors.
The report was harsher on the company's paid inclusion practices. "InfoSpace uses paid inclusion but did not disclose this fact. Like other meta-engines tested, InfoSpace strips out any disclosure language from the search engines whose results it displays. Unless one knows the business practices of these engines which should not be expected of the average consumer there is no way to tell if InfoSpace uses paid inclusion."
Melissa Turtel, Sr. Product Manager - Search & Directory for InfoSpace, says "We do not 'strip out' any disclosure information whatsoever. Our partners do not flag paid inclusion results to distinguish them from others. In our user testing, we have found that the results provided by partners with paid inclusion programs to be as relevant as and sometimes more relevant than - those without."
Turtel also says that the company agrees with many of the criticisms in the report and will be making changes to both layout and design of result pages as well as disclosure language in an upcoming release.
Bottom line: InfoSpace exceeds FTC disclosure guidelines for paid placement, but does not adequately disclose paid inclusion listings—a shortcoming that the company says it intends to address very soon.
MSN Search uses paid placement, with results provided by Yahoo-owned Overture. This is disclosed by a heading and hyperlink to a disclosure pop-up. However, while testers approved of the language of the "Sponsored Sites" heading, they criticized the light gray font, which was difficult to spot. In addition, it required multiple clicks to ultimately access the full disclosure.
Since the report was written, MSN Search has changed the layout and function of result pages. Now the "Sponsored Sites" heading is a hyperlink (though it is not underlined and its not apparent that it is a link unless you mouse over the words). This link now leads directly to a disclosure page.
During the testing period in late April and early May MSN did use paid inclusion, both from partner Yahoo and from its own program. In July, MSN eliminated its own paid inclusion program, and modified its agreement with Yahoo so that the natural search listings provided by Yahoo are free of paid inclusion listings.
Nonetheless, the report took MSN to task for not disclosing that it does not use paid inclusion, in much the same way that it was critical of Google.
Bottom line: MSN Search adequately discloses paid placement listings and does not use paid inclusion.
Yahoo Search uses paid placement results from its Overture subsidiary. The report applauded Yahoo's disclosure practices, finding placement, use of fonts and the language in disclosure pages to be more than adequate.
Yahoo's paid inclusion policies generally received poor marks, however. Testers had mixed feelings about Yahoo's paid inclusion disclosure, ranging from praise to harsh criticism. While one tester said that paid inclusion disclosure was easy to find and understand, another said the disclosure is "buried beyond some puffery about the service."
Bottom line: Yahoo exceeds FTC disclosure guidelines for its paid placement programs, and is borderline in its disclosure of its paid inclusion policies.
Other Search Engines Covered
The report covered nine additional search engines, none of which garner the traffic of those listed above. Some have been acquired by larger companies and have similar disclosure practices (AltaVista and AlltheWeb by Yahoo, My Web Search and My Way Search by Ask Jeeves).
In general disclosure was found to be adequate for most engines, but the report singled out 1stBlaze, CNETs Search.com and WebSearch.com as having poor disclosure practices. The report even went so far as to recommend avoiding 1stBlaze because of inadequate or absent disclosures that undermine the integrity of search results.
For full details, the report itself is available as an 82 page PDF download.
Complete Search Engine Watch coverage of the disclosure issue is available on the Buying Your Way In: Search Engine Advertising Chart.Want to discuss or comment on this story? Join the Rating Search Engine Disclosure discussion in the Search Engine Watch forums.
NOTE: Article links often change. In case of a bad link, use the publication's search facility, which most have, and search for the headline.