The Big Four Meta Search Engines

Though there are dozens of useful meta search engines, InfoSpace is the industry gorilla, operating the four arguably best known and most heavily used properties.

Dogpile and Metacrawler are the two best known InfoSpace meta search engines. Less well known is that InfoSpace purchased the Excite and Webcrawler properties when Excite failed, and quietly re-engineered their back ends as meta search engines.

In the past, Search Engine Watch has been critical of the InfoSpace services (see "Meta Search or Meta Ads" below). But the company has overhauled its business model during the past six months, and has renewed its commitment to providing quality search results.

"We took a turn where we were a meta-paid engine," admitted Richard Pelley, InfoSpace's vice president and general manager, search properties.

A key component of InfoSpace's new focus is still "monetization," but the company has also refocused on two other core goals: To improve quality of results and to improve the performance of the search engines.

As part of this process, InfoSpace has cut deals with Fast, Google and Inktomi to include results from their sites in addition to the paid listings provided by Overture, FindWhat, Ah-Ha, and other results from LookSmart, About, Ask Jeeves, SearchHippo, the Open Directory Project, and InfoSpace's own directories.

Query analysis has also been improved. The engines first try to determine whether the user is looking for commercial or non-commercial results, according to Tasha Irvine, InfoSpace product unit manager, search.

"We don't believe paid is bad," said York Baur, InfoSpace executive vice president of wireline and broadband. "Our approach is to present the right thing at the right time."

Test queries show that there is a fairly good balance between sponsored and "pure" search results. Nonetheless, some of the paid result links are still labeled with euphemisms like "Dogpile suggests" or "Featured Search Results" -- vague labels that don't clearly delineate the links as advertisements.

Euphemistic labeling of paid placement links has drawn the scrutiny of the Federal Trade Commission. The FTC "recommends that if your search engine uses paid placement, you make any changes to the presentation of your paid-ranking search results that would be necessary to clearly delineate them as such," the FTC wrote in a recommendation on June 27, 2002.

InfoSpace says that while it plans to add more disclosure information, it has no current plans to change the labels for its paid placement links. "Our belief is that the consumer is smart," said Baur, adding that it would be difficult to label individual links provided by paid link partners such as Overture and FindWhat.

Issues about labeling aside, what differentiates the InfoSpace meta search engines from one another?

The underlying technology is similar for each of the four engines. But there are important differences that will influence which one is the best choice for your own needs. Here's a look at the unique aspects of each engine.


Dogpile is InfoSpace's "destination meta property," according to Baur. It's designed as a mainstream consumer site, with a simple presentation. Results are grouped by provider, typically with Overture and FindWhat results presented first. This presentation makes it easy to compare and contrast results from different search engines for the same query, and is one of Dogpile's most useful features.

Dogpile offers no advanced search capabilities, though you can limit your search to Web search; Images from Ditto and Fast Image; Audio/MP3 from Astraweb, Fast Audio and MP3Board; Files from Fast FTP; News from Dogpile Newscrawler and Fast News; or Multimedia files from Fast.

You can also select, to an extent, which search engine results you want to be displayed first, or not displayed at all, using the "custom search" form.


"Metacrawler is a hardcore site for the sophisticated searcher," said InfoSpace's Baur. Unlike Dogpile's approach of presenting results grouped by their original source, Metacrawler blends results based on relevancy and performance. Speed is emphasized -- if a particular engine isn't processing a query quickly, its results won't be included in your Metacrawler results.

The source of each result is clearly labeled. This lets you see that a particular result came from several search engines, or perhaps just a single one. Seeing the attribution displayed like this offers clues that can help you decide whether to click through to the underlying page or not.

For example, if a particular result is attributed to Fast, the Open Directory, and FindWhat, you can be relatively confident that it's an authoritative site that they've all "agreed" is a highly relevant match for your query.

You can also sort results by source, similar to Dogpile's approach. The key difference is that results from all engines are displayed, rather than spread out over several pages.

Like Dogpile, you can limit your search to the same kinds of files or sources noted above. But Metacrawler also provides an advanced search form with a number of useful options. You can set your keyword default to "any," "all," or "phrase," and whether results are sorted by relevance or source.

Checkboxes allow you to specify specific search engines to use, though even if you specifically exclude Overture you'll still get three Overture results labeled "Featured Search Results" if your search is deemed "commercial" in nature. This is because Overture does have some preferred placement on commercial-only terms, according to Baur.

You can choose the number of results to be displayed on each page and the maximum number of results from each engine. You can also specify the "timeout" length, or how long Metacrawler will wait for a response from all engines queried before aggregating results.

Excite and Webcrawler

When InfoSpace purchased Excite and Webcrawler, the intention was to keep the user experience as similar as possible to those provided by Excite, primarily to maintain the high traffic levels enjoyed by each site. To a large extent, the company has succeeded in its goal. Excite still offers most of the portal features that have long made it an appealing start page, such as news, stock quotes and so on. Webcrawler still has its clean, simple look.

What's changed, though, is the underlying search engine for both properties. Search results for both are now blended metasearch results, similar to Metacrawler results. Neither Excite nor Webcrawler offer Metacrawler's advanced customization features, though you can limit your search to the web, news or photos.

Results are identical for both properties, with one key exception: Webcrawler results are ad-free, apart from the links served by Overture, FindWhat and Ah-Ha. No banners; no annoying pop-ups. Of course, you don't have access to the personalization features offered by Excite, but the ad-free environment is refreshing.

Bottom line: The meta search products offered by InfoSpace have improved since the last time Search Engine Watch looked at them. If you find the meta search concept attractive, both Dogpile (for casual searching) and Metacrawler (for serious searching) are worthy of your time and attention.

Since you have more control with Dogpile and Metacrawler, there's really no reason to use Excite or Webcrawler unless you are already familiar with them and like Excite's extensive portal features or Webcrawler's clean look.

"We're trying to get the products restored not only to their former glory but what we believe to be the next generation of meta search," said InfoSpace's Baur. A worthy goal, and the company is off to a good start toward achieving it.


Dogpile Custom Search
Use this form to select the order in which results from individual search engines appear.


Metacrawler Advanced Search
Use this form to select customize and/or save your settings.



Meta Search Or Meta Ads?
The Search Engine Report, June 4, 2001

A review of meta search services by Search Engine Watch shows that some are providing results where more than half of their listings are paid links. A guide to what's paid, what's not and how to get the most from your meta search service.

InfoSpace Launches Guaranteed Search Inclusion Program

InfoSpace has announced the launch of its Guaranteed Search Inclusion program. The new program is a Paid Inclusion only index searched by InfoSpace powered meta-search engines.

Further information about the program, as well as a sign-up form can be found by clicking on the "Submit Your Site" link at Excite, Dogpile, WebCrawler or MetaCrawler, using the links above.

Search Headlines

NOTE: Article links often change. In case of a bad link, use the publication's search facility, which most have, and search for the headline.

Online search engines news
Search engines in China face balancing act...
San Francisco Chronicle Sep 17 2002 5:34AM GMT
Google Uses Meta Tags Sparingly, But Should You? (Enough Already, Part 2)...
Traffick Sep 17 2002 3:30AM GMT
Web developer news
Content 2.0...
WebTechniques Sep 17 2002 2:26AM GMT
Online legal issues news
School Communities Give Internet Filtering Law Failing Grade...
Electronic Frontier Foundations Sep 17 2002 1:45AM GMT
Online portals news
Can Yahoo Do a Number on the Skeptics?...
Business Week Sep 17 2002 1:32AM GMT
Internet: international news
Web and State Face Off in China...
New York Times Sep 17 2002 1:14AM GMT
Domain name news
U.S. likely to renew ICANN's authority...
CNET Sep 16 2002 10:44PM GMT
A New Internet Domain for Europe...
DW_WORLD.DE Sep 16 2002 10:11PM GMT
Online search engines news
Specifying Depth of Results in Google...
Research Buzz Sep 16 2002 6:31PM GMT
AlltheWeb's Search Engine Offers New Options...
Research Buzz Sep 16 2002 11:51AM GMT
Domain name news
ICANN delays naming .org successor...
ZDNet Sep 16 2002 11:41AM GMT
Internet features
Store the front page...
Media Guardian Sep 16 2002 10:28AM GMT
Online search engines news
Update: Chinese Google still not itself... Sep 16 2002 9:09AM GMT
powered by

About the author

Chris Sherman is a frequent contributor to several information industry journals. He's written several books, including The McGraw-Hill CD ROM Handbook and The Invisible Web: Uncovering Information Sources Search Engines Can't See, co-authored with Gary Price. Chris has written about search and search engines since 1994, when he developed online searching tutorials for several clients. From 1998 to 2001, he was's Web Search Guide.