NPD Search and Portal Site Study

By Danny Sullivan
July 6, 2000

Every three months, NPD New Media Services conducts a survey at the major search engines to measure how visitors use those services. Below are key results from the latest survey, as well as charts that go back to when the survey first began in Spring 1997.

The latest study involved 33,000 respondents, who were picked at random during the first quarter of 2000. When these respondents came to a participating search engine, a new browser window popped up inviting them to take part in the survey.

The survey is conducted on behalf of the participating search engines. In the last one, 13 major search sites were involved: AltaVista, AOL Search, Ask Jeeves, Excite, Go, Google,, HotBot, Lycos, MSN Search, Netscape Search, WebCrawler, and Yahoo.

Search Success Rate

The chart below shows how successful respondents said they were in locating information. The answers from each respondent, for the particular search engine they used, were consolidated to produce an average score for that search engine. Then the scores for each search engine were averaged to produce figures for all of them, as a whole.

How Often Do You
Find What You Are Looking For?
(Winter 2000 - In Percentage)

Of particular interest are the "Every Time" and "Most of the Time" figures. Obviously, everyone wants to find what they are looking for every time. However, I believe most users would also be pleased if their search engine at least managed to find what they were looking for most of the time. Combining these two figures gives what I call an overall "success rate" for the search engines. The latest survey found search engines to have a success rate of 81 percent. That's the highest the success rate has been for two years, as the next chart shows:

Search Success Rate
Information found Every Time + Most of the Time
(Summer 1997 - Winter 2000, In Percentage)
Key: Sm = Summer; F = Fall; W = Winter; Sp = Spring. Also use for
charts below. NOTE: There was no survey for Summer 1998.

Why has the search success rate increased? NPD suspects it may be related to more searchers using multiple keywords (see chart below). The more specific someone is when searching, the more likely it is that they'll get better results.

Which Is The Best Search Engine?
It is up to each search engine to release their NPD figures. Only those performing well tend to do this (see article for more). Below are top scores from the latest survey that have been released.

Second quarter in a row with best overall success rate, with
97 percent of respondents saying they found what they were looking for either Every Time or Most of the Time.

Ranked second for finding information Every Time (29 percent)
and fifth for overall success rate (82 percent).

Breaking News
Results from the Spring 2000 survey should be released soon. Google and iWon have already issued press releases detailing their successes in that survey. Links to these releases are below:

Google Release on NPD Study

iWon Release on NPD Study

The next chart shows two measures of search failure, those who find things only "Some of the Time" or "Never."

Search Failure Rates
Information found "Some of the time" or "Never"
(Summer 1997 - Fall 1999, In Percentage)

The rise in search success has brought about a corresponding drop in the failure rates. The Some of the Time figure has dropped to its lowest point in over two years. The Never figure, while improved, is still far higher than when the survey first began.

Action After Search Failure

The chart below shows what people do if their search fails to find what they are looking for. Again, the scores for each search engine were averaged to produce figures for all of them, as a whole.

What Do You Do If You Don't Find
What You Are Looking For?
(Winter 2000 - In Percentage)

The results show that people are clearly more likely to try again searching at the same place in a different way than to try the same query at another search engine. This action has stayed relatively steady over time:

What Do You Do If You Don't Find
What You Are Looking For?
(Summer 1997 - Winter 2000, In Percentage)

The increase in Same Search, Different Site for Winter 2000 is likely due to the dropping of a fourth choice, "Different Source," from the possible answers to this question. So rather than an increase, it is likely that users interpreted Same Search, Different Site to mean the same as Different Source. That would mean that Same Search, Different Site would have been at a higher level on previous surveys, though still well below those choosing Same Site, Different Search.

Search Method

The chart below shows the method in which people seek information, be it using multiple keywords, a single keyword, asking questions or browsing for answers by clicking on "predefined options" such as channels or directories.

How Do You Search?
(Winter 2000 - In Percentage)

Multiple keyword searching is most popular, and the results would be even higher if the usage of "Related Searches" features (see Search Assistance page) were counted in the total. This allows you to enter a single word, then click on a list of multiple keyword suggestions in order to refine your search. That type of behavior instead is part of the "one keyword" figure. Least popular is question asking, but that method has dramatically increased over the past year:

How Do You Search?
(Summer 1997 - Fall 1999, In Percentage)

For the first time since the survey began, multiple keyword searching has shown a significant rise. NPD was uncertain why this has happened. One reason might be that search engines are doing a better job of educating their users to be more specific with their queries.

As for the general rise in question asking, what I call the "Ask Jeeves" factor has probably encouraged users to try phrasing their queries as questions at all search engines in general, rather than just at Ask Jeeves. All major search engines can process queries phrased as questions, and doing so often yield better results, as the queries often contain several keywords.

More Resources


Search Satisfaction And Behavior Results Released
The Search Engine Report, April 3, 2000

Provides more analysis about the facts and figures described above, plus why there are no figures for each individual search engine.