Why Good Sites Lack Search Engines

A new survey attempts to explain the inexplicable: why some perfectly good web sites fail to provide a search engine to improve user navigation.

"We wanted to learn more about the relationship of search engines and web sites, and how web site managers view search engines," wrote Avi Rappoport of Search Tools consulting, lead investigator of the study. In a surprising finding, the majority of site managers surveyed had not installed a site search engine.

More than 1,600 webmasters responded to the survey, which covered the topics of why site managers have or have not installed search engines, correlations of the sizes of sites and the installation of search engines, frequency of updates, file formats served, languages, and number of languages used on sites.

Most sites with a search tool installed wanted to provide better navigation and a professional look for the site.

Other key findings:

- Sites with more pages tend to have search engines installed.
- Sites which are updated hourly or daily are much more likely to have search installed than those which update less frequently.
- Sites with non-English text are more likely to have search engines installed.

Why haven't the majority of web managers surveyed installed search engines? Time and complexity were the most frequently cited reasons.

"Search engine requirements are more complex and idiosyncratic than they appear at first," wrote Rappoport in a summary of the survey findings. "A web site may have dynamic pages, or be missing page descriptions, or change often, requiring a flexible indexer to adjust to these conditions... A topical portal may have customers who perform many single-word searches. No one search engine is best for everyone, but some have consistently happy users while others are rated very badly."

Rappoport also noted that installing a site search tool is easiest on a local server. Administrators working on co-located and hosted servers have a harder time installing site search tools, and are significantly less likely to do so.

The report offers detailed analysis of all of the major survey findings, generously illustrated with graphs and charts. It's a fascinating glimpse at an important aspect of web navigation that's often overlooked, much to the detriment of users.

Search Tools Survey Results
An overview of the findings of the recent survey done by Search Tools consulting.

Reports on Key Findings:

Why Site Managers Install Search Engines

Reasons Why Search Is Not Installed

Site Sizes and Search Engines

Search and Update Frequency

Server Location Search

Search and Languages on Sites

Search Engines & Multiple Languages

Search and File Formats

Site Search Tools Product Ratings Summary
The 16 most popular search tools (6 or more entries) as of July 31, 2002, with a list of less popular search tools (five or fewer responses).

Search Headlines

NOTE: Article links often change. In case of a bad link, use the publication's search facility, which most have, and search for the headline.

Domain name news
Spielberg in domain name dispute...
BBC Aug 22 2002 8:49AM GMT
XML and metadata news
Weblogs of the stars...
San Francisco Chronicle Aug 22 2002 0:17AM GMT
Top internet stories
New Salvo in Piracy, Privacy War...
Wired News Aug 21 2002 5:53PM GMT
Online search engines news
Independence of Search Engines questioned...
IT-Analysis.com Aug 21 2002 3:35PM GMT
Online legal issues news
'Stupid' linking policies come under fire...
ZDNet Aug 21 2002 2:43PM GMT
Domain name news
Domain name holders targeted once again...
IDGNet New Zealand Aug 21 2002 2:07PM GMT
Tech latest
Media chief decries Net's moral fiber...
CNET Aug 21 2002 1:26PM GMT
Online portals news
Content is king again at AOL...
ZDNet Aug 21 2002 12:59PM GMT
Online legal issues news
Is your website illegal?...
Silicon.com Aug 21 2002 10:38AM GMT
US Justice Department ready to prosecute file-swappers...
ZDNet Aug 21 2002 7:57AM GMT
Online marketing news
Ruder Finn, Ad Council Relaunch Smokey Bear Web Site...
Design In Motion Aug 21 2002 6:33AM GMT
Domain name news
New .Org Nominee an Old ICANN Face...
AtNewYork Aug 21 2002 6:08AM GMT
Online marketing news
Spam fighters shouldn't tread on the innocent...
SiliconValley.com Aug 21 2002 5:42AM GMT
powered by Moreover.com

About the author

Chris Sherman is a frequent contributor to several information industry journals. He's written several books, including The McGraw-Hill CD ROM Handbook and The Invisible Web: Uncovering Information Sources Search Engines Can't See, co-authored with Gary Price. Chris has written about search and search engines since 1994, when he developed online searching tutorials for several clients. From 1998 to 2001, he was About.com's Web Search Guide.