Search engines are essentially massive full-text indexes of web pages. The quality of the indexes, and how the engines use the information they contain, is what makes -- or breaks -- the quality of search results.
Search engines are essentially massive full-text indexes of web pages. The quality of the indexes, and how the engines use the information they contain, is what makes — or breaks — the quality of search results.
We’re all familiar with back-of-the-book indexes. They’re simply alphabetized lists of the important words in the book, and the pages on which they appear.
Search engine indexes are similar, but vastly more complex that back-of-the-book indexes. Although most of us will never want to become experts on web indexing, knowing even a little bit about how they’re built and used can vastly improve your searching skills.
A good way to learn about web indexing is to spend some time with a page compiled by The School of Library, Archival and Information Studies at the University of British Columbia. Indexing Resources on the WWW is a lengthy list of links to research, articles and web sites concerned with the process of indexing of all kinds.
SearchDay Readers will want to investigate two sections in particular. The first, Information Retrieval, looks at all types of searching.
Although most of the links point to technical information, there are a number of excellent articles for non-professionals, such as Vannevar Bush’s classic As We May Think, the recent U.C. Berkeley research on How Much Information, and others.
The second section, Indexing Resources on the WWW is focused on indexing specifically for the world wide web. Three parts of this page will be of most interest:
Search Headlines
NOTE: Article links often change. In case of a bad link, use the publication’s search facility, which most have, and search for the headline.
| University plans bot museum… USA Today May 5 2003 12:41PM GMT |
| Google: An engine of change… SiliconValley.com May 5 2003 12:34PM GMT |
| Google listens to your questions… The Register May 5 2003 9:55AM GMT |
| Signs of a Revival for Online Ads… New York Times May 5 2003 5:52AM GMT |
| Microsoft Takes XML Mainstream… EContent May 5 2003 4:02AM GMT |
| UK internet surfers are weird… Web-User May 5 2003 0:16AM GMT |
| Hints on tracing stock splits on Yahoo… San Francisco Chronicle May 4 2003 12:32PM GMT |
| eBizSearch Search Engine for E-Business Info… Research Buzz May 3 2003 12:43PM GMT |
| U.S. may add to copyright confusion… ZDNet May 2 2003 12:58PM GMT |
| Are spam blacklists killing legitimate emails?… Silicon.com May 2 2003 11:44AM GMT |
| Report: Web Traffic Rises With Consumer Confidence… dmnews.com May 2 2003 5:24AM GMT |
| Search Engine Optimization for Pure Content Sites… High Rankings May 2 2003 4:02AM GMT |
| Is Yahoo making eyes at Overture?… CNET May 1 2003 9:15PM GMT |
 |