Use these outstanding online resources to get the equivalent of an introductory semester of “search engines 101″ — without having to go back to school for your education.
Here at Search Engine Watch, our goal is to keep you up to date with the world of search engines. Although we provide detailed information about how search engines work, if you really want to “go deep” you should check out some of the academic resources focused on the broader field of information retrieval.
Web search engines are an outgrowth of academic research into information retrieval. Work on information retrieval systems began in the late 1960s, long before the web came into being in the 1990s.
Reading the history of information retrieval systems, and how they evolved over the decades, is not only a fascinating exercise in its own right, it can also help you improve your own searching skills.
For an excellent overview, check out Professor Osmar R. Zaïane’s paper “From Resource Discovery to Knowledge Discovery on the Internet.” It’s an engaging history of information retrieval, and discusses many of the basic principles that underlie the operations of today’s search engines.
Despite being a technical report published by Simon Fraser University, the paper is eminently readable. Part one describes basic information retrieval concepts, including brute force document scanning, conventional document retrieval techniques, and the particular challenges of indexing and searching hypertext documents and multimedia content on the web.
Next, Dr. Zaïane offers a survey of “resource discovery on the Internet.” This is a technical glimpse of how search engines and directories are created and maintained. Although the paper was written in the pre-Google era, it still offers solid information to help you understand how your favorite search tools work.
The last section is dedicated to data mining or knowledge discovery on the Internet — what needs to be done to help improve search results.
If you really want to dig deeply into learning about search technology, wander over to the web site of the School of Information Management and Systems at the University of California at Berkeley. Most of the courses taught on campus are supported by web sites, and have lots of links to lecture notes, presentations, articles, and other course materials.
In particular, check out the pages for “SIMS 202: Information Organization and Retrieval.” This truly is “Search Engines 101,” starting from basic concepts like “What is Information” and ending up with “Web Search Architecture and Crawling.” Many of the course materials are available online.
You can find many other similar courses at other university web sites. Use Google’s University Search, with keywords such as “search engines” or “information retrieval” to find them.
Finally, if you really get hooked and want a “fire hose” of information about information retrieval and search engines, try CiteSeer. This specialized search engine, from the NEC Research Institute, is focused on computer science research papers.
CiteSeer can index and make available papers in Postscript format, a capability most other search engines lack (at this point). It also creates interesting linkages to help you find information, such as citations, related documents, links to author home pages, and many other useful features.