A Search Engine Gateway to the Invisible Web

The Invisible Web is an enchanted realm for searchers, but only if you know how to access its abundant treasures. The Resource Discovery Network (RDN) is an outstanding gateway to thousands of Invisible Web sites that’s as close to a search engine for the hidden web as you’re likely to find.

The RDN is a web directory compiled by subject and information experts in colleges, universities and related organizations throughout the United Kingdom. These individuals identify, catalogue and describe high quality Internet resources relevant to teaching, learning and research.

Like the U.S. based Librarian’s Index to the Internet, the RDN is not a “pure” invisible web directory, but a considerable portion of its high quality content consists of material indexed poorly (if at all) by conventional search engines.

The RDN is structured as a cooperative network consisting of a central organization and a number of independent service providers called hubs.

While the hubs can be accessed independently, browsing the RDN lets you easily access all of them under a unified interface. Even better, the site’s search function provides cross-disciplinary querying of all RDN resources with a single search.

The service currently links to more than 35,000 human selected resources organized into eleven topical categories.

The RDN also offers a news service called “Behind the Headlines” that offers links to in-depth resources and information for a wide range of current events. It’s an excellent way to get information not always provided by the mainstream media. For example, related to the current instability in Zimbabwe, there are links to both government controlled web sites and independent groups advocating democratic reform in the country.

The RDN’s “Virtual Training Suite” is another useful resource. This is a set of online tutorials designed to help students, lecturers and researchers improve their Internet information skills. The tutorials take around an hour each to complete, and include quizzes and interactive exercises. The tutorials provide both an excellent way to sharpen research skills and to learn what’s available online for specific subject areas.

The RDN is also pushing the envelope when it comes to resource discovery, according to Simon Jennings, Manager
of the Resource Discovery Network Centre.

“In the medium term we will be developing an advanced search and a search engine based on harvesting one hop away from all the links in our 35,000 hand selected and described records,” says Jennings. “The software will store (and we hope, in future, utilise) the linking relationships between all items in the database.”

In other words, the RDN is applying Google-like techniques to find additional web resources based on the “recommendations” made by links in its existing database of selected sites. This “focused crawler” approach to resource discovery is providing excellent results, when a bit of filtering is applied, says Jennings.

The RDN is a first-rate gateway to some of the best resources available on the Web. And, given that it points the way into numerous regions of the Invisible Web, it’s a tremendously valuable pathfinder for all of us.

Related reading

Simple Share Buttons