Google Adds Directory

Users continue to rave about the quality of Google's search results, and that's testimony that its link analysis system works well at delivering relevant information. Now Google is applying link analysis in a new way, to the human powered information of the Open Directory.

All I can say is hurray! I've been chanting for an end to the tyranny of the alphabet when it comes to directory listings for some time. Why should sites that begin with an A appear at the top of list, oppressing better quality sites that may simply begin with a letter further down in the alphabet? In many cases, this offers no help to the searcher.

For instance, imagine you want to find a comparison shopping service. Over at Yahoo, "Shopping Agents" seems to be the most suitable category. In it, the widely-praised service mySimon appears 15th on the list, which is ranked in alphabetical order. The situation is worse when reviewing the Open Directory's "Price Comparisons" category, where mySimon appears 37th on the list.

The trouble here is that it already takes time for human editors to review and list sites. It takes even more time to rank them in order of quality, even though that's exactly what users would like. That's where Google's use of link analysis adds the perfect blend of technology to the human effort of the Open Directory.

For instance, that same Open Directory category mentioned above is now available in Google's edition of the Open Directory. It's basically the same core list of sites (the Open Directory list has a few more, as it is fresher), but Google has reranked the sites in the list so that those with link popularity come at the top. Result? mySimon rises to 5th on the list, which is appropriate for such a popular and well-regarded site.

How to make use of this new directory? One means is to search Google, just as you might ordinarily. As before, that will bring back matching pages of information based on Google's automated crawling of the web. The difference is that you may now get suggested "categories" of information at the top of the search results. Clicking on one of these will take you into the Google Directory. You'll be shown a list of sites for that category that were compiled by the volunteer editors of the Open Directory and ranked according to Google's link analysis system.

When you search, you'll also discover that some pages now have new "Category" links appearing below their listings. For example, in a search for "comparison shopping," mySimon is listed in the top results, along with a link to "Home > Consumers > Consumer Information > Price Comparisons." Selecting this link will take you to the area within the Google Directory where mySimon is listed. Naturally, you'll also find other sites in this category along with mySimon. This means that you can use the category links as a way to find sites similar to those listed in the main search results.

You can browse the Google Directory by selecting the "browse web pages" link underneath the search box, on the Google home page. Browsing the directory also gives you the ability to focus Google's crawler-based results toward any topic you are interested in. This is an incredibly powerful feature that's worth exploring.

Let's say you live in the United Kingdom (I do) and are interested in information about genetically modified foods, which is a hot topic in the UK. You search for "gm foods" on Google, and now you are shown a list of matching pages from all over the web. Great, but now let's say you want to narrow your results to sites specifically related to the UK. Go to the Google Directory, click on Regional, then Europe, then United Kingdom. Now search for "gm foods" from within the category (you'll see that the option is set to "Search only in United Kingdom"). The results which come back will be much more specific to the UK.

What's happening is that Google will return any matching sites for that topic which the editors have listed, plus it goes beyond and brings back any additional relevant pages from these sites that it has found from crawling the web. "You can look through the entire contents of those web sites," explained Google President Sergey Brin.

So by navigating to a specific area of Google, you are essentially turning it into a specialty search engine of your choice. The possibilities are endless. Go to Home > Houses, search for "windows," and you get pages and sites about windows in your home, not Microsoft Windows. Go to World > Deutsch and search for "star trek," and you get links about the series written in German or aimed at German-speakers (trivia tip: Dr. "Bones" McCoy is called "Pill" when Star Trek is translated into German).

In other news, Google has announced that it is planning a family filter and has new international editions in the works. The service also says it has introduced a page clustering feature, so that at most, only two pages per web site will appear in its top results. New Google "buttons" let you search Google or use its GoogleScout feature directly from within your browser.

Finally, Google becomes the latest major service to pay affiliates 3 cents for each search request sent to them. With these programs becoming so widespread, it might even make sense (no pun intended) for libraries or even individuals to consider becoming affiliates of the search engines they use often. If you've got to search, why not get paid?

By the way, we might be seeing Yahoo make a move away from the alphabet. A sharp-eyed reader alerted me to a category where Yahoo first presented a "Most Popular Sites" area, followed by its standard listing format. That page has since gone back to normal. I'll bring more details, as they are available.


Google Directory

Google Buttons

Google Affiliate Program

Search Assistance Features

Explains more about page clustering and ways to find related pages, including using the GoogleScout feature.

Go Guides

Hate the alphabet? Go's volunteer directory also rates sites in order of popularity, with three stars being best.

AT&T WorldNet

Features a version of the Open Directory where results have been organized in terms of what people click on, using Direct Hit's technology.

Yahoo Shopping Agents

Open Directory Price Comparisons

Google Price Comparisons


Need to comparison shop? This is a wonderful service, especially improved by the fairly recent and needed addition of a search box to the home page. You no longer need to navigate first to a category and then search.

Webbys Technical Achievement Nominees

Like Google? They've been nominated for a Webby -- you can vote for them here. So has AltaVista's translation service. Like another search engine? Write-ins are accepted, but don't get your hopes up.

Google: We're down with ODP
Salon, March 24, 2000

Another look at the Google-Open Directory partnership