The Search Engine Update, Oct. 18, 1999, Number 63

October 18, 1999 - Number 63

By Danny Sullivan
Editor, Search Engine Watch

About The Update

The Search Engine Update is a twice-monthly update of search engine news. It is available only to those people who have subscribed to Search Engine Watch,

Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.

In This Issue

+ General Notes
+ AltaVista Relaunches Soon, To Use Open Directory
+ Directories Power On
+ Subscribing/Unsubscribing Info

General Notes

Hello Everyone--

I just got back from Internet World, where I met with the people who run AOL Search. From that, I'll be writing up my promised review of the new service for the next newsletter. I'll also give you a recap of interesting things related to search that I saw at the conference.

It's also just a month until the Search Engine Strategies '99 conference in November, an all-day event that I'm organizing which covers aspects of search engine promotion and marketing. Sponsorships are also still available. More information on attending or sponsorships can be found via the link below.

Search Engine Strategies '99

Search Engine News

AltaVista Relaunches Soon, To Use Open Directory

Expect to see a new AltaVista next Monday, Oct. 25. I've been briefed but can't bring you more details until the public launch -- after that happens, expect a detailed review. Also, in the edition of the Update that was mailed out, I said that AltaVista's main search results would come from the Open Directory. It turns out that the slides I was provided by AltaVista of its new service were mislabeled, which lead to confusion over AltaVista's use of the Open Directory's information. AltaVista's main search results will continue to be derived from crawling the web, with matching categories of human-compiled information from the Open Directory presented after that.



Directories Power On

In the search battle of human versus machine, 1999 is the year that the tide turned in favor of humans. One-by-one, major services that served up predominately crawler-based results have switched over to using human-power for their main listings.

Let's recap. In April, Lycos changed to the Open Directory. In June, Netscape Search made the same move. September saw AOL Search begin using Open Directory information, while MSN Search changed to using LookSmart-powered results. Later this month, AltaVista becomes the latest Open Directory convert, listing its categories within its main results.

Here's some perspective. At the beginning of 1996, only one of the six major search services provided human-dominated results -- Yahoo. The others -- Excite, Infoseek, Lycos, WebCrawler and AltaVista -- served up crawler-dominated results.

Now at the end of 1999, only three of the top six major search services remain predominately crawler-based. These are Excite, Go (Infoseek) and AltaVista. Expand the list to the top ten major services, and you only pick up one additional crawler-based service -- HotBot. (For those keeping score, the human-powered ones are Yahoo, MSN Search, Netscape Search, Lycos, AltaVista, LookSmart and Snap).

Crawlers aren't obsolete, of course. Humans are used for the main results at the services above, but crawlers provide the necessary and welcome backup for when there aren't any matches from a human-compiled database.

Let's look at the remaining "holdouts" and make some predictions. I'd say the next to switch is Go, which already has a substantial, quality directory that it could leverage at any time. HotBot already floats Open Directory categories at the top of its listings, but I suspect its deal with Direct Hit means that we won't see Open Directory sites supplanting Direct Hit information in the main results in the near future. AltaVista has just recommitted to using crawler-based results, so it seems unlikely that they will change over soon. That leaves Excite. Given that most of its competitors have changed over to a human results first, crawled results second model, I think it will be inevitable that Excite will also make the change. Moreover, Excite recently cut a deal with LookSmart to license that company's directory. That gives Excite the ability to more easily promote human-powered results.

Of course, the big question is, why has this change happened? For one thing, the Open Directory has sparked a revolution by making human results available for free. That's given Lycos and AltaVista the easy ability to change. And since AOL owns the Open Directory, it makes sense for Netscape Search and AOL Search to use it.

A bigger reason is that directories tend to provide better results because of their nature. A user searches for "travel," and they get a list of categories that are related to that incredibly broad topic. They select a specific category of interest, and in doing so, they painlessly narrow their search without even being aware of doing so. In contrast, a user searching for "travel" on most of the crawler-based services will get millions of matches. They can feel overwhelmed by this large number, and it's worse if the top results aren't relevant to what they are looking for.

So what's next? Better ranking. It's not enough to categorize web sites. They need to be ranked in order of quality. Currently, users do a search, see a category of interest and click through. There, they discover a list of sites where the alphabet is the main ranking criteria involved. Maybe a few "editor choices" top the list. But pretty much, sites that begin with A come first, then the Bs, then the Cs, and so on.

Enough of this. It's like telling someone to get something to drink by pointing them at a soda fountain with 100 different taps, one for each brand. The variety is nice, but some guidance would be appreciated.

Go does guidance right. Check out its directory, via the home page. Sites are ranked in order of quality, three stars being best. I find myself continually drawn back because of this feature. It helps me sort through the mess and decide which sites are worth my time.

What does this change to the directory format mean for the webmaster? I've always said that the most important promotional activity that you can do is concentrate on building a quality web site. Doorway pages are not going to help you impress an editor, when you submit your site for review. Instead, editors want to see a polished web site that loads quickly, is easy to navigate and which offers substantial content to visitors. Build your content. Make your site a place to visit that's related to your underlying products or services. For instance, if you sell car tires online, then ensure that your site has lots and lots of good general information about buying car tires -- in addition to information about your actual products.

The popularity of the Open Directory also means you need to adopt a specific strategy there: multiple listings with different descriptions. At any directory, you have to submit to a particular category. The best way to find the right category is to search for your most important search terms, see which category comes up first and try to get listed there. The problem is that the Open Directory now powers four different services, each of which uses slightly different ranking systems. So, if you pick the "right" category via AOL Search, you may miss out on being in the right category over at Netscape Search.

Here's what to do. Go to each of the Open Directory powered services. Search for your top terms and review the categories that appear at each of them. Submit to all the appropriate ones for your site. There will be some overlap -- for instance, let's say all of them list a category for online car sales. That's fine -- submit to that category once, and you'll be covered on all of them for it. But you may notice that one of them also lists a category about tire buying tips. Perfect, here's another appropriate place for you to submit a relevant subsection of your web site. Now you've expanded your presence and the chances that you will rank well at various Open Directory-backed services.

Furthermore, don't use the same description for all of your submissions. Use a slightly different or even a dramatically different description for each of your different listings, but always ensuring to make use of your key search terms. By doing so, you increase the odds of doing well at the different services, which as I mentioned, have their own ranking criteria.


How The Open Directory Works

Learn more about submitting to the Open Directory.

Password Finder

In case you forget your password, when trying to reach the article above.


How do I unsubscribe?
+ Follow the instructions at the very end of this email.

How do I subscribe?
The Search Engine Update is only available to paid subscribers of the Search Engine Watch web site. If you are not a subscriber and somehow are receiving a copy of the newsletter, learn how to subscribe at:

How do I see past issues?
Follow the links at:

Is there an HTML version?
Yes, but not via email. View it online at:

How do I change my address?
+ Send a message to

I need human help with my subscription!
+ Send a message to DO NOT send messages regarding list management or site subscription issues to Danny Sullivan. He does not deal with these directly.

I have feedback about an article!
+ I'd love to hear it. Use the form at

This newsletter is Copyright (c) corp., 1999