Searching The AOL Time Warner Way

Netscape Search used to be a powerhouse in the search world, because so many people were routed to it via the Netscape browser. Those heady days have declined, as Microsoft's Internet Explorer browser now commands the lion share of web users. However, Netscape Search has a new lease on life, via the AOL Time Warner network.

The what network? Cast your mind back to last January. That's when AOL and Time Warner completed their merger, creating a single company that owns properties such as CNN, Time, the WB Network and many others -- not to mention AOL itself.

Potentially, AOL Time Warner could have tried to force all of these companies into a single portal, imitating the similar and failed attempt by Disney, which tried to make diverse sites such as ABC News, ESPN and Infoseek all part of the ill-fated Go portal.

Instead, AOL Time Warner is wisely letting its sites operate as independent entities. However, the company recognizes the advantage of somehow uniting these entities into a network -- and that uniting factor is Netscape.

In March, AOL Time Warner rolled out the "Netscape Toolbar." It's a small graphic that now appears on many AOL Time Warner web sites. It contains icons leading to the Netscape portal, Netscape Mail, the Netscape-edition of AOL's Instant Messenger and Netscape Search.

The toolbar initially appeared at the magazine web sites of Time, People and Money, as well as Warner Bros. Today, it has been extended to appear at other places, such as CNN, HBO and the WB Network.

The toolbar essentially makes Netscape Search the "official" search engine of AOL Time Warner -- with the notable exception of at itself. But we'll get back to that in a bit....

The Netscape presence at all of these sites means that potentially, the Netscape portal may receive a new and steady stream of visitors. The same is true for the Netscape Search service, especially as it has its own dedicated button. Certainly Netscape says the toolbar, or "hat," has helped:

"As one might expect, the increased exposure of Netscape web services on AOL Time Warner brands such as CNN and Time has had a positive contribution of traffic to Netscape. All of the services displayed on the Netscape 'hat' have seen their numbers rise accordingly, but Netscape Search has experienced the greatest benefit from this added presence," said Myron Rosmarin, senior product manager of Netscape Search.

Reaquaint Yourself With Netscape Search

So what are these people being directed to Netscape Search likely to find? Despite a relaunch this summer, Netscape Search remains pretty much the same as it has been for the past two years, a source of human-powered search results.

Netscape continues to use the Open Directory Project as its primary catalog for answering queries. Though owned by AOL Time Warner, the Open Directory project is a volunteer-based effort to catalog the web. It now has over 40,000 editors and nearly 3 million URLs listed.

Matches from the Open Directory appear in the Reviewed Web Sites section of Netscape Search's results page. Below this, the "Web Site Categories" section will lead you into topics from the Open Directory that match your search. Selecting a category will bring up a list of sites from that topic, out of the Open Directory.

Of course, the Open Directory's editors haven't categorized everything. In cases where no ODP information is available, Netscape Search then "falls through" to providing matches from Google.

It's a partnership that has been maintained since mid-1999. You can also choose to search directly against Google by selecting the "Search again with Google" link that appears near the bottom of the Netscape Search results page.

Be aware that in either case, you won't hit the entire Google database when searching "through" Netscape Search. Instead, only a portion of the Google database will be queried.

For example, a search for "yucaipa" brought back 6,100 Google matches when done via the Netscape Search results page, while when done at Google directly, there were 25,700 matches found.

Shocked? Well, Yahoo does the same thing with its partnership with Google. In both cases, the companies only hit a portion of the Google index, because it costs them less to do so. Google's never been extremely clear on exactly what gets hit in these situations, but it is likely that only the most popular pages get queried.

Because of this situation, it is best to go directly to Google, if you feel your query would best benefit from seeing results from across the entire web. However, if you are primarily looking for a human-oriented view, then Netscape Search (or Yahoo) provide this experience, with some backup available from Google.

New Features Available

So what new features came out of the last August's relaunch? The most dramatic was the change to the Netscape Search home page. It gained buttons allowing visitors to search using other search engines that have cut deals with Netscape. Specifically, these are Ask Jeeves, Google, LookSmart, Lycos and Overture (though it is still listed under the former name of GoTo).

Annoyingly, searching against any of the services listed on the Netscape Search home page, including Netscape Search itself, now pops open a new window with your results.

By the way, when using the Google option from the Netscape Search home page, you will hit all of Google's results. This is because you'll be querying Google directly, rather than through Netscape Search.

"Tabs" along the left-hand side of the Netscape Search home page give easy access to a variety of vertical searches. For example, the "Classifieds" tab gives you the ability to job search via (if you further select the "Careers" option), while the "Money" tab provides access to stock quotes, and the "References" tab puts dictionary, almanac or encyclopedia information at your finger tips. Well, supposedly. I couldn't get the encyclopedia option to work, when I tried it.

It would be nice if you had a better idea of what was powering some of these searches. For instance, the "Travel" search tab brought up some useful information when I tried a test query, but I had no idea how it was compiled.

Similarly, the "News" tab brought up some good headlines on test queries I ran, but it was impossible to easily see what the underlying source material was. Netscape told me when I asked that it is primarily material from the Associated Press and Reuters. However, it would have been better for anyone to easily discern this.

A useful new addition to the results page is related searches functionality. When you do a search, you'll often find that other popular searches related to your original query can be found at the top of the results page, in the "Most popular related searches" area.

For example, a search for "cars" brings up related searches such as "used cars," "car insurance" and "car rental." Just select one of the related searches shown, and your search will be rerun with those terms.

"We found that the users are really gravitating toward these search results," Rosmarin said. "When related search terms are available for a query, we've been noticing double-digit click-thru rates."

It's a great way to narrow in on what you are looking for, and similar functionality is also offered by other search engines (a page below lists them). Ironically, Netscape Search-sibling AOL Search removed its own related searches functionality earlier this year.

Paid Listings At Netscape Search

Sadly, Netscape Search did not choose to change the title of its paid listings during the redesign process. These may appear below the related searches area and remain labeled "Partner Search Results."

This label is simply a euphemism for paid listings provided by Overture. In contrast, AOL Search, which also uses Overture links, clearly calls them "Sponsored Links." You'd think Netscape would do the same and avoid potential criticism, but the company says user testing has found the labeling is clear enough.

"We tested two attributes of these listings. Was the user aware that these links were paid listings? What was the user's inclination to click on the link? We found that calling them 'Sponsored Links' versus 'Partner Web Sites' didn't make much of a difference in the user's awareness that the links had been paid for but that it did make a difference in the user's inclination to click on the link. That helped govern our decision to leave the labeling as it was," Rosmarin said.

There's also a Netscape Recommends section that may appear in the results. This is relevant content within the Netscape site or AOL Time Warner that Netscape wishes to promote.

You'll also find this is the area to look if you want to find the "official" web sites of major companies. For instance, a search for "microsoft" brings up a "Microsoft Corporation - Official Web site" link. Similar links appear for searches on "Avis" or "Sears."

Behind the scenes, a key change to Netscape Search is that it now goes beyond just checking the title, URL and description of sites listed in the Open Directory. It also has indexed the full-text of each page listed. This means that it may return some URLs for a search, even if the search terms don't appear in the Open Directory's description. It's something that sibling service AOL Search has done for over a year.

For example, a search for "roddenberry" at Netscape Search (or AOL Search ) brings up several sites about Gene Roddenberry-created television shows, even though the word "roddenberry" doesn't appear in the site descriptions. This is possible because the word does appear on the actual pages listed.

In other words, the page indexing has taken the card catalog of the Open Directory and added many more words about the URLs listed. As a result, the ODP listings are more likely to come up than in the past.

"You'll have more sites provided from the ODP and fewer fall thoughts," Rosmarin said.

For webmasters, this means it is important to ensure that the pages you have listed in the ODP adequately describe the content of your web site. By doing this, you can enhance the chances of getting found for various terms.

Of course, this also means that you may get some unexpected results when doing a search. For example, a query for "discount cameras" in mid-October brought back some sites that were clearly off-topic. I even examined the body copy of some of these sites and found the terms didn't appear. Netscape followed up on this and came back with the explanation that they'd been spammed. Despite this, the sites remain listed.

Search And The Netscape Browser

Netscape is more than a web site, of course -- it is also a browser. The latest version of that browser, Netscape 6.1, was released in the summer, and search is deemed a key component of it.

Indeed, the address bar has a big "search" button sitting right next to it, so that the address bar essentially doubles as a search box that feeds into Netscape Search. Just enter your queries into the bar, push "search," and results from Netscape Search will appear.

You'll also see that the listings for your search will also appear in the "Search" area of the "My Sidebar" portion of the browser. This makes it easy to see both your results list and choices from it. Just select a listing from My Sidebar, and it will load in the main browser window. Don't like what you get? Then just choose another result from My Sidebar.

You can also query Netscape Search directly from the My Sidebar area, or change the default "Using" setting to instead query Google, Lycos or Overture (GoTo).

You'll also find that Netscape Search gets used if you don't enter a full-formed domain name. For instance, enter "california" rather than (or, etc.), and you'll get back matching results from Netscape Search.

In some instances, however, you'll be delivered directly to a web site that Netscape believes is the best match. For instance, "amazon" takes you to, while "microsoft" delivers you to

Oddly, if you push the "search" button in Netscape, you end up at an old version of the Netscape Search page, which uses a "tab" format for displaying search partners and even still lists NBCi among the choices.

Netscape says the display of the older page is intentional, but the reason why wasn't exactly clear. The impression I got is that there are some partnerships on that page that are still running, so they need to keep pointing people at the old format, for the time being.

AOL Search For AOL Users

So what about AOL Search? How does it fit into AOL Time Warner's plans, now that Netscape Search is being revitalized? It continues to have its own unique role, that of serving AOL users with search needs.

AOL Search is the only search engine that can allow AOL users to search across all of the proprietary content within the AOL service and the web itself. Anyone who accesses the service from within the AOL software gets this unique hybrid.

There's also an "external" version available on the web lacks the AOL content links. Other than that, it retains the same look and feel of the regular AOL Search service, which is good, considering the primary audience of is AOL members seeking to access email or other AOL features when away from their software.

As AOL puts it, is the "home away from home" for AOL members who are without their software. And, in case you were wondering, the "vast majority" of AOL Search users make use of the internal version, said Gerry Campbell, AOL's director of search and navigation

Like Netscape Search, AOL Search uses the Open Directory for some of its listings. However, it uses its own ranking algorithms to sort through this material. It also uses Inktomi as its crawler-based partner, rather than Google.

In addition, AOL Search recently abandoned its traditional delineation between these two listing sources. Previously, you would get first OPD listings, then Inktomi listings. Today, listings from either service are mixed together.

Inktomi's Bigger Role At AOL Search

AOL Search says it has been operating in this "fusion" mode since late July. However, it has only been until this past month or so that it has seemed more noticeable. Indeed, I had a stream of messages come in last month from people who assumed that the ODP was no longer being used, because they saw so many Inktomi links now.

Part of this seems to stem from the fact that AOL Search began using the Inktomi "GigaDoc" index in late August, AOL says. That index contains about 500 million pages from across the web.

Previously, AOL Search only used the Inktomi "Best of the Web" index, which contains about 110 million pages. The addition of so many more Inktomi listings naturally gives them a greater chance of appearing in the new fused format.

The AOL Search-Inktomi relationship was recently renewed. AOL Time Warner doesn't cite any particular reason for using Google for one of its search properties and Inktomi for another, simply saying that folks are happy with the status quo.

"Netscape has been very happy with their relationship with Google, and AOL has been very happy with their relationships with Inktomi," Netscape's Rosmarin said.

A key reason for having both is that AOL Time Warner is probably happy to play the companies off of each other, keeping the threat of dumping one of them as a way to get the most favorable deal for their services. In the same way, Terra Lycos uses Inktomi for its HotBot service while FAST Search gets to power the Lycos properties.

Another reason for expanding the use of Inktomi listings at AOL could be because of Inktomi's paid inclusion program. As Inktomi adds more paid inclusion links, especially of the pay per click variety, its search partners like AOL stand to earn more of this revenue, if they expose more of the Inktomi data.

For its part, AOL Search said that bringing in the additional Inktomi information was done primary to increase hits in the "tail" of the query stream, those queries that come up only once or twice per month. By having the larger Inktomi index online, AOL Search says it hopes to provide better answers for these queries.

This mixing follows on the redesign change that AOL Search made last June, when it dropped a "tabbed" structure that let people choose to see either ODP or Inktomi listings. The tabs weren't found to be effective.

"People didn't get what they were doing," said AOL's Campbell. "The best way to hide a results set is to put them on a tab. We saw people don't click on tabs."

That's ironic given that Google has just moved to a tabbed interface. However, there is a greater difference in what the tabs do at Google than what they did at AOL Search, so Google may have more luck with them.

A more recent and welcome redesign was the return to listing 10 editorial results. AOL Search dropped to only five editorial results in June, but the company increased this amount back in mid-September.

"We found our members wanted more web results and were willing to scroll down a longer page in order to get them," Campbell said.

As seen, the Open Directory is a crucial part of listings for both Netscape Search and AOL Search. This means that being listed with the OPD is crucial, for webmasters. To help them in this task, the Open Directory has recently added some new information in its help section for site owners. Within this area are general tips on submitting, getting pages updated, guidelines on category placement and more.

Here are also some additional tips for the Open Directory, to questions that have come in from readers recently.

I can't get listed and followed your resubmission instructions.

I won't repeat the instructions here, for space reasons, but you'll find them on the How The Open Directory Works page, listed below. Instead, let's go beyond them.

You've resubmitted after three weeks and also sent email to the editor of that category, with no luck. Next, go "up" a level to the category above the one you submitted to, and send a polite message to that editor (you would also do this if you submitted to a category that has no editor).

Still no luck? Then use the ODP Staff Feedback page (listed below) or email That will put your message in front of an ODP staff member.

I can't get my listing changed or I disagree with how I appear!

You should first have used the Update option (see the How ODP Works page). If you did that and two weeks have gone by, then send feedback to the editor of the category. Still no luck after about two weeks? Then go up a level and send a message to that category editor. If that doesn't work, then use the Staff Feedback page.

There's an editor who clearly is abusing the ODP!

Unfortunately, the weakness to a volunteer directory is that there will be some people who try to use it for their own gain. The Open Directory doesn't flat out ban those with an interest in a particular category from running it. For example, if you sold collectable toys, you might also be the perfect person to run the collectable toys category, because you know so much about the subject. The ODP might allow you to do this, as long as you didn't favor your own site over others.

Should you suspect that someone is going beyond the situation above and abusing the ODP for their own benefit, then assemble your evidence succinctly and send it to the ODP using the Staff Feedback page. Make sure you use "Abuse Report" as the title of your message. Save a copy before sending it. That way, if your browser should glitch (as can happen), you can easily resend.

Someone is spamming the Open Directory!

As with possible editor abuse, assemble your evidence, then use the feedback form and put "ODP Spam" as the subject.

Finally, you may occasionally notice at AOL Search that there is no standard "green box" at the bottom of category listings, allowing you to submit to the Open Directory. This is an occasional bug that pops up, which AOL says it is working to correct.

Should this happen to you, make a note of the category name, search for it at the Open Directory, and submit to the category from within the Open Directory web site.

Netscape Search

AOL Search (Internal Version)

Ordinarily, you would reach this version of AOL Search using the AOL software, such as via the "Search" button, but it can also be reached via the URL above. This version will contain links to content that can only be viewed from within AOL.

AOL Search (External Version)

Open Directory

Home of the volunteer web cataloging project that provides results not just to AOL Time Warner but also to anyone who wants them. Even non-AOL Time Warner search engines such as Google and Lycos make use of Open Directory information

How AOL Search Works

Updated slightly to reflect the new fusion listings, and the page also provides more information on how AOL factors "popularity" into creating its results. For Search Engine Watch members, this page explains more about how the AOL Search service operates, especially from a submission point of view.

Open Directory Help Central

See the "For Submitters and Site Owners" section for general tips on submitting.

How The Open Directory Work

More tips and information on submitting to the Open Directory, for Search Engine Watch members.

Open Directory Project Staff Feedback

Use this form for sending feedback to the Open Directory's staff.

Open Directory Guidelines

Designed primarily for ODP editors, this page is nonetheless used as a reference by the service for submitters who want to learn more about how to write good titles and descriptions for their web sites. The page also offers useful tips such as what's deemed to be spam, why sites might get deep linked, the type of editors that exist with the service and why editors might be selected, rejected or removed.

OPD Editing Guidelines
Even more guidelines for particular categories can be found here.

DMOZ Newsletter

This is a newsletter for OPD editors which also can be viewed by the general public. It's an interesting way to learn more about the service, from an editor's perspective. For example, an article like "To Deeplink or Not to Deeplink" gives you a better idea of what editors will look for or reject when it comes to deep links. That can perhaps help you understand what content is worth submitting and what is likely to be rejected. Unfortunately, links to articles in the current issue don't appear to work correctly. However, they do work if you browse from the second URL listed above.

Review: Netscape 6.1
BrowserWatch, Aug. 23, 2001

Closer look at features in the new Netscape browser.

Search Assistance Features

More about related searches functionality can be found here.

Netscape's New Mission
Washington Post, Aug. 21, 2001

How the Netscape brand is being rebuilt to serve AOL Time Warner's needs.

AOL Meddling in ODP Causes Shift in Balance of Editorial Power
Traffick, Sept. 4, 2001

Long criticique of perceived problems with the Open Directory, in terms of how paid staff allegedly override the wishes of senior volunteer "meta" editors and other issues. Written by someone said to be a high-ranking ODP editor, though a pseudonym is used.

Digital Windmill

Want to add the Open Directory to your own web site? A free tool from this company lets you do it via just three lines of JavaScript into your web pages.

Chicken's World

Another solution for placing Open Directory information into your own web site.