The Search Engine Update, August 2, 2000, Number 82

August 2, 2000 - Number 82

By Danny Sullivan
Editor, Search Engine Watch
Copyright (c) 2000 corporation

About The Update

The Search Engine Update is a twice-monthly update of search engine news. It is available only to those people who have subscribed to Search Engine Watch, Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.

In This Issue

+ About The Search Engine Watch site
+ Conference News

+ LookSmart First To Charge Commercial Sites
+ iWon Expands Search Offerings
+ Inktomi To Offer Paid Submit Option
+ Ask Jeeves Adds Directory, Direct Hit Results
+ Google Gains More Features
+ AltaVista Redesigns, Again
+ Invisible Web Gets Deeper
-- (full story online, summary and link provided)
+ Now, It's The "Vectories" That Are Coming!
+ Search Success Rate Increases
-- (full story online, summary and link provided)

+ FAST Gets Advanced Search, Customize Pages
+ Excite Moves To Your Toolbar

+ List Info (Subscribing/Unsubscribing)

Site News

Hello Everyone--

I've posted a complete revision of the How Excite Works page, within the Subscribers Only area of the site. If you've kept up with past newsletters, much of this will be familiar. However, I did add new details about how I believe the two index system at Excite works. The page is here:

How Excite Works

Password Finder

In case you forgot your password, the finder will help you access the article above and some links mentioned elsewhere in this newsletter.

All you Boolean junkies will be happy to know that Excite now supports a NOT query by just specifying NOT, rather than AND NOT. However, you still must put Boolean queries in uppercase.

Also, Excite also appears to be quietly testing RealNames links. Try a search for "Microsoft" and look at the embedded URL. You'll see that it redirects through RealNames. I've seen a few other examples of this.

Conference News

The next Search Engine Strategies conference is less than two weeks away!

Coming to San Francisco on August 14, I'll be presenting and moderating sessions at the conference that feature experts on search engine marketing issues and panelists from the various major search engines themselves. Search engines participating in the panels include, AltaVista, Ask Jeeves, Excite, Google, Inktomi, LookSmart, mySimon, NBC Internet/Snap, Netscape/The Open Directory and Yahoo. There will be a special session on shopping search, which should be of interest to any online retailers. Details about the conference, for attendees or potential sponsors and exhibitors, can be found via the URL below.

Search Engine Strategies 2000 - San Francisco


LookSmart First To Charge Commercial Sites

After months of beta testing, LookSmart has become the first major search service to charge commercial sites to be reviewed for inclusion in its listings. In addition, the revenue-generating move appears to have helped LookSmart win back its former position as AltaVista's directory provider. It has also strengthened LookSmart's ability to compete as a provider against the Open Directory, which allows any search site to use its human-compiled listings for free.

LookSmart's "Express Submit" service does not guarantee commercial sites a listing or better rankings. This isn't pay for placement, as at, where a site can pay more to appear higher in the search results. Nor is it pay for inclusion, where a site would only be included in listings if it paid a fee. Rather, this is pay for consideration. The payment levied against commercial sites ensures that they will be considered for inclusion into the index in a timely manner, but they should ultimately only be listed if they also meet editorial criteria.

In LookSmart's system, commercial sites can pay US $199 to be listed within the directory, assuming they meet acceptance criteria. The vast majority of sites will meet these, so spending the fee isn't much of a gamble.

"The percentage of rejection is pretty low, but we do hold them to the same standards," said Kristin Morse, LookSmart's director of ecommerce.

If a site is rejected, refunds are no longer offered, as was the case in the past. However, there remains a 30 day appeal period. Sites can make changes and then reapply during this time, without having to pay again.

The $199 option has been available since January, but a new $49 option has just been added. The only difference is the response time. Those paying $49 may have to wait up to eight weeks for their sites to appear, assuming the sites meet acceptance standards.

The lower price point is a compromise between LookSmart wanting to accommodate sites with little money and the problem of commercial sites continuing to use the free submit option, when it was available to them alongside the pay submit option. Morse estimated that of the over 1,000 submissions sent through the free option per day, 80 to 90 percent of them were from commercial sites. By requiring all commercial sites to use the pay option, this should make it easier for non-commercial sites to be listed.

I certainly hope that this proves to be the case. If so, it should benefit both searchers and webmasters, because some non-commercial sites that were lost in the blizzard of commercial submissions may now get the attention they deserve.

LookSmart doesn't guarantee how long it will take non-commercial sites using the free submit option to appear. However, Morse said that any qualified site really should show up within eight weeks. I'd rather hear that this will happen within a week or less, which would show that removing the commercial stream of sites really will benefit the non-profits.

If you run a non-commercial site and don't appear within eight weeks, resubmit. Alternatively, non-commercial sites can use either of the paid options, to speed up the process. By the way, non-commercial means that your organization is formally registered with the US government to have not-for-profit status. Unfortunately, even if your site is a labor of love that earns you no money, you cannot use free submit unless you are registered.

Should businesses get up in arms over the new charges? I'd say not. This is the price of doing business on the web, just like paying for a domain name or for a professional web hosting service. Yes, it is possible to run a web business using free web space and without getting a domain name of your own, but you will ultimately be more successful in the long run by doing so.

In fact, if you are serious about doing business on the web, using free web space is a false economy. One reason is that it is often difficult to ensure that all the traffic going to your free web site can be redirected to your new site, when you finally make the move to your own space. Do it right from the beginning, and you won't regret it.

Also, unlike a domain name, the LookSmart submission fee is not recurring. You pay once, and that's it. Moreover, a LookSmart listing will generate traffic for your web site, which is not the case for registering a domain name (unless its a good, generic one). Your site will be listed in LookSmart's main results, the main results of MSN Search, and will also be available through other ways at iWon and AltaVista, to name only LookSmart's major partners.

Do the LookSmart submit changes mean that some deserving commercial sites may go unlisted, because they cannot afford to pay or refuse to pay. Possibly, but not necessarily. LookSmart strongly emphasizes that the submission system is only one way editors find potential listings. Editors are also watching various resources across the web, looking out for new web sites to add independently of the submission service.

"We are keeping intact our usual editorial policy of finding the best of the web," said Scott Stanford, LookSmart's vice president of ecommerce and distribution. "If luck favors you, your site will be found, as soon as our editors get to it.

I'd strongly advise those with commercial or non-commercial sites not to leave it to luck. Use the paid option rather than relying on editors to find you. You'll probably get listed faster, and more importantly, you can give them suggestions about how your site should be described and where it should be listed. If you leave it to the editors, they may not include important terms relevant to your site, in your description.

Charging a fee should also limit the amount of spam and irrelevant listings that LookSmart editors receive, which anyone who operates a search engine can tell you is a serious problem. In fact, many webmasters who have become editors at the Open Directory have reported gaining a much greater appreciation of how few people submit correctly, after they've had to step into the editor role.

One serious concern is that LookSmart's standards might drop in order to allow more sites in and thus ultimately make more money. LookSmart has previously said it has no intention of letting standards slip and that listings would continue to be handled by the editorial staff, rather than becoming an advertising option.

LookSmart's search partners will share in the wealth the new submission system will produce. MSN Search, Excite, iWon and AltaVista now or soon will have options allowing people to submit directly to LookSmart and thus appear within the LookSmart-powered portions of their web sites. LookSmart said these partners will receive some of the submission fees, in relation to the submissions they generate.

iWon and AltaVista are recent partner wins by LookSmart, with AltaVista being most notable. LookSmart had provided some of AltaVista's results from January 1998 through October 1999, when the Open Directory took over as the primary directory provider. The attraction of the Open Directory is that it doesn't charge for its listings, while LookSmart was engaging in licensing deals or ad share revenues for those who wished its services. Because of this, it seemed as if LookSmart couldn't continue to attract new partners. After all, the logic went, who would pay for LookSmart's listings when they could get listings from the Open Directory for free?

Now the submission revenue share option flips that argument around 180 degrees. Why go with the Open Directory, when going with LookSmart might make you money? This isn't the only reason future partners might choose LookSmart, but it is an important factor. The article below touches on the revenues AltaVista expects to make. LookSmart takes over from the Open Directory as AltaVista's exclusive directory partner later this month.

Another factor here is LookSmart's recent redesign, made at the end of June. The company recently deemphasized features such as free email, news headlines and horoscopes, designed to help it compete as a portal, and the directory structure has been left in place primarily as a demonstration site for potential partners such as iWon or AltaVista. In many ways, it reflects LookSmart's goal to be a behind-the-scenes directory provider, similar to how Inktomi is a behind-the-scenes crawler-based search engine provider.

"The message for our partners is that we are not in the business to compete with them but rather to support them," said spokesperson Liz Connaghan.

LookSmart Submit A Site

Information from LookSmart about its various submission options.

Free Listings Gone At LookSmart
The Search Engine Update, June 2, 2000

Previous article on the LookSmart paid submit service, which touches on issues about editorial staff remaining in charge. It also provides tips on why you should select a proper category at LookSmart.

How LookSmart Works

More background on submitting properly to LookSmart, including how to submit subsections of your web site.

Yahoo Opens Express Submission Service
The Search Engine Report, March 3, 1999

Past article about the launch of Yahoo's own express service last year.

Yahoo Business Express Help

Information from Yahoo about its express service, similar to LookSmart's but less extensive.

LookSmart pays portals to use its service, July 26, 2000

Good details on the revenue relationship between AltaVista and LookSmart.

Wherewithal Hires Former Open Directory 'Editall'
Wherewithall, July 31, 2000

In another challenge to the Open Directory, Wherewithal has gained a former Open Directory editor to help it in its quest to go further by paying editors to list sites. More details from the company's press release.

Community Search Blossoms - Part 1
The Search Engine Report, July 5, 2000

More details about Wherewithal and a link to an article from the former Open Directory editor that they've hired.


iWon Expands Search Offerings

Over the past three months, iWon has been adding new search resources to its site, which offers users the chance to win money as they search the web or undertake other portal activities, such as checking email or stock quotes. It's a move away from the less-is-more trend that you see at places such as Google, AltaVista's Raging Search and recently revamped Excite search engine. However, iWon feels providing a variety of results serves its users better.

"A lot of people think simple is better and web pages [only” are better," said George Nimeh, iWon's vice president of search and special projects. "But for the time being, we are trying to surface multiple data in a clean and uncrowded way.

Previously, search at iWon brought up information primarily from Inktomi's crawler-based index of web pages. Now information from several search providers, such as LookSmart, Direct Hit and Moreover, is being presented.

Inktomi's crawler-based index remains the power behind the main portion of iWon's listings. Do a search from the home page, and it is information from Inktomi's index that appears in the "Matching Web Sites" section that makes up the bulk of the iWon results page. iWon is also now making use of Inktomi's new 500 million page index, so you should consider it among your choices when seeking unusual or obscure information.

Previously, Inktomi also provided iWon with categorized results. Now, LookSmart fulfills this function. At the top of the results page is a "Related Web Directory Categories" section. The categories listed here come from LookSmart, and clicking on a category link displays an editor-compiled list of web sites related to your search topic.

Although LookSmart information is being used, the listings and categories may not match those exactly at LookSmart. iWon is still using the directory building tools that Inktomi provides to shape the LookSmart directory into a format that best suits iWon. For example, after searching for travel, a category called "Reservations" appears. LookSmart also has a "Reservations" category, but no actual sites are listed within it. Instead, you'd have to drill down an extra level to come to web sites. At iWon, a change has been made that ensures some sites exist at the top level.

On the far right hand side of the results page is a "Most Popular" result section. This provides access to listings from the Direct Hit service, which shows what web pages are most popular based on what people click on in search results (see article below, for more about Direct Hit). For any search, suggested topics will appear. Selecting a topic brings up web page listings for that subject. For example, if you search for "science," the Most Popular section will suggest "science projects" as one of several popular results categories. In turn, clicking on that category up Direct Hit's actual listings related to science projects.

At the bottom of the results page is the "Matching News Articles" section. These are matching stories from across the web, as found by the Moreover news search engine (see article below for more about Moreover).

There's also an "iWon Search" box that appears on the right-hand side of the results page that lets you do a specific searches. "Full search" runs your query against all the providers already mentioned. "Top 10 web sites" just runs the search against Direct Hit results. "News articles" lets you search Moreover's news listings. "MP3/audio" is a new option, which allows you to search for multimedia content. Those listings are provided by Inktomi, based on its crawls of the web.

RealNames is also present at the service. On the results page, you'll see an "Internet keywords" link. Clicking on that will take you either directly to a site that has an exact match in RealNames listings or to a list of sites that have RealNames which contain your search term. The near matching will be a boon to webmasters with registered RealNames, as it can generate significant traffic, as does AltaVista because of its near matching. For searchers, it would be better if only exact matches were reported, in my opinion.

iWon has also just announced a new partnership with the Fact City facts search engine will also be added. Information from Fact City will be available at iWon later this year.

How to get listed in iWon? It's a matter of being listed with its search providers. You should submit your site to Inktomi, and using the HotBot Add URL page is still the best way to do this. For LookSmart, also use its submit service. You can also submit to Direct Hit, through ranking well in Direct Hit will primarily depend on whether your site attracts clicks via Direct Hit's partner sites. Finally, the recent article about Moreover covers how sites with news-related content might get picked up by this service.


iWon Wins Users
The Search Engine Update, Feb. 3, 2000

Past article about iWon that covers information about how the site gives away prizes to attract users.

How Inktomi Works

More information on submitting to Inktomi.

How LookSmart Works

More information on submitting to LookSmart

Using Direct Hit Popularity Results

General information on how Direct Hit results work

How Direct Hit Works

More specific information on how Direct Hit works, with submission tips.

Breaking The News Barrier
The Search Engine Update, July 25, 2000

More about the Moreover news search engine, with submission tips.

Using RealNames Links

General information about using RealNames links to reach web sites.

How RealNames Work

More specific information about how RealNames works, with tips on how near matching can generate traffic via AltaVista (and now, iWon).

Fact Search Engines Available
The Search Engine Report, July 5, 2000

A bit more about Fact City and what a "fact" search engine provides.


Inktomi To Offer Paid Submit Option

In the wake of LookSmart's new pay-for-consideration system, Inktomi has just announced that it will offer a pay for inclusion service later this year. The program will initially launch in conjunction with Network Solutions, but it will likely be expanded to allow Inktomi's search engine partners and perhaps selected submission services to participate, according to Inktomi.

The service will ensure that web site owners can find their home page listed in Inktomi-powered search engines with 48 hours after submitting, and Inktomi will revisit the home page every 48 hours to check for changes. The service will not give the pages any ranking boost.

"This is just about getting into the index. It's still up to the site owners to make sure their content is relevant," said Troy Toman, general manager of Inktomi's search division. "There's no preferential treatment," Toman said.

A free submit option will be retained, but there are no guarantees on how quickly sites will be added or if they will be retained in the index. Inktomi's indexing system may drop pages if they don't appear to be satisfying search queries. The paid option would help webmasters overcome this by keeping their pages listed permanently. It would also offset Inktomi's cost of retaining pages that might seem to have no particular search value.

In particular, Inktomi determines how a page satisfies queries through clickthrough measurements. If your newly submitted pages never get clicked on in response to searches at Inktomi-powered search engines, then the pages may be dropped. iWon is an example of a Inktomi-powered search engine where clickthrough measurements are taken, so searching over there and clicking on your own site may help keep your important pages in.

The main issue here is that any good search engine ought to be listing the home pages of web sites in a timely manner, period. The paid submit system will help site owners know that they can quickly get listed, but searchers might be penalized if Inktomi takes longer to add new sites, in order to boost the value of its paid submit system. Inktomi has not said this would happen.

A more useful system might be one where sites that pay may get spidered to a greater depth. That potentially gives them the opportunity to attract more visitors, because each page is a potential entry point for different topics. This could add value to a paid submit system without penalizing sites that don't pay. Inktomi says that the system could be expanded this way, after it launches.

Overall, it's a difficult balance that has to be struck. There's a real plus to site owners knowing that they'll quickly be added, but a search engine must also be inclusive of all important sites, lest it begin to lack comprehensiveness. However, a paid submit system might reduce the incredibly serious problem of search engine spamming. The free submit systems that most crawler-based search engines operate leave them open to indexing pages with little value and which take away from the good content they'd prefer to add. Some type of paid system might ease the massive pressure on free submit and actually allow non-paying sites better representation. Whether this will happen remains to be seen.

The program is expected to launch in September or October, probably through the Network Solutions site. Pricing has not yet been set, and it's likely the program details could also change from what was announced today.

As noted above, it is likely that the search engines that Inktomi powers, such as MSN Search and iWon, might also offer a paid submit service through their sites. Additionally, submission services are also other potential outlets. I'm planning a review of two likely candidates to appear in the next newsletter, but some basic details about them are below.

Inktomi Paid Submit Announcement


If you have content locked behind password protected areas of your site, this service can actually create pages that search engines can spider and yet which do not give away your content. The company can also spider existing sites and submit them to search engines. The company has also been talking with some search engines about a paid submit option similar to what Inktomi is proposing.

Position Pro

I've written about Position Pro before, but the service has recently added new features to spider dynamic content and create static pages that search engines can spider, as well as having added some meta tag generating tools.


Ask Jeeves Adds Directory, Direct Hit Results

Ask Jeeves has redesigned its home page and search results page. From the home page, you'll find category links just below the search box. These lead to the Ask Jeeves version of the Open Directory, where sites are ranked in order of popularity. That popularity is as determined by Ask Jeeves-owned Direct Hit, which uses clickthrough measurements to determine what are the most popular pages on the web.

Of course, you can still do keyword searching at Ask Jeeves. After you do a search, the results page has several key sections. At the top are the "normal" Ask Jeeves results, which come from their editors seeking the best sites that match a query. They come under a section that says, "I have found answers to the following questions."

After that, the next section says, "People with similar questions have found these sites relevant." The matches here come from the Direct Hit index of popular pages.

At the bottom of the page, the section that begins "I have also found these sites through other search engines" are meta search results that come from other major search engines.

Finally, along the right hand side of the page are new paid links. These come out of the former Direct Hit paid text ads system, which has now been rebranded the Ask Jeeves Text Sponsorship Network. If you tried Direct Hit paid links before and were disappointed because of poor clickthrough, things might change now that the links are appearing at Ask Jeeves. That service is far more popular than Direct Hit, giving your ads more exposure, and it is possible this may also translate into better clickthrough.

Be careful, however -- the page estimates for terms you bid on will be based on the previous months traffic. With Ask Jeeves now also delivering ads, the estimates are likely to underestimate the number of views you'll pay for. Consider monitoring your bids more closely, until the traffic estimates have normalized.

Ask Jeeves

Paid Links At MSN Search And Direct Hit
The Search Engine Update, April 24, 2000

More about paid links at Direct Hit. The MSN Search program also described has now closed.


Google Gains More Features

The folks at Google have been busy, adding a new advanced search page, a preferences page, dictionary links and a Korean-language version.

The new advanced search page allows to perform some searches without knowing any search commands. For example, you can do a phrase search or search for pages just within a particular domain, all by filling out a form. You can also specify that you want pages in a particular language and increase the number of results returned per page, up to 100. Links to Google's specialty search engines can also be found here, such as its Linux or US Government search engines.

The new preferences page lets Google remember your favorite settings. For example, you can make it always display 100 results per page when you come to the site, rather than this being a one-off choice that the advanced search page allows. Language preferences can be set, and you are finally able to set Google's "SafeSearch" porn filter to work before you do a search.

Google now also provides dictionary definitions, served out of Do a search, then look at the status message that appears above the search results. What's the status message? That's my made-up word for the line that tells you how many pages were found for a search. For example, the status message for a search on "onomatopoeia" says this:

Google results 1-10 of about 6,680 for onomatopoeia. Search took 0.05 seconds.

In the status message, the search word will usually be an underlined clickable link -- in this case, "onomatopoeia." Selecting it takes you to, where the word's meaning will be displayed. You can also switch to use the thesaurus option, once there.

I love this new addition, because it's a fast way to get word meanings. The main flaw is that if you don't know how to spell the word, you won't get the definition, much less the correct spelling.

Finally, Google is beta testing a Korean-language version of its service.


Google Advanced Search

Google Preferences Page

Google in Korean


AltaVista Redesigns, Again

AltaVista has once again undertaken another major redesign of its service, the fourth in less than a year. AltaVista says the latest changes are primarily aimed at making it easier for users to navigate within the service and then outwards to what they are looking for. While AltaVista's heart is in the right place, I don't feel this latest attempt makes life any easier. If anything, there's an overload of information. Links, tabs and color boxes seem to be everywhere, competing for attention.

Rather than tell you how to find the new and interesting stuff among the clutter, I'll just highlight the important points and provide direct links at the end of this article.

+ Perform a search from the home page, and you'll discover that AltaVista now highlights your search words with either italics or bold print, where they appear in the web page listings.

+ The results page itself is actually cleaner, focusing mainly on matches from the crawler-based index. Suggested directory categories and Ask AltaVista responses no longer seem to appear.

+ Use the "Find Results In" box that appears near the top of the results page to send your query to one of several specialty search engines that AltaVista operates. For example, a search for "toshiba portege" could be sent to AltaVista shopping search engine by clicking on the "Products" link or against human-compiled listings by clicking on the "Directories" link.

+ A new Power Search page allows you to finally make use of AltaVista's powerful commands without having to know any special syntax. A nice touch is a pop-up box that displays what domains belong to particular countries, which can be helpful when trying to narrow down searches. Most people will be far better served by this page than the "Advanced Search" page, which remains offered.

+ A new Search Guide area is designed to provide examples of searching in different topics areas, such as the Summer Olympics.

+ A new Entertainment search area is designed to allow you to search and get matching image files, audio and MP3 files, video clips and web pages and radio stations, all on the same page. AltaVista has also made changes that it says should provide more relevant entertainment results. They've done this by generally defaulting the searches to bring back pages from sites they've automatically identified as being related to entertainment, rather than from the web as a whole. This only works when you are within the entertainment area itself.

+ Radio stations are a complete new media type that can be found, but the functionality sounds better than it seems. Internet or terrestrial radio sites listed in response to a query may not actually be playing the song or artist you searched for, even though they are listed. Instead, they may appear simply because they've defined themselves as somehow relevant to that artist or song.

+ A great new feature is the "Search Trends" area, also called AltaVista A-List. Here, AltaVista is highlighting what we are searching for. Lycos has been doing something similar for over a year, and it's always fun reading (yesterday's headline -- Pokemon finally slips from the number one search spot after 30 weeks running). I'm glad to see AltaVista also offering a glimpse into what we search for and how we do it. Hopefully, they'll also add an archive of past postings.


AltaVista Power Search Page

AltaVista Search Guides

AltaVista Entertainment

AtlaVista A-List

Raging Search

If you agree AltaVista is too cluttered, don't worry -- that's why they offer you the clean, fast Raging Search site.

Go's Pure Search Site

Go appears to have quietly rolled out a pure search site similar to AltaVista's Raging Search.

Lycos 50

What People Search For

More ways to see how we search.


Invisible Web Gets Deeper

I've written before about the "invisible web," information that search engines cannot or refuse to index because it is locked up within databases. Now a new survey has made an attempt to measure how much information exists outside of the search engines' reach. The company behind the survey is also offering up a solution for those who want tap into this "hidden" material.

The study, conducted by search company BrightPlanet, estimates that the inaccessible part of the web is about 500 times larger than what search engines already provide access to. To put that another way, Google currently claims to have indexed or know about 1 billion web pages, making it the largest crawler-based search engine, based on reported numbers. Using Google as a benchmark, that means BrightPlanet would estimate there are about 500 billion pages of information available on the web, and only 1/500 of that information can be reached via traditional search engines.

The article below provides full details about this story:

Invisible Web Gets Deeper
The Search Engine Report, Aug. 2, 2000


Now, It's The "Vectories" That Are Coming!

Back in April, I wrote an article about an expected growth in vertical search engines or "vortals," as they are sometimes called. This was due to the emergence of new services that I feel make it cheaper and easier for sites to create their own crawler-based search engines that focus on particular topics. Now another group of new services are emerging, those that offer to automatically categorize web pages into Yahoo-like directories. For example, point these services at a set of pages from different sites about cars, and they'll attempt to automatically group the pages into categories that make sense. As a result, we may see more vertical directories, or "vectories," as I'll dub them, also appearing across the web in the near future.

In general, the vectory products aren't magic. You typically have to provide "training documents" for the major categories you wish to populate. You give the software some good examples of pages about cars and trucks, and it will then use them to group other pages into either a cars or trucks category. Human editors may also be involved to create keyword definitions for some categories and to filter out obviously incorrect matches.

Here's a look at some of the players in the vectory market:


Mohomine is aimed at both the vertical portal market and those needing Intranet search solutions. It can create vectories for consumers or organize a company's internal documents into categories. Mohomine also has document discovery capabilities. Show it examples of documents you like, and it can mine the entire web, if you desire, to recover similar matches. Mohomine can classify more than web pages. It can extract textual information from objects such as computer code, allowing such items to be classified alongside HTML documents. The company launched services in March.


Mohomine-powered search engine that helps software developers track down source code.


Twirlix offers both a crawler-based search engine and an automated directory to its clients. Sites can be ranked by using criteria such as link analysis, the size of a site and freshness of content to determine relevancy. Thumbnail images of each page can also be presented. The company launched its US services in May and has been open in the German market since last November ( The Twirlix US and German home pages serve both as a technology demo to potential clients and as a web search site for consumers.


Autodesk vertical portal designed for designers and engineers. Twirlix recently signed an agreement to provide services for PointA, but I don't believe any new material has yet gone live.


Founded by two students from Carnegie Mellon University, the birthplace of Lycos, Hubat offers automated directory building. The site serves as a demonstration of the company's technology and as a resource for consumers to search the web.

Hubat: A promising automatic directory builder
TBTF, Feb. 7, 2000

Tasty Bits From The Technology Front author Keith Dawson came away impressed with Hubat's automatically generated page descriptions, in this review.

WorldFree SeeAll

Released in April, the SeeAll tool is designed to create directories within an Intranet enviroment. It can also be applied against web-wide documents.


RuleSpace offers the ability to categorize documents for purposes of searching. However, the product really seems positioned as a classification tool for networks. For example, you might use RuleSpace to identify and prevent porn documents from being accessed on corporate networks or to filter out certain types of documents from a directory. The company has been operating since 1996. It lists AOL and Amazon's Alexa as companies using its technology to help categorize information.

Inktomi Directory Engine

Inktomi has offered its "Directory Engine" since July 1999. The product that automatically classifies web sites into categories and makes use of link and caching data to help determine which documents are most popular. The company also recently acquired Ultraseek, which has categorization technology designed for use on Intranets.

The Vortals Are Coming!
The Search Engine Update, April 4, 2000

The earlier article I wrote about vertical search engines, with players mentioned.

Vortal Services Now Offered
The Search Engine Report, July 5, 2000

Short follow-up to the vortal article.


Search Success Rate Increases

The success rate of finding information using major search engines has hit its highest point in two years, according to figures from the last NPD Search and Portal Site Study to have been released. A new version of the study was also just completed, so I expect to update this page again sometime in August. Google and iWon have already issued press releases detailing their successes in this latest survey, and links to these are below.

NPD Search and Portal Site Study, July 6, 2000

Google Release on NPD Study

iWon Release on NPD Study,11873,071200,00.html


FAST Gets Advanced Search, Customize Pages

Google and AltaVista aren't the only ones with new advanced search pages. FAST also added one in July. It helps you perform custom queries, enable a porn filter, see more results on one page and more. The new FAST Search Customize page lets you make some settings permanent.

Fast Search

FAST Search Advanced Page

FAST Search Customize Page


Excite Moves To Your Toolbar

Those using Internet Explorer can now quickly add a new toolbar that gives direct search access to Excite, plus a button to brings up links to Excite's references tools, such as an online dictionary and encyclopedia. The toolbar also gives access to other features of the Excite portal.

Excite Toolbar

List Info

How do I unsubscribe?
+ Follow the instructions at the very end of this email.

How do I subscribe?
The Search Engine Update is only available to paid subscribers of the Search Engine Watch web site. If you are not a subscriber and somehow are receiving a copy of the newsletter, learn how to subscribe at:

How do I see past issues?
Follow the links at:

Is there an HTML version?
Yes, but not via email. View it online at:

How do I change my address?
+ Send a message to

I need human help with my subscription!
+ Send a message to DO NOT send messages regarding list management or site subscription issues to Danny Sullivan. He does not deal with these directly.

I have feedback about an article!
+ I'd love to hear it. Use the form at