The Search Engine Update, July 2, 2001, Number 104


About The Update

The Search Engine Update is a twice-monthly update of search engine news. It is available only to Search Engine Watch members. Please note that long URLs may break into two lines in some mail readers. Cut and paste, should this occur.


In This Issue

+ Site News
+ Search Engine Strategies Comes To San Francisco
+ The Evolution Of Paid Inclusion
-- (full story online, link provided)
+ AltaVista Opens Paid Inclusion
+ Changes Coming To FAST
+ LookListings Go Live
+ Make Room For Teoma
-- (full story online, link provided)
+ Excite Makes Changes
+ iWon Brings Back Inktomi
+ Google Adds Picture Search, New Languages
-- (full story online, link provided)
+ Google: No IPO Imminent
-- (full story online, link provided)
+ Microsoft Smart Tags Abandoned
-- (full story online, link provided)
+ Search Resources
+ SearchDay Articles
+ Search Engine Articles
+ List Info (Subscribing/Unsubscribing)


Hello Everyone--

I hope all my Canadian readers had a happy Canada Day yesterday, and a great Fourth of July to all my fellow American readers. In site news:

+ The How MSN Search Works page has been fully updated, with one key exception. All Asian-language web page information is now coming from Inktomi, rather than AltaVista, through a new deal that was just announced.

+ Both the Directory Submission Chart and the Search Engine Submission Chart have been updated for all major services.

+ The How Yahoo Works page has been updated with information previously reported in past newsletters on mandatory listing fees and new promotion of the Yahoo Shopping area.

+ The More About Link Analysis page has been entirely rewritten. Previously, it has some summary tips from past articles I've written. Now, it has been greatly expanded to explain how link analysis works and how to build appropriate links to your web site in a way that should please link analysis systems.

+ The How Google Works page has been completely updated with material previously covered in past newsletters. However, even if you've kept up with your reading, you should review this page. Some additional new material has been posted to it.

In particular, Google is planning a harder look in the future at those who try to spam by using invisible text, excessive word repetition and other tactics that it traditionally has not been concerned with. I've never recommended that my readers do any of these things, so if you've followed that advice, you'll be fine.

In addition, Google says that in the near future, it may consider banning pages from those who run position checking reports. This would be a substantial change from how the service has operated in the past.

I don't have more details on this policy change yet nor any alternatives Google may have in mind for those who need position reports. I got this information late on Friday, after Google reviewed the How Google Works page for suggestions and corrections. This was one of the changes that came back, and there was no time to do a follow up before this latest newsletter had to go out.

If you must run position reports, review the article I've previously written about minimizing your risk -- there's a link from the appropriate place on the Google page. I also will follow up with Google about this issue.

Links to all the pages mentioned can be found via the Members-Only What's New page, below:

Members-Only What's New

Password Finder

In the public site, the Jupiter MMXI European Search Engine Ratings page has been updated with listings of the most popular search engines and portals for various European countries. A link to the page can be found via the What's New page, below. On What's New, I also expect to post another link to new figures for US search engines and portals, from Jupiter Media Metrix.

Search Engine Watch What's New


SES Comes To San Francisco

Coming to San Francisco on August 16th and 17th, the Search Engine Strategies conference features a day devoted to search engines and promotion issues, with a second day featuring panels on promotion, sessions for web developers and two tracks with panels designed to help Internet searchers.

I'll be speaking at the conference, along with other search engine marketing and research experts. There will also be speakers from the search engines themselves, including, AltaVista, FAST Search, Google, GoTo, Inktomi, LookSmart, MSN Search and Netscape/The Open Directory.

Exhibiting companies include, EasyAsk, Fast Search & Transfer, Inceptor, Firespout, LexiQuest, LingoMotors, NuTech Solutions, Position Technologies, SiteLab, Sprinks, Web Ignite, WebSeed and WebGenius.

Those interested in sponsoring or exhibiting should contact Frank Fazio Jr,, for more information. Attendees can find the agenda or sign-up for the conference via the URL below. Do so by July 18th, in order to save up to US $200 on registration.

Search Engine Strategies: San Francisco 2001


The Evolution Of Paid Inclusion

Paid inclusion has always been a tricky concept to explain, but understanding it is important to both webmasters and searchers, as recent changes have suddenly made paid inclusion commonplace with search engines. For webmasters, do these new programs mean that you won't get listed, unless you pay? And for searchers, do these programs present a threat to the quality of your results? See the article below for more about some new developments in paid inclusion.

The Evolution Of Paid Inclusion
The Search Engine Report, July 2, 2001


AltaVista Opens Paid Inclusion

After months of talking about it, AltaVista finally unveiled a paid inclusion program last week. Similar to the program Inktomi began offering last September, this guarantees that submitted pages will be listed within AltaVista's web page index in a prompt manner and revisited on a regular basis.

AltaVista has hinted that it was going to offer paid inclusion since late last year and even accidentally posted pages promoting its "bulk inclusion" program when it introduced its "ransom note" Add URL puzzle system back in March. The company has finally followed through on this by launching its new "Express Inclusion" program.

The program allows you to submit up to 500 URLs, which will be visited on a weekly basis. This means that a brand new page submitted to AltaVista through the program should show up in about week, or changes to existing pages should be reflected in about a week.

Notice I said changes will show up in "about a week," not weekly. While the spider will revisit weekly, it could take longer than this for the changes to be reflected in the index. This is because the index might be refreshed before any new changes can be reported by the spider. Hard to understand? Consider this example:

On Friday, the paid inclusion spider visits your page. Then on Monday, you make a change. On Thursday, AltaVista refreshes its index, unaware of your change, because it happened after the paid inclusion spider made its visit. Now its Friday again, and the spider returns to your site on its weekly schedule and finds the changes. However, the changes don't appear until AltaVista again updates its index on the following Thursday.

In all, 10 days have elapsed from when your page was changed to when the changes appeared in AltaVista, despite the fact that the paid inclusion spider did visit you each week, as promised.

AltaVista says that it plans to continue updating its index each week, so in the worst case, you'd be looking at up to two weeks before the changes appear, should the spidering and index refresh cycles align badly. On the flip side, the cycles could also align so that you might find your changes appear in AltaVista in less than a week.

AltaVista's pricing is more expensive than Inktomi's paid inclusion program -- much more -- starting at US $78 per year for the first URL and dropping in price as follows:

URL 1: $78
URLs 2-10: $48.00
URLs 11-100: $38.00
URLs 101-500: $24.00

AltaVista actually sells its service on a six month basis, so you can get started with paid inclusion for half the prices shown on the list above. I've changed the figures to yearly pricing, so you can more easily compare to Inktomi's paid inclusion prices, where service is sold by the year.

Here's Inktomi's current charges, as sold through Position Technologies:

URL 1: $30
URLs 2-20: $15
URLs 21-1000: $12

As you can see, AltaVista's prices are easily double what Inktomi charges. To include 25 pages for a year with Inktomi would be $375, while at AltaVista, the same number of pages for the same time period would be $1,080 -- almost three times what Inktomi charges. In addition, you cannot swap URLs, as you can with Inktomi. However, there is no problem with completely changing the page content as often as you like, AltaVista says

Is AltaVista's program worth this premium? To AltaVista's credit, its crawler-based listings are the main information presented on its results page. In contrast, Inktomi's listings tend to be used by portal partners as backup to other results. For instance, you will get Inktomi information at MSN Search, but usually only for more obscure or unusual results. This means it is possible your AltaVista listings might get seen by more people. However, it's also true that Inktomi's results are distributed to a wide range of portals.

Still, more than $1,000 for 25 pages and no guarantee that they will bring in any traffic? Rather than that gamble, it might make more sense to put the money into AltaVista's separate paid listing program or into GoTo listings, which are carried on AltaVista's pages.

Remember, you can continue to use the free Add URL page to add new pages to AltaVista. Now called "Basic Submit," the downside is that you have to enter a ransom note message for each batch of five URLs that you add. In addition, the former weekly insertion schedule has now been downgraded to 4 to 6 weeks, to make the paid inclusion program more attractive. Of course, that's not out of line with the delay Inktomi has when using free Add URL pages to submit to it. The biggest drawback is simply the uncertainty. There's no guarantee that your pages will get picked up.

Before you panic, remember that AltaVista is also continuing to add pages for free, through its regular crawling. It also still promises to revisit pages in its index each month. That means you may already have plenty of pages listed, and they should stay there despite the addition of the paid inclusion program. Nor will those non-paid pages be downgraded in relevancy.

"We're not going to penalize in relevance the people who are adding through free Add URL," said Chris Kermoian, director of search and web marketing services for AltaVista. "We'll also still go through our standard crawling process."

At the prices being charged, the AltaVista program only seems worthwhile if you have new pages you absolutely must get listed right away or you have pages that you absolutely must get refreshed on a weekly basis. Neither situation is probably true for the small and medium-sized businesses that AltaVista says this program is aimed at, probably aren't in this category.

As for big businesses, another program for them that probably will promise better pricing is being considered.

"This is the first of the programs that we will be rolling out," Kermoian said. "We'll have more programs for larger sites later on."

Is it anything goes for submitting with the paid program? Nope. AltaVista says it won't accept pages that do things such as use invisible or tiny text, those that attempt to mislead, mirror pages, doorway-style pages with no real content or those that only link or redirect to other pages.

By the way, should you enroll in the program and then decide not to renew, your URL is not supposed to be dropped, AltaVista says. It simply won't be revisited as often as the program allows.

infoSpider is the company that is handling the paid inclusion system for AltaVista. infoSpider is also a sister company to submission services of WorldSubmit, ProBoost, and ProBoostGold. However, you needn't use any of these other services to take advantage of AltaVista's program.

How about non-US editions of AltaVista. How does the paid inclusion system work for them? I didn't get an answer back in time for the newsletter, so I'll let you know what I hear in the future. My assumption is that paid inclusion only inserts your pages into the global index that all AltaVista sites use for worldwide results.

Speaking of non-US editions, AltaVista is taking over direct control of AltaVista Canada, which has been run under license by Telus since its launch several years ago. The site will continue to operate and index pages like AltaVista's other non-US editions.

Back to US-based, there were two other developments of note last month.

First, AltaVista has dropped the "Directory" search option that was previously available. This makes it much harder for people to encounter the LookSmart listings that AltaVista licenses.

"The directory link won't be coming back," said AltaVista spokesperson Kristi Kaspar. "We've found historically with directories that people aren't certain exactly what to do with them."

Instead, AltaVista will continue to make directory listings available to those who browse links from its home page and in vertical search products that it may develop.

The promotion of submitting to LookSmart has also been dropped from AltaVista's Add URL page, with AltaVista's new paid inclusion program being promoted, instead.

Finally, the paid links area at AltaVista has continued to be a bouncing ball. It was briefly changed to being called "Sponsored Listings" in June, and I would have loved to see AltaVista stick with this clearer definition. However, it was changed back to being called "Featured Listings" later in the month.

"Featured listings is the title that not only seems to get a high clickthrough rate for us but also doesn't seem to bother the users," Kaspar said.

Meanwhile, the number of listings that appear there has continued to fluctuate. Sometimes there have been two links, sometimes three links, and sometimes there are no links -- even for queries where there are GoTo bids. I have no doubt we'll continue to see this fluctuate, as AltaVista continues to experiment.


AltaVista Express Inclusion

AltaVista Free Add URL

IMPORTANT! If you fail to include the ? at the end of the URL above, you will only get a blank page.

How AltaVista Works

I'll be adding the latest changes to this page later in July, but the explanations on how pages are ranked, how to submit via free Add URL, how non-US editions work are all still valid.

Password Finder

AltaVista reclaims Canadian portal
The Globe and Mail, June 20, 2001,C/20010620/

More about the end of the AltaVista-Telus relationship in Canada. Sorry about the long URL.


Changes Coming To FAST

FAST Search is planning to release a new look and functionality to its site later this week. The revamped service is designed to automatically present information from appropriate indexes as required. For instance, if you do a picture-oriented search, FAST hopes to present relevant pictures along with matching web page listings. The new changes are supposed to go live on July 6, and I'll be doing a longer review of the new service in a following newsletter.

FAST Search


LookListings Go Live

The LookListings program I wrote about last issue for LookSmart has now gone live. If you are an existing Subsites customer, you will be rolled into the new program and apparently told how to take out contracts on search terms of interest. LookSmart says new customers can also get started with the program now, though nothing is mentioned about it on the LookSmart site. My advice would be to use the Subsites contact page, below.

LookSmart Readies Paid Placement Program
The Search Engine Update, June 18, 2001

Further background about how the new LookListings program works.

LookSmart Subsites Program


Make Room For Teoma

After the loss of Go earlier this year and the expected departure of NBCi, you might have thought it was all over in the search engine game. However, just as consolidation seemed inevitable, new player Teoma has stepped up with an impressive debut of its new search service. Opened to the public last month, Teoma leverages link structures from across the web to provide not only relevant results but to allow present different views of information automatically. See the article below for more about this new service.

Make Room For Teoma
The Search Engine Update, July 2, 2001

Password Finder


Excite Makes Changes

Excite has finished making changes to its search interface, mostly minor alterations that that service hopes will improve usability. Here's what you will find that's different:

+ More emphasis on the "Zoom In" search refinement tool.

+ Results are now numbered, to help those who wish to remember their place when returning to the search results page.

+ Search page numbers are now at the bottom of the screen, allowing you to jump directly to the second, third or tenth page of results, should you desire.

+ Site listings now put more of an emphasis on title and summary.

+ Page clustering is now automatically performed, so that only one top page per web site is displayed.

+ The site ranking algorithm has been tweaked, to take the proximity of search words into greater account.

Excite is also working on tightening up its spam detection, in particular to watch for mirror pages or those who run multiple web sites in hopes of dominating the top results. This has been a big problem for Excite over the past year or so. In addition to automatic detection, Excite says it also depends on customer reports of spam.

"It does help us a lot for people to report when they find spam," said Lynne Mariani Pogue, who oversees search for Excite@Home. She cited one extreme example where someone reviewed the top 750 URLs that appeared for a particular search, calling out specific examples of spam.

"That's definitely above and beyond the call of duty, but it was helpful to us," Pogue said.

Should you wish to report spam to Excite, note the search term, the URL in question, its position and a succinct explanation of why you think it is spam. Send this using the Excite feedback form, below.

Excite also refreshed its index in the middle of June, and it hopes to stay on a four to six week refresh schedule going forward. Directory listings from LookSmart are now also updated each week.

By the way, Excite is not reading frames, as I said it was to do back in March. That was originally supposed to be the plan, but working on making this possible hasn't finished.

In other news, Excite@Home's WebCrawler service is to transform this month into a more pure search service. Chris Sherman will be following up on the change in Search Engine Watch's SearchDay newsletter. You can sign-up for it via the URL listed below.

All this work on the search front might seem a bit odd, given that Excite@Home has sent out past signals that it might dump its portal properties to concentrate on broadband access. Despite those signals, search apparently remains important.

"It's a confusing time, but there's definitely a strong commitment to support the search product," said Pogue. "There will be more clarity at the earnings call in July. [Search” is one of our company's main assets, and we are investing in it and moving forward."


Excite Feedback Form



Good News, Bad News for Excite@Home
WebSideStory, June 26, 2001

Despite Excite@Home's ISP operations rising, usage of its search engine has plunged, according to search referral traffic measured by StatMarket.

Excite UK MD: We are not closing down
NetImperative, June 25, 2001

In June, Excite@Home said it would close all but its Italian and UK operations in Europe. This articles covers rumors that the UK site would also be closed, which Excite@Home denies.


iWon Brings Back Inktomi

Last month, I wrote about how iWon had switched to delivering pure paid placement results from GoTo, rather than the Inktomi-dominated listings it had historically used. iWon said this was a temporary move, and to its credit, Inktomi listings were returned in mid-June. They now appear midway down the results page, after the "Featured Listings" section, which are paid listings from


iWon, Go Paid Placement
The Search Engine Update, June 4, 2001

Past article about the iWon change.


Google Adds Picture Search, New Languages

Google has added a new picture search facility to its service, along with the ability to search in 48 languages, such as Bengali, Welsh, Telugu, Elmer Fudd and Pig Latin. Search Engine Watch's associate editor Chris Sherman has reviews of both new features available online, via the URLs below:

Speaking in Tongues at Google
SearchDay, June 25, 2001

Google's gone multilingual, allowing you to search in 48 languages. Soon, you'll be able to search in any language, and automatically translate results from one language to another.

Google Polishes its Image
SearchDay, June 26, 2001

Google has taken the wraps off its new specialized image search engine, allowing you to search and browse more than 150 million digital images.


Google: No IPO Imminent

Following an story in the UK's Sunday Business newspaper, it was widely reported by other media outlets that Google was expecting to go public by the end of the year. The company strongly denies that this is the case. More details can be found via the URL below:

Google: No IPO Imminent
The Search Engine Report, July 2, 2001


Microsoft Smart Tags Abandoned

Microsoft has given up on its plan to add "Smart Tags" to web pages viewed by those using Internet Explorer and Windows XP, following bad publicity over the system. More about the proposed system can be found via the URL below:

Microsoft Smart Tags Abandoned
The Search Engine Report, July 2, 2001

Search Engine Resources


WebLens offers access to over 4,500 Internet research tools and resources. It's the companion site for writer Pam Blackstone's weekly technology column for the Victoria Times-Colonist newspaper.



Diggit is a new image search engine that lets you search for images using other images and keywords. Image matching is based on color, shape, and texture. You can even paint your queries right in your browser window using the Diggit FX (ActiveX) or Diggit Graffiti (Java) applets.


Linking And Crawling Issues

This article covers some of the legal aspects of linking and crawling that may have an impact on both web site developers and search engines. Written by attorney Ivan Hoffman who specializes in online content and intellectual property issues.



The WordTracker service now offers the ability to research what people are searching for via the popular meta search services of MetaCrawler and Dogpile. It provides access to query logs stretching back for two months, which amounts to 350 million queries or 40 million unique search terms. The database is also kept constantly updated, with new data added each week.

SearchDay Articles

Here are some recent articles that may be of interest from Search Engine Watch's daily SearchDay newsletter:

+ Backwash: A Web Concierge - Backwash is a hybrid portal, community and search site that organizes the Internet by personality, not by subject matter like most traditional search engines and directories.

+ 12 Cool Sites and Tools for Searchers - A roundup of interesting search-related sites, services and tools, with this issue heavy on browser companions and add-ons that must be downloaded prior to use.

+ What Search Engines See Isn't Always What You Get - Cloaking is a technique used by some webmasters to deliver one page to a search engine for indexing, while serving an entirely different page to everyone else.

+ Web Archaeology: Yahoo Relics - Do you remember what Yahoo looked like in 1994? Digging into the web reveals several fascinating relics of the service that later morphed into the portal we're familiar with today.

You can find all of these articles and more via the archives page, below. You can also sign-up for SearchDay on that page to get more articles like these during the workweek, along with search engine headlines from across the web.

SearchDay Archives

Search Engine Articles

Cyber Law Journal: Invisible Publishing Sparks a Lawsuit
New York Times, June 29, 2001

We've had metajacking and pagejacking, and now here's a new one -- a case involving allegations of copyjacking. Copyjacking? A new term I've made up for taking HTML body copy from someone else's page and posting it to your own in hopes of attaining higher search engine rankings. It involves the horoscope site alleging that InternetHoroscopes, a site, used text from EasyScopes to boost its rankings. By the way, the "tricks" used by EasyScope as mentioned at the end of the article aren't aimed at search engines but instead at potential copyjackers, the company says. They include spelling errors to make it easier for EasyScopes to track down unauthorized use of its material.


Optimizing for International Search Engines
ClickZ, June 27, 2001

A look at major non-US search engines and tips on submitting.


Intelliseek Gains $1.4 Million From CIA-Backed Firm
Washington Post, June 25, 2001

The US Central Intelligence Agency thinks that invisible web search firm Intelliseek's technology might come in handy in tracking down information, so its venture capital firm has made a large investment in the company.


Google ` go-go
Salon, June 21, 2001

Interview with Google director of research, on how the service operates.


Selling Yourself Short on Traffic
ClickZ, June 18, 2001

Not all traffic is the same, and you want traffic that converts. A guide to understanding conversion, which may help you understand that more visitors from search engines is not necessarily better than fewer, targeted visitors.


Napster, Gnutella have competition
ZD Net Australia, June 15, 2001,2000020814,20232639,00.htm

A look at Napster alternatives, focusing on Audiogalaxy.


Search the Web Like a Map Web Search Guide, June 18, 2001

Review of tools that let you see web search information in visual or graphical form.

List Info

How do I unsubscribe?
+ Follow the instructions at the very end of this email.

How do I subscribe?
The Search Engine Update is only available to paid members of the Search Engine Watch web site. If you are not a member and somehow are receiving a copy of the newsletter, learn how to become a member at:

How do I see past issues?
Follow the links at:

Is there an HTML version?
Yes, but not via email. View it online at:

How do I change my address?
+ Send a message to

I need human help with my subscription!
+ Send a message to DO NOT send messages regarding list management or membership issues to Danny Sullivan. He does not deal with these directly.

I have feedback about an article!
+ I'd love to hear it. Use the form at