Is AltaVista's Stale Index The Last Blow?

Earlier this year, it emerged that AltaVista had not refreshed any of its country-specific web page indexes for months. Now, it turns out that the main global database is also several months old.

It's a PR black eye that the service didn't need. AltaVista has been losing its audience steadily over the past year, with Google looking to be the big winner. To regain that audience, AltaVista has continued to proclaim for over a year that it is focused on search. Unfortunately, it's failing to deliver on that promise.

Back in March, AltaVista made a lot of noise about a new crawling system that was going to revisit each page in its index on a monthly basis or even sooner, if a page required it. Here we are in November, and that system doesn't seem to be working. Such a long delay doesn't promote confidence that AltaVista will get its act together.

To be fair, some parts of AltaVista's index were fresher than you could find at Google. For example, in mid-October, the listing for the CNN home page was only a few days old. This could easily be spotted by looking at the date of the listing. In contrast, Google's listing for CNN was still dating back to Sept. 13 -- though you would only know this by viewing Google's cached copy of the page.

That's more the exception that the rule, however. The vast majority of AltaVista's content out of date, which the company itself admits. There is one exception. Anyone who stumped up money over the past few weeks got their pages either included or revisited in a current fashion.

It's worth recalling that when AltaVista rolled out paid inclusion, it promised that regular spidering of "free" content would continue, albeit at a slower monthly refresh rate. That pledge has been broken.

Freshness is a key component for any crawler-based search engine, but verifying freshness can be difficult. One standard that all the crawlers should immediately implement would be to list the date that they last visited each page. Then, at a glance, any user could easily tell exactly how fresh -- or how old -- a search engine's listings are.

I tried to follow up with AltaVista on the freshness issue and its future in general last week, but the company couldn't get an interview scheduled. That was probably exacerbated by the fact that one of its long-time press relations people also left last week.

The departure reminded me about a similar situation last year. I took a briefing on how AltaVista was revitalized and set to do better -- only to have the person giving me the pitch add at the end of the interview that they were leaving. Other tech-related individuals I spoke with in the past have also left. The "rats off a sinking ship" image is the one that keeps coming to mind, after each departure.

I'm not normally one for predictions, but I'll ease myself out onto a limb and make one for AltaVista. I believe that Google will acquire AltaVista within a year.

No, there's no source within Google or AltaVista suggesting this to me, so don't take this as anything other than my own musings. However, it makes a lot of sense.

Google wants to play in the enterprise search space, making intranets searchable. This would give the company some insurance against the vagaries of selling adverting aimed at a consumer market. Google's big problem is that many corporations want a software-based search solution. They don't want to allow an company to "remote host" sensitive data.

In contrast, AltaVista does have a software-based tool -- and by many accounts, it's a great one, with plenty of customers. What AltaVista lacks is a decent web-wide index.

A logical step for AltaVista would be to dump its web-wide indexing effort and concentrate on the corporate market. However, I think there's a fear in the company that doing this would hurt the reputation of its enterprise product. The thought would be, "If you can't index the web, how can you handle my intranet?"

The answer is that web-wide indexing and enterprise searching are completely different creatures. While there can be some synergies in doing both, they still have different requirements. Poor web wide indexing does not mean poor enterprise searching, and vice versa.

If Google were to purchase AltaVista, it would pick up a decent software-based enterprise solution that could take it forward into the corporate space. It would also pick up what remaining traffic AltaVista has, along with those search patents that AltaVista likes to crow about. The patents probably don't pose any real threat to Google or other search engines, but if you can pick them up as part of a bargain rate package, why not?

There's a great precedent for this: Inktomi's acquisition of Ultraseek, back in June 2000. Ultraseek was the enterprise search subsidiary of (the former Infoseek). Go decided to concentrate on the consumer market and so wanted to offload Ultraseek. By picking it up, Inktomi gained big inroads into the enterprise space.

Of course, Google could build its own software solution from scratch. The new emphasis on including Microsoft Office documents and other file formats in its regular search results does suggest that Google could be doing this in order to have a more robust enterprise product.

AltaVista serving up out-of-date listings, Oct. 23, 2001

Good coverage about AltaVista's freshness problem.

CMGI restructures $220 million debt from AltaVista deal
Chicago Tribune, Oct. 31, 2001

CMGI still owes Compaq $220 million, apparently for the purchase of AltaVista. A new deal reduces this to $75 million in cash, $7 million in CMGI stock and a 49 percent stake for Compaq in CMGI's B2E Solutions.

Fired AltaVista Employee sues CMGI for $70 million, Oct. 31, 2001

Adding to AltaVista's woes, a former employee is now suing the company and parent CMGI for more than $70 million in damages, over his termination.

AltaVista Listing Enhancements

In a new program aimed at earning revenue from web site owners, AltaVista will now allow listings to be "enhanced" with icons and other customized features. Basic details can be found here, directly from AltaVista. Chris Sherman is also planning a longer look in the SearchDay newsletter, later this week.