Inktomi Increases Size, Introduces Anti-Proximity

Last month Inktomi rolled out a larger, 2 billion web page index, as well as making some internal changes designed to improve the relevancy of its listings.

Previously, Inktomi had divided its index into "Best Of The Web" versus "Rest Of The Web" portions, which helped it serve partners who didn't want to search against its entire index. Instead, all pages now reside in a single index, and all partners are said to dig through the entire index.

Clickthrough data from all partners is now also said to be given to paid inclusion partners. However, not all paid inclusion partners may provide their customers with access to this data. If your partner doesn't offer it, then ask for it -- or find a new partner, if this is important to you.

Inktomi reiterates that paid inclusion does not improve ranking: "It does not do that, and it will not do that," said Vishal Makhijani, vice president and general manager of web search at Inktomi.

As for freshness, the company says it plans to revisit every page in the index at least once during a 2 to 3 week period -- with the average being around every 10 days. Some pages may be revisited far more frequently than this. Obviously, those in paid inclusion get refreshed regularly. Otherwise, pages may be tagged for fast refresh if they are seen as popular or if they've been observed to change often over time.

As part of the new release, the company also has introduced what it calls better "conceptual" searching, though I think a better name would be "anti-proximity."

For example, take a search for "york." On Google, 8 out of the top 10 results all have to do in some way with New York. At, all 10 results have to do with New York. However, with Inktomi, 8 out of the top 10 results all have to do with places that in some way are related to the single word "York," such as the Canadian and English cities of York.

How is Inktomi doing this? It has compiled a dictionary of common terms that all contain the word "york" in them, such as New York. When a search is done on the single word, it will scan the documents retrieved and may downgrade those from the common term list. In other words, it can see terms that are often in close proximity with the word York and use this anti-proximity filter to let pages for just the single word York rise to the top.

Similarly, a search for "mexico" via Inktomi has a more Mexican-feel to the results, since the many links to New Mexico sites that you'll find on Google and have been suppressed. The same is true for "hampshire," where links for New Hampshire sites don't appear.

It's pretty impressive, but those are examples that Inktomi suggested. A search for "england" still lets plenty of "new england" material through. In a search for "california," I got pages not purely about the state generally but instead of various things including the word in it, such as the University of Southern California -- and that was true with Google and AllTheWeb, as well.

In response, Inktomi says that its concept database is growing, so you may see these type of searches work better in the near future. Other examples the company suggests that either work well or will work in the near future include "depressants" not picking up pages about "anti-depressants" or "voter rise" not getting results about "non-voters."

MSN Search: Inktomi Results Via Advanced Page

A search using MSN Search's advanced search page brings back results only from Inktomi, which are called "Web Page Matches."

PositionTech Pure Inktomi

PositionTech is a reseller of paid inclusion for Inktomi and makes available a direct line into the Inktomi search engine. Easier to use than dealing with MSN Search, for those who just want to see what's happening solely within Inktomi.

Inktomi Search Submit

This program is designed for those with small web sites, typically where they have only a few pages they'd like to submit. Pricing is on a flat-fee, annual basis.

Inktomi Index Connect

This program is designed for those with thousands of pages they'd like listed and offers CPC pricing. Major partners are listed here.