Inktomi Debuts Self-Serve Paid Inclusion

Inktomi Debuts Self-Serve Paid Inclusion

By Danny Sullivan
From The Search Engine Report
November 3, 2000

A longer, more detailed version of this article is available to
Search Engine Watch "site subscribers."
Click here to learn more about becoming a site subscriber.

In a landmark move, Inktomi announced a new partnership with Position Technologies this week that allows site owners to pay for guaranteed inclusion in the Inktomi index. While paid submission (or "pay for consideration") systems have operated at human-powered Yahoo and LookSmart for some time, we've never had a major crawler-based service offer something similar. In addition, paid inclusion means that pages will absolutely be guaranteed to be listed, while in paid submission systems, no such assurances are offered.

"The biggest challenge web site owners face is getting found on the web," said Troy Toman, general manager of Inktomi's search solutions division. "Inktomi Search/Submit is the first time we're allowing Web site owners to publish directly into the Inktomi index, helping them get found faster and stay up-to-date in the database."

The program promises to add any URL submitted to the Inktomi index within 2 days and keep revisiting the URL at that frequency for up to a year. In return, site owners pay on a sliding scale: US $20 for the first URL, $10 for the next 2 to 100 URLs and $6 for any URLs over 100. For a 150 page web site, the costs work out like this:

Page 1: $20
Pages 2-100: $990 (99 pages @ $10 each)
Pages 101-150: $300 (50 pages @ $6 each)
Total: $1,310 per year

The pricing is more expensive than I would have expected. A few dollars per URL, sliding down into less than a dollar for large orders, feels fairer. It's one thing to ask web site owners to help share in the cost of indexing the web, but this is a huge jump for even small sites of 50 pages. In addition, the pricing feels high in contrast to fees charged by Yahoo and LookSmart. At those services, $199 essentially purchases you a listing (if approved) for life, or at least several years. Inktomi charges on an annual basis. True, it has to keep revisiting the pages, but as a crawler-based service, that's something it was always supposed to do. However, the pricing could also change.

"The paid submission program was designed for Web site owners to submit the most relevant pages for entry to their site, not all of their content," said Toman. "This pricing structure is an introductory offer with the new service and we will be adjusting it over the coming months to determine the most equitable price point."

Inktomi says that the new paid inclusion system won't alter the normal crawling that it does. Pages from across the web will continue to be listed for free, especially those that Inktomi deems to be popular or which seem to satisfy queries.

Free Add URL pages that feed into Inktomi, such as the one offered by HotBot, are eventually to be phased out. Inktomi finds that the vast majority of submissions sent via these pages are spam. Other crawler-based services also report the same problem. A small number of people submit an incredibly large number of poor-quality URLs through these pages. By eliminating them and moving to a paid inclusion model, Inktomi (and others) could arguably do a better job of indexing the web.

Such a shutdown won't occur until Inktomi can establish a means for non-commercial sites to submit to the search engine. "We want to make sure we have adequate non-fee submission mechanisms for them," said Troy Toman, general manager of Inktomi's search solutions division. "Our goal is not to charge anybody and everybody. We want submissions from non-profits and non-commercial sites on the web, and we'll wait until those are covered."

Finally, while pages are guaranteed to be listed, there's no guarantee that they will rank well for particular searches. Inktomi will continue to apply its regular relevancy algorithms to paid inclusion pages, just as it would to those it lists for free.

Inktomi Paid Inclusion Program

Describes pricing and leads you to an order form

Inktomi Paid Inclusion FAQ

Q&A directly from Inktomi.

Position Pro

Operated by Position Technologies, Inktomi's new paid inclusion partner, Position Pro has tools especially suited to help those large web sites optimize their pages and submit to crawler-based search engines. However, even small site owners may want to take a look.


Another Inktomi paid inclusion partner, MediaDNA's eLuminator spiders web sites and creates optimized pages for search engines. The eLuminator product is especially designed for sites that have content locked behind password protected areas, as it can make that content visible to search engines without giving it away for free to unregistered users.

Pay For Inclusion Advances
The Search Engine Report, Nov. 3, 2000

Inktomi isn't the only major search service with a paid inclusion program. This article looks at programs run by Ask Jeeves and LookSmart, plus touches on some issues that paid inclusion raises.

A longer, more detailed version of this article is available to
Search Engine Watch "site subscribers."
Click here to learn more about becoming a site subscriber.