Inktomi: One Database, But Different Results
From The Search Engine Report
August 4, 1998
A number of people have noticed that Yahoo's Inktomi-powered web results are different from those that appear when the same search is performed at Inktomi-powered HotBot. Likewise, some minor differences also appear at other services where Inktomi provides results.
HotBot was the first Inktomi-powered service, and so that's the benchmark most people are using to compare the Inktomi results on other services. It pulls its results from the Inktomi database in California.
In contrast, Yahoo is pulling its results from a new database that Inktomi has been building in Virginia, according to Inktomi marketing director Kevin Brown.
These results appear after matches from Yahoo's own listings, or if a search is performed and the "Web Pages" link at the top of the results page is then selected.
The Virginia database is roughly half the size of the California database, which is why Yahoo's web searches often yield a smaller number of matches than at HotBot. It was decided to use the Virginia database for Yahoo temporarily, in order to satisfy the huge traffic demands of the popular service without impacting Inktomi's other partners, Brown said.
The database will be expanded over the coming months until it is a mirror of the California index. Then Inktomi will spread queries from all its partners across both databases, in order to balance the load, Brown said.
Even when this happens, there are still likely to be differences between results at the various Inktomi-powered services. This is because Inktomi gives its partners the ability to make various tweaks and changes to the way results are ranked, a crucial part of marketing the same database to competing companies, Brown said.
A good example is at GoTo.com. Its Inktomi results, which come below paid listings, look identical to HotBot's with one important exception: only one page per web site makes it into the top results at GoTo.
This is similar to the clustering feature at Infoseek, where matching pages in the top results are grouped together and only the best page is presented. It allows more web sites to have a shot at the top 10 -- or the top 40, in GoTo's case.
At Snap, you may notice that multiword queries bring back more results than at HotBot. This is because Snap performs a broader "match any word" search, while HotBot performs a "match all words" search. Changing HotBot's settings to "match any" brings the count up to Snap's level, though top results usually remain the same.
As with Yahoo, results from Inktomi appear after Snap's own listings. However, you can also query the Inktomi database directly from Snap's advanced search page.
Overall, it's best to remember that each service that Inktomi powers is distinct, even though they use the same core database. Brown says that over time, the services may grow even more distinct, as different features are enabled. Inktomi also plans to segment its database, which will produce differences.
For example, a portion of the database might contain pages only from high-quality web sites. One partner may decide to use just this segment, which would be cheaper than querying the entire database. Another partner may decide they want to be comprehensive and so tap into the entire database each time.
As a reminder to webmasters, the best way to ensure you are listed within Inktomi's database is to submit to HotBot. Just keep in mind that this is not a guarantee you'll appear in Yahoo or elsewhere, or that your rank at HotBot will be the same as elsewhere.
Snap Advanced Search
This Year's Premier Digital Marketing Event is #CZLSF
ClickZ Live San Francisco (Aug 11-14) will bring together the industry's leading online marketing practitioners to deliver 4 days of educational sessions and training workshops. From Data-Driven Marketing to Social, Mobile, Display, Search and Email, the comprehensive agenda will help you maximize your marketing efforts and ROI. Register today!