Searching Inside Of Images

From The Search Engine Report
Dec. 4, 2000

The Holy Grail of image searching is to actually "see" what images are about, rather than understanding them based only on the words appearing around them. A new generation of multimedia search tools aims to change this. In particular, Ereo, Cobion and are all companies with new products touted to truly see inside images and recognize patterns efficiently.

To understand the importance of this, consider a traditional image search tool, such as what AltaVista runs. If you use its image search and look for "eagles," you'll get pictures of eagles or graphics with the word "eagles" in them. However, AltaVista didn't retrieve these pictures because it could recognize what an eagle looks like or because it could read the text inside an image.

Instead, AltaVista (and most crawler-based image search engines) remain mostly blind to what the actual image shows. Instead, they rely on the words that appear around the image or in the file name of the image to understand its content. So, pictures with the word "eagles" in the file name or pictures that appear on web pages that make use of the word "eagles" in the HTML text give AltaVista the clues it needs to display results.

Since this is guesswork, it's not surprising that an image search may miss what you are looking for. In contrast, the new crop of products aims to use both surrounding text and an analysis of the image file itself to determine what is relevant. Interestingly, none of these new companies is trying to launch an image search service on their own. Instead, they plan to partner with those who may have image search requirements -- either large portals and search engines, vertical portals or even those with visual intranet content that needs to be indexed.

In addition to Ereo, Cobion and, I've also noted a few other interesting companies doing recent work in the multimedia space worth watching.


Ereo just landed a deal to power Excite's web-wide multimedia search service, plus Ereo says that relevant image and multimedia finds may be integrated into Excite's regular search results. The joint service is set to go live in the first quarter of next year. In addition to providing web-wide image search for portals, Ereo is also applying its technology to those with media collections that need an indexing solution, as with its just announced deal with Minden Pictures.

The Start-up: A story of dot-com survival
Denver Post, July 30, 2000

Long article discussing the launch of Ereo and touching on the technological issues involved in extracting information from images to improve search.


Cobion is another new company that has launched image scanning services. These are being primarily targeted at those with brand protection needs. Coupled with Cobion's web crawling, it's designed to help ferret out unauthorized use of logos and other trademarked symbols. Cobion's technology is also being positioned as a filtering tool. For example, it could help detect pornographic images on pages that are pretending to be about innocent topics. Finally, Cobion is also available as a regular image search engine. In contrast to Ereo and, Cobion is also attempting to extract any words it finds within images, rather than primarily sticking to pattern recognition. introduced its image search products last month at Internet World in New York. As with Cobion, is pushing its technology as a filtering tool. And, similar to Ereo, it suggests that by letting people upload an image of what they want, or clicking on a image (such as a lamp or a couch), more matching products can be found by looking for similar images. Finally, offers its technology for those needing an image search tool.


Launched in 1999, like Cobion, ImageLock is aimed to recognize unauthorized use of logos and copyrighted materials. A client's content is "fingerprinted," then ImageLock's spider watches content across the web for unauthorized use of the fingerprinted materials.

Originally launched as a standalone image search engine, Ditto made a position change earlier this year to distribute its results to other partners. It currently provides image search results to NBCi and Go2Net's MetaCrawler and Dogpile meta search services. You can still search for images directly at Ditto's own site, but it now it is more a showcase for its technology than aimed at consumers.


Taalee has been primarily focusing on those with audio-visual search needs. It will index multimedia content and understand what it is about by the associated text it finds on the page and from any meta data that may be provided along with the content. When users search, it also goes beyond the actual search words to include related concepts or meanings that may not have been specified. This semantic expansion of search queries is also something Taalee will apply to those needing more traditional text-based search.

Launched earlier this year, has developed what it claims to be the largest index of MP3 and streaming media files. Its first portal partners are supposed to be announced later this month.

Start-up Boasts Of 99.9%-Accurate Video, Music Search
Newsbytes, June 12, 2000

More technical details about

Virage Internet Video Guide

Rolled out earlier this year by video search specialist Virage, this product is available to portals and others with web-wide video search needs. Use the "Got Video" icon to let them know if you have video content that should be included.

Virage Showcase

Various places to see Virage's video search technology in action.

He Said What? Clinton Video Search Available
The Search Engine Report, October 5, 1998

Past story about Virage's video indexing service.

AltaVista Image Search

Web-wide image search service from AltaVista.