SEO News
Search

Searching for Images by Similarity

author-default
by , Comments

Searching for pictures needn't take a thousand words. An experimental search engine helps you find images based on colors, textures or characteristics similar to a reference image.

Generally speaking, images pose more of a challenge for search engine developers than words on web pages. Why? Words on a web page are called text strings, made up of individual characters, and characters are stored in computer memory as unique numbers. It's comparatively easy for a search engine to recognize and manipulate text strings, because they form recognizable patterns that can be compared to one another.

By contrast, images on the web are made up of hundreds or thousands of discrete points (pixels), each with its own color and brightness, which together make up a gestalt whole that our brain perceives as a picture. Like alphabetic characters, each point in an image is also represented by a number, but these numbers don't form the same kind of recognizable patterns that text strings do, because each picture is unique.

The image search offered by Google, Yahoo and other search engines therefore relies heavily on other clues to identify the subject of an image—things like filename, image ALT tags that describe the image, text immediately above or below an image (which is often a caption of sorts), and links pointing to images. With enough of these clues, a search engine can make a decent guess about the content of the image.

Photo sharing sites like Flickr take this a step farther, allowing users to tag images with descriptive labels. This helps improve search results, but it's far from a perfect solution. If someone uses the tag "orange," does it describe a sunrise or a fruit? Or a pumpkin or an orange house or...

Another approach to image search involves analyzing certain characteristics of an image, attempting to "see" the image in the same way humans do. And taking this approach, you can also ask a search engine to, in essence, show you images that have similar characteristics to the one you're currently viewing.

Tiltomo is an experimental search engine that works in just this fashion, allowing you to find similar images in two ways. You can search for images that have similar color and texture characteristics, or you can look for images with similar "themes," which adds an analysis of subject matter to the color/texture mix.

Tiltomo searches a sample collection of images from Flickr, both from the general images group and those included in the "catchy colors" group. You start your search by entering a tag into a search box; results display thirty largish thumbnails from Flickr.

Here are some examples using color and texture:

And others that try to group images by "themes:"

Links beneath each image in these search results allow you to explore other sets of images, either by characteristics or theme.

As you see, search results aren't perfect, but that's not the point. Tiltomo's developers want to provide a way to easily explore similar images, and perhaps stumble upon images you might not otherwise see. If you like images, it's a great way to have a fun romp through Flickr.

Another experimental image search service is found at Russia's State Hermitage Museum. You can search for images with certain color characteristics or layout characteristics using photoshop-like controls.

Results are 12 thumbnail images based on the criteria you selected. Clicking on a result displays a larger image of the artwork with text describing it—for example, van Gogh's Boats at Saintes-Maries. Want more images like this? Use the menu at the bottom of the page to view similar artwork in the collection by the same artist, of the same style, from the same country/region, using the same technique, of the same genre or with a similar visual layout.

Why aren't the major search engines using these types of image search techniques? They are, but mostly behind the scenes. As image search becomes more popular, however, I expect we'll gradually see more and more of these types of query by image content search tools emerge into the mainstream.

Want to know more about content-based image retrieval? This Wikipedia entry is a good place to start.

Search Headlines

NOTE: Article links often change. In case of a bad link, use the publication's search facility, which most have, and search for the headline.

Google Search Captures Lion's Share Of Visitors...
Media Post May 22 2006 10:02PM GMT
For the New Search Workforce, Offer A Career...
Media Post May 22 2006 10:01PM GMT
Stats Say Google Losing Something...
Internet News May 22 2006 9:46PM GMT
New service scans search results for viruses, spyware...
tom's hardware guide May 22 2006 9:09PM GMT
Photos: Inside Google Australia...
CNET News.com May 22 2006 6:12PM GMT
Bill Gates Outlines His Answer To Google...
Information Week May 22 2006 4:03PM GMT
Internet searches: Librarians do it better...
Reuters May 22 2006 4:03PM GMT
Google retains lead in U.S. search market...
ZDNet May 22 2006 4:02PM GMT
Snap mixes ads in with search results...
Australian IT May 22 2006 4:01PM GMT
Google Responds to Orkut Child Porn Allegations...
Publish.com May 22 2006 2:30PM GMT
Google Class Action Emails Going Out...
Search Engine Guide May 22 2006 2:24PM GMT
Firms will embrace the web or die, claims Google...
PC Magazine UK May 22 2006 1:07PM GMT
Beyond Google and Yahoo: Advanced Search...
LLRX.com May 22 2006 11:02AM GMT
Was Google's Summer of Code a Boon or Bust?...
Internet News May 22 2006 10:27AM GMT
Analyzing Search Behavior for SEO...
ClickZ Today May 22 2006 3:28AM GMT


ClickZ Live San Francisco This Year's Premier Digital Marketing Event is #CZLSF
ClickZ Live San Francisco (Aug 11-14) will bring together the industry's leading online marketing practitioners to deliver 4 days of educational sessions and training workshops. From Data-Driven Marketing to Social, Mobile, Display, Search and Email, the comprehensive agenda will help you maximize your marketing efforts and ROI. Register today!

Recommend this story

comments powered by Disqus