Searching for pictures needn’t take a thousand words. An experimental search engine helps you find images based on colors, textures or characteristics similar to a reference image.
Generally speaking, images pose more of a challenge for search engine developers than words on web pages. Why? Words on a web page are called text strings, made up of individual characters, and characters are stored in computer memory as unique numbers. It’s comparatively easy for a search engine to recognize and manipulate text strings, because they form recognizable patterns that can be compared to one another.
By contrast, images on the web are made up of hundreds or thousands of discrete points (pixels), each with its own color and brightness, which together make up a gestalt whole that our brain perceives as a picture. Like alphabetic characters, each point in an image is also represented by a number, but these numbers don’t form the same kind of recognizable patterns that text strings do, because each picture is unique.
The image search offered by Google, Yahoo and other search engines therefore relies heavily on other clues to identify the subject of an image—things like filename, image ALT tags that describe the image, text immediately above or below an image (which is often a caption of sorts), and links pointing to images. With enough of these clues, a search engine can make a decent guess about the content of the image.
Photo sharing sites like Flickr take this a step farther, allowing users to tag images with descriptive labels. This helps improve search results, but it’s far from a perfect solution. If someone uses the tag “orange,” does it describe a sunrise or a fruit? Or a pumpkin or an orange house or…
Another approach to image search involves analyzing certain characteristics of an image, attempting to “see” the image in the same way humans do. And taking this approach, you can also ask a search engine to, in essence, show you images that have similar characteristics to the one you’re currently viewing.
Tiltomo is an experimental search engine that works in just this fashion, allowing you to find similar images in two ways. You can search for images that have similar color and texture characteristics, or you can look for images with similar “themes,” which adds an analysis of subject matter to the color/texture mix.
Tiltomo searches a sample collection of images from Flickr, both from the general images group and those included in the “catchy colors” group. You start your search by entering a tag into a search box; results display thirty largish thumbnails from Flickr.
Here are some examples using color and texture:
And others that try to group images by “themes:”
Links beneath each image in these search results allow you to explore other sets of images, either by characteristics or theme.
As you see, search results aren’t perfect, but that’s not the point. Tiltomo’s developers want to provide a way to easily explore similar images, and perhaps stumble upon images you might not otherwise see. If you like images, it’s a great way to have a fun romp through Flickr.
Another experimental image search service is found at Russia’s State Hermitage Museum. You can search for images with certain color characteristics or layout characteristics using photoshop-like controls.
Results are 12 thumbnail images based on the criteria you selected. Clicking on a result displays a larger image of the artwork with text describing it—for example, van Gogh’s Boats at Saintes-Maries. Want more images like this? Use the menu at the bottom of the page to view similar artwork in the collection by the same artist, of the same style, from the same country/region, using the same technique, of the same genre or with a similar visual layout.
Why aren’t the major search engines using these types of image search techniques? They are, but mostly behind the scenes. As image search becomes more popular, however, I expect we’ll gradually see more and more of these types of query by image content search tools emerge into the mainstream.
Want to know more about content-based image retrieval? This Wikipedia entry is a good place to start.
NOTE: Article links often change. In case of a bad link, use the publication’s search facility, which most have, and search for the headline.