With the explosion of visual content orientated apps and social networks such as Instagram and Pinterest, visual search seems to be back on the agenda. One only has to look at the subsequent explosion of image orientated socially enabled shopping sites such as ShopCade, Fancy and Glimpse (from The Find) to see that e-commerce vendors are betting on being able to cash-in on a more visually orientated online shopping experience.
Visual search company, Superfish, have announced a technology update that could improve search on image heavy content sites.
Visual search, done right, is more than simply the image search feature on Google, Bing or Yahoo. Whereas "traditional" image search relies on keyword context and meta data to define the content of an image, visual search aims to use computation to solve the problem of machine recognition - image by image. The core difference is that image search uses typed in keywords as the query mechanic, whereas visual search uses an actual image as the query.
The simplest way to explain the difference in output is that visual search should be able to identify dogs and then sort them by breed, based on one picture of a dog, whereas image search would require the breed of 'dog' to be named as part of the keyword query.
In a nutshell, visual search aims to compute the dissonance between what is similar and what is identical. To complicate this problem, there are three vectors by which something can be similar and identical – namely color, shape and texture. Targeting one of these vectors of innovation, today Superfish announced their "finest technological development" yet, called ShapeRank.
Superfish's ShapeRank examines the geometric relationships between images, to more accurately present images with visually similar content. Superfish say that this additional dimension of geometry improves visual search’s precision and performance "by several orders of magnitude" and allows them to deliver more precise results – which can be filtered by similarity or simply to match identical images.
From their press release, Superfish says:
"Current technologies employ categorization, object or scene recognition, or near-duplicate image matching techniques in order to perform a visual search. While these approaches can be effective in narrow use cases or for detecting exact image matches, they cannot detect and rank visually similar images. ShapeRank allows users to go beyond exact image matching and identify images that are visually similar and sort them by relevance in their web searches. The closer to “near identical” these images are, the higher they rank in search priority, and the faster people will find what they are seeking online."
In a phone conversation, Joe Dew, Head of Product at Superfish, told Search Engine Watch that the typical type of problem Superfish is trying to tackle is how to get a machine to recognize the difference between a flat metal washer (as in seal or nut) and a ring (as in jewelry).
For example, the image based Google search query for a wrist watch below shows identical matches in the main index, yet the visually similar matches, while visually similar are entirely different objects – a coin, a fan and some wheels.
Ironically, as a human being with the innate ability to recognize objects, such a problem is one that we cannot "compute" – we know instantly they are different objects without thinking about it. Yet this underlines how such advancements in machine recognition could eventually contribute to the web of things (much like facial recognition software is used to verify real names on identity platforms like Google+ and Facebook).
Where Superfish's technology demonstrates its real value is in the ability to recognize a caricature of someone or something as being visually similar to the original. This is a problem that cannot be solved simply through better meta tagging.
The advantage that Superfish have is that they are able to crawl and index these visual matches "at Internet scale" without the need for contextual data.
What's more, Superfish see themselves as a platform rather than a destination search engine. In fact there is no destination site – just a browser plugin.
Webmasters and app developers can build Superfish into their own site and even earn revenue for doing so. Such technology could be a boon for the latest breed of image led social networks looking to monetize with search ads.
As these sites become havens for tastemakers it doesn't seem too much of a stretch to imagine Pinterest or ShopCade users looking for similar and identical matches of the latest item they 'pinned'.