Mobile Search: Time for a Hearing and Vision Checkup

During BIA/Kelsey’s November Interactive Local Media conference in Los Angeles, Google Product Manager Surojit Chatterjee asserted that the mobile device has eyes, ears and, in some cases, touch.

His point: Why not utilize these features? It’s a philosophy that has shown through in much of Google’s mobile product development. Goggles, for example, takes a picture with your phone which is matched to image databases to return information on the object in question.

Though Goggles has ways to go, it’s indicative of the mobile innovation we’re starting to see beyond the Googleplex walls. This includes voice search, barcode scans, augmented reality, and other search “inputs” more intuitive than the typing that ruled the PC environment.

Augmenting your senses

Say It

We’ll see this concept increasingly apply to product innovations and monetization strategies in the coming months. Mobile pay-per-call, for example, is being developed by Google and others to take advantage of the obvious fact that the connected device is, after all, a phone.

Augmenting your sensesVoice search is likewise accelerating on the mobile device. AT&T’s YPMobile app last month integrated voice search for category and business name queries — a format to which directory assistance has long familiarized consumers.

Meanwhile, voice search start-up Siri specializes in natural language search (i.e., “Where is the closest liquor store?”). It was acquired by Apple last month for a rumored $200 million, further signaling Apple’s desire to deliver content and search on its flagship device.

Microsoft earlier this week launched a voice-powered mobile navigation app powered by Tellme and available on Windows Mobile (6.x) devices. This was Redmond’s answer to Google’s navigation app, launched with the Droid and available on Android (1.6.x) devices.

Google meanwhile continues developing speech-to-text processing on iPhone apps and embedded deeper in the Android OS. With Google’s scale — and speech processing’s natural tendency to get smarter over time — these products will only improve.

And in addition to local search, Google is thinking globally with ambitions to tackle languages such as Mandarin Chinese. Due to the complexity of typed character sets in such languages (not to mention the size of the Chinese market), this is an opportune area.

Scan It

Augmenting your sensesVisual search” is likewise developing, tied to improving smartphone capabilities. In addition to Goggles mentioned above, bar code scanning software allows you to scan 2D bar codes in traditional media using the phone’s camera, to receive more info or promotions.

UPC product code scanners are also gaining popularity due to the format’s ubiquity in retail environments. The opportunity is to push information, promotions, and advertising to consumers whose proximity to the point of purchase indicates higher levels of buying intent.

This information includes specs, reviews, and even point of sale inventory for nearby stores that might have better prices. Monetization hasn’t been fully developed, but this inferred buying intent is a foundation for premium ad inventory of various formats.

Treating scans as a form of search, for example, a bid marketplace could develop where advertisers bid on scans the same way they currently bid on search keywords. Sponsored results could come in the form of banners, text ads, or actionable coupons.

So far the leaders in the space include Scanbuy, ShopSavvy (on Android), and Red Laser (on iPhone), which this week announced 2 million app downloads, 50 million scans, and nearly 1 million active users.

Pimp My Reality

Augmenting your sensesAugmented reality (AR) is another form of visual search getting lots of attention. For those unfamiliar, it displays the physical world around you (through the phone’s viewfinder) with graphical overlays representing different data sets like public transit or merchant details.

This can mean holding your phone up to a busy street and seeing floating icons with storefront information or promotions. Though this has sex appeal, the reality of augmented reality is tied to myriad technical and logistical challenges such as data aggregation.

But the underlying technology is making strides. Out ahead is Amsterdam-based Layar, with iPhone and Android AR apps. Yelp’s Monacle (iPhone only) is another, which overlays geotagged listings and reviews on the viewfinder screen to show the closest merchants.

All of the above technologies will continue to be applied to new areas as they mature on the smart phone and beyond. This will eventually include other mobile hardware, wearable technologies and even biometric interfaces in the long term.

But back to the foreseeable future, innovation will continue to be fueled by one giant feedback loop between users, ODMs, and app developers. Through this, mobile software will more substantially “grow into” the inherent capabilities of the hardware. Finally.

Related reading