Delving into the SearchMonkey

Yahoo announced today the general public availability of their SearchMonkey program. This is a program that has been in beta testing with limited partners. It allows the partner to provide Yahoo with structured data that provides advanced information about a web page. This information is then used by Yahoo to influence the presentation of organic search listing results for that page.

This is a very powerful concept in that a modified search listing can surely influence click through rates. Imagine your search listing with an image and several related links built in. Let's look at a quick example:

SearchMonkey

You can see additional examples in my interview with Yahoo Chief Scientist Andrew Tomkins. The interview was published this past Monday and focuses on SearchMonkey.

The basic process for creating SearchMonkey applications is straightforward. SearchMonkey supports multiple formats, including microformats, RDFa, eRDF, XML feeds, and APIs such as OpenSearch, so publishers have many options for exposing the data.

In addition, developers can build sophisticated applications into the search results. An example of this is the notion of an InfoBar. With an InfoBar, you can actually put an active control in your search listing result. When users click on the control, you mini application will run and can present additional data that displays inline right on the Yahoo search results page.

Here is what it looks like:

InfoBar

The InfoBar provides a very powerful mechanism for managing complex interactions with users right on the Yahoo search results screen. This should have significant value from a branding and click through perspective.

Here is a summary of the development process:

  1. Application Type – Decide what type of app you want to build (Enhanced Result or Infobar) and enter basic info such as application name, description, and icon.
  2. Trigger URLs – Decide the URL patterns that will trigger your app.
  3. Data Services – Data Services are the structured data on which SearchMonkey apps are based. They can be created using data available in the Yahoo! Search index (via data feeds or page mark-up such as microformats or RDF) or by using APIs or page extraction.
  4. Appearance - Use PHP to configure how structured data should appear in the application.

Commentary

Note step 2, the one in which your application gets activated. A critical part of the program will be determining when and where you would like your enhanced result to show up.

One key element of the program is that creating an enhanced result, or an InfoBar, does not mean that all users will be exposed to them. Users need to enable the enhanced listings on a publisher by publisher basis. In addition, users can change their minds later and remove your SearchMonkey application from their results.

I spoke to Amit Kumar, Director of Product Management at Yahoo, this past Tuesday, and he indicated that in the future that select SearchMonkey applications may get exposed to all comers. Applications that are adopted by lots of users, and not remove by many at all would be more likely to make this leap to general availability. This however, is not a certainty.

Amit also told me that Yahoo is going to setup a Gallery of such applications for users. This will be a place where the user can go to select an application and enable it. It will be interesting to see how much exposure the Gallery gets. This will play a critical role in the rate of adoption of these types of results. The publisher can, of course, promote their own application, and try to drive people to sign up for it.

Another thing that Amit emphasized during our conversation was that the effort level for developers to engage with SearchMonkey is quite low. The platform makes it really easy for them to engage. This could play a critical role in broadening adoption.

One thing I learned in my interview with Andrew, and also from his presentation at SES New York, is that building SearchMonkey applications will not help you improve your rankings. The program is not intended to be used for that purpose.

Personally, I'd like to see a stronger move towards exposing some of the applications to all users. This maybe a difficult thing to implement at some level, and it makes it far more susceptible to spam. But it would certainly accelerate the exposure of these types of applications to the general public.

The early action (in terms of users) will likely be driven by early adopters. Then we will need to see how widely it penetrates the market, and how aggressively Yahoo pushes it forward.

That said, this is exciting stuff. I have long been a believer that search engines should get more information from the publishers, in a structured format. Yahoo has taken a big step in that direction with this program.