Yahoo's Shashi Seth Calls comScore Data Misleading, Blames Google Instant

In a blog post yesterday, Shashi Seth, Sr. VP, Yahoo Search & Marketplaces called the latest comScore results, which showed Google’s search market share increase and Yahoo’s dip, misleading. He said the increase was likely due to people who…well, type too slowly using Google Instant.

Here’s Seth’s spin:

One example to consider is Google Instant. This user experience does not fit the mold for traditional search measurement – when a user starts typing, and pauses for a few seconds without finishing their thought, should that be counted as a search? For example, I want to find Sprinkles Cupcakes. When I start typing “S-P,” Google Instant offers up “Sprint,” unrelated to what I was looking for. However, this would count as a search in comScore’s measurements.

Going further and typing “S-P-R-I,” I still see a suggestion for “Sprint,” not quite what I had in mind. In the mean time, this counts as another search. If I typed slower, more searches would be counted.

Looking at comScore’s report, it appears to me that a majority of Google’s query growth in September (a month in which Google Instant was live for 20 days) came from precisely these kinds of interactions. I bet even the folks at Google are mystified by this kind of accounting.

I’d argue that with this type of counting, third party measurement companies are potentially misleading the consumers of their data, despite encouragements to take a more careful look at how to most accurately evolve their collection, verification, and analysis of the numbers. It is time to think about ways to more accurately measure searches in the long term, so that all of the players in the industry can focus on driving innovation that will ultimately benefit consumers and advertisers alike.

Seth also wrote that people are searching Yahoo “more frequently” and user engagement is up since Yahoo launched its new search features last week.

A post on comScore’s blog explained how they defined searches with Google Instant:

…we have developed a priority scoring system that allows us to identify search results with explicit user action and interstitial results with a sufficiently long pause to suggest some level of implicit engagement. User actions that qualify an explicit query include those where a user hits “enter,” clicks on an algorithmic or sponsored result, clicks one of the various refinement links (such as past 24 hours, etc.) or clicks on a vertical search tab to execute the query in a different channel (such as News or Image Search). Any query with such explicit action is counted in Explicit Core Search. Query result pages without explicit user action, but with a pause of at least 3 seconds, are considered as indicating ‘implicit’ engagement and will count towards Total Core Search.

After the launch of Instant in September, Seth made use of the Yahoo Search Blog to note that Yahoo did Instant first (back in 2005, even though Yahoo never rolled it out, instead launching it on AllTheWeb). He also has hinted that Instant infringes on many of Yahoo’s patents.