comScore Figures Show First Google Decline For Nearly A Year, But What To Believe?

The latest search engine share figures from comScore are now out, and
Google’s nearly year-long continued rises have came to a halt in July 2006,
according to comScore. But how much can you trust any of the figures that
ratings services provide? In this post, I look at the latest comScore stats and
begin a series about how to critically evaluate search share ratings.

I’ve been dealing with search ratings for 10 years now. I’ve seen all types
of things in that time. I’ve watched as ratings services would suddenly pull
their publicly reported information, only to resume it when a competitor came
along. I’ve watched things like AltaVista get classified as a portal rather than
a search engine or important subdomains of a particular search engine not get
counted. I’ve been bemused at how a search engine will heartily endorse figures
from a ratings service when they are positive then tell investors not to trust
the figures when they go against them.

My best overall advice to anyone looking at these figures is:

  • Look for long-term trends. You want to view stats for several
    months in a row, not two isolated months compared to each other. Stats can and
    will plunge from one month to the next for all types of reasons, not the least
    due to a ratings service itself having some counting glitch. Similarly,
    comparing back from one month to the same time the previous year might not
    reflect counting changes that may have happened or been refined over that
    time. I want a trend line — and a long one.
     
  • Look at figures from multiple services. For several months,
    comScore has painted a pretty bleak figure for Yahoo, showing share decreases.
    At least twice this year (January
    & July), Yahoo has
    had to warn analysts not to trust the comScore figures too much (and oh
    the irony
    of Yahoo now having hired the former comScore CEO this month). In contrast,
    NetRatings was showing Yahoo as pretty stable. If I’m going to declare Yahoo
    in trouble (and
    I didn’t
    ), I’m more likely to do that if more than one ratings service is
    reflecting a plunge of some time. If it’s only one of them, then I’m more in
    "watch and see" mode.

Now let’s jump into
today’s figures
out of comScore, which are for share of all searches within the United States:

July 2006

Share

Searches (Millions)

Searches Per Day (Millions)

Google

43.7%

2,753

91.8

Yahoo

28.8%

1,814

60.5

MSN

12.8%

806

26.9

AOL

5.9%

372

12.4

Ask

5.4%

340

11.3

Others

3.4%

214

7.1

NOTE: My figures for searches will vary slightly from comScore’s as
I’m working off the overall rounded 6.3 billion searches figure they provide
multiplied by share percentages. I have to do that to calculate the share of
"others" searches, which comScore does not provide. The comScore press release
has precise figures.

Google’s still shown as well above the rest. But the press release headline
from comScore highlights a Yahoo victory:

Yahoo! Sites Register a Moderate Share Gain for the Second Consecutive
Month

Indeed, some good news for Yahoo out of comScore for once. They have a gain
that seems to come off of Google’s loss.

Let’s look at a big trend chart, then we’ll do a drill down. Pulling the
chart from my now updated
comScore Media
Metrix Search Engine Ratings
page, we get this:

Who Gained & Lost In June & July

You can see how comScore’s shown Yahoo having slight drops, then a bigger
drop in January, then rises in June and July. In particular, if you go back to
May 2006, Yahoo had a 28 percent share. In June, it rose to 28.5 percent. In
July, a further rise to 28.8 percent.

What’s fueling that gain? Maybe
Yahoo Answers
being popular and getting
heavy promotion?
Maybe something else? The answer is, we don’t really know. But we can try to see
who is losing.

Is Google the loser? Not in June. When Yahoo saw a gain in June, so did
Google. Google went up from 44.2 percent in May to 44.7 percent. in June In
contrast, MSN dropped from 13.1 percent to 12.8 percent and Ask dropped from 5.3
percent to 5.1 percent. Those drops fueled some of the gains for both Yahoo and
Google.

AOL is the mystery player. comScore gave no figure for May 2006. Actually,
it did — 6.7
percent. But the following month, it clawed back that figure,
saying:

Due to a definitional change occurring with June 2006 data, trended data
for the Time-Warner Network are not available

Since we don’t have a May 2006 figure, we can’t tell if the June 2006 figure
for AOL — 5.6 percent — was a gain from or loss to Google and Yahoo.

I’ve been through this type of thing before. Usually, it means that
methodology has changed so much that the ratings service doesn’t feel
comfortable comparing figures under a new system to the old one. This note is
unusual in that it’s particular to AOL, rather than the methodology over all.
It’s also disturbing, because it suggests that all those months of AOL figures
previously reported perhaps can’t be trusted.

I’m asking comScore about this and will update. But let’s move ahead to July.
Again, the goal is to figure out who lost to Yahoo’s gain. In July, it really
was Google, mostly. Google dropped a full percentage point, 44.7 percent in June
to 43.7 percent in July. Yahoo got some of that, a 0.3 percent rise to 28.8
percent. AOL got a 0.3 percent rise as well, coming up to 5.9 percent. The Ask
network of sites also came up 0.3 percent, to 5.4 percent. MSN didn’t budge.

Crisis for Google? Way, way too early to be saying stuff like that. As I
said, I want to see several months of trending data from a particular player
before I start issuing panic calls. For all I know, next month comScore will
quietly reissue these figures that shows Google doing better.

Revising Data After The Fact

For instance, let’s go back to May 2006. Did you know that a month after
comScore released figures, they revised those? There was no big press release
about it. It was an asterisk mention as part of the June 2006
figures.

What was different? Let’s compare:

Company

Original

Revised

Difference

Google

44.1%

44.2%

0.1%

Yahoo

27.9%

28.0%

0.1%

MSN

12.9%

13.1%

0.2%

AOL

6.7%

n/a

???

Ask

5.3%

5.3%

0.0%

MySpace

0.7%

n/a

???

Others

2.4%

9.4%

7.0%

Aside from AOL, there was no really big change among the players. But then
again, this revision suddenly changed MSN from a fifth month in a row of either
decreases or no gain to the different story of the first gain in five months.

MySpace The Search Monster?

Also notice how MySpace disappears off the chart, in the revised figures.
What happened? Why did comScore stop reporting figures there? I’m checking on
that as well, but my guess is that close attention to the MySpace figures may
have backfired on comScore.

Let’s go back to April 2006. That’s the first time that out of the blue, we
get search figures for MySpace. comScore
told us:

MySpace.com has been added to the search engine rankings for April 2006,
coming in at 6th place with 43 million search queries performed (0.6 percent
share of the U.S. search market). Will this smaller player eventually be able to
grab a substantial share of the search market due to the site’s remarkable
popularity?

That’s press release bait, OK? That’s a nugget you’re putting out in hopes
that the press will eat it up. And they do, eventually. For example, in

MySpace, The 27.4 Billion Pound Gorilla
over at TechCrunch, the article
notes at the end:

MySpace also has the sixth largest market share among search engines, even
though they aren’t, actually, a search engine.

I’ll get to another more important press reference in a moment, but let me
diverge on the "MySpace As Search Engine" idea. What’s comScore counting as
searches? From the May 2006 release:

qSearch includes Web searches originating from the search engines reported,
other Web-based searches such as News and Image searches and channel searches
conducted on portal sites (e.g., Finance and Movies). qSearch does not include
Yellow Pages or Maps searches.

OK, people on MySpace searching for other MySpacers might be doing what’s
considered a "channel search." But that activity is extremely unlikely to
translate into a search behavior that sends people to an external site or to an
ad. It’s simply a site search within MySpace. If that’s the traffic being
counted, then searches at a place like Microsoft for software support might
warrant it a place on the list (and FYI, these exact types of searches have
indeed gotten Microsoft mistakenly on these types of "popular search engines"
lists in the past).

MySpace does allow people to search the web. Perhaps the searches counted
really are only web searches (and I’m checking on this). If so, I’m still
surprise to see the sudden emergence.

Why? Because of that 2.4 percent of traffic going to "others" in May 2006.
Who are these others? Among them will be players like Infospace or Earthlink.
For June 2006, I have NetRatings figures giving Dogpile (an Infospace property)
a 0.5 percent share. That’s NetRatings, not comScore, and for a different month
as well. Still, it’s really unusual for a new player like MySpace to just leap
out of the blue like this over a more established lower tier player. When I’ve
seen this in the past, it’s simply been because suddenly, a metrics company has
changed a definition and decided to count some new player that they previously
ignored.

Back to MySpace. Last month, BusinessWeek cut loose with a big

article
about who it was seeking a search suitor, since it was such a big
search powerhouse:

MySpace already drives a huge amount of traffic to search engines. It
generates 100 million searches a month. In fact, 5% of all searches on the Web
and 8% of all searches on Google are originated by people who come directly from
MySpace.

Wow, pretty stunning numbers. And figures that frankly, I didn’t believe. As
I wrote in my
review
:

The story also gives new, amazing stats that MySpace generates 5 percent of
all searches on the web. Hmm. Just a month ago, this was said to be 0.6% of all
searches in the US, according to comScore. And 8 percent of all searches on
Google come via MySpace? I’m checking with the BusinessWeek author, because
those stats just don’t make sense.

I did email the reporter, but I never heard back. Perhaps the email went
astray (I mean that in all seriousness — email does get lost or stuck in spam
filters). But five percent of all searches on the web — eight percent of all
Google searches — those are incredible numbers, if true. But they seem so
untrue when you look at the 100 million figures per month stat that BusinessWeek
also gives us.

The Ask network — with a 5 percent share of just US searches — generates
340 million searchers per month according to comScore (see the chart above). If
MySpace really had 5 percent of ALL SEARCHES WEB-WIDE, then it should have far,
far more than 300 million searches per month, much less 100 million.

I don’t know that the BusinessWeek story came off comScore’s press release
bait. Heck, the stats aren’t sourced at all. But when the figures start to fly
and don’t make sense, that’s a good time for a ratings service to stop releasing
them. The BusinessWeek figures weren’t adding up, and perhaps that’s one reason
why comScore decided that MySpace no longer deserved the limelight. After
listing it as a top search engine for April and May, from June onward, it fell
back into the unitemized "Other" category.

I really like BusinessWeek, by the way, but this is the second time recently
that I’ve also seen them do stumbles with figures. Many people couldn’t believe
the
Valley Boys
write-up where BusinessWeek gave Digg a $60 million valuation.
Hey, I couldn’t get past this section:

Digg’s stature changed dramatically that day. It is now the 24th-most popular
Web site in the U.S., nipping at the New York Times’ (No. 19) and easily beating
Fox News (No. 62), according to industry tracker Alexa.com.

Alexa? Alexa! BusinessWeek is going to use Alexa to tell us how popular a
site is? Alexa, the toolbar that so many say is so easily manipulated? Honestly,
I couldn’t’ believe it. In this story, I’m poking at comScore — and later this
week, I’ll poke at Hitwise and NetRatings. But I won’t even waste my time on
looking at Alexa’s data, since I long ago wrote it off as being useful. But if
you don’t believe me, check out Matt Cutts and his
A Word About
Metrics, Part II
post. Why, if Digg can be so popular, so can Matt. He’s
almost as popular as Ask. Shortly, Barry Diller will be looking to acquire him.

MySpace The Google Enabler?

comScore’s not the only one to play the MySpace press release bait game.
Hitwise has been having fun with it, as well. Note what Om Malik
wrote about why
Google felt it had to partner with MySpace:

According to data collected by Hitwise, an Internet traffic tracking service,
nearly 10.8% of Google’s traffic was coming from MySpace.com for the week ending
July 29, 2006. Had Fox gone with Yahoo or Microsoft, it could have been a
serious blow to Google.

Now let’s step back a minute. MySpace has shot up in traffic over time, but
until recently, I never heard any connection suggesting that Google was somehow
gaining because of that popularity. Google was popular because it was Google and
generated its own weather. Now, suddenly, Google might potentially lose 10
percent of its traffic to its site?

That’s especially amazing to digest, considering that Google had no search
partnership with MySpace at the time. The web search box at MySpace was (and
still is for the moment, despite
the
Google-MySpace deal
) powered by Yahoo. If anyone would lose this "massive"
amount of traffic, it should be Yahoo. And since Yahoo clearly didn’t fight
tooth and nail for MySpace, I don’t think that search box was generating much.

So how’s Google getting all this supposed traffic from MySpace. I don’t know
— and Hitwise doesn’t seem to know, either. Let’s go to the Hitwise post on
MySpace traffic:

MySpace and Google: What do the Numbers Say?
:

In January of this year, Google received a little over 4% of their traffic
from MySpace in our U.S. sample. As of the week ending July 29th, that
percentage has grown to almost 11%, making MySpace the #1 upstream site for
Google.

Since over the last year there has been no formal relationship between the
two companies, this high volume of traffic flowing from MySpace to Google would
most likely be the result of users using their Google toolbar or manually
navigating to Google while engaged in their MySpace session. It would seem that
this represents a great opportunity for Google to monetize a portion of that
audience through AdSense listings and increase search traffic through greater
exposure on MySpace.

To be clear, all Hitwise knows is that for the ISP data is monitors, it sees
that a bunch of people who leave MySpace go to Google next. It doesn’t know why
but can only guess that people for some reason decide to go to Google after
MySpace.

If those people change their minds, would that hurt Google? Maybe — but then
again, it’s not 10 percent of ALL Google’s traffic, as Om wrote. It’s 10 percent
of the US audience. Google’s not given out country-by-country breakdowns that
I’ve seen for some time, but in the past, they’d stressed that half their
traffic came from outside the US. So that would reduce the blow. What reduces
the blow even further is the fact that if people are doing this out of habit,
they aren’t likely to break that habit regardless of who ultimately got the
MySpace deal.

Poking At The Other Metrics Services

Enough of MySpace — let’s get back to the Google death plunge. One of the
other things I said to look at was how the other ratings services are viewing
them. For now, I’m going to give you the

highlights
of what Hitwise had to say about July 2006:

As you can see, the top engines combined account for over 94% of all search
volume in the US. Since we last released these numbers in June 2006, Google
(www.google.com) has increased its share from 59.3% to 60.2%, Yahoo! Search
(search.yahoo.com) has increased from 22.0% to 22.5% and MSN Search
(search.msn.com) has decreased slightly from 12.1% to 11.8%.

As you can see, Hitwise is finding that all is good with Google plus
supporting the Yahoo gain that comScore also sees. So is Google in trouble? I’d
say it’s way, way too early to predict anything like that.

Tomorrow, I’ll take a closer look at Hitwise, poke some fingers into the
service and do some trending data. On Wednesday, I’ll spend time with
NetRatings. Then hopefully on Thursday, I’m going to drag out my special
patented three-on-one ratings chart that I haven’t used since the days of
NetRatings, comScore and RelevantKnowledge to help you understand how to tap
into multiple services to see the trends you might want to believe.

Postscript: I sent comScore some
follow-up questions, and here’s what I received back.

Q. General Comments?

We agree with your assessment that a single-month decline does not
constitute a trend. In fact, comScore also observed a similar seasonal decline
for Google during the same period last year. Fewer work days, more vacations,
and reliance on academia could all contribute to Google’s core user group
showing lower online activity and conducting fewer searches during this time
period. Bottom line – Google could be more impacted by seasonality than other
engines.

Q. What happened to AOL figures in May? Am I correct that you changed your
methodology and so earlier figures are no longer comparable to recent ones. If
so, what exactly changed?

In June, there was a dictionary change for AOL that contributed to some of
the M/M decline in Time Warner searches in the US (the dictionary defines how
the Time Warner property breaks out its sites into titles, channels and
sub-channels). This change does not materially alter the year-over-year market
share decline that Time Warner has experienced (down nearly 3 full share
points from May 05 to May 06, the last full month before the dictionary
change).

Q. What happened to MySpace? You started reporting it in April 2006 out of
the blue, then by June, you stopped. Was MySpace traffic being counted before
April 2006, or is that the first month you looked at searches there and counted
them into the searches pie? Why did they stop being broken out in June? And were
they really generating more searches than someone like the Infospace network?
Finally, what searches are being counted — even those internal to MySpace to
find MySpacers?

MySpace is still included in qSearch market data, we’ve simply chosen not
to report it on our monthly market share press releases. We typically issue
market share data to the press for the top 5 search sites, and simply added
MySpace for a couple of months to highlight to the market that we now have the
site under measurement for Web search activity. In fact, comScore data show
MySpace search activity trending very nicely over the last few months.

To clarify the specifics of our MySpace reporting, we provide data for Web
search only; searches done on MySpace content (e.g., user profiles, video) are
not include in the MySpace totals reported in qSearch.

Q. I notice you aren’t counting yellow pages or maps searches. But Google
considers any local search to be part of Google Maps. Does that mean all this
traffic is getting omitted now?

We do not include map-only searches in qSearch or any of our other search
tracking services. There is the initial question of whether a map request
actually constitutes a search, but ultimately, low marketplace demand is the
primary reason for our not reporting map searches. We have, instead, chosen to
concentrate our efforts and resources in the areas of greatest importance to
our major search clients (i.e., all of the major search engines).

However, maps that are accompanied by business listings (as you would find
in Google Maps, if you searched for a business name category) are included in
comScore’s Internet Yellow Pages (IYP) Search Share report. This report
includes the leading IYP sites like SUperpages.com, plus the YP/business list
search activity taking place on the portals. In other words, if the result is
just a map, then comScore does not report that activity in any of its search
reports; if the map includes or is part of Yellow Page listings, that activity
is reported in our IYP Search Share report.

Related reading

hp
clickstream-data
vaccine-impact-1024x730
Search Console Search Analytics