A new study from Slingshot SEO re-examines the value of the top position in organic ranking. Their estimates for the number one position are roughly half of figures provided by previous studies. This article examines potential reasons why.
The Study and Methodology
Slingshot SEO's organic CTR study, titled Mission ImposSERPble: Establishing Google Click-Through Rates, took place over the course of six months and examines 324 total keywords. Each of those keywords yielded a site that was stable in position for at least 30 days, and the stable site had to be in a position ranging from 1 to 10 in the organic results. Slingshot looked at a minimum of 30 keywords for each position.
Most important, the volume of actual user visits tracked was substantial. As specified by the study itself, "The data is based on more than 170,000 actual user visits, making it one of the largest studies of its kind."
Slingshot SEO used a lineup of tools to get this info, including Authority Labs (used to track stable positions), Google Analytics, the Google AdWords Keyword Tool, and Slingshot's client database. Some data elements were harder to track down than others. To get the 324 keywords actually used, Slingshot had to examine data from more than 10,000 keywords.
The click-through rate for a website in the top spot is 18.2 percent, with second position at 10.05 percent and third at 7.22 percent. It's still clear that the number one position is the ideal place to be (number one got more clicks than two and three combined, and over a third of all clicks on the home page). While the gap between first, second, and third is smaller than in previous studies, this holds with the common logic of research that's been done in the past.
However, if you've been keeping track of those SERP organic CTR studies, you know that the exact numbers are a good deal different than what we've seen previously. (See "Top Google Result Gets 36.4% of Clicks [Study]" for a great, comprehensive article on those studies from April.) Slingshot SEO themselves devoted a section to the differences between their study, the Optify study done in December, and the Enquiro study in 2007.
What we're looking at is a conclusion that position one is only half as valuable as the Optify study claimed it was. But why?
The Reasons Behind the Dissonance
What could be impacting the results in such a substantial way? To find out, Search Engine Watch contacted Evan Fishkin, Slingshot's head of research and development, and Brian Goffman, Optify's CEO.
Was It Seasonality? Maybe
Slingshot SEO cites one of the key factors in the 18.2 percent variance from the Optify study: "Optify’s insightful and thorough study was conducted during the holiday season of December 2010," reads the study. "There are significant changes in Google’s rankings during the holiday season that many believe have a significant impact on user behavior as well as the inherent change in user intent."
However, Goffman didn't believe this explained it.
"There's definitely seasonality around trending terms," he said. But, he continued, "I don't see why the percent clicks on the top results versus other results would be different based on time of year." He added that Optify would need to conduct studies at different times of the year – which they intend to do.
Was It Because of Long-Tail Terms? Maybe
One theory is that Slingshot's study of long-tail keywords – which showed a substantially lower CTR than exact matches, at just 5.8 percent for position one – was tilting the study after those terms were averaged in. Fishkin confirmed, however, that no such averaging was happening. The 18.2 percent was for exact matches only.
Of interest, Goffman felt that the long-tail results from the Slingshot study didn't match with Optify's understanding or his own experience. Goffman stated that the more long-tailed a term was, the more concentrated the clicks would be on the top results.
Was It Because of UI Changes? Probably Not
While the idea of UI changes would have gone a long way to explaining the differences between the studies, it quickly became clear that there was no such convenient excuse. Google Instant had deployed two months prior to Optify's study, and the only other UI changes since had been the blended pages – which showed local results from Google Places with standard results – and that black bar at the top.
Could the blended pages have made such a substantial impact? Almost definitely not. Since both Slingshot and Optify were using their client data for the study, and neither is working with small local businesses who use Google Places, the approximate impact of Places is nil.
Was It Because of User Perceptions? Possibly
Could user interactions with search have changed because the quality of the SERP was different in a post-Panda world? Neither Fishkin nor Goffman believed Panda has changed how users interact with the SERP, though both acknkowledged there might have been some user perception changes.
Fishkin feels that user behavior changes gradually over time as people get used to how search works, and that user adaptation to things like Google Instant may have had an impact. Goffman, meanwhile, said that Google's algorithms were among "the multitude of ranking variables and the infinite ways in which people search" that "make it very hard to predict an accurate CTR curve and also compare them."
Was It Because of the Datasets? Possibly
Since Optify and Slingshot were using very different sets of data, it's possible they were getting industry-specific CTRs that didn't translate well.
"Who's their target audience? Are they B to B or B to C? All of those are going to be factors," Goffman said.
However, Slingshot and Optify are similar companies in many regards. While Slingshot and Optify certainly were looking at data from different types of businesses, there shouldn't be a massive shift in the percentages.
Was It Because of the Math? Probably
After digging around all of the reasons listed above, I cracked back down on the numbers and methodology of both studies. What I found was that the data used – which is based on the Google AdWords Keyword Tool predictions for search volume – has the possibility for gaping inaccuracies.
As the Slingshot study puts it, "One of the confounding variables is that the data from Google Adwords is likely to be overstated due to rounding, making our click-through-rates slightly understated."
Because both groups were looking at monthly search volume, and Optify was running their study in the holiday season, it's likely that Optify is overstating their results while Slingshot is understanding theirs. As I went back through the percentages, I found the Slingshot study shows only 52 percent of searchers click on an item on the home page, compared to nearly 90 percent from the Optify study. Optify's total on the second and third page then brings the total CTR on organic results to such a high point that there's no room left for non-clicks or paid clicks.
What that leaves us with is the important conclusion: Rather than comparing the figures, we should take a look at the ratios to understand the importance of the various positions.
How It Boils Down: Some Final Thoughts
There are still some fairly substantial differences between the studies, but what we see is a dramatic advantage in reaching the first two to three positions on the Google SERP. According to Slingshot, 35 percent of clicks on the home page went to the first position, while Optify showed 40 percent for the same placement. Position two and three received 19 and 14 percent for Slingshot and 14 and 11 percent for Optify, respectively.
Slingshot's study concludes that the second and third positions especially are getting a more attention these days – with any of the reasons mentioned above factoring into the reasons why. In the end, though, the conclusion is the same: Get to the first page, and as close to the top as possible!
In their study, Slingshot encourages users to note the differences between the Optify and Slingshot study and take them into account, rather than seeing the two studies as battling each other. Goffman mirrored these sentiments, saying that this was "a great debate to have," and that he looks forward to seeing how both Optify's and Slingshot's continued studies help the industry gain a better understanding of organic CTR.
Both companies are definitely continuing on that path. Slingshot's six-month study will become an ongoing examination of the organic CTR trends, while Optify will be exploring new angles of SERP activity (currently, Optify is doing "research on the differences between SEO and PPC in conversion rate and quality"). As an SEO or webmaster, both groups are certainly worth paying attention to.
Meet Your Favorite Search Engine Watch Contributors
Many of SEW's leading expert contributors will be at ClickZ Live, the new online and digital marketing event kicking off in New York (March 31-April 3). Hear from the likes of: Thom Craver, Josh Braaten, Lisa Barone, Simon Heseltine, Josh McCoy, Lisa Raehsler, Greg Jarboe, Dan Cristo, Joseph Kerschbaum, John Gagnon, Eric Enge and more!