Lies, Damned Lies, and Statistics

09 Jan 09 | Gregg Stewart

When Mark Twain penned, “There are three kinds of lies: lies, damned lies, and statistics,” he must have foreseen online campaign analytics.

Over the past year, I’ve been in numerous meetings with advertisers, agencies, and online media vendors. I’m often amazed at the confusion surrounding how local search campaign analytics are reported, and sometimes twisted, to create a desired story.

While most of today’s column will center on analytics as it relates to local search campaigns, you can use the same concepts for online, and many offline, lead-generating capabilities.

The Basics: Where to Start

In analytics, all metrics can be traced to two standard units of measure: counts and ratios.

Counts are whole numbers, and as the name connotes, are a total number of some item. Ratios are generally the relation between two counts, or the proportion in relation to the whole. In layman’s terms: two counts divided by each other.

Don’t worry, that’s the entire statistics lesson for today. Just remember these basic concepts when embarking on your journey into statistical analysis.

In local search, we’ll receive a large number of counts and ratios from media vendors, or observe them via our own Web site analytics package:

Impressions
Visits
Unique visitors
Clicks
Click-through rate (CTR)
Average ad position
Cost per click (CPC)

(Instead of defining each term individually, you can find a great resource on the Internet Advertising Bureau’s glossary page to help you with each term’s meaning.)

Calculating ROI

One of the simplest methods for determining ad effectiveness is to measure campaign ROI. In an earlier column, I provided some simple formulas associated with this key performance indicator:

Metrics Table 1

One of the most common errors I see in calculating ROI is when a marketer or their agency simply divides ad cost by total sales and then declares it ROI. Actually, that formula represents the return on ad spend (ROAS). While it remains an important basic measure, it doesn’t take into account the marketer’s cost of delivering the goods or services sold.

One of the most important parts of the above ROI calculation is how the “Number of Leads” count is captured. Your leads count should include all of the clicks from the specific campaign, as well as the phone traffic measured through call tracking, and walk-in purchases (if you measure these).

For marketers who desire to move to a higher level of precision in calculating ROI, consider breaking out the lead counts into their component response channels (e.g., clicks, calls, and walk-ins) and apply conversion rates based on their respective response channels.

Keep in mind that conversion rates by response channel may vary dramatically by category. That being said, as a starting point we generally see a 3:1 ratio of call to click sales conversion.

Now let’s look at the same ROI calculation from above, breaking down leads and applying a differential conversion rate between clicks and calls.

Metrics Table 2

The more precision you use in measuring and correctly attributing conversion rates to individual lead sources and response channels, the better you’ll understand the true value of your local search campaigns.

A word of caution on a common issue: some marketers inflate a campaign’s ROI to misapply a lifetime value of customer (LTV) in order to make a negative ROI campaign go positive. Now I’m not saying LTV should never be used, but I suggest you go with concrete facts where possible.

For example, to correctly assign LTV in an ROI calculation, the marketer needs to understand what percentage of the customers responding to a campaign are new customers to their place of business. If they’re repeat buyers, for example, using LTV will double count some of the sales, creating an inflated ROI.

Hopefully, this information helps clear up how to use base-level counts and ratios to help measure the effectiveness and optimize your local search initiatives. Please feel free to contact me for any additional clarification. Now get counting.