How to Kill Self-Reported Data


Self-reported data is collected when a website/company asks a user any iteration of “How did you hear about us?”

While self-reported data is leaps and bounds better than no-reported data, companies must always strive to create a much more accurate picture of the channels and campaigns that are creating engagement and revenue.

The Problem With Self-Reported Data

Self-reported data is frequently subjected to a spectrum of limitations that can ultimately result in a skewed data set that is ignored by the savvy and consumed by the mislead.

The following is likely to erode the reliability of even the best-intentioned data surveys.

1. People Bend Reality

Mary McCarthy coined the famous expression “We are the hero of our own story.” People tend to view themselves through the lens of the protagonist, and are inclined to spit out whatever fits this predisposed view, regardless of the reality they may or may not be distorting in the process.

How can you really trust a user to tell you if they found your product in a less-sexy medium (like that 1 a.m. infomercial), especially when people so frequently carry an “advertising doesn’t work on me” worldview?

2. People Have Terrible Memories

The uncertainty of memory contributes heavily to the fallibility of self-reported data. Many users don’t remember the exact touch points that lead them to a sale, nor have the capacity to communicate them given the technical limitations of a dropdown selection or radio button form.

Moreover, awareness can occur on the periphery of consciousness. Take an extreme example: can you honestly remember the first time you became familiar with Coca-Cola? Here’s some light reading on the 7 Sins of Memory.

3. Users Don’t Care

Everyone who has ever made a purchase online has probably experienced the following in some form: you’ve just finished filling out a purchase form and clicked buy, only to see that an error has been thrown because you didn’t check a “how did you hear about us?” box. In that brief moment, you hastily check a random box just to complete the transaction. The sooner we can all admit to doing this, the better.

4. Most Self-Reported Data Suffers From Ordering Bias

If we can agree that users don’t always report honestly, it’s safe to assume that the order your department appears in the survey might significantly impact the results. I’d much rather have my department at the top of a dropdown list than the bottom.

5. The Survey Might be Asking the Wrong Question

Is “how did you hear about us?” the most important question? What creates awareness and what convinces a user to buy might be two very different channels.

For example, you might hear about a restaurant from a friend, but only make the reservation after seeing a string of positive reviews on Yelp. Both interactions play very different, yet equally important roles in the creation of a sale.

6. Non-Web Self-Reported Data is an Even More Junky Process

Many websites are designed with the end goal of converting traffic into phone calls to further nurture a lead. At the end of a call, sales specialists are often prodded to ask the question, “how did you hear about us?” to help aggregate lead source data.

In my experience, this data is among the most difficult to cull. That’s because a caller will often answer this question with “the Internet.” Even if the awkward follow-ups are asked, it’s virtually impossible to get any actionable data without an uncomfortable cross examination of your new customer or prospect.

Minimizing the Need for Self-Reported Data

As much as possible, I try to ensure that my clients are unchained from self-reported data. Here are the three best ways to get a clear and more reliable picture of the channels that are driving business.

1. Campaign Parameters

URL parameters are the ideal way to measure responses from paid campaigns, social media, email, and forum outreach. You can quickly attach source, medium, content, and campaign data to URLs. Be sure to create company-wide standards for URL creation if more than one person is responsible for constructing them.

2. Custom URLs

Using custom URLs for traditional media buys can be a great way of determining lead source, especially if they are coupled with a special offer that can be redeemed to further encourage usage of the full URL.

3. Call Tracking Analytics

Phone call analytics are a game changer for any business that relies on offline lead nurturing to close a sale. Phone call analytics platforms charge a monthly fee to utilize multiple 800 or local numbers, which are then associated directly to a campaign to measure the impact.

It isn’t uncommon to utilize dozens of toll free numbers in PPC campaigns. These platforms integrate directly with larger bid management platforms like Kenshoo/Marin or can be exported to excel and combined with other reporting bulk sheets like AdWords or Bing.

Push Beyond Last Click Attribution Models

Beyond inaccuracies, most self-reported data surveys are a one-touch point attribution model. Users tasked with digesting complex value propositions or deals that require an investment of more than $100 typically utilize multiple touch points such as forums, reviews, and key influencer blogs.

Tying these touch points directly to phases in the buy cycle can go a long way in creating an appropriate attribution model that helps dictate a logical flow of resources in the future.

Related reading

guide to google analytics terms
SEO travel mistakes to avoid in 2019