Data is dangerous.
More specifically, the amount of data to which we now have access to is dangerous. Beyond the obvious analysis paralysis challenges and other big data pitfalls, the abundance of data creates a world in which we are led to believe that there are hidden silver bullets that we can find to magically solve all of our problems. Like a hidden short cut in a Super Mario video game (sorry for the ancient cultural reference). If we only found that short key, we could advance to the final stage without having to go through the rigors of completing all the stages. Our challenge, therefore, is not to do better or work harder through the challenges of marketing, but to spend the time analyzing the data so we can find that elusive variable that will make everything fall into place.
Let me help you with it right now – as Ben Horowitz brilliantly puts in his book "The Hard Thing About Hard Things" - there are no silver bullets in the data, only a lot of lead bullets. No hidden pathways, no short keys, no magic wands, just hard work and a succession of smart, data-driven decisions and flawless execution.
But, what’s the right approach to learning from the overflowing amount of available data to analyze? Do we need a data scientist to go through the mountains of data to provide us with the answers?
I strongly oppose this approach. Data scientists are the luxury of big brands and corporate behemoths and are a danger for us marketers. It’s not that I oppose the concept of data analysis; the exact opposite is true. Heck, I work for a data company. I just believe that it’s our duty as marketers to posses the basic skill set for analyzing our own data so we can come up with the actionable insights and the recommended action plan. We need to know the data well enough to identify all the lead bullets needed to slay our next monstrous challenge.
Control Your Fate; Own the Measurement Plan
Whether you’re the person who reports or the recipient of those reports, to truly control your fate you need to own the measurement and reporting plan. It all starts with setting the right expectations and clearly defining goals and key performance indicators (KPIs). The sole purpose of data analysis and reporting is to enable smarter decisions that will lead to improvements. If the reports you put together or receive don’t allow you to do that, something is broken.
From my experience, these weak points can usually be traced back to lack of clear expectations. The following steps outline how to avoid these problems and set a measurement plan that will serve all the stakeholders involved.
5 Steps to Owning Your Measurement Plan
Step 1: Define goals and KPIs, and make sure they support the objective.
This step should be a no-brainer, but I see too many cases where it gets neglected or mistreated. The key to success is to agree on a goal and identify all the KPIs you will measure and use as levers to control the outcome.
Potential Pitfalls: There are two potential pitfalls in this step: (1) goal and KPIs don’t support the overall objective, and (2) the definitions of the metrics aren’t clear and agreed upon.
To avoid the first pitfall, go through the following process. Ask yourself if achieving all your KPIs will it ensure achieving the goal. Then, figure out if achieving all of your goals will it mean that you achieved the objective. If the answer to each of these questions is anything but "Absolutely yes," then you need to revisit your goals and KPIs. The worst thing that could happen to you is that you achieve all your goals but no one is happy because the goals weren’t set up well to start.
To handle this second pitfall, I suggest creating a definition document in which you clearly define each metric and measurement, and what it means, so there are no arguments or unspoken misunderstandings about terminology or definitions. For example, marketing qualified lead could mean one thing to you and something completely different to your counterpart.
Step 2: Make sure the baseline is clear so you can report on improvements. If there’s no baseline, set one.
The sole purpose of data analysis and reporting is to enable smarter decisions that lead to improvements. You will not be able to understand the improvements without establishing a baseline. Avoid disagreements on the extent of the improvements by recording the baseline every single time so you can reference it.
Potential Pitfall: You’ll run into trouble if you do not establish a baseline to start. In cases where it’s a new initiative or a new business, there might not be a baseline. In these cases you can either use zero as your baseline or define the establishment of a baseline as the first goal in your plan. In those cases, you need to make sure you capture all the right KPIs and then re-evaluate the goals and KPIs to make sure they are still valid.
Step 3: Ensure access to data and verify data accuracy and hygiene.
I can’t stress enough how important it is to make sure you have the access to the right data. In a recent survey we conducted with social marketers, we found that one of the biggest challenges in reporting is access to data. Even today, with so many data sources and public information, access to the right data can still be challenging.
Potential Pitfall: Your data could be inaccurate or misconstrued. Access is not enough, since most data sources will provide only raw data that needs to go through an aggregation and standardization process. These processes, if mishandled, can distort the data, which can result in catastrophic decision-making. Think of the representation of the data like levers on a faucet. If you put the cold lever on the hot water pipe, when you try to cool down the water, you will get burned.
Step 4: Agree on the reporting template. Verify full consensus on all report elements.
This is possibly the most important step in this process and the one that tends to get the ignored most often. The concept of this step is all about expectations setting. Whether you’re the one reporting or the recipient of the reports, to avoid wasting time reading useless reports that will lead you nowhere, spend time defining the reporting template and creating consensus on what is expected to be communicated in the reports.
Decide in advance what level of data you would like to see/report and what the importance of that data is in the report. It’s easy to get distracted by data points that have no importance, or marginal impact on your business decisions, and miss the important insight. For example, as a chief marketing officer (CMO), do you really care what your organic SERP rank for a set of keywords is or is it more important to know that you drove X revenue from organic search? Is the audience reach on Facebook really important or whether or not your message resonates with your target audience? Sometimes the most "sexy" metrics are the least important ones.
Potential Pitfall: Beware of defining the metrics for a report but neglecting the format. Defining the type of metrics you want to measure is only half of the way to crafting a useful report. You must also be clear about the template in which you want to see it and what format that works for you. For example, if you want to measure the total number of leads generated, is it important to just see the total number per week, or would you like to see a trend graph and a week-over-week rate to make sure you’re showing upward progress? Or perhaps you just want to see the total for the month compared to plan? Try to wireframe the report and clearly define how you would like it to be presented and delivered. I recommend reading Avinash Kaushik’s post "Digital Dashboards: Strategic & Tactical: Best Practices, Tips, Examples to get some guidance on reporting templates.
Step 5: Define your reporting cadence.
Daily, weekly, monthly, or quarterly – these cadences are both important to define to decide on the depth of reporting but also because they might determine a different reporting type and format. You might need to see different data and analysis for a weekly report than what you would like to see for a quarterly report.
Potential Pitfall: You can end up with low incremental value or diminishing return in high cadence. Remember, analysis and reporting take time to construct and deliver. Make sure the investment in them has the right level of return before you decide on the cadence and extent of your measurement plan.
If you follow these steps when you put together your measurement plan, I assure you will see better results over time. This approach will allow you to focus on what’s really important and the true purpose of reporting – to make smart decisions that lead to improvements. Don’t look for hidden silver bullets in the data. You’re not going to find them.