Perhaps one of the most overlooked aspects of ad testing is formulating a hypothesis. Some may choose to skip the hypothesis because it is easier to create a few different ad or landing page variations, set them live, play the waiting game and hope for some positive changes. Others, whom I dare call the worst optimizers, are simply afraid of being wrong.
What we often fail to remember, though, is that the purpose of a hypothesis is not for you to prove it right through your test, but rather to use it as a framework for executing the test and a reference point for understanding and interpreting the results.
Always be cognizant of the fact that there is no right and wrong in testing. There is trying, learning and repeating it all over again.
You can't screw up when it comes to testing. If you implement a particular change that doesn't yield positive results, you haven't lost. You will stop the test (assuming that you are keeping a close eye on your metrics) and know that this particular change is harmful to your business.
In this sense, you haven't lost because your hypothesis was proven incorrect. You have gained knowledge about something that does not work and will avoid it in the future. That knowledge is worth far more than the value we often give it.
Every test must begin with a clear, specific, and informed hypothesis. To create a good hypothesis, start with a question you are curious about. For instance, "why aren’t people clicking my ad?" The answer to this question could be something like "maybe because I don’t have a free trial offer."
Your hypothesis, then, is the outcome you expect if you implemented your answer through a specific change. More precisely, your hypothesis in this case would be "Adding a free-trial offer in my CTA will increase my ad’s CTR."
The more specific the hypothesis is, the easier it will be to create a clear framework for testing. Here is an example of how you could quickly improve a hypothesis:
- "Let’s see what happens when we add a free trial in our ad copy." --> Needs work.
- "If we put ‘Free Shipping’ in our ads, we will get better results." --> A little better.
- "Adding ‘Free Shipping’ to our branded ad campaigns will generate higher CTRs and a better Quality Score." --> Good enough.
If you don’t know where to start in terms of creating a hypothesis for your test, here are some ad components you could use:
- Promotional Language
- Branding Language
- Dynamic Data
- Display URLs
Developing a good hypothesis is not a random act and should always be informed by some degree of knowledge, irrespective of any certainty around what the outcome may be. Best practices provide a great starting point for developing hypothesis, however they don’t always yield positive results.
You should be especially wary of absolutes such “free trials always improve CTR!” It is important to be skeptical and to test when determining whether absolutes are applicable to your unique set of products or services presented in the context of your ads.
So, do you always start your tests with a hypothesis or are you afraid of being wrong?
At SES London (9-11 Feb) you'll get an overview of the latest tools, tips, and tactics in Paid, Owned, Earned, Integrated Media and Business Intelligence to streamline your marketing campaigns in 2015. Register by 31 October to take advantage of Early Bird Rates.