If you’re new to AdWords this article is for you. I’m going to talk you through how much data you need to be able to make decisions about your campaigns.
The first and most important point to note is that no campaign will ever perform the same from day to day, even if you make no changes. The vagaries of search mean that your campaign will have a certain variability from hour to hour, day to day, week to week, and even month to month (you’ll have lots of month to month variability if your campaign is subject to seasonal changes).
What this means is that if you look at the data from the previous day, there is a good chance that what you’re looking at isn’t representative of “usual” performance.
The metric you’re interested in here is your campaign’s “standard deviation.” Reach your mind back to school statistics lessons now. If you can remember it well, then skip the next paragraph.
Standard deviation is a way of measuring the average amount that your data diverges from its mean average. What that means is that if you have data that is clustered around a certain level but is normally off by a certain amount, the standard deviation will tell us how much offset is normal. Imagine you had two days worth of click volume data. Day 1: 5 clicks. Day 2: 9 clicks. Your average is 7 clicks per day. But you are normally 2 clicks away from your average. A small standard deviation means that your campaign has a low variability, and vice versa.
I’m not going to suggest at this point that you actually calculate the standard deviation of your campaign metrics, since in all likelihood you have been making too many changes to necessarily know how much of the change in your data is because of search variability and how much is because of your own deliberate changes (new bids, new ad texts, etc). If you’re interested, most spreadsheet packages have built-in functions for calculating the mean and standard deviation of a dataset.
What you really need to know is that if you don’t have enough data before you make decisions about your campaign, you may well be trusting data that is just undergoing a natural amount of up and down.
Do You Need a Long Timeline to Have Enough Data?
Well, sort of.
Ceteris paribus, the more data you have the better, and getting more data is as easy as increasing your timeline.
But we don’t live in an “all other things being equal” world. At least I hope not. I hope that your AdWords campaigns are undergoing regular adjustments, bid changes, new ad texts, landing page tests, negative keyword reviews, etc. Enough changes are (or should be) going on in your account that you can’t necessarily trust your long timeline data.
If I were to look at my campaign over the last six months I’d get a really good idea of my overall conversion rates for certain sites in my display campaigns. I’d find the poor performers and reduce the bid on those sites.
But if I already reduced that bid last week (trust me, you won’t be able to remember every change you make) then you may have over-reacted. You’ve already fixed that problem, so don’t fix it twice!
OK, So You Need a Short Timelines?
Well, sort of.
If I want to know I’m not looking at data influenced by problems I’ve already dealt with, then I need to use a more recent timeline. If I’ve selected 6 months of data but last month I made major changes, then I’ll only want to look at the performance since those changes.
But we’ve already established the problem there. Too short, and the data is susceptible to variability. Maybe this week was just odd. Good weather, bad weather, school holidays, political turmoil, economic uncertainty, predicted upcoming tax changes… the list of things that can affect your campaign in the short term and are out of your control is nearly endless.
How do you know if your change helped your campaign or if people just went on a back-to-school spree? You can’t.
Some things you’ll know about and can anticipate the effects of, but whenever you’re comparing one timeline to another there may be other changes. Particularly over very short timelines. The higher your standard deviation, the worse this will be.
What’s the Answer?
Use both! Long timelines and short timelines. From now on whenever you’re analyzing your data, consider a minimum of two date ranges: one short, one long.
It doesn’t matter which you look at first. If you start with a short timeline and you see a keyword with a high CPA, your first thought should be: “Is this a chronic problem? Or has this been an unusual period?” Then you note that keyword and see what its performance has been like over a longer timeline.
Conversely, if you start with a long timeline and see an underperforming keyword, your first thought should be “Is it still bad? Or is this already better?” Shorten your timeline and see if the situation has improved recently.
How Much Data is Enough?
So far I’ve stayed away from the question of what would be considered long or short time ranges. The reason for that is because there aren’t any hard and fast rules on that. Your judgment will have to come into play.
Option 1: You download all your data on daily, weekly, and monthly timelines. You find the standard deviation of each, and take long term to be the first date range where the standard deviation is below X percent of your mean.
So if your weekly click volumes have a mean of 2,500 and a standard deviation of 500, then it would mean you expect to fluctuate by around 25 percent away from your average each week. But if your monthly click volumes are 10,000 and you have a standard deviation of 1,000, then you can expect to fluctuate by 10 percent away from your mean each month.
How much percentage to use is up to you and depends on what metric you’re measuring. If you’re measuring clicks then you’d hope for a lower percentage than if you’re measuring conversions (because conversion volumes will always be lower, and therefore will have a higher relative standard deviation). You should be perfectly happy with a date range where your standard deviation of click volumes is 15 percent of your mean. You could use that sort of range as a long timeline.
To get this data, navigate to the level of hierarchy you’re interested in (e.g., account, campaign, or ad group level) and go to the dimensions tab. Set the date ranges and choose daily data, then click to download.
Option 2: Pluck a figure out of the air that seems sensible. I know this goes against the overall statistical approach that I’m suggesting you use, but in reality it will only be a little bit worse than Option 1, and a lot less work!
It’s typical that on a large campaign you’d consider a daily date range as short and a weekly date range as long. On a more common campaign you might consider a weekly date range as short and a monthly or three monthly date range as long.
You’ll be able to use your judgment after a while of running any campaign. You’ll swiftly know how wildly things change from day to day or week to week, and that those date ranges can’t be used as long timelines.
Bid Management: The Complication
Bid management relies on having trustworthy data to make decisions. If you’re using an off-the-shelf, well respected bid management system then this won’t be a problem. The software will already be doing all of these things. It will be calculating the standard deviation (and more) of different date ranges when deciding what data is safe to use.
- If you’re doing it yourself (by spreadsheet, automated rules or another method) then you’ll need to take this into account. You simply can’t make judgments on the previous day’s data if you already know that your standard deviation could be a huge proportion of your mean.
- If you’re targeting a specific CPA or ROI, then you’ll need to use your long timeline for conversion rate measurements. If you’re position targeting then it gets even more complex.
- If you’re position targeting, then you can’t use data from a longer date range than your frequency of changes. If you do so you’re guaranteed to be making decisions based on outdated data, and you will overshoot your targets. So you either need to make frequent changes (based on short timelines) or infrequent changes (based on a longer timeline). The problem with short timelines is that you may make a decision based on one-off data, and the problem with infrequent changes is that you won’t react quickly enough to the market. Make sure you consider all possibilities and consider separating your frequency of changes (and the timeline used) by how many impressions that keyword has per period.
I haven’t given you any hard and fast guidance here for how to get around the problems I’ve outlined. You either need very sophisticated systems to be able to make these judgments for you, or you need to make an informed judgement.
Humans are notoriously bad at judging this sort of thing, so always apply the two-timeline process above before making any decisions, and think to yourself “Do I trust this data?”