Fair Isaac Click Fraud Report Spreads False Alarm

Media reports (including Search Engine Watch’s) saying Fair Isaac Corp. (FIC) reported industry-wide click fraud at 10 to 15 percent are not accurate. FIC decided to put out a press release and speak at its user conference about data that was extremely preliminary, based on a small sample size, and not statistically significant.

While FIC’s press release and public statements were carefully worded, the data was inevitably taken out of context, especially by mainstream media that may not understand the intricacies of click fraud, or of statistics and research. This is compounded by headline editors, who don’t always go for nuance when they write headlines.

So that meant that much of the news coming out of Friday’s announcement ended up being misleading, such as the reports from AP, Information Week. And industry and blog reports, based on these mainstream media articles, followed suit.

I spent a good portion of my day yesterday trying to get the facts, collected in yesterday’s post, Fair Isaac Pegs Click Fraud at 10-15%. I’ve edited that to reflect the latest facts, but I feel that it needs even more clarification and explanation to put it in perspective, so I’ll try to do that here.

OK, here’s the situation: Fair Isaac Corp. (FIC) is in the early stages of applying its substantial fraud-detection skills and technology to the world of click fraud. It’s been planning a study, with SEMPO’s help, for the past year. FIC started collecting click fraud data in August.

The data has been hard to come by, with advertisers either interested but not able to convince their company to share all the data, or advertisers not having adequate data to meet FIC’s strict quality standards. That means any preliminary findings are of limited value, which FIC did say in its press release, and in talks with journalists. But as I said, data and research are often misrepresented in news stories, not because of any malicious intent, but because it’s not always easy for a reporter to dig that deeply on a deadline, and many reporters are not equipped to do much more than write down the findings as told to them by the researchers.

So was it really a good idea for FIC to go public with data, which they admit is not significant and should not be extrapolated to apply to the broader industry? I really don’t think so. I can’t say definitely what their motivations were, but I don’t think they were malicious. It seems they wanted to have something to talk about at their InterACT conference. They may have been feeling pressure to produce some results, since this data collection phase has been dragging on much longer than they expected. They may also be trying to attract more advertisers to the study, so they can get more usable data. Good intentions, but this was misguided.

They’re implying that 10-15 percent of advertisers’ PPC budgets go to click fraud. That’s a huge jump from what the search engines and other click fraud detection firms have said:

  • In March, Google reported that “under 10-percent” of clicks could be categorized as “invalid clicks,” which Google catches before advertisers are charged. The amount of invalid clicks that are not proactively detected and are caught by advertisers is less than 0.02 percent.
  • Also in March, Yahoo reported its “network discard rate,” representing the average number of clicks (in aggregate) that its clickthrough protection filters identify, tag and do not bill to advertisers, is between 12 and 15 percent.
  • In January, Click Forensics put overall click fraud at 14.2 percent, which would compare to Google’s “under 10 percent” number. Click Forensics does not offer a corresponding number to Google’s “less than 0.02 percent” figure of invalid clicks that are billed to advertisers. That’s the number that Fair Isaac’s numbers would correspond to.

The huge difference between FIC’s findings and the previous findings should have given them enough pause to wait until they had better data to report. Technically, everything they said was correct, but they had to know, or suspect, that it would be taken out of context and applied as a broader figure about the entire market. So chalk this up to a media-relations mistake, an error in judgment, or a mis-played hand.

I think FIC can have a tremendous influence on the PPC space, if they can get enough data. They have years of experience in detecting fraud, first on credit and debit cards, and then on other fields, like insurance and provider fraud, and network assurance, which monitors signals over telephone networks. But until they get that data, and can perform a full investigation into click fraud across markets and across verticals, I wouldn’t put much weight on these preliminary findings.

UPDATE: Google’s Shuman Ghosemajumder spoke with FIC as well, and has more of his response on his blog