A recent survey of 1,231 journalists in traditional and non-traditional media, including newspaper and magazine journalists, television, radio and online reporters, and bloggers seemed to show that most journalists are optimistic about the future of their publications. But a closer look at the respondents may reveal some unintended bias in the sample.
Some of the findings of the “2008 PRWeek/PR Newswire Media Survey” ratify what other media surveys have found. For example, when asked how they acquire information about a company, the journalists who were surveyed cited company Web sites (89%), Google (73.8%), e-mailed press releases (72.7%), conversation or personalized e-mail from a PR person (70.9%), newswires (49.5%), and RSS feeds (13.9%.
However, other media survey findings were surprising – even to the sponsors. As Frank Washkuch of PRWeek noted in his special report, one response in particular – “63.5% of print magazine or newspaper journalists saying their publication will continue ‘indefinitely’ in its current state” – surprised Dave Armon, COO of PR Newswire.
“[It’s” more of an optimism than I would have expected about the life expectancy of publications and mainstream media outlets,” Armon told Washkuch.
It surprised me, too.
So, over the weekend, I took a closer look at the 2008 PRWeek/PR Newswire Media Survey. And I discovered why there may be a potential for inaccuracy in the findings.
It could be unintentional coverage bias caused by using a sample that isn’t representative of the entire population.
The classic example of coverage bias was the 1936 survey conducted by The Literary Digest, which mailed out 10 million postcards to subscribers asking who they planned to vote for in that year’s presidential election. While 2.3 million “voters” responded to the survey, the magazine’s subscribers were more affluent than the national average – they could afford to subscribe to a literary magazine in the middle of The Great Depression.
When The Literary Digest reported a week before the election that Alf Landon was far more popular than Franklin D. Roosevelt, no one suspected that affluent Americans might tend to be disproportionately Republican. Only after Roosevelt carried 46 states, while Landon only carried Maine and Vermont, did it occur to anyone that the magazine’s sample might have been skewed.
So, look at the sample used for the PRWeek/PR Newswire media survey: “E-mail notification was sent to approximately 8,675 traditional journalists and 956 bloggers. A link to the survey was also posted to a journalist group message board on Facebook. A total of 1,231 journalists (1,152 traditional journalists and 79 bloggers) completed the survey online between January 14-30, 2008. Results aren’t weighted.”
And, why might this sample be skewed?
First, while the sample was large, the response rate was low. Less than 13% of those contacted responded to the survey – and there’s no way of knowing if the 87% who didn’t respond would have answered the questions the same way as those who did.
Second, bloggers made up only 10% of the sample. This may not be representative of the entire population of “traditional and non-traditional media” that public relations professionals should be targeting.
According to BurrellesLuce, there are 400,000 staff listings at more than 100,000 media outlets in North America. But, according to Technorati, there are more than 15.5 million active blogs, or blogs that have been updated in the past 90 days.
And, as I mentioned last December in my article, “Blogs Are the New Trade Press,” if you look at the top referrers in your web analytics software, don’t be surprised to see that a disproportionate percentage of your web traffic comes from blogs.
So, maybe bloggers were under-represented in the sample.
Third, the e-mail notifications sent to traditional journalists were sent to ones who are still employed. That’s certainly a primary audience for public relations professionals, but it may not be representative of the entire population of journalists.
According to The State of the News Media 2008, “Most news organizations have fewer people to gather and report.” It adds, “We calculate the industry has lost a net of 3,000 print jobs from 2000 through 2006. With all appropriate caveats, we estimate as a reasonable guess for 2007 that somewhere in the neighborhood of 1,000 to 1,500 print jobs will be lost.”
So, maybe print journalists who still have jobs are more optimistic than the 4,000 to 4,500 ones that have lost them over the past eight years.
The net net: There’s lots of good data to mine in the 2008 PRWeek/PR Newswire Media Survey. But, if some of the findings appear a little too hard to believe, remember the quip by FDR’s campaign manager James Farley: “As Maine goes, so goes Vermont.”
Greg Jarboe is the president and co-founder of SEO-PR, a search engine optimization and public relations firm. He covers news search, blog search and PR correspondent for the Search Engine Watch Blog. Greg is one of the 25 successful online marketing gurus interviewed by Michael Miller for the new book, Online Marketing Heroes.