*One blogger surveyed, not at random, and offered chocolate to participate.
I’ve touched on bad polls before. Sometimes it’s the interpretation that’s faulty; having earned a master’s degree in mass comm research, I can say with authority that statistics can be interpreted in whatever way needed.
Sometimes it’s the questions, such as asking respondents if they’d prefer a budget plan that cuts the national debt by $5 trillion and doesn’t raise taxes or one that raises taxes and doesn’t cut debt. When the questions themselves are skewed, the statistics interpretation doesn’t need to be, yet at times questions and interpretations employ shadiness.
Sometimes its simply the survey sample itself that’s biased. Pew, one of the most trusted research companies, discovered last year that fewer than 10 percent of those contacted for surveys actually responded, so how can many of the samples be that representative?
Basically, statistics can say exactly what you want them to say.
As Mark Twain said, “There are three kinds of lies: lies, damn lies and statistics.” OK, he probably didn’t really say that; in his autobiography, he credited Benjamin Disraeli, though even that attribution is arguable.
On the news side at my newspaper, there’s a rule that if you cite a poll, you have to cite its source and its methodology: sample size, margin of error, randomization, etc. Elsewhere, it’s pretty much just citing the source and you’re good, so if Rush Limbaugh surveys three of his lackeys and two out of three say Barack Obama was born somewhere in Alpha Centauri, as long as you credit Limbaugh, you could say 67% of respondents believe Obama is an alien.
So how can you tell if the poll results you’re reading are biased? Start with if sample size and randomization, and margin of error are disclosed; if not, red flag. Next, check if the actual poll report is included in some way; if you can’t read the questions and check the sample breakdown (preferably by question), that’s another red flag. And often most importantly, check the disclosure on who paid for the survey; often that will tell you everything you want to know about the poll’s trustworthiness (or lack thereof). And yeah, I’m mean … I don’t let letter-writers or columnists cite statistics without citing the source … if they want to attribute bad statistics to their source so people can understand exactly where their sympathies lie, more power to ’em.
As for me, I’m stocking up on chocolate … my poll dupes demand it.
- 5 Ways to Avoid Being Fooled By Statistics (iacquire.com)
- Most Americans think a government shutdown would hurt both the economy and their own families, new poll shows – @HuffingtonPost (huffingtonpost.com)
- YouGov Poll Says Nearly 60% of Americans Believe the Devil is Real (patheos.com)
- Public Policy Polling and the performance of polls (talkingpoliticsjomc.wordpress.com)
- Americans’ Belief That Gov’t Is Too Powerful at Record Level (counterinformation.wordpress.com)
- When Discussing Confidence Level With Others… (r-bloggers.com)