100% of bloggers agree this is awesome!*

*One blogger surveyed, not at random, and offered chocolate to participate.

****

wonkanator

I’ve touched on bad polls before. Sometimes it’s the interpretation that’s faulty; having earned a master’s degree in mass comm research, I can say with authority that statistics can be interpreted in whatever way needed.

Sometimes it’s the questions, such as asking respondents if they’d prefer a budget plan that cuts the national debt by $5 trillion and doesn’t raise taxes or one that raises taxes and doesn’t cut debt. When the questions themselves are skewed, the statistics interpretation doesn’t need to be, yet at times questions and interpretations employ shadiness.

Sometimes its simply the survey sample itself that’s biased. Pew, one of the most trusted research companies, discovered last year that fewer than 10 percent of those contacted for surveys actually responded, so how can many of the samples be that representative?

Basically, statistics can say exactly what you want them to say.

Chromolithograph from the 1898 oil portrait by Ignace Spiridon.

Chromolithograph from the 1898 oil portrait by Ignace Spiridon.

As Mark Twain said, “There are three kinds of lies: lies, damn lies and statistics.” OK, he probably didn’t really say that; in his autobiography, he credited Benjamin Disraeli, though even that attribution is arguable.

On the news side at my newspaper, there’s a  rule that if you cite a poll, you have to cite its source and its methodology: sample size, margin of error, randomization, etc. Elsewhere, it’s pretty much just citing the source and you’re good, so if Rush Limbaugh surveys three of his lackeys and two out of three say Barack Obama was born somewhere in Alpha Centauri, as long as you credit Limbaugh, you could say 67% of respondents believe Obama is an alien.

So how can you tell if the poll results you’re reading are biased? Start with if sample size and randomization, and margin of error are disclosed; if not, red flag. Next, check if the actual poll report is included in some way; if you can’t read the questions and check the sample breakdown (preferably by question), that’s another red flag. And often most importantly, check the disclosure on who paid for the survey; often that will tell you everything you want to know about the poll’s trustworthiness (or lack thereof). And yeah, I’m mean … I don’t let letter-writers or columnists cite statistics without citing the source … if they want to attribute bad statistics to their source so people can understand exactly where their sympathies lie, more power to ’em.

As for me, I’m stocking up on chocolate … my poll dupes demand it.

Did someone say chocolate?

Did someone say chocolate? Photo credit: latesthdwallpaper.com

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s