HOME | POLITICS | SPORTS | LIFE | SCI/TECH | OPEDS | HELPFUL TIPS

Useless-Knowledge.com
Articles


How to Distort a Public Survey Poll

By Tom Pain
Jan. 3, 2007

I saw headlines for the results of a survey on Yahoo! today – “Poll shows support for Democrats' goals.”  Naturally, it caught my attention with its sweeping proclamation and the implication that Democrats are united in their goals, so I opened the link.  I was not happy with what I read.

 

First, the article did not, like most such articles, actually provide the poll questions.  Readers are expected to just trust that polls are fair and balanced.  There was a mention of the polling company in the last paragraph, but the link to those details was broken – hmm, what a surprise.  So I Google’d the company name, but still had to drill through their site and download a PDF to learn the facts. 

 

The article claimed that “two of the Democrats' top goals — a higher minimum wage and federal funding of embryonic stem cell research — enjoy broad public support...." The problem is that the survey questions were even broader than that support, to wit:

 

Poll question:  “Do you favor or oppose an increase in the minimum wage?”

 

Surprisingly, only 80% responded in favor of an increase.  I would have predicted a higher number, more in line with the percentage who would vote no to the question “Do you favor war?”  Who doesn’t want people to make more money??? 

 

The problem with that question is that it presumes the respondent understands the implications of their favorable vote.  What do you think the results would be if the question were restated thusly:

 

“Understanding that the increased costs would be passed on to consumers in the form of higher prices, would you favor or oppose an increase in the minimum wage?”

 

Nothing untrue or biased about that question, and it provides the consequence to the action rather than presuming that the respondent understands that.  I would predict a significant drop in the favorable responses if the question were asked that way.

 

Next question: “Should the government ease the restrictions on use of federal money to research embryonic stem cells, or not?”

 

The article was even more biased than the poll question.  After glorifying the supposed benefits of the research, it characterized the opposition as one-dimensional:  “Bush and other opponents say the embryos from which the cells are extracted are human lives that should not be destroyed in the name of science.”  It mentions no other argument against the research and makes sure to smear Bush with its bias. 

 

But look at the question.  Immediately that insertion of “or not” sticks out.  What is its purpose, isn’t the ‘yes’ or ‘no’ answer going to tell us that?  Also note the use of the phraseology – “ease the restrictions.”  Our natural instinct is to oppose “restrictions,” but at the same time, in order to capture those who have concerns, “restrictions” implies some measure of oversight.  And note the reversed, elongated sentence structure.  It is important to know that this was a telephone survey as well.  You have to imagine how someone answering a phone call, without advance notice, hears such a sentence.  Positive but reassuring (ease restrictions), positive (research), then ending with that dire warning, “OR NOT.”  There has to be sufficient spacing for those subliminal concepts to sink in, thus the elongation.

 

Understand, I’m not saying everyone falls for this BS, but it only takes a few to significantly impact the survey.  Here’s how that question could have been asked to produce much different results: 

 

Do you believe taxpayer money should be used to fund embryonic stem cell research by private, for profit, drug companies?

 

Of course, the article also should have included the legal justification for those restrictions – for profit companies should fund their own research.

 

Next, the article notes that importing prescription drugs is illegal, but qualifies it by stating:  “but the Food and Drug Administration generally does not bar individuals from bringing in small amounts for personal use.”   In other words, it is really already OK.  In the survey, 56% responded favorably to this question:

 

Do you favor or oppose the federal government making it easier for people to buy prescription drugs from other countries?

 

Notice it doesn’t mention the illegality?  Where is that “ease restrictions” phrase now?  To a respondent who is unaware of the current laws, and what they seek to protect by making importation illegal, this is a no-brainer – “what, we can do it now and you want to know if I think it should be easier?”  Of course, anyone would want to make a permitted activity easier.  Why don’t they give the respondent enough information for them to know what they’re voting for?  Here’s a better version of that question:

 

“Personal importation of prescription drugs from other countries is illegal, despite the cost savings to consumers, to insure that the drugs meet American purity standards; do you favor or oppose legalizing the importation of prescription drugs by individuals?”

 

Phrased that way, many more respondents would have, undoubtedly, opposed the proposal.

 

The last poll question was not policy oriented.  It sought to gauge the popularity of incoming House Speaker Nancy Pelosi.  From its phraseology, it would appear the pollsters sought to minimize any negative opinion about the Representative.  Here’s the question:

 

Do you have a favorable or unfavorable opinion of incoming House Speaker Nancy Pelosi, or haven’t you heard enough about Nancy Pelosi yet to have an opinion? 

 

Strange that they include those first two answers in the question, but neither is one of the five possible choices from which respondents are asked to choose – those being ‘very favorable,’ ‘somewhat favorable,’ ‘somewhat unfavorable,’ ‘very unfavorable,’ and ‘haven’t heard enough yet.’  Their trick is to then add that clause to the end of the sentence – about not having heard enough yet.  While the question seems easy, respondents were forced to decide – quickly, because of the tendency to want to end an interrupting phone call – between the indefinite adjectives ‘somewhat’ and ‘very’ if they did, in fact, lean one way or the other.  So, by emphasizing the ‘haven’t heard enough yet” answer with that clause, the pollster leads the respondent towards an easy out.  And, since most people want to be positive, this bias is sure to eliminate more of the ‘unfavorable’ votes than it would ‘favorable.’  A more accurate result would have resulted if the question had been asked this way:

 

What is your opinion, if any, of incoming House Speaker Nancy Pelosi?

 

Seriously, isn’t that just the obvious way to ask that question?  Don’t you know that professional pollsters, experts in the art of asking questions, have very good reasons for asking it that other, unnatural way?

 

Finally, as biased as this survey appears once you go to the trouble to find the source data, I’m sure it is even more biased.  What other questions were asked that the pollsters chose not to publish?  Definitely, I would guess that a separate poll first established that Nancy Pelosi was not popular, and then this poll was formulated to produce the results they needed.  It is not too far-fetched to believe that the firm actually conducted different versions of this survey, using different formulations of those questions.  Once they confirmed which questions produced the best results, they conducted a final survey using those.

 

The Associated Press commissioned the survey.  I have noted their liberal bias many times and quit reading my local paper, The Dallas Morning News, in part because of its heavy reliance on AP content.

 

After analyzing a number of these surveys now, I can honestly warn you not to believe any of them.  I’ve yet to see one that was not promoting an agenda with bias or distortion of some kind.

 

Yahoo article:  http://news.yahoo.com/s/ap/20070102/ap_on_go_co/congress_poll

 

Survey data:  http://www.ipsos-na.com/news/client/act_dsp_pdf.cfm?name=mr070102-1topline.pdf&id=3322

 

------------

About the author Tom Pain: Just an American boy with so much common sense, it hurts.

Email: thomas.pain@hughes.net


Comment on this article here!

------------

All articles are EXCLUSIVE to Useless-Knowledge.com. Please link to this article rather than copying and pasting it onto your site (which would be unauthorized and illegal).

Google
 
Web useless-knowledge.com

Useless-Knowledge.com © Copyright 2002-2006. All rights reserved.