07 Mar 2011

How Useful are Polls that Don’t Include Context?

A Wall Street Journal/NBC News poll (Wall Street Journal, March 3, 2011, page A5) asked Americans about what they believe should be done to fix the federal budget. Large majorities (76-77%) in both parties believe it is unacceptable to make major cuts in entitlement programs and K-12 education, but smaller majorities (51-57%) believe that it is acceptable to cut federal assistance to states, the EPA’s budget and nuclear power plant subsidies. For those questions, only 1-3% of people answered “not sure” (detailed document here). Most people were “sure”. But, were the “sure” people’s opinions based on understanding the question’s context?

How many people who expressed an opinion in the WSJ poll knew, for example, (i) the size of the 2011 federal budget, (ii) how much federal nuclear power subsidies cost taxpayers and (iii) what damage (or benefit), if any, accrues to our economy or society at large from that spending? Ditto for spending on the EPA, spending on K-12 education or medicare. What if an unbiased cost-benefit analysis showed that nuclear power subsidies or medicare amounts to the single least (or most) effective spending in the entire federal government? What about medicare spending in the context of defending the American standard of living? Good, bad or indifferent?

It is a fair bet that less than less than 1% of people who answered the WSJ poll about federal nuclear power subsidies, EPA spending, K-12 spending or maybe even social security understand the real (unspun) cost-benefit of the spending that they claim to like or dislike. Nonetheless, they have opinions. Asking for fiscal restraint based on understanding the cost-benefit makes sense. Asking for restraint based on ignorance is a crap shoot. It could help, hurt or be neutral.

Pollsters ask Americans all kinds of questions about all kinds of complex topics all the time. The topics are usually shrouded in sophisticated spin by sophisticated interests on all sides. What if the WSJ poll had asked those same questions twice, once with no context and once with meaningful cost-benefit context, i.e., how much cost, what benefit? That would probably give us significant shifts in some or all of the WSJ poll results.

If that is true, do polls like this help or hurt public discourse? Does that help or hurt America? What is the value of polls like that when politicians use misinformed public opinion to guide their all-important quest for re-election? When politicians pander to uninformed public opinion to set policy, do we win or not?

Author Details
This centrist community blogger has chosen not to reveal much about themselves in their bio – as is their right.
This centrist community blogger has chosen not to reveal much about themselves in their bio – as is their right.

5 thoughts on “How Useful are Polls that Don’t Include Context?”

  1. You touch on a real pet peeve of mine about polls on issues that involve substantial detail. They ask for peoples opinions without regard for how informed they are.

    Then they treat all such opinions equally. Given how voting works, that’s not entirely indefensible, granted. But still… .

    I would love to see the results if polls included questions that asked people to rank how well informed they were on a topic before stating their opinion. I’d love it even more if folks who said they were very well informed were required to take a brief quiz to prove it.

    I would not go so far as to say that polls hurt public discourse. They are simply tools, as good or bad as the people who use or abuse them.

    If I were the President, I’d do lots and lots of polling as part of a good faith effort to better understand and respond to both the public’s views and needs on the one hand, and as a stealthy way to inform them better in objective ways. This would certainly be viewed as duplicitous by the party not holding the white house, but who gives an FF? The President is nailed to the cross by the opposition within days of taking office anyways.

  2. I find most poll results to be largely worthless because questions are simplistic, with simplistic answers. It’s like the “Do you like the Health Care Act?” question. Asked as one question, polls show disfavor. Asked in separate questions, giving specific highlights of the bill, the results turn around. But even THAT is misleading, because different aspects of the bill could be weighted differently, with respect to importance of each.

    Another poll that was frustrating was the poll regarding satisfaction with HCR. Overall, results were negative, but right wing pundits completely ignored the percentage of those who disliked the bill because it didn’t go far enough (for them). So they lumped in the righties and lefties into one group.

    I hate polls.

    1. I distrust most poll questions and answers too. Nonetheless, they are powerful opinion drivers and make great, cheap “news”.

      What can reasonable people do in the face of that? I struggle to find better, more efficient and humane ways in politics, but am always crushed by the status quo. Do you have any low-cost suggestions about how to fix this?

      As long as people like you and me are in the minority and vastly outspent, I suspect that our opinions will get nowhere unless things get much worse economically. I don’t want to just complain about the situation. I want to go on the attack. I just don’t know how to do that without tens or hundreds of millions of dollars behind me to purchase politicians and public opinion.

      Logic and reason should win political arguments, but they don’t. Sophisticated spin wins, and that includes inane opinion polls most politicians use to weasel into the next re-election cycle.

      1. Actually… issue wise… even with all the lack of political pushing of centrist ideas we still are where the American people gravitate towards overall.

        You do what you can, with what you have. This site is just a blog, but it wont be just a blog for long, and even the next stage in its evolution is just step two. It WILL take millions, and a nationwide network of activists, to form an opposition capable of really fighting them, but that isn’t a reason not to do what you can, with what you have, right now.

        Start small… something you can start building a track record of success with. People focus too much on the big things, thinking they can have an impact on macro level issues, when they really should be focusing on something they can have an impact on, however small.

        Pick a candidate, cause, group, etc… and put more time, energy and money into it than your opponents are willing to. THAT is what we need… because the two parties have that. That’s why I put about 40+ hours a week into this stuff.

  3. 1. Always read the entire poll’s phrasing.

    2. Ask yourself, what is the intensity of the opinion being polled? Opinions shift quickly according to phrasing or context, until such time as long-neglected topics suddenly get lots of coverage and debate, and only then do opinions (a) intensify and (b) lock in (to a degree).

Leave a Reply