How Not to Interpret Polls
The Washingtonpost.com, Quinnipiac, and the Wall Street Journal have a new poll, which is advertised as showing this: “McCain Makes Significant Gains in Four Key Battleground States.” (Here is virtually the same headline at the Wall Street Journal and Quinnipiac. I would bet both the Washingtonpost.com and the WSJ were basically transcribing Quinnipiac’s press release.) At least one prominent left-wing blog, Talking Points Memo, calls it a “reality check.”
This analysis is wrong. For two reasons:
1) In 2 of these 4 key states, there has been no meaningful change, given the inherent sampling error in polls. In both Michigan and Wisconsin, McCain’s share is unchanged; Obama’s is down 2 points. See here.
2) Repeat after me: ONE POLL DOES NOT MAKE A TREND. Look at Pollster.com’s poll aggregations for another of these four states: Minnesota.
Only in Colorado does this poll’s numbers appear to conform to a trend. See again Pollster.com.
This kind of analysis may generate clicks for the Washingtonpost.com — which, it must be noted, is largely separate from the Washington Post and its polling outfit — and for the WSJ.com, but this analysis is terrible.
Comments
John,
I don' t even know if the Colorado interpretation is that accurate. Of the Colorado polls taken since Obama clinched the nomination (six polls), this is the only one showing McCain in the lead. The Pollster graphic does at least show McCain gaining in his share of the preference in recent polls. However, my average still has Colorado at about Obama +3, which is down but only slightly.
Then again, the real problem may be the issue of the change Rasmussen has made recently to include leaners in his state-by-state analyses. I don't have a problem with the decision, but when the "with leaner" polls are being compared to previous "without leaners" polls there is a problem. That problem is compounded when the new "with leaners" data is incorporated into all these electoral college analyses (mine included). Colorado is an excellent example of this. The recent poll with leaners had Obama up by 3, but the without leaners version had Obama up 7. That four point difference has implications for properly analyzing these things.
But it looks like even "proper analysis" is a bit too much to ask for during this cycle. Ha!
Posted by: Josh Putnam | July 24, 2008 05:40 PM
Josh:
Always remember MOE. All four July polls for Colorado have McCain and Obama statistically tied.
Since Obama won the nomination, in Colorado, only Quinnipac/June showed an Obama lead close to MOE limits (5%), but I attribute that to a primary-race-win-bump, which has since almost disappeared. Except for Quinnipac's MN, McCain has not gained or his gain is within MOE (as John points out).
I compared the Quinnipac numbers for June and July with the Gallup aggregate for swing states here:
http://www.pollster.com/blogs/poll_quinnipiac_colorado_michi.php#comment-40412
Posted by: RS![[TypeKey Profile Page]](http://www.themonkeycage.org/nav-commenters.gif)
|
July 26, 2008 01:19 PM