…the polls are dirty.
Hugh Hewitt cross-examines the pollsters and those who gleefully report them:
August 2, 2012 interview of Peter Brown of Quinnipiac, exposing the pollster’s circular reasoning in just two moves:
HH: I mean, when does it become unreliable? You know you’ve just put your foot on the slope, so I’m going to push you down it. When does it become unreliable?
PB: Like the Supreme Court and pornography, you know it when you see it.
HH: Well, a lot of us look at a nine point advantage in Florida, and we say we know that to be the polling equivalent of pornography. Why am I wrong?
PB: Because what we found when we made the actual calls is this kind of party ID.
September 14, 2012 interview of Dr. Lee Miringoff of Marist Institute, who suggests he has never seen a suspicious poll as he refuses to answer Hewitt’s repeated question (eight by my count in a one-segment interview) as to when disparity of Republicans and Democrats in a poll might suggest a problem in the poll.
September 26, 2012 interview of National Journal polling reporter Steven Shepard, who admits that “Nothing bothers me.” Really, he explains:
HH: You know, how long have you, you’ve been a reporter for eight years. Have you ever run into a source that wouldn’t answer the simple question? I mean, would it cause you concern?
SS: I think I am answering your question. No, it wouldn’t cause me concern if there were 80% Democrat or 80% Republican.
HH: It would not cause you concern.
SS: I would write that and say there’s something going on here.
HH: I got my answer. It would not cause you concern.
SS: And that would run both ways.
HH: But I understand. But it would not cause you concern. If it was 100% Democrats, and they said Obama was ahead, it wouldn’t cause you concern?
SS: I would write that if in a random sample of voters in a given state, or across the country, if 99% were identifying themselves as Democrats, but the poll was adequately weighted according to race, according to gender, according to age, I would look at education, I would look at income. And if everything else checked out, I would say well, maybe there’s an important shift going on. Obviously, this is, you know, a big exaggeration.
Here’s Michael Barone with the cold water:
Having said all that, looking at, for example, these Quinnipiac results, as you note, we see that they are more Democratic now than went Democratic in the 2010 electorate, which nationally was 35% Democratic, 35% Republican in party identification, but more Democratic than the 2008 electorate, which was 39% Democratic, 32% Republican by party ID. That’s out of line with what most political observers would have expected the outcome to be this year. Up until the Democratic convention, polling showed a consistently higher degree of enthusiasm, significantly higher, among people identifying as Republicans than among people identifying as Democrats. That gap has diminished, and I think I’ve seen one Gallup poll that said that self-identified Democrats actually expressing more enthusiasm after the Democratic convention. So it’s possible that we, you know, that Democrats are more likely to pass through the screens as likely voters or registered voters, or people interested in voting through the pollsters’ screens than they were prior to the Democratic convention. But we’ve seen these kind of polls all along, and they’re, you know, I think that you want to look at them with an asterisk in mind.
HH: Now Michael, I want to remind myself of something I asked, or you told me the last time we talked, which is that no presidential candidate has polled grater than 3% of the turnout for his party in the previous off year election. Am I recalling that correctly?
MB: I’m not sure if that’s exactly what I said, or the point that I addressed. Oh, yeah, I know what you’re saying. Basically, in the last three presidential elections, what we’ve seen is that the percentage for the winning candidate has been equal to or within one percent of the percentage for his party’s vote, share of the popular vote two years before.
HH: Okay, one percent. Wow.
MB: One percent. Now that’s not an in an electable rule. It did not turn out to be true in 1994 and 1996, where Bill Clinton outpolled his party two years before, and of course, he shifted policy. And if you look at the data before 1994, it doesn’t work until you go back years and years, because basically, there were a lot more split ticket voting. White southerners were typically voting Democratic for president, Republican for president, Democratic for Congress, and there were other anomalies in a countervailing direction. So the rule only applies to the last three presidential elections. But I think it’s, you know, looking at that, it suggests some peril for the Obama candidacy.
HH: Jay Cost…
MB: Yeah.
HH: …who I have tremendous esteem for, said look, what he cares about most in all of these polls is not what they suggest the prediction is, but how the independents are breaking. And he keeps looking at Quinnipiac, at Marist, at all of them, and the independents are breaking for Romney. In some cases, it’s a little bit just above a tie, but Romney is winning the middle and the independents. And he finds this to be the most significant factor in the polls. Do you agree with his assessment on that?
MB: I think that that’s one good way to look at it. And it’s just another way of saying that those polls are showing a significant Democratic party ID edge among those who were polled, and one that in many cases is greater than Democrats enjoyed in the 2008 results.
I confess this idea that circulated about the blogosphere today has quite confused me. Something is being expressed here that I’ve missed. If a poll of random respondents comes back with 39% identifying as Democrats, and 32% identifying as Republicans, isn’t the immediately obvious conclusion that 39% of the electorate are Democrats, and 32% are Republicans?
I realize that in 2010, some polls revealed very close to equal results of partisan self-identification, and a 4% shift is a big deal. But at the same time, when you look at data from the Census Bureau, specifically table 4 here about how many registered voters there are by state, and you compare that to reported partisan registrations from the several secretaries of state around the country you get a result that 39.42% of voters nationally are registered as Democrats and 34.10% of voters nationally are registered as Republicans. That’s just math. I’ve saved the excel spreadsheet assimilating the data from these two sources, which are, it seems to me, not only non-partisan but functionally apolitical; I’ll post it if you like. (I excluded the District of Columbia from my math, DC is overwhelmingly Democratic but quite small in terms of population.)
Most national polls have a margin of error of between three to five percent, and a result showing 39% Democrat and 32% Republican is well within such a margin for the poll results which the Republican cheerleaders today seemed to all call fishy after Jim Geraghty’s interview with John McLaughlin was posted on National Review Online Friday. The raw numbers seem to indicate a seven-and-a-third point national registration lead for Democrats. If I were charged with Republican GOTV efforts, I’d find those numbers, which aren’t from polling organizations at all, daunting.
Besides that, what incentive do Gallup, Pew, Marist, etc. have to skew results to favor the Democrats? A polling organization lives and dies by its reputation for accuracy. If something is wrong with the way these groups are randomly sampling, McLaughlin and the rest of the GOP’ers singing in chorus haven’t explained that in any way I have ascertained. What I’ve heard and read is that they complain that the Obama campaign seems to want to cause a wave of depression, fatalism, and futility on the part of likely Republican voters in order to depress GOP turnout on election day. Of course the Obama campaign would like to see GOP voters stay home, but that doesn’t mean the polls which the campaign points to in its propaganda are inaccurate.
I just don’t get it. The polls look as accurate as can reasonably be expected. And as I hope you’ll recall from my post on the front page on Tuesday, I’ve no particular enthusiasm for either Obama or Romney so I might as well call it like I see it and I do not consider myself a partisan for either side in this cycle.
Polling involves a lot of self selection. And there’s of course the issue of pollster bias in exit polling at least. It’s extraordinarily difficult to know who’s going to actually show up to vote for whom on the big day versus who who happened to get a pollster call and what they happened to say in they call. It’s at least relevant what happened in the most recent actual election, and it’s bizarre to say the least that these so called empiricists would claim exactly the opposite.
But we’re not talking about exit polls. We’re talking about “nowcast” polls used to create forecasts, or more directly forecast polls. Stipulated that separating “likely voters” from “poll respondents” is a more complex task than it seems and that some pollsters are better at it than others, to a material degree. Notwithstanding that stipulation, I still don’t see a causal link between the difficulty of this task and a purported inaccuracy in either the anticipated behavior of these people or the reporting of their partisan self-identification.
Let’s just settle this on the FF field this weekend, Burt.
Bring it, dude.
Pollsters generally weight towards the census — making sure they have a representative sample that way.
The actual breakdown of D/R/I comes from the poll itself, and they do it by…well, asking. There’s generally a series of questions designed to tease out party identification and likelihood (or enthusiasm) for voting.
And indeed it can be wrong. However, like any poll — if multiple polls are showing rough agreement on the numbers, the odds of them all being outliers are low.
Pragmatically speaking, arguing there’s a vast polling conspiracy is the ‘denial’ stage of grief. It is very unlikely there is such a vast conspiracy, but accepting that Romney appears to be losing is hard to swallow.
I watched Democrats react the same way in 2004.
I’m not claiming conspiracy. I did say the polls were “dirty,” and that was probably over the top on my part, an effort to add color. I’m agnostic on why this is occurring. I don’t know. I just think it’s very weird that we have a very respected pollster like Barone saying it’s out of step with expectations to see these kinds of numbers, and yet many pollsters and reporters call that wholly irrelevant to the poll.
Look, say you had a bunch of scientists who created a weather chamber to create certain conditions in order to predict what would happen if those conditions occur in the real world. And let’s say they look at barometric pressure and humidity in their controlled environment, and concluded it was going to monsoon tomorrow. Then some respected meteorologist who’s studied actual weather in the real world says, no, we haven’t seen monsoons following those conditions in the real world. The scientists then scoff and claim the real world is irrelevant, because their artificial chamber and artificially created conditions show what they show.
Wouldn’t that seem weird? Not a conspiracy, but perhaps hubris or too much attachment to the experiment, or something. But not terribly reliable, at least.
If you have a bunch of pollsters finding the same (roughly, within MOE) numbers on party identification….
It might be weird in the sense of “I didn’t expect that” but not weird in the sense of “can’t be true”.
The worst thing you could do, as a pollster, is weight those numbers to “something that seems right” — you weight towards things you KNOW (census figures on income, race, gender) and/or things you have polled on before and trust (which requires multiple polls to avoid MOE problems).
The problem here seems to be that multiple, independent pollsters are finding party ID responses that are confusing and different, but — and this is critical — agree with each other on MOE.
Your analogy doesn’t hold true — pollsters are finding these results in the ‘real world’. They do party ID by asking about it. Either large, large, LARGE numbers of people are consistently lying about it (pollsters or respondents, take your pick) — or the ‘real world’ isn’t what you’re expecting.
But yes, I find multiple agreement among polls to be ‘terribly reliable’ snapshots as long as I trust the bunch of them not to be lying in cahoots. What’s more likely — a conspiracy of pollsters, a cluster of extreme outliers, or that the electorate doesn’t look the way you think?
When you’re questioning the basics of polling, the basics of statisics — you really only have two places to go. Outright incompetence or conspiracy. Outliers take care of themselves through multiple surveys.
What annoys me about many of these critiques of the pollseters is that everyone is pointing at the outlier polls with the outlier numbers and saying it’s a problem with the methodology. There are, literallyjoebiden, hundreds of polls on this election, there are going to be some really funky funky ones, even with 95% confidence. The ones with the D+11 are interesting because they are outliers, but, IIRC, the average poll is D+5 or D+6, which is a totally reasonable number.
Makes me think of this.
http://www.xkcd.com/882/
I’m going to take Nate Silver’s record of prognostication over Hugh Hewitt’s if it’s all the same to you.
(better, more on the point link)
Yeah, I remember him predicting a McCain win right up until election day. Hewitt is a hack even among hacks.
I’m starting to wonder how everyone on the right is going to process events when Romney fails to capture the White House. Will there be a lot of “no, if you look closely you an see he actually *won*” talk?
Well, the Democrats will have cheated, of course.
The real question would be the turnout of Ds vs.Rs. If it resembles what these polls are showing, i.e., a more decisive D turnout than even 2008, then bazinga on me.
This is why Tim (and almost all of our other posters) are worth 100 professional pundits: the willingness to say “If I’m wrong, it will demonstration that I was wrong.”
Yeah, this. If we had he tech, I would seriously clone Tim.
What do you think all this talk about voter fraud is for?
Exactly.
FTR, I’ve never heard Hewitt try to blame anyone for the outcome of an election. He blamed himself for not working harder in the 2008 election. And I’ve heard him say to the effect that, if Obama is elected a second term — if he’s not elected, for that matter — it’s nothing other than a reflection of the people’s will — not the media, not dirty campaign tricks, etc.
There are some general ad hominem arguments here, which is to be expected. But I’ll just point out that the OP is not about Hewitt’s predictions or opinions. He’s simply playing the role of a lawyer cross-examining the witness. Barone’s is the only “expert” opinion offered into evidence here.
Hewitt has an agenda, which just happens to sync with that of the numerous other conservative pundits who are jumping all over this poll issue. Cross-examination is the legal version of trying to score points for your side.
I assume what he’s doing is trying to make sure that conservatives, and particularly the “base” (which would comprise most of Hewitt’s audience), doesn’t become so disheartened that they stay away from the polls in large numbers. If they don’t vote, then Romney has no chance, whereas at the moment his chances are only small (even Gallup, which tends to skew right, though not as much as Rasmussen, has Obama with a sizable lead now). Plus, when the base doesn’t come out, the contested Senate and House races become more problematic for Republicans.
That said, he’s playing with the numbers, loosely if not fast.
I find this whole right wing concern about polling to be disingenuous to say the least. If polls showed Romney ahead, I doubt there’d be any concern. With a rabid winger like Hewitt, I’m guessing that he simply cannot believe Romney is behind the dreaded black Kenyian anti-colonial Marxist and is gravitating toward any explanation that seems vaguely plausible.
My biggest concern with this latest right wing meme is that it is an attempt to set up a narrative for an Obama win blaming the “liberal” media for deliberately skewing poll results to discourage conservative turnout. This explanation, along with whatever “evidence” of widespread voter fraud and Black Panther intimidation they can muster, will be used to prove that Obama didn’t actually win a second term.
Conversely, if voter ID laws and other supression tactics succeed in keeping Obama voters from the polls, the right wing commentariat can claim that they were correct all along about the MSM and the polls having a liberal bias.
“If polls showed Romney ahead, I doubt there’d be any concern.”
Probably no concern on the right, but there would be on the left. And rightly so! In the law, we call that part of the doctrine of standing: only the one who has been wronged can bring the claim.
I understand the doctrine of standing, but I don’t recall the left creating an alternate polling site to show results they wanted when Kerry trailed in the polls, or when the polls indicated that the Democrats faced an epic wipeout in 2010.
Nor do I recall the left generating a meme about skewed polls like the right is now doing. Heck, even the clowns on Fox and Friends joined in, despite the fact that even Fox’s polls show Obama pulling farther away from Romney.
So yeah–all of this handwringing about the polls strikes me as incredibly disingenuous.
Just for fun–Fox and Friends doubling down on teh stupid as only they can:
http://2012.talkingpointsmemo.com/2012/09/fox-friends-polls-rigged-conspiracy.php?ref=fpa
I remember individuals muttering about 2010. but citing evidence, and talking about cellphone users.
We all know the old saying that the best way to predict tomorrow’s weather is to look at today’s weather. (If you young’uns didn’t know it, now you do.)
In the same vein, the best way to predict the next election is to look at the last one.
If the last election had X% turnout for Republicans, Y% turnout for Democrats, and Z% turnout for Team Crayzeee, then you’re more likely to be right than wrong if you figure that the next election will be around there.
But which election do we want to look at? 2008? There are reasons to do that and reasons to suspect that 2008 was an outlier. 2010? There are reasons to do that as well and reasons to suspect that that was an outlier.
Part of the problem is that since 9/11, there have been so many damned outliers that there’s every good reason in the world to suspect that there’s some serious climate change going on. Anthropogenic, even.
You know how I’d do it?
I’d poll the electorate. Multiple times. Designed to figure out which party they’re leaning towards and how likely they are to vote.
I’d weight the results to the US census results to ensure I’d gotten a represenative sample (I’d probably even include some with cell phones and some without, trying to make sense of that).
And if the numbers were different than 2008 and 2010, well, I might repoll a few more times and check my results against my competitors, but I certainly wouldn’t weight them to party ID’s a few elections back.
How about 2000? I can’t rule out Romney winning 5.4.
Between 1990 and now, there were a lot of elections with a lot of dynamics that strike me as one-shots.
I rather expect 2016 to be the first “normal” election in a long, long time.
Really? I expect the opposite. I figure Romney will lose, the GOP base will decide (as per usual) that he wasn’t ‘conservative enough’ (conservatism never fails, it is only failed) and nominate Santorum in 2016 who will run on a policy that does, indeed, make Romney look like a RINO.
Which will make the electorate quite odd, I have no doubt.
I’m betting on Christie or Rubio at this point. Whosever’s backer’s stab Romneyt he hardest I bet.
I expect 2016 to look a lot like 2008. Neither VP Biden nor Secretary Clinton will be in a position to succeed termed-out Obama. There’s going to be a level of “Democrat fatigue” combined with what promises to be a strong crop of GOP contenders so the CW would give the edge to the Republicans — but I expect that both parties will have crowded, bruising primaries.