Friday, October 03, 2008

A Question for the Peanut Gallery

Let's say you watched a debate. And then you read a poll after the debate that said that someone won, and someone lost, and somebody else had a positive impression, etc. etc.

You'd believe it, right?

But what would you do if you found out that the poll was done halfway through the debate:

For everyone who hates public opinion polls, and the effect they have on campaigns, this first-out-of-the-gate survey "following" the English-language leaders' debate is all they'll ever need to make their case.

It's not that the results are wildly out of whack with the general sense one gets talking to people. But that's beside the point. This poll on who won the debate was taken halfway through the debate.

All (or at least the vast majority) of us in the media are undoubtedly guilty of taking the horse-race thing too far. But this is really a whole other level.

I'm sure you understand why. In case you don't, here it is.

The polls effectively set the narrative for the campaign. Who "won" or who "lost" the debate, and thus who we cast as gaining momentum or losing it at the most pivotal moment of the race, will thus be overwhelmingly determined by what surveys of viewers tell us. But if someone had performed startlingly well or startlingly poorly in the second half of the measly two hours, the impact of that performance would have been negated because from the perspective of the poll driving the narrative the next day, it never happened.

I understand that polling is an enormously competitive industry, and I have a lot of respect for the smarts of the people in it. But there have to be limits, and this surely exceeded them.
There's an ass-covering response from Ipsos-Reid there saying that "the questionnaire didn't close until the last 30 minutes of the debate". But that's immaterial. Nobody should be polled on a debate until after it's over; that's just common sense.

Online polling is questionable enough. It's demographically sketchy and in terms of self-selection it's pretty much worthless. But this sort of scheme? Oh, you have to be kidding.

Sorry, Ipsos-Reid. But even if Harper really did win that debate--and it's a reasonable argument to make that he did--your poll cannot, and does not, effectively prove it.

No comments:

Post a Comment