A lesson in poll reading

baldheadeddork · 09-04-2008, 09:34 PM

Aside from McCain's acceptance speech, the top story tonight is a CBS/NYT poll released tonight that Obama has gone from a 48-40 lead on September 1 to a 42-42 tie today.

The talking heads have already run with it as evidence that McCain is getting his convention bump and the race will be essentially tied again at the least when it's over. Since the poll was mostly taken before Palin's speech, and totally before McCain's speech, McCain could even jump into a lead.

A swing that big in just three days, when the driving event was only half over when most of the respondents were called, should make anyone skeptical. And it turns out there's a big reason why, but CBS doesn't tell you about it in their story.

I've said several times that the backbone of any poll is how you measure up the electorate for your model. If you think the electorate is equally divided between Republicans, Democrats and Independents, then you should have equal numbers of each in your poll results, regardless of how they answer questions.

If you get this wrong, you'll underestimate the actual results from one group and overestimate another. This is something all of the polling companies are wrestling with this year because the electorate has changed since 2004 by all accounts, both in polling and in voter registration numbers.

That's one thing. But you can also get a big change in the results if you change the size of the sample for each group between polls. If you do one poll with one-third Republicans and one-third Democrats, but the next poll you make it 40% Republicans and cut the number of D's and Independents in the sample. If you put more of one group in the poll, and they lean in one direction, you'll create a big change in the results.

And that's what CBS did between these two polls.

They don't mention it in this story or in the story from Monday that showed an eight point Obama lead. But if you look at the very bottom of the full report for the first poll to the weighted respondents and grab a calculator, the model breaks down as R-26/D-35/I-39.

In the new poll, it's R-31/D-34/I-34.

According to CBS, there has been a 6% swing in the electorate between Republicans and Democrats, and a six percent swing between Republicans and Independents, in just 72 hours.

Small wonder there was an eight point swing in the final results.

I don't know if CBS ordered a change in the model, or if it was an accident or a mistake. But to not note it in their reporting is hugely misleading.

F&D · 09-04-2008, 10:06 PM

Smell the panic. Taste the terror.

Gaolee · 09-04-2008, 10:13 PM

No panic, no terror. Just a matter of looking at how things are. If anything, the methodology shift should have resulted in a greater than 8% change in results, but since it didn't, my take is it is positive news for the Democrats. If you overrepresent Republicans and it is no better than tied, McCain has a problem.

I don't see anything here to justify a return to the smug crowing Republican dimwits have been doing for the past eight years.

SlowGoose · 09-05-2008, 12:13 AM

As I've said before, I have no problem with a McCain lead right now. Front runners, in this cycle, seem to have a way falling by the wayside. I just as soon see McSame and the Caribou Barbie lead for a while.

Dakota Kid · 09-05-2008, 09:33 AM

When Sarah Palin burst onto the scene, it changed the landscape in a big, big way. Yes, there is angst in the Dem circles, but it is still early. If I was a Dem, I would be more concerned that my guy did not have a double digit lead...after all...the sky IS falling, right?

Yeah...a landslide.

I don't know who is gonna prevail....but I do believe it will be very, very close.

SlowGoose · 09-05-2008, 12:52 PM

I think it's going to be close too.

Dakota Kid · 09-05-2008, 12:55 PM

Lets hope it is not SCOTUS close, though. This country does not need that again.

SlowGoose · 09-05-2008, 12:56 PM

Quote:

Originally Posted by Dakota Kid

Lets hope it is not SCOTUS close, though. This country does not need that again.

I agree completely.

Mooosman · 09-05-2008, 12:58 PM

It's going to be close because of the "I'd like to have a beer with him..." crowd.

Among people who actually follow politics, it won't be close at all.

Nick

cbrenthus · 09-05-2008, 01:33 PM

Personally, I think polls are full of crap. It doesn't matter who is leading now, it only matters who wins. There is still quite some time before the election, and alot can happen.

I also think polls are bad for elections. People should vote if they want to for who they want to. I think polls help shape election results when people look and say "Oh, candidate A is not doing so well, so instead of voting for this independent guy, I'm throwing my vote over here". I really how election day is handled, as well. I do not think there should be any exit polling, or at least exit polling results untill ALL the polls are closed.

But hey, I guess that's how it is, and I don't think they should be illegal either.

Gaolee · 09-05-2008, 02:24 PM

Polls are good. They keep us entertained and give us something to talk about. Without polls, we would be stuck discussing whether Sarah Palin's grandkid will be born with or without hair.

SlowGoose · 09-05-2008, 02:32 PM

Bah!

flylooper · 09-05-2008, 02:40 PM

Quote:

Originally Posted by cbrenthus

Personally, I think polls are full of crap. It doesn't matter who is leading now, it only matters who wins. There is still quite some time before the election, and alot can happen.

I also think polls are bad for elections. People should vote if they want to for who they want to. I think polls help shape election results when people look and say "Oh, candidate A is not doing so well, so instead of voting for this independent guy, I'm throwing my vote over here". I really how election day is handled, as well. I do not think there should be any exit polling, or at least exit polling results untill ALL the polls are closed.

But hey, I guess that's how it is, and I don't think they should be illegal either.

I agree on them being full of crap. There's something really funky going on with the polls. I can't prove it. I just intuit it. I'm not sensing a McCain candidacy that makes it a close one.

I'm stickin' with my prognosis that it'll be a lopsided win for Obama. I feel especially confident since Dakota won't put his money where his mouth is and take my offer of a $50 even money bet.

As they say, money talks and bullshit walks. I'll even go a hundred with you, Deek. Waddayasay?

BigArn · 09-05-2008, 02:45 PM

I'll be the official money holder.

cbrenthus · 09-05-2008, 02:49 PM

Quote:

Originally Posted by flylooper

I agree on them being full of crap. There's something really funky going on with the polls. I can't prove it. I just intuit it.

Well, while I don't think the polls are very accurate, what's worse is I think they are extremely irrelevent.

Think about it like this:

If you are watching a football game, and at half time, your team is up by 1 point, are you going to start jumping around and cheering that your team has won?

Its the same thing. What the polls say right now doesn't matter, there is only one "poll" that counts, and that is the official election.

And where is the bookie one here? We need a betting thread about now!