Golly, The Polls Might Be Wrong Again?

Audio Version

Gee, how can that be?

Okay, okay, less sarcasm, more explanation. There are several major factors and a host of minor ones. We’ll cover the majors.

First, most polls aren’t done to actually measure voter preference. This is a feature, not a bug. MSNBC, CNN and FOX just want something to fill the dead space. Polls are quick, cheap and sound like something – without being easily checked. Well, at least most folks don’t know how. Five bajillion percent increase in public dissatisfaction sounds like it is important. It’s not. But it sounds important and news worthy. it’s REALLY not. When the government doesn’t do anything stupid all day or there hasn’t been a crisis all week and Trump is no longer making mean tweets, a poll that tells you nothing at all about the electorate or its behavior but has numbers and sounds scientific is just the thing to fill in the gap. At least if you are a journalist.

There is a more sinister aspect to this. By harping on the polls that show whatever the media is trying to sell, the media tries, sometimes successfully, to influence the public rather than just report the facts. The reality is, if the polls all seem to go one way, it suppresses turnout on the other side. Unless the gap is too large or the public has caught on, hence little of that appears this cycle. The backlash did occur – the media managed to dishearten Democrats which will probably depress their turnout – the opposite of what most media wanted. Still, polls can affect elections and just because the public has wised up for now does not mean it can’t work later down the line.

Second, modern polling is pretty much crap. It’s not all the pollster’s faults – the media has damaged the heck out of pooling as an industry. No one liked pollsters ever. When you decide not to participate, we call that mortality. Pollsters then have to keep calling until someone does participate. This is a normal problem for polling but it is getting ridiculous thanks to the media. The incessant polling drives mortality up. The sampling is partially to blame – it’s easier to sample in cities even by phone. As more and more surveys try to sample even in large cities, more and more people get tired of the calls and stop answering. The effect is that the sample will be badly skewed toward people dumb enough to answer and not rude enough to hang up.

Or Democrats which have an historically activist base – they are more likely to opt in to the survey because they feel freer to share their views – often because they believe they have the politically correct views. Yep, skews the daylights out of any poll. This is part of why we want likely voters over general public – strangely, willing to march in a protest doesn’t always translate into willing to vote.

And yes, cell phones also contribute to the problem. Getting a good number is just the start – by the time the pollster is finished they won’t have a random sample of cellphone users and will have no representation from those who don’t use cellphones of just don’t answer strange numbers.

My personal favorite pet peeve is the sample size. Back in the Stone Age, you know, the Eighties, we were taught that a minimum sample size should be 30% to get good representation and avoid weighting. No modern nationwide polls even remotely come close and I doubt any statewide polls do, either. The dominate poll nowadays is the micropoll. Used to, they tried for between 1000 and 1200 participants but lately I’m seeing below 800. This is insane.

Yes, micropolls can work but ONLY if the electorate is fairly stable – basically, everyone is going to vote pretty much the same way they did last election. Micropolls depend on a statistical balancing act called ‘weighting’. Using the past performance (i.e. the last election) as an indicator, statisticians assign new values to the poll results. It’s basically trying to fix a messed up scale by sticking your thumb on it with just the right amount of pressure. Can it work? Sure. Can you really muck it up? Absolutely.

But what happens if everyone gets all riled up about something between elections? It pretty much destroys the basis for the weights assigned because people aren’t acting the same as they did. A great example was the overturn of Roe. Polls showed a high level of upset because that’s what pollsters expected from their oversampled female and Democrat polls. Trouble was, this was a once in a lifetime thing – the past election isn’t good guidance here. Coming into the midterms, there’s literally no sign of any significant effect on the election. The polls measured something that wasn’t really there because they weren’t set up to handle the oddball event.

Yes, we can be confident that the polls then were off because it simply hasn’t been long enough nor has there been a big enough event to overshadow Roe. If the polls had been right, they would still be showing the same effect. They aren’t.

So, short answer. most polls are crap. All media polls are crap. If Gallup, Pew or Zogby have done the poll, It’s worth a look but even they are having trouble doing good polling in the modern world. it’s not even that the media is deliberately skewing polls – they stopped doing that after getting caught big time in the late Nineties – it’s just that there are a lot of hurdles and the huge number of polls make it unlikely that we’ll get better any time soon.

But the media will have plenty of filler. Too bad you can’t compost it.

Spread the word!

Author: Archena

Cranky old lady with two degrees in Political Science and she ain't afraid to use 'em!