By Frank Blechman
A weekly column is an invitation to preach. As a former teacher, I strongly feel the pull to lecture. Yet I like to think that for the most part, I have used these columns to share my perspectives and opinions, rather declare my eternal truths.
This week’s column is different. I feel a need to explain some things about political polling. Coverage in the media of this year’s elections, and questions asked of me by those involved in campaigns, indicate that many people who should know better fundamentally do not understand how polling works or what it is.
Many people who should know better fundamentally do not understand how polling works.
I was never formally trained as a research methodologist, but I have been around political polling for over 50 years, and have picked up a few things. If you understand how polling works, you can skip the rest of this column.
- Polls do not “show” or “prove” or even “say” anything. When the media reports that a recent survey demonstrates something or other, they are mis-stating the facts. A survey is a snapshot in time. Whatever the data may capture about the moment (or more accurately, the time period in which the results were collected), those data do not predict what will happen in a different period, any more than a result of a baseball game predicts the results the next time. Polls are samples that at best “suggest” certain things. The poll data indicates a range of possibilities
- Any sample has the possibility of error. The larger the sample, the more statistical confidence we can have in the results. Beyond statistics, however, the validity, predictive power, and usefulness of poll data can vary a lot. Here are a few of the most important factors.
The pool. Simply put, does the makeup of the sample match the actual total population? If not, the sample cannot represent the larger group. In politics, we do not know what the final pool of voters will be. So, we guess. We construct models. We imagine that demographic “A” (registered voters under age 30, for example) will turn out at a certain rate, while demographic “B” (voters age 30 or over) will vote at a different rate. Then, we try to get participants in our survey who match that model. If the guess is wrong, the results of the poll will be less valid.
The sample. Fewer people answer their hones or reply top written interviews these days than did in the past. So, far more people need to be approached to get the desired number of responses. That raises the costs. If the respondents in the sample do not match the model in the projected pool, and we not want to spend the money tom get the right balance, mathematical adjustments will have to be made to weight some of the answers more than others. This multiplies the problems created if our projected pools is wrong.
Veracity. People who have opinions that they know are unpopular. Sometimes do not answer polls honestly. This is a small factor, but one that cam be critical in a close race (see polling in MI, WI, or PA in the 2016 Presidential race).
The interpretation. Folks want to know, “What does this mean?” “What should I DO?” Pollsters are under pressure from clients to interpret all the numbers in a way pleasing to those paying the bills. Pollsters try to resist this pressure, but are not always successful. Usually, this is not a case of lying or distorting data. More likely, this involves emphasizing the positive and downplaying the negative.
- Even a series of polls taken over time cannot really show a trend or reliably predict future actions. Charts with curves seem impressive, but all they tend to show is that results oscillate based on many complex changes of environment and perception.
In Virginia, we have suffered with polling errors illustrating every one of these factors. Some of the worst problems have come from internal over-interpretation of questions about issues. For example, a poll may suggest that more (likely) voters express passion for a certain position on an issue (such as reproductive choice or gun control) than for an alternative one. A candidate may then be advised to emphasize that issue in the campaign in a way that aligns with popular passion. The risks are that while people care about the issue a lot, and want to hear a candidate affirm support for their position, they may not need or want to be beaten over the head with it. A mention affirming the candidate’s values might do, leaving time for the campaign to help people make up their minds about other, less settled issues. Further, those supporting the position may already be committed to this candidate. The issue may not be as important to swing or undecided voters.
[Voting by mail] adds another level of uncertainty. We not only don’t know who will vote, we don’t know when a person will vote. That means that late-breaking events and last-minute appeals will have less impact on election day than in the past because a large proportion of votes will have already been cast.
Finally, this year we expect a record number of people to vote by mail, rather than in-person on ‘election day.’ From our current perspective, this adds another level of uncertainty. We not only don’t know who will vote, we don’t know when a person will vote. That means that late-breaking events and last-minute appeals will have less impact on election day than in the past because a large proportion of votes will have already been cast. It makes a poll taken in July or August even less reliable as a predictor of outcomes in November.
Depending on our point of view, as political voyeurs, we can enjoy media reports about the ‘horse race’ (who’s ahead, by how much) as entertainment, but should not get carried away by them.
Instead, we should focus on questions much closer to home, such as “What can I do to help my family, friends, and neighbors vote in the 2020 elections?” “What can I do to ensure that this election is safe and credible for voters, and decisive for my preferred candidates?” “What can I do to help my candidate stay grounded?”
Polls tell us about great foggy clouds of probabilities. Our own personal actions can be a lot clearer than that.