Political Polling | The Art of Science

Political Polling: Art Behind the Numbers

 

Everyone has read about the results of political polls. As we near any major election, the media breathlessly reports on the ups and downs of various candidates. And because they get so much attention, partisans from across the political spectrum love to attack polls. They point out where a poll may be lacking if their candidate of choice doesn’t look like the winner. Or they spin why the person who dominates the poll may have secret weaknesses that the survey did not capture.

 

When faced with all the ways polls get picked apart, why should a campaign do its own polling?

 

Let’s look at what polls are really for…  The most important use of political polling isn’t really for public consumption or newspaper fodder. Instead, it’s for campaigns to set their strategy.

 

• Polling can identify the size of a candidate’s base,
• The campaign should target opposition and swing groups, and
• The campaign positions and talking points resonate with voters.

 

But you can only use that data if you trust the results of your poll. That’s where science meets art in strong project design. A well-designed poll considers both the science of sampling and the art of stratifying the sample. An experienced researcher can help you through all the decisions you must make to achieve quality results within your polling budget.

 

Here are the most important factors to consider in designing a political poll…

 

Respondent Source

 

Where do you find the people to respond to your poll? The most common sources are random digit dialing, voter lists, and internet panels, and each has pros and cons.

Random digit dialing is the most expensive option but is often best for national samples as it gives everyone with a telephone an equal chance of participating. Voter lists are a popular way to reach respondents because they provide certainty that a household is on the voter rolls. You know, things like political affiliation if you are looking to survey, say, registered Republicans. Internet panels are the most cost-effective way to poll likely voters but often have limitations for recruiting respondents for an election in a limited geographic area.

 

Data Collection Methodology

 

Once you’ve determined where you will find people to poll, how do you collect all that data?

Three major methods dominate political polling: live telephone interviews, internet surveys, and interactive voice response polls.

Cost is a major driver of this choice – telephone polling is the traditional method, but it is the most expensive because of the need to pay for callers to conduct the interviews at phone centers. Internet and interactive voice response (IVR) are both less expensive because they are self-directed. Internet polling is advantageous as it allows researchers to show respondents stimuli such as candidate videos and advertisements. Meanwhile, IVR robocalls provide an easy way to contact voters efficiently for shorter surveys.

 

Sample Size

 

Determining the right number of interviews to conduct is one of the most important political polling project design decisions. An experienced political pollster can help you decide how to maximize your polling dollars by surveying only the number of respondents you actually need to get accurate answers.

More interviews provide a smaller margin of error and a greater ability to examine the data by subgroup but cost more for data collection. A typically national survey will have a sample size of roughly 1,000 voters with a margin of error at roughly plus or minus three percent at the 95th percentile confidence interval (meaning 19 out of 20 surveys will be within that range), providing a high degree of statistical accuracy. Typically, we recommend conducting at least 400 interviews for a political survey because this is a point on the bell curve. The margin of error becomes manageable at plus or minus roughly five percent.

 

Screening

 

If you’ve determined how many people to contact, where that sample is coming from, and what technology you are using to reach out, you still have other decisions to make. Screening in a poll is important because the definition of who is likely to vote in an election is part art and part science.

Determining whether to conduct a poll with registered voters or likely voters often depends on how close to the election the survey is conducted. Months out from an election, surveys are more apt to be conducted with registered voters since the general public is not considering whether they will vote. Closer to an election, you can use screening questions to predict how likely a voter will participate in an election producing a survey of likely rather than registered voters. Those who are less likely to vote are then excluded from the survey and provide a turnout model of who is likely to participate in the election.

 

Sample Stratification

 

There is no subject matter on which people have a wider range of opinions than politics. And opinions differ greatly based upon demographics – a respondent’s location, gender, age, income, education, and race can make a huge difference in their outlook.

As a result, it is critical when planning a political poll to make sure that the sample is stratified by these demographics. By stratified, we mean making sure we have an appropriate number of respondents in key demographic groups to ensure the survey represents the entire electorate.

Typically, we will use targets to ensure we have a minimum number of respondents in each cohort that we believe needs to be represented. Determining exactly what these targets are is one of the critical decisions a researcher makes in designing the polling project. Typically, these respondent minimums are set on only a handful of the most important variables because recruiting difficult to reach subgroups can significantly increase the cost of research.

 

Sample Weighting

 

Because sample stratification is only possible over a limited number of variables and with limited accuracy, most political research is adjusted demographically using a weighting statistical process.

Weighting involves using a mathematical algorithm that adjusts the importance of each respondent depending upon their demographic responses such that the entire sample represents a series of target demographic results.

The target demographic results are usually part of the turnout model and are based on a mix of previous election results, polls, census data, and a researcher’s sense of how these demographics are changing over time. The demographic targets of the weighting scheme typically are one of the elements in a pollster’s secret sauce that they do not publicly disclose. Nonetheless, these weighting targets are critical in impacting the results of a political survey, as the demographic makeup of an electorate is often determinative of the outcome of an election.

 

 

Questionnaire Design

 

Finally, and perhaps most critically, questionnaire design must be done by researchers thoughtfully.

The wording of questions and what information to include can directly impact the results of a poll. A seasoned political researcher can decide if it is appropriate to include a candidate’s party or their current role, boosting name recognition and impact results. Similarly, where the question is placed in a survey can also impact how respondents answer it.

Experience with questionnaire design is critical to get this step in the process right and provide accurate results that can drive smart choices for a campaign.