Political Polling: Art Behind the Numbers

Everyone has read about the results of political polls. As we near any major election, the media breathlessly reports on the ups and downs of various candidates. And because they get so much attention, partisans from across the political spectrum love to attack polls. They point out where a poll may be lacking if their candidate of choice doesn’t look like the winner. Or they spin why the person who dominates the poll may have secret weaknesses that the survey did not capture.

 

When faced with all the ways polls get picked apart, why should a campaign do their own polling?

 

Let’s look at what polls are really for…  The most important use of political polling isn’t really for public consumption or for newspaper fodder. Instead, it’s for campaigns to set their strategy.

 

  • Polling can identify the size of a candidate’s base,
  • Opposition and swing groups that should be targeted, and
  • What positions and talking points resonate with voters.

 

But you can only use that data if you trust the results of your poll. That’s where science meets art in strong project design. A well-designed poll considers both the science of sampling and the art of stratifying the sample. An experienced researcher can help you through all the decisions you must make to achieve quality results within your polling budget.

 

Here are the most important factors to consider in designing a political poll:

 

Respondent Source

Where do you find the people to respond to your poll? The most common sources are random digit dialing, voter lists, and internet panels, and each has pros and cons.

Random digit dialing is the most expensive option but is often best for national samples as it gives everyone with a telephone an equal chance of participating. Voter lists are a popular way to reach respondents because they provide certainly that a household is on the voter rolls, and you know things like political affiliation if you are looking to survey say, registered Republicans. Internet panels are the most cost-effective way to poll likely voters, but often have limitations for recruiting respondents for an election taking place in a limited geographic area.

 

Data Collection Methodology

Once you’ve determined where you’re going to find people to poll, how do you collect all that data?

Three major methods dominate political polling: live telephone interviews, internet surveys, and interactive voice response polls.

Cost is a major driver of this choice – telephone polling is the traditional method, but it is the most expensive because of the need to pay for callers to conduct the interviews at phone centers. Internet and interactive voice response (IVR) are both less expensive because they are self-directed. Internet polling has an advantage as it allows researchers to show respondents stimuli such as candidate videos and advertisements. Meanwhile, IVR robocalls provide an easy to way efficiently contact voters for shorter surveys.

 

Sample Size

Determining the right number of interviews to conduct is one of the most important decisions in political polling project design. An experienced political pollster can help you decide how to maximize your polling dollars by surveying only the number of respondents you actually need to get accurate answers.

More interviews provide for a smaller margin of error and a greater ability to examine the data by subgroup but cost more for data collection. A typically national survey will have a sample size of roughly 1,000 voters with a margin of error at roughly plus or minus three percent at the 95th percentile confidence interval (meaning 19 out of 20 surveys will be within that range), providing a high degree of statistical accuracy. Typically, we recommend conducting at least 400 interviews for a political survey because it this is a point on the bell curve where the margin of error becomes manageable at plus or minus roughly five percent.

 

Screening

If you’ve determined how many people to contact, where that sample is coming from, and what technology you are using to reach out, you still have other decisions to make. Screening in a poll is important, because the definition of who is likely to vote in an election is part art and part science.

Determining whether to conduct a poll with registered voters or likely voters often depends on how close to the election the survey is conducted. Months out from an election, surveys are more apt to be conducted with registered voters since voters are not yet considering whether they will vote. Closer to an election, screening questions can be used to predict how likely a voter will be to participate in an election producing a survey of likely rather than registered voters. Those who are less likely to vote are then excluded from the survey and provide a turnout model of who is likely to participate in the election.

 

Sample Stratification

There is no subject matter on which people have a wider range of opinions than politics. And opinions differ greatly based upon demographics – a respondent’s location, gender, age, income, education, and race can make a huge difference in their outlook.

As a result, it is critical when planning a political poll to make sure that the sample is stratified by these demographics. By stratified, we mean making sure we have an appropriate number of respondents in key demographic groups to ensure the survey is representative of the entire electorate.

Typically, we will use targets to ensure we have a minimum number of respondents in each cohort that we believe needs to be represented. Determining exactly what these targets are is one of the critical decisions a researcher makes in the design of the polling project. Typically, these respondent minimums are set on only a handful of the most important variables, because recruiting difficult to reach subgroups can increase the cost of research significantly.

 

Sample Weighting

Because sample stratification is only possible over a limited number of variables and with limited accuracy, most political research is adjusted demographically using a statistical process called weighting.

Weighting involves using a mathematical algorithm which adjusts the importance of each respondent depending upon their demographic responses such that the entire sample represents a series of target demographic results.

The target demographic results are usually part of the turnout model and based on a mix of previous election results, polls, census data, and a researcher’s sense of how these demographics are changing over time. The demographic targets of the weighting scheme typically are one of the elements in a pollster’s secret sauce that they do not publicly disclose. Nonetheless, these weighting targets are critical in impacting the results of a political survey, as the demographic makeup of an electorate is often determinative of the outcome of an election.

 

Questionnaire Design

Finally, and perhaps most critically, questionnaire design must be done thoughtfully.

The wording of questions, and what information to include, can directly impact the results of a poll. A seasoned political researcher can decide if it is appropriate to include a candidate’s party or their current role, which can boost name recognition and impact results. Similarly, where the question is placed in a survey can also impact how respondents answer it.

Experience with questionnaire design is critical to get this step in the process right and provide accurate results that can drive smart choices for a campaign.

 

Andrew Ribner is Chief Research Officer at OvationMR and is responsible for Public Opinion Research, Online Panel and Online Sample Data Quality. He also consults with clients on research and sampling design for various types of online research campaigns and political polling. Andrew can be reached at andrew.ribner@ovationmr.com

Ovation MR is a global provider of first party data for those seeking solutions that require data for informed business decisions. Ovation MR is a leader in delivering insights and reliable data across a variety of industry sectors around the globe consistently for market research professionals and management consultants. Visit: https://www.ovationmr.com