Sampling Bias Considerations for Online Research

Jim Whaley

Understanding Types and Sources of Sampling Bias in Online Research

Reach Your Target Market
Market Research » Sampling Bias Considerations for Online Research

 

What is sampling bias?

 

To describe what is sampling bias is and what are the various types including selection bias and data bias this article will provide examples as well as provide explanations. Therefore sampling bias occurs when a probability or random sample is collected in such a way that some members from the set of the target population have a lesser chance of being selected than others, or with a probability of no chance at all.

This causes your random sample to be biased.  As with all forms of sampling, when conducting online research it is likely that a true randomly selected sample would be either impractical or impossible to achieve. It is more likely that a balance of sample selection measures is taken to achieve your sample size which will also yield broad representational coverage of the target population. Like any non-random sample of a population in which all participants were not equally likely to have been selected, it is essential to account for this in the sampling error rather than the study results.

 

 

 

What are the sampling bias types in online research?

 

There are two primary categories of sampling bias in online research and they fall into the area of things concerned with:

  • selection of the population being sampled whether it is a coverage issue related to the design of the sample frame, self-selection bias, or non-response bias which can be due to a variety of factors
  • data loss or review of record removals due to screener issues, questionnaire errors, or perceived data quality issues during the fielding and editing stage.

 

Selection Bias Data Bias
Coverage Bias Screener Bias
Non-Response Bias Questionniare Bias
Self Selection Bias Data Review Bias

 

 

What is sampling selection bias in online research?

Coverage Bias

Sample selection bias may occur at multiple levels throughout the research process. Coverage bias is concerned with the sample design and the plan to execute it.

Here are some areas to pay close attention to as you design and execute your study:

  • Excluding segments of the target population during sampling frame design.
  • Using a panel that does not provide adequate coverage for study population and sample frame.
  • Not providing accessibility to all target individuals either because of day, time of day, device, disability, or economic circumstances.

Non-Response Bias

Non-response bias may occur for a variety of reasons and is problematic for any study.  This type of bias can cause significant sample underrepresentation of segments of the target population, regardless of sampling method.

Here are some of the reasons which contribute to non-response bias:

  • Excessive survey length
  • Concern for loss of anonymity, fear or judgment
  • Study topic or sensitive subject matter
  • Availability of respondent during fielding periods
  • Economic, cultural or social pressures

Self Selection Bias

Like non-response bias self selection or voluntary response bias creates a disproportionate representation of response from the population. It differs in that it occurs most frequently due to a participant’s affinity to the subject matter.

Here are some common observations to watch out for with self selection bias:

  • Does that online panel you use allow anyone to join or is it invitation only?
  • Is the study link and open link that anyone can self select to participate?
  • Do individuals participating have a vested interest in the outcome of the poll or research?
  • Polarizing Topic

 

 

What is sampling data bias in online research?

 

Another area to control for sampling bias when conducting online research is during the questionnaire design, fielding and data review process:

  • Questionnaire Design Considerations and choices are extremely important to how screeners are designed and configured in order to ensure they both qualify the correct participants and do not lead non-qualified participants to qualify for a study inadvertently or deliberately. A survey should be organized in a logical way and be of reasonable length as to avoid respondent drop-offs which cause non-response complications.
  • Proper Field Sequencing and Targeting of the target population is critical. For example, it is important that you don’t fill up a total male quota-cell without paying attention to the male/teen sub-quota.
  • Data Edit and Review often yields a wide range of record removals, which varies on a case by case basis due to study dynamics and client preference. Be careful when removing cases either because you don’t like open-end responses, or the respondent did not provide a “Hemingway” quality response to the question. Removal of data for a single-dimension only creates sampling bias. Not all respondents provide open-end responses, (especially in the mobile medium) or have something to say concerning every question you may present to them. Data cleaning and edits are part of any research process, just be sure it is for the right reasons and understand the tradeoffs.

About Us

Jim Whaley is CEO of OvationMR and posts frequently on The Standard Ovation and other Industry Blogs. Ovation MR is a global provider of first-party data for those seeking solutions that require information for informed business decisions. Ovation MR is a leader in delivering insights and reliable results across a variety of industry sectors around the globe consistently for market research professionals and management consultants. Visit: https://www.ovationmr.com.

Need help with your project?

 

We are ready to get to work for you by providing:

N

Our latest Panel Book, ESOMAR28 response

N

An open dialogue about your requirements

info@ovationmr.com

+1.212.653.8750

N

A project estimate/proposal

39 Broadway, Suite 2010, New York, NY 10006 USA

M-F: 8am-5pm, S-S: Closed

OvationMR Case Studies sampling bias logo

OvationMR Market Research

           Visit Us:

 

           39 Broadway

               Suite 2010

New York, NY 10006

 

OvationMR Market Research

     Visit Us:

  • 39 Broadway
  • Suite 2010
  • New York, NY 10006

Share This