What sampling methods should be in every practical researcher’s toolkit?
If you are looking to design a survey and have online research sampling methods considerations, or you are attempting any other type of research methodology…
you will often find yourself asking the following sampling-method related questions…
What types of sampling methods should you consider for research?
What kind of sampling method is an online survey?
How does each technique affect the sampling process?
What confidence should you have in the results?
Can you draw statistical inferences from the data to project results?
However, when it comes to considering types of sampling,
in research, you often have to take a step back and consider the big picture.
Begin with finding the answers to these kinds of questions…
- Will my product or service sell?
- Is my candidate’s message moving the needle?
- How will my employees respond to a compensation change?
- Which fundraising campaign will be most accepted?
These and many similar questions typically drive an organization’s desire to conduct market research, political research, or organizational research. If you are planning to embark on such an effort, one of the first questions you’ll need to answer is:
How you will collect your sample?
Collecting your sample implies you have a source or list of all the members of the target population from which to draw a sample and a method by which you will select the sample. A source can be any resource that contains the information required to contact every member of the target population. Examples may include…
- A list of registered voters from the county,
- A database of members of a national association,
- Geographically defined exchange blocks for computer-generated RDD telephone sample
- A representative online panel of a country or other defined group
These are also referred to as your sample frame.
Important things a research practitioner should consider during sampling frame selection are the following kinds of questions…
- How many responses do I need? (sample size). Are they the right people? (sample representative of a total population). There are several paths to achieve various degrees of precision for different population sizes, classes, and categories, which we discuss in more depth.
- How do I ensure all qualified subjects are included in the selection process? (sampling bias). How how do I avoid excluding subgroup subjects from the sampling frame? (selection bias), and properly use quotas to ensure all subgroup individuals are represented? (non-response bias).
- With what degree of precision (margin of error) can I forecast or state my conclusions? This sampling error is about the statistical precision of estimates from the target population, confidence interval, and sample size.
- All of which can be answered by your online panel company or research consultant.
We now have a clear picture of who we are looking for and
we know how many cases we need to acheive the level of precision for stating our findings.
And since we also know the degree of statistical certainty required to meet the standard required for the situation or case…
So now it’s time to explore...
What type of sampling methods are used in online research?
Simply stated sampling theory on the internet is not different than on the phone or via the mail. Yet we cling to these methodologies now for decades despite their high cost, non-response bias, and other inefficiencies, and there are still legitimate use cases for these tools today.
Because of the popularity of online research, principally due to the speed, relatively lower costs, and access to targeted audiences many research practitioners ask:
What kind of sampling method is an online survey?
The answer to this question highlights the main point of what this sampling article seeks to address. Which is that regardless of medium; internet, phone, mail, door to door, or mental telepathy – it is the way you sample and not the medium that is important. All mediums have limitations and strengths with regard to each sampling method for each given use case.
Which is why if you are doing online qualitative in,
- Cincinnati or,
or quantitative online surveys with,
- N=100 Alaskan crab fishermen or,
- Brand tracking with N=7000 13 to 39-year-old millennials every month,
any of those project samples may be online research, and each of those may be different sampling methodologies. A panel provider you work with can address how they design and build sample frames for your specific research.
Use this Sample Size Calculator to Estimate Your Sample Size
- Population total: Give the number of people that your survey results will be representing. This is not your sample size but the entire population you are trying to represent. Examples would be: The State of California (39.51 million) or Pacifc Gas & Electic (PG&E) customers (16 million). Enter your target population total.
- Margin of error: The margin of error represents how precise your results will be. The margin of error represents the difference of proportion from the true proportion that is acceptable to the researcher. In the social sciences, 5 percent is an acceptable margin of error. You can change the margin of error depending on your precision needs. If you have no preference, choose 5 percent default. Sample size increases as margin of error decreases.
- Confidence level: Select the reliability level. In statistics this a measure of the reliability of a result. A confidence level of 95 percent means there is a probability of at least 95 percent the result is within a given range. Increasing the confidence level will increase sample size.
What types of sampling methods for research should you consider for your study design?
The two primary methods employed by market research experts are probability and non-probability sampling.
Within each of these categories are subgroups that more clearly define how to obtain your sample, each carrying its own advantages and disadvantages to the researcher seeking the data.
What are probability sampling methods and how are they used?
In probability sampling, every individual in the entire population being considered has an equal chance of being randomly selected for the survey, interview, or questionnaire. The selection process is completely random, and therefore, the sample is likely to be closely representative of the whole population.
Probability sampling is best used in market research requiring quantitative results. Typically, the results of probability sampling involve a great deal of number crunching and statistical charts and graphs.
Four probability sampling methods used in market research
Simple Random Sampling Method
There are generally two principal ways to make a random selection when building a sampling frame. One standard method when the sample size is smaller is to use simple random sampling. Which gives every individual in the target population an equal chance of being selected by generating a series of random numbers. Or when you have a larger population, deploy a systematic approach described below or use cluster or stratified methods.
Cluster sampling involves two stages. In stage one, the market researcher selects a certain number of groups or clusters of people to question or interview. In stage two, a random sample within each cluster is selected for the actual study.
Cluster sampling works best in situations where a random sampling method of an entire population would be too expensive, impossible, or extremely complicated. This method is a less expensive and faster way to collect market research information. However, since it’s not a completely random sampling, you are more likely to generate sampling error.
One example of good use for this research sampling method may involve collecting customer preferences for a large, national hotel chain. It would be difficult, expensive, and time-consuming to collect information about every customer visiting every location of a hotel chain. However, you can select a dozen locations around the country in stage one of a cluster sample, and then randomly select guests at each of those 12 locations over the course of a month for your b2b research.
Perhaps you are collecting insights about new hotel services you’re thinking about adding. Customer preferences shared through such a cluster sample would probably be reasonably representative and usable for making such decisions. Precision is not extremely important in this case, and therefore, the cost and time savings would outweigh the need for conducting a completely randomized survey.
Stratified sampling is a method where the overall population is divided into mutually exclusive groups before a random sample is selected. You might want to sub-divide your group by gender, race, income levels, or age. Each person can only belong to one stratum or group.
Businesses or organizations looking for a high level of precision or the ability to analyze information within the smaller subgroups in addition to the overall population may want to invest in stratified sampling. Since a representative group will be selected from each stratum, the actual sample can be smaller, which will save time and money.
Depending on your population and research goals, you’ll want to decide if you will use a proportionate or disproportionate stratification. Proportionate stratification can increase your precision because the actual sampling fraction of people will be proportionate to your entire population, which may not be the case in a completely randomized sample.
Disproportionate stratification can help market researchers when there are significant variances among the strata. You may be able to gain precision for a particular survey measure; however, this precision may not carry across other components of the research.
Systematic sampling is an easy version of probability sampling because researchers select every nth individual on a population list. As long as the population list does not contain any pre-organization, the resulting sampling should be representative. This method of sampling is simple, fast, and effective in most market research situations. All that is necessary is a list of the population, a starting point, and a sampling interval.
For example, if you want to collect data from a trade association with 10,000 members, you can select every 100th person (sampling interval) on a membership roster to create a survey group of 100. One example of a potential problem with systematic sampling would be a list that is organized before the sample is selected.
For instance, if you are questioning coaches and players of an adult sporting league about tournament locations, and the list is made up of team sub-lists that always place two coaches followed by 20 team members, you run the risk of either soliciting feedback from all coaches or no coaches depending on your interval selection.
What are non-probability sampling methods and when are they used?
Non-probability sampling is, obviously, the opposite of probability sampling. Often referred to as purposive sampling collectively the selection process within the various methods is not random, and therefore, subject to research to greater bias and more sampling errors. The results of non-probability sampling are often helpful before or after a market research project involving probability sampling. For instance, the ideas generated can be used to create a quantitative survey for a randomized sample of your population, or a non-probability sample can help flesh out and clarify topics that come out of a randomized survey.
Three non-probability sampling methods used in market research
Convenience Sampling Method
Convenience sampling is a quick-and-easy way to select your research subjects. Because they are the ones most convenient for your particular research project. This factor means that it’s faster, easier, and cheaper to conduct your research. The major disadvantage is that (depending on the type of convenience sampling you are using) you can introduce significant bias or sampling errors using this method.
One common example of convenience sampling is in a university setting where graduate students use volunteer undergraduate students as subjects for experiments. In other cases, a researcher may select the people who happen to shop at a particular store on one day, mall shoppers, or the first dozen clients on a business’ customer list.
There are hybrids for convenience samples in online research where you draw random samples from a universe of participants with certain characteristics using behavioral data and other targeting methods. It’s a targeted convenience sample but still random. The question becomes for low incidence categories; is it really better to screen through 10,000 people to get 100 people who qualify for a 1 percent incidence study? Most likely it’s cost and time prohibitive if you consider all the pros and cons.
Quota sampling is a non-probability sampling method that can be valuable in particular market research efforts. Using this technique, a researcher will sub-divide a cohort to trial a specific group within that cohort. The best use of quota sampling is to research a particular trait within a larger group or how one trait affects another trait in the same group.
For example, if a researcher wishes to examine the disease profile of a group of senior citizens across gender, age, or socioeconomic lines, quota sampling may be ideal. The main disadvantage of this technique is that is limited to the traits that you’re studying. Other factors within the sample may be over- or under-represented, and therefore, the scope of your results will be limited. Researchers should be careful about generalizing traits outside the actual trial to the larger population.
Judgmental sampling is a technique where researchers use their own judgment to select a sample based on personal knowledge and expertise. This produces a bias in the sample, but purposive sampling research methods can be useful when studying very specific groups within the population.
For example, if a researcher is collecting insights on patients suffering from a rare disease, it would make more sense to find those individuals directly. The advantage of judgmental sampling is that you question the exact type of person you’re seeking for your research project. The major disadvantage is that you will most likely introduce human error and researcher bias into the results. As a result, it would be unwise to make generalizations to a greater population based on the results of a judgmental sample study.
Sampling methods conclusions
Once you’ve determined the goals and objectives of your market research project, you can make a smarter decision about which type of sampling methodology to use. As you can see from this primer, you need to balance your requirements for precision against the cost and time requirements of each sampling method. If you’re seeking quantitative results, it’s best to use one of the probability sampling methods or hybrids.
If you are looking for qualitative information, one of the non-probability sampling methods may deliver the information you need at a much lower cost and time investment. Working with a market research expert can help you better match your research requirements to the research sampling method that will give you the greatest return on your investment.
Jim Whaley is CEO of OvationMR and posts frequently on The Standard Ovation and other Industry Blogs. Ovation MR is a global provider of first-party data for those seeking solutions that require information for informed business decisions. Ovation MR is a leader in delivering insights and reliable results across a variety of industry sectors around the globe consistently for market research professionals and management consultants. Visit: https://www.ovationmr.com.
You might also like…
Understanding Types and Sources of Sampling Bias in Online Research What is sampling bias? In an effort to describe what is sampling bias and what are the various types including selection bias and data bias this article will provide examples...
When designing a market research study and considering goals and objects the next step is to determine the target population to be studied. The target population the entire population, or set, that will be considered qualified for data...
A vital component of any successful marketing campaign is having an in-depth knowledge of the market you're trying to target. Time and effort taken to identify and learn about a market will pay dividends when it comes to making real...