The importance of sample size determination

Are you ready to survey your research target? Research surveys can help you gain insights from your target audience. The data you collect gives you insights to meet customer needs, leading to an increase in sales and customer loyalty. Sample size determination is imperative to the researcher to determine the right number of respondents, keeping in mind the quality of the research study.

So, how do you know who should get your survey? How do you decide on the number of the target audience?

Sending out too many surveys can be expensive without giving you a definitive advantage over a smaller sample. But if you send out too few, you won’t have enough data to draw accurate conclusions.

This is where knowing how to determine sample size can give you an edge over your competitors. Let’s take a look at what a good sample includes. Also, let’s look at the sample size determination formula so you can use the perfect sample size for your next survey.

Select your respondents

What is a population sample?

A population sample is a fraction, or sampling, of a group that represents the usual attributes of the entire group. Determining the size of your population sample is equally as important as the sample size.

Establishing the right population sample will help your survey gain credibility and reach research goals. Population samples allow you to quickly define the different characteristics of each population.

Setting up population samples in advance is economical for your business. Each time you need to send a survey, you can use your population samples as a starting point in sample size determination. This reduces the time it takes to conduct market research surveys.

What is ‘sample size’?

A ‘sample size’ is a market research term used for defining the number of individuals included to conduct research. You’ll need to choose your sample based on certain characteristics or demographics, such as age, gender, or physical location.

Your sample can be vague or specific. For example, you may want to know what people within the 18-25 age range think of your product. Or, you may only require your sample to live in the United States, which gives you a wide range of the population. The total number of individuals in a particular sample is the sample size.

A large sample size can yield better and more accurate study results. Research is a multi-step process that allows you to achieve the desired results if all the steps are conducted in a systematic manner.

The most important of all the steps is determining how many responses are actually required to be able to derive a conclusive result. Larger sample sizes tend to increase the precision of your study.

For example, you want to know the proportion of children infected by a certain pathogen within a specific community. We would generally get a more precise estimate of this proportion if we chose to sample 300 children rather than 50.

Let’s say you are a market researcher in the US and want to send out a survey or questionnaire. The purpose of the survey is to understand the feelings of your audience toward a new cell phone you are about to launch. You want to know what people in the US think about the new product to predict the success or failure of the phone before launch.

Hypothetically, you choose the population of New York, which is 8.49 million. You use a sample size determination formula to choose a sample of 500 individuals that fit into the consumer panel requirement. You can use the responses to help you determine how your audience will react to the new product.

However, knowing how to determine a sample size requires more than just throwing your survey at as many people as you can. If your sample size is too big, it could be a waste of resources, time, and money. A sample size that’s too small doesn’t allow you to gain maximum insights, leading to inconclusive results.

Select your respondents

What are the terms used around the sample size?

Before we jump into sample size determination, let’s take a quick look at the terms you should know:

  1. Population Size: Population size is how many people actually fit your demographic. For example, you want to get information on doctors residing in North America. Your population size is the total number of doctors in North America. Don’t worry! Your population size doesn’t always have to be that big. Smaller population sizes can still give you accurate results as long as you know who you’re trying to represent.
  2. Confidence Level: Confidence level tells you how sure you can be that your data is accurate. It is expressed as a percentage and aligned to the confidence interval. For example, if your confidence level is 90% then it is most likely that your results will be 90% accurate.
  3. The Margin of Error (Confidence Interval): When it comes to surveys, there’s no way to be 100% accurate. Confidence intervals tell you how far off from the population means you’re willing to allow your data to fall. A margin of error describes how close you can reasonably expect a survey result to fall relative to the true population value.
  4. Standard of Deviation: Standard deviation is the measure of the dispersion of a set of data from its mean. It measures the absolute variability of a distribution. The higher the dispersion or variability, the greater the standard deviation and the greater the magnitude of the deviation. For example, you have already sent out your survey. How much variance do you expect in your responses? That variation in response is the standard of deviation.

What is the sample size determination formula? Learn how to calculate a sample size

With all the necessary terms defined, it’s time to learn how to determine sample size using a sample calculation.

Your confidence level corresponds to a Z-score. This is a constant value needed for this equation. Here are the z-scores for the most common confidence levels:

90% – Z Score = 1.645

95% – Z Score = 1.96

99% – Z Score = 2.576

If you choose a different confidence level, there are various online tools that can help you find your score.

Necessary Sample Size = (Z-score)2 * StdDev*(1-StdDev) / (margin of error)2

Here is an example of how the math works assuming you chose a 90% confidence level, .6 standard deviation, and a margin of error (confidence interval) of +/- 4%.

((1.64)2 x .6(.6)) / (.04)2

( 2.68x .0.36) / .0016

.9648 / .0016

603

603 respondents are needed and that becomes your sample size.

How is a sample size determined?

Determining the right sample size for your survey is one of the most common questions researchers ask when they begin a market research study. Luckily, sample size determination isn’t as hard to calculate as you might remember from an old high school statistics class.

There is no magic bullet or single number that works in every situation. Before you can determine your sample size, make sure you have these things in place:

Your goals and objectives: What do you hope to do with the survey?  Are you planning on projecting the results onto a whole demographic or population? Do you just want to see how a specific group thinks? Are you trying to make a big decision or just set a direction?

If you’re going to be projecting the results of your survey on a larger population, the sample size is critical. You’ll want to make sure that it’s balanced and reflects the population as a whole. If you’re just trying to get a feel for preferences, then the sample size isn’t as critical.

For example, you’re surveying homeowners across the US on the cost of cooling their homes in the summer. A homeowner in the South probably spends a lot more money cooling their home in the humid heat than someone in Denver, where the climate is more dry and cool. For the most accurate results, you’ll need to get responses from people in all areas and climates of the US. If you only collect responses from one extreme, such as the warm South, your results will be skewed.

How precise must you be? How close do you want the survey results to mimic what the true value would be if everyone responded? Again, if this survey is going to determine how you’re going to spend millions of dollars, then your sample size determination should be very precise. The more accurate you need to be, the more sample you’re going to want to have and the more your sample will have to represent the overall population. If your population is small, say, 200 people, then you may want to survey the entire population rather than cutting it down with a sample.

How confident or sure do you want to be in the results? Think of confidence from the perspective of risk. How much risk are you willing to take on? This is where your Confidence Interval numbers become important. How confident do you want to be — 98% confident, 95% confident? Understand that the confidence percentage you choose has a big impact on the number of completions you’ll need for accuracy. This can possibly increase the length of the survey and how many responses you need, which means increased costs to your survey. Knowing the actual numbers and amounts behind percentages can help make more sense of your sample size needs vs survey costs. For example, you want to be 99% confident. After using the sample size determination formula, you find you need to collect an additional 1000 respondents. This, in turn, means you’ll be paying for samples or keeping your survey running for an additional week or two. You have to determine if the increased accuracy is more important than the cost.

What kind of variability exists in your population? In other words, how similar or different is the population?

If you are surveying consumers on a broad topic you may have lots of variations. You’ll need a larger sample size to get the most accurate picture of the population. 

However, if you’re surveying a population with similar characteristics, your variability will be less and you can sample fewer people. More variability equals more sample and less variability equals less sample. If you’re not sure, you can start with 50% variability.

Estimate your response rate: You want everyone to respond to your survey. Unfortunately, every survey comes with targeted respondents who either never open the survey or drop out halfway. Your response rate will depend on how engaged your population is with your product, service organization, or brand.

The higher the response rate, the higher the level of engagement from your population. Your base sample size is the number of responses you must get for a successful survey.

In order to meet the goal of your survey, you may have to try a few methods to increase the response rate, such as:

  • Increase the list of people who receive the survey.
  • Use multiple distribution channels, such as SMS surveys, website surveys, and email surveys to reach a wider audience.
  • Send reminders to survey participants to complete the survey.
  • Offer incentives for completing the survey, such as an entry into a prize drawing or a discount on the respondent’s next order.
  • Consider your survey structure and find ways to make your questions more straightforward. The less work someone has to do to complete the survey, the more likely they are to finish it. For example, longer surveys tend to have lower response rates due to the length of time it takes to complete the survey. In this case, you can reduce the number of questions in your survey to try to increase responses.

Select your respondents

Consider Your Audience: In addition to the variability within your population, you need to make sure your sample doesn’t include people who won’t benefit from the results. One of the biggest mistakes you can make in sample size determination is forgetting to consider your actual audience. You don’t want to send a survey asking about the quality of local apartment amenities to a group of homeowners, for example.

Focus on the objectives of your survey and what you hope to gain to get a more accurate idea of who to target. You may start with general demographics and characteristics, but can you narrow those characteristics down even more? Narrowing down your audience makes it easier to get a more accurate result from a small sample size.

For example, you want to know how people will react to new automobile technology. Your current population includes anyone who owns a car in a particular market. However, you know your target audience is people who drive cars that are less than 5 years old. You can remove anyone with an older vehicle from your sample because they’re unlikely to purchase your product.

Once you know what you hope to gain from your survey and what variables exist within your population, you can decide how to determine sample size. Using the formula for sample size is a great starting point to get accurate results. After calculating your sample size, you’ll want to find a reliable survey software platform to help you accurately collect survey responses and turn them into analyzed reports.