Sample Size for Your Survey
Determining the right sample size for your survey has become one of the most common questions for DIY market researchers. The process for determining the best sample size to collect so that you can make a good decision isn’t as complicated as you might think (or remember from your stats classes).
Because there is no magic bullet or single number, there are a few things you’ll want to have determined before you start figuring out what your sample size is:
- Your goals and objectives – What do you hope to do with the survey. Are you planning on projecting the results onto a whole demographic or population? Do you just want to see how a specific group thinks? Are you trying to make a BIG decision or just set a direction. If you’re going to be PROJECTING the results of your survey on a larger population — then sample size is critical and you’ll want to make sure that it’s balanced and reflects the population. If you’re just trying to get a feel for preferences – then it’s not as critical.
- How precise do you need or want to be? How CLOSE do you want the survey results to mimic what the true value would be — if EVERYONE responded? Again, if this survey is going to determine how you’re going to spend millions of dollars — you want to be VERY precise. The more precise you want to be, the more sample you’re going to want to have and the more your sample will have to represent the overall population. So if your population is small, say 200 – then you may want to do a complete census rather than get just a sample.
- How confident or sure do you want to be in the results? Think of Confidence from the perspective of RISK. How much risk are you willing to take on? This is where your Confidence interval numbers become important. How confident do you want to be? 98% confident? 95% confident. Understand that this confidence percentage that you choose has a BIG impact on the number of completions you’ll need and that can possibly increase the length of the survey, how much sample you ultimately have to get and that means increased costs to your survey. It helps to understand the actual numbers behind the percents and the costs associated with reaching them. For example – if you want to be 99% confident and that means collecting an additional 1000 respondents and this means paying for sample or keeping your survey running for an additional week or so — you will have to ask yourself if you’re willing to make that tradeoff for the precision.
- What kind of variability are you looking at? In other words, how similar or different is the population. If you are surveying consumers on a broad topic – then you may have LOTS of variability and will need MORE sample. But if you’re surveying a fairly homogenous population, then your variability will be less and you can sample FEWER people. So more variability equals more sample and less variability equals less sample. If you’re not sure, you can start with 50% variability.
- Estimate your response rate. Of course you want everyone to respond to the survey, but we know that isn’t going to happen. Your response rate will depend on how engaged your sample or population is with your product, service organization or brand. The higher the response rate, the more engaged your list. Your BASE sample size is the number of responses you MUST get. In order to meet those goals, you will have to increase your list size, remind people to complete the survey and also look at your survey structure. Sometimes people will not respond to long surveys, for example. Then if you reduce the number of questions in your survey, your response rate may increase.
Getting to the magic number
Now you are ready to figure out your sample size!
Here is a reference chart you can use to get started: