Last week, QuestionPro hosted a webinar about increasing response rates. During the webinar, some tips were given about survey design that can contribute to your audience being more likely to take the survey. Let’s review those and look at a few more tips on survey design to get more responses from your audience.
From the webinar
The tips about survey design from the webinar were:
- Use multiple-choice as opposed to text boxes (note: this doesn’t mean to completely ignore using open-ended questions, but rather be sure that you aren’t overloading the survey with them)
- Include an “other” option to help your respondents have a voice
- Avoid huge tables with lots of choices.
These are great tips! One of my pet peeves for surveys are large matrix questions (large meaning 15+ rows). If your respondents have to scroll down to keep answering the questions included in the table, you’re running the risk of the respondent losing interest, and thus getting data that isn’t accurate. And I’ve often taken surveys where I wished there had been an “other” option because none of the options presented applied. This has been especially frustrating when the question required an answer. In some cases, I dropped out of the survey because I didn’t want to provide false data just to keep answering!
More tips for increasing response rates via survey design
I was curious about other survey design tips that other researchers have found increase response rates. In a summary of an article published in the journal Social Science Computer Review in 2010, authors Paula Vicente and Elizabeth Reis examined some web survey design items and how they affected nonresponse (in other words, how they should be done to increase responses). Here’s a general overview of what they found.
- Opt for multiple pages instead of scrolling. While the difference in response rates between the two designs was not statistically significant, they observed more items were skipped when respondents scrolled as opposed to going to more pages of the survey.
- Attaching a survey for respondents to email back gained far fewer responses than a link included in an email. In other words: make it as easy as possible for respondents to access your survey.(Note: this is from research published in the past 5 years, so this is relevant advice for any survey being done today.)
- Length matters. However, being upfront about how long your survey will take (and being accurate about it) will reduce drop-out rates (people who start but don’t finish the survey).
- Including some type of progress indicator is helpful. Interestingly, they found that for long questionnaires, having a permanently-displayed progress bar actually led to an increase in drop-out rates. Showing the same progress bar intermittently during the progress of the survey was better. Generally, though, the research design wasn’t too clear on this, so rule of thumb: show progress in the survey somehow, especially for longer surveys.
- Keep the survey simple in terms of visual design. When the survey is image-heavy, you can affect download speeds, which increases drop-out rates. So keep the survey simple, from design to the number of images and videos you include in the survey. (And a plug here: test your surveys on multiple devices to check things like download speeds, especially when using multimedia in your surveys.)
- Find ways to keep your survey engaging. This could include any type of hover-text to help explain something to your respondents, “next” buttons, text indicating when a respondent missed a question, and piping text into the survey so that the questions are updated based on answers given prior in the survey. And while there isn’t much research showing actual impact of these types of items in surveys, one trick – requiring responses – does work, though it also increased drop-out rates, so use these judiciously.
- Keep the open-ended text to a minimum in an online survey – but don’t exclude them entirely. Too many open-ended text questions leads to increased drop-out rates. A good rule of thumb is about a 1:5 ratio (5 closed-ended questions, 1 open-ended question).
- If you can keep the answer type to a radio button over a text input (such as for an “other” option) or a drop-down list, go for the radio buttons. Again, the key is to make the survey as simple as possible for your respondents to answer.
LEARN ABOUT: Open-Ended Questions
As you may have heard in the webinar, keeping things simple is the best general rule for questionnaire design. This can be easy to overlook, but it can have a significant impact on your response rates. What have you found increases or decreases response rates for questionnaires?
LEARN ABOUT: Survey Sample Sizes
Here is an other article to design a survey people will love to take.
