Survey Length on Mobile
Among one of the more ubiquitous survey tips you’ll see is this: don’t make a long survey. The question is, though, what exactly does that mean?
Defining: “long” surveys
In the market research industry, you’ll find that “long” can range anywhere from 25 minutes to 45 minutes. It also doesn’t always take into account the number of open-ended text questions that are included in a self-administered survey.
With an increase of respondents accessing surveys via mobile devices (such as smartphones and tablets), I personally think “long” is becoming shorter. For example, I participated in a survey that was advertised in the email invitation as taking “about 15 minutes.” I decided to try it via my smartphone out of curiosity on how long that 15 minutes would feel.
The answer? At least for me, that 15 minutes felt long.
Interestingly, 15 minutes on a laptop doesn’t feel as long as on a smartphone. I can spend 30+ minutes on my smartphone doing a bunch of things: checking social media, reading articles, posting on social media, checking email, adding to a grocery list, adding to a to-do list. But that time is typically spent among a variety of apps, not entirely focused on one app. Even when I am focused on one app (such as Facebook), I’m reviewing multiple pages, reading articles, and watching videos. In other words – I’m interacting with the content, which is keeping me engaged in the content.
So why is it that I can spend that long on my smartphone without ever once looking up, but ask me to take a 15-minute survey on the same smartphone, and I feel like it’s taking forever? I think it boils down to this: engagement. Remember, the time I spend on my device is spent engaged in a variety of ways, with multiple media options. A survey is typically far less engaging than any social media site.
Increase engagement or decrease time?
When it comes to questionnaire design, increasing engagement can frankly be difficult to do. For starters, including videos in a questionnaire can be cool, but you’ll need to double- and triple-check a few items on multiple devices: load speed, ability to control playback, and rendering (does it auto-fit to the screen, or is the size static?). And when it comes to images, similar items need to be reviewed: load speed and screen fit.
If you can manage to get all of those items to work well on your survey (so that no matter the device, the survey content renders well), then please contact me and tell me your tips!
In other words, that leaves us with decreasing the length of the survey so that respondents don’t spend a lot of time on a survey on their mobile device.
“But how will I get all of the data that I need from my audience?” you might ask? My answer: first, broaden the scope of where and how you’re acquiring data. If you’re doing a survey about a purchase experience, can you tie the information about the purchase to the survey response so that you don’t have to ask the respondent to answer 5 questions about what it was they purchased and when? How about including that information in a QR code on a receipt so that the respondent just has to use the QR code question type, snap the code, and the data is already included in their response? You’ve now shortened the survey by a good 2 minutes or more.
Additionally, especially when it comes to mobile, I think it’s fair to say that you can field multiple, shorter surveys, and get a better response than if you field a single long survey. This is a great time to exercise the “what is it I actually need to know?” muscle when designing the survey project.
What about non-mobile?
Because of the impact mobile is having, I think “long” is becoming shorter for desktop and laptop devices, as well. I’m still surprised any time I see a survey that is reportedly going to take me 45 minutes. Sadly, this happens often. I think 25 minutes is the maximum length of time you’ll want to keep a self-administered online survey to minimize risk of drop-out. Unless your survey is incredibly engaging (and I have taken a handful of those in the hundreds I’ve taken in recent years), our attention just won’t hang in there for much longer. A great feature to increase your response rate and manage the lenght of your survey is survey timer. It tells your respondents how long it will take them to complete the survey, so they know what they are committing to and complete the survey.
What about open-ended text questions?
If you’re designing a survey with a lot of open-ended text questions, I’d recommend instead doing a focus group – either moderated online or else done in-person. Online surveys just aren’t the place for having many open-ended text questions. Too many will result in a decrease in response rates and an increase in drop-out rates. What is too many? A good rule of thumb is 1 open-ended text question per 5 closed-ended questions. And even then, just because you have 15 multiple choice questions doesn’t mean you can necessarily include 3 open-ended text questions. Qualitative data is very important, but acquiring it via an online survey should be done carefully. Other places to acquire qualitative data can be forums, social media, and as mentioned earlier, focus groups (both online moderated and in-person).
