In a previous post about designing effective mobile surveys, we had highlighted an important statistic – that for any given project market researchers can now expect up to 40% survey response on smartphones. This percentage will continue to increase. The question then becomes one of what do we know about mobile respondents and can this be applied to our sampling plans?
Mobile Respondents
First and foremost, mobile responders are less likely to actually complete the surveys they start. Depending on the source cited people taking surveys on smartphones are between 2.6 and 4.0 times MORE likely to drop out of a survey before completion. Mobile surveys can help reduce this rate, but it is wise to oversample if you want a balanced sample. Consider oversampling by a rate of 3:1 for mobile respondents compared to desktop or tablet respondents.
Speeding is another behavior to watch with mobile respondents. There is conflicting evidence regarding the completion time. Some sources report that they will complete their surveys in less time than desktop or tablet respondents. Other authors indicate the opposite. The question raises – does a faster response time mean lesser quality data? If you are asking for gut reactions or attitudes on familiar topics then speed is fine. However, in other cases, speed can easily equate to not reading instructions. It may be unreasonable to expect mobile respondents to complete the same discrete choice modeling exercise as their desktop counterparts.
In regard to the discrete choice, it is possible to lighten the load by reducing the number of tasks but exposing the experiment to a larger set of respondents. This would make for a more effective approach if you are allowing mobile respondents into the survey.
A final point of consideration is the need to begin using the response device as a variable for screening data quality. Would you assume differences between mobile and non-mobile respondents makes the mobile respondents of lesser quality? We need to ascertain if mobile respondents are different because they are mobile or if they are from a different demographic. Most, if not all, online survey platforms will export device as a variable. Are you using it in your data quality assessments? If not, then you should be!
