It’s a mantra we hear often, but we seem to forget as often as we hear it: test your surveys before you go live with them. A little time testing up front can save you a lot of money redoing survey projects in the end, or worse, getting and acting on the wrong information to guide your business decisions.
Testing: a best practice
Testing your surveys might be the single best practice when it comes to the laundry list of best practices associated with survey design. I am just as guilty as the next person of failing to include any time for testing a survey in my project schedule. It can take so long tweaking the questions with your stakeholders, trying to nail down exactly what you’re trying to achieve with the survey, and making all of your stakeholders happy with the end result, that by the time you’ve programmed the survey into your survey software, the last thing on your mind and on your schedule is sending it out for testing. After all, you just barely got everyone to agree on the questions!
However, testing should be one of the primary things on your schedule, and there should be ample time given to test, tweak, and test again if necessary (and let’s face it, it will likely be necessary to have at least one update to your survey after it’s been tested).
Unbiased testing: an even better practice
I am a fan of having surveys tested by as many people as possible before it goes out. However, that usually means I’m turning to friends and colleagues, some of whom may feel hesitant to tell me all of the issues with my questionnaire.
And while I have a few key friends who I know will be sure to tell me every issue they find, I don’t want to run the risk of either their suffering test exhaustion because they are the group I turn to every time, or having them become immune to issues because they have become accustomed to my particular style of survey design.
I also don’t have the opportunity to know if there was something they originally stumbled over, then were able to work through just fine after a moment, or if they had to make assumptions about what I was asking based on the responses I’d listed.
To be more certain of an unbiased test result, then, I need as unbiased a group as I can find to test my survey, and having a video of them taking my survey would be fantastic, so I could catch any of those assumptions or stumbling blocks that I didn’t know my respondents would run into along the way.
Integrated testing in QuestionPro
We’ve rolled out integrated survey testing in QuestionPro in partnership with TryMyUI. This survey testing gets us over a few of the testing hurdles.
- I get completely unbiased testers. They don’t know me, and I don’t know them. They’re not worried about hurting my feelings if they tell me something is wrong with my survey.
- I get video of them taking my survey. Now, I can hear if they are needing to make assumptions about what I’m trying to ask in my survey. I can also hear if the way I’ve set up the survey is confusing.
- My test results will be back within a day. Sometimes, I’ll even get results back within a few hours, so I can definitely work that into my research project schedule.
And, as we always strive for with any QuestionPro feature, requesting a test of my survey is as easy as clicking the testing link found under the Live Survey Link. I then enter a detailed scenario (the more detailed, the better), set up payment, and look forward to the results.
As we talk more about this integrated testing feature and a new Survey Respondent Score that will give a quantitative measure of the survey design. So, you’ll get quantitative data and qualitative data to help you design a great survey.
