10 million users
Designing an effective survey needs to be a perfect balance of linguistic art and science. Art in using precise words to form impactful questions and science in getting an equilibrium between close-ended and open ended questions, understanding the target audience, number of questions to add etc. while curating a survey.
Open and close-ended questions have their own set of pros and cons. The correct inclusion of both results in a better performing survey. While close-ended questions come with limitations like restriction of selecting from mentioned options, open ended questions give a platform for the respondents to say what they want to.
High-value surveys are those that bring in desired results like increase in sales or rise in number of signups. It’s observed that these high-value surveys have more open ended questions and not that many close-ended ones.
Unlike a closed ended question that leaves your responses narrow and limited, a free-form question allows you to probe deep into the respondent's answers, gaining valuable information about the subject at hand. The responses to these questions can be used to attain detailed, descriptive information, otherwise unattainable.
When designing surveys, we often need to decide whether to use an open ended question versus close ended questions to get specific information. Yet we need to be aware of the fact that open ended question and close ended questions each have their own strengths and weaknesses and perform in different ways.
Open Ended Questions Example
In a study conducted by the Pew Research in their November 2008 post-election survey, respondents were asked, “What one issue mattered most to you in deciding how you voted for president?” One group was asked this question in a close ended questions format while the other group was asked this question in an open ended question format. The results are displayed below:
In the close ended questions format, 58% of respondents chose “The economy,” while in the open-ended format only 35% wrote in a response that indicated “The economy.” Note also that in the close ended questions format, only 8% of respondents selected “Other,” while in the open-ended format 43% of respondents wrote in a response that would have been categorized as “Other.”
Open ended questions motivate the respondents to put their feedback into words without restricting their thoughts. They aren’t as objective and dominant as close-ended questions.
By using open ended questions, the researcher gets to understand the true feelings their respondents have. They have an element that will give you information about different thought processes across your clientele, troubleshooting suggestions, get a peek into their inhibitions too.
So what does this mean for us? If you can, do qualitative research first and make sure your close ended questions represent the items that are in people’s heads. We need the list of items to be complete since few respondents will select the “Other” category. It may also be necessary to list items that are not readily available to respondents if those items are important to you
Consider doing split sample studies where half of the respondents see the question in the open-ended format while the other half sees the question in the close ended questions format.
When presenting results, I have found it helpful to explain to the audience, in a sentence or two, the fundamental differences between open ended question and close ended questions. This helps them understand that these are not necessarily precise measurements, but measurements that require some interpretation relative to other questions in the survey and other information from qualitative research. Hence, that is why they need an analyst like you or me!
Always think before designing a survey as to what your objective is. Scrutinize the objective, evaluate the positives and negatives of using open ended questions. Try it by sending out to a selected database, analyze the results and plan improvements for the next round of surveys.
Where can I see reports/collected data?
Please note that analysis for open ended text questions is not included in the Real-Time Summary Report. To view analysis on open-ended questions, you can view the Word Cloud report.
To make the question mandatory you can click on Settings and you will find a toggle button to turn ON the validation. WHY? When you add questions to a survey, by default, required option is turned off. When required option is not enabled, respondents can continue with the survey without selecting answers. If respondents go through all the pages in the survey without selecting answers, the response is still considered as complete. You can enable required option to make a question required so that respondents can continue with the survey only after responding to the questions.
With many ways to create a survey and so many options for customization, whatever kind of questionnaire you need, QuestionPro can help you create it. If you're looking at other survey creators, think about these following questions:
Quickly create open ended question surveys using our powerful web-based platform. You can make a survey with your own open ended questions, copy a professionally written open ended survey template, or upload your survey with open ended questions directly from a Microsoft™ Word™ file. QuestionPro has over 30 question types, advanced logic, dozens of color themes and other customization options.
Reach respondents wherever they are using our email distribution system, embedding in your website or blog, with pop-up and exit questionnaires, promoting using our built-in QR Code generator or posting to social networks like Facebook, Twitter and Google+. QuestionPro Questionnaires look great on mobile devices or tablets, and you can even collect responses offline using our SurveyPocket iPad & Android tablet or Smartphone app.
With our full suite of reporting features such as real-time summary, pivot tables, segmentation tools, trend analysis, and text analytics, analysis is efficient and effective. Need more in-depth options to comb through the data? Export your survey data directly to Excel, SPSS, or CSV. And it's easy to share results with formatted Word™ and PowerPoint™ reports or by generating great looking infographics and word clouds.