I don’t know about you, but my default question type to use for any survey is the multiple choice question type. It’s easy, it doesn’t require a lot of work and I thought that it would give me more or less the same result as any other kind of question type.
Then QuestionPro updated it’s ordering and ranking question types by adding all kinds of bells and whistles such as the “Drag and Drop” question type, and the “start rating” question type. This made me wonder about what the difference was between each type and when I should use each.
I did a little digging and here is what I’ve learned:
A rating question type is used to compare different items using a common rating scale such as 1-10 for example.
A ranking question asks you to compare a list of items to each other.
Use a rating scale when you suspect and want to allow people to assign similar values to items. This requires less thought on the part of the respondent. On the other hand, if your intention is to get an ordered list of top rated items similar to a ranking — this may not happen, especially if you have a large number of respondents giving numerous items the same rating.
If it’s critical for you to get to a specific order of items on your list where there can only be a single top-ranked item, then you should consider rank order questions.
Rating and ranking question types from QuestionPro
Luckily, QuestionPro has numerous options for both rating and ranking questions for you to choose from:
Rank order is the first most basic question type. With this question type you ask the respondent to rank all the items in the order they prefer. So if you have 10 items, they would rate each option from 1 – 10 and enter those number in themselves.
This option is the traditional rank-order question type. But one downside of this question type is that it can frustrate respondents because they actually have to devote more thought to filling it out. And if they give any two items the same rating – they will be asked to go back and fix their answers and this can lead to people either dropping out of the survey.
Drag and drop. This is a new feature on QuestionPro that actually allows respondents to manipulate the items on your list into the order they would like to see it in. The reporting outcome or results are the same as in the rating scale. It’s simply an improvement in the user experience and interface because it eliminates the confusion and the option of giving two items the same ranking.
Constant sum is another traditional question type that is best used when you want to understand how the respondent would weigh or trade off competing priorities. A typical example might be asking the respondent to assign a percentage number to the list of items so that they add up to 100%. For example: What percentage of your day is spent doing the following activities?
- Eating _____
- Sleeping_____
- Working_____
- Hobby ______
The key to having a good experience with constant sum questions is to keep the choices down to a minimum. Many respondents find it difficult to do the addition in their head and may drop out of the survey before completing it.
Star ratings. This is another new QuestionPro question type that plays into bringing user interface and engagement together into a more visual question type. It’s most similar to a rating scale that runs from 1-5 or 1-4.
Choosing the right question type for the right result
I don’t need to mention the whole “garbage in – garbage out” cliche do I? Of course not. The key is to take a moment when developing your questions to think about exactly what kind of information you need to make your decision. What is it that matters in the results of your study and let that guide you to the appropriate question type.
Speaking for myself, you won’t see me defaulting to the multiple choice question type without exploring and testing the other question types that are available with QuestionPro.
