Rank Order: Definition, Survey Questions, Characteristics, Types and Examples

Rank Order Definition

Rank order scale is a defined as survey question type, that allows respondents to rearrange and rank multiple choice options in a specific order. Rank order questions are close-ended questions that allow respondents to evaluate multiple row items in relation to one column item or a question in a ranking survey and then rank the row items. They are multiple-choice questions represented in a single column format. They are used extensively both in pen and paper questionnaires as well as online surveys due to the accuracy of responses and the insights that can be collected.

Rank order scale questions are an integral part of a ranking survey where the person conducting a survey would like to understand the order of importance of items when there are multiple items. This ranking could be on the basis of features, want, likes or dislikes, effectiveness and so on. The ranking is by a simple ordinal position where one variable or option is higher than another. The ranking could also be by a relative position where one variable has a higher relative rating than another and subsequent ones.

Characteristics of Rank Order Scale

  • Help to collect quick feedback: Rank order questions are most effective in collecting feedback about what one is doing right and what enhancements can make to an offering to make it even more robust. The other scenario is where you are an airline company collecting feedback on the customer journey. You would like to improve the overall experience, and you have many ideas of possible solutions, but you can’t implement them all. You could start by presenting ideas to the customers and asking to rate them on a scale from “not at all” to “very” important options but would it be helpful if they all said that all these aspects are “very important”? Probably not.
  • Justification of secondary research: These survey questions derive the highest value when there’s some background research before the question is administered to potential respondents. For example, if preliminary research has been conducted and possible areas of product enhancement have been marked out, it builds a platform to validate the findings. You could present customers with a list of possible product improvements and ask them to select what product tweaks matter the most to them within the product. But what if they select many of the options, how would you find out which are they most interested in? This is where rank order scaling questions help prioritize.
  • Collect actionable data: Rank order question asks respondents to order the presented options from high to low. By asking this, respondents cannot say that all things are equally important. They are forced to prioritize and that provides you insightful and actionable data.

Learn more: 300+ Survey Templates and Questionnaires

Rank Order Scale by Methods of Sorting

There are multiple ways of sorting ordinal data using rank order scaling questions. Some of the major sorting types are:

  • Priority sorting: Priority sorting looks for the most important and subsequent options in order of importance. This method is widely used in customer satisfaction surveys (CSAT) to capture customer insights. For example during a new product launch or a product upgrade, priority sorting is used to capture what customers perceive as the important features in a descending order.
  • Block sorting: Block sorting is used when there are many options that can be bucketed into distinctive blocks of quantitative data. This gives the interviewer a granular insight into exactly what a customer wants. For example if a steakhouse wants to understand what meats, cuts and rubs to have on the menu asks a respondent to sort options by bunching the above three parameters.
  • Score sorting: Score sorting is one of the most widely used methods of rank order scaling questions. Respondents are expected to add one unique number to each option and the options are then sorted from 1 being the most wanted feature. For example if an airline conducts a survey where respondents are required to score options like seat size and pitch, in-flight food, luggage allowance, lounge access and frequent flier program.

Learn more: Qualitative Data

Types of Rank Order Survey Questions

  • Rank Order Scaling Question: The rank order scaling question type provides respondents an unique opportunity to rank a set of items against each other. Rank order scaling based multiple-choice questions allow a certain set of brands or products to be ranked based on a specific attribute or characteristic. The ranking is one in which a numerical drop down box can be created. By asking this question in a ranking survey, the survey creator can understand the order of preference of the respondent and conclude their choice preference.

    Rank-Order-Rank-Order

  • Constant Sum Question: Constant sum multiple choice questions provide respondents a way to enter numeric data. A constant sum question allows respondents to enter numerical values for a set of variables, but requires the values to add up to a pre-specified total. Each numeric entry is summed and can be displayed to the respondent. This is a great question type to use when asking financial, budget related questions or percentage based questions.

    Constant-Sum-Rank-Order

  • Drag and Drop Question: Respondents in this type of survey question can drag and drop the answer choices to re-order them as they choose. This question type gives complete freedom to the respondent to order the answer options to best suit to their preference. This drag and drop option questions is interactive and will help the respondents choose wisely.

    Drag-and-Drop-Rank-Order

Reasons to use a Rank Order Scale Questions?

As mentioned earlier, when a researcher intends to collect information about features, products or services in the order that customers would like to consume them, rank order scale questions offer the best insights. Some of the other reasons to use them are:

  • Reduce survey dropout rate: Rank order scale questions are easy to understand and they are also easy to answer which decreases the survey dropout rate. This question type also throws light into the needs and wants of a consumer and due to the nature of the question, the answer collected help in product and service tweaks at both a macro and micro level.
  • Increased participation: Respondents are generally hesitant to provide information due to a variety of reasons. The rank order scale question though gives the power to a respondent to put forth to a brand the needs, wants and preferences thereby offering a heightened sense of importance. This increases the participation in a survey.
  • Validating any prior research: Rank order scale questions work best when the responses to a question are validated with prior secondary research. Hence, the options are not vague and helps to elicit responses only on absolutely critical features or upgrades in a product or service.

Examples of Rank Order Scale Questions

Fitness-Tracker-Survey-Rank

This is best asked if you have determined, in a previous question, that respondent is currently or planning to evaluate fitness trackers within three months. Otherwise, the question is way too hypothetical.

Fitness-Tracker-Survey-Rank-Order

If you are looking to offer a new, differentiated product, a good first step is to know what the “table stakes” are. If you know certain features are well-liked in current products than those may be the items you need to match (at minimum). Then, knowing the aspects that are least well-liked gives you input on what possible differentiators you may be able to seize.

Fitness-Tracker-Survey-Rank-Order-Question1

While this approach is wordier, it reflects a reality: choosing between products in many product categories is often ultimately driven by one or two items, and all other features are not all that differentiated.

Tips for Rank Order Scale Questions

  • Use rank order scaling questions judiciously: Rank order scaling questions allow the respondents to rank responses on the basis of preferences. This is an easy survey question type for a respondent but also one where they have to think. Adding too many such questions in a survey can possibly cause higher drop outs in a survey.
  • Conduct prior in-depth secondary research: To come up with options for a rank order scale question, there is a need for a lot of secondary research. Not doing your homework can negate the complete exercise of conducting the survey.
  • Answer options need to be well thought out: To collect insightful answers that contain actionable analytics, the options in an answer have to be well thought out. This ensures that the responses from the question can be actioned on.

Learn more: Primary Research

Advantages of Rank Order Questions

  • Advanced consumer insights: Rank order survey questions help to delve deeper into the profile of the respondent. This question not only talks about the what but also the why of a research subject. There is a value tied into each individual response and this helps derive statistical analysis about a certain subject. The data from such a question also helps with making business decisions and also help satisfy your audience with what is important to them.
  • Higher response rate: Due to the nature of the answers, the respondent feels a heightened sense of responsibility towards the question being asked. The rank order survey question also pushes the respondent towards providing as realistic responses as possible.
  • Easy of use: The instructions of a rank order survey question are simple and easy to understand. The respondents are comfortable with ranking the different options as they are comfortable with the options provided and have a heightened sense of belonging.
  • Validating secondary research: Rank order survey questions help validate any prior secondary research. The options provided in such a question are brief inferences from the research. The respondent data from this question helps the researcher understand if the initial theories you set out with are true or not.

Rank Order Scale : The QuestionPro Process

To add a Rank order question, go to: Login >> Edit >> Workspace

  • Click on the Add Question button.
  • Now you will see various options, under Basic >> click on Ordering >> Rank Order
    rank-order_1
  • Add Option: Use this option to add more answer choices in the answer field.
    rank-order_2
  • Add/Edit Answers in Bulk: This option helps to add answer choices in bulk. We can copy and paste the list of answer options directly in this field.
    rank-order_3
  • Required: When you add questions to a survey, by default, required option is turned off. When required option is not enabled, respondents can continue with the survey without selecting answers. If respondents go through all the pages in the survey without selecting answers, the response is still considered as complete. You can enable required option to make a question required so that respondents can continue with the survey only after responding to the questions.
    rank-order_4