Card sorting is a research technique designed to analyze how people understand and categorize information. The card sorting data gives organizations a better sense of how information should be presented to a target audience. What makes sense to an organization may not always translate to the customer. The card sorting survey feature closes this gap and ensures everyone is on the same page. As a question type, it is highly interactive and allows respondents to sort, categorize, and rank based on the order of preference. 

When to use cart sorting

  • Grouping features and attributes together of new products/services
  • Designing or improving existing websites – what site info should go where
  • Understand and compare different concepts and ideas
  • Find out how people rank or arrange concepts given based on a set of criteria.

Two types of card sorting

The card sorting survey feature comes with two answer types – closed and open.

  • Closed card sorting: In the closed card sorting question, a survey designer has to define the categories.
  • Open card sorting: In an open card sorting question, survey respondents can define their very own categories.

We will go over both options for you to help you determine which type of card sorting is the best for you.

Example: If you want your respondents to categorize a list of Hollywood movies into different genres, then you can choose the open card sorting to let the respondent define the genres, or you can predefine the categories of genres using the closed card sorting question type.

List of movies:

  • The Conjuring
  • Due Date
  • Terminator: Dark Fate
  • Annabelle
  • Hangover
  • Avengers: Endgame

If there are categories that already exist that the audience understands, then using the closed card sorting question would be the best option. If category names are not universally recognized, then using the open card sorting option would get the best understanding of what words people associate with the items being categorized and ranked. 

Closed card sorting set up and analysis

Instructions to add the closed card sort question:

  • Go to: Login » Surveys (select a survey) » Edit » Workspace.
  • Click on the ‘Add question’ button.
  • Under advanced question types » ordering, click on ‘Card Sorting’ question.

add cards in survey

  • After the template is selected, add the question text, predefined categories, and answer options
  • Use the edit in bulk options to do a quick copy/paste of content
  • Click on ‘Settings’ and select answer type as ‘Closed’

card sorting in survey

Under settings, update the answer display order from the following drop-down options:

  • Default
  • Ascending
  • Descending
  • Random (with the option to apply a specific number of randomization to overall answer options)
  • Advanced randomization

On the bottom left corner of the question template, apply validation options to either force a response or request a response from a specific number of answer options.

What respondents see – Closed card sorting

closed card sorting

Here, the respondents need to drag the options from the left-hand side box to the respective categories on the right-hand side. For example, “The Conjuring” will go into the ‘horror’ category and “Hangover” will go into ‘comedy’.

Analyzing the results for closed card sorting

survey card sorting analysisAnalyzing the results above,  you can see the items sorted into given categories – comedy, horror, and action. For “The Conjuring,” 91.47% of respondents categorized it as a horror movie while 8.33% categorized it as an action film.

The count and percentage of responses received and its average ranking is given for every item. For “The Conjuring,” we see 11 people selected it as a horror film while only one selected it as an action film. And although “Annabelle” also has 11 counts of horror, “The Conjuring” ranks in the top spot for horror in the list of movies given. 

Open card sorting set up and analysis

Let’s go back to the movie example again and this time, instead of predefining the categories, we will let the respondent create their own categories and rank according to the list of movies given.  Important things to note:

  • For open card sorting, make sure to give instructions in the question section for respondents to create their own categories before dragging and ranking the answer options
  • The percentage calculation dashboard will look different than the closed card sorting results but will still show results for each item out of 100%.

To add an open card sorting question:

  • Go to: Login » Surveys (select a survey) » Edit » Workspace.
  • Click on the ‘Add question’ button.
  • Under advanced question types » ordering, click on ‘Card sorting’ question.

use cards in survey

  • After the template is selected, add the question text, predefined categories, and answer options
  • Use the edit in bulk options to do a quick copy/paste of content
  • Click on ‘Settings’ and select answer type as ‘Open’

card sort in surveyUnder settings, update the answer display order from the following drop-down options:

  • Default
  • Ascending
  • Descending
  • Random (with the option to apply a specific number of randomization to overall answer options)
  • Advanced randomization

On the bottom left corner of the question template, apply validation options to either force a response or request a response from a specific number of answer options.

What respondents see – Open card sorting

sort cards in survey

When respondents take a survey with the open card sorting question,  the respondents need to drag answer options one-by-one from the left-hand side box into the box “drop an item here to create a new category” on the right-hand side. A new box will be created below. Respondent has to name the category as shown in the above image. 

For example:

  • Respondents need to drag “The Conjuring” into the red box – ‘drop-in item here to create a new category’. They can name the new category as ‘horror’ in the grey text box. Now, they can drag and drop horror movies from the answer options into the horror category.
  • Similarly, respondents need to drag other movie titles into the red box – ‘drop-in item here to create a new category’ if they want to create a new category and name the category as applicable. Now, they can drag and drop remaining movie titles from the answer options into their respective categories.

Analyzing the results for open card sorting

To view the report for card sorting question:

  • Go to: Login » Surveys (select a survey) » Analytics » Dashboard.

card sort reportWith the open card sorting results,  you can see the categories created by the respondents – comedy, horror and action and their respective movies along with a percentage calculation report by the movie.

View the count and percentage of the responses received for each item and its average ranking. For “Due Date,” you can see 12 total counts with 100% being respondent-categorized as a comedy while “The Hangover” had 11 categorizations as a comedy and one as an action film. Reviewing the total average rank, “Due Date” came in as the top-ranked comedy in the list of films.

The card sorting question type is available with QuestionPro’s Enterprise license. If you are interested in the card sorting question and would like to learn more about additional enterprise features and services, contact us here and we’ll be happy to answer any questions or set you up with a free trial.