A card sorting question is an interactive survey format that asks respondents to sort, categorize, and rank a set of items based on how they naturally understand them, rather than how a company assumes they should be grouped. It exists because what makes sense internally to an organization often does not match how customers actually think.
Card sorting closes that gap directly. Instead of guessing which website navigation labels or product categories will make sense to users, you ask them to sort real items into groups themselves using a card sorting survey feature and watch the patterns that emerge.
This guide covers the two types of card sorting questions, worked examples of each, how to set them up in QuestionPro UX, and how to read the resulting data. Card sorting is available alongside standard survey software tools, so it fits naturally into a broader research project rather than requiring a separate platform.
What is card sorting?
Card sorting is an interactive research technique that analyzes how people understand and group information, based on how they sort a set of items into categories.
The output gives organizations a clearer picture of how to structure information for a target audience, whether that is a website’s navigation, a product catalog, or a set of feature names. As a question type, it is highly interactive: respondents drag, drop, and rank items in an order that reflects their own mental model, not a predefined one.
When should you use a card sorting question?
Card sorting works best whenever you need to understand how your audience naturally groups or prioritizes information, rather than confirming a structure you have already decided on.
Common situations where it adds real value, as documented by the Nielsen Norman Group, include:
- Grouping features or attributes for a new product or service
- Deciding what information belongs where on a new or redesigned website
- Comparing how people understand and differentiate similar concepts or ideas
- Learning how people rank a set of items against a specific criterion
What are the two types of card sorting?
Card sorting comes in two formats, closed and open, and the choice between them depends on whether your categories already exist or need to be discovered.
- Closed card sorting gives respondents a predefined set of categories and asks them to sort items into those categories. This works well when categories already exist and are widely understood, since it measures how well items fit an established structure.
- Open card sorting gives respondents no predefined categories at all. They create and name their own groups as they sort. This is the better choice when category names are not yet standardized, since it reveals the actual words and mental groupings your audience uses.
What does a card sorting example look like?
A closed card sorting example might ask respondents to sort a list of movies, such as The Conjuring, Due Date, Terminator: Dark Fate, Annabelle, Hangover, and Avengers: Endgame, into predefined genres like action, horror, and comedy.
Because the categories already exist and are universally understood, closed sorting here simply measures agreement: does the audience place “The Conjuring” in horror the way you expect, or do a meaningful share of respondents see it differently?
An open card sorting example uses the same list of movies, but respondents create their own category names instead of choosing from action, horror, and comedy. This version is more useful when you are not certain your assumed categories match how your audience actually thinks about the content, or when you want to capture the specific language people use to describe each group.
How do you set up a closed card sorting question?
Setting up closed card sorting starts with defining your categories before you add any items, since respondents will sort cards into those fixed groups.
- Go to Login, then Surveys, select your survey, then Edit and Workspace.
- Click Add Question, then under advanced question types and ordering, select Card Sorting.
- Add your question text, predefined categories, and answer options. Use the bulk edit option to paste in content quickly.
- Under Settings, select Closed as the answer type.
- Choose an answer display order: default, ascending, descending, random, or advanced randomization.
- Apply validation settings to require a response or a minimum number of sorted items.
Respondents see the answer options in a box on one side of the screen and drag each one into the appropriate category box on the other side. In the movie example, “The Conjuring” would move into the horror category and “Hangover” into comedy.
How do you analyze closed card sorting results?
Closed card sorting results show, for each item, the count and percentage of respondents who placed it in each category, along with its average ranking within that category.
If 91% of respondents categorized “The Conjuring” as horror and 8% categorized it as action, that tells you the item’s category assignment is strong but not unanimous. Comparing count and rank across similar items, such as two movies that both received horror votes, shows which one respondents consider the stronger fit for that category overall.
How do you set up an open card sorting question?
Open card sorting follows nearly the same setup process as closed sorting, with one key difference: you skip defining categories in advance and instead instruct respondents to create their own.
- Go to Login, then Surveys, select your survey, then Edit and Workspace.
- Click Add Question, then under advanced question types and ordering, select Card Sorting.
- Add your question text and answer options, using bulk edit to paste content quickly.
- Under Settings, select Open as the answer type.
- Choose an answer display order, the same options available for closed sorting.
- Apply validation settings to require a response.
Make sure your question instructions clearly explain that respondents need to create categories themselves before dragging items into them. Respondents drag an item into a box labeled to create a new category, name that category, and repeat the process, building their own structure as they go.
How do you analyze open card sorting results?
Open card sorting results show the categories respondents created themselves, along with each item’s count, percentage, and average rank within the categories the audience generated.
Using the movie example again, if respondents created comedy, horror, and action on their own without being prompted, and “Due Date” received a 100% comedy categorization with the highest average rank, that tells you not just where the item belongs, but that your audience’s natural category structure closely matches an industry-standard genre system. When it does not match, that mismatch is often the most valuable finding of the entire exercise.
What mistakes should you avoid with card sorting?
A handful of avoidable mistakes weaken card sorting data more than any single setup error.
- Using too many items.
More than 30 to 40 cards makes the task tedious and increases dropout.
- Mixing item types that do not belong together.
Sorting products, features, and marketing phrases in one list confuses respondents about what they are actually organizing.
- Choosing closed sorting when categories are still unclear.
This forces respondents into groups that do not reflect how they actually think, producing misleading agreement rates.
- Skipping a pilot test.
A small test group catches confusing item wording before it affects your full sample.
Putting Card Sorting to Work
Card sorting replaces assumptions with real user data, bridging the gap between how your team organizes content and how your audience actually processes it. Whether you opt for open sorting to uncover fresh user mental models or closed sorting to validate an existing setup, capturing these insights early on saves you from costly redesigns down the road.
Keep your card counts manageable, stick to one item type at a time, and listen to the patterns that emerge. As a result, your navigation, product categories, and overall user experience will be much better.
Frequently Asked Questions (FAQs)
Most card sorting studies work best with 30 to 60 items total. Going much higher increases respondent fatigue and lowers the quality of the sorting decisions, particularly toward the end of the task.
No. While website information architecture is a common use case, card sorting also works for grouping product features, comparing brand concepts, and understanding how customers rank priorities across almost any category-based decision.
A ranking question asks respondents to order a single list from most to least preferred. Card sorting asks respondents to group items into categories first, and, in some setups rank items within those categories, which captures structure as well as preference.
Run an open card sort first if you have any doubt about whether your assumed categories match your audience’s mental model. You can follow up with closed card sorting later to confirm and refine the categories the open sort revealed.
Yes, though shorter item lists perform better on smaller screens. Dragging and dropping many items on a phone screen is more tedious than on desktop, so consider trimming the list for mobile-heavy audiences.
