10 Things to Know about Card Sorting

Jeff Sauro • April 3, 2012

1. Card Sorting involves having users (not designers, developers or VP's) sort a list of pages or categories together based on their understanding of the structure and relationship.
   
   Card sorting is useful for such things as finding out how users would group merchandise at an ecommerce website that has a huge selection of products and categories.
   
   
2. The goal of card sorting is to generate a user-centered taxonomy which, among other things, should result in a better navigation structure for a website (less time to find products, higher conversion rates and a happier world for all of us).
   
   
3. There are two types of card sorts:
   
   Open Sort: Users are asked to sort items into a group and make up their own groups and give the groups a name.
   Closed Sort: Users sort items into previously defined category names.
   
   
4. Roughly half (52%) of UX professionals report using card sorting, about the same as in 2009 and up from 46% in 2005 (the 13% increase is statistically higher) [Source UPA annual Salary Survey].
   
   
5. In a survey of 217 attendees of Usability Week 2008, Nielsen Norman Group reported that the average number of card sorts conducted per year was 2. 
   
   
6. How long does it take to perform a card sort?  Well it depends on the number of cards and complexity of the categorization but William Hudson offers some guidance:
   
   • 20 minutes for 30 items
   • 30 minutes for 50 items
   • 60 minutes for 100 items
     
     
7. Card sorting can be done by hand (hence the term card) but is now usually performed using software which allows for users from around the world to sort and generates fancy graphs and percentages. Popular packages include: OptimalSort, Websort and Userzoom. Note: We have business relationships with OptimalSort and Userzoom.
   
   
8. Identifying groups of categories is often done using cluster analysis and results in a figure called a dendrogram. An example dendrogram from OptimalSort is shown below. Interpreting the groups in dendrograms is notoriously subjective, especially when there isn't high agreement between users. Using Factor Analysis may lead to better discrimination. 
   
   
   
   
   
9. Confidence : Use confidence intervals around the percentage of items that were placed within a group to estimate the likely percentage of the total user population that would also group the cards that way. For example, if 32 out of 70 participants placed "Getting a New Person Started" in the "Hiring" category of an HR application, we can be 95% confident between 35% and 58% of all users also would place the card there (a margin of error or 11%).
   
   
   
10. Sample Size: There is surprisingly little guidance on finding the right sample size for a card sorting study. Tullis and Wood (2004)[pdf] performed a resampling study with one large card sort involving 168 users and found the cluster results would have been very similar (correlations above .93) at sample sizes between 20-30.
    
    Another approach is based on finding the desired amount of precision around the percent of users who place cards in each category. It's basically working backwards from the confidence intervals.   For example, to have a margin of error of around 7% you should plan on having around 193 participants.
    
    

    Sample Size	Margin of Error (+/-)
    10	27%
    21	20%
    30	17%
    39	15%
    53	13%
    93	10%
    115	9%
    147	8%
    193	7%
    263	6%
    381	5%
    597	4%
    1064	3%

    
    
    The computations are explained in Chapter 3 and Chapter 6 of Quantifying the User Experience.

Learn More

UX Bootcamp: Sept 25th-27th in Denver, CO
Best Practices for Remote Usability Testing
The Science of Great Site Navigation: Online Card Sorting + Tree Testing Live Webinar

You Might Also Be Interested In:

Three easy metrics for improving website navigation

Quantifying the User Experience: Practical Statistics for User Research the Book (TOC)

10 Things to Know about Usability Problems

10 Things to Know about Completion Rates

Rate this Blog

Avg. Rating 4.97 (73)

Poor										Excellent
0	1	2	3	4	5	6	7	8	9	10

Related Topics

Card Sorting, Information Architecture

.

Posted Comments

There are 2 Comments

April 1, 2013 | Candace wrote:

I am a graduate student at Arizona State University and am attempting to find a card sorting software to perform a closed sorting task using an image pool of 200 teapots. I would like the participants to sort 50 of the images only at a time, but to have these images randomly drawn from the total image pool for each participant. Does anyone know of a software that can rapidly perform this task? Thanks for your time and information.  

---

May 4, 2012 | S wrote:

It seems appropriate to mention here Remote Card Sorting (done online). There's a number of tools currently available to those interested in conducting this test remotely. One of such tools is: http://www.usabilitest.com/CardSorting 

---

Post a Comment

Comment:

Your Name:

Your Email Address:

.

To prevent comment spam, please answer the following :
What is 1 + 1: (enter the number)