10 Things to Know about Completion Rates

Jeff Sauro • September 6, 2011

1. Completion rates are the fundamental usability metric: A binary measure of pass and fail (coded as 1 or 0) provides a simple metric of success. If users cannot complete a task, not much else matters with respect to usability or utility.
   
   
2. Easy to understand: They are easy to collect and easy to understand for both engineers and executives. You don't need to be a statistician to understand what it means when only 20% of your users can complete key tasks on an application.
   
   
3. They are popular: Completion rates are used in approximately 86% of summative usability tests and around 79% of formative tests.
   
   
4. Can be collected at any stage of development: Completion rates work well for low-fidelity prototypes through full fidelity benchmark testing.
   
   
5. Have pre-defined success criteria: Prior to testing, determine what things must happen to consider a task successful. Focus on the outcome of the task (did the user get to the right page, submit the right information) not necessarily the path to completion (although that is important too).
   
   
6. Context Matters: What makes a good task completion rate depends on the context. Where the consequences of failure are high (loss of money, life, reputation etc.,) you'll want high completion rates.   Past studies or other tests using a similar interface in a similar field can provide guidance on what's acceptable.
   
   
7. The average task completion rate is 78%: If you have absolutely no clue as to what a good benchmark for a completion rate is, use 78%. This is the average from a database of almost 1200 software and website tasks across several products and domains. As you get more information, update the benchmark.
   
   
8. Compute a Confidence Interval around Completion rate using the adjusted Wald confidence interval. It is accurate for any sample size.  For example, if 1 out of 5 users complete a task, the 95% confidence interval is 2% to 64%. While the interval width is wide (over 60 percentage points), its highly improbable that the completion rate will ever exceed 70%.
   
   
9. To ensure a task-completion rate is at least 70%, plan on testing at least 8 users: If all 8 users complete the task, you can conclude with 95% confidence that the completion rate exceeds 70%  (to ensure a 90% completion you plan on testing a minimum of 30 users).
   
   
10. Around 14% of users who fail a task still rate it as super easy to do: The correlation between task success and task difficulty ratings is around .5. This means that task-ratings only predict around 25% of completion rates. The explanatory power isn't higher because some users don't know they failed the task and others complete the task but still think it's difficult.
    

Learn More

UX Bootcamp: Sept 25th-27th in Denver, CO
Best Practices for Remote Usability Testing
The Science of Great Site Navigation: Online Card Sorting + Tree Testing Live Webinar

You Might Also Be Interested In:

What is a good task-completion rate?	What metrics are collected in usability tests?	10 Things to Know about Usability Problems	10 Things to Know about Task Times
Sample Size Calculator for a Completion Rate

Rate this Blog

Avg. Rating 5.56 (36)

Poor										Excellent
0	1	2	3	4	5	6	7	8	9	10

.

Posted Comments

There are 1 Comments

September 7, 2011 | John Romadka wrote:

I love your articles and blog postings. They are so succinct and valuable!!! Thanks! 

---

Post a Comment

Comment:

Your Name:

Your Email Address:

.

To prevent comment spam, please answer the following :
What is 1 + 2: (enter the number)