Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

6 Steps to Identifying Usability Problems

Jeff Sauro • September 17, 2013

While humans are error prone, many of the unintended actions, delays, frustrations and confusions come from interfaces:
  • Pushing instead of pulling a door
  • Not being able to set the time on a clock
  • Unintentionally voting for the wrong candidate on a ballot
  • Turning on the wrong burner on a stove
  • Not understanding an error message

A usability problem is anything in a product or website that leads a user to an undesirable outcome. It's relatively easy to spot when users have problems in an interface. It's a lot harder to know what, if anything, to do about them.  Making usable interfaces starts with understanding what in an interface is causing problems and helping to fix them.

Here are six steps to help identify and fix usability problems more efficiently.

  1. Record the undesirable outcome: Did users make a mistake, not notice an element, take a long time to complete a task, or fail to complete the task altogether? These actions are symptoms, but not guarantees of usability problems. Errors are undesirable, but they aren't necessarily caused by a problem with the interface (think typos). Both first-time and experienced users commit errors, although different types of errors[pdf].

  2. Identify what in the interface is causing the problem:  Sometimes it's obvious (e.g. users don't know what an icon means), but other times knowing the context, what the user was doing and what they were asked to do, is essential for extracting a usability problem.  There have been attempts at more systematically identifying and cataloging problems. See, for example the SUPEX and CUP approaches. While these systems aren't in wide use by practitioners, largely because of their complexity, perhaps a customized and streamlined approach can be beneficial.  There is a well-documented history of variability in the type and number of problems discovered by different usability evaluators. There is some hope these more structured problems discovery methods will lead to more consistent results (even though consistency isn't necessarily the goal of usability evaluations).

  3. Determine if the Issue is global or local: It can often be helpful to identify if a problem affects the entire interface or just a section of it. For example, if users have problems turning on filters on a mobile TV app, it's likely local to the filtering functionality. However, if users don't understand what a name in the global navigation means, it'll affect the entire application. Just because an issue is local doesn't make it less important. Rather, it helps you and the design team understand both the scope of the problem and the potential impact of the change. 

  4. Assign a severity rating: You want to separate the catastrophic problems from the cosmetic ones. One of the best ways is to assign a severity rating. We use a simple, three-level scale, calling out insights. You can use almost any number of levels or system that works for your stakeholders or organization. Don't obsess of the "best" way to assign severity ratings, just pick one and use it.

  5. Recommend possible solutions if appropriate:  Every usability issue doesn't have to have a proposed solution, but if there is an obvious fix, then suggest it. We often recommend solutions when multiple issues can be fixed by a new design.  Usability problem descriptions should be usable and useful. Often just a well-described usability problem can help developers generate ideas for design alternatives.

  6. Present usability problems: While the "right" way to find and present usability problems will depend on the purpose of the study (finding and fixing versus comparing to previously identified problems) it's ultimately about balancing priorities, design and business constraints to help determine what to fix.  Providing a list of usability problems is the simplest and most straightforward approach to presenting usability problems. Include the percent of users that encountered each problem and the assigned severity. 

    Problem lists work fine if the stakeholders know what the problem is without much context or description. This is especially the case if we know design teams watched the usability tests with us or we discussed the issues in a debrief. Adding screen shots, videos and additional descriptions is necessary to help everyone understand the issues—especially if the usability issues are controversial. In fact researchers at the University of Copenhagen[pdf] found that developers preferred this more detailed method over simpler problem lists.  We usually find adding both a simplified problem list at the beginning of a report and a more detailed follow up section with screen shots helps accomplish the need to be succinct while still providing the necessary context when needed.

About Jeff Sauro

Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 4 books on statistics and the user-experience.
More about Jeff...


Learn More



You Might Also Be Interested In:

Related Topics

Usability Problems, Usability Testing
.

Posted Comments

Newsletter Sign Up

Receive bi-weekly updates.
[4241 Subscribers]

Connect With Us

.

Jeff's Books

Quantifying the User Experience: Practical Statistics for User ResearchQuantifying the User Experience: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.