Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

Blogs & Articles

Before measuring anything, especially a construct that

5 Questions to Answer Before Measuring Anything

Jeff Sauro • April 22, 2014

Before measuring anything, especially a construct that's not well defined or used in practice, we answer these five questions: How is this being measured already?, What will we compare this to? How reliable and valid are the measures? How precise do we need to be? What will we do with the results? [Read More]

Missing data in a survey is a pain. What

Identifying the 3 Types of Missing Data

Jeff Sauro • April 15, 2014

Missing data in a survey is a pain. What's worse is if the missing values are systematically excluding a segment of your customers. This blog describes a technique to understand if your data is missing at random, missing completely at random or systematically missing.[Read More]

UX researchers have developed many techniques over the years for testing and validating their ideas.  Here are ten essential methods you can employ on your next project.

10 Essential User Experience Methods

Jeff Sauro • April 7, 2014

UX researchers have developed many techniques over the years for testing and validating their ideas. Here are ten essential methods you can employ on your next project.[Read More]

Measuring Usability has partnered with leading user experience firms to create Cindy (Consistent INterview Drone for usabilitY), the first autonomous usability test facilitator.

Automated Lab Based Usability Testing

Jeff Sauro • April 1, 2014

Measuring Usability has partnered with leading user experience firms to create Cindy (Consistent INterview Drone for usabilitY), the first autonomous usability test facilitator.[Read More]

Every time researchers conduct a usability test to uncover problems they

What the NCAA Tournament & Usability Testing Have in Common

Jeff Sauro • March 24, 2014

Every time researchers conduct a usability test to uncover problems they're also working with probabilities, even if they tell you they hate math! To understand the role of probabilities in usability testing it helps to see how they are used when picking winning teams for the NCAA tournament.[Read More]

Most people are comfortable with the concept of an average or percentage as a measure of quality. An equally important component of measuring the user experience is to understand variability. Here are 10 things to know about measuring variability in the user experience.

10 Things to Know About Variability in the User Experience

Jeff Sauro • March 18, 2014

Most people are comfortable with the concept of an average or percentage as a measure of quality. An equally important component of measuring the user experience is to understand variability. Here are 10 things to know about measuring variability in the user experience.[Read More]

When using statistics to make comparisons between designs, it

Understanding Effect Sizes in User Research

Jeff Sauro • March 11, 2014

When using statistics to make comparisons between designs, it's not enough to just say differences are statistically significant. Effect sizes provide a standardized way to understand the size of the difference across all types of metrics.[Read More]

To get a better understanding of problems and opportunities for your customers, you

4 Types of Customer Analytics Data to Collect

Jeff Sauro • March 4, 2014

To get a better understanding of problems and opportunities for your customers, you'll want to collect data from each of the following four customer analytics groups: descriptive, behavioral, interaction, and attitudinal.[Read More]

It

Is Observing One User Worse Than Observing None?

Jeff Sauro • February 25, 2014

It's possible for product stakeholders to be misled from watching a single session during a usability study. This leads teams to have a 2 or more rule when observing. However, watching just a single user still provides information on the impact of an interface problem. Over time, if stakeholders are consistently watching a single random user in each usability study, odds are they will be more likely to see common problems than freak occurrences.[Read More]

A lot of planning goes into a usability test.  Part of good planning means being prepared for the many things that can go wrong. Here are the ten most common problems we encounter in usability testing and some ideas for how to avoid or manage them when they inevitably occur.

10 Things That Can Go Wrong in a Usability Test

Jeff Sauro • February 18, 2014

A lot of planning goes into a usability test. Part of good planning means being prepared for the many things that can go wrong. Here are the ten most common problems we encounter in usability testing and some ideas for how to avoid or manage them when they inevitably occur.[Read More]

Inevitably one or a few of these survey response rate killers will creep into your next survey project, but sometimes knowing more about them will help reduce or prevent them to help keep your response rate high.

10 Ways to Get a Horrible Survey Response Rate

Jeff Sauro • February 11, 2014

Inevitably one or a few of these survey response rate killers will creep into your next survey project, but sometimes knowing more about them will help reduce or prevent them to help keep your response rate high.[Read More]

Here are four steps to take to get your questionnaire ready for a different language than it was written in: Translate, Back Translate, Pretest and Validate.

4 Steps to Translating a Questionnaire

Jeff Sauro • February 4, 2014

Here are four steps to take to get your questionnaire ready for a different language than it was written in: Translate, Back Translate, Pretest and Validate.[Read More]

While there are books written on measuring usability, it can be easy to get overwhelmed by the details and intimidated by the thought of having to deal with numbers. If I had to use five words to describe some best practices and some core principles of measuring usability, here they are: multi-method, triangulate, redundancy, actions, and attitudes.

Measuring Usability Best Practices in 5 Words

Jeff Sauro • January 28, 2014

While there are books written on measuring usability, it can be easy to get overwhelmed by the details and intimidated by the thought of having to deal with numbers. If I had to use five words to describe some best practices and some core principles of measuring usability, here they are: multi-method, triangulate, redundancy, actions, and attitudes.[Read More]

It

10 Ways to Conduct Usability Tests with Credit Cards & Personal Data

Jeff Sauro • January 21, 2014

It's difficult, and often prohibited, to ask users to use their own information during a usability test. Here are some options that help us get good usability data while working within the constraints of purchasing, logging in, or working with users' sensitive personal data.[Read More]

There are many legitimate criticisms from conclusions drawn using quantitative methods. But often disagreements are rooted more in politics than from p-values. Here are some thoughts on dealing with civil and uncivil criticism when presenting quantitative methods.

How to Deal With Quantitative Criticism

Jeff Sauro • January 14, 2014

There are many legitimate criticisms from conclusions drawn using quantitative methods. But often disagreements are rooted more in politics than from p-values. Here are some thoughts on dealing with civil and uncivil criticism when presenting quantitative methods.[Read More]

An analysis of 500 users

Usability & Net Promoter Benchmarks for Health Insurance Websites

Jeff Sauro • January 7, 2014

An analysis of 500 users' recent experiences across four major healthcare provider websites provides usability and Net Promoter Score benchmarks. We also collected data on what users were trying to accomplish and the major areas for improvement.[Read More]

In 2013 we featured both primary research and relevant research published in peer reviewed journals and websites. Our articles were served-up 2.2. million times to 895k visitors in 2013. Here are the ten articles that received most of those page views.

The Top 10 UX Articles of 2013

Jeff Sauro • December 30, 2013

In 2013 we featured both primary research and relevant research published in peer reviewed journals and websites. Our articles were served-up 2.2. million times to 895k visitors in 2013. Here are the ten articles that received most of those page views.[Read More]

There are a lot of misconceptions about when it is and when it is not appropriate to test with five users. Here are five examples of what you can and cannot learn from just a handful of users in a usability test.

5 Reasons You Should and Should Not Test With 5 Users

Jeff Sauro • December 17, 2013

There are a lot of misconceptions about when it is and when it is not appropriate to test with five users. Here are five examples of what you can and cannot learn from just a handful of users in a usability test.[Read More]

How important is the first choice users make when navigating a website? Turns out it

The Importance of the First Choice in Website Navigation

Jeff Sauro • December 10, 2013

How important is the first choice users make when navigating a website? Turns out it's even more important than we thought. Across 8 tree-test studies we found users were more than six times as likely to succeed in finding an item in a navigation structure if their first choice was down the optimal path.[Read More]

A one year longitudinal study of iPhone usage found interesting patterns in behavior and differences by socioeconomic status.

How Smartphone Usage Changes Over Time and By Income Group

Jeff Sauro • December 3, 2013

A one year longitudinal study of iPhone usage found interesting patterns in behavior and differences by socioeconomic status.

[Read More]

A detailed infographic covering many of the key events, publications and people that have shaped the profession of usability for the last 100 years.

Timeline of Usability Infographic

Jeff Sauro • November 20, 2013

A detailed infographic covering many of the key events, publications and people that have shaped the profession of usability for the last 100 years.[Read More]

See how well you can answer some questions on the core concepts of usability, with an emphasis on usability evaluation.

Usability 101 Quiz

Jeff Sauro • November 12, 2013

See how well you can answer some questions on the core concepts of usability, with an emphasis on usability evaluation.

[Read More]

Focus on decisions and not deliverables. It

5 Tips for Applying Lean UX to User Research

Jeff Sauro • November 5, 2013

Focus on decisions and not deliverables. It's one of the main concepts around Lean thinking in general, and Lean UX in particular. Bloated test plans, power point presentations, pixel-perfect concepts and all the clutter that comes from assembling a team to develop an application or website. Here are five tips for helping make your user research lean.[Read More]

An analysis of problems from nine usability test across different devices revealed no correlation between problem frequency and severity. This means that the first few users in a usability test are just as likely to encounter minor as much as critical problems.

Are Severe Problems Uncovered Earlier in Usability Tests?

Jeff Sauro • October 30, 2013

An analysis of problems from nine usability test across different devices revealed no correlation between problem frequency and severity. This means that the first few users in a usability test are just as likely to encounter minor as much as critical problems.[Read More]

When designing an application, website or product, three things help generate a more usable experience: an early focus on the users and tasks, empirical measurement, and iterative design. These principles were articulated almost 30 years ago in one of the most influential papers in usability. They are just as relevant today.

Designing for Usability: 3 Key Principles

Jeff Sauro • October 22, 2013

When designing an application, website or product, three things help generate a more usable experience: an early focus on the users and tasks, empirical measurement, and iterative design. These principles were articulated almost 30 years ago in one of the most influential papers in usability. They are just as relevant today.[Read More]

There are four major design types with relevance to user research : experimental, quasi-experimental, correlational and single subject. These research designs proceed from a level of high validity and generalizability to ones with lower validity and generalizability.

4 Experiment Types for User Research

Jeff Sauro • October 15, 2013

There are four major design types with relevance to user research : experimental, quasi-experimental, correlational and single subject. These research designs proceed from a level of high validity and generalizability to ones with lower validity and generalizability.[Read More]

There are a lot of methods to use to improve the user experience. We detailed many of them in an earlier blog. We know many of you are visual learners so we created an infographic of when to use which UX method.

UX Methods Infographic

Jeff Sauro • October 9, 2013

There are a lot of methods to use to improve the user experience. We detailed many of them in an earlier blog. We know many of you are visual learners so we created an infographic of when to use which UX method.[Read More]

A usability test plan should not only lay out the framework of the study, but also help identify problems with the methodology, metrics or tasks while something can still be done to fix things. This blog describes the common sections we use.

What Goes into a Usability Test Plan?

Jeff Sauro • October 1, 2013

A usability test plan should not only lay out the framework of the study, but also help identify problems with the methodology, metrics or tasks while something can still be done to fix things. This blog describes the common sections we use. [Read More]

It

6 Steps to Identifying Usability Problems

Jeff Sauro • September 17, 2013

It's relatively easy to spot when users have problems in an interface. It's a lot harder to know what, if anything, to do about them. To identify usability problems more efficiently, start by recording the unintended user actions, identify what in the interface is causing them, assign a severity rating and report the frequency in a problem list with screen shots and context to generate design solutions.[Read More]

Who are the users and what are they trying to do? Answering those two questions are essential first steps to measuring and improving the right things on an interface. A top-tasks analysis is a quick and effective method for obtaining which critical few tasks, among dozens, are the ones a website or software product needs to do well.

How to Conduct a Top Task Analysis

Jeff Sauro • September 10, 2013

Who are the users and what are they trying to do? Answering those two questions are essential first steps to measuring and improving the right things on an interface. A top-tasks analysis is a quick and effective method for obtaining which critical few tasks, among dozens, are the ones a website or software product needs to do well.[Read More]

Showing the latest 30 posts. Show All 256 Blog Posts

Newsletter Sign Up

Receive bi-weekly updates.
[3813 Subscribers]

Connect With Us

Our Supporters

Usertesting.com

Loop11 Online Usabilty Testing

Use Card Sorting to improve your IA

Userzoom: Unmoderated Usability Testing, Tools and Analysis

About Jeff Sauro

Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 4 books on statistics and the user-experience.
More about Jeff...

.

Jeff's Books

Quantifying the User Experience: Practical Statistics for User ResearchQuantifying the User Experience: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.