Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

Blogs & Articles

There are some good metrics to collect and a lot to keep track of when measuring, managing and improving the customer experience. Here are 10 methods we use to improve the customer

10 Methods To Improve the Customer Experience

Jeff Sauro • July 30, 2014

There are some good metrics to collect and a lot to keep track of when measuring, managing and improving the customer experience. Here are 10 methods we use to improve the customer's experience on websites, mobile apps, software and hardware.[Read More]

There are some common criticisms for using the popular Net Promoter Score as a measure of company growth. Despite its shortcomings, the idea of measuring word of mouth and having a simple metric that companies can compare provides more gain than pain.  The challenge is not in how to write the questions or response options, the challenge is in actually doing something with the information.

Should the Net Promoter Score Go? 5 Common Criticisms Examined

Jeff Sauro • July 22, 2014

There are some common criticisms for using the popular Net Promoter Score as a measure of company growth. Despite its shortcomings, the idea of measuring word of mouth and having a simple metric that companies can compare provides more gain than pain. The challenge is not in how to write the questions or response options, the challenge is in actually doing something with the information.[Read More]

A comparative usability evaluation requires more planning and thought to account for unwanted variables than a stand-alone study. Carefully setting up the study and analyzing the results is important if you plan on publishing your results, or if the findings have significant consequences. This blog discussed 5 common threats to reliability and validity and 3 things to do about them.

5 Variables to Manage in a Comparative Usability Study

Jeff Sauro • July 15, 2014

A comparative usability evaluation requires more planning and thought to account for unwanted variables than a stand-alone study. Carefully setting up the study and analyzing the results is important if you plan on publishing your results, or if the findings have significant consequences. This blog discussed 5 common threats to reliability and validity and 3 things to do about them.[Read More]

When companies profit from bad experiences they are considered bad profits. Customers who are dissatisfied with the service or quality of a product are not only less likely to repurchase it, they are also more likely to tell their friends about the bad experience.  Ideally, less than 10% of revenue should come from bad profits.

Getting Revenue From Unhappy Customers

Jeff Sauro • July 9, 2014

When companies profit from bad experiences they are considered bad profits. Customers who are dissatisfied with the service or quality of a product are not only less likely to repurchase it, they are also more likely to tell their friends about the bad experience. Ideally, less than 10% of revenue should come from bad profits.[Read More]

While the "best" metrics depend on your goals and specific context, here is a list of 10 that most organizations should collect. They include a mix of the four types of customer analytics to collect: descriptive, behavioral, interaction and attitudinal.

10 Customer Metrics You Should Collect

Jeff Sauro • July 1, 2014

While the "best" metrics depend on your goals and specific context, here is a list of 10 that most organizations should collect. They include a mix of the four types of customer analytics to collect: descriptive, behavioral, interaction and attitudinal.[Read More]

Understanding your customers--their similarities, their differences--is one of the most fundamental and important steps in quantifying the customers

Why and How To Segment Your Customers

Jeff Sauro • June 24, 2014

Understanding your customers--their similarities, their differences--is one of the most fundamental and important steps in quantifying the customers' relationship with your product and company. It leads to better products, more targeted marketing and meeting unmet needs.[Read More]

An analysis of 1,023 users

The User Experience of Airline & Aggregator Websites

Jeff Sauro • June 17, 2014

An analysis of 1,023 users' responses to their most recent experiences on 9 US based airline or aggregator websites. This blog reviews overall UX Quality scores (SUPR-Q), Net Promoter Scores and details on the positive and negative interactions that contribute to the overall experience.[Read More]

There are a number of methods to improve the usability of an interface.  While it

The Most Fundamental Concept in Usability

Jeff Sauro • June 10, 2014

There are a number of methods to improve the usability of an interface. While it's hard to identify one overarching concept that's fundamental to the whole idea of usability, I think there's one that underlies most methods and desirable outcomes. That concept is that the developer is not the user.[Read More]

While conducting a usability test on a mobile device shares a lot of the same characteristics as a desktop usability study, there are some additional factors to deal with. Here are five tips to help make the most of your mobile study.

5 Tips for Conducting a Mobile Usability Test

Jeff Sauro • June 3, 2014

While conducting a usability test on a mobile device shares a lot of the same characteristics as a desktop usability study, there are some additional factors to deal with. Here are five tips to help make the most of your mobile study.[Read More]

Technology is the amazing result of humanity

10 Surprisingly Common But Painful User Experiences

Jeff Sauro • May 27, 2014

Technology is the amazing result of humanity's evolution from self-sufficient hunter-gathering societies to increasingly interconnected social organizations. Yet sometimes you feel like there's no humanity in some of our online and offline experiences. Inspired by some more general frustrations, we've quantified 10 surprisingly common but generally painful experiences you've almost certainly encountered.[Read More]

Finding and fixing usability problems is one of the most effective ways for improving the user experience on websites, applications and hardware.  But just because a problem is identified in a usability test doesn

7 Reasons Usability Problems Don't Get Fixed

Jeff Sauro • May 20, 2014

Finding and fixing usability problems is one of the most effective ways for improving the user experience on websites, applications and hardware. But just because a problem is identified in a usability test doesn't mean it gets fixed. Here are seven reasons why usability problems often don't get fixed.[Read More]

This blog presents the highlights of an analysis of UX metrics, Net Promoter Scores and insights from 800 users across 10 popular US retail websites from data collected in Q4 2013.

Retail Website Usability & Net Promoter Benchmarks

Jeff Sauro • May 13, 2014

This blog presents the highlights of an analysis of UX metrics, Net Promoter Scores and insights from 800 users across 10 popular US retail websites from data collected in Q4 2013.[Read More]

While similar to usability in the Business to Consumer (B2C) sector, Business to Business (B2B) usability differs in six key ways: user is not the buyer, customization, hard to find users, training, switching costs and poorer experiences.

6 Differences Between B2B and B2C Usability

Jeff Sauro • May 5, 2014

While similar to usability in the Business to Consumer (B2C) sector, Business to Business (B2B) usability differs in six key ways: user is not the buyer, customization, hard to find users, training, switching costs and poorer experiences.[Read More]

A customer journey map is a natural extension of the sales and marketing funnel  but has an impact beyond the marketing or sales teams.  It

10 Steps to Mapping the Customer Journey

Jeff Sauro • April 29, 2014

A customer journey map is a natural extension of the sales and marketing funnel but has an impact beyond the marketing or sales teams. It's a document meant to unify fragmented efforts and, most importantly, identify points of friction and opportunities for improvement. Finding and fixing the pain points in a customer's journey isn't just about damage control, it's about the innovation that comes from fixing the pain.[Read More]

Before measuring anything, especially a construct that

5 Questions to Answer Before Measuring Anything

Jeff Sauro • April 22, 2014

Before measuring anything, especially a construct that's not well defined or used in practice, we answer these five questions: How is this being measured already?, What will we compare this to? How reliable and valid are the measures? How precise do we need to be? What will we do with the results? [Read More]

Missing data in a survey is a pain. What

Identifying the 3 Types of Missing Data

Jeff Sauro • April 15, 2014

Missing data in a survey is a pain. What's worse is if the missing values are systematically excluding a segment of your customers. This blog describes a technique to understand if your data is missing at random, missing completely at random or systematically missing.[Read More]

UX researchers have developed many techniques over the years for testing and validating their ideas.  Here are ten essential methods you can employ on your next project.

10 Essential User Experience Methods

Jeff Sauro • April 7, 2014

UX researchers have developed many techniques over the years for testing and validating their ideas. Here are ten essential methods you can employ on your next project.[Read More]

Measuring Usability has partnered with leading user experience firms to create Cindy (Consistent INterview Drone for usabilitY), the first autonomous usability test facilitator.

Automated Lab Based Usability Testing

Jeff Sauro • April 1, 2014

Measuring Usability has partnered with leading user experience firms to create Cindy (Consistent INterview Drone for usabilitY), the first autonomous usability test facilitator.[Read More]

Every time researchers conduct a usability test to uncover problems they

What the NCAA Tournament & Usability Testing Have in Common

Jeff Sauro • March 24, 2014

Every time researchers conduct a usability test to uncover problems they're also working with probabilities, even if they tell you they hate math! To understand the role of probabilities in usability testing it helps to see how they are used when picking winning teams for the NCAA tournament.[Read More]

Most people are comfortable with the concept of an average or percentage as a measure of quality. An equally important component of measuring the user experience is to understand variability. Here are 10 things to know about measuring variability in the user experience.

10 Things to Know About Variability in the User Experience

Jeff Sauro • March 18, 2014

Most people are comfortable with the concept of an average or percentage as a measure of quality. An equally important component of measuring the user experience is to understand variability. Here are 10 things to know about measuring variability in the user experience.[Read More]

When using statistics to make comparisons between designs, it

Understanding Effect Sizes in User Research

Jeff Sauro • March 11, 2014

When using statistics to make comparisons between designs, it's not enough to just say differences are statistically significant. Effect sizes provide a standardized way to understand the size of the difference across all types of metrics.[Read More]

To get a better understanding of problems and opportunities for your customers, you

4 Types of Customer Analytics Data to Collect

Jeff Sauro • March 4, 2014

To get a better understanding of problems and opportunities for your customers, you'll want to collect data from each of the following four customer analytics groups: descriptive, behavioral, interaction, and attitudinal.[Read More]

It

Is Observing One User Worse Than Observing None?

Jeff Sauro • February 25, 2014

It's possible for product stakeholders to be misled from watching a single session during a usability study. This leads teams to have a 2 or more rule when observing. However, watching just a single user still provides information on the impact of an interface problem. Over time, if stakeholders are consistently watching a single random user in each usability study, odds are they will be more likely to see common problems than freak occurrences.[Read More]

A lot of planning goes into a usability test.  Part of good planning means being prepared for the many things that can go wrong. Here are the ten most common problems we encounter in usability testing and some ideas for how to avoid or manage them when they inevitably occur.

10 Things That Can Go Wrong in a Usability Test

Jeff Sauro • February 18, 2014

A lot of planning goes into a usability test. Part of good planning means being prepared for the many things that can go wrong. Here are the ten most common problems we encounter in usability testing and some ideas for how to avoid or manage them when they inevitably occur.[Read More]

Inevitably one or a few of these survey response rate killers will creep into your next survey project, but sometimes knowing more about them will help reduce or prevent them to help keep your response rate high.

10 Ways to Get a Horrible Survey Response Rate

Jeff Sauro • February 11, 2014

Inevitably one or a few of these survey response rate killers will creep into your next survey project, but sometimes knowing more about them will help reduce or prevent them to help keep your response rate high.[Read More]

Here are four steps to take to get your questionnaire ready for a different language than it was written in: Translate, Back Translate, Pretest and Validate.

4 Steps to Translating a Questionnaire

Jeff Sauro • February 4, 2014

Here are four steps to take to get your questionnaire ready for a different language than it was written in: Translate, Back Translate, Pretest and Validate.[Read More]

While there are books written on measuring usability, it can be easy to get overwhelmed by the details and intimidated by the thought of having to deal with numbers. If I had to use five words to describe some best practices and some core principles of measuring usability, here they are: multi-method, triangulate, redundancy, actions, and attitudes.

Measuring Usability Best Practices in 5 Words

Jeff Sauro • January 28, 2014

While there are books written on measuring usability, it can be easy to get overwhelmed by the details and intimidated by the thought of having to deal with numbers. If I had to use five words to describe some best practices and some core principles of measuring usability, here they are: multi-method, triangulate, redundancy, actions, and attitudes.[Read More]

It

10 Ways to Conduct Usability Tests with Credit Cards & Personal Data

Jeff Sauro • January 21, 2014

It's difficult, and often prohibited, to ask users to use their own information during a usability test. Here are some options that help us get good usability data while working within the constraints of purchasing, logging in, or working with users' sensitive personal data.[Read More]

There are many legitimate criticisms from conclusions drawn using quantitative methods. But often disagreements are rooted more in politics than from p-values. Here are some thoughts on dealing with civil and uncivil criticism when presenting quantitative methods.

How to Deal With Quantitative Criticism

Jeff Sauro • January 14, 2014

There are many legitimate criticisms from conclusions drawn using quantitative methods. But often disagreements are rooted more in politics than from p-values. Here are some thoughts on dealing with civil and uncivil criticism when presenting quantitative methods.[Read More]

An analysis of 500 users

Usability & Net Promoter Benchmarks for Health Insurance Websites

Jeff Sauro • January 7, 2014

An analysis of 500 users' recent experiences across four major healthcare provider websites provides usability and Net Promoter Score benchmarks. We also collected data on what users were trying to accomplish and the major areas for improvement.[Read More]

Showing the latest 30 posts. Show All 270 Blog Posts

Newsletter Sign Up

Receive bi-weekly updates.
[4120 Subscribers]

Connect With Us

Our Supporters

Usertesting.com

Userzoom: Unmoderated Usability Testing, Tools and Analysis

Loop11 Online Usabilty Testing

Use Card Sorting to improve your IA

About Jeff Sauro

Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 4 books on statistics and the user-experience.
More about Jeff...

.

Jeff's Books

Quantifying the User Experience: Practical Statistics for User ResearchQuantifying the User Experience: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.