Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

Blogs & Articles

At our company, we collect and examine a lot of data from various studies: usability tests, branding studies, customer-segmentation analyses, and so on. How do we get the most out of that data? We use the following six techniques to gather the most from our data.

6 Tips for Better Data Analysis

Jeff Sauro • September 29, 2014

At our company, we collect and examine a lot of data from various studies: usability tests, branding studies, customer-segmentation analyses, and so on. How do we get the most out of that data? We use the following six techniques to gather the most from our data.[Read More]

An innovative product doesn

8 Ways to Identify Unmet Customer Needs

Jeff Sauro • September 23, 2014

An innovative product doesn't come from a law passed by the government. It also doesn't come from venture capitalists looking for a higher return on an investment. Innovation comes from identifying customers' needs and providing a solution that meets that need. Here are 8 ways to help identify unmet customer needs and develop innovative solutions.[Read More]

With the recent adoption of unmoderated remote testing methods, usability testing can be done for a fraction of the cost and time. Advances in technology mean we can also see and hear participants as they actually use the Internet. But the reduced time and cost of data collection does not mean a sloppy study design. The more accurate picture of users is painted by research that balances the tenets of good experimental design with the new means of data collection. While research into the reliability of panel research continues to evolve, we can now put a face to all that data!

The New Face of Usability Testing

Jeff Sauro • September 16, 2014

With the recent adoption of unmoderated remote testing methods, usability testing can be done for a fraction of the cost and time. Advances in technology mean we can also see and hear participants as they actually use the Internet. But the reduced time and cost of data collection does not mean a sloppy study design. The more accurate picture of users is painted by research that balances the tenets of good experimental design with the new means of data collection. While research into the reliability of panel research continues to evolve, we can now put a face to all that data![Read More]

If you

7 Techniques for Prioritizing Customer Requirements

Jeff Sauro • September 9, 2014

If you're introducing or improving a product or website, you probably have an impossibly long list of features you'd like to fix, improve, or add. Prioritizing this list is essential to the success of your product and potentially business. But what's the best way to approach this? Here are seven techniques to help you along the way.[Read More]

Confidence intervals are your frenemies. They are one of the most useful statistical techniques you can apply to customer data. At the same time they can be perplexing and cumbersome. Here is a peek behind the statistical curtain to assure you it

How to Compute a Confidence Interval in 5 Easy Steps

Jeff Sauro • September 3, 2014

Confidence intervals are your frenemies. They are one of the most useful statistical techniques you can apply to customer data. At the same time they can be perplexing and cumbersome. Here is a peek behind the statistical curtain to assure you it's not black magic or quantum mechanics. Use these five steps to compute your next confidence interval.[Read More]

Privacy, disclosure and anonymity are necessary but not sufficient for the ethical treatment of users. As user advocates we can help companies take a step further by improving what users comprehend. At some point bad business practices turn into unethical practices. When that happens is not always clear ahead of time. With some transparency and improved comprehension coming from the industry itself, I suspect many of these problems introduced by the technology will self-correct.

The Experiment Requires That You Continue: On The Ethical Treatment of Users

Jeff Sauro • August 26, 2014

Privacy, disclosure and anonymity are necessary but not sufficient for the ethical treatment of users. As user advocates we can help companies take a step further by improving what users comprehend. At some point bad business practices turn into unethical practices. When that happens is not always clear ahead of time. With some transparency and improved comprehension coming from the industry itself, I suspect many of these problems introduced by the technology will self-correct.[Read More]

The field of user experience encompasses a wide array of individual jobs including designers, researchers, information architects and product developers.  If you

28 Resources for Getting Started In UX

Jeff Sauro • August 18, 2014

The field of user experience encompasses a wide array of individual jobs including designers, researchers, information architects and product developers. If you're interested in breaking into the field, or just want to see what all the fuss is about, here is a list of resources to get started.[Read More]

In an ideal world, users would be involved in every stage of product development, including requirements gathering, iterative prototype testing and post release testing. But sometimes testing with users just doesn

What To Do When You Can't Run a Usability Test

Jeff Sauro • August 5, 2014

In an ideal world, users would be involved in every stage of product development, including requirements gathering, iterative prototype testing and post release testing. But sometimes testing with users just doesn't happen. Here are three alternatives we use to understand the user experience without usability testing.[Read More]

There are some good metrics to collect and a lot to keep track of when measuring, managing and improving the customer experience. Here are 10 methods we use to improve the customer

10 Methods To Improve the Customer Experience

Jeff Sauro • July 30, 2014

There are some good metrics to collect and a lot to keep track of when measuring, managing and improving the customer experience. Here are 10 methods we use to improve the customer's experience on websites, mobile apps, software and hardware.[Read More]

There are some common criticisms for using the popular Net Promoter Score as a measure of company growth. Despite its shortcomings, the idea of measuring word of mouth and having a simple metric that companies can compare provides more gain than pain.  The challenge is not in how to write the questions or response options, the challenge is in actually doing something with the information.

Should the Net Promoter Score Go? 5 Common Criticisms Examined

Jeff Sauro • July 22, 2014

There are some common criticisms for using the popular Net Promoter Score as a measure of company growth. Despite its shortcomings, the idea of measuring word of mouth and having a simple metric that companies can compare provides more gain than pain. The challenge is not in how to write the questions or response options, the challenge is in actually doing something with the information.[Read More]

A comparative usability evaluation requires more planning and thought to account for unwanted variables than a stand-alone study. Carefully setting up the study and analyzing the results is important if you plan on publishing your results, or if the findings have significant consequences. This blog discussed 5 common threats to reliability and validity and 3 things to do about them.

5 Variables to Manage in a Comparative Usability Study

Jeff Sauro • July 15, 2014

A comparative usability evaluation requires more planning and thought to account for unwanted variables than a stand-alone study. Carefully setting up the study and analyzing the results is important if you plan on publishing your results, or if the findings have significant consequences. This blog discussed 5 common threats to reliability and validity and 3 things to do about them.[Read More]

When companies profit from bad experiences they are considered bad profits. Customers who are dissatisfied with the service or quality of a product are not only less likely to repurchase it, they are also more likely to tell their friends about the bad experience.  Ideally, less than 10% of revenue should come from bad profits.

Getting Revenue From Unhappy Customers

Jeff Sauro • July 9, 2014

When companies profit from bad experiences they are considered bad profits. Customers who are dissatisfied with the service or quality of a product are not only less likely to repurchase it, they are also more likely to tell their friends about the bad experience. Ideally, less than 10% of revenue should come from bad profits.[Read More]

While the "best" metrics depend on your goals and specific context, here is a list of 10 that most organizations should collect. They include a mix of the four types of customer analytics to collect: descriptive, behavioral, interaction and attitudinal.

10 Customer Metrics You Should Collect

Jeff Sauro • July 1, 2014

While the "best" metrics depend on your goals and specific context, here is a list of 10 that most organizations should collect. They include a mix of the four types of customer analytics to collect: descriptive, behavioral, interaction and attitudinal.[Read More]

Understanding your customers--their similarities, their differences--is one of the most fundamental and important steps in quantifying the customers

Why and How To Segment Your Customers

Jeff Sauro • June 24, 2014

Understanding your customers--their similarities, their differences--is one of the most fundamental and important steps in quantifying the customers' relationship with your product and company. It leads to better products, more targeted marketing and meeting unmet needs.[Read More]

An analysis of 1,023 users

The User Experience of Airline & Aggregator Websites

Jeff Sauro • June 17, 2014

An analysis of 1,023 users' responses to their most recent experiences on 9 US based airline or aggregator websites. This blog reviews overall UX Quality scores (SUPR-Q), Net Promoter Scores and details on the positive and negative interactions that contribute to the overall experience.[Read More]

There are a number of methods to improve the usability of an interface.  While it

The Most Fundamental Concept in Usability

Jeff Sauro • June 10, 2014

There are a number of methods to improve the usability of an interface. While it's hard to identify one overarching concept that's fundamental to the whole idea of usability, I think there's one that underlies most methods and desirable outcomes. That concept is that the developer is not the user.[Read More]

While conducting a usability test on a mobile device shares a lot of the same characteristics as a desktop usability study, there are some additional factors to deal with. Here are five tips to help make the most of your mobile study.

5 Tips for Conducting a Mobile Usability Test

Jeff Sauro • June 3, 2014

While conducting a usability test on a mobile device shares a lot of the same characteristics as a desktop usability study, there are some additional factors to deal with. Here are five tips to help make the most of your mobile study.[Read More]

Technology is the amazing result of humanity

10 Surprisingly Common But Painful User Experiences

Jeff Sauro • May 27, 2014

Technology is the amazing result of humanity's evolution from self-sufficient hunter-gathering societies to increasingly interconnected social organizations. Yet sometimes you feel like there's no humanity in some of our online and offline experiences. Inspired by some more general frustrations, we've quantified 10 surprisingly common but generally painful experiences you've almost certainly encountered.[Read More]

Finding and fixing usability problems is one of the most effective ways for improving the user experience on websites, applications and hardware.  But just because a problem is identified in a usability test doesn

7 Reasons Usability Problems Don't Get Fixed

Jeff Sauro • May 20, 2014

Finding and fixing usability problems is one of the most effective ways for improving the user experience on websites, applications and hardware. But just because a problem is identified in a usability test doesn't mean it gets fixed. Here are seven reasons why usability problems often don't get fixed.[Read More]

This blog presents the highlights of an analysis of UX metrics, Net Promoter Scores and insights from 800 users across 10 popular US retail websites from data collected in Q4 2013.

Retail Website Usability & Net Promoter Benchmarks

Jeff Sauro • May 13, 2014

This blog presents the highlights of an analysis of UX metrics, Net Promoter Scores and insights from 800 users across 10 popular US retail websites from data collected in Q4 2013.[Read More]

While similar to usability in the Business to Consumer (B2C) sector, Business to Business (B2B) usability differs in six key ways: user is not the buyer, customization, hard to find users, training, switching costs and poorer experiences.

6 Differences Between B2B and B2C Usability

Jeff Sauro • May 5, 2014

While similar to usability in the Business to Consumer (B2C) sector, Business to Business (B2B) usability differs in six key ways: user is not the buyer, customization, hard to find users, training, switching costs and poorer experiences.[Read More]

A customer journey map is a natural extension of the sales and marketing funnel  but has an impact beyond the marketing or sales teams.  It

10 Steps to Mapping the Customer Journey

Jeff Sauro • April 29, 2014

A customer journey map is a natural extension of the sales and marketing funnel but has an impact beyond the marketing or sales teams. It's a document meant to unify fragmented efforts and, most importantly, identify points of friction and opportunities for improvement. Finding and fixing the pain points in a customer's journey isn't just about damage control, it's about the innovation that comes from fixing the pain.[Read More]

Before measuring anything, especially a construct that

5 Questions to Answer Before Measuring Anything

Jeff Sauro • April 22, 2014

Before measuring anything, especially a construct that's not well defined or used in practice, we answer these five questions: How is this being measured already?, What will we compare this to? How reliable and valid are the measures? How precise do we need to be? What will we do with the results? [Read More]

Missing data in a survey is a pain. What

Identifying the 3 Types of Missing Data

Jeff Sauro • April 15, 2014

Missing data in a survey is a pain. What's worse is if the missing values are systematically excluding a segment of your customers. This blog describes a technique to understand if your data is missing at random, missing completely at random or systematically missing.[Read More]

UX researchers have developed many techniques over the years for testing and validating their ideas.  Here are ten essential methods you can employ on your next project.

10 Essential User Experience Methods

Jeff Sauro • April 7, 2014

UX researchers have developed many techniques over the years for testing and validating their ideas. Here are ten essential methods you can employ on your next project.[Read More]

Measuring Usability has partnered with leading user experience firms to create Cindy (Consistent INterview Drone for usabilitY), the first autonomous usability test facilitator.

Automated Lab Based Usability Testing

Jeff Sauro • April 1, 2014

Measuring Usability has partnered with leading user experience firms to create Cindy (Consistent INterview Drone for usabilitY), the first autonomous usability test facilitator.[Read More]

Every time researchers conduct a usability test to uncover problems they

What the NCAA Tournament & Usability Testing Have in Common

Jeff Sauro • March 24, 2014

Every time researchers conduct a usability test to uncover problems they're also working with probabilities, even if they tell you they hate math! To understand the role of probabilities in usability testing it helps to see how they are used when picking winning teams for the NCAA tournament.[Read More]

Most people are comfortable with the concept of an average or percentage as a measure of quality. An equally important component of measuring the user experience is to understand variability. Here are 10 things to know about measuring variability in the user experience.

10 Things to Know About Variability in the User Experience

Jeff Sauro • March 18, 2014

Most people are comfortable with the concept of an average or percentage as a measure of quality. An equally important component of measuring the user experience is to understand variability. Here are 10 things to know about measuring variability in the user experience.[Read More]

When using statistics to make comparisons between designs, it

Understanding Effect Sizes in User Research

Jeff Sauro • March 11, 2014

When using statistics to make comparisons between designs, it's not enough to just say differences are statistically significant. Effect sizes provide a standardized way to understand the size of the difference across all types of metrics.[Read More]

To get a better understanding of problems and opportunities for your customers, you

4 Types of Customer Analytics Data to Collect

Jeff Sauro • March 4, 2014

To get a better understanding of problems and opportunities for your customers, you'll want to collect data from each of the following four customer analytics groups: descriptive, behavioral, interaction, and attitudinal.[Read More]

Showing the latest 30 posts. Show All 278 Blog Posts

Newsletter Sign Up

Receive bi-weekly updates.
[4332 Subscribers]

Connect With Us

Our Supporters

Userzoom: Unmoderated Usability Testing, Tools and Analysis

Loop11 Online Usabilty Testing

Use Card Sorting to improve your IA

Usertesting.com

About Jeff Sauro

Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 4 books on statistics and the user-experience.
More about Jeff...

.

Jeff's Books

Quantifying the User Experience: Practical Statistics for User ResearchQuantifying the User Experience: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.