Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

Usability as a Key Driver of Revenue

Jeff Sauro • March 8, 2011

Many companies understand the importance of user centered design in theory.

Few can track how money invested in improving the user experience translates into profitability.

By quantifying design efforts and outcomes all organizations can benefit from understanding how improving the user experience can improve revenue.

The Engine of Growth

If you can imagine revenue being the final gear in a business machine designed to produce profits, we can think of the smaller gears as all the possible variables that crank out the revenue.  While there are a large number of possible variables: competition, branding, the economy, price and perceived worth to name a few, there are usually a few critical variables that disproportionally explain a product's success or failure.

While there's not a one-size fits-all formula, I have found perceptions of utility, quality, value, usability and loyalty to have the most explanatory power after examining survey and revenue data from several software products and websites.



The green gears are the outcomes that really matter for business investments. I've segmented them into short-term and long-term profits because many business decisions can cut-costs or boost sales in the short-term but have a deleterious effect on long term viability (e.g. cutting support or advertising).

The blue gears in the figure are measured from closed-ended survey questions. In more detail they are:

Utility: The gateway driver. If your product doesn't do something with perceived value the gears don't move.

Quality & Value: Once there is perceived value you have to deliver a consistent and high-quality product at the right price to continue generating revenue

Usability: Perceived usability turns with quality and value in driving both short-term revenue and as a key driver of customer loyalty.

Loyalty: Typically measured using the Net Promoter Score, such measures of the customer experience are leading indicators of revenue.  The NPS isn't a perfect metric but some research has established it as a reasonable link to long term growth.  Many companies now use it as a proxy for revenue and pay bonuses on how much it increases.

Short & Long Term Revenue: For products with short purchase cycles and repeat purchases (e.g. e-commerce websites) it is easier to measure revenue directly. For products with longer purchase cycles or one-time purchases, changes in revenue are better measured as increases or decreases in maintenance contracts or upgrades.

The gray gear represents unmeasured variables or difficult to quantify factors that are either out of our control or that we are unaware of.  It reminds us that even a systematic process of quantifying inputs cannot perfectly predict future outcomes.

Key Drivers

Separating the critical few variables from the trivial many that best explain a product's success or failure is determined using Key-Driver Analysis.

Key-Driver Analysis uses Multiple Regression Analysis to mathematically determine which variables predict your outcome variable the best (using standardized beta-coefficients) and they provide the relative importance of each variable.

Typically the outcome variable is the likelihood to recommend question (as used in the NPS) as a proxy for revenue and the predictor variables are questions that ask about usability, value, quality, utility and a potpourri of other product and service questions. 

For a variable to be a key driver means it needs to add information not contained in the other variables. I have consistently found this to be the case with ease of use questions. Customer attitudes about how difficult or easy a product is tend to go beyond their attitudes about quality, value, functionality and price.

The gears above are shown in approximate proportion to their contribution. This means that utility, quality and value are typically bigger contributors that usability.  Despite being a lessor contributor, there are typically a few easy-to-identify fixes that can move this gear.

This is one of the benefits of a Key Driver Analysis, instead of prioritizing based on company politics or intuition, you use data from your customers.

Moving the Gears

Each gear in the machine represents an outcome that can't be moved directly. Instead, you have to change the product, service, customer support, pricing, features or other factors to move the gears.

Correlation is of course not causation. So while it is simpler to think in terms of the gears moving in one-direction to drive profits, it is also likely that higher revenue from successful products cast a Halo on attitudinal metrics.  That means high satisfaction scores could just be a result of a successful product and not a cause of the success.

So while there are flaws with this model (as there are with all models) I find it useful for prioritizing tactical activities to support long term strategic goals and helping establish a program for measuring ease of use and ultimately tracking the return on UX investments.

Moving the Usability Gears

Reductions in the number of interaction problems, time to complete tasks and increases in task completion rates are examples of metrics tied to tactical activities that ultimately move the usability gear. These changes can be tracked using a questionnaire like the System Usability Scale (SUS) or Standardized Usability Percentile Rank (SUPR-Q) which are loosely tied to task-performance metrics and user-interface problems. 

You can think of these tactical changes as very-small gears inside the usability gear.  By making small incremental changes over-time and measuring the movement in SUS scores, these small movements to the gears will help drive the engine of profits.

Measuring the Impact of Usability Investments

Here's the minimum you need to do to get started measuring the impact of usability on your product:
  1. Measure Revenue or another key business metric: repeat business, renewed maintenance agreements or new customers. If you don't have access to revenue data find a good proxy like Net Promoter Scores (ideally use both).

  2. Survey customers about their attitudes around product features, customer support and services. Be sure to include questions about usability, value and quality--keep it as short and simple as possible.

  3. Conduct a Key-Driver Analysis to identify which attributes have the biggest impact on loyalty and revenue. Usability will likely be one of your key-drivers and something you can take actionable steps to improve. Despite asking your customer many questions, you'll be surprised how just a few attributes predict most of the changes in loyalty.

  4. Track UX inputs and outputs: There is always a long list of fixes and enhancements. Instead of picking the oldest or easiest ones to implement, identify the high-impact changes by examining the verbatim comments from the survey responses. Then setup metrics to track the positive impact on design changes.  Finally, track the time and money spent on testing users, conducting site-visits and other UX activities.

Following these steps allows you to more effectively prioritize your UX efforts and establish a link to revenue.  Contact Measuring Usability LLC for help getting started surveying your customers, conducting a Key Driver Analysis or establishing an effective system for measuring your product's user experience.

About Jeff Sauro

Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 4 books on statistics and the user-experience.
More about Jeff...


Learn More


UX Bootcamp: Aug 20th-22nd in Denver, CO
Best Practices for Remote Usability Testing
The Science of Great Site Navigation: Online Card Sorting + Tree Testing Live Webinar


You Might Also Be Interested In:

Related Topics

Return on Investment, Key Drivers, UX
.

Posted Comments

There are 6 Comments

June 5, 2012 | Tactical Gear wrote:


excellent information keep up your good work thanks.
 


June 5, 2012 | Tactical Gear wrote:

Hello. impressive job. I did not anticipate this. This is a fantastic story. Thanks!
 


March 20, 2011 | Carolyn King wrote:

Yes I think the gears would get stuck in that diagram! But the point is well made - all of the gears have to turn in order to get results, and if any one of them stops turning the system grinds to a halt. It looks like a very useful model for assessing dependencies and priorities. 


March 18, 2011 | Jeff Sauro wrote:

Alexix,
LOL. Great point. I never tested the mechanical integrity of that diagram! 


March 17, 2011 | Ravi Kumar Verma wrote:

What a awsome explanation. Gr8 appreciate your simple and deep language 


March 17, 2011 | Alexis wrote:

The gears diagram says "total system lock" to me. 


Post a Comment

Comment:


Your Name:


Your Email Address:


.

To prevent comment spam, please answer the following :
What is 5 + 2: (enter the number)

Newsletter Sign Up

Receive bi-weekly updates.
[4080 Subscribers]

Connect With Us

UX Bootcamp

Denver CO, Aug 20-22nd 2014

3 Days of Hands-On Training on User Experience Methods, Metrics and Analysis.Learn More

Our Supporters

Loop11 Online Usabilty Testing

Use Card Sorting to improve your IA

Userzoom: Unmoderated Usability Testing, Tools and Analysis

Usertesting.com

.

Jeff's Books

Quantifying the User Experience: Practical Statistics for User ResearchQuantifying the User Experience: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.