Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

Is that a Usability Problem or a Feature?

Jeff Sauro • July 19, 2011

If you ask independent usability evaluators to run a usability test and report the problems found you'll get largely different lists of problems.

While there are many causes for the differences, one major reason is that evaluators disagree on what constitutes a problem.

Usability is often at odds with security and business interests—what's best for the user may not be the best for the organization. 

If a clear source of revenue or customer-leads interferes with users goals, is it a usability problem? And should these events even be reported by usability evaluators?


A common place where this collision occurs is with default choices.   The simple decision on what choice to preselect (if any) has real consequences. There are books written about defaulting retirement options and organ donation. It's no wonder that such issues often get called usability issues by some and features by others.

Default Check Box for Sign Up

Email newsletters continue to be a major channel for online marketing and sales. Is pre-selecting the check box a usability problem or a feature?

Optional Insurance Protection

Default Option Selected

In the recent Comparative Usability Evaluation, Rolf Molich reported that only 1 of 19 professional usability experts that watched the same video of users renting a U-Haul reported the following as a usability issue. The default option for Damage Protection doesn't cover the costs of the damage to other people or property and can be insufficient.

Is this a usability problem or is providing the default with some coverage the right choice?

No Option Selected

When you purchase an airline ticket, you're often presented with an option to purchase Trip Cancellation insurance.  On, there's no choice defaulted.  I suspect they could really increase revenues by defaulting the optional insurance to "Yes."  But would that make it a usability issue?


An Encouraged Option

As an interesting compromise, offers the "highly recommended" option for the optional Loss Damage Waiver protection. It isn't selected, but it's as encouraged as the GPS.

Links to Ads

When you click on a link to Yahoo Mail from a page, it doesn't take you to your Inbox as you'd expect, it instead takes you to a "What's New" page loaded with ads and requires another link to your Inbox. 

Pageviews matter a lot for ad-revenue. But will this frustrate too many users?  Is this a feature or usability problem?

The Upsell

In other cases it's not the default options, it's by-passing all the add-ons and up-sells.  For example, if you want to renew your hosting with GoDaddy, you have to pass through at least 4 pages and on one page alone you have to scroll past 15 check-boxes, 15 drop-down lists to get to the "No Thanks" button. 

Requiring all this scrolling and distracting options increases tasks times and may lead to more cart-abandonment. But I suspect these are lucrative revenue opportunities for GoDaddy.  Is this a Usability Problem or Feature?

The UpSell with Ambiguous Default

Jared Spool discussed balancing business and usability and provided the example of the Award accelerator.  The upsell is both defaulted and the opt-out selection is clearly dwarfed by the giant "Add Option" button. Is this a usability problem or a feature?

 Image used shamelessly without permission.

Security Concerns

Online banking and credit cards are on the front-lines in the battle between usability and security. Just to login you usually have to remember usernames, passwords, security questions and security images.

Then, to make a payment you often have to agree to terms and conditions, submit the payment then confirm the payment.    Is requiring 3-4 steps to make a payment a usability issue or a security feature?

Usability and Business Interests aren't mutually exclusive

There has to be a balance between usability, business and security.  Even the most intransigent usability wonk would agree to that. But when there's too little usability and too much business someone has to sound the alarm. And while security is essential, it shouldn't discourage usage.

Successful companies know how to provide long-term value to a customer. If users feel like you tricked them into a purchase or signing-up, you may have helped your numbers in the short-term, but at what cost? 

Revenue is a lagging indicator and it's often too late to fix problems once they show up on balance sheets.  Measures of customer loyalty are driven by usability, so a usability evaluator should report on usability problems (or any problem) even when they conflict with clear streams of income.

However, it should come as no surprise that business leaders want to understand the impact on revenue before making design changes. That's an opportunity to change the conversation from usability versus revenue to short-term revenue at the expense of long-term profits.


About Jeff Sauro

Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 4 books on statistics and the user-experience.
More about Jeff...

Learn More

You Might Also Be Interested In:

Related Topics

Usability Problems

Posted Comments

There are 1 Comments

September 22, 2011 | Ryan Hickman wrote:

I found your article very insightful. Great use of screenshots to convey your points. I can certainly relate to wanting to abandon my GoDaddy cart after being harassed for upsells.

You may be interested in a usability article posted by my boss here at Back40 Design. 

Post a Comment


Your Name:

Your Email Address:


To prevent comment spam, please answer the following :
What is 1 + 4: (enter the number)

Newsletter Sign Up

Receive bi-weekly updates.
[4292 Subscribers]

Connect With Us

Our Supporters

Use Card Sorting to improve your IA

Loop11 Online Usabilty Testing

Userzoom: Unmoderated Usability Testing, Tools and Analysis


Jeff's Books

Quantifying the User Experience: Practical Statistics for User ResearchQuantifying the User Experience: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download