Jeff Sauro • June 18, 2013
The system usability scale is a ten-item questionnaire administered to users for measuring the perceived ease of use of software, hardware, cell phones and websites. Here are ten things to know about it, including: industry benchmarks, sample sizes, predicting customer loyalty, effects of prior experience and normality.[Read More]
Jeff Sauro • June 11, 2013
Is a beautiful website more usable? There have been consistent findings that show perceptions of usability and appearance are correlated. Some recent research suggests that a more usable website improves perceptions of beauty, rather than beauty improving perceptions of usability. Appearance isn't just a beauty contest; there is strong evidence that more attractive websites generate higher opinions of trust which impacts loyalty and sales.[Read More]
Jeff Sauro • June 4, 2013
The Net Promoter Score is intended to be a proxy for revenue and future growth and is easier to make associations with using user-experience metrics. You can then indirectly tie changes made through early development research efforts to later measurements of customers' likelihood to recommend through such metrics.[Read More]
Jeff Sauro • May 28, 2013
It can be a small scale do-it-yourself test or a large sample corporate test, but finding available users can be a burden. The process by which you find your users will vary depending on what you are testing, the types of users you need, and the stage of testing (early versus late). But to help lessen the burden (and remove an excuse not to conduct usability testing) here are seven sources to help find users.[Read More]
Jeff Sauro • May 21, 2013
While the fundamental metric of findability is whether users find an item or not, often other metrics provide clues to problems in the hierarchy even if users manage to find an item. Here are 10 metrics to collect and dig into when you're looking to improve website navigation.[Read More]
Jeff Sauro • May 14, 2013
You don't need to be a trained psychotherapist to conduct effective usability sessions, but it always helps to refine the art of understanding human behavior and intentions when looking to improve the customer experience. Here are five techniques to work on: Not asking why, not planting ideas, minimizing yes/no questions, reducing the "Would You?" questions and turning questions back to the user. [Read More]
Jeff Sauro • May 7, 2013
For as long as user interfaces have had icons, there have been strong opinions about what makes a good icon. Instead of making decisions based on the pay-grade of the people in a meeting, consider using these seven data driven approaches to help make icons more effective.[Read More]
Jeff Sauro • May 1, 2013
Almost all research contains mistakes in methodology, measurement or interpretation. Rarely do the mistakes render the research completely useless. This blog discussed five of the more common ones. But don't let the fear of mistakes or shortcomings prevent you from conducting new user research.[Read More]
Jeff Sauro • April 23, 2013
To measure whether users understand a price, concept or design you can't just ask them. To measure comprehension use the three R's: recognition (multiple choice), recall (open-response) and recounting (explaining to a friend).[Read More]
Jeff Sauro • April 16, 2013
Crafting task scenarios is a balance between providing enough information and not leading the user. Here are seven tips for crafting a better scenario for usability testing including providing enough context, being specific, using the users' language and having correct solutions.[Read More]
Jeff Sauro • April 9, 2013
There are two common uses of the term learnability, one is the ability of an interface to allow users to accomplish tasks on the first attempt. The second is usability over time. A more learnable system is one that allows users to complete tasks more quickly with less practice with a system.[Read More]
Jeff Sauro • March 26, 2013
A tree test is like a usability test on the skeleton of your navigation with the design "skin" removed. It allows you to isolate problems in findability in your taxonomy, groups or labels that are not attributable to the designs or search engine. Here are some common questions and answers about the method including when to use it, how to interpret the output, the number of items to test and how long it takes.[Read More]
Jeff Sauro • March 19, 2013
Card sorting is a popular method for improving the organization of websites and software. Here are some common questions and answers about the method including when to use it, how to interpret the output, the number of items to test and how long it takes.[Read More]
Jeff Sauro • March 12, 2013
There are a number of popular methods used in improving the user experience at all phases of research and design. Here are over twenty of the more popular methods we use and when we use them in the product development cycle.[Read More]
Jeff Sauro • March 5, 2013
Usability is hardly physics or chemistry. But there are some important principles from decades of research in Human Computer Interaction (HCI) that apply to design and user research. Here are five famous laws that can be applied to improving the user experience of applications and websites: Miller's Law, Fitts' Law, Hick-Hyman Law, Power Law of Practice and the Pareto Law.[Read More]
Jeff Sauro • February 26, 2013
Face-to-face usability sessions provide rich feedback without the biases introduced by multi-person sessions, but at a cost. One-on-one in person sessions can be time consuming, hard to recruit for, biased, expensive and a pain to schedule. For the same cost we recommend a mix of surveys, moderated (remote and in-person) and unmoderated testing to maximize usability testing budgets. [Read More]
Jeff Sauro • February 19, 2013
Six Sigma is a methodology for improving the quality of everything from the manufacturing of minute electronic parts to the development of complex software. While there are many tools in the six sigma toolbox, here are six that we apply when improving the user experience of websites, software and hardware.[Read More]
Jeff Sauro • February 11, 2013
The profession of usability as we know it largely started in the 1980s. Many methods have their roots in the earlier fields of Ergonomics and Human Factors which began near the beginning of the 20th century and had a strong influence through the World War II. While not exhaustive, the following is a timeline of several key events, people and publications that have shaped the history and future of usability.[Read More]
Jeff Sauro • February 5, 2013
Hypothesis testing is at the heart of modern statistical thinking and a core part of the Lean methodology. Instead of approaching design decisions with pure instinct and arguments in conference rooms, form a testable statement, invite users, define metrics, collect data and draw a conclusion.[Read More]
Jeff Sauro • January 29, 2013
In building a better experience, there are many questions about mobile device usage and how designers can best meet users' needs with apps and responsive designs. From usage and ownership to purchasing behavior, here are 15 data points to help in the decision making.[Read More]
Jeff Sauro • January 22, 2013
The process of thinking through potential problems and knowing how to detect and prevent problems before they create a UX catastrophe is what the Failure Modes Effect Analysis (FMEA) is all about. It takes into account problem frequency, severity and ability to detect into a single priority number.[Read More]
Jeff Sauro • January 15, 2013
When we have a frustrating or poor experience, we can hate the product, tell our friends about our bad experience, post it to Twitter, and we probably will not use the service or product again. Using some familiar usability metrics and re-framing them, we can understand and eliminate poor experiences.[Read More]
Jeff Sauro • January 8, 2013
Too much to do and too little time—that's product design. If you tried to fix or improve everything, you'd never get any new products released. You have to prioritize. Prioritize is exactly what the Quality Function Deployment (QFD) method does. It provides a structured way to prioritize features, functions or even website content by taking into account both business priorities (the voice of the company) and the customer/user priorities (the voice of the customer).[Read More]
Jeff Sauro • January 2, 2013
On MeasuringUsability.com our servers saw 596k visitors and 1.6 Million page views in 2012. We wrote 49 articles and created two calculators. The 10 most popular articles each received at least 10,000 page views with the top three seeing more than 20k.[Read More]
Jeff Sauro • December 18, 2012
Surveys are ubiquitous. We answer them and we develop them because they are an efficient means for collecting feedback and insight from current and prospective customers. Here are a mix of ten tips, insights and philosophies to consider for your next survey.[Read More]
Jeff Sauro • December 11, 2012
You don’t need to be a mathematician to quantify the problems and improvements in user interfaces. Often the most compelling metrics are simple to compute and require no more than arithmetic and basic algebra. While most of us were exposed to these concepts in 8th and 9th grade, they are easy to forget and probably didn’t seem applicable when not learned in context. Here are a few fundamental concepts to help with quantifying the user experience.[Read More]
Jeff Sauro • December 5, 2012
Few things are more revealing than watching just a few users attempt tasks on a website or software.While watching users you should ask them some key questions to help put the observations into perspective. Here are eight recommendations for helping quantify both attitudes and put the insightful observations into context. [Read More]
Jeff Sauro • November 27, 2012
Whether you're new to A/B testing or a seasoned practitioner, here are 10 things you should know about this essential method for quantifying the user experience: sample size, statistical significance, stopping early and practical significance from confidence intervals around the difference.[Read More]
Jeff Sauro • November 13, 2012
Unmoderated usability testing is one of the fastest growing usability methods. Here are 10 things to know about this essential approach.[Read More]
Jeff Sauro • November 7, 2012
In a survey of over 2000 users from a selection of 30 high traffic websites, we asked what one thing users would improve. We found some usual suspects: search, navigation, appearance along with security and information content.[Read More]
Jeff Sauro • October 30, 2012
After users attempt a task, record how difficult they found it using the Single Ease Question (SEQ). We use the SEQ on websites, mobile devices, software and paper-prototypes to generate a reliable measure of perceived task difficulty over time and against comparable interfaces.[Read More]
Jeff Sauro • October 23, 2012
In order to improve findability, use the same core usability metrics to assess how well users can find items. Define what users are trying to find, measure how well they can currently find it, make improvements to the navigation and include cross-linking, then measure the same items and compare statistically.[Read More]
Jeff Sauro • October 16, 2012
Quantifying the user experience is the first step to making measured improvements. One of the first questions with any metrics is "what's a good score?". Like in sports, a good score depends on the metric and context. Here are 10 benchmarks with some context to help make your metrics more manageable.[Read More]
Jeff Sauro • October 9, 2012
An analysis of 122 usability studies with data from 833 users found that within a test, task performance and System Usability Scale scores have a low correlation. When aggregated at a higher level, SUS scores tracked completion rates better, suggesting SUS scores alone can estimate, at least crudely, task performance.[Read More]
Jeff Sauro • October 2, 2012
By estimating the percent of customers that recommend a website or product, those who were referred by someone, plus the "likelihood to recommend" question, you can estimate how many promoters it takes to generate enough positive word-of-mouth to net a new customer.[Read More]
Jeff Sauro • September 25, 2012
As UX continues to mature it's becoming harder to avoid using statistics to quantify design improvements. Here are five of the more critical but challenging concepts that take practice and patience but are worth the effort to understand.[Read More]
Jeff Sauro • September 12, 2012
The Pareto Principle, also called the 80/20 rule shows that a small percentage of things explain a large percentage of outcomes. In user research, an effective strategy is to separate the vital few from the trivial many to improve the user experience—whether it be usability problems, feature requests, calls into support, software bugs or revenue.[Read More]
Jeff Sauro • September 6, 2012
A heuristic evaluation of two websites from four evaluators found 32% of the issues 50 users had during a usability test. Heuristic evaluations typically find between 30% and 50% of issues found in usability tests. In general, HE is best used in addition to usability testing to generate the most complete picture of potential and actual interface problems.[Read More]
Jeff Sauro • August 28, 2012
In user research, it's hard to know which method or which metric will generate the right insights a head of time. By using multiple methods and measures and in effect "averaging" the results you provide the most reliable picture of the user experience.[Read More]
Jeff Sauro • August 21, 2012
Usability testing, like most studies of human behavior has biases. Many of these biases stem from the observer effect--participants know they're being watched and act differently. Biases can never be entirely eliminated. The best remedy is often being aware of them and communicating their potential impact on decisions.[Read More]
Jeff Sauro • August 14, 2012
We review an experiment where four independent evaluators inspected two websites (Budget.com and Enterprise.com). The results are consistent with earlier research: different evaluators find different issues, experts find more but not all issues compared to novices, and concerns about false alarms and misses persist with inspection methods compared to usability testing.[Read More]
Jeff Sauro • August 7, 2012
The focus group has its place in UX research but is usually the wrong tool for the job. Combining some aspects of focus groups with usability testing in a 1:1 session allows the researcher to gather both motivations as well as task-based outcomes with an interface.[Read More]
Jeff Sauro • July 31, 2012
Personas are popular in UX research with 65% of practitioners using them. A persona is more than just a user or something the marketing department made up. Here are seven core ideas everyone should know about personas in UX.[Read More]
Jeff Sauro • July 24, 2012
Lean is not a new diet plan. It's keeping the quality while reducing the waste in the processes. It's not doing fewer methods to improve the user experience--it's refining the methods so we spend less time processing, producing and waiting.[Read More]
Jeff Sauro • July 17, 2012
Card sorting is a popular method for understanding users' mental models for improving website navigation. The method is also best used in conjunction with tree-testing (reverse card sorting). This blog reviews the combined approach we used in the Target.com navigation case study of an open card sort and tree-test. It also includes answers to the questions asked during the webinar.[Read More]
Jeff Sauro • July 10, 2012
Have you wanted to purchase something online, but couldn't? Here are nine things we see rather often in our usability tests of eCommerce websites: not showing shipping +tax prior to checkout, registration, saving form information and the quality of user reviews.[Read More]
Jeff Sauro • July 4, 2012
While the look of our interfaces has changed, the constant demand to respond to alerts hasn't. There are friendlier ways of alerting users, however, it's less about how the message is displayed but the fact that it is displayed. In the end it's finding ways of having fewer messages, with less content, less often.[Read More]
Jeff Sauro • June 26, 2012
The one question everyone using statistics asks is : What statistical procedure do I use? In this blog I've created a clickable decision map which links to the correct procedure and online calculator after answering a couple questions.[Read More]
Jeff Sauro • June 19, 2012
If they can find it they will buy it. But how will they find it? An evaluation of 1500 users across 9 websites and 25 tasks found that about 14% of users start with search. While the percent of users browsing versus searching varies based on the task, website and prominence of the search box, in general most users start browsing.[Read More]
Jeff Sauro • June 12, 2012
In a study of 881 users across 5 usability studies and 38 tasks, we found men were about 6% more confident than women when they failed a task. As data from other studies suggest while both genders tend to be overconfident in their abilities, in the absence of a clear indication of failure, men tend to assume they were successful.[Read More]
Jeff Sauro • June 6, 2012
Organizing usability problems into a user by problem matrix helps estimate the prevalence of problems, the percent of problems uncovered and increases the clarity and credibility of usability testing methods.[Read More]
Jeff Sauro • May 30, 2012
There is no usability thermometer to tell you how easy to use a website or software application is. By averaging together the common metrics of effectiveness (completion rates, errors), efficiency (time) and satisfaction (task-level satisfaction) we generate an accurate and actionable picture of usability.[Read More]
Jeff Sauro • May 22, 2012
My thoughts on measuring the customer experience, including challenges to getting a company wide measure and some best practices for measuring and improving.[Read More]
Jeff Sauro • May 15, 2012
Errors can be categorized as slips (like a typo) or a mistake (incorrect goal) and are common occurrences in usability tests. Errors are often caused by problems in an interface and lead to longer task times, higher task failure and lower satisfaction ratings. While errors can't be entirely eliminated, they can often be reduced substantially by reducing the opportunity for an error.[Read More]
Jeff Sauro • May 9, 2012
Regardless of whether you're more on the research side or more on the design side of the User Experience, here are five skills that will make you more valuable and effective in your job.[Read More]
Jeff Sauro • May 1, 2012
When it comes to testing websites there are many unmoderated and moderated solutions. But if you've ever tried to evaluate an app or website on a mobile phone or tablet there are fewer options. Here's our solution.[Read More]
Jeff Sauro • April 24, 2012
Increasingly companies are adopting the Net Promoter Score as the corporate metric. All metrics, including user experience metrics should roll up to the Net Promoter Score. Here are 10 things to know about the Net Promoter Score if you're concerned about improving the user experience.[Read More]
Jeff Sauro • April 17, 2012
One of the simplest ways to measure any event is a binary metric coded as a 1 or 0. It's at the heart of computing and plays a critical role in user research.[Read More]
Jeff Sauro • April 17, 2012
Compare two independent proportions for A/B testing or comparing completion rates or conversion rates for small and large sample sizes. Uses the N-1 Two proportion test and Fisher Exact test to generate p-values.[Read More]
Jeff Sauro • April 10, 2012
Despite the rise in unmoderated usability testing, the bulk of evaluations are still done with a facilitator. Whether you are sitting next to the user in a lab or sharing screens with someone thousands of miles away, here are 20 practical tips for your next moderated usability test.[Read More]
Jeff Sauro • April 3, 2012
When you need a user-centered view of categories, labels or groups to improve findability, card sorting is the way to go. Here are at least 10 things you should know about this popular user research method.[Read More]
Jeff Sauro • March 27, 2012
Standardized Usability Questionnaires offer the advantage of higher reliability, validity and sensitivity and some offer the advantage of a normalized database which allows you to compare a score to hundreds of others.[Read More]
Jeff Sauro • March 21, 2012
Compute confidence intervals around continuous data using either raw or summary data.
[Read More]
Jeff Sauro • March 21, 2012
You don't need a PhD in statistics to understand and use confidence intervals. Here are 10 things to know to get you started using and interpreting confidence intervals for your next user research project.[Read More]
Jeff Sauro • March 13, 2012
Rarely can we talk to all users in the population we're studying, instead we sample. Here are 7 S's to help in your sampling: Simple Random, Starbucks, Stratified, Snowball, Spot, Sequential and Serial sampling.[Read More]
Jeff Sauro • March 7, 2012
Many of the reasons people don't use statistics with usability data are based on misconceptions about what you can and can't do with statistics and the advantage they provide in reducing uncertainly and clarifying recommendations. Here are nine of the more common misconceptions I've heard.[Read More]
Jeff Sauro • February 29, 2012
After the successful webinar on Best Practices for Remote Usability Testing, we received many questions about how I performed the analysis: sample size questions, time on task and other logistic issues are covered.[Read More]
Jeff Sauro • February 21, 2012
The right measure will: identify problem areas, track improvements over time, be meaningful to the customer. The wrong measure can: identify wrong areas of focus, miss problems all together, lead to unintended consequences and alienate customers. Finding the right measure means taking multiple measures and seeing which one best tracks other customer sentiments and revenue.[Read More]
Jeff Sauro • February 14, 2012
There are more than five challenges facing UX professionals today, but here are five that tend to cross projects and products: Time, Costs, Tools & Techniques, Finding Representative Users and Deliverables. [Read More]
Jeff Sauro • February 7, 2012
An analysis of 4000 users across 59 websites found that when users fail to accomplish their goals, they are 5 times less likely to return to the website and 3 times more likely to tell their friends not to visit the website.[Read More]
Jeff Sauro • January 31, 2012
An analysis of SUPR-Q data from users of popular social networking sites reveals users generally don't trust social networks. Facebook, Twitter and Google+ all fall within the bottom quintile of website trust. Facebook leads the pack with the highest Net Promoter Score and usability.[Read More]
Jeff Sauro • January 24, 2012
The distinction between a qualitative study and quantitative study is a false dichotomy. It doesn't cost more money to quantify or use statistics. It just takes some training and confidence. Not only can qualitative data be categorized into quantities, but it can prompt further questions and discovery for usability improvement.[Read More]
Jeff Sauro • January 17, 2012
There are advantages and disadvantages to the different usability testing methods: lab-based, remote moderated and remote unmoderated. A combination of methods provides a more comprehensive picture of the user experience but is not always possible. Consider these nine factors when deciding on a method.[Read More]
Jeff Sauro • January 10, 2012
Despite UX being a field with wide ranging skills, the average salaries for individual contributors are about the same across job functions at about $85k and haven't changed much over two years.[Read More]
Guest Post By Jim Lewis • January 3, 2012
System Usability Scale (SUS) scores are often collected along with Net Promoter Scores in evaluations of software and website usability. An examination of 81 datasets from 2200 users shows that dividing SUS scores by 10 does a decent job of predicting the Net Promoter Score.[Read More]
Jeff Sauro • December 27, 2011
Thank you to the 585k visitors and 1.3 Million page views on MeasuringUsability.com in 2011. Of the 52 articles written in 2011, in ascending order, here are the 15 most popular.[Read More]
Jeff Sauro • December 20, 2011
Improving the user experience means starting with the right measure or measures to manage. Here are 10 of the more common ones I've written about in 2011.[Read More]
Jeff Sauro • December 14, 2011
The only thing worse than users failing a task is users failing a task and thinking they've completed it successfully. This is a disaster. Disasters can be tracked by measuring task completion rates and task-level confidence. Data from 174 tasks show the likely prevalence of disasters in consumer software and websites.[Read More]
Jeff Sauro • December 7, 2011
What sample size do i need? It's usually the first and most difficult question to answer when planning a usability evaluation. There are actually good ways for estimating the sample size that don't rely on intuition, dogma or conventions.[Read More]
Jeff Sauro • November 30, 2011
There isn't a usability thermometer to tell you how usable your software or website is. Instead we rely on the impact of good and bad usability to assess the quality of the user experience. Assessing that impact starts by knowing and collecting these 10 metrics.[Read More]
Jeff Sauro • November 22, 2011
The reported number of UX professionals in both small and large organizations has increased between 20% and 30% over the last two years. This can be attributed to both an increase in actual UX professionals and a broadening of the jobs that fall under the UX umbrella.[Read More]
Jeff Sauro • November 15, 2011
For as long as there have been websites it seems that there's been a call to reduce the number of clicks to improve the user experience. An analysis of click counts and task times across 3 eCommerce website usability tests and 1200 users found that clicks and time have a correlation of .5. Clicks predict around 25% of task time, meaning it's better to directly measure or estimate task times than click counts when improving website efficiency.[Read More]
Jeff Sauro • November 8, 2011
There are many great methods for gathering insights from users and many more software tools. Here are the tools and services I use when conducting user research.[Read More]
Jeff Sauro • November 1, 2011
A product must be useful and usable to be adopted and both have been shown to predict reported usage. However, perceived usefulness is 1.5 times more important than usability when predicting technology acceptance.[Read More]
Jeff Sauro • October 26, 2011
Build it and they might come. Build trust and they will stay. Make it usable and credible and they may tell their friends. Measuring credibility and trust along with usability should be part of benchmarking efforts. Is your website trusted as little as Facebook or as much as PayPal and Apple?[Read More]
Jeff Sauro • October 19, 2011
Good website navigation is essential for task success. When users' first click is down an incorrect path, only 46% eventually succeed. Here are some ideas on testing first click success.[Read More]
Jeff Sauro • October 11, 2011
Recent changes in Netflix pricing, services and poor communication substantially affected key customer metrics. A longitudinal analysis of customers shows a drop of 80 percentage points in the Net Promoter Score and a drop in credibility rankings from the 99th percentile to the 62nd percentile. What's bad for Netflix is a reminder to measure early and often.[Read More]
Jeff Sauro • October 4, 2011
A successful website needs to be usable, credible and visually appealing. This will generate positive word of mouth, repeat visitors and ultimately a more successful website. The SUPR-Q (Standardized Universal Percentile Rank) measures these concepts in 13 items along with a robust database of 200 websites to know where your scores ranks.[Read More]
Jeff Sauro • September 28, 2011
These are three of the most important words for anyone trying to make better decisions with data. For most measures of customer experience there are at least three good places to start to look for meaningful comparables: a prior version, an industry average or a leading competitor.[Read More]
Jeff Sauro • September 20, 2011
It's not easy finding and scheduling users to participate in a usability test. It gets a lot easier to observe even difficult to find users when the commitment is just a few minutes in front of their computer.[Read More]
Jeff Sauro • September 13, 2011
While overall men tend to make around 4% more than women, the difference is largely explained by age and years of experience. Women make the same or more than men below age 35 but tend to make less for the higher age cohorts.[Read More]
Jeff Sauro • September 6, 2011
Completion rates are the fundamental usability metric. They are easy to collect and understand and should be reported with confidence intervals at every stage of development.[Read More]
Jeff Sauro • August 30, 2011
The results of the recent UPA Salary survey show 82% of UX professionals use some form of usability testing, 75% perform expert reviews and only 16% have the budget to perform eye-tracking studies. Almost 40% use focus groups and more than two-thirds use prototyping.[Read More]
Jeff Sauro • August 23, 2011
The median UX professional salary for 2011 is $90k, up around $5,500 (7%) from 2009. The key factors that drive 40% of salary variation are: years of experience, having a PhD, being a manager and whether you live in the Western or Northeastern US.[Read More]
Jeff Sauro • August 16, 2011
There isn't a single silver bullet technique or tool which will uncover all usability problems. Instead, practitioners should quadrangulate using the four corners of usability measurement: user testing, inspection methods, cognitive modeling and standardized questionnaires.[Read More]
Jeff Sauro • August 9, 2011
Perhaps it's something about the precision of minutes and seconds that demands greater scrutiny. There's a lot to consider when measuring and analyzing task times in usability tests: among them are how to report the average time, what to do with failed task-attempts, thinking aloud and figuring out how long the task should take.[Read More]
Jeff Sauro • August 2, 2011
Heuristic Evaluations and Cognitive Walkthroughs are both usability Inspection Methods that can be used early and often in design to detect many of the problems seen in usability testing. Cognitive Walkthoughs have more of an emphasis on the learnability of a task. Both methods are best performed with multiple evaluators.[Read More]
Jeff Sauro • July 26, 2011
If you collect nothing else in a usability test it should be a list of problems encountered by users. It seems so simple yet there is a rich history of how many users you need to test, what constitutes a problem and which method to use.[Read More]
Jeff Sauro • July 19, 2011
A usability problem to one usability expert is a feature to another. There is a balance between business interests and user interests but the two aren't mutually exclusive. The discussion should be about short-term revenue at the expense of long term profits--not usability versus revenue.[Read More]
Jeff Sauro • July 12, 2011
The mean response to the likelihood to recommend question predicts the Net Promoter Score very well. Net Promoter Scoring loses about 4% of the response information. It may be more beneficial to report the Net Promoter Score to executives but use the mean for statistical comparisons.[Read More]
Jeff Sauro • July 5, 2011
I've listed 14 of the more frequent/painful burdens I experience in the hope we can shift the burden more from the human back to the computer: passwords, account numbers and confirmation dialogues get special attention.[Read More]
Jeff Sauro • June 28, 2011
Measure the percent of users that find the content, the percent who suggest a category label and the percent who place a card in a group along with confidence intervals to improve website navigation.[Read More]
Jeff Sauro • June 21, 2011
It's easy to get overwhelmed by methods, tools and technology when making design improvements. Be sure to get the critical few tasks right by defining the users, what they're trying to do and how they do it on your application.[Read More]
Jeff Sauro • June 15, 2011
Using the lower-boundary of a confidence interval on a pre-test response rate will provide an accurate estimate of your full-survey's response rate. You can use this to estimate how many total invites you need to maintain a margin of error in your responses.[Read More]
Jeff Sauro • June 8, 2011
Many software companies track and use the Net Promoter Score as a gauge of customer loyalty. In the largest study of consumer and productivity software, over 1700 current customers provide Net Promoter Scores and Usability scores for 17 products.[Read More]
Jeff Sauro • June 1, 2011
It's fine to compute means and statistically analyze ordinal data from rating scales--the numbers don't know where they came from. But just because one rating is twice as high as another does not mean users are really twice as satisfied.[Read More]
Jeff Sauro • May 24, 2011
When we look to improve the user experience of software or websites, sometimes the best improvements aren't slight tweaks to the interface but involve eliminating steps altogether. Here are four examples of terrific and terrible experiences from the physical world to inspire the digital one.[Read More]
Jeff Sauro • May 17, 2011
In measuring perceived usability of five popular websites, a single difficult task will lower post-test usability scores by 8%. This is largely driven by the least experienced users whose scores dropped by almost 20%. A difficult task doesn't appear to affect the most experienced users' attitudes.[Read More]
Jeff Sauro • May 10, 2011
Closed ended rating scale data is easy to summarize and hard to interpret. These five approaches provide meaning to raw responses and often generate similar results.[Read More]
Jeff Sauro • May 4, 2011
It's easy to get derailed when writing a survey or questionnaire when measuring the user experience. You need to worry about what to ask, who to ask and how what you're asking affects the responses. These eight insights will help make things a bit smoother.[Read More]
Jeff Sauro • April 26, 2011
There is a long tradition of including items in questionnaires that are phrased both positively and negatively to minimize extreme response and acquiescent biases. An analysis of an all positively worded version of the SUS found little evidence for these biases. This suggests response bias effects are small and outweighed by the real effects of miscoding by researchers and misinterpreting by users.[Read More]
Jeff Sauro • April 19, 2011
Research generally shows that including a neutral response will affect the distribution of responses and sometimes lead to different conclusions. However, this is less important when assessing usability as you're usually more concerned about comparisons over time or against a benchmark than the percent of users who agree to statements[Read More]
Jeff Sauro • April 12, 2011
Task time is the best way to measure the efficiency of a task and it is a metric that everyone understands. Task-times will differ (often substantially) between lab tests, remote unmoderated tests and estimates from Keystroke Level Modeling. The trick is to be consistent. Use the same method in both comparisons and focus on the improvement instead of the absolute time regardless of the method.[Read More]
Jeff Sauro • April 5, 2011
You can use the cause and effect diagram at all stages of development to help brainstorm root causes for interaction problems and other unwanted outcomes. It becomes an excellent input for finding design solutions and turns negative thinking into positive user experiences.[Read More]
Jeff Sauro • March 28, 2011
Every field has its set of hot-button issues and usability is no exception. Here are six topics that tend to generate passionate discussions: Quantifying Usability, Reliability of Evaluators, Certification, Unmoderated Testing Quality, Heuristic Evaluations and Sample Size.[Read More]
Jeff Sauro • March 21, 2011
Context matters in deciding what a good completion rate is for a task, however, knowing what other task completion rates are can be a good starting point given no other data. An analysis of almost 1200 usability tasks shows that the average task-completion rate is a 78%. Anything above a 78% is above average and can be a good starting point for goal setting. I'm not suggesting you use this information blindly when establishing a goal for a task-completion rate.[Read More]
Jeff Sauro • March 15, 2011
Asking users to rate the difficulty/ease of a task-scenario (without even attempting the task) can predict around half the variation in ratings from users that actually attempted the task. Users tend to overestimate how difficult a task is but on average were able to predict task difficulty ratings to within 17%. The gap between expectation and experience can be a powerful predictor of an interaction problem--something recently seen on eBay.[Read More]
Jeff Sauro • March 8, 2011
By quantifying design efforts and outcomes all organizations can benefit from understanding how improving the user experience can improve the bottom line. You need to measure tactical design changes, customer loyalty and some indicator of product revenue to identify key drivers of growth.[Read More]
Jeff Sauro • March 1, 2011
A lot of effort goes into simplifying interactions, reducing bugs and enhancing features. By providing simple quantitative measures of improvements in the user experience you have the data to both justify design efforts and get a better idea of what methods worked.[Read More]
Jeff Sauro • February 21, 2011
Quantifying the frequency of comments with a binomial confidence interval helps you estimate a sentiment in the total user population and prioritize findings.[Read More]
Jeff Sauro • February 14, 2011
To know if design changes improved the usability of an application, you first need a baseline measure of usability from a benchmark test. Benchmark usability tests don't happen a lot, so to help you in your next test I've assembled a list of ten tips to help you get the most out of your effort.[Read More]
Jeff Sauro • February 8, 2011
Observing customer behavior is an excellent way for discovering opportunities for product innovation. The number of customers you need to observe can be determined using the binomial probability formula and will vary depending on how common customer behaviors are and how certain you need to be.[Read More]
Jeff Sauro • February 2, 2011
At only 10 items, SUS may be quick to administer and score, but data from over 5000 users and almost 500 different studies suggests that SUS is far from dirty. Its reliability and validity are as high as or higher than commercial questionnaires. Its versatility, brevity and wide-usage means that despite inevitable changes in technology, we can probably count on SUS being around for at least another 25 years.[Read More]
Jeff Sauro • January 26, 2011
Responses to rating scale data typically don’t follow a normal distribution. However, this is unlikely to affect the accuracy of statistical calculations because the distribution of error in the measurement is normally distributed.[Read More]
Jeff Sauro • January 19, 2011
SUS scores from 1100 users from 62 websites showed that repeat users rate the website as 11% more usable than first time users. SUS scores from 800 users of 16 consumer software products showed that the most experienced users (5+ years of experience) also rate the software as 11% more usable than the least experienced users (less than 3 years of experience).[Read More]
Jeff Sauro • January 11, 2011
At most 8% of users read license agreements on software programs. Users typically spend no more than a few seconds on the license page. Users breeze by the page probably because they really have no choice if they want to use the program and cannot negotiate terms.[Read More]
Jeff Sauro • January 4, 2011
Keystroke Level Modeling can predict error-free experienced user times to an accuracy of between 10-20%. KLM is ideally suited for comparing the efficiency of alternative designs early in the design stage when it's difficult to measure task time from user-testing.[Read More]
Jeff Sauro • December 29, 2010
There were many interesting findings reported on MeasuringUsability.com in 2010. Here is my top-10 list to get you thinking for 2011.[Read More]
Jeff Sauro • December 21, 2010
Links to 25 books, papers, chapters and questionnaires on measuring usability.
[Read More]
Jeff Sauro • December 21, 2010
This is the working outline of our book with Morgan Kaufmann. It will bring together almost a decade of research on finding the best statistical approaches to solving the most common issues in user research. Anticipated publication date Spring 2012.[Read More]
Jeff Sauro • December 14, 2010
Top-box and top-two-box scoring is appealing for summarizing responses in the absence of benchmarks or comparisons. Top-box scoring has the disadvantage of losing information about precision and variability. Reducing 5, 7 or 11 response options to two or three options can mask real changes in attitudes.[Read More]
Jeff Sauro • December 7, 2010
Around 10% of paid respondents to online surveys are rushing through surveys to get the honorarium. With longer surveys and more complicated questions, closer to 20% of responses will be unusable.[Read More]
Jeff Sauro • November 30, 2010
Users rate tasks as more difficult when given 5 or 60 seconds compared to no time limit. These lower task-ratings don't appear to translate to overall website ratings. Concurrent ratings may pinpoint interaction problems more precisely than post-task ratings but add time to test-sessions and task-times.[Read More]
Jeff Sauro • November 23, 2010
Survey results show completion rates and UI problems dominate formative usability tests. Task time data is more common in summative usability tests, yet still prevalent in formative tests. At least half of respondents use some form of usability questionnaire in both types of tests.[Read More]
Jeff Sauro • November 17, 2010
In his book Outliers, Malcolm Gladwell identifies the years around 1955 as the best time to become a software millionaire. Bill Gates, Steve Jobs and Eric Schmidt all have birthdays in 1955. However, an examination of a larger dataset of 41 software millionaires reveals twice as many were born outside the 1950s. CEOs from all industries happen to be born in the 1950s--a likely consequence of the baby-boom and not particular birth years.[Read More]
Jeff Sauro • November 9, 2010
System Usability Scale (SUS) Scores from users who had only 5 seconds to assess the usability of a website were statistically indistinguishable from users who had no time limit. Users who had only 60 seconds on a website tend to rate websites as more usable than those who had only 5 seconds or no time limit.[Read More]
Jeff Sauro • November 2, 2010
The results of an email survey found 80% of Formative usability tests have less than 15 users. Summative usability test sample sizes are around 3 times larger for respondents who conducted both types of tests.[Read More]
Jeff Sauro • October 27, 2010
In a usability test we're taught to be neutral observers of user behavior. But what actually happens in practice? A review of 14 usability testing sessions from seven companies suggests that what we preach is a bit different than what we practice.[Read More]
Jeff Sauro • October 19, 2010
UX and marketing can compete for the same limited resources in an organization and at times can be at odds on product direction. In my experience they are an unlikely couple and have a lot in common.[Read More]
Jeff Sauro • October 12, 2010
Three books on usability testing from the early 90's, one on usability ROI and another on estimating experienced user task times without testing any users.[Read More]
Jeff Sauro • October 6, 2010
When users document the usability problems they encounter during a usability test, they find around half as many problems as trained professionals watching users in a lab. This low-cost method is a good supplement to heuristic evaluations and user testing.[Read More]
Jeff Sauro • September 29, 2010
Usability problem frequencies from 24 usability tests show that users are almost ten-times more likely to encounter a usability problem in a business application than a website. Users are about half as likely to encounter a problem in consumer software than a business application.[Read More]
Jeff Sauro • September 21, 2010
Items in the System Usability Scale (SUS) were rephrased to either all extreme positive or all extreme negative wording and average scores were compared. Users disagree more with items that are worded in the extreme and resulted in significantly different SUS Scores.[Read More]
Jeff Sauro • September 14, 2010
There is a modest bias towards the left side of a rating scale. This matters most for stand-alone surveys or questionnaires when no comparisons are being made. In such cases, the number of "agree" or top-box statements will be higher if placed on the left-side of a scale.[Read More]
Jeff Sauro • September 7, 2010
Users, surveys, tasks and metrics: there's a lot to know about usability but here's a resource for experts and novices.
[Read More]
Jeff Sauro • August 31, 2010
Maybe you already have heard-of and use Heuristics Evaluations. Here are six things you might NOT know about this popular usability method.[Read More]
Jeff Sauro • August 25, 2010
7 point scales tend to perform slightly better than 5 point scales. The benefit is too small to change your existing questionnaires if you have historical data. Having more points will provide the biggest benefit when you have only a few or one question in your questionnaire. Focus more on what you'll do with the results than whether 5 or 7 points is better.[Read More]
Jeff Sauro • August 18, 2010
Different usability teams find different problems in websites and applications. Would you get different diagnoses if different radiologists read your x-ray or MRI? It turns out you would. While there is a lot of room for improvement in the reliability of usability evaluations much of the variability is due to human judgment--a problem that also plagues the medical field.[Read More]
Jeff Sauro • August 10, 2010
Love them, hate them, admire them or ignore them. These seven living legends aren't one-hit wonders. Their work has had and will continue to have a large impact on the field of usability for some time.[Read More]
Jeff Sauro • August 4, 2010
Confidence intervals, like statistics in general, are powerful because they are both consistent with our experience and provide a level of precision we can't articulate. You should use them with your usability test data.[Read More]
Jeff Sauro • July 27, 2010
A study conducted by Nielsen on reading speeds was criticized for going beyond the statistics to support the claim that books are faster than tablets. The crux of this point comes down to considering a finding "statistically significant" only when the p-value is below .05. This criterion is a convention not a commandment and context should always be considered when deciding the role of chance in applied research.[Read More]
Jeff Sauro • July 21, 2010
Wondering about the origins of the sample size controversy in the usability profession? Here is an annotated timeline of the major events and papers which continue to shape this topic from 1982-2010.[Read More]
Jeff Sauro • July 15, 2010
This common question mixes two concepts: representativeness and sample size. It is more important to ask a few of the right people what they think than a lot of the wrong people. Once you're talking to the right people identify the highest margin of error you can tolerate to compute the right sample size.[Read More]
Jeff Sauro • July 7, 2010
These five papers have had a large and lasting influence on the field of Usability and User Experience.
[Read More]
Jeff Sauro • June 30, 2010
It is tempting to just find, fix and forget usability problems. It is tempting to skip one milkshake and think you're losing weight. Systematic measuring uncovers patterns and helps prevent major problems.[Read More]
Jeff Sauro • June 23, 2010
Whether you're conducting an early stage test of a prototype or late validation, these five tips can make any usability test more credible. The tips both temper skepticism about small samples and help you avoid overstating your findings.[Read More]
Jeff Sauro • June 16, 2010
Web analytics has transformed the problem of understanding user behavior from a puzzle to a mystery. Where we once didn’t have enough information, we now can have too much to make sense of. Small sample user testing tells helps answer the "why" mystery. There will be a continued demand for user-researchers who can quantify observational data and make the most of analytic data.[Read More]
Jeff Sauro • June 10, 2010
Usability is not adhering to guidelines. Usability is measuring whether users are actually having a more usable experience. Users might want emotion-evoking software, but if they do there should be some evidence for it.[Read More]
Jeff Sauro • June 3, 2010
Is the term UX becoming an overused dot-com term like B2C or clicks-to-mortar? The field of User Experience has matured enough now that it deserves its own BS generator replete with terms that have more use than meaning--as well as a PDF Report with an ROI graph.[Read More]
Jeff Sauro • May 26, 2010
Mechanical Turk is seeing a shifting of its demographics from stay-at-home moms in the US to younger males in India. Around 60% of the Mechanical Turk workforce will put forth a conscientious effort in completing HITs while around 12% are likely rushing through tasks indiscriminately.[Read More]
Jeff Sauro • May 19, 2010
It takes about 2 to 3 times as long to send an email on the iPhone as on the Desktop. However, users were able to find percentages faster on the iPhone calculator, showing you can have both portability and efficiency for certain tasks.[Read More]
Jeff Sauro • May 12, 2010
You don't need a random sample to use statistics to make better decisions from your usability data. You do need to know if the users who aren't in your usability tests are different enough than those who are.[Read More]
Jeff Sauro • May 6, 2010
The sample size formula for finding usability problems only works for a specific set of users and closed-ended tasks. With five users you will only find the more obvious problems.[Read More]
Jeff Sauro • April 29, 2010
For most cases exclude the users who fail the task and call it the average task completion time. Keep the failed task times to report Mean Time to Failure and Average Time on Task.[Read More]
Jeff Sauro • April 21, 2010
For small sample sizes, the geometric mean provides a better estimate of the middle task time that the sample median or mean.
[Read More]
Jeff Sauro • April 7, 2010
QA testers are not adequate substitutes for real users and usability tests are not adequate substitutes for good QA.
[Read More]
Jeff Sauro • April 1, 2010
Retrospective probing of user actions and intentions allows you to get a reliable benchmark and identify problems with an interface.[Read More]
Jeff Sauro • March 25, 2010
While testing with five users might reveal 85% of problems that impact 31% of users (given a set of tasks and user-type), it doesn't mean you're finding 85% of the critical problems. Assume that the severity of a problem is not related to how often it occurs.[Read More]
Jeff Sauro • March 18, 2010
One consequence of analyzing user data is having to reconcile conflicting data-points. How would you display links to PDF files on a web-page?[Read More]
Jeff Sauro • March 8, 2010
For finding usability problems with an interface, testing with five users is fine to find problems that affect 31% to 100% of all users. If a problem is more elusive (affects fewer than 31% of users) then you need to increase your sample size. This sample size does not apply to comparing designs or generating a precise estimate of completion rates or task-times.[Read More]
Jeff Sauro • March 2, 2010
A single 7-point likert-type question asked after a task-scenario provides a quick but sensitive and reliable estimate of task level difficulty and ease.[Read More]
Jeff Sauro • February 24, 2010
Post-task ratings capture satisfaction with task-performance and are great for identifying problem areas in an interface. Conversely, post-test questionnaires provide overall attitudes about the application and don't provide much diagnostic information.[Read More]
Jeff Sauro • February 16, 2010
At some point usability practitioners began using the terms quantitative and Summative interchangeably. That's a bad thing as metrics and quantitative methods should be used when finding and fixing UI problems as well as establishing a usability benchmark.[Read More]
Jeff Sauro • February 9, 2010
In a comparative test, satisfaction scores and completion rates from professional usability test-takers were nearly identical to lab-based users. However, time on task data differed significantly and showed much higher variability. For testing websites intended for a general audience the use of professional testers appears to provide mostly reliable data quickly and for a fraction of the price.[Read More]
Jeff Sauro • February 1, 2010
Insurance companies do it, drug companies do it and so should usability testers. When you observe a problem from a small sample test, it is unlikely the problem only affects a tiny percentage of users.[Read More]
Jeff Sauro • January 25, 2010
Performance data such as task times and completion rates explain around 26% of the user's perception of the ease of use. Gather both performance data and satisfaction data to triangulate around the task-level user experience.[Read More]
Jeff Sauro • January 18, 2010
The System Usability Scale (SUS) is the most popular standardized usability questionnaire because it's free and short. It was designed over 20 years ago before the web existed. Should it be used on websites?[Read More]
Jeff Sauro • January 7, 2010
I examined the relationship between customer loyalty as measured by the Net Promoter Score (NPS) and System Usability Scale (SUS) questionnaire from several usability tests. I found that perceptions of usability account for about 1/3 of the changes in customer loyalty. Increasing your usability will lead to increased loyalty.[Read More]
Jeff Sauro • January 4, 2010
Is there such thing as usability? This might sound like a silly question considering the industry around usability testing and user experience consulting (not to mention this website). But you can't touch usability and there is no usability thermometer to measure its presence or absence. While we can talk about usability and know it when we see it (or really, know it when we don't see it), what data is there that shows usability exists?[Read More]
Jeff Sauro • December 17, 2009
Imagine a marketing department asking for more money to conduct a direct-mail campaign and their only justification was that marketing is a critical business advantage.[Read More]
Jeff Sauro • December 8, 2009
Is it possible to get the same data from lab-based tests by having users test themselves? Unmoderated testing appears to provide a cost effective alternative for gathering a lot more usability data with considerably less effort. Additional time is required to filter invalid data such as unrealistically short task times.[Read More]
Jeff Sauro • November 5, 2009
Does a full-time PhD pay off in the Usability Profession? The UPA Salary data from 2009 and 2005 is analyzed. It shows that while a PhD may open doors, being out of the work-force for five years is an opportunity cost that is unlikely to made up for.[Read More]
Jeff Sauro • October 9, 2009
Only 14% of users who fail a task rate it at maximum level of satisfaction. In general there is an 80/20 rule of satisfaction and completion rates: 80% of users who rate at the maximum level of satisfaction will pass and 80% of users who rate at the minimum satisfaction level will fail the task.[Read More]
Jeff Sauro • August 6, 2009
How many users will complete the task and how long will it take them? If you need to benchmark an interface, then a summative usability test is one way to answer these questions. Summative tests are the gold-standard for usability measurement. But just how precise are the metrics?[Read More]
Jeff Sauro • June 5, 2009
Used for comparing 2 small sample binary completion rates (it uses a statistical test called the Fisher Exact Test)
[Read More]
Jeff Sauro • June 6, 2008
Time-on-task can be used as a valuable diagnosis and comparative tool during formative evaluations.
[Read More]
Jeff Sauro • January 4, 2008
Use this interactive calculator to understand how the sample size changes will affect the confidence interval around a completion rate.[Read More]
Jeff Sauro • December 14, 2007
Use this interactive normal curve to understand how the z-score and the area of the curve are related. Offers both one and two-sided options.[Read More]
Jeff Sauro • December 4, 2007
Enter the area under the normal curve (a proportion between 0 & 1) and get the Z-critical value.
[Read More]
Jeff Sauro • December 3, 2007
Look up the area under the normal curve (1 or two-sided areas) from a standard score (Z-score).
[Read More]
Jeff Sauro • June 1, 2007
The UsabilityScorecard web-application will take raw usability metrics (completion, time, sat, errors and clicks) and calculate confidence intervals, z-scores, quality levels and graph the results automatically. You can also combine any combination of the metrics into a 2, 3 or 4 measure combined score. Data can be imported from Excel (.csv) and exported to Word(.rtf).[Read More]
Jeff Sauro • October 1, 2006
Use this calculator to determine the number of users you'd need to test given the probability of detecting a problem. If the probability of detecting the problem is unknown, this calculator also allows you to estimate the problem occurrence (p) from sample data.[Read More]
Jeff Sauro • February 6, 2006
Visualizing your task time data is an essential step in understanding its distribution and computing accurate confidence intervals. This calculator creates a dot-plot of your task times, transforms the raw data to adjust for non-normality and computes the intervals.[Read More]
Jeff Sauro • October 1, 2005
If you've wanted to provide a confidence interval around a small sample completion rate but just didn't have time to do the math, this calculator does the work for you.[Read More]
Jeff Sauro • June 14, 2005
Do you need to feel more confident about using statistics? Dismayed by overly complicated "introduction" courses that focus on theory and not application? Do the "basic" books assume you know where to look for your answer? The first module in this series is on using confidence intervals in usability testing.[Read More]
Jeff Sauro • June 11, 2005
This paper identifies the limitations of traditional usability metrics and presents a process to increase their meaning by adapting Six Sigma methods. We define how common usability metrics can be evaluated in terms of a standardized defective rate or quality level and explore the benefits of this data transformation. Use the Usability Scorcard or the excel-based SUM calculator to standardize your metrics.[Read More]
CHI paper which explains the theoretical foundations.' title='SUM is a single usability metric that summarize the majority of variation in four common summative usability metrics. Download the calculator to convert raw metrics to a SUM score or read the CHI paper which explains the theoretical foundations.' />Jeff Sauro • April 17, 2005
SUM is a single usability metric that summarize the majority of variation in four common summative usability metrics. Download the calculator to convert raw metrics to a SUM score or read the CHI paper which explains the theoretical foundations.[Read More]
Jeff Sauro • December 1, 2004
You can use measures such as confidence intervals, sample size calculations—and other statistics normally associated with more premium usability methods—without the high costs. These methods require no money to compute yet provide a wealth of information. Even better, you can still provide these quantitative qualifiers while using most discount methods. Pre-Published PDF Version[Read More]
Jeff Sauro • October 18, 2004
Adding confidence intervals to completion rates in usability tests will temper both excessive skepticism and overstated usability findings. Confidence intervals make testing more efficient by quickly revealing unusable tasks with very small samples.[Read More]
Jeff Sauro • September 17, 2004
Conventional six sigma commonly adds a 1.5 sigma buffer to account for the shifting of a process over time. Does this make sense for software usability?[Read More]
Jeff Sauro • September 17, 2004
The complexity and depth of this popular quantitative measurement is often only given cursory thought. There's much more to task times than using a stopwatch.[Read More]
Jeff Sauro • September 17, 2004
Is attaining Six Sigma a reasonable or even attainable goal for usability? A product's usability is the sum of several usability measures. Each Relative movement in your sigma value is good predictor of usability improvements.[Read More]
Jeff Sauro • September 17, 2004
Many popular usability testing techniques are the right method to gather user data, however, their results alone will only scratch the surface of the true state of usability. Often their results can be misleading.[Read More]
Jeff Sauro • September 17, 2004
The order in which a task is administered during a usability test can have an effect on the user's performance especially as measured by task time. By randomizing task order the effects of this lurking variable can be mitigated.[Read More]
Jeff Sauro • September 17, 2004
I've begun to collect a list of articles and publications that relate to the quantitative measures of usability.
[Read More]
Jeff Sauro • September 17, 2004
Often the most reported measures of usability is task success. How does task success translate into a quality sigma value that can be compared to other reported sigma values?[Read More]
Jeff Sauro • September 17, 2004
This common statistical way of describing data can be used in usability testing to standardize disparate data types to allow easy comparison between products or versions and providing a universal way of assessing quality.[Read More]
Jeff Sauro • September 17, 2004
Jeff Sauro • September 17, 2004
One of the most common concerns about the six sigma methodology is that it cannot apply to something as byzantine as the interactions of humans with software. One of the major tenets of both Six Sigma and Human Factors is that the customer or user determine what's considered "quality."[Read More]
Jeff Sauro • September 17, 2004
Jeff Sauro • June 14, 2004
The basics of z-scores are discussed plus an example of raw usability data converted into z-scores including three of Nielsen's five usability attributes.[Read More]
Jeff Sauro • March 8, 2004
Shows the history and computation of deriving a sample size for discovering problems in an interface.
[Read More]
Jeff Sauro • March 8, 2004
The discerning usability analyst should employ a mix of both qualitative and quantitative methods when discovering usability problems. The risks of relying heavily on a qualitative approach can lead to a severe misdiagnosis especially when usability problems are difficult to detect[Read More]
Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to
Fortune 1000 companies.
He is the author of over
15 journal articles and 4 books on statistics and the user-experience.
More about Jeff...
10 Things to Know About the System Usability Scale (SUS)
Measuring The Visual Appeal of Websites
97 Things to Know about Usability
10 Things to Know about Usability Problems
Does better usability increase customer loyalty?
How to Conduct a Usability test on a Mobile Device
5 Examples of Quantifying Qualitative Data
8 Ways to Show Design Changes Improved the User Experience
Confidence Interval Calculator for a Completion Rate
Nine misconceptions about statistics and usability
Why you only need to test with five users (explained)
A Brief History of the Magic Number 5 in Usability Testing
What five users can tell you that 5000 cannot
Should you use 5 or 7 point scales?
 | Quantifying the User Experience: Practical Statistics for User Research The most comprehensive statistical resource for UX Professionals (JUST RELEASED) Buy on Amazon |
 | Excel & R Companion to Quantifying the User Experience Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R Buy on Amazon | Download |
 | A Practical Guide to the System Usability Scale Background, Benchmarks & Best Practices for the most popular usability questionnaire Buy on Amazon | Download |
 | A Practical Guide to Measuring Usability 72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software Buy on Amazon | Download |