Quantifying the user experience through the statistical analysis of human behavior.
6 things you didn’t know about Heuristic Evaluations :
Maybe you already have heard-of and use Heuristics Evaluations. Here are six things you might NOT know about this popular usability method.
Should you use 5 or 7 points scales? :
7 point scales tend to perform slightly better than 5 point scales. The benefit is too small to change your existing questionnaires if you have historical data. Having more points will provide the biggest benefit when you have only a few or one question in your questionnaire. Focus more on what you'll do with the results than whether 5 or 7 points is better.
Usability Evaluators: Reliable as Radiologists? :
Different usability teams find different problems in websites and applications. Would you get different diagnoses if different radiologists read your x-ray or MRI? It turns out you would. While there is a lot of room for improvement in the reliability of usability evaluations much of the variability is due to human judgment--a problem that also plagues the medical field.
7 Living Legends of Usability :
Love them, hate them, admire them or ignore them. These seven living legends aren't one-hit wonders. Their work has had and will continue to have a large impact on the field of usability for some time.
Memory versus Math in Usability Tests :
Confidence intervals, like statistics in general, are powerful because they are both consistent with our experience and provide a level of precision we can't articulate. You should use them with your usability test data.
Books Faster than Tablets…or not? :
A study conducted by Nielsen on reading speeds was criticized for going beyond the statistics to support the claim that books are faster than tablets. The crux of this point comes down to considering a finding "statistically significant" only when the p-value is below .05. This criterion is a convention not a commandment and context should always be considered when deciding the role of chance in applied research.
What is a Representative Sample Size for a Survey? :
This common question mixes two concepts: representativeness and sample size. It is more important to ask a few of the right people what they think than a lot of the wrong people. Once you're talking to the right people identify the highest margin of error you can tolerate to compute the right sample size.
Survey Sample Size Package :
A 23 page how-to-guide and Excel calculator that computes sample sizes for rating scales and binary response items. This package allows you to easily answer the common question: What sample size do I need for my survey? It includes plenty of examples and tips for computing the right sample size based on the question type and response analysis.
What five users can tell you that 5000 cannot :
Web analytics has transformed the problem of understanding user behavior from a puzzle to a mystery. Where we once didn’t have enough information, we now can have too much to make sense of. Small sample user testing tells helps answer the "why" mystery. There will be a continued demand for user-researchers who can quantify observational data and make the most of analytic data.
Quantitative Starter Package for Usability Testing :
A package of the popular How to Conduct a Quantitative Usability Test Report and four of the most commonly used excel calculators for getting started measuring and reporting usability with confidence.
User Experience BS Generator :
Is the term UX becoming an overused dot-com term like B2C or clicks-to-mortar? The field of User Experience has matured enough now that it deserves its own BS generator replete with terms that have more use than meaning--as well as a PDF Report with an ROI graph.
Problem Frequency Calculator :
Estimate how many users will experience a problem you see in a small sample usability test by entering the number who encountered the problem and your sample size. Also estimate the probability you can detect problems given your sample size.
iPhone vs. Desktop: Which is Faster? :
It takes about 2 to 3 times as long to send an email on the iPhone as on the Desktop. However, users were able to find percentages faster on the iPhone calculator, showing you can have both portability and efficiency for certain tasks.
Average Task Time Calculator :
An excel calculator for task times that provides an easy way to compute the geometric mean, confidence intervals and a 1-sample t-test to compare a sample against a benchmark.
How to conduct a Quantitative Usability Test :
Do you know how to measure usability? Do you have questions about the benefits and process for conducting a quantitative usability test? I've assembled answers to the 72 most common questions that arise from measuring usability. In this 64 page report I provide concrete examples and plenty of data from a dataset of 120 usability tests, the latest usability research and my decade of experience conducting quantitative usability tests.
This graph was generated using the Usability Scorecard web-application. The Usability Scorecard takes any combination of raw usability metrics (time on task, satisfaction, completion rates and errors), standardizes them into z-scores, then summarizes them into a composite Single Usability Metric (SUM)
Premium Usability: Getting the Discount without Paying the Price
The Risks of Discounted Qualitative Studies:
| Single Usability Metric (SUM) |
| SUM is a single usability metric that summarize the majority of variation in four common summative usability metrics. The theoretical foundations of the model are explained in the CHI paper. A more detailed practitioners guide was presented at UPA and a case study on using SUM to compare the usability of competing products was presented at HCII |
| Confidence Intervals |
| Confidence Intervals show the uncertainty of an estimate and can help Restore Confidence in Usability Results. Calculators are available for Completion Rate (Binomial) or Task Time Confidence Intervals. |
| Sample Size |
| The sample size will affect the amount of uncertainty in an estimate. Its calculation depends on the type of data. Calculators are available for Task Completion, Time on Task and Discovering Problems in an Interface. In usability studies, the task context is important for sample sizes. |
Six Sigma
How Do You Calculate a Z-Score/ Sigma Level?
Making Sense of Usability Metrics: Usability and Six Sigma
Calculating a Sigma Level from Task Success
What's a Z-Score and Why Use it in Usability Testing?
More Six Sigma...