Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

Sample Size Calculator for Discovering Problems in a User Interface

Jeff Sauro • October 1, 2006

Sample Size from an estimate of Problem Occurrence (p)

If the probability of detecting a UI problem is known in advance, use this portion of the calculator to estimate the total number of users needed to uncover on average the specified percentage of problems (e.g. 90%). The calculator is based on the binomial probability formula.
Input
Discover of all Problems.
Problem Occurrence (values between 0 and 1)

Results


Estimate Problem Occurrence (p) then Sample Size
This portion of the calculator first builds an estimate of the probability of detecting a UI problem (from sample data). It then produces an estimate of the number of users needed to discover the specified percent of total problems. It uses the Good-Turing and Normalization procedure as outlined by Lewis (2001) and further discussed in (Turner, Lewis & Nielsen 2006).

Input
Discover of all Problems.
Total participants
Problems Discovered:

Results
References
Lewis, James (2001) "Evaluation of Procedures for Adjusting Problm-Discovery Rates Estimated from Small Samples" in The International Journal of Human-Computer Interaction 13(4) p. 445-479

Turner, C. W., Lewis, J. R., and Nielsen, J. (2006). Determining usability test sample size. In W. Karwowski (Ed.), International Encyclopedia of Ergonomics and Human Factors (pp. 3084-3088). Boca Raton, FL: CRC Press.

Rate this Calculator

Avg. Rating 58.36 (102)

Poor         Excellent
012345678910

.

Posted Comments

There are 5 Comments

February 2, 2012 | Ivan wrote:

Jeff,
the "Estimate Problem Occurrence (p) then Sample Size" calculator doesn't work now.
Uncaught TypeError: Object [object Object] has no method 'prettyGallery' 


June 17, 2009 | Davant G Bryant wrote:

6. Why is population shape of concern when estimating a mean? What does sample size have to do
with it? 


April 30, 2008 | Raul R Wells wrote:

Very useful, I could not move the bar, so it reported 50% against a 100% I wished to mark 


January 4, 2008 | John Romadka wrote:

It might help if you could give a real world example of how these calculators could be used. Because I think I get it, but an example would make it more concrete. 


January 4, 2008 | John Romadka wrote:

Jeff,
I wonder if you could add a little contextual help for the Problem Occurance field (".30"). Specifically, when would I use a number closer to 0 or a number closer to 1. 


Post a Comment

Comment:


Your Name:


Your Email Address:


.

To prevent comment spam, please answer the following :
What is 3 + 2: (enter the number)

.
.