Log-in | Contact Jeff | Email Updates

Email Updates | Contact
Measuring Usability Homepage
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro
Sample Size Calculator for Discovering Problems in a User Interface
by Jeff Sauro | October 1, 2006 :: RSS Subscribe to RSS Feed of Measuring Usability Updates

Follow MeasuringUsability.com on Twitter

RSS Subscribe to RSS Feed of Measuring Usability Updates

Get Email Updates

Sample Size from an estimate of Problem Occurrence (p)

If the probability of detecting a UI problem is known in advance, use this portion of the calculator to estimate the total number of users needed to uncover on average the specified percentage of problems (e.g. 90%). The calculator is based on the binomial probability formula.
Input
Discover of all Problems.
Problem Occurrence (values between 0 and 1)

Results


Estimate Problem Occurrence (p) then Sample Size
This portion of the calculator first builds an estimate of the probability of detecting a UI problem (from sample data). It then produces an estimate of the number of users needed to discover the specified percent of total problems. It uses the Good-Turing and Normalization procedure as outlined by Lewis (2001) and further discussed in (Turner, Lewis & Nielsen 2006).

Input
Discover of all Problems.
Total participants
Problems Discovered:

Results
References
Lewis, James (2001) "Evaluation of Procedures for Adjusting Problm-Discovery Rates Estimated from Small Samples" in The International Journal of Human-Computer Interaction 13(4) p. 445-479

Turner, C. W., Lewis, J. R., and Nielsen, J. (2006). Determining usability test sample size. In W. Karwowski (Ed.), International Encyclopedia of Ergonomics and Human Factors (pp. 3084-3088). Boca Raton, FL: CRC Press.

If you'd like an email when a new article or calculator is posted sign up for Email Updates.

Rate this Article

Avg. Rating 5.918 (70)

Poor         Excellent
012345678910



Comments
Name
Email Address
Not Published, sold or abused

To prevent comment spam, please answer the following question before submitting (tags not permitted) :
What is 3 + 2: (enter the number)
June 17, 2009 | Davant G Bryant wrote:
6. Why is population shape of concern when estimating a mean? What does sample size have to do with it?
 
April 30, 2008 | Raul R Wells wrote:
Very useful, I could not move the bar, so it reported 50% against a 100% I wished to mark
 
January 4, 2008 | John Romadka wrote:
It might help if you could give a real world example of how these calculators could be used. Because I think I get it, but an example would make it more concrete.
 
January 4, 2008 | John Romadka wrote:
Jeff, I wonder if you could add a little contextual help for the Problem Occurance field (".30"). Specifically, when would I use a number closer to 0 or a number closer to 1.