Measuring Usability: Sample Size Calculator for Discovering Problems in a User Interface

Sample Size Calculator for Discovering Problems in a User Interface
by Jeff Sauro | October 1, 2006 :: 10 Related Questions How useful was this calculator?
Avg. Rating: 64 ( 14 ) | 3 Comments

Page Tags

Tag Name	#	Vote
Calculator	9
Binomial Probability	7
Sample Size	6
UI Problems	6
binomial	1

New Tag:

Sample Size from an estimate of Problem Occurrence (p)

If the probability of detecting a UI problem is known in advance, use this portion of the calculator to estimate the total number of users needed to uncover on average the specified percentage of problems (e.g. 90%). The calculator is based on the binomial probability formula.

Input

Discover of all Problems.

Problem Occurrence (values between 0 and 1)

Results

Estimate Problem Occurrence (p) then Sample Size
This portion of the calculator first builds an estimate of the probability of detecting a UI problem (from sample data). It then produces an estimate of the number of users needed to discover the specified percent of total problems. It uses the Good-Turing and Normalization procedure as outlined by Lewis (2001) and further discussed in (Turner, Lewis & Nielsen 2006).

Input

Results

References
Lewis, James (2001) "Evaluation of Procedures for Adjusting Problm-Discovery Rates Estimated from Small Samples" in The International Journal of Human-Computer Interaction 13(4) p. 445-479

Turner, C. W., Lewis, J. R., and Nielsen, J. (2006). Determining usability test sample size. In W. Karwowski (Ed.), International Encyclopedia of Ergonomics and Human Factors (pp. 3084-3088). Boca Raton, FL: CRC Press.

If you'd like an email when a new article or calculator is posted sign up for Email Updates.

Related Questions

Ask a Question

	A local bakery determines that the probability of a customer ordering a blueberry muffin is 50%. 5 customers come into the bakery. A. What is the probability that 3 out of the 5 order a blueberry muffin? B. Explain why this scenario would meet the criteria of a binomial distribution. A random sample of 100 students who took a statistics exam were asked their score on the exam. The mean score on the test was 50; the standard deviation was 10. The scores are normally distributed. A) What percent of the students scored below 30? B) What percent scored between 30 and 55?
	A surgical technique is performed on eight patients. You are told that there is an 80% chance of success. a) Construct a binomial distribution. b) Find the mean, variance, and standard deviation of the probability distribution. c) Find the probability that the surgury is successful for exactly six patients. d) Find the probability that the surgury is successful for fewer than six patients
	How to determine the sample size for comparing mulltiple parameters like Height , weight, Blood pressure , Blood parameters like blood glucose, total cholesterol, etc in two different populations?
	Suppose that in NY State, 35% of voters are Democrats, 30% are Republicans, and 11% are independent. You are conducting a poll by randomly calling registered voters. In your first four calls, what is the probability that you talk to: a)all Republicans? b)exactly 2 Republicans? c)no Independents? d)all Independents? e)at least one independent?
	In 1992, the FAA conducted 86,991 pre-employment drug tests on job applicants who were to be engaged in safety and security relatd jobs and found that 1,143 were positive a. construct a 95% confidence interval for the population proportion of positive drug test. b. why is the normality assumption not a problem, despite the very small value?
	A doctor knows from experience that 20% of the patients to whom he gives a high blood pressure drug will have undesirable side effects. Assume the doctor gives that drug to ten of his patients. Find the probability that among the ten patients to whom he gives the drug, at most two will have undesirable side effects.
	State the main points of the Central Limit Theorem for a mean. B. Why is population shape of concern when estimating a mean? What does sample size have to do with it?
	How do you determine the sample size for data for which the mean and standard deviation are not known?
	A machine cuts circular filters from large rolls of material. If 7.3% of the filters fail to meet specifications, use the normal approximation to the binomial to compute the probability that a sample of 100 of the filters will contain 5 or fewer that fail to meet specifications.
	I am charged with sampling expense report submissions for accuracy. We get 8000 T&E claims, and want to sample a subset making inferences about the population. I can probably get a good estimate of the populations' SD, how would i calculate required sample size? I think i would be measuring the delta between actual and claimed - the majority be 0. Is this a one sided issue? Thanks in advance - FT Also, would i need the acceptable level first? ie. we would accept an average difference of $5? How can i work backwards if the first sample size yields an average of $1? (if that made sense)

Ask a Question

April 30, 2008 | Raul R Wells wrote:

Very useful, I could not move the bar, so it reported 50% against a 100% I wished to mark

January 4, 2008 | John Romadka wrote:

It might help if you could give a real world example of how these calculators could be used. Because I think I get it, but an example would make it more concrete.

January 4, 2008 | John Romadka wrote:

Jeff, I wonder if you could add a little contextual help for the Problem Occurance field (".30"). Specifically, when would I use a number closer to 0 or a number closer to 1.

Comments
Name
Email Address
To prevent comment spam, please answer the following question before submitting (tags not permitted) : What is 2 + 5: (enter the number)