Problem Frequency Calculator

Download an excel calculator to estimate a problem's impact on the whole user population from any sample size. Just enter the number of users who experience a problem and the sample size. This calculator also estimates the probability you can detect problems given your sample size.

Click for more details on this screen shot

$22.99 Now :

If you will be making copies or distributing the product, please purchase the site license

Features & Benefits

Estimate a problem's impact on all users: If you see 2 out of 10 users experiencing a problem this calculator will provide the most likely percentage of users who will experience the problem (between 5% and 52%).
Test the problem frequency seen in a sample against a test percentage: If 2 out of 10 users have a problem locating a product, the probability exactly 20% of all users will also have this problem is 30%.
Find the chance you'll be able to detect problems given your sample size: At a sample size of 10, the probability of seeing a problem in a usability test that affects 30% of users is 97%. The probability of seeing problems that affect only 1% of users is 9.6%.

Who Should Buy

Anyone who needs to understand both the power and limits of small sample data for detecting UI problems or defects.This calculator will provide you with:

How many users are likely experiencing a particular problem
The chance you have of seeing a problem in a test given a small sample size
The probability a problem affects a specific percent of all users.

Select Sample Screen Shots

Simple Data Entry

You only need to enter two number to get immediate results: The number of users who encountered a problem and the total number tested. The calculator will then tell you the best estimate of the total user population that is impacted by this problem

What percent of problems will you see with your sample size

With smaller sample sizes you can still find problems, you are just limited to seeing the more obvious problems (problems that will affect most users). The calculator automatically generates a graph to show you the probability of detecting certain problems in a usability test given your sample size.

Is your sample size large enough?

The larger your sample size the better chance you will have of seeing problems that affect a smaller and smaller percentage of your users. There is a diminishing return in testing more users and this calculator helps you prioritize your resources by showing you the probability of detecting classes of problems given a certain frequency in the total user population.

A Detailed Explanation of the Numbers

It's easy to get confused with all the numbers, that's why I included a "How to Report" section that explains in simple language what the numbers mean. As your sample size and problem occurrences change, the text in the reporting section automatically updates.

What Customers are Saying:

Thanks for the calculator. It hits the spot perfectly. I played with it for 15 minutes and all the elements are understandable and I can see what's happening.

Philip Hodgson Ph.D.
Blueprint Usability