What's a Z-Score and Why Use it in Usability Testing?

Jeff Sauro • September 17, 2004

The Power of Z

A common statistical way of standardizing data on one scale so a comparison can take place is using a z-score. The z-score is like a common yard stick for all types of data. Each z-score corresponds to a point in a normal distribution and as such is sometimes called a normal deviate since a z-score will describe how much a point deviates from a mean or specification point.

In Six Sigma parlance, z-score and process sigma are used interchangeably and are sometimes called z-equivelents. Strictly speaking, the process sigma and z-equivalents are loosely tied to the statistical z-score. The statistical z-score has very strict definitions derived from the rules of the normal distribution. For most applications in Six Sigma, ignoring some of those constraints is innocuous. In usability testing the benefit of the standardization from process sigmas allow us to meaningfully compare disparate measures like task completion and time on task.

The z-score/process sigma is calculated by subtracting your sample mean from a target data point and dividing by the target standard deviation. This value is a measure of the distance in standard deviations of a sample from the mean and is expressed using the Greek letter σ. If your sample is 3 standard deviations from the spec limit, you would describe your process as 3 sigma. or 3σ

The further away a sample is from the spec limit the higher the z-score and process sigma. A higher process sigma means a less defective process. The term Six Sigma originates from the z-score. 6σ means that six standard deviations lie between the mean of a sample and the nearest specification limit. To visualize the Z-score see the Interactive Graph of the Standard Normal Curve

Each process sigma has two equivalent values which provide a meaningful way to compare data and understand how defective a process is:

1. DPMO: Each expresses the probability of a defect in terms of a defect per million opportunities or DPMO. That is, if a condition were to occur one million times, how many times out of that one million would a defect occur? A process sigma of .5 is equal to 308,000 defects per million opportunities. And a process sigma of 2.5 means that 6,210 out of 1 million times there will be a defect. For a sample that is 6σ, the DPMO is .0.001. Some organizations prefer to think in terms of defects per opportunities instead of the more abstract "standard deviations above the spec limit."
   
2. Probability of a Defect: The process sigma can also be described in terms of a probability of a defect. A z-score of .5 means there is a 30% probability of encountering a defect. A z-score of .25 means there is a 40% probability of a defect. For a sample that is 6σ, the probability of a defect is .0000001%. Note: Values do not include a 1.5σ shift.

Why use a Process Sigma?

The process sigma is helpful in three ways:

1. It allows you to compare disparate types of data (seconds, which are a continuous measurement with task completion which is binary with errors which are discrete count data)
   
2. It provides you with a probability of a defect
   
3. You can meaningfully compare two different products or processes:
   1. The process sigma for one release of a software product can be compared to subsequent versions
   2. You can compare two different products' process sigmas
   3. You can compare one module of the same product with a different module on the same product
   4. You can use the properties of the normal distribution to aide in assessing and improving your data set.

Learn More

UX Bootcamp: Sept 25th-27th in Denver, CO
Best Practices for Remote Usability Testing
The Science of Great Site Navigation: Online Card Sorting + Tree Testing Live Webinar

Rate this Blog

Avg. Rating 53.87 (298)

Poor										Excellent
0	1	2	3	4	5	6	7	8	9	10

Related Topics

Z-score

.

Posted Comments

There are 13 Comments

January 25, 2013 | Soma wrote:

I want to comparision of two Z score but I have not getting more idea about the comparision of the Z score. 

---

October 3, 2012 | anonymous wrote:

Comprehensive 

---

September 11, 2012 | Khalil wrote:

I am UX (QoE) Scientist and I found this post easy to understand and informative. Thanks 

---

July 5, 2012 | MaraCantu28 wrote:

Don't you recognize that this is the best time to get the <a href="http://goodfinance-blog.com">loans</a>, which can make your dreams come true.  

---

April 18, 2012 | pramod wrote:

It is very good to find these kind of learning material on web. it makes the learning easier. 

---

October 13, 2011 | anonymous wrote:

The Interactive graph drives the point home like a nail. 

---

February 24, 2011 | PALASH ROY wrote:

Very good thing 

---

November 23, 2010 | Dr. John wrote:

wonderful! simple and comprehensive. 

---

September 11, 2009 | Barbara wrote:

Discuss types of research where using the t statistic may be an appropriate alternative to using a z-score. 

---

August 25, 2008 | DERYA TERZIOGLU wrote:

what is the difference between z score and sigma value? 

---

May 28, 2008 | samar wrote:

that was great job thank you very much 

---

May 28, 2008 | samar wrote:

that was great job thank you very much 

---

March 22, 2008 | Michael wrote:

How do you compare two z scores to see if there is a significant difference between the two? would that be a t-test, or something else? 

---

Post a Comment

Comment:

Your Name:

Your Email Address:

.

To prevent comment spam, please answer the following :
What is 3 + 4: (enter the number)