What's a Z-Score and Why Use it in Usability Testing?

by Jeff Sauro | September 17, 2004 :: RSS

The Power of Z

A common statistical way of standardizing data on one scale so a comparison can take place is using a z-score. The z-score is like a common yard stick for all types of data. Each z-score corresponds to a point in a normal distribution and as such is sometimes called a normal deviate since a z-score will describe how much a point deviates from a mean or specification point.

In Six Sigma parlance, z-score and process sigma are used interchangeably and are sometimes called z-equivelents. Strictly speaking, the process sigma and z-equivalents are loosely tied to the statistical z-score. The statistical z-score has very strict definitions derived from the rules of the normal distribution. For most applications in Six Sigma, ignoring some of those constraints is innocuous. In usability testing the benefit of the standardization from process sigmas allow us to meaningfully compare disparate measures like task completion and time on task.

The z-score/process sigma is calculated by subtracting your sample mean from a target data point and dividing by the target standard deviation. This value is a measure of the distance in standard deviations of a sample from the mean and is expressed using the Greek letter σ. If your sample is 3 standard deviations from the spec limit, you would describe your process as 3 sigma. or 3σ

The further away a sample is from the spec limit the higher the z-score and process sigma. A higher process sigma means a less defective process. The term Six Sigma originates from the z-score. 6σ means that six standard deviations lie between the mean of a sample and the nearest specification limit. To visualize the Z-score see the Interactive Graph of the Standard Normal Curve

Each process sigma has two equivalent values which provide a meaningful way to compare data and understand how defective a process is:

1. DPMO: Each expresses the probability of a defect in terms of a defect per million opportunities or DPMO. That is, if a condition were to occur one million times, how many times out of that one million would a defect occur? A process sigma of .5 is equal to 308,000 defects per million opportunities. And a process sigma of 2.5 means that 6,210 out of 1 million times there will be a defect. For a sample that is 6σ, the DPMO is .0.001. Some organizations prefer to think in terms of defects per opportunities instead of the more abstract "standard deviations above the spec limit."
   
2. Probability of a Defect: The process sigma can also be described in terms of a probability of a defect. A z-score of .5 means there is a 30% probability of encountering a defect. A z-score of .25 means there is a 40% probability of a defect. For a sample that is 6σ, the probability of a defect is .0000001%. Note: Values do not include a 1.5σ shift.

Why use a Process Sigma?

The process sigma is helpful in three ways:

1. It allows you to compare disparate types of data (seconds, which are a continuous measurement with task completion which is binary with errors which are discrete count data)
   
2. It provides you with a probability of a defect
   
3. You can meaningfully compare two different products or processes:
   1. The process sigma for one release of a software product can be compared to subsequent versions
   2. You can compare two different products’ process sigmas
   3. You can compare one module of the same product with a different module on the same product
   4. You can use the properties of the normal distribution to aide in assessing and improving your data set.

View All Articles | Subscribe to RSS | Follow on Twitter | Get Email Updates

Rate this Article

Avg. Rating 5.841 (155)

Poor										Excellent
0	1	2	3	4	5	6	7	8	9	10

---

Comments

Name

Email Address Not Published, sold or abused

To prevent comment spam, please answer the following question before submitting (tags not permitted) : What is 4 + 2: (enter the number)

---

Most Popular Content

GoDaddy Web Hosting for $1.99

Six Sigma Training

Featured Product