Guest Post By Jim Lewis • January 3, 2012

Introduced in 2003 by Fred Reichheld, the Net Promoter Score (NPS)The NPS uses a single Likelihood to Recommend question ("How likely is it that you would recommend our company to a friend or colleague?") with 11 scale steps from 0 (Not at all likely) to 10 (Extremely likely), as shown below.

In NPS terminology, respondents who select a 9 or 10 are "Promoters," those selecting 0 through 6 are "Detractors," and all others are "Passives". The NPS from a survey is the percentage of Promoters minus the percentage of Detractors, making the NPS a type of top-box-minus-bottom-box type of metric (actually, top 2 minus bottom 7 boxes) – thus, the "net" in Net Promoter.

For example, suppose you've collected 100 LTR ratings for a company for which 25 ratings fall between 0 and 6 (25% Detractors), 25 fall between 7 and 8 (25% Passives), and 50 fall between 9 and 10 (50% Promoters). The resulting NPS is the percentage of Promoters minus the percentage of Detractors, in this case, 25%. The developers of the NPS hold that this metric is easy for managers to understand and to use to track improvements over time, and that improvements in NPS have a strong relationship to company growth[pdf]. The metric becomes especially valuable when compared to industry benchmarks.

Since its introduction, the NPS has generated controversy. For example, Keiningham et al. (2007, 2008) challenged the claim of a strong relationship between NPS and company growth. In general, top-box and top-box-minus-bottom-box metrics lose information during the process of collapsing measurements from a multipoint scale to percentages of a smaller number of categories, and thus lose sensitivity (although increasing sample sizes can make up for lack of sensitivity in a metric).

Despite these criticisms, it is unlikely that the popularity of the NPS will diminish at any time in the near future due to its simplicity and intuitiveness.

* Our friends at Satmetrix want us to remind you that Net Promoter, NPS, and Net Promoter Score are trademarks of Satmetrix Systems, Inc., Bain & Company, and Fred Reichheld

The SUS accounted for 43% of post-test questionnaire usage in a recent study of a collection of unpublished usability studies. Research conducted on the SUS has shown that although it is fairly quick, it is probably not all that dirty.

LTR = 0.52 + 0.09(SUS)

LTR = 1.33 + 0.08(SUS)

Although the parameters of the equation are somewhat different, this equation isn't dramatically different from the initial one. The intercept is somewhat greater (1.33 instead of 0.52) and the slope is a little less steep (8% instead of 9%). The percentage of variation in LTR explained by SUS is slightly higher (about 39%, corresponding to a statistically significant correlation between LTR and SUS of .623). When you change the data from which you derive a regression equation, you expect some change in the parameters, so this shouldn't be shocking news – especially with this sample size roughly 15 times the size of the initial sample.

One of the things Jeff and I noticed with both regression equations was that the slope was almost equal to 0.1 (10%), so we wondered what would happen to the quality of the regression equation if we dropped the intercept (mathematically, forcing its value equal to 0). If it turned out that this changed the slope to 10%, then it would result in a very easy to remember relationship between LTR and SUS – if you know the SUS score, just divide it by 10 to get an estimate of the user's likelihood to recommend.

LTR = 0.1(SUS)

LTR = SUS/10

Whenever you deviate from the parameters indicated by least squares regression, you expect the quality of the regression formula as measured by its coefficient of determination (the percentage of variance explained) to decline. The question was how much quality we would lose as a consequence of this simplification. LTR = SUS/10

It turned out that the percentage of variation in LTR explained by SUS for the simplified equation was about 37% (corresponding to a statistically significant correlation of about .606) – a drop of only 2%. If you don't remember the updated equation, you will get almost as good a prediction with the easier-to-remember simplified equation.

This can be helpful if your company is using NPS from other voice-of-the-customer sources as a consistency check. If you're serious about using LTR in the future, it is a small effort to collect ratings of the NPR item in addition to the SUS rather than estimating it.

About Jim Lewis PhD : Jim has worked as a human factors engineer and usability practitioner at IBM since 1981. He has published influential research on the measurement of usability satisfaction, use of confidence intervals, and sample size estimation for usability studies.

He is a BCPE Certified Human Factors Professional, an IBM Master Inventor, and a member of UPA, HFES, APS and APA. He is the author of Practical Speech User Interface Design (Taylor & Francis 2011) and is co-author of the forthcoming book Quantifying the User Experience (Morgan Kaufmann 2012).

The Experiment Requires That You Continue: On The Ethical Treatment of Users

28 Resources for Getting Started In UX

The Five Most Influential Papers in Usability

Should you use 5 or 7 point scales?

5 Examples of Quantifying Qualitative Data

Why you only need to test with five users (explained)

97 Things to Know about Usability

How to Conduct a Usability test on a Mobile Device

A Brief History of the Magic Number 5 in Usability Testing

How common are usability problems?

Nine misconceptions about statistics and usability

Does better usability increase customer loyalty?

8 Ways to Show Design Changes Improved the User Experience

What five users can tell you that 5000 cannot

.

Quantifying the User Experience: Practical Statistics for User ResearchThe most comprehensive statistical resource for UX Professionals Buy on Amazon | |

Excel & R Companion to Quantifying the User ExperienceDetailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R Buy on Amazon | Download | |

A Practical Guide to the System Usability ScaleBackground, Benchmarks & Best Practices for the most popular usability questionnaire Buy on Amazon | Download | |

A Practical Guide to Measuring Usability72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software Buy on Amazon | Download |

.

.

.