Can you use the SUS for websites?

Jeff Sauro • January 18, 2010

The System Usability Scale (SUS) is the most popular standardized usability questionnaire. SUS was developed about 20 years ago at Digital Equipment Corporation by John Brooke. It's popular for two reasons: it's free and short (at only 10 questions). The process of taking a set of ordinary questions and making it into a psychometrically valid and reliable "standardized" questionnaire essentially involves having many users answer a large set of questions against many products and interfaces. You then analyze the responses to see which questions correlate well, tend to a cohesive structure and elicit about the same pattern of responses from people. You throw out questions with low correlations and unusual response patterns to whittle the set down to a manageable set that purports to measure something, in this case perceptions of usability.  This process of standardization was performed on the original super-set of SUS candidate questions some twenty years ago before the web (as we know it) existed. 

I've seen SUS data for desktop applications, printers, cell-phones, Interactive-Voice Response systems and most recently on websites and web-applications.  Some have argued that the 10 SUS questions just don't lend themselves well to effectively measuring websites. Websites are typically walk-up-and use customer facing interfaces. Does a SUS score on a website mean the same thing as a SUS score on a word-processor?

1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system.

Exhibit 1: The 10 SUS questions.

There are certainty some questions which don't seem well-suited for websites (e.g.Question 4, "I think I'd need the support of a technical person to be able to use this website.") and I'm sure some of the candidate questions which were thrown out in the original design phase of the questionnaire might be better. So there is good reason to be skeptical about the validity of SUS scores on website. With that said, websites are still interfaces and while some of the questions aren't the best, they seem to be close enough. Of the data I've seen so far, websites with high-perceived usability and applications with high-perceived usability tend to have similar high SUS scores. As I get more data on SUS scores from websites, I'll be interested to see how well this heavily used psychometric instrument holds up to the changing times.

You Might Also Be Interested In:

Measuring Usability with the System Usability Scale (SUS)	5 Second Usability Tests	Should you use 5 or 7 point scales?	That's the worst website ever!: Effects of extreme survey items
If you could only ask one question, use this one.

Rate this Blog

Avg. Rating 7.5 (18)

Poor										Excellent
0	1	2	3	4	5	6	7	8	9	10

.

Posted Comments

There are 2 Comments

August 10, 2010 | Carolyn Snyder wrote:

I've used the SUS on a bunch of web sites, and in my opinion the scores are usually well aligned with the qualitative data. Your post makes me wonder if "I would need help from a customer support person" is the solution to #4, though even as is I think it's a useful question. Although it reads a bit strangely for a site that works well, I've seen instances where people have agreed with it, either because the site was buggy or required specialized domain knowledge.

The question I'd most like to overhaul is #5. Some people don't know the meaning of "integrated," and "functions" implies software. I've used a variant: "The various parts of this site fit together well." I have also, at times, replaced "cumbersome" with "awkward," for similar reasons.

I realize that tweaking validated questionnaires is the kind of thing that drives gurus like you nuts, but ultimately I don't really care about comparing a web site to a word processor. I care about comparing a web site with similar sites intended for similar audiences. I never use a SUS score alone as justification for a recommendation, and in the report I provide caveats for interpreting it.

It's like what they say about democracy - it's the worst form of government... except compared to everything else. When something better comes along, I'll switch. Until then, you can have my SUS when you pry it from my cold, dead hands :-). 

---

January 19, 2010 | Asbjorn Folstad wrote:

Thanks for an interesting and relevant post. It seems to be no easy solution to the challenge of finding user satisfaction measurements that are bot valid and relevant. It would be interesting to have your thoughts on recommendable measurements (if any) that are particularly tailored to measure the user satisfaction of websites. 

---

Post a Comment

Comment:

Your Name:

Your Email Address:

.

To prevent comment spam, please answer the following :
What is 2 + 4: (enter the number)