The Five Most Influential Papers in Usability
Jeff Sauro • July 7, 2010
I compiled a list of papers that have had a large and lasting influence on the field of Usability and User Experience. I then asked Jim Lewis and Joe Dumas, two pioneers in this field for their top five. There was considerable overlap in both the papers and topics suggesting that while there may be some disagreement with the conclusions of the papers there is strong agreement on their impact.
1. Designing for usability: key principles and what designers think. Gould, J. D. and Lewis, C. (1985)
Go to any UX conference and you'll hear the same points as those raised in this pioneering paper: early and continual focus on users; empirical measurement, iterative design. View Paper [pdf]
Two honorable mentions for pioneering work include:
Al-Awar, J., Chapanis, A., and Ford, R. (1981). Tutorials for the first-time computer user. IEEE Transactions on Professional Communication, 24, 30-37. This is one of the first descriptions of formative usability testing. Prior to this paper most user testing efforts were more summative (benchmark testing).
Shackel, B. (1990). Human factors and usability.
In J. Preece and L. Keller (Eds.), /Human-Computer Interaction, Selected Readings/ (pp. 27-41). This paper defined usability as a function of efficiency, effectiveness & satisfaction (the ISO 9241 pt 11 standard). Despite many proposed extensions, we still think of usability in terms of these three aspects
2. Heuristic evaluation of user interfaces, Nielsen, J., and Molich, R. (1990)
Despite much criticism, Heuristic Evaluation a "discount usability method" still plays a major role in interface evaluation. HE is best conducted with multiple trained professionals (ideally experts in both UX and the domain) and done prior to and in addition to a user-test. View Paper[pdf]
3. Damaged merchandise? A review of experiments that compare usability evaluation methods. Gray, W. D., and Salzman, M. C. (1998).
With the proliferation of discount methods such as Heuristic Evaluation came a number of papers determining which method was "better": User Testing or Heuristic Evaluation? While there were a lot of good insights from these papers, Damaged Merchandise pointed out in detail that the methodologies used to make the comparisons were poorly designed and lead to fallacious conclusions. View Paper[pdf]
4. Refining the test phase of usability evaluation: how many subjects is enough?. Virzi, R. A. (1992)
Want to get a reaction? Just mention testing with five users
to anyone in software development. Few topics elicit more emotion than small sample sizes in user testing
. Robert Virzi's paper codified this number into UX history. It was one of a few papers which showed how to use the Binomial (or Poisson) distributions to arrive at the now famous formula
. View Paper [Paid Link]
Other Honorable mentions in this category include:
Lewis, James (1982) "Testing Small System Customer Setup[pdf]
" in Proceedings of the Human Factors Society 26th Annual Meeting p. 718-720 (1982).This is probably the first to use the binomial for usability problem discovery.
5. Comparative Evaluation of Usability Tests Molich et al (1998).
Do usability evaluators agree on what usability problems are? There is considerable variability between usability evaluators. The Comparative Usability Evaluations (CUE) papers and panels have been helping us better understand the limits of human judgment as well as suggesting improvements for usability methods and reports. View Paper [pdf]
In the same year as the first CUE paper at UPA, Jacobsen, Hertzum and John published a paper using a more controlled setup but found similar findings. Since then there have been 8 CUE's and a number of papers written about the Evaluator Effect.
What research papers have influenced you ?