The Methods UX Professionals Use
Jeff Sauro • August 30, 2011
There are many tools in the UX professional's toolkit.
It's always interesting to know the popular and neglected methods.
The recent
UPA biennial survey asked members which UX methods they use
in addition to their salaries and I've summarized some results here.
I've grouped the responses into larger categories so they're more digestible, but clearly many methods can fall into different groups. The yellow error bars seen on the graphs represent the 95% confidence intervals around the percentage.
Usability Testing & Expert Reviews
Not surprisingly, some form of usability testing dominates the methods used by UX professionals with 82% of respondents reporting they perform lab, remote or informal usability testing.
Expert reviews are close second at 75% (which includes
Heuristic Evaluations and Cognitive Walkthroughs). Expert reviews and usability testing are being used together by 71% of respondents and over 95% of respondents who report performing an expert review also report performing usability testing.
Growth in Remote Unmoderated Testing
One category that continues to grow is testing users asynchronously over the Internet. Despite the clunky name, remote unmoderated usability testing is a low cost and effective technique for collecting task based usability data. Services like
Loop11 and
Userlytics provide results for a fraction of the time and without the inconvenience of the lab (
Disclosure: Both companies are sponsors of MeasuringUsability.com).
In the
2009 UPA Survey[pdf], 18% of respondents reported using remote unmoderated testing--showing growth of around 28%. It wasn't even listed as a method to select from in the 2007 survey--suggesting new techniques can still a have major impact on the field.
And in case you think remote unmoderated testing will replace the need for lab-testing, the data seems to suggest it will take a while. Roughly the same percentage of respondents report performing lab based testing in 2007 (54%), 2009 (54%) and 2011 (52%). The slight down-tick in the latest survey is still within the margin of error (95% CI between 50% and 55%).
A more likely replacement will be moderated remote testing. It's hard to replace the richness of interacting with a user and the advantage of not needing people to drive to a lab. There was decent growth in the use of moderated remote testing with an 8 percentage point increase over the last survey from 42% to 50% (a 19% increase).
Usability Testing For the Rich and Famous
While eye-tracking has gotten a lot cheaper and less clumsy over the last decade, performing these studies still requires a lab, training and a good-sized budget to afford the specialized equipment. Roughly 16% of respondents reported using the method that generates those really cool heatmaps--up 3 percentage points from 2009.
Surveys & General Research
This is a broad category that involves methods that cross departments. Interestingly enough even the much
maligned method of focus groups is employed by almost 40% of respondents.
Defining Users & Requirements
Who are your users? What are their goals? What tasks do they perform? All the expensive lab equipment in the world won't help if you're testing the wrong users doing the wrong things. Roughly the same percentage of respondents in the 2009 survey report using these early stage-methods. Maybe we already know the users and what they're trying to do, or perhaps we should put a bit more effort in early.
Information Architecture
You can have well defined users and tasks, but user's also need to find things on a website. Information architecture isn't just for
Louis Rosenfeld anymore with more than half of respondents performing the technique of
card-sorting. (Disclosure: Optimal Workshop is a sponsor of MeasuringUsability.com). 
Design
Without a design you have nothing to test. The responses show that prototyping (low and high-fidelity) is a popular way to get feedback from users early and often. More than two-thirds of respondents (which includes designers and researchers) report using some form of prototyping.
Accessibility
Less than 20% of respondents report using some form of accessibility testing. This may be because this function also crosses into more QA roles which are less represented in this survey or we're all neglecting accessibility.
Training & Advising
Just talking about UX and the methods is itself a method reported by around a third of respondents.
Predictions for 2013
I suspect the 2013 results won't differ too much from the 2011 or 2009 results. Lab-based usability testing will still play a major role for many practitioners as will expert reviews, card-sorting and prototyping.
I do expect to see a continued increase in both moderated and unmoderated testing. The biggest reason for not performing usability testing has been the cost of testing users. If you can recruit users and have them attempt structured tasks on a website or web application the incremental cost of testing additional users in negligible.
I think moderated remote testing will continue to grow (perhaps more than unmoderated). Desktop sharing software is ubiquitous (I regularly use GoTo Meeting) to record usability sessions with users from all over the world. But there will always be a need for the judgment of the UX professional to interact with a user, ask the right questions, interpret the data and summarize findings in a usable way.
