Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

5 Benefits of Remote Usability Testing

Jeff Sauro • September 20, 2011

Remote usability testing has become increasingly popular among user researchers and grew by 19% since 2009.

Even though I have easy access to a brick and mortal usability lab, I particularly like remote testing  for at least 5 reasons:


1. Availability: Do you have time next week to drive to a place you've never been, spend an hour doing a test, then spend time coming home all for $100? Maybe, but a lot of the users you want to hear from don't.

If you want to recruit even a moderately specialized skill (such as accounting) it gets hard to find professionals who are willing to take a half-day off to participate. Your prospects are better if you only have to ask for an hour and they don't even have to leave their office.


2. Geography: Want to test users from Europe, Asia and Australia all for the same product in the same week?  Not a problem, just setup a web-conference, have the users share their screen and have them attempt your tasks. With remote usability testing you don't have to worry about jet-lag, just time-zone differences (scheduling users in Australia from the US is always an interesting exercise in sleep deprivation).


3. Speed:  It would be nice if you could plan weeks ahead of time to get feedback on a prototype then schedule users accordingly. In reality (and in my experience) most design teams want feedback now,  maybe next week and definitely on a Saturday at 10pm.

When users need to take a day off work you need to give them a lot of notice. If it's just a few minutes on their computer, days and times become less constricting. In fact, recently when a user didn't show up for an in-person test I was able to send a few emails and conduct a remote moderated test for a client in the same time slot.


4. The Unmoderated Testing Option: You have the ability to collect data without the need for a facilitator's time. There are many remote tools like Loop11 and UserZoom and low-cost services like Usertesting.com and Userlytics which also recruit and send you videos of users interacting with the site. Some tools can even collect user errors like WebnographerDisclaimer: Measuring Usability LLC has a professional relationship with all services mentioned.


5. You see what they see:  I've conducted a lot of lab-based usability tests over the years. The artificial nature of the tasks in a usability test are one thing, but a more immediate difference is often the computer and browser you have the participant using: Things I hear a lot are:
  • My monitor resolution at home is (bigger, smaller)
  • I usually use (Firefox, IE, Crome, Safari)
  • I use a (PC, Mac, Sparc)
  • My keyboard/mouse are (faster/slower or bigger/smaller)

With remote testing users have the familiarity and comfort of their own setup. This is probably most noticeable in both the screen resolution and browser configuration (like all those lovely tool bars that seem to magically appear).  Here are 6 screens from a usability test I conducted last week. Each screen shot was scaled the same to retain the proportions and show the dramatic differences in how users see just a simple starting message in UserZoom (and your website) differently.

   


About Jeff Sauro

Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 4 books on statistics and the user-experience.
More about Jeff...


Learn More


UX Bootcamp: Aug 20th-22nd in Denver, CO
Best Practices for Remote Usability Testing
The Science of Great Site Navigation: Online Card Sorting + Tree Testing Live Webinar


You Might Also Be Interested In:

Related Topics

Remote Usability Testing
.

Posted Comments

There are 6 Comments

March 5, 2013 | Cris wrote:

next tool: <a href="http://beeux.com">beeux</a> 


March 5, 2013 | Cris wrote:

next tool: http://beeux.com 


November 9, 2011 | Ricca @ Remote Access wrote:

I spent a lot of time on choosing the right one and would recommend thanks for this post!This fits into my budget which is the most important factor fot me. 


September 23, 2011 | UsableInsights wrote:

I like remote testing but have not needed specialized observation software. Recording the session is good for later referral, but I get most of my mileage from screen caps ... I love using screen caps within the results-reporting presentation for observations and recommendations. While Morae is great for recording the session (and when possible, the user's facial expressions as well as the screen), the time it takes to boil down the videos to some meaningful "snippets" makes the analysis time unacceptable (unless there are two folks, one analyzing the results and the other editing the video.
 


September 21, 2011 | S wrote:

Another tool not mentioned above is: usabiliTEST.com 


September 21, 2011 | Sergey wrote:

I think the #2, Geography and time-difference are the two big reasons to consider remote testing! 


Post a Comment

Comment:


Your Name:


Your Email Address:


.

To prevent comment spam, please answer the following :
What is 1 + 5: (enter the number)

Newsletter Sign Up

Receive bi-weekly updates.
[3788 Subscribers]

Connect With Us

UX Bootcamp

Denver CO, Aug 20-22nd 2014

3 Days of Hands-On Training on User Experience Methods, Metrics and Analysis.Learn More

Our Supporters

Use Card Sorting to improve your IA

Userzoom: Unmoderated Usability Testing, Tools and Analysis

Loop11 Online Usabilty Testing

Usertesting.com

.

Jeff's Books

Quantifying the User Experience: Practical Statistics for User ResearchQuantifying the User Experience: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.