Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

Measuring Website Visitors' True Intent

Jeff Sauro • July 23, 2013

Who are your users and what are they doing?

Measuring the user experience starts with understanding who your users are and what tasks they are trying to accomplish on your website. It's one of the first things we'll cover at the UX Boot Camp.

Visitor profile and intent are not something you can derive easily from Google Analytics or log files.

These sources provide an immense amount of data on how many people are visiting your website, and their geography, browsers and device (which of late is especially important). See Louis Rosenfeld's new book, Search Analytics, for advice on making the most of these sources.

You also know which pages are popular and how long visitors stay for. If things are set up right, you might even know how many browsers convert to customers. 

But for the vast majority of users who don't purchase something, it's unclear what they came to the website for and whether they accomplished their goal. 

True Intent

One of the most cost effective and easiest ways to understand who your users are and what they are trying to do is to conduct a true intent study.   As the name suggests, such studies aim to understand the intended goals of actual visitors to a website.   Data is collected by intercepting users while on the website of interest.  No doubt you've encountered some of these popups while browsing a website.

We use software from UserZoom that inserts a single line of javascript in some or all pages on the website we are interested in (usually in an include-file next to the Google Analytics javascript code). The UserZoom software allows us to control the percent of visitors that receive the pop-up, only delivers it once to each visitor, and controls the delay before the popup appears.



The popup intercept directs participants to a short survey (no more than 3-5 minutes) asking them to describe what they intended to accomplish on their site (their true intent), some questions about demographics (who they are), branding questions, and often the standardized questions on usability, credibility, and appearance from the SUPR-Q questionnaire.

Who are the users?

In many organizations, even large, mature ones, there is still a bit of a mystery as to the demographic composition of the website visitors. There's usually more data on customers from market research, but unless you've actually surveyed users on the live website, it's hard to know if your customer profile matches your browser profile.

Usually, just a few quick questions (age, gender, occupation, income etc.) quickly narrow the profile picture and can help build the customer segments and provide that ever-sought after data for personas

For a more in-depth assessment of your visitors, you can ask questions about purchasing behavior or affiliation. For example, universities have prospective and current students, faculty, alumni and media all browsing their website. Organizations like the National Multiple Sclerosis Society (NMSS) and National Cancer Institute (NCI) also benefit from knowing the percentage of visitors who are living with MS or Cancer, volunteers, physicians, or those with friends and family living with the disease.

What are they doing?

To understand the intent of your visitors, you can simply ask visitors to briefly describe why they came to your website, using an open-ended textbox.  While we use open-ended boxes, depending on the website and your users, you may get a lot of less than helpful information, like ("I'm researching information" or "Browsing items").   For that reason, we often recommend having a finite set of options for users to select from in addition to the textbox or instead of it.

The options should be at the level of granularity that's both meaningful to the visitor and actionable to the organization.  On a consumer ecommerce website, most visitors are there to browse or buy, but there are a lot of phases in the purchasing process.

Knowing what percent of visitors are in each group can help inform design.   For example, "I know exactly what I want and I'm ready to purchase it" or "I'm narrowing my search to a few brands" may be different options to present. For organizations like NMSS and NCI, while many visitors will be those living with the diseases, understanding their stages is informative and prioritizes content. Knowing the percent that are unsure if they have the disease, are recently diagnosed, or are living with it or recovering, is important, as all have different content needs.

Top Tasks

If you already have some idea about the types of tasks users perform (for example, from a previous true intent study), present these tasks to users in a randomized list, and ask them to select only the 3-5 most important tasks they want to be able to accomplish. This top-task analysis becomes a crucial ingredient for identifying the tasks you test in usability tests and can help prioritize design efforts.

What do they think?

True intent surveys are also a great place to obtain baseline data on how visitors view your website on the key dimensions of usability, credibility, appearance and loyalty. We've began including the 13 items from the SUPR-Q in many true intent  surveys to measure both who users are and what they are doing, as well as what they think about the website. Having these three dimensions allows us to slice the data and understand how differing profiles are being met or neglected.  With a reoccurring true intent study including the SUPR-Q, we can also assess the impact of design changes over time.

Key Drivers

Including a handful of questions around features or content allows you to conduct a key-driver analysis on the data to understand what features are having the biggest impact on usability or appearance for example. 

Don't cram too many questions into true intent surveys, as you'll have a significant drop-off in response rates (remember, you're essentially interrupting your users). If you find your true intent surveys are longer than five minutes, consider randomly assigning users to only some questions. The nice thing about the software is that once you have the line of javascript included, you can add, modify and remove questions at any time.

Recruiting for Usability Tests

Finally, true intent surveys are an excellent way to recruit participants for usability studies. These participants are clearly qualified to provide more focused feedback on tasks or features.  We've recruited website visitors to participate immediately or at a future date in tree-tests, card-sort studies, or 10-20 minute unmoderated usability studies.  


About Jeff Sauro

Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 4 books on statistics and the user-experience.
More about Jeff...


Learn More



You Might Also Be Interested In:

Related Topics

Survey
.

Posted Comments

There are 1 Comments

August 6, 2014 | Ranjani wrote:

Awesome information Jeff, am going to be a regular reader of your articles. This was very insightful.  


Post a Comment

Comment:


Your Name:


Your Email Address:


.

To prevent comment spam, please answer the following :
What is 2 + 4: (enter the number)

Newsletter Sign Up

Receive bi-weekly updates.
[4275 Subscribers]

Connect With Us

.

Jeff's Books

Quantifying the User Experience: Practical Statistics for User ResearchQuantifying the User Experience: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.