Measuring Usability
Quantitative Usability, Statistics & Six Sigma by Jeff Sauro

Are men overconfident users?

Jeff Sauro • June 12, 2012

There are some interesting known differences between men and women in the psychological literature.

For example, women tend to be better judges of emotion when looking at faces for just 0.2 of a second[pdf]!

And across many measures of ability, while both men and women tend to exhibit overconfidence, men are generally more overconfident than women  and this is especially the case when men do things incorrectly.

But does this also apply to the user experience on websites?

Measuring Confidence

As part of our usability tests we ask users how confident they are they completed the task successfully on a 7 point scale.  When users rate that they are extremely confident (a 7) yet fail the task it's a disaster. Not surprisingly, you want to minimize disasters in the user experience. We wanted to know though, are men overconfident in their assessment of task success?

To find out I aggregated data across five recent task-based website usability evaluations where we collected confidence scores.  In total we had 38 tasks and 881 total participants: 55% Women and 45% men.  Across all studies we had 5246 task attempts to analyze.

Overall, men and women completed roughly the same proportion of tasks successfully: Women completed 63% and Men Completed 62% of task attempts successfully (no statistical difference).

There also wasn't a significant difference in the average confidence ratings between men and women. On a 7 point scale where 7 is extremely confident, men on average rated their confidence in completing the task as 5.47 and women at 5.45.

However, things got interesting when we looked at failed task attempts. When tasks attempts weren't successful, men were more confident than women that they actually completed the task successfully!

Men's average confidence was 6% higher than women's (p=.02) 4.3 (sd=2.2) vs. 4.1 (sd=2.3).


Figure 1: When users fail a task, men are about 6% more confident than women that they completed the task successfully. Error bars are 90% confidence intervals.

In other words, while both sexes were overconfident in the face of task failure, men were even more overconfident.

So in addition to men insisting they know where they're going even though they're lost, you can add overconfidence in completing tasks online to the list.

It turns out that this behavior has been noted for some time now.  In 1977 Lenney found that gender differences are most pronounced when there is a lack of clear and unambiguous feedback. When feedback is "unequivocal and immediately available, women do not make lower ability estimates than men.   However, when such feedback is absent or ambiguous, women seem to have lower opinions of their abilities and often do underestimate relative to men." 

In usability tests, there's often little obvious feedback that the user did the task incorrectly. This would likely explain part of why both genders have an average confidence score above the nominal midpoint on the confidence scale (above 4).  This data suggests that in the face of this uncertainty, men are more likely than women to assume they did the task correctly. 

So it appears this male overconfidence not only manifests itself in higher confidence for things like getting higher returns in the stock market[pdf] but also for the more modest but ubiquitous experience of attempting tasks on websites.



About Jeff Sauro

Jeff Sauro is the founding principal of Measuring Usability LLC, a company providing statistics and usability consulting to Fortune 1000 companies.
He is the author of over 20 journal articles and 4 books on statistics and the user-experience.
More about Jeff...


Learn More


UX Bootcamp: Aug 20th-22nd in Denver, CO
Best Practices for Remote Usability Testing
The Science of Great Site Navigation: Online Card Sorting + Tree Testing Live Webinar


You Might Also Be Interested In:

Related Topics

Confidence, UI Disasters, Usability
.

Posted Comments

There are 2 Comments

April 4, 2013 | Jim wrote:

Im so confident that you didnt convince me one bit 


June 17, 2012 | Joe wrote:

You failed to convince me, but you thought you were confident you would, eh? 


Post a Comment

Comment:


Your Name:


Your Email Address:


.

To prevent comment spam, please answer the following :
What is 3 + 2: (enter the number)

Newsletter Sign Up

Receive bi-weekly updates.
[4079 Subscribers]

Connect With Us

UX Bootcamp

Denver CO, Aug 20-22nd 2014

3 Days of Hands-On Training on User Experience Methods, Metrics and Analysis.Learn More

Our Supporters

Use Card Sorting to improve your IA

Loop11 Online Usabilty Testing

Userzoom: Unmoderated Usability Testing, Tools and Analysis

Usertesting.com

.

Jeff's Books

Quantifying the User Experience: Practical Statistics for User ResearchQuantifying the User Experience: Practical Statistics for User Research

The most comprehensive statistical resource for UX Professionals

Buy on Amazon

Excel & R Companion to Quantifying the User ExperienceExcel & R Companion to Quantifying the User Experience

Detailed Steps to Solve over 100 Examples and Exercises in the Excel Calculator and R

Buy on Amazon | Download

A Practical Guide to the System Usability ScaleA Practical Guide to the System Usability Scale

Background, Benchmarks & Best Practices for the most popular usability questionnaire

Buy on Amazon | Download

A Practical Guide to Measuring UsabilityA Practical Guide to Measuring Usability

72 Answers to the Most Common Questions about Quantifying the Usability of Websites and Software

Buy on Amazon | Download

.
.
.