The Faces in the Crowdsourcing

A Discussion of Amazon's Mechanical Turk

Jeff Sauro • May 26, 2010

Who would toil away for pennies, completing trivial tasks and answering surveys? A quarter-million users registered with Amazon's Mechanical Turk service.  Just who exactly are the people behind the Mechanical Turk and can we rely on the data we get from the nameless faceless masses ?  In this blog posted on Oracle.com I review two recent studies published and presented at CHI 2010. In one study we see a shift in the demographics of the crowd from stay-at-home moms in the US to an increasingly growing younger male worker from India. 

Are people really taking a task seriously when they only get paid a dime for their time? Another study set out to find out what percent of the crowd was in it just for the money and not taking the tasks seriously or just rushing through to get paid.  This study found most (88%) completed a simple task that required only a modicum of attention (presumably the 12% were rushing through or making careless errors). Sixty-one percent conscientiously answered both an easy and more involved question, suggesting the majority (but far from all) of this global workforce are providing an honest effort. Whether this effort will provide representative and reliable results for all studies is a subject of future crowdsouricng research.  

Rate this Blog

Avg. Rating 5 (2)

Poor										Excellent
0	1	2	3	4	5	6	7	8	9	10

.

Posted Comments

Post a Comment

Comment:

Your Name:

Your Email Address:

.

To prevent comment spam, please answer the following :
What is 5 + 1: (enter the number)