Amazon Mechanical Turk ( Mturk )
description
Transcript of Amazon Mechanical Turk ( Mturk )
![Page 1: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/1.jpg)
Amazon Mechanical Turk (Mturk)
• What is MTurk?– Crowdsourcing Internet marketplace that utilizes
human intelligence to perform tasks that computers are unable to do
– Mturk samples are…• Fast • Cheap• Reliable• Diverse
![Page 3: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/3.jpg)
First Things First…• Sign in with existing Amazon.com account or
create one
![Page 4: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/4.jpg)
Getting StartedYou are called a requester
![Page 5: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/5.jpg)
Design
• The space that hosts your Qualtrics survey link is called a HIT
• Your HIT is published to the Mturk site for workers to see
• The first step is to design the layout of your HIT
![Page 6: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/6.jpg)
![Page 7: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/7.jpg)
Properties
The title and description are seen by workers
Keywords categorize your HIT and make it searchable
![Page 8: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/8.jpg)
PropertiesSet time restrictions
Chose Mturk Masters option
Set qualifications
Choose qualification requirement
![Page 9: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/9.jpg)
Properties
Choose payment amount per HIT
Choose number of participants
The next step will be to design the layout of the HIT
![Page 10: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/10.jpg)
Layout
• Uses html code • Give instructions and brief description of study
to workers • Provide survey link
![Page 11: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/11.jpg)
Layout
Template name should identify your study. This is not displayed to workers.
Keep any of this you want and delete the rest.
![Page 12: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/12.jpg)
Layout
Code word and study instructions
Qualtrics link
Confirmation code
User comments
![Page 13: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/13.jpg)
Layout• Use a code word to prevent workers from repeating similar surveys• Lets say you create a survey about prejudice and use code word
ALPHA for the HIT• Inform workers before they begin that the HIT uses code word
ALPHA• After data collection you would like to re-run the study after making
some changes on Qualtrics• You don’t want workers who completed the first round to do the
second round also• Again, inform workers that the HIT uses code word ALPHA and that if
they had taken an ALPHA HIT before, they should not continue
![Page 14: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/14.jpg)
Layout
• Other ways of controlling for repeats– In Qualtrics, create an item at the beginning of the survey
that asks the participant if they have completed a HIT from this requester before• If no, okay• If yes, ask if the HIT they’ve done uses undesired code word (in
this case, its ALPHA)– If yes, send participant to end of survey message– If no, okay
– Ask at end of survey if participant had completed a similar survey in the past• Yes, no, I don’t remember
![Page 15: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/15.jpg)
Layout
Confirmation code
![Page 16: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/16.jpg)
Layout
• Confirmation Code– Create a space for workers to enter the
confirmation code you provide at the end of Qualtrics survey
– Code can be made up– Used to check that workers completed entire
survey
![Page 17: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/17.jpg)
Layout
![Page 18: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/18.jpg)
Layout
SpaceLink
Space
![Page 19: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/19.jpg)
Layout
SpaceInstructions
Text box
![Page 20: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/20.jpg)
Layout
SpaceInstructions
Text box
![Page 21: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/21.jpg)
Layout – html code<p><a href="http://kuclas.qualtrics.com/SE/?SID=SV_5pfmnKdhILnpOfi" target="_blank">Click here to take the survey.</a></p> !survey link<p> </p> !space<p>At the end of the survey, you will be provided with a confirmation code. Please enter the code below to verify that you have completed the task:</p><p><textarea rows=“1" cols="80" name=“Code"></textarea></p> !code box<p> </p> !space<p>Please provide any comments you may have below, we appreciate your input!</p><p><textarea rows="3" cols="80" name="comment"></textarea></p> !comment box
![Page 22: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/22.jpg)
Preview and Finish
![Page 23: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/23.jpg)
Publish
![Page 24: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/24.jpg)
Publish
Publish HIT
Name the batch
Negative balance
![Page 25: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/25.jpg)
Publish
• Note: – If your Amazon account is funded, the HIT will be
published right away– If not, you will be notified of this and be able to
click a link which will allow you to fund your account using a variety of methods
![Page 26: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/26.jpg)
Manage
• Watch completion rate• Accept or reject work• Cancel batch
![Page 27: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/27.jpg)
Manage Click here to accept work (pay participants)
![Page 28: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/28.jpg)
ManageConfirmation code
Marking worker and clicking approve will initiate payment
Clicking worker ID allows you to block worker and pay bonuses
Approved 1 HIT out of 1 for this worker
![Page 29: Amazon Mechanical Turk ( Mturk )](https://reader033.fdocuments.net/reader033/viewer/2022061500/56815f85550346895dce8ad3/html5/thumbnails/29.jpg)
MTurk – Things to consider
• Blocking workers is BAD• Average pay anywhere from $.10 to $1.00• Command line tools• Professionalism– Turkernation.com
• SPSP Seminar on MTurk