Understanding the Costs of MTurk: A Detailed Breakdown
Amazon's Mechanical Turk (MTurk) has become a popular platform for running online behavioral and cognitive experiments in recent years. However, understanding the cost structure of this service can be a somewhat complex task, and often, researchers are not fully aware of the potential hidden costs that can inflate their budgets. In this article, we will go over the basics of MTurk's license model, give an example cost of an experiment with 100 participants, delve into potential hidden costs, and provide some tips on how to reduce costs.
MTurk License Model
MTurk utilizes a pay-as-you-go model, meaning that you only pay for the tasks (referred to as Human Intelligence Tasks or HITs) that are completed. The costs are calculated on a per-task basis and may vary depending on the complexity and duration of the task. On top of this, Amazon charges a typically 20% fee for each HIT.
Example Cost of an Experiment with 100 Participants
Suppose we plan to run an online cognitive experiment via MTurk, with a participant pool of 100. If we decide to pay each participant $5 for their time and effort in completing this test, the total payment to the participants would be $500.
The fees that MTurk would charge on top of this equate to 20% of the total compensation to participants. In this case, that would be an extra $100 (20% of $500), bringing the total cost of the experiment to $600.
Hidden Costs
Additional fees apply if we're to require more than ten participants to complete the same HIT. Amazon charges an additional 20% for such cases. Furthermore, for projects that involve qualitative research methods such as interviews or open-ended questions, additional costs for data analysis software, transcriptions, and codings might add up.
Tips to Reduce Cost
Designing your study to maximize the benefits while minimizing the costs requires careful planning. Running pilot experiments can help fine tune your experiment design and verify that tasks are understandable and can be completed in a reasonable amount of time. Tools like jsPsych, which is a JavaScript library for creating behavioral experiments in a web browser, and platforms like www.cognition.run, which come with an array of cognitive tasks, can help simplify this process and hold the cost down by offering seamless remote study solutions.
Strategic considerations such as conducting shorter surveys or performing HITs in batches can also result in cost-savings. Remember, it is not just about reducing costs but also about ensuring you get reliable and quality data for your psychological research.
We hope this guide to understanding MTurk's costing provides valuable insights to scientific researchers embarking on conducting remote studies. Happy experimenting!