Amazon's Mechanical Turk (MTurk) has become a popular platform for running online behavioral and cognitive experiments in recent years. However, understanding the cost structure of this service can be a somewhat complex task, and often, researchers are not fully aware of the potential hidden costs that can inflate their budgets. In this article, we will go over the basics of MTurk's license model, give an example cost of an experiment with 100 participants, delve into potential hidden costs, and provide some tips on how to reduce costs.
MTurk utilizes a pay-as-you-go model, meaning that you only pay for the tasks (referred to as Human Intelligence Tasks or HITs) that are completed. The costs are calculated on a per-task basis and may vary depending on the complexity and duration of the task. On top of this, Amazon charges a typically 20% fee for each HIT.
Suppose we plan to run an online cognitive experiment via MTurk, with a participant pool of 100. If we decide to pay each participant $5 for their time and effort in completing this test, the total payment to the participants would be $500.
The fees that MTurk would charge on top of this equate to 20% of the total compensation to participants. In this case, that would be an extra $100 (20% of $500), bringing the total cost of the experiment to $600.
Additional fees apply if we're to require more than ten participants to complete the same HIT. Amazon charges an additional 20% for such cases. Furthermore, for projects that involve qualitative research methods such as interviews or open-ended questions, additional costs for data analysis software, transcriptions, and codings might add up.
Strategic considerations such as conducting shorter surveys or performing HITs in batches can also result in cost-savings. Remember, it is not just about reducing costs but also about ensuring you get reliable and quality data for your psychological research.
We hope this guide to understanding MTurk's costing provides valuable insights to scientific researchers embarking on conducting remote studies. Happy experimenting!