Hostname: page-component-6587cd75c8-rlsgg Total loading time: 0 Render date: 2025-04-24T05:56:24.888Z Has data issue: false hasContentIssue false
  • English
  • Français

Turking overtime: how participant characteristics and behavior vary over time and day on Amazon Mechanical Turk

Published online by Cambridge University Press:  17 January 2025

Antonio A. Arechar Open the ORCID record for Antonio A. Arechar [Opens in a new window] ,
Gordon T. Kraft-Todd  and
David G. Rand
Antonio A. Arechar*
Affiliation:
Department of Psychology, Yale University, 2 Hillhouse Ave., New Haven, CT 06511, USA
Gordon T. Kraft-Todd
Affiliation:
Department of Psychology, Yale University, 2 Hillhouse Ave., New Haven, CT 06511, USA
David G. Rand*
Affiliation:
Department of Psychology, Yale University, 2 Hillhouse Ave., New Haven, CT 06511, USA Department of Economics, Yale University, 28 Hillhouse Ave., New Haven, CT 06511, USA School of Management, Yale University, 165 Whitney Ave., New Haven, CT 06511, USA
*
e-mail: [email protected]
e-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Online experiments allow researchers to collect datasets at times not typical of laboratory studies. We recruit 2336 participants from Amazon Mechanical Turk to examine if participant characteristics and behaviors differ depending on whether the experiment is conducted during the day versus night, and on weekdays versus weekends. Participants make incentivized decisions involving prosociality, punishment, and discounting, and complete a demographic and personality survey. We find no time or day differences in behavior, but do find that participants at nights and on weekends are less experienced with online studies; on weekends are less reflective; and at night are less conscientious and more neurotic. These results are largely robust to finer-grained measures of time and day. We also find that those who participated earlier in the course of the study are more experienced, reflective, and agreeable, but less charitable than later participants.

Keywords

CooperationHonestyDecision-makingTime of dayMTurkSelf-control

JEL classification

C80: General C90: General
Type
Original Paper
Information
Journal of the Economic Science Association , Volume 3 , Issue 1 , July 2017 , pp. 1 - 11
Copyright
Copyright © Economic Science Association 2017

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Electronic supplementary material The online version of this article (doi:10.1007/s40881-017-0035-0) contains supplementary material, which is available to authorized users.

References

Amir, O., Rand, D. G., Gal, Y. K. (2012). Economic games on the Internet: The effect of $1 stakes. PLoS ONE, 7(2),e31461. 10.1371/journal.pone.0031461CrossRefGoogle ScholarPubMed
Arechar, A. A., Molleman, L., Gachter, S. (2017). Conducting interactive experiments online. Experimental Economics.Google Scholar
Aviv, A. L., Zelenski, J. M., Rallo, L., Larsen, R. J. (2002). Who comes when: Personality differences in early and later participation in a university subject pool. Personality and Individual Differences, 33(3), 487–496. 10.1016/S0191-8869(01)00199-4CrossRefGoogle Scholar
Berinsky, A. J., Huber, G. A., Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s mechanical turk. Political Analysis, 20(3), 351–368. 10.1093/pan/mpr057CrossRefGoogle Scholar
Capraro, V., Jordan, J. J., Rand, D. G. (2014). Heuristics guide the implementation of social preferences in one-shot Prisoner’s Dilemma experiments. Scientific Reports, 4, 6790. 10.1038/srep06790CrossRefGoogle ScholarPubMed
Casey, L. S., Chandler, J., Levine, A. S., Proctor, A., & Strolovitch, D. Z. (2016). Intertemporal differences among MTurk worker demographics. https://osf.io/preprints/psyarxiv/8352x.Google Scholar
Chandler, J., Paolacci, G., Peer, E., Mueller, P., Ratliff, K. A. (2015). Using nonnaive participants can reduce effect sizes. Psychological Science, 26(7), 1131–1139. 10.1177/0956797615585115CrossRefGoogle ScholarPubMed
Deetlefs, J., Chylinski, M., & Ortmann, A. (2015). MTurk ‘Unscrubbed’: Exploring the good, the ‘super’, and the unreliable on Amazon’s mechanical turk. http://ssrn.com/abstract=2654056.Google Scholar
Frederick, S. (2005). Cognitive reflection and decision making. The Journal of Economic Perspectives, 19(4), 25–42. 10.1257/089533005775196732CrossRefGoogle Scholar
Gosling, S. D., Rentfrow, P. J., Swann, W. B. Jr. (2003). A very brief measure of the Big-Five personality domains. Journal of Research in Personality, 37(6), 504–528. 10.1016/S0092-6566(03)00046-1CrossRefGoogle Scholar
Gunia, B. C., Barnes, C. M., Sah, S. (2014). The morality of larks and owls: Unethical behavior depends on chronotype as well as time of day. Psychological Science, 25(12), 2272–2274. 10.1177/0956797614541989CrossRefGoogle ScholarPubMed
Horton, J. J., Rand, D. G., Zeckhauser, R. J. (2011). The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14(3), 399–425. 10.1007/s10683-011-9273-9CrossRefGoogle Scholar
Kirby, K. N., Petry, N. M., Bickel, W. K. (1999). Heroin addicts have higher discount rates for delayed rewards than non-drug-using controls. Journal of Experimental Psychology-General, 128(1), 78–87. 10.1037/0096-3445.128.1.78CrossRefGoogle ScholarPubMed
Kouchaki, M., Smith, I. H. (2014). The morning morality effect: The influence of time of day on unethical behavior. Psychological Science, 25(1), 95–102. 10.1177/0956797613498099CrossRefGoogle ScholarPubMed
Paolacci, G., Chandler, J., Ipeirotis, P. G. (2010). Running experiments on amazon mechanical turk. Judgment and Decision Making, 5(5), 411–419.CrossRefGoogle Scholar
Rand, D. G. (2012). The promise of mechanical turk: How online labor markets can help theorists run behavioral experiments. Journal of Theoretical Biology, 299, 172–179. 10.1016/j.jtbi.2011.03.004CrossRefGoogle ScholarPubMed
Rand, D. G., Peysakhovich, A., Kraft-Todd, G. T., Newman, G. E., Wurzbacher, O., Nowak, M. A. et al., (2014 ). Social heuristics shape intuitive cooperation. Nature Communications, 5, 3677. 10.1038/ncomms4677CrossRefGoogle ScholarPubMed
Shenhav, A., Rand, D. G., Greene, J. D. (2012). Divine intuition: Cognitive style influences belief in God. Journal of Experimental Psychology: General, 141(3), 423–428. 10.1037/a0025391CrossRefGoogle ScholarPubMed
Stagnaro, M. N., Arechar, A. A., Rand, D. G. (2017). From good institutions to generous citizens: Top-down incentives to cooperate promote subsequent prosociality but not norm enforcement. Cognition.CrossRefGoogle Scholar
Peysakhovich, A., Nowak, M. A., Rand, D. G. (2014). Humans display a ‘cooperative phenotype’ that is domain general and temporally stable. Nature communications, 5, 4939. 10.1038/ncomms5939CrossRefGoogle ScholarPubMed
Supplementary material: File

Arechar et al. supplementary material

Arechar et al. supplementary material
Download Arechar et al. supplementary material(File)
File 1.5 MB