Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-745bb68f8f-f46jp Total loading time: 0 Render date: 2025-01-26T08:47:38.422Z Has data issue: false hasContentIssue false

Part I - Quantitative Data Collection Sources

Published online by Cambridge University Press:  12 December 2024

Edited by
John E. Edlund  and
Austin Lee Nichols
John E. Edlund
Affiliation:
Rochester Institute of Technology, New York
Austin Lee Nichols
Affiliation:
Central European University, Vienna
Get access
Type
Chapter
Information
The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
Volume 2: Performing Research
 , pp. 1 - 186
Publisher: Cambridge University Press
Print publication year: 2024

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

References

Amaya, A., & Presser, S. (2016). Nonresponse bias for univariate and multivariate estimates of social activities and roles. Public Opinion Quarterly, 81(1), 1–36.Google Scholar
Aggarwal, R., & Ranganathan, P. (2019). Study designs: Part 2 – descriptive studies. Perspectives in Clinical Research, 10(1), 34–36.CrossRefGoogle ScholarPubMed
Ashraf, R., & Merunka, D. (2017). The use and misuse of student samples: An empirical investigation of European marketing research. Journal of Consumer Behaviour, 16(4), 295–308.CrossRefGoogle Scholar
Babbie, E. (1992). The Practice of Social Research, 6th ed. Wadsworth.Google Scholar
Baláž, V., Bačová, V., Drobná, E., Dudeková, K., & Adamík, K. (2013). Testing prospect theory parameters. Ekonomicky časopis, 61, 655–671.Google Scholar
Banyard, P., & Hunt, N. (2000). Reporting research: Something missing? The Psychologist: Bulletin of the British Psychological Society, 13(2), 68–71.Google Scholar
Barr, S. H., & Hitt, M. A. (1986). A comparison of selection decision models in manager versus student samples. Personnel Psychology, 39(3), 599–617.CrossRefGoogle Scholar
Basil, M. D. (1996). The use of student samples in communication research. Journal of Broadcasting and Electronic Media, 40, 431–440.CrossRefGoogle Scholar
Basil, M. D., Brown, W. J., & Bocarnea, M. C. (2002). Differences in univariate values versus multivariate relationships: Findings from a study of Diana, Princess of Wales. Human Communication Research, 28, 501–514.Google Scholar
Beebe, L. H. (2007). What can we learn from pilot studies? Perspectives in Psychiatric Care, 43(4), 213–218.CrossRefGoogle ScholarPubMed
Beets, M. W., Weaver, R. G., Ioannidis, J., Geraci, M., Brazendale, K., Decker, L., & Milat, A. J. (2020). Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: A systematic review and meta-analysis. International Journal of Behavioral Nutrition and Physical Activity, 17(1), 1–20.CrossRefGoogle ScholarPubMed
Bello, D., Leung, K., Radebaugh, L., Tung, R. L., & Van Witteloostuijn, A. (2009). From the editors: Student samples in international business research. Journal of International Business Studies, 40(3), 361–364.CrossRefGoogle Scholar
Berkowitz, L., & Donnerstein, E. (1982). External validity is more than skin deep: Some answers to criticisms of laboratory experiments. American Psychologist, 37(3), 245.CrossRefGoogle Scholar
Bowen, G. L. (1994). Estimating the reduction in nonresponse bias from using a mail survey as a backup for nonrespondents to a telephone interview survey. Research on Social Work Practice, 4, 115–128.CrossRefGoogle Scholar
Burnett, J. J., & Dune, P. M. (1986). An appraisal of the use of student subjects in marketing research. Journal of Business Research, 14(4), 329–343.CrossRefGoogle Scholar
Cappelen, A. W., Nygaard, K., Sørensen, E. Ø., & Tungodden, B. (2015). Social preferences in the lab: A comparison of students and a representative population. Scandinavian Journal of Economics, 117(4), 1306–1326.CrossRefGoogle Scholar
Casadevall, A., & Fang, F. C. (2008). Descriptive science. Infection and Immunity, 76(9), 3835–3836.CrossRefGoogle ScholarPubMed
Caspaldi, D., & Patterson, G. R. (1987). An approach to the problem of recruitment and retention rates for longitudinal research. Behavioral Assessment, 9, 169–177.Google Scholar
Clara, I. P., Cox, B. J., Enns, M. W., Murray, L. T., & Torgrudc, L. J. (2003). Confirmatory factor analysis of the multidimensional scale of perceived social support in clinically distressed and student samples. Journal of Personality Assessment, 81(3), 265–270.CrossRefGoogle ScholarPubMed
Coleman, S. (2007). Testing theories with qualitative and quantitative predictions. European Political Science, 6(2), 124–133.CrossRefGoogle Scholar
Compeau, D., Marcolin, B., Kelley, H., & Higgins, C. (2012). Research commentary – Generalizability of information systems research using student subjects – A reflection on our practices and recommendations for future research. Information Systems Research, 23(4), 1093–1109.CrossRefGoogle Scholar
Crabbe, B. D., & Pinkerton, K. A. (1992). Sources of bias in Health Commission and tobacco industry surveys in Australia. Australian Psychologist, 27, 103–108.CrossRefGoogle Scholar
Crandall, C. S., & Sherman, J. W. (2016). On the scientific superiority of conceptual replications for scientific progress. Journal of Experimental Social Psychology, 66, 93–99.CrossRefGoogle Scholar
Deffner, D., Rohrer, J. M., & McElreath, R. (2022). A causal framework for cross-cultural generalizability. Advances in Methods and Practices in Psychological Science, 5(3), 25152459221106366.CrossRefGoogle Scholar
Dura, J. R., & Kiecolt-Glaser, J. K. (1990). Sample bias in caregiving research. Journals of Gerontology, 45, P200–P204.CrossRefGoogle ScholarPubMed
Edgington, E. S. (1966). Statistical inference and nonrandom samples. Psychological Bulletin, 66(6), 485–487.CrossRefGoogle ScholarPubMed
Edlund, M. J., & Swann, A. C. (1989). Continuing in treatment as a form of selection bias. American Journal of Psychiatry, 146, 254–256.Google ScholarPubMed
Efron, B., & Tibshirani, R. (1991). Statistical data analysis in the computer age. Science, 253(5018), 390–395.CrossRefGoogle ScholarPubMed
Espinosa, J. A., & Ortinau, D. J. (2016). Debunking legendary beliefs about student samples in marketing research. Journal of Business Research, 69(8), 3149–3158.CrossRefGoogle Scholar
Falk, A., Meier, S., & Zehnder, C. (2013). Do lab experiments misrepresent social preferences? The case of self-selected student samples. Journal of the European Economic Association, 11(4), 839–852.CrossRefGoogle Scholar
Ferber, R. (1977). Research by convenience. Journal of Consumer Research, 4(1), 57–58.CrossRefGoogle Scholar
Fienberg, S. E., & Tanur, J. M. (1996). Reconsidering the fundamental contributions of Fisher and Neyman on experimentation and sampling. International Statistical Review/Revue Internationale de Statistique, 64(3), 237–253.Google Scholar
Frame, C. L., & Strauss, C. C. (1987). Parental informed consent and sample bias in grade-school children. Journal of Social and Clinical Psychology, 5, 227–236.CrossRefGoogle Scholar
Gold, R. L. (1997). The ethnographic method in sociology. Qualitative Inquiry, 3(4), 388–402.CrossRefGoogle Scholar
Goyal, R. (2015). Animal testing in the history of anesthesia: Now and then, some stories, some facts. Journal of Anaesthesiology, Clinical Pharmacology, 31(2), 149–151.CrossRefGoogle ScholarPubMed
Greenberg, J. (1987). The college sophomore as guinea pig: Setting the record straight. Academy of Management Review, 12(1), 157–159.CrossRefGoogle Scholar
Guttman, I. (1973). Care and handling of univariate or multivariate outliers in detecting spuriosity – a Bayesian approach. Technometrics, 15(4), 723–738.Google Scholar
Hallingberg, B., Turley, R., Segrott, J., et al. (2018). Exploratory studies to decide whether and how to proceed with full-scale evaluations of public health interventions: A systematic review of guidance. Pilot and feasibility studies, 4(1), 1–12.CrossRefGoogle Scholar
Hanel, P. H., & Vione, K. C. (2016). Do student samples provide an accurate estimate of the general public? PLOS ONE, 11(12), e0168354.CrossRefGoogle ScholarPubMed
Harrison, D. A. (1995). Volunteer motivation and attendance decisions: Competitive theory testing in multiple samples from a homeless shelter. Journal of Applied Psychology, 80(3), 371–385.CrossRefGoogle Scholar
Heggestad, E. D., Rogelberg, S., Goh, A., & Oswald, F. L. (2015). Considering the effects of nonresponse on correlations between surveyed variables. Journal of Personnel Psychology, 14(2), 91–103.CrossRefGoogle Scholar
Henry, P. J. (2008). Student sampling as a theoretical problem. Psychological Inquiry, 19(2), 114–126CrossRefGoogle Scholar
Honigmann, J. J. (2003). Sampling in ethnographic fieldwork. In Burgess, R. G. (ed.), Field Research: A Sourcebook and Field Manual (pp. 134–152). Routledge.Google Scholar
Jackson, S. (1992). Message Effects Research: Principles of Design and Analysis. Guilford Press.Google Scholar
Jackson, S., & Jacobs, S. (1983). Generalizing about messages: Suggestions for design and analysis of experiments. Human Communication Research, 9(2), 169–191.CrossRefGoogle Scholar
James, W. L., & Sonner, B. S. (2001). Just say no to traditional student samples. Journal of Advertising Research, 41(5), 63–71.Google Scholar
Jennings, W., & Wlezien, C. (2018). Election polling errors across time and space. Nature Human Behaviour, 2(4), 276–283.CrossRefGoogle ScholarPubMed
Kahneman, D., Slovic, R., & Tversky, A. (1982). Judgment under Uncertainty: Heuristics and Biases. Cambridge University Press.CrossRefGoogle Scholar
Kardes, F. R. (1996). In defense of experimental consumer psychology. Journal of Consumer Psychology, 5, 279–296.CrossRefGoogle Scholar
Kim, H., Schimmack, U., Oishi, S., & Tsutsui, Y. (2018). Extraversion and life satisfaction: A cross‐cultural examination of student and nationally representative samples. Journal of Personality, 86(4), 604–618.CrossRefGoogle ScholarPubMed
Kish, L. (1957). Confidence intervals for clustered samples. American Sociological Review, 22(2), 154–165.CrossRefGoogle Scholar
Krackardt, D. (1987). QAP partialling as a test of spuriousness. Social Networks, 9(2), 171–186.CrossRefGoogle Scholar
Krupnikov, Y., Nam, H. H., Style, H., Druckman, J. N., & Green, D. P. (2021). Convenience samples in political science experiments. In Druckman, J. and Green, D. (eds.), Advances in Experimental Political Science (pp. 165–183). Cambridge University Press.CrossRefGoogle Scholar
Kruskal, W., & Mosteller, F. (1980). Representative sampling, IV: The history of the concept in statistics, 1895–1939. International Statistical Review/Revue Internationale de Statistique, 48(2), pp. 169–195.Google Scholar
Lamb, C. W. Jr., & Stem, D. E. Jr. (1980). An evaluation of students as surrogates in marketing studies. Advances in Consumer Research, 7(1), 796–799.Google Scholar
Lesser, V. M., & Kalsbeek, W. D. (1999). Nonsampling errors in environmental surveys. Journal of Agricultural, Biological, and Environmental Statistics, 4(4), 473–488.CrossRefGoogle Scholar
Lucas, J. W. (2003). Theory-testing, generalization, and the problem of external validity. Sociological Theory, 21, 236–253.CrossRefGoogle Scholar
Lynch, D. L., Stern, A. E., Oates, R. K., & O’Toole, B. I. (1993). Who participates in child sexual abuse research? Journal of Child Psychology and Psychiatry and Allied Disciplines, 34, 935–944.CrossRefGoogle ScholarPubMed
Lynch, J. G. (1982). The role of external validity in theoretical research. Journal of Consumer Research, 10, 109–111.CrossRefGoogle Scholar
McNemar, Q. (1946) Opinion attitude methodology. Psychological Bulletin, 43, 289–374.CrossRefGoogle ScholarPubMed
Meyer, J., Kohn, I., Stahl, K., Hakala, K., Seibert, J., & Cannon, A. J. (2019). Effects of univariate and multivariate bias correction on hydrological impact projections in alpine catchments. Hydrology and Earth System Sciences, 23, 1339–1354.CrossRefGoogle Scholar
Mishra, S. I., Dooley, D., Catalano, R., & Serxner, S. (1993). Telephone health surveys: Potential bias from noncompletion. American Journal of Public Health, 83, 94–99.CrossRefGoogle ScholarPubMed
Monin, B., & Oppenheimer, D. M. (2014). The limits of direct replications and the virtues of stimulus sampling. Social Psychology, 45(4), 299–300.CrossRefGoogle Scholar
Nichols, A. L., & Maner, J. K. (2008). The good-subject effect: Investigating participant demand characteristics. Journal of General Psychology, 135(2), 151–166.CrossRefGoogle ScholarPubMed
Nielsen, M., Haun, D., Kärtner, J., & Legare, C. H. (2017). The persistent sampling bias in developmental psychology: A call to action. Journal of Experimental Child Psychology, 162, 31–38.CrossRefGoogle ScholarPubMed
Norden, K. A., Klein, D. N., Ferro, T., & Kasch, K. (1995). Who participates in a family study? Comprehensive Psychiatry, 36, 199–206.CrossRefGoogle Scholar
Payne, B. K., & Chappell, A. (2008). Using student samples in criminological research. Journal of Criminal Justice Education, 19(2), 175–192.CrossRefGoogle Scholar
Pernice, R. E., Ommundsen, R., Van Der Veer, K., & Larsen, K. (2008). On use of student samples for scale construction. Psychological Reports, 102(2), 459–464.CrossRefGoogle ScholarPubMed
Peterson, R. A. (2001). On the use of college students in social science research: Insights from a second-order meta-analysis. Journal of Consumer Research, 28, 450–461.CrossRefGoogle Scholar
Peterson, R. A., & Merunka, D. R. (2014). Convenience samples of college students and research reproducibility. Journal of Business Research, 67(5), 1035–1041.CrossRefGoogle Scholar
Potter, W. J., Cooper, R., & Dupagne, M. (1993). The three paradigms of mass media research in mainstream communication journals. Communication Theory, 3, 317–355.CrossRefGoogle Scholar
Rindfuss, R. R., Choe, M. K., Tsuya, N. O., Bumpass, L. L., & Tamaki, E. (2015). Do low survey response rates bias results? Evidence from Japan. Demographic Research, 32, 797–828.CrossRefGoogle Scholar
Sears, D. O. (1986). College sophomore in the laboratory: Influences of a narrow data base on social psychology’s view of human nature. Journal of Personality and Social Psychology, 51, 515–530.CrossRefGoogle Scholar
Shapiro, M. A. (2002). Generalizability in communication research. Human Communication Research, 28(4), 491–500.CrossRefGoogle Scholar
Simonson, I., Carmon, Z., Dhar, R., Drolet, A., & Nowlis, S. M. (2001). Consumer research: In search of identity. In Fiske, S. T., Schacter, D. L., & Zahn-Waxler, C. (eds.), Annual Review of Psychology (vol. 52, pp. 249–275). Annual Reviews.Google Scholar
Smith, T. M. F. (1983). On the validity of inferences from non‐random samples. Journal of the Royal Statistical Society: Series A (General), 146(4), 394–403.CrossRefGoogle Scholar
Stephan, F. F. (1948). History of the uses of modern sampling procedures. Journal of the American Statistical Association, 43(241), 12–39.CrossRefGoogle Scholar
Tabachnick, B. G., & Fidell, L. S. (2018). Using Multivariate Statistics. Pearson.Google Scholar
Thomas, R. W. (2011). When student samples make sense in logistics research. Journal of Business Logistics, 32(3), 287–290.CrossRefGoogle Scholar
Traugott, M. (2011). The accuracy of opinion polling and its relation to its future. In Shapiro, R. I. & Jacobs, L. R. (eds.), The Oxford Handbook of American Public Opinion and the Media (pp. 316–331). Oxford University Press.CrossRefGoogle Scholar
Van Teijlingen, E., & Hundley, V. (2010). The importance of pilot studies. Social Research Update, 35(4), 49–59.Google Scholar
Walsch, J. P., Sproull, L. S., & Hesse, B. W. (1992). Self-selected and randomly selected respondents in a computer network. Public Opinion Quarterly, 56, 241–244.CrossRefGoogle Scholar
Wells, W. D. (1993). Discovery-oriented consumer research. Journal of Consumer Research, 19(4), 489–504.CrossRefGoogle Scholar
Wesiner, C., Schmidt, L., & Tam, T. (1995). Assessing bias in community-based prevalence estimates: Towards an unduplicated count of problem drinkers and drug users. Addiction, 90, 391–405.CrossRefGoogle Scholar
Wolfe, B. E. (2013). The value of pilot studies in clinical research: A clinical translation of the research article titled “In search of an adult attachment stress provocation to measure effect on the oxytocin system.” Journal of the American Psychiatric Nurses Association, 19(4), 192–194.CrossRefGoogle ScholarPubMed

References

Aguinis, H., Villamor, I., & Ramani, R. S. (2021). MTurk research: Review and recommendations. Journal of Management, 47(4), 823–837. https://doi.org/10.1177/0149206320969787CrossRefGoogle Scholar
Ahler, D. J., Roush, C. E., & Sood, G. (2021). The micro-task market for lemons: Data quality on Amazon’s Mechanical Turk. Political Science Research and Methods, 1–20. https://doi.org/10.1017/psrm.2021.57CrossRefGoogle Scholar
American National Election Studies. (2020). The ANES Guide to Public Opinion and Electoral Behavior. https://electionstudies.org/data-tools/anes-guide/anes-guide.html?chart=lib_con_identification_7_ptGoogle Scholar
Arechar, A. A., Gächter, S., & Molleman, L. (2018). Conducting interactive experiments online. Experimental Economics, 21(1), 99–131. https://doi.org/10.1007/s10683-017-9527-2CrossRefGoogle ScholarPubMed
Arechar, A. A., Kraft-Todd, G. T., & Rand, D. G. (2017). Turking overtime: How participant characteristics and behavior vary over time and day on Amazon Mechanical Turk. Journal of the Economic Science Association, 3(1), 1–11. https://doi.org/10.1007/s40881-017-0035-0CrossRefGoogle ScholarPubMed
Bai, H. (2018, August 8). Evidence that a large amount of low quality responses on MTurk can be detected with repeated GPS coordinates. MaxHuiBai.com. www.maxhuibai.com/blog/evidence-that-responses-from-repeating-gps-are-randomGoogle Scholar
Barr, J. (2011, June 23). Get better results with Amazon Mechanical Turk Masters. Amazon Web Services. https://aws.amazon.com/blogs/aws/amazon-mechanical-turk-master-workersGoogle Scholar
Benndorf, V., Moellers, C., & Normann, H.-T. (2017). Experienced vs. inexperienced participants in the lab: Do they behave differently? Journal of the Economic Science Association, 3(1), 12–25. https://doi.org/10.1007/s40881-017-0036-zCrossRefGoogle Scholar
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis, 20(3), 351–368. https://doi.org/10.1093/pan/mpr057CrossRefGoogle Scholar
Berry, C., Kees, J., & Burton, S. (2022). Drivers of data quality in advertising research: Differences across MTurk and professional panel samples. Journal of Advertising, 51(4), 1–15. https://doi.org/10.1080/00913367.2022.2079026CrossRefGoogle Scholar
Bohannon, J. (2011). Social science for pennies. Science, 334(6054), 307. https://doi.org/10.1126/science.334.6054.307CrossRefGoogle ScholarPubMed
Boynton, M. H., & Richman, L. S. (2014). An online daily diary study of alcohol use using Amazon’s Mechanical Turk. Drug and Alcohol Review, 33(4), 456–461. https://doi.org/10.1111/dar.12163CrossRefGoogle ScholarPubMed
Buchanan, E. A., & Hvizdak, E. E. (2009). Online survey tools: Ethical and methodological concerns of human research ethics committees. Journal of Empirical Research on Human Research Ethics, 4(2), 37–48. https://doi.org/10.1525/jer.2009.4.2.37CrossRefGoogle ScholarPubMed
Buhrmester, M. D., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5. https://doi.org/10.1177/1745691610393980CrossRefGoogle Scholar
Buhrmester, M. D., Talaifar, S., & Gosling, S. D. (2018). An evaluation of Amazon’s Mechanical Turk, its rapid rise, and its effective use. Perspectives on Psychological Science, 13(2), 149–154. https://doi.org/10.1177/1745691617706516CrossRefGoogle ScholarPubMed
Burleigh, T., Kennedy, R., & Clifford, S. (2018). How to screen out VPS and international respondents using qualtrics: A protocol. SSRN, 3265459. https://papers.ssrn.com/abstract=3265459CrossRefGoogle Scholar
Campbell, D. S., & Reiman, A.-K. (2022). Has social psychology lost touch with reality? Exploring public perceptions of the realism and consequentiality of social psychological research. Journal of Experimental Social Psychology, 98, 104255. https://doi.org/10.1016/j.jesp.2021.104255CrossRefGoogle Scholar
Casey, L. S., Chandler, J., Levine, A. S., Proctor, A., & Strolovitch, D. Z. (2017). Intertemporal differences among MTurk workers: Time-based sample variations and implications for online data collection. SAGE Open, 7(2), 215824401771277. https://doi.org/10.1177/2158244017712774CrossRefGoogle Scholar
Chandler, J. J., & Paolacci, G. (2017). Lie for a dime: When most prescreening responses are honest but most study participants are impostors. Social Psychological and Personality Science, 8(5), 500–508. https://doi.org/10.1177/1948550617698203CrossRefGoogle Scholar
Chandler, J. J., Paolacci, G., Peer, E., Mueller, P., & Ratliff, K. A. (2015). Using nonnaive participants can reduce effect sizes. Psychological Science, 26(7), 1131–1139. https://doi.org/10.1177/0956797615585115CrossRefGoogle ScholarPubMed
Chandler, J. J., Rosenzweig, C., Moss, A. J., Robinson, J., & Litman, L. (2019). Online panels in social science research: Expanding sampling methods beyond Mechanical Turk. Behavior Research Methods, 51(5), 2022–2038. https://doi.org/10.3758/s13428-019-01273-7CrossRefGoogle ScholarPubMed
Chandler, J., Sisso, I., & Shapiro, D. (2020). Participant carelessness and fraud: Consequences for clinical research and potential solutions. Journal of Abnormal Psychology, 129(1), 49–55. https://doi.org/10.1037/abn0000479CrossRefGoogle ScholarPubMed
Chmielewski, M., & Kucker, S. C. (2020). An MTurk crisis? Shifts in data quality and the impact on study results. Social Psychological and Personality Science, 11(4), 464–473. https://doi.org/10.1177/1948550619875149CrossRefGoogle Scholar
Clifford, S., & Jerit, J. (2014). Is there a cost to convenience? An experimental comparison of data quality in laboratory and online studies. Journal of Experimental Political Science, 1(2), 120–131. https://doi.org/10.1017/xps.2014.5CrossRefGoogle Scholar
Converse, B. A., & Epley, N. (2007). With God on our side. TESS: Time-Sharing Experiments for the Social Sciences. www.tessexperiments.org/study/converse561Google Scholar
Coppock, A. (2019). Generalizing from survey experiments conducted on Mechanical Turk: A replication approach. Political Science Research and Methods, 7(3), 613–628. https://doi.org/10.1017/psrm.2018.10CrossRefGoogle Scholar
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLOS ONE, 8(3), e57410. https://doi.org/10.1371/journal.pone.0057410CrossRefGoogle Scholar
Dennis, J. M. (2001). Are internet panels creating professional respondents? Marketing Research, 13(2), 34–38.Google Scholar
Dennis, S. A., Goodson, B. M., & Pearson, C. (2020). Online worker fraud and evolving threats to the integrity of MTurk data: A discussion of virtual private servers and the limitations of IP-based screening procedures. Behavioral Research in Accounting, 32(1), 119–134.CrossRefGoogle Scholar
Difallah, D., Filatova, E., & Ipeirotis, P. (2018). Demographics and dynamics of Mechanical Turk workers. In Chang, Y. (ed.) Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining (pp. 135–143). ACM. https://doi.org/10.1145/3159652.3159661CrossRefGoogle Scholar
Downs, J. S., Holbrook, M. B., Sheng, S., & Cranor, L. F. (2010, April). Are your participants gaming the system? Screening Mechanical Turk workers. In Kaye, J. (ed.), Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (pp. 2399–2402). ACM.Google Scholar
Edlund, J. E., Lange, K. M., Sevene, A. M., Umansky, J., Beck, C. D., & Bell, D. J. (2017). Participant crosstalk: Issues when using the Mechanical Turk. Tutorials in Quantitative Methods for Psychology, 13(3), 174–182. http://dx.doi.org/10.20982/tqmp.13.3.p174CrossRefGoogle Scholar
Edlund, J. E., Nichols, A. L., Okdie, B. M., Guadagno, R. E., Eno, C. A., Heider, J. D., et al. (2014). The prevalence and prevention of crosstalk: A multi-institutional study. Journal of Social Psychology, 154(3), 181–185. https://doi.org/10.1080/00224545.2013.872596CrossRefGoogle ScholarPubMed
Enyon, R., Fry, J., & Schroeder, R. (2016). The ethics of online research. In Fielding, N. G., Lee, R. M., & Blank, G. (eds.), The SAGE Handbook of Online Research Methods (pp. 19–37). SAGE Publications.Google Scholar
Fair Crowd Work. (n.d.). Amazon Mechanical Turk. http://faircrowd.work/platform/amazon-mechanical-turk (retrieved June 9, 2022).Google Scholar
Fordsham, N., Moss, A. J., Krumholtz, S., Roggina, T., Robinson, J., & Litman, L. (2019). Variation among Mechanical Turk Workers across time of day presents an opportunity and a challenge for research [preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/p8bnsCrossRefGoogle Scholar
Fort, K., Adda, G., & Bretonnel Cohen, K. (2011). Amazon Mechanical Turk: Gold mine or coal mine? Computational Linguistics, 37(2), 413–420. https://doi.org/10.1162/COLI_a_00057CrossRefGoogle Scholar
Gallo, J., & Gran-Ruaz, S. (2021, October 1). Racial trauma scale: Creative interview strategies employed in the development of a new clinical tool for measuring race-based stress and trauma. CloudResearch Innovations in Online Research Conference (virtual). www.youtube.com/watch?v=hKCK_dWfUfI&t=1568sGoogle Scholar
Gallup News. (2007, September 20). Party affiliation. https://news.gallup.com/poll/15370/Party-Affiliation.aspxGoogle Scholar
Garbinsky, E. N., Gladstone, J. J., Nikolova, H., & Olson, J. G. (2020). Love, lies, and money: Financial infidelity in romantic relationships. Journal of Consumer Research, 47(1), 1–24. https://doi.org/10.1093/jcr/ucz052CrossRefGoogle Scholar
Gray, M. L., & Suri, S. (2019). Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass. Houghton Mifflin Harcourt.Google Scholar
Haggbloom, S. J., Warnick, R., Warnick, J. E., Jones, V. K., Yarbrough, G. L., Russell, T. M., et al. (2002). The 100 most eminent psychologists of the 20th century. Review of General Psychology, 6(2), 139–152. https://doi.org/10.1037/1089-2680.6.2.139CrossRefGoogle Scholar
Hall, M. P., Lewis, N. A. Jr., Chandler, J., & Litman, L. (2020). Conducting longitudinal research on Amazon Mechanical Turk. In Litman, L. & Robinson, J. (eds.), Conducting Online Research on Amazon Mechanical Turk and Beyond (pp. 198–216). SAGE Publications.Google Scholar
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018). A data-driven analysis of workers’ earnings on Amazon Mechanical Turk. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1–14). ACM. https://doi.org/10.1145/3173574.3174023Google Scholar
Hauser, D. J., Moss, A. J., Rosenzweig, C., Jaffe, S., & Robinson, J. (2022). Evaluating CloudResearch’s approved group as a solution for problematic data quality on MTurk. Behavioral Research Methods, 55(8), 3953–3964. https://doi.org/10.31234/osf.io/48yxjCrossRefGoogle Scholar
Hauser, D. J., Paolacci, G., & Chandler, J. (2019). Common concerns with MTurk as a participant pool: Evidence and solutions. In Kardes, F. R., Herr, P. M., & Schwarz, N. (eds.), Handbook of Research Methods in Consumer Psychology (pp. 319–337). Routledge.Google Scholar
Hauser, D. J., & Schwarz, N. (2016). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1), 400–407. https://doi.org/10.3758/s13428-015-0578-zCrossRefGoogle ScholarPubMed
Hillyguys, D. S., Jackson, N., & Young, M. (2014). Professional respondents in non-probability online panels. In Callegaro, M. et al. (eds.), Online Panel Research: A Data Quality Perspective, 1st ed. (pp. 219–237). Wiley.CrossRefGoogle Scholar
Hitlin, P. (2016). Research in the crowdsourcing age, a case study. Pew Research Center. www.pewresearch.org/internet/2016/07/11/what-is-mechanical-turkGoogle Scholar
Horton, J. J., Rand, D. G., & Zeckhauser, R. J. (2011). The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14(3), 399–425. https://doi.org/10.1007/s10683-011-9273-9CrossRefGoogle Scholar
Huff, C., & Tingley, D. (2015). “Who are these people?” Evaluating the demographic characteristics and political preferences of MTurk survey respondents. Research & Politics, 2(3), 205316801560464. https://doi.org/10.1177/2053168015604648CrossRefGoogle Scholar
Kaplan, T., Saito, S., Hara, K., & Bigham, J. P. (2018, June 15). Striving to earn more: A survey of work strategies and tool use among crowd workers. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 6, 70–78. https://doi.org/10.1609/hcomp.v6i1.13327CrossRefGoogle Scholar
Kees, J., Berry, C., Burton, S., & Sheehan, K. (2017). An analysis of data quality: Professional panels, student subject pools, and Amazon’s Mechanical Turk. Journal of Advertising, 46(1), 141–155. https://doi.org/10.1080/00913367.2016.1269304CrossRefGoogle Scholar
Keller, L., Kabengele, M.-C., & Gollwitzer, P. M. (2021). The self-regulation of face touching – a preregistered experiment testing if-then plans as a means to promote COVID-19 prevention. Psychology & Health, 38(8), 1–19. https://doi.org/10.1080/08870446.2021.2005793Google ScholarPubMed
Kennedy, R., Clifford, S., Burleigh, T., Waggoner, P., & Jewell, R. (2018). How Venezuela’s economic crisis is undermining social science research – about everything. Washington Post, November 7, www.washingtonpost.com/news/monkey-cage/wp/2018/11/07/how-the-venezuelan-economic-crisis-is-undermining-social-science-research-about-everything-not-just-venezuela (retrieved May 27, 2022).Google Scholar
Kittur, A., Chi, E. H., & Suh, B. (2008). Crowdsourcing user studies with Mechanical Turk. CHI ’08: Proceedings of the Twenty-Sixth Annual CHI Conference on Human Factors in Computing Systems (pp. 453–456). ACM. https://doi.org/10.1145/1357054.1357127CrossRefGoogle Scholar
Krupnikov, Y., & Levine, A. S. (2014). Cross-sample comparisons and external validity. Journal of Experimental Political Science, 1(1), 59–80. https://doi.org/10.1017/xps.2014.7CrossRefGoogle Scholar
Levay, K. E., Freese, J., & Druckman, J. N. (2016). The demographic and political composition of Mechanical Turk samples. SAGE Open, 6(1), 215824401663643. https://doi.org/10.1177/2158244016636433CrossRefGoogle Scholar
Litman, L. (2019, May). Composition of Online Participant Pools Moderates Effect Sizes of Experimental Manipulations. Association for Psychological Science.Google Scholar
Litman, L., & Robinson, J. (2020a). Conducting ethical online research: A data-driven approach. In Litman, L. & Robinson, J. (eds.), Conducting Online Research on Amazon Mechanical Turk and Beyond (pp. 234–263). SAGE Publications.Google Scholar
Litman, L., & Robinson, J. (2020b). Introduction. In Litman, L. & Robinson, J. (eds.), Conducting Online Research on Amazon Mechanical Turk and Beyond (pp. 1–26). SAGE Publications.Google Scholar
Litman, L., Robinson, J., & Abberbock, T. (2017). TurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences. Behavior Research Methods, 49(2), 433–442. https://doi.org/10.3758/s13428-016-0727-zCrossRefGoogle ScholarPubMed
Litman, L., Robinson, J., & Rosenzweig, C. (2015). The relationship between motivation, monetary compensation, and data quality among US- and India-based workers on Mechanical Turk. Behavior Research Methods, 47(2), 519–528. https://doi.org/10.3758/s13428-014-0483-xCrossRefGoogle ScholarPubMed
Litman, L., Rosenzweig, C., Jaffe, S. N., Gautam, R., Robinson, J., & Moss, A. J. (2021). Bots or inattentive humans? Identifying sources of low-quality data in online platforms [preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/wr8dsCrossRefGoogle Scholar
Litman, L., Rosenzweig, C., & Moss, A. J. (2020, July 15). New solutions dramatically improve research data quality on MTurk. CloudResearch [blog]. www.cloudresearch.com/resources/blog/new-tools-improve-research-data-quality-mturkGoogle Scholar
Loepp, E., & Kelly, J. T. (2020). Distinction without a difference? An assessment of MTurk worker types. Research & Politics, 7(1), 205316801990118. https://doi.org/10.1177/2053168019901185CrossRefGoogle Scholar
Matsakis, L. (2018, March 22). A window into how YouTube trains AI to moderate videos. Wired. www.wired.com/story/youtube-mechanical-turk-content-moderation-aiGoogle Scholar
Matthijsse, S. M., de Leeuw, E. D., & Hox, J. J. (2015). Internet panels, professional respondents, and data quality. Methodology, 11(3), 81–88.CrossRefGoogle Scholar
McCredie, M. N., & Morey, L. C. (2019). Who are the Turkers? A characterization of MTurk workers using the Personality Assessment Inventory. Assessment, 26(5), 759–766. https://doi.org/10.1177/1073191118760709CrossRefGoogle ScholarPubMed
Mehrotra, D. (2020). Horror Stories From Inside Amazon’s Mechanical Turk. Gizmodo, January 28. https://gizmodo.com/horror-stories-from-inside-amazons-mechanical-turk-1840878041Google Scholar
Meyers, E. A., Walker, A. C., Fugelsang, J. A., & Koehler, D. J. (2020). Reducing the number of non-naïve participants in Mechanical Turk samples. Methods in Psychology, 3, 100032. https://doi.org/10.1016/j.metip.2020.100032CrossRefGoogle Scholar
Miller, J. D., Crowe, M., Weiss, B., Maples-Keller, J. L., & Lynam, D. R. (2017). Using online, crowdsourcing platforms for data collection in personality disorder research: The example of Amazon’s Mechanical Turk. Personality Disorders: Theory, Research, and Treatment, 8(1), 26–34. https://doi.org/10.1037/per0000191CrossRefGoogle ScholarPubMed
Moss, A. J. (2021, December 1). Five years of Mechanical Turk data in five figures. CloudResearch [blog]. www.cloudresearch.com/resources/blog/mechanical-turk-data-five-years-in-five-figuresGoogle Scholar
Moss, A. J. (2022, March 4). How CloudResearch and IARPA completed the largest longitudinal online research project ever. CloudResearch [blog]. www.cloudresearch.com/resources/blog/the-largest-longitudinal-online-research-projectGoogle Scholar
Moss, A. J., Rosenzweig, C., Robinson, J., Jaffe, S. N., & Litman, L. (2023). Is it ethical to use Mechanical Turk for behavioral research? Relevant data from a representative survey of MTurk participants and wages. Behavior Research Methods, 55(8), 4048–4067.CrossRefGoogle ScholarPubMed
Mullinix, K. J., Leeper, T. J., Druckman, J. N., & Freese, J. (2015). The generalizability of survey experiments. Journal of Experimental Political Science, 2(2), 109–138. https://doi.org/10.1017/XPS.2015.19CrossRefGoogle Scholar
Newman, A. (2019, November 15). I found work on an Amazon Website. I made 97 cents an hour. New York Times. www.nytimes.com/interactive/2019/11/15/nyregion/amazon-mechanical-turk.htmlGoogle Scholar
Nichols, A. L., & Edlund, J. E. (2020). Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants. International Journal of Social Research Methodology, 23(6), 625–638. https://doi.org/10.1080/13645579.2020.1719618CrossRefGoogle Scholar
Ogletree, A. M., & Katz, B. (2021). How do older adults recruited using MTurk differ from those in a national probability sample? International Journal of Aging and Human Development, 93(2), 700–721. https://doi.org/10.1177/0091415020940197CrossRefGoogle Scholar
Ophir, Y., Sisso, I., Asterhan, C. S. C., Tikochinski, R., & Reichart, R. (2020). The Turker blues: Hidden factors behind increased depression rates among Amazon’s Mechanical Turkers. Clinical Psychological Science, 8(1), 65–83. https://doi.org/10.1177/2167702619865973CrossRefGoogle Scholar
Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23(3), 184–188. https://doi.org/10.1177/0963721414531598CrossRefGoogle Scholar
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. SSRN, 1626226. https://papers.ssrn.com/abstract=1626226CrossRefGoogle Scholar
Peer, E., Vosgerau, J., & Acquisti, A. (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods, 46(4), 1023–1031. https://doi.org/10.3758/s13428-013-0434-yCrossRefGoogle ScholarPubMed
Pontin, J. (2007, March 25). Artificial intelligence, with help from the humans. New York Times. www.nytimes.com/2007/03/25/business/yourmoney/25Stream.htmlGoogle Scholar
Rand, D. G. (2018). Non-naïvety may reduce the effect of intuition manipulations. Nature Human Behaviour, 2(9), 602. https://doi.org/10.1038/s41562-018-0404-6CrossRefGoogle ScholarPubMed
Rivera, E. D., Wilkowski, B. M., Moss, A. J., Rosenzweig, C., & Litman, L. (2022). Assessing the efficacy of a participant-vetting procedure to improve data-quality on Amazon’s Mechanical Turk. Methodology, 18(2), 126–143. https://doi.org/10.5964/meth.8331CrossRefGoogle Scholar
Robinson, J., & Litman, L. (2020). Conducting a study on Mechanical Turk. In Litman, L. & Robinson, J. (eds.), Conducting Online Research on Amazon Mechanical Turk and Beyond (pp. 49–78). SAGE Publications.Google Scholar
Robinson, J., Litman, L., & Rosenzweig, C. (2020). Who are the Mechanical Turk workers? In Litman, L. & Robinson, J. (eds.), Conducting Online Research on Amazon Mechanical Turk and Beyond (pp. 121–147). SAGE Publications.Google Scholar
Robinson, J., Rosenzweig, C., Moss, A. J., & Litman, L. (2019). Tapped out or barely tapped? Recommendations for how to harness the vast and largely unused potential of the Mechanical Turk participant pool. PLOS ONE, 14(12), e0226394. https://doi.org/10.1371/journal.pone.0226394CrossRefGoogle ScholarPubMed
Rouse, S. V. (2020). Reliability of MTurk data from masters and workers. Journal of Individual Differences, 41(1), 30–36. https://doi.org/10.1027/1614-0001/a000300CrossRefGoogle Scholar
Ryan, T. J. (2018, August 12). Data contamination on Mturk [blog]. https://timryan.web.unc.edu/2018/08/12/data-contamination-on-mturkGoogle Scholar
Semuels, A. (2018, January 23). The internet is enabling a new kind of poorly paid hell. The Atlantic. www.theatlantic.com/business/archive/2018/01/amazon-mechanical-turk/551192Google Scholar
Simonton, D. K. (2000). Methodological and theoretical orientation and the long-term disciplinary impact of 54 eminent psychologists. Review of General Psychology, 4(1), 13–24. https://doi.org/10.1037/1089-2680.4.1.13CrossRefGoogle Scholar
Stagnaro, M., Pennycook, G., & Rand, D. G. (2018). Performance on the cognitive reflection test is stable across time. SSRN, 3115809. https://papers.ssrn.com/abstract=3115809CrossRefGoogle Scholar
Stewart, N., Ungemach, C., Harris, A. J. L., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015). The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers. Judgment and Decision Making, 10(5), 479–491.CrossRefGoogle Scholar
Thomson, K. S., & Oppenheimer, D. M. (2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making, 11(1), 99–113.CrossRefGoogle Scholar
TurkerView. (2019, November 18). Writer who never learned to drive works for Uber. Makes $0.97/hr. https://blog.turkerview.com/writer-who-never-learned-to-drive-works-for-uberGoogle Scholar
US Census Bureau. (2020, February 5). Marital status in the United States. www.census.gov/library/visualizations/interactive/marital-status-in-united-states.htmlGoogle Scholar
US Census Bureau. (2021, November 29). Census Bureau releases new estimates on America’s families and living arrangements. www.census.gov/newsroom/press-releases/2021/families-and-living-arrangements.htmlGoogle Scholar
US Census Bureau. (2022, February 24). Census Bureau releases new educational attainment data. www.census.gov/newsroom/press-releases/2022/educational-attainment.htmlGoogle Scholar
US Census Bureau. (n.d.). QuickFacts: United States. www.census.gov/quickfacts/fact/table/US/POP010220Google Scholar
Walters, K., Christakis, D. A., & Wright, D. R. (2018). Are Mechanical Turk worker samples representative of health status and health behaviors in the U.S.? PLOS ONE, 13(6), e0198835. https://doi.org/10.1371/journal.pone.0198835CrossRefGoogle ScholarPubMed
Wessling, K., Huber, J., & Netzer, O. (2017). MTurk character misrepresentation: Assessment and solutions. Journal of Consumer Research, 44(1), 211–230. https://doi.org/10.1093/jcr/ucx053CrossRefGoogle Scholar
Williamson, V. (2016). On the ethics of crowdsourced research. PS: Political Science & Politics, 49(01), 77–81. https://doi.org/10.1017/S104909651500116XGoogle Scholar
Zwaan, R. A., Pecher, D., Paolacci, G., Bouwmeester, S., Verkoeijen, P., Dijkstra, K., & Zeelenberg, R. (2018). Participant nonnaiveté and the reproducibility of cognitive psychology. Psychonomic Bulletin & Review, 25(5), 1968–1972. https://doi.org/10.3758/s13423-017-1348-yCrossRefGoogle ScholarPubMed

References

Acker, A., & Kreisberg, A. (2020). Social media data archives in an API-driven world. Archival Science, 20, 105–123.CrossRefGoogle Scholar
Albertson, B., & Gadarian, S. (2014, July 1). Was the Facebook emotion experiment unethical? Washington Post. www.washingtonpost.com/news/monkey-cage/wp/2014/07/01/was-the-facebook-emotion-experiment-unethical/?utm_term=.15088275b53cGoogle Scholar
Anderson, C. A., Allen, J. J., Plante, C., Quigley-McBride, A., Lovett, A., & Rokkum, J. N. (2019). The MTurkification of social and personality psychology. Personality and Social Psychology Bulletin, 45(6), 842–850.CrossRefGoogle ScholarPubMed
Antoun, C., Zhang, C., Conrad, F. G., & Schober, M. F. (2016). Comparisons of online recruitment strategies for convenience samples: Craigslist, Google AdWords, Facebook, and Amazon Mechanical Turk. Field methods, 28(3), 231–246.CrossRefGoogle Scholar
Arif, A., Stewart, L. G., & Starbird, K. (2018). Acting the part: Examining information operations within #BlackLivesMatter discourse. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 1–27.CrossRefGoogle Scholar
Ashokkumar, A., & Pennebaker, J. W. (2021). Social media conversations reveal large psychological shifts caused by COVID-19’s onset across US cities. Science Advances, 7(39), eabg7843.CrossRefGoogle Scholar
Barrie, C., & Ho, J. C. (2022). Using the Twitter Academic API with R for Social Science Research. SAGE Publications. https://dx.doi.org/10.4135/9781529609233CrossRefGoogle Scholar
Bayer, J. B., Triệu, P., & Ellison, N. B. (2020). Social media elements, ecologies, and effects. Annual Review of Psychology, 71, 471–497.CrossRefGoogle ScholarPubMed
Bentley, F. R., Daskalova, N., & White, B. (2017, May). Comparing the reliability of Amazon Mechanical Turk and Survey Monkey to traditional market research surveys. In Proceedings of the 2017 CHI conference extended abstracts on human factors in computing systems (pp. 1092–1099). ACM.CrossRefGoogle Scholar
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political analysis, 20(3), 351–368.CrossRefGoogle Scholar
Boyd, R. L., Ashokkumar, A., Seraj, S., & Pennebaker, J. W. (2022). The Development and Psychometric Properties of LIWC-22. University of Texas at Austin.Google Scholar
Boyd, R. L., & Pennebaker, J. W. (2015). A way with words: Using language for psychological science in the modern era. In Dimofte, C. V., Haugtvedt, C. P., & Yalch, R. F. (eds.), Consumer Psychology in a Social Media World (pp. 222–236). Routledge.Google Scholar
Boyd, R. L., & Pennebaker, J. W. (2017). Language-based personality: A new approach to personality in a digital world. Current Opinion in Behavioral Sciences, 18, 63–68.CrossRefGoogle Scholar
Buchanan, E. A., & Hvizdak, E. E. (2009). Online survey tools: Ethical and methodological concerns of human research ethics committees. Journal of Empirical Research on Human Research Ethics, 4(2), 37–48.CrossRefGoogle ScholarPubMed
Buchanan, T., & Williams, J. E. (2010). Ethical issues in psychological research on the internet. In Gosling, S. D. & Johnson, J. A. (eds.), Advanced Methods for Conducting Online Behavioral Research (pp. 255–271). American Psychological Association.CrossRefGoogle Scholar
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5.CrossRefGoogle Scholar
Burger, J. M. (2014). Personality. Cengage Learning.Google Scholar
Butler, L., Lamont, P., Wan, D. L. Y., Prike, T., Nasim, M., Walker, B., et al. (2022). The (mis) information game: A social media simulator. Behavior Research Methods. https://doi.org/10.3758/s13428-023-02153-xCrossRefGoogle Scholar
boyd, D. M., & Ellison, N. B. (2007). Social network sites: Definition, history, and scholarship. Journal of Computer-Mediated Communication, 13(1), 210–230. https://doi.org/10.1111/j.1083-6101.2007.00393.xCrossRefGoogle Scholar
Cadwalladr, C., Khalili, M., Phillips, C., Silver, M., Jenkins, A., Search, J., et al. (2021). Cambridge Analytica whistleblower: “We spent $1m harvesting millions of Facebook profiles” [video]. Guardian, March 17. www.theguardian.com/uk-news/video/2018/mar/17/cambridge-analytica-whistleblower-we-spent-1m-harvesting-millions-of-facebook-profiles-videoGoogle Scholar
Casler, K., Bickel, L., & Hackett, E. (2013). Separate but equal? A comparison of participants and data gathered via Amazon’s MTurk, social media, and face-to-face behavioral testing. Computers in Human Behavior, 29(6), 2156–2160.CrossRefGoogle Scholar
Chambers, C. (2014). Facebook fiasco: Was Cornell’s study of “emotional contagion” an ethics breach? Guardian, July 1. www.theguardian.com/science/head-quarters/2014/jul/01/facebook-cornell-study-emotional-contagion-ethics-breachGoogle Scholar
Chambers, M., Bliss, K., & Rambur, B. (2020). Recruiting research participants via traditional snowball vs Facebook advertisements and a website. Western Journal of Nursing Research, 42(10), 846–851.CrossRefGoogle ScholarPubMed
Chandler, J. (2023). Participant recruitment. In Nichols, A. & Edlund, J. E. (eds.), Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences (vol. 1, pp. 179–201). Cambridge University Press.CrossRefGoogle Scholar
Chandler, J., & Shapiro, D. (2016). Conducting clinical research using crowdsourced convenience samples. Annual Review of Clinical Psychology, 12, 53–81.CrossRefGoogle ScholarPubMed
Clark, T., & Blackhart, G. (2023). Debriefing and post-experimental procedures. In Nichols, A. & Edlund, J. E. (eds.), Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences (vol. 1, pp. 244–265). Cambridge University Press.CrossRefGoogle Scholar
Clifford, S., Jewell, R. M., & Waggoner, P. D. (2015). Are samples drawn from Mechanical Turk valid for research on political ideology? Research & Politics, 2(4), 2053168015622072.CrossRefGoogle Scholar
Cohn, M. A., Mehl, M. R., & Pennebaker, J. W. (2004). Linguistic markers of psychological change surrounding September 11, 2001. Psychological Science, 15(10), 687–693.CrossRefGoogle ScholarPubMed
Dawson, P. (2014). Our anonymous online research participants are not always anonymous: Is this a problem? British Journal of Educational Technology, 45(3), 428–437.CrossRefGoogle Scholar
De Choudhury, M., Gamon, M., Counts, S., & Horvitz, E. (2013). Predicting depression via social media. ICWSM, 13, 1–10.Google Scholar
Dobber, T., Ó Fathaigh, R., & Zuiderveen Borgesius, F. J. (2019). The regulation of online political micro-targeting in Europe. Internet Policy Review, 8(4). https://policyreview.info/articles/analysis/regulation-online-political-micro-targeting-europeCrossRefGoogle Scholar
Edlund, J. E., Lange, K. M., Sevene, A. M., Umansky, J., Beck, C. D., & Bell, D. J. (2017). Participant crosstalk: Issues when using the Mechanical Turk. Tutorials in Quantitative Methods for Psychology, 13(3), 174–182.CrossRefGoogle Scholar
Foster, M. D. (2015). Tweeting about sexism: The well‐being benefits of a social media collective action. British Journal of Social Psychology, 54(4), 629–647.CrossRefGoogle ScholarPubMed
Gosling, S. D., & Mason, W. (2015). Internet research in psychology. Annual Review of Psychology, 66, 877–902.CrossRefGoogle ScholarPubMed
Graham-Harrison, E., & Cadwalladr, C. (2021). Revealed: 50 million Facebook profiles harvested for Cambridge Analytica in major data breach. Guardian, September 29. www.theguardian.com/news/2018/mar/17/cambridge-analytica-facebook-influence-us-electionGoogle Scholar
Grimm, P. (2010). Social desirability bias. In Cooper, C. L. (ed.), Wiley International Encyclopedia of Marketing. Wiley.Google Scholar
Guadagno, R. E. (2019). Using the internet for research. In Edlund, J. E. & Nichols, A. L. (eds.), Advanced Research Methods for the Social and Behavioral Sciences (pp. 68–82). Cambridge University Press.Google Scholar
Guadagno, R. E. (in press). Psychological Processes in Social Media: Why We Click. Academic Press.Google Scholar
Guadagno, R. E., Loewald, T. A., Muscanell, N. L., Barth, J. M., Goodwin, M. K., & Yang, Y. (2013). Facebook history collector: A new method for directly collecting data from Facebook. International Journal of Interactive Communication Systems and Technologies (IJICST), 3(1), 57–67.CrossRefGoogle Scholar
Guadagno, R. E., Muscanell, N. L., & Pollio, D. E. (2013). The homeless use Facebook?! Similarities of social network use between college students and homeless young adults. Computers in Human Behavior, 29(1), 86–89. https://doi.org/10.1016/j.chb.2012.07.019CrossRefGoogle Scholar
Guadagno, R. E., Muscanell, N. L., Rice, L. M., & Roberts, N. (2013). Social influence online: The impact of social validation and likability on compliance. Psychology of Popular Media Culture, 2(1), 51–60.CrossRefGoogle Scholar
Guadagno, R. E., Rempala, D. M., Murphy, S., & Okdie, B. M. (2013). What makes a video go viral? An analysis of emotional contagion and internet memes. Computers in Human Behavior, 29(6), 2312–2319.CrossRefGoogle Scholar
Hellström, T., Dignum, V., & Bensch, S. (2020). Bias in machine learning: What is it good for? [preprint]. Arxiv. https://doi.org/10.48550/arXiv.2004.00686CrossRefGoogle Scholar
Hill, K. (2014). Facebook manipulated 689,003 users’ emotions for science. Forbes, June 28. www.forbes.com/sites/kashmirhill/2014/06/28/facebook-manipulated-689003-users-emotions-for-science/#593f5cbf197cGoogle Scholar
Ireland, M. E., & Henderson, M. D. (2014). Language style matching, engagement, and impasse in negotiations. Negotiation and Conflict Management Research, 7(1), 1–16.CrossRefGoogle Scholar
Jacobucci, R., & Grimm, K. J. (2020). Machine learning and psychological research: The unexplored effect of measurement. Perspectives on Psychological Science, 15(3), 809–816. https://doi.org/10.1177/1745691620902467CrossRefGoogle ScholarPubMed
Jagayat, A. (2022, October 4). Mock social media website tool. OSF. https://osf.io/m2xd8Google Scholar
Jenders, M., Kasneci, G., & Naumann, F. (2013, May). Analyzing and predicting viral tweets. In Schwabe, Daniel (ed.), Proceedings of the 22nd International Conference on World Wide Web (pp. 657–664). ACM.CrossRefGoogle Scholar
Kendall, C., Kerr, L. R., Gondim, R. C., Werneck, G. L., Macena, R. H. M., Pontes, M. K., et al. (2008). An empirical comparison of respondent-driven sampling, time location sampling, and snowball sampling for behavioral surveillance in men who have sex with men, Fortaleza, Brazil. AIDS and Behavior, 12, 97–104.CrossRefGoogle ScholarPubMed
Kern, M. L., McCarthy, P. X., Chakrabarty, D., & Rizoiu, M. A. (2019). Social media-predicted personality traits and values can help match people to their ideal jobs. Proceedings of the National Academy of Sciences, 116(52), 26459–26464.CrossRefGoogle ScholarPubMed
Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 110(15), 5802–5805.CrossRefGoogle ScholarPubMed
Kramer, A. D., Guillory, J. E., & Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(24), 8788–8790.CrossRefGoogle ScholarPubMed
Ksiazek, T. B. (2015). Civil interactivity: How news organizations’ commenting policies explain civility and hostility in user comments. Journal of Broadcasting & Electronic Media, 59(4), 556–573.CrossRefGoogle Scholar
Lee, J., Gillath, O., Kimbrough, A. M., & Guadagno, R. E. (2017, January). Development and validation of helping in gaming scales. Poster presented at the Media Psychology Preconference, San Antonio, TX.Google Scholar
Lee, J., & Spratling, R. (2019). Recruiting mothers of children with developmental disabilities: Adaptations of the snowball sampling technique using social media. Journal of Pediatric Health Care, 33(1), 107–110.CrossRefGoogle ScholarPubMed
Markham, A., & Buchanan, E. (2012). Recommendations from the AoIR Ethics Working Committee (Version 2.0).Google Scholar
Merriam-Webster. (n.d.). Crowdsourcing. In Merriam-Webster.com [dictionary]. www.merriam-webster.com/dictionary/crowdsourcing (retrieved March 6, 2024).Google Scholar
Miller, J. D., Crowe, M., Weiss, B., Maples-Keller, J. L., & Lynam, D. R. (2017). Using online, crowdsourcing platforms for data collection in personality disorder research: The example of Amazon’s Mechanical Turk. Personality Disorders: Theory, Research, and Treatment, 8(1), 26–34.CrossRefGoogle ScholarPubMed
Murray, M. (2014). Users angered at Facebook emotion-manipulation study. Today, June 30. www.today.com/health/users-angered-facebook-emotion-manipulation-study-1D79863049Google Scholar
Muscanell, N. L., & Guadagno, R. E. (2012). Make new friends or keep the old: Gender and personality differences in social networking use. Computers in Human Behavior, 28(1), 107–112. https://doi.org/10.1016/j.chb.2011.08.016CrossRefGoogle Scholar
Nichols, A., & Edlund, J. E. (2020). Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants. International Journal of Social Research Methodology, 23(6), 625–638. https://doi.org/10.1080/13645579.2020.1719618CrossRefGoogle Scholar
Oinas-Kukkonen, H., & Oinas-Kukkonen, H. (2013). Humanizing the Web: Change and Social Innovation. Palgrave Macmillan.CrossRefGoogle Scholar
Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding Mechanical Turk as a participant pool. Current Directions in Psychological Science, 23(3), 184–188.CrossRefGoogle Scholar
Park, G., Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Kosinski, M., Stillwell, D. J., et al. (2015). Automatic personality assessment through social media language. Journal of Personality and Social Psychology, 108(6), 934–952.CrossRefGoogle ScholarPubMed
Parker, C., Scott, S., & Geddes, A. (2019). Snowball Sampling. SAGE Publications. https://dx.doi.org/10.4135/9781526421036831710Google Scholar
Pennebaker, J. W., Chung, C. K., Ireland, M., Gonzales, A., & Booth, R. J. (2007). The development and psychometric properties of LIWC2007. LIWC. www.liwc.net/LIWC2007LanguageManual.pdfGoogle Scholar
Pennebaker, J. W., Francis, M. E., & Booth, R. J. (2001). Linguistic inquiry and word count: LIWC 2001. LIWC. www.researchgate.net/publication/246699633_Linguistic_inquiry_and_word_count_LIWCGoogle Scholar
Phing, A. N. M., & Yazdanifard, R. (2014). How does ALS ice bucket challenge achieve its viral outcome through marketing via social media? Global Journal of Management and Business Research, 14(E7), 57–64.Google Scholar
Roozenbeek, J., & van der Linden, S. (2019). Fake news game confers psychological resistance against online misinformation. Palgrave Communications, 5(12). https://doi.org/10.1057/s41599-019-0279-9CrossRefGoogle Scholar
Ross, J., Zaldivar, A., Irani, L., & Tomlinson, B. (2009). Who are the Turkers? Worker demographics in Amazon Mechanical Turk. In CHI ’10 Extended Abstracts on Human Factors in Computing Systems (pp. 2863–2872). ACM.Google Scholar
Rouse, S. V. (2015). A reliability analysis of Mechanical Turk data. Computers in Human Behavior, 43, 304–307.CrossRefGoogle Scholar
Prevency. (n.d.). The solution for a realistic social media simulation. https://socialmediasimulator.comGoogle Scholar
Solon, O. (2018). Facebook says Cambridge Analytica may have gained 37 m more users’ data. Guardian, April 4. www.theguardian.com/technology/2018/apr/04/facebook-cambridge-analytica-user-data-latest-more-than-thoughtGoogle Scholar
Stephens-Davidowitz, S. (2017). Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us about Who We Really Are. HarperLuxe.Google Scholar
Vasalou, A., Gill, A. J., Mazanderani, F., Papoutsi, C., & Joinson, A. (2011). Privacy dictionary: A new resource for the automated content analysis of privacy. Journal of the Association for Information Science and Technology, 62(11), 2095–2105.Google Scholar
Vaughn-Nichols, S. J. (2014). We’re all just lab rats in Facebook’s laboratory. ZDNet, June 30. www.zdnet.com/article/were-all-just-lab-rats-in-facebooks-laboratory/Google Scholar
Verma, I. M. (2014). Editorial expression of concern: Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(29), 10779–10779.Google ScholarPubMed
Wang, T., Brede, M., Ianni, A., & Mentzakis, E. (2017, February). Detecting and characterizing eating-disorder communities on social media. In de Rijke, M. (ed.), Proceedings of the Tenth ACM International Conference on Web Search and Data Mining (pp. 91–100). ACM.CrossRefGoogle Scholar
Wright, S. A., & Goodman, J. K. (2019). Mechanical Turk in consumer research: Perceptions and usage in marketing academia. In Kardes, F. R., Herr, P. M., & Schwarz, N. (eds.), Handbook of Research Methods in Consumer Psychology (pp. 338–357). Routledge.Google Scholar
Zhou, H., & Fishbach, A. (2016). The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. Journal of Personality and Social Psychology, 111(4), 493–504.CrossRefGoogle ScholarPubMed
Zhou, L., Pan, S., Wang, J., & Vasilakos, A. V. (2017). Machine learning on big data: Opportunities and challenges. Neurocomputing, 237, 350–361.CrossRefGoogle Scholar
Zimmer, M. (2018). Addressing conceptual gaps in big data research ethics: An application of contextual integrity. Social Media + Society, 4(2), 2056305118768300.CrossRefGoogle Scholar

References

Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014). Separating the shirkers from the workers? Making sure respondents pay attention on self‐administered surveys. American Journal of Political Science, 58(3), 739–753.CrossRefGoogle Scholar
Chandler, J. J., Mueller, P., & Paolacci, G. (2014). Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods, 46(1), 112–130.CrossRefGoogle ScholarPubMed
Chandler, J. J., & Paolacci, G. (2017). Lie for a dime: When most prescreening responses are honest but most study participants are impostors. Social Psychological and Personality Science, 8(5), 500–508.CrossRefGoogle Scholar
Chandler, J. J., Paolacci, G., Peer, E., Mueller, P., & Ratliff, K. A. (2015). Using nonnaive participants can reduce effect sizes. Psychological Science, 26(7), 1131–1139.CrossRefGoogle ScholarPubMed
Chandler, J. J., Rosenzweig, C., Moss, A. J., Robinson, J., & Litman, L. (2019). Online panels in social science research: Expanding sampling methods beyond Mechanical Turk. Behavior Research Methods, 51(5), 2022–2038.CrossRefGoogle ScholarPubMed
Douglas, B. D., Ewell, P. J., & Brauer, M. (2023). Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. PLOS ONE, 18(3), e0279720.CrossRefGoogle ScholarPubMed
Gerlach, P., Teodorescu, K., & Hertwig, R. (2019). The truth about lies: A meta-analysis on dishonest behavior. Psychological Bulletin, 145(1), 1–44.CrossRefGoogle Scholar
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26(3), 213–224.CrossRefGoogle Scholar
Goodman, J. K., & Paolacci, G. (2017). Crowdsourcing consumer research. Journal of Consumer Research, 44(1), 196–210.CrossRefGoogle Scholar
Hauser, D. J., Moss, A. J., Rosenzweig, C., Jaffe, S., & Robinson, J. (2022). Evaluating CloudResearch’s approved group as a solution for problematic data quality on MTurk. Behavioral Research Methods, 55(8), 3953–3964. https://doi.org/10.31234/osf.io/48yxjCrossRefGoogle Scholar
Nichols, A. L., & Edlund, J. E. (2020). Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants. International Journal of Social Research Methodology, 23(6), 625-638.CrossRefGoogle Scholar
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4), 867–872.CrossRefGoogle Scholar
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision making, 5(5), 411–419.CrossRefGoogle Scholar
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017). Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70, 153–163.CrossRefGoogle Scholar
Peer, E., Rothschild, D., & Gordon, A. (2023). Behavioral Lab 3.0. Towards the next generation of online behavioral research [preprint]. PsyArXiv.Google Scholar
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2022). Data quality of platforms and panels for online behavioral research. Behavior Research Methods, 54, 1643–1662. https://doi.org/10.3758/s13428-021-01694-3CrossRefGoogle ScholarPubMed
Peer, E., Vosgerau, J., & Acquisti, A. (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods, 46(4), 1023–1031.CrossRefGoogle ScholarPubMed
Stewart, N., Ungemach, C., Harris, A. J., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015). The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers. Judgment and Decision Making, 10(5), 479–491.CrossRefGoogle Scholar
Thomas, K. A., & Clifford, S. (2017). Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments. Computers in Human Behavior, 77, 184–197.CrossRefGoogle Scholar
Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453–458.CrossRefGoogle ScholarPubMed

References

Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.CrossRefGoogle Scholar
Aronow, P.M., & Samii, C. (2017). Estimating average causal effects under general interference, with application to a social network experiment. Annals of Applied Statistics, 11, 1912–1947.CrossRefGoogle Scholar
Astuti, R., & Bloch, M. (2010). Why a theory of human nature cannot be based on the distinction between universality and variability: Lessons from anthropology. Behavioral and Brain Sciences, 33(2–3), 83–84.CrossRefGoogle Scholar
Beaman, L., Chattopadhyay, R., Duflo, E., Pande, R., & Topalova, P. (2009). Powerful women: Does exposure reduce bias? Quarterly Journal of Economics, 124(4), 1497–1540.CrossRefGoogle Scholar
Beshears, J., Dai, H., Milkman, K. L., & Benartzi, S. (2021). Using fresh starts to nudge increased retirement savings. Organizational Behavior and Human Decision Processes, 167, 72–87.CrossRefGoogle ScholarPubMed
Blair, G., Littman, R., & Paluck, E. L. (2019). Motivating the adoption of new community-minded behaviors: An empirical test in Nigeria. Science Advances, 5(3), eaau5175.CrossRefGoogle ScholarPubMed
Blair, G., & McClendon, G. (2021). Conducting experiments in multiple contexts. In Druckman, J. (ed.), Advances in Experimental Political Science (pp. 411–428). Cambridge University Press.CrossRefGoogle Scholar
Blair, G., Weinstein, J. M., Christia, F., Arias, E., Badran, E., Blair, R. A., et al. (2021). Community policing does not build citizen trust in police or reduce crime in the Global South. Science, 374(6571), eabd3446.CrossRefGoogle ScholarPubMed
Broockman, D., & Kalla, J. (2016). Durably reducing transphobia: A field experiment on door-to-door canvassing. Science, 352(6282), 220–224.CrossRefGoogle ScholarPubMed
Bullock, J., Green, D., & Ha, S. (2010). Yes, but what’s the mechanism? (Don’t expect an easy answer). Journal of Personality and Social Psychology, 98, 550–558.CrossRefGoogle ScholarPubMed
Carpenter, S. M., Menictas, M., Nahum-Shani, I., Wetter, D. W., & Murphy, S. A. (2020). Developments in mobile health just-in-time adaptive interventions for addiction science. Current Addiction Reports, 7(3), 280–290.CrossRefGoogle ScholarPubMed
Chang, E. H., Milkman, K. L., Gromet, D. M., Rebele, R. W., Massey, C., Duckworth, A. L., & Grant, A. M. (2019). The mixed effects of online diversity training. Proceedings of the National Academy of Sciences, 116(16), 7778–7783.CrossRefGoogle ScholarPubMed
Cialdini, R. B. (1980). Full-cycle social psychology. Applied Social Psychology Annual, 1, 21–47.Google Scholar
Cialdini, R. B., Demaine, L. J., Sagarin, B. J., Barrett, D. W., Rhoads, K., & Winter, P. L. (2006). Managing social norms for persuasive impact. Social Influence, 1(1), 3–15.CrossRefGoogle Scholar
Cui, Z., Liu, L., Li, D., Wu, S. J., & Zhai, X. (2022). Safety messaging boosts parental vaccination intention for children ages 5–11. Vaccines, 10, 1205.CrossRefGoogle ScholarPubMed
Dai, H., Saccardo, S., Han, M. A., Roh, L., Raja, N., Vangala, S., et al. (2021). Behavioural nudges increase COVID-19 vaccinations. Nature, 597(7876), 404–409.CrossRefGoogle ScholarPubMed
DiNardo, J., McCrary, J., & Sanbonmatsu, L. (2006). Constructive proposals for dealing with attrition: An empirical example. Working paper, University of Michigan.Google Scholar
Dolan, P., & Galizzi, M. M. (2014). Getting policy-makers to listen to field experiments. Oxford Review of Economic Policy, 30(4), 725–752.CrossRefGoogle Scholar
Duflo, E., & Banerjee, A. (2017). Handbook of Field Experiments. Elsevier.Google Scholar
Duflo, E., Glennerster, R., & Kremer, M. (2007). Using randomization in development economics research: A toolkit. Handbook of Development Economics, 4, 3895–3962.CrossRefGoogle Scholar
Dunning, T. (2016). Transparency, replication, and cumulative learning: What experiments alone cannot achieve. Annual Review of Political Science, 19, 541–563.CrossRefGoogle Scholar
Eden, D. (2017). Field experiments in organizations. Annual Review of Organizational Psychology and Organizational Behavior, 4, 91–122.CrossRefGoogle Scholar
Ferraro, P. J., & Agrawal, A. (2021). Synthesizing evidence in sustainability science through harmonized experiments: Community monitoring in common pool resources. Proceedings of the National Academy of Sciences, 118(29), e2106489118.CrossRefGoogle ScholarPubMed
Ferrero, M., & Pinto, I. (2023). A regenerative tourism approach for the development of marginalised areas: Insights from two best practices in Southern Italy. Turistica – Italian Journal of Tourism, 32(1), 128–149.CrossRefGoogle Scholar
Gantman, A., Gomila, R., Martinez, J. E., Matias, J. N., Elizabeth, L. P., Starck, J., et al. (2018). A pragmatist philosophy of psychological science and its implications for replication. Behavioral and Brain Sciences, 41.CrossRefGoogle ScholarPubMed
Gerber, A. S., & Green, D. P. (2000). The effects of canvassing, telephone calls, and direct mail on voter turnout: A field experiment. American Political Science Review, 94(3), 653–663.CrossRefGoogle Scholar
Gerber, A. S., & Green, D. P. (2012). Field Experiments: Design, Analysis, and Interpretation. W. W. Norton.Google Scholar
Gerber, A. S., Huber, G. A., Doherty, D., Dowling, C. M., & Ha, S. E. (2010). Personality and political attitudes: Relationships across issue domains and political contexts. American Political Science Review, 104(1), 111–133.CrossRefGoogle Scholar
Gneezy, U., & Rustichini, A. (2000). A fine is a price. Journal of Legal Studies, 29(1), 1–17.CrossRefGoogle Scholar
Graham, J. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576.CrossRefGoogle ScholarPubMed
Hansen, J. A., & Tummers, L. (2020). A systematic review of field experiments in public administration. Public Administration Review, 80(6), 921–931.CrossRefGoogle Scholar
Harrison, G. W., & List, J. A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009–1055.CrossRefGoogle Scholar
Haushofer, J., & Shapiro, J. (2016). The short-term impact of unconditional cash transfers to the poor: Experimental evidence from Kenya. Quarterly Journal of Economics, 131(4), 1973–2042. https://doi.org/10.1093/qje/qjw025CrossRefGoogle Scholar
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83.CrossRefGoogle ScholarPubMed
Hudgens, M. G., & Halloran, M. E. (2008). Toward causal inference with interference. Journal of the American Statistical Association, 103(482), 832–842.CrossRefGoogle ScholarPubMed
Hussey, M. A., & Hughes, J. P. (2007). Design and analysis of stepped wedge cluster randomized trials. Contemporary Clinical Trials, 28(2), 182–191.CrossRefGoogle ScholarPubMed
International Telecommunication Union. (2021, November). Facts and figures 2021: 2.9 billion people still offline. https://www.itu.int/hub/2021/11/facts-and-figures-2021-2-9-billion-people-still-offlineGoogle Scholar
James, W. (1907). Pragmatism’s conception of truth. Journal of Philosophy, Psychology and Scientific Methods, 4(6), 141–155.CrossRefGoogle Scholar
Kapiszewski, D., MacLean, L. M., & Read, B. L. (2015). Field Research in Political Science: Practices and Principles. Cambridge University Press.CrossRefGoogle Scholar
King, G., Pan, J., & Roberts, M. (2013). How censorship in China allows government criticism but silences collective expression. American Political Science Review, 107(2), 326–343.CrossRefGoogle Scholar
Lewin, K. (1944/1997). Problems of research in social psychology. In Lewin, , Resolving Social Conflicts; & Field Theory in Social Science. American Psychological Association.Google Scholar
Lewin, K. (1947). Frontiers in group dynamics: II. channels of group life; social planning and action research. Human Relations, 1(2), 143–153.CrossRefGoogle Scholar
Marrow, A. J. (1977). The Practical Theorist: The Life and Work of Kurt Lewin. Teachers College Press.Google Scholar
Moore-Berg, S. L., Bernstein, K., Gallardo, R. A., Hameiri, B., Littman, R., O’Neil, S., & Pasek, M. H. (2022). Translating social science for peace: Benefits, challenges, and recommendations. Peace and Conflict: Journal of Peace Psychology, 28(3), 274–283.CrossRefGoogle Scholar
Nickerson, D. W. (2008). Is voting contagious? Evidence from two field experiments. American Political Science Review, 102(1), 49–57.CrossRefGoogle Scholar
Paluck, E. L. (2009). Reducing intergroup prejudice and conflict using the media: A field experiment in Rwanda. Journal of Personality and Social Psychology, 96, 574–587.CrossRefGoogle Scholar
Paluck, E. L., & Cialdini, R. B. (2014). Field research methods. In Judd, C. M. & Reis, H. T. (eds.), Handbook of Research Methods in Social and Personality Psychology, 2nd ed. (pp. 81–98). Cambridge University Press.CrossRefGoogle Scholar
Paluck, E. L., & Shafir, E. (2017). The psychology of construal in the design of field experiment. In Banerjee, A. V. & Duflo, E. (eds.), Handbook of Economic Field Experiments (vol. 1, pp. 245–268). North-Holland.CrossRefGoogle Scholar
Pan, J. (2019). How Chinese officials use the internet to construct their public image. Political Science Research and Methods, 7(2), 197–213.CrossRefGoogle Scholar
Pew Research Center. (2024, April). Internet/Broadband Fact Sheet. www.pewresearch.org/internet/fact-sheet/internet-broadbandGoogle Scholar
Radsch, C. (2009). From cell phones to coffee: Issues of access in Egypt and Lebanon. In Sriram, C. L., King, J. C., Mertus, J. A., Martin-Ortega, O., & Herman, J. (eds.), Surviving Field Research: Working in Violent and Difficult Situations. Routledge. https://doi.org/10.4324/9780203875278Google Scholar
Read, B. L., Kapiszewski, D., & MacLean, L. M. (2015). Field research in political science: Practices and principles. In Read, , Kapiszewski, , & MacLean, (eds.), Field Research in Political Science: Practices and Principles (pp. 1–33). Cambridge University Press.Google Scholar
Roberts, S. O., Bareket-Shavit, C., Dollins, F. A., Goldie, P. D., & Mortenson, E. (2020). Racial inequality in psychological research: Trends of the past and recommendations for the future. Perspectives on Psychological Science, 15(6), 1295–1309.CrossRefGoogle ScholarPubMed
Ross, L., & Nisbett, R. E. (1991). The Person and the Situation: Perspectives of Social Psychology. McGraw-Hill.Google Scholar
Rozin, P. (2010). The weirdest people in the world are a harbinger of the future of the world. Behavioral and Brain Sciences, 33(2–3), 108–109.CrossRefGoogle ScholarPubMed
Rubin, D. B. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services and Outcomes Research Methodology, 2(3), 169–188.CrossRefGoogle Scholar
Rubin, D. B. (2005). Causal inference using potential outcomes. Journal of the American Statistical Association, 100(469), 322–331.CrossRefGoogle Scholar
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin.Google Scholar
Shafer, K., & Lohse, B. (2005). How to conduct a cognitive interview: A nutrition education example. US Department of Agriculture, National Institute of Food and Agriculture.Google Scholar
Silver, L. (2019, February 5). Smartphone ownership is growing rapidly around the world but not always equally. Pew Research Center. www.pewresearch.org/global/2019/02/05/digital-connectivity-growing-rapidly-in-emerging-economiesGoogle Scholar
Tankard, M. E., & Paluck, E. L. (2017). The effect of a Supreme Court decision regarding gay marriage on social norms and personal attitudes. Psychological Science, 28(9), 1334–1344. https://doi.org/10.1177/0956797617709594CrossRefGoogle ScholarPubMed
Voigt, R., Camp, N. P., Prabhakaran, V., Hamilton, W. L., Hetey, R. C., Griffiths, C. M., et al. (2017). Language from police body camera footage shows racial disparities in officer respect. Proceedings of the National Academy of Sciences, 114(25), 6521–6526.CrossRefGoogle ScholarPubMed
Walton, G. M., & Cohen, G. L. (2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science, 331(6023), 1447–1451.CrossRefGoogle ScholarPubMed
Willis, G. (2004). Cognitive Interviewing. SAGE Publications. https://doi.org/10.4135/9781412983655Google Scholar
Wood, E. J. (2009). Field research. In Boix, C. & Stokes, S. C. (eds.), The Oxford Handbook of Comparative Politics (pp. 123–146). Oxford University Press.CrossRefGoogle Scholar
Wu, S. J., Mai, M., Yi, F., Truex, R., & Zhuang, M. (2023). Bootstrapping participation: A field experiment on participatory budgeting and civic engagement in China. Working paper.Google Scholar
Wu, S. J., & Paluck, E. L. (2020). Participatory practices at work change attitudes and behavior toward societal authority and justice. Nature Communications, 11(1), 2633.CrossRefGoogle ScholarPubMed
Wu, S. J., & Paluck, E. L. (2021). Designing nudges for the context: Golden coin decals nudge workplace behavior in China. Organizational Behavior and Human Decision Processes, 163, 43–50.CrossRefGoogle Scholar
Wu, S. J., & Paluck, E. L. (2022). Having a voice in your group: Increasing productivity through group participation. Behavioural Public Policy, First View, 1–21.Google Scholar
Wu, S. J., Yuhan Mei, B., & Cervantez, J. (2022). Preferences and perceptions of workplace participation: A cross-cultural study. Frontiers in Psychology, 13, 806481.CrossRefGoogle ScholarPubMed

References

Adamovic, M., Sojo, V., Schachtman, R., & Vargas, A. (2022). Explaining the relationship between ethnicity and depressive symptoms: The roles of climate for inclusion, job self-efficacy, and job demands. Asia Pacific Journal of Management, 40, 903–928. https://doi.org/10.1007/s10490-022-09834-9CrossRefGoogle Scholar
Aguinis, H., Ramani, R. S., & Villamor, I. (2019). The first 20 years of organizational research methods: Trajectory, impact, and predictions for the future. Organizational Research Methods, 22(2), 463–489. https://doi.org/10.1177/1094428118786564CrossRefGoogle Scholar
Aguinis, H., & Vandenberg, R. J. (2014). An ounce of prevention is worth a pound of cure: Improving research quality before data collection. Annual Review of Organizational Psychology and Organizational Behavior, 1, 569–595. https://doi.org/10.1146/annurev-orgpsych-031413-091231CrossRefGoogle Scholar
Antonakis, J., Banks, G. C., Bastardoz, N., Cole, M. S., Day, D. V., Eagly, A. H., et al. (2019). The Leadership Quarterly: State of the journal. Leadership Quarterly, 30(1), 1–9. https://doi.org/10.1016/j.leaqua.2019.01.001CrossRefGoogle Scholar
Aquino, K., & Lamertz, K. (2004). A relational model of workplace victimization: Social roles and patterns of victimization in dyadic relationships. Journal of Applied Psychology, 89, 1023–1034. http://dx.doi.org/10.1037/0021-9010.89.6.1023.CrossRefGoogle ScholarPubMed
Australian Bureau of Statistics. (2022). Business longitudinal analysis data environment (BLADE). www.abs.gov.au/about/data-services/data-integration/integrated-data/business-longitudinal-analysis-data-environment-blade (retrieved October 17, 2022).Google Scholar
Beal, D. J. (2015). ESM 2.0: State of the art and future potential of experience sampling methods in organizational research. Annual Review of Organizational Psychology and Organizational Behavior, 2(1), 383–407. https://doi.org/10.1146/annurev-orgpsych-032414-111335CrossRefGoogle Scholar
Becker, T. E. (2005). Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8(3), 274–289. https://doi.org/10.1177/1094428105278021CrossRefGoogle Scholar
Beckman, N., Birney, D., Beckman, J., Wood, R., Sojo, V., & Bowman, D. (2020). Inter-individual differences in intra-individual variability in personality within and across contexts. Journal of Research in Personality, 85, 1–22. https://doi.org/10.1016/j.jrp.2019.103909CrossRefGoogle Scholar
Bellé, N. (2015). Performance-related pay and the crowding out of motivation in the public sector: A randomized field experiment. Public Administration Review, 75(2), 230–241. https://doi.org/10.1111/puar.12313CrossRefGoogle Scholar
Black, T. R. (2002). Understanding Social Science Research. SAGE Publications. https://dx.doi.org/10.4135/9780857020208.n3CrossRefGoogle Scholar
Bono, J. E., & McNamara, G. (2011). Publishing in AMJ – Part 2: Research design. Academy of Management Journal, 54(4), 657–660. https://doi.org/10.5465/amj.2011.64869103CrossRefGoogle Scholar
Brewerton, P., & Millward, L. (2001). Organizational Research Methods. SAGE Publications. https://doi.org/10.4135/9781849209533CrossRefGoogle Scholar
Calderwood, L., & Lessof, C. (2009). Enhancing longitudinal surveys by linking to administrative data. In Groves, R. M., Kalton, G., Rao, J. N. K., Schwarz, N., Skinner, C., & Lynn, P. (eds.), Methodology of Longitudinal Surveys (pp. 55–72). https://doi.org/10.1002/9780470743874.ch4CrossRefGoogle Scholar
Callegaro, M., Baker, R., Bethlehem, J., Göritz, A. S., Krosnick, J. A., and Lavrakas, P. J. (2014). Online panel research. In Callegaro, M., Baker, R., Bethlehem, J., Göritz, A. S., Krosnick, J. A., and Lavrakas, P. J. (eds.), Online Panel Research (pp. 1–22). Wiley. https://doi.org/10.1002/9781118763520.ch1CrossRefGoogle Scholar
Campbell, D. T., & Stanley, J. C. (1963). Experimental and Quasi-Experimental Designs for Research. Houghton Mifflin.Google Scholar
Caplow, T. (1964). Principles of Organization. Harcourt Brace & World.Google Scholar
Chandler, J., Mueller, P., & Paolacci, G. (2014). Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods, 46(1), 112–130. https://doi.org/10.3758/s13428-013-0365-7CrossRefGoogle ScholarPubMed
Charlesworth, S., McDonald, P., & Cerise, S. (2011). Naming and claiming workplace sexual harassment in Australia. Australian Journal of Social Issues, 46(2), 141–161. https://doi.org/10.1002/j.1839-4655.2011.tb00211.xCrossRefGoogle Scholar
Clark, E., & McCann, T. V. (2005). Researching students: An ethical dilemma. Nurse Researcher, 12(3), 42–51. https://doi.org/10.7748/nr2005.01.12.3.42.c5947CrossRefGoogle ScholarPubMed
Cortina, L. M., Fitzgerald, L. F., & Drasgow, F. (2002). Contextualizing Latina experiences of sexual harassment: Preliminary tests of a structural model. Basic and Applied Social Psychology, 24(4), 295–311. https://doi.org/10.1207/S15324834BASP2404_5CrossRefGoogle Scholar
Cortina, J. M., & Landis, R. S. (2013). Introduction: Transforming our field by transforming its methods. In Cortina, J. M. & Landis, R. S. (eds.), Modern Research Methods for the Study of Behavior in Organizations (pp. 1–25). Routledge. https://doi.org/10.4324/9780203585146CrossRefGoogle Scholar
Cunliffe, A. L. (2022). Must I grow a pair of balls to theorize about theory in organization and management studies? Organization Theory, 3(3), 26317877221109277.CrossRefGoogle Scholar
Eden, D. (2017). Field experiments in organizations. Annual Review of Organizational Psychology and Organizational Behavior, 4(1), 91–122. https://doi.org/10.1146/annurev-orgpsych-041015-062400CrossRefGoogle Scholar
Edwards, J. R. (2010). Reconsidering theoretical progress in organizational and management research. Organizational Research Methods, 13(4), 615–619. https://doi.org/10.1177/1094428110380468CrossRefGoogle Scholar
Eldridge, J. E. T., & Crombie, A. D. (2013). A Sociology of Organisations. Routledge.Google Scholar
Fisher, G. G., & Sandell, K. (2015). Sampling in industrial–organizational psychology research: Now what? Industrial and Organizational Psychology, 8, 232–237. https://doi.org/10.1017/iop.2015.31CrossRefGoogle Scholar
Garud, N., Pati, R., Sojo, V., Bell, S. J., Hudson, R., & Shaw, H. (2022). 3 ways hospitals can boost worker engagement. Harvard Business Review, February 16. https://hbr.org/2022/02/3-ways-hospitals-can-boost-worker-engagement.Google Scholar
Goodman, J. K., & Paolacci, G. (2017). Crowdsourcing consumer research. Journal of Consumer Research, 44(1), 196–210. https://doi.org/10.1093/jcr/ucx047CrossRefGoogle Scholar
Grosser, T. J., Lopez-Kidwell, V., & Labianca, G. (2010). A social network analysis of positive and negative gossip in organizational life. Group & Organization Management, 35(2), 177–212. https://doi.org/10.1177/1059601109360391CrossRefGoogle Scholar
Gulati, R., Lawrence, P. R., & Puranam, P. (2005). Adaptation in vertical relationships: Beyond incentive conflict. Strategic Management Journal, 26(5), 415–440. https://doi.org/10.1002/smj.458CrossRefGoogle Scholar
Harley, B. (2018). Sociology, the labour process and employment relations. In Wilkinson, A., Dundon, T., Donaghey, J., & Colvin, A. (eds.), The Routledge Companion to Employment Relations (pp. 81–92). Routledge. https://doi.org/10.4324/9781315692968CrossRefGoogle Scholar
Hauser, D. J., & Schwarz, N. (2016). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48(1), 400–407. https://doi.org/10.3758/s13428-015-0578-zCrossRefGoogle ScholarPubMed
Heath, C., & Sitkin, S. (2001). Big-B versus Big-O: What is organizational about organizational behavior? Journal of Organizational Behavior, 22, 43–58. https://doi.org/10.1002/job.77CrossRefGoogle Scholar
Heath, C., & Staudenmayer, N. (2000). Coordination neglect: How lay theories of organizing complicate coordination in organizations. Research in Organizational Behavior, 22, 153–191. https://doi.org/10.1016/S0191-3085(00)22005-4CrossRefGoogle Scholar
Hedrick, T. E., Bickman, L., & Rog, D. J. (1993). Applied Research Design. SAGE Publications. https://dx.doi.org/10.4135/9781412983457CrossRefGoogle Scholar
Holman, M. (2022). Citations as power. #MHAWS: Mirya Holman’s Aggressive Winning Scholars Newsletter. https://miryaholman.substack.com/p/citations-as-power (retrieved April 28, 2022).Google Scholar
Hunter, S. B., Miles, J. N. V., Paddock, S. M., & D’Amico, E. J. (2013). Evaluating treatment efficacy. In Miller, P. M. (ed.), Interventions for Addiction: Comprehensive Addictive Behaviors and Disorders (pp. 589–597). Academic Press. https://doi.org/10.1016/B978-0-12-398338-1.00061-0CrossRefGoogle Scholar
Jeong, S.-H., Mooney, A., Zhang, Y., & Quigley, T. J. (2022). How do investors really react to the appointment of Black CEOs? Strategic Management Journal, 44(7), 1733–1752. https://doi.org/10.1002/smj.3454CrossRefGoogle Scholar
Johns, G. (2018). Advances in the treatment of context in organizational research. Annual Review of Organizational Psychology and Organizational Behavior, 5(1), 21–46. https://doi.org/10.1146/annurev-orgpsych-032117-104406CrossRefGoogle Scholar
Kerlinger, F. N., & Lee, H. B. (1999). Foundations of Behavioral Research. Harcourt College.Google Scholar
Kozlowski, S. W. J., & Ilgen, D. R. (2006). Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest, 7(3), 77–124. https://doi.org/10.1111/j.1529-1006.2006.00030.xCrossRefGoogle ScholarPubMed
Landers, R. N., & Behrend, T. S. (2015). An inconvenient truth: Arbitrary distinctions between organizational, Mechanical Turk, and other convenience samples. Industrial and Organizational Psychology, 8(2), 142–164. https://doi.org/10.1017/iop.2015.13CrossRefGoogle Scholar
MacKenzie, S. B. (2003). The dangers of poor construct conceptualization. Journal of the Academy of Marketing Science, 31(3), 323–326. https://doi.org/10.1177/0092070303031003011CrossRefGoogle Scholar
Maurer, C. C., & Qureshi, I. (2021). Not just good for her: A temporal analysis of the dynamic relationship between representation of women and collective employee turnover. Organization Studies, 42(1), 85–107. https://doi.org/10.1177/0170840619875480CrossRefGoogle Scholar
McAdams, D. P., & Olson, B. D. (2010). Personality development: Continuity and change over the life course. Annual Review of Psychology, 61(1), 517–542. https://doi.org/10.1146/annurev.psych.093008.100507CrossRefGoogle ScholarPubMed
Minbashian, A., Wood, R. E., & Beckmann, N. (2010). Task-contingent conscientiousness as a unit of personality at work. Journal of Applied Psychology, 95(5), 793–806. https://doi.org/10.1037/a0020016.CrossRefGoogle ScholarPubMed
Mitchell, T. R. (1985). An evaluation of the validity of correlational research conducted in organizations. Academy of Management Review, 10(2), 192–205. https://doi.org/10.5465/amr.1985.4277939CrossRefGoogle Scholar
Mulfinger, N., Sander, A., Stuber, F., Brinster, R., Junne, F., Limprecht, R., et al. (2019). Cluster-randomised trial evaluating a complex intervention to improve mental health and well-being of employees working in hospital – a protocol for the SEEGEN trial. BMC Public Health, 19(1), 1694. https://doi.org/10.1186/s12889-019-7909-4CrossRefGoogle ScholarPubMed
Osis, J., & Donins, U. (2017). Structure analysis and design. In Osis, & Donins, (eds.), Topological UML Modeling (pp. 205–224). Elsevier. https://doi.org/10.1016/B978-0-12-805476-5.00008-3Google Scholar
Overbeck, J. R., Neale, M. A., & Govan, C. L. (2010). I feel, therefore you act: Intrapersonal and interpersonal effects of emotion on negotiation as a function of social power. Organizational Behavior and Human Decision Processes, 112(2), 126–139. https://doi.org/10.1016/j.obhdp.2010.02.004CrossRefGoogle Scholar
Park, M., Leahey, E., & Funk, R. J. (2023). Papers and patents are becoming less disruptive over time. Nature, 613(7942), 138–144. https://doi.org/10.1038/s41586-022-05543-xCrossRefGoogle ScholarPubMed
Porter, C. O. L. H., Outlaw, R., Gale, J. P., & Cho, T. S. (2019). The use of online panel data in management research: A review and recommendations. Journal of Management, 45(1), 319–344. https://doi.org/10.1177/0149206318811569CrossRefGoogle Scholar
Porter, M. E. (1991). Towards a dynamic theory of strategy. Strategic Management Journal, 12(S2), 95–117. https://doi.org/10.1002/smj.4250121008CrossRefGoogle Scholar
Reed, M. (2006). Organizational theorizing: A historically contested terrain. In Clegg, S., Cynthia, H., Lawrence, T., & Nord, W. (eds.), The SAGE Handbook of Organization Studies (pp. 19–54). SAGE Publications. https://doi.org/10.4135/9781848608030.n2CrossRefGoogle Scholar
Roe, R. (2008). Time in applied psychology. European Psychologist, 13(1), 37–52. https://doi.org/10.1027/1016‐9040.13.1.37CrossRefGoogle Scholar
Schwab, D. P. (2004). Research Methods for Organizational Studies. Routledge.Google Scholar
Schwerdt, G., & Woessmann, L. (2020). Empirical methods in the economics of education. In Bradley, S. & Green, C. (eds.), The Economics of Education, 2nd ed. (pp. 3–20). Academic Press. https://doi.org/10.1016/B978-0-12-815391-8.00001-XCrossRefGoogle Scholar
Smith, N., Sabat, I., Martinez, L., Weaver, K., & Xu, S. (2015). A convenient solution: Using MTurk to sample from hard-to-reach populations. Industrial and Organizational Psychology, 8(2), 220–228. https://doi.org/10.1017/iop.2015.29CrossRefGoogle Scholar
Sojo, V., & Roberts, V. L. (2019). From apples and cases to barrels and orchards: Macro-level drivers of workplace abuse. Academy of Management Proceedings, 2019(1), 12323. https://doi.org/10.5465/AMBPP.2019.12323symposiumGoogle Scholar
Sojo, V., Wood, R., & Genat, A. (2016). Harmful workplace experiences and women’s occupational well-being: A meta-analysis. Psychology of Women Quarterly, 40(1), 10–40. https://doi.org/10.1177/0361684315599346CrossRefGoogle Scholar
Spector, P. E. (2006). Method variance in organizational research: Truth or urban legend? Organizational Research Methods, 9(2), 221–232. https://doi.org/10.1177/1094428105284955CrossRefGoogle Scholar
Spector, P. E., Zapf, D., Chen, P. Y., & Frese, M. (2000). Why negative affectivity should not be controlled in job stress research: Don’t throw out the baby with the bath water. Journal of Organizational Behavior, 21(1), 79–95. https://doi.org/10.1002/(SICI)1099-1379(200002)21:1<79::aid-job964>3.0.CO;2-G3.0.CO;2-G>CrossRefGoogle Scholar
Stratemeyer, M., Sojo, V., Wheeler, M., Rozenblat, V., Lee, I., Peter, D., et al. (2018). Recruit Smarter [technical report]. Victorian Government (Australia). www.vic.gov.au/recruit-smarterGoogle Scholar
Trzebiatowski, T. M., Wanberg, C. R., & Dossinger, K. (2020). Unemployed needn’t apply: Unemployment status, legislation, and interview requests. Journal of Management, 46(8), 1380–1407. https://doi.org/10.1177/0149206318823952CrossRefGoogle Scholar
Van Quaquebeke, N., Salem, M., van Dijke, M., & Wenzel, R. (2022). Conducting organizational survey and experimental research online: From convenient to ambitious in study designs, recruiting, and data quality. Organizational Psychology Review, 12(3), 268–305. https://doi.org/10.1177/20413866221097571CrossRefGoogle Scholar
Vantilborgh, T., Hofmans, J., & Judge, T. A. (2018). The time has come to study dynamics at work. Journal of Organizational Behavior, 39(9), 1045–1049. https://doi.org/10.1002/job.2327CrossRefGoogle Scholar
Vetter, T. R., & Chou, R. (2014). Clinical trial design methodology for pain outcome studies. In Benzon, H. T., Rathmell, J. P., Wu, C. L., Turk, D. C., Argoff, C. E., & Hurley, R. W. (eds.), Practical Management of Pain (pp. 1057–1065). Mosby. https://doi.org/10.1016/B978-0-323-08340-9.00080-3CrossRefGoogle Scholar
Walter, S. L., Seibert, S. E., Goering, D., & O’Boyle, E. H. (2019). A tale of two sample sources: Do results from online panel data and conventional data converge? Journal of Business and Psychology, 34(4), 425–452. https://doi.org/10.1007/s10869-018-9552-yCrossRefGoogle Scholar
Wang, M., Zhou, L., & Zhang, Z. (2016). Dynamic modeling. Annual Review of Organizational Psychology and Organizational Behavior, 3(1), 241–266. https://doi.org/10.1146/annurev‐orgpsych‐041015‐062553CrossRefGoogle Scholar
Wheeler, M., Wood, R., Sojo, V., & McGrath, M. (2016). A Question of Ethics: Navigating Ethical Failure in the Banking and Financial Services Industry. Chartered Accountants Australia and New Zealand and Centre for Ethical Leadership.Google Scholar
Wickert, C., Post, C., Doh, J. P., Prescott, J. E., & Prencipe, A. (2021). Management research that makes a difference: Broadening the meaning of impact. Journal of Management Studies, 58(2), 297–320. https://doi.org/10.1111/joms.12666CrossRefGoogle Scholar

References

Abbasi, G., Zadeh, S. S., Janfaza, E., Assemi, A., & Dehghan, S. S. (2012). Language, translation, and culture. International Conference on Language, Medias and Culture, 33(2), 83–87.Google Scholar
Adams, G., Dobles, I., Gómez, L. H., Kurtiş, T., & Molina, L. E. (2015). Decolonizing psychological science: Introduction to the special thematic section. Journal of Social and Political Psychology, 3, 213–238. https://doi.org/10.5964/jspp.v3i1.564CrossRefGoogle Scholar
Adams, V., Miller, S., Craig, S., Sonam, , Nyima, , Droyoung, , et al. (2007). Informed consent in cross-cultural perspective: Clinical research in the Tibetan Autonomous Region, PRC. Culture, Medicine and Psychiatry, 31, 445–472. https://doi.org/10.1007/s11013-007-9070-2CrossRefGoogle ScholarPubMed
Akaliyski, P., Welzel, C., Bond, M. H., & Minkov, M. (2021). On “nationology”: The gravitational field of national culture. Journal of Cross-Cultural Psychology, 52(8–9), 1–23. http://doi.org/10.1177/00220221211044780CrossRefGoogle Scholar
Arnett, J. J. (2008). The neglected 95%: Why American psychology needs to become less American. American Psychologist, 63(7), 602–614. https://psycnet.apa.org/doi/10.1037/0003-066X.63.7.602CrossRefGoogle ScholarPubMed
Ashdown, B. K., Dixe, A., & Talmage, C. A. (2021). The potentially damaging effects of developmental aid and voluntourism on cultural capital and well-being. International Journal of Community Well-Being, 4(1), 113–131. https://doi.org/10.1007/s42413-020-00079-2CrossRefGoogle Scholar
Barrett, H. C. (2020). Deciding what to observe: Thoughts for a post-WEIRD generation. Evolution and Human Behavior, 41(5), 445–453. https://doi.org/10.1016/j.evolhumbehav.2020.05.006CrossRefGoogle Scholar
Berry, J. W. (1999). Emics and etics: A symbiotic conception. Culture & Psychology, 5(2), 165–171. https://doi.org/10.1177/1354067X9952004CrossRefGoogle Scholar
Berry, J. W. (2013). Global psychology. South African Journal of Psychology, 43(4), 391–401. https://doi.org/10.1177%2F0081246313504517CrossRefGoogle Scholar
Boehnke, K., Lietz, P., Schreier, M., & Wilhelm, A. (2011). Sampling: The selection of cases for culturally comparative psychological research. In Matsumoto, & van de Vijver, (pp. 101–129).Google Scholar
Bou Zeineddine, F., Saab, R., Lášticová, B., Kende, A., & Ayanian, A. H. (2021). “Some uninteresting data from a faraway country”: Inequity and coloniality in international social psychological publication. Journal of Social Issues, 78(2), 320–345. https://doi.org/10.1111/josi.12481CrossRefGoogle Scholar
Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1(3), 185–216. https://doi.org/10.1177%2F135910457000100301CrossRefGoogle Scholar
Brislin, R. W., & Freimanis, C. (2001). Back-translation: A tool for cross-cultural research. In Sin-Wai, C. & Pollard, D. E. (eds.), An Encyclopedia of Translation (pp. 22–40). Chinese University Press.Google Scholar
Broesch, T., Crittenden, A. N., Beheim, B. A., Blackwell, A. D., Bunce, J. A., Colleran, H., et al. (2020). Navigating cross-cultural research: Methodological and ethical considerations. Proceedings of the Royal Society B: Biological Sciences, 287(1935), 20201245. https://doi.org/10.1098/rspb.2020.1245CrossRefGoogle ScholarPubMed
Broesch, T., Lew-Levy, S., Kärtner, J., Kanngiesser, P., & Kline, M. A. (2022). A roadmap to doing culturally grounded developmental science [Unpublished manuscript].CrossRefGoogle Scholar
Brouwers, S. A., van Hemert, D. A., Breugelmans, S. M., & van de Vijver, F. J. R. (2004). A historical analysis of empirical studies published in the Journal of Cross-Cultural Psychology 1970–2004. Journal of Cross-Cultural Psychology, 35(3), 251–262. https://doi.org/10.1177%2F0022022104264121CrossRefGoogle Scholar
Burton, M., & Kagan, C. (2005). Liberation social psychology: Learning from Latin America. Journal of Community & Applied Social Psychology, 15(1), 63–78. https://doi.org/10.1002/casp.786CrossRefGoogle Scholar
Cha, E.-S., Kim, K. H., & Erlen, J. A. (2007). Translation of scales in cross-cultural research: Issues and techniques. Journal of Advanced Nursing, 58(4), 386–395. https://doi.org/10.1111/j.1365-2648.2007.04242.xCrossRefGoogle ScholarPubMed
Cheon, B. K., Melani, I., & Hong, Y. (2020). How USA-centric is psychology? An archival study of implicit assumptions of generalizability of findings to human nature based on origins of study samples. Social Psychological and Personality Science, 11(7), 928–937. https://doi.org/10.1177/1948550620927269CrossRefGoogle Scholar
Cheung, F. M., van de Vijver, F. J. R., & Leong, F. T. L. (2011). Toward a new approach to the study of personality in culture. American Psychologist, 66(7), 593–603. https://psycnet.apa.org/doi/10.1037/a0022389CrossRefGoogle Scholar
Cheung, G. W., & Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. Journal of Cross-Cultural Psychology, 31(2), 187–212. https://doi.org/10.1177/0022022100031002003CrossRefGoogle Scholar
Clancy, K. B. H., & Davis, J. L. (2019). Soylent is people, and WEIRD is white: Biological anthropology, whiteness, and the limits of WEIRD. Annual Review of Anthropology, 48, 169–186. https://doi.org/10.1146/annurev-anthro-102218-01133CrossRefGoogle Scholar
Clark, M. J. (2012). Cross-cultural research, challenge and competence. International Journal of Nursing Practice, 18(s2), 28–37. https://doi.org/10.1111/j.1440-172X.2012.02026.xCrossRefGoogle ScholarPubMed
Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., & Billiet, J. (2014). Measurement equivalence in cross-national research. Annual Review of Sociology, 40, 55–75. https://doi.org/10.1146/annurev-soc-071913-043137CrossRefGoogle Scholar
Davies, S. E. H. (2020). The introduction of research ethics review procedures at a university in South Africa: Review outcomes of a social science research ethics committee. Research Ethics, 16(1–2), 1–26. https://doi.org/10.1177/1747016119898408CrossRefGoogle Scholar
Diener, E., & Diener, M. (1995). Cross-cultural correlates of life satisfaction and self-esteem. Journal of Personality and Social Psychology, 68, 653–663. https://doi.org/10.1037/0022-3514.68.4.653CrossRefGoogle ScholarPubMed
Douglas, S. P., & Craig, C. S. (2007). Collaborative and iterative translation: An alternative approach to back translation. Journal of International Marketing, 15(1), 30–43. https://doi.org/10.1509%2Fjimk.15.1.030CrossRefGoogle Scholar
Dulin, M. F., Tapp, H., Smith, H. A., Urquieta de Hernandez, B., & Furuseth, O. J. (2011). A community based participatory approach to improving health in a Hispanic population. Implementation Science, 6, Article 38. https://doi.org/10.1186/1748-5908-6-38CrossRefGoogle Scholar
Ebert, T., Gebauer, J. E., Brenner, T., Bleidorn, W., Gosling, S. D., Potter, J., & Rentfrow, P. J. (2022). Are regional differences in psychological characteristics and their correlates robust? Applying spatial-analysis techniques to examine regional variation in personality. Perspectives on Psychological Science, 17(2), 407–441. https://doi.org/10.1177/1745691621998326CrossRefGoogle ScholarPubMed
Ember, C. R., & Ember, M. (2009). Cross-Cultural Research Methods, 2nd ed. Altamira Press.Google Scholar
Fischer, R., & Fontaine, J. R. J. (2011). Methods for investigating structural equivalence. In Matsumoto, & van de Vijver, (pp. 179–215).Google Scholar
Fischer, R., & Poortinga, Y. H. (2018). Addressing methodological challenges in culture-comparative research. Journal of Cross-Cultural Psychology, 49(5), 691–712. https://doi.org/10.1177/0022022117738086CrossRefGoogle Scholar
Fontaine, J. R. J., & Fischer, R. (2011). Data analytic approaches for investigating isomorphism between the individual-level and the cultural-level internal structure. In Matsumoto, & van de Vijver, (pp. 273–298).Google Scholar
Foulks, E. F. (1989). Misalliances in the Barrow Alcohol Study. American Indian and Alaska Native Mental Health Research, 2(3), 7–17.CrossRefGoogle ScholarPubMed
Fraser, G. (2018). Evaluating inclusive gender identity measures for use in quantitative psychology research. Psychology & Sexuality, 9(4), 343–357. https://doi.org/10.1080/19419899.2018.1497693CrossRefGoogle Scholar
Freitag, M., & Bauer, P. C. (2013). Testing for measurement equivalence in surveys: Dimensions of social trust across cultural contexts. Public Opinion Quarterly, 77(S1), 24–44. https://doi.org/10.1093/poq/nfs064CrossRefGoogle Scholar
Friesen, P., Kearns, L., Redman, B., & Caplan, A. L. (2017). Rethinking the Belmont Report? American Journal of Bioethics, 17, 15–21. https://doi.org/10.1080/15265161.2017.1329482CrossRefGoogle ScholarPubMed
Gaines, S. O. Jr., Marelich, W. D., Bledsoe, K. L., Steers, W. N., Henderson, M. C., Granrose, C. S., et al. (1997). Links between race/ethnicity and cultural values as mediated by racial/ethnic identity and moderated by gender. Journal of Personality and Social Psychology, 72(6), 1460–1476. https://doi.org/10.1037//0022-3514.72.6.1460CrossRefGoogle Scholar
Gjorgjioska, M. A., & Tomicic, A. (2019) The crisis in social psychology under neoliberalism: Reflections from social representations theory. Journal of Social Issues, 75, 169–188. https://doi.org/10.5964/jspp.v1i1.97CrossRefGoogle Scholar
Gurven, M. D., & Lieberman, D. E. (2020). WEIRD bodies: Mismatch, medicine and missing diversity. Evolution and Human Behavior, 41(5), 330–340. https://doi.org/10.1016/j.evolhumbehav.2020.04.001CrossRefGoogle ScholarPubMed
Halttunen, L. (2017). Whose culture? Monolithic cultures and subcultures in early childhood settings. Journal of Early Childhood Research, 15(1), 73–82. https://doi.org/10.1177%2F1476718X15579742CrossRefGoogle Scholar
Hambleton, R. K., & Zenisky, A. L. (2011). Translating and adapting tests for cross-cultural assessments. In Matsumoto, & van de Vijver, (pp. 46–70).Google Scholar
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2–3), 61–83. https://doi.org/10.1017/S0140525X0999152XCrossRefGoogle ScholarPubMed
Hofstede, G. (1994). Cultures and Organizations: Software of the Mind. HarperCollins.Google Scholar
Hofstede, G. (2011). Dimensionalizing cultures: The Hofstede Model in context. Online Readings in Psychology and Culture, 2(1). https://doi.org/10.9707/2307-0919.1014CrossRefGoogle Scholar
Hollins, E. A. (1990). Debunking the myth of a monolithic White American culture; or, moving toward cultural inclusion. American Behavioral Scientist, 34(2), 201–209. https://doi.org/10.1177%2F0002764290034002008CrossRefGoogle Scholar
Hruschka, D. J. (2020). “What we look with” is as important as “what we look at”. Evolution and Human Behavior, 41(5), 458–459. https://doi.org/10.1016/j.evolhumbehav.2020.07.011CrossRefGoogle Scholar
Hruschka, D. J., Munira, S., Jesmin, K., Hackman, J., & Tiokhin, L. (2018). Learning from failures of protocol in cross-cultural research. Proceedings of the National Academy of Sciences, 115(45), 11428–11434. https://doi.org/10.1073/pnas.1721166115CrossRefGoogle ScholarPubMed
Johnson, T. P., Shavitt, S., & Holbrook, A. L. (2011). Survey response styles across cultures. In Matsumoto, & van de Vijver, (pp. 130–175).Google Scholar
Kağıtçıbaşı, Ç. (1996). Family and Human Development across Cultures: A View from the Other Side. Psychology Press.Google Scholar
Kankaraš, M., & Moors, G. (2010). Researching measurement equivalence in cross-cultural studies. Psihologija, 43(2), 121–136.CrossRefGoogle Scholar
Klausner, S., & Foulks, E. (1979). Alcohol and the Future of Ukpiagvik. Center for Research on the Acts of Man.Google Scholar
Klausner, S., & Foulks, E. (1980). Social Change and the Alcohol Problem in the Alaskan North Slope. Center for Research on the Acts of Man.Google Scholar
Klein, V., Savas, Ö., & Conley, T. D. (2022). How WEIRD and androcentric is sex research? Global inequities in study populations. Journal of Sex Research, 59(7), 810–817. https://doi.org/10.1080/00224499.2021.1918050CrossRefGoogle ScholarPubMed
Klitzman, R. (2011). How local IRBs view central IRBs in the US. BMC Medical Ethics, 12, Article 13. https://doi.org/10.1186/1472-6939-12-13CrossRefGoogle ScholarPubMed
Klitzman, R. (2012). US IRBs confronting research in the developing world. Developing World Bioethics, 12(2), 63–73. https://doi.org/10.1111/j.1471-8847.2012.00324.xCrossRefGoogle ScholarPubMed
Klitzman, R., Pivovarova, E., Murray, A., Appelbaum, P. S., Stiles, D. F., & Lidz, C. W. (2019). Local knowledge and single IRBs for multisite studies: Challenges and solutions. Ethics & Human Research, 41(1), 22–31. https://doi.org/10.1002/eahr.500003CrossRefGoogle ScholarPubMed
Ledgerwood, A., da Silva Frost, A., Kadirvel, S., Maitner, A. T., Wang, Y. A., & Maddox, K. B. (in press). Methods for advancing an open, replicable, and inclusive science of social cognition. Chapter to appear in Hugenberg, K., Johnson, K., & Carlston, D. E. (eds.), Oxford Handbook of Social Cognition. Oxford University Press.Google Scholar
Ledgerwood, A., Hudson, S. T. J., Lewis, N. A., Maddox, K. B., Pickett, C. L., Remedios, J. D., et al. (2022). The pandemic as a portal: Reimagining psychological science as truly open and inclusive. Perspectives on Psychological Science, 17(4), 937–959. https://doi.org/10.1177/17456916211036654CrossRefGoogle ScholarPubMed
Leong, F. T. L., & Lyons, B. (2010). Ethical challenges for cross-cultural research conducted by psychologists from the United States. Ethics & Behavior, 20(3–4), 250–264. https://doi.org/10.1080/10508421003798984CrossRefGoogle Scholar
Liu, C., Cox, R. B. Jr., Washburn, I. J., Croff, J. M., & Crethar, H. C. (2017). The effects of requiring parental consent for research on adolescents’ risk behaviors: A meta-analysis. Journal of Adolescent Health, 61(1), 45–52. https://doi.org/10.1016/j.jadohealth.2017.01.015CrossRefGoogle ScholarPubMed
Maitner, A. T., DeCoster, J., Andersson, P. A., Eriksson, K., Sherbaji, S., Giner-Sorolla, R., et al. (2022). Perceptions of emotional functionality: Similarities and differences among dignity, face, and honor cultures. Journal of Cross-Cultural Psychology, 53(3–4), 263–288. https://doi.org/10.1177/00220221211065108CrossRefGoogle Scholar
Matsumoto, D., Kim, J. J., Grissom, R. J., & Dinnel, D. L. (2011). Effect sizes in cross-cultural research. In Matsumoto, & van de Vijver, (pp. 244–272).Google Scholar
Matsumoto, D., & van de Vijver, F. J. R. (eds.). (2011). Cross-Cultural Research Methods in Psychology. Cambridge University Press.Google Scholar
Metro, R. (2014). From the form to the face to face: IRBs, ethnographic researchers, and human subjects translate consent. Anthropology & Education Quarterly, 45(2), 167–184. https://doi.org/10.1111/aeq.12057CrossRefGoogle Scholar
Morren, M., Gelissen, J. P. T. M., & Vermunt, J. K. (2012). Response strategies and response styles in cross-cultural surveys. Cross-Cultural Research, 46(3), 255–279. https://doi.org/10.1177%2F1069397112440939CrossRefGoogle Scholar
National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. (1979). The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research. www.hhs.gov/ohrp/regulations-and-policy/belmont-reportGoogle Scholar
Neil, M., & Saenz, C. (2020). Advancing research ethics systems in Latin America and the Caribbean: A path for other LMICs? The Lancet: Global Health, 8(1), E23–E24. https://doi.org/10.1016/S2214-109X(19)30441-3Google ScholarPubMed
Newson, M., Buhrmester, M., Xygalatas, D., & Whitehouse, H. (2021). Go WILD, not WEIRD. Journal for the Cognitive Science of Religion, 6(1–2), 80–106. https://doi.org/10.1558/jcsr.38413CrossRefGoogle Scholar
Nezlek, J. B. (2011). Multilevel modeling and cross-cultural research. In Matsumoto, & van de Vijver, (pp. 299–345).Google Scholar
Oishi, S., Diener, E. F., Lucas, R. E., & Suh, E. M. (1999). Cross-cultural variations in predictors of life satisfaction: Perspectives from needs and values. Personality and Social Psychology Bulletin, 25, 980–990. https://doi.org/10.1177/01461672992511006CrossRefGoogle Scholar
Ozolins, U., Hale, S., Cheng, X., Hyatt, A., & Schofield, P. (2020). Translation and back-translation methodology in health research – a critique. Expert Review of Pharmacoeconomics & Outcomes Research, 20(1), 69–77. https://doi.org/10.1080/14737167.2020.1734453CrossRefGoogle ScholarPubMed
Perry, K. H. (2011). Ethics, vulnerability, and speakers of other languages: How university IRBs (do not) speak to research involving refugee participants. Qualitative Inquiry, 17(10), 899–912. https://doi.org/10.1177%2F1077800411425006CrossRefGoogle Scholar
Pratt, B., Van, C., Cong, Y., Rashid, H., Kumar, N., Ahmad, A., et al. (2014). Perspectives from South and East Asia on clinical and research ethics: A literature review. Journal of Empirical Research on Human Research Ethics, 9(2), 52–67. https://doi.org/10.1525/jer.2014.9.2.52CrossRefGoogle Scholar
Presidential Commission for the Study of Bioethical Issues. (2011). Research across Borders: Proceedings of the International Research Panel of the Presidential Commission for the Study of Bioethical Issues.Google Scholar
Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004CrossRefGoogle ScholarPubMed
Rad, M. S., Martingano, A. J., & Ginges, J. (2018). Toward a psychology of Homo sapiens: Making psychological science more representative of the human population. Proceedings of the National Academy of Sciences, 115, 11401–11405. https://doi.org/10.1073/pnas.1721165115CrossRefGoogle Scholar
Rashwan, B., & Jenkins, J. C. (2017). Fatalism and revolution: Expanding our understanding of fatalism during a unique political opening in Egypt. Journal of North African Studies, 22(4), 645–664. https://doi.org/10.1080/13629387.2017.1316716CrossRefGoogle Scholar
Rhodes, S. D., Daniel, J., & Alonzo, J. (2012). A systematic community-based participatory approach to refining an evidence-based community-level intervention: The HOLA intervention for Latino men who have sex with men. Health Promotion Practice, 14(4), 604–616. https://doi.org/10.1177%2F1524839912462391Google ScholarPubMed
Rosling, H., Rosling, O., & Rönnlund, A. R. (2018). Factfulness. Flatiron Books.Google Scholar
Ross, C. E., & Mirowsky, J. (1984). Socially-desirable response and acquiescence in a cross-cultural survey of mental health. Journal of Health and Social Behavior, 25(2), 189–197. https://doi.org/10.2307/2136668CrossRefGoogle Scholar
Saab, R., Ayanian, A., & Hawi, D. (2020). The status of Arabic social psychology: A review of 21st-century research articles. Social Psychological and Personality Science, 11, 917–927. https://doi.org/10.1177/1948550620925224CrossRefGoogle Scholar
Schroeder, D., Chatfield, K., Singh, M., Chennells, R., & Herissone-Kelly, P. (2019). Equitable Research Partnerships: A Global Code of Conduct to Counter Ethics Dumping. Springer.CrossRefGoogle Scholar
Shweder, R. A. (1990). Cultural psychology: What is it? In Stigler, J. E., Shweder, R. A., & Herdt, G. (eds.), Cultural psychology: Essays on Comparative Human Development (pp. 1–43). Cambridge University Press.Google Scholar
Sireci, S. G. (2011). Evaluating test and survey items for bias across languages and cultures. In Matsumoto, & van de Vijver, (pp. 216–240).Google Scholar
Skewes, M. C., & Lewis, J. P. (2016). Sobriety and alcohol use among rural Alaska Native elders. International Journal of Circumpolar Health, 75(1), Article 30476. https://doi.org/10.3402/ijch.v75.30476CrossRefGoogle ScholarPubMed
Smith, P. B. (2002). Levels of analysis in cross-cultural psychology. Online Readings in Psychology and Culture, 2(2). https://doi.org/10.9707/2307-0919.1018CrossRefGoogle Scholar
Smith, P. B., & Fischer, R. (2008). Acquiescence, extreme response bias and culture: A multilevel analysis. In van de Vijver, F. J. R., Van Hemert, D. A., & Poortinga, Y. H. (eds.), Multilevel Analysis of Individuals and Cultures (pp. 285–314). Lawrence Erlbaum Associates.Google Scholar
Smith, P. B., Vignoles, V. L., Becker, M., Owe, E., Easterbrook, M. J., Brown, R., et al. (2016). Individual and culture-level components of survey response styles: A multi-level analysis using cultural models of selfhood. International Journal of Psychology, 51(6), 453–463. https://doi.org/10.1002/ijop.12293CrossRefGoogle Scholar
Solberg, B., & Eikemo, H. (2021, January 11). Passive consent for passive participation? Tidsskr Nor Legeforen, 141. https://doi.org/10.4045/tidsskr.20.0654Google ScholarPubMed
Son, J. (2018). Back translation as a documentation tool. Translation & Interpreting, 10(2), 89–100.CrossRefGoogle Scholar
Spence, S., White, M., Adamson, A. J., & Matthews, J. N. S. (2015). Does the use of passive or active consent affect consent or completion rates, or dietary data quality? Repeat cross-sectional survey among school children aged 11–12 years. BMJ Open, 5(1), Article e006457. https://doi.org/10.1136/bmjopen-2014-006457CrossRefGoogle ScholarPubMed
Spencer-Oatey, H. (2008). Culturally speaking: Culture, Communication and Politeness Theory. Continuum.Google Scholar
Ssali, A., Poland, F., & Seeley, J. (2016). Exploring informed consent in HIV clinical trials: A case study in Uganda. Heliyon, 2(11), Article e00196. https://doi.org/10.1016/j.heliyon.2016.e00196CrossRefGoogle ScholarPubMed
Steltenpohl, C. N., Montilla Doble, L. J., Basnight-Brown, D. M., Dutra, N. B., Belaus, A., Kung, C.-C., et al. (2021). Society for the Improvement of Psychological Science Global Engagement Task Force Report. Collabra: Psychology, 7(1). https://doi.org/10.1525/collabra.22968Google Scholar
Sun, H. (2011). On cultural differences and translation methods. Journal of Language Teaching and Research, 2(1), 160–163. https://doi.org/10.4304/jltr.2.1.160-163CrossRefGoogle Scholar
Tay, L., Newman, D. A., & Vermunt, J. K. (2011). Using mixed-method item response theory with covariates (MM-IRT-C) to ascertain observed and unobserved measurement equivalence. Organizational Research Methods, 14(1), 147–176. https://doi.org/10.1177%2F1094428110366037CrossRefGoogle Scholar
Thalmayer, A. G., Toscanelli, C., & Arnett, J. J. (2021). The neglected 95% revisited: Is American psychology becoming less American? American Psychologist, 76(1), 116–129. https://psycnet.apa.org/doi/10.1037/amp0000622CrossRefGoogle ScholarPubMed
Tiokhin, L., Hackman, J., Munira, S., Jesmin, K., & Hruschka, D. (2019). Generalizability is not optional: Insights from a cross-cultural study of social discounting. Royal Society Open Science, 6(2), 181386. https://doi.org/10.1098/rsos.181386CrossRefGoogle Scholar
Tremblay, M.-C., Martin, D. H., McComber, A. M., McGregor, A., & Macaulay, A. C. (2018). Understanding community-based participatory research through a social movement framework: A case study of the Kahnawake School Diabetes Prevention Project. BMC Public Health, 18, 487. https://doi.org/10.1186/s12889-018-5412-yCrossRefGoogle ScholarPubMed
Tucker, B. (2017). From risk and time preferences to cultural models of causality: On the challenges and possibilities of field experiments, with examples from rural southwestern Madagascar. Nebraska Symposium on Motivation, 64, 61–114.CrossRefGoogle ScholarPubMed
Tusino, S., & Furfaro, M. (2021). Rethinking the role of research ethics committees in the lift of Regulation (EU) No 536/2014 on clinical trials and the COVID-19 pandemic. British Journal of Clinical Pharmacology, 88(1), 40–46. https://doi.org/10.1111/bcp.14871CrossRefGoogle Scholar
van de Vijver, F. J. R., & Leung, K. (2011). Equivalence and bias: A review of concepts, models, and data analytic procedures. In Matsumoto, & van de Vijver, (pp. 17–45).Google Scholar
van de Vijver, F. J. R., & Tanzer, N. K. (2004). Bias and equivalence in cross-cultural assessment: An overview. European Review of Applied Psychology / Revue Européenne de Psychologie Appliquée, 54(2), 119–135. https://doi.org/10.1016/j.erap.2003.12.004CrossRefGoogle Scholar
Van Hemert, D. A. (2011). Cross-cultural meta-analysis. In Matsumoto, & van de Vijver, (pp. 348–378).Google Scholar
Vargas, J. H., & Kemmelmeier, M. (2013). Ethnicity and contemporary American culture: A meta-analytic investigation of horizontal–vertical individualism–collectivism. Journal of Cross-Cultural Psychology, 44(2), 195–222. https://doi.org/10.1177/0022022112443733CrossRefGoogle Scholar
Welzel, C., & Inglehart, R. F. (2016). Misconceptions of measurement equivalence: Time for a paradigm shift. Comparative Political Studies, 49(8), 1068–1094. https://doi.org/10.1177%2F0010414016628275CrossRefGoogle Scholar
Wexler, L. (2011). Intergenerational dialogue exchange and action: Introducing a community-based participatory approach to connect youths, adults, and elders in an Alaska Native community. International Journal of Qualitative Methods, 10(3), 248–264. https://doi.org/10.1177%2F160940691101000305CrossRefGoogle Scholar
White, M. T. (1999). Guidelines for IRB review of international collaborative medical research: A proposal. Journal of Law, Medicine & Ethics, 27(1), 87–94. https://doi.org/10.1111/j.1748-720X.1999.tb01440.xCrossRefGoogle ScholarPubMed
Wong, J. P-H., & Poon, M. K-L. (2010). Bringing translation out of the shadows: Translation as an issue of methodological significance in cross-cultural qualitative research. Journal of Transcultural Nursing, 21(2), 151–158. https://doi.org/10.1177%2F1043659609357637CrossRefGoogle Scholar
Wundt, W. (1900). Elemente der Völkerpsychologie / Elements of Folk Psychology, trans. Schaub, E. L.. George Allen & Unwin.Google Scholar

References

Anguera, M. T., Blanco-Villaseñor, A., Losada, J. L., Sánchez-Algarra, P., & Onwuegbuzie, A. J. (2018). Revisiting the difference between mixed methods and multimethods: Is it all in the name? Quality & Quantity, 52(6), 2757–2770. https://doi.org/10.1007/s11135-018-0700-2CrossRefGoogle Scholar
Bazeley, P. (2018). Integrating Analyses in Mixed Methods Research. SAGE Publications.CrossRefGoogle Scholar
Bazeley, P., & Kemp, L. (2012). Mosaics, triangles, and DNA: Metaphors for integrated analysis in mixed methods research. Journal of Mixed Methods Research, 6, 55–72. https://doi.org/10.1177/1558689811419514CrossRefGoogle Scholar
Bryman, A. (2006). Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6(1), 97–113. https://doi.org/10.1177/1468794106058877CrossRefGoogle Scholar
Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8–22. https://doi.org/10.1177/2345678906290531CrossRefGoogle Scholar
Bryman, A. (2008). Why do researchers integrate/combine/mesh/blend/mix/merge/fuse quantitative and qualitative research? In Bergman, M. M. (ed.), Advances in Mixed Methods Research (pp. 87–100). SAGE Publications. https://doi.org/10.4135/9780857024329.d9Google Scholar
Clark, A. (2005). Ways of seeing: Using the Mosaic approach to listen to young children’s perspectives. In Clark, A., Kjørholt, A. T., & Moss, P. (eds.), Beyond Listening: Children’s Perspectives on Early Childhood Services (pp. 29–49). Policy Press.Google Scholar
Clark, A., & Moss, P. (2005). Spaces to Play: More Listening to Young Children Using the Mosaic Approach. National Children’s Bureau.Google Scholar
Creamer, E. G. (2018). An Introduction to Fully Integrated Mixed Methods Research. SAGE Publications.CrossRefGoogle Scholar
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and Conducting Mixed Methods Research, 3rd ed. SAGE Publications.Google Scholar
Fetters, M. D., & Molina-Azorin, J. F. (2017). The journal of mixed methods research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research, 11(3), 291–307. https://doi.org/10.1177/1558689817714066CrossRefGoogle Scholar
Finlay, W. M. L., Rohleder, P., Taylor, N., & Culfear, H. (2015). “Understanding” as a practical issue in sexual health education for people with intellectual disabilities: A study using two qualitative methods. Health Psychology, 34(4), 328–338. https://doi.org/10.1037/hea0000128CrossRefGoogle ScholarPubMed
Flick, U. (2018). Doing Triangulation and Mixed Methods. SAGE Publications.CrossRefGoogle Scholar
Greene, J. C. (2008). Is mixed methods social inquiry a distinctive methodology? Journal of Mixed Methods Research, 2(1), 7–22. https://doi.org/10.1177/1558689807309969CrossRefGoogle Scholar
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274. https://doi.org/10.2307/1163620CrossRefGoogle Scholar
Guest, G. (2013). Describing mixed methods research: An alternative to typologies. Journal of Mixed Methods Research, 7(2), 141–151. https://doi.org/10.1177/1558689812461179CrossRefGoogle Scholar
Guetterman, T. C., & Fetters, M. D. (2022). Data visualization in the context of integrated analyses. In Hitchcock, J. H. & Onwuegbuzie, A. J. (eds.), The Routledge Handbook for Advancing Integration in Mixed Methods Research (pp. 301–323). Routledge. https://doi.org/10.4324/9780429432828-6CrossRefGoogle Scholar
Ivankova, N. V. (2015). Mixed Methods Applications in Action Research: From Methods to Community Action. SAGE Publications.CrossRefGoogle Scholar
Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3–20. https://doi.org/10.1177/1525822x05282260CrossRefGoogle Scholar
Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133. https://doi.org/10.1177/1558689806298224CrossRefGoogle Scholar
Lee, Y.-J., & Greene, J. (2007). The predictive validity of an ESL placement test: A mixed methods approach. Journal of Mixed Methods Research, 1(4), 366–389. https://doi.org/10.1177/1558689807306148CrossRefGoogle Scholar
Maxwell, J. A. (2013). Qualitative Research Design: An Interactive Approach, 3rd ed. SAGE Publications.Google Scholar
Maxwell, J. A., & Loomis, D. M. (2003). Mixed methods design: An alternative approach. In Tashakkori, A. & Teddlie, C. (eds.), Handbook of Mixed Methods in Social & Behavioral Research (pp. 241–271). SAGE Publications.Google Scholar
Maxwell, J. A., & Mittapalli, K. (2010). Realism as a stance for mixed methods research. In Tashakkori, A. & Teddlie, C. (eds.), SAGE Handbook of Mixed Methods in Social & Behavioral Research, 2nd ed. (pp. 145–167). SAGE Publications.CrossRefGoogle Scholar
Mertens, D. M. (2018). Mixed Methods Design in Evaluation. SAGE Publications.CrossRefGoogle Scholar
Mertens, D. M., Bazeley, P., Bowleg, L., Fielding, N., Maxwell, J., Molina-Azorin, J. F., & Niglas, K. (2016). The Future of Mixed Methods: A Five Year Projection to 2020. Mixed Methods International Research Association task force report. https://mmira.wildapricot.org/resources/Documents/MMIRA%20task%20force%20report%20Jan2016%20final.pdfGoogle Scholar
Morse, J. M., Bowers, B. J., Charmaz, K., Clarke, A. E., Corbin, J., & Porr, C. J. (eds.). (2021). Developing Grounded Theory: The Second Generation Revisited, 2nd ed. Routledge.CrossRefGoogle Scholar
Morse, J. M., & Niehaus, L. (2009). Mixed Method Design: Principles and Procedures. Left Coast Press.Google Scholar
Moseholm, E., & Fetters, M. D. (2017). Conceptual models to guide integration during analysis in convergent mixed methods studies. Methodological Innovations, 10(2), 1–11. https://doi.org/10.1177/2059799117703118CrossRefGoogle Scholar
Nzabonimpa, J. P. (2018). Quantitizing and qualitizing (im-)possibilities in mixed methods research. Methodological Innovations, 11(2), 1–16. https://doi.org/10.1177/2059799118789021CrossRefGoogle Scholar
Sandelowski, M., Voils, C. I., & Knafl, G. (2009). On quantitizing. Journal of Mixed Methods Research, 3(3), 208–222. https://doi.org/10.1177/1558689809334210CrossRefGoogle ScholarPubMed
Schoonenboom, J. (2018). Mixed methods in early childhood education. In Fleer, M. & van Oers, B. (eds.), International Handbook of Early Childhood Education (pp. 269–293). Springer. https://doi.org/10.1007/978-94-024-0927-7_11CrossRefGoogle Scholar
Schoonenboom, J. (2022). Developing the meta-inference in mixed methods research through successive integration of claims. In Hitchcock, J. H. & Onwuegbuzie, A. J. (eds.), The Routledge Handbook for Advancing Integration in Mixed Methods Research (pp. 55–70). Routledge. https://doi.org/10.4324/9780429432828-6CrossRefGoogle Scholar
Schoonenboom, J. (2023). Ten mixed methods integration strategies for obtaining a detailed understanding. In Tierney, R., Rizvi, F., Ercikan, K., & Smith, G. (eds.), International Encyclopedia of Education, 4th ed. Elsevier. https://doi.org/10.1016/B978-0-12-818630-5.11045-0Google Scholar
Schoonenboom, J., & Johnson, R. B. (2017). How to construct a mixed methods research design. Kölner Zeitschrift für Soziologie und Sozialpsychologie, 69(2), 107–131. https://doi.org/10.1007/s11577-017-0454-1CrossRefGoogle ScholarPubMed
Schoonenboom, J., & Johnson, R. B. (2021). The case comparison table: A joint display for constructing and sorting simple tables as mixed analysis. In Onwuegbuzie, A. J. & Johnson, R. B. (eds.), The Routledge Reviewer’s Guide to Mixed Methods Analysis (pp. 277–288). Routledge. https://doi.org/10.4324/9780203729434-24CrossRefGoogle Scholar
Schoonenboom, J., Johnson, R. B., & Froehlich, D. E. (2018). Combining multiple purposes of mixing within a mixed methods research design. International Journal of Multiple Research Approaches, 10(1), 271–282. https://doi.org/10.29034/ijmra.v10n1a17CrossRefGoogle Scholar
Tashakkori, A. M., Johnson, R. B., & Teddlie, C. B. (2021). Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences, 2nd rev. ed. SAGE Publications.Google Scholar
Teddlie, C. B., & Tashakkori, A. (2009). Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. SAGE Publications.Google Scholar
Uprichard, E., & Dawney, L. (2019). Data diffraction: Challenging data integration in mixed methods research. Journal of Mixed Methods Research, 13(1), 19–32. https://doi.org/10.1177/1558689816674650CrossRefGoogle ScholarPubMed
Visser, L., Korthagen, F. A., & Schoonenboom, J. (2018). Differences in learning characteristics between students with high, average, and low levels of academic procrastination: Students’ views on factors influencing their learning. Frontiers in Psychology, 9(808). https://doi.org/10.3389/fpsyg.2018.00808CrossRefGoogle ScholarPubMed
Yin, R. K. (2006). Mixed methods research: Are the methods genuinely integrated or merely parallel? Research in the Schools, 13(1), 41–47.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×