Introducing Our Associate Editor for Reproducibility, Dr. Alan Farahani, Editorial Board Member
This year we are introducing a new option for American Antiquity (AAQ) authors of accepted articles: to have your quantitative or data-rich research officially (and computationally) reproduced and for its successful reproduction to be publicly recognized. The proposal to have a board member take on the responsibilities of this initiative was discussed extensively by the entire AAQ Editorial Board, and it was unanimously approved. Dr. Alan Farahani (founder, Sci-Scope Solutions) has agreed to serve as the first Associate Editor for Reproducibility (AER). He has extensive experience in computational methods and data analysis in archaeology and anthropology. He will hold this position for at least a year; we see this as a rotating position that taps others with the expertise and commitment to data transparency.
The initiative is entirely optional and is available only for AAQ papers that are accepted for publication. After acceptance, authors can voluntarily submit their free- and open-source code and dependent datasets for any number of specified figures or analyses. The AER will then attempt to reproduce those specific figures and analyses over a set number of days. Those analyses that do reproduce will be acknowledged via a “reproducibility statement.” This statement will include the names of the individuals responsible for the creation of the analyses and figures in the following format: “The Associate Editor of Reproducibility downloaded all materials relevant to Figures 4 and 6, and the analyses presented in Table 2, and was able to reproduce the results as presented by the authors.”
These data and analyses will then be published alongside the article as supplemental information, unless authors have already made them available through an accessible repository. Analyses that have data that cannot be shared for ethical or community-based reasons are still eligible for review, except for the final step of data publication.
If the results do not reproduce, the AER will work with the author(s) over a set timeframe to identify why the analyses/figures could not be replicated and to address any issues before publication. No paper will be rejected because of a failure to reproduce unless significant problems with the data are identified.
What Are the Benefits of Instituting an Associate Editor for Reproducibility?
One of the primary reasons for this new position is to facilitate approaches that foster transparency and integrity in quantitative research that journals across the social (Lindsay Reference Lindsay2023), natural (Powers and Hampton Reference Powers and Hampton2018), and physical (Van de Lindt and Narasimhan Reference Van de Lindt and Narasimhan2024) sciences are now embracing. Many researchers across disciplines ranging from social psychology to cancer biology have noted that there is a “crisis of reproducibility” (Baker Reference Baker2016) in that many highly cited research results have failed to reproduce, which, following the 2019 National Research Council Consensus Study, means “obtaining consistent computational results using the same input data, computational steps, methods, code, and conditions of analysis” (National Academies of Sciences, Engineering, and Medicine 2019:1). This failure may be due to a combination of several factors: mundane errors, specific choices made during data analysis (Oza Reference Oza2023), or even intentional data fabrication (Simonsohn et al. Reference Simonsohn, Simmons and Nelson2021). Regardless of the causes, the inability for other researchers to reproduce results, whether due to inaccessible data (Stodden et al. Reference Stodden, Seiler and Ma2018) or other reasons, undermines peer and stakeholder trust in quantitative and empirical archaeological research. Transparency in our analyses reaffirms our collective commitment to making the research enterprise a community effort that depends on the cumulative aggregation of knowledge shared by our peers.
We emphasize again that the option for a reproducibility review is optional. But its benefits are many! In lengthy multiauthored publications, any author or authors who have taken the time to make their data and analyses reproducible will be acknowledged. It is, in effect, not only a public statement of the very hard work that a team has done to make its results reproducible but also a demonstration to our research community of the importance of the accessibility of these routines and data for the archaeological (and scientific) process and a commitment to transparent empirical research conducted with integrity.
This initiative is part of the AAQ mission to be more inclusive (see Editor's Corner, vol. 89, no. 1). It acknowledges that there are many kinds of archaeologies and archaeologists, including those who value sharing their data and creating reproducible analyses. This aligns with a greater emphasis on making archaeological data FAIR: findable, accessible, interoperable, and reusable (Nicholson et al. Reference Nicholson, Kansa, Gupta and Fernandez2023). It will enable our research community to have the tools and resources to address large-scale questions that are dependent on a variety of datasets. Finally, it furthers the call for “open science” in archaeology (Marwick et al. Reference Marwick, d'Alpoim Guedes, Barton, Bates, Baxter, Bevan and Bollwerk2017), by celebrating, via acknowledgment, all those individuals (Marwick Reference Marwick2022) and institutions (Begley et al. Reference Begley, Buchan and Dirnagl2015) committed to it.
