Hostname: page-component-cd9895bd7-8ctnn Total loading time: 0 Render date: 2024-12-19T04:46:21.917Z Has data issue: false hasContentIssue false

Open science and epistemic pluralism: A tale of many perils and some opportunities

Published online by Cambridge University Press:  27 January 2023

Andrea Bazzoli*
Affiliation:
Washington State University Vancouver, Vancouver, WA, USA
Rights & Permissions [Opens in a new window]

Abstract

Type
Commentaries
Copyright
© The Author(s), 2023. Published by Cambridge University Press on behalf of the Society for Industrial and Organizational Psychology

Broadly, open science can be defined as “transparent and accessible knowledge that is shared and developed through collaborative networks” (Vicente-Saez & Martinez-Fuentes, Reference Vicente-Saez and Martinez-Fuentes2018, p. 434). Hence, it refers to a broad range of practices aimed at detecting scientific fraud and enhancing transparency and replicability of research. In their focal article, Guzzo et al., (Reference Guzzo, Schneider and Nalbantian2022) highlighted several tensions between these values and applied research in organizations. In this commentary, we develop a slightly different argument: the open science movement, as a direct offspring of (post)positivist research paradigmsFootnote 1, has the potential to stifle epistemological and scientific pluralism and reproduce historical scientific hierarchies it purports to redress. In doing so, we distinguish between the spirit of open science (i.e., promoting participation, transparency, and access to science) and its implementations (e.g., OSF badges, TOP guidelines, and multi-laboratory research, but also sexist attacks on social media and podcasts by other scholars in the field [e.g., the Twitter pile-on in November 2021 regarding Roxanne Felig and her coauthors’ paper], and a general disregard of feminist epistemologies; Brabeck, Reference Brabeck2021). In the first part of this commentary, we focus on open science’s ideals and examine a few unstated assumptions, advancing a set of equally valid assumptions based on constructionist thought, and then we discuss how unchecked implementations of open science practices can marginalize scholars that do not subscribe to its epistemic premises. We conclude with a few thoughts to improve the open science movement.

Taken-for-granted assumptions

Undeclared and unexamined assumptions that are, by definition, self-evident to a community can be seen as impositions by other communities that do not share them (Romaioli & McNamee, Reference Romaioli and McNamee2021). In turn, a given community and its members will defend as reasonable arguments that logically descend from those assumptions against competing communities. We suggest that it is not qualitative methods, per se, that are at odds with open science principles (as Guzzo et al., Reference Guzzo, Schneider and Nalbantian2022, and Pratt, Kaplan, & Whittington, Reference Pratt, Kaplan and Whittington2020, among others, argued), but the universalistic and all-encompassing claims that are tied to open science are incompatible with epistemic stances most often embraced by qualitative scholars.

Consider the following excerpt from one of the first articles advocating for open science in psychology, which defined our discipline as hinging on “testing statistical hypotheses using empirical observations and data” (Shrout & Rodgers, Reference Shrout and Rodgers2018, p. 487). Industrial–organizational (I-O) psychologists went a step further and argued that open science principles are the hallmark of a “robust” science and I-O psychologists “must actively pursue the tenants of open science” (McAbee et al., Reference McAbee, Grubbs and Zickar2018, p. 59). However, these seemingly innocuous statements encompass several assumptions on what science is and what it is not. The primacy of numerical data and quantitative methods, large samples, successful replications as a measure of scientific merit, and testing pre-registered hypotheses are foundational of open science and are hailed as a solution to improve research quality for the entirety of I-O psychology field and broader psychological science (Banks et al., Reference Banks, Field, Oswald, O’Boyle, Landis, Rupp and Rogelberg2019). We advance that this is position is disingenuous, as it further marginalizes critical organizational psychologists and constructivist research, for which we should instead create space (Bazzoli & Probst, Reference Bazzoli and Probst2022a,b). Further, we should characterize open science for what it actually is: a response to a set of challenges raised for the most part by positivist and quantitative psychologists.

For example, constructionist epistemologies are not concerned with replicability of findings at all; rather, borrowing a page from the book of American Pragmatism (e.g., Dewey, Reference Dewey and Thayer1982), “what is true is what works,” in the sense that forms of knowledge are assessed on their ability to meet their stated goals with a focus on the consequences that sharing that type of knowledge will have on participants, their communities, society as a whole, and other scientific disciplines. Similarly, the social reality we inhabit is not necessarily objective and measurable and can be instead understood as the by-product of negotiations people carry out with others and the environment, considering historical, local, and cultural contexts. Following Wittgenstein (Reference Wittgenstein1953), when a community defines a “reality,” its members are bound to a cultural and historical tradition that informs how they know that reality through their shared negotiations, norms, and symbols.

Given this, organizational theories need not be evaluated only on whether their predictions hold true to empirical investigations and formal statistical testing. A theory’s generative potential can also be explored (Gergen, Reference Gergen1978), that is, its capability to challenge assumptions of culture, raise questions relevant to social life, and eventually generate alternatives for social action. Last, every form of knowledge is a social construction, included—indubitably—critical and constructionist perspectives; therefore, open science is not necessarily better science, it is just another way of doing science and approaching psychological phenomena. This point, however, has not yet been recognized by open science advocates, who have uncritically adopted a postpositivist framework and a realist ontology.

Perils

Principles of open science are already embedded in certain funding mechanisms; for instance, open access publication of data (with very limited exceptions), protocols, and the article itself are mandated for projects funded by the Horizon Europe (2021–2027) framework. Furthermore, there is a movement, especially in Northern Europe, that aims to link adherence to open science practices to tenure and promotion. Those policies are not value neutral and can be understood as manifestations of funding agencies’ deep-seated values, paradigmatic assumptions, and beliefs. A slavish adoption of these principles will lead to an exclusionary and hostile notion of what counts as science, what is deemed valuable, and what should be done in the future to tackle societal challenges. In short, by imposing and implementing a set of universalistic principles that legitimize the cultural authority of the dominant postpositivist ontology and epistemology, open science serves as a tool to reinforce postpositivism’s hegemonic position in I-O psychology and beyond, to the detriment of other epistemologies.

Rewarding open science practices is a sensible stance to enhance rigor for some quantitative organizational research, but it poses challenges for other types of research. Others argued that core open science tenets might be unethical (Brabeck, Reference Brabeck2021), especially when doing research with vulnerable and minoritized people (e.g., women victim of abuse, undocumented immigrants) and might discourage altogether other types of research. Action research and ethnographies might disappear as they completely reject the idea of testing hypotheses and the presence of a bias-free, independent researcher, which are requirements of many open science practices. These methods instead focus on doing research with participants, not on them. Similarly, constructionist research approaches insist on researcher’s reflexivity (McNamee & Hosking, Reference McNamee and Hosking2012), which is severely limited in favor of following pre-registered protocols.

As noted earlier, unchecked implementation of these principles will further marginalize scholars doing research that does not fit positivist epistemologies. This marginalization will look like exclusion from funding opportunities, limited career opportunities, and adverse reputational consequences. On a more theoretical note, such an uncritical implementation is a threat to the pluralism of scientific research, narrowing the available discourses and methods to analyze relevant organizational phenomena. Unfortunately, certain qualitative researchers have already begun embracing methodological simplicity to produce mechanistic accounts of social life to fit the dominant epistemology (Donmoyer, Reference Donmoyer2012).

A way forward?

Considering open science not as valid in itself, but as a tool that can influence—in a powerful way—psychological science is key to this commentary. We resist open science’s contention to be a gold standard for all psychological research, as evaluating a way of doing research using criteria that are foreign to its own epistemic premises or using assumptions that are derived from a competing epistemology is meaningless, given that epistemic systems are incommensurable. Each research tradition would in fact emphasize some aspects over other because they better align with their stated (and unstated) values and beliefs. In other words, there is no birds’ eye when it comes to theories and paradigms as we cannot step out of our own linguistic systems. Instead, our aim is to encourage scientists to reflect and clarify what “open” and “science” mean and suggest ways for scholars to reflect on the wider impact that their research can have.

On the one hand, coming to terms that today’s open science movement is not a movement for all psychologists and it only seems to work for European and North American quantitatively oriented researchers (e.g., South African researchers resisted sharing their data because of high income countries’ well-known history of exploitation; Traynor & Foster, Reference Traynor and Foster2017) is of particular importance considering the recent expectation that all social scientists engage in open science practices can easily be misunderstood by those tasked with evaluating research and decide what lines of research deserve funding, recognition, and ultimately what counts as good science. On the other hand, we would like to suggest one useful (we hope) reflection about how constructionist discourses can complement the open science movement. Disclosing researchers’ own positionality seems a sensible topic to include under the umbrella of transparency: a reflection on how research questions and data were generated and how results were interpreted could be published alongside the paper, following the lead of Nosek et al., (Reference Nosek, Alter, Banks, Borsboom, Bowman, Breckler and Yarkoni2015). In other words, we encourage scholars to analyze the boundaries within which a paper has been produced, acknowledging that those boundaries are not value-free, and were shaped by researcher’s environment and beliefs. Doing so, we believe, will enhance research transparency by uncovering whose experiences are being valued and investigated as well as what assumptions are being made. In the end, we are not claiming that epistemic assumptions that are foundational of open science principles are ill conceived or unreasonable; we are instead asking whether they are always and universally useful, concluding that perhaps they are not, and warning that scientific endeavors that purport to be value-free can be dangerous in certain circumstances.

Acknowledgements

We have no known conflicts of interest to disclose.

Footnotes

1 Throughout this commentary, we use (post)positivism and constructionism following Guba and Lincoln (Reference Guba, Lincoln, Denzin and Lincoln1994). Postpositivism can be defined as a research paradigm holding that a knowable, tangible, and measurable reality exists (i.e., naïve or critical realism), knowledge claims about this reality can be developed objectively, and verification/falsification of a priori hypotheses is the most prevalent methodological choice. Conversely, constructionism developed in the latter part of the 20th century with the sociological work of Berger and Luckmann (Reference Berger and Luckmann1966). Since then, it has developed into a scholarly movement that holds that reality is the result of communicative processes that create a sense of shared reality (i.e., it is locally co-constructed), emphasizes that objectivity is also co-constructed through communicative processes, and aims to examine taken-for-granted realities that might be oppressive or dysfunctional through future-forming, dialogic, hermeneutical, and dialectical research to generate new functional realities.

References

Banks, G. C., Field, J. G., Oswald, F. L., O’Boyle, E. H., Landis, R. S., Rupp, D. E., & Rogelberg, S. G. (2019). Answers to 18 questions about open science practices. Journal of Business and Psychology, 34(3), 257270. https://doi.org/10.1007/s10869-018-9547-8 CrossRefGoogle Scholar
Bazzoli, A., & Probst, T. M. (2022a). Taking stock and moving forward: A textual statistics approach to synthesizing four decades of job insecurity research. Organizational Psychology Review (Advance online publication). https://doi.org/10.1177/20413866221112386 CrossRefGoogle Scholar
Bazzoli, A., & Probst, T. M. (2022b). Vulnerable workers in insecure jobs: A critical meta-synthesis of qualitative findings. Applied Psychology: An International Review (Advance online publication). https://doi.org/10.1111/apps.12415 Google Scholar
Berger, P. L., & Luckmann, T. (1966). The social construction of reality: A treatise in the sociology of knowledge. Anchor Books.Google Scholar
Brabeck, M. M. (2021). Open science and feminist ethics: Promises and challenges of open access. Psychology of Women Quarterly, 45, 457474. https://doi.org/10.1177/03616843211030926 CrossRefGoogle Scholar
Dewey, J. (1982). The unit of behavior (The reflex arc concept in psychology). In Thayer, H. (Ed.), Pragmatism: The classic writings (pp. 262274). Hackett.Google Scholar
Donmoyer, R. (2012). Two (very) different worlds: The cultures of policymaking and qualitative research. Qualitative Inquiry, 18, 798807. https://doi.org/10.1177/1077800412453128 CrossRefGoogle Scholar
Gergen, K. J. (1978). Toward generative theory. Journal of Personality and Social Psychology, 36, 13441360. https://doi.org/10.1037/0022-3514.36.11.1344 CrossRefGoogle Scholar
Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In Denzin, N. K. & Lincoln, Y. S. (Eds.), Handbook of qualitative research (pp. 105117). Sage.Google Scholar
Guzzo, R. A., Schneider, B., & Nalbantian, H. R. (2022). Open science, closed doors: The perils and potential of open science for research in practice. Industrial and Organizational Psychology: Perspectives on Science and Practice, 15, 495515. CrossRefGoogle Scholar
McAbee, S. T., Grubbs, J. B., & Zickar, M. J. (2018). Open science is robust science. Industrial and Organizational Psychology: Perspectives on Science and Practice, 11, 5461. https://doi.org/10.1017/iop.2017.85 CrossRefGoogle Scholar
McNamee, S., & Hosking, D. M. (2012). Research and social change: A relational constructionist approach. Routledge. CrossRefGoogle Scholar
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., … & Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 14221425. https://doi.org/10.1126/science.aab2374 CrossRefGoogle ScholarPubMed
Pratt, M. G., Kaplan, S., & Whittington, R. (2020). Editorial essay: The tumult over transparency: Decoupling transparency from replication in establishing trustworthy qualitative research. Administrative Science Quarterly, 65(1), 119. https://doi.org/10.1057/s41599-019-340-8 Google Scholar
Romaioli, D., & McNamee, S. (2021). (Mis)constructing social construction: Answering the critiques. Theory & Psychology, 31, 315334. https://doi.org.107.171/0779/5093595345342302909667757 Google Scholar
Shrout, P. E., Rodgers, J. L. (2018). Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual Review of Psychology, 69, 487510. https://doi.org/10.1146/annurev-psych-122216-011845 CrossRefGoogle ScholarPubMed
Traynor, C. & Foster, L. (2017). Principles and practice in open science: Addressing power and inequality through “situated openness.” Natural Justice. https://naturaljustice.org/principles-practice-open-science-addressing-power-inequality-situated-openness/ Google Scholar
Vicente-Saez, R., & Martinez-Fuentes, C. (2018). Open science now: A systematic literature review for an integrated definition. Journal of Business Research, 88, 428436. https://doi.org/10.1016/j.jbusres.2017.12.043 Google Scholar
Wittgenstein, L. (1953). Philosophical investigations. Macmillan.Google Scholar