Open scholarship in applied linguistics: What, why, and how

1. Introduction

In November 2021, the UNESCO Recommendation on Open Science was adopted by the UNESCO General Conference. This marks a significant milestone in the development of Open Science as the new norm for research, with intended influences on national laws and practices of the 193 UN Member States and far-reaching implications beyond the UN. It is against such a background that Open Applied Linguistics (openappliedlinguistics.org), a new Research Network affiliated with the International Association of Applied Linguistics (AILA), was established in early 2022 and a two-day online symposium on open scholarship in applied linguistics was held in June 2022.

Organized by Meng Liu and Sin Wang Chong, convenors of Open Applied Linguistics, the symposium was the inaugural event of the network and was sponsored by Cambridge University Press & Assessment. There were 19 sessions by 31 speakers and presenters across career stages, covering a wide range of topics. While our symposium included speakers from neighbouring fields such as psychology, due to limited space, we selected six sessions that directly concentrated on applied linguistics to include in this report.

2. What do we mean by open scholarship?

Open scholarship is the theme of our symposium. Specifically, we follow the UNESCO (2021) definition with our own adaptations:

Open [scholarship] is defined as an inclusive construct that combines various movements and practices aiming to make multilingual [scholarly] knowledge openly available, accessible, and reusable for everyone; to increase [scholarly] collaborations and sharing of information for the benefits of [scholarship] and society; and to open the processes of [scholarly] knowledge creation, evaluation, and communication to societal actors beyond the traditional [scholarly] community. It comprises all [scholarly] disciplines and aspects of scholarly practices, including basic and applied sciences, natural and social sciences and the humanities, and it builds on the following key pillars: open [scholarly] knowledge, open [scholarship] infrastructures, research communication, open engagement of societal actors, and open dialogue with other knowledge systems. (p. 7)

While we mostly use ‘open scholarship’, ‘open research’, and ‘open science’ interchangeably, we deliberately replaced the term ‘science’ with ‘scholarship’ in the above definition to highlight our aim to promote a more inclusive and critical understanding of open science. Readers familiar with the history of open science in psychology know that it has traditionally been associated with quantitative research and it was only recently that alternative, equally valid epistemologies have received more attention in this space (see Bennett, 2021, for example, for a discussion on open science from the perspective of feminist psychology). A recent survey within applied linguistics (Liu & De Cat, in press) also revealed some concerns from qualitative researchers on open science having unduly quantitative connotations. Nonetheless, both the UNESCO recommendations and our symposium demonstrate that open science can be and should be understood as more inclusive. By using open scholarship, a term less commonly used than open science, we hope to inspire curiosity, mitigate misconceptions, and facilitate a more inclusive understanding of open scholarship/research/science (henceforth OS).

3. Selected talks and panels from the symposium

3.1 Open research practices: Value versus sustainability

Marsden and Bolibaugh's session focused on open research practices. Marsden started the session with the rationales for and the value of OS. Rationales for OS can be philosophical, social, and scientific, ranging from epistemic responsibility and social equity to enhanced rigour, reliability, and validity. Within applied linguistics, there have been several initiatives to promote OS. IRIS (Instruments and Data for Research in Language Studies; Marsden et al., 2016), an open repository for instruments and data established in 2011, is one of the first initiatives to promote open materials and data, and soon IRIS will increase its scope from a focus on second languages to all language-related studies. The Open Accessible Summaries in Language Studies (OASIS; Marsden et al., 2018a) is another exemplary initiative for research accessibility. While much progress has been made and journals are continuing to join the initiative, there are persisting challenges in terms of sustainability (e.g., active engagement from journal editors, cost of infrastructures, time required for data preparation). It is also important to acknowledge that epistemology may affect the rationales for OS and its potential benefits – some epistemologies and methodologies are not necessarily compatible with, for example, replication and fully anonymized data sharing.

In Part II of this session, Bolibaugh zoomed in on open data. Reiterating the rationales for open data, she called for a move from the verification-focused approach (Open Research 1.0) to a more synthetic and collaborative approach (Open Research 2.0). In the case of open data, Open Research 2.0 would entail reusing data for purposes such as synthesis and reconceptualization, in addition to verification. Surveying the IRIS database, it was found that while the prevalence of open data is on the rise, the usability and completeness judged according to the FAIR (findable, accessible, interoperable, and reusable) principles are less than ideal. To move to Open Research 2.0, we still need consensus on what constitutes data for reproducing published findings, recognition for OS efforts at the institutional and award levels, training support, and funding for infrastructure.

To conclude, to keep applied linguistics at the forefront of OS, sustained commitment is needed from all levels in the community, which will understandably be a gradual process.

3.2 TESOLgraphics: Opening up scholarly communication

The session by Chong extended the notion of OS in applied linguistics to include practitioners. This session reported on an ongoing initiative, TESOLgraphics (tesolgraphics.com), which produces open-access, one-page infographic summaries of secondary research in all topics of TESOL (Teaching English to Speakers of Other Languages) to create opportunities for English teachers to engage with research and develop their evidence-based teaching practices.

Chong began the session by reviewing research on researcher-practitioner relationships in language education, highlighting that both researchers and practitioners are willing to engage in professional exchange. Chong discussed the usefulness of secondary research, in particular research syntheses (i.e., literature reviews conducted using systematic methodologies), in fostering the research-practice nexus (Chong, 2019; Chong & Plonsky, 2021).

Introducing the notion of ‘praxis’ from dialectical materialism (Poehner & Inbar-Lourie, 2020), Chong problematized the nature of current research-practice dialogues. Most research-practice activities are unidirectional, with researchers imparting knowledge to practitioners. This points to the need for two-way communication between researchers and practitioners in TESOL, forming an evidence ecosystem comprising producers, synthesisers, and consumers of evidence (Shepherd, 2014): while researchers offer empirical and theoretical insights into issues pertaining to teaching and learning, experience and pragmatic knowledge of practitioners evaluate ecological validity of research findings, making the focus of educational research more relevant to practitioners.

Responding to the need for more researcher-practitioner dialogues, TESOLgraphics will enter a new phase of development, inviting English teachers to produce teaching materials and resources (e.g., lesson plans) inspired by specific infographic summaries. Going alongside the teaching resources are short video clips where teachers designing the materials reflect on how research findings are translated into concrete teaching ideas.

3.3 Replication research in applied linguistics: What, why, and how

McManus’ talk focused on replication research in applied linguistics. Several new initiatives in our field point to an increased awareness about the need, place, and value of replication research in the growth and credibility of the discipline (e.g., methodological guides, journal special issues). Yet, as reviews have noted (e.g., Marsden et al., 2018b; Porte, 2012), support and resources are needed so that researchers can design, conduct, and report high quality replication studies. This session responded to that need by introducing novice and experienced researchers to the replication research process. The key topics introduced were: why replication studies are needed, what a replication study is, what distinguishes a replication study from other types of research studies, and how to go about designing a replication study in the field of applied linguistics (see McManus, 2021, in press-a, in press-b; Porte & McManus, 2019). A brief summary of this session's main parts is provided here.

To start, a replication repeats a study's methodology with or without changes followed by systematic comparison to better understand the nature, repeatability, and generalizability of its findings. Replication therefore aims to systematically reconsider, refine, extend, and sometimes limit previous research findings. Because replication is defined by comparison, replication researchers must systematically draw comparisons at each stage of the process (design, data handling, reporting), including using comparative language in reporting and planning how comparisons will be handled in the analyses. In addition, the more that we change in a replication, the more difficult comparison becomes. We should therefore aim to keep as much similarity between the studies as possible. Lastly, because replication is important for all empirical disciplines, clear labelling should be used throughout, including the use of ‘replication’ in the study's title and abstract.

3.4 Registered reports as an open research practice

In this talk, Morgan-Short discussed Registered Reports (RRs; Center for Open Science, n.d.; Chambers & Tzavella, 2022; Marsden et al., 2018c) as an open research practice that can improve the validity, reliability, and replicability of applied linguistics research. RRs is an article type in which authors motivate and design their study and submit it for peer review before data collection. If a study is well motivated and has a valid design protocol, it receives ‘In Principle Acceptance’, meaning that the journal agrees to publish the study after data collection regardless of the outcomes of the results, as long as the authors adhere to the planned protocol and make reasonable conclusions about the results. Upon receiving In Principle Acceptance, researchers preregister their protocol. Then, they collect and analyse their data and write their full manuscript, which is reviewed for adherence to the preregistered protocol and the reasonableness of the conclusion before being published.

Morgan-Short presented emerging metascience suggesting that RRs are effective in attenuating publication bias and are associated with higher computational reliability, validity, and increased replication efforts (for more information, see Chambers & Tzavella, 2022). In addition, common concerns about RRs were addressed. For example, exploratory analyses are not restricted in RRs as long as preregistered confirmatory analyses are reported as planned. Morgan-Short also discussed the concern that the publication time course of RRs is shifted because of the peer review process that occurs before data collection, which may be a particular challenge for researchers with inflexible deadlines. Finally, ideas, examples, and resources for moving forward with RRs were considered, especially the need for funding support and for more journals to offer RRs, following the lead of Language Learning, Bilingualism: Language and Cognition, and Second Language Research, where RRs are available as an article type. To conclude, Morgan-Short noted that the benefits of RRs often greatly outweigh the challenges and encouraged researchers to adopt this practice to bring about increased validity, reliability, and replicability to their own research and to our field.

3.5 Open science initiatives: The Postprint Pledge

The session by Al-Hoorie, Hiver, and Nosek was concerned with OS initiatives, specifically promoting open access in order to democratize scholarship, provide equitable access to researchers from the Global South, and remove obstacles to accessing the latest literature. According to the classic technology adoption lifecycle model (Rogers, 1995), a sociological framework describing the process of innovation acceptance, individuals vary in the speed at which they adopt innovations and initiatives. The first groups are innovators and early adopters. These are then followed by early majority and later majority, and finally by laggards. Building on this model, the session launched the ‘Postprint Pledge’, an initiative for applied linguists and second language acquisition researchers to share the ‘accepted versions’ of their manuscripts. According to the policies of journal publishers, authors are usually permitted to post their postprints online. Authors may not be aware that they still have the right to share the accepted versions of their manuscripts, believing that once they sign off the copyright agreement, they lose all rights to the manuscript.

As part of this initiative, and to make it more concrete for authors wishing to share the accepted versions of their manuscripts, a list of 60 applied linguistics journals was compiled and their copyright policies were reviewed through Sherpa Romeo, an online resource that aggregates the open access policies of various publishers. Analysis of the copyright agreements of these 60 journals showed that most publishers have no restrictions on authors sharing postprints, whether on their personal websites and on nonprofit repositories, though a minority of publishers require an embargo period of one year and in a few cases two years. The list detailing the copyright policies of each journal in relation to sharing postprints is currently hosted at ali-alhoorie.com/postprint-pledge, where interested researchers can see the list of signatories and pledge to share their postprints.

3.6 Speaking openly on open science: A panel discussion

The panel discussion by Plonsky, Winke, Huensch, and Hui sought to bring in a range of voices that might help us to reckon with some of the new ways of thinking and action that accompany a way forward for the field. There are many facets to embracing an ethos of OS. For the full range of benefits associated with OS to accrue, we as a culture and community need to shift not only our actions but our thinking as well. These adjustments, both major and minor, can be seen and felt in virtually all we do as scholars. The points of view that the panel shared included, among others, those of researcher, journal editor, manuscript reviewer, researcher trainer, grant writer, grant reviewer, graduate program director, mentor, and recent graduate.

In an event such as this one, instead of focusing on what OS can bring to our field, the panel spoke openly on the pushback that has been and continues to be observed in the face of OS initiatives. For instance, one commonly raised concern is the fear of being ‘scooped’ or having a result overturned via re-analysis. There may also be the hesitation to freely share something (e.g., data) that one has very painstakingly collected, cleaned, coded, and prepared. The panel argued that more of a community or synthetic mindset is needed and that it is an ethical duty to engage in practices that are best for the collective construction of knowledge rather than solely for career advancement.

In addition to deliberating on potentially thorny issues applied linguists all face such as data sharing, the panel also luxuriated in response to a prompt to dream about where the field might be in terms of OS ten years from now. Among other visions and hopes, the group imagined scholarly and institutional structures, incentives, workflow tools, and training protocols with an OS orientation baked in. Achieving consistency in code books (e.g., variable naming conventions) and in script annotation, for instance, present serious challenges to these dreams, but we see them as worthwhile if not necessary.

These discussions, though useful, can sometimes drift into the very conceptual and abstract. In order to reorient to more practical territory, the session was concluded by each panellist citing concrete steps toward embracing OS. These included, among others, posting materials and data on IRIS or OSF, writing an OASIS summary, identifying as part of the OS movement on personal websites, posting about OS practices on social media, taking the Postprint Pledge (see Section 3.5), and encouraging journal editors to integrate OS practices into their journals’ policies. Such a list of activities, though incomplete, may seem daunting. Consequently, the panel urged those who might feel overwhelmed to put aside the notion that OS is an all or nothing endeavour. The path that will lead to the greatest uptake in OS at both the field and individual levels and the one that will ultimately prove most effective for applied linguistics is one of incrementalism.

4. Conclusion

Our symposium addressed the what, why, and how of OS: Initiatives such as IRIS, OASIS, TESOLgraphics, and the PostPrint Pledge directly promote research accessibility and social equity, and they indirectly facilitate OS practices such as replication and registered reports, which, in turn, directly improve research validity, reliability, and reproducibility. A common thread across all is an ethos of openness, not merely in terms of research practices, but also in the open acknowledgement of and engagement with the many tensions and challenges regarding OS in applied linguistics, from epistemological and cultural to psychological and practical. This, we believe, is the ethos that can truly build us up as a field. While our symposium represents only one of the initial steps in a long journey, we hope to inspire more aligned and sustained efforts across all levels to incrementally and effectively advance our field towards a more open future.