Future-proofing I-O psychology: The need for updated graduate curriculum

Hyland (2023) provides a model for reflection and reflexivity to prevent industrial-organizational (I-O) psychology research from growing stale. Our focus is to expand upon Hyland’s model by first reflecting on the recent sociohistorical forces that have shaped I-O psychology and then by proactively future-proofing our field through graduate education focused on transparency, software accessibility, and multidisciplinary collaboration. Recent history has seen an upsurge of unprecedented macro events such as COVID-19, nationwide racial division, political unrest, and mental health crisis; these events make us aware of blind spots within our societal, scientific, and economical systems. Such events force us as a field to be reactive and adaptive by transitioning from old methods to new and developing methods (e.g., work shifting from in-person to online). However, as humans, we tend to cling to what is familiar and comfortable, and likewise, our field has often chosen to remain comfortable (Kruglanski, 2001). We believe that the proclivity to resist change results in an overreliance on outdated practices and to combat this, we suggest a grassroots approach to transformation by focusing on future-proofing graduate coursework. In line with the Society of Industrial Organizational Psychology’s (SIOP) strategic goals, we envision a future that equips future generations of researchers and practitioners with the skills and knowledge to be lifelong learners, so they are prepared for ever-changing challenges.

We suggest updating the I-O graduate course curriculum by (a) implementing open science practices throughout courses, (b) embracing the latest open-source coding technologies (e.g., R and Python), and (c) advancing inferential inclusivity by teaching Bayesian statistics in addition to traditional methods. This three-pronged approach addresses the need for transparency, software accessibility, and multidisciplinary research to prepare graduate students to theorize, plan appropriate study design, thoughtfully consider necessary analyses, interpret meaningful results, and share those results in a clear and far-reaching manner. Researchers can then prepare for (rather than react to) unprecedented macro events, clarifying our collective identity and future-proofing the field with an updated skill set to overcome obstacles.

Updating curriculum

Open science

Open science is a collaborative network to future-proof our field by promoting transparency, replicability, and accessibility. Open science has many benefits, such as reducing or resolving questionable research practices, promoting collaboration and data sharing, increasing the accessibility of nonpublished research, and extending the public’s trust in science. Some actions to promote open science include preregistration of quantitative and qualitative studies (which can reduce questionable research practices such as p hacking), sharing code and data (which can increase research replicability; Hagger, 2021), and contributing to online archives for nonpublished studies (Aguinis et al., 2020). Additionally, open science has the potential to enable us to verify the exact procedures and justification for the analysis while also increasing the reliability of results.

Our field has wrestled with the file drawer problem (i.e., the threat of bias within empirical literature due to “null effect” studies not being disseminated; Card, 2010); however, using open science practices allows nonpublished or “null-effect” studies to be uploaded and shared online, which promotes the publication of the findings regardless of the statistical effect (Allen & Mehler, 2019). Furthermore, open science allows for increased accessibility to other datasets on platforms such as Open Science Framework (OSF; osf.io). Requiring graduate students to preregister studies and share research plans, data, and code via the OSF as part of coursework helps build a culture of open science and promotes multidisciplinary research by facilitating collaboration with other disciplines and researchers. Graduate students can benefit from learning the values of open science while also gaining practical experience by practicing in courses. Ultimately, open science practices make our work more credible, reliable, and collaborative. We envision the future of I-O research to be a more accessible and collaborative process while increasing the trust in the field as a science. The first step is to provide graduate students with tangible opportunities to engage with open science practices.

Open-source software

Despite the acknowledgment that R is a highly capable, flexible, and free software that could become the “default” statistical program for I-O researchers, many undergraduate programs, graduate programs, and research labs still use paid-license alternatives like SPSS, SAS, Stata, MPlus, and so on as their go-to software for training and use. Those options certainly have unique strengths and weaknesses for specific projects or processes, so exposure to multiple statistical programs should be encouraged. However, I-O psychology departments can work to future-proof statistical software by implementing more comprehensive coursework and training in open-source software like R, Jamovi, JASP, and/or Python. R is a particularly appealing software due to its variety of shared libraries and searchable code. Instruction in these languages as distinct topics, rather than as a “means to an end” (i.e., to complete research projects or course assignments within I-O), should shift students’ views on coding and data analysis more broadly too (i.e., show its applicability to jobs, research, and hobbies outside of I-O).

Improved graduate training in open-source software future-proofs the field in multiple regards. First, greater fluency in coding provides the next generation of researchers with the skill set to adapt more quickly to constant changes in technology. Second, collectively training the next generation of researchers in the same software can make collaboration and scrutinization of research more efficient. Finally, the resulting shift in the field to free (or low-cost) software options like R, Jamovi, or JASP could make methodological processes more accessible to a broader population in the future. In other words, a more diverse group of academics, practitioners, students, and members of the public can learn and implement the field’s newest methodological techniques regardless of their financial resources or institutional affiliation.

Bayesian statistics

As pointed out by Hyland, overreliance on null hypothesis significance testing (NHST) may be limiting our ability to theorize and examine research questions. As mentioned above, graduate statistics rely heavily upon the frequentist paradigm of statistical thinking and NHST. However, the Bayesian statistical paradigm offers several benefits that have yet to be fully appreciated in I-O psychology such as providing direct support for any model, incorporating prior empirical evidence, and representing uncertainty with probability distributions. More recently, statistical software, such as JASP (JASP Team, 2021) and R, has greatly improved the accessibility of Bayesian analysis. Along with the technological gaps, most I-O psychologists are entrenched in frequentist philosophical thinking and are unable to provide Bayesian courses to students.

Bayesian methods provide additional benefits that can improve graduate statistical reasoning and analysis. First, Bayesian methods view probability as a degree of belief and represent uncertainty in probabilistic distributions that are often more thoughtful representations of events. This contrasts with the dichotomous, all-or-nothing thinking of p values used in NHST where we are limited to “rejecting the null” or “failing to reject the null hypothesis” (Hoekstra et al., 2006; Rosnow & Rosenthal, 1989). Second, a Bayesian analysis can support the finding of no difference and even compare multiple alternative models through Bayes factors that tell us how much better one model is at explaining the observed data relative to another model. One way to facilitate a transition to Bayesian thinking might be to teach null hypothesis Bayesian testing to graduate students in addition to or as an alternative to NHST so that students can quantify results and support null findings (e.g., Dienes, 2014). Third, a Bayesian analysis allows for the inclusion of known information to inform the data analysis (e.g., data from a meta-analysis or possible parameter values). Not only is this beneficial for explicitly modeling assumptions and prior beliefs, but it also is beneficial for teaching graduate students how to handle smaller samples. Oftentimes datasets with small samples are not used due to an increase in Type II error (i.e., failing to reject the null when it is false), but in the Bayesian framework, we can supplement small samples with carefully selected prior distributions (e.g., van de Schoot et al., 2015).

There is a growing consensus among various disciplines that we need to train graduate students to be successful in their statistical reasoning. Taking a multidisciplinary approach by providing Bayesian statistics courses would future-proof statistical inference by equipping graduate students with another statistical tool that has many potential advantages for I-O psychology.

Conclusion

As we take time to be reflexive and to reflect, we all must consider the future that we desire. We envision a field that is more transparent and collaborative through open science, accessible and advanced through open-source software, and statistically sound and multidisciplinary through Bayesian statistics. Though not exhaustive, implementing these tools creates a culture of implementing the most updated practices that will keep us ahead of the curve in the future. Enthusiastic consideration of these suggestions by the I-O community will bring in a fresh perspective, reinvigorating the field, clarifying our collective identity, and promoting continued relevancy for years to come.