Structural problems require structural solutions

Abstract

Chater & Loewenstein criticize behavioral scientists' reliance on individual-level (“i-frame”) analysis, observing that this impoverishes policy interventions and stymies scientific progress. We extend their analysis to argue that structural factors bias and perpetuate behavioral science toward the i-frame. Addressing this problem fully will require structural changes to the training, peer review, and granting structures that confront research scientists.

Chater & Loewenstein (C&L) have offered a useful criticism of behavioral scientists' reliance on individual-level (“i-frame”) analysis to investigate social problems, neglecting potential structural (“s-frame”) interventions. They focus on the example of nudges, but the problem is applicable to the behavioral sciences generally. For example, the emphasis on reducing of individual-level racial prejudice has masked the much broader potential impact of addressing racism at the systemic level (Paluck, Porat, Clark, & Green, 2021).

C&L provide an important first step in drawing attention to this critical problem. We suggest building on their analysis by applying the content of their insight to the target of their criticism: Behavioral scientists' reliance on i-frame analysis. The priorities and activities of researchers throughout the behavioral sciences are affected by the field's structures and the social system in which those structures are embedded, not just by intellectual mistakes made by researchers about which frameworks to apply to a subject matter. We argue that structural changes are necessary for making good on the analysis they present, because the tendencies toward i-frame research and policy interventions that C&L identify are ones that are selected for and incentivized by these institutions.

Behavioral science research is structurally biased in favor of i-frame research in several respects. A hierarchically organized system of higher education trains and credentials most behavioral scientists, and a likewise organized system of research funding organizes their research activities.

The way researchers are trained ensures that i-frame research remains the predominant way of approaching psychological questions. Most published research in the behavioral sciences focuses on individuals rather than structural relationships. This includes the subdisciplines of psychology that aim to make contact with public policy interventions, like behavioral economics and social psychology. Extant research furnishes material with which new researchers are taught. Moreover, early career researchers are trained using an apprenticeship model. Senior investigators train lab members junior to them, which influences trainees' intellectual development (Feldon et al., 2019).

Beyond the training process for new scientists, the incentive structures of peer review further entrench the i-frame. “Peer review” can be interpreted generously here: It refers not only to the process of evaluating work for publication in academic journals, but evaluating job candidates, grant proposals, tenure and promotion cases, and award-worthiness. All of these are a form of peer review, and as such subject to structural limitations.

At a minimum, peer review places considerable practical obstacles in the way of generating pathbreaking s-frame work. Reviewers will be more likely to recommend for publication papers that are on topics and hypotheses that they study and support (NCR, 2005); tenure and promotion are subject to the same logic. This lends an inherent conservatism to the peer review process (Kuhn, 1970).

This problem is reinforced by the steadily rising median age of National Institutes of Health (NIH) grant recipients over the past four decades, which suggests an increase in the difficulty of accessing these grants (Lauer, 2021; Lauer & Roychowdhury, 2021; National Research Council, 2005). Researchers respond to this heightened risk by proposing more conservative projects that reflect the status quo (Luukkonen, 2012; National Research Council, 2005).

The risks of the bias toward the i-frame in training and peer review go beyond the question of which ideas get financial support – they also constrict what ideas are generated (Stanford, 2019). Grant-making processes are not simply making pathbreaking work harder to get funded. They are also selecting against the time, effort, and intellectual habits that might be required to come up with novel research in the first place. The greater prevalence of i-frame studies and training methods makes i-frame ways of interrogating problems more practically available than s-frame approaches. This in turn makes s-frame studies costlier to pursue for those few who still generate s-frame research projects after i-frame heavy professional training.

We should expect all of these factors to be robust against even the most cogently argued target articles. Behavioral scientists will likely continue to run the studies that earn them career advancement and research funding, despite compelling arguments against the predominant frames of their research.

There is nothing inherent to human behavior that requires the behavioral sciences to focus on the individual. There are a number examples of s-frame research that the field can look to. These include social dominance theory (Pratto, Sidanius, & Levin, 2006; Sidanius & Pratto, 2001); how class and social power affect prosocial behavior (Keltner, Van Kleef, Chen, & Kraus, 2008; Kraus & Torrez, 2020); how basic decision-making processes are influenced by resource scarcity and social location (Farah & Hook, 2017; Morton, 2017; Morton & Paul, 2019). We believe this research should complement rather than replace i-frame research.

Building a field that is hospitable to more s-frame research requires, at a minimum, reconstruction of the field's training and reward structures. Existing theoretical and empirical work suggests alternatives that could address these structural problems. There have been longstanding calls for the reintegration of approaches like action research and community psychology into academic institutions, which could help train new researchers in ways that encourage s-frame analysis (Lykes, 2017; Simon & Wilder, 2018). Possible alternatives to preproduction peer review of articles include postpublication review (Heesen & Bright, 2021; Rowbottom, 2022), and possible alternative research funding models include the use of grant lotteries rather than evaluated submissions (Adam, 2019; Ahmed, 2019; Avin, 2019; Currie, 2019). Regardless of whether we adopt these particular interventions, we will need to implement structural-level solutions if we want to make good on the promise of C&L's analysis.