Where next for behavioral public policy?

Abstract

Our target article distinguishes between policy approaches that seek to address societal problems through intervention at the level of the individual (adopting the “i-frame”) and those that seek to change the system within which those individuals live (adopting the “s-frame”). We stress also that a long-standing tactic of corporations opposing systemic change is to promote the i-frame perspective, presumably hoping that i-frame interventions will be largely ineffective and more importantly will be seen by the public and some policy makers as a genuine alternative to systemic change. We worry that the i-frame focus of much of behavioral science has inadvertently reinforced this unhelpful focus on the individual. In this response to commentators, we identify common themes, build on the many constructive suggestions to extend our approach, and reply to concerns. We argue, along with several commentators, that a key role of behavioral public policy is to clarify how to build support for systemic reforms for which there is a broad consensus in the policy community, but which are opposed by powerful special interests.

Time spent arguing is, oddly enough, almost never wasted.

— Christopher Hitchens, Letters to a Young Contrarian

R1. Introduction

Academic papers are rarely born out of personal struggles. Our target article was. For two decades or so, the two of us have been trying to apply what we know about the science of human nature to real-world policy challenges. Our natural focus was on the individual – for example, on how to encourage people to lose weight, exercise more, take medications, save more, cut back on high-interest credit cards and pay-day loans, escape problem gambling, and to lead more environmentally sustainable lives. We began with great optimism, but found ourselves increasingly disillusioned, at the point where our presumed fund of behavioral insights made contact with policy-making reality. In some cases, our interventions simply didn't work; in others, they worked, but their effects seemed like proverbial drops in the bucket when compared to the problems they were intended to mitigate.

We wanted to understand why. Was it just our choice or execution of projects, or was there a larger problem with the direction of the field? And, if so, how might behavioral public policy most effectively move forward, to help change the world for the better? So we wrote this paper, deliberately targeting behavioral and brain sciences (BBS) for its interdisciplinary open-peer commentary, hoping both to clarify our thinking and to see how far our diagnosis and proposals resonate, or clash, with others in behavioral public policy and neighboring fields. We hoped to open a productive debate about the best role for the behavioral and social sciences in public policy. The breadth and quality of the commentaries convinces us that we have accomplished this goal.

Our commentators represent diverse viewpoints, some aligning with, and others unconvinced by, our analysis. But our overwhelming impression is one of common purpose and willingness constructively to debate how our field can move forward. Openness to diverse viewpoints has been crucial to the development of the behavioral sciences, whether in integrating economic theory, experimental psychology, and neuroscience, or in harnessing the power of large-scale field studies, big data, and machine learning. We need the same openness when it comes to public policy. We thank our insightful commentators for joining this debate. We have learned a lot in thinking through their arguments; and their insights will, we hope, be valuable for the behavioral public policy community at large.

Our target article reflects our concern that our own policy-related thinking, along with much of our field, has gone off track, and that, collectively, we need a rethink and a reset (see Lamberton for a valuable historical context). The two of us have become convinced that many of the growing number of problems faced by the United States, Britain, and other countries – including financial insecurity in retirement, climate change, obesity, gun violence, inequality, and many others – stem not from the limitations of individuals, but from misguided policies. We have also become concerned that “nudging” has become synonymous in the public eye, and even among some academics, with behavioral public policy. We made this point in an earlier paper (Loewenstein & Chater, 2017) and pursue it in our target article. In this response, we highlight the broader role that we believe behavioral science can and should play in public policy.

Some of our commentators take exception to our central arguments. They argue that our concerns are misplaced or overplayed (Hallsworth; Sunstein; Thaler), or agree with our diagnosis but argue that behavioral public policy faces more fundamental challenges (e.g., Edelman; Hertwig; Osman). We were gratified and even somewhat surprised by the much larger number of commentaries that expressed support for our central idea, albeit accompanied by constructive criticisms and insights (for example, Thomas, Kay & Laurin persuasively argue that an i-frame bias may be built into human moral psychology, which primarily focuses on praise and blame for individuals). Before taking up broader themes, some specific points are worth noting:

1. (1) Walton & Yeager rightly point out that our concern regarding the overemphasis on the individual has a long and distinguished history in social psychology and sociology, disciplines that have long recognized the primacy of the s-frame. Indeed, i-frame interventions in behavioral economics can seem radical and exciting precisely because they break away from such conventional (although, we believe, in retrospect, largely correct) wisdom.

2. (2) Several commentators point to inequality and discrimination as vitally important topics to which our analysis applies, but which we barely discuss. For example, He & Kang argue that “Inequality is not an individual-level issue, but rather a systemic problem that requires systemic solutions. Absent any other supporting systemic intervention, changing individual behaviors is unlikely to close inequality gaps; the systems in which individuals are nested must be fundamentally altered.” We couldn't have said it better! He & Kang point out that equity, diversity, and inclusion (EDI) has been dominated by the i-frame perspective (including, they note, a $8 billion/year diversity training industry), distracting, they argue, from addressing deep systemic challenges for organizations and government.¹ Relatedly, at a recent conference on gender inequality, a leading researcher on this topic, Lise Vesterlund, memorably said: “We don't need to fix the woman, we need to fix the system.” We agree wholeheartedly. And, referring specifically to economic inequality, Ruggeri, Tutuska, Ladini, Al-Zahli, Alexander, Andersen, Bibilouri, Chen, Doubravová, Dugué, Durrani, Dutra, Farrokhnia, Folke, Ge, Gomes, Gracheva, Grilc, Gürol, Heidenry, Hu, Krasner, Levin, Li, Messenger, Nilsson, Oberschulte, Obi, Pan, Park, Pelica, Pyrkowski, Rabanal, Ranc, Recek, Pascu, Symeonidou, Vdovic, Yuan, Garcia-Garzon, & Ashcroft-Jones (Ruggeri et al.) note that, given extreme existing levels of inequality in income and wealth, expecting most people to save for their own retirement is unrealistic. We agree, and discuss financial provision for retirement in detail below.²

3. (3) Although embracing Hertwig's critique of behavioral public policy's focus on human weakness as the cause of societal problems, we remain unconvinced that “boosts” – his proposed policy solution – are likely to have much impact on the problems we discuss. Boosts are the type of targeted education interventions that Kristal & Davidai, in their commentary, argue almost never change behavior. Although doubtful of the efficacy of educational interventions to change targeted behaviors, we see education in general as of huge importance: An educated polity is, almost certainly, one more likely to support, and vote for, wise policies.

4. (4) We also question Hertwig's claim that the widespread “rhetoric of irrationality” necessarily reinforces the presumption that people are primarily the authors of their own problems (as opposed to the systems within which they operate). This focus on irrationality does not, on its own, explain the focus on the i-frame. Indeed, one might equally well conclude that human irrationality argues for more rigid regulation, an argument used, largely uncontroversially, to justify limits on children's choices about whether to be educated, drink alcohol, smoke cigarettes, and so on.

5. (5) We agree with those commentators (e.g., Bingley, Haslam, Haslam, Hornsey, & Mols [Bingley et al.]; Caggiano, Constantino, Lees, Majumdar, & Weber [Caggiano et al.]; Mermelstein & Preston) who endorse our call for system-level change, but stress the importance of understanding the forces, including bottom-up community mobilization, that often drive such change. We agree that behavioral public policy should not be seen as a purely a top-down technocratic exercise of advising current policy makers, but also as informing the social movements and public debates providing the bottom-up impetus for real change.

6. (6) Further commentaries focus on how behavioral science can, and already has, helped understand systemic problems (Bowles; Ockenfels). We strongly endorse the emphasis by Ockenfels and Bowles on the crucial importance of behavioral mechanism design: Setting the “rules of the game” likely to produce the best outcomes must be based not on assuming that people can be treated as ideally rational individuals but on a realistic picture of human nature (Bolton & Ockenfels, 2012). Bowles outlines a helpful “mini-manifesto” for a behavioral science of s-frame reform. We believe that ideas of this kind are vital for preserving open debate and democratic institutions (cf., Bak-Coleman et al., 2021; Lessig, 2019; Stewart et al., 2019).

7. (7) A number of commentators (e.g., Brown; Heath; Johnson & Dana) are sympathetic to our central argument, but, aligning with arguments put forward by Oliver (2023) and Sugden (2017) have ethical qualms about our enthusiasm for s-frame reform, particularly regarding behaviors that primarily impact the individual (e.g., regarding health) rather than imposing social costs (e.g., regarding pollution). We agree that caution is required, but make two points in response. First, it is highly dangerous if behavioral science is deployed only by one side of a market transaction. Currently people are making decisions in environments (e.g., engineered processed foods; slot machines) that are deliberately and carefully designed to steer their “free choice” to ends that are good for companies and bad for individuals. To quote Tariq Fancy, the former Chief Investment Officer for sustainable investing at Blackrock, from an article describing Environmental, Social, and Governance (ESG) Investing as a “deadly distraction” (Fancy, 2021): “No ‘free market’ truly exists. A market economy is, at its core, a collection of rules. No rules mean no market. Nor is there one set of standard rules. Every rule, including corporate tax rates, patent protection and fines against pollution, is a deliberate decision that has an impact on the system. If a government changes the rules, we get different results – all of which can be defined as market outcomes. Changing rules is no more an ‘intervention on the free market’ than creating them in the first place.”

Second, we are not proposing that consumer protection should be imposed by fiat by a technocratic regulator. How we, as a society, are regulated, should be determined by the normal processes of democratic debate. There is a balance between consumer protection and individual freedom – and that balance should be set, we believe, by democratic consensus (uninfluenced, as far as possible, by corporate lobbying).

In the remainder of our response, we turn to broader questions raised in the diverse and insightful range of perspectives we received on our paper. We organize the points below on a number of discrete themes, discussing each in turn.

R2. Why do deep societal problems persist? Two competing perspectives

In our paper, we talk about individual- and system-oriented policies, drawing upon the widely used metaphor according to which social, economic, and political life is viewed as a game (or rather a complex system of interlocking games). The players of each game – citizens, politicians, corporations, think tanks, university researchers, governments, corporations, and many others – interact with one another, seeking to further their objectives, which might take any form, from complete selfishness, to the promotion of the interests of the group, to universal altruism (see Bowles, for an insightful discussion of why a single concept of utility may be insufficient to capture the diversity of such objectives). The outcome of any game depends both on the rules of the game itself (the focus of the s-frame), and how – and how well – the players can play according to those rules (the focus of the i-frame).³

Persistent societal problems arise when particularly crucial games “go wrong,” leading to outcomes such as climate change, inadequate financial provision for retirement, the obesity “epidemic,” plastic pollution, and the spiraling costs of health care. Substantial societal problems typically develop over decades and continue to grow despite widespread alarm, active programs of research, and highly motivated and informed groups campaigning for change.

Why do such problems persist? One possibility is that, although the players all genuinely want to fix the problem, they can't figure out, or agree on, what to do. More research, perhaps, is required. On this reading, the various players, despite their diverse concerns, would be willing to help to reduce carbon emissions, plastic pollution, inequality, or spiraling healthcare costs, if only they knew how. We believe that this “good faith” perspective is often implicit in policy thinking, including behavioral public policy.

But there is a second, very different explanation, which assumes that societal problems persist because of conflicts of interests between the players. What may seem to be deeply pathological outcomes of social and economic games may in fact be highly beneficial to some participants in the game. If solving a societal problem damages powerful and concentrated interests, these interests will work to block reform.

According to this conflictual perspective, the key obstacle to solving many of society's problems lies not with helping well-intentioned actors find better policies. Indeed, we suggest that the well-intentioned often know very well what to do, at least in general terms. What is lacking is the political influence and popular support to drive through reform, in the teeth of powerful and well-funded opposition. Smoking provides a relatively uncontroversial case. After early and compelling evidence about the harms of tobacco, it was clear that smoking needed to be reduced; and also relatively uncontroversial what measures (s-frame taxes and bans, combined with large and consistent public information campaigns) would be effective. But these measures (as well as the underlying science) were relentlessly contested by “big tobacco,” with their considerable financial and lobbying power. Indeed, tobacco companies continue to aggressively promote smoking in many countries around the world, even as profitable Western markets decline. Thus, more than 20% of the global population now smokes, and, according to the World Health Organization (WHO),⁴ half of these people will die of smoking-related diseases. As the case of smoking illustrates, where conflicts of interest are creating or maintaining major societal problems, the solution will typically require campaigning and building political coalitions to change the “rules of the game” so that the wishes of concentrated special interests do not prevail.

Johnson & Dana strongly endorse such a conflictual perspective, and review a substantial body of research corroborating our point that existing, concentrated, interests are often behind legislation and regulations that support their interests. They provide, however, a useful caution to our call for s-frame reform (see also Heath): “C&L are surely right that traditional regulations, whether through bans or incentives, will change behavior more than nudging. Yet a public choice analysis suggests that this is a reason for more, not less, caution in proposing regulation: Poor nudging can waste resources; poor regulation can lay waste to us all.” We agree, but stress that the poorest regulations of all are those that, as is so often the case, have been crafted by powerful interests opposed to change – the phenomenon of “regulatory capture” (e.g., Laffont & Tirole, 1991). It is therefore especially important to apply behavioral science to design, and build support for, better s-frame policies.

Some commentators take the conflictual perspective for granted, and stress the need for reforms in different areas, and at different scales, to address them (e.g., Bright, Parry, & Thoma [Bright et al.]; Edelman; Strohminger & Táíwò). Other commentators, rather to our surprise, downplay the conflictual origins of social problems. Indeed Sunstein, fresh from an earlier commentary accusing us of being “reactionaries” (Sunstein, 2022; for our response, Chater & Loewenstein, 2022) now makes the rather mystifying claim that our conflictual analysis can be dismissed as a “conspiracy theory.” Sunstein's charge would, if valid, apply to almost all academic studies of the political and policy-making process, where the conflictual analysis is taken for granted across the ideological spectrum, from Karl Marx (2004/1867) to Chicago School Economics (e.g., Becker, 1983; Stigler, 1971). Indeed, the conflictual perspective is entirely standard in fields such as political science, political economy, public health, climate policy, the sociology of science, and many more (Bartels, 2016; Brownell & Warner, 2009; Mann, 2021; Oreskes & Conway, 2011).⁵

The difference between the good faith versus conflictual perspective is crucial in understanding political debate concerning persistent social challenges – and the role of i- and s-frame interventions. The good faith perspective takes the superficial content of this debate at face value, accepting as genuine tobacco companies' expressed doubts that smoking kills, fossil-fuel companies' dismissal of the idea that greenhouse gases cause rising global temperatures, the gun lobby's questioning that the availability of assault weapons impacts the scale of mass shootings, and the insurance industry's view that the US model of private health insurance provides good value health care for the average citizen. The conflictual perspective suggests, instead, that for these and many other apparent debates about “the facts” are phony. The facts are widely and long agreed upon by any serious individual who doesn't have an economic stake in disbelieving them. Uncertainty and confusion arises less from genuine uncertainty than from deliberate obfuscation and disinformation from powerful interests that would be disadvantaged by s-frame reform.

It doesn't require a conspiratorial mindset to appreciate that many current policy debates regarding persistent social problems are similarly phony. Powerful interests benefitting from the status quo will, of course, continually attempt to raise doubts about the “quality of the evidence,” or will agitate endlessly for “more research” before action is taken (while often blocking the ability to carry out that research). But, according to the conflictual perspective, these are moves in an economic and political game – aimed at delaying s-frame reform.⁶ If persistent social problems typically arise from political log-jams caused by competing interests, better insights into how to change individuals' behaviors are likely to have only marginal impacts. From this perspective, behavioral science will contribute to better policy primarily by helping to overcome the special interests that block s-frame reform.

R3. The definitions of i-frame and s-frame

Some commentators (e.g., Hallsworth; Madva, Brownstein, & Kelly [Madva et al.]) question the clarity of our i-frame/s-frame distinction. For example, Hallsworth worries that “the distinction does not offer much clarity and holds up poorly under scrutiny.” By contrast, we suggest that the core distinction is simple, clear, and is indeed already deeply woven into the social sciences. S-frame interventions involve changing the rules of the game; i-frame interventions attempt to modify the actions of the players, within the existing rules. Distinguishing rules and actions is basic to seeing the social and economic world in terms of games at all: A game is, after all, simply an interaction between agents (players) governed by some set of rules.

Interestingly, libertarian paternalism (Sunstein & Thaler, 2003; see also Camerer, Issacharoff, Loewenstein, O'Donoghue, & Rabin, 2003, and Heath for insightful discussion) relies on the same distinction, but with a different slant. Here, changing the rules of the game (s-frame change) is viewed as infringing individual autonomy; “nudging,” or providing information, advice, or education (i-frame change), is viewed as preserving individual autonomy. Where we differ from libertarian paternalism is not on the i-frame/s-frame distinction, but on which approach should be prioritized when dealing with major social challenges.

The natural viewpoint of policy makers (and we suspect the general public) is that when games go wrong, we need to change the rules of the game or “system” (e.g., introducing regulations, subsidies, and taxes to decarbonize the economy). The libertarian paternalist perspective offers a superficially attractive “light-touch” alternative (e.g., for climate change, defaulting people into green electricity tariffs; or helping people compare their energy consumption with that of their neighbors). Our contention is that light-touch liberty-preserving alternatives are rarely a meaningful alternative to, and can distract from, much-needed rule changes. But more to the point here: The very concept of libertarian paternalism requires distinguishing between interventions that change the rules of the game (thus impacting individual autonomy); and “nudges” that don't change the rules but encourage players to respond differently.

Now any such discussion (whether from a libertarian perspective, or from our own) is relative to which game, and which players, we are focusing on. Banning displays of cigarettes in shops is an i-frame nudge if we focus on consumer choice (consumers can still buy cigarettes, but it is now harder); it is an s-frame nudge if we focus on the interaction between retailers and the legal system (the retailers aren't merely discouraged from displaying cigarettes – they are legally required not to do so). But whichever game is the focus, changing the rules of the game, rather than nudging the players, is likely to be more effective. Focusing on consumers, bans and taxes will be more impactful than nudges that make cigarettes less salient in shops. Focusing on retailers, banning cigarette displays outright will be more impactful than nudging them to make displays less salient. If any game is going persistently wrong, the policy maker's, and the public's, first thought should be how to change the rules. This typically means facing up to a trade-off between welfare and liberty, to be resolved by the normal political processes. For most persistent social problems, to follow the libertarian paternalist in hoping this trade-off can be dodged, by helping players play better within the existing rules, is to pursue a mirage.

Some commentators propose introducing a third, intermediate level of analysis (a group-frame, Bingley et al. or community-frame, Caggiano et al.). We are very sympathetic to this emphasis on groups of many kinds, including popular movements (Mermelstein & Preston; see also Cole, 2016). Regarding theory and terminology, though, we believe it is simpler to stick to the basic distinction (changing the rules of the game vs. the actions of the players), while allowing that the players in the game (i.e., system) under study can be groups (or other aggregate entities) rather than individuals (just as we might model a wage negotiation game between labor unions and businesses, rather than individual workers and managers). Similarly, there may be interesting i-frame/s-frame issues within groups or organizations (e.g., concerning struggles for control within a social movement).

R4. Crowd-out, crowd-in, and the impetus for s-frame reform

Several commentators doubt our assertion in the target article and in prior work (Hagmann, Ho, & Loewenstein, 2019) that nudges may “crowd out” support for more substantive policies. We will turn later to an obvious, material, crowd-out effect: That armies of researchers testing nudge interventions are not engaged in other policy-related research. Here, we focus on the narrower question of whether exposure to i-frame interventions, whether in the news or in practice, diminishes support for structural changes.

Sunstein, who has extensive experience in public policy, claims to have never seen such crowd-out. This conflicts with other top-level policy makers we have interacted with who, reading our paper or hearing us present it, have reported on situations in which the (false) promise of nudges diminished support for more substantive reforms. Sunstein dismisses the systematic pattern we identify of corporate support for the i-frame as no more than “arresting stories,” but seems to have complete confidence in his own personal observations from his time in the White House.

Sunstein seems to misunderstand our worry that behavioral scientists' focus on i-frame interventions can weaken support for systemic reform. He claims “If we were making a list of 100 reasons why system reform has not happened in some important area (such as climate change), the fact that some behavioral scientists have been enthusiastic about i-frame interventions could not possibly make the list.” We are certainly not claiming that crowd-out is one of the most important causes of lack of progress on climate change (or any other issue). This would be to radically overestimate the power of behavioral insights, or indeed any other source of policy recommendations, when compared with the vast political and commercial forces battling for control of the climate agenda. We are not expecting behavioral public policy to single-handedly change the world; we are hoping that more reflection on our focus as a discipline may increase the degree to which we can collectively contribute in a positive direction as far as possible.

We are surprised that Sunstein expresses such confidence that i-frame research findings will not reduce the perceived need for s-frame change. To requote a passage we highlighted, he recently wrote (Sunstein, 2021, p. 548):

It has long been thought that to reduce environmental harm, the best approach is an economic incentive, perhaps a corrective tax. In recent years, however, increasing attention has been given to non-monetary interventions including “nudges,” such as information disclosure, warnings, uses of social norms, and default rules. A potentially promising intervention would automatically enrol people in green energy, subject to opt-out.

This very description pitches i-frame interventions as an alternative to what was “long thought” to be the best approach: A carbon tax. Sunstein seems to imply that the best approach may not be a carbon tax, or similar s-frame reform, but that i-frame nudges may provide an alternative. Behavioral scientists need to be very cautious about conveying such an impression, whether intentionally or not.

Sunstein also dismisses our research as “unreliable non-evidence, including surveys finding that if you tell people about an i-frame intervention, you can reduce support for an s-frame intervention.” We assume Sunstein cannot be suggesting that survey evidence is always unreliable – indeed, he has a long track-record of using, and drawing strong conclusions from, survey methods. But if that is not the implication of his statement, then why are the diverse empirical studies we review – all showing large and highly statistically significant crowd-out effects – viewed as “unreliable non-evidence?”⁷

Zhao & Chen address the empirical question more substantively, and point to a recent meta-analysis concluding that there is no systematic negative spillover between different environmental behaviors (although also showing weak or non-existent positive spillovers). They cite Thomas, Sautkina, Poortinga, Wolstenholme, and Whitmarsh's (2019) study, which found that the English plastic-bag charge changed behavior and increased support for other charges to reduce plastic waste. Of course, as a plastic-bag charge is a paradigm s-frame policy, this demonstrates a positive spillover from s-frame reform to i-frame reform, rather than the opposite.⁸ Zhao & Chen note that positive spillovers might be more likely when i-frame interventions target intrinsic motivation. Koppel, Robertson, Doell, Javeed, Rasmussen, Rathje, Vlasceanu, & Van Bavel (Koppel et al.) acknowledge the lack of empirical support for positive spillover, but claim that positive, crowd-in, effects could be obtained if nudges played on social identity, paralleling Walton & Yeager's claim that nudges could have large effects if more accurately targeted.⁹

We acknowledge that there are situations in which positive spillovers could, and very likely do, occur – in which being nudged to engage in a certain behavior increases public support for more substantive measures. Newell, Vigouroux, & Greenwell (Newell et al.) provide a persuasive example: Although there is little if any evidence that carbon-footprint calculators reduce personal emissions, they suggest that “knowledge about how our personal actions can collectively make a difference in tackling environmental problems can be a powerful motivator for supporting proenvironmental action.”

In closing this subsection, we note that Hagmann et al. (2019) did not intend the conclusion of their paper, “Nudging out support for a carbon tax,” to be that nudges always crowd out more substantive policies, but as a response to claims that such effects do not occur (e.g., Sunstein; Thaler). Indeed, as we note in our target article, Hagmann et al.'s final study showed that these effects disappear when participants are informed of the likely small impact of the nudge (green-energy defaults).¹⁰ We continue to believe, along with many commentators (e.g., Bright et al.; Hertwig; He & Kang) and, apparently, many companies and their public relations agencies, that the i-frame can provide a potent distraction from s-frame change, unless we are very clear about the modest impacts that i-frame interventions are likely to have.¹¹

R5. i-Frame and s-frame interventions: Complements or substitutes?

A number of commentators (e.g., Cherry & Kallbekken; Collier, Harris, Jecks, & Bendtsen; Hagger & Hamilton; Koppel et al.; Madva et al.; Newell et al.; Ruggeri et al.; Sunstein; Zhao & Chen) note that i-frame and s-frame policies should be viewed as complements, not substitutes: That is, that surely both are required. We agree, and regret if we failed to convey this clearly. We did go some distance in that direction in the target article, stating for example, that:

uncontroversially, s-frame policies should be as “ergonomic” as possible, and they frequently fail badly in this regard… A valuable lesson from the behavioral insights movement has been that ergonomics matters just as much for government policies as for the personal computer (PC) or smart phone. Designing policy around the consumer can frequently make the difference between success and failure, and policy design should be guided primarily by behavioral insights.

Among the illustrations we provide is the case of a carbon tax (or, nearly equivalently, a cap-and-trade scheme), widely recognized as an essential part of any successful response to climate change. Designing and implementing a carbon tax that is both effective and acceptable to the public will, as we discuss in our paper, require key decisions (some involving i-level issues) that can usefully be informed by behavioral research.

Although i-level and s-level interventions can and should be synergistic, a very real danger arises when i-frame interventions (typically with modest scope and effect sizes) are framed as alternatives to s-frame change. The problem is not primarily that nudges, information provision (for example, calorie labels, labels for recycling, kite-marks for sustainability), boosts (Hertwig), financial education, computational tools for assisting decision making (Johnson & Mrkva) and the like are being oversold by their inventors. The danger is rather that they are being overbought by policy makers hoping that difficult s-frame policy challenges can be avoided by supposed i-frame alternatives.

Indeed, our central theme is that powerful interests opposed to s-frame reform exploit the prospect of i-frame change as a substitute for s-frame change. Thus, as we document, fossil-fuel companies have promoted individual carbon footprints; pension companies frame long-term retirement provision as a matter of personal prudence; food companies focus on individual choice, and especially levels of exercise, as the root cause of the obesity epidemic; companies generating vast quantities of plastic waste sponsor advertising campaigns focusing on individual responsibility for littering; the gun lobby promotes the slogan that “guns don't kill people, people kill people,” and so on. Viewed through the lens of the conflictual analysis of persistent social problems, this is to be expected, as a tactic to reduce pressure for s-frame solutions and to focus instead on i-frame interventions, which are likely to have only marginal (although often worthwhile and positive) impacts.

In the debate over the substitutability versus complementarity of i-frame and s-frame policies, an obvious point gets lost: Researchers have limited time and resources. As Roberto writes, “Resources spent developing, pursuing, and touting relatively ineffective i-frame interventions draw resources away from the development and implementation of more effective s-frame solutions.” “Attentional and physical resources are limited. A researcher spending time investigating or promoting an i-frame solution is not spending that time investigating or promoting an s-frame solution. Funding dollars spent on i-frame research is not spent on s-frame work.” Roberto concludes that “Behavioral scientists who want to develop solutions to the world's biggest problems should focus their efforts on s-frame solutions.” We agree. Similarly, Newell et al. also point to the “scarcity of academic or bureaucratic resources” as reasons for why i-frame and s-frame interventions can act as substitutes rather than complements.

R6. Are i-frame interventions really so ineffective?

Surprisingly few commentators (e.g., Johnson & Mrkva) challenge our conclusions about the (in)effectiveness of nudges.¹² We stress that in some policy contexts, i-frame ergonomics, and in particular choice architecture, can be important (e.g., Johnson, 2022). But often better s-frame design is more crucial. Thus, no amount of nudging to help people to save can compensate for a fundamentally flawed pension system (see below). We completely agree with Johnson & Mrkva that policy makers need to be alert to, and regulate against, profitable but welfare-destroying “dark nudges.”

Beyond the studies we cite (e.g., by DellaVigna & Linos), it is useful to consider a more recent analysis (Saccardo et al., 2022) of two randomized controlled trials (RCTs) of nudges to increase coronavirus disease-2019 (COVID-19) vaccinations (N = 187,134) and 111 nudge RCTs ranging across policy areas involving 22 million people. The paper concludes that “nudges’ estimated efficacy is higher when outcomes are more narrowly (vs. broadly) defined and measured over a shorter (vs. longer) horizon, which can partially explain why nudges evaluated by academics show substantially larger effect sizes than nudges evaluated at scale by a government agency.”¹³

Relatedly, and consistent with this theme, Thaler notes that: “the range of interventions studied by behavioral scientists is truncated by what I call permission bias: you can only test what you can get the approval to try. It is wrong to infer from this fact of life that behavioral scientists are using the wrong ‘frame.’ Rather, they face constraints! It also makes it problematic to judge the potential impact of possible behavioral policy interventions based on the set of randomized controlled experiments behavioral scientists have been allowed to run.” It is indeed possible that nudges might be more effective if unconstrained by policy-making realities. But these same constraints inevitably bind not only on what RCTs are possible, but also what policies can be implemented. Moreover, if Saccardo et al.'s (2022) findings hold more broadly, we might anticipate that large-scale and longer-term trials, if and when they could be run, would produce disappointingly modest effects. More generally, our argument, echoing others (e.g., Deaton, 2020; Deaton & Cartwright, 2018a, 2018b), is that the focus on RCTs itself is extremely restrictive, and largely excludes s-frame reform. Systemic changes typically affect everyone, and usually cannot be randomly allocated to some people and not others. We discuss this point further in section R8, dealing with research methods.

Walton & Yeager suggest that nudges might be more effective if more accurately targeted. They cite two meta-analyses finding that a specific type of nudge, mindset interventions (which they acknowledge that we don't discuss but, somewhat mysteriously, claim that we nevertheless mischaracterize) has modest overall effects. But they argue that: “One should not expect strong main effects but variable effects in different contexts (for i-frame interventions) … C&L ignore this, claiming that small average effects for nudges indict the field. Yet the question should be: how and under what circumstances can effects be optimized?” Perhaps, but the proof will be in the pudding.

Newell et al. propose that discouragement about small nudge effect sizes arises from excessive expectations. We agree that “behavioural scientists should not overhype the potential impact of i-frame interventions beyond what is justified by their typically modest results.” Indeed, as shown in Hagmann et al. (2019), this is crucial for encouraging complementarity rather than rivalry between s- and i-frame approaches – because it is critical for policy makers, opinion formers, and the general public to appreciate that, in most policy domains, i-frame interventions are too weak to substitute for s-frame change.

R7. Have behavioral scientists been engaging with the s-frame all along?

Some commentators (Hallsworth; Sunstein; Thaler) stress that behavioral insights have already often been applied to inform systemic change.¹⁴ According to this critique, where behavioral insights researchers are in, or are close to, government, they have frequently focused on systemic policy (e.g., laws, taxes, mechanism design). Where this occurs, it is to be applauded and extended.

Perhaps differences in perspective partly reflect a disagreement on what counts as substantive s-frame reform. Focusing on the crucial case of what he terms “saving for retirement” Thaler writes “In four decades of behavioral economic research on this topic, the focus has always been on making the system work better for humans. Isn't that the s-frame?” Not necessarily. Indeed, the very framing of retirement provision as a problem of “saving” reflects an individualist perspective. Retirement is, in most countries with successful pension systems, not a matter of individuals saving (and making investment decisions) for their own retirement; it is often a process that is primarily organized by the state.¹⁵

Substantial s-frame reform for pension provision in the United States needs to be far more radical than the innovations presented by auto-enrolment and auto-escalation. The entire defined-contribution retirement system, with its unrealistic assumptions about individual saving, highly regressive tax breaks, requirement that individuals make their own (uninformed) investment choices, ease in permitting withdrawals and borrowing against retirement savings, and ignorance of the realities of life at low or even median income-levels in the United States, is fundamentally flawed.

The benefits of defined-benefit schemes (and other collective schemes) are clear: They pool risks across many workers. By contrast, defined-contribution schemes place the risk regarding life-expectancy and investment performance firmly with the individual. Most people attempting to provide for their retirement under such schemes have little idea what level of savings will be sufficient, or how to draw down the money once they are retired. As it happens, across the population, defined-contribution pensions are also seriously underfunded: Many people with such schemes face severe financial hardship in retirement.¹⁶

Thaler claims: “Private sector defined-benefit plans, like typewriters and dial telephones, are obsolete technologies few people pine for.” But the transition has not occurred because of public enthusiasm for defined-contribution schemes; but because they are cheaper for employers and more profitable for pension providers (Hassel, Naczyk, & Wiß, 2019). The outcome has been disastrous. The median 401(k) balance in the United States in 2019 for people who have a 401(k) in the age range 55–64 was an utterly inadequate $144,000.¹⁷ Worse, 48% of that age group have no 401(k) at all. If defined-benefit schemes were the typewriters of their day, it seems we must now make do with a broken pencil.^{18, 19}

The challenge of s-frame pension reform is substantial, but is dwarfed by the technological and social transformations required to tackle problems such as climate change or healthcare reform. This “inconvenient truth” is rarely confronted in behavioral public policy (Roberto, see also Lamberton). For example, Jackson notes how little social science policy research (including behavioral science) has focused on addressing the growing problem of inequality in the United States, United Kingdom, and many other countries. Of course, where there are small-scale “quick wins,” we should embrace them.²⁰ But we must not lose sight of the scale of reforms required to bring about really substantive change; nor of the inevitability that such reforms will be vigorously opposed by those benefitting from the status quo.

There is doubtless room for further debate on how usefully and distinctively behavioral science can shape government policy. But this issue is largely independent of our argument that: (1) that a large fraction of the behavioral science community has focused on i-frame interventions to the detriment of exploring routes to sorely needed systemic reform; and (ii) the biggest stumbling block to reform is not lack of insight (behavioral or otherwise) about what to do, but achieving the political momentum to overcome the powerful interests opposing system change. A major role of behavioral science should, therefore, be to help identify how to mobilize popular support to overcome powerful vested interests.

R8. Why is there so little s-frame behavioral public policy research?

A theme that we touch upon, but do not discuss in detail, in our target article is the tendency for the most widely used social science methods to bias policy toward the i-frame. Jackson concurs, arguing that “the social sciences are increasingly ill-equipped to design system-level reforms. Long-standing trends in social science – in particular, increasing specialization, emphasis on causal inference, and the replication crisis – are barriers to system-level policy development.” Strohminger & Táíwò likewise argue that “structural factors bias and perpetuate behavioral science toward the i-frame”; we agree, though we are skeptical of some of their specific proposals (e.g., choosing which research projects to fund by lottery). Gal & Rucker have elsewhere noted that “applied behavioural science tends to overvalue interventions that can be readily tested using experiments,” and argued that such constraints on research “drives the popularity of light interventions and nudges and unnecessarily limits the scope and ambition of the field” (Gal & Rucker, 2022). Thaler also points out the biasing effect of constraints on research: “An important point to stress is that behavioral scientists, whether they are in academia or nudge units, do not have the authority to experiment with most of the rules and regulations in a given domain. No nudge unit has the ability to say, hey, let's try a carbon tax in half the country and strict emission rules in the other and see how it goes. In practice they are often limited to messaging campaigns, which are less impactful.”

Note, though, that the limits on s-frame research go beyond what Thaler refers to as “permission bias” – that researchers need political permission to test their ideas. Commercial funding for science is very substantial, and, given severe limits on government funding of research, inevitably influences the types of research that gets done. Worse, commercial interests can suppress science. Koerth (2023) notes that there is little evidence on effective gun controls in the United States because laws banning the use of gun-tracing information for research purposes was enacted by gun-lobby-influenced legislators.²¹ In consequence, Koerth notes, a recent major Rand report (Smart et al., 2023) that adopts strict criteria to evaluate studies on gun control finds just three studies between 1995 and 2020 meet these criteria. But these are, in any case, the wrong standards for s-frame policies, where experimental studies are generally impractical (and well-controlled “natural” experiments are rare); instead, historical and cross-national (and sub-national) comparisons, while imperfect, are hugely informative.

More broadly, s-frame policy innovation is inevitably, as Cherry & Kallbekken observe in their insightful commentary, a process involving intuition and exploration, as is true for systemic changes ranging from restructuring a business, changing a law, to widening the franchise, expanding civil rights, or setting up entirely new institutions. Any experimentation in system-level reform typically occurs in a fairly ad hoc manner, as the new systemic change is trialed and continually adapted to deal with the inevitable stream of hitches and unintended consequences (Hausmann & Rodrik, 2003; Mulgan, 2021; Sanger & Levin, 1992).

A cautious approach to policy innovation relying on experimental, or similar, evidence as a precondition would have ruled out most of the major transformational developments in human history. Consider the transitions toward democracy, revolutions in agriculture, manufacturing, and the invention of information technology (IT) (and their economic and organizational consequences), the modern financial system, the creation of the welfare state, or the international institutions of the United Nations (UN), and many more. All these huge s-frame innovations would have been hopelessly hamstrung if each move forward had to be grounded in a solid basis in RCTs.

Research and its influence on policy can be skewed in other ways, too. Scientists, ourselves included, are naturally driven by curiosity. But what is most interesting is not always what is most important. Roberto writes:

typically, scientists ask questions they are curious about and that other scientists find interesting. This approach works well if you want to learn something about human psychology or offer self-help ideas or treatments for people. But if your goal is to contribute population-level solutions (which are required for most big challenges), a scientist must begin the research process by asking: (1) What is known about the problem drivers, (2) what has been tried, and (3) what solutions are most promising?

Without abandoning curiosity as an important and legitimate criterion for the selection of research projects, a “nudge” for researchers in the direction of policy impact might be beneficial.

R9. Final thoughts: Behavioral science and the struggle against special interests

We agree that behavioral insights are key to enacting and implementing successful policy reform (e.g., Hallsworth; Sunstein; Thaler). A key, but under-appreciated role that the behavioral sciences can play is in winning the “political battle” against special interests (which are not seriously attempting to engage with the truth or find policy solutions to maximize human welfare). Radical systemic change often comes from the bottom-up, as well as from top-down. Understanding which policies gather popular support (Fitzgerald, Lamberton, & Walsh, 2016) and how to design policies (e.g., carbon taxes, health care, or pension reform) to maximize that support are key challenges.

Major societal problems require, we believe, major systemic change. Early social and economic theorists as varied as Thomas Hobbes, Jean-Jacques Rousseau, Adam Smith, David Ricardo, Auguste Compte, and Karl Marx, focused at least as much on changing the social world as on understanding it (Mulgan, 2021). But major s-level change, even where it is widely agreed to promote human welfare overall, will typically be bitterly opposed by those benefitting from the status quo. The challenge in formulating, and building support for, reforms that address the fundamental challenges that face our societies is formidable, but more than worth confronting. All available tools must be deployed, and where i-frame solutions can contribute, behavioral scientists should pursue these enthusiastically. But to really make a difference, behavioral public policy needs to refocus its insights and energy on s-frame reforms: Almost always, deep policy problems require us not just to nudge the players, but to change the game.