Cost-Benefit Analysis, Who’s Your Daddy?

Abstract

If policymakers could measure the actual welfare effects of regulations, and if they had a properly capacious sense of welfare, they would not need to resort to cost-benefit analysis, which gives undue weight to some values and insufficient weight to others. Surveys of self-reported well-being provide valuable information, but it is not yet possible to “map” regulatory consequences onto well-being scales. It follows that at the present time, self-reported well-being cannot be used to assess the welfare effects of regulations. Nonetheless, greatly improved understandings are inevitable, and current findings with respect to reported well-being – above all the serious adverse effects of unemployment – deserve to play a role in regulatory policymaking.

1 Two illustrative problems

Suppose that the Environmental Protection Agency (EPA) is considering a new regulation, designed to reduce levels of particulate matter in the ambient air. Suppose that the total cost of the regulation would be $900 million. Suppose that the mortality benefits would be precisely equal to that, because the regulation would prevent 100 deaths, each valued at $9 million. Suppose as well that if the EPA includes morbidity benefits (in the form of nonfatal illnesses averted), the regulation would have an additional $150 million in benefits, ensuring that the monetized benefits significantly exceed the monetized costs.

Now assume two further facts. First, the mortality and morbidity benefits of the regulation would accrue mostly to older people – those over the age of 80.² Second, the rule would have significant unemployment effects, imposing a statistical risk of job loss on a large number of people, and ultimately causing 3,000 people to lose their jobs.³ Environmental Protection Agency believes that the overwhelming majority of those 3,000 people would find other jobs, and probably do so relatively soon, but it does not have a great deal of data on that question, and it cannot rule out the possibility of long-term job loss for many people. In accordance with standard practice, the EPA does not include either of those further facts in its cost-benefit analysis.⁴

If the goal is to promote social welfare, we might well think that it is far too simple for EPA to conclude that because the monetized benefits exceed the monetized costs, it should proceed with the regulation. One question is whether and how to take into account, in welfare terms, the relatively fewer additional life-years that the regulation will generate. In those terms, is a rule that “saves” people over 80 to be deemed equivalent to one that “saves” an equivalent number of people who are much younger⁵? Another question is the welfare consequences of the $900 million expenditure. Suppose that concretely, that cost will be spread across at least 200 million people, who will be spending, on average, a little over $4 annually for the regulation. What are the welfare consequences of that modest expenditure? Might they be relatively small? A further question is the unemployment effect. We know that in terms of subjective welfare, it is extremely bad to lose one’s job.⁶ We know too that in terms of money, a loss of a job often creates a nontrivial long-term loss in income.⁷ We know that a long-term loss of employment has more severe adverse consequences than a short-term loss – but that both are bad. Shouldn’t those welfare effects be included?⁸

Now suppose that the Department of Transportation is considering a regulation that would require all new automobiles to come equipped with cameras, so as to improve rear visibility and thus to reduce the risk of backover crashes.⁹ Suppose that the total estimated annual cost of the regulation is $1.2 billion (reflecting an average cost of $50 for a large number of vehicles, to be sold over a relevant time period). Suppose that the regulation is expected to prevent 60 deaths annually, for monetized annual savings of $540 million, and also to prevent a number of nonfatal injuries and cases of property damage, for additional annual savings of $200 million. On the basis of these numbers, the Department is inclined to believe that the benefits of the rules are significantly lower than the costs.

At the same time, suppose that the Department is aware of four facts that it deems relevant, but is not at all clear how to handle. First, a majority of the deaths would involve young children – between the ages of one and five. Second, a majority of those deaths would occur as a result of the driving errors of their own parents, who would therefore suffer unspeakable anguish. Third, the cost of the rule would be diffused across a large population of new car purchasers, who would not much notice the per vehicle cost. Fourth, the cameras would improve people’s driving experience, by making it much easier for them to navigate the roads, even when it does not prevent crashes. (The Department speculates that many consumers do not sufficiently appreciate this improvement when deciding which cars to buy.) Is it so clear, in light of these four facts, that the agency should not proceed?

In principle, cost-benefit analysis is best defended as the most administrable way of capturing the welfare effects of policies¹⁰ (including regulations). But if we actually knew those effects, in terms of people’s welfare, and thus could specify the actual consequences of policies for welfare, we would not have to trouble ourselves with cost-benefit analysis. An initial problem is that cost-benefit analysis depends on willingness to pay, and people might be willing to pay for goods that do not have substantial positive effects on their welfare (and might be unwilling to pay for goods that would have substantial positive effects). Willingness to pay is based on a prediction, and at least some of the time, people make “hedonic forecasting errors.”¹¹ A separate problem involves the incidence of costs and benefits, which can complicate the welfare analysis even if we put “pure” distributional considerations to one side.¹² Suppose that a regulation would impose $400 million in costs on relatively wealthy people and confer $300 million in benefits on relatively poor people. Even if the losers lose more than the gainers gain in monetary terms, we cannot exclude the possibility that the losers will lose less that the gainers gain in welfare terms.

The most general problem is that once agencies specify costs and benefits, the resulting figures will inevitably have an ambiguous relationship to what they should care about, which is welfare.¹³ To be sure, it is possible that some of the problems in the two cases I have given could be significantly reduced with improved cost-benefit analysis. If children should be valued differently from adults, and elderly people differently from younger, cost-benefit analysis might be able to explain why and how.¹⁴ Perhaps parental anguish could be monetized as well.¹⁵ The same might well be true of the increased ease of driving. But even the best proxies remain proxies, and what matters most is the thing itself.

Cost-benefit analysis, meet your Daddy.¹⁶

2 Administrability and crude proxies

The great promise of self-reported well-being is that it might be able to offer a more direct, and more accurate, measure of welfare¹⁷ than could possibly come from an account of costs and benefits (especially if that account depends on willingness to pay). Suppose that we agree, with Paul Dolan,¹⁸ that welfare consists in significant part of people’s feelings of pleasure (broadly conceived) and purpose (also broadly conceived). If so, we might be able to ask people about those two variables. How much pleasure do people get from certain activities? How much of a sense of purpose? Dolan has in fact asked such questions, with a range of illuminating results.¹⁹ We are learning a great deal about what kinds of activities are pleasurable or not, and also about what kinds of activities seem to give people a sense of meaning or purpose.

2.1 Evaluation and experience

With respect to subjective well-being, the most popular of the existing measures take two forms. First, researchers try to assess people’s “evaluative” welfare by asking questions about overall life satisfaction (or related concepts).²⁰ With such measures, it is possible to test the effects of a range of variables such as marriage, divorce, disability, and unemployment.²¹Second, researchers try to assess people’s “experienced” welfare, through measures of people’s assessments of particular activities (working, commuting, being with friends, watching television).²²

In fact, researchers have uncovered some systematic differences between people’s overall evaluations and their assessments of their particular experiences.²³ Marital status is more closely associated with evalutive well-being than with experienced well-being, though there is conflicting evidence on this point.²⁴ French people report significantly lower levels of satisfaction in their lives than do Americans, but the French appear to show equal or even higher levels of experienced well-being.²⁵ Health states are more closely correlated with experienced well-being, though they also affect evaluative well-being.

There is a lively debate about the choice between the two measures.²⁶ The current consensus appears to be that useful but different information is provided by each.²⁷ On one view, questions about experienced welfare focus people on their existing emotional states, and thus provide valuable information about those states. By contrast, questions about evaluative welfare encourage people to think more broadly about their overall goals or aspirations. On this view, evaluative welfare “is more likely to reflect people’s longer-term outlook about their lives as a whole.”²⁸ If this is so, then the two measures do capture different kinds of values, and both are important. In my view, an important question remains: We do not yet know whether people’s answers to questions about evaluative well-being in fact reflect their broader aspirations, or whether they are instead an effort to summarize experienced well-being, in which case the latter is the better (because more accurate) measure.

There is also an active debate about the reliability of both measures of welfare; some people are skeptical about how much we can learn from them.²⁹ Just as with willingness to pay,³⁰ it is important to consider issues related to the quality, reliability, and applicability of different studies.³¹ Making a frontal assault on the whole program, Deirdre McCloskey disparages what she calls “the 1-2-3 hedonists,” who celebrate “happyism.”³² In my view, the disparagement is a mistake, for the measures tell us a great deal.³³ It is true that of “welfare” leaves a great deal of ambiguity, and if it is invoked for policy purposes, any particular account is highly likely to end up in contested normative terrain.³⁴ As Dolan makes clear, a purely hedonic measure, focused only on pleasure and pain, would be inadequate; people’s lives should be meaningful as well as pleasant.³⁵ But even if we adopt a measure that goes beyond pleasure to measure a sense of purpose as well, we might be capturing too little.

Mill’s objections to Bentham are worth quoting at length:³⁶

Nor is it only the moral part of man’s nature, in the strict sense of the term – the desire of perfection, or the feeling of an approving or of an accusing conscience – that he overlooks; he but faintly recognizes, as a fact in human nature, the pursuit of any other ideal end for its own sake. The sense of honor, and personal dignity – that feeling of personal exaltation and degradation which acts independently of other people’s opinion, or even in defiance of it; the love of beauty, the passion of the artist; the love of order, of congruity, of consistency in all things, and conformity to their end; the love of power, not in the limited form of power over other human beings, but abstract power, the power of making our volitions effectual; the love of action, the thirst for movement and activity, a principle scarcely of less influence in human life than its opposite, the love of ease…. Man, that most complex being, is a very simple one in his eyes.

These points suggest the importance of having a capacious conception of welfare, one that is alert to the diverse array of goods that matter to people. Amartya Sen and Martha Nussbaum have elaborated a “capabilities approach,” which is objective rather than subjective, and which emphasizes the importance of providing people with “capabilities” that support “functionings.”³⁷ A particular concern, stressed by both Sen and Nussbaum (and also Elster³⁸) has to do with people’s capacity for adaptation and, in particular, the problem of adaptive preferences. Suppose that people adapt to their situations, so that even under conditions of serious disability or acute deprivation, people’s subjective well-being remains relatively high. Indeed, the well-being literature finds that many serious disabilities produce relatively small hedonic losses.³⁹ Ought we to conclude that for those who are seriously disabled, or who suffer some form of acute deprivation, there is little or no real loss, if self-reported well-being so suggests? That would be an odd conclusion. Part of what matters is what kinds of lives people are able to live. Their subjective experience of their lives matters too, but it is not the whole picture.

Consistent with Mill’s plea and the emphasis on capabilities, a large survey by Daniel Benjamin and his coauthors tests people’s concern for a list of factors that includes not only “measures widely used by economists” (e.g., happiness and life satisfaction), but also “other items, such as goals and achievements, freedoms, engagement, morality, self-expression, relationships, and the well-being of others.”⁴⁰ The central (and not especially surprising) result is that people do indeed care about those other items.⁴¹ The perhaps ironic conclusion that if measures of reported well-being neglect those items, they will end up losing information that cost-benefit measures ought to be able to capture. A significant advantage of the willingness to pay measure is that it should, in principle, take account of everything that people care about, including those things that matter for Mill’s reasons. If people value cell phones because they want to connect with their children, or if they want to save (rather than spend) money so they can give it to poor children in Africa, or if they want to spend money on a vacation because of their love of nature, their concerns, however diverse in qualitative terms, should be adequately captured by the willingness to pay criterion, however unitary.⁴²

That is a point for cost-benefit analysis, and it suggests that that form of analysis might have advantages over some measures of happiness or subjective welfare, at least if they are insufficiently refined; but it should not be taken to mean that that form of analysis has priority over good or full measures of welfare. (Having cell phones, giving to charity, and enjoying nature also have effects on welfare, and these should be measurable in principle.) What is required are measures that are sufficiently reflective of the diverse set of goods that matter to people, but that avoid the various problems, sketched above, with cost-benefit analysis.

2.2 The largest problem

With respect to regulatory policy, the largest problem with invoking self-reported well-being is this: Even if such surveys provide a great deal of information, we cannot easily “map” any particular set of regulatory consequences onto changes in welfare.⁴³ Self-reported well-being is a measure of well-being, not the thing itself; the numbers that emerge from coefficients on well-being, and then are translated into monetary terms, are not an exact measure of what matters. They may offer us valuable information about how much (for example) unemployment affects well-being as compared to a health shock, across a population. But they do not give us a clear indication of how the multiple impacts of (say) air pollution and automobile safety regulation affect welfare as such.

2.2.1 Coarseness

Although we are learning a great deal about what increases and what decreases welfare, most of what we have learned to date remains relatively coarse⁴⁴; it frequently involves the consequences of large life events, such as marriage, divorce, and unemployment. We do not know enough about how to answer hard questions about the welfare effects of health and safety regulations.⁴⁵ How much happier are people when the level of ozone in the ambient air is decreased from 70 parts per billion to 60 parts per billion? For the median person, what is the welfare effect of having to spend $50 or $100 or $300 on a particular regulatory initiative, noting that the money could have been used for other purposes? In terms of “welfare units,” how should we think about a loss of a job, or a life-year? Should we use those units, or some other kind (monetary?), in conducting analyses on the basis of studies of self-reported well-being? If we use those units, what, exactly, is the relevant scale?⁴⁶

Return to the two problems with which I began. We have seen that in terms of welfare, cost-benefit analysis, at least in its current form, may not adequately handle: (1) unusually large or unusually small numbers of life-years saved⁴⁷; (2) adverse unemployment effects; (3) questions about the welfare effects of small economic losses faced by large populations; (4) intense emotions associated with certain outcomes, such as parental anguish⁴⁸; and (5) hedonic benefits associated with increased ease and convenience. As I have suggested, improved forms of cost-benefit analysis might be able to reduce these problems.⁴⁹ But ideally, we would want to know about welfare itself. The problem is that at the current time, measures of self-reported well-being are too crude to enable us to do that.

No one should doubt that cost-benefit analysis itself presents serious challenges, sometimes described under the rubric of “the knowledge problem”; agencies have to compile a great deal of information to make sensible extrapolations.⁵⁰ But to map regulatory outcomes onto self-reported well-being, the challenges are far more severe.⁵¹

It is important to emphasize that those challenges might be met over time. Subjective well-being metrics are becomingly increasingly refined, and it should be possible to control for character traits, seasonal effects, and cultural differences. In principle, we might be able to test not only for the effects of large life changes, but also for such things as having a cell phone or not, donating to charity, and experiencing different levels of pollution, as well as experiencing different health conditions. Indeed, some research is doing precisely that.⁵² As such refinements are undertaken, the problem of coarseness will be reduced, perhaps dramatically. But for the problems that typically concern regulatory agencies, we are not nearly there yet.

2.2.2 An example

For purposes of assessing regulation, creative efforts have been made to extrapolate relevant values from measures of self-reported well-being.⁵³ Examining an EPA pulp and paper regulation from the late 1990s, Bronsteen et al. urge that cost-benefit analysis and reported welfare analysis point in quite different directions, and that as a proxy for welfare the latter is superior to the former. In particular, they conclude that while the regulation would be expensive in monetary terms (with EPA’s options ranging from $262 million to $1 billion in cost), that expense would have trivial effects on welfare. Under their estimate, the regulation would require affected individuals to bear several hundred dollars in annual costs.

To capture the welfare effects of that cost, they enlist a study finding that those with a threefold increase in income experience a gain in well-being units (WBUs) of 0.11, and those with a two thirds decrease in income experience a 0.11 decrease in such units.⁵⁴ On the basis of this study, Bronsteen et al. find near 0 in aggregate welfare losses for a modest annual expenditure of several hundred dollars. As they put it, “the monetary costs of the regulation, which dominated the cost-benefit analysis, are nearly irrelevant here.”⁵⁵ To this extent, the monetary costs that loomed so large in EPA’s analysis end up playing essentially no role in a welfare analysis.

By contrast, the unemployment effects matter a great deal in that analysis. On the basis of another study, Bronsteen et al. conclude that those who are unemployed lose 0.83 WBUs per year, and that even after finding new employment, they lose an average of 0.34 WBUs per year during the next seven years after they begin working again.⁵⁶ Because the EPA regulation would result in several thousand lost jobs, it would produce a massive welfare loss. Stunningly, the estimated welfare loss, from lost jobs, is well over 400,000 times the estimated welfare loss from the total monetary cost. Bronsteen et al. also note that the average American has a life satisfaction of 7.4 on a scale of 0 to 10. To capture the welfare loss from avoided fatal cancers, they invoke that number in their welfare equation, which means that the welfare gains of mortalities averted are very high (a shorthand for their analysis: 7.4 times a significant number of life-years).⁵⁷

The problem with this impressively creative exercise is that in light of the current state of the art, it involves far too much guesswork to be invoked credibly by government regulators. Some of the central numbers are both speculative and unclear, provoking a range of questions and doubts. (1) Can we really extrapolate, from one or two (or more) studies of the effects on reported well-being of massive increases or decreases in annual income, the welfare effects, across a large population, of losing several hundred dollars a year? (2) Is it credible to suppose that regulations that impose such costs, on such large populations, should be treated as having essentially no adverse effects on welfare? Would a tax of $200, on say 200 million Americans, have no such adverse effects? (3) Should the same be said of five, or ten, or fifty regulations of this kind? (4) What does it even mean to say that the average worker loss is 0.83 WBUs while unemployed? If an employed person’s experience of his life is 7.4, is an unemployed person’s 6.57? In terms of what, exactly?

At the present time, these questions do not seem to have good answers. The most sensible conclusion is that studies of reported well-being cannot be used as anything like a substitute for cost-benefit analysis, and that they should not yet play a significant role in regulatory analysis.

3 Conclusion

Does this conclusion mean that today and in the near future, regulators should rest content with cost-benefit analysis, and put entirely to one side, as speculative and unreliable, whatever we might learn from directly considering welfare? That would be far too strong. Most important, unemployment effects deserve serious consideration,⁵⁸ not least because of the significant adverse welfare effects of losing one’s job.⁵⁹ It is also at least relevant to know whether a regulation would protect children, and hence provide a large number of life-years, or instead protect older people, and hence provide a relatively smaller number of life-years. The Department of Transportation was correct to refer to this point in the rear visibility case.⁶⁰ It is indeed possible that a large cost, spread over a very large population, might turn out to have relatively modest adverse effects on welfare. Agencies have already taken account of that possibility in undertaking breakeven analysis; it might be considered in cases in which costs and benefits are otherwise fairly close.⁶¹ If agencies would help people who suffer from mental illness of one or another kind, the welfare gain might be substantial,⁶² even if the benefits cannot be adequately captured in willingness to pay figures.

Emphasizing the promise of research on subjective well-being, Raj Chetty contends: “Further work is needed to determine whether and how subjective well-being metrics can be used to reliably measure experienced utility, but they appear to offer at least some qualitative information on ex post preferences than can help mitigate concerns about paternalism in behavioral welfare economics.”⁶³ Chetty’s conclusion is sound, but it could be stronger. Work on subjective welfare⁶⁴ can serve not only to mitigate concerns about paternalism, but at least on occasion, to inform analysis of the welfare effects of regulations (and policies in general). The point has nothing to do with paternalism, but with accurate measurement of what really counts.

At the present time, cost-benefit analysis remains the best proxy for what matters, but it is not too optimistic to think that in the fullness of time, it will actually have a chance to meet its Daddy.