Three Criteria for Evaluating Social Programs

Abstract

This article examines the economic foundations of three criteria used for evaluating the costs and benefits of social programs. Some criteria do not consider the scale of programs or address the costs associated with programs that expand or contract the total government budget. A recent addition to the list of evaluation criteria – the marginal value of public funds – does not adopt a social optimality perspective. It evaluates the optimality of expenditures assuming a predetermined aggregate budget without considering the social costs of raising that budget.

1. Introduction

Many criteria are currently used to evaluate the effectiveness of policies. This article examines three: (i) the venerable benefit-cost ratio (BCR) (e.g., Boardman et al., 2018); (ii) the net social benefit (NSB) (see Feldstein, 1964); and (iii) the marginal value of public funds (MVPF), recently reintroduced to the literature in Hendren and Sprung-Keyser (2020).

The basic principles of social policy evaluation are straightforward but often not well understood. For a given project $ j $ , define the benefits resulting from it as $ {B}_j $ . In principle, these benefits are discounted over the life of the project. ¹ Benefits include productivity-enhancing gains directly received by intended project beneficiaries as well as any productivity spillovers to society at large, including both the values that nonparticipants place on those gains associated with “welfare weights” and the psychic benefits to beneficiaries. A policy that only directly benefits some may be valued by many not directly receiving benefits.

Measuring the full range of benefits of a project for current and future generations is a challenging task addressed in the literature. These benefits include gains generated by the program (e.g., projects in the private sector activated by the program). Social gains do not include revenue payoffs to governments (e.g., tolls or taxes raised on any income increases of participants, although they are revenue gains to governments). Social gains include savings in welfare losses associated with revenue raising and hence reduced tax avoidance, although these are not transfers to governments.

Many possible projects compete for attention, $ j\in \left\{1,\dots, J\right\} $ . These may include the same project at different scales or different versions of the same program. Some programs may be mutually exclusive (e.g., the same program at different scales), complimentary, or independent.

Offsetting benefits $ {B}_j $ are costs that consist of direct costs $ {D}_j $ (consumption foregone) and any resulting welfare costs from financing the program, including labor supply effects and other tax avoidance behaviors (Feldstein, 1999). Define the total costs of funds to finance $ {D}_j $ as $ \Omega \left({D}_j\right) $ .

Taxes collected on enhanced social output are merely social transfers from the public to the government, although their welfare costs are not. ² Such transfers are irrelevant for computing NSB but, as we shall see, matter greatly for computing MVPF. The level of taxes raised from enhanced productivity crucially depends on institutional details about tax schedules determined by administrative or legislative considerations. Note further that any revenue raised by a program is not necessarily applied to the programs generating it and might well be returned to the public in the form of lower taxes.

The full social cost $ \Omega \left({D}_j\right) $ need not be an affine function of $ {D}_j $ (i.e., it can be a nonproportional, nonlinear function). A proper accounting of costs requires the analyst to estimate the full social costs of financing the program, including any distortions induced by taxation, distress sale of assets, or borrowing costs. In practice, the full array of costs and the source for financing any project is often unknown. Feldstein (1999) discusses the costs of revenue raised by taxes and presents widely used estimates of welfare costs, often stated as a proportion of project expenditure. In general, these costs are not proportional in scale, nor does Feldstein (1999) claim they are. Investigating the full array of avoidance costs of taxation is an active research topic.

The literature on MVPF focuses on a new and highly artificial “offset” to $ {D}_j $ : the “revenue” raised $ {R}_j $ . Revenue raised $ {R}_j $ is an administrative consideration; a legislative artifact. The revenue raised by boosting benefits in the form of taxable income is not a benefit to society but instead may be a boost to the budget of the government, and crucially depends on particular details of tax schedules or administrative or legislative decisions. $ {R}_j $ is not properly counted as a component of $ {B}_j $ . It is not an additional social benefit. In addition, “revenue” as defined by proceeds of tax collections ignores untaxed psychic benefits. Documenting $ {R}_j $ may be a useful practice for keeping track of the potential revenue base, especially if governments at any level operate under revenue constraints.

2. Determining optimal programs

The academic literature proceeds as if an enlightened social planner is choosing and evaluating projects. In truth, individual agencies often initiate and evaluate projects and face different budgeting constraints determined by legislatures or higher administrative units. Taxation and revenue raising are often determined by distinct agencies from those determining expenditure. The perspective of a social planner may well differ from that of an agency administrator with a fixed budget to spend on a collection of projects. Criteria useful for bureaucrats may differ sharply from those needed to evaluate social optimality.

Converting utils into dollar equivalents, as is common in the literature, the NSB of the project $ j $ is:

(1) $$ {\mathrm{NSB}}_j\hskip0.35em =\hskip0.35em {B}_j-\Omega \left({D}_j\right). $$

$ {\mathrm{NSB}}_j $ is a measure of the net expansion or contraction of the social possibility frontier. Projects may vary in scale and content. A project $ j $ at one scale when implemented at another scale may produce benefits and costs, nonproportional to the expansion of scale of costs and benefits, a feature captured by (1) but rarely addressed in computations of the BCR or MVPF.

García et al. (2020, 2021) are examples of studies that properly define and estimate $ {\mathrm{NSB}}_j $ . Both studies quantify the long-term benefits and costs of productivity-enhancing early childhood education programs, which increase labor income, reduce criminal activity, and promote behaviors that improve the long-term health of their participants. Their estimate of $ {\mathrm{NSB}}_j $ includes the reduction in welfare costs due to lower costs of the criminal justice system (e.g., costs of imprisonment). It also includes private gains due to better health (e.g., better quality of life) and reductions in welfare costs of taxation due to lower costs for the public health system. Enhanced tax revenue due to, for example, reduced participation in safety-net programs because of higher earned (gross) income induced by the programs, should not be part of $ {B}_j $ – it is a transfer between two parties (the government and the public), if these parties are assumed equally weighted. However, any welfare-cost changes generated by such tax enhancement should be counted as part of $ \Omega \left({D}_j\right) $ . García et al. (2020, 2021) quantify these changes in excess burden costs.

Provided inclusive costs and benefits are properly accounted for, from a social planner’s point of view, all projects $ j\in \left\{1,\dots, J\right\} $ that satisfy $ {\mathrm{NSB}}_j>0 $ are worth undertaking. Note further that arbitrary decisions of what goes into costs and benefits are irrelevant for computing NSBs using (1), provided double counting is avoided. Projects with the highest NSBs should be selected in rank order. Note that benefits include welfare evaluations of the project for the larger society, but, in practice, these are rarely measured and dollar values for direct benefits are used on pragmatic grounds. The set of possible projects considered should recognize mutual exclusivity across some projects. Let $ {J}^{\ast } $ be the set of all possible projects. Note that there may be additional political constraints such as restrictions on direct costs $ \left({\sum}_{j\in {J}^{\ast }}{D}_j\hskip0.35em \le \hskip0.35em \overline{D}\right) $ or restrictions on net revenue $ \left({\sum}_{j\in {J}^{\ast }}\left({D}_j-{R}_j\right)\le \hskip0.35em \overline{R}\right) $ that limit the set of politically attainable projects.

3. Benefit-cost ratio

The BCR is widely used. It counts the full social benefits $ {B}_j $ (which of course include expansions of the potential tax base). The full expansion of the social possibility frontier $ {B}_j $ is the appropriate total benefit. The BCR is

$$ {\mathrm{BCR}}_j\hskip0.35em =\hskip0.35em \frac{B_j}{\Omega \left({D}_j\right)}. $$

If $ {\mathrm{NSB}}_j\hskip0.35em =\hskip0.35em {B}_j-\Omega \left({D}_j\right)>0 $ , $ {\mathrm{BCR}}_j>1 $ .

The rule $ {\mathrm{BCR}}_j>1 $ is not a sufficient guide to the optimality of choosing $ j $ over some alternative $ {j}^{\prime } $ . Evidence that $ {\mathrm{BCR}}_j>1 $ is only evidence that program $ j $ pays for itself in the sense of a Kaldor criterion including the utility of equally weighted altruistic agents, but does not necessarily indicate optimality. $ {\mathrm{BCR}}_j $ ignores the scale of net benefits. This is a general property of any ratio measure, including BCR and MVPF. However, rarely do economists value alternative projects by using NSB to establish optimality and this limits the validity of many current criteria. Hendren and Sprung-Keyser (2020) is a recent example. They use, instead, the following ratio criterion.

4. Marginal value of public funds

The MVPF focuses attention on one component of the total benefits: namely, the revenues of the government raised by the program. As previously noted, such revenues depend on institutional details regarding taxation. These revenues are transfers unless the government is more efficient than the public in using funds. MVPF adopts a peculiar posture of evaluating the social benefits of projects assuming a predetermined social cost of public revenue collected or, alternatively, ignoring it. The criterion might be defended from the standpoint of an administrative agency or set of such agencies with a total fixed pool of funds, determined by the legislature that gets to keep (and spend) any revenue benefits $ {R}_j $ of its programs and which are free to disregard the welfare costs of taxation. ³ The MVPF criterion is

$$ {\mathrm{MVPF}}_j\hskip0.35em =\hskip0.35em \frac{B_j}{D_j-{R}_j}. $$

If the agency or collection of agencies providing the program capture the generated “revenue” $ {R}_j $ and can spend against it, MVPF selects projects with the highest “bang for the buck” from available funds so determined. However, there is, in general, no logical necessity for determining the funds available to an agency in this manner. MVPF assumes that the opportunity cost of $ j $ , $ {D}_j $ , is the amount spent by the agency, which might have gone to another agency or returned to the public, and ignores the excess burden of taxes. Notice that

$$ {\displaystyle \begin{array}{l}{\mathrm{MVPF}}_j>1\Rightarrow {B}_j>{D}_j-{R}_j\\ {}\hskip7.5em \Rightarrow {B}_j+{R}_j-{D}_j>0.\end{array}} $$

This expression ignores the social cost of raising revenue. In application, one should not include $ {R}_j $ in $ {B}_j $ or $ \Omega \left({D}_j\right) $ because revenue transfers are not a social benefit or offset to social cost.

Any application of MVPF consistent with basic economics requires that any new choice of a project takes into account any expansion of total expenditure induced by it, by accounting for the social costs of raising those funds. This is not done by Hendren and Sprung-Keyser (2020), who compare projects with vastly different scales and budgetary requirements, and, hence, social costs of funds.

For an administrative agency with a prespecified budget or with access to a pool of funds somehow determined, MVPF characterizes a certain type of efficiency in spending its budgeted funds. Ignoring the social costs of raising the funds allocated, it violates guidelines for the evaluation of federal programs that recognize such costs and require that evaluations quantify them. ⁴ If the social cost of funds is unaffected by a particular policy, it can safely be ignored.

MVPF shares with BCR the property that it ignores the scale of programs evaluated. It further ignores the welfare costs of taxation. A large MVPF could justify expenditure on programs generating low benefits (NSB) at a high direct cost as long as $ {D}_j\to {R}_j $ . It favors programs that operate in institutional tax and expenditure environments that produce government revenue that might or might not be used to offset direct costs and not necessarily programs with the greatest social benefits, including the social opportunity costs of public funds.

5. Conclusions

Neither BCR nor MVPF adequately characterizes the appropriate social criterion for evaluating policies. Neither accounts for the scale of the project being evaluated. MVPF has the additional defect of ignoring the social cost of raising revenue to fund them. The appropriate criterion is $ \mathrm{NSB} $ , but it is rarely implemented because of the challenges of evaluating the full benefits (including “welfare weights” or the value society places on the benefits received by program participants) and the true social costs of funding government expenditure.