In Defence of Cost-Benefit Analysis

Henry Ergas

22 June 2010

Overview

This presentation:

• Outlines the nature of cost-benefit analysis (CBA)

• Considers three current challenges to CBA:

– Consideration of ‘wider economic benefits’

– Use of ‘economic impact’ methods

– The trend to replacing CBA by various forms of multi-criteria analysis (MCA)

• Reviews some criticisms of CBA

• Concludes by considering the state of project appraisal

Overview of Cost Benefit Analysis

What is Cost-Benefit Analysis?

• Cost-benefit analysis (CBA) is a technique for evaluating collective decisions

• CBA hinges on the comparisons of the costs of a proposal to its benefits, where costs and benefits are valued in monetary terms.

• CBA asks whether the sum of the amounts the community at issue would be willing to pay for the project to proceed exceeds the costs of that project.

• Generally, a project enhances wealth — in the sense of the aggregate monetary valuation of the community’s resources — if it meets a properly specified cost-benefit test.

• Whether enhancing wealth in this sense is either necessary or sufficient for a project to be worthwhile is a complex issue. However it seems reasonable to suggest that projects that fail properly specified cost-benefit tests should be looked at very carefully, and be found to have other, significant, redeeming features, before they are allowed to proceed.

• By the same token, if a project has benefits that (evaluated in monetary terms) clearly exceed its costs, it seems reasonable to presume that, in the absence of compelling reasons to the contrary, society would gain were it to proceed.

• CBA is an important element in addressing principal/agent problems in the public sector

– Public projects involve one group of people spending other people’s money

– When poor quality projects are selected, the community loses twice:– Scarce capital is wasted– Taxes are required to fund those projects. Those taxes alter economic

behaviour, imposing a welfare loss– Even when a project is privately funded, as in PPPs, that does not reduce

the social cost of bad projects– In fact, social costs can be every bit as high, especially if the operators

of the PPP are allowed to charge monopoly prices to consumers– Moreover, the result can be to create incentives for governments and

infrastructure builders to collude in promoting projects that are privately profitable but socially wasteful, undermining the efficiency with which capital is used in the economy as a whole

Mirrlees-Little Rule: the social value of good project appraisal is ~>10% of project value

Why is CBA important?

Challenge 1: Wider economic benefits

TOTAL BENEFITS

Social savings

Wider economic benefits

Pecuniary externalities that arise from transport investments in markets that are imperfectly competitive

Travel time savings, reductions in pollution or other externalites

Agglomeration economies

Increased tax revenues resulting from higher LF participation

What is involved in CBA?

Agglomeration economies

Estimated using UK estimates of impact of ‘effective economic distance’ on productivity

Marshallian external economies that arise from colocation of economic entities

Case Study: East West Rail Evaluation (Victoria and IA)

Both theory and empirical analysis suggests that a substantial share of agglomeration economies arise from spill-over effects in human capital

But moving skilled people from one area to another is only advantageous if the impacts of those skills differ across areas – resources should, in other words, be pushed to areas that are more productive and where the elasticity of productivity with respect to agglomeration is higher

The resulting gain in the new equilibrium is not the gross gain in the destination area (assuming there is such a gain), but rather the net gain taking account of the loss of agglomeration economies in the origin area

Also, over the long run, locational choices adjust in ways that re-equilibrate the bid/rent curve and that – if there are unpriced local public goods subject to congestion – can impose negative externalities

The estimated gains are over-stated and in any event, very unreliable

Conclusions on wider economic benefits

• There is nothing inherently wrong with focusing on the wider impacts of transport projects.

• However, our understanding of how to assess these in a rigorous way is still in its early stages and most of the Australian estimates made of these impacts are very unreliable and in some cases frankly incorrect.

• As a result, there is a substantial risk that these will be used as ‘fudge factors’, distorting the outcome in ways that allow socially inefficient, but politically attractive, projects to proceeed.

Challenge 2: Economic impact assessments

Economic impact assessments typically involve feeding a project into a large scale model of the economy and assessing what are described as effects on factors such as income and employment

In theory, the results should be the same as those of a properly undertaken CBA. In practice, they are often very different, with the economic impact assessment generating far higher benefits than the CBA

This is almost always because the economic impact assessment makes assumptions that are difficult to accept – for instance, about what would happen to the labour employed on the project absent the project. What from an economic perspective are costs (such as labour displaced from other uses) are often treated as if they were benefits

Moreover, the assumptions being made are often extremely opaque

The estimated gains are invariably over-stated and in any event, very unreliable

Challenge 3: Multi-Criteria Appraisal

Multi-criteria analysis

• Multi-criteria analysis (MCA), rather than seeking an overall monetary valuation of project effects, identifies salient elements of those effects (be they costs or benefits), scales them and then places subjective weights upon them.

• In that sense, it is a technique for scoring project attributes.

• The project is expected to be approved if the weighted sum of the desirable effects (that is, the scaled value of the various dimensions of project benefits) exceeds the weighted sum of the project’s undesirable effects (the scaled value of its costs and harmful outcomes).

The differences between CBA and MCA

The six critical differences between CBA and competitors to it such as MCA.

1. Evaluative standpoint. There is a presumption in CBA that the value policy-makers place on a policy’s effects ought to be derived from the valuations of the individuals who will bear those effects.

2. Decision-relevance. A project is wealth-increasing if its benefits exceed its costs (the ‘total test’), including the opportunity cost of pursuing alternative projects or of pursuing that project on a different scale (the ‘marginal test’).

3. Comparability. An advantage of the systematic use of CBA is that it allows consistent values to be used across projects for assessing particular project inputs or outcomes.

4. Verifiability. The assumptions used in a CBA are capable of being tested in terms of their consistency with market valuations, the way in which market valuations have been altered, and the methods used to measure and weight non-marketed inputs and outputs.

The differences between CBA and MCA (cont)

5. Accountability. As CBA estimates impacts and costs and benefits in quantitative terms and lays out all of its underlying assumptions, it can be found to be wrong after the fact.

It might turn out, for instance, that people value a new transport link by less than was assumed by a study recommending its creation. In contrast, unless a CBA had actually been undertaken it is difficult to see how one could determine that an MCA assessment was wrong after the fact. This may be a reason why MCA is gaining popularity — as a device for reducing accountability for decisions.

6. Scientific progression. CBA gets better over time because it attempts to systematically measure impacts and the values placed upon them and does so in a way that is comparable from study to study.

Criticisms of CBA

Criticisms of CBA

1. Non-commensurability.

CBA cannot deal with the non-commensurable dimensions of a policy or project evaluation, particularly those dimensions that cannot (or, for ethical reasons, should not) be given a monetary valuation.

There are circumstances in which CBA cannot be a complete answer to the problem of policy or project evaluation.

Even in those cases, however, monetizing the costs and benefits associated with those effects that are capable of being valued can still provide a better basis for public decision-making than could occur in the absence of that quantification since the decision then provides a de facto valuation of those elements that are not capable of monetary valuation.

It may be one of CBA’s strengths that it can focus attention on, and provide for explicit political resolution of, those aspects of policy-making that defy expression in monetary form.

Criticisms of CBA (cont)

2. Income distribution.

CBA treats a dollar as a dollar, regardless of who it is removed from, or accrues to.

This gives a greater weight to higher-income consumers, who have a lower marginal utility of income and hence can ‘pay more’ to secure a benefit or avoid a loss.

While this is correct:

– Infrastructure projects are a poor tool for income redistribution

– And if income distribution considerations are relevant, there are many ways of transparently taking them into account in CBA

The criticism of CBA on grounds of income distribution is not particularly telling; and even to the extent to which it does have bite, it hardly seems to justify replacing CBA by MCA.

Criticisms of CBA (cont)

3. Accuracy.

A third and final criticism of CBA is that it is based on complex assumptions, and hence likely to be inaccurate.

While this is correct, it simply reflects the difficulty inherent in assessing and valuing the effects of public projects. To that extent, it is not an argument against CBA; rather, it is an argument against public projects.

Even less is it an argument for replacing CBA by MCA:

– there is no reason whatsoever to think that MCA is any more accurate than CBA

– Rather, as noted above, the valuation weights and assumptions used in CBA are capable of being compared across projects, and as between initial projections and out-turns; neither of these can be done in MCA

– As a result, use of MCA is more likely to undermine accuracy in project appraisal than to advance it.

Conclusions: The current state of Australian project evaluation

• There are grounds for being concerned about the state of project evaluation in Australia.

• For example, there is increasingly widespread use of ‘triple bottom line’ approaches that somehow combine CBA with assessments of environmental and social impacts.

• However, CBA has always considered relevant environmental and social impacts in assessing project net benefits; to consider them twice, as occurs in ‘triple bottom line’ evaluations where projects are ranked both on the results of a CBA and on environmental and social grounds, is double counting.

• This is, of course, the antithesis of proper cost-benefit appraisal.

• The risks of poor quality assessments are compounded by the fact that there is very little transparency in project appraisal – no jurisdiction systematically publishes CBAs and even when published, it is impossible to gain access to the underlying data

• The situation is even worse for PPPs, where commercial confidentiality is used to limit disclose

Given the billions of dollars being spent, the risks are very great and merit urgent attention

About Deloitte

Deloitte provides audit, tax, consulting, and financial advisory services to public and private clients spanning multiple industries. With a globally connected network of member firms in more than 140 countries, Deloitte brings world class capabilities and deep local expertise to help clients succeed wherever they operate. Deloitte‘s more than 168,000 professionals are committed to becoming the standard of excellence.

Deloitte's professionals are unified by a collaborative culture that fosters integrity, outstanding value to markets and clients, commitment to each other, and strength from diversity. They enjoy an environment of continuous learning, challenging experiences, and enriching career opportunities. Deloitte's professionals are dedicated to strengthening corporate responsibility, building public trust, and making a positive impact in their communities.

Deloitte refers to one or more of Deloitte Touche Tohmatsu, a Swiss Verein, and its network of member firms, each of which is a legally separate and independent entity. Please see www.deloitte.com/au/about for a detailed description of the legal structure of Deloitte Touche Tohmatsu and its member firms.

About Deloitte Australia

In Australia, Deloitte has 12 offices and over 4,500 people and provides audit, tax, consulting, and financial advisory services to public and private clients across the country. Known as an employer of choice for innovative human resources programs, we are committed to helping our clients and our people excel. Deloitte's professionals are dedicated to strengthening corporate responsibility, building public trust, and making a positive impact in their communities. For more information, please visit Deloitte’s web site at www.deloitte.com.au.

Confidential This document and the information contained in it is confidential and should not be used or disclosed in any way without our prior consent.

Liability limited by a scheme approved under Professional Standards Legislation.

General information only

This presentation is provided as general information only and does not consider your specific objectives, situation or needs. You should not rely on the information in this presentation or disclose it or refer to it in any document. We accept no duty of care or liability to you or anyone else regarding this presentation and we are not responsible to you or anyone else for any loss suffered in connection with the use of this presentation or any of its content.

© 2010 Deloitte Touche Tohmatsu. All rights reserved.