Option appraisal: making informed decisions in operate a review process during the appraisal cycle...

download Option appraisal: making informed decisions in operate a review process during the appraisal cycle and

of 36

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Option appraisal: making informed decisions in operate a review process during the appraisal cycle...

  • Option Appraisal: Making informed decisions in government


    informed Government

  • 2 Option Appraisal: Making informed decisions in government

    Our vision is to help the nation spend wisely.

    we apply the unique perspective of public audit to help Parliament and government drive lasting improvement in public services.

    The National Audit Office scrutinises public spending on behalf of Parliament. The Comptroller and Auditor General, Amyas Morse, is an Officer of the House of Commons. He is the head of the NAO, which employs some 880 staff. He and the NAO are totally independent of government. He certifies the accounts of all government departments and a wide range of other public sector bodies; and he has statutory authority to report to Parliament on the economy, efficiency and effectiveness with which departments and other bodies have used their resources. Our work led to savings and other efficiency gains worth more than £1 billion in 2010-11.

  • Contents Summary 4

    Part One

    Introduction to option appraisal in government 9

    Part Two

    Quality of option appraisal 13

    Part Three

    Use of appraisals in the decision-making process 20

    Part Four

    Management of the appraisal process 23

    Appendix One

    Methodology 33

  • 4 Summary Option Appraisal: Making informed decisions in government

    Summary introduction

    An informed government collects high quality 1 information on context, activities and results; analyses it to expose issues or opportunities; and presents informed options to decision-makers internally, and candid assessments of plans and performance externally. Without high quality information government will not be well placed to respond to the immediate challenge of cuts to funding and longer- term challenges of providing sustainable high-quality public services and creating the right climate for economic growth.

    Option appraisal is crucial to ensure government 2 interventions are fully informed and based on robust evidence. The core method of economic appraisal, cost-benefit analysis, has been designed to inform comparison between options for government interventions, recognising that in the public sector appraisals need to go beyond traditional financial analysis, and pick up broader social, environmental and economic effects which may not have ready market values. Appraisal techniques can be applied at project or programme levels, and have particular value when there is pressure to reduce costs while minimising effects on front line services or the wider economy.

    There are two primary option appraisal 3 processes in government: business cases and Impact Assessments. Generally, the former deals with expenditure, and the latter with policy impacts and regulatory burdens. They follow similar, but not the same, processes and both are based on the principles set out in government’s central guidance on appraisal, the Green Book (see Figure 1).

    The National Audit Office’s review of option appraisal

    We have regularly concluded in our value for 4 money reports that departments could either not demonstrate the value for money of government interventions or that interventions provided poor value for money due to an absence of option appraisals. For example, in our report on Reorganising Central Government we concluded that the value for money of central government reorganisations could not be demonstrated because, among other concerns, “three-fifths of arm’s length bodies did not conduct investment appraisals to compare expected costs and benefits of alternative options before taking a decision to reorganise”.1 In our report on the Coal Health Compensation Schemes we concluded that the scheme provided poor VFM for the Department of Trade and Industry because it “set out the pay compensation without a systematic in-depth option appraisal being considered at more senior levels within the Department”.2 We have also reviewed Impact Assessments annually since 20023 and our most recent report (July 2010) found that the quality of Impact Assessments had improved but remained variable.4

    This inconsistency in appraisal prompted us to 5 review a broader set of recent appraisals, and seek views on the appraisal process from Chief Economists and their staff, and from policy and finance officials. Our methodology is presented in Appendix One. This report is aimed at staff involved in option appraisal at working level or responsible for options appraisal within a department. There are specific findings which will be of interest to the Treasury and Better Regulation Executive as they revise guidance for option appraisal across government.

    1 Comptroller and Auditor General, Reorganising Central Government, HC 452, National Audit Office, March 2010, paragraphs 7 and 10. 2 Comptroller and Auditor General, Coal Health Compensation Schemes, HC 608, National Audit Office, July 2007, paragraph 9. 3 http://www.nao.org.uk/areas_of_specialist_expertise/regulation/about_regulation.aspx 4 Assessing the Impact of Proposed New Policies, HC 185, National Audit Office, July 2010, paragraph 6.

  • Option Appraisal: Making informed decisions in government Summary


    Figure 1 The current option appraisal landscape

    Green Book sets out appraisal principles (owned and managed by HM Treasury)

    Decisions ultimately approved by HM Treasury and/or government depending on delegated authority

    Decisions ultimately approved by Reducing Regulation Committee and/or Parliament

    Source: National Audit Offi ce

    Business case guidance lays out structured framework for designing and assessing spending interventions

    Impact Assessment guidance and template lays out structured framework for assessing policy impacts and regulatory burdens

    Primarily regulated within departments with major business cases scrutinised by HM Treasury spending teams, Gateway review teams, Major Project Authority depending on level of risk, cost and delegated authority

    All Impact Assessments scrutinised by Regulatory Policy Committee

    Investment regime guidance owned primarily by HM Treasury (and in part by OGC – now MPA)

    Regulatory regime framework owned by Better Regulation Executive

  • 6 Summary Option Appraisal: Making informed decisions in government

    Key findings

    Established systems for option appraisal seek to promote good decision-making

    Central guidance on cost-benefit analysis is 6 thorough, and there are established systems to make sure that policy, regulation and expenditure projects are appraised. Many departments supplemented central guidance with material relevant to their sector. Most staff we consulted understood the need for appraisal, and the main administrative routes that they had to follow. Structural developments, such as the creation of the Regulatory Policy Committee, and in some departments, Investment Committees, had tended to emphasise the importance of good appraisal. Significant failures to implement appraisal guidance or take account of policy commitments can result in judicial reviews.5

    The quality of the appraisals we reviewed varied 7 and the findings below highlight the need to sharpen the requirements for completing appraisals, and tighten management of its application, to secure full value from the process. While we judged 23 of 53 cost-benefit analyses presented in the appraisals we reviewed to be good or better, 30 contained weaknesses. We did not find any significant link between the size of a project and the quality of appraisal. Others have also noted problems with appraisals; for example, a recent review of 189 Impact Assessments by the Regulatory Policy Committee judged 44 per cent of appraisals not fit for purpose.6 The main weaknesses we found were:

    Inadequate development of options against ¬¬

    which to judge the preferred course of action;

    Lack of monetisation of burdens and benefits ¬¬

    – over 40 per cent of those involved in the appraisal process we surveyed did not agree that sufficient time and effort went into monetising impacts;

    Unstructured qualitative analysis – while ¬¬

    qualitative arguments were influential in a large proportion of cases, few followed guidance on ways to structure that analysis, or applied a qualitative structure consistently to all options considered.

    Staff perceived business cases to be more 8 relevant to decisions than Impact Assessments. They told us that this was partly because Impact Assessment requires standard processes to be applied to all policies, regardless of their potential size or impact, and was therefore very inflexible. Business cases, by contrast, were subject to a series of delegations graded by financial size and risk, and associated differences in the level of scrutiny. Both processes, however, provide for the depth of analysis to be ‘proportionate’ to the significance of the decision. The challenge has been to define ‘proportionate’ logically and in a well-informed way: the central guidance does not cover this issue well.

    Management of the appraisal processes by departments

    Good planning helps to integrate appraisals 9 into the decision-making process. Many departments had no standard process across their organisation to manage the development of Impact Assessments, but those that had such a process produced, on average, stronger appraisals. When regulation and spending were required within the same intervention