Using Performance Information for Management and Budgeting - Canada’s Experience

Lee McCormack, Executive Director

Results-based Management

OECD Presentation, May 3, 2006

2

Overview

The Canadian context

Problems, priorities and implementation challenges

Lessons learned

3

The Canadian context

4

Canada is a big country with relatively few people and a highly decentralized government

• 32 million citizens in 5 ½ time zones

• Decentralized federation with 10 provinces and 3 territories

• Provinces are equal to the federal government and have real power

• Hundreds of federal-provincial agreements and federal programming is weighted towards transfer payments

• Over 90 departments and agencies in the federal government– 46 Crown corporations– 172 000 public servants

• Eight consecutive federal surpluses

• New Conservative minority government

5

The majority of federal expenditures are transfer payments…

• Total net expenditures in 2004-05 were $172.8 B, of which 56% ($96.8B) were transfer payments (including major transfers & grants and contributions)

International Organizations and Foreign Countries

16%

Provinces and Territories

16%

Industry13%

Persons19%

Municipalities and Local Govts

3%

Non-Profit Organizations

33%

Major Transfers to Persons (EI, OAS, GIS and

spouse's allowance)22%

Public Debt Charges17%

Other Program Expenses27%

Grants and Contributions13%

Major Transfers to other levels of Government

(CHST, Fiscal arrangements, etc)

22%

Total Net Expenditures: $172.8 B Grants and Contributions: $22.5B

Transfer payments

6

The federal expenditure management system is also decentralized

Element Responsibility

Developing an annual Budget and a multi-year fiscal framework

Department of Finance

Establishing annual departmental reference levels

Treasury Board Secretariat and departments – approved by Treasury Board

New policy or program approval phase Privy Council Office supporting Cabinet

Allocating and reallocating to ensure alignment with priorities and aggregate expenditure control

Departments routinely – plus three central agencies in special cases

Seek Parliamentary approval of spending plans Treasury Board Secretariat

Reporting to Parliament on spending plans, actual expenditures, and results achieved

Treasury Board Secretariat and Departments

7

The Finance department sets the Fiscal Framework and manages the annual budget process

Budget Speech

Budget Bill

Main Estimates

Release of Budget Consultation Papers

(Fall Update)

Cabinet Review of Budget Strategy

Cabinet Review of Priorities

Budget Consultation Process

Final Decisions on Budget

Focus on the “increment”

No formal mechanism requiring incorporation of performance information into Budget process

Not many targets

February

October

Aug-Sept

Oct-Dec

Dec-Jan

Jan-Feb

8

Treasury Board Secretariat monitors the ongoing “A-Base” and seeks Parliamentary supply

Main Estimates

Interim Supply

Departmental Performance

Reports, Canada’s Performance and Public Accounts

Reports on Plans and Priorities

Full Supply Bill

Pressure Management Dept Baseline

Funding

A-Base Perspective Input to New Policy

Proposals

Supplementary Estimates and

Supply Bill

A-Base Perspective Input to Budget

Supplementary Estimates and

Supply Bill

Substantial planning and performance information given to Parliament

Much performance information available, especially on G and C programs

Room to develop more systematic cyclical review

February

Aug-Nov

Aug-Nov

March-June

Nov-Dec

Oct-Jan

Jan-Feb Feb-Mar

9

Canada has a relatively mature performance measurement and reporting infrastructure

• All major departments and agencies– Have internal audit and evaluation units– Publish their audits and evaluations– Produce and explain to Parliament upon request their reports on plans and priorities and

departmental performance reports– Are developing outcome frameworks that are consistently based across government

• Treasury Board Secretariat– Reviews program and management performance, focusing primarily on G and C

programs– Has a strong policy centre for results based management– Produces an annual performance report – Canada’s Performance– Assesses management performance across government, including results-based

management capacity

• Office of the Auditor General– Produces performance audits of departments and agencies

10

Although there is long results management history …

Management, Resources and Results Structures – regaining detailed program level knowledge

Possible Federal Accountability Act 2006 – cyclical evaluation of all transfer payment programs

Improved Reporting to Parliament – moving to whole of government planning and reporting

Management Accountability Framework – assessing management performance across government

Canada’s Performance – linking programs to societal performance

Results for Canadians – results-based management as a stated priority of government

Transfer Payment Policy – performance frameworks and evaluations reviewed by TBS

Modern Comptrollership – investment in management practices and controls

Program Review – dealing with a large deficit through major expenditure cuts

Departmental Reporting to Parliament – moving to results-based plans and reports on performance

Planning Reporting and Accountability Structures – unsuccessful try at linking resources to results

First Program Evaluation Policy – government-wide implementation

… some real challenges remain

Early to Mid 80s

Mid 90s

2000

Today’s Agenda

11

Problems, priorities and implementation challenges

12

Our agenda focuses on four main problems that are limiting us from getting performance information into our budget and management processes

• To regain our detailed whole-of-government understanding of the ongoing program base we are implementing the Management, Resources and Results Structure (MRRS)

• To have better ongoing measurement, periodic evaluation and use of performance information we are renewing the evaluation function

• To help Parliamentarians fulfil their role of holding the Government to account we are transforming the existing reporting regime through the Improved Reporting to Parliament Project (IRPP)

• To ensure the right management incentives are in place we are implementing the Management Accountability Framework (MAF)

Agenda

• MRRS

• Evaluation

• IRPP

• MAF

13

How do you regain a detailed understanding of the program base?

• To manage its programs, the government needs to know:

– What they are– How they align to intended outcomes– What they cost– What results they achieve

• This should support more informed decisions on value for money, program effectiveness and possible ways of improving

• Treasury Board approved the Management, Resources and Results Structure policy (MRRS) to remedy this situation

• The policy is mandatory for appropriated organizations and entering the second year of implementation

MRRS

Evaluation

IRPP

MAF

14

The approach requires consistent, structured information for every department and agency

Program Activity Architecture (PAA) + Financial and non financial information

Planned and Actual Results

Planned & Actual Financial information

Governance

Required for all elements and all levels of the PAA

Program Activity Architecture

D - 1

D - 2

D - 3

D - 4

D

Sub - Sub – Sub – Program Activity

Sub - Sub – Program Activity

Sub – Program Activity

Program Activity

Departmental Strategic Outcome

M

Department or Agency

Recoding of departmental information systems will be necessaryCentral Expenditure Management Information System (under development)

Required for the highest or “program activity” level

MRRS

Evaluation

IRPP

MAF

15

MRRS is the basis for planning, resource allocation and reporting across the government

DEPARTMENTSManagement Tool

Improve program performance Better alignment of resources & priorities

Easier reallocationMore evidence based reporting

FinancePrivy

Council Office

Budget Plan

Budget Office FunctionInformed decisions and

investment choices based on priorities and value for money

PARLIAMENTReporting Tool

Stronger accountability on spending and results

Common basis for planning &

reporting inside & outside

More logical & consistent basis

for interaction

MRRS

MRRS

Evaluation

IRPP

MAF

16

Implementation faces four main challenges

• Cultural– Some departments have had difficulty in adopting a performance

measurement culture– Many tend to identify too many measures and actually measure too few– Transparency leads to accountability and that can be scary– Estimate it will take 3 – 5 years to establish a government wide

performance measurement framework

• Institutional– Departments are managed in vertical, organizational silos and some

are having difficulty converting to logic based program structures– Over the last 2 years 30% of departments have made changes to the

basic program activity architecture – not unexpected– Performance measurement capacity – demographic concerns – 3-5 years to stabilize to long term and enduring structures

MRRS

Evaluation

IRPP

MAF

17

Implementation faces four main challenges

• DefinitionalWhat is a program?

How do you measure the effectiveness of corporate services?

How do you treat overheads consistently?

What is the lexicon of program descriptions? (dozens of other questions)

• Technical– Developing a system capable of collecting, sorting and linking

information

– Making sure it talks to other systems – annual budget, public accounts and others

– Recoding of departmental systems is a large job – pilot projects

MRRS

Evaluation

IRPP

MAF

18

The Federal Accountability Act, if passed, will increase scrutiny of program performance …

• Auditor General to inquire into the use of funds by third parties under any funding agreement with the federal department

• Parliamentary Budget Office to provide analysis to Parliament about the nation’s finances and trends in the national economy

• Departments to evaluate, at least once every five years, the relevance and effectiveness of its grants and contributions programs

… and this will put a strain on evaluation capacity

MRRS

Evaluation

IRPP

MAF

19

There are some real challenges for the evaluation function

• Quality, timeliness and strategic focus issues make it difficult to

use evaluation to support decision-making:– often focused on small programs and not strategic – can take too long to complete and are difficult to understand– can be self serving when funded by program managers

• Government wide capacity issues have made it difficult to deliver– Lack of trained evaluators – No consistent competencies for those who lead the

evaluation function – Definition of the evaluation “product” hasn’t changed much in

20 years

MRRS

Evaluation

IRPP

MAF

20

Step #1 is to renew our policy so that evaluation is…

• Better recognized as a core function of public management

• Embedded in the decision-making process

• Linked to budgeting and expenditure management

• A neutral and independent function, aligned to government management priorities

• Timely in its production of a quality product

MRRS

Evaluation

IRPP

MAF

21

Step #2 is to zone in on implementation

• Make sure the function is properly resourced• Build capacity by re-establishing training and recruitment programs

and partnering with universities on certification• Ensure objectivity – evaluators fund evaluations, period

• Redefine the “product” to promote relevance and better coverage:– Strategic policy evaluation– Impact evaluation– Value-for-money assessments– Implementation evaluation

• Treasury Board Secretariat to assess quality and set out clear competencies for Heads of evaluation and evaluators

• Introduce a Government wide evaluation plan

MRRS

Evaluation

IRPP

MAF

22

Canadian evaluators are ready for change but they will face real implementation challenges

• Demographic - High turnover rates will need to be met with targeted whole of government recruitment

• Workload – Will be difficult to manage without a realistic plan

• Skills – Will take time to rebuild training programs and partner with universities

• Jurisdictional – Continually reinforcing the distinction between internal audit and evaluation

• Relevance – Good quality, timely products need to be used to inform allocation and reallocation decisions

MRRS

Evaluation

IRPP

MAF

23

Parliamentarians want better public performance reporting, ideally …

• Simpler, more integrated information

• More context and analysis (trends, graphs, including reporting on horizontal issues)

• High level overviews with the ability to drill-down to more detail

• Clearer logic between planning and performance reporting documents

• More balanced reporting

• Better links between programs, resources and results

• More clarity between parliamentary reports and estimates votes

MRRS

Evaluation

IRPP

MAF

24

Whole of Government planning and reporting is part of the solution …

Government could better communicate its overall “strategic plan”

This “strategic plan” could be akin to a whole of Government Report on Plans and Priorities:– Outlining plans and priorities for the coming fiscal year– Linking planned results to planned expenditures– Electronic, in every parliamentary office

Logically, a fall whole of Government performance report could present the results and actual spending vis-à-vis such a Plan

… but this solution depends on a stable, whole of Government framework

MRRS

Evaluation

IRPP

MAF

25

This framework has been tested in Canada’s Performance over the past few years

MRRS

Evaluation

IRPP

MAF

26

This agenda depends on better management overall: the Management Accountability Framework

Public Service Values

By their actions departmental leaders continually reinforce the importance of PS Values and Ethics in the delivery of results to Canadians (e.g.: democratic, professional, ethical and people values).

Learning, Innovation and Change Management

The department manages through continuous innovation and transformation, promotes organizational learning, values corporate knowledge, and learns from its performance.

Results and Performance

Relevant information on results (internal, service & program) is gathered and used to make departmental decisions, and public reporting is balanced, transparent, and easy to understand.

Stewardship

The departmental control regime (assets, money, people, services, etc.) is integrated and effective, and its underlying principles are clear to all staff.

Policy and Programs

Departmental research and analytic capacity is developed and sustained to assure high quality policy options, program design and advice to Ministers.

Accountability

Accountabilities for results are clearly assigned and consistent with resources, and delegations are appropriate to capabilities.

Citizen Focused Service

Services are citizen-centred, policies and programs are developed from the ‘outside in’, and partnerships are encouraged and effectively managed.

People

The department has the people, work environment and focus on building capacity and leadership to assure its success and a confident future for the Public Service of Canada.

Risk Management

The executive team clearly defines the corporate context and practices for managing organizational and strategic risks proactively.

Governance & Strategic Direction

The essential conditions – internal coherence, corporate discipline and alignment to outcomes -- are in place for providing effective strategic direction, support to the Minister and Parliament, and the delivery of results.

MRRS

Evaluation

IRPP

MAF

27

The Management Accountability Framework is results based and demands attention

• All departments are assessed against a set of indicators including the quality of Management, Resources and Results Structures, program evaluation and reports to Parliament

• Discussions with senior officials identify management improvement priorities

• Treasury Board Secretariat feeds this input into performance appraisals for the most senior officials

• So far, so good but there are some implementation issues:– Process takes time– Resource intensive– Debates on ratings with departments

MRRS

Evaluation

IRPP

MAF

28

Lessons Learned

29

There are several lessons learned from recent Canadian experience

• There is no end point on results based management – persistence over many years is required and you never get it “right”

• You need central leadership to build capacity – someone with authority must set the game plan, make and enforce policies and invest

• A detailed understanding of the program base is essential – it is very easy to lose and not easy to get back

• Everybody knows about cultural barriers but don’t underestimate the challenges with systems

• You need a common whole of government planning and reporting framework if you want to do real strategic planning and reporting

• There is no substitute for evaluation but you need to give it some backbone and support

• Parliamentarians and the external auditor demand better public performance information and this is good

• There is no managing for results without sound management practices period – you need clear expectations and annual assessment

30

Useful resources

• Government of Canada - http://canada.gc.ca/main_e.html

• Treasury Board Secretariat - http://www.tbs-sct.gc.ca/index_e.asp

• Finance (Budget) - http://www.fin.gc.ca/fin-eng.html

• Reports on Plans and Priorities (2005-06) - http://www.tbs-sct.gc.ca/est-pre/20052006/deptlist_e.asp

• Departmental Performance Reports (2004-05) - http://www.tbs-sct.gc.ca/rma/dpr1/04-05/0405dpr-rmr_e.asp

• Canada’s Performance - http://www.tbs-sct.gc.ca/report/govrev/05/cp-rc_e.asp

• Results-Based Management (MRRS, Evaluation, Improved Reporting to Parliament) http://www.tbs-sct.gc.ca/rma/rbm-gar_e.asp

• Management Accountability Framework - http://www.tbs-sct.gc.ca/maf-crg/index_e.asp

• EMIS - http://www.tbs-sct.gc.ca/emis-sigd/index_e.asp

31