1

Delivering on Results – Evidence-Based Decision Making through Better Metrics Organisation for Economic Co-operation and Development November 2015

UNCLASSIFIED

Kiran Hanspal Executive Director - Planning, Performance, and Evaluation Treasury Board of Canada Secretariat

Current Context

2

UNCLASSIFIED

The newly elected Federal Government’s platform signalled that transparent, evidence-based decision-making will be integral to how Canada will be governed

- Prime Minister Justin Trudeau

‘‘

’’

We will make decisions using the best data

available and will invest only in programs proven

to offer good value.

‘‘

’’

Responsible governments rely on sound data to make

their decisions. We will release to the public key

information that informs the decisions we make.

3

SECRET

Advice to Ministers

The President of the Treasury Board (TB) has been tasked by the Prime Minister with leading efforts to:

• Ensure that departments and other federal organizations are able to use the best available information; and

• Strengthen the culture of measurement, evaluation and innovation in program and policy design and delivery

Within Canada’s overall Expenditure Management System (EMS), TBS supports the TB Cabinet Committee in effectively allocating spending in a manner that ensures efficiency, effectiveness and accountability by: • Establishing and monitoring adherence to TB management policies (financial & non financial); • Supporting TB approval of detailed operational plans and recommendation of resource

appropriations for new programs ; and • Supporting TB determination of resource needs and investment opportunities for existing

programs

The EMS relies on sound performance data to provide TB Ministers with the information required to make good decisions • However, this data is often incomplete, inconsistent and not focussed on real results for

Canadians

3

Role of the Treasury Board Secretariat (TBS) and Mandate of the President

UNCLASSIFIED

4

SECRET

Advice to Ministers

4

Four Decades of Evidence Based Performance Management

• New Government - Emphasis on evidence-based decision-making, transparency, and accountability

• Deficit Reduction Action Plan – Moving the government towards budgetary balance

• Financial Crisis – Fiscal stimulus

• Cyclical spending reviews – Controlling spending which had grown since Program Review

• Federal Accountability Act 2006 - Cyclical evaluation of all transfer payment programs

• Management, Resources and Results Structures – Detailed program level knowledge

• Management Accountability Framework – Assessing management performance across government

• Canada’s Performance – Linking programs to societal performance

• Results for Canadians – Results-based management as a stated priority of government

• Transfer Payment Policy – Performance frameworks and evaluations reviewed by TBS

• Modern Comptrollership – Investment in management practices and controls

• Program Review – Dealing with a large deficit through major expenditure cuts

• Improved Reporting to Parliament– Moving to results-based plans and reports on performance

• Planning Reporting and Accountability Structures – Unsuccessful try at linking resources to results

• First Program Evaluation Policy – Government-wide implementation Late 70’s

Mid 90’s

2000

Mid 2000’s

Last Ten Years

2015

UNCLASSIFIED

5

SECRET

Advice to Ministers

5

Current Strengths

• Management Resources and Results Structure (MRRS) – presents a well-

established and effective mechanism for parliamentary reporting and

public accountability • Reports on Plans and Priorities (RPP) – Forward Looking

• Departmental Performance reports (DPR) – Retrospective

• Program Alignment Architecture (PAA) & Performance Measurement

Framework (PMF) – provide a coherent picture of government

programming and of how each program’s expected results are organized

to achieve a department’s mandate

• Complementary measuring and performance reporting processes –

Estimates Process, Evaluation Process and Management Accountability

Framework Process

UNCLASSIFIED

6

SECRET

Advice to Ministers

Quality • Varying quality of indicators and data – including the lack of standardization – have

hindered assessments of program effectiveness and efficiency • Since 2009-10, nearly half of all evaluations have noted insufficient data to assess

program effectiveness and efficiency

Access • High quality, integrated financial and non-financial data and information is not readily

available or accessible to key decision-makers (TB, Deputy Heads)

Utility • The existing PAA/PMF structure has been effective in terms of its utility for reporting

but it has been less useful for decision-making due to a lack of relevant metrics • Large proportion of proposal for new funding do not provide adequate plans for

measuring performance and are not sufficiently evidence-based

Administrative and Reporting Burden • The present requirements for measuring and reporting require a significant

investment in time and resources • The perceived value of measuring and reporting is not commensurate with this

investment

6

Opportunities for Improvement

UNCLASSIFIED

7

UNCLASSIFIED

7

Focusing on the whole

The majority of annual program spending occurs without detailed Treasury Board scrutiny

More Comprehensive Tools Are Required to Build on Our Strengths, and Capture Opportunities

We need better tools for … • credible, regular assessments of key aspects of performance

• measuring and improving the back office (internal services)

• comparing performance across disparate organizations and types of programs

• facilitating the use of performance information in decision making

• transparency and increased access to performance information

$6.9Bn Closely Scrutinized by TB*

*2013-14 Supplementary Estimates (A, B, C)

$232.5Bn Not Closely Scrutinized by TB**

*2013-14 Main Estimates ** 2013-14 Statutory Spending

8

SECRET

Advice to Ministers

8

‘‘

’’

Leveraging and creating relevant information to

support sound evidence-based decision-making,

accountability and transparency … founded on

benchmarks, advanced costing methods, and

performance metrics

The Vision

• Enhancing the, utility and accessibility of information products which describe government operations

• Improving the reliability and standardization of key underlying data and information

UNCLASSIFIED

9

SECRET

Advice to Ministers

9

Three Key Elements to Our Approach

1. Metrics

• What we mean: Fundamental data and information (e.g., $, personnel, time, inputs/outputs, etc.).

• Standardized costing and standard measures for efficiency, effectiveness, and service

2. Analytics

• What we mean: Information derived from metrics (e.g., roll-up indices, correlations, etc.), and tools for comparing like with like (benchmarking).

• For departmental management, and the budget office

3. Information Products

• What we mean: Incorporating the metrics and analytics into existing data systems and tools which support decision-making

• Extending the breadth and depth of routine oversight within departments, and by central agencies of government

UNCLASSIFIED

10

SECRET

Advice to Ministers

Moving forward with a ‘Metrics Agenda’

10

Standardized metrics

Outcomes

Efficiency Service Standards

< ------- All Government Programs ------ >

Other programming types

Services to Citizens (including Transfer

Payments) Internal Services

Functions

Standardization work is well advanced on back office/internal services functions, and is being followed-up for service-oriented programs, including

those which administer transfer payments

Applied to …

UNCLASSIFIED

11

SECRET

Advice to Ministers

Standardized Metrics Internal Services and Services to Canadians (including Transfer Payments)

For Back Office/Internal Services: • Guide on Internal Services Expenditures: Recording, Reporting and Attributing which is

being modified to better support comparison of like-with-like • Updated draft standard performance measures for 7 of the 10 categories of internal

services for implementation in fiscal 2016-17, and with an initial emphasis efficiency measures • operating cost of the function / operating cost of the department • personnel delivering the function / personnel in the department

For Service to Canadians: • Developed two standardized efficiency measures and one timeliness measure for

significant services to citizens (including transfer payments) • Phased government-wide implementation of measures with the most materially-

significant organizations scoped for implementation in fiscal 2016-17

11

UNCLASSIFIED

12

SECRET

Advice to Ministers

The ‘Metrics Agenda’ is part of our increasing focus on evidence-driven decision making

Manage operations better - Better context on departments’ operating situations = better assessment/challenge of their spending proposals

Shape government-wide policy together - Everyone has the same starting point, the same snapshot of current conditions

Take stock of enterprise-wide trends - Impact of specific measures or initiatives at the government level can sometimes be best identified and understood through graphical presentation styles

12

UNCLASSIFIED

13

SECRET

Advice to Ministers

Contacts

• Expenditure Management Sector (EMS)

mrrs-sgrr@tbs-sct.gc.ca

• Office of the Comptroller General (OCG)

fin-www@tbs-sct.gc.ca

UNCLASSIFIED

13

14

14 http://www.tbs-sct.gc.ca/ems-sgd/edb-bdd/index-eng.html#start

TBS InfoBase provides a quick and easy access to detailed information on government spending and people management

Annex A TBS InfoBase

UNCLASSIFIED