Haiti: Paying NGOs for results

January 29, 2010Jaipur, India

Rena Eichler, PhDBroad Branch Associates

renaeichler@broadbranch.com

.

2

Overview

• USAID funded project (1995-2010) to strengthen capacity to provide essential health services.

• 2.7 million people covered by 2005• NGOs are contracted to provide a defined package of services.

Technical assistance is provided to enhance institutional capacity.

• Beginning with a pilot in 1999, progressively more NGOs are paid partly based on results each year.

• Design changes happened along the way (indicators, targets, validation and payment rules) as more was learned.

• Results from 1999-2005 will be presented.• RBF approach continued until earthquake.

Model is relevant for public payment to public facilities as well as for contracting private providers.

3

Funding mechanism before RBF

• Extensive and detailed budget negotiations • One-year contract for service provision• Initial (1 month) advance of funds• NGO submission of monthly financial vouchers• Cost reimbursement up to a negotiated budget

ceiling• Routine financial verifications and financial audits

4

Limitations of initial funding mechanism

• Emphasis on securing and accounting for funds.

• Little motivation to improve cost-effectiveness, encourage innovation, and promote accountability for results.

• Weak incentives to improve management and promote organizational development.

• Weak incentives to improve clinical quality and quality as perceived by customers.

5

How did these limitations play out?

• Wide variation in performance and results: 1997 Population-based survey found variations in performance (23 NGO service areas), which indicated improvement was possible.

– Full immunization coverage ranged from 7% to 70%.– Contraceptive prevalence rates ranged from 7% to 25%. – Prenatal care ranged from 21% to 43%. – Institutional delivery ranged from 53% to 87%.

• Little or no correlation between costs and results– average costs per visit range from $1.35 to $51.93!

• Communities health needs not met • “Legitimate” excuses for non-performance• Business as usual, few innovations

6

Launch of a three-pronged strategy to improve results

• Technical assistance for capacity building• Cross-fertilization for learning and technical

exchange within the NGO Network• A change in payment structure from

reimbursement for documented expenditures to payment based partly on achievement of negotiated performance targets

7

Results Based Financing model

• Utilizes pre-established, negotiated objectives• Increased management flexibility for strategy

formulation and resource allocation• Financial incentives for achieving performance

targets• Financial risks for not achieving targets• Relies on independent verification of performance

data by a third party • Strengthened performance management and data

validation systems

8BROAD BRANCH ASSOCIATES

Pilot phase in 1999- Stakeholder consultations were critical• NGOs perceived to be well managed were

consulted.• Incentive based approach was presented for

feedback.• Agreed that if there was the potential to earn

more, some financial risk would be acceptable.

• Three NGOs were chosen and agreed to be part of an initial pilot.

9

Payment formula (pilot phase in 1999)

Potential payment received by NGO = 95% of historical budget + award fee.

Maximum award fee = 10% of historical budget.

(Note: 5% is “withheld” and an additional 5% can be earned)

In later phase, NGOs received 94% fixed and 12% “at risk”

10

Award Fee is earned when targets are reached1999 pilots and award fee distribution

11

Simple example

• Historic budget = $1000• Fixed portion: .95 (1000) = 950• Award fee: .05 (1000) + .05 (1000) = 100• Maximum potential earnings = 1050

Payment Frequency: quarterly payments for fixed portion. Award fee paid once per year.

12

Immunization coverage (1999 pilot)

13

3 + prenatal visits according to MOH norms (1999 pilot)

14BROAD BRANCH ASSOCIATES

Award Fee paymentAttained Award Fee

Award Fee

attained* NGO 1 90% NGO 2 70% NGO 3 80%

* If actual performance was within one

confidence interval, the NGO was awarded the bonus for that indicator.

15

Five design and implementation phases• Phase I: Pilot phase (pilot):1999

– Standard list of health indicators– Performance verified by independent firm with

community-based surveys• Phase II: April 2000-December 2001

– Standard list of health indicators– NGO self-reporting complemented by validation by an

independent firm• Phase III: January 2002-December 2003

– Random selection of health indicators from an expanded list

– Addition of a standard list of management indicators– 50-50 split between health and management indicators for

payment

16

Five design and implementation phases

• Phase IV: January –December 2004

– Random selection from 2 “packages” of indicators including both health and management

• Phase V: January-December 2005

– All NGOs in RBF, payment conditional on benchmarks plus performance targets that were the same for all NGOs, amount at risk increased to 12%.

17

Administration

1. Selecting Recipients: Before 2004 NGOs had to meet preconditions related to management, financial management and information systems, and results achieved.

2. MOUs, performance agreements and contracts: Contract Management unit designs contract template and oversees compliance with terms.

3. Results reporting, monitoring and validation: Post pilot-NGOs report results monthly to monitoring unit. Monitoring unit enters data into a tracking system. Independent auditors contracted to conduct random audits on reported data.

18

Administration continued

4. Payment Generation: Financial Management unit receives information from contract unit about targets attained as specified in contract. Contract unit receives info about achieved results from monitoring unit. Payment is triggered when results specified in contracts are achieved.

5. Assessment and Revision: Combination of consultation with NGOs, assessment of results and processes, and cross-network learning.

19BROAD BRANCH ASSOCIATES

Selected indicator change (1999-2004)

NGO Immunization 3 prenatal visits Years 1 49to 88 49 to 36 5 2 40 to 90 32 to 72 5 3 35 to 98 18 to 98 5 4 37 to 84 17 to 57 4 5 73 to 88 38 to 88 4 6 54 to 102 25 to 76 4

7 27 27 to 21 1 (contract cancelled)

8 50 to 107 44 to 54 3 9 78 to 71 36 to 40 3

11 49 to 65 61 to 74 1 12 69 to 94 69 to 93 1 13 N/A 53 to 75 1

20

Results after six years of implementation in Haiti

• Introduction of RBF marked a significant improvement in NGO documented performance (against all odds)

• NGOs in the project performed consistently better than all of Haiti

• NGOs under RBF consistently outperformed those under cost reimbursement (in same project)

• NGOs usually showed a large jump in performance between the year before RBF and the first year under RBF

• Overall Project performance improved significantly in 2005 when almost all NGOs were in RBF

21

A glimpse of NGO feedback and what happens inside the black box…• Consensus that the focus on results provides

incentives to strengthen information and to use it.• Financial incentives for staff implemented: One NGO

establishes targets for each service delivery point, reviews data and holds discussions monthly, helps establish strategies to reach performance goals. Staff receive a 14th month of pay if SDP target is reached.

• Another NGO views RBF as a “catalyst for improved management”.

• Expressed disadvantages include “institutional stress” generated by having to achieve targets and that payment for achieving each target is “all or nothing”.

Major Accomplishment

Despite fragile government, weak infrastructure, security problems and natural disasters, innovative and flexible implementation efforts continued to provide essential health services & deliver results

USAID external evaluation - 2006

Thank You!