Practical Issues in Implementing Performance-Based Contracting Health System Innovations Workshop...

Practical Issues in Implementing Performance-Based Contracting

Health System Innovations Workshop Abuja, Jan. 25-29, 2010

A Few Thoughts to Start• Don’t Panic: Not everything has to be perfect,

be creative! Make new & interesting mistakes – don’t repeat old ones

• Be Systematic: Write things down in a contract and contracting manual

• The Limits of Planning: Endless planning and analysis can get in the way of action & learning

• Humility: a) don’t be too sure of thingsb) knowledge must be larger than our experiencec) give people at local levels sufficient autonomyd) keep learning, evaluating, adapting

• :

Performance-Based Contracting for Health Services in Developing Countries-A Toolkit

Outline of the Toolkit1. Summary of the Toolkit – pages 1-82. What is performance-based contracting?

Definitions and Concepts – pages 9-173. How to Contract? Going through the 7 steps

of the contracting cycle – pages 19-664. Checklist for Contracting – quick summary of

tasks in contracting – pages 67-685. Whether to contract? Review of the global

experience with contracting – pages 69-96

Outline of the Toolkit• Appendix A: Example of a Contracting Manual

including an example of a contract• Appendix B: World Bank procurement approach

to contracting for health services• Appendix C: Description of Evaluated Contracting

Experiences• Appendix D: TORs for 3rd party evaluation• Appendix E: TORs for contracts involving: (i) PHC

delivery; and (ii) HIV prevention for CSWs• WWW.rbfhealth.org - tools and guidelines

http://www.rbfhealth.org/

2. Define the Services

3. Design the Monitoring and Evaluation

5. Arrange for Contract Management

7. Carry out Bidding Process and Manage the Contracts

1. Dialogue with Stakeholders

4. Decide how to Select Contractors and Establish Price

6. Draft Contract & Bidding Documents

The Contracting Cycle: A Systematic Approach (page 20)

The 6 most important mistakes1. Not clearly defining the objectives of the contract

and the indicators of success 2. Allowing contractors & purchasers to forget the

stated objectives & targets3. Limiting the managerial autonomy of contractors4. Not defining the size & location of each “lot”5. Not having a contracting plan: not indicating how

contracts will be managed & M&E will be done 6. Setting prices irrationally and not choosing the

best contractors

Defining Objectives – More than SMART

• Big advantage of contracting is results focus so concentrate on outputs not inputs.

• The purchaser should objectively define: – Quantity of services (e.g. % DTP3 coverage, skilled

birth attendance)– Technical Quality (national technical guidelines) – Equity (ensuring the poor receive services)


• Need to address the most important challenges– In Ghana 70% of HIV transmitted through FSWs,

guess what % of grants went to FSWs?

• Focus on a Few!! Indicator inflation if > 10 loss of focus, less data collection

• Bias towards outputs & outcomes not inputs & processes e.g. DPT3 coverage better than vaccine availability or “micro-planning” – the exception is in measuring “quality”


• They need to be defined in a measurable way – define numerator & denominator precisely.

• Avoid undefined terms like “functional” health facility

• Set targets broadly: 20% 58% DPT3 coverage is good even if the target was 60%

• Within the “span of control” of the contractor, e.g., measuring availability of staff if contractor not allowed hire, transfer, decide payments to health workers

Some Examples of Indicators – What’s Wrong with Them?

1. % of county hospitals with functioning x-ray machines

2. Number of health workers receiving appropriate training

3. % of patients using a PHC Center who are satisfied with the services

4. Low rate of health workers leaving their positions


• Refer to Tasks 4 and 5 (pages 26-31)• Look at table 3.2 (page 28)• Congratulations!! You’ve avoided the first

important mistake





best contractors

Ensuring Focus on Outputs• Regular (quarterly) discussion of progress on

indicators between purchaser and contractor• Carry out independent M&E• Credible threat of sanctions:– Embarrassment works!! Special meetings– Letters to NGO board– Replacement of key managers– Termination of contract

• RBC!!!! (Read the bloody contract)

Ensuring Focus on Outputs – Performance Bonuses

• Sends signals to contractors about important indicators

• Challenges of performance bonuses:– Finding indicators that are important & can be

measured reasonably frequently– Amount of bonus – enough to signal but not too

expensive – about 10%– What contractor can do with the bonus– Design of bonus: (i) improvement from baseline; (ii)

absolute target

Design of Bonus SystemLot A Lot B Lot C

Score year 0 40 65 50

Score Year 1 50 70 70Score Year 2 60 75 75Change from previous best (+10 points)

Yes, Yes

No, No

Yes,No

Absolute Target (=70 points) No,No

Yes,Yes

Yes,Yes

Change and/or above 70 points Yes,Yes

Yes,Yes

YesYes

Ensuring Focus on Outputs-Results-Based Financing

• “Transfer of money or material goods conditional on taking a measurable health related action or achieving a predetermined performance target.” - Eichler and Levine

• Contractors are provided payments based on the amount of services they actually deliver



stated objectives & targets3. Limiting the managerial autonomy of

contractors4. Not defining the size & location of each “lot”5. Not having a contracting plan: not indicating how


best contractors

Ensuring Managerial Autonomy• Decentralize management to people who are

closest to the ground reality - • Purchaser can hold contractors accountable

for results when managers have responsibility & autonomy - avoids the “blame game”

• Encourages innovation – RBF, sub-centers• Take advantage of private sector’s flexibility –

that’s why they’re called Non-Governmental Organizations

Impediments to Managerial Autonomy

• Telling contractors “how” they should deliver services (define objectives, “what”)

• Line item budgets with reimbursement of actual expenditures– Limit flexibility to move money where it’s

needed– Encourages micro-management– Increases transaction costs & arguments– Encourages focus on inputs

Lump-sum allows proper FM!!!

Example of a Line-Item Budget

Item Amount1. Staff salaries $150,0002. Drugs $40,0003. Medical Equipment $20,0004. Medical supplies $30,0005. Maintenance & Repair $10,000

TOTAL $250,000

Impediments to Managerial Autonomy

• Following Government procedures for staff hiring, firing, transfer, & pay.

• Unclear authority of purchaser’s officials– Will try to force certain approaches– Will claim power over more decisions including

staff recruitment• Government procures important inputs (allow

contractors to do procurement, use private auditors)

Drug Availability Index (max = 100) according to who is responsible for drug procurement -

Afghanistan

A Tale of 2 Countries: Afghanistan & DRC

• Both severely affected by conflict• Poor countries with limited infrastructure• Lots of donor money flowing in• Lots of NGOs• Both started contracting with NGOs to deliver

health services around 2004• Used different approaches to contracting

Government Controls on Contracted NGOs

Type of Control Afghanistan DRC

Plans and Fiduciary Controls

Annual work plans approved by Government NO YES

Annual procurement plans approved by Government NO YES

Centralized procurement of goods by Government NO YES

Reimbursement for individual expenditures NO YES

Ex ante approval of payments to health workers NO YES

Innovations have to be pre-approved NO YES

Monitoring & Evaluation

Regular quarterly review of reported results YES NO

High quality annual health facility surveys YES NO

Frequent field supervision YES NO

Change in the utilization rate for curative care (per 100 population per year) during the first

year of the contracts

2007 2008DRC

2004 2005Afghanistan

Increasing Managerial Autonomy

• See tasks 26 – 29 pages 55-59





best contractors

Size of “Lots” – Economies of Scale

• Economies of scale in price per beneficiary– Fixed management and admin. costs.

• Likely more competition lower prices• Large packages facilitates contract management• Easier & cheaper to monitor and evaluate

contractor performance with fewer lots• Likely to reduce opportunities for corruption– Unscrupulous officials will intimidate small

contractors– Easier to avoid “ghost” NGOs

Scale of Contracts – Provinces vs. Clusters in Afghanistan

• Provinces: 300,000 – 900,000 population• “Clusters” of districts – 100,000 – 180,000• Issues related to NGO capacity, equity• Bid price of provinces = $4.05 per capita per

year vs. $7.80 for clusters, high admin. costs• MOH decided cluster approach was too

expensive

Size of “Lots” – Arguments for Smaller Scale

• Concerns regarding contractor “capacity”– hard to predict & compared to what alternative

• Increased diversity – no oligopoly • No disruption to existing providers– But could suffer from “Swiss cheese”

Distribution of NGO HCs in Afghanistan

un-served

Size of “Lots” - Recommendations

• Lots should cover at least 0.5 million population

• 7-20 contracts• See task 8, pages 34-36



stated objectives & targets3. Limiting the managerial autonomy of contractors4. Not defining the size & location of each “lot”5. Not having a contracting plan: not indicating

how contracts will be managed & M&E will be done

6. Setting prices irrationally and not choosing the best contractors

Use a Contracting Plan

• People tend to focus on the contract itself and the recruitment procedures

• Fail to systematically address:1.How monitoring and evaluation will be

carried out (otherwise indicators in contract are meaningless)

2.How contracts will be managed (client’s activities not described in the contact)

Monitoring and Evaluation

• Ensure that contracts remain output and outcome focused

• Learn lessons and improve performance• Definitions: – “Monitoring” tracking the performance of

individual contractors– “Evaluation” tracking the overall progress in

service delivery of all contractors (comparison to other service providers)

The Process of M&E

• Collect baseline data early on & provide to all stakeholders (helps contractors identify important issues)

• Recruit 3rd party to help with M&E design & data collection while maintaining government oversight & involvement

• Clear responsibility for analysis of M&E data

• Ensure there’s data on effectiveness (coverage), equity, quality of care, cost

Recommendations

• M&E key to achieving good results –worth the investment needed.

• Use different sources of data – none are perfect

• Make M&E somebody’s job• See tasks 10-15, pages 37-44

Contract Management

• Often done poorly• Ministries of Health often have limited

experience and understanding of contracting• Often not clear who in the MOH is responsible

for contract management

Major Issues and Challenges in Contract Management

• Paying contractors on time• Avoiding corruption• Proper supervision, monitoring & evaluation• Solving problems – many related to

relationships• Maintaining government ownership,

oversight, & involvement while avoiding micro-management

Some Possible General Solutions to Contract Management

• Recruiting sufficient number of talented people to manage contracts

• Allow sufficient budget• Computerized contract management systems• Provide incentives to contract managers based

on results achieved by contractors, timely payment, lack of audit objections, etc.





best contractors

Determining the Price of a Contract

There are basically 3 options, each with benefits and issues:

1. Competition at least partly on the basis of price

2. Negotiation of price with selected bidder3. Fixed price where client sets price in

advance

Competition, at least partly on Price

Advantages• lowest price• transparent• encourages

innovation• reflects local realities• benefits local NGOs

Disadvantages• may take longer• may end up with

prices that are inconsistent

• may end up with prices that are too low or too high

Negotiations with Contractor

Advantages• fairly quick• reflects local

realities• if purchaser well

motivated & savvy can be low cost

Disadvantages• not transparent!!!• may end up with

prices that are inconsistent

• may end up with prices that are too low or too high

Fixed Price

Advantages• fairly quick• transparent• uniform, hence

“fair” (?)

Disadvantages• rigid, “one size fits

all”• discourages

innovation or cost savings

• hard to estimate real costs (be humble!)

Example of Competition (LCS or QCBS)

• Bidders submit technical and financial proposals• technical proposals scored by evaluation

committee• financial proposals opened publicly• In LCS, lowest price among technically responsive

bidders is selected.• In QCBS technical “score” combined with financial

score using a weight (e.g. 80/20)

Example of LCS: Minimum Acceptable Technical Score = 60

Name of

NGO

Technical Score (St)

Financial Proposal

($)

Financial Score Sf =100

x Fm/F

Total Score (S) = St x T+ Sf

x P

A* 60 $2.0 million

100 68.0

B 70 $2.5 million

80 72.0

C 75 $4.0 million

50 70.0

Example of Competition (QCBS)

Name of

NGO

Technical Score (St)

Financial Proposal

($)

Financial Score Sf =100

x Fm/F

Total Score (S) = St x T+ Sf

x P

A 60 $2.0 million

100 68.0

B* 70 $2.5 million

80 72.0*

C 75 $4.0 million

50 70.0

* 2.0m/2.5m x 100=80 and (70 x 0.8) + (80 x 0.2) = 72.0

Contractor Selection – Competition vs. Sole Source

• Bidders want transparent process with a “level playing field” competition

• Competitive process will generally lead to “best” managers, most innovative ideas, “best” organizations, best prices

• Sole source selection is quick but not fair, not transparent, creates resentment, leads to “fat & happy” contractors, limits innovation & creative thinking

Situations Where Sole Source Might Make Sense

• Where a mission clinic has been providing services for 50 years

• Limited competition possible due to ongoing armed conflict

• Contractor bringing significant funds into a partnership

Selection Criteria

• Need to be clearly defined in advance• Should not be excessively detailed • Should not be excessively difficult• Should look at:– Experience of organization (track record)– Key staff (quality of managers)– Work-plan/strategy (lowest % of score)

Evaluation Process

• Independent evaluation committee• Committee should include members external

to client/purchaser:– technical agency (e.g. WHO, UNICEF)– representative of NGO community (obviously

need to avoid conflicts of interest)

Maximizing Participation

• Prior consultations with many NGOs• Advertise widely• Hold information sessions• Don’t make process too daunting• Establish reasonable selection criteria • Allow NGOs to form consortia (< 3 or 4)• Avoid bid & performance bonds, guarantees• “Grow” local NGOs by having a few small

packages

Recommendation

• Use competition based at least partly on price whenever it’s feasible

• Modify approach to minimize disadvantages, e.g. fixed price is adjusted to reflect local variation

• See tasks 16-20 pages 44-51

Practical Issues in Implementing Performance-Based Contracting Health System Innovations Workshop...

Documents

Transcript of Practical Issues in Implementing Performance-Based Contracting Health System Innovations Workshop...