Monitoring & Evaluation for Equity & Effectiveness ME3
description
Transcript of Monitoring & Evaluation for Equity & Effectiveness ME3
Monitoring & Evaluation for Equity & Effectiveness
ME3The F1 M & E System
Rationale
• Need to measure progress, not merely on reform strategies, but more on how it has helped us attain the goals and results we want to achieve
• Need to provide data to stakeholders for outcomes valuable to the poor and the Filipino people, for which they are accountable for
•The F1 reforms were instituted to help us achieve goals faster and better for the poor
II. GOAL
To establish how reforms have improved outcomes for the poor, and how reforms have equitably and effectively achieved goals in the health systems
.
Measuring: EquityEquity
how reforms have improved outcomes for the poor
EffectivenessEffectiveness how reforms have achieved goals in the health systems
Purpose
1. Make available to all stakeholders of F1 relevant data on the health sector’s progress towards attaining improvements in F1 goals
2. Establish, improve, develop and maintain systems for generating, collecting, reporting, analyzing and using data to enhance attainment of F1 goals
Objectives
1. Defining Performance Indicators
2. Establish contribution of stakeholders to outcomes and outputs
3. Report to clients and stakeholders
4. To assess F1 policy levers for revision or augmentation
Performance Indicator Framework
HEALTH STATUS
Financial Risk Protection
ResponsivenessFinal
Performance Outcomes
SERVICEDELIVERY
Governance
Financing Regulation
Major Final Outputs
Access Quality Efficiency Financial Burden
Intermediate Performance Outcomes
HEALTH STATUS FINANCIAL RISK PROTECTION (FRP)
RESPONSIVENESS
Improved Health and Reduced Deaths(eg . CMR)
Equitable Financing or FRP for all
Client Satisfaction
Respect for Clients
ACCESS QUALITY
EFFICIENCY
FINANCIAL IMPACT
Protect Children from Immunizable Diseases(eg FIC)
Full govt. subsidy
SERVICE DELIVERY REGULATION
FINANCING
GOVERNANCE
Immunization Services to cover all 0-18 mos.(eg.# hours immunization services per RHU)
Delivery Services (eg.%LGUs received vaccines on time)
Vaccine Procurement Services (eg. #vials procured)
FINAL OUT-COMES
INTERMEDIATEOUT-COMES
MFO-LGUMFO-CHDMFO-CO
PIF
IV. COMPONENTS OF THE ME3
B. Scorecards and Performance Assessment• A tool for measuring and reporting on performance of
stakeholders for outcomes and outputs that can be attributed to them and for which they are accountable for
• Measure performance on outcomes valuable to the stakeholders clients and convey performance in a manner that clients and stakeholders can easily comprehend
• Serves as Tool for Performance Benchmarks, and for Accountability to clients
• Subset of indicators taken from overall PIF
Scorecards
• Categories of performance levels– Color Coded: Green, Yellow, Red– Easy Label of Status and Link to Decisions
• Challenge:– Selecting 10 Indicators from PIF
• That reflect performance
• Valuable & Understandable to clients
– 10 Themes with 2-3 summated PIs under it
IV. COMPONENTS OF THE ME3
C. Methods for collecting, storing and reporting on results
• Population or household surveys, facility surveys, routine records and reports in the health system, special studies, etc. shall be utilized
• Additional data sources shall be established• Data banking and reporting shall also be
developed
IV. COMPONENTS OF THE ME3
D. Baseline studies and parallel scorecards
• A baseline study shall be done utilizing indicators and assigned data source developed (2007 Q1)
• The baseline study shall provide information on
1) status of performance of the health system, particularly for incoming Local Chief Executives
2) areas of the PIF that need to be improved or revised
3) targets that need to be revised or set
IV. COMPONENTS OF THE ME3
E. System of governance of the F1 ME3
• Governance system shall be established which shall cover concerns such as level of independence of the ME3, internal rules, decision making process
VI. Functional Structure
DOH CLUSTERS TWGs(development of F1 ME3)
DOH EXECOM(Policy decisions)
TCG(Technical Approval)
DOH Task Force on F1 M&E(Coordination and Guiding Work)
Cluster TWGs
TWG PIF SCORECARD
PSDO Yes Central Office
Regulation Yes Central
SMCO Yes Central Office
Donors (BIHC)
CHD(BLHD)
IMCO Yes Central Office
FICO Luzon Yes DOH Hospitals
FICO Vis-Min Yes LGU
Technical Exchange FellowsUSAID
TWG PIF SCORECARD
PSDO-SD(PH) Yes Central Office
FICO Luzon Yes DOH Hospitals
FICO Vis-Min Yes LGU
Level/Stakeholder Assesment OversightClient HPDPB &
MIS
(Through a contracted external agent)
LGUProviders
LGU-Municipality ProvinceLGU-Province CHDCHD FICO LVMDOH Hospitals
BFAD, BBHFS, BHDT DOH PSDO Regulation
NCDPC DOH PSDO NCDPC
HPDPB,BIHC,BLHD,HRDB DOH SMCO
FS, PS, MIS DOH IMCO
Donors BIHC
ME3 IMPLEMENTATION
Implementation Structure
DOH EXECOM(Policy decisions)
TCG(Technical Approval)
HPDPB MIS
FICOs, PSDOs, SMCO, SMCT
LEVELS OF STAKEHOLDERSLEVELS OF STAKEHOLDERS
LEVELS OF STAKEHOLDERSLEVELS OF STAKEHOLDERS
LEVELS OF STAKEHOLDERS
IV. COMPONENTS OF THE ME3
F. Capability Building
• Competent units and personnel
• IT and MIS systems
• Funds for surveys and special studies
V. Phases of Activities and Timelines
Phase
1Drafting of PIF, Scorecards, and Data Sources for the 11stst set set of Final Outcomes
FOCUS: MDG
Develop Templates,Orientation Module
October-December 2006
Phase
2
•Baseline Studies
•Initial Publication of Scorecards
•Refinement of PIF for first set and ME3
January-April 2007
Phase
3
•Develop lower level output /process/ input indicators for 1st Set1st Set of FO
•Develop PIF for subsequent set of Final Outcomes consistent with F1 ME3
•Design of MIS, Capacity Building
2007
TASKS
Draft PIF & Scorecards
Consultations
Pretest
Baseline Survey
1. Final Outcomes2. Intermediate Outcomes3. Major Final Outputs4. Select key Indicators for
Scorecard1. Small Groups2. E-Mail3. National
Consultation(Dec.4-5)
For Feasibility
Q1 2007
Health outcome
Family/individual
Provider
SupplierLocal governm
ent
Central governm
ent
regional offices
FINAL OUTCOME INDICATOR NATIONAL EQUITY
Life expectancy Increase to…Baseline: 69.8 years in 2003 (Projections based on 1995 CPH)
LE of lowest 2 quintiles to approximate LE of highest 2 quintiles
Infant Mortality RatePneumoniaBacterial sepsis of newbornDiarrhea and gastroenteritis of presumed infectious origin
Reduce IMR to 17/1000 livebirths (MTPDP) Baseline: 29/1000 livebirths in 2003 (NDHS 2003)
Reduce IMR in lowest 2 quintiles to 18.5/1000 livebirths (TF) Baseline: 37/1000 lb in lowest 2 quintiles (NSO, ORC macro, NDHS 2003/Herrin 2006)
Under Five Mortality RatePneumoniaDiarrheas and gastroenteritis or presumed infectious originMeasles
Reduce U5MR to 32.24 (MTPDP) Baseline: 40 in 2000 (NSCB 2000)
UFMR in lowest 2 quintiles to approximate national target of 32.24 (TF)Baseline: 56.5 UFMR in lowest 2 quintiles (NSO, ORC Macro, NDHS 2003/Herrin 2006)
Maternal Mortality RateHypertension complicating pregnancy childbirth and puerperiumPostpartum hemorrhage
Reduce MMR to 90/100,000 livebirths (MTPDP)Baseline: 172/100,000 (NDHS 1998)
Reduction in baseline rates in lowest 2 quintiles by 70%
Malnutrition rate:Prevalence of underweight preschool children aged 0-5 years old
Reduce to 21.6% (MTPDP)Baseline: 27.6%(NNS 2003)
FINAL OUTCOME INDICATOR NATIONAL EQUITY
Total Fertility rate Reduce to 3.1 (MTPDP)Baseline: 3.5 in 2003 (NOH 2005-2010)
TFR in lowest 2 quintiles to approximate national target of 3.1 (TF)Baseline: 5.25 TFR among lowest 2 quintiles (NSO, ORC Macro, NDHS 2003/Herrin 2006)
HIV prevalence Contain ≤ 1% prevalence (MTPDP)Baseline(NOH): ≤ 1 % (DOH)
Hospital Net Death Rate
EQUITABLE FINANCING OR FINANCIAL RISK PROTECTION FOR ALL
Ratio of health expense to total non-food spending
Reduce ratio of health expense to total non-food spending to --%Baseline:---
Ratio of health expense to total non-food spending among lowest 2 income quintiles to be approximate to highest 2 income quintilesBaseline:---
Out of pocket spending for health of total health expenditure
Reduce out-of-pocket spending to 20% of total health expenditure (NOH/PNHA 2003)Baseline: 45% out-of-pocket spending out of total health expenditure (PNHA 2003)
Reduce out of pocket spending of lowest 2 income quintiles to ---% of total health expenditureBaseline: ---% out-of-pocket spending of lowest 2 income quintiles out of total health expenditure
FINAL OUTCOME INDICATOR
NATIONAL EQUITY
RESPONSIVENESS OF HEALTH SYSTEM
Client satisfaction on:Health facilityHealth agencyHealth programClient participation
Health facilityBaseline (WB/SWS 2000):Net satisfaction rate forFor- profit hospitals +96Traditional healers +94Nonprofit hospitals +91RHU +82Government hospital +79BHS +74
Net satisfaction rate to be equal/similar between lowest 2 income deciles and highest 2 income decilesBaseline: ---
Provider’s respect for personsDignityConfidentialityAutonomy
Scores to be equal/similar between lowest 2 income deciles
and highest 2 income deciles
Client orientationPrompt attentionAmenitiesAccess to social supportChoice of provider
Scores to be equal/similar between lowest 2 income deciles
and highest 2 income deciles
MDG-F1 Linkage Infant & Child Mortality Rate
ACCESS QUALITY FINANCIAL IMPACT EFFICIENCY
Pneumonia
Mortality Rate
Diarrhea Mortality
Rate
B. Sepsis Mortality
Rate
Malnutriton Rate
Water & Sanitatio
n
DOH HOSPITALS
CHD PSD-SD SMCO IMCOLGU
PROVIDER
(Public & Private
CLIENT
NCDPC
NCHFD
NCHP
NEC
HPDPB
BIHC
BLHD
HHRDB
Finance
Procurement
MIS
LGU Scorecard FrameworkHealth MDGs
Intermediate Outcomes
FINAL OUTCOME
Intermediate Outcome
Providers/StaffActions/Decisions
FacilityCharacteristics
ClientsActions/Decisions
ClientCharacteristics
LGU Actions/Decisions
InputsOutputs
Implementation Schedule
FO FO
FO FO FO
IO IO IO
MFOInput/Process
/Structure
MFOInput/Process
/Structure
MFOInput/Process
/Structure
MFOInput/Process
/Structure
MFOInput/Process
/Structure
Year 1 2 3 4 5
2007 2008 2009 2010 2011