Health system metrics

Introduction

Glion September 28-29 2006

www.healthmetricsnetwork.org

Demand for health systems monitoringDemand for health systems monitoring

• Global Health Partnerships and other donors: enhance accountability and monitor progress of investments in health systems – Global Fund, GAVI, Child Survival Countdown 2015 etc.

• Universal Access to health services: HIV/AIDS, reproductive health, drugs (MDGs, G8 etc.)

• Assess / monitor health system performance

• Communication: Make health systems tangible by focusing on actionable (measurable) items

– "The commitment to invest in health systems as a means of scaling up is unprecedented. It will not last unless it is possible to show results. We need to be able to show what works and why."

DemandSupplyGoalsFrameworkObjectivesDashboardPerformance

Supply side of health systems monitoringSupply side of health systems monitoring

• WHO /World Bank: 2004 meeting; World Health Statistics 2005 (9 indicators) and World Health Report annexes (financing)

– Financing, HR, information, service delivery– Governance

• High and upper middle income countries: OECD quality of care, US National scorecard on health systems performance, Mexico effective coverage work

• Global Health Partnerships (GAVI, GFATM), WHO departments: developing shortlist of indicators for (suggested) reporting

DemandSupplyGoalsFrameworkObjectivesDashboardPerformance

GAVI and Global Fund Health System Strengthening monitoring

GAVI and Global Fund Health System Strengthening monitoringGAVI

• Process indicators (short term)– N of people trained, N of

vehicles purchased, etc.

• Output and capacity indicators (medium term)

– N of facilities with trained health workers in place, % of districts with vehicles operational etc.

• Outcome indicators (long term)– District level DPT 3 coverage

(equity indicator)– Measles coverage, – Under 5 mortality

Global FundService delivery: 4 output (facilities with specific services), 3 outcome indicators (% of population covered by GF interventions)

Human resources: 4 output (HW density, training output), 2 outcome (patient satisfaction)

Community systems strengthening: 4 output (training community workers), 1 outcome (availability basic package community services)

Information system & operational research: 4 output (reporting facilities), 4 outcome (surveys, HIV prevalence)

Infrastructure: 2 output (health facilities with specific arrangements), 1 outcome (geographic access)

Procurement and supply management: 3 output (trained technicians, stock out days TB drugs), 1 outcome (facility supply status)

Health system metrics - goalsHealth system metrics - goals

• Develop a common strategy to monitor health systems in countries that:

1.Guides and advocates for investment in a data generation strategy to provide accurate statistics for health system indicators

2.Includes a core set of health system indicators, if possible with baselines and targets or thresholds

3.Promotes the incorporation of health system monitoring in health information systems and planning cycles (e.g. PRSP, annual health sector reviews, strategic planning)

DemandSupplyGoalsFrameworkObjectivesDashboardPerformance

Basic framework for health system metrics (2)Basic framework for health system metrics (2)

INPUTS & PROCESSES

Governance & leadership

FinancesHuman resourcesInfrastructureProcurementLogistics & suppliesInformation

INPUTS & PROCESSES

Governance & leadership

FinancesHuman resourcesInfrastructureProcurementLogistics & suppliesInformation

OUTPUTS

Services

(availability, affordability, quality)

OUTPUTS

Services

(availability, affordability, quality)

UTILIZATION

Coverage

Behavioural change

UTILIZATION

Coverage

Behavioural change

OUTCOME

Improved health

Reduced mortality

OUTCOME

Improved health

Reduced mortality

OTHER DETERMINANTS OF HEALTH (ECONOMIC, SOCIAL, POLITICAL)

OTHER DETERMINANTS OF HEALTH (ECONOMIC, SOCIAL, POLITICAL)

Expanding the contents of health system metrics to measure performance

Expanding the contents of health system metrics to measure performance

Areas of focus• Governance: defining sector

strategies, clarifying roles, managing competing demands

• Financing: ensuring fair and sustainable financing

• Human resources: having a sufficient and productive workforce

• Information and knowledge: ensuring the generation and use of information

• Service provision: ensuring adequate drugs, equipment, infrastructure

• Coverage: use of service by those in need

Universal concerns• Equity: to protect and improve

health, as equitably as is possible.

• Being responsive to users

• Financial protection: to ensure people are able to avoid impoverishing health expenditures

• Health service quality and safety: more equitable access and use of services, ensure that effective interventions are used; services of adequate quality and safe

• Efficiency: to ensure that resources are used 'wisely' and not wasted

WHO Health System Performance FrameworkAdapting it for Health System Metrics

WHO Health System Performance FrameworkAdapting it for Health System Metrics

Stewardshipdefining sector strategies, clarifying roles, managing competing demands

Stewardshipdefining sector strategies, clarifying roles, managing competing demands

Healthoutcomes

Healthoutcomes

Coveragereaching those who need it

Coveragereaching those who need it

FUNCTIONS

Service provisionensuring adequate drugs, equipment, infrastructureimproving organization, management and quality of services

Service provisionensuring adequate drugs, equipment, infrastructureimproving organization, management and quality of services

Financingensuring fair and sustainable financing, with financial protection

Financingensuring fair and sustainable financing, with financial protection

Quality & safetyservices of adequate quality and safe harmful practices are reduced

Quality & safetyservices of adequate quality and safe harmful practices are reduced

Efficiency ensure that resources are used 'wisely' Interventions that are relevant

Efficiency ensure that resources are used 'wisely' Interventions that are relevant

GOALS OF THE SYSTEM

Financial protectionensure people are able to avoid impoverishing health expenditures.

Financial protectionensure people are able to avoid impoverishing health expenditures.

Responsiveness by treating people with dignity, and ensuringconfidentiality, irrespective of who they are

Responsiveness by treating people with dignity, and ensuring confidentiality, irrespective of who they are

Human resourceshaving a sufficient and productive workforce

Human resourceshaving a sufficient and productive workforce

Informationensuring the generation and use of Information and knowledge

Informationensuring the generation and use of Information and knowledge

Equitable distributionEquitable distribution

Glion Meeting objectivesGlion Meeting objectives

• Develop a dashboard to monitor health systems– Define the contents– Advise on reporting mechanisms

• Take stock of progress in health systems performance assessment

– New methods– Discuss priority areas for further work, including

linking dashboard to HSPA

• Identify priority areas for data collection and analysis investments at the country level

– Clear message to international partners

DemandSupplyGoalsFrameworkObjectivesDashboardPerformance

Dashboard for health systems monitoring: focusDashboard for health systems monitoring: focus

• Simplicity > comprehensiveness

• Parsimonious subset of indicators > full range of M&E indicators

• Measurability but not shying away from promoting new investments

• Comparable data over time and between countries

• National focus but include statistics within the country

• Low and lower middle income country focus

DemandSupplyGoalsFrameworkObjectivesDashboardPerformance

M&E indicators

Dashboard

Financingcurrent situationFinancingcurrent situation

• General agreement around indicators: financing of health services (revenue collection) and financial protection

• WHO – annual production of statistics

• Data availability inadequate– 70 countries ever done a full National Health Account

exercise– Very few countries have regular surveys to monitor

financial protection

• No thresholds or benchmarks for most indicators

FinancingDashboard exampleFinancingDashboard example

 

IndicatorYearValueRank*

Total health expenditure per capita) int $ rate(2003$2976

Total expenditure on health as % of GDP20034.3%95

General gov expenditure on health as % of total general gov expenditure200312.7%36

Health workforceCurrent situationHealth workforceCurrent situation

• Some consensus on indicators, but problems with worker classifications affecting comparability

• Poor quality of country data as reflected by the WHO HR database

• Lack of consensus about data collection strategies and reconciliation of data sources

– Professional data bases– Administrative data public sector– Population census– Facility census

• Distribution of health workforce within country: equity issues

Health workforceDashboardHealth workforceDashboard

STOCKActive health workers per 1,000 population

2.28 /1,000 (WHR 2006)

ENTRY

TRAININGAnnual output

per 1,000 population

EXIT

Indirect

Census;migration Records;Mortality

WITHIN COUNTRY DISTRIBUTIONUrban – rural ratio

Upper – lower quintile

InformationCurrent situationInformationCurrent situation

• Coverage civil registration systems: percent of deaths covered – limited picture and lack of change over time

• HMN assessment and scoring tool for all components of health information system – comparability issues

• Scoring of availability and quality of health statistics

• Scoring of data sources

Health Information System scoreDashboard exampleHealth Information System scoreDashboard example

0

Surveys Vital registration Census Service records Administrative records

32%

46%

13%

36%

25%

42%

Governance / stewardshipCurrent situationGovernance / stewardshipCurrent situation

• Multiple dimensions, some are difficult to measure: strategic vision, rule of law, participation etc.

• Country Policy and Institutional Assessment (CPIA):– 20 indicators, instrument to guide IDA funds

allocation– Criticisms:

• methodologically weak• Not conducted in transparent manner

– Potential for health sector:• Lift relevant components out of CPIA• Develop separate methodology and implementation

Service provisionCurrent situationService provisionCurrent situation

• Includes technology and infrastructure – focus on peripheral point of service provision

• Many dimensions – availability, quality, safety, quality, effectiveness, efficiency => use of a score card

• Developed countries: quality indicators, score card in the USA; score card in Afghanistan

• Focus on basic services – e.g. availability of essential drugs, basic equipment

– Facility census– Facility survey

Service provisionDashboard potential?Service provisionDashboard potential?

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1%

of

he

alt

h f

ac

iliti

es

Infra-structure

Basicequipment

Drugavailability

Infectioncontrol

Lab support

2005 2007

Coverage of interventionsCurrent situationCoverage of interventionsCurrent situation

• Immediate output of health systems: preventive and curative interventions; non-personal interventions

• Combining interventions into one index to assess health system status and performance

– Co-coverage: individual data on preventive interventions for children 1-4 years (Victora et al, 2005) – requires survey data

– Effective coverage: aggregate measures based on range of indicators – subnational analysis Mexico (Lozano et al. 2006)

• Explore possibility to combine current data on coverage into comparable indicator

• Investments in data collection to obtain more frequent quality coverage data

Co-coverage scoreDashboard potential?Co-coverage scoreDashboard potential?

Below 40%40-50%50-60%60-70%70-80%>80%

ChadSierra LeoneGuinea BissauGuatemalaEl SalvadorMexico

AfghanistanTogoBangladeshBeninColombiaMongolia

NigeriaBurundiKenyaIndonesiaTurkmenistanIran

SomaliaCambodiaGabonZimbabweUzbekistanSlovakia

NigerYemenZambiaGuyanaGeorgiaDR Korea

CARPakistanBhutanParaguayNicaraguaSri Lanka

LaoBurkina FasoGhanaNamibiaHondurasThailand

EritreaMozambiqueSwazilandTurkeyEgyptPanama

Timor-LesteEq GuineaTajikistanBoliviaBotswanaJordan

HaitiRwandaDR CongoMyanmarMoroccoTunisia

LiberiaMauritaniaDjiboutiAzerbaijanRussiaOman

GuineaCote d'IvoireIndiaPhilippinesJamaicaAlgeria

MaliAngolaSenegalSyriaChinaLebanon

UgandaTanzaniaRomaniaSerbiaBosnia

MadagascarMalawiDominican RepUruguay

NepalLesothoSouth AfricaCuba

IraqPeruChile

GambiaArmeniaBulgaria

QatarCosta Rica

Venezuela

DashboardDashboard

Mortality trendsBurden of disease

Overall performance(Life expectancy by GDP per capita)

Coverage trends (index)

Service provision status and trends (indexes)

Financing HR Information Governance

Distribution with country (equity)

Stewardship: policy & governance monitoring – special data collectionHR: census, labour force surveys, administrative recordsFinancing: household surveys, administrative records (budget, exp.), NHA, PERInformation: draws upon all data sourcesInfrastructure / technology / services: service records, administrative records

Emphasis on investment in data collectionEmphasis on investment in data collection

Health systems performance assessmentHealth systems performance assessment

• World Health Survey– 71 countries, results on coverage, responsiveness,

input into efficiency analysis– Malaysia example

• Subnational assessments– District health barometer: South Africa– Efficiency analysis: Indonesia

• Country practices in HSPA

• Issues and way forwardDemandSupplyGoalsFrameworkObjectivesDashboardPerformance

www.healthmetricsnetwork.org