Measurement Issues Measurement Issues Regarding Estimation of Regarding Estimation of

Sub-National Health Sub-National Health System EfficiencySystem Efficiency

Ajay TandonAjay Tandon

Asian Development BankAsian Development BankDevelopment Indicators and Policy Research Development Indicators and Policy Research

DivisionDivision

Economics and Research DepartmentEconomics and Research Department

September 29September 29thth 2006 2006

Asian Development Bank

Sub-National Health System Sub-National Health System EfficiencyEfficiency

Entails measurement of health system Entails measurement of health system outcomes relative to resource inputs at outcomes relative to resource inputs at lower administrative tiers (e.g., at district lower administrative tiers (e.g., at district or province level). or province level).

Arguably, of greater local relevance and Arguably, of greater local relevance and often more useful than cross-country often more useful than cross-country analyses for policy-makers.analyses for policy-makers.

Poses significant measurement Poses significant measurement challenges: relevant data often not challenges: relevant data often not available sub-nationally.available sub-nationally.

Asian Development Bank

Sub-National Data ChallengesSub-National Data Challenges

Sub-nationally representative surveys Sub-nationally representative surveys are expensive to conduct and – even if are expensive to conduct and – even if available – such surveys are not available – such surveys are not conducted as regularly as may be conducted as regularly as may be needed. needed.

Relevant sub-national data may only be Relevant sub-national data may only be available from administrative records available from administrative records and this may not be reliable.and this may not be reliable.

Data may not be available for pertinent Data may not be available for pertinent variables.variables.

Asian Development Bank

Possible SolutionsPossible Solutions

Use of innovative sampling strategies Use of innovative sampling strategies to reduce cost of conducting surveys, to reduce cost of conducting surveys, e.g., EPI cluster surveys and Lot Quality e.g., EPI cluster surveys and Lot Quality Assurance Sampling (LQAS). Assurance Sampling (LQAS).

Use of econometric models and Use of econometric models and Bayesian data combinatorial techniques Bayesian data combinatorial techniques for estimation purposes.for estimation purposes.

Use factor analytical techniques to Use factor analytical techniques to compute proxy indexes to capture compute proxy indexes to capture relevant quantities of interest.relevant quantities of interest.

Asian Development Bank

Econometric ModelsEconometric Models

Quantity of interest (Y) may be Quantity of interest (Y) may be available at a national level, but its available at a national level, but its determinants (X’s) are available determinants (X’s) are available bothboth at at the national and sub-national level.the national and sub-national level.

Y = Y = ff (X): model at national level. (X): model at national level. Predict Y at sub-national level using X.Predict Y at sub-national level using X.

Disadvantage: model dependence is Disadvantage: model dependence is high as estimates of Y at the sub-high as estimates of Y at the sub-national level will be sensitive to choice national level will be sensitive to choice of predictors.of predictors.

Asian Development Bank

Bayesian Data Combination Bayesian Data Combination TechniquesTechniques

Derive “priors” of quantity of interest Derive “priors” of quantity of interest using national survey data.using national survey data.

Augment priors with micro-samples at Augment priors with micro-samples at the sub-national level (the “likelihood”).the sub-national level (the “likelihood”).

Combine the priors with the likelihood to Combine the priors with the likelihood to estimate “posterior” estimates at the estimate “posterior” estimates at the sub-national level.sub-national level.

Borrow strength using prior information.Borrow strength using prior information.

Asian Development Bank

Bayesian Data Combination Bayesian Data Combination TechniquesTechniques

Proportion poor

Prior Likelihood Posterior

0 .2 .4 .6 .8 1

0

.5

1

1.5

Asian Development Bank

Proxy IndexesProxy Indexes

Constraint Constraint typetype

VariableVariable IndicatorIndicator

FinancialFinancial Lack of demand due to insufficient Lack of demand due to insufficient purchasing power; Low levels of purchasing power; Low levels of public/private spendingpublic/private spending

GDP per GDP per capita, PPPcapita, PPP

HouseholdHousehold Lack of demand for healthLack of demand for health Female Female literacyliteracy

Health Health services services deliverydelivery

Shortage and maldistribution of staff; Shortage and maldistribution of staff; Weak systems for managing supplies Weak systems for managing supplies and staff; Poor accessibilityand staff; Poor accessibility

DPT3 DPT3 coverage; coverage; Access to Access to health health servicesservices

EnvironmentaEnvironmental l characteristiccharacteristics (1)s (1)

Corruption; Weak governance; Lack Corruption; Weak governance; Lack of public accountabilityof public accountability

Government Government effectiveness effectiveness indexindex

EnvironmentaEnvironmental l characteristiccharacteristics (2)s (2)

Geographical/epidemiological Geographical/epidemiological constraintsconstraints

Proportion of Proportion of population in population in tropicstropics

Source: Ranson, Hanson, Oliveira-Cruz, and Mills (2003)

Asian Development Bank

Indonesia Sub-National Indonesia Sub-National ApplicationApplication

Output index by district:Output index by district: Coverage of complete immunization.Coverage of complete immunization. Skilled birth attendance rate.Skilled birth attendance rate. Iodized salt consumption.Iodized salt consumption. Life expectancy.Life expectancy. Extent of protection from catastrophic spending.Extent of protection from catastrophic spending.

Input index by district:Input index by district: Income.Income. Female education.Female education. Nurses per 100,000.Nurses per 100,000. Out-of-pocket health expenditure.Out-of-pocket health expenditure. Access to health facilities.Access to health facilities.

Asian Development Bank

Indonesia Sub-National Indonesia Sub-National ApplicationApplication

-4-2

02

4O

utp

ut

ind

ex

-4 -2 0 2 4 6Input index

Indonesia HSPA: Outcomes vs. inputs across districts

Magelang (Jawa Tengah)Kulon Progo (DI Yogyakarta)

Asian Development Bank

Indonesia Sub-National Indonesia Sub-National ApplicationApplication

-4-2

02

4O

utp

ut

ind

ex

-4 -2 0 2 4 6Input index

Indonesia HSPA: Outcomes vs. inputs across districts

W onogiri (Jawa Tengah)

Kulon Progo (DI Yogyakarta)

Soppeng (Sulawesi Selatan)

Jayawi Jaya (Papua)

Madiun (Jawa Timur)

Manokwari (Papua)

Blitar (Jawa Timur)

Asian Development Bank

Indonesia Sub-National Indonesia Sub-National ApplicationApplication

-4-2

02

4D

istr

ibut

ion

of o

utpu

t in

dex

1 2 3 4 5 6 7 8 9 10

Deciles of input index