Linxiu Zhang, Chinese Academy of Sciences "Better Nutrition and Health through Impact Evaluation"
-
Upload
independent-science-and-partnership-council-of-the-cgiar -
Category
Health & Medicine
-
view
77 -
download
0
description
Transcript of Linxiu Zhang, Chinese Academy of Sciences "Better Nutrition and Health through Impact Evaluation"
Better nutrition and healththroughimpact evaluation!?
Response to “How to evaluate nutrition and health impacts of agricultural innovations?”
Linxiu Zhang
Center for Chinese Agricultural Policy
Chinese Academy of Sciences
Contents
• Why we want evaluate impacts
• Managing impact evaluations
• Nutrition and health – more than ag. Innovations
• Communicating findings to policy makers –scaling up, REAP example
Challenge/ Excuses
• We don’t have time
• It costs too much to do rigorous impact evaluation
• It is unethical
• Project implementation is completely site and context specific
• We already know!
• So many of the innovations/programs that we heard about on the news …
… how many of the new technologies/programs that we have become excited about …
… how many have been rigorously evaluated?
• Do we have empirical evidence, based on a carefully constructed counterfactuals, that these breakthroughs/programs work, can positively affect the lives of the poor and do so in a cost effective way? Yes but not many
KEY to IE: the attribution problem:factual and counterfactual
Impact varies over time
Impacts also are defined over time … Little attention has been given to the dynamics over time … though people think about this …
With project
Withoutproject
Project impact
Change in the CAL program effect on the standardized math test scores over time
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Baseline (Sept. 1) Midterm (early Nov) Final (late Dec)
Time
Stan
dard
ized
mat
h te
st s
core
The CAL program effect occurred by the midterm evaluation, less than two months after the start of the program.
0
0.120.11
Change in the CAL program effect on the standardized math test scores over time
-0.04
-0.02
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Baseline (Sept. 1) Midterm (early Nov) Final (late Dec)
Time
Stan
dard
ized
mat
h te
st s
core
There is no improvement between month two and month three..
0
0.120.11
Impact of nutrition intervention at infancy in Guatemala
• After 2 years � greater BMI
• After 10 years � higher grades in school
• After 15 years � higher school attainment
• After 40 years � higher wages / income
Source: Maluccio et al. 2009
When to use what methods?• RCTs Gold standard
– Where random assignment is possible
• Matching methods– Quasi-experimental methods (construct a
comparison group)– Propensity score matching (PSM)– Regression discontinuity design (RDD)
– ‘Intuitive matching’
• Regression-based/Modeling– Instrumental variables: need to be well-
motivated
Managing Impact Evaluation
• The importance of ex ante designs with baseline (building evaluation into design)- A theory-based approach (using causal chain) allowing analysis of causal pathways
• The importance of a credible design with a strong team.– Clear objectives, simple interventions
• Ensure management feedback loops
Budget and timeline
• Ex post or ex ante?• Existing data or new data?• Qualitative and quantitative?• How many rounds of data collection?• How large is sample?• When is it sensible to estimate impact?
Nutrition and health: morethanag. innovations
Example of China:
•Annual growth rate at 8-10%
•Agricultural growth rate at 3-4%
•High speed of urbanization andindustrialization
•Income for both rural and urban increasedrapidly
•Yet, China’s challenge in nutrition and healthis HUGE
Between 2008 and 2012 we tested nearly 60,000 children across
China for iron-deficiency anemia
In fact, anemia is all over China
Luo, R., X. Wang, C. Liu, et al. (2011) “Alarmingly High Anemia Prevalence in Western China.” Southeast Asian Journal of Tropical Medicine and Public Health Vol. 42 No. 5
Total
Total 33.7Shaanxi—2008 (Dataset 1) 37.5Shanxi—2009a (Dataset 2) 31.6Gansu—2010 (Dataset 3) 31.2Qinghai—2009 (Dataset 4) 51.1Ningxia—2009 (Dataset 5) 25.4Sichuan—2010 (Dataset 6) 24.8Guizhou—2010 (Dataset 7) 33.1
Poor areas of China
Children with anemia (≈ 33%)
Children with out ≈ 30 to 35 million school aged
children are estimated to be suffering from malnutrition!
Non-poor areas
of China
Children with anemia (≈ 8%)
Children with out (92%) < 5 million school aged children in
all of the rest of China
THE SCOURGE WITHIN:INTESTINAL WORMS IN RURAL CHINA
We have tested more than 4000 children for:
Incidences of Intestinal Worms, Guizhou Province, 2010& 2013
3 to 5 year olds 8 to 10 year olds
33.9% with worms
40.1% with worms
Without Without
Zhang et al., 2013… millions of children are infested with these …
Even earlier(malnutrition during the first 1000 days)
• Testing ≈1000 babies and their Mom’s in Southern Shaanxi (11 counties)
(these areas are 2 to 3 hour drive from Xi’an – one of China’s fastest growing cities)
Main Outcomes of InterestAt the time of enrollment and at each follow-up survey, we test babies for:
• Hemoglobin (anemia)
• Length & weight
• Gross & fine motor skills
• Cognitive development
• Social-Emotional development
One of these test is the Bayley Scales of Infant Development
Like an IQ test for babies
Some Initial Results from the Baseline Survey
Surveyed about 1,000 mothers and babies in rural Shaanxi Province
Of the nearly 1,000 babies tested….
55% were anemic� But less than 20% were stunted/wasted, indicating that this
is a micronutrient problem—the babies are getting enough calories, but not enough nutrients
Of the nearly 1,000 babies tested….
35% were cognitively delayed – scored outside of the “normal” range for other babies their age around
the world
Of the nearly 1,000 babies tested….
57% were significantly delayed in their motor development
Ultimate Consequences:
If the micronutrient deficiencies of infants / toddlers are not corrected before baby is 30 months old, there will be permanent and irreversible effects on:
– IQ
– Mental health
– Height
– Weight
– Health
We hope that we can help to address these issues through our intervention project, and ultimately bring national policy attention
to this important problem!
What does this mean?
In harshest terms:
Between 20 to 30 percent of China’s future population are in danger of becoming PERMANENTLY physically and mentally HANDICAPPED
What’s the challenge?1. Rural China’s missing
babies –broken chain – During our survey we
found that many families with babies had moved out of the villages.
– Where did they go?– What is the nutritional
status of “migrant babies”?
– If you cannot find them, how to help?
To sum up the “China story”
• Nutrition and health improvement does notnecessary go hand in hand with income and productivity growth
• We need to identify and map complete causal-chain between agriculture/income –nutrition/health, that would include care,knowledge, health systemreform, genderand governance and MORE
Communicating findings to policy makers- scalingup
Lessdiscussedtopicat thisforum
Find solutions by helping 100s and 1000s …
… and work with the government to help 1,000,000s …
Our experience of doingproject and its evaluationis to:
REAP Experiments (Projects) in China’’’’s Poor Rural Areas (and
Migrant Communities)
>30
REAP has touched the lives of thousands of China's poorest and most needy children
REAPhastestedmorethan schoolkidsonanemia
Passedoutmorethan multivitamins(toaddressanemia)
Testedmorethan schoolkidsforvisionproblems
Passedoutmorethan pairsofglasses
REAPhastestedmorethan ruralkidsonworms
Introducedcomputerassistedlearning(CAL)tomorethan schoolkids
Sofarhasgivenout millionUSDscholarshipstokeepkidsinschool
From Results to Policy Action
Approximately Millionchildreneatdailyegg
Over millionchildreneatafreeheathermealaday
Over kidshavebeenpreventedfromdroopingoutofschoo
schools have introduced CAL into regular clas
Inthepast years,REAPsubmitted policybriefstoGovernmen
Thank you!
Please visit us / contact us:• http://www.reapchina.com
• http://reap.stanford.edu