White paper and CRP6: Co-learning on Impact Evaluation Design in NRM Research Programmes
-
Upload
worldfish -
Category
Investor Relations
-
view
2.584 -
download
3
description
Transcript of White paper and CRP6: Co-learning on Impact Evaluation Design in NRM Research Programmes
White paper and CRP6: Co-learning on Impact Evaluation Design in NRM
Research Programmes
Presented to NRM Impact Community of Practice The Worldfish Center, Penang, Malaysia, Sept. 4-5, 2012
Brian Belcher, CIFOR/Royal Roads University
THINKING beyond the canopy
CRP6 Aims
to enhance livelihoods through forestry, agroforestry and other uses of forest resources while sustaining environmental services and resource resilience.
THINKING beyond the canopy
CRP6 Components
THINKING beyond the canopy
• Stronger collaboration, focus and coherence within CG
• More, stronger and more diverse partnerships
• Results focus, shared responsibility for outcomes
• Sentinel landscapes
• Emphasis on learning by doing and on verification of progress
What’s new in CGIAR/CRP6?
THINKING beyond the canopy
CRP6 Proposal & MEIA Strategy
Discuss complexity in NRMR Recognize experimental design IE approaches not
necessarily appropriate or best Appreciate evaluation for learning & for accountability Emphasize clarifying causal assumptions through
participatory impact pathway development Recognize multiple impact pathways and multiple scales Propose using tools of Outcome Mapping, PIPA,
integrating monitoring, evaluation and impact assessment, and collaboration with other CRPs and experts
THINKING beyond the canopy
White Paper and process
White Paper and COP helps explain the need for alternative approaches and provides legitimacy for mixed methods approaches
Develop/promote consistent definitions and approaches Forum for sharing ideas, perspectives, expertise Overview and access to a range of relevant methods and
related literature Simultaneously developing “IDOs” Presentation will reflect on experience to date within
CRP6 evaluation planning and implications for WP
THINKING beyond the canopy
Concept of NRMR
CRP6 primarily contributes to SLO 1(reducing rural poverty) and 4 (sustainable NRM)
Supplementary contributions to 2 (improving food security) and 3 (improving nutrition and health)
Bottom-up approaches (“trajectories”): develop and support technology and institutions to benefit small-holders, communities)
Intermediate approaches: influence research agenda; support capacity strengthening; mainstream gender analysis)
Top-down approaches: influence policy at level of conservation orgs, development orgs, national and international policy)
THINKING beyond the canopy
Research for policy influence
predict (ex ante) or measure (ex post) effects of policy options and policy tools
provide knowledge for forming, implementing or contesting policy
identify and explain trends raise awareness of a problem improve understanding of underlying causes of economic
behaviour and environmental outcomes challenge conventional wisdom develop/influence research methods develop useful theory or conceptual framework
THINKING beyond the canopy
Implications for WP
Begin with definition of NRMR that encompasses and explains the range of ways that NRMR contributes to improvements in social and natural systems
Explicitly recognize that research processes and products contribute to change
Make stronger argument that we need alternative ways to establish “counterfactuals”
THINKING beyond the canopy
Terminology
Major source of confusion – need good, clear definitions and consistent use of key terms
e.g. SLOs are really “impacts” by contemporary definitions
still unclear whether IDOs should be defined as changes in behaviour or changes in state
THINKING beyond the canopy
Implications for WP
Distinguish “intervention” from “programme” (programme can include/support many interventions/kind of intervention)
Use “boundary partner” instead of “working partner” (consistent with K to A literature)
Distinguish & clarify “target groups” & “beneficiary partners” Distinguish between results from the use of new knowledge
and results from the process of doing the research Define outcomes as behavioural change Intended impacts include changes/conservation of
biophysical resources
THINKING beyond the canopy
Scale issues
Nested impact pathways at several scales (project, theme, component, “sentinel sites”, CRP6, SLOs)
Project-scale boundary partners & outcomes typically more direct, more tangible
Component-scale boundary partners and outcomes more difficult to define, identify and measure
THINKING beyond the canopy
Implications for WP
Need more attention to program-level issues and approaches
How to define, identify, and measure outcomes that are manifold and diffuse
How will adopting the framework change M&E and IE practice in CRP6?
supports and strengthens MEIA approaches in development in CRP6
Provides structure for building, testing methods
What benefits will this bring?
Encourages “impact culture”, clearer and more comprehensive project/program conceptualization, design and implementation
Support and encourage broader range of partnerships and interventions
Learning, feedback (monitoring for adaptive management)
What enabling changes are needed?
Recognition and legitimacy of TOC approaches within CGIAR and donors
Capacity building internally
Next steps for implementation?
TOC approaches to be used in forthcoming evaluations of past work
Recognition and legitimacy of TOC approaches within CGIAR and donors
Capacity building internally
THINKING beyond the canopy
CRP6 IDO Template
IDO
SLO
Quantified targets (10 Years)
BaselineGeographic
focus /scope
Comments [on ability to quantify
targets/baseline/geo...]
# How?
Enhanced contribution of FAT to income, food security, and nutritionIntermediate Development Impacts
Increased revenue from sale of tree products
1 Higher productivity of trees and forests coupled with improved markets and policy environment enables more rural households to participate in tree product markets and to earn more money from them. Higher productivity comes from improved germplasm and management.
Incomes from tree and forest products doubled for target households
ICRAF: new data from Sahel; data on fruits in Kenya and Malawi; range of tree products in Cameroon, Cote d’Ivoire and Sumatra; timber in NW India and fruits elsewhere;
ICRAF: new projects in Mali, BF, Niger, Sierra Leone, CDI, Malawi, Ethiopia, Rwanda, Burundi, Uganda, Kenya; Allanblackia project in Tanzania and Ghana, timber in Sulawezi, fruits in Vietnam
We have households who already sell and may enjoy better incomes; we will have households new to receiving income from FAT. We can estimate this once we decide on the locations. As for the projects mentioned, most have baselines but there is need to collate results in 2012/13.
THINKING beyond the canopy
www.cifor.cgiar.orgwww.cifor.cgiar.org