Download - Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Transcript
Page 1: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Agenda

Welcome Remarks

Introduction

Overview of Measurement Options

Panel Discussion

Questions and Answers

Closing Remarks

Twitter info:

@USAID @REAL_Award

@USAIDFFP @FeedtheFuture

@TANGOInt @MercyCorps

#resilience

#resiliencemeasurement

#measuringresilience

Resilience and Resilience Capacities Measurement Options

November 8, 2018 at 9:00 – 10:30 am ET

Page 2: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Presenters and Panelists

Tiffany M. Griffin

Adviser, Monitoring, Evaluation, and Strategic Analysis

USAID Center for Resilience

RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018

Katherine Arnold

Associate Director, Monitoring, Evaluation, and Learning

Dept. of Hunger and Livelihoods, Save the Children

Olga Petryniak

Senior Director, Global Resilience

Mercy Corps

Arif Rashid

Monitoring and Evaluation Team Lead

USAID Office of Food for Peace

Tim Frankenberger

President and Co-founder

TANGO International

Janina Mera

Monitoring and Evaluation Advisor

USAID Bureau for Food Security

Page 3: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Customized Standardization

• NOT an oxymoron!

• Context is important

• SOME uniformity has advantages

Practicality

• Documents designed with the users in mind

• Resource-intensity and 6 years of testing drove the options

Complement to Other Resources

• Guidance Notes

• Short Course Series/Webinars

• Evidence Briefs

• Assessment Tools

• Case Studies

Important Caveats / Considerations

• Food security-centric

• HH/Community-centric

• Resist the urge to adopt the “easiest” way forward

• Categories are not “fixed”

RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS

Page 4: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

USAID-funded food security

activities require consistent and

context-specific ways to measure

resilience and resilience capacity

in order to ensure effective

resilience programming and

continuous improvement based

on evidence.

Resilience /Resilience Capacities

Measurement Options

Photo

cre

dit: Z

achar

ias

Abubeker/

Save

the C

hild

ren

Page 5: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Resilience/Resilience Capacities Measurement Options

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Resilience and Resilience

Capacities Measurement

Options

Household

QuestionnaireCommunity

Questionnaire

Methodological

Guidance

Enumerator

Guidance

Page 6: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

• Provides an introduction to and comparison of several measurement approaches for analyzing resilience (i.e., Light, Intermediate, and Full), each of which reflects a different level of effort and budget in terms of the survey instrument and analysis used (i.e., number of questions asked and analyzed).

• Intermediate and Light approaches use subsets of the questions, responses, and analysis of the Full approach.

Save

the C

hild

ren

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Resilience/Resilience Capacities Measurement Options

Photo

cre

dit: T

AN

GO

Inte

rnat

ional

Page 7: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Full approach – resilience focus countries; full “menu” of

questions, responses, and analyses

Intermediate approach – programs in countries promoting

resilience but not resilience focus countries; subset from Full menu

Light approach – programs in countries that are not resilience

focus countries; minimal subset from Full menu

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Resilience/Resilience Capacities Measurement Options

Page 8: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Comparison of full, intermediate, and light approaches

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Page 9: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Protocol considerations for the Light, Intermediate and Full resilience approaches

NOVEMBER 2018USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS

Page 10: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Household and Community Questionnaires

• Together, the household and

community questionnaires make up

the resilience instrument.

• Must be used in conjunction with

the FFP core questionnaire.

• In the FFP questionnaire, certain

modules or questions must remain

and others must be deleted when

used with the resilience

instrument.

• Replacement questions included in

an annex if not used with FFP

questionnaire.

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Photo

cre

dit: K

aryn

Fox, T

AN

GO

Inte

rnat

ional

Page 11: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Enumerator Guidance

• Provides specific instructions for enumerators to help them interpretquestions and responses in the household and community questionnaires.

• Not all questions are included:

− Yes/no questions that do not require further explanation.

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Page 12: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Methodological Guidance

• Provides specific instructions for calculating individual components of resilience capacity as well as the three resilience indices absorptive, adaptive, and transformative capacity and an overall resilience index.

• Represents the Full approach (i.e., complete “menu” of questions, responses, and analyses).

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Page 13: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Resilience Indices and Components of Resilience Capacity: Full Approach

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Page 14: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Methodological Guidance

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Page 15: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

1. Access to cash savings. This indicator is a binary (dummy) variableequal to 1 if the respondent reported that a household memberregularly saves cash.

• Survey question: R601

2. Access to remittances. This indicator is a binary (dummy) variableequal to 1 if the respondent reported that the household receivesremittances.

• Survey questions: R1101, R1103, R1105, R1107

3. Asset ownership index. Asset ownership is measured using thenumber of consumer durables, productive assets, and livestock owned.

• Survey questions: BL H7.02, H7.03, R201, R201A

USAID RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOVEMBER 2018

Methodological Guidance

Page 16: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Thank You

TANGO International

Tim Frankenberger [email protected]

www.fsnnetwork.org/REAL

[email protected]

Page 17: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Panel Discussion

Page 18: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Resilience M&E in Implementation

Examples from the Sabal program in Nepal

1) Inform program design and TOC

2) Inform M&E system and tools

- Baseline survey

- Routine monitoring

3) Review and revise TOC

4) Frame how we talk about resilience

Resilience M&E tools used to:

Page 19: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Ethiopia

PRIME

Page 20: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

PRIME Ethiopia

Baseline and Counterfactual challenges…

Page 21: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

PRIME Ethiopia

PRIME Post-Shock Measurement Framework

Page 22: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Nepal

PAHAL

Page 23: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

PAHAL Nepal

PAHAL Models

Page 24: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

PAHAL Nepal

PAHAL Models

Page 25: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

PAHAL Nepal

PAHAL Measures of Resilience Strategies

Page 26: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Key Considerations

Page 27: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Questions

& Answers

Photo

cre

dit: T

AN

GO

Inte

rnat

ional

Photo

cre

dit: T

AN

GO

Inte

rnat

ional

Page 28: Resilience and Resilience Capacities Measurement Options...RESILIENCE AND RESILIENCE CAPACITIES MEASUREMENT OPTIONS NOV 8, 2018 Katherine Arnold Associate Director, Monitoring, Evaluation,

Thank You

www.fsnnetwork.org/REAL

www.tangointernational.com

www.mercycorps.org/research/resilience

www.savethechildren.org