Extreme Realities and Imagination Barriers: Research in Somalia

Savina Tessitore, Julie Lawson-McDowall, Giuseppe Simeon, FAO Somalia & FAO Rome.

With Mohammed Jama Aden, Hoosh Banadir Sheikh, Omar Mohammed Hassan, Mire Abdullaahi Elmi, Sowdo Mude Billow, Farah Osman, Liban Mohammed, Mohyadin Omer, Khadar Abiib

Outline

• Background: the Somali research conundrum• How has accountability in M&E in Somalia

been attempted until now?• 2 cases of qualitative research around the FAO

Cash for Work programme• Key methodological learning

The Somali research conundrum• The context demands and

impedes intense learning and oversight

• History of extensive diversion imposes compliance monitoring

• Community targeting is a black box for outsiders

• Gatekeepers e.g. local authorities “tax” incoming resources

• Customary redistribution distorts implementation objectives

• Security imposes remote management employment of local researchers, but scarce research skills at local level

• Clan affiliation of researchers, a key to access, may prove problematic

Accountability in M&E in Somalia• Quantitative impact evaluations with Randomized

Control Trials are the gold standard for Cash Transfers assessment. Always the best solution?

• In Somalia they inspired recent M&E of humanitarian interventions

• Pros: large-scale, replicability, generalization and comparability between agencies and kinds of interventions

• Cons: extremely “dirty” data, difficulties of remote managing large-scale data collection, inputs not always high enough value to be measurable, final results useful but too general?

• How can in-depth qualitative work complement this methodology?

Researching FAO’s CFW: a virtuous cycle of learning

Case 1: Evaluation of FAO Somalia 2007/2012. Qualitative impact study of FAO’s CFW programmeGedo, South-Central Somalia

• Team: inexperienced male and female researchers with local access, led and supervised by experienced researcher

• Methodology: adaptation of a familiar body of participatory tools

• Capacity building: substantial training input

• Learning: constant mentoring, debriefing and interrogation of research outputs. Virtuous cycle of growth in the quality of researchers and their work

• Sampling: reiteration of research in several sites to build up a more robust understanding

Case 2: Scoping mobile banking to poor rural beneficiaries for next phases of CFW implementationHargeisa, Somaliland

CASE 1 CFW impact study: aims and methodology

Context

• Rapid and huge expansion of FAOSO CFW in response to crisis: 2011: $2m to 2012: $30m+

• Shifting emphasis: ag productivity food access + ag productivity

Objectives

• Study to focus on outcome and impact, beneficiary perspectives• + Learning - accountability• Feeding evidence into the programme’s new learning mechanisms

Methodology

• Tools: social mapping; wealth ranking and targeting analysis; income, expenditure and coping strategies matrix; ranking of preferred interventions; SSI with non-beneficiaries

• Sampling: 7 villages, restricted access due to security and investigation of partner

• Flexibility, build trust and capacity of team, rely on local knowledge, incorporate team’s suggestions and observations

Impact• CFW valued for its short and long-term benefits, relevant

impact on FS and Ag productivity, multiplier effects, some impact on productive activities, some unintended negative impacts

• Little community consultation (women, Bantus)• Systematic Inclusion and Exclusion Errors• Better planning required for work component and choice of

infrastructuresMethodological learning• Qualitative approach allowed tight feedback loops into

evolving methodology• In a research context where so much is unknown, being open

to emerging issues is vital• Approach permitted interrogation of complex social relations

CASE 1 CFW impact study: findings

Hypothesis: mobile banking can be useda) as an alternative delivery mechanism b) to enhance M&E through direct access

Methodology:• Desk study and preliminary research mission• Identification and training of team• Research in 3 communities around Hargeisa, 5 communities in Sanaag, 5

communities in Burao• Research tools: village mapping, wealth ranking, FGD on phone use

Methodological learning:• Male and female team must be able to work in specific localities• Training, practice, debrief, review, practice• Daily debrief, by phone if necessary and systematize data collection for

comparison• Keep interrogating key issues • Request phone numbers and call interviewees back if necessary

CASE 2 scoping the use of mobile banking: aims and methodology

Topic Findings

Network Coverage Erratic, poor to non-existent in remote sites. Combined with charging problems makes use of phones a challenge

Charging Dependent on generators, solar chargers, normal batteries Current use of Zaad Spreading downwards from better off/shop owners, saves transport costs

Registration Energetic marketing by phone companyOwnership and Access

Ownership spreading fast amongst better off,

Illiteracy Only functional numeracy/literacy requiredBuying phone $20 upwards. Many second hand and inheritedCharging phone Major expense - $0.13-25 per chargeAirtime Relatively low cost

CASE 2 scoping the use of mobile banking: findings

CONCLUSION: Can be piloted where network exists. Risky technology given infrastructural challenges. Partnership with mobile bank provider necessary

Key methodological learning I• The “extreme reality” of Somalia sheds light on

methodological issues encountered elsewhere, and provides indications to overcome “imagination barriers”

• Intense learning, supervision, search for gaps, reiteration and testing results in steady improvement in team performance and research outcomes

• Mixed gender team crucial to learning

Key methodological learning II

• Some questions particularly resistant to interrogation (clan, polygamous households, intra-household dynamics) will be easier to crack through refinement of qualitative methods

• Though quantifying and attributing causality remains a challenge, this approach allows in-depth learning and acknowledges this complexity in social relations.

Conclusions

• Cost-effective approach.

• Repetition makes highly indicative findings more robust.

• Rapid feedback of evidence into programming