Evaluation of the Winter Cash Programme for Lebanese Poor ... · 1.2 Programme overview- The...

Evaluation of the Winter Cash Programme for Lebanese Poor

Children and their Families

Evaluation Report

Submitted by

Awny Amer Morsy

Independent Monitoring, Research and Evaluation Consultant

Submitted to

UNICEF Lebanon- MENA Region

Jan, 2017

Disclaimer: The opinions expressed in this report are those of the evaluation consultant who

conducted and led the evaluation process that was captured from the views of the project

stakeholders interviewed. The evaluation consultant takes responsibility for any errors reported

herein that are based on its own independent data collection.

[email protected]

Egypt

Mobile: 002-01020525500

Skype: awny2210

mailto:[email protected]

2

The final evaluation report of the Winter Cash Programme-UNICEF Lebanon. Developed by AwnyAmer- Jan, 2017

Acknowledgements

The consultant would like to thank all those who provided their input, feedback and genuine support

during the different phases of the evaluation process.

Most importantly, the consultant would like to thank all the actors and key partners working with

UNICEF Lebanon on the Winter Cash programme, especially the Ministry of Social Affairs (MoSA),

the National Poverty Targeting Program (NPTP) team, the Coordination Management Unit (CMU) at

the Presidency of Council of Ministries (PCM), and the Word Food Programme (WFP). In this

regard, the consultant is grateful for the assistance provided by the programme section at UNICEF as

well as the Planning, Reporting, Innovations and M&E section (PRIME) for the whole-hearted

support during the evaluation process. Appreciation also goes out to those who facilitated the

logistical requirements for the completion of this task and who provided professional guidance to the

evaluation process.

Special thanks are extended to the Directors of the Social Development Centres (SDCs) and their

teams, including the field coordinators and social workers who played a vital role in the evaluation

process, whether in organising and coordinating the facilitation of the Focus Group Discussions

(FGDs) with the direct beneficiaries or by providing their time, insights, views and feedback during

the interviews conducted with them. Their high level of transparency, commitment and

understanding for the evaluation purpose were indispensable to the success of the evaluation.

Finally, the consultant is very grateful to the direct beneficiaries who took part in the evaluation

process and provided their feedback, views, insights and recommendations during the FGDs, which

were facilitated in the four governorates and the sub-areas during the data collection phase of the

evaluation process.

3


Table of Contents

LIST OF ACRONYMS …………………………………………………………………………………..4

0. Executive summary…………………………………………………………………………………….6

1. OBJECT OF EVALUATION………………………………………………………………………..13

1.1 Introduction/background…………………………………………….…….………………………13

1.2 Programme overview- The expected results chain………………………………………………..13

1.3 Key partners……………………………………………………………………………………….15

2. EVALUATION PURPOSE, OBJECTIVES AND SCOPE OF WORK…………………………16

2.1 Overall goal of evaluation………………………………………………………………………..16

2.2 Specific evaluation objectives……………………………………………………………………16

2.3 Scope of work…………………………………………………………………………………….16

2.4 Evaluation criteria/questions……………………………………………………………………..18

2.5 HRBA/Gender equality & equity principles in the current evaluation……………………….….19

3. EVALUATION METHODOLOGY Methodology………………………………………………..20

3.1 Introduction/overview…………………………………………………………………………….20

3.2 UNEG Norms & Ethical standards……………………………………………………………….20

3.3 Data collection methodologies and tools…………………………………………………………22

3.4 Limitations and challenges of evaluation…………………………………………………………25

4. KEY FINDINGS…………………………………………………………………………………….26

4.1 Relevance………………………………………………………………………………………..26

4.2 Effectiveness…………………………………………………………………………………….35

4.3 Efficiency………………………………………………………………………………………..46

4.4 Sustainability…………………………………………………………………………………….49

4.5 Coordination……………………………………………………………………………………..51

5. CONCLUSION………………………………………………………………………………………54

6. LESSONS LEARNED………………………………………………………………………………55

7. KEY RECOMMENDATIONS………………………………………………………………….….56

Annexes………………………………………………………………………………………………….60

Annex 1: The Terms of Reference (TOR) .................................................................................................. 60

Annex 2: List of stakeholders interviewed ................................................................................................. 68

Annex 3: List of documents consulted ........................................................................................................ 70

Annex 4: Data collection instruments – Methodologies & tools’ guides ................................................... 71

Annex 4.1 - Focus Groups Discussions (FGDs) Guide .............................................................................. 71

Annex 4.2 – Semi-Structured Interview Guide ........................................................................................... 72

Annex 4.3 – Semi-Structured Group Interview Guide................................................................................ 73

Annex 5: Evaluator Bio Data ...................................................................................................................... 75

Annex 6: Evaluation Matrix ........................................................................................................................ 76

Annex 7: Results framework ...................................................................................................................... 80

Annex 8: Summary of the stakeholders’ feedback and insights about the restricted cash modality versus

unrestricted cash modality .......................................................................................................................... 81

Annex 9: Summary overview about the Post Distribution Monitoring (PDM) process ............................ 84

4


List of Acronyms

ALNAP Active Learning Network for Accountability and Performance

ATM Automated Teller Machine

CMU Central Management Unit

CPD Country Programme Document

CRC Convention on the Rights of the Child

CTP Cash Transfer Programming

DAC Development Assistance Criteria

FGD Focus Group Discussion

GDP Gross Domestic Product

HACT Harmonised Approach to Cash Transfer

HH Household

HRBA Human Rights Based Approach

HRRP Humanitarian Response and Resilience Plan

IT Information Technology

M&E Monitoring and Evaluation

MDG Millennium Development Goal

MEHE Ministry of Education and Higher Education

MoJ Ministry of Justice

MoSA Ministry of Social Affairs

MOU Memorandum of Understanding

NPTP National Poverty Targeting Programme

OECD Organisation for Economic Cooperation and Development

OMR Optical Marketing Recognition

PCM Presidency of the Council of Ministries

PDM Post Distribution Monitoring

PIN Personal Identification Number

PMT Proxy Means Test

SDC Social Development Centre

SDG Sustainable Development Goal

SMS Short Message Service

TOR Terms of Reference

UNEG United Nations Evaluation Group

5


UNHCR United Nations High Commissioner for Refugees

UNICEF UN Children’s Fund

USD United States Dollar

WFP World Food Programme

6


0. Executive summary Addressing the seasonal needs of families in Lebanon requires a multi-sectorial approach. During the

harsh winter months, vulnerable families rely on winter cash assistance to address their basic needs, often

at the expense of purchasing other key items. Poor Lebanese and refugee households are highly affected

by inconsistent flows of income, which shift season to season and year to year. Employment is largely

concentrated in the informal jobs market and remains unreliable. In particular, during the winter months,

the availability of work is less which directly impacts the earning ability of vulnerable families (reduced

by 40 per cent as reported by Lebanese poor households1).

Cash Transfer Programming (CTP) has rapidly become an important and often preferred mode of

assistance for humanitarian actors around the world. Organisations operating in areas with conflict-

affected or displaced persons and with functioning markets are increasingly using cash-based assistance

to meet the needs and improve livelihood outcomes of targeted populations.

The Winter Cash programme supports economically vulnerable Lebanese children and their families. It

was designed through a collaborative process between UNICEF, National Poverty Targeting Programme

(NPTP), Ministry of Social Affairs (MoSA), and World Food Programme (WFP), the Office of the Prime

Minister and the Lebanese Presidency of the Council of Ministers (PCM). The objective of the

programme was to introduce cash-based assistance as a pilot to further strengthen the existing social

safety nets and social protection mechanisms in Lebanon. Implementation is the responsibility of the

NPTP, Central Management Unit (CMU) at the presidency of the Council of Ministries and MoSA, with

technical support provided by UNICEF and WFP.

The Winter Cash programme relied on the Proxy Means Test (PMT) score to target the most socio-

economically vulnerable Lebanese households with children. MoSA social workers updated data in the

NPTP database which identifies the most vulnerable households. They visited these households to gather

updated information (specifically the number of children in the household and telephone numbers) which

was then entered into an Optical Marking Recognition (OMR) form, which ranked and selected the most

vulnerable families corresponding to the target of 75,000 children. At the end of this process, 26,052

families were selected as beneficiaries of the Winter Cash programme. A cash grant of USD 40 per child

was transferred to beneficiary households. This value is based on the estimated cost of the winter

clothing kit which was distributed during the 2014/2015 UNICEF winter campaign. The payment was

transferred through ATM cards issued by WFP and Banque Libano-Francaise. Distribution of cards took

place between February 15th and March 10th 2016 by MoSA social workers in 22 selected SDCs, out of a

total 220 existing in the country.

The specific objectives/expected outcomes of the Winter Cash programme were focused on two key objectives - (1) Poor Lebanese children and families provided with the means to cover a portion of their winter expenses to help their children cope with the winter cold in Lebanon exposed to seasonal hazards and unexpected winter emergencies are able to maintain safe access to goods and services; (2) Strengthened delivery mechanisms of the NPTP through the incorporation of new systems such as a Cash-based programme. The overall purpose of the programme

evaluation was to assess (or measure) the outcome and effectiveness of the winter cash programme in

order to strengthen the joint planning, design and implementation capacity of the NPTP, MoSA, PCM and

UNICEF as well as meet the needs of the children by providing appropriate quality assistance in a

dignified manner.

1Poverty, Inequality and Social Protection in Lebanon, January 2016, Oxfam, AUB

7


The key objectives of the evaluation were guided by OECD/DAC criteria with special focus on assessing

the overall effectiveness of the winter cash modality in Lebanon; and the appropriateness and

acceptance of the programme from the perspective of the target population. Additionally, the evaluation

aimed to assess the efficiency of joint collaboration among the different actors in addition to providing

concrete recommendations for the improvement of planning and implementation of joint programme

targeting for vulnerable Lebanese children.

The key programme beneficiaries (vulnerable Lebanese poor children and their families) were selected

from the following targeted working areas: Bekaa, Mount Lebanon, Nabatieh, Akkar, Tripoli and South.

The key target audience of the evaluation was the NPTP and MoSA as well as other stakeholders

including the PCM, CMU, SDC directors, social workers & field coordinators. WFP in addition to the

households under “Hala” programme having benefited from the programme; children boys & girls and

their families.

Evaluation Methodology:

In order to address the key questions outlined in the TOR, the consultant used formative evaluation

practices combined with participatory approaches, depending on the type and category of the different

stakeholders (including programme beneficiaries). The evaluation process was initiated by carrying out a

desk review of ten of the key documents outlined in the TOR. Thirteen semi-structured individual

interviews were conducted with UNICEF staff and key partners (Five males and eight females). Seven

semi-structured group interviews were conducted with a total of 24 participants. The participatory

approach was applied by conducting Focus Group Discussions (FGDs) to capture the outcomes and key

changes impacting the quality of life of the targeted groups. A total of 90 stakeholders were interviewed

during the evaluation process in addition to 101 who were interviewed during the Post Distribution

Monitoring (PDM) process which was conducted in 2015. Therefore, a total of 191 individuals were

interviewed in order to assess the winter cash programme.

Additionally, the survey that was conducted by a third party during the PDM, using Computer Assisted

Telephone and face to face interviews, was also used in this evaluation as part of the triangulation process

to validate the data collected and the key findings captured. A total of 457 out of 500 beneficiaries were

interviewed in Bekaa, Mount Lebanon, North and South Lebanon.

Key Findings and Conclusions

Relevance

The Winter Cash programme focused mainly on addressing the rights and the needs of Lebanese poor

children and their families in targeted areas across Lebanon. Overall, the evaluation found that the

programme intervention reached the most socio-economically vulnerable Lebanese households and met

most of their basic winter needs.

The evaluation revealed the absence of the capacity building component from programme intervention

strategies. Despite the fact that the Winter Cash programme is a pilot project, this discovery led to the

recommendation for the inclusion of capacity development in future programmes. Capacity needs to be

strengthened to ensure fairness in the provision of cash assistance.

The cash grant of USD 40 per child was found to be insufficient to meet the basic needs of the most

socio-economically vulnerable Lebanese households. Moreover, the programme excluded a large number

of vulnerable families who were also in need of financial assistance due to their harsh economic

conditions owing to the pilot nature of the Winter Cash programme in addition to the available financial

resources.

8


The selection process and criteria were developed jointly by the NPTP, UNICEF, and CMU/PCM.

However, four major pitfalls were reported – firstly, there lacked clear documentation and understanding

of how the selection process worked. Secondly social workers and field coordinators did not participate in

setting the selection criteria or other relevant questionnaires. Thirdly, the data used in the selection

process relied on outdated sources (as far back as 2011) in addition to omitting some of the important

variables and criteria in the selection process. Lastly, the age of selected children (aged up to 15 years)

was the key selection criterion. The programme did not consider gender as one of the selection variables

and therefore a gender perspective was not incorporated in the targeting process. Although the payment

list of beneficiaries includes the sex of the ATM cardholder, there exists no disaggregated data for the

children supported per family. As a result, no sex-disaggregated data for children who benefitted from the

programme is available.

The programme was successful in identifying the most relevant partners for the project, with each of the

five partners (UNICEF, MoSA, CMU, NPTP and WFP) playing a significant role in bringing an added

value to the implementation of the programme.

Effectiveness

The evaluation revealed that the majority of beneficiaries who took part in the FGDs expressed a high

degree of satisfaction with their participation in and benefits from the Winter Cash programme.

Dissatisfaction regarding the value of the cash transfer was reported amongst a number of beneficiaries

who believed it to be insufficient for meeting their needs. Others suggested that the assistance should be

recurrent on a monthly or quarterly basis throughout the year. The evaluation also found that the

programme did not take into account the special needs of families who have children with disabilities and

did not adequately consider other vulnerable groups such as working women, orphans and families who

do not have children.

Due to a delay in the implementation of the project, ATM cards were not distributed until March instead

of during the previous winter. This hindered the achievement of the main planned objective of the

programme which was “to provide the aid in the winter season” in order to reduce the vulnerability of

Lebanese poor families. UNICEF, with the support of other key partners, developed a well-structured

system to track the distribution cohorts, delivery system process, hotline utilisation, and question and

answer mechanism, and complaint mechanism, which are guided and led by WFP. However, there is a

lack of monitoring and post follow-up activities by social workers and field coordinators to assess the

changes in the lives of the beneficiaries as a result of their participation in the programme and collection

of information on the spending patterns of the cash amounts.

Some beneficiary families (particularly in the Akkar and Mount Lebanon governorates) received their

ATM cards without a pin code. To respond to this issue, a complaint mechanism was introduced.

However, it was weak and the programme ended before the issues were resolved.

The distribution mechanism was considered as one of the key challenges in the Winter Cash programme

especially for those living far from SDC selected distribution points and who incurred additional

transportation fees.

Communication issues relating to the SMS notification which alerted beneficiaries to the distribution of

ATM cards were reported. Many selected families could not understand the SMS for three primary

reasons: (1) They were illiterate, (2) They presumed the message to be an advertisement, and (3) The

model of their phone could not read Arabic messages.

Although the grant assisted families in securing some basic needs for their children, it cannot be

substantiated that the cash grant of USD40 per child resulted in significant changes, or directly impacted

on the lives of the beneficiaries.

9


Based on these findings, no clear conclusion can be drawn on the success of unrestricted cash transfer as

opposed to restricted cash transfer. Some stated their stance based on their points of view, insights and

previous experiences from other similar programmes with poor vulnerable groups. All partners including

WFP, NPTP, MoSA staff, six SDC directors, five field coordinators as well as 14 out of the 20 social

workers interviewed during the evaluation process expressed their preference for restricted cash transfer

in e-voucher form in exchange for food, clothes or fuel. They indicated that if it was not possible for the

unrestricted cash transfer assistance to be recurrent, it would be better to have the restricted e-vouchers

for food, fuel, clothes or other essentials.

On the other hand, the majority of direct beneficiaries and few of the social workers and field

coordinators interviewed during the evaluation process expressed their preference for

unconditional/unrestricted cash transfers. This corresponds with the results of the PDM survey showing

that the majority of beneficiaries were satisfied with the cash assistance modality (63 per cent)2 compared

to those who were very satisfied (32 per cent) and those who were not satisfied (5 per cent). MoSA staff

and the direct beneficiaries interviewed during the PDM FGDs indicated that the cash modality was

perceived as a better tool, allowing beneficiaries to purchase the essentials most applicable for their

personal needs. Spending patterns of the priority five spending areas3 among the participants interviewed

are highlighted as follows:were: Winter Household expenses (56 per cent), food (55 per cent), education

fees-including the transporation fees to school (30 per cent), repaying of debts (20 per cent), health

expenses (20 per cent), electercity bills (12 per cent), heating- fuel or others (11 per cent) and rent (11 per

cent).

Capacity building activities are crucial for programme development and investment in enhancing the

skills and abilities of social workers in the field is crucial. Not only will this lead to improvement of

programme delivery, but it will also promote ownership of the NPTP.

Efficiency

The successful collaboration between implementing partners demonstrated an excellent example of the

partnerships model. Each of the key stakeholders were well-informed, roles and responsibilities were

agreed upon in advance, and specific measures were set in order to maintain cost-efficiency during each

phase of the programme. Most of the participants stated that they consider the unrestricted cash transfer as

the preferred modality of assistance due to its cost-efficiency, in comparison to the restricted cash transfer

(e-vouchers). The prevailing opinion of beneficiaries, captured during the FGDs, was that the cash value

meant that they felt more respected and were treated “like human beings”.

As the beneficiaries could withdraw the cash grant directly, using ATM cards, the need to engage a third

party was null, thus reducing costs and risk of wasted resources. The programme was less efficient

during the verification phase, which took much longer than planned leading to the delayed delivery of the

cash grant.

Sustainability The absence of a sustainable strategy to address the graduation approach in some of the targeted areas

was unveiled during the evaluation. Lack of skills training and activities aimed at improving

beneficiaries’ ability to secure employment opportunities were not evident. Although beneficiaries

learned how to use the ATMs, illiteracy impacted negatively on the results of this training.

2Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 72 3Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 69

10


Coordination

Coordination during each implementation phase was aided by communication between all partners.

Weekly meetings were organised and attending by representatives from each of the five partners to ensure

follow up on the implementation of the programme, resolution of problems in a timely manner, and

maintain effective communication between staff. This mechanism will be referred hereafter as the

“steering committee”. However, they lacked a well-designed management and coordination system with

clearly defined roles and responsibilities of each partner.

Although the SMS as a communication mechanism was very well received by the respondents, MoSA

staff did not approve this mode of communication due to the belief that beneficiaries did not read or

understand the messages. The MoSA staff did not provide full approval of the brochures and flyers being

distributed to beneficiaries for reasons including the fact that social workers had arrived late in this

process and sometimes during the phase of distribution of the ATM cards which means that it was not

helpful and came at the wrong time.

The complaint mechanism was identified as one of the gaps in the programme. Issues such as the absence

of ATM pin codes, deposit of incorrect cash amounts and other technical issues, were not resolved

through this structure. Another key challenge in the coordination of the programme was the poor selection

of distribution centres which resulted in lengthy and difficult travel conditions for many beneficiaries.

Key Recommendations

Below are the key recommendations following the evaluation:

Programme approach, design and interventions Adopting a multi-dimensional approach when developing cash-based programme interventions

within NPTP is highly recommended. This approach comprises an integrated and comprehensive

package of activities and does not focus only on the cash transfer amount as a single activity. This

means that programme developers during the design phase should consider the “Cash for

what?” question, as the cash modality should not be seen as an end in itself or as a stand-alone

intervention.

Special attention should be given to developing a separate, well-structured programme proposal

or concept note for the Winter Cash programme, considering that it is part of the Country

Programme Document (CPD).

A gender sensitive strategy should be devised for the programme. The strategy should state how

the programme will be guided by equality and equity measures during each phase and should

outline the importance of disaggregated data for all the beneficiaries.

The target age group of children who are eligible to benefit from cash transfer assistance should

be revised and extended to include children aged up to 18 years. (In compliance with the

definition of the child adopted by UNICEF).

The cash transfer amount should be revised, ensuring that it is sufficient to secure winter basic

needs and also considers the transportation fees of those who must travel to selected distribution

centres.

Capacity building Investment in capacity building activities for field coordinators and social workers is highly

recommended. Enhancing their abilities and skills in the areas of communication, field work,

monitoring, mentoring, follow-up, data collection, data integrity, data validity, child rights, and

research ethics will lead to greater institutional and human resource sustainability. This could be

organised in the form of a four to five days training workshop to cover basic information and

skills which they can also apply to other projects managed by the NPTP.

Awareness raising sessions should be provided for parents to educate them on how to spend the

cash grant appropriately, focusing on the essential needs of their children.

11


Selection process In order to gather accurate and updated baseline data, it is recommended to conduct a

vulnerability assessment to inform on specific vulnerability and welfare indictors of families and

communities registered in the NPTP. The information collected will provide important data

which is vital for implementation of similar programmes in addition to the winter cash

programme.

Promoting greater involvement of field coordinators, social workers and other stakeholders in the

selection process and development of selection criteria is recommended in order to strengthen

their sense of cooperation - not only being seen as implementers but as implementing partners.

The questionnaire currently does not include the minimum assessment requirements and

variables. Revision of the questionnaire tool will enable the appropriate identification of

vulnerable poor families referring to the vulnerability variables.

Coordination A well designed communication and coordination plan is recommended. The plan should include

specific milestones and measures outlining the specific roles and responsibilities to guide each

phase of the programme: planning, distribution, monitoring and post-delivery activities.

Establishing unified guidelines and principles before the implementation phase will help to avoid

addressing issues which arise in an ad hoc manner.

Cash modality Joint partnership with other organisations, such as WFP, where each partner complements the

other to address the needs of the NPTP beneficiaries is recommended. This means that UNICEF,

for example, can provide cash assistance while WFP can provide conditioned e-vouchers under

the same programme in a systematic way throughout the year, combining both restricted and

unrestricted cash transfer modalities.

It is highly recommended that NPTP adopts the unrestricted cash transfer, which is considered to

be the most favourable cash modality by a majority of beneficiaries. Certain conditions are

encouraged to be incorporated including a revision of the cash grant amount; adoption of a

recurrent policy throughout the year and replacement of cash grants with an e-food voucher on

quarterly basis.

Monitoring & Evaluation A well-structured M&E system should be developed comprising various methodologies and tools,

as well as community reflection tools which are linked to a combination of qualitative and

quantitative indicators.

Best practices should be supported, documented and shared regularly for enhanced learning

purposes.

Special attention should be given to increasing the involvement of field coordinators and social

workers in the different phases of programme implementation.

Joint partnerships

Developing joint partnerships with other organisations working with the same vulnerable families

registered in the targeted areas is strongly recommended. Through collaboration and utilisation of

data collected during the vulnerability assessment, integrated, joint interventions can be

developed reducing duplication of efforts.

The evaluation suggests investigating the possibility of targeting the winter needs of groups

whose profiles do not meet the current selection criteria or the mandate of partner organisations.

12


Scaling up opportunities

In order to promote the institutional sustainability of the NPTP and strengthen its ownership of other

similar programmes, special attention should be given to adopt a well-structured strategy that includes

the following sub-strategies:

Develop a comprehensive, well-designed capacity building training in advance of project

implementation.

Provide awareness sessions for targeted parents and/or caregivers to ensure the correct use of cash

transfer assistance to secure the basic needs of their families during the winter season.

Conduct a vulnerability assessment to collect accurate and updated data on vulnerability and

welfare variables.

Promote collaboration with the other organisations working with the same targeted groups to

maximise the use of resources.

Set clear and structured communication and coordination mechanisms for all stakeholders

including key partners and the participating bank. This will increase the efficiency of the

programme; improve timely delivery of activities and help to identify and address challenges

promptly.

Develop a well-structured monitoring and quality assurance framework with clear mechanisms,

methodologies and tools to track the progress during the project lifecycle.

13


1. Objective of the Evaluation

1.1 Introduction/Background

Today, Lebanon’s four million citizens struggle to withstand the social, economic and political

impact of the Syrian conflict. A stagnant economy, coupled with the threat of insecurity, increases

fears that children and families living in poverty will sink further into deprivation. As a consequence

of the on-going Syrian crisis, population augmentation, of both host and displaced communities has

resulted in gradually shrinking spaces for livelihood and income-generation which significantly

impacts the ability for vulnerable children and their families to secure their basic needs.

In 2016, President Michel Aoun was elected as Head of State after a 29-month political deadlock and

presidential vacuum. Despite a magnitude of challenges, the Government with UNICEF Lebanon’s

financial and technical investments in strengthening public institutions and their delivery systems

contributed to over one million vulnerable children being reached.

Nearly half of those affected by the Syrian crisis are children and adolescents and are currently

growing up at risk, deprived, and with an acute need for basic services and protection4. Vulnerable

families are subject to seasonal hazards and a substantial proportion of these have had to increase

their spending patterns to cover winter needs such as heating fuel, shelter repairs and winter clothing.

Addressing the seasonal needs of families in Lebanon requires a multi-sectorial approach. During the

harsh winter months, vulnerable families rely on the winter cash assistance to address their basic

needs, often at the expense of purchasing other key items. Poor Lebanese and refugee households are

highly affected by inconsistent flows of income, which shift season to season and year to year.

Employment is largely concentrated in the informal jobs market and remains unreliable. In particular,

during the winter months, the availability of work is less which directly impacts the earning ability of

vulnerable families (reduced by 40 per cent as reported by Lebanese poor households5).

Cash Transfer Programming (CTP) has rapidly become an important and often preferred mode of

assistance for humanitarian actors around the world. Organisations operating in areas with conflict-

affected or displaced persons and with functioning markets are increasingly using cash-based

assistance to meet the needs and improve livelihood outcomes of targeted populations.

1.2 Programme overview

The Winter Cash programme supports economically vulnerable Lebanese children and their families.

It was designed through a collaborative process between UNICEF, National Poverty Targeting

Programme (NPTP), Ministry of Social Affairs (MoSA), and World Food Programme (WFP), the

Office of the Prime Minister and the Lebanese Presidency of the Council of Ministers (PCM). The

objective of the programme was to introduce cash-based assistance as a pilot to further strengthen the

existing social safety nets and social protection mechanisms in Lebanon. A cash-based intervention

was considered to be the most relevant and effective mechanism for reaching 75,000 vulnerable

children across Lebanon. Implementation is the responsibility of the NPTP, Central Management

Unit (CMU) at the presidency of the Council of Ministries and MoSA, with technical support

provided by UNICEF and WFP. The programme was designed to be implemented by the NPTP, the

4LCRP 2015-2016 p.9 5Poverty, Inequality and Social Protection in Lebanon, January 2016, Oxfam, AUB

14


Central Management Unit (CMU) at the presidency of the Council of Ministries and Ministry of

Social Affairs (MoSA) with technical support from UNICEF and WFP. The NPTP established in

2011, aims to support vulnerable Lebanese families in meeting their most urgent needs. To date,

86,000 Lebanese households have been identified as living below the poverty line (USD3.84/day), of

which nearly one-third (25,000 households) live in absolute poverty (below USD2/day)6. The NPTP

provides health and education subsidies to eligible beneficiaries, and food assistance through e-

vouchers is provided to 10,000 (27,000 extremely poor individuals).

The NPTP, PCM and the World Bank developed a Proxy Means Test (PMT) formula to target NPTP

beneficiaries. Inclusion of applicant households was based on a proxy score that calculates the

welfare of individuals by estimating their consumption per day as a proxy measure of income per

day. The test included questions to assess the applicant’s standard of living, such as the employment

status, level of education, marital status, physical ability, housing conditions, assets owned and their

geographical location. It is agreed that if the score is under a certain determined threshold that

corresponds to the Lower Poverty Line, then the household was included as a beneficiary of the

NPTP. This was followed by providing the head of the household or the applicant with a “Halla”

card, which can be used to access the education and health services at any of the 220 SDCs, and local

clinics.

The Winter Cash programme relied on the Proxy Means Test (PMT) score to target the most socio-

economically vulnerable Lebanese households with children. MoSA social workers updated data in

the NPTP database which identifies the most vulnerable households. They visited these households

to gather updated information (specifically the number of children in the household and telephone

numbers) which was then entered into an Optical Marking Recognition (OMR) form, which ranked

and selected the most vulnerable families corresponding to the target of 75,000 children. At the end

of this process, 26,052 families were selected as beneficiaries of the Winter Cash Programme. A cash

grant of USD 407 per child was transferred to beneficiary households. This value is based on the

estimated cost of the winter clothing kit which was distributed during the 2014/2015 UNICEF winter

campaign. The payment was transferred through ATM cards issued by WFP and Banque Libano-

Francaise. Distribution of cards took place between February 15th and March 10th 2016 by MoSA

social workers in 22 selected SDCs, out of a total 220 existing in the country.

Expected outcomes/results

Subsequent to the Winter Cash programme project proposal, the Humanitarian Response and

Resilience Plan 2016 (HRRP) and other project documents, the following key expected outcomes

were developed: which targets

Key Objectives/ Expected Results (medium and short-term results)

Outcome 1:

Lebanese children and the families vulnerable to seasonal hazards and unexpected winter

emergencies are able to maintain safe access to goods and services.

Expected Outputs:

1.1: Households in the targeted areas at risk of seasonal and unexpected hazards survive without

6 Poverty, Inequality and Social Protection in Lebanon, January 2016, Oxfam, AUB 7 The US$40 was the monetized amount of the children’s winter clothing kits and e-vouchers which UNICEF had provided to children of NPTP

support families in previous years.

15


adverse effects.

Planned activities

1.1.1: Provide cash grants for seasonal hazards affected households (USD40/child in the targeted

families)

Outcome 2:

Strengthened delivery mechanisms for the NPTP via the incorporation of new systems: Cash-

based programme

Expected Outputs:

2.1: Strengthened child-focused delivery mechanisms for NPTP via the incorporation of new child-

sensitive systems, including cash-based programmes

Planned activities

2.1.2: Conduct a joint study with MoSA/NPTP on outcomes and impacts of winter cash-based

programme for socio-economically vulnerable children and their households.

1.3 Key programme stakeholders – including key partners

Table 1 below shows a brief mapping of the stakeholders involved in the programme. It lists the

different key actors/stakeholders – including the key partners who were targeted during the data

collection phase.

# Key stakeholder Actors involved

1 UNICEF Lebanon

staff

UNICEF Programme Specialist Cash and Vouchers

UNICEF Winter Focal Points

UNICEF Chief of Field Operations

2 NPTP Director of the NPTP

NPTP Business IT

SDC directors

3 MoSA MoSA Winter Cash Transfer Programme Focal Point

MoSA Field work coordinators

MoSA social workers

Adviser of the Minister of Social Affairs

4 PCM/CMU CMU Director

PCM/CMU Statistician

5 WFP WFP Cash and Vouchers Unit Programme Associate(s)

6 Direct beneficiaries Sampled groups from the poor vulnerable families (parents and

caregivers)

Table1. Key Stakeholders

16


2. Evaluation Purpose, Objectives and Scope of Work

Referring to the agreed scope of work with UNICEF & other stakeholders, UNICEF commissioned

the consultant to conduct “The final evaluation of the winter cash transfer programme for

Lebanese poor children & their families”.

2.1 Overall Goal of Evaluation

The overall goal of the programme evaluation is to assess (or measure) the outcome and

effectiveness of the Winter Cash programme in order to strengthen the joint planning, design and

implementation capacity of the NPTP, MoSA, PCM and UNICEF as well as meet the needs of the

children by providing appropriate quality assistance in a dignified manner.

2.2 Specific Evaluation Objectives

In order to provide a robust evaluation of the joint NPTP / UNICEF winter cash response, the

specific objectives of the programme evaluation are as follows:

Assess the overall effectiveness of the Winter Cash programme modality in Lebanon.

Assess the appropriateness and acceptance of the Winter Cash programme modality from

the perspective of the target population.

Assess the joint collaboration of the overall implementation of the Winter Cash programme.

Provide concrete recommendations to improve planning and implementation of joint programmes

and deliver optimal results for the most economically vulnerable Lebanese children.

2.3 Scope of Work

In order to achieve the specific objectives, the evaluation covered the entirety of the programme

including the following components (in reference to the project cycle process):

The design & planning phase: This includes the selection process, payment modality and

repartition of the duties and responsibilities among the different actors as well as the planning

phase.

The Implementation phase: This includes all training conducted, distribution of ATM

cards, feedback mechanisms and monitoring.

It is important to note that the scope of work in the current evaluation focused mainly on poor

vulnerable Lebanese families only and did not consider Syrian and Palestine refugees addressed

under the winter HRRP-2016.

17


The geographic scope of the evaluation:

The evaluation focused primarily on the targeted areas of programme implementation shown in Table

2 below.

The key programme beneficiaries (vulnerable Lebanese poor children & their families) were selected

from the following targeted working areas8:

S Governorate Cazas Targeted # Of SDCs involved

1 Beirut Beirut 4

Sub-Total 4

2 Bekaa Baalbek 9

Hermel 1

Rashaya 4

West Bekaa 4

Zahle 3

Sub-Total 21

3 Mount-Lebanon Aley 6

Baabda 6

Chouf 10

El Metn 5

Jbeil 4

Kesrouan 5

Sub-Total 36

4 Nabatieh Hasbaya 1

Marjayoun 3

Nabatiyeh 3

Sub-Total 7

5 Akkar Akkar 10

Batroun 4

Bchare 1

Koura 2

Sub-Total 17

6 Tripoli Minieh-Dannieh 3

Tripoli 3

Zgharta 2

Sub-Total 8

6 South Bint Hbeil 4

Jezzine 1

Saida 5

Sour 5

Sub-Total 15

Total 26 108

Table2. Geographic scope of the evaluation

8List of Distribution of UNICEF_NPTP beneficiaries by Governorate_Gaza_SDC.

18


Time period covered by the evaluation: The evaluation covered the period between August 2015

and May 2016, starting from the first meetings held between PCM, MoSA, WFP and UNICEF

pertaining to the design of the programme and ending with the last Post Distribution Monitoring

(PDM) process.

2.4 Evaluation Criteria & Questions

With reference to the TOR and the approved inception report, the evaluation is primarily based on

the DAC criteria with a special focus on Relevance (including appropriateness), Effectiveness

(including acceptance), Efficiency, Sustainability and Coordination reflected in the key questions

outlined in Table3 below.

Evaluation

criteria Key questions / areas of discussion

Relevance

(including

Appropriateness)

1. To what extent were the project intervention strategies, project design and

approaches appropriate and relevant to promoting the intended objectives of

the programme? And to what extent were the needs of the most socio-

economically vulnerable Lebanese households met?

2. To what extent were the criteria of the selection process developed to ensure

that the most vulnerable and poor children were reached and benefited

properly from the programme?

3. To what extent has the programme been successful in identifying the most

relevant partner/actors for this project and what was the added value of these

partnerships?

4. How gender sensitive was the programme when reaching the most

vulnerable children?

5. Do the current Log frame and/or results framework and relevant indicators

in the Winter Cash programme document need improvement to better serve

the programme objectives?

Effectiveness

6. To what extent were the vulnerable children and their families targeted by

the Winter Cash programme reached (as planned)? How satisfied were they

about the programme?

7. To what extent did the programme have an impact on beneficiaries in

improving access to essential goods and services of their choice in a safe,

dignified, and empowered manner? (Key changes affecting the lives of

beneficiaries as result of the programme)?

8. Are there other unintended or unforeseen changes evident in the lives of the

targeted groups of poor children & their families? Are there any unintended

beneficiaries of the programme?

19


9. What was the level of acceptance of the cash modality as a tool to support

poor Lebanese households to cover their winter needs (winterisation) from

the point of view of all the different actors: MoSA, NPTP, SDCs, field staff

and the beneficiaries?

10. To what extent were the capacities of the different actors involved -

particularly the NPTP and MoSA - adequate and efficient to ensure the

quality of the programme implementation? What can be improved in this

regard?

11. To what extent does the programme have an effective monitoring and

evaluation mechanism (including reporting, quality assurance and

reflection)? Are there proper processes on different levels (UNICEF, NPTP,

and CMU/PMC) in place? How has the programme used information

generated to inform and make programmatic adjustments & corrective

actions during the life of the project?

12. To what extent did the project consider gender equality in the programme

implementation – including the selection process (e.g. is USD40 an

equitable amount for both boys and girls? On what basis was the amount

calculated and does it take into account gender-based requirements?)

Efficiency 13. To what extent was the Winter Cash programme cost-efficient – including

optimal usage of programme resources & time efficiency as opposed to

other modalities/programmes that could be designed to cover the same

needs of the Lebanese vulnerable poor children and their families?

Sustainability 14. To what extent did the programme equip service providers (NPTP/MoSA)

and beneficiaries with the required capacities (skills, knowledge and

sustainability), resources (connections) and confidence (institutional and

financial sustainability) to sustain the programme implementation?

Coordination

15. How effective was the management and coordination system - as part of

joint programme partnership between UNICEF, NPTP, MoSA, CMU/PCM

and WFP - in achieving results for the target population?

Table3. Evaluation Criteria and Key questions / areas of discussion

2.5. HRBA, Gender Equality and Equity Principles in the Current Evaluation The current evaluation took into consideration the following key Human Rights Based Approach

(HRBA), gender equality and equity principles to ensure that they were mainstreamed in the

evaluation process as follows9:

9 United Nations Evaluation Group (2014). Integrating Human Rights and Gender Equality in Evaluations. New York: UNEG- page 32.

20


Inclusion: The evaluation investigated which groups benefitted from the programme and

targeted them during the assessment process. The evaluation developed disaggregated data

of all the groups interviewed as shown in Annex 2 of this report. Participation: The evaluation adopted and followed a participatory evaluation approach,

applying methodologies and data collection tools engaging the programme stakeholders in a

consultative manner. The evaluation measured the level of their participation and

involvement in the project design, planning, and monitoring activities. Fair power relations: The evaluation focused specifically on assessing the degree to which

power relations among the different programme stakeholders and implementers supported or

undermined programme results. Additionally, the evaluation considered the status of groups

in the targeted population within the programme.

3. Evaluation Methodology

3.1 Overview

With reference to the TOR, the evaluation process was guided by the following four internationally

recognised standards:

Utility – Evaluation findings and recommendations will serve the different stakeholders.

Feasibility – Realistic, prudent, diplomatic and frugal.

Propriety – Conducted legally, ethically and with due regard for the welfare of those involved in the

evaluation process, as well as those affected by its results.

Accuracy – Reveal and convey technically adequate information about the features that determine

the worth or merit of the programme being analysed in the evaluation process.

3.2 UNEG Norms and Ethical Standards

On the other hand, the evaluation was guided in all its processes with the key UNEG norms and

ethical standards10 as follows:

Intentionality of evaluation - by ensuring the utility and necessity of the information and

results captured from the current evaluation for the different stakeholders and target

audiences.

Key obligations11 to the evaluator:

Independence - both behavioural and organisational independence.

Impartiality- objectivity, professional integrity and absence of bias.

Credibility – transparent evaluation processes and inclusive approaches involving relevant

stakeholders.

Conflict of interest- Ensure neutrality in all activities related to the programme design,

planning and implementation.

Honesty and integrity

Accountability

10Linda G. MorraImas&Ry C. Rist: The Road to Results- The world bank – Washington- 2009- Page 509-510 11United Nations Evaluation Group (2016). Norms and Standards for Evaluation. New York: UNEG- page 11-12.

21


Key obligations to participants:

Respect for dignity & diversity

Rights

Confidentiality

Avoidance of harm

In order to address the key questions outlined in the TOR, the consultant used formative evaluation

practices combined with participatory approaches when engaging the different stakeholders and

beneficiaries.

As the programme was a pilot project, the scope of the evaluation was mainly focused on the

learning purpose rather than applying it for accountability processes.

The evaluation process was initiated by conducting a desk review of ten key documents

including: the TOR; basic assistance/winter HRRP 2016; implementation plan; PDM final

products; distribution documents; forms and sheets; lessons learnt document as part of the

PDM process; complaint mechanism document; and NPTP card management in addition to

the respective questionnaires. (See Annex 3).

Thirteen semi-structured individual interviews were conducted with relevant UNICEF staff

and key partners. (Five males and eight females).

Seven semi-structured group interviews were conducted with a total of 24 participants (Five

males and nineteen females) with relevant MoSA staff in the five targeted working areas

located in four governorates.

One group interview took place with relevant WFP staff.

The participatory approach was applied by conducting FGDs to capture the outcomes of

programme activities and any significant impact on the quality of life of the beneficiaries.

Five FGDs were facilitated with five sample groups of beneficiaries from five working areas

affiliated to five SDCs located in four governorates with a total of 53 parents & caregivers.

The current evaluation used eight FGDs (men and women) and four key informant interviews

(with SDC directors, social workers and field coordinators) which were conducted last year

during the Post Distribution Monitoring process.

Ninety stakeholders were interviewed in the current evaluation process in addition to the 101

who were interviewed during the Post Distribution Monitoring process last year. A total of

191 were therefore interviewed to assess the Winter Cash assistance programme. Those

interviewed were involved in the programme either as partners or as direct programme

beneficiaries (Refer to Annex 2 for more details).

Additionally, the survey that was conducted by a third party during the Post Distribution

Monitoring - using Computer Assisted Telephone and face to face interviews - was also used

in this evaluation. A total of 457 out of 500 beneficiaries were interviewed in Bekaa, Mount

Lebanon, North and South. Triangulating the data to draw comparisons from the information

was essential in order to validate the findings.

Table4 below shows the number of stakeholders interviewed and consulted during the current

evaluation process disaggregated by sex and type of evaluation methodology/tool. The details of this

list are presented in Annex 2.

22


S Type of evaluation

methodology/tool

# of

interviews/FGDs

Classification of participants by

sex

Male Female Total

1 Individual semi-structured interviews 13 5 8 13

2 Group semi-structured interviews 7 5 19 24

3 Focus Group Discussions 5 16 37 53

Total 25 26 64 90

Table4. Type of evaluation methodology/tool

3.3 Data Collection Methodologies and Tools

3.3.1. SOURCES OF DATA COLLECTION

The following are the key sources of information referred to in the current evaluation process:

- Information collected and captured from FGDs with direct beneficiaries.

- Information collected from the individual and group interviews conducted with the key

stakeholders including the main partners of the Winter Cash Programme.

- Post Distribution Monitoring (PDM) findings.

The following key criteria were applied in order to select the sample of SDCs that were to

facilitate the interviews and FGDs:

The SDCs which have the highest number of the beneficiaries referring to the above actual

distribution schedule.

Ensure that the sample of SDCs, representing each of the rural/semi-rural/urban/semi-urban

and/or slums, is useful for analysing the spending patterns and sufficiency of the cash transfer

amount to secure their winter basic needs.

This sample was developed after the initial recommendation from UNICEF o and the adviser of the

NPTP and MoSA staff.

The following criteria were applied for the selection of participants in FGDs: The

participants were selected from the beneficiaries of the Winter Cash programme in each

SDC, ensuring that all targeted working areas were represented under the SDC.

The number of children per household who benefitted from the Winter Cash programme was

considered during the selection process to ensure a diverse group of beneficiaries from those

with only one child to those with several children aged up to 15 years. This proved to be

helpful in capturing different views during the evaluation process.

Access to targeted working areas was a factor due to security issues in some areas.

Other basic criteria are outlined in the FGD guide included in Annex 4.

23


3.3.2. DATA COLLECTION TOOLS

A. SEMI-STRUCTURED INTERVIEWS (GROUP AND INDIVIDUAL INTERVIEWS)

The evaluation included a series of semi-structured interviews with key partners and stakeholders

involved in the programme implementation. These interviews were conducted with NPTP, MoSA,

WFP, SDCs, field coordinators, Social Workers, and CMU/PMC in addition to UNICEF. Participants

were selected based on TOR requirements and consultations with UNICEF and the desk review

findings. The selected individuals were interviewed for their first-hand knowledge about the

programme and their involvement in the programme implementation itself.

The interviews were roughly structured with a list of topics for discussion, and an interview guide to

allow a free flow of ideas and information. The consultant framed questions spontaneously, probed

for information and took notes accordingly. (Details of the interviews are available in the tools’

guides & protocol section in Annex 4).

Twenty interviews (13 individual interviews and 7 group interviews) were conducted with a total of

37 stakeholders (Refer to Annex 2 for more details).

B. FOCUS-GROUP DISCUSSIONS (FGDS)

The consultant facilitated FGDs in four selected governorates across Lebanon. The purpose of the

FGDs was to capture the most significant impacts, if any, of participation in the programme and to

gain a better understanding of their opinions, insights, ideas, and challenges encountered, and

recommendations based on their level of satisfaction.

The advantage of FGDs is that they allow for open dialogue and discussions that helps to gather

insightful information for the evaluation. .

Table 5 below shows the number of beneficiaries/households (HHs) per governorate, Caza and SDC

and was used as a guiding reference to identify the targeted working areas and the beneficiaries who

took part in the FGDs.

S Governorate Caza Targeted # Of SDCs

involved

# of HHs

benefitted

1 Beirut Beirut 4 46

Sub-Total 4 46

2 Bekaa Baalbek 9 2982

Hermel 1 534

Rashaya 4 122

West Beqaa 4 552

Zahle 3 572

Sub-Total 21 4762

3 Mount-Lebanon Aley 6 737

Baabda 6 921

Chouf 10 306

El Meten 5 33

Jebeil 4 69

24


Kesrouan 5 105

Sub-Total 36 2171

4 Nabatieh Hasbaya 1 8

Marjayoun 3 3

Nabatiyeh 3 149

Sub-Total 7 160

5 North Akkar 10 7568

Batroun 4 124

Bchare 1 59

Koura 2 182

Minieh-Dannieh 3 3085

Tripoli 3 4959

Zgharta 2 679

Sub-Total 25 16656

6 South Bint Hbeil 4 322

Jezzine 1 4

Saida 5 1069

Sour 5 862

Sub-Total 15 2257

Total 26 108 26052

Table 5. Number of beneficiaries/households per governorate, Caza and SDC

The following table (Table 6) shows the results of the selection process and number of beneficiaries

per target area. The consultant selected four governorates with the highest number of HHs that

benefitted from the programme. Referring to Table 5 above, the key specific areas in the targeted

governorates with the highest administrative level and highest number of beneficiaries were: Baalbek

SDC – Bekaa governorate (2,982 families), Rahba SDC- Akkar governorate (7,568 families), Tripoli

SDC – North governorate (4,959 families) in addition to the others mentioned above.

After consultations with UNICEF and considering any possible security & safety limitation during

the data collection phase, Table 6 below shows the breakdown of the sample:

S Governorate CAZA(s)

Targeted

SDC(s)

targeted

# of FGDs/

interviews Target groups # of

participants

1 Mount

Lebanon

Baabda Ghobeiri 4

SDC director

Field coordinators

Social workers

Direct beneficiaries

1

2

5

8

Sub-total 16

2 Bekaa Baalbek Baalbek 5 SDC director

Director of SDCs -

Beqaa

Field coordinators

Social workers


1

1

1

7

13

Sub-total 23

3 Akkar Akkar Rahba 4 SDC director 1

25


Field coordinators

Social workers


1

2

10

Sub-total 14

4 Tripoli Tripoli El-Mina 3 SDC director

Social workers


1

2

22

Bab El

Tabbaneh 3 SDC director

Field coordinator

Social workers

1

1

4

Sub-total 31

Total 19 84

Table 6. Breakdown of the sample

Specific FGD guide was developed and included under annex 4 attached to this report.

3.4 Limitations and Challenges of the Evaluation Process

The following are some of the limitations faced during the evaluation process:

Availability of programme documents: During the initial phase of the evaluation, it was not possible for the consultant to access or

reference the selection criteria. This was rectified when they were provided by the CMU and

the SDCs.

There was difficulty accessing relevant versions of the questionnaire pertaining to the

analysis of the selection process, which were also later provided by the CMU and SDCs.

The HRRP 2016 includes the results and the respective indicators for poor Lebanese families,

Syrian and other refugees. This was in fact one of the challenges encountered especially for

the logical framework that should be referred to in the current evaluation. Therefore, if we

have one programme document addressing the winterization basic needs targeting both

Syrian or other refugees and the poor Lebanese, we have to have separate results framework

and respective activities to be referred during the evaluation process.

It was difficult to conduct planned FGDs due to security reasons in some of the targeted areas

(e.g. Sir El-Doniyeh-Tripoli). However, this issue was addressed by conducting additional

FGDs in other similar areas (El-Mina-Tripoli).

The FGDs relied on participants’ memory only to recall the impact of the programme which

was implemented approximately one year ago. However, the information previously

collected during the PDM process played a vital role in triangulating all of the data. There

was a strong correlation and similarity between the key findings captured from the two

processes.

High turnover of staff involved in programme implementation, especially at MoSA and

NPTP, led to certain gaps in information. However, this issue was overcome by organising

meetings with them and other key staff members such as field coordinators who had

transferred to another SDC.

26


4. Key findings

The following sections review the data collected and provide an analysis of the key findings based on

relevance, effectiveness, efficiency, coordination and sustainability criteria. Refer to Annex 6 for a

detailed copy of the evaluation framework & matrix with the respective questions for each indicator.

4.1. Relevance

Question1: To what extent were the project intervention strategies, project design,

interventions, and approaches suited to the priorities, needs and rights of the targeted children

and their families; as well as appropriate (and relevant) to promoting the intended objectives of

the programme? And to what extent were the needs of the most socio-economically vulnerable

Lebanese Households met?

The overall goal of the Winter Cash programme was to introduce cash assistance to serve as a pilot to

further strengthen existing social safety nets and social protection mechanisms in Lebanon. In order

to do that, the Winter Cash programme provided one-off cash assistance to 75,000 children through a

cash grant of USD 40 per child to secure basic winter needs. The expected objectives/outcomes of the

winter cash programme mainly focused on the following two objectives:

Enable Lebanese children and their families to cover a portion of their winter expenses to help

their children cope with the winter cold in Lebanon.

The Winter Cash programme aimed to strengthen delivery mechanisms of the NPTP via the

incorporation of new child-sensitive systems, including cash-based programmes like the

unconditioned cash transfer assistance modality.

The strategy adopted in the Winter Cash programme mainly focused on the collaboration between

UNICEF, NPTP, MoSA, WFP, the office of the Prime Minister and the Lebanese Presidency of the

Council of Ministries (PCM). The programme relied on the PMT score to target the most socio-

economically vulnerable Lebanese households with children, referring to some key variables in the

final selection process such as the employment status, level of education, marital status, physical

ability, housing conditions, assets owned and their geographical location.

Analysis of cash transfer modalities adopted in other similar programmes addressing the winter

basic needs for vulnerable groups revealed that alternative conditioned cash transfers are used,

mostly by other organisations. These modalities include the e-voucher for food, paper vouchers for

food, and paper vouchers for fuel or for clothing kits. The current Winter Cash programme adopted

the unconditioned cash transfer modality as a pilot intervention managed by NPTP/MoSA with the

technical support of UNICEF, WFP and other actors. Families have the opportunity to secure and

purchase their basic winter essentials in a more flexible, unrestricted manner, respecting both their

rights and needs.

Overall, the Winter Cash programme has focused on addressing the rights and the needs of the

Lebanese Poor vulnerable children and their families in the targeted areas across Lebanon. In

general, the evaluation found that the programme intervention could reach the most socio-

economically vulnerable Lebanese Households significantly and was successful in meeting most of

their basic winter needs.

27


The evaluation found that the intervention strategy implemented by UNICEF and its partners helped

vulnerable children and their families to meet some of their basic winter and emergency needs such

as food, medication and winter clothes. Without this assistance, their level of poverty would be

further exacerbated. The evaluation found that the NPTP succeeded through the implementation of

the Winter Cash programme in securing most of the basic winter needs of the targeted families.

The two areas of intervention highlighted as needing additional support were: (1) Capacity building

and (2) Raising awareness.

Capacity building

The evaluation revealed that the capacity building component was missing in the following phases of

programme design, intervention, and activities, which directly affected achievement of programme

objectives. Strengthening this component is considered a pressing issue, to promote fairness in

delivery mechanisms and must be addressed in future programme design.

Capacity building of Field Coordinators and Social Workers

Although the Winter Cash programme aimed at strengthening delivery mechanisms and enhancing

the skills of NPTP staff, the evaluation found that the capacity development component was largely

absent. Through interviews conducted with staff from NPTP and MoSA, it was reported that not all

of the social workers had a scientific professional qualification, thus limiting their ability to play a

full role in programme implementation, particularly when working directly with beneficiary families.

An Advisor at MoSA indicated that the supply of social workers is also much greater than the

demand for their skills. This was validated through interviews conducted with three SDC directors

during the evaluation process.

“There are a large number of human resources represented in the category of the social workers

and field workers but they are not utilised completely, fairly and effectively” - CMU/PCM staff

interviewed during the evaluation process commented.

Limitations found in the knowledge, capabilities and skills of social workers indicate an urgent need

to include a training component in the programme design in order to ensure the quality of the

delivery process and to arrive at improved socio-economic results for the beneficiaries.

SDC directors, NPTP staff, MoSA staff and WFP partners recommended the following areas in

which to develop capacity, enabling improved performance of social workers to effectively deliver

the Winter Cash programme: communication skills, social work & research ethics, data collection,

data reliability and validity, data integrity, and child rights.

Such capacity building activities are essential to reduce the gaps and individual differences among

social workers who represent the key personnel in this programme in terms of knowledge, skills and

their performance in the field where they work with targeted communities and vulnerable families.

One SDC reported that questionnaires distributed as part of the selection process were filled by

telephone by two social workers, indicating a possible lack of credibility of certain information

collected during the implementation phase. Although this was found in only one out of four SDC

visited it highlights the need to include sufficient training for social workers and field coordinators to

ensure that they suitably reach the most vulnerable families. The capacity building component must

also include a monitoring mechanism enabling greater awareness of social workers to the needs of

28


beneficiary families and establish strong relations with them. This will help to ensure accurate

reporting on the status and changing conditions of the families.

Training and workshops

When social workers obtained the ATM cards at the beginning of the distribution process, they also

received an introductory administrative awareness session which included an explanation on the

complaint mechanism. NPTP and MoSA staff interviewed during the evaluation process indicated

that they would - rather have received a capacity building training on communication and facilitation

skills which they deemed to be more appropriate for the distribution process.

Beneficiaries’ awareness session

Feedback collected from MoSA staff, SDC directors and field coordinators, during the PDM process

indicated that although the workshops were well received, the beneficiaries were not interested in

taking part. The majority of beneficiaries did not attend these sessions. It is therefore highly

recommended that a demonstration on the use of ATM cards be provided during the ATM card

distribution itself, rather than in a separate workshop12.

Capacity building of MoSA/NPTP staff

To ensure the sustainability of the programme; promote its ownership at the level of MoSA and the

NPTP; and to enable them to replicate the programme it in the future, MoSA staff should receive

capacity building to enable them to monitor the implementation of the programme and raise their

awareness on the importance of follow-up and presence in the field. This will maintain the accuracy,

effectiveness, efficiency of the programme and increase its credibility among beneficiaries.

Parents’ awareness

During FGDs in Mount Lebanon, Akkar and Tripoli it was reported that on a few occasions, parents

did not use the cash grant appropriately, leading their children to miss out on the benefits of the

assistance. Examples of how the grant was spent are cigarettes, and going to the hairdresser.

Although these cases cannot be generalized, the inclusion of awareness raising sessions for parents is

recommended for future programmes, to educate them on how to appropriately use the cash grant to

meet the needs of the child - the main objective of the programme. Outlining the correct use of the

cash grant was largely conducted in an informal manner, when parents received the ATM cards. This

proved to be inadequate for communicating the intended purpose of the grant. Revisiting the “Cash

for What?” question for multi-purpose and integrated programmes, to better identify the expectations

of parents are advised.

Definition of the child in the Selection criteria of the Winter Cash Programme

At the global level there is considerable debate around the definition of a child, compared to that of

an adolescent. UNICEF considers a child to be under the age of 18 - as defined in the Convention on

the Rights of the Children (CRC). However, the Winter Cash programme targeted children aged up

to 15 years due to limited funding and the priorities of the most vulnerable age groups.

12InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 13

29


The justification and the rationale behind this age limit was not present in any of the programme

documents reviewed, nor was it explained to the social workers, field coordinators or the SDCs

directors.

“I have to treat my children equally and that’s why I added and/or borrowed additional money to

buy the winter clothes for all my children instead of buying it for all my younger children up to 15

years” - participants in Akkar, Mount Lebanon and Tripoli.

It is recommended to provide a clear explanation on how the target age was reached, to both

implementers and beneficiaries, to reduce confusion and encourage a better understanding of the

programme.

The needs of most socio-economically vulnerable Lebanese families

It was concluded that the objective of the programme in meeting the seasonal needs of the most

socio-economically vulnerable Lebanon households through the provision of USD 40 per child was

not achieved. The cash grant was insufficient in securing the basic needs related to seasonal hazards,

such as food, medication and clothes. This was particularly highlighted by participants during FGDs

in Mount Lebanon and Bekaa. All SDCs’ directors, field coordinators and social workers interviewed

during the current evaluation process indicated that despite the fact that the programme managed to

benefit many families, it excluded a large number of vulnerable families who were also in need of

financial assistance. This was validated again by the beneficiaries during the FGDs.

The majority of stakeholders and direct beneficiaries interviewed in the evaluation process reported

that in order to solve this, there are two possible suggested scenarios:

1- To increase the amount of money and decrease the number of families so as to ensure that

more of their needs are met;

2- To include all families who carry the “Hala” card and are classified as vulnerable families,

while decreasing the amount of the grant with the aim of serving more.

Two solutions are recommended in order to meet the needs of vulnerable families: (1) Increase the

total value of the grant provided, to cover the needs of entire families, and (2) Engage more partners

in the programme with special support from MoSA and other ministries to provide cash or in-kind

assistance to the programme, such as vouchers for food, clothes, fuel, or medications, ensuring that

the all needs of the most socio-economically vulnerable families are met sufficiently and in a

dignified manner.

Meeting the needs of more vulnerable families in the future would only be possible by ensuring

accurate and updated data to better inform programme decisions. The data used in the selection

process of the current programme was outdated and was not validated by social workers.

Question2: To what extent were the criteria of the selection process developed to ensure

that the most vulnerable and poor children were reached and benefited properly from the

program?

The selection process was developed jointly between the NPTP, UNICEF, and CMU/PCM

including the selection criteria. However, there was no clear documentation or understanding of

how the selection process worked. Social workers & field coordinators did not participate in

setting the selection criteria and other relevant questionnaires. Additionally, the data referred in

30


the selection process relied on outdated sources (from2011) in addition to omitting some of the

important variables and criteria in the selection process.

Selection criteria

The evaluation found that there was no clear documentation or understanding of how the selection

process worked. It is worth noting that the NPTP, PCM and the World Bank developed a Proxy

Means Test (PMT) formula to target beneficiaries whether through the current Winter Cash

programme or other programmes. However, the Winter Cash programme relied on the PMT score to

target the most socio-economically vulnerable Lebanese households with children to include them in

the programme. In order to proceed to the targeted HHs, MoSA social workers updated the NPTP

database of the 35,000 identified most vulnerable households and entered it into an Optical Marking

Recognition (OMR) form, which was then inserted into the live database.

Thereafter, the selection process was purely scientific and based on a statistical, computer-based

system developed by experts. This system analysed 60 indicators and criteria which defined the

situation of each household through information collected from field and household visits, and which

was then entered into the computer system to generate a final score. This score dictated eligibility for

the programme. It was found that the data used in the selection process was outdated. In addition, the

new questionnaire, which was used as a validation tool to select the poor vulnerable families, did not

take into consideration new-borns.

The use of an outdated list during the selection process meant that the poorest families were not

always reached. MoSA staff who were interviewed in the PDM process indicated the strong belief

that the selection process was “unfair”, and that many families who should have been eligible were

excluded, while others who were less in need received the cash assistance.

In addition, some SDC directors, social workers, field coordinators and FGD participants - especially

in Akkar, Tripoli and Beqaa/Baalbek - indicated that the equitability of the selection process was an

issue of concern. They implied the existence of political bias and that many families who were

eligible to receive the grant did not benefit from it, while others who were less needy did. Moreover,

NPTP field workers assured the 35,000 households that were surveyed and verified using the NPTP

PMT formula that they would receive the ATM cards, however only 27,000 cards were printed.

Furthermore, not all benefitted because of additional challenges encountered during the distribution

phase, such as distribution without pin codes or other technical problems. In the final analysis the

actual number of families who benefitted from the Winter Cash programme was 24,601 (94 per

cent3) out of the 26,052 selected households. Therefore, 5.6 per cent of the targeted children and

their families were not reached.

This figure is considered as a no-show rate because families called for the distribution phase did not

show up after two rounds of notification SMSs, telephone calls and home visits. It is also probable

that some families did not go to the distribution centres to receive their ATM cards for reasons other

than what was reported by SDC staff (e.g. families involved in disputes, and individuals who

separated or divorced during the distribution process. The latter would lead to a case in which the

mother would have the “Hala” card while the father (the beneficiary name) could not receive the

ATM card without the “Hala” card). The evaluation found that 5.6 per cent, which represents

approximately 5,000 children and their families, could be accepted as a normal rate. However,

31


additional steps should be taken by WFP and NPTP to use more effective means of communication

with beneficiaries in future similar cash programmes.

Findings from the PDM process showed that the majority of social workers and field coordinators

were not aware of who would receive cash assistance and created unrealistic expectations amongst

possible recipient households. 13. This shows the weakness of the training provided by NPTP to the

social workers and field coordinators and highlights the need for quality capacity building activities

for implementing staff. This would help to ensure more effective communication and interaction with

the targeted families.

Some social workers in the North (including Akkar and Tripoli governorates) who interviewed

during the PDM14 also reported that members of the police force were interfering in the selection

process to help people that they knew, indicating favouritism and exclusion towards others15. Other

political and community considerations were also revealed.

The questionnaire used in the selection process

The evaluation found that the questionnaire used in the selection process was inadequate to identify

vulnerable families and their children.

As explained above, the winter cash programme relied on the Proxy Means Test (PMT) formula was

developed by the NPTP, PCM and the World Bank to target the most socio-economically vulnerable

Lebanese households with children to include them as a programme beneficiary. Then, MoSA social

workers updated the data of 35,000 most vulnerable identified households included in the NPTP

database and entered them into an Optical Marking Recognition (OMR) form. After updating the

assessment performed, the CMU ranked and selected the most vulnerable families corresponding to

the target of 75,000 children (with total 26,052 families) referring to the PMT formula.

Another concern raised by MoSA staff was the irrelevance of some of the selection criteria while

other crucial elements were missing completely from the questionnaire.

The questionnaire is linked to a PMT formula, including 60 variables identified by the data collection

team. Most of the MoSA staff, field coordinators and social workers who were interviewed reported

the following missing variables in the selection process:

Deceased husband

Children or persons with disability in the family

Unemployment rate of the family

The employment type and whether it is permanent or seasonal type

The income level(s) of the family

The illness history of the family- how many family members are suffering from chronic

diseases

The educational status of the family – how many children are studying

Whether the house is rented or owned by the family

The status of orphans in the family (due to a deceased husband)

13InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 9 14Ibid_ page 9 15Ibid _ page 10

32


It should be noted that the size of a beneficiary’s apartment or house is not considered as a robust

criterion since the house could be inherited.

The new questionnaire, administrated during the selection process, was used as a verification tool to

update some of the basic existing data included in the NPTP database. The existing data had been

used to issue the “Hala” card and included similar vulnerability and welfare variables. The NPTP

recognised some issues which had been reported in the interviews conducted during the evaluation

process and accordingly, they started to redevelop the PMT, questionnaires and other forms to ensure

an improved selection process. Most of the MoSA staff and the field staff indicated the importance of

involving and consulting them during the programme design and implementation phases and to take

into account their feedback on the selection criteria and the decisions taken. Their insights would be

useful 16as they work directly with the targeted families17.

Other selection criteria/variables

The selection process was mainly based on the Proxy Mean Test (PMT) formula to assess and

estimate the families’ consumption per day as a proxy means referring to some variables such as the

employment status, level of education, marital status, physical ability, housing conditions and

geographical locations. This led to a selection process based on data that was not updated with

regards to any significant change in some of the key variables such as the employment status, level of

income, housing conditions and other relative sub-variables as indicated above.

Social workers and field coordinators’ involvement in setting the selection criteria and

questionnaire

The evaluation found that the lack of involvement of the social workers, field coordinators and other

SDC staff in reviewing and checking the questionnaire greatly impeded the selection process and was

considered to be a major gap. All of the 20 social workers and field coordinators reported that they

were not informed about the selection criteria variables, except that the beneficiary family should

have children aged up to 15 years.

Although they received and reviewed the revised questionnaires, providing feedback and

highlighting certain issues, this was not taken into consideration. This inevitably has a negative

impact and created unfavourable relations between the targeted families and the SDCs – particularly

the social workers and field coordinators as they were unable to respond adequately to the families

who dissatisfied with their exclusion from the programme.

Although, the questions and answers mechanism was developed to respond to these issues, there was

a noticeable gap in the capacity of social workers and field staff to adequately respond. This, once

again, highlights the need to train personnel before programme implementation.

16Beneficiaries’ feedback document__Winter cash programme_ MoSA staff- Summary report page 4. 17Ibid_page 10

33


Question 3: To what extent has the programme been successful in identifying the most relevant

partners/actors for this project and what was the added value of these partnerships?

Partners’ identification

The evaluation found that UNICEF consistently selected appropriate and relevant partners for this

programme and it was evident that UNICEF had a clear vision regarding its partner selection.

UNICEF has been working with Government partners represented in NPTP/MoSA, and has also

been supplementing this work by collaborating with a series of key partners with considerable

expertise and strong commitment to the implementation of the Winter Cash programme such as

WFP, and the CMU/PCM.

Complementarity component among the different key partners

The National Poverty Targeting Programme was established in 2011, with the aim of supporting

vulnerable Lebanese families in meeting their needs. Although the NPTP works under MoSA, it has

its own separate, permanent structure as a semi-entity. For this reason, the Winter Cash programme

was designed in a collaborative manner between UNICEF, NPTP, MoSA), WFP, and CMU/PCM

(with separate roles for each). For example, MoSA and the CMU were the official entities

responsible for the provision of information and data on the most socio-economically vulnerable

families in Lebanon needed during the selection process.

The programme was successful in identifying the most relevant partners for implementation and each

of the five partners (UNICEF, MoSA, CMU, NPTP and WFP) played a significant role bringing their

own added value.

The NPTP is the official Government arm supporting underprivileged Lebanese families living in

harsh economic conditions (below the poverty line). Its importance has increased, especially due to

the ongoing Syrian conflict that has resulted in further deterioration of conditions for many Lebanese

families. Syrians willing to work for fewer wages than their Lebanese counterparts have gained

access to the labour market. This has undermined the situation of many Lebanese wage-workers,

threatening already vulnerable families with falling further into poverty.

In addition, WFP has a good reputation and strong outreach and technical expertise, particularly

among the group targeted by the Winter Cash programme where they implement e-food programmes.

Overall, the evaluation found that the selection of the partners was well-informed and successfully

boosted the trust of the beneficiaries in the programme.

Question 4: How gender sensitive was the programme when reaching the most vulnerable

children?

Based on the findings of the evaluation, the programme did not incorporate a gender perspective

when targeting the most vulnerable children. This is due to the fact that the selection criteria of the

families in the poor vulnerable families did not consider gender as one of the selection variables. The

age of the selected children (up to 15 years) was the key selection criterion. No sex-disaggregated

data for children who benefitted from the programme was available. The payment list of beneficiaries

of the Winter Cash programme included the sex of the ATM card holder without having

disaggregated data of the children supported per family.

34


Moreover, the gender criterion was not included in the questionnaire that was administrated to the

target groups during the selection process. As a result, little or no information was collected on the

number of female children who benefited from the programme. There was a noticeable gap in the

knowledge of when one should apply gender sensitive and equity measures. It is therefore

recommended that a gender component be included in any future programmes.

Question 5: Do the current log frame and/or results framework and relevant indicators in the

winter cash programme document need improvement to better serve the programme objectives

Programme proposal/concept note

A basic Concept Note on the 2015/2016 Winter Programme was developed and endorsed by the UNICEF

Country Management Team and shared with the NPTP management. The Lebanese component of the

UNICEF assisted winter programme was also included in the HRRP and Lebanon Crisis Response Plan.

At the start of the planning, a detailed action plan was developed by UNICEF and NPTP with the

partners.

The evaluation, however, found that a programme proposal, considered to be a fundamental

document, was missing from the programme. The programme proposal, plays a major role in

outlining the programme objectives, activities, outcomes, monitoring and evaluation (M&E)

mechanism, duration, target group and sustainability mechanism for the partners and other

stakeholders.

Providing this document at the start of the programme ensures that the role of each partner is well

explained and that the partners are aware of their responsibilities in the implementation. Despite the

absence of a concept note, the consultant managed to collect most of this information during the

inception phase of the evaluation and during the desk review of secondary data.

The Humanitarian Response and Resilience Plan (HRRP) 2016

The evaluation found that the HRRP 2016 lacks a separate log frame with specific objectives for the

Winter Cash programme targeting the poor Lebanese families, without reference to Syrian refugees.

This is considered to be a shortcoming in the design of the programme. To ensure efficient strategic

planning for the programme in the future, a well-structured log frame should be incorporated from

the very beginning, with a results framework which follows the results chain approach. There is no

clear documentation that shows the results framework as part of a complete Results-Based

Monitoring and Evaluation system or that includes an M&E matrix explaining the roles and

responsibilities for each stakeholder (who will do what, when, how and where).

However, the consultant extracted the key expected outcomes (results chain) from other documents

that combine two programmes, the first addressing and targeting Lebanese poor vulnerable families,

and the other addressing Syrian refugees. It is valuable to focus on the scope of work of the

programme component that addresses vulnerable Lebanese children and their families. Therefore, it

is critical to differentiate between dealing with and addressing the priorities and needs of refugees

versus tackling these same needs of the vulnerable poor Lebanese families as explained above.

This is important because for a programme that addresses refugees’ basic needs, evaluation criteria

must be followed (mainly the ALNAP criteria as part of the OECD-DAC criteria). This is different

from those used for programmes addressing the poorest Lebanese vulnerable groups (mainly DAC

35


criteria). The normal DAC criteria includes: Relevance, effectiveness, efficiency, sustainability and

impact, while the OECD-DAC criteria for humanitarian aid programmes are18:

Relevance/appropriateness, connectedness, coherence, coverage, efficiency, effectiveness and

impact. The TOR for the current evaluation includes a combination between the OECD/DAC criteria

with the absence of the connectedness, coherence, and impact while the current evaluation addressed

the coordination as a separate indicator.

The evaluation found that an operational plan was developed and updated on a bi-weekly basis.

However, it was revealed that there was a delay in distribution of ATM cards. This is not due to lack

of unclear timeliness but was because of the time allocated to the data collection by the field workers

and social workers in addition to the time spent in the long negotiation process with the SDCs to

define where the distribution will take place. However, one of the lessons learnt- considering that this

is pilot project- is that the estimated time needed for each of the different programme phases should

be re-visited for better time management in the future implementation of similar cash programmes.

4.2. Effectiveness

Question 6: To what extent were the vulnerable children and their families targeted by the

Winter Cash programme reached- (as planned)? How satisfied were they about the

programme?

Level of satisfaction among the direct beneficiaries

FGDs revealed that the majority of beneficiaries expressed a high degree of satisfaction with their

participation in the Winter Cash programme. These results were consistent with satisfaction shown

by beneficiaries who either experienced or observed changes in their sons, daughters or other family

members.

‘We really appreciate the efforts and aid provided by UNICEF through the cooperation of NPTP

via the “Hala” card which was our channel to benefit from the Winter Cash programme,” –

participants in the FGDs.

The evaluation indicates that the programme was very useful and that beneficiaries were satisfied

that it helped them secure some of their children’s and families’ basic needs such as food, clothes,

and medication.

Other FGD participants from the programme beneficiaries (particularly in Mount Lebanon and Bekaa

governorates) expressed how the cash transfer was exceptionally helpful and necessary given the fact

that it covered a large segment of the most vulnerable and disadvantaged families who were

registered in the NPTP. This was particularly true in the two above mentioned governorates where

scarcity of work opportunities, particularly in the winter season is a regular characteristic.

One of the rewarding observations that emerged during the FGDs was that the beneficiaries

associated the “Hala” card as the key reference for the Winter Cash programme rather than UNICEF.

This attitude corresponds with the key findings captured during the PDM19. The majority of

18Evaluating Humanitarian action using the OECD-DAC criteria- Overseas Development Institute- London, March 2006- page

20-21

36


beneficiaries discussed their appreciation of the cash assistance during the FGDs and how much the

support was necessary.

The adequacy of the cash transfer amount

The evaluation found that some of the beneficiaries were not satisfied with the value of the cash

grant, deeming it insufficient in meeting their children’s needs, while others suggested that it should

be distributed on a monthly or quarterly basis throughout the year. These responses correspond to

those collected during the PDM which showed that 42 per cent of beneficiaries believed the

assistance to be sufficient in covering their children’s winter needs, while the majority, 58 per cent20,

thought it was insufficient.

One of the interesting comments reported by some of the participants during the FGDs - especially in

Baalbeck was “I’m earning at least 20,000 LL on daily basis as I’m a taxi driver and that’s why

the amount allocated in the Winter Cash programme per child up to 15 years means nothing”.

Participants in all five FGDs indicated the need to either increase the amount of the cash grant, or

receive the assistance on a recurrent basis (i.e. quarterly).

Timing of the winter cash assistance

A delay in the implementation of the programme led to the distribution of ATM cards in March

instead of winter. This hindered achieving the main planned objective of the programme which is “to

provide the aid in the winter season to reduce the vulnerability of Lebanese poor families that is

exacerbated by the increase of expenditures and prices and is paralleled with a reduced flow of

income due to low availability of work opportunities during the winter”. This corresponds to the

results of the PDM survey that revealed 59 per cent21 of beneficiaries believed that the assistance

was performed in a timely manner, 24 per cent stated that it was a bit too late, 14 per cent stated that

it was quite late, and 3 per cent stated that it was really too late.

Families who have children with disabilities – Other vulnerable groups

Other beneficiaries, especially in Bekaa, Mount Lebanon and Akkar governorates expressed their

dissatisfaction as the programme did not take into account the special needs of families who have

children with disabilities. They emphasised that this should be included as one of the key selection

criteria, especially because these families incur higher expenses for additional care and medication.

Family members spend more caring for children with special needs, impeding their employment

opportunities and ability to earn a steady income.

Despite the fact that the number of children with disabilities in Lebanon is not documented, it is

highly recommended to consider this category as part of the key selection criteria for the Winter

Cash programme.

Working women and orphan children

The evaluation concluded that the Winter Cash programme did not give special consideration to

categories such as working women or orphans. Participants on the FGDs suggested that this should


37


be included as one of the key variables in the main vulnerability assessment selection criteria. This

variable was omitted from the questionnaire when selecting the most vulnerable families. In addition,

the number of employed family members who earn an income inside the family was also not

considered.

“We don’t know on what basis those who benefited from the Winter Cash programme were

selected,” – One FGD participant. .

Other vulnerable families- families with no Children

Other concerns were recorded which raised dissatisfaction over omitting families who do not have

children and who were excluded completely from the programme. Despite the fact that the Winter

Cash programme with the support of UNICEF Lebanon targets poor children and their families (in

reference to the mandate and scope of work of UNICEF), the FGD participants in addition to the

SDC staff pointed to the importance of considering them in the other similar future projects.

Other fairness aspects in the selection process

Some families who benefitted from the Winter Cash programme (especially in Akkar and Mount

Lebanon governorate) - received their ATM cards without a pin code. To address this issue, the

programme adopted a complaint mechanism to report these cases (with the support of the field

coordinators and social workers in the respective SDCs). However, these problems were not

resolved. Furthermore, these families did not benefit from the programme due to the weaknesses of

communication within the complaint mechanism - including delays in response. The programme

ended before these issues were resolved.

“What is the mistake I made that deprived my children from the Winter Cash programme like

the rest of the children in my community/neighbourhood? “ - Two participants of the FGDs in

Akkar governorate.

Distribution mechanism – Organisational issues

The distribution mechanism was considered as one of the main challenges of the Winter Cash

programme as reported by some of the participants, especially those living far from the SDC selected

as a distribution point. This corresponds with the results of the PDM process that showed: 69 per

cent stated that it was too far from their place of residence, 14 per cent stated that it was unsafe, 13

per cent waited too long, 3 per cent stated that it was not accessible to elderly, pregnant women, or

disabled individuals, and 1 per cent stated that they had to go to the site several times22.

During the current evaluation, some of the FGD participants reported that additional transportation

fees were incurred by families living far from the distribution centre. The following examples were

captured:

In Akkar Governorate, only three SDCs were selected to distribute the ATM cards

The number of ATM cards distributed in Rahba SDC covered beneficiaries from five SDCs -

Bebnin (1,534 ATM cards), El Bireh (551 ATM cards), Halba (1,025 ATM cards), Qoubaiyet

22Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 39

38


(269 ATM cards) and Rahba (465 ATM cards) totalling 3,844 ATM cards. The distance

between Rahba SDC and Qoubaiyat SDC is at least one-hour drive and a family travelling

between the two would incur additional transportation fees of approximately 20,000 LL

(USD 13.50). The same was found in Bekaa governorate (Baalbeck SDC).

“If I have only one child up to 15 years old, I would receive USD40 This means that I have to

spend 35 per cent of this amount before coming to receive the ATM card,” - Participants of the

FGDs commented in both Akkar and Baalbek.

These comments corresponded with the results of the PDM process. The results of the survey

conducted during the PDM23 show that 89 per cent of participants confirmed that the distribution site

was satisfactory in comparison to 11 per cent who found it to be unsatisfactory. With respect to why

they felt dissatisfied, the PDM results showed that 78 per cent24 indicated that the organisation of the

distribution process was weak. The related issues were reported by the beneficiaries interviewed

during the PDM process as follows:

The distance between beneficiaries’ homes and their assigned centres was an issue (cost of

the trip).

The distribution was cumbersome, often disorganised and extremely exhausting for social

workers.

Some centres were overloaded and could not cope with the large number of beneficiaries

coming every day. Staff had to deal with problems on the spot and be reactive rather than

proactive25.

The evaluation considers this issue as one of the gaps in the coordination among different actors and

a drawback negatively impacting the cost efficiency of the programme.

SMS

Another coordination issue that was raised related to UNICEF’s role in informing the eligible

families about the ATM cards. The evaluation found that many of the selected families did not open

the SMSs that were sent to them, which notified them about the distribution of the cards. The reasons

behind this were that they were either illiterate and could not read the SMS, they thought that the

message was an advertisement, or because the models of their phone could not read Arabic messages.

This corresponds with the findings of the PDM process as reported by MoSA staff26.

Question 7: To what extent did the programme have an impact on beneficiaries in improving

access to essential goods and services of their choice in a safe, dignified, and empowered

manner while decreasing socio-economic vulnerability?: Key changes affecting the lives of

beneficiaries as result of the programme.

23Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 40 24Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 41 25 Beneficiaries’ feedback document__Winter Cash programme_ MoSA staff- Summary report page 4 26InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 14

39


Key changes

The programme succeeded in serving 75,000 children from the most socio-economically vulnerable

families in Lebanon and provided each child, aged 0-15 years with a USD40 cash grant to help

secure food, clothing and other winter basic needs. Those beneficiaries who were interviewed

expressed their satisfaction with the programme, stating that the money came exactly on time as they

were in a serious need of extra support due to their economic conditions and lack of employment

opportunities during the winter season. The evaluation revealed that the majority of the beneficiaries

spent most of the cash transfer on medical care, education and food followed by heating oil, winter

clothes27, rental fees and repayment of some loans. The cash grant gave beneficiaries the freedom to

spend their money as they needed and according to their own priorities in a safe, dignified and

empowered manner. Nevertheless, although the grant helped families to secure basic needs for their

children, it cannot be confirmed that the USD 40 managed to invoke key changes, or significantly

impact the lives of the beneficiaries, other than fulfilling a serious need for some basic services

which they could not afford due to their poverty.

On the other hand, some beneficiaries felt that the amount was insufficient for securing their basic

needs, especially as the price of food and clothing in the winter tends to increase. In addition, others

stressed that the assistance should be provided on a continual basis and if it was merely a one-time

grant then no significant change would be incurred. It was also reported that because of the delay in

delivering the ATM cards, after the winter season, some families used the cash grant to cover other

basic needs.

Spending patterns

The current evaluation revealed that the majority of beneficiaries spent the cash amount on food and

rent, namely in Mount Lebanon and Tripoli, and on winter clothing especially in areas like Baalbek

and parts of Tripoli. A minority of beneficiaries reported that they spent the cash assistance on

medical expenses - in Mount Lebanon, Akkar and Baalbek - to address chronic cases of illness within

the family. Finally, few beneficiaries reported that they spent the cash support on tution fees and

purchasing fuel for heating purposes in the winter season in addition to other purposes.

“The cash transfer I received through the Winter Cash programme came on time. It protected

the health of two of my sons and saved the life of my husband as I used all the amount to buy

the medicines they need,” Participant in a FGD.

This confirms the results captured in the PDM survey. The results showed that the five most common

expenditures 28 among the beneficiaries interviewed were as follows: Winter Household expenses

(56 per cent), food (55 per cent), education fees-including the transporation fees to schools (30 per

cent), repaying debts (20 per cent ), health expenses (20 per cent), electercity bills (12 per cent),

heating- fuel or others (11 per cent) and rent (11 per cent).

Question 8: Are there other unintended or unforeseen changes shown in the lives of the

targeted groups of poor children & their families? Also, are there any unintended beneficiaries

in the programme?

27InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 28-29. 28Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 69

40


Although, the Winter Cash programme was designed to secure the basic winter needs of poor

vulnerable children and their families, the unrestricted cash transfer was used as an added value to

cover other expenditures. The below chart # 7 shows how the unrestricted cash transfer was used to

cover other needs including education, health, rent and others. The spending patterns revealed that

the cash grant was used only for the basic needs of winter. Participants indicated that the cash

assistance had a positive effect on the child’s schooling, through positive impacts29 such as

purchasing clothes for school (29 per cent), paying transportation fees (27 per cent), paying school

fees (22 per cent), purchasing stationery (20 per cent), and purchasing books (16 per cent). However,

the chart below shows how the programme, by adopting the unconditioned cash transfer modality,

helped in securing other spending aspects as a kind of unintended change.

Moreover, the unrestricted cash transfer assistance in the current Winter Cash programme that was

designed to secure the winter basic needs for the vulnerable children up to 15 years and their families

was helpful in securing the winter and other basic needs for the other family members.

Table 7. Summary of the spending and expenditures patterns by order of priority30

Question 9: What was the level of acceptance of the cash modality as a tool to support poor

Lebanese households to cover their winter needs (winterisation) from the points of view of all

the different actors: MoSA, NPTP, SDCs, field staff and the beneficiaries?


41


The level of acceptance of the unrestricted cash transfer

Cash support is considered unrestricted and this form of assistance was praised by the majority of

respondents as their preferred means of support. The same attitude was also expressed by MoSA

staff, who felt that this was a more appropriate wayto help those in need and allow them to address

their own priorities. This opinionwas applicable in all of the groups, with no exception.

“It allowed me to prioritise and buy what the family needed”, said one man in Baalbek.

Men in Halba also agreed that the needs vary from one household to another, and that while one

might need clothes, others might need food products.

“With the money you can always purchase anything that you need”, said one man in Mount

Lebanon.

“We can always secure food, but how can we go to the doctor’s if we do not have money?” commented a woman from Mount Lebanon.

However, others pointed to the need for other types of assistance. Men in Mount Lebanon agreed

that the e-food voucher was also a great tool that could secure food and beverages on a monthly basis

and that other types of support were also helpful provided that they were appropriate and needed. A

man from Tripoli said that if the cash assistance was only going to reach them once, then it would

make no difference.

The findings of the evaluation show that opinions vary with regards to unrestricted cash transfer and

restricted cash transfer.

Partners from WFP, NPTP, and MoSA, as well as six SDC directors, five field coordinators and 14

social workers who were interviewed during the evaluation reported their preference for restricted

cash transfers (in the form of e-vouchers). They indicated that if it was not possible to implement the

unrestricted cash assistance on a recurrent basis, then it would be better to provide e-vouchers for

food, fuel and clothing. During the FGDs participants commented that the one-off winter cash grant

of USD40 per child once a year was not enough to secure their basic needs. They also commented

that since the cash grant was at the end of the winter season, it was instead used to purchase other

essentials and was still not adequate to meet all basic needs. These participants recommended that the

cash assistance be provided on a recurrent basis.

On the other hand, the majority of the direct beneficiaries and a few social workers and field

coordinators who were interviewed expressed their preference for the unrestricted cash transfer. This

corresponds with the results of the PDM survey showing that the majority of beneficiaries are

satisfied with the cash modality (63 per cent)31 compared to those who are very satisfied (32 per cent)

and those who are not satisfied (5pe cent). The MoSA staff and direct beneficiaries interviewed

during the PDM indicated that the unrestricted cash modality was a better tool, to allow beneficiaries

spend according to their personal needs.

During the PDM, other participants indicated the need for other types of support and expressed

criticism of the “Hala card” which they believe was not useful in accessing medical care32

particularly to cover medication fees.

31Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 72 32InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 31

42


However, some of the programme’s actors have gone for the restricted cash modality (vouchers for

food, clothes, medication or fuel) in order to ensure the delivery of the assistance to its target group,

without leaving a space for the irrationality of some parents who might misuse the money and spend

it on inappropriate items, and consequently, hinder the delivery of the money to the children. The

results show that the ones who preferred other kinds of modalities mainly chose vouchers (59%), in

kind (30%), and fee waiver (11%) as reported by the FGDs and survey respondents targeted in the

PDM process33.This indicates their preference for restricted cash modality (e- voucher for food, fuel,

clothes, etc.) in the case of not having the possibility to receive the cash transfer assistance in a

recurrent way because of the highest guarantee in benefiting vulnerable children and avoiding some

of the bad practices of misuse of the cash transfer amount by few parents as reported earlier in this

report and as summarized below.

For more details on the advantages of using the unrestricted/or the restricted cash transfer as

captured and collected from the interviews with the different stakeholders and FGDs with the

programme beneficiaries, please refer to annex 8 attached to this report.

Six social workers in addition to the majority of direct beneficiaries interviewed during the

evaluation showed great acceptance of the cash modality as a support tool, giving families the

freedom of choice and power to purchase the goods and services they need according to their own

priorities.

Combined modalities recommended by the beneficiaries

Some suggested combining the in-kind and the cash modality so as to ensure that the children and

families receive the in-kind assistance -while not depriving them from receiving cash to choose and

purchase what they need according to their personal priorities. Some commented that the cash

transfer assistance is the best cash modality, but on one condition -that it is received regularly on a

quarterly or monthly basis.

One interesting observation emerging from the FGDs in Mount Lebanon was that the husband

preferred the restricted cash transfer, especially the e-food voucher on a regular basis, while the wife

favoured the unrestricted cash transfer as the preferred cash modality, allowing them to set and

prioritise their own needs.

Some participants in FGDs, in Baalbek and Mount Lebanon, highly recommended introducing the e-

health voucher where families can receive medical services especially medication in case of its

unavailability within the health section in the SDC. In this manner, the parent/caregiver can continue

to work and earn money while he/she remains in a good health.

Finally, the evaluation found that introducing a multi-dimensional cash modality based on accurate

and updated data using a well-structured vulnerability assessment is highly recommended.

Question 10: To what extent were the capacities of the different actors involved, particularly

the NPTP and MoSA, adequate and efficient in ensuring the quality of the programme

implementation? What can be improved in this regard?


43


Although NPTP and MoSA have a large number of social workers in the field, their skills need to be

improved and strengthened in order to deliver quality programme activities. Many do not possess a

scientific degree in social services and sciences creating a gap in skills and knowledge. As previously

discussed, there is a need to strengthen and enhance their capacities so as to promote programme

ownership of the NPTP and establish it as a model for replication and scale-up in the future. In

addition, the number of senior NPTP and MoSA staff involved in the programme was limited

compared to the number of social workers and the number of beneficiaries. The evaluation concludes

that investments should be made to improve the skills of field coordinators and social workers in

order to deliver a quality programme.

Capacity building – What is needed?

The following areas in need of capacity building are those specified by key partners and

stakeholders:

Communication skills - to improve interaction with the other stakeholders in the programme

ensuring enhanced cooperation.

Social work skills - including case management skills to enable social workers to

professionally interact with the beneficiaries during the programme implementation phase.

Field work skills - including field visits and working with vulnerable groups. Research ethics

- to ensure credibility, privacy and accuracy when distributing questionnaires to targeted

families.

Data collection methodologies and tools - including surveys, questionnaires, FGDs and

interviews.

Data reliability, validity, data integrity.

Knowledge of child rights - to enable a better understanding of the linkages between the

impact of the programme and the rights of the child. This will be helpful in the design of

other questionnaires/assessment forms that include different indicators.

Basic monitoring skills

The recommendations listed above could be organised in 4-5 working days to ensure basic

knowledge of each area and that all social workers and field coordinators have the same level of

understanding of the scope of work.

Other specific and structured short training packages should be provided for other MoSA staff and

partners involved in the programme.

Question 11: To what extent does the programme have an effective monitoring & evaluation

mechanism (including reporting, quality assurance and reflection)? Are there proper processes

on different levels (UNICEF, NPTP, CMU/PMC) in place and how has the programme used

information generated to inform (make) programmatic adjustments and corrective actions

during the life of the project?

Monitoring and Evaluation (M&E)Mechanisms

The Winter Cash programme has specific reporting and monitoring activities:

UNICEF Monitoring & Reporting component including FGDs, third monitoring

process and UNICEF programme monitoring activities.

44


Partner reporting component including Government reports, quarterly and final

narrative reports and Activity info reporting.

The evaluation found that UNICEF, with the support of key partners, developed a well-structured

system to track the distribution cohorts, delivery system process, and the questions and answers

mechanism, in addition to the complaint mechanism, which was guided and led by WFP. There are

86,000 families registered in the NPTP and other vulnerable lists of the poorest vulnerable families in

the current Winter Cash programme who were selected using the PMT formula with technical

support from the CMU/PCM (guided by the questionnaire and associated forms).

One of the notable M&E achievements was the PDM process which included FGDs with direct

beneficiaries, and key informant interviews conducted with MoSA staff- including the social

workers, field coordinators and the SDC directors. Additionally, a survey was conducted with

beneficiaries using the computer assisted telephone and face to face interviews.

The results framework of the Winter Cash programme referred mainly to the HRRP 2016 where

expected outcomes were identified with planned activities under each objective. The evaluation

recommended that a separate results framework be used, instead of combining it with other activities

addressing the winter basis needs of both vulnerable Lebanese and refugees in Lebanon.

Risk and assumption analysis

Considering the nature of the Winter Cash programme, some of the challenges encountered during

the implementation phase were as follows:

As noted before, 5.6 per cent (approximately 5,000 HHs) of the selected families was not

reached during the distribution phase after two rounds of SMS, telephone calls and field

visits. This highlights the importance of conducting a simple risk analysis and assumption

component as part of the M&E system during the programme design phase.

The evaluation revealed that the reason beneficiaries who received ATM cards, but had

issues with their pin code or other technical issues was due to the lack of an effective

monitoring and follow-up mechanism to track the communication between WFP and the

bank when responding to the complaint mechanism cases reported.

45


Table8. Monitoring and reporting mechanism

Post distribution monitoring activities

There is a lack of monitoring and post follow-up activities by social workers and field coordinators

that assess changes in the lives of the beneficiaries as a result of their participation in the programme

as well as information on the spending patterns of the cash grants. Based on the absence of a proper

follow-up mechanism, families were not provided with sufficient guidance on the proper use of these

grants to secure the basic needs of their children during the winter season.

Involvement of field coordinators and social workers in M&E activities

The current evaluation highlights a need for the NPTP to enhance the role of social workers and field

coordinators in setting and developing the selection criteria and respective questionnaires and forms

in addition to considering their feedback in a meaningful manner. This corresponded with the results

found in the PDM.

There is an urgent need to train field staff, especially field workers and social workers, on the key

monitoring skills for tracking changes during the life of the programme as a community M&E team.

In this way, the database of targeted families would be updated on a bi-annual or annual basis.

Moreover, they should be involved in the development of the vulnerability assessment process that

includes all the vulnerability indicators.

46


Question 12: To what extent did the programme consider gender equality in the programme

implementation – including the selection process (e.g. is USD40 an equitable amount for both

boys and girls? On what basis was it calculated and does it take into account gender-based

requirements?)

Gender and selection process

As indicated previously the Winter Cash programme did not specify gender as a variable of the

selection criteria of poor vulnerable families. The Proxy Mean Test formula was used instead to

estimate a family’s consumption per day by referring to other variables such as the employment

status, level of education, marital status, physical ability, housing conditions and geographical

location. Both boys and girls aged 0-15 years were eligible for the cash grant.

Additionally, the evaluation demonstrated the absence of sex disaggregated data among the children

who benefited from the programme. The final payment transfer list includes sex disaggregated data

for the ATM card holder, but not for the beneficiary children.

Following the evaluation, it is highly recommended that the NPTP includes a gender sensitive

component in all cash transfer programmes and that gender be considered not only in the programme

design and planning, but also during the programme implementation phase with special emphasis on

sex disaggregated data.

Gender sensitivity and the adequacy of the cash transfer amount

Regarding the adequacy of the cash transfer amount of USD40, for covering and securing basic

winter needs, the majority of the participants interviewed agreed that there was no significant

difference between the cost of winter clothes items for boys and for girls. They reported that the main

winter item bought in response to the seasonal hazards and unexpected winter emergencies was an

outer winter jacket which can be worn by both male and female children and which can be

exchanged between the two genders.

Gender sensitivity and spending patterns

Interestingly, there were divergent views between men and women in terms of the preferred cash

modality. Direct male beneficiaries, particularly in Mount Lebanon, preferred the restricted cash

transfer especially the e-food voucher on regular basis, while female beneficiaries favoured the

unrestricted cash transfer to set and prioritise their needs. This corresponded to the results of the

PDM process34.

The female participants (parents/caregivers) who took part in the FGDs expressed that they shared

each other’s’ experiences and views that they were the decision makers in terms of the family’s

needs and priorities. It could be concluded that women preferred the unrestricted cash modality as it

was a source of empowerment and engagement in decision-making activities at home. Restricted

cash transfers would isolate a woman from any decision-making process.

34InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 31

47


4.3. Efficiency

Question 13: To what extent was the Winter Cash programme cost-efficient – including

optimal use of the programme resources and time efficiency as opposed to other

modalities/programmes that could be designed to cover the same needs of the Lebanese

vulnerable poor children and their families?

Based on the evaluation, there was a good selection of partners and it was considered as a good

example of a partnerships model. Each of the key partners and actors were well-informed, their main

roles and responsibilities were agreed upon in advance, and specific measures were set to maintain

cost-efficiency during the different phases of the winter cash programme. This model ensures that the

minimum requirements can be realized.

As explained previously in the report, the winter cash programme was designed to provide monetized

assistance to 75,000 children all across Lebanon through a one-off winter grant of USD 40 per child.

Since the programme includes only one activity targeting poor vulnerable Lebanese children and

their families, Table 9 shows the cost transfer per child/family:

Total amount of cash transferred to the bank

Planned

Transferred

Actual delivered

USD

3,000,360

USD

2,800,000

Planned Actual

1 Total number of programme beneficiaries - HHs 26,052 24,601

2 Total number of vulnerable beneficiaries of children 75,009 70,000

3 The average number of children/family 2,87 2.84

4 The average cost per family(HH) USD 115.10 USD 113.80

5 The average cost per child USD 40 USD 40

Table9. Cost transfer per child/family

Cost efficiency and the cash modality

As indicated previously, 88 per cent of the surveyed respondents (programme beneficiaries)35 as well

as the majority of the FGD participants stated their preference for the unrestricted cash transfer

indicating that in their opinion it was more cost-efficient than the restricted cash transfer (e-vouchers)

especially in the absence of WFP as an effective implementing partner.

Cost efficiency and programme outreach

Despite the fact that the majority of the participants during the FGDs and other key partners indicated

that it would be better to target all the families (86,000 HHs) registered in the NPTP, even with an

amount less than USD 40 per child, this would not acceptable in most cases because most, if not all

of the families already have e-cards from WFP and other organisations in addition to the winter cash


48


grant. This would therefore not be the most cost effective way of reaching vulnerable families.

Offering an amount less than USD 40 might also result in additional problems, since Palestine and

Syrian refugees receive USD 40 per child up to 15 years and could lead to unnecessary competition

and increased tensions between vulnerable groups.

The most prevalent response from beneficiaries regarding the cash grant was the comment that they

felt more respected and treated “like human beings”. “The card is a more respectful tool rather than

having cash handed out to us,” said the SDC director of Baalbek. However, beneficiaries

recommended better management of the distribution process, advising to minimise the flow of people

so that huge numbers do not show up on the same day at the same distribution centre. Respect for

human dignity implies commitment to creating conditions under which individuals can develop a sense of self-worth and security in such situations36.

Cost efficiency and the selection process

It is highly recommended to allocate additional resources for the design of a well-structured,

comprehensive vulnerability assessment which includes all those registered in the NPTP and which

covers all vulnerabilities and welfare variables. This would reduce the risk of selecting those families

who are not in need of the grant and would ensure optimal use of resources.

Cost efficiency and capacity building

It is strongly recommended to provide capacity building for implementing stakeholders - especially

field coordinators, social workers and other MoSA staff. Developing skills in the areas of

communication, interaction, reporting, monitoring, and field work in addition to child rights and

research ethics is most necessary. By building capacity of these stakeholders, implementation of the

programme would be more efficient, reducing errors and leading to greater cost efficiency.

Cost efficiency and the cash transfer process

Collaborating directly with Banque Libano-Francaise enabled beneficiaries to directly withdraw the

cash grant using an ATM card thus reducing any third party costs. This not only ensured the security

of resources, but also reduced the risk of wasted resources during the implementation process,

excluding the need for procurement operations.

Cost and time efficiency

The evaluation found that the verification phase took much longer than planned, leading to a

significant delay in the distribution of ATM cards. Issues included cash withdrawal errors, lack of a

functioning complaint mechanism and lack of coordination and efficient communication between the

bank and implementing staff. The objective of the programme required timely delivery of cash

support to secure winter basic needs. This delay undermined the quality of programme

implementation and beneficiary satisfaction.

Cost effectiveness and value for money

There was a surplus amount of approximately USD 200, 360, reported by the MoSA focal point in

the NPTP for the Winter Cash programme. However, the surplus was a combination of an

36http://hdr.undp.org/en/content/human-development-and-human-dignity- Human development and human dignity- By Aung San

SuuKyi, Winner of the Nobel Peace Prize, 1991

http://hdr.undp.org/en/content/human-development-and-human-dignity-

49


overestimation of the WFP budget and the funds corresponding to the no-show cases of

approximately 5.6 per cent (5,000 children). Greater attention should be given to ensure the full

amount is allocated to the selected vulnerable families as planned or a selection of other eligible

families is conducted in order to reduce the surplus amounts, especially considering the large number

of poor vulnerable Lebanese families.

Due to the large number of beneficiaries visiting the SDC distribution centres, it was considered

impossible to organise the distribution process in a sufficient manner to reach all families with ATM

cards in one day. Additional transportation fees were incurred by families living far from the

distribution centres, which in some cases cost 33 per cent (20,000 LL) of the cash grant. In this

regard, some believed that it would be better to have 50,000 LL (less than the grant amount of USD

40) instead of withdrawing part of this amount and leaving the remainder as some of the other

families had commented. 37.

Some of the key partners (particularly WFP staff) interviewed during the evaluation indicated that

setting up an ATM-card or e-card/voucher system is too costly to for it be cost efficient for a one–off

transfer. Card issuance fees are between USD 4-5 per card in Lebanon in addition to other bank

charges and operational costs. This further confirms the need to maximise the coordination of NPTP,

Government and other actors serving the same vulnerable groups, to consolidate the vulnerability

needs of the beneficiaries based on the results of the vulnerability assessment and (as recommended

above) to ensure that the minimum requirements are taken into consideration and measures are set for

the provision of cash for greater cost-efficiency.

Cost efficiency and the distribution mechanism

As indicated earlier in this report, the distribution mechanism was considered one of the key

challenges in the Winter Cash programme, negatively impacting the cost effectiveness and value for

money of the cash transfer amounts allocated and invested in this programme.

Cost efficiency and risk analysis

The programme design lacked a risk analysis component, which is considered critical for any type of

resources transfer process. Risk analysis is a method that is used to guide the development of

standard operating procedures for cash transfer including threat analysis and acceptable risk

threshold. One relevant example which emerged from the evaluation was the illiteracy of some

beneficiaries who could not use the ATM cards, especially in Akkar and other areas. This

observation confirms that such challenges were not addressed during the trainings provided to social

workers and coordinators before the implementation of the programme.

Finally, there are both advantages and disadvantages to restricted and unrestricted cash assistance.

Unrestricted support shows that distribution is more timely and cost effective, whereas restricted

assistance incurs other logistic activities with additional costs.

37 Beneficiaries’ feedback document__Winter cash programme_ MoSA staff- Summary report page 4

50


4.4. Sustainability

Question 14: To what extent did the programme equip service providers (NPTP/MoSA) and

beneficiaries with the needed capacities (skills and knowledge) (Human resources'

sustainability), resources (connections) and confidence (Institutional and financial

sustainability) to sustain the programme implementation?

Within the framework of the Winter Cash programme, the NPTP social workers were trained on the

distribution of ATM cards. However, the programme failed to provide well-designed capacity

building training on essential topics, such as: communication skills, teamwork, social work, research

ethics, data collection, data reliability and validity, data analysis, child rights, and monitoring skills.

Without these skills, the programme is not sustainable. It is highly advised to raise the capacities of

the NPTP and MoSA workers to ensure full programme ownership by the NPTP. The aim is to

strengthen capacity and empower personnel in such a way that they can carry out similar future

programmes on their own without receiving technical assistance from partners such as WFP or

UNICEF. The Winter Cash Programme, as a pilot, shed light on the essential capacities if a more

holistic programme is rolled out in the future.

Human resources and institutional sustainability

The evaluation affirms that the capacity building component promoting human resource

sustainability is the most appropriate mechanism in the medium and long-term and would also

contribute in the bolstering institutional sustainability. However, a capacity building approach

requires significant investment in order to provide on-going training for partners in order to develop a

team of field coordinators and social workers who can deliver effective policy making, programming

and application of cash transfer support to poor vulnerable families. Well-trained social workers and

field coordinators optimise key human resources not only within the programme, but also would

enable them to play a vital role as a community monitoring & evaluation team.

Considering the winter cash grant is a one-off cash transfer there is a difficulty in developing

sustainability measures. However, other scaling up mechanisms should be developed on the

institutional and human resources levels (as indicated above).

Although the scope of work of the Winter Cash programme focused on providing monetised

assistance through a one-off winter cash grant of USD 40 per child to secure their basic winter needs

and to help mitigate the effects of seasonal hazards and unexpected emergencies, the evaluation

found that there was an absence of a sustainability strategy to address the graduation approach in

some of the targeted areas – to raise the skills of the targeted groups and helping them to secure

employment opportunities. The evaluation regards this aspect as one of the fundraising mechanisms

to promote financial sustainability where the budget can be invested and allocated to serve other

vulnerable groups.

The programme provided beneficiaries with training on how to use the ATM cards to facilitate

withdrawal of the cash grant. Nevertheless, illiteracy among some beneficiaries- especially in the

Akkar governorate - impacted negatively on the result of the training, and they continued to have

difficulties using the cards. This corresponded with the results of the PDM survey which showed one

51


of the main reasons for not finding information on the card was illiteracy (16 per cent of interviewed

beneficiaries)38.

4.5. Coordination

Question 15: How effective was the management and coordination system - as part of the joint

programme partnership among UNICEF, NPTP, MoSA, CMU/PCM and WFP- in delivering

the set results to the target population?

Coordination and complementarity among partners

Given the fact that the Winter Cash programme is a pilot project, the evaluation found that the

programme succeeded in identifying and selecting a genuine and strong model of partnership

combining Government entities represented by the NPTP, and the CMU under the Presidency of

Council of Ministries (PCM) in addition to other UN organisations such as UNICEF and WFP. In

this regard, the roles and responsibilities for each partner were more or less defined clearly and

contributed to the realisation of complementarity principles in this joint partnership. Overall, partners

and other different actors had smooth communication between one another throughout the lifecycle

of the programme. However, there was a lack of a well-designed management and coordination

system, particularly with respect to clear identification of the roles and responsibilities of each

partner.

Steering committee

Regular meetings were organised on a weekly basis by representatives from the five partners to

follow-up on the implementation of the programme, ensure that problems were solved in a timely

manner, and maintain effective communication between the implementing staff. The steering

committee also served as a reference for the implementing staff whenever they needed advice,

assistance, or certain exceptional interventions to solve problems or deal with challenges

encountered.

Communication mechanisms

1. SMS

One area of concern was the role of UNICEF in informing the selected eligible families about

the distribution of the ATM cards. The evaluation found that many of the selected families

did not open the SMSs that were sent to them notifying them of the distribution of the cards.

The reasons for this were: they were either illiterate or unable to read the SMS; they thought

that the message was an advertisement; or because their phones could not read Arabic

messages. This corresponded with the findings of the PDM process as reported by the MoSA

staff39.

This is considered one of the mitigation measures taken by UNICEF to ensure the highest attendance

rate of beneficiaries during the distribution process. Consequently, social workers telephoned the

families who were not present at the distribution centres, and visited those who did not have

38Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 29 39InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 14

52


telephones to inform them about the distribution of the cards. This corresponded with the results of

the PDM survey, which showed that 92 per cent40 of individuals received an SMS to collect their

cards compared to 8 per cent who did not.

Although the SMS was very well received by the respondents, many of the MoSA staff disapproved,

as reported in the FGDs facilitated in the PDM41, due to a variety of factors among which was the

belief that many beneficiaries did not read the SMSs or understand them. Additionally, in some

cases, information brochures arrived late and SMSs were delayed. On the other hand, the entire

process had a very short timeframe and that also affected the delivery of information and hindered

the opportunity of creating awareness about the programme in advance as reported by the majority of

beneficiaries in the FGDs facilitated in the PDM process.42

2. Brochures and flyers:

The MoSA staff interviewed in the PDM process indicated that they did not fully approve the

brochures and flyers distributed to beneficiaries for reasons including their late inclusion in this

process; as well as some not being involved in the distribution process until much later thus limiting

their inputs. Other reasons included the illiteracy of many beneficiaries which led to a lack of

interest in the brochure and flyer. This created a feeling of disappointment and lack of trust among

many of the community members towards the coordinators since they expected to benefit from the

programme through the flyers and visits43.

3. Complaint mechanism/Hotline:

The MoSA staff interviewed in the PDM process indicated that many of the beneficiaries had called

the complaint mechanism/hotline phone numbers several times and had to pay money for the

communication, without solving their problems44. Also, many participants reported that they would

have preferred one to one interactions or reaching out to the centres and people they knew better for

help45.

Moreover, the current evaluation observed the absence of following some of the coordination steps

which should have been taken, as well as lack of activation of the complaint mechanism in a timely

manner; this resulted in some families receiving cash transfers that were less than the planned. In this

regard, there were problems in terms of withdrawing the entire amount of money, as beneficiaries

were notified to withdraw the cash in USD 20 notes, and yet these were not available at all ATMs

due to the variation of the denominations from one machine to another. One of the reasons behind

this was the absence of daily or even weekly updates of the bank account balances, demonstrating a

vital communication obstacle with the bank.

Additionally, one of the participants in the FGDs indicated that not all villages and areas of residence

had ATMs close by, which meant that beneficiaries had to travel to the ATMs, sometimes for long

distances, spending a significant amount on transportation fees..

40Statistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 16 41InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 14 42 Beneficiaries’ feedback document__Winter cash programme_ MoSA staff- Summary report page 4 43InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 14 44Ibid_page 15 45Ibid_page 24

53


The evaluation found that the complaint mechanism was one of the gaps identified under the

coordination component. In practice, very few issues (such as the absence of pin codes , or the

incorrect cash amount or other technical ATM related issues) were addressed or resolved when

following the existing complaint mechanism with delays experienced by many.

This observation corresponded with the results of the PDM findings. According to the results, out of

those who confirmed that their case was solved (56 per cent), 60 per cent 46 of them verified that the

processing of their complaint was done in a timely manner, while 40 per cent stated that the process

was not time efficient. For those who were informed about the problem, the majority confirmed (82

per cent) that the interlocutor took their claim into consideration, while the remaining (18 per cent)

did not.

Coordination and distribution process

The current evaluation revealed that the geographical distribution of centres selected as distribution

points was considered a key challenge in terms of good communication. This was associated with

lengthy and difficult travel conditions for many beneficiaries. The distance between many of the

beneficiaries’ homes and their assigned centres was too long, which made the trip tedious and costly.

This resulted in many not attending the distribution of ATM cards, particularly those who had only

one child and others who went to the centres and spent time, effort and money on the trip and who

were dissatisfied and confused by the programme.

Moreover, the operation of distribution at the centres was a cumbersome process, often disorganised

and extremely exhausting for social workers. Some centres were often overloaded and could not cope

with the large number of beneficiaries coming every day. Therefore, it is recommended- as reported

in the current evaluation process and in the PDM process - that other mechanisms be devised either

for wider distribution of all the SDCs or covering the transportation fees for those for whom it has

been proved that they are too far from the distribution centre.

Coordination and follow-up meetings

Weekly meetings were held among the key partners through the steering committee to manage and

steer the programme activities in a timely manner. A similar meeting should be organised on a

regular basis between the WFP and the bank to get ensure cash transfer balances of the bank accounts

of the ATM holders. This would enable those responsible for the complaints mechanism to track

complaints concerning any of the ATM related issues. It is worth noting that the WFP staff

interviewed in the current evaluation indicated that specific agreements were consequently made

with the bank to ensure updates are received on these balances for any future cash transfer

programmes.


54


5. Conclusion

Overall, UNICEF and its key partners have successfully achieved the outcomes anticipated for the

Winter Cash programme. There was a high level of satisfaction among the direct beneficiaries

commenting that the assistance came at a right time considering the challenges that are being

encountered to find job opportunities during winter in some regions. However, complaints

highlighted the insufficiency of the cash amount. On the other hand, the ATM card was perceived as

tool that made them feel proud, free, respected, safe and valued not to have to wait in line and beg

for support as happens with other modalities. However, some beneficiaries commented that they

were unable to withdraw the full amount due to the different denominations. Others who could not

use the ATM cards sought help from their neighbours, children and family members.

Both MoSA staff and the direct beneficiaries perceived the selection process as unfair and having left

many of those in need with no support, while some who were very less vulnerable received the

assistance. Additionally, the selection process was not well understood by social workers and field

coordinators who were not aware of the selection criteria. They expressed their willingness to take

part in the selection process and the added value their knowledge of communities and vulnerable

families would bring to the process. Many believed that the selection process should be more based

on more accurate information and closer observation of each household’s needs and priorities.

The flier and SMS, used as communication tools, were criticized as being insufficient in meeting the

needs of the beneficiaries. Illiteracy among vulnerable groups and lack of feedback from the

complaint mechanism led to dissatisfaction among beneficiaries and MoSA staff. In some cases, the

brochures arrived late and SMSs were delayed. The programme was conducted within a short time

frame and this hindered the ability to create much need awareness about the process and objectives.

In terms of the distribution process, the majority of beneficiaries were satisfied with the steps taken

including the SMS, direct contact by telephone and direct communication in some cases. However,

the selection process was widely criticised by other beneficiaries and MoSA staff because of the

distribution locations and regrouping of several smaller community-based centres into one large

centre, making travel burdensome, and internal organisation issues difficult to manage. It was

described as chaotic and disorganised in some SDCs. Other tensions occurred between members of

communities and the SDCs workers when individuals did not understand why they had been

excluded from the programme and SDC staff was unable to explain the selection criteria and process.

Finally, the unrestricted cash transfer modality was perceived as a positive method of support by

most of the beneficiaries and SDC staff enabling families to spend the cash on what they think is

important to them at any time, based on their priorities. However, they indicated the need to revisit

the cash amount and the possibility of receiving it on a recurrent basis. Contrary to this, most of the

MoSA, NPTP and WFP staff in addition to some SDC workers who were interviewed preferred the

restricted cash transfer, especially the e-food voucher and e-vouchers for clothing or fuel in order to

ensure the best usage of the cash amount to secure the basic needs of the vulnerable groups

particularly children.

55


6. Lessons Learned

During the implementation of the current Winter Cash programme evaluation the following lessons

learned were identified based on several observations gleaned from the key informant interviews and

FGDs conducted in addition to the data captured from the PDM process-including the survey

findings and FGDs facilitated with a sample of the programme beneficiaries:

Selection of the most appropriate cash modality and delivery mechanism should be made in

consultation with the field coordinators, social workers and other relevant actors, including a

sample of the direct beneficiaries, given the mixed opinions that exist between beneficiaries

and the NPTP and considering the risks associated with both the restricted and unrestricted

cash modalities.

The lack of involvement of field coordinators, social workers and other SDC staff in the

selection criteria and process is a lesson to be learned. It is strongly recommended that they

be involved and consulted through different feedback and reflection activities in order to

ensure a higher level of fairness in the data collection and selection processes.

The absence of accurate, up-to-date and complete data on the poorest vulnerable families

registered in the NPTP led to concerns about the relevancy of the selection process, which

might not have taken into account the poorest vulnerable families- including children with

disabilities, orphans, the unemployed parents and/or caregivers, and others.

Lack of coordination and the importance of the complaint mechanism as a means for

effective management requires improved follow-up with the bank, by WFP, in order to

receive updated balances of the bank accounts.

The lack of relevant skills of the field coordinators and social workers played a vital role in

understanding the training provided by NPTP. This requires special attention to strengthen

the human and institutional capacities of the NPTP towards improved management of future

cash programmes, particularly with regards to skills related to teamwork, communication,

monitoring and field work.

NPTP should develop their own effective monitoring system that includes quality assurance

and a performance management component. In addition, there is a need for better knowledge

management within the NPTP to ensure the quality of the data and information.

56


7. Key recommendations

Based on this review and assessment of the Winter Cash programme, the consultant proposes the

following recommendations for improvement of the programme, should it be expanded or

replicated in other areas in the future. The majority of these recommendations were derived from

the analysis of the data collected during the interviews and FGDs with the different actors. These

recommendations will be presented in sub-categories in bullets as follows:

Programme approach, design and interventions

It is highly recommended that a multi-dimensional approach be adopted when developing

cash-based programme interventions within NPTP. This approach comprises an integrated

and comprehensive package of activities. Programme developers should answer the “Cash

for what?” question during the design phase, as the cash modality should not be seen as an

end in itself or as a standalone programme. (Relevance & & appropriateness findings #1-page 26)

Special attention should be given to developing a separate, well-structured programme

proposal or concept note, considering that it is part of the Country Programme Document

(CPD). This plays a major role in explaining the programme to partners and other

stakeholders. This document should include the programme’s objectives, activities, outputs,

outcomes, M&E mechanism, logical framework, duration, target groups and sustainability

mechanism. (Relevance & appropriateness findings#5-page 34)

Having a complete form of the theory of change that combines between vertical & horizontal

logic template and the traditional logical framework matrix is helpful in humanitarian aid

programmes in order to track whether the outputs are achieved and to identify the means of

verification through which data and evidence are communicated to confirm the indicators. In

addition, risks and assumptions that might be faced or encountered as a result of external

factors during the implementation of the program should be stated.

A gender component should be devised for the programme, and managed and led by NPTP.

This should state how the programme will be guided by equality and equity measures relating

to gender during the different phases and should include the need for disaggregated data. (Relevance & appropriateness findings#4-page 33 + Effectiveness findings # 12-page46)

The age of eligibility of children should be revisited, with the possibility to extend assistance

to children aged up to 18 years (in compliance with the definition of a child adopted by

UNICEF) and/or agree jointly on the maximum number of children who will be supported

per family considering the available financial resources. (Relevance & appropriateness findings#1-

pages 28-29)

The cash transfer amount should be revised to ensure that it is adequate to secure basic winter

needs of vulnerable children. Consideration should also be given to the supplementation of

transportation fees of those who travel to collect the ATM card from their identified SDC.

)Effectiveness findings # 6-page 36)

Capacity building

It is highly recommended that the NPTP provide capacity building activities for field

coordinators and social workers and invest in raising their capacities and skills in the areas of

communication, field work, monitoring, mentoring, follow-up, data collection, data integrity,

data validity, child rights, and research ethics. With these skills, they can play a vital role in

updating the existing data regarding vulnerable families in Lebanon and work as a permanent

community M&E team not only for the Winter Cash programme but also for other future

programmes. This could be organised in the form of a 4-5 day training workshop to cover the

57


basic information and skills under each of these areas. Investing in capacity development

will also promote enhance institutional and human resources sustainability. (Relevance &

appropriateness findings#1-page 27 + Effectiveness findings # 10-page 43)

Awareness raising sessions for parents should be included in the design of the programme to

educate them about spending the money appropriately on the needs of the children, and

explain to them the main objective of the programme, highlighting children as the main target

group. (Relevance & appropriateness findings#1-page 28)

Selection process

Immediate resources should be allocated by the NPTP to develop a well-structured

vulnerability assessment for the communities and families registered in the NPTP programme

to gather accurate baseline data that can be referred to when needed in other similar

programmes.. This requires the compilation of accurate, complete and updated data that

addresses all the vulnerability and welfare indicators (employment and income status,

number of family members, illness status, disability, gender variables, education status,

orphans’ status, house conditions and others).The NPTP has already started this process. (Relevance & appropriateness findings#2-pages31- 33 + Effectiveness findings pages 35-37)

It is highly advised to promote and increase the involvement of the field coordinators, social

workers and other actors in the SDCs in the selection process starting from jointly developing

the selection criteria and including the SDCs in the implementation phase as partners and not

only as implementing mechanisms. (Relevance & appropriateness findings#2-page 32)

The questionnaire must be revised as it currently does not include the minimum assessment

requirements and variables; this will enable the accurate identification of vulnerable poor

families referring to specific vulnerability variables. (Relevance & appropriateness findings#2-page

31 + Effectiveness findings #1- page 37)

Coordination

It is recommended to continue working with the key partners following the format of the

steering committee - which represents all the partners and different actors. - A

communication and coordination plan outlining specific roles and responsibilities to guide

them during the different phases of the programme should be developed. The steering

committee should meet regularly and on a timely basis to immediately deal with the

challenges and problems encountered. UNICEF has announced that these meetings will

continue in future cash programmes. (Relevance & appropriateness findings#2-page 32 + Effectiveness

findings page 39)

It is highly advised during the implementation to set unified guidelines and principles for all

the SDCs to avoid inconsistency when finding solutions to issues. For example, in some of

SDCs field coordinators monitored the social workers in the field while others did not follow

this approach. (Coordination findings # 15-pages 51-52 + Efficiency findings- pages 36-37)

Cash modality

If in the future the restricted cash transfer (e-vouchers) - based on the recommendations of

some groups interviewed in the current evaluation process – is adopted, it is highly

recommended to engage in a joint partnership with other organisations such as WFP and

UNHCR where each partner can complement the other to address the needs of beneficiaries. (Effectiveness findings # 9- pages 41-42 + Annex 8 attached to this report)

It is highly recommended that NPTP continues to implement the unrestricted cash transfer,

which is the preferred cash modality of the majority of beneficiaries, but with some

conditions/or alternatives included:

58


Ensure the selection process reaches the poorest vulnerable families fairly.

Provide the cash transfer on a monthly or quarterly basis to secure basic needs.

Increase the cash transfer amount significantly in order to secure basic needs especially in the

winter season when employment opportunities are scarce. (Effectiveness findings # 9- page 42 +

Annex 8 attached to this report)

The majority of the participants indicated that they prefer the e-food voucher if they can

receive it regularly and not once per year, as it represents the best e-voucher modality

compared to the e-voucher for fuel or clothes as mentioned in the report. This preference is

explained by the fact that they need to have control in the malls to ensure that they deliver

food items not other non-food items in a corrupt manner. (Effectiveness findings # 9- pages 41-42 +

Annex 8 attached to this report)

Monitoring & Evaluation

Special attention should be given at the NPTP level to developing a well-structured M&E

system that comprises methodologies and tools, including some of the community reflection

tools linked to a combination of qualitative and quantitative indicators.. Accordingly, the

selected field coordinators and social workers in each community/area per SDC will work as

a community M&E team by incorporating a reflection and reporting component when

meeting with the direct beneficiaries and make corrective action when needed. (Effectiveness

findings # 11- pages 43-44)

Best practices should be supported and documented. These examples or success stories can

be shared with others for learning purposes. (Effectiveness findings # 11- pages 43-44)

Special attention should be given to increasing the involvement of the field coordinators and

social workers in the different phases of programme implementation. (Effectiveness findings #

11- page 45)

Joint partnerships It is recommended to develop joint partnerships with organisations who work with the same

vulnerable families registered in the NPTP in the targeted areas. Through this partnership, the

results of the vulnerability assessment can be used and the areas of programme interventions

can be jointly developed referring to the scope of work of each organisation. Based on such

practice, duplication of services can be avoided and organisations can complement each other

through an integrated approach. (Relevance & appropriateness findings#2-pages 31- 33)

The evaluation suggests investigating the possibility of targeting the winter needs of some

groups whose profile might not be consistent with the scope of work or the mandate of some

of these organisations. (Effectiveness findings # 6- page 36)

Scaling up opportunities In order to promote the institutional sustainability of the NPTP and strengthen its ownership in other

similar programmes, special attention should be given to adopt a well-structured strategy that

includes the following sub-strategies (Sustainability findings # 14-page 50):

Implement a comprehensive well-designed capacity building training in advance, before the

beginning of the project implementation.

Provide awareness sessions for targeted parents and/or caregivers to ensure that they will use

the cash transfer assistance appropriately and secure the basic needs of their children during

the winter season.

Conduct a vulnerability assessment to collect accurate and updated data on vulnerability and

welfare variables.

59


Promote integration with the other organisations working with the same targeted groups to

maximise the optimal use of financial resources.

Set clear, structured communication and coordination mechanisms among the different actors

including the key partners and the bank. This will be helpful in effective implementation of

programme activities and in a timely manner and identify and manage gaps.

Develop a well-structured monitoring and quality assurance framework with clear

mechanisms, methodologies and tools to track the progress of the programme.

60


Annexes

Annex 1: The Terms of Reference (TOR)

61


62


63


64


65


66


67


68


Annex 2: List of stakeholders interviewed

S Name Sex Position Affiliation /

Governorate

Date of

interview/F

GD

M F Total

(1) Key partners ( Individual & group interviews) 1 Mrs. Marie Camaghia 0 1 1 Winter Cash programme Focal

Point NPTP/MoSA 16/11/2016

2 Mr. Ramzy Fanous 1 0 1 CMU/PCM statistician CMU/PCM 24/11/2016

3 Mr. Michele Sarkis 1 0 1 IT Business officer NPTP/MoSA 28/11/2016

4

Mrs. Fouz Quebesi 0 1 1 Programme Policy officer-

Formerly cash & transfer associate WFP 29/11/2016

Mr. Charbel Habib 1 0 1 Head of Cash Transfer

Programming Unit - Formerly cash

& transfer associate

5 Dr. Bashir Osmat 1 0 1 The adviser of the MoSA MoSA 29/11/2016

Sub-total 4 2 6

(2) Other stakeholders ( Individual, group interviews & FGDs)

A. Individual interviews:

6 Mrs. Naziha Dakroub 0 1 1 Chiyah SDC Director Mount Lebanon 24/11/2016

7 Mrs. Rajia Bitar 0 1 1 Rahbe SDC Director Akkar 2/12/2016

8 Mrs. Sahar Alawani 0 1 1 Al Mina SDC Director Tripoli 6/12/2016

9 Ms. Hoda Said 0 1 1 Baalbek SDC Director Bekaa/MoSA

20/12/2016

10 Mr. Wael Assaf 1 0 1 Director of SDC directors of

Bekaa 11 Mrs. Fatima Sawrli 0 1

1 Ex- Field coordinator - Chiyah

SDC

Mount Lebanon

24/11/2016

12 Mrs. Fatma Zeater 0 1 1 Field Coordinator - Chiyah SDC 1/12/2016

13 Mr. Hayssam Habib 1 0 1 Field Coordinator – Rahbe SDC Akkar 2/12/2016

14 Mona Raad 0 1 1 Field Coordinator- Baalbek SDC Bekaa 20/12/2016

Sub-total 2 7 9

B. Group interviews: 15 1. Hussein

Tabaja

2. Amira Khalil

3. Fatma Gaafar

4. Maha

Dakrouni

5. Amel Merwa

1 4 5 Social workers-Goberiri SDC

Mount Lebanon

1/12/2016

16 1. Khetam

Mohamed

2. Wafaa Elias

0 2 2 Social workers-Rahbe SDC

Akkar 2/12/2016

17 1. Selvia

Antonios

2. Dina Gamous

0 2 2 Social workers-Al-Mina SDC

Tripoli 6/12/2016

18 1. Amira

Sharamnt

2. Wael Negro

3. Belal Kadri

0

1

1

1

0

0

1

1

1

- SDC Director- Bab E-

Tabaenh

- Field coordinator – “ “

- 4 Social workers - “

Tripoli 13/12/2016

69


4. Fawaz

Moreaby

5. Nivine El-

Gendy

6. Farah Saleh

1

0

0

0

1

1

1

1

1

“

19 1. Mary

Ghanem

2. Walaa El-

Nemr

3. Nour El-Hoda

4. Laila Asaf

5. Rawia Saleh

6. Waad El-Haj

7. Zeinab Ismail

0

0

0

0

0

0

0

1

1

1

1

1

1

1

1

1

1

1

1

1

1

Social workers- Baalbek SDC

Bekaa 20/12/2016

Sub-total 4 1

8

2

2

C. Focus Group Discussions with sample of programme beneficiaries: 20 FGD with sample of

Programme

Beneficiaries

(All were parents

and/or caregivers)

2 6 8 Ghobeiri SDC Mount Lebanon 1/12/2016

21 2 8 10 Rahbe SDC Akkar 2/12/2016

22 1 12 13 Al-Mina SDC Tripoli 6/12/2016

23 0 9 9 24 11 2 13 Baalbek SDC Bekaa 20/12/2016

Sub-total 1

6

3

7

5

3

Total 2

6

6

4

9

0

Classification of stakeholders interviewed per category

S Stakeholders category/type # of

interviewed

1 Key partners 6

2 SDC/MoSA directors 6

3 Field Coordinators 5

4 Social Workers 20

5 Beneficiaries 53

Total 90

70


Annex 3: List of documents consulted

The following are the key documents consulted and referred to during the evaluation process:

TOR of the evaluation of the Winter Cash programme.

Overview about the Winter Cash programme “Leaflet”.

Basic Assistance/winter HRRP 2016.

NPTP – UNICEF Winterisation programme Implementation Plan.

PPT about the Post Distribution Monitoring- Lessons learnt document- April, 2016.

PDM Final products including FGD Summary Reports and Survey findings.

Distribution documents including: banners, card issuance (household registry statement),

card loading (payment list), card transportation (delivery notes, Clustering (clustering of

NPTP UNICEF beneficiaries, distribution cohorts, questions and answers and programmatic

information for social workers, templates and standards operation procedures distribution

and timeline & distribution rate tracking.

Complaint mechanism hotline including logbook & procedures.

Card Management NPTP.

Questionnaire administrated to select the poor vulnerable families.

71


Annex 4: Data collection instruments – Methodologies & tools’ guides

Annex 4.1 - Focus Groups Discussions (FGDs) Guide

(With sample of programme beneficiaries: Children & their families)

Key Guidelines

Focus group discussions each had a maximum of 8-12 participants with an average of 10

persons including parents and/or caregivers from the Lebanese poor vulnerable families. The

actual number of participants ranged between 8-13 participants.

At the beginning of the FGD, the consultant/facilitator provided a short introduction to

explain the purpose of the evaluation process, and the objective of conducting the FGD and

why they were selected to take part in the activity.

The FGD was conducted by one facilitator and one junior researcher, who documented the

responses, allowing the facilitator to focus on managing and facilitating the FGD.

The facilitator asked participants for their consent to use the information collected during the

FGD for inclusion in the evaluation.

The facilitator conducted the FGD in an interactive manner, giving participants the

opportunity, to express their views, insights and ideas freely.

The seating arrangement was shaped in a semi-circle to allow all participants to take part in

the FGD and to allow the facilitators to observe any non-verbal communications.

A documentation form was developed to document the name, age, profession and sex of

participants and other basic data about the participants in the FGDs (the names will be

referred for internal use only- when needed).

The duration of the FGD was between 60-90 minutes.

At the end of the FGD, the facilitator summarised the information recorded and thanked the

participants for their feedback and time.

Selection criteria

The total number of participants involved in each of the FGDs ranged from 8-13 participants.

All the participants were selected from the parents and caregivers of poor vulnerable families.

Gender of participants was considered equitably to include both male and female participants.

Despite the fact the all participants are supposed to be among the most/vulnerable poor

children and their families, they were in fact selected from different socio-economic status.

Despite the fact that the initial criterion was intended to include at least one person with

disabilities, this was not implemented in practice as it was not possible.

Key issues/questions that were discussed in the FGDs facilitation process

Discuss the orientation received about the selection & distribution processes.

Discuss the support received from the SDCs and social workers involved in the distribution

process

How did you spend the cash transfer amounts and in which aspects of your lives (e.g.

education, services, health, transportation, food, clothes, etc.)

What are the key changes captured in your lives as a result of your participation in this

programme? Elaborate.

72


As one of the recipients of cash transfer assistance, are you satisfied about the programme

implementation?

To what extent do you think that the selection process of the targeted poor families followed

what was explained at the beginning of the process through the orientation sessions?

What do you think about the timing of the cash transfer process referring to the winter

season?

What do you think about the level of support received from the MoSA (SDCs social workers

during the different phases of the programme implementation?

What were the key challenges and weaknesses encountered and how were they addressed?

Who did you prefer to contact to deal with this issue?

How well did the complaint mechanism work from your point of views? Explain with some

examples?

What is the level of acceptance of the cash transfer modality from your point of view?

If yes, why do you see this modality as the best one in the winter cash assistance programme?

If not, from your point of view, what are the other modalities that can work much better? And

why? Explain with some examples.

From your point of view, in case of replicating the same programme, what we can be done in

a better way to improve the quality of the programme implementation?

--------------------------------------------------------------------------------------------------------------------------

Annex 4.2 – Semi-Structured Interview Guide

(Key partners/stakeholders: NPTP, MoSA, CMU/PMC, WFP and others)

Interviewee Name: Age:

Position: Sex:

Affiliation:

Date:

The venue of the interview:

Interviewer:

Note Taker (Documenter

At the beginning of the interview, the consultant/interviewer provided a short introduction to

explain the purpose of the evaluation process, and the objective of conducting the interview

and why they were selected to take part in this activity and how the information collected will

be used.

The interview lasted between 60 to 75 minutes, in a location preferable to the interviewee.

Key issues/questions that were discussed with the interviewee

Discuss the role of the interviewee and his/her affiliation in the programme implementation?

From your point of view, to what extent were the programme design and planned activities

adequate promoting the achievement of the programme objectives?

To what degree did the programme meet the needs of the most socio-economically

vulnerable Lebanese Households?

To what extent was the criteria of the selection process developed while ensuring that the

most vulnerable and poor children were reached and benefited properly from the programme?

73


What is the added value of having such an integrated approach with the different selected

partners/actors in this joint programme?

If there are some specific successes, what are the key factors that led to that success?

What are the key challenges/constraints (hindering factors) that influenced the achievements

of the programme objectives? How were they addressed?

How effective was the management and coordination system - as part of the joint programme

partnership among UNICEF, NPTP, MoSA, CMU/PCM and WFP in delivering the set

results to the target population?

From your point of view, to what extent has the planned and implemented response reduced

vulnerability among the targeted groups?

What is your level of acceptance of the cash transfer modality from your point of view?

If yes, why do you see this modality as the best one in the Winter Cash programme?



To what extent were the capacities of the different actors- particularly the NPTP and MoSA -

adequate and efficient to ensure the quality of the programme implementation? What can be

done in a better way in this regard?

To what extent was the Winter Cash programme cost-efficient – including the optimal usage

of the programme resources & time efficiency as opposed to other modalities that could be

designed to cover the same needs of the Lebanese vulnerable poor children and their

families?

To what extent did the programme equip service providers (NPTP/MoSA) and beneficiaries

with the capacities (skills and knowledge) (Human resources' sustainability), resources

(connections) and confidence (Institutional and financial sustainability) to sustain the

programme implementation?

What are the key lessons learnt from the programme implementation?

What can be done in a better way to improve the programme effectiveness?

Others (E.g. consider the gender equality during the evaluation process?

Annex 4.3 – Semi-Structured Group Interview Guide

(With sample of MoSA social workers and SDCs coordinators)

Date:

The venue of the interview:

Interviewer:

Note Taker (Documenter

S Interviewee name Age Sex Position Affiliation

74


At the beginning of the interview, the consultant/interviewer provided a short introduction to

explain the purpose of the evaluation process, the objective of conducting the interview, why

they were selected to take part in this activity and how the information collected will be used.

The interview lasted between 60 to 75 minutes, in a location preferable to the interviewee.

Key questions/issues will be discussed with the interviewee

The role of the interviewee and his/her affiliation in the programme implementation?

From your point of view, to what extent were the programme design and planned activities

adequate in promoting the achievement of the programme objectives?

To what degree did the programme meet the needs of the most socio-economically

vulnerable Lebanese households?

To what extent were the capacities of the different actors- particularly the NPTP & MoSA -

adequate & efficient to ensure the quality of the programme implementation? What can be

done in a better way in this regard?

To what extent were the criteria of the selection process developed while ensuring that the

most vulnerable and poor children were reached and benefited properly from the programme?

What are the key challenges/constraints (hindering factors) that influenced the achievements

of the programme objectives? How were they addressed?

From your point of view, to what extent has the planned and implemented response reduced

vulnerability among the targeted groups?

What is your level of acceptance of the cash transfer modality from your point of view?

If yes, why do you see this modality as the best one in the Winter Cash programme?



What are the key lessons learnt from the programme implementation?

What can be done in a better way to improve the programme effectiveness?

75


Annex 5: Evaluator Bio Data

Awny Amer is working as an independent Research, Monitoring & Evaluation consultant, based in

Egypt. He has 30 years of experience in development & evaluation. Before, he worked with Plan

International Egypt for about 22 years before starting his role as independent R&E consultant starting in 2009.

Through his career in the R&E field, he had the opportunity to lead and conduct different R&E

processes, surveys, and situations analysis for diverse programmes addressing children, youth,

women and other vulnerable groups with most of the UN and INGOs at global, regional and national

levels- including UNICEF, UNDP, UN Women, WFP, ESCWA, Oxfam, IDRC, CARE, Save the

Children, Inter News, Search for Common Ground (SFGG), UNRWA, and Plan International among

others. He has good regional experience (MENA) region where he worked in different MENA

regional countries.

In this regard, he always adopts the application and usage of a combination of the qualitative &

quantitative with special focus on the participatory M&E approach, methodologies and tools

including Most Significant Change (MSC) technique, storytelling techniques, longitudinal study

processes, FGDs, case studies, community reflection tools and others in a consultative manner during

the evaluation process to promote meaningful participation and strengthen the accountability practices on different levels.

Currently, he is acting as board member of the International Development Evaluation Association

(IDEAS), African Policy Centre (APC), the Egyptian R&E network in Egypt (EREN) and the

Community of Evaluators (CoE) in addition to acting as one of the founders of the EvalMENA and

active member at AfrEA. He had the opportunity to develop different papers and abstracts where

they were submitted to different international evaluation conferences. Also, he had the opportunity to

translate some of the key R&E publications such as the “The Road to Results” book, the Most Significant Change technique (MSC) guide among others.

76


Annex 6: Evaluation Matrix

Criteria Evaluation Question(s) Method(s) Target

Groups(s)

Triangulation

Relevance 1. To what extent were the

scope of project

intervention strategies,

project design and

approaches appropriate and

relative to promoting the

intended objectives of the

programme? And to what

extent the needs of the most

socio-economically

vulnerable Lebanese

households were met?

2. To what extent were the

criteria of the selection

process developed while

ensuring that the most

vulnerable and poor

children were reached and

benefited properly from the

program?

3. How gender sensitive was

the programme in reaching

the most vulnerable

children?

4. To what extent has the

programme been successful

in identifying the most

relevant partner/actors for

this project and what was

the added value of these

partnerships?

5. Do the current Log frame

and/or results framework

and relevant indicators in

the Winter Cash

programme document

need improvement to

better serve the

programme objectives?

Semi-

structured

interview

Relative

programme staff

of UNICEF

Lebanon

Semi-structured

interviews with

key partners (in

some/all parts of

these issues)

FGDs with the

targeted groups of

the vulnerable

families

The individual and

group interviews

with the field

coordinators and

social workers.

77


Effectiveness

6. To what extent did the targeted

vulnerable children & their

families of the Winter Cash

programme were reached by

the programme as planned?

How satisfied were they about

the programme?

7. To what extent did the

programme have an effect on

beneficiaries in improving

access to essential goods and

services of their choice in a

safe, dignified, and empowered

manner while decreasing socio-

economic vulnerability? (Key

changes impacted on the lives

of beneficiaries as result of the

programme implementation)?

8. Are there other unintended or

unforeseen changes shown in

the lives of the targeted groups

of poor children & their

families? Also, are there

unintended beneficiaries in the

programme?

9. What was the level of

acceptance of the cash modality

as a tool to support poor

Lebanese households to cover

their winter needs

(winterisation) from the point

of views of all the different

actors: MoSA, NPTP, SDCs,

field staff and the beneficiaries?

10. To what extent the capacities of

the different actors- particularly

the NPTP & MoSA actors

involved were adequate and

efficient to ensure the quality of

the programme

implementation? What can be

done in a better way in this

regards?

11. To what extent does the

programme have an effective

Focus-

Group

Discussion

s

Key direct

beneficiaries

(Lebanese poor

vulnerable

children and their

families)

(Some of the

questions

discussed with

the beneficiaries

will be discussed

with partners:

UNICEF, WFP

staff)

1-Semi-Structured

interviews with the

concerned

UNICEF Lebanese

Staff and WFP

2-Secondary

analysis review

3- Semi-structured

interview with

decision makers in

MoSA and NPTP

78


monitoring & evaluation

mechanism (including

reporting, quality assurance and

reflection) processes on

different levels (UNICEF,

NPTP, CMU/PMC) in place

and how has the programme

used information generated to

inform programmatic

adjustments and corrective

actions during the life of the

project?

12. To what extent did the project

consider gender equality in the

programme implementation –

including the selection process

(e.g. is USD40 an equitable

amount for both boys and girls?

On what basis was it calculated

and does it take into account

gender-based requirements?)

Efficiency 13. To what extent was the

Winter Cash programme

cost-efficient – including

the optimal usage of the

programme resources &

time efficiency as opposed

to other

modalities/programmes that

could be designed to cover

the same needs of the

Lebanese vulnerable poor

children and their families?

Semi-

structured

interviews

Key partner(s)

(NPTP and

MoSA)

Secondary data

analysis & review

Semi-structured

interviews with

key UNICEF and

WFP programme

staff involved in

the program

Sustainability 14. To what extent did the

programme equip service

providers (NPTP/MoSA) and

beneficiaries with the capacities

(skills and knowledge) (Human

resources' sustainability),

resources (connections) and

confidence (Institutional and

financial sustainability) to

sustain the programme

implementation?

Semi-

structured

interviews

Key partner(s)

(NPTP & MoSA)

Secondary data

analysis & review

Semi-structured

interviews with

key UNICEF &

WFP programme

staff involved in

the programme

79


Coordination

15. How effective was the

management and

coordination system - as

part of the joint programme

partnership among

UNICEF, NPTP, MoSA,

CMU/PCM and WFP in

delivering the set results to

the target population?

Semi-

structured

interviews

Key partner(s)

(NPTP and

MoSA- including

the SDCs

directors & field

coordinators)

Secondary data

analysis & review

Semi-structured

interviews with

key UNICEF &

WFP programme

staff involved in

the programme

80


Annex 7: Results framework

Results Chain (Operational & Developmental results)

Overall goal: “Improve the current conditions of the vulnerable & most vulnerable Lebanese poor children

and their families”

Outcome 1: Lebanese children and the families vulnerable to seasonal hazards and unexpected winter

emergencies are able to maintain safe access to goods and services.

Outcome 2 – Strengthened delivery mechanisms for the NPTP via the incorporation of new systems : Cash

based programme

A.1 Inputs A.2 activities A.3 Outputs A.4 Outcomes A.5 Impacts

Funding and

human resources

on different levels

(UNICEF staff,

NPTP, MOSA

and other actors &

partners) to

support the

implementation of

the programme

activities aimed to

improve the

current situation

of the Lebanese

poor children &

their families in

the targeted areas

of the programme.

Cash transfer

processes and other

relative supportive

guidance steps

provided by/and to

the key partners to

promote use of these

resources to help

achieve programme

outcomes & outputs

and establish efficient

distribution and

delivery

unconditioned cash

systems & respective

mechanisms.

Overall Rating:

MOUs

developed,

budget

allocated, ATM

cards

distribution

completed and

complaint

process and

mechanism set

in the targeted

working areas

in Lebanon

Overall Rating:

Increased the

efficiency of the

Delivery and

Distribution

systems on the

NPTP/MoSA

level.

Overall Rating:

Increased

sustainability

& ownership

(on the

national &

governmental

level) of

programmes

that targeting

the

vulnerable

poor

Lebanese

children and

their families

Overall

Rating:

81


Annex 8: Summary of the stakeholders’ feedback and insights about the restricted cash modality versus unrestricted cash

modality

S Unrestricted cash modality

Restricted cash Modality

The following is a summary of the justified reasons for the preference

for the unrestricted cash transfer from the point of view of the

majority of the direct beneficiaries (parents/caregivers) compared to

the other stakeholders interviewed in the current evaluation process.

This corresponded with the PDM findings as the majority does not prefer

other kinds of modalities in comparison to those who do (88per cent

versis 12 per cent)47.

The following is a summary of the justified reasons of

stakeholders preferences for the restricted cash transfer

assistance to cover the winter needs provided to the poor

Lebanese households (advantages of adopting the e-vouchers or

restricted cash transfer as opposed to the restricted cash transfer

modality as reported by all the partners (WFP, NPTP and MoSA

staff) interviewed and 12 per cent48 of the FGD participants in the

PDM process as indicated inside the report :

1 The unrestricted cash transfer assistance modality gives

families the freedom and flexibility of choice, and the

power to purchase the goods and services they need as per

their own priorities instead of receiving clothes or others

items they do not need. This is particularly of great value

in the case of spending this cash on covering medication

expenses for chronic diseases or disabilities within the

family especially if this is not considered in the selection

process.

The possibility to misuse the cash amounts in purchasing

inappropriate items by the cardholder.

2 It is worth noting that most of the targeted poor vulnerable

households in the programme have access to other e-

vouchers, provided by WFP, UNHCR or others, which

create duplication of the services, provided resulting in a

gap in the cost effectiveness.

When cash transfer is adopted as the preferred modality,

there is no assurance that the children will benefit from this

programme because of the unconditional nature of this

method.

3

Discussions with two of the SDC directors interviewed

reported that the unrestricted cash transfer is cost-effective

The burden of additional expenses such as bank charges in

addition to other relative fees to issue the ATM cards on

47S4tatistics Lebanon Ltd_ Results of the survey findings _Winter Cash Programme-PDM Report-2016_Slide# 74 48 Ibid- slide # 74

82


in comparison to the other alternatives of the restricted

cash transfer (restocking, seed distribution, etc.) especially

in view of the presence of the WFP as an effective partner

and which already has such assets through other similar

programmes. This was validated by some of the SDC

directors (e.g. Baalabek SDC) when interviewed in the

current evaluation report.

the accounts created for the holders of the ATM cards.

4 Additionally, two of SDC directors interviewed during the

evaluation process indicated that the unrestricted cash

transfer assistance is a human-based approach compared

to the e-voucher to buy clothes- for example. “ It is not

acceptable to let children wear the same type of winter

jacket, for example, which makes them feel like they

are stigmatised and conveys the image that they are

either poor/vulnerable or refugee. This has a negative

impact on social cohesion within the community

especially if they feel that you are violating their

dignity,” one of the interviewers commented.

It might lead to inflation and increased local prices in the

markets.

5 In this regard, other technical distribution related issues

such as the low quality level of materials and what could

be provided is too expensive. “With the e-voucher

provided to buy the clothes from specific shops, we can

buy the same and maybe double the number of items

with the same amount”. “Actually, this is against

respecting our privacy” other participants commented.

This means that the participants feel a great deal of

independence and privacy when they can buy the clothes

wherever they like and select the models they prefer

instead of going to specific places that offer models they

might not like.

Women may not retain control over the income or on the

amounts transferred. During the FGDs, some cases

revealed disputes between the two parents which resulted

in losing the benefit of the cash transfer assistance

especially if the ATM card was with the husband while the

“Hala” card was with the mother. Children may suffer the

consequences and be deprived from benefiting from the

programme. In fact, this was reported by some of the FGD

participants as explained earlier in this report.

6 Potential benefits of cash injection on local markets and

trade.

Potential security issues especially for those who cannot

read and write well.

7 There could be some kind of corruption in the restricted

cash transfer (ex. e-food voucher). Some of the

participants in the FGDs reported that “some markets

Two of the SDC directors interviewed indicated that in

some cases, they noticed that the unrestricted cash transfer

may result in dependency behaviours from some of the

83


provide different non-food items to the holder of the e-

voucher - including cigarettes as per his request while

it is registered as foods”.

parents who are not working seriously or might result in

preference to bearing more children, in order to benefit

from this programme. 8

It is easily invested in livelihood security as long as the

transfers are sufficient.

9 Can improve the status of women and marginalised groups

if they use it to raise their own small projects or are even

being involved in the decision-making process inside the

family. Two of the female FGD participants in Mount

Lebanon and Tripoli- Al-Mina SDC indicated: “I feel

empowered when I have the cash transfer amount in my

hands. I can select what my children need myself and

this makes me feel that I’m the owner of the decision

taken in this regard.”

Sometimes, the provision of cash may cause other social

problems such as family disputes and domestic violence.

(E.g. some of the families that took part in the FGDs

indicated that some of their neighbours divorced and

separated with the father possessing the ATM card while

the mother holds the Hala card. As a consequence, they

were not able to use the cash transfer amount). Other

family members pointed to the occurrence of family issues

because of the bad use of the cash transfer amount by the

father. However, this happened rarely and in few cases as

indicated before in this report.

10 By adopting the unrestricted cash transfer assistance,

projects benefit the community as a whole and promote

the internal trade and market movement within a small

community.

11 Finally, it could be quicker to mobilise than alternatives

such as food purchase and transport. This means that the

other unrestricted cash transfer might include logistical

steps with additional cost compared with the unrestricted

one.

84


Annex 9: Summary overview about the Post Distribution Monitoring (PDM) process

Feedback from the Ministry of Social Affairs (MoSA) staff and beneficiaries of the Winter Cash

programme is a crucial matter for UNICEF in order to evaluate its performance, redirect, improve

and upgrade the programme for any future assistance it might provide.

UNICEF engaged InfoPro Research (a third party entity) to conduct a series of focus group

discussions (FGDs) to assess the perception of the target population and field actors; programme

implementation; processes; use of assistance; and the general satisfaction and perception of

effectiveness of the cash modality.

The qualitative information collected through the FGDs was triangulated with the quantitative data

collected through the questionnaires on a statistically representative sample of the beneficiary

population. The objective was also to gather information on the possible areas of conflict and/or

sensitivities regarding the programme and more specifically on possible tensions that could have

arisen among the communities.

InfoPro held twelve focus group discussions; four with MoSA staff and eight with beneficiaries. The

focus groups with the MoSA staff were split as follows: one with directors, one with coordinators,

and two with social workers, one of which was in North Lebanon and the other in Mount Lebanon.

As for the beneficiaries, the focus groups were split as follows: four with benefiting females and four

with benefiting males residing in Mount Lebanon, Tripoli, Halba and Baalbek. Each focus group was

attended by six to nine participants whose ages ranged between 24 and 53 years49.

InfoPro was provided with contact lists of MoSA staff and Lebanese beneficiaries by UNICEF. Prior

to each discussion group session, InfoPro screened and evaluated all the lists provided in order to

confirm that all participants had the appropriate profile. A total of ten MoSA staff and an average of

twelve Lebanese people were invited to each session in order to ensure a minimum of eight

attendees. The focus group discussion guide was drafted and developed by the UNICEF

Communication for Development (C4D) section in Arabic and translated into English by InfoPro. It

is included in the Appendix. Each focus session lasted a maximum of one and a half hours. As for the

FGDs, before the start of the discussions, InfoPro team and UNICEF’s Monitoring and Evaluation

Consultant, C4D section’ representative and Programme Specialist assisted in a training session to

elucidate the content of the discussion guide.

49 InfoPro_ Winter Cash Assistance- Final report -Post Distribution Monitoring Report-2016_page 6 & 35

Evaluation of the Winter Cash Programme for Lebanese Poor ... · 1.2 Programme overview- The...

Documents

Transcript of Evaluation of the Winter Cash Programme for Lebanese Poor ... · 1.2 Programme overview- The...