Impact Assessment – Linking Learning to Earning Approach

Final Evaluation Report

December 15, 2017

Presented by: Halcyon Louis, International Development Consultant halcyon.louis@gmail.com

i

Executive Summary

Introduction

This Final Evaluation Report is hereby presented to Finn Church Aid (FCA) by Ms Halcyon Louis,

international development consultant, for the contracted consultancy assignment: Impact Assessment –

Linking Learning to Earning Approach (LL2E), which is hereafter referred to as an evaluation) The

purpose of the evaluation was to explore to what extent, and how, LL2E activities in two countries of

focus, Nepal and Uganda brought about changes in BTVET quality and the lives of rights-holders, and

created actual linkages with employment and/ or entrepreneurship. In line with the terms of reference

(ToR), the scope of the evaluation covered LL2E implementation from 2015 to 2017 in both countries.

Description of the Intervention: LL2E

The LL2E) Approach uses market analyses and partnerships with the private sector, to facilitate BTVET

delivery for youth from vulnerable communities, and establish effective linkages between trainings and

the world of work. This approach is designed primarily for youth and women from the target

communities, to provide them with an accessible skill set for use in earning a decent living through wage

employment or entrepreneurship. Based on a funded timeframe of January 2015 to December 2017,

elements of the LL2E Approach have been systematically tested in Nepal and Uganda through the Skills

Training and Technical Education for Employment Programme (STEP) Nepal, and the Rwamwanja BTVET

Project for Congolese refugee youth and youth from the host community in Uganda. As a follow-up to

the testing of project component in both countries, as well as developing nations, FCA has developed an

indicative Theory of Change for the LL2E Approach, which covers the entire programmatic cycle, from

quality BTVET to gainful employment, and includes post-graduation support from the private sector for

wage employment and entrepreneurship.

Methodology

Overview

The technical approach to the consultancy was participatory, and was supported by the use of

utilisation-focused principles and mixed methods. All evaluation activity was also guided by a client-

approved, evaluation matrix. In lieu of an impact assessment, which concentrates on outcomes

achievement only, the piloted components the LL2E Approach were subjected to a comprehensive

evaluation that incorporated summative (backward-looking) and formative (forward-looking) elements.

Participatory Approach

The application of a participatory approach to the evaluation was appropriate on two levels. First,

evaluations that use a participatory approach to data collection and analysis are pragmatic; they allow

evaluators to identify and seek input for the design and implementation the of evaluation from the main

project partners, as key informants. The contribution of key informants increases the possibility for

better quality data; a better understanding of collated data; more appropriate recommendations; and a

better uptake of evaluation findings. Second, from an ethical perspective, key informants have a right to

ii

be involved in decision-making that affects them. A participatory approach was therefore used to

conduct an evaluation that is useful; ethical; and of high quality.

Utilisation Focused Evaluation Principles

Utilisation-focused evaluation principles use participatory methods to determine what the end-users

want from the evaluation, and thereby support ownership of the evaluation by its intended users. To

ensure that the evaluation is owned by its intended end-users, all categories of project stakeholders

who were involved in LL2E activities were engaged during the implementation of the evaluation. The

intention, here, was to capture the unique experience and perspectives of all project stakeholders, who

were recognised as key informants on the effects and workings of the LL2E Approach. The evaluation

maintained its independence and impartiality, however, as the consultant was responsible for the final

analysis of data; the interpretation of results; and the synthesis of findings.

Mixed Methods

Mixed-methods, entailing qualitative and quantitative techniques, were used, where applicable, to

ensure that: i) the methodological techniques were suitable for collecting and analysing the type of data

that was required; ii) data was triangulated from multiple sources, to increase the credibility of the

results generated during data analysis, to inform the development of the main evaluation findings; and

iii) the results of data analysis can be used to identify lessons and develop recommendations that can be

used to inform future project activities, as applicable.

Evaluation Matrix

In accordance with the ToR, including the specific evaluation questions that were suggested, the main

evaluation issues were categorised under seven dimensions, which were aligned to OECD-DAC

definitions; the Paris Declaration, and the Accra Agenda for Action. Based on the emphasis placed on

learning within the ToR, as well as the formative element of the evaluation, two additional dimensions

were incorporated into the evaluation matrix, namely lessons learned; and recommendations. The

evaluation matrix was finalised in collaboration with FCA during the Inception Phase, and was included

in the final Inception Report and Work Plan, as a guide for all evaluation activity

Sampling

Purposeful sampling based on a sequential approach was used to select target groups and activities for

inclusion in data collection activity. The rationale for this approach was two-fold. First, one of the main

limitations to the evaluation was resource scarcity (human; time; financial) which acted as a constraint

to the engagement of all persons who were involved in LL2E activities in the focus countries. Second, in

order to achieve the level of rigour required for a robust evaluation process, purposeful sampling was

structured around the main evaluation issues, to support the generation of accurate responses to the

key evaluation questions. The application of a sequential approach had the further advantage of

allowing data collection to evolve in tandem with emergent findings, by creating the flexibility for

additional data collection at any given stage of the evaluation. The approach to sampling placed the

evaluation questions at the forefront of the sampling process, and focused on pre-defined, strategic

criteria and categories, to increase the quality and accuracy of collated responses.

iii

Methods

The evaluation was implemented over five distinct, but overlapping phases: Phase 1: Inception; Phase 2:

Data collection; Phase 3: Data analysis and synthesis; Phase 4: Reporting; and Phase 5: Assignment

management. The main deliverable of the Inception Phase of the evaluation was the Inception Report

and Work Plan, comprising the approved evaluation methodology. Data collection involved an in-depth

desk review of relevant documents, guided by the approved evaluation matrix. Key informant

consultations were conducted by interview and focus group discussion, including during country

missions to Nepal and Uganda. Data analysis involved cross-referencing and triangulation from multiple

data sources, using descriptive, content, quantitative and comparative analyses, following which,

preliminary observations and findings were shared with key informants during validation sessions.

Feedback from the discussions informed further data collection, where required, and report

development. Assignment management occurred throughout the evaluation, and adhered to the quality

assurance procedures that have been established by OECD-DAC and UNEG.

Limitations

At the level of project design, the evaluation was challenged by resource scarcity of, in terms of the

availability of human and financial resources, and the designated timeframe Purposeful sampling based

on a sequential approach was used to mitigate this limitation. At the level of evaluation implementation,

data collection was challenged by the unavailability of some stakeholders for consultation. By way of

mitigation, consultations were held with key informants who were accessible during the evaluation, and

document review was used to triangulate collected data.

Main Findings

Impact

Finding 1: The LL2E Approach has contributed strongly to intended behavioural and attitudinal change

among graduates of LL2E project activities. Central to this development has been the facilitation of

access by the target groups of youths to BTVET opportunities and follow-up support.

By creating increased opportunities for youths from vulnerable and socially marginalised communities,

to access opportunities for skill-building and decent work, the LL2E Approach has contributed, in great

measure, to changes in behaviour and attitudes among the target group of young persons

Finding 2: Notwithstanding the ‘trickle-down effect’ of the LL2E Approach on the families and

communities of LL2E project graduates, the intended impact of forging effective linkages with the

private sector is a work in progress.

As a direct result of facilitating increased livelihood opportunities through improved skillsets, the LL2E

Approach can be seen to generate a ‘trickle-down effect’ on the communities where youths live as they

apply their newly-acquired skills. Importantly, however, there is evidence of inadequate use of

supportive resources that may be accessible through the private sector in both countries.

iv

Relevance

Finding 3: Given its aim to link vulnerable young persons to opportunities for improved livelihoods,

through BTVET and decent work, the LL2E Approach has strategic global- and country-level relevance, to

complement its alignment with the FCA institutional mandate.

LL2E further directly aligns with two FCA strategic priorities, namely, R2QE and the R2L. In light of this

strategic focus, LL2E can be positioned within the 2030 Agenda for Sustainable Development, and

national plans for sustainable social sector development in Nepal and Uganda

Finding 4: From a conceptual standpoint, the proposed LL2E programmatic design responds to the

situation of unemployed youth from vulnerable/ marginalised communities. Piloted components have

not, however, established adequate linkages between the trainings and the private sector for the

employment of graduates/ ex-trainees.

The LL2E Approach has been designed to address the unique circumstances of the target groups of

youths in each country. Evidence points, however, to the under-utilisation of private sector partnerships

to support the acquisition of relevant skills and opportunities for sustainable employment by youths.

Effectiveness

Finding 5: By combining classroom training with practical application and opportunities for certification,

the LL2E Approach has increased access by HKK women and youths to quality BTVET services. There is

room, however, for further enhancement of trainings to increase individual marketability and improve

livelihood (STEP Nepal, Outcome 1.1).

The LL2E Approach has been instrumental in the progress made by the STEP Nepal project towards

results achievement. Of concern to graduates and key actors in the private sector, however, is the level

of training that is provided, as LL2E project activities provide CTEVT Level I certification, which equates

to basic training.

Finding 6: The need to systematically document and disseminate knowledge and experiences at project-

level may conflict with the underlying objectives of LL2E. There is an equal possibility for misalignment

between the approach to capacity-strengthening for BTVET delivery and the scope of LL2E (STEP Nepal,

Outcomes 1.2; 1.3)

The application of the LL2E Approach to project activities has involved: i) the facilitation of a systematic

process for documenting and disseminating knowledge and experiences to all concerned; and ii)

strengthened capacity of public and private TVET service providers for quality service delivery. If the

objectives of LL2E is to facilitate increased access to quality BTEVT and employment opportunities, there

is a question as to whether project outcomes that aim to build the capacity of TVET institutions in the

country (by national seminar or other modality) are too ambitious in scope for LL2E project activities.

Finding 7: LL2E project activities have equipped Congolese youths with marketable vocational skills and

opportunities for industrial training. Access to opportunities for employment and apprenticeship are

highly dependent, however, on vocational specialisation (Rwamwanja BTVET Project, Outcomes 1.1; 1.2)

v

Youths have gained access to vocational and entrepreneurial skills, industrial training and certification of

training based on the use of LL2E for the implementation of project activities. There was need to re-visit

the project design, however, in response to challenges faced by graduates from certain specialisations in

finding work upon completion of the trainings.

Efficiency

Finding 8: The LL2E Approach is strongly supported by standard procedures for the management of

results, risk and financial resources. Efficient project implementation is contingent, however, on

resource availability.

At the level of project management, the LL2E Approach is built on strong supportive systems for the

management of risk and financial resources. The capacity of the LL2E Approach to facilitate results

achievement is lacking in the area of high project staff capabilities in monitoring and evaluation.

Finding 9: The LL2E Approach provides some value for money (VfM) from a theoretical perspective.

There is potential for increased VfM, however, through greater private sector engagement and

increased capacity-building for implementing agencies.

The LL2E Approach demonstrates some value-for-money, in as far as resources available for LL2E project

implementation have contributed to development results. There is scope to increase LL2E value-for-

money, however, through closer collaboration between individual projects and the private sector, and

the institutional strengthening of the LL2E project implementing agencies.

Cross-cutting theme: Gender

Finding 10: While the application of the LL2E Approach has allowed for the integration of gender

considerations at project-level, participation by young women in the trainings is, at times, challenged by

domestic commitments.

The emphasis placed on gender, through LL2E Approach application, draws on the FCA human rights-

based approach. In spite of the fixation of LL2E on integrating gender considerations into project

activities, participation by young women in the trainings is largely challenged by their family

commitments.

Participation and Ownership

Finding 11: The full national ownership of the LL2E Approach through policy uptake and programme

planning is a work in progress. Discussion and collaboration between government and LL2E project

implementing agencies is ongoing.

Uptake of the LL2E Approach by national policy is indicative of, and of necessity, required for its long-

term integration into social sector investment programming in Nepal and Uganda. The process requires

ongoing efforts at collaboration between government and the project implementing agencies, in the

interest of country ownership of the Approach and its related project activities.

Finding 12: The use of advocacy as a core element of LL2E has been actively pursued as a mechanism for

awareness-raising within project communities. It is debatable, however, whether the potential

contribution of advocacy to LL2E has been fully explored.

vi

Significantly, advocacy has been central to the implementation of LL2E activities in both project

countries, given its role in awareness-raising for BTVET at community-level. While the importance of

policy uptake, and advocacy in support of uptake is noted, the implementation of the LL2E Approach in

Nepal and Uganda has not focused on this activity.

Sustainability

Finding 13: BTVET ‘skilling’ acquired through LL2E lends itself to results sustainability into the medium-

term. Results sustainability into the long-term is conditional, however, on the establishment of support

structures for the marketability of acquired skills

The proposed LL2E programme includes a timeline of six to twelve months of post-graduation support

involving mentorship, coaching, facilitation of networks, etc. The question that emerges though, is

whether the LL2E Approach is able to incorporate supportive mechanisms, to address the post-

graduation needs of graduates who require additional assistance beyond the support timeframe.

Finding 14: Based on initial steps taken during project planning and implementation, there is potential

for the programmatic sustainability of LL2E beyond its funding cycle. The sustainability of the approach

is highly reliant, however, on policy uptake by government and/ or the private sector.

From a financial, policy and practical perspective, the sustainability of the LL2E Approach is highly

dependent on intervention by government and/ or the private sector. While uptake by government and/

or private sector enterprises has the potential to sustain project activities, the re-visiting of wholly

subsidised trainings is a possibility.

Conclusions, Lessons learned and Recommendations

Conclusions

Youths who participated in LL2E project activities in Nepal and Uganda gained BTEVT and inter-personal

skills, and were empowered to become change-makers by setting, pursuing and achieving personal goals

for improved livelihoods. The effectiveness and efficiency of the LL2E Approach into the medium to

long-term, however, requires inter-sectoral collaboration that aligns with the need for the strategic

linking of theoretical and practical learning to workplace earning.

Lessons learned

• The under-estimation of the role of the private sector in LL2E project design and

implementation is counter-productive to outcomes achievement, including the process of

linking learning to earning.

• While it is important to facilitate access to BTEVT training to individuals and communities where

there is considerable lack of marketable skills to secure gainful employment, access to decent

work is determined by the extent of skill marketability.

• The capacity for LL2E to support the achievement of development outcomes is increased

through the strategic alignment of project-level development targets with the objectives of

LL2E.

vii

• Inadequate assessment of the needs of the labour market can lead to labour market saturation

in an identified vocational specialty or situation where there are insufficient support structures

to facilitate employment.

Recommendations

Recommendation 1: There is need to re-visit the role of the private sector in facilitating LL2E, with a

focus on the early engagement of private sector enterprises in the LL2E programmatic cycle, and the re-

working of the partnership for industrial placement. (Related to Finding #2; #4; #5; #9; #11; #12; #13;

#14)

Evaluation findings on the application of the LL2E Approach to development interventions have

indicated that the role of the private sector is being largely underutilised. Importantly, however, private

sector enterprises have emphasised that they have the potential to make a viable contribution to LL2E

implementation, by providing guidance for trainee selection and facilitating on-the-job-training that

responds to existing labour market needs.

Recommendation 2: The LL2E implementing agency should re-visit its approach to advocacy to

incorporate the lobbying of government, or as applicable, private sector enterprises, for LL2E uptake by

government policy or private sector programming. (Related to Finding #11; #12; #14)

Further to the foreseen role of the private sector (Recommendation 1), it would be essential to ensure

alignment between the objectives of the programming and the mandate of the Ministry to which

responsibility for oversight would be transferred. This process has added advantage for the

sustainability of programming and results, into the medium to long-term.

Recommendation 3: In order to better support the participation of young women during LL2E

programming, project implementing agencies should explore the establishment of child-caring facilities.

(Related to Finding #10)

Young women continue to face constraints to their participation in the trainings, which arise from their

domestic responsibilities, and in particular, the need to look after young children. This challenge can be

addressed through the establishment of child-care facilities to accommodate the children of young

mothers who are enrolled/ wish to enrol for training.

Recommendation 4: The project implementing agency, with support from the project donor, where

applicable, should invest in capacity-building at the level of project management, coordination and

implementation to support the establishment of efficient supportive structures for results achievement.

(Related to Finding #6; #7; #8; #9)

To better enable the use of LL2E in managing for development results, it would be worthwhile for the

implementing agencies, in collaboration with the project donor or oversight agency, to invest in

capacity-strengthening activities in: professional development for BTVET instructors; knowledge

exchange through communities of practice; M&E training for project staff; and the assessment of labour

market needs. The combination of trainings/ guidance has the potential to strengthen the capacity of

the implementing agency for enhanced oversight at the level of management, coordination and

implementation; and the achievement of LL2E objectives.

viii

Acronyms

AAFreedHAKK Action for Advancing Freedom of Haliyas, Ex-Kamaiyas and Kamlaris

BTVET Business, Technical, Vocational Education and Training

CBCC Community-based Childcare Centre

CDP Continuous Professional Development

FCA Finn Church Aid

LL2E Linking Learning to Earning

MSC Most Significant Change

n.d. No date

OECD - DAC Organisation for Economic Co-operation and Development – Development Assistance

Committee

PME Planning, Monitoring and Evaluation Coordinator

STEP Nepal Skills Training and Technical Education for Employment Programme in Nepal

ToR Terms of Reference

TVET Technical, Vocational Education and Training

UCEP Underprivileged Children’s Educational Programme

UN United Nations

UNEG United Nations Evaluations Group

UNHCR United Nations High Commissioner for Refugees

ix

Table of Contents

Executive Summary ............................................................................................................ i

1. Introduction ............................................................................................................... 1

2. Description of the Intervention ................................................................................. 3

2.1 The Linking Learning to Earning Approach (LL2E) ...................................................................... 3

3. Methodology .............................................................................................................. 5

3.1 Overview .................................................................................................................................... 5

3.2 Participatory Approach .............................................................................................................. 5

3.2.1 Utilisation-focused Evaluation Principles ........................................................................ 6

3.2.2 Mixed Methods ................................................................................................................ 6

3.3 Evaluation Matrix ....................................................................................................................... 7

3.4 Sampling ..................................................................................................................................... 8

3.5 Methods ................................................................................................................................... 11

3.6 Limitations ................................................................................................................................ 14

4. Main Findings ........................................................................................................... 15

4.1 Overview .................................................................................................................................. 15

4.2 Impact ....................................................................................................................................... 15

4.3 Relevance ................................................................................................................................. 17

4.4 Effectiveness............................................................................................................................. 19

4.5 Efficiency .................................................................................................................................. 21

4.6 Cross-cutting theme: Gender ................................................................................................... 26

4.7 Participation and Ownership .................................................................................................... 27

4.8 Sustainability ............................................................................................................................ 28

5. Conclusions, Lessons learned and Recommendations ............................................ 31

5.1 Conclusions............................................................................................................................... 31

5.2 Lessons learned ........................................................................................................................ 31

5.3 Recommendations ................................................................................................................... 32

x

Exhibits

Exhibit 3.1 Evaluation issues ................................................................................................... 7

Exhibit 3.2 Sampling criteria ................................................................................................... 9

Exhibit 4.1 Assessment of LL2E value-for-money ................................................................. 24

List of Appendices

Appendix I Terms of Reference ...................................................................................... 39

Appendix II Evaluation Matrix ........................................................................................ 45

Appendix III List of Key Informants Consulted ............................................................... 81

Appendix IV Fieldwork Itinerary ..................................................................................... 87

Appendix V List of Documents Reviewed ....................................................................... 89

Appendix VI Data Collection Tools/ Protocols................................................................ 91

Appendix VII Map of Findings and Recommendations ................................................ 112

1

1. Introduction

This Final Evaluation Report is hereby presented to Finn Church Aid (FCA) by Ms Halcyon Louis,

international development consultant, for the contracted consultancy assignment: Impact Assessment –

Linking Learning to Earning Approach (LL2E). It responds to the terms and reference (ToR) for the

impact assessment, as commissioned by signed contractual agreement on October 3, 2017.

As its main objective, the consultancy entailed conducting an assessment and analysis (hereafter

referred to as an evaluation) of the long-term outcomes and impact of the FCA LL2E Approach for rights-

holders. The scope of the evaluation covered LL2E activities implemented from 2015 to 2017 in two

countries, namely, Nepal and Uganda. A key element of the evaluation process was the identification of

factors that contributed to successes and challenges in each country, and as applicable, within each

community context.

In accordance with the ToR, the evaluation of the LL2E Approach involved:

1) Assessing the long-term impact of LL2E activities in each country;

2) Analysing the results in different contexts to draw lessons learnt;

3) Analysing different methods used in LL2E, to determine what has worked well and which

methods are in need of strengthening;

4) Documenting gaps in the current TVET programming and suggesting remedies;

5) Developing recommendations for future LL2E programming; and

6) Examining gender aspects, where relevant.

The purpose of the evaluation, therefore, was to explore to what extent, and how, LL2E activities in

Nepal and Uganda brought about changes in BTVET quality and the lives of rights-holders, and created

actual linkages with employment and/ or entrepreneurship.

The evaluation report is structured as follows:

• Section 1 introduces the overall evaluation objective and purpose;

• Section 2 describes the intervention;

• Section 3 outlines the client-approved evaluation methodology;

• Section 4 is a discussion of the main findings of the evaluation;

• Section 5 presents the evaluation conclusions, lessons learned and recommendations; and

2

• Appendices I to VII present supportive material that expand on the main report content:

– Appendix I Terms of reference;

– Appendix II Evaluation matrix;

– Appendix III List of Key informants consulted;

– Appendix IV Fieldwork itinerary;

– Appendix V List of documents reviewed;

– Appendix VI Data collection tools/ protocols; and

– Appendix VII Map of findings and recommendations.

3

2. Description of the Intervention

2.1 The Linking Learning to Earning Approach (LL2E)

FCA launched the LL2E Approach in 2013, following its decade-long experience in business technical

vocational education and training (BTVET), and small business development, across several developing

nations.1 The LL2E Approach uses market analyses and partnerships with the private sector, to facilitate

BTVET delivery for youth from vulnerable communities, and establish effective linkages between

trainings and the world of work. This approach is designed primarily for youth and women from the

target communities, to provide them with an accessible skill set that is relevant to labour market needs.

The intention is to increase the opportunity for skilled graduates of the training to earn a decent living

through wage employment or entrepreneurship.

Of importance, the approach aligns with two strategic priority areas of the FCA operational portfolio2:

namely, the Right to Quality Education (R2QE); and the Right to Livelihood (R2L). The R2QE strategy

focuses on the design and implementation of innovative solutions for education sector development,

with emphasis on the delivery of quality education in fragile contexts. In devising practical ways to

facilitate youth accessibility to quality vocational education, the R2QE strategic approach further entails

establishing links between vocational education and the job market in support of employment creation.

One of the main elements of this process is teacher education, given the role of teachers as the most

important resource for improving education. The R2L strategy, by extension, is geared towards

promoting sustainable livelihoods by developing resilient communities in situations where communities

are affected and created by migration in fragile contexts. R2L focuses, therefore, on facilitating

emergency livelihood recovery, entrepreneurship and employment through the linking of learning and

earning.

LL2E is learner-centred, practical and flexible, allowing for the tailoring of training activities to the

situational context of each learner community, to yield maximum benefit. In order to increase the

interplay between skills, vocational education and employment, trainings are based on the use of

blended modules, comprising:

1) Competence -based curriculum and qualification systems;

2) Practical application of learnings; and

3) On-the job training.

1 BTVET activities, in particular, were first started in three countries: Congo (DRC); Sierra Leone; and Uganda.

2 FCA operations include long-term development cooperation; humanitarian assistance; and advocacy.

4

As a result, close and continual partnership with private sector enterprises throughout the training

lifecycle, is central to the implementation of LL2E project activities. Post-training support, in the form of

career counselling, is further integrated into the core components of LL2E, to provide trained graduates

with job-seeking or entrepreneurial skills, and linkages to key labour market actors of relevance to their

vocational specialty.

Based on a funded timeframe of January 2015 to December 2017, elements of the LL2E Approach have

been systematically tested in the two countries of focus for this assignment, Nepal and Uganda. In

Nepal, the focal project for the consultancy, the Skills Training and Technical Education for Employment

Programme (STEP) Nepal was designed to build the technical, vocational and entrepreneurial skills of

youths who complete the trainings, as well as provide follow-up support for employment, where

applicable. STEP Nepal has aligned directly with the FCA R2QE and R2L, given the project focus on the

strengthening of TVET, to create access by women and youths to quality vocational skills training and

technical education. In Uganda, the LL2E Approach has been used to develop and implement the

Rwamwanja BTVET Project, which has aimed to improve the standard of living for Congolese refugee

youth and youth from the host community, by enabling youth to access marketable vocational skills.

Project activities in Nepal have been implemented through a partnership between FCA and a local non-

governmental organisation, Underprivileged Children’s Educational Programme (UCEP), Nepal. In

Uganda, the Rwamwanja BTVET Project has been self-implemented by the FCA Uganda Country Office.

As a follow-up to the testing of project component in both countries, as well as developing nations, FCA

has developed an indicative Theory of Change for the LL2E Approach, which covers the entire

programmatic cycle, from quality BTVET to gainful employment.3 In addition to providing core BTVET

elements, namely trainings, on-the-job training and career counselling, the proposed LL2E programme

will include training for BTVET instructors (Training of Trainers) and post-graduation support from the

private sector for wage employment and entrepreneurship.

3 The LL2E indicative Theory of Change is included in the evaluation ToR; see Appendix I, p. 40 of the current report.

5

3. Methodology

3.1 Overview

The technical approach to the consultancy was participatory, and was supported by the use of

utilisation-focused principles and mixed methods. All evaluation activity was also guided by a client-

approved, evaluation matrix. ToR specifications requested an impact assessment to examine the ‘what’

and ‘how’ of LL2E activity management and implementation in Nepal and Uganda. This was interpreted

by the consultant as a need to identify the extent to which the LL2E Approach facilitated results

achievement at the level of project outcomes, and as well, the need to evaluate the viability of the

project management and implementation process. In lieu of an impact assessment, therefore, which

concentrates on outcomes achievement only, the piloted components the LL2E Approach were

subjected to a comprehensive evaluation that incorporated summative (backward-looking) and

formative (forward-looking) elements. The core elements of the proposed approach are described

further in sub-sections 3.2 to 3.5.

3.2 Participatory Approach

In line with the ToR, the evaluation focused on assessing and documenting project performance based

on the application of the LL2E Approach, and the capturing of key learnings. As a result, it is anticipated

that the main evaluation findings will inform decision-making; improve knowledge; and account for

results achieved. The approach to the evaluation was, therefore, participatory, which was appropriate

on two levels. First, evaluations that use a participatory approach to data collection and analysis are

pragmatic. This approach allows evaluators to identify the key categories of project partners/

stakeholders/ key informants, who have been involved in project activities, to seek their input for the

design and implementation of evaluation activities. The contribution of key informants increases the

possibility for better quality data; a better understanding of collated data; more appropriate

recommendations; and a better uptake of evaluation findings.4 Second, from an ethical perspective, key

informants have a right to be involved in decision-making that affects them.5 By extension, therefore,

the client approved methodology involved applying a participatory approach to the execution of an

evaluation that is useful; ethical; and of high quality.

4 Guijit, I, 2014. Participatory Approaches. Florence: UNICEF.

5 United Nations, 2003. The Human Rights-based Approach to Development Cooperation: Towards a Common Understanding

among UN Agencies. Available at: http://hrbaportal.org/the-human-rights-based-approach-to-development-cooperation-

towards-a-common-understanding-among-un-agencies [Accessed online 1 November 2016].

6

3.2.1 Utilisation-focused Evaluation Principles

The use of utilisation-focused evaluation principles complements the participatory approach, as it is

based on the understanding that the evaluation should be useful to its end-users6. Utilisation-focused

evaluation principles use participatory methods to determine what the end-users want from the

evaluation, and thereby support ownership of the evaluation by its intended users. To ensure that the

evaluation is owned by its intended end-users, all categories of project stakeholders who were involved

in LL2E activities were engaged during the implementation of the evaluation. The intention, here, was to

capture the unique experience and perspectives of all project stakeholders, who were recognised as key

informants on the effects and workings of the LL2E Approach. As a result, consultations were held with

key informants from the following categories: BTVET providers (the management and staff); LL2E

graduates/ rights-holders (from at least 12 months, but not more than 36 months); relevant private

sector representatives (employers; businesses; and their associations); government authorities and

other relevant education authorities engaged in BTVET (e.g. camp management; and UNHCR); FCA staff

(based at FCA Head Office and Country Offices); and other key informants, as relevant. By incorporating

utilisation-focused evaluation principles, the evaluation was conducted for and by its intended end

users, the project stakeholders/ key informants, to ensure that their expressed needs are met. The

evaluation maintained its independence and impartiality, however, as the consultant was responsible

for the final analysis of data; the interpretation of results; and the synthesis of findings.

3.2.2 Mixed Methods

Mixed-methods, entailing qualitative and quantitative techniques, were used to structure data

collection and analysis during the evaluation. The use of mixed methods responded to the evaluation

ToR, including the key evaluation questions, which suggested that data analysis would generate

qualitative and quantitative results. Mixed-methods were used, where applicable, to ensure that: i) the

methodological techniques were suitable for collecting and analysing the type of data that was required;

ii) data was triangulated from multiple sources, to increase the credibility of the results generated

during data analysis, to inform the development of the main evaluation findings; and iii) the results of

data analysis can be used to identify lessons and develop recommendations that can be used to inform

future project activities, as applicable.7 (The methods of data collection and analysis are described under

Section 3.5: Methods).

6 Patton, M.Q., 2010. Utilisation-focused evaluation. 4th edition. Thousand Oaks, California: Sage Publications.

7 Nagy- Hesse-Biber, S., 2010. Mixed Methods Research: Merging Theory with Practice. London: The Guilford Press.

7

3.3 Evaluation Matrix

An evaluation matrix was developed to articulate the main evaluation issues, including: the key

evaluation questions and sub-questions; performance indicators; and means of verification. In

accordance with the ToR, including the specific evaluation questions that were suggested,8 the main

evaluation issues were categorised under seven dimensions, which were aligned to OECD-DAC9

definitions; the Paris Declaration, and the Accra Agenda for Action.10 Based on the emphasis placed on

learning within the ToR, as well as the formative element of the evaluation, two additional dimensions

were incorporated into the evaluation matrix, namely lessons learned; and recommendations. The

evaluation matrix was finalised in collaboration with FCA during the Inception Phase, and was included

in the final Inception Report and Work Plan, as a guide for all evaluation activity. Exhibit 4.1 provides an

overview of the main evaluation issues/ dimensions, and the client-approved evaluation matrix is

appended at Appendix II.

Exhibit 3.1 Evaluation issues

Evaluation Dimension OECD DAC Definitions

1. Impact Positive and negative, primary and secondary long-term effects produced by a

development intervention, directly or indirectly, intended or unintended

2. Relevance The extent to which the objectives of a development intervention are consistent

with beneficiaries’ requirements, country needs, global priorities and partners’

and donors’ policies

3. Effectiveness The extent to which the development intervention’s objectives were achieved, or

are expected to be achieved, taking into account their relative importance

4. Efficiency A measure of how economically resources/inputs (funds, expertise, time, etc.)

are converted to results

5. Participation and

Ownership

Developing countries set their own strategies for poverty reduction, improve

their institutions and tackle corruption

Countries have more say over their development processes through wider

participation in development policy formulation, stronger leadership on aid co-

ordination and more use of country systems for aid delivery

8 ToR, pp. 3-4

9 OECD-DAC – Organisation for Economic Co-operation and Development – Development Assistance Committee

10 OECD-DAC develops guidelines and reference documents to inform the execution of development programmes. The 2005 Paris

Declaration was endorsed by donor and recipient governments during the Second High-level Forum on Aid Effectiveness, and is

centred around the core principles of: ownership; alignment; harmonisation; managing for development results; and mutual

accountability. The 2008 Accra Agenda for Action was established to accelerate the achievement of measurable targets of the

Paris Declaration.

8

Evaluation Dimension OECD DAC Definitions

6. Sustainability The continuation of benefits from a development intervention after major

development assistance has been completed

The probability of continued long-term benefits; the resilience to risk of the net

benefit flows over time

7. Cross-cutting theme:

Gender

The process of assessing the implications for women and men of any planned

action, including legislation, policies or programmes, in any area and at all levels;

it is a strategy for making women’s as well as men’s concerns and experiences an

integral dimension in the design, implementation, monitoring and evaluation of

policies and programmes in all political, economic and social spheres

8. Lessons learned Generalisations based on evaluation experiences with projects, programs, or

policies that abstract from the specific circumstances to broader situations;

frequently, lessons highlight strengths or weaknesses in preparation, design, and

implementation that affect performance, outcome, and impact

9. Recommendations Proposals aimed at enhancing the effectiveness, quality, or efficiency of a

development intervention; at redesigning the objectives; and/or at the

reallocation of resources.

Source: OECD DAC Glossary of Key Terms in Evaluation and Results-based Management (#1 -4; #6; #8; #9); 2005

Paris Declaration, and 2008 Accra Agenda for Action (#5); UN Women (2014), Gender mainstreaming in

development programming (#7)

3.4 Sampling

FCA has applied and tested LL2E activities in several projects in Nepal and Uganda over the past four

years. As such, the ToR for the consultancy noted the large scale of the assessment/ evaluation, and the

need for proper sampling to ensure quality assessment. Given this context and requirement, purposeful

sampling based on a sequential approach was used to select target groups and activities for inclusion in

data collection activity. The rationale for this approach was two-fold. First, one of the main limitations to

the evaluation was resource scarcity (human; time; financial) which acted as a constraint to the

engagement of all persons who were involved in LL2E activities in the focus countries. This limitation

was by no means unique to the evaluation, however, as it is a common challenge to evaluations.

Second, in order to achieve the level of rigour required for a robust evaluation process, purposeful

sampling was structured around the main evaluation issues, to support the generation of accurate

responses to the key evaluation questions. The application of a sequential approach had the further

advantage of allowing data collection to evolve in tandem with emergent findings, by creating the

flexibility for additional data collection at any given stage of the evaluation.

9

Of note, purposeful sampling does not permit generalisation to a population. The purposeful sampling

approach mitigated this limitation, however, by placing the evaluation questions at the forefront of the

sampling process. As the aim of this evaluation did not centre on establishing generalisations that are

based on statistical significance, purposeful sampling was used to focus on pre-defined, strategic criteria

and categories, to increase the quality and accuracy of responses that were acquired on the key

evaluation questions. The anticipated result of this process was an evaluation of high quality that would

be useful for the identified end users.

In light of this context, Exhibit 4.3 outlines the objective criteria that were used to engage a sample of

LL2E project stakeholders in each project country. This is followed by an explanation of the underlying

rationale.

Exhibit 3.2 Sampling criteria

Project

Country Stakeholder Category Sampling Criteria

Nepal Rights-holders • Consultations held with rights-holders from the following

categories

Male graduates under the age of 18 (16 – 17 years)

Female graduates under the age of 18 (16 – 17 years)

Male graduates between the ages of 18 and 22 years,

inclusive

Female graduates between the ages of 18 and 22 years,

inclusive

Male graduates over the age of 22 (23 – 29 years)

Female graduates over the age of 22 (23 – 29 years)

• Priority selection for all age groups targeted:

Graduates from urban areas (Kathmandu and environs)

Graduates from remote and inaccessible areas

Graduates with physical challenges

Graduates whose participation in project activities was

prioritised (youths from very poor families; youths facing

social and family problems)

Non-graduates who dropped out of a project

BTVET providers • Consultations pursued with representatives of BTVET

providers in rural and urban settings

Private sector representatives • Consultations pursued with representatives of private sector

agencies in rural and urban settings

10

Project

Country Stakeholder Category Sampling Criteria

Government authorities/

Education Authorities

• Consultations pursued with representatives of government

authorities/ education authorities involved in LL2E project

activities at the local/ national level, and/ or in rural and

urban settings

UCEP/ FCA Nepal • Consultations pursued with all staff members who were

involved in LL2E Project activities

Country Manager

Programme Coordinator/ Partner liaison - FCA Nepal

Country Office

Programme Officer

Accountant

Uganda Rights-holders • Consultations held with rights-holders within the following

categories

Male graduates under the age of 18 (Youths 14 – 17

years)

Female graduates under the age of 18 (Youths 14 – 17

years)

Male graduates over the age of 18 (18 – 25 years)

Female Youths over the age of 18 (18 – 25 years)

Non-graduates who dropped out of the project

BTVET providers • Consultations pursued with representatives of BTVET

providers

Private sector representatives • Consultations pursued with representatives of private sector

agencies

Government authorities/

Education Authorities

• Consultations pursued with representatives of government

authorities/ education authorities involved in LL2E project

activities

FCA Staff • Consultations pursued with all staff members who were

involved in LL2E Project activities

Country Manager

Programme Coordinator

Programme Officer

Regional Education Adviser

Accountant

11

Drawing on the purposeful approach to sampling, consultations were actively pursued with all

categories of stakeholders who were involved in LL2E project activities. This approach was designed to

generate accurate responses to the main evaluation questions, as well as support data triangulation.

With particular reference to the project rights-holders, criteria for sampling were based on the unique

features of LL2E project activities in Nepal and Uganda. To illustrate, in Nepal, project activities targeted

youths aged 17 to 29 years, and youths aged 14 to 25 years were targeted in Uganda. To create greater

uniformity in the sampling of rights-holders, the selection criteria in both countries were further

informed by:

1) The criteria that were used to select trainees for LL2E projects in each country;

2) The need to incorporate gender considerations into the data collection process; and

3) The anticipated positive effect of group consultations that engage persons who are closer in

age.

3.5 Methods

A detailed overview of the proposed evaluation methodology is presented in this sub-Section. The

evaluation was implemented over five distinct, but overlapping phases: Phase 1: Inception; Phase 2:

Data collection; Phase 3: Data analysis and synthesis; Phase 4: Reporting; and Phase 5: Assignment

management.

Phase 1: Inception (5.5 days)

The Inception Phase of the evaluation commenced with an initial start-up discussion, and follow-up

exchanges between the consultant and FCA, to finalise contractual details and launch the evaluation

process. This discussion was also used by the consultant to acquire a greater understanding of the

evaluation context and client expectations. To support this process, the consultant requested

background documentation on the LL2E approach from FCA, including documents on activities

implemented in the two countries of focus, for preliminary review. The main deliverable for this stage of

the evaluation was the Inception Report and Work Plan, a draft version of which was submitted to FCA

for review and comment. Feedback that was received informed the finalisation of the document, which

was used as the client-approved guideline for the evaluation. The final Inception Report and Work Plan

included the approved evaluation methodology, comprising a description of the technical approach and

methods; the evaluation matrix; the sampling strategy and data collection tools/ protocols.

Phase 2: Data Collection (16.5 days)

Data collection involved the review of documents; stakeholder consultations (comprising interviews and

focus group discussions), and field visits during country missions to Nepal and Uganda. An in-depth desk

review of relevant documents was guided by the key questions and sub-questions of the approved

evaluation matrix. Consultations with key informants were conducted by interview (in-person; Skype)

and focus group discussion, using data collection protocols that were tailored to each category of key

informant. The selection of key informants and project activity sites, as potential sources of data, was

based on purposeful sampling using a sequential approach. As indicated in Section 4.4, this approach

12

allowed for the identification and selection of additional key informants and sites throughout the

evaluation, for follow-up data collection, as applicable.

In order to document the perceived impact of LL2E activities on rights-holders in Nepal and Uganda, the

consultant used a variation of the Most Significant Change (MSC) technique, to engage this category of

key informants in a reflection on their involvement in project activities. The MSC technique has been

used during development programming to isolate the main changes that have resulted from an

intervention.11 The application of this technique takes time, however, as it involves a thorough review of

project activities during interactive reflection among rights-holders, to determine which effect(s) of the

project can be considered as the most profound. Given the short timeframe for data collection during

the evaluation, rights holders in each country were engaged in a participatory, interactive session.

During each session, they were asked to re-trace their ‘steps’ to situations that existed prior to their

involvement in LL2E activities, and then recount the situations that could be observed after their

graduation. This process was used to guide rights-holders towards identifying the most significant

changes generated by LL2E, from an individual, and as applicable, a group standpoint. The proceeds of

this exercise were used to develop case study vignettes for inclusion in the evaluation report.12 Of

significance, in anticipation of some difficulty in convening rights-holders in a central location, for

example if they moved away from the host training community, focus groups were used, where

possible, to engage small groups of graduates. As an alternative, individual graduates were engaged

using interviews. In both instances, the variation to the MSC technique was applied, and the support of

translators was acquired, as necessary.

Phase 3 – Data Analysis and Synthesis (5 days)

The analysis of data was aligned to the approved technical approach to the evaluation, and complied

with the international evaluation standards of the OECD-DAC, and the United Nations Evaluation Group

(UNEG). Data validity was assured through cross-referencing and triangulation from multiple data

sources. In order to purposefully influence analysis through triangulation and produce credible

evaluation findings, the following methods of analysis were used:

• Descriptive analysis of LL2E to understand and describe its main components, including

activities, partnerships, modalities of delivery, etc. Descriptive analysis was used as a first step

that preceded more interpretative approaches during the evaluation;

• Content analysis of documents, and notes arising from interviews and focus group discussions

with key informants, to identify common trends and themes, and patterns for each of the key

evaluation issues (as the main units of analysis). Content analysis was also used to flag diverging

views and opposite trends, and determine whether there might be need for additional data

collection. Emerging issues and trends were synthesised to inform each stage of the reporting

process (validation; draft and final evaluation reports);

11 Davies, R. and Dart, J. 2005. The 'Most Significant Change' Technique - A Guide to Its Use,

http://www.mande.co.uk/docs/MSCGuide.pdf

12 Case study vignettes will be described under Data Analysis and Synthesis and will be included in the Final Report

13

• Quantitative analysis of data on project achievement of quantitative targets, and resource use

during project design and implementation was conducted where applicable. Simplified analyses

were conducted using spreadsheet software (Excel), to generate summary statistics. The

statistics that were generated were further used to develop emergent findings and inform

comparative analysis; and

• Comparative analysis was used to examine findings across emerging themes and identify good

practice, innovative approaches and lessons learned, where applicable. Information was

organised according to hypotheses generated. Emerging from this process were the main

evaluation findings, as well as case study vignettes that documented examples of the impact of

LL2E activities on the lives of the rights-holders. The case study vignette is a short, descriptive,

summary example of the effects of project implementation.13 It can be used to provide a

detailed description of the most significant changes that have resulted from the project.

Vignettes will vary in length and detail according to the specific example and the amount of data

available. They are not intended to be aggregated, or consolidated into an account of overall

impact. Rather, they are intended to provide rich qualitative data on selected examples in given

settings.

In line with the use of utilisation-focused principles, the emergent evaluation findings were shared with

key informants in Nepal and Uganda, and with FCA Head Office, in the interest of validation.

Consolidated feedback that was received was used to further inform evaluation reporting.

Phase 4 – Reporting (19 days)

As indicated, Phase 3 of the evaluation included reporting on preliminary observations and preliminary

evaluation findings to key informants in each country of focus, and FCA Head Office during validation

sessions. Feedback from the discussions were used to initiate further data collection, where required,

and inform the development of the draft evaluation report. The draft report was submitted to the PME

Coordinator of FCA, for review and feedback, for use in report finalisation. Evaluation reporting involved

the use of jargon-free language and followed all guidelines specified in the ToR.

Phase 5 – Assignment management (1 day)

Assignment management occurred throughout the evaluation, and adhered to the quality assurance

procedures that have been established by OECD-DAC and UNEG. This phase of the evaluation entailed

timely implementation of all evaluation activity, and the provision of informal updates on the progress

of the consultancy to FCA at scheduled intervals. In addition, any emergent incident that had bearing on

the evaluation was reported to FCA immediately.

13 Patton, M. 2001. Qualitative Research and Evaluation Methods. California: Sage Publications.

14

3.6 Limitations

There were two main limitations to the evaluation. At the level of project design, the evaluation was

challenged by resource scarcity of, in terms of the availability of human and financial resources for

evaluation activity, and the timeframe that was contractually designated for the same. As indicated in

Section 3.4, purposeful sampling based on a sequential approach was used to mitigate this limitation, to

ensure that the evaluation was structured around the key evaluation questions, for greater accuracy in

the responses generated. This approach further allowed for increased flexibility in the engagement of

key informants, as it allowed for the inclusion of relevant persons at any stage of the evaluation process.

At the level of evaluation implementation, data collection was challenged by the unavailability of some

stakeholders for consultation. Of particular note, the evaluation coincided with a two-week sessional

break of BTVET classes in Uganda, which was introduced into the programming to allow trainees to

prepare for module examination by the Directorate of Industrial Training. As a result of this sessional

break, BTVET instructors were also on leave from the training facility. In both Nepal and Uganda, efforts

to engage youths who had dropped out of the training were also unsuccessful. By way of mitigation,

consultations were held with key informants who were accessible during the evaluation. In the interest

of response accuracy, document review was also used to triangulate data collected by key informant

interview.

15

4. Main Findings

4.1 Overview

The main findings of the evaluation were generated from the interpretation of the results of data

analysis. In line with the utilisation-focused approach to the evaluation, emergent preliminary

observations from data collection and initial analysis were shared during debriefing sessions at country-

level to support validation. The participants at these sessions were key informants who were

responsible for the implementation of LL2E project activities, namely UCEP, in collaboration with FCA

Country Office Nepal; and FCA Country Office Uganda. Country-level validation was followed by a

debriefing and validation session with FCA Head Office, which was conducted by Skype. Feedback

received from the key informants was used to convert country-specific evaluation observations on the

LL2E Approach into the main evaluation findings, as aligned to the core evaluation issues. The main

evaluation findings are presented in sub-sections 4.2 through 4.8.

4.2 Impact

Finding 1: The LL2E Approach has contributed strongly to intended behavioural and attitudinal change among graduates of LL2E project activities. Central to this development has been the facilitation of access by the target groups of youths to BTVET opportunities and follow-up support.

By creating increased opportunities for youths from vulnerable and socially marginalised communities,

to access opportunities for skill-building and decent work, the LL2E Approach has contributed, in great

measure, to changes in behaviour and attitudes among the target group of young persons. Self-

expressed reporting by trained graduates, on the effect of the training on their lives, has emphasised

youth engagement in the re-prioritisation of personal goals to improve the standard of living of their

respective households. In general, youths have gained access to vocational skills that they can use to

secure a safe and reliable source of income, through wage employment or small-business start-up. The

future livelihood prospects for trained youths have also been increased through certification in their

vocational specialty based on national standards of quality.

Beyond the acquisition of new learnings and levels of certified competence, of relevance here is the

contextual importance of the trainings and follow-up support for the target category of youths.

Specifically, the LL2E has facilitated access to improved livelihoods to youths who have had limited

opportunities, pre-training, to pursue or continue their education. To illustrate, LL2E project activities in

Nepal under the STEP Nepal Project have engaged youths from the Far West Nepal, who are members of

the Halaiya, Kamaiya and Kamlari (HKK) communities. The Far West of Nepal is a remote region, in which

wage and self-employment prospects for unskilled persons are limited. In addition, the HKK are former

bonded labourers, whose history includes labour exploitation; lack of land ownership; and engagement

in risky and/ or seasonal work as low-paid migrants to India. In Uganda, the target group for LL2E has

16

largely been Congolese youth refugees14, who face several challenges to social integration, including:

unfamiliarity with the official language of Uganda; lack of proof of prior certification and schooling; and

post-traumatic stress from their experience with conflict in their home-country.

Importantly, the flexibility of the LL2E Approach, denoted by the tailoring of LL2E project activities to the

situational context of the target community, has enhanced the capacity for youths to benefit from the

trainings. Of note, trainings have been refined to:

• Address the challenge of distance, between the vocational centre and the communities where

trainees live, through mobile trainings in Nepal and community-based trainings in Uganda;

• Provide a business-skills component for post-training guidance on job-applications; customer

service skills; etc.;

• Include tracer studies on graduates and facilitate post-training support to graduates who are

unemployed or in need of other forms of support (e.g. additional training; access to networks;

etc.)

• It is unclear, however, whether follow-up support is provided within a given timeframe (e.g. up

to six months post-training) or whether the LL2E Approach advocates support to graduates into

the long-term upon request or based on systematic follow-up.

Finding 2: Notwithstanding the ‘trickle-down effect’ of the LL2E Approach on the families and communities of LL2E project graduates, the intended impact of forging effective linkages with the private sector is a work in progress.

As a direct result of facilitating increased livelihood opportunities through improved skillsets, the LL2E

Approach can be seen to generate a ‘trickle-down effect’ on the communities where youths live as they

apply their newly-acquired skills. Communities have benefited from graduates of LL2E trainings by

gaining access to:

• New services from graduates who choose to establish their own business;

• New employees, arising from the hiring of graduates who pursue wage employment; and

• Quality service delivery from graduates who receive certification upon graduating from the

trainings.

In addition, included among the communal gains generated by LL2E are the effects of the trainings on

the households of trained graduates. Consultations with LL2E graduates have shown that their families

benefit from financial and material support, which in combination contribute to an improved standard

of living within the households.

14 In accordance with regulations established by the Government of Uganda, the trainings also engaged youths from the host

community

17

Of interest, the programmatic structure of LL2E involves close collaboration with the private sector to

facilitate job placement/ industrial training and employment upon graduation. Document review and

consultations with project staff and training graduates have, however, identified instances of

unemployment or low-wage employment among graduates. Several factors for consideration emerge in

response:

4) The pre-training assessment of labour market needs was inadequate, relative to the

consultation of labour market actors for an accurate determination of needs;

5) Youths harbour unrealistic expectations of their marketability immediately upon graduation,

and therefore expect higher wages;

6) Collaboration between LL2E and the private sector is inadequate, in relation to the creation of

strong linkages between skills; training and the world of work.

It is submitted, here, that items #1 and #2 above can be subsumed under #3, inadequate collaboration

with the private sector.

Consultations with key informants within the private sector indicated a willingness by enterprises to

support the implementation of the LL2E Approach from the point of trainee selection, through to-job

training and post-training employment or support for small business start-up. Of note, LL2E

implementing agencies have taken steps to engage the private sector during the application of the LL2E

Approach to project activities. As examples, Prospective Employers’ Meetings (Nepal) and partnerships

with private sector enterprises (Uganda) were used to facilitate on-the-job placement/ industrial

training within private sector enterprises, as well as increase understanding by trainees and

implementing agencies alike, of the main labour market needs. Importantly, however, there is evidence

of inadequate use of supportive resources that may be accessible through the private sector in both

countries. On one hand, enterprises in Nepal are not engaged at the point of trainee selection, during

which information can be shared on labour market needs and the availability of ongoing support for

trainees. On the other hand, partnerships with enterprises in Uganda are exclusive in their

specifications, focusing, for example, on industrial training only, as opposed to encouraging job creation

for graduates who have completed their industrial training.

4.3 Relevance

Finding 3: Given its aim to link vulnerable young persons to opportunities for improved livelihoods, through BTVET and decent work, the LL2E Approach has strategic global- and country-level relevance, to complement its alignment with the FCA institutional mandate.

Of necessity, the LL2E Approach is informed by the vision and mission of FCA, which entails facilitating

human dignity for people in vulnerable situations through the use of a rights-based approach. LL2E

further directly aligns with two FCA strategic priorities, namely, R2QE and the R2L. In light of this

strategic focus, LL2E can be positioned within the 2030 Agenda for Sustainable Development

(Sustainable Development Goals – SDGs), which includes among its global goals, the need to end hunger

and poverty (thereby supporting the implementation of livelihood strategies); and the need to ensure

inclusive and equitable quality education (with support, by implication, for BTVET delivery). In addition,

18

the SDGs advocate gender equality, including the empowerment of women and girls, which aligns with

the primary targeting of women and youth by LL2E programming.

By adopting the 2030 Agenda, developing nations have committed to cross-sectoral development,

including social sector investment that supports a sustainable future. National plans for sustainable

social sector development in Nepal and Uganda, with emphasis on education and livelihoods, are

particularly noted in:

• The 2007 TVET Skill Development Policy 2064 (Nepal), which has emphasised the need to

expand training access to women and vulnerable social groups;

• The 2002 Bonded Labour (Prohibition) Act (Nepal), which emancipated all labourers bonded

under the HKK system and included provisions for their rehabilitation;

• The 2011 – 2020 BTVET Strategic Plan (Uganda), which advances the need for employable skills

and competencies that are relevant to the labour market; and

• The 2006 Refugee Act, and 2010 Refugee Regulations (Uganda) which allow for the

accommodation and integration of refugees into Ugandan society (including access to

education; healthcare; land; and employment)

Overall, both LL2E project countries have taken steps towards the strategic elimination of poverty and

the promotion of human development.

Finding 4: From a conceptual standpoint, the proposed LL2E programmatic design responds to the situation of unemployed youth from vulnerable/ marginalised communities. Piloted components have not, however, established adequate linkages between the trainings and the private sector for the employment of graduates/ ex-trainees.

The LL2E Approach has been designed to address the unique circumstances of the target groups of

youths in each project country. As indicated in Finding 1, the provision of training based on LL2E has

been tailored to resolve some of the challenges faced by trainees, which impede their participation in

the training programmes. Of added importance, the proposed conceptual design of the full LL2E

programmatic cycle is informed by the need for skill-building, on-the-job-training, and career

counselling, including post-training support, for youths from vulnerable communities who have not

previously had access to these facilities. As discussed under Finding 2, however, evidence points to the

under-utilisation of private sector partnerships/ collaboration, to support the acquisition of relevant

skills and opportunities for sustainable employment (including self-employment) by the targeted

category of youths.

Consultations with the private sector have underscored the possibility for trainees to gain access to

mentorship; employment; and seed capital for small business start-up, if private sector collaboration

were incorporated at an earlier stage of the LL2E programmatic cycle. Notably, the proposed approach

to LL2E (the full LL2E model/ programme) will include an extended period of post-training support,

which will engage private sector enterprises for a period of six to 12 months. The model will further

include basic training for workplace instructors. It is unclear, however, whether key actors from the

19

private sector will be engaged during the early stages of the programme, in particular, during trainee

selection.

Significantly, increased collaboration with the private sector, prior to and during training activities, has

the potential to effectively address criticisms of the inadequacy of skills acquired by youths from the

trainings. The major issue here has not been the quality of the trainings, including the quality of training

equipment and tools, but rather the level of training acquired as of course completion, and the

relevance of the training curriculum to labour market needs. As an illustration, both employers and

training graduates in Nepal have indicated that skills acquired through the BTEVT training provide

Level 1 certification, which corresponds to basic capabilities in each vocational area. Consultations with

graduates cite the need for more advanced skills that are acquired through Level 2 training, to equip

them with more marketable skills and greater competence in their areas of speciality, both of which are

useful for wage employment and small-business start-up. Employers have confirmed that Level 2

certification would support the competence of graduates in relation to the completion of workplace

tasks. In Uganda, similarly, the issue was not one of training quality, but rather, a lack of alignment with

the trainings that were provided in some areas (namely, agriculture15; catering and cookery; and

motorcycle and bicycle repair) and labour market requirements. Specifically, the capacity of the labour

market to absorb graduates in these specialities as wage employees was low, which made it difficult for

these graduates to find employment.

4.4 Effectiveness

Finding 5: By combining classroom training with practical application and opportunities for certification, the LL2E Approach has increased access by HKK women and youths to quality BTVET services. There is room, however, for further enhancement of trainings to increase individual marketability and improve livelihood (STEP Nepal, Outcome 1.1).

At the level of outcomes achievement, the LL2E Approach has been instrumental in the progress made

by the STEP Nepal project towards results achievement. The STEP project has targeted increased access

by women and youth from the HKK communities to quality BTVET services. As noted in a previous

finding, the target communities for the LL2E intervention have largely comprised unskilled persons with

limited educational prospects, including BTVET, and employment options. This situation has been

compounded by the remote location of the project communities in Far West Nepal, and the system of

social stratification that has isolated HKK communities from the rest of society.

HKK women and youth have gained increased access, however, to nationally certified BTVET, which can

be used to improve their livelihood through decent employment. Of concern to graduates and key

actors in the private sector, however, is the level of training that is provided, as LL2E project activities

provide CTEVT Level I certification, which equates to basic training. Increased access to certified training

notwithstanding, of significance here, is the reduced marketability of graduates in their search for good

wages, and the skill limitations faced by those graduates who have intentions towards small-business

start-up. Importantly, document review and consultations with key informants at CTEVT indicate that

15 Agriculture Vocational Education and Training

20

Level 2 certification is attainable by graduates following one year of work experience after the Level 1

Skill Test certificate was awarded in a relevant occupation/ trade.

Finding 6: The need to systematically document and disseminate knowledge and experiences at project-level may conflict with the underlying objectives of LL2E. There is an equal possibility for misalignment between the approach to capacity-strengthening for BTVET delivery and the scope of LL2E (STEP Nepal, Outcomes 1.2; 1.3).

Significantly, LL2E is founded on skill-building, through BTEVT, and the creation of linkages between

skills, trainings and employment. Of relevance, therefore, the application of the LL2E Approach to

project activities has involved: i) the facilitation of a systematic process for documenting and

disseminating knowledge and experiences to all concerned; and ii) strengthened capacity of public and

private TVET service providers for quality service delivery. In line with the LL2E mandate and project-

level targets at outcome-level, there is an undeniable need for institutional strengthening to support

BTEVT delivery, and knowledge-sharing, the latter of which can be used to enhance the former.

Importantly therefore, LL2E has been used to raise awareness on BTEVT in the target communities,

through partnerships with community fora and groups. Additional proposed actions under STEP Nepal

have included: a regional forum/ community of practice conference on BTVET; and a national seminar

for TVET policy-makers, planners and practitioners.

The aim of the regional forum was to facilitate the sharing of LL2E project experiences and lessons from

one country (Nepal) with LL2E partners, across Asia, including FCA offices in the region. This forum was

cancelled on account of insufficient funding. The national seminar was designed to facilitate discussion

among TVET practitioners and authorities on related education issues. The objectives of the seminar, as

published in the proceedings, entailed sharing experiences and lessons from the TVET sub-sector;

initiating constructive dialogue among key TVET decision-makers and practitioners; and drawing

conclusions and developing recommendations for the strengthening of the TVET sub-sector. In essence,

the intention was to inculcate a culture of critical reflection on past experiences, as a guide towards

future actions. The seminar proceedings were published with the support of the TVET Knowledge

Management Network (Nepal).

Of interest, the target outcomes of STEP Nepal, as articulated in the project logical framework, included

the dissemination of TVET sub-sector knowledge and experiences ‘among the concerned’; and

strengthened capacity of TVET service providers (public and private) for quality service delivery. The

merits of the regional forum and national seminar are acknowledged by the evaluation, in light of their

potential to facilitate knowledge exchange for institutional strengthening and enhanced TVET service

delivery. What is unclear from the target outcomes, and the modalities of knowledge exchange, is the

extent to which key actors in BTEVT delivery and skills acquisition, namely the instructors and rights-

holders are to benefit. By way of explanation, if the objectives of LL2E is to facilitate increased access to

quality BTEVT and employment opportunities, there is a question as to whether project outcomes that

aim to build the capacity of TVET institutions in the country (by national seminar or other modality) are

too ambitious in scope for LL2E project activities. Would scarce project resources be better used in

building the internal capacity of the implementing agency for BTEVT delivery, and the management of

development results, to ensure results achievement and sustainability in line with LL2E? Of particular

21

interest is the extent to which the national seminar has been able to enhance BTVET instruction, and by

extension, instructor capability, to increase the marketability of rights-holders upon graduation.

Finding 7: LL2E project activities have equipped Congolese youths with marketable vocational skills and opportunities for industrial training. Access to opportunities for employment and apprenticeship are highly dependent, however, on vocational specialisation (Rwamwanja BTVET Project, Outcomes 1.1; 1.2)

Key informant consultations (with ex-trainees/ graduates, project staff and stakeholders), in

combination with document review, have shown that youths have gained access to vocational and

entrepreneurial skills based on the use of LL2E for the implementation of project activities. LL2E has also

facilitated industrial training and certification by national Directorate of Training TEVT standards. In

addition, career counselling throughout the project cycle, and follow-up support post-training, have

been used to guide youths towards wage employment or small-business start-up.

Of interest, the BTEVT trainings have evolved from their initial design to include an increased focus on

entrepreneurship. As part of the training in agriculture (AVET), for example, trainees are encouraged to

sell their produce in seasonal markets. Post-training support for small-business start-up has also

included the establishment of a trainee restaurant and garage, and a Demonstration Plot. Importantly,

the re-visiting of the project design was in response to challenges faced by graduates from certain

specialisations (as indicated in Finding 4) in finding work upon completion of the trainings. By way of

contrast, graduates who specialised in brick-laying and concrete practices did not face similar

employment challenges. While there is a question, here, of the adequacy of the pre-training labour

market needs assessment, the location of project activities is also at issue. Specifically, trainings are

conducted in a remote area in which there are limited opportunities for employment.

4.5 Efficiency

Finding 8: The LL2E Approach is strongly supported by standard procedures for the management of results, risk and financial resources. Efficient project implementation is contingent, however, on resource availability.

At the level of project management, the LL2E Approach is built on strong supportive systems for the

management of risk and financial resources. The assessment of risk is incorporated into LL2E project

activities at the point of design and includes the identification of mitigation measures to minimise the

negative effects of anticipated risks to results achievement. By extension, project implementation has

demonstrated a sound capacity to resolve emergent, unanticipated issues, which have the ability to

impair the achievement of expected results. As an example, given the need for an increased focus on

entrepreneurship as part of the Rwamwanja BTVET Project, training instructors received business

development training to enable them to, in turn, facilitate training in this area. Similarly, the

establishment of a Placement Unit within UCEP and the appointment of a Career Counsellor at

Rwamwanja responded to the need to guide trainees and graduates towards gainful employment.

22

In the area of project implementation, the application of the LL2E Approach to project activities is

strongly supported by qualified and committed staff, who enable the process of managing for

development results. Interestingly, in spite of the difference in implementation modality in each project

country, specifically, an organisational partnership in in Nepal, and FCA self-implementation in Uganda,

both projects have made progress towards the achievement of expected outcomes. Operational

coordination within the project communities, combined with results achievement in the same, has

benefitted from the representation of the implementing agency at community-level. Notably, UCEP

presence in Far West Nepal has been supported by the appointment of a District Coordinator, whose

responsibilities include local oversight of project implementation. Project implementation in

Rwamwanja has also benefitted from the establishment of the FCA Sub-project Office, led by a Project

Coordinator, which coordinates BTEVT delivery, including follow-up, and stakeholder collaboration at

local level.

Importantly, results management at local level is enhanced by the ongoing collaboration between local

and national-level personnel. There are operational constraints pertaining to resource availability at

local-level, however, which pose challenges to outcomes achievement. Given the focus of this

evaluation on the viability of the LL2E Approach, these challenges will not be addressed here, as they

will be more appropriate for the evaluation of the country-specific projects.16 It should be mentioned,

however, that the capacity of the LL2E Approach to facilitate results achievement is lacking in the area

of high project staff capabilities in monitoring and evaluation (M&E). The lack of up-to-date, M&E skills,

including skills that are informed by the specific requirements of LL2E, has affected staff capacity to

manage for development results. In particular, there is a notable gap in staff capacity to establish

realistic project targets that align with LL2E; identify appropriate performance indicators; and report

adequately on results achievement at the level of project outcomes.

Finding 9: The LL2E Approach provides some value for money (VfM) from a theoretical perspective. There is potential for increased VfM, however, through greater private sector engagement and increased capacity-building for implementing agencies.

By design, the LL2E Approach creates opportunities for vulnerable and socially marginalised youths to

gain technical and vocational skills for making better career decision to improve their livelihood,

including that of their immediate household. Youths are engaged in wholly subsidised trainings, as all

associated training costs are met by the implementing agency. In consequence, youths who enrol in the

trainings are not faced with any financial responsibility relative to the payment of fees; accommodation

costs; and / or the purchase of course materials.

16 Following commissioning of the consultancy, terms of reference for the evaluation of STEP Nepal and the Rwamwanja BTVET

Project were in process of development.

23

The notion of VfM in development programming is not limited to considerations of least cost, but

rather, involves assessing the extent to which financial resources are converted into tangible (or

anticipated) development outcomes. Using the 3Es Framework for VfM assessment, the purpose of VfM

is to create a better understanding of costs and results to: i) maximise the impact of monies spent; and

ii) facilitate informed, evidence-based decision-making to manage for development results.17 By

extension, therefore, the estimation of VfM is based on the application of the following criteria to the

assessment of the same:

• Criterion 1: Economy, which is used to determine whether required inputs of appropriate

quality are being purchase at suitable prices;

• Criterion 2: Efficiency, where emphasis is placed on how well project inputs are converted into

results at the output-level;

• Criterion 3: Effectiveness, which focuses on the extent to which, and process whereby, outputs

are converted into expected outcomes.

VfM assessment of LL2E shows that the Approach demonstrates some value-for-money, in as far as

resources available for LL2E project implementation have contributed to development results. Of

interest, here, the development results that have been achieved are not necessarily articulated in the

logical frameworks of the projects of focus for the evaluation. Importantly, however, the nature of

identified development results, in particular, the increased confidence and resourcefulness of youth in

establishing and pursuing livelihood goals, outweigh the costs associated with LL2E. It should be noted,

nevertheless, that there is scope to increase LL2E value-for-money, through closer collaboration

between individual projects and the private sector, and the institutional strengthening of the LL2E

project implementing agencies. In the area of institutional strengthening, kay areas for capacity-building

include the implementation of the M&E function; and professional development for BTVET staff. Of

equal importance is the establishment of communities of practice for the sharing of knowledge and

experience among BTVET instructors; as well as between project staff.

The results of the assessment of VfM relative to LL2E is summarised in Exhibit 4.1.

17 See Department for International Development (2011). DFID’s Approach to Value for Money. Available at:

http://www.betterevaluation.org/en/resources/guide/dfid_appraoch_to_value_for_money

24

Exhibit 4.1 Assessment of LL2E value-for-money

VfM Criteria Assessment Results

Criterion 1:

Economy

NGO Partnership Modality:

• Cost of subsidising vocational trainings > 60% of project budget (STEP Nepal, 2017)

• Cost of training and certification = 11% of project budget (STEP Nepal, 2017)

• Cost of advocacy and local partnerships: up to 6% of project budget (STEP Nepal, 2017)

Self-implementation Modality:

• Cost of training and certification = 13% of project budget (Rwamwanja BTVET Project,

2017)

• Cost of providing tools and materials for trainings = 14% of project budget (Rwamwanja

BTVET Project, 2017)

• Cost of student welfare = 8% of project budget (Rwamwanja BTVET Project, 2017)

• Cost of outreach through advocacy = 3% of project budget (Rwamwanja BTVET Project,

2017)

• Cost of post-training support (Trainee garage and restaurant; Demonstration garden) =

6% of project budget (Rwamwanja BTVET Project, 2017)

Criterion 2:

Efficiency

NGO Partnership Modality:

• LL2E has facilitated sound project management (including risk and financial

management) by international standards

• LL2E has contributed to the achievement of development results

• Implementing agency, including BTEVT instructors, has had insufficient opportunities for

capacity-building/ institutional strengthening

Self-implementation Modality:

• LL2E has facilitated sound project management (including risk and financial

management) by international standards

• LL2E has contributed to the achievement of development results

• Implementing agency, including BTEVT instructors, has had access to opportunities for

capacity-building/ institutional strengthening

Additional training is required to support results management

25

VfM Criteria Assessment Results

Criterion 3:

Effectiveness

NGO Partnership Modality:

• In general, youths have gained:

Livelihood and entrepreneurial skills

On-the-job training

Increased levels of confidence and resourcefulness

Access to decent employment, including wage and self-employment

• Not all youths have become high-wage earners

• Communities have gained access to:

New sources of service delivery through small businesses

Information on BTEVT

Model examples of successful youths

Partnerships between Community Fora and UCEP

• Private sector enterprises have gained access to:

Newly trained and certified workers

• Insufficient engagement of private sector enterprises has, in some instances, reduced

the marketability of youths on account of a need for further training or higher

certification

Self-implementation Modality:

• Youths have gained access to:

BTEVT skills

Business development/ entrepreneurial skills

On-the-job/ Industrial training

Support for small-business start-up, including collaborative start-up

Support for post-training wage Placement

Access to decent employment, including wage and self-employment

• Youths demonstrate increased levels of confidence and resourcefulness

• Communities have gained access to:

New sources of service delivery through small businesses

Information on BTEVT

Model examples of successful youths

Partnerships between Community Groups and FCA

• Private sector enterprises have gained access to:

Newly trained and certified workers

Industrial partnerships with FCA

• The terms of engagement of private sector enterprises for Industrial Training has

reduced the opportunity for post-training employment of youths within the enterprises

where the Training occurred

26

Source: Assessment of data gathered by document review and key informant consultations.

4.6 Cross-cutting theme: Gender

Finding 10: While the application of the LL2E Approach has allowed for the integration of gender considerations at project-level, participation by young women in the trainings is, at times, challenged by domestic commitments.

As young women feature prominently in the implementation of the LL2E Approach, given its primary

focus on women and youth, it is conceivable that project activities will seek to prioritise enrolment and

successful completion of the trainings by this target group of trainees. Relatedly, therefore, trainee

selection for LL2E project activities under the STEP Nepal and Rwamwanja BTVET projects has been

informed by male to female ratios that were established by each implementing agency.

In Nepal, the selection of STEP trainees was guided by a male to female ratio of 60 to 40. Participation

by young women was also actively encouraged by Results Area #1, which targeted enhanced access of

women and youths to quality TVET services. In addition, advocacy during the project involved the

marketing of training activities through civil society organisations, which included women’s groups; and

mothers’ groups. UCEP, in applying the LL2E Approach to the STEP project, also supported enrolment by

young women in training specialities that have not been traditionally explored by women, such as

Plumbing; and Motorcycle Repair18. In Uganda, trainee selection is informed by a male to female ratio of

55 to 45. Importantly, in applying the LL2E Approach to the selection process gender is not used as the

basis for selection. Much attention is given to young women, however, as project staff are aware of the

vulnerability of young women in the communities that are targeted by the Rwamwanja BTVET

intervention. Overall, the emphasis placed on gender, through LL2E Approach application, draws on the

FCA human rights-based approach, which is based on a commitment by FCA programming to address

the structural factors that underlie gender-based inequality and discrimination, among other factors.

In spite of the fixation of LL2E on integrating gender considerations into project activities, participation

by young women in the trainings is largely challenged by their family commitments. By way of

illustration, in 2015, 31% of the graduates of STEP Nepal were young women, this being the first year of

project implementation. During the second year of training (2016), and as of June 2017, none of the

graduates of the residential training were young women. Consultations with project staff and the review

of relevant documents showed that although young women from the targeted project communities had

interest in the trainings, they were reluctant to re-locate to Kathmandu to participate in the residential

trainings, because of their domestic obligations. Similarly, in Uganda, one of the main reasons for

absenteeism by young women during the trainings is the lack of childcare facilities. Importantly, in the

effort to make the training more accessible to all young persons who express interest, including young

women with family commitments, both projects have designed trainings for delivery at community-

level, specifically Mobile Trainings in Far West Nepal, and Community-based Trainings within the

Rwamwanja Refugee Settlement.

18 Of interest, the STEP project facilitated an innovative training for 100 women in four-wheeler light-vehicle driving

27

4.7 Participation and Ownership

Finding 11: The full national ownership of the LL2E Approach through policy uptake and programme planning is a work in progress. Discussion and collaboration between government and LL2E project implementing agencies is ongoing.

Conceivably, uptake of the LL2E Approach by national policy is indicative of, and of necessity, required

for its long-term integration into social sector investment programming in the focal countries for the

evaluation. The suggestion, here, is that policy uptake is the outcome of ongoing efforts at collaboration

between government and the project implementing agencies in the interest of country ownership of the

Approach and its related project activities. Importantly, therefore, LL2E project planning activities have

included initial discussions between public sector ministries and LL2E management. In Nepal,

representatives from the Ministry of Social Welfare and the Ministry of Education were informed about

project activities and were engaged in early discussions on implementation. In particular, five meetings

were held with representatives of the District and Central Advisory Councils, to share the approach that

was being used to implement LL2E project activities. In Uganda, initial discussions took the format of a

scoping exercise that engaged key actors in education and refugee programming at district level (in

particular, UNHCR, Office of the Prime Minister and the Kibale County District Education Department19).

The discussions included considerations on the feasibility of the intervention, including the type of

educational instruction that should be provided, in the interest of reducing the possibility for

duplication. Of interest, the discussion also considered possible uptake (and, in essence, continuation) of

project activities by the Ministry of Education, in light of the intention of the Ministry to establish a

vocational centre in each county, and the absence of a nationally-owned TVET centre in Kibale County.

In response to ongoing discussion between government and the project implementing agencies, BTVET

certification by national standards, through CTEVT in Nepal and DIT in Uganda, was incorporated into

project activities. FCA Uganda has also continued to engage government in discussions on the ownership

of the Rwamwanja TVET Centre in relation to District-level ownership.

Finding 12: The use of advocacy as a core element of LL2E has been actively pursued as a mechanism for awareness-raising within project communities. It is debatable, however, whether the potential contribution of advocacy to LL2E has been fully explored.

Significantly, advocacy has been central to the implementation of LL2E activities in both project

countries, given its role in awareness-raising for BTVET at community-level. In Nepal, as indicated in

Findings #6 and #10, partnerships with community fora/ groups were used to disseminate information

on BTEVT in the target communities, as well as engage the support of these groups in the selection of

trainees. A similar approach to advocacy in Uganda engaged refugee groups, and included follow-up

discussions with the parents of trainees, to emphasise the importance of the trainings and request

parental support for encouraging trainees to complete the trainings.

19 Project activities under the Rwamwanja BTVET Project are implemented in Rwamwanja Village, Kibale County, in which the

Refugee Settlement is located.

28

Notwithstanding initial, and as applicable, ongoing discussions between UCEP / the FCA Uganda Country

and Sub-project Office and national LL2E project stakeholder agencies (specifically, government

agencies), there is limited evidence of the use of advocacy to encourage policy uptake of the LL2E

Approach. In Nepal, the activities of another FCA project (which was implemented by Lutheran World

Federation Nepal)20 used advocacy to secure funding for small-business start-up within the communities

that were targeted under STEP Nepal. This project is not exclusively based, however, on the LL2E

Approach to programming. While the importance of policy uptake, and advocacy in support of uptake is

noted, the implementation of the LL2E Approach in Nepal and Uganda has not focused on this activity. A

question that arises though, is whether resources available for the implementation of the LL2E Approach

can and should be used to incorporate an advocacy component for policy-uptake, or whether the LL2E

focus should concentrate solely on skill-building and the practical application of skills, and employment.

4.8 Sustainability

Finding 13: BTVET ‘skilling’ acquired through LL2E lends itself to results sustainability into the medium-term. Results sustainability into the long-term is conditional, however, on the establishment of support structures for the marketability of acquired skills.

Arguably, skill-building is sustainable in itself, as it entails imparting or acquiring a skillset that is retained

for life. An alternative view can be used to argue that skills need to be nurtured through practical

training, refinement and /or refresher courses, in order for the skill-holder to remain relevant to a

dynamic labour market, in terms of individual marketability. Within the context of LL2E, while the post-

training nurturing of acquired skills is not a mandated element of the Approach, it is to be noted that the

career counselling/ placement component of project activities has been used for graduate follow-up. On

one hand, this activity focuses on: nature of employment and level of earnings, as well as on possible

reasons for unemployment, where it occurs. On the other hand, the activities conducted under career

counselling/ placement include the provision of follow-up support to enhance or initiate employment,

including self-employment. The importance of these activities is especially noted, in light of the LL2E

context relative to geography. Specifically, given that the target group(s) for LL2E project activities

comprise young persons from marginalised communities, it is understandable that most trainees are

based in remote/ hard-to-reach areas. As indicated in Finding 7, therefore, within the context of LL2E,

BTVET trainings are likely to be conducted in remote locations, where opportunities for wage

employment are limited.

Interestingly, the proposed LL2E programme includes a timeline of six to twelve months of post-

graduation support involving mentorship, coaching, facilitation of networks, etc. This implies that there

is an assumption that graduates are well-established in their careers beyond this period. Undeniably,

resource limitations confine the implementation of the Approach, including its post-training supportive

activities, to a given timeframe. It is equally undeniable, however, that some LL2E project graduates may

require follow-up support beyond the established timeframe. The need for this support is compounded

by the remoteness of the communities in which graduates live. While some graduates may choose to re-

20 AAFreedHAKK - Action for Advancing Freedom of Haliyas, Ex-Kamaiyas and Kamlaris

29

locate to cities with greater employment prospects, other may opt to remain in their communities of

origin. The question that emerges here, pertains to whether the LL2E Approach is able to incorporate

supportive mechanisms, to address the post-graduation needs of ex-trainees/graduates who require

additional assistance beyond the support timeframe. At issue would be the possibility for:

1) Extended follow-up of graduates, including the availability of resources (human; financial; time;

etc.) to facilitate the same;

2) Continued, and intensified, investment in the assessment of labour market needs, to ensure

that trainings respond to the requirements of the labour market to increase employment

prospects for graduates; and

3) Investment in (or continued investment, as applicable) in entrepreneurial education to guide

graduates towards the option of small-business start-up, as an alternative to wage

employment.

Overall, items 1) to 3) centre on the post-graduation support that seeks to carefully match BTVET skills

and training with labour market needs, in the interest of the labour market absorption of each cohort of

new graduates.

Finding 14: Based on initial steps taken during project planning and implementation, there is potential for the programmatic sustainability of LL2E beyond its funding cycle. The sustainability of the approach is highly reliant, however, on policy uptake by government and/ or the private sector.

The LL2E Approach has been applied to projects that are managed and implemented through FCA

financial support. It is realistic to assume that project activities cannot be continued indefinitely through

the current source of funding. In this regard, sustainability planning was incorporated into the STEP

Nepal and Rwamwanja BTVET projects from inception. Section 12 of the project proposal for STEP Nepal

drew reference to a sustainability/ exit plan that was to be prepared as a separate document.21

Sustainability considerations for financial; institutional; social and cultural; and environmental

sustainability were also included in the project proposal for the Rwamwanja BTVET project. The steps

taken at project level can be used to contribute towards the implementation of a sustainability strategy

to maintain the positive outcomes achieved through the LL2E Approach. These initial efforts towards

sustainability can also be used to sustain project programming into the long-term in the projects of

focus for the evaluation, as well as future projects.

Based on the findings that have emerged from the evaluation, from a financial, policy and practical

perspective, the sustainability of the LL2E Approach is highly dependent on intervention by government

and/ or the private sector. Consultations with employers from the private sector have shown that

private enterprises are willing to provide support for trainee selection through to employment, and have

expressed having access to resources to facilitate this process. In addition, policy uptake by government

has the potential to underscore the importance of the approach for development programming,

including the management of development results. Actions taken by the Government of Nepal and

21 This document was not accessed during the evaluation.

30

Uganda, in the area of BTVET, further suggest that policy uptake has the potential to facilitate

transparent, sustainable support for:

• Training and certification of youths in the target communities;

• Hiring of the existing BTVET instructors;

• Expansion and maintenance of the vocational training facilities; and

• Facilitation of start-up capital for new, small-business owners.

Regarding the sustainability of LL2E project activities, however, a point for consideration is whether

government-managed BTVET trainings will continue to be free or highly-subsidised, given the economic

status of the communities that are targeted for training. As indicated in Finding 9, LL2E trainings are

subsidised by the implementing agencies, through the use of donor funding, to minimise the cost of

trainee-participation in BTVET trainings. In essence, the full cost of the trainings is absorbed by the

implementing agencies. This approach is practical, in terms of facilitating access by young persons from

the target communities to the trainings. It is questionable whether uptake by government and/ or the

private sector will apply the existing subsidised modality. Government funding would likely derive from

taxation at source or overseas development assistance, neither of which can be used as unlimited and

sustainable funding. As the private sector is also driven by a profit-motive, private sector partners may

likely anticipate a return on investments made into LL2E. While uptake by government and/ or private

sector enterprises has the potential to sustain project activities, the re-visiting of wholly subsidised

trainings is a possibility.

31

5. Conclusions, Lessons learned and Recommendations

5.1 Conclusions

The LL2E Approach has facilitated empowerment programming. Youths who participated in LL2E project

activities in Nepal and Uganda gained BTEVT and inter-personal skills, and were empowered to become

change-makers by setting, pursuing and achieving personal goals for improved livelihoods. The

responsiveness of BTVET skill-building to issues that emerged during training delivery has further

allowed for the improvement of the training modality, to ensure that youths, as rights-holders continue

to be the main focus of the programming. As a result, trainings offered at the main vocational centre in

each project country have been complemented by training that is more accessible to youths in the

targeted communities through training delivery at community-level. At issue has been the need to

ensure that youths continue to have continuous access to trainings that are relevant to labour market

needs to support youth employability and marketability, and capacity to enhance sustainable livelihoods

for youths and their families.

In light of the need for ‘skilling’ that remains relevant and responsive to a dynamic labour market, the

evaluation has unearthed several key issues that have the potential to challenge the capacity for LL2E to

support the achievement of its fundamental development targets. Of central focus is the question of

whether there is a potential role for private sector involvement in the provision of support and

mentorship for youth employment and small business start-up. Further, there is need to determine

whether the finalised version of the LL2E Approach can, by design, provide enhanced value-for-money

by ensuring that: i) results achievement at outcomes level continues to outweigh resource investment in

LL2E project activities; and ii) programmatic sustainability is maintained through the uptake of project

delivery by government practice or private sector initiative. Irrespective of the decisions that are taken

with regards to results sustainability and programming continuity, it is undisputed that the effectiveness

and efficiency of the LL2E Approach into the medium to long-term requires inter-sectoral collaboration

that aligns with the need for the strategic linking of theoretical and practical learning to workplace

earning.

5.2 Lessons learned

The under-estimation of the role of the private sector in LL2E project design and implementation is

counter-productive to outcomes achievement, including the process of linking learning to earning.

Private sector enterprises are astutely aware of the needs of the labour market, and have the capacity

to provide practical guidance for matching skill-building with occupational requirements, and creating

opportunities for post-training employment.

While it is important to facilitate access to BTEVT training to individuals and communities where there

is considerable lack of marketable skills to secure gainful employment, access to decent work is

determined by the extent of skill marketability. In addition to providing basic training in areas of BTVET

specialty, the level of training acquired is a main determinant of individual capacity to access good

working conditions, including higher wages, for improving livelihood.

32

The capacity for LL2E to support the achievement of development outcomes is increased through the

strategic alignment of project-level development targets with the objectives of LL2E. If the

performance measurement framework is insufficiently aligned with the underlying objective of LL2E,

results achievement will be loosely related to the need to establish key linkages between skill-building

and livelihood improvement.

Inadequate assessment of the needs of the labour market can lead to labour market saturation in an

identified vocational specialty or situation where there are insufficient support structures to facilitate

employmentThe marketability of BTVET training is highly dependent on the needs of the labour market.

The selection of training course offerings must be preceded by an in-depth assessment of the existing

labour market needs.

5.3 Recommendations

The recommendations in this section comprise strategic and operational guidelines for the finetuning of

the LL2E Approach, with emphasis on elements of the existing Approach that need to be addressed in

the interest of strengthening. It follows therefore, that these recommendations are informed by the

main findings and lessons learned of the evlautaion. Of importance, as the evaluation mandate has

concentrated on the workings of LL2E, the recommendations here are not project- or country-specific,

but are geared towards the overall strengthening of the LL2E Approach.

Recommendation 1: There is need to re-visit the role of the private sector in facilitating LL2E, with a focus on the early engagement of private sector enterprises in the LL2E programmatic cycle, and the re-working of the partnership for industrial placement. (Related to Finding #2; #4; #5; #9; #11; #12; #13; #14)

Evaluation findings on the application of the LL2E Approach to development interventions have

indicated that the role of the private sector is being largely underutilised. Private sector enterprises are

engaged in up to three ways: i) during informational sessions for contract signature for the provision of

on-the-job training; ii) the provision of employment; and/ or iii) the engagement of trainees in

motivational talks and discussions on employment requirements or motivational talks. Feedback

received from the private sector has, in some instances, included claims of inadequate knowledge of

LL2E, as well as late involvement of firms in LL2E activities, usually, following the completion of

classroom-based training. Importantly, however, private sector enterprises have emphasised that they

have the potential to make a viable contribution to LL2E implementation, by providing guidance for

trainee selection and facilitating on-the-job-training that responds to existing labour market needs. Two

issues are worth considering here:

1) Given their immersion in the labour market, private sector enterprises are potential employers,

who are experts on the realities of the labour market. An extended role for employers during

the post-graduation period is also incorporated into the proposed LL2E programme. Based on

feedback received from potential employers, however, the timing of their engagement under

this arrangement is late. Private sector actors are more than capable of making a real and

appropriate contribution to the development of trainees into workers who are equipped with

required workplace skillsets. The role foreseen for the private sector includes contribution to

33

the selection of trainees for LL2E intervention. There may be concern, by implementing and

donor agencies alike, of the effect of private sector involvement at this stage, on the definition

of the target group(s) for LL2E intervention and the prioritisation of vulnerability and

marginalisation. In order to strike a balance between LL2E emphasis on vulnerability and

employer needs, it is important, here to underscore the need for partnership between the each

implementing agency and private sector firms. In essence, implementing agencies should be

open and transparent about the objectives of LL2E programming, with emphasis on the need to

engage young persons from vulnerable and marginalised communities. Of necessity, this

approach should be followed up by a signed memorandum of understanding between each

implementing agency and private sector firm to ensure adherence to LL2E principles. As

required, private sector support for LL2E programming can be further informed by the

incorporation of a selection ratio to support the engagement of young persons from the target

group(s) (e.g. 50% of trainees should meet the vulnerability/ marginalisation criterion). In this

regard, engagement in an LL2E partnership would not affect the capacity of private sector firms

to engage trainees who do not meet the vulnerability criterion.

2) There has been ongoing debate as to whether the length of LL2E trainings are too short at

three months, with an additional month of on-the-job-training or too long at five months, with

an additional month of on-the-job-training. The underlying issues here are: i) level of training,

as the three-month period results in basic level certification; and ii) loss of income, given the

extended period of classroom based training over the five-month period. The trainings that are

offered align with the requirements of the national training boards that are responsible for

regulating training quality and certification. Importantly, however, it is feasible for BTVET

training to explore the possibility of on-going training within the workplace over the duration of

classroom learning, through partnerships with private sector enterprises. This approach will

need to be finetuned with the key actors involved in BTEVT, namely potential trainees or

representatives from their communities, the BTVET boards; the vocational centres (comprising

management and instructors), the implementing agencies, and the funding body. The main

item to be explored is whether there is preference for this approach and whether trainees can

be provided with an ongoing stipend, to earn while they learn in the workplace. Of additional

note, the advantages of this approach include:

– A lengthier period of on-the-job-training, which would address employer criticism of

inadequately trained graduates under the current BTVET arrangement; and

– Trainees would be closer to achieving a higher level of certification following completion of

blended training and post-training employment

The approach that has been described is a variation of the LL2E blended approach that can be used to

yield quality BTVET and improved, sustainable livelihood.

Recommendation 2: The LL2E implementing agency should re-visit its approach to advocacy to incorporate the lobbying of government, or as applicable, private sector enterprises, for LL2E uptake by government policy or private sector programming. (Related to Finding #11; #12; #14)

34

As Recommendation 1 has outlined a possible role for the private sector, moving forward, the focus

here will be on advocacy for policy uptake by government. At the outset, it would be essential to ensure

alignment between the objectives of the programming and the mandate of the Ministry to which

responsibility for oversight would be transferred. This process is straight-forward and has added

advantage for the sustainability of programming, and by extension results, into the medium to long-

term. Complications may arise, however, if the modalities of delivery and oversight are not finalised

prior to hand-over to determine:

1) Whether the objectives of the development intervention will remain unchanged;

2) How development objectives will be achieved;

3) Whether there is a perceived role for existing implementing agencies, or a move towards self-

implementation by government;

4) Whether the programme will continue to be cost-free to registered participants; and

5) Whether the programme will continue to be offered exclusively to the current target groups.

Overall, these considerations have been outlined to emphasise the need for an extended period of

negotiation, collaboration and transference, which cannot be conducted, effectively, during a restricted

or hastened timeframe. It is crucial that proper systems are put in place to ensure programming

continuity and results sustainability. The role of advocacy is important in this context, to increase

awareness among key decision-makers/ policy planners at the level of government to ensure

sustainability. It would be necessary, therefore, to establish an agenda for advocacy that identifies key

stakeholders at the level of local/ national government, who can be engaged in strategic discussions on

the transfer of oversight for project coordination and implementation.

There is a risk, however, of untimely adjustment of the programming that can result from the

transference of programming responsibility. On one hand, government will not be obligated to retain all

elements of the existing programme, if there is a change of government mandate or priority areas of

programming focus. These adjustments may be contrary to the original programming design. In order to

mitigate this risk, advocacy that targets government should be used to underscore the merits of the

programming in its current format and the need to maintain the existing approach to project

implementation.

Recommendation 3: In order to better support the participation of young women during LL2E programming, project implementing agencies should explore the establishment of child-caring facilities. (Related to Finding #10)

The LL2E Approach has incorporated actions that have provided support for the integration of gender

considerations during programming. Young women continue to face constraints to their participation in

the trainings, however, which arise from their domestic responsibilities, and in particular, the need to

look after young children. An approach that can be taken to address this challenge involves the

establishment of child-care facilities to accommodate the children of young mothers who are enrolled/

wish to enrol for training. In order to maximize the potential benefit of introducing the facility, however,

a needs assessment should be conducted to determine level of demand, as well as whether there is

35

interest among trainees in a BTVET course offering in childcare. In addition to expanding the trainings

that offered by the vocational centres, and providing services to young mothers who are enrolled

trainees, the training can be used to secure a source of revenue through small-business start-up as

discussed below.

Conceivably, the establishment of childcare facilities within the context of LL2E is viable at the project

level, through the use of project-related funding. In the interest of encouraging and increasing

participation by young women in LL2E trainings, however, it is a worthwhile investment. The main issue

to be addressed is the implementation and sustainability of childcare facilities at the LL2E programmatic

level. Interestingly, therefore, the East African developing nation of Malawi has had some measure of

success with the establishment of community-based childcare centres (CBCCs).22 The introduction of the

CBCCs was primarily intended to address the situation of idle children; lack of caregivers; lack of

proximity to child care facilities; and the possible exploitation of children left unattended at home.

While the children who were initially targeted by the CBCC Approach were aged three to five years, the

facility was also used to engage children who were older or younger than this age range. Of relevance to

the LL2E Approach, the CBBC facility was developed to create time for parents/ guardians of young

children to pursue economically productive activities, as well as protect children from threats associated

with being left alone. In addition to providing adult supervision for children, CBCCs also engage children

in learning activities and provide nutritious meals. Importantly, each CBCC is community-owned and

managed, using a mixture of resources that include human, financial and physical resources, as well as

time. CBCCs can be established in permanent physical facilities or can be operated in shared community

structures, such as places of worship; a community centres or other gathering place. Importantly,

although funding for the CBCC can be provided by government or donor agencies, community-

ownership is central to CBCC sustainability. Community ownership allows for resource mobilisation,

including fundraising and the management of CBCCs by community groups, which also has the potential

to decrease operating costs. The emphasis on community-ownership is central to the sustainability of

the facility, as there if minimal reliance on external funding. CBCCs in Malawi do receive governmental

support, however, based on the observed viability of the facility for early childhood education.

In order to fully integrate the CBCC facility into LL2E project activities, including at the programmatic

level, as indicated, BTVET training and certification in childcare would be required. This creates an

avenue for graduates of the childcare training to seek employment at established CBCCs or open their

own facility to provide similar services and acquire a source of revenue. Of additional necessity, in order

to facilitate certification of trainings and uptake by government, the planning stages for CBCC

establishment should engage key decision-makers at the level of community leadership; community-

based organisations; local government; and BTVET quality assurance.

22 See Munthali et al, 2014 https://link.springer.com/content/pdf/10.1186%2F2193-1801-3-305.pdf

36

Recommendation 4: The project implementing agency, with support from the project donor, where applicable, should invest in capacity-building at the level of project management, coordination and implementation to support the establishment of efficient supportive structures for results achievement. (Related to Finding #6; #7; #8; #9)

The LL2E Approach has received support from well-established structures to ensure quality during the

management through coordination of project-level activities. The evaluation has noted, however, that

there is room for capacity-strengthening across the programming, in the interest of efficiency and

results achievement. To better enable the use of LL2E in managing for development results, it would be

worthwhile for the implementing agencies, in collaboration with the project donor or oversight agency

to invest in the following capacity-strengthening activities:

1) Professional development courses for BTVET instructors: The instructors of BTVET serve to

benefit from capacity-building through professional development, where the focus will be on

innovative methods of teaching, and the use of new technology. Specifically, in the short-

through longer-term, implementing agencies can arrange mandatory, refresher trainings/

workshops for BTVET instructors, in the format of continuous professional development (CPD)

on new developments in BTVET that are of direct relevance to classroom instruction, including

the facilitation of the practical, classroom based component of the trainings. The emphasis,

here, would be on facilitating a student/ trainee-centred approach though engagement and

support for learning using teaching aids and innovative techniques, including new technology.

In addition, it would be necessary to provide guidance to instructors to ensure that trainings

respond to the needs of the target group of trainees, on one hand, and are tailored to the

learning abilities of individual trainees, on the other hand (competence-based education).

Importantly, the facilitation of CPD requires close collaboration with the Ministry of Education

in the focal countries for LL2E implementation, and the oversight agency for BTEVT for the

purpose of quality assurance and relevance to country priorities and labour market needs.

Collaborating with ministries of education and BTVET oversight agencies increases the potential

for ownership and sustainability of CPD by the implementing agencies and governments. By

extension, institutional and national ownership of CPD allows for the assumption of

responsibility for the sourcing and validation of CPD trainings, with attention to quality

assurance and certification of the same. Of note, FCA has designed a professional development

package for BTVET instructors, comprising basic pedagogical instruction; on-the-job-training;

and support for the provision of career guidance counselling. The stated intention is to address

the possible challenge of BTVET staff who might be qualified in their area of speciality, but

might not have had any pedagogical training.

37

2) Knowledge exchange through communities of practices for BTVET instructors; and project

coordination staff: Both categories of LL2E personnel can benefit from the exchange of

learnings/ knowledge and experience (in separate groupings) for use in enhancing practical

application of the LL2E Approach. Communities of practice can further be used to invest in the

planning stages of advocacy for the uptake of the LL2E at various levels, such as, media

campaigns to raise community awareness; the identification and promotion of LL2E graduates

as good examples of the benefits of BTVET; the investment in the visibility of LL2E among the

private sector and key decision-making agencies at the level of government (local and/ or

national). The establishment of communities of practice can allow for the informal or

structured/ systematic exchange of learning and experiences to support BTVET enhancement

and CPD at country- and/ or regional-level;

3) M&E training for project staff: Training in this area has the potential to strengthen staff

capacity for results monitoring and reporting on results achievement, with the added

advantage of developing capacity to adapt to changing circumstances that affect performance

measurement and results achievement. Of importance, here, is the need for increased staff

capacity at the level of the implementing agencies, to establish development targets that

reflect LL2E objectives, and ensure that available resources are directed towards this end.

Specifically, at the point of requesting project proposals / concept notes for LL2E project

activities, an established criterion for project selection should be the alignment of project

results, as articulated in the project-specific logical framework, with LL2E objectives. It would

be insufficient to select projects based on thematic alignment only. Using the current projects

in Nepal and Uganda as examples, both projects align thematically with LL2E given the

communities that are targeted for the interventions; the focus on young persons; and the aim

of facilitating access by youths to quality BTVET trainings to improve their livelihoods. At the

level of the logical frameworks, however, the embedded theory of change for each project

should be strengthened in some areas. To illustrate, with reference to the Rwamwanja BTVET

project, there is no logical progression from outputs through to outcomes that outlines the

LL2E ideals of relevant pre-training activities that determine labour market needs; subsequent

training activities; and anticipated results at the level of outputs and outcomes. The logical

framework for the STEP Nepal project is stronger in this regard, as it clearly articulates progress

from relevant project activities through to results. Greater clarity was required, however, in the

identification of relevant indicators for project-level outcomes, as the indicators listed at

outcome-level are not all applicable to each outcome. It would be worthwhile therefore for

LL2E project selection to involve a pre-selection (or pre-implementation) phase during which

implementing agencies receive support to refine the logical matrix to ensure that targeted

results are measurable and are in line with LL2E objectives, in the interest of achieving

meaningful and sustainable results.

38

4) Assessment of labour market needs: A pre-condition for all training activities, including the

identification of areas of speciality; the development of training curricula; and the

establishment of memoranda of understanding for on-the-job trainings, should be the accurate

assessment of labour market needs. In terms of the capacities and resources that are required

for this exercise, it might be more cost-effective for implementing agencies to outsource the

needs assessment to an individual consultant or research firm. It is worthwhile, however, for

the implementing agency to develop a list of considerations/ guidelines that should inform the

assessment of needs. Notably, the assessment should explore:

– Prospects for employment in the vicinity of the BTVET training centre and the communities

where trainees live; questions to consider include which vocational specialties are in

demand in the area; and whether there are more opportunities for wage – or self-

employment;

– The needs of established enterprises in relation to job creation, with emphasis on required

skills; organisational capacity to absorb graduates; wages levels and prospects for

individual growth through long-term employment;

– Opportunities for partnerships between the trainings/ implementing agencies and the

business community/ private sector for the provision of mentorship; on-the-job training;

and employment opportunities, through early collaboration at project inception;

– Availability of resources to support small-business start-up through micro-loans or grants,

with emphasis on potential sources of seed capital that are external to the project (e.g.

from government sources; private sector enterprises; local cooperatives; credit unions;

financial institutions23; etc.); and

– The resources and/ or support that will be required at project level to guide graduates

towards employment, namely, whether there is a context-specific role for implementing

agencies during post-graduation support.

Upon completion, the combination of trainings/ guidance has the potential to strengthen the capacity of

the implementing agency for enhanced oversight at the level of management, coordination and

implementation; and the achievement of LL2E objectives.

23 Access to funding from financial institutions would be context- / country-specific, given established banking criteria for the

issuance of financial loans.

39

Appendix I Terms of Reference

40

41

42

43

44

45

Appendix II Evaluation Matrix

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

1.0 Impact 1.1 What is the

overall impact of

the LL2E

Approach to

project

implementation

in Nepal and

Uganda (including

intended/

untended; short-/

long-term;

positive/ negative

impact)?

1.1.1 To what

extent have LL2E

project activities

in Nepal/

Uganda added

value to the

lives of target

beneficiaries?

• Evidence of:

improved standard of living

increased access to employment

entrepreneurial growth/ initiatives

increased networking/ growth of

professional networks

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Tracer study reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

46

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

1.1.2 In what

ways have LL2E

project activities

in Nepal/

Uganda

enhanced the

lives of the

communities in

which project

activities

occurred,

and/or the

communities in

which

beneficiaries

live?

• Evidence of:

improved standard of living

reduced unemployment

entrepreneurial growth/ initiatives

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Tracer study reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

47

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

1.1.3 What

effect has LL2E

project activities

had on the

private sector in

Nepal/ Uganda?

• Evidence of:

Increased job creation

Mentorship and coaching for new

and/or potential employees

Support for business incubation

Reduced job turn-over

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Tracer study reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

• Document

review

• Consultations

with key

informants

48

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

1.1.4 To what

extent have LL2E

project activities

achieved their

intended

impact?

• Evidence of:

Enhanced access of women and

youths to quality vocational skills

training and technical education by

strengthening the TVET sub-sector

in Nepal

Improved standard of living of

Congolese refugee and host

community youth in Rwamwanja

Refugee Settlement through access

to marketable vocational skills

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Tracer study reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

49

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

1.1.5 What are

the unintended

impacts

achieved by the

LL2E projects?

• Evidence of unforeseen and

unexpected effects from project

implementation in Nepal

• Evidence of unforeseen and

unexpected effects from project

implementation in Uganda

• Perceptions of key informants

• Annual reports

• Project management

reports

• Project monitoring

reports

• Tracer study reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

50

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

1.2 What are the

main reasons that

led to the impact

generated by the

LL2E Approach?

1.2.1 How were

project activities

in Nepal/

Uganda able to

have the effect

that they did on:

• the labour

market

• the lives of

beneficiaries

• the

communities

in which

beneficiaries

live

• the private

sector

• government

authorities

and other

education

authorities

• Etc?

• Identified/ documented reasons for

the effects of LL2E project activities in

Nepal

• Identified/ documented reasons for

the effects of LL2E project activities in

Uganda

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Tracer study reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

51

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

2.0 Relevance 2.1 To what

extent is the LL2E

Approach

relevant to the

national and local

contexts of Nepal

and Uganda?

2.1.1 How

relevant are the

priorities and

goals of LL2E

project activities

to the needs of

the labour

market in

Nepal/ Uganda?

• Evidence of alignment with labour

market needs in Nepal

• Evidence of alignment with labour

market needs in Uganda

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

52

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

2.1.2 What

methods, if any,

were used to

create a link

between BTVET

and

employment?

• Evidence of the creation of links

between BTVET and employment

• Examples of methods used to create

links between BTVET and employment:

Memoranda of understanding for

job placement; mentorship;

coaching; etc.

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

53

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

2.1.3 In what

ways have LL2E

project activities

met the needs

of trainees/

beneficiaries?

• Evidence of alignment with the

expressed needs/ goals of project

trainees in Nepal

• Evidence of alignment with the

expressed needs/ goals of project

trainees in Uganda

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Tracer study reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

54

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

2.1.4 What

evidence is

there to show

that LL2E project

activities have

been suitable

for the culture/

context of: the

project

countries?

• Evidence of integration of cultural /

contextual considerations in project

design in Nepal

• Evidence of integration of cultural /

contextual considerations in project

design in Uganda

• Evidence of integration of cultural /

contextual considerations during

project implementation in Nepal

• Evidence of integration of cultural /

contextual considerations during

project implementation in Uganda

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Tracer study reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

55

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

2.1.5 How

suitable have

LL2E project

activities been

for the culture/

context of: the

communities in

which trainees/

beneficiaries

live?

• Evidence of integration of community-

specific cultural / contextual

considerations in project design

• Evidence of integration of community-

specific cultural / contextual

considerations during project

implementation

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

56

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

2.1.6 What gaps,

if any, were

there in the

training that was

provided, and

how were these

gaps addressed?

• Evidence of unaddressed, yet relevant,

subject matter

• Evidence of steps taken to address

gaps in BTVET training

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

57

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

3.0 Effectiveness24 3.1 To what

extent have LL2E

projects in Nepal

achieved

expected

outputs?

3.1.1 What

evidence is

there of

enhanced access

of women and

youth to quality

TVET service in

Nepal? (STEP

NEPAL:

Outcome 1.1)

• Graduates of TVET program have been

placed into job opportunity of at least

NRs. 6,000/month, their living standard

is improved by this.

• Local level training institutions are

competent to impart integrated quality

TVET services for long run.

• E-library established by TVET actors

and supported by STEP project has

documented and disseminated

prominent issues of the sub-sector.

• Seminar, workshop and interaction

programs have been instrumental to

feed TVET policy makers and program

designers to adopt the best practices

of past project and avoid the failure

lessons.

• Guidelines on soft skills, confidence-

building and training management has

been accessible to all TVET providers

and contributed for quality

improvement of the services.

• Study report

• Job placement data

sheet.

• Publications

• Third party evaluation

reports

• Document

review

• Consultations

with key

informants

24 Indicators and Sources of data extracted from project logical frameworks

58

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

3.1.2 To what

extent have

TVET sub-sector

knowledge and

experiences

been

systematically

documented

and widely

disseminated

among women

and youth in

Nepal (STEP

NEPAL:

Outcome 1.2)

• Graduates of TVET program have been

placed into job opportunity of at least

NRs. 6,000/month, their living standard

is improved by this.

• Local level training institutions are

competent to impart integrated quality

TVET services for long run.

• E-library established by TVET actors

and supported by STEP project has

documented and disseminated

prominent issues of the sub-sector.

• Seminar, workshop and interaction

programs have been instrumental to

feed TVET policy makers and program

designers to adopt the best practices

of past project and avoid the failure

lessons.

• Guidelines on soft skills, confidence-

building and training management has

been accessible to all TVET providers

and contributed for quality

improvement of the services.

• Study report

• Job placement data

sheet.

• Publications

• Third party evaluation

reports

• Document

review

• Consultations

with key

informants

59

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

3.1.3 Is there

evidence to

demonstrate

strengthened

capacity of

public and

private TVET

service

providers to

deliver quality

service in

Nepal? (STEP

NEPAL:

Outcome 1.3)

• Graduates of TVET program have been

placed into job opportunity of at least

NRs. 6,000/month, their living standard

is improved by this.

• Local level training institutions are

competent to impart integrated quality

TVET services for long run.

• E-library established by TVET actors

and supported by STEP project has

documented and disseminated

prominent issues of the sub-sector.

• Seminar, workshop and interaction

programs have been instrumental to

feed TVET policy makers and program

designers to adopt the best practices

of past project and avoid the failure

lessons.

• Guidelines on soft skills, confidence-

building and training management has

been accessible to all TVET providers

and contributed for quality

improvement of the services.

• Study report

• Job placement data

sheet.

• Publications

• Third party evaluation

reports

• Document

review

• Consultations

with key

informants

60

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

3.2 To what

extent has the

Rwamwanja

BTVET Project

achieved its

expected

outcomes?

3.2.1 What

evidence is

there to show

that Congolese

youth in

Rwamwanja

Refugee

Settlement and

the host

community

gained

increased access

to marketable

Vocational skills

by 2017?

(Outcome 1.1)

• 95% of the youth enrolled were able to

complete the training

• Perceptions of key informants

• Training reports

• Trainee attendance

sheets

• Document

review

• Consultations

with key

informants

61

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

3.2.2 To what

extent did the

trained

Congolese and

host community

youth gain

increased access

to employment

opportunities

through

placement and

apprenticeship

by 2017?

(Outcome 1.2)

• 95% of the trained youth practice what

they learnt through access to

employment

• Perceptions of key informants

• Tracer study forms

• Impact assessments and

evaluation reports

• Document

review

• Consultations

with key

informants

4.0 Efficiency 4.1 To what

extent does the

LL2E Approach

allow for efficient

project

management in

host countries?

4.1.1 What

evidence is

there to show

that LL2E

projects have

been managed

by approved

financial policies

and procedures

in Nepal/

Uganda?

• Evidence of use of internationally

approved procedures for financial

management

• Evidence of financial management

controls (budgeting, verification and

payment controls, authorisation,

record-keeping, auditing, reporting

etc.)

• Evidence of achievement of financial

milestones and/or objectives

• Evidence of timely disbursements

• Perceptions of key informants

• Audited financial reports

• Project management

reports

• Annual reports

• Strategic reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

• Document

review

• Consultations

with key

informants

62

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

4.1.2 In what

ways has

capacity for

future project

management

been built in

Nepal/ Uganda?

• Evidence of training/ coaching for

project staff in:

Financial management

Human resource management

Project management

Monitoring and evaluation

Perceptions of key informants

• Project management

reports

• Project logframe

• Project monitoring

reports

• Annual reports

• Strategic reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

• Document

review

• Consultations

with key

informants

63

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

4.1.3 To what

extent have LL2E

projects in

Nepal/ Uganda

provided value-

for-money?

• Evidence of production of intended

outputs

• Evidence of outcomes achievement

• Cost of inputs/ resources relative to

outputs

• Capacity to generate quality outputs at

least cost

• Capacity to generate quality outputs

on time and within budget

• Project logframe

• Project monitoring

reports

• Project management

reports

• Internal financial reports

and revised budgets

• Annual reports

• Strategic reports

• Project proposals

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

• Document

review

• Consultations

with key

informants

64

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

4.2 How suitable

is the LL2E

Approach for

enabling projects

in host countries

to manage for

development

results?

4.1.4 What

success have

projects had in

Nepal/ Uganda

in achieving

expected

results?

• Evidence of results achievement in

alignment with project logframe

(Nepal)

• Evidence of results achievement in

alignment with project logframe

(Uganda)

• Thematic strategic

documents

• Project logframe

• Project monitoring

reports

• Project proposals

Annual reports

• FCA Strategic reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

65

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

4.1.5 What were

the main

challenges

experienced by

LL2E projects in

Nepal/ Uganda

(including the

reasons for

these

challenges)?

• Documented/ Expressed challenges to

results achievement

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

66

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

4.1.6 What

measures, if

any, were used

to improve the

quality of BTVET

in Nepal/

Uganda?

• Evidence of steps taken to improve

BTVET quality

• Examples of improve measures

• Perceptions of key informants

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

• Document

review

• Consultations

with key

informants

67

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

4.1.7 To what

extent were

projects in

Nepal and

Uganda

different/

similar, and how

did these

differences/

similarities

affect results

achievement?

• Observed/ expressed differences in

LL2E project elements within Nepal

• Observed/ expressed differences

between LL2E projects in Nepal and

Uganda

• Observed/ documented effects of

project similarities/ differences on

results achievement within Nepal

• Observed/ documented effects of

project similarities/ differences on

results achievement in Nepal and

Uganda

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

68

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

4.1.8 To what

extent was

results

achievement in

Nepal affected

by differences in

location/

context (e.g.

urban/ rural)?

• Observed/ documented effects of

project similarities/ differences on

results achievement within Nepal

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

69

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

5.0 Participation

and Ownership

5.1 To what

extent have Nepal

and Uganda

demonstrated full

ownership

(support/

ownership/

participation) of

the LL2E

Approach?

5.1.1 Is there

any evidence of

support and

participation by

government

(national

government/

local

authorities/

local

government) or

the private

sector in Nepal/

Uganda in LL2E

project

activities?

• Evidence of government/ private

sector support for project

implementation (e.g. financial/ human/

physical resources) in Nepal

• Evidence of government/ private

sector support for project

implementation (e.g. financial/ human/

physical resources) in Uganda

• Evidence of policy uptake/ steps

towards policy uptake by government/

private sector agencies in Nepal

• Evidence of policy uptake/ steps

towards policy uptake by government/

private sector agencies in Uganda

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

70

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

5.1.2 To what

extent was

advocacy built

into project

activities and

actively

pursued?

• Evidence of advocacy component in

project design

• Evidence of advocacy during project

implementation:

Lobbying of government/ education

authorities; private sector agencies;

communities; etc.

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

71

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

5.1.3 To what

extent, and

under what

conditions (as

applicable),

have project

beneficiaries

become

involved in

planning, design

and/or

implementation

of LL2E project

activities?

• Evidence of beneficiary involvement in

project decision-making

• Evidence of beneficiary volunteering

activities during project planning/

design / implementation

• Evidence of provisions (stipends; travel

cost reimbursement; child care

facilities, etc.) for beneficiary

involvement in project decision-making

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

72

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

5.1.4 How

inclusive are

project activities

to persons from

vulnerable

communities/

groups (e.g.

residents of

hard-to-reach

areas; members

of cultural

communities,

persons with

physical

challenges; etc.)

• Evidence of recruitment of trainees

from vulnerable communities/ groups

• Evidence of inclusion of trainees from

vulnerable communities/ groups in

project activities

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

73

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

6.0 Sustainability 6.1 How

successful is the

LL2E Approach in

facilitating results

sustainability into

the long-term?

6.1.1 Is there

evidence to

show that LL2E

is likely to have

impact (results

sustainability)

beyond each 12-

month period of

funding?

• Evidence of sustainability planning in

Nepal

• Evidence of sustainability planning in

Uganda

• Evidence of project ownership by

government/ authorities in Nepal

• Evidence of project ownership by

government/ authorities in Uganda

• Evidence of project ownership by

beneficiaries in Nepal

• Evidence of project ownership by

beneficiaries in Uganda

• Evidence of advocacy for project

activities in Nepal

• Evidence of advocacy for project

activities in Uganda

• Evidence of policy uptake in Nepal

• Evidence of policy uptake in Uganda

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

74

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

6.1.2 How can

LL2E projects be

used to

generate

unexpected, but

sustainable

development

results (e.g. new

models of

thinking/ doing;

examples for

further action;

etc.)?

• Examples (documented or observed) of

project capacity to generate

unexpected but sustainable results in

Nepal

• Examples (documented or observed) of

project capacity to generate

unexpected but sustainable results in

Uganda

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

75

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

6.1.3 What are

the main risks to

project

sustainability in

Nepal/ Uganda?

• Evidence of lack of sustainability

planning in Nepal

• Evidence of lack of sustainability

planning in Uganda

• Evidence of lack of advocacy for

project activities in Nepal

• Evidence of lack of advocacy for

project activities in Uganda

• Evidence of lack of policy uptake for

project activities in Nepal

• Evidence of lack of policy uptake for

project activities in Uganda

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

76

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

7.0 Cross-cutting

theme: Gender

7.1 In what ways

has the LL2E

Approach

contributed to

the integration of

gender

considerations?

7.1.1 What

evidence is

there to show

that gender was

addressed

during the

design of LL2E

projects in

Nepal/ Uganda?

• Evidence of the integration of gender

equality considerations and practices

(equal participation; advancement of

women’s; access and control by

women) in project design

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

77

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

7.1.2 How was

gender

integrated into

LL2E project

implementation

activities in

Nepal/ Uganda?

• Evidence of the integration of gender

equality considerations and practices

(equal participation; advancement of

women’s; access and control by

women) during project

implementation

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

78

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

8.0 Lessons

learned

8.1 To what

extent has the

application of the

LL2E Approach

generated lessons

learned to inform

future

programming?

8.1.1 What are

the main lessons

from LL2E

project

implementation

in Nepal/

Uganda?

• Lessons learned from project

implementation in Nepal

• Lessons learned from project

implementation in Uganda

• Synthesis of results of

data analysis

• Review of

evidence from

analysed results

8.1.2 What are

the specific

lessons that

show project

elements in

Nepal/ Uganda

that work well?

• Examples of projects elements in Nepal

that have worked well

• Examples of projects elements in

Uganda that have worked well

• Synthesis of results of

data analysis

• Review of

evidence from

analysed results

8.1.3 Which

lessons highlight

project

elements in

Nepal/ Uganda

that need to be

strengthened in

the future

projects?

• Examples of projects elements in Nepal

that require strengthening

• Examples of projects elements in

Uganda that require strengthening

• Synthesis of results of

data analysis

• Review of

evidence from

analysed results

79

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

8.1.4 Is there

any evidence to

show that

project

implementing

teams in Nepal/

Uganda

systematically

collected and

shared lessons

learned?

• Evidence of systematic collection and

sharing of lessons learned from project

implementation in Nepal

• Evidence of systematic collection and

sharing of lessons learned from project

implementation in Uganda

• Perceptions of key informants

• Project proposals

• Annual reports

• Project management

reports

• Project monitoring

reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

80

Issues Key Evaluation

Questions

Examples of

Sub-Questions Indicators Main Sources of Data

Means of

Verification

8.1.5 Using the

Nepal/ Uganda

experience, how

can lessons

learned from

LL2E project

implementation

be documented

and shared with

other countries?

• Examples of ways in which project

implementation lessons learned can be

shared

• Perceptions of key informants

• Project proposals

• Project management/

monitoring reports

• Consultations with key

informants:

FCA, Head Office

FCA, Country Office

UCEP

Project beneficiaries

BTVET providers

Private sector

representatives

Government/

Education authorities

• Document

review

• Consultations

with key

informants

9.0

Recommendations

9.1 What are the

main

recommendations

for the

finalisation of the

comprehensive

LL2E Programme?

9.1.1 In what

ways can the

LL2E programme

be strengthened

strategically?

• Evidence of LL2E strategic limitations

and/ or strengths

• Main evaluation findings

and lessons learned

• Review of

evidence from

analysed results

9.1.2 How can

the LL2E

programme be

strengthened

operationally?

• Evidence of LL2E operational

limitations and/ or strengths

• Main evaluation findings

and lessons learned

• Review of

evidence from

analysed results

81

Appendix III List of Key Informants Consulted

Nepal:

NAME

(SURNAME, First Name) TITLE - AFFILIATION SEX

METHOD OF

CONSULTATION

ACHARYA, Bipin Executive Director – Underprivileged

Children’s Educational Programs, Nepal Male

Individual

interview

BAJRACHARYA, Sanju Placement Officer – Underprivileged

Children’s Educational Programs, Nepal Male

Individual

interview

BHANDARI, Ram Bahadur General Secretary – Kailali Chamber of

Commerce Male

Focus group

discussion

BHANDARI, Yam

Director – Council for Technical Education

and Vocational Training, National Skill

Testing Board

Male Small-group

interview

BHATTA, Jyoti Member – Kailali Chamber of Commerce Female Focus group

discussion

BHATTCIRAI, Mahesh

Director, Training Development Division –

Council for Technical Education and

Vocational Training

Male Small-group

interview

BISLOOKARMA,

Siddhartha Representative – Metal Industries Male

Focus group

discussion

CHAH, Rona Contractor/ Representative, Construction

Association Male

Focus group

discussion

CHAPAI, Parasram Chairman – Kailali Chamber of Commerce Male Focus group

discussion

CHAUDHARY, Dhanmarath

Assistant TVET Instructor, Masonry –

Underprivileged Children’s Educational

Programs, Nepal, Mobile Training

Male Small-group

interview

CHAUDHARY, Gopaul Contractor Male Focus group

discussion

CHAUDHARY, Indra Programme Coordinator – Freed Kamaiya

Women Development Forum Male

Individual

interview

CHAUDHARY, Jyothi

TVET Instructor, Masonry – Underprivileged

Children’s Educational Programs, Nepal,

Mobile Training

Male Small-group

interview

CHAUDHARY, Khusiram Contractor/ Member – Chamber of

Commerce Male

Focus group

discussion

82

NAME

(SURNAME, First Name) TITLE - AFFILIATION SEX

METHOD OF

CONSULTATION

CHAUDHARY, Krishna

Chandra Mason Male

Focus group

discussion

CHAUDHARY, Rajendra Owner – Raj Tailors Male Focus group

discussion

CHAUDHARY, Rukmani Vice-President – Chamber of Commerce Male Focus group

discussion

CHAUDHARY, Sural Mobile Shop Owner Male Focus group

discussion

DALI, Lokesh Raj Managing Director – Pioner Consultants Ltd Male Focus-group

discussion

DAS, Sookdeo Kuran Contractor/ Representative, Construction

Association Male

Focus group

discussion

DHUNGNA, Chandra

Kumar

Principal – Underprivileged Children’s

Educational Programs, Nepal, Sano Thimi

Technical School

Male Small-group

interview

DHUNGNA, Pradumna

TVET Instructor, Motorcycle Mechanics –

Underprivileged Children’s Educational

Programs, Nepal, Sano Thimi Technical

School

Male Focus group

discussion

FULARI, Shiv Kumar Representative – Metal Industries Male Focus group

discussion

GIRI, Surendra Vice Chairman – Kailali Chamber of

Commerce Male

Focus group

discussion

HADA, Binita Assistant Accounts Officer - Underprivileged

Children’s Educational Programs, Nepal Female

Small-group

interview

JOSHI, Krisham Owner – Construction Dhangadi Male Focus group

discussion

KAINI, Bhim District Coordinator – Underprivileged

Children’s Educational Programs, Nepal Male

Individual

interview

KARKI, Hari Programme Coordinator – Finn Church Aid,

Nepal Country Office Male

Individual

interview

KHATRI, Sailesh

TVET Instructor, Plumbing and Sanitation –

Underprivileged Children’s Educational

Programs, Nepal, Sano Thimi Technical

School

Male Focus group

discussion

83

NAME

(SURNAME, First Name) TITLE - AFFILIATION SEX

METHOD OF

CONSULTATION

MANGRANTI, Rohit

TVET Instructor, Mobile Repair –

Underprivileged Children’s Educational

Programs, Nepal, Sano Thimi Technical

School

Male Focus group

discussion

MUKTI, Badhur Bisha

TVET Instructor, Automobile –

Underprivileged Children’s Educational

Programs, Nepal, Sano Thimi Technical

School

Male Focus group

discussion

MUNANKARMT,

Purushotiam

Accounts Officer – Underprivileged

Children’s Educational Programs, Nepal Male

Small-group

interview

NEUPANE, Dependra

Vice-Principal– Underprivileged Children’s

Educational Programs, Nepal, Sano Thimi

Technical School

Male Small-group

interview

NEUPANE, Saroj Programme Coordinator – Underprivileged

Children’s Educational Programs, Nepal Male

Individual

interview

RAHUL, Netra Bhadur Contractor/ Representative, Construction

Association Male

Focus group

discussion

RANA, Jay Raj Owner – Jay Auto Male Focus group

discussion

SAUD, Kedar Automobile Mechanic Male Focus group

discussion

SHAH, Rishi Kesh Automobile Mechanic Male Focus group

discussion

SINGH, Pushkar Owner – JME Motors Male Focus group

discussion

SITARAM Contractor Male Focus group

discussion

SUNAR, Hikmat Bahadur Member – Chamber of Commerce Male Focus group

discussion

UHUMIRE, Santoshi

TVET Instructor, Electrical Wiring –

Underprivileged Children’s Educational

Programs, Nepal, Sano Thimi Technical

School

Male Focus group

discussion

84

AGENCY - AFFILIATION REPRESENTATION METHOD OF CONSULTATION

Haliya Foundation

Federation 18 members

Focus group discussion

(11 men; 7 women)

Haliya Foundation

Federation 7 members

Focus group discussion

(Women only)

Skills Training and Technical

Education for Employment

Program in Nepal (STEP

Project)

20 trainees

(Mobile Training – Masonry)

Focus group discussion

(Young men)

Skills Training and Technical

Education for Employment

Program in Nepal (STEP

Project)

25 graduates

Interviews/ Focus group

discussions

(21 young men; 4 young women)

Uganda:

NAME

(SURNAME, First Name) TITLE - AFFILIATION SEX

METHOD OF

CONSULTATION

ADJEARI, Nasaba TVET Instructor, Hairdressing – Finn Church

Aid BTVET Centre, Rwamwanja

Female Telephone

interview

AGABA, Hudson Field Associate – United Nations High

Commissioner for Refugees

Male Individual

interview

BYARUGABA, Elisa Farmer Male Individual

interview

HUHTELA, Kaisa Humanitarian Coordinator – Finn Church Aid

Uganda Program

Female Individual

interview

KANYESIGYE, Robert Manager – Rwamwanja Country Resort Male Individual

interview

KUTAMBA, Winnie Programme Officer – Finn Church Aid Uganda

Program

Female Individual

interview

MAJWALA, William Senior Instructor – Finn Church Aid BTVET

Centre, Rwamwanja

Male Individual

interview

MUGABI, Caphas Project Coordinator – Finn Church Aid

Uganda Program

Male Unstructured

discussion; Email

exchange

MUHIIRWE, Ronald Farmer Male Small-group

interview

85

NAME

(SURNAME, First Name) TITLE - AFFILIATION SEX

METHOD OF

CONSULTATION

MUTAMBUZI, Coleb Farmer Male Individual

interview; Small

group interview

NABANKEMA, Milly Manager – Uganda Association of Private

Vocational Institutions

Female Small-group

discussion

NAZZIWA, Olivia

Nanyonga

Career Counsellor – Finn Church Aid Uganda

Program

Female Individual

interview

NDEMERE, Adrian National Chairman – Uganda Association of

Private Vocational Institutions

Male Small-group

discussion

NSHEKA, Wycliffe Country Director – Finn Church Aid Uganda

Program

Male Individual

interview

NYAKATO, Loy Hotel Manager Female Individual

interview

OBWORI, Emmanuel Roving Financial Advisor – Finn Church Aid Male Individual

interview

OKUMU, Michael Ag. Deputy Director, Assessment and

Certification – Ministry of Education and

Sports, Directorate of Industrial Training

Male Individual

interview

RUHINDA, Ivan Protection Officer – Office of the Prime

Minister

Male Individual

interview

SANDIBERG, Helena Regional Educational Specialist – Finn Church

Aid Uganda Program

Female Individual

interview

SSENKIMA, Stephen Programme Coordinator – Finn Church Aid

Uganda Program

Male Individual

interview

TURYASIIMA, Deus

Dickens

Project Officer – Finn Church Aid Uganda

Program

Male Individual

interview

AGENCY - AFFILIATION REPRESENTATION METHOD OF CONSULTATION

Rwamwanja Refugee Project

3 ex-trainees

(Agriculture Vocational Education and

Training – AVET)

Focus group discussion

(Young women)

Rwamwanja Refugee Project

3 ex-trainees

(Agriculture Vocational Education and

Training – AVET)

Focus group discussion

(Young men)

86

AGENCY - AFFILIATION REPRESENTATION METHOD OF CONSULTATION

Rwamwanja Refugee Project 4 trainees

(Welding)

Focus group discussion

(Young men)

Rwamwanja Refugee Project

1 ex-trainee

(Agriculture Vocational Education and

Training – AVET)

Individual interview

(Young woman)

Rwamwanja Refugee Project 1 ex-trainee

(Catering)

Individual interview

(Young woman)

Rwamwanja Refugee Project 1 ex-trainee

(Bricklaying and Concrete practices)

Individual interview

(Young man)

Finland:

NAME

(SURNAME, First Name) TITLE - AFFILIATION SEX

METHOD OF

CONSULTATION

AUBRET, Nathalie Desk Officer, East Africa – Finn Church Aid Female Individual

interview

FESSARD, Frédéric Planning Monitoring and Evaluation

Coordinator Male

Individual

interview

PELTOLA, Minna Education Thematic Advisor – Finn Church

Aid Female

Individual

interview; Email

exchange

SARASALMI, Ulla Livelihoods Thematic Advisor – Finn

Church Aid Female

Individual

interview

WACKLIN, Ville Project Manager – Finn Church Aid Male Individual

interview

WAISMAA, Aila Desk Officer, Asia – Finn Church Aid Female Individual

interview

87

Appendix IV Fieldwork Itinerary

Nepal

DATE ACTIVITY

DAY 1:October 29, 2017

STEP Project Presentation and Question and Answer Session

Consultations with Key informants - UCEP

Consultations with Training instructors – Sano Thimi Technical School

Consultations with Project Trainees – Residential Training

Observation of Training Sessions – Residential Training

DAY 2: October 30, 2017 Field visits to Enterprises – Consultations with Training Graduates,

Kathmandu Valley

DAY 3: October 31, 2017 Transit to Far West Region – Dhangadi, Kailali; Dadeldhura, Amargadi

DAY 4: November 1, 2017

Field visit to Local Partner Headquarters – Haliya Federation

Presentation by Haliya Federation

Consultations with Local Partner – Haliya Federation

Field visit to Enterprises – Consultations with Training Graduates, Dadeldhura

DAY 5: November 2, 2017 Field visit to Enterprises – Consultations with Training Graduates, Dadeldhura

DAY 6: November 3, 2017

Field visit to Training Site – Mobile Training

Consultations with Project Trainees – Mobile Training

Field visits to Enterprises – Consultations with Training Graduates, Kailali

Consultation with Key informant – Finn Church Aid Country Office

DAY 7: November 4, 2017

Field visits to Enterprises – Consultations with Training Graduates, Kailali

Consultations with Private Sector Enterprises – Tikapur, Kailali

Consultations with Key Informants - Freed Kamaiya Women Development

Forum

DAY 8: November 5, 2017 Consultations with Private Sector Enterprises –Dhangadi, Kailali

Return to Kathmandu

DAY 9- 10: November 6 -7, 2017 Consolidation

DAY 11: November 11, 2017

Consultation with Stakeholder Organisation – Council for Technical Education

and Vocational Training

Debriefing

88

Uganda

DATE ACTIVITY

DAY 1: November 13, 2017

Country Programme Briefing

BTVET Project Briefing

Consultations with Key informants – Finn Church Aid Uganda Country Office

DAY 2: November 14, 2017

Consultations with Key informants – Finn Church Aid Uganda Country Office

Consultations with Stakeholder Organisation – Directorate of Industrial Training

Consultations with Stakeholder Organisation – Uganda Association of Private

Vocational Institutions

Informal Discussion with Key informant – Finn Church Aid Sub-project Office

Rwamwanja

DAY 3: November 15, 2017

Field visit to Project Community – Rwamwanja Refugee Settlement

Settlement and Project Briefing

Consultations with Stakeholder Organisation – Office of the Prime Minister

Consultations with Stakeholder Organisation – United Nations High Commissioner

for Refugees

Visit to Training Centre

Consultation with Key informant – Head of Training, Rwamwanja Refugee

Settlement BTVET Centre

Consultations with Training Graduates

Consultations with Project Trainees – Residential Training

Field visit to Trainee Restaurant – Consultation with Project Graduate

Consultation with Key informant – Finn Church Aid Sub-project Office

Rwamwanja

DAY 4: November 16, 2017

Field visits to Enterprises – Consultations with Training Graduates

Field visits to Industrial Training Sites – Consultations with Private Sector

Enterprises

Consultation with Training Graduate

Consultation with Training Instructor – Rwamwanja Refugee Settlement BTVET

Centre

DAY 5: November 17, 2017

Return to Kampala

Consolidation

Debriefing

89

Appendix V List of Documents Reviewed

Council for Technical Education and Vocational Training, 2017. National Occupational Skill Standard.

Madhyapur, Thimi: Council for Technical Education and Vocational Training

Council for Technical Education and Vocational Training, 2013. Technical and Vocational Education and

Training Development Journal. 1(13)

Directorate of Industrial Training, 2016. Flexible Modular Assessment Results 2016

Finn Church Aid, 2015. Right to Quality Education. Helsinki: Finn Church Aid

Finn Church Aid, n.d. Action for Human Dignity: FCA Education Strategy: R2QE Linking Learning to

Earning. PowerPoint document

Finn Church Aid, n.d. Action for Human Dignity: Right to Livelihood Linking Learning to Earning.

PowerPoint document

Finn Church Aid, n.d. Pilot: Linking Learning to Earning for Gainful Employment and Entrepreneurship

Finn Church Aid, n.d. Right to Quality Education Leaflet

Finn Church Aid, n.d. Teacher Education as a Tool in Contributing to Quality and Linking Learning to

Earning (LL2E) in FCA

Finn Church Aid, n.d. FCA Teacher Training Package: Protective Quality Education for Every Learner.

Finn Church Aid, n.d. Vocational Teacher Education Programme in FCA: Teacher Education as a Tool in

Contributing to Quality and Linking Learning to Earning (LL2E). PowerPoint document

Finn Church Aid Rwamwanja, 2016. Workplan for 1st January - 31st December 2016

Finn Church Aid Uganda, 2017. FCA Quarterly Monitoring Report 1st January – 31st March 2017

Finn Church Aid Uganda, 2017. FCA Quarterly Monitoring Report 1st July, August, September 2017

Finn Church Aid Uganda, 2017. Rwamwanja BTVET Project Budget and Workplan, 1st January – 31st

December 2017

Finn Church Aid Uganda, 2016. Rwamwanja BTVET Project for Congolese Refugee Youth and Host

Community. Third Quarter Report

Finn Church Aid Uganda, 2016. Rwamwanja BTVET Project for Congolese Refugee Youth and Host

Community. Second Quarter Report

Finn Church Aid Uganda, 2016. End of Quarter One Rwamwanja Report for the BTVET Project 2016

Finn Church Aid Uganda, 2015. Rwamwanja TVET Project: Annual Report

Finn Church Aid Uganda, 2015. Logical Framework for Rwamwanja TVET Project

90

Finn Church Aid Uganda, 2015. Rwamwanja TVET Project for Congolese Refugee Youth and Host

Community. Project Proposal

Finn Church Aid Uganda, n.d. Selection Criteria: BTVET Selection Criteria for Trainees

Finn Church Aid Uganda, 2016. The Annual Report for Rwamwanja BTVET Project for the Congolese

Refugees and Host Community Youths for 2016

Munthali, A., Mvula, P., and Silo, L., 2014. Early Childhood Development: The Role of Community-based

Childcare Centres in Malawi. SpringerPlus, [online] Available at:

https://link.springer.com/content/pdf/10.1186%2F2193-1801-3-305.pdf [Accessed 14 December 2014].

Peltola, M., n.d. Action for Human Dignity: Right to Quality Education in R2QE Orientation. PowerPoint

document

Peltola, M., n.d. Linking Learning to Earning Guidance (draft)

Prad Initiative Ltd, 2014. Final BTVET Assessment Report to Finn Church Aid. Kampala: Prad Initiative Ltd

The Lutheran World Federation Nepal, 2013. Promotion of Women’s Economic, Social and Cultural

Rights in Lalitpur, January 1, 2014 – December 31, 2016

TVET Knowledge Management Network, 2014. National Seminar on TVET: Potential and Pitfalls in the

TVET Sub-Sector – A Learning Exchange Opportunity. Madhyapur, Thimi: Council for Technical Education

and Vocational Training

Underprivileged Children's Educational Programs Nepal, n.d. Strides. Sanothimi, Bhakthapur:

Underprivileged Children's Educational Programs Nepal.

Underprivileged Children's Educational Programs Nepal, 2015. A Proposal on Skills Training and

Technical Education for Employment Programme in Nepal (STEP Nepal), January 2015 – December 2017

Underprivileged Children's Educational Programs Nepal, n.d. Logframe: Socio-Economic Empowerment

of Disadvantage Women and Girls (SEED) Project, January 2017 – December 2019

Underprivileged Children's Educational Programs Nepal, n.d. Project Summary: Action for Advancing

Freedom of Freed Haliyas, Ex- Kamaiyas and Kamlaris-(AAFreedHAKK), 2015 – 2017.

United Nations High Commissioner for Refugees, 2014. Uganda: UNHCR Global Appeal 2014 – 2015.

91

Appendix VI Data Collection Tools/ Protocols

Interview Protocol: BTVET Providers

Introduction:

Finn Church Aid (FCA), the largest Finnish organisation for development cooperation, and the largest

provider of humanitarian assistance, has developed several approaches in the areas of education and

livelihoods, to increase the resilience of vulnerable communities. Included among that approaches that

have been developed is the Linking Learning to Earning (LL2E) Approach, which has been developed

primarily for youth and women in vulnerable situations, including post-conflict and refuges contexts.

FCA has tested various elements of the LL2E Approach in project activities in several countries over the

past four years, including Nepal and Uganda. FCA is now engaged in the process of consolidating the

pilot elements into a comprehensive LL2E programme. To support this process, FCA has contracted Ms

Halcyon Louis, international development consultant to conduct an impact assessment of the LL2E

Approach. As its main purpose, the assessment will be used to determine which elements of the LL2E

Approach have worked well, and which areas require further strengthening. The assessment will be

conducted in Nepal and Uganda during October – December 2017, and will assess the following aspects

of the LL2E Approach: impact; relevance; effectiveness; efficiency; participation and ownership;

sustainability; and gender integration. In addition, the assessment has been designed to generate key

lessons and recommendations for future programme activities.

All discussions conducted during the assessment will be kept confidential. The consultant will only share

generalised findings and anonymous comments, so you will not be identified in any material that is

produced. You are therefore encouraged to speak openly and honestly about your experiences. Our

discussion will last for a maximum of 60 minutes.

Before we begin, do you have any general questions about the assessment?

Background:

• Could you tell me your exact position? How long have you been in this position?

• What are your current responsibilities?

• Please describe your involvement with the STEP Nepal Project / Rwamwanja Refugee Project?

Main Questions:

Impact

• In your opinion, how has the STEP Nepal Project / Rwamwanja Refugee Project added value to

the lives of the project trainees?

• Has the project had an impact on communities in which trainees live? If so, what impact has the

project had on these communities? How did it achieve this impact?

92

• Do you believe the project has had an impact on the communities in which the training took

place? Can you provide a reason for your response? (If an impact on communities is perceived):

How did the project achieve its impact on these communities?

• How, if at all, has the private sector benefitted from the activities of this project?

• Has the project had any impact on the labour market? If so, what impact has it had? How did the

project achieve this impact? If not, what is the reason for your response?

• How familiar are you with the goal of the project? In your opinion, has the project achieved its

intended impact? Can you explain your response?

• Were any of the impacts of the project unintended? If so, which impacts can be identified as

being unintended? How were they achieved?

• Are the project impacts short-term or will they last into the long-term? Which impacts are short-

term, and which are long-term? Please explain your response.

Relevance

• How relevant are the priorities and goals of the STEP Nepal Project / Rwamwanja Refugee

Project to the needs of the labour market? What is the reason for your response?

• In your opinion, have project activities met the needs of the trainees? If so, how have the needs

of trainees been met? If not, can you explain your response?

• In what ways has the project been suitable for the culture of community/ies in which trainings

occur?

• Have project activities been suitable for the context/ culture of the communities where the

trainees live? Can you provide a reason for your response?

• How did the project create links between the training and employment?

• Were there any subject areas or issues that were not addressed by the training but should be

included in future training?

Effectiveness

STEP Nepal Project:

• Have women and youth gained increased access to quality TVET services because of this/these

project (s)? If so, how has the project enhanced access for them? If not, what is the reason for

your response?

• Is there any evidence to suggest that TVET sub-sector knowledge and experiences have been

systematically documented because of this project? If so, how has systematic documentation

occurred? Have TVET sub-sector knowledge and experiences been widely disseminated among

women and youth? If so, can you describe the dissemination process? Do you believe this

process has been successful? Please explain your response.

93

• How, if at all, have project activities strengthened the capacity of public and private TVET service

providers to deliver quality services?

Rwamwanja Refugee Project:

• In your opinion, has the project caused Congolese youth and youth in Rwamwanja Village to

gain increased access to marketable vocational skills? If so, how did the project enable this? If

not, can you provide a reason for your response?

• Has the project enabled Congolese and Rwamwanja Village youth to gain increased access to

employment opportunities through placement and apprenticeship? Please explain your

response.

Efficiency

• In your opinion, how has the STEP Nepal Project / Rwamwanja Refugee Project provided value-

for-money?

• How successful has the STEP Nepal Project / Rwamwanja Refugee Project been in achieving its

expected results? Can you provide an explanation for your response?

• What, in your opinion, have been the main challenges experienced by the STEP Nepal Project/

Rwamwanja Refugee Project? Can you identify the reason(s) for these challenges?

• What measures, if any, were used to improve the quality of BTVET in Nepal/ Uganda?

• For Nepal only: Is there any evidence to suggest that project results were affected by

differences in location or context? Can you explain your response? Were there any similarities

between project locations that affected results achievement? If so, can you provide a reason for

your response?

Participation and Ownership

• In what ways, if at all, have government authorities supported and participated in project

activities?

• In what ways, if at all, has the private sector supported and participated in project activities?

• Can you provide examples of project advocacy, including which groups or agencies were

targeted?

• Is there any evidence to show that project beneficiaries have been involved in project planning,

design or implementation? If so, can you provide examples of their involvement?

• If project beneficiaries have been involved in project planning and decision-making, have they

been compensated in any way for their time? Have they received any provisions to support their

participation (e.g. access to child care facilities)?

94

• In what ways do project activities accommodate beneficiaries from vulnerable communities/

groups (e.g. residents of hard-to-reach areas; members of cultural communities, persons with

physical challenges)?

Sustainability

• What evidence is there to suggest that the results of project activities will be sustained into the

long-term?

• What, in your opinion, are the main risks to project sustainability? How can these risks be

addressed?

Cross-cutting theme: Gender

• Can you provide examples of how gender has been integrated into project activities?

Lessons learned

• What, in your opinion, are the main lessons from project implementation? How can these

lessons be best addressed?

• Have projects engaged in the systematic collection and sharing of lessons learned? If so, how, if

at all, have projects benefitted from this activity?

• How can the lessons learned from the STEP Nepal Project/ Rwamwanja Refugee Project be

documented and shared with other countries? In what ways can other countries benefit from

this process?

Recommendations

• What are your recommendations for improving future project activities?

• Are there any further comments or suggestions you wish to make?

Thank you for your time and participation.

95

Interview Protocol: Private Sector Representatives

Introduction:

Finn Church Aid (FCA), the largest Finnish organisation for development cooperation, and the largest

provider of humanitarian assistance, has developed several approaches in the areas of education and

livelihoods, to increase the resilience of vulnerable communities. Included among that approaches that

have been developed is the Linking Learning to Earning (LL2E) Approach, which has been developed

primarily for youth and women in vulnerable situations, including post-conflict and refuges contexts.

FCA has tested various elements of the LL2E Approach in project activities in several countries over the

past four years, including Nepal and Uganda. FCA is now engaged in the process of consolidating the

pilot elements into a comprehensive LL2E programme. To support this process, FCA has contracted Ms

Halcyon Louis, international development consultant to conduct an impact assessment of the LL2E

Approach. As its main purpose, the assessment will be used to determine which elements of the LL2E

Approach have worked well, and which areas require further strengthening. The assessment will be

conducted in Nepal and Uganda during October – December 2017, and will assess the following aspects

of the LL2E Approach: impact; relevance; effectiveness; efficiency; participation and ownership;

sustainability; and gender integration. In addition, the assessment has been designed to generate key

lessons and recommendations for future programme activities.

All discussions conducted during the assessment will be kept confidential. The consultant will only share

generalised findings and anonymous comments, so you will not be identified in any material that is

produced. You are therefore encouraged to speak openly and honestly about your experiences. Our

discussion will last for a maximum of 60 minutes.

Before we begin, do you have any general questions about the assessment?

Background:

• Could you tell me your exact position? How long have you been in this position?

• What are your current responsibilities?

• Please describe your involvement with the STEP Nepal Project / Rwamwanja Refugee Project?

Main Questions:

Impact

• In your opinion, how has the STEP Nepal Project/ Rwamwanja Refugee Project added value to

the lives of the project trainees?

• How, if at all, has the private sector benefitted from the activities of this project?

• Has the project had any impact on the labour market? If so, what impact has it had? How did the

project achieve this impact? If not, what is the reason for your response?

96

• Were any of the impacts of the project unintended? If so, which impacts can be identified as

being unintended? How were they achieved?

• Are the project impacts short-term or will they last into the long-term? Which impacts are short-

term, and which are long-term? Please explain your response.

Relevance

• How relevant are the priorities and goals of the STEP Nepal Project/ Rwamwanja Refugee

Project to the needs of the labour market? What is the reason for your response?

• In your opinion, have project activities met the needs of the trainees? If so, how have the needs

of trainees been met? If not, can you explain your response?

• In what ways has the project been suitable for the culture of community/ies in which trainings

occur?

• Have project activities been suitable for the context/ culture of the communities where the

trainees live? Can you provide a reason for your response?

• How did the project create links between the training and employment?

• Were there any subject areas or issues that were not addressed by the training but should be

included in future training?

Effectiveness

STEP Nepal Project:

• Have women and youth gained increased access to quality TVET services because of this/these

project (s)? If so, how has the project enhanced access for them? If not, what is the reason for

your response?

• Is there any evidence to suggest that TVET sub-sector knowledge and experiences have been

systematically documented because of this project? If so, how has systematic documentation

occurred? Have TVET sub-sector knowledge and experiences been widely disseminated among

women and youth? If so, can you describe the dissemination process? Do you believe this

process has been successful? Please explain your response.

• How, if at all, have project activities strengthened the capacity of public and private TVET service

providers to deliver quality services?

Rwamwanja Refugee Project:

• In your opinion, has the project caused Congolese youth and youth in Rwamwanja Village to

gain increased access to marketable vocational skills? If so, how did the project enable this? If

not, can you provide a reason for your response?

97

• Has the project enabled Congolese and Rwamwanja Village youth to gain increased access to

employment opportunities through placement and apprenticeship? Please explain your

response.

Efficiency

• In your opinion, how has the STEP Nepal Project / Rwamwanja Refugee Project provided value-

for-money?

• What, in your opinion, have been the main challenges experienced by the STEP Nepal Project/

Rwamwanja Refugee Project? Can you identify the reason(s) for these challenges?

Participation and Ownership

• In what ways, if at all, has the private sector supported and participated in project activities?

• Can you provide examples of project advocacy, including which groups or agencies were

targeted?

• Is there any evidence to show that project beneficiaries have been involved in project planning,

design or implementation? If so, can you provide examples of their involvement?

• In what ways do project activities accommodate beneficiaries from vulnerable communities/

groups (e.g. residents of hard-to-reach areas; members of cultural communities, persons with

physical challenges)?

Sustainability

• What evidence is there to suggest that the results of project activities will be sustained into the

long-term?

• What, in your opinion, are the main risks to project sustainability? How can these risks be

addressed?

Cross-cutting theme: Gender

• Can you provide examples of how gender has been integrated into project activities?

Lessons learned

• What, in your opinion, are the main lessons from project implementation? How can these

lessons be best addressed?

• How can the lessons learned from the STEP Nepal Project/ Rwamwanja Refugee Project be

documented and shared with other countries? In what ways can other countries benefit from

this process?

98

Recommendations

• What are your recommendations for improving future project activities?

• Are there any further comments or suggestions you wish to make?

Thank you for your time and participation.

99

Interview Protocol: Government Authorities and Other Education Authorities

Introduction:

Finn Church Aid (FCA), the largest Finnish organisation for development cooperation, and the largest

provider of humanitarian assistance, has developed several approaches in the areas of education and

livelihoods, to increase the resilience of vulnerable communities. Included among that approaches that

have been developed is the Linking Learning to Earning (LL2E) Approach, which has been developed

primarily for youth and women in vulnerable situations, including post-conflict and refuges contexts.

FCA has tested various elements of the LL2E Approach in project activities in several countries over the

past four years, including Nepal and Uganda. FCA is now engaged in the process of consolidating the

pilot elements into a comprehensive LL2E programme. To support this process, FCA has contracted Ms

Halcyon Louis, international development consultant to conduct an impact assessment of the LL2E

Approach. As its main purpose, the assessment will be used to determine which elements of the LL2E

Approach have worked well, and which areas require further strengthening. The assessment will be

conducted in Nepal and Uganda during October – December 2017, and will assess the following aspects

of the LL2E Approach: impact; relevance; effectiveness; efficiency; participation and ownership;

sustainability; and gender integration. In addition, the assessment has been designed to generate key

lessons and recommendations for future programme activities.

All discussions conducted during the assessment will be kept confidential. The consultant will only share

generalised findings and anonymous comments, so you will not be identified in any material that is

produced. You are therefore encouraged to speak openly and honestly about your experiences. Our

discussion will last for a maximum of 60 minutes.

Before we begin, do you have any general questions about the assessment?

Background:

• Could you tell me your exact position? How long have you been in this position?

• What are your current responsibilities?

• Please describe your involvement with the STEP Nepal Project/ Rwamwanja Refugee Project?

Main Questions:

Impact

• In your opinion, how has the STEP Nepal Project/ Rwamwanja Refugee Project added value to

the lives of the project trainees?

• Has the project had an impact on communities in which trainees live? If so, what impact has the

project had on these communities? How did it achieve this impact?

• Do you believe the project has had an impact on the communities in which the training took

place? Can you provide a reason for your response? (If an impact on communities is perceived):

How did the project achieve its impact on these communities?

100

• How, if at all, has government benefitted from the activities of this project?

• Has the project had any impact on the labour market? If so, what impact has it had? How did the

project achieve this impact? If not, what is the reason for your response?

• In your opinion, has the project achieved its intended impact? Can you explain your response?

• Were any of the impacts of the project unintended? If so, which impacts can be identified as

being unintended? How were they achieved?

• Are the project impacts short-term or will they last into the long-term? Which impacts are short-

term, and which are long-term? Please explain your response.

Relevance

• How relevant are the priorities and goals of the STEP Nepal Project/ Rwamwanja Refugee

Project to the needs of the labour market? What is the reason for your response?

• In your opinion, have project activities met the needs of the trainees? If so, how have the needs

of trainees been met? If not, can you explain your response?

• In what ways has the project been suitable for the culture of community/ies in which trainings

occur?

• Have project activities been suitable for the context/ culture of the communities where the

trainees live? Can you provide a reason for your response?

• How did the project create links between the training and employment?

• Were there any subject areas or issues that were not addressed by the training but should be

included in future training?

Effectiveness

STEP Nepal Project:

• Have women and youth gained increased access to quality TVET services because of this/these

project (s)? If so, how has the project enhanced access for them? If not, what is the reason for

your response?

• Is there any evidence to suggest that TVET sub-sector knowledge and experiences have been

systematically documented because of this project? If so, how has systematic documentation

occurred? Have TVET sub-sector knowledge and experiences been widely disseminated among

women and youth? If so, can you describe the dissemination process? Do you believe this

process has been successful? Please explain your response.

• How, if at all, have project activities strengthened the capacity of public and private TVET service

providers to deliver quality services?

101

Rwamwanja Refugee Project:

• In your opinion, has the project caused Congolese youth and youth in Rwamwanja Village to

gain increased access to marketable vocational skills? If so, how did the project enable this? If

not, can you provide a reason for your response?

• Has the project enabled Congolese and Rwamwanja Village youth to gain increased access to

employment opportunities through placement and apprenticeship? Please explain your

response.

Efficiency

• In your opinion, how has the STEP Nepal Project/ Rwamwanja Refugee Project provided value-

for-money?

• What, in your opinion, have been the main challenges experienced by the STEP Nepal Project/

Rwamwanja Refugee Project? Can you identify the reason(s) for these challenges?

Participation and Ownership

• In what ways, if at all, have government/ education authorities supported and participated in

project activities?

• Can you provide examples of project advocacy, including which groups or agencies were

targeted?

• Is there any evidence to show that project beneficiaries have been involved in project planning,

design or implementation? If so, can you provide examples of their involvement?

• If project beneficiaries have been involved in project planning and decision-making, have they

been compensated in any way for their time? Have they received any provisions to support their

participation (e.g. access to child care facilities)?

• In what ways do project activities accommodate beneficiaries from vulnerable communities/

groups (e.g. residents of hard-to-reach areas; members of cultural communities, persons with

physical challenges)?

Sustainability

• What evidence is there to suggest that the results of project activities will be sustained into the

long-term?

• What, in your opinion, are the main risks to project sustainability? How can these risks be

addressed?

Cross-cutting theme: Gender

• Can you provide examples of how gender has been integrated into project activities?

102

Lessons learned

• What, in your opinion, are the main lessons from project implementation? How can these

lessons be best addressed?

• How can the lessons learned from the STEP Nepal Project/ Rwamwanja Refugee Project be

documented and shared with other countries? In what ways can other countries benefit from

this process?

Recommendations

• What are your recommendations for improving future project activities?

• Are there any further comments or suggestions you wish to make?

Thank you for your time and participation.

103

Interview Protocol: FCA Staff (Head Office)

Introduction:

Finn Church Aid (FCA), the largest Finnish organisation for development cooperation, and the largest

provider of humanitarian assistance, has developed several approaches in the areas of education and

livelihoods, to increase the resilience of vulnerable communities. Included among that approaches that

have been developed is the Linking Learning to Earning (LL2E) Approach, which has been developed

primarily for youth and women in vulnerable situations, including post-conflict and refuges contexts.

FCA has tested various elements of the LL2E Approach in project activities in several countries over the

past four years, including Nepal and Uganda. FCA is now engaged in the process of consolidating the

pilot elements into a comprehensive LL2E programme. To support this process, FCA has contracted Ms

Halcyon Louis, international development consultant to conduct an impact assessment of the LL2E

Approach. As its main purpose, the assessment will be used to determine which elements of the LL2E

Approach have worked well, and which areas require further strengthening. The assessment will be

conducted in Nepal and Uganda during October – December 2017, and will assess the following aspects

of the LL2E Approach: impact; relevance; effectiveness; efficiency; participation and ownership;

sustainability; and gender integration. In addition, the assessment has been designed to generate key

lessons and recommendations for future programme activities.

All discussions conducted during the assessment will be kept confidential. The consultant will only share

generalised findings and anonymous comments, so you will not be identified in any material that is

produced. You are therefore encouraged to speak openly and honestly about your experiences. Our

discussion will last for a maximum of 60 minutes.

Before we begin, do you have any general questions about the assessment?

Background:

• Could you tell me your exact position? How long have you been in this position?

• What are your current responsibilities?

• Please describe your involvement with the STEP Nepal Project/ Rwamwanja Refugee Project?

Main Questions:

Impact

• In your opinion, how has the STEP Nepal Project/ Rwamwanja Refugee Project added value to

the lives of the project trainees?

• Has the project had an impact on communities in which trainees live? If so, what impact has the

project had on these communities? How did it achieve this impact?

• Do you believe the project has had an impact on the communities in which the training took

place? Can you provide a reason for your response? (If an impact on communities is perceived):

How did the project achieve its impact on these communities?

104

• How, if at all, has the private sector benefitted from the activities of this project?

• How, if at all, have government/ education authorities benefitted from the activities of this

project?

• Has the project had any impact on the labour market? If so, what impact has it had? How did the

project achieve this impact? If not, what is the reason for your response?

• In your opinion, has the project achieved its intended impact? Can you explain your response?

• Were any of the impacts of the project unintended? If so, which impacts can be identified as

being unintended? How were they achieved?

• Are the project impacts short-term or will they last into the long-term? Which impacts are short-

term, and which are long-term? Please explain your response.

Relevance

• How relevant are the priorities and goals of the STEP Nepal Project/ Rwamwanja Refugee

Project to the needs of the labour market? What is the reason for your response?

• In your opinion, have project activities met the needs of the trainees? If so, how have the needs

of trainees been met? If not, can you explain your response?

• In what ways has the project been suitable for the culture of community/ies in which trainings

occur?

• Have project activities been suitable for the context/ culture of the communities where the

trainees live? Can you provide a reason for your response?

• How did the project create links between the training and employment?

• Were there any subject areas or issues that were not addressed by the training but should be

included in future training?

Effectiveness

STEP Nepal Project:

• Have women and youth gained increased access to quality TVET services because of this/these

project (s)? If so, how has the project enhanced access for them? If not, what is the reason for

your response?

• Is there any evidence to suggest that TVET sub-sector knowledge and experiences have been

systematically documented because of this project? If so, how has systematic documentation

occurred? Have TVET sub-sector knowledge and experiences been widely disseminated among

women and youth? If so, can you describe the dissemination process? Do you believe this

process has been successful? Please explain your response.

105

• How, if at all, have project activities strengthened the capacity of public and private TVET service

providers to deliver quality services?

Rwamwanja Refugee Project:

• In your opinion, has the project caused Congolese youth and youth in Rwamwanja Village to

gain increased access to marketable vocational skills? If so, how did the project enable this? If

not, can you provide a reason for your response?

• Has the project enabled Congolese and Rwamwanja Village youth to gain increased access to

employment opportunities through placement and apprenticeship? Please explain your

response.

Efficiency

• What evidence is there to show that project activities have been managed according to

approved financial policies and procedures?

• In what ways, if at all, has the project built the capacity of UCEP/ FCA Country Office Uganda for

future project management?

• In your opinion, how has the STEP Nepal Project/ Rwamwanja Refugee Project provided value-

for-money?

• How successful has the STEP Nepal Project/ Rwamwanja Refugee Project been in achieving its

expected results? Can you provide an explanation for your response?

• What, in your opinion, have been the main challenges experienced by the STEP Nepal Project/

Rwamwanja Refugee Project? Can you identify the reason(s) for these challenges?

• What measures, if any, were used to improve the quality of BTVET in Nepal/ Uganda?

• For Nepal only: Is there any evidence to suggest that project results were affected by

differences in location or context? Can you explain your response? Were there any similarities

between project locations that affected results achievement? If so, can you provide a reason for

your response?

Participation and Ownership

• In what ways, if at all, have government authorities supported and participated in project

activities?

• In what ways, if at all, has the private sector supported and participated in project activities?

• How, if at all, was advocacy built into the project design? Can you provide examples of project

advocacy, including which groups or agencies were targeted?

• Is there any evidence to show that project beneficiaries have been involved in project planning,

design or implementation? If so, can you provide examples of their involvement?

106

• If project beneficiaries have been involved in project planning and decision-making, have they

been compensated in any way for their time? Have they received any provisions to support their

participation (e.g. access to child care facilities)?

• In what ways do project activities accommodate beneficiaries from vulnerable communities/

groups (e.g. residents of hard-to-reach areas; members of cultural communities, persons with

physical challenges?

Sustainability

• What evidence is there to suggest that the results of project activities will be sustained into the

long-term?

• Is it possible to use projects to generate sustainable development results that were not included

in the project logframe? If so, what types of results do you anticipate? What would be their

purpose? How can these results be achieved?

• What, in your opinion, are the main risks to project sustainability? How can these risks be

addressed?

Cross-cutting theme: Gender

• In what ways, if at all, was gender addressed during project design for the STEP Nepal Project/

Rwamwanja Refugee Project

• Can you provide examples of how gender has been integrated into project activities?

Lessons learned

• What, in your opinion, are the main lessons from project implementation? How can these

lessons be best addressed?

• Have projects engaged in the systematic collection and sharing of lessons learned? If so, how, if

at all, have projects benefitted from this activity?

• How can the lessons learned from the STEP Nepal Project/ Rwamwanja Refugee Project be

documented and shared with other countries? In what ways can other countries benefit from

this process?

Recommendations

• What are your recommendations for improving future project activities?

• Are there any further comments or suggestions you wish to make?

Thank you for your time and participation.

107

Interview Protocol: UCEP, Nepal and FCA Staff (Nepal/ Uganda Country Office)

Introduction:

Finn Church Aid (FCA), the largest Finnish organisation for development cooperation, and the largest

provider of humanitarian assistance, has developed several approaches in the areas of education and

livelihoods, to increase the resilience of vulnerable communities. Included among that approaches that

have been developed is the Linking Learning to Earning (LL2E) Approach, which has been developed

primarily for youth and women in vulnerable situations, including post-conflict and refuges contexts.

FCA has tested various elements of the LL2E Approach in project activities in several countries over the

past four years, including Nepal and Uganda. FCA is now engaged in the process of consolidating the

pilot elements into a comprehensive LL2E programme. To support this process, FCA has contracted Ms

Halcyon Louis, international development consultant to conduct an impact assessment of the LL2E

Approach. As its main purpose, the assessment will be used to determine which elements of the LL2E

Approach have worked well, and which areas require further strengthening. The assessment will be

conducted in Nepal and Uganda during October – December 2017, and will assess the following aspects

of the LL2E Approach: impact; relevance; effectiveness; efficiency; participation and ownership;

sustainability; and gender integration. In addition, the assessment has been designed to generate key

lessons and recommendations for future programme activities.

All discussions conducted during the assessment will be kept confidential. The consultant will only share

generalised findings and anonymous comments, so you will not be identified in any material that is

produced. You are therefore encouraged to speak openly and honestly about your experiences. Our

discussion will last for a maximum of 60 minutes.

Before we begin, do you have any general questions about the assessment?

Background:

• Could you tell me your exact position? How long have you been in this position?

• What are your current responsibilities?

• Please describe your involvement with the STEP Nepal Project/ Rwamwanja Refugee Project?

Main Questions:

Impact

• In your opinion, how has the STEP Nepal Project/ Rwamwanja Refugee Project added value to

the lives of the project trainees?

• Has the project had an impact on communities in which trainees live? If so, what impact has the

project had on these communities? How did it achieve this impact?

• Do you believe the project has had an impact on the communities in which the training took

place? Can you provide a reason for your response? (If an impact on communities is perceived):

How did the project achieve its impact on these communities?

108

• How, if at all, has the private sector benefitted from the activities of this project?

• Has the project had any impact on the labour market? If so, what impact has it had? How did the

project achieve this impact? If not, what is the reason for your response?

• In your opinion, has the project achieved its intended impact? Can you explain your response?

• Were any of the impacts of the project unintended? If so, which impacts can be identified as

being unintended? How were they achieved?

• Are the project impacts short-term or will they last into the long-term? Which impacts are short-

term, and which are long-term? Please explain your response.

Relevance

• How relevant are the priorities and goals of the STEP Nepal Project/ Rwamwanja Refugee

Project to the needs of the labour market? What is the reason for your response?

• In your opinion, have project activities met the needs of the trainees? If so, how have the needs

of trainees been met? If not, can you explain your response?

• In what ways has the project been suitable for the culture of community/ies in which trainings

occur?

• Have project activities been suitable for the context/ culture of the communities where the

trainees live? Can you provide a reason for your response?

• How did the project create links between the training and employment?

• Were there any subject areas or issues that were not addressed by the training but should be

included in future training?

Effectiveness

STEP Nepal Project:

• Have women and youth gained increased access to quality TVET services because of this/these

project (s)? If so, how has the project enhanced access for them? If not, what is the reason for

your response?

• Is there any evidence to suggest that TVET sub-sector knowledge and experiences have been

systematically documented because of this project? If so, how has systematic documentation

occurred? Have TVET sub-sector knowledge and experiences been widely disseminated among

women and youth? If so, can you describe the dissemination process? Do you believe this

process has been successful? Please explain your response.

• How, if at all, have project activities strengthened the capacity of public and private TVET service

providers to deliver quality services?

109

Rwamwanja Refugee Project:

• In your opinion, has the project caused Congolese youth and youth in Rwamwanja Village to

gain increased access to marketable vocational skills? If so, how did the project enable this? If

not, can you provide a reason for your response?

• Has the project enabled Congolese and Rwamwanja Village youth to gain increased access to

employment opportunities through placement and apprenticeship? Please explain your

response.

Efficiency

• Have project activities been managed according to approved financial policies and procedures?

Please provide examples to support your response.

• In what ways, if at all, has the project built the capacity of UCEP/ FCA Country Office Uganda for

future project management?

• In your opinion, how has the STEP Nepal Project/ Rwamwanja Refugee Project provided value-

for-money?

• How successful has the STEP Nepal Project/ Rwamwanja Refugee Project been in achieving its

expected results? Can you provide an explanation for your response?

• What, in your opinion, have been the main challenges experienced by the STEP Nepal Project/

Rwamwanja Refugee Project? Can you identify the reason(s) for these challenges?

• What measures, if any, were used to improve the quality of BTVET in Nepal/ Uganda?

• For Nepal only: Is there any evidence to suggest that project results were affected by

differences in location or context? Can you explain your response? Were there any similarities

between project locations that affected results achievement? If so, can you provide a reason for

your response?

Participation and Ownership

• In what ways, if at all, have government authorities supported and participated in project

activities?

• In what ways, if at all, has the private sector supported and participated in project activities?

• How, if at all, was advocacy built into the project design? Can you provide examples of project

advocacy, including which groups or agencies were targeted?

• Is there any evidence to show that project beneficiaries have been involved in project planning,

design or implementation? If so, can you provide examples of their involvement?

• If project beneficiaries have been involved in project planning and decision-making, have they

been compensated in any way for their time? Have they received any provisions to support their

participation (e.g. access to child care facilities)?

110

• In what ways do project activities accommodate beneficiaries from vulnerable communities/

groups (e.g. residents of hard-to-reach areas; members of cultural communities, persons with

physical challenges?

Sustainability

• What evidence is there to suggest that the results of project activities will be sustained into the

long-term?

• Is it possible to use the project(s) to generate sustainable development results that were not

included in the project logframe? If so, what types of results do you anticipate? What would be

their purpose? How can these results be achieved?

• What, in your opinion, are the main risks to project sustainability? How can these risks be

addressed?

Cross-cutting theme: Gender

• In what ways, if at all, was gender addressed during project design for the STEP Nepal Project/

Rwamwanja Refugee Project?

• Can you provide examples of how gender has been integrated into project activities?

Lessons learned

• What, in your opinion, are the main lessons from project implementation? How can these

lessons be best addressed?

• Have projects engaged in the systematic collection and sharing of lessons learned? If so, how, if

at all, have projects benefitted from this activity?

• How can the lessons learned from the STEP Nepal Project/ Rwamwanja Refugee Project be

documented and shared with other countries? In what ways can other countries benefit from

this process?

Recommendations

• What are your recommendations for improving future project activities?

• Are there any further comments or suggestions you wish to make?

Thank you for your time and participation.

111

Interview/ Focus Group Protocol: Project Graduates

Introduction:

Good morning/ Good afternoon. My name is Ms Halcyon Louis. I have been asked by Finn Church Aid to

speak with some of the persons who have received training through the STEP Nepal Project/

Rwamwanja Refugee Project. Finn Church Aid has collaborated with persons in Nepal/ Uganda to

develop the training material that was used, and now wishes to get your comments on whether the

training has worked or whether it needs to be improved in some areas.

The stories you provide today will be kept confidential. You will not be identified by name when they are

forwarded to Finn Church Aid. You are therefore encouraged to speak openly and honestly about your

experiences. Our discussion will last for a maximum of 60 minutes.

Before we begin, do you have any general questions?

Prompts/ Questions:

• Please tell me a story. Tell me about yourself.

• How did you learn about this project and why did you decide to participate?

• What has this project done for you?

• Has the project helped your family and your community in any way? (If yes) how has it helped

them?

• Please describe the most significant thing this project has done for you?

• Why is this significant to you?

• What can be done to improve the project so that other persons can benefit?

Thank you for your time and participation.

112

Appendix VII Map of Findings and Recommendations

Evaluation

Issues Findings

Related ToR

Impact

Assessment

Questions*

Related

Recommendation

Impact Finding 1: The LL2E Approach has contributed

strongly to intended behavioural and

attitudinal change among graduates of LL2E

project activities. Central to this development

has been the facilitation of access by the target

groups of youths to BTVET opportunities and

follow-up support.

#1; #2

Finding 2: Notwithstanding the ‘trickle-down

effect’ of the LL2E Approach on the families

and communities of LL2E project graduates, the

intended impact of forging effective linkages

with the private sector is a work in progress.

#3; #7 Related to

Recommendation 1

Relevance Finding 3: Given its aim to link vulnerable young

persons to opportunities for improved

livelihoods, through BTVET and decent work,

the LL2E Approach has strategic global- and

country-level relevance, to complement its

alignment with the FCA institutional mandate.

#2; #7

Finding 4: From a conceptual standpoint, the

proposed LL2E programmatic design responds

to the situation of unemployed youth from

vulnerable/ marginalised communities. Piloted

components have not, however, established

adequate linkages between the trainings and

the private sector for the employment of

graduates/ ex-trainees.

#1; #5; #7 Related to

Recommendation 1

Effectiveness Finding 5: By combining classroom training with

practical application and opportunities for

certification, the LL2E Approach has increased

access by HKK women and youths to quality

BTVET services. There is room, however, for

further enhancement of trainings to increase

individual marketability and improve livelihood.

#1; #2; #4; #5;

#7

Related to

Recommendations 1,

3, 4

113

Evaluation

Issues Findings

Related ToR

Impact

Assessment

Questions*

Related

Recommendation

Finding 6: The need to systematically document

and disseminate knowledge and experiences at

project-level may conflict with the underlying

objectives of LL2E. There is an equal possibility

for misalignment between the approach to

capacity-strengthening for BTVET delivery and

the scope of LL2E.

#7 Related to

Recommendation 4

Finding 7: LL2E project activities have equipped

Congolese youths with marketable vocational

skills and opportunities for industrial training.

Access to opportunities for employment and

apprenticeship are highly dependent, however,

on vocational specialisation.

#7 Related to

Recommendation 4

Efficiency Finding 8: The LL2E Approach is strongly

supported by standard procedures for the

management of results, risk and financial

resources. Efficient project implementation is

contingent, however, on resource availability.

#7 Related to

Recommendation 4

Finding 9: The LL2E Approach provides some

value for money (VfM) from a theoretical

perspective. There is potential for increased

VfM, however, through greater private sector

engagement and increased capacity-building

for implementing agencies.

#7

Related to

Recommendations 1

and 4

Gender Finding 10: While the application of the LL2E

Approach has allowed for the integration of

gender considerations at project-level,

participation by young women in the trainings

is, at times, challenged by domestic

commitments.

#7 Related to

Recommendation 3

Participation

and Ownership

Finding 11: The full national ownership of the

LL2E Approach through policy uptake and

programme planning is a work in progress.

Discussion and collaboration between

government and LL2E project implementing

agencies is ongoing.

#7

Related to

Recommendation 1

and 2

114

Evaluation

Issues Findings

Related ToR

Impact

Assessment

Questions*

Related

Recommendation

Finding 12: The use of advocacy as a core

element of LL2E has been actively pursued as a

mechanism for awareness-raising within

project communities. It is debatable, however,

whether the potential contribution of advocacy

to LL2E has been fully explored.

#6; #7

Related to

Recommendations 1

and 2

Sustainability Finding 13: BTVET ‘skilling’ acquired through

LL2E lends itself to results sustainability into

the medium-term. Results sustainability into

the long-term is conditional, however, on the

establishment of support structures for the

marketability of acquired skills.

#3; #7 Related to

Recommendation 1

Finding 14: Based on initial steps taken during

project planning and implementation, there is

potential for the programmatic sustainability of

LL2E beyond its funding cycle. The sustainability

of the approach is highly reliant, however, on

policy uptake by government and/ or the

private sector.

#3; #7

Related to

Recommendations 1

and 2

* Key Questions to be answered by the assessment (ToR, Section 5)

#1. How was FCA’s approach to LL2E suitable to the contexts selected by the projects?

#2. What were the major components of the approach that led to sustainable improvement in the lives

of the beneficiaries?

#3. What limited the impact and subsequent sustainability of the intervention., comparing the different

settings and countries?

#4. What measures took place in order to improve the quality of BTVET?

#5. What kinds of methods were used to link BTVET to employment (e.g. study visits; workplace earning;

career counselling; business incubation)

#6. What kinds of advocacy components re: LL2E were used, if any?

#7. What were the main challenges?