ENGINE II External Evaluator

Attachment A. Statement of Work

Name of Project: ENGINE IILocation: Mercy Corps Abuja NigeriaRecipient of Services: Mercy Corps

1. Program SummaryMercy Corps Nigeria is seeking to procure the services of an Independent Evaluation Team to conduct an impact evaluation of Mercy Corps’ Educating Nigerian Girls in New Enterprises (ENGINE) II programme, which is in its transition (second) phase and is funded through the Girls Education Challenge (GEC) of the Department for International Development (DFID). The programme started in April 2017 and run through to March 2020.

The project has three major outcomes which are:● Improve learning outcomes (literacy, numeracy, financial literacy and life skills) ● Support girls to transition through key stages of education, training and employment ● Ensure sustainability of changes that are in line with learning and transition.

ENGINE II seeks to transform the future of over 21,000 marginalised girls who are aged between 17 and 23 years old by fulfilling their potential in education and work. The programme is implemented by a consortium which includes Mercy Corps - lead implementer, an external evaluator, Civil Society Organisations (CSOs), private and public sector organisations.

The Evaluation Team once engaged will provide an independent and rigorous evaluation and research function, implementing a framework which will assess the process of delivery, effectiveness, Value for Money (VfM) and impact of the programme, additionally the Evaluation Team will report findings and lessons learnt throughout the process. The overall aim is to evaluate the programmes’ outcomes and intermediate outcomes during the project’s Baseline, Midline and Endline evaluations using rigorous quantitative and qualitative methods.

It is against this background that bidders (evaluation firms) are invited to submit a proposal to evaluate ENGINE II in accordance with the guidelines outlined below.

1.1. Purpose and Type of Evaluation

1

ENGINE II External Evaluator1.1.1. Rationale The overall aim of evaluating ENGINE II is to evaluate the programmes’ outcomes and intermediate outcomes using rigorous quantitative and qualitative methods.

The findings from the evaluation will primarily be used:● By the project management team, project partners and stakeholders to inform

improvements in the delivery of the project during its lifetime;● To demonstrate accountability for the funding received to DFID, other UK Government

Departments, UK taxpayers, UK media; ● By the project management team to leverage additional resources from existing and

new partners and stakeholders in order to scale-up and sustain the activities /benefits delivered by the project;

● By the project management team to support the ongoing development and implementation of the project’s sustainability and succession strategies;

● By partners, stakeholders and the Government to learn lessons from the project for the purpose of replicating what works elsewhere and/or taking up approaches and activities that have proven to work in order to scale up the project;

● By the Fund Manager to feed into and identify insights in order to inform programme level questions; and

● By other donors, academic institutions and education networks to inform the wider policy debate concerning the education of girls and marginalised girls.

1.1.2. Evaluation ObjectiveENGINE II shall conduct a mixed-method, gender-sensitive evaluation that is inclusive of persons with disabilities of the project over the period of project implementation. The evaluation will assess the delivery, effectiveness, VfM and impact of the project and report the findings and lessons learnt throughout the process.

1.1.3. Evaluation ApproachThe overall evaluation approach requires the Evaluation Team to design, plan and conduct a mixed-methods evaluation that is longitudinal in nature. The Evaluation Team will need to consider the following:

● The programme’s evaluation objectives and evaluation questions and the programmes relationship to these;

● The complexity and clarity of the programme’s logframe, design, evaluation questions and the measurability of the intended outcomes and the effect this has on its long-term evaluation;

● The requirements of the GEC Programme Evaluation Strategy and the planned approach to evaluation and data collection;

● Availability and quality of existing evidence and data sources; and

2

ENGINE II External EvaluatorThe proportionate amount of time and resources that should be allocated to evaluation given the type of project interventions, operational context and the reporting requirements of the GEC.

2. Background and Relevant Documents ENGINE II is a follow-on to ENGINE I, which was implemented by Mercy Corps (managing partner) in three states of Kano, Kaduna and Lagos and the Federal Capital Territory (FCT) through CSOs. The project targeted 18,000 marginalised (in and out of school) girls aged 16 to 19 years old. The project improved learning outcomes, increased girls’ economic assets and influenced gatekeepers by working with them to create enabling environments for girls to thrive.2.1. GEC Programme Background

The Department for International Development (DFID) leads the UK’s work to end extreme poverty. DFID is tackling the global challenges of our time including poverty and disease, mass migration, insecurity and conflict. DFID’s work is building a safer, healthier, more prosperous world for people in developing countries and in the UK.

DFID is working to reach the Sustainable Development Goals (SDGs) by 2030. Progress on girls’ education is critical to the achievement of these targets. SDGs 4 and 5 specifically relate to education and achieving gender parity. SDG 4 specifically notes ‘inclusive and quality education for all and promote lifelong learning’.

Globally 31 million primary age girls, have never been to school1. And the majority of these girls come from the poorest and most marginalised communities in the most disadvantaged locations, ethnic groups etc2. Over the last 20 years primary enrolments for girls have improved along with boys but completion rates are equally low for both sexes. At the secondary level the differences between boys and girls participation rates really start to show. Significant disparities exist within countries, with the poorest girls from rural areas most severely subject to educational disadvantage - even at the primary level3.

The Girls' Education Challenge (GEC) is helping the world’s poorest girls improve their lives through education and supporting better ways of getting girls in school and ensuring they receive a quality of education to transform their future.

PricewaterhouseCoopers LLP (PwC) and alliance partners have been contracted as the dedicated Fund Manager (FM) and is responsible for the day-to-day operation of the GEC. This includes establishing the recipient tendering process, supporting bidders, sifting and scoring

1 United Nations, 2015. The World's Women 2015: Trends and Statistics. New York: United Nations, Department of Economic and Social Affairs, Statistics Division. Sales No. E.15.XVII.8.

2 Idem3 Idem

3

ENGINE II External Evaluatorproposals, monitoring Value for Money (VfM) and making project funding recommendations for DFID approval. The FM also manages the relationships with the selected projects and oversees their Monitoring, Evaluation, and Learning operations.

Through the GEC, DFID provided funds between 2012 and 2017 to the FM to disburse to 37 individual projects across 18 countries across sub-Saharan Africa and South Asia to help girl’s education. In 2016 the GEC Transition window has been set up with additional DFID funding to support the original GEC beneficiaries continue their journey through stages of education and further improve their learning4.

2.2. Existing Program Information Sources Bidders should refer to the DFID GEC website: https://www.gov.uk/guidance/girls-education-challenge for general information concerning the Girls’ Education Challenge. Bidders should also refer to relevant country data and information, as required, to prepare the proposal.

The successful bidder would be required to review the following GEC programme documentation:

● Grant Recipient Handbook● Evaluation Guidance● Logframe and workplan guidance● Additionally, the successful bidder would be required to review the following

documentation specific to ENGINE II:● Project logframe;● Full Project Application as included in the Accountable Grant Arrangement; and● Project’s MEL framework.

3. Evaluation and Design

3.1. Evaluation QuestionsThe Evaluation Team will be required to develop an evaluation approach that answers the following overarching questions as a minimum:

● Process – Was the ENGINE II project successfully designed and implemented?● Impact – What impact did the ENGINE II project have on the learning and transition of

marginalised girls, including girls with disabilities? How and why this impact was achieved across project implementation locations.

● VfM – Did the project demonstrate a good VfM approach?● Effectiveness – What worked (and did not work) to increase the learning and transition

of marginalised girls that benefitted from the ENGINE II project?

4 https://www.gov.uk/international-development-funding/girls-education-challenge#overview4

ENGINE II External Evaluator● Sustainability – How sustainable were the activities funded by the GEC and was ENGINE

II successful in leveraging additional interest and investment?

3.2. Evaluation Approach & MethodologyThe Evaluation Team, in partnership with the implementation team will be required to revise and finalise on an evaluation approach that is complementary to the programme’s implementation approach using a representative population household survey to capture the prevalence of different risk factors and literacy and numeracy skills at the baseline stage amongst the target population, an assessment of levels of exposure to the programme’s intervention and changes in intermediary outcomes at the midline stage and assessment of the outcomes achieved at the endline stage.

● Comparison groups: bidders are required to outline their approach to evaluating the impact of the project. This should include consideration of the most rigorous approach to establishing a counterfactual. This should enable comparison of the outcomes achieved by the target group who were affected by the ENGINE II intervention with the outcomes achieved by a group who are similar in every way to the target group, except that they have not in any way been exposed to or affected by the project intervention i.e. a comparison/control group. Careful consideration should be given to the use of experimental or quasi-experimental methods for this purpose.

● Cohort tracking: the project is required to track a learning cohort and a transition cohort – defined as a group of individuals who progress through life (community or school) together. Bidders should outline their approach to tracking these cohorts in both the control and intervention areas.

● Measuring outcomes: bidders are expected to understand the projects’ key and intermediate outcomes and suggest the most appropriate data collection approach to evaluate each outcome. This should include a mixture of quantitative and qualitative approaches. The Evaluator will be expected to pilot tools that will be used for data collection and refine as necessary.

3.2.1. Project Sampling FrameworkThe Evaluation Team, working with the project implementation team, will be required to review and finalise the sampling frameworks for both qualitative and quantitative samples. These should be of a sufficient size and representativeness to allow:

● Reasonable levels of certainty that the findings are representative for the target population;

● Reasonable ability to generalise the intervention’s effectiveness to similar contexts; and

5

ENGINE II External Evaluator● Reasonable ability to generalise the insights into what works and why for similar

contexts.

The Evaluation Team will be required to manage and work within a sampling framework that allows individual-level measurement of marginalisation at the girl level, since the risk factors are often located in the family or community and will not be measurable in school. This should enable the measurement of the expected risk factors that are present in the programme target area, an assessment of who is at risk of poor educational outcomes and help identify the positive benefits of the programme for its target group.

The sampling framework will allow measurement of how much intervention each of the marginalised girls is exposed to/receives. This means being able to record who has participated in the programme or has been exposed to an area level initiative, especially at the girl level. The more accurate and detailed the information is, the more likely the evaluation will be able to demonstrate a clear link to the programmes outcomes.

At the end of ENGINE I, Mercy Corps retained 21,162 beneficiaries. Across project areas, 14,034 of these were out-of-school girls and 7,128 were in school; bidders are to propose representative samples for both groups, stratified by state that would allow for statistical assessments and tracking. This includes consistent identification of (1) the same cohort at different points in time (programme impact assessment) and (2) tracking equivalent cohorts at similar stages in other contexts. Research should employ a mixed-methods design combining experimental or quasi-experimental survey data with semi-structured interviews. Propensity score matching should be employed to establish a comparison group from which to produce reliable counterfactual evidence on individual level programme impact. The sampling methodology will be finalised by the external evaluation partner and the ENGINE II team.

3.2.2. Collation of Control Schools (Kano, Kaduna, and FCT)The external evaluator in partnership with the project team will revise and finalise the list of control schools in selected local government areas of project states. This is to replace initial control schools which were included in ENGINE but unequally ranked with treatment schools and hence with more learning advantage than the treatment schools posing a challenge to comparing treatment and control schools. Schools that will be selected to be in the control local government areas in project states for ENGINE II will have similar characteristics (grades, number of students, teachers and other indicators) with the treatment schools.

6

ENGINE II External EvaluatorThe project did not work with in-school girls in its first phase of implementation in Lagos, hence there were no control schools. Since the programme will provide academic intervention to the existing out-of-school girls in Lagos, the external evaluator will be required to select a control group in Lagos.

3.3. Monitoring StrategyThe Evaluation Team will be required to support the Programme Management Team to design, establish and implement a comprehensive monitoring strategy, including a data collection strategy to support the implementation of the evaluation. Technical support should include guidance concerning the programme monitoring processes. The Evaluation Team will be expected to communicate regularly with the Programme Management Team.

3.4. Cohort TrackingThe Evaluation Team is required to track a cohort of the sampled population – defined as a group of individuals who transit on their journey through the ENGINE II programme (school, work or business). Cohort tracking is an important tool for monitoring and evaluating changes in outcomes for a particular population. In order to identify and assess the impact of the programme on the programme’s target group, it is necessary to track and measure the changes experienced by those who are not exposed to the programmes’ activities in order to compare and contrast these effects. Details will be provided in the M&E framework.

3.5. Baseline StudyThe Evaluation Team will be required to design and implement a gender sensitive mixed method baseline study as an integrated part of the overall Monitoring, Evaluation and Learning strategy and plan for the project. The Evaluation Team will be required to conduct a household mapping survey to identify sampled girls, the team will be responsible for identifying representative samples for control groups. Girls in the treatment groups must have the same or very similar characteristics to those in control groups with the only difference being the intervention of the programme.

The baseline study should identify the number of beneficiaries with disabilities as well as the type and severity of their disability, following the UN Washington Group methodology5. Bidders should set out their approach to this for the baseline study.

3.6. Impact EvaluationThe Evaluation Team will be required to design and implement a programme impact evaluation involving primary quantitative and qualitative research of beneficiaries and non-beneficiaries of

5 http://www.washingtongroup-disability.com/7

ENGINE II External EvaluatorENGINE II as a control or comparison group for the purpose of establishing a counterfactual, against which the attributable impacts of the programme can be evaluated.

Bidders should outline their approach to evaluating the impact of ENGINE II. This should include consideration of the most rigorous approach to establishing a counterfactual that enables comparison of the outcomes achieved by a target group who were affected by the programme’s intervention with the outcomes achieved by the control group who are similar in every way to the target group, except that they have not in any way been exposed to or affected by the intervention. Careful consideration should be given to the use of experimental or quasi-experimental methods for this purpose.

3.7. Research MethodsThe Evaluation Team will be required to conduct quantitative research that enables an assessment of the attributable impacts of the programme based on evidence that the target population have realised improved educational outcomes as a result of the programmes’ outputs. Although aggregate evidence from a representative selection of schools is important for some outputs and for some outcomes, such as retention, ultimately what is important is to study a representative selection of target girls’ experiences and outcomes. This involves identifying a representative sample of the target population and showing that their educational outcomes are (positively) related to programme activity. Given that out of school and in school girls will be studied, the evaluation should be at the household and school levels.

Furthermore, the team will be required to design and conduct qualitative research methodologies for the purpose of exploring and explaining the nature of causality and resulting changes that are observed. The type of methods and samples used may vary depending on the cultural constraints or opportunities presented by the programme’s context.

The qualitative research methodology should seek to examine the theory of change and intervention logic of the programme as a means of understanding and confirming the processes measured through the quantitative research, namely the linkages between marginalisation (risk, barriers), the programmes intervention and outcomes that result.

3.8. Ethical Protocols

3.8.1. Child ProtectionThe evaluation approach must consider the safety of participants and especially children at all stages of the evaluation. The evaluation team will need to demonstrate how they have

8

ENGINE II External Evaluatorconsidered the protection of children through the different evaluation stages, including recruitment and training of research staff, data collection and data analysis and report writing.

3.8.2. Research Ethics PlanBidders are required to set out their approach to ensuring complete compliance with international good practice with regards to research ethics and protocols particularly with regards to safeguarding children, vulnerable groups (including people with disabilities) and those in fragile and conflict affected states. Consideration should be given to:

● Administrative, technical and physical safeguards to protect the confidentiality of those participating in research;

● Physical safeguards for those conducting research; ● Data protection and secure maintenance procedures for personal information;● Parental consent concerning data collection from children or collation of data about

children;● Age- and ability-appropriate assent processes based on reasonable assumptions about

comprehension for the ages of children and the disabilities they intend to involve in the research;

● Age-appropriate participation of children, including in the development of data collection tools.

3.9. Risk and Risk Management

3.9.1. Risk Management PlanIt is important that the successful bidder has taken all reasonable measures to mitigate any potential risk to the delivery of the required outputs for this evaluation. Therefore, bidders should submit a comprehensive risk management plan covering:

● The assumptions underpinning the successful completion of the proposals submitted and the anticipated challenges that might be faced;

● Estimates of the level of risk for each risk identified;● Proposed contingency plans that the bidder will put in place to mitigate against any

occurrence of each risk identified;● Specific child protection risks and mitigating strategies, including reference to the child

protection policy and procedures that will be in place; ● Health and safety issues that may require significant duty of care precautions.

9

ENGINE II External Evaluator3.10. Data Quality Assurance

3.10.1. Quality Assurance Plan Bidders are required to submit a quality assurance plan that sets out the systems and processes for quality assuring the evaluation and research process and deliverables from start to finish of the project. This plan should include the proposed approaches to:

● Piloting of all research activities;● Training of enumerators and researchers conducting the mixed-methods primary

research, including training in research ethics;● Logistical and management planning;● Field work protocols and data verification including back-checking and quality control by

supervisors; and● Data cleaning and editing before any analysis.

4. Team Composition

4.1. Professional Skills and Qualifications, Experience and Eligibility Criteria

The proposed evaluation team should include evidence of the technical expertise and practical experience required to deliver on the scope of work and evaluation outputs, in particular, with regards to:

ExperienceEvaluation design: Experience with gender-sensitive design, management and implementation of primary quantitative and qualitative research in the three geopolitical zones of Nigeria is required (Southwest, Northwest and Northcentral Nigeria). The eligible evaluator should have experience in the design of longitudinal household panel surveys, EGRA /EGMA tests, in-depth interviews (IDIs), focus group discussions (FGDs), etc. Previous programme experience for the provision of similar evaluation services and the design and implementation of similar evaluation activitiesConducting evaluations of DFID funded development programmes.

Quality control: Experience in implementing quality control approaches and processes for all stages of

10

ENGINE II External Evaluatorthe evaluation process, in order to anticipate challenges and and address them with detailed mitigation plan.Evaluation Steering Committee: As a requirement of the programme, it is expected that the chosen Evaluation Team is familiar with working with reporting and management structures.Quantitative and qualitative data collection and analysis, drawing findings from multiple sources and handling potential contradictions between data sets.Primary research: Conducting and reporting a baseline assessment to writing a final programme evaluation report, as well as tracking a representative cohort over time in challenging project environments, such as fragile and conflict affected states. – IE The design of longitudinal household panel surveys, information about familiarity and experience administering literacy and numeracy surveys, experience in the administration of the Early Grade Reading Assessment (EGRA) and Early Grade Maths Assessment (EGMA) learning instruments in the past (as well as assessments for Secondary Schools) and how you intend to use these instruments to assess learning under ENGINE II, in-depth interviews (IDIs), focus group discussions (FGDs), etc.

Local presence The staff assigned as Lead in the Evaluation Team should be resident in Nigeria for the duration of the evaluation, whether through the prime company or through the local branch or sub-contracted technical consultants.Support staff that would also work on the project do not have to be resident in Nigeria, however the requisite experience in research, fieldwork and report writing is expected of all members of the evaluation team (evidence-CVs and references, of this should be included in the application).

Administrative Requirements The eligible evaluation team must provide evidence of being a legally registered company.If the prime evaluation firm is operating directly in Nigeria it must provide proof of its registration as a Nigerian company/agency (CAC registration) as well

11

ENGINE II External Evaluatoras tax clearance for the year preceding this tender.

Relevant subject matter knowledge and experience:

Knowledge and experience required on conducting gender sensitive research with children, the education sector, disability, non-state actors, and private sector education to ensure that the evaluation design and research methods are as relevant and meaningful as possible given the aims and objectives of the project and the context in which it is being delivered

Evaluation management: Research agencies interested in bidding should have capacity to design, plan, conduct and manage a large-scale mixed-method complex impact evaluation and research process, potentially using experimental or quasi-experimental techniques.

Country experience: It is particularly important that the field evaluation team, mainly the Project Manager/Team Lead, Supervisors and Enumerators are based in Nigeria and has the appropriate knowledge/experience of the geopolitical zones in Nigeria, selected to be a part of ENGINE II as well as the appropriate language proficiency required to conduct research in the different geographical areas.

Information management: Capacity to design and manage sex and disability disaggregated data and information systems capable of handling large datasets for MEL purposes. The successful Evaluation Team shall work with Mercy Corps to design and manage data and information systems capable of handling large datasets for M&E purposes, including a plan and advice on how data will be physically and electronically stored and disposed to maintain the privacy and confidentially of all programme participants, while also providing safeguards for the integrity, reliability and cross- checking of all data. The fund manager or DFID has the right to request for and review the data collected from the field and it is essential that data integrity is maintained;

Statistical analysis: Sampling, a range of statistical modelling and analysis of impact data; highly proficient user(s) of: SPSS or STATA; and qualitative data analysis techniques, including the use of software e.g. ATLAS.ti, NVivo or equivalent where needed.

12

ENGINE II External EvaluatorVfM assessment of education projects:

Education economics expertise to conduct cost benefit analysis and cost effectiveness analysis as part of the assessment of the project’s VfM;

Effective data collection The selected evaluation team should provide evidence of effective data collection, storage devices with a strong backup system to prevent loss of data, and s system that can undertake daily backup of data collection and weekly upload of reports is strongly preferred.

Safety considerations: Ensuring the whole evaluation process adheres to best practice for research with children including the implementation of child protection policy and procedures to ensure safety of participants. Note that all bidders are expected to be able to show that they have a child protection policy in place to safeguard children that the research team would come into contact with through the research activities.

4.1.1. Evidence of eligibility and experienceBidders should provide evidence of previous programme experience and qualifications for the provision of similar evaluation services and the design and implementation of similar evaluation activities.

● Bidders are required to clearly identify and provide CVs for all staff proposed in the Evaluation Team, clearly stating their roles and responsibilities for this evaluation.

● Submit a clear organogram of the staff structure that will be operational for the programme.

● Information on the number of individuals (staff, enumerators and field supervisors with proof of competence) that will be designated to evaluate ENGINE II and the firm’s ability to meet all evaluation requirements during short windows of time.

● Submit successful completion certificate as evidence from previous program experience.

5. Roles, Responsibilities and Requirements

5.1. Evaluation LeadThe Evaluation Team will be expected to identify an Evaluation (Project) Director or Manager for communication and reporting purposes. At the inception meeting the Evaluation Team’s

13

ENGINE II External EvaluatorProject Director or Manager will be expected to submit a list of all staff involved in the evaluation and this list is to include their contact information.

The Evaluation Team will be expected to attend and report to the Evaluation Steering Group and attend all meetings as agreed with Mercy Corps’ Senior Manager on Knowledge Management.

5.2. Evaluation Governance Arrangements - Programme Evaluation Steering Group

The evaluation process will be guided by a dedicated Evaluation Steering Group from the start to finish of the programme. The Evaluation Steering Group will play a critical role in the coordination between the independent Evaluation Team and the Programme Team, as well as in the process of implementation. The Group will:

● Support the evaluation by facilitating access to the documentation and data required for the purpose of evaluation;

● Regularly assess and assure the quality of the design, research and deliverables;● Provide a source of validation for the findings emerging from the evaluation; and● Ensure that findings and lessons learnt are fed back to the ENGINE programme team

and other relevant audiences in a timely way in order to maximise the utility of the evaluation process.

The Evaluation Steering Group will meet regularly (timetable to be confirmed), particularly at stages in the evaluation process when deliverables are produced, including (at the very minimum):

● Submission of a Baseline Study Report;● Design and later submission of the Midline Study Report;● Submission of emerging findings from the evaluation fieldwork; and● Design and later submission of the Final Programme Evaluation Report.

The steering group will include:● GEC Programme Evaluation Team;● GEC In-Country Coordinator;● Programme Manager ENGINE II;● Senior Manager, Knowledge Management – ENGINE II;● Country MEL Manager - Mercy Corps Nigeria● Program Director or Manager of the evaluation firm

t

14

ENGINE II External Evaluator5.3. Reporting requirements

The selected bidder shall work with Mercy Corps on the review of survey (Baseline, Midline and Endline) and field monitoring reports. This will be the responsibility of Mercy Corps; however, the external evaluator will be required to submit monthly reports in addition to actual field based reports.

All reports will be submitted in electronic form and should be submitted in English. The Evaluation Team will be required to provide face-to-face presentations in-country of all deliverables as an integral part of the submission process.

The Team will be required to submit to the Senior Manager bi-weekly progress reports (by email) during the study periods summarising activities/tasks completed to date (percent achieved), time spent, etc.

The Evaluation Team will be expected to provide a fully ‘cleaned-up’ dataset in SPSS file format accompanied by the code used to carry out analysis and a variable codebook, with full cross-tabulations of the results. It is expected that 10 - 12 sets of cross - tabulations breaking down the results for all questions and including appropriate statistical tests that reflect significant differences will be provided.

6. Expected ENGINE II Evaluation Timeframe and Sequencing

6.1. Typical Programme Milestones/Outputs DeadlineCall for expression of interest sent out 14th July 2017Deadline for submission of expression of interest and Full proposal 27th July 2017Evaluation of tenders and shortlisting complete 31st July 2017Invitation to tender is shared (Tender explanation/discussion) 31st July 2017Evaluation Firm Appointed 8th August 2017Inception PhaseOn boarding Meeting Held 9th August 2017Design and review of data collection tools 10th to 18th August 2017Draft inception report developed 21st August 2017Review of data collection tools by Mercy Corps 21st to 25th August 2017Review of tools and instruments by Fund Manager 25th-25th August 2017

15

ENGINE II External EvaluatorRevision of tools and instruments based on feedback 28th-1st September 2017Submission of final versions of data collection tools 15th September 2017Final inception report submitted 18th-22nd September 2017

Implementation PhaseBaseline Software formatting (digital scripting of data collection tools) August 2017Selection of Control Schools and control groups August 2017Training of enumerators September 2017Fieldwork and data collection Oct-Nov 2017Development of protocol and schedule for Cohort tracking October 2017Data cleaning and analysis Dec 2017Topline Report highlighting key evaluation findings Dec 2017First draft of detailed report shared with Mercy Corps January 2018Review of detailed report and feedback by Mercy Corps January 2018Revision of detailed report and submission by evaluation team February 2018Final Report shared with Fund Manager February 2018Report Presentation to project team by evaluation firm February 2018

Cohort tracking Training of enumerators Baseline+ refresherField work and data collection TBD (to be determined)Data cleaning and analysis TBDFirst draft of report shared with Mercy Corps TBDReview of draft report and provision of feedback by Mercy Corps TBDRevision of draft report and submission by evaluation team TBDFinal Report shared with Fund Manager TBDReport Presentation to project team by evaluation firm TBD

Midline Training of enumerators TBDFieldwork and data collection TBDData cleaning and analysis TBDTopline Report highlighting key evaluation findings TBD First draft of detailed report shared with Mercy Corps TBDReview of draft report and provision of feedback by Mercy Corps TBDRevision of draft report and submission by evaluation team TBD

16

ENGINE II External EvaluatorFinal Report shared with Fund Manager TBDReport Presentation to project team by evaluation firm TBD

Endline Training of enumerators TBDFieldwork and data collection TBDData cleaning and analysis TBDFirst draft of report shared with Mercy Corps TBDReview of draft report and provision of feedback by Mercy Corps TBDRevision of draft report and submission by evaluation team TBDFinal Report shared with Fund Manager TBDReport Presentation to project team by evaluation firm TBD

Exit PhaseDrafting of final evaluation report December 2019First draft of report shared with Mercy Corps January 2020Review of draft report and provision of feedback by Mercy Corps January 2020Revision of draft report and submission by evaluation team February 2020Final Report shared with Fund Manager February 2020Report Presentation to project team by evaluation firm February 2020

7. Description of Project DeliverablesThe main deliverables for this project are as follows:

● Inception report: The successful bidder shall work with the project team to set out the design of the evaluation strategy and plan and associated planning, logistics, quality assurance, child protection measures and risk management information including gender analysis.

● Baseline study report: Design and conduct a baseline study and submit a report that describes the initial conditions (before the start of the programme’s second phase – ENGINE II in April 2018) against which progress can be measured or comparisons made to show the effects and impact of the programme in the final programme evaluation report. A final report structure will be produced or provided at the inception meeting with the successful research firm. Mercy Corps will review and provide feedback on a draft evaluation report prior to finalisation.

● Midline project evaluation report: Design and conduct a midline evaluation with a report that assesses the effectiveness, impact and VfM of ENGINE II at the midline point. A final report structure will be produced or provided during the inception meeting. Mercy Corps will review and provide feedback on both the evaluation design and draft report prior to finalisation.

17

ENGINE II External Evaluator● Cohort tracking: The Evaluation Team is required to track a cohort of the population –

defined as a group of individuals who transit on their journey (school, work or business) through the ENGINE programme. It is necessary to track and measure the changes experienced by those who are not exposed to the programmes’ activities in order to compare and contrast these effects. Details will be provided in the M&E framework.

● Final project evaluation report: Design and conduct a final programme evaluation, and submit an endline report that assesses the effectiveness, impact and VfM of the programme. A final report structure will be produced or provided at the inception meeting. Mercy Corps will review and provide feedback on the evaluation design and draft report prior to finalisation.

● Detailed work plan: All bidders are required to provide a detailed work plan incorporating all relevant tasks and milestones from start to finish of the evaluation.

● Project milestones: bidders are required to include in their detailed work plans the milestones set out above (Typical Programme Milestones/Outputs ) .

8. Project BudgetThe proposed costs should reflect the costs should reflect costs that would be incurred over the life of the programme. Mercy Corps Nigeria will accept submissions for the total budget for this evaluation request for proposal in Naira, US Dollars or British Pounds. If budget quotations are submitted in a currency that is not Naira, the exchange rate Mercy Corps will use is the internal rate that is updated monthly and accepted by the donor.

The budget should be inclusive of all unit costs covering the bidders’ staff remuneration costs, travel, research costs and any other costs associated with the completion of the evaluations. Bidders are required to organise and fund their own duty of care arrangements as required. Bidders are required to provide a proposal which reflects full costs in the form of a price schedule that, as a minimum, should include:

● Sub-total of fees for the delivery of any task or deliverable;● Sub-total for number of days per partner organisation (as applicable);● Expenses and overheads broken down by the programme cost categories;

1) Fees/salaries2) Travel and accommodation 3) Training costs 4) Supplies 5) Equipment 6) Communication costs 7) Other direct costs8) Overhead

18

ENGINE II External Evaluator● Total costs before and after any taxes that are applicable. Bidders are required to

provide a payment schedule on the basis of milestone payments for the successful delivery of each deliverable.

9. Contracting Terms● The prime evaluation firm must be a registered legal entity with the ability to carry out

the evaluation in Nigeria either directly or through sub-contracted commercial arrangements.

● Payment will be submitted within 30 days of completing the milestone for the successful delivery of each deliverable. Terms and modalities of payment to be finalised during contractual negotiations.

● Payment(s) will be subject to statutory deductions as per the Nigerian law (for Nigerian registered companies), e.g WHT (Withholding Tax), etc.

● Mercy Corps will not be responsible for payment for work that does not meet pre-established specifications, criteria, and/or compliance expectations.

● Companies wishing to be considered for this solicitation must; o Be prepared to provide examples of selected materials (where applicable) upon

request by Mercy Corps.o Not be bankrupt or in the process of going bankrupt.o Not been convicted for an offense concerning professional conduct (either in

Nigeria or outside of Nigeria).o Not been guilty of grave professional misconduct (proven by any means which

the contracting authorities can justify).o Have fulfilled obligations related to payment of social security and taxes.o Not been guilty of serious misinterpretation in supplying information. o Not been in situation of conflict of interest (with prior relationship to project or

family or business relationship to parties on Commission). o Not been declared as serious fault of implementation owing to a breach of their

contractual obligations.o Not employ personnel below the legal working age.o Provide basic social rights and fair working conditions to their employees.o Not be on any list of sanctioned parties issued by the United States Government,

United Nations and/or European Union.● Companies should provide any documentation of their status as (preferred)

supplier/evaluator to government/institutional donors. Conversely companies must identify whether they are, or have ever been, excluded (“blacklisted”) as a service/goods provider by any institutional and/or government donor.

19