INDEPENDENT IN-DEPTH EVALUATION · Web view1400 Vienna, Austria Telephone: (+43-1) 26060-0 Email:...

PROJECT TITLE

INDEPENDENT IN-DEPTH EVALUATION

PLEASE ADD FULL PROGRAMME

TITLE HERE

Programme number

Month year

viii

iii

INSTRUCTIONS

YELLOW TEXT BOXES PROVIDE INSTRUCTIONS. PLEASE DELETE THEM BEFORE SUBMITTING THE REPORT. (JUST CLICK ON THE BORDER OF THE TEXT BOX AND DELETE IT.)

TEXT AREAS HIGHLIGHTED IN GREEN ARE PLACEHOLDERS. YOU WILL NEED TO REPLACE THEM WITH ACTUAL DATA AND REMOVE THE HIGHLIGHTING.

THIS TEMPLATE IS FORMATTED USING THE REQUIRED STYLES FOR HEADINGS, BODY TEXT, ETC. PLEASE FOLLOW THESE FORMATS AND STYLES BY WRITING ON TOP OF THE HEADINGS AND BODY TEXT. IF PASTING DATA FROM ANOTHER DOCUMENT, PLEASE MAKE SURE TO MATCH DESTINATION FORMATTING.

FOR THE FRONT COVER: PLEASE SELECT A SUITABLE IMAGE TO REPLACE THE STANDARD ONE IN THIS TEMPLATE.

Please visit IES’s website for additional information:

http://www.unodc.org/unodc/en/evaluation/normative-tools.html

Please consult the IES Handbook:

https://www.unodc.org/unodc/en/evaluation/evaluation-handbook.html

or contact IES directly for more guidance.

This independent evaluation report was prepared by an evaluation team consisting of (names and titles of external evaluators). The Independent Evaluation Section (IES) of the United Nations Office on Drugs and Crime (UNODC) provides normative tools, guidelines and templates to be used in the evaluation process of projects.

Please find the respective tools on the IES web site: http://www.unodc.org/unodc/en/evaluation/evaluation.html

The Independent Evaluation Section of the United Nations Office on Drugs and Crime can be contacted at:

United Nations Office on Drugs and CrimeVienna International CentreP.O. Box 5001400 Vienna, AustriaTelephone: (+43-1) 26060-0Email: [email protected]: https://www.unodc.org/unodc/de/evaluation/index.html

Disclaimer

Independent In-Depth Evaluations are managed by the Independent Evaluation Section (IES) and conducted by external independent evaluators. It is the responsibility of IES to respond to the commitment of the United Nations Evaluation Group (UNEG) in professionalizing the evaluation function and promoting a culture of evaluation within UNODC for the purposes of accountability and continuous learning and improvement.

The views expressed in this independent evaluation report are those of the evaluation team. They do not represent those of UNODC or of any of the institutions or Member States referred to in the report. All errors and omissions remain the responsibility of the evaluation team.

© United Nations, Month Year. All rights reserved worldwide.

The designations employed and the presentation of material in this publication do not imply the expression of any opinion whatsoever on the part of the Secretariat of the United Nations concerning the legal status of any country, territory, city or area, or of its authorities, or concerning the delimitation of its frontiers or boundaries.

This publication has not been formally edited.

CONTENTS

CONTENTSiii

ABBREVIATIONS AND ACRONYMSiv

EXECUTIVE SUMMARYvi

MANAGEMENT RESPONSE NARRATIVE…………………………………………………………………………………………………………vii

SUMMARY MATRIX OF FINDINGS, EVIDENCE, RECOMMENDATIONS AND MANAGEMENT RESPONSEix

I. INTRODUCTION1

Background and context1

Evaluation methodology2

Limitations to the evaluation2

II. EVALUATION FINDINGS3

Relevance3

Efficiency4

Coherence4

Effectiveness4

Impact5

Sustainability5

Human Rights, Gender Equality and leaving no one behind5

III. CONCLUSIONS7

IV. RECOMMENDATIONS8

V. LESSONS LEARNED AND BEST PRACTICES10

Lessons Learned10

Best Practices10

ANNEX I: TERMS OF REFERENCE11

ANNEX II: EVALUATION TOOLS: QUESTIONNAIRES AND INTERVIEW GUIDES12

ANNEX III: DESK REVIEW LIST13

ANNEX IV: STAKEHOLDERS CONTACTED DURING THE EVALUATION14

ANNEX V: OPTIONAL15

IN-DEPTH EVALUATION OF THE PROJECT TITLE

iii

ix

ABBREVIATIONS AND ACRONYMS

Abbreviationor Acronym

Full name

Abbreviationor Acronym

Full name

Text

Text

Text

Text

iii

ABBREVIATIONS AND ACRONYMSix

EXECUTIVE SUMMARY

INSTRUCTIONS

The Executive Summary should not be a repetition of the text of the main body but be drafted in a crisp, short and clear manner, with the objective to convey the key and most important information about the evaluation to a wide array of readers.

This section should consist of no more than four pages (equalling up to 14,000 characters including spaces) that includes:

· Short introduction with a brief description of the project including its objectives, the purpose and scope of the evaluation, the evaluation methodology and the composition of the external independent evaluation team (maximum 2 short sentences on background, expertise, number and gender);

· The main findings of the evaluation for each of the evaluation criteria;

· The main conclusions;

· The main recommendations (ranked by importance, each with a topic and implementing recipient(s), referring to the total number in the matrix and the main body. There should be a clear illustration of how the recommendations build upon the conclusions and in turn upon the findings. See more guidance in relation to the Summary Matrix and the chapter on recommendations in the main body.

· The main lessons learned and main best practice, referring to the total number in the main body of the report.

INTRODUCTION

Text

PROJECT DESCRIPTION and objectives

Text

PURPOSE, SCOPE and methodology OF EVALUATION

Text

Main findings

Text

Main conclusions

Text

Main recommendationsRecommendation 1 – TOPIC

Text

Recommendation 2 – TOPIC

Text

Recommendation 3 – TOPIC

Text

MAIN Lessons learned and best practice

IN-DEPTH EVALUATION OF THE PROGRAMME TITLE

Tex

iii

EXECUTIVE SUMMARYix

MANAGEMENT RESPONSE NARRATIVE

The Programme/Project Manager provides a Management Response narrative text in a separate document, from which IES copies and pastes below.

Text

MANAGEMENT RESPONSE NARRATIVEix

SUMMARY MATRIX OF FINDINGS, EVIDENCE, RECOMMENDATIONS AND Management response

INSTRUCTIONS

FINDINGS, EVIDENCE AND RECOMMENDATIONS

The part below should include the most significant findings with related evidence and all recommendations (each with a topic).

The recommendations should be brief and clear, as well as appropriately directed to the specific target group of implementing recipient(s) at UNODC to take action (e.g. Project Management at the specific Office/Section/Unit). The recommendations could also be directed to Project Management and refer to cooperation with specific other responsible UNODC Offices/Sections/Units. Utility is promoted when the recommendations are SMART (Specific, Measurable, Actionable, Results-oriented, Time-bound).

The maximum number of recommendations is 10. They should be ranked by importance, beginning with the most important recommendations and finish with the least important ones.

Recommendations need to be clearly derived from the findings and conclusions.

The evidence column should illustrate the general sources of the findings that lead to the recommendations (e.g. desk review of project documents and progress reports; interviews with internal stakeholders; interviews with recipients; etc.).

PLEASE NOTE: The findings and recommendations should be brief and succinct to match the format of a matrix and not be directly copied from the findings or recommendations chapter in the main body.

MANAGEMENT RESPONSE

The Programme/Project Manager completes an Evaluation Follow-up Plan with a Management Response in a separate document, from which IES copies information relating to the Management Response and pastes below in the last column.

Findings

Evidence[footnoteRef:1] [1: General sources that substantiate the findings.]

Recommendations[footnoteRef:2] [2: Should include the specific target group of implementing recipient(s) at UNODC.]

Management Response[footnoteRef:3] [3: Accepted/partially accepted or rejected for each recommendation. For any recommendation that is partially accepted or rejected, a short justification is to be added.]

1. Text

Text

1. Text

2.

2.

3.

3.

4.

4.

5.

5.

6.

6.

7.

7.

8.

8.

9.

9.

10.

10.

Note: A finding uses triangulated evidence from the data collection to allow for a factual statement.

Note: Recommendations are proposals aimed at enhancing the effectiveness, quality, or efficiency; at redesigning the objectives; and/or at the reallocation of resources. For accuracy and credibility, recommendations should be the logical implications of the findings and conclusions and need to clearly identify the responsible implementing recipient(s).

iii

SUMMARY MATRIX OF FINDINGS, EVIDENCE, RECOMMENDATIONS AND MaNAGEMENT rESPONSEix

INSTRUCTIONS

The main body of the report should not exceed 25-30 pages, excluding the four annexes (up to 100,000 characters including spaces). The report should further include relevant maps, graphs, pictures etc. of project implementation. Please note that title and source for each is required.

I. INTRODUCTIONBackground and context

INSTRUCTIONS

This sub-section should include:

· The overall concept and design of the project, including an assessment of its strategy; duration; financial and human resources; as well as clarity, logic and coherence of the project document;

· The purpose (objective) and scope (coverage) of the evaluation;

· The composition of the external independent evaluation team (including short background; expertise; number and gender of evaluators);

· A map of the countries in which the project is active (please refer to the UN Geospatial information website: http://www.un.org/Depts/Cartographic/english/htmain.htm)

OVERALL CONCEPT AND DESIGN

Text

Purpose and scope

Text

The composition of the evaluation team

Text

MAP OF PROJECT COUNTRIES

INSERT MAP HERE

MAP CAPTION

Evaluation methodology

INSTRUCTIONS

This section should describe the approach and methods used to obtain, collect and analyze the data. This part is very important as it provides the basis for the credibility of the evaluation results. Reference should be made to the annex encompassing the evaluation tools.

The evaluation methodology should support the purpose of the evaluation. It should further answer each of the evaluation questions posed in the TOR and further refined in the Inception Report.

The evaluation report should contain a logical sequence; evidence-assessment: assessment-findings; findings/conclusion-recommendations.

Reference must be made to the desk review, identification and inclusion of stakeholders in the evaluation process, sampling strategy and triangulation.

Please include at least the following graphics:

· Number and type of stakeholders interviewed (e.g. pie chart), including sex of interviewees, and number and type of stakeholders surveyed (if applicable). Title and source for each graphic is required.

· Countries included in the data collection.

Text

Limitations to the evaluation

INSTRUCTIONS

The report should highlight major constraints that had an impact on the evaluation process, i.e. limited budget, limited time and unavailability of some major stakeholders for interviews. This section should further include how these limitations were overcome.

Text

INDEPENDENT EVALUATION OF: XXXX

9

INTRODUCTION2

II. EVALUATION FINDINGS

INSTRUCTIONS

This section is the most important one since it covers the analysis of information and articulates the findings of the evaluation.

It is the longest and most detailed section of the report and it should be based on facts, proven by reference to source/methodology. The other sections of the report should draw on and make references to this part.

A finding uses evidence from several sources to allow for a factual statement.

All questions for each criteria from the cleared Inception Report need to be included in the box at the top. These should be followed by the key findings relating to the questions.

When writing this section, please keep in mind that findings are presented for both documentation purposes and for substantiating the conclusions, recommendations and lessons learned of the evaluation.

Please also clearly relate the evaluation to the role of UNODC in supporting the implementation of the SDGs, UNODC’s role in the country, UNODC’s cooperation with other UN agencies, etc. – in line with the Evaluation ToR.

A box at the end should include the main findings/summary for each criteria. The text should be no longer than three sentences.

PLEASE NOTE: Relevant maps, graphics, statistics, pictures, etc. should be included where possible. Please further note that title and source for each is required.

RelevanceEvaluation questions:

Insert questions from the cleared Inception Report.

Text

Summary - RELEVANCE

Insert maximum three synthesized sentences of the key findings.

EfficiencyEvaluation questions:


Text

Summary – Efficiency


CoherenceEvaluation questions:


Text

Summary – Coherence


EffectivenessEvaluation questions:


Text

Summary – EFFECTIVENESS


ImpactEvaluation questions:


Text

Summary – IMPACT


SustainabilityEvaluation questions:


Text

Summary – SUSTAINABILITY


Human Rights, Gender Equality and leaving no one behindEvaluation questions:


Human Rights

Text

Gender Equality

Text

Leaving no one behind

Text

Summary – Human Rights, Gender Equality and leaving no one behind


EVALUATION FINDINGS2

III. CONCLUSIONS

INSTRUCTIONS

The report must draw overall conclusions based on the evaluation findings. Conclusions should add value to the findings and draw on data collection and analyses undertaken through a transparent chain of arguments.

Conclusions point out the factors of success and failure of the evaluated project, with special attention paid to the intended and unintended results and impacts, and more generally to any other strength or weakness.

PLEASE NOTE: There must be a clear link between the findings, the conclusions and the recommendations.

Text

9

CONCLUSIONS2

IV. RECOMMENDATIONS

INSTRUCTIONS

This part of the report should provide more detailed information on the recommendations than in the Summary Matrix.

The recommendations should be clear, useful, time-bound and actionable, aimed at enhancing the project performance and improving the sustainability of results.

The recommendations should be aimed, for example, at improving project design, delivery and overall management. In addition, be directed on the topic under evaluation, as well as towards changing policy.

The recommendations should clearly build upon the conclusions, which in turn build upon the findings.

Each recommendation should clearly indicate the action to be undertaken or the decision to be made, as well as the specific group of implementing recipient(s) (same as in the Summary Matrix).

PLEASE NOTE: Recommendations are not directed at Governments as the objective of the evaluation is the respective project.

The maximum number of recommendations is 10. They should be ranked by importance, beginning with the most important recommendations and finish with the least important ones. The topic of each recommendation should be clearly indicated with a sub-heading. Please also provide the related finding/conclusion for each recommendation.

Utility is promoted when the recommendations are SMART (Specific, Measurable, Actionable, Results-oriented, Time-bound), prioritized and appropriately pitched to the target group responsible for taking action. To further support utilization of evaluation results, the recommendations should easily allow the implementing recipient(s) to address them in the Management Response and the Evaluation Follow-up Plan.

RECOMMENDATION 1 - TOPICRecommendation

Text


Text


Text


Text


Text


Text


Text


Text


Text


Text

9

RECOMMENDATIONS2

V. LESSONS LEARNED AND BEST PRACTICES

INSTRUCTIONS

Lessons learned are “Generalizations based on evaluation experiences with projects, programs, or policies that abstract from the specific circumstances to broader situations. Frequently, lessons highlight strengths or weaknesses in preparation, design, and implementation that affect performance, outcome, and impact.”

Lessons learned are a key component of any knowledge management system and they are important for continuously improving the performance of organizations like UNODC. Sometimes these lessons will be derived from success and sometimes they will be derived from areas where there is room for improvement.

The purpose of a lesson learned is to see what works and what does not. Lessons can be success stories that should be repeated, or they can be areas in which change towards improvement is to take place. They can offer advice on how to improve processes (how things were done) or products (outputs).

The evaluation report should focus on the most important lessons, especially those with wider applicability and those that have the following characteristics:

· The lessons learned from a specific project should highlight the strengths and weaknesses in preparation, design and implementation that affect performance, outcomes and impact. They are also applicable to other projects and programs, as well as policies, and have the potential to improve future actions.

· Lessons learned should be based on findings and evidence presented in the report. Lessons learned should neither be written as a recommendation, nor as an observation or description.

Lessons Learned

Text

Best Practices

Text

9

LESSONS LEARNED AND BEST PRACTICES2

INSTRUCTIONS

Apart from the mandatory four annexes: Terms of reference of the evaluation; Evaluation tools: questionnaires and interview guides; Desk review list; and, Stakeholders contacted during the evaluation - additional annexes may be included in case they add value to the evaluation report (e.g. case studies; list of outputs and outcomes achieved/not achieved; etc.)

PLEASE ENSURE THAT THE SAME CORRECT FORMAT AND STYLES OF THE MAIN BODY OF THE REPORT ARE USED WHEN FORMATTING THE ANNEXES.

ANNEX I: TERMS OF REFERENCE

Insert text from the cleared Terms of Reference. Do not include annexes from the ToR.

9

ANNEX I: TERMS OF REFERENCE2

ANNEX II: EVALUATION TOOLS: QUESTIONNAIRES AND INTERVIEW GUIDES

9

ANNEX II: EVALUATION TOOLS: QUESTIONNAIRES AND INTERVIEW GUIDES2

ANNEX III: DESK REVIEW LIST UNODC documents

Number

Title

Text

Total number of UNODC documents reviewed:

External documents

Number

Title

Text

Total number of external documents reviewed:

9

ANNEX III: DESK REVIEW LIST2

ANNEX IV: STAKEHOLDERS CONTACTEDDURING THE EVALUATION

Number of interviewees

Organisation

Type of stakeholder (see note below)

Sex disaggregated data

Country

Text

Text

Male:

Female:

Text

Male:

Female:

Male:

Female:

Male:

Female:

Male:

Female:

Male:

Female:

Male:

Female:

Male:

Female:

Total:

Male:

Female:

Note: A stakeholder could be a Civil Society Organisation; Project/Programme implementer; Government recipient; Donor; Academia/Research institute; etc.

9

ANNEX IV: STAKEHOLDERS CONTACTED DURING THE EVALUATION2

ANNEX V: OPTIONAL

9

ANNEX V:2

INDEPENDENT IN-DEPTH EVALUATION · Web view1400 Vienna, Austria Telephone: (+43-1) 26060-0 Email:...

Documents

Transcript of INDEPENDENT IN-DEPTH EVALUATION · Web view1400 Vienna, Austria Telephone: (+43-1) 26060-0 Email:...