TOOLKIT ON EVALUATION - EvalPartners 2016. 7. 7.¢  UNICEF/ ECO has compiled this...

download TOOLKIT ON EVALUATION - EvalPartners 2016. 7. 7.¢  UNICEF/ ECO has compiled this Evaluation toolkit

of 134

  • date post

    08-Sep-2020
  • Category

    Documents

  • view

    2
  • download

    0

Embed Size (px)

Transcript of TOOLKIT ON EVALUATION - EvalPartners 2016. 7. 7.¢  UNICEF/ ECO has compiled this...

  • TOOLKIT ON EVALUATION

    UNITED NATIONS CHILDREN’S FUND/ EGYPT COUNTRY OFFICE

    November 2008

  • i

    UNICEF/ ECO/ SPME/2008/ Technical Paper

    01/Toolkit

    Cover photo was borrowed from: United Nations Systems Staff College (UNSSC) Workshop on: “What a UN Evaluator Needs to Know” Copyright reserved, 2008 UNICEF/ Egypt website: www.unicef.org.egypt

  • ii

    List of Abbreviations and Acronyms

    CIDA Canadian International Development Agency DAG Development Assistance Group DAC Development Assistance DFID Department for International Development ECO Egypt Country Office HRBA Human Rights Based Approach IMEF Integrated Monitoring and Evaluation Framework IMEP Integrated Monitoring and Evaluation Plan M&E Monitoring and Evaluation MDGs Millennium Development Goals MTSP Mid Term Strategic Plan NGOs Non-Governmental Organizations OIA Office of Internal Audit RBM Results Based Management TOR Terms of Reference UN United Nations UNFPA United Nations for Population Agency UNEG United Nations Evaluation Group UNICEF United Nations Fund for Children USAID United States Aid

  • iii

    Table of Contents Abbreviations and Acronyms Introduction CHAPTER 1: UNICEF EVALUATION POLICY AND STANDARDS

    1.1 Scope of UNICEF Evaluation Policy ……………………………………………….. 2 1.1.1 Purpose and Use of Evaluation ……………………………………………… 5 1.1.2 Guiding Principles of Evaluation at UNICEF …………………………. 5 1.1.3 Evaluation in context of Medium-Term Strategic Plan ………… 7 1.1.4 Review of Evaluation Policy …………………………………………………… 7 1.1.5 UNICEF Program Evaluation Standards ………………………………. 8 1.1.6 Evaluation Criteria …………………………………………………………………. 11

    1.2 Organization of Evaluation in UNICEF ………………………………………….. 13

    1.2.1 Decentralized Evaluation Function ………………………………………. 13 1.2.2 Roles and Responsibilities for Evaluation in UNICEF ………..… 14 1.2.3 Evaluation with Partners …….………………………………………………… 17

    1.3 Measures to Strengthen the Evaluation System ……………………….…... 19

    1.3.1 Evaluation and Results Based Management ………………………..…. 19 1.3.2 Evaluation and Human Rights Based Approach …………….……….. 19 1.3.3 Evaluation and Knowledge Generation System ….……………..…… 20 1.3.4 Gender Sensitivity …………………….……………………………………………… 20 1.3.5 Evaluation and Professionalization of Evaluators …………….…….. 20 1.3.6 Evaluation and Special Supplementary Funds …………..………….. 21 1.3.7 Evaluation and Accountability to OIA ……………………………………… 22

    CHAPTER TWO: EVALUATION THEORIES, TYPES AND TRENDS

    2.1 Evaluation Theories and Models …………………………….……………………….. 23 2.2 Types of Evaluation ………………………………………………………..……………….. 25

    2.2.1 Type of Evaluation by Time ………………………………………………………… 26 2.2.2 Type by Evaluation Process …………………………………………………………. 27 2.2.3 Type by subject ……………………………………………………………………………. 27

  • iv

    2.3 Participatory Evaluation ……………………………………………………………………. 28 2.3.1 What is a Participatory Evaluation? .......................................... 28 2.3.2 Characteristics of Participatory Evaluation …………………………………… 28 2.3.3 Steps of Conducting Participatory Evaluation ………………………………. 29

    2.4 Evaluation Trends ……………………………………………………………………………….. 30

    2.4.1 Paradigmatic Shift in the Concept of Evaluation …………………….…….. 30 2.4.2 Internalizing Evaluation Culture …………………………………………………... 32

    2.4.3 Common Concerns and Directives about Evaluation ……….…………… 33 2.4.4 Factors to Determine the Focus of Evaluation ………………………………. 35 2.4.5 Cost of Evaluation ….….…………………………………………..…………………...... 36

    CHAPTER THREE: PLANNING EVALUATION 3.1 Purpose for Conducting Evaluation ………………….……………………………….. 38 3.2 Scope and Focus of Evaluation …………………..……………………………………. 39 3.3 Resources to Commission Evaluation ………………..……………………………. 39 3.4 Development of Terms of Reference (TOR) ……………..……………………. 41 3.5 Stakeholders’ Participation ………………………..…….…………….………………… 42

    CHAPTER FOUR: CONTRACTING AND MANAGING EVALUATORS 4.1 Selection of Appropriate Evaluator(s) …………………………………………... 44 4.2 Contracting Evaluator(s) …………..…………………………………………..…….. 45 4.3 Brief Evaluation Team and Set Ground for Action ……...………………. 46 4.4 Roles and Responsibilities of Evaluation Team …………………………….. 47 4.5 What to Do When you have Trouble Hiring an Evaluator ………….... 50 4.6 Possible Problems with Evaluators and Suggested Solutions ………. 51

    CHAPTER FIVE: CONDUCTING EVALUATION 5.1 Agree on the Design of the Evaluation Plan …………………………………….…. 53 5.2 Conducting Field Missions …………................................………………… 55 5.3 Data Collection ……………………………………………………………………………….. 58 5.4 Data Analysis and Interpretation ……………………………………………………. 59 5.5 Identifying Results …………………………………………………………………………… 59

  • v

    CHAPTER SIX: REPORTING, DISSEMINATION AND USING OF RESULTS 6.1 Reporting ………………………………………………………………………………………….. 61 6.2 Types of Reporting and Roles of Parties Involved …………………………… 61 6.3 Evaluation Final Report Format ………………….……………………………………. 63 6.4 Assessment of Final Evaluation Report ………………………………………….… 64 6.5 Utilization Focused Approach in Evaluation ….…………………………………. 69 6.6 Follow up Evaluation …………………………………………………………………………. 70 6.7 Using Results to Obtain Support ………………………………………………………. 71 6.8 Dissemination Strategy for Evaluation Results ………………………………… 73

    GLOSSARY OF KEY TERMS IN EVALUATION AND RBM …………………….. 78

    ANNEXES Annex-1 Checklist on Assessing HRBAP in Monitoring and Evaluation Annex-2 Tips for Sensitizing Evaluation Annex-3 Evaluation Plan Checklist Annex-4 Model Template for developing an Evaluation TOR Annex-5 UNICEF Detailed TOR Model Annex-6 Evaluation Contracts Checklist Annex-7 Code of Conduct for Evaluation Consultants Annex-8 Matrix for Facilitating and Managing Evaluations Annex-9 Checklist for Roles of Evaluation Managers and Teams Annex-10 Key steps in carrying Evaluation Activities Annex-11 Scoring Evaluation Plans Annex-12 Data Collection Methods Annex-13 Writing a Good Executive Summary Annex-14 Generic Structure for Evaluation Reports Annex-15 Matrix for Quality Criteria of Evaluation Report

  • 1

    Introduction UNICEF/ ECO has compiled this Evaluation toolkit with the aim to improve the quality of evaluation reports to enhance results based programming and better serve children, and to facilitate an improved application of UNICEF evaluation policy and standards. The information included in this toolkit is brought together from different sources in particular UNICEF PPP Manual and guides on Monitoring and Evaluation, UNEG, DFID, CIDA, UNFPA and other sources. Indeed, it is more reasonable and sound to draw on the existing resources, optimally use what other partners produced and broaden our scope of research and evaluation. No doubt that exchanging knowledge and generating collective ideas would enrich evidence based research and boost development efforts to better serve people. The toolkit is developed to serve as a useful reference with regards to the UNEG evaluation standards and UNICEF evaluation policy. The kit will take the reader step by step through the practice of planning and designing, managing, reporting on, disseminating and using the results of an evaluation. The prime audience of the kit is basically UNICEF Staff, research entities, partners an