Independent Evaluation Office (IEO) Engagement with ...
Transcript of Independent Evaluation Office (IEO) Engagement with ...
Independent Evaluation Office (IEO) Engagement with International Program for Development Evaluation Training (IPDET)
Independent Evaluation Office Networking and Participation Report
2
TABLE OF CONTENTS
I. Background .............................................................................................................................. 3
II. IEO participation at IPDET 2014 .............................................................................................. 4
III. On the margin meetings .......................................................................................................... 5
IV. Annex ....................................................................................................................................... 7
3
I. BACKGROUND
The International Program for Development Evaluation Training (IPDET) is a leading executive
training program that was established in 2001. As an initiative of the Independent Evaluation
Group of the World Bank and the Faculty of Public Affairs at Carleton University, an IPDET
flagship training program is offered each summer in Ottawa, Canada. The program is directed
at managers and practitioners and aims to provide the tools required to evaluate development
policies, programs, and projects at local, national, regional, and global levels. To date, the
program has graduated close to 3,500 participants from over 125 countries. IPDET is taught by
renowned international faculty and is co-directed by Ray Rist and Linda Morra Imas, both
leading evaluation experts and former Evaluation Capacity Building Advisors at the Independent
Evaluation Group of the World Bank.
Representing the Independent Evaluation Office (IEO) of UNDP and United Nations Evaluation
Group (UNEG), the Director of the IEO, who is also a Vice-Chair of UNEG, remains engaged with
IPDET as visiting faculty. He previously developed and co-delivered courses from 2008 on the
evaluation of governance with the Director of the IEO of the Global Environmental facility (GEF).
Engagement with IPDET remains important and useful, and the IEO will continue to build this
relationship.
Since joining UNDP in 2012 he has been presenting the lunch time guest lecture and
participating in panels and workshops sharing insights into the evaluation function and IEO of
the UNDP. For example, in 2013, the Director’s participation included a co-presentation at
workshops led by Dr. Ray Rist (on setting up M&E systems) and Professor RM Levin (on
evaluating governance in Africa). These interactive workshops included an exploration of the
use of relevant techniques, instruments and methodologies.
Many leading global and national actors were represented at IPDET 2014, including participants
from government ministries, national commissions, non-governmental organizations, private
sector, think tanks and international development agencies. For example, participants over the
fourth week (when the Director of the IEO was present) included senior representatives from:
Ministry of Environmental Protection China, Ministry of Health Ghana, Ministry of Planning &
Administrative Development Palestine, Office of the Public Service Commission South Africa,
International Atomic Energy Agency Austria, Gross Happiness Commission Bhutan, African
Union Commission Ethiopia, World Health Organization, Swiss Agency for Development and
Cooperation Mongolia, World Bank Group, The Danish Evaluation Institute, SOS Children’s
Villages Georgia, Caribbean Development Bank, Asian Development Bank, Aga Khan Foundation
Tajikistan, and International Development and Research Centre Canada.
4
II. IEO PARTICIPATION AT IPDET 2014
Key note address by Director
At this year’s summer training program, the Director of the IEO delivered a lecture entitled
“Independence, Credibility and Use’. The presentation shared experiences of the IEO and
UNEG in evaluation, emphasizing the importance of ensuring independence, credibility, and
utility in evaluation. For example, topics discussed included the significance of quality
assurance, the role of institutional linkage and capacity, and the need for stakeholder
engagement. Some of the key messages conveyed in the presentation included the notion that
nothing is too complex to be measured and that evaluations must address issues surrounding
efficiency and management performance. The declaration of EvalPartners on 2015 as the Year
of Evaluation was also introduced to participants. Powerpoint slides from the presentation are
included here:
Ipdet 2014 Indran
Naidoo.pptx
Overall, the lecture was well received by participants. The audience was engaged and attentive
to hearing about the work and experiences of the UNDP IEO and UNEG. Following the lecture,
members of the audience commented on the professional nature of the presentation and
relevance of insights shared to their own work. Questions posed related to questions of
independence and management uptake of recommendations. The presentation was video
recorded and will soon be on the international IPDET and other websites.
Participation of Director on a Heads of Agencies panel
As in previous years, the Director of the IEO participated on an international panel on
“Managing and Evaluation Unit” alongside the following:
Rob van den Berg (Director: IEO – GEF)
Goberdhan Singh (Former Director: CIDA)
Maliha Khan (Oxfam USA)
Niels Dabelstein (former Director, DANIDA Evaluation unit – and former Peer Review of
the UNDP Evaluation Policy)
The workshop addressed issues surrounding setting up a new evaluation unit, including issues
surrounding structuring, staffing, budgeting, and creating good products. Panel members were
5
asked to share knowledge and insights into these and other key issues. Discussion included
topics of structural and behavioural independence and management uptake of
recommendations.
Participation of Deputy Director and Staff Member of IEO, UNDP
In IPDET 2014, Juha Uitto, Deputy Director of the IEO, UNDP was a co-instructor with Robert
van den Berg (Director of GEF IEO) for a workshop on “Challenges in Evaluating Environment
and Development Issues”. The workshop tackled difficulties of scope, timing, and linkages
between climate change, natural resources management, poverty and development in
evaluations. The presentation covered some of the topics discussed in a recent book on
“Evaluating Environment in International Development”, edited by Juha Uitto with a forward by
Indran Naidoo
Roberto La Rovere from the IEO UNDP was a participant at IPDET 2014. Several staff from the
UN system (for example, UNICEF, UNEP, UN Women) and members of UNDP Country Offices
(for example Bosnia and Herzegovina, Congo and Zimbabwe) also participated in this year’s
program.
III. ON THE MARGIN MEETINGS
Planning for the 2015 National Evaluation Capacity (NEC) conference
Preparatory work is underway for the next NEC event that will be held in the Asia region in
September/October of 2015. The previous NEC event partnered with the Government and
Brazil and EvalPartners, and a proceedings has just been released.
It has been decided to extend the reach further for the next event, and plans are afoot to
partner with the International Development Evaluation Association (IDEAS). The IEO has
supported IDEAS previously, and participated in the IDEAS Global Assembly in Barbados in May
2013, where the Director presented a key note address on ‘Evaluation and its role in advancing
a measurement discourse’. Note: an article based on this address, ‘Evaluators as agents in
development change’ was published in the September 2013 European Evaluation Society
Newsletter (page five). The newly elected President of IDEAS, Rob van den Berg is keen to
participate on this, with a back-to-back event in the region, that shall make it the largest NEC
and also largest evaluation event in the UN, as it will bring together the global network of
development evaluation thinks on an event that will have training and other relevant side
events. This will be strategic from a funding and profile point of view.
6
Meeting with Professor Levin – Public Service Commission (PSC) Head, South Africa - to
discuss collaboration around evaluation training in Africa
As a follow-up to a meeting held in December 2013 at the PSC, the Director met with Professor
Levin to discuss a course proposal for an Africa initiative, linked to AAPSCOMS for a UN and
other training programme. Discussions were also held with Dr. Rist, who is keen to partner with
IPDET for this event. A proposal has been developed and has been shared with IPDET and the
Swiss, for possible seed funding for this collaboration.
Meeting with Dr. Rist on mini-IPDET for UN
A meeting was held to discuss the possibilities for developing a UNDP course for the regional
M&E officers. The IEO is also looking to develop its own training course in 2015, as it would
provide an opportunity to enhance the UNDP. There is a willingness to support this initiative,
and a proposal has been submitted for funding.
Meeting with Swiss on funding proposal
A meeting was held with Lucien Aegerter, the Programme Manager, Management for
Development results, of the Swiss Agency for Development and Cooperation (SDC). The IEO
submitted a proposal for funding a training course, and this was accepted in principle. The
purpose of the meeting was to discuss details and chart a way forward, on both direct support
to IEO, the UNDP and the regions.
Meeting with Dr. Rist as member of the Advisory Panel of IEO, UNDP
The opportunity was used to discuss IEO developments, and in particular the 2014-2017 work
plan and how members will support it. Dr. Rist will also contribute to the office on issues
relating to strategic positioning of IEO and UNDP.
Meeting with Ms. Jankovic on UNEG
A series of meetings were held between the Director of the IEO, in his role as the Vice-Chair of
UNEG, Evaluation function, and Ms. Judita Jankovic (International Civil Aviation Organization –
ICAO, Canada) who is the convener of the area. The UNEG work plan for this area calls for
deliverables on the area of professionalization, norms and standards and Peer Reviews. The aim
is to move towards certification of evaluators, and an occupational class within the UN, as is the
practice globally. The participation of Ms. Jankovic was important as she gained an
understanding of what needed to be done, and why. The other area of work – Peer Review is
proceeding well, and the Director of the IEO has just led a review of the evaluation function of
UNEG and a report shall be released shortly.
7
IV. ANNEX
Networking with participants after Director of IEO’s
lecture
Participants at the ‘Managing an Evaluation Unit’
panel.
Richard Levin, Indran Naidoo, Julia Tompson, and
Judita Jankovic
Linda Morras , Ray Rist and Richard Levin
8
Presentation by Indran Naidoo (Director, IEO)
Presentation by Michael Patton (Faculty, IPDET) Director of the IEO featured as ‘Speakers’ Lunch
Independent Evaluation Office
EVA
LUA
TIO
N
THE INDEPENDENT EVALUATION OFFICE OF UNDP
INDEPENDENCE, CREDIBILITY AND USE
IPDET, 30 JUNE 2014
Indran A. NaidooDirector
Independent Evaluation Office
Outline
Make evaluation count!
Dynamic triad of independence, credibility & utility
Experience from the IEO of UNDP
2
Independence
CredibilityUtility
Independent Evaluation Office
Evaluand
3
Country Presence
UNDP in 177 countries
Budget
Annual Budget is more than 5 Billion dollars
FOCUS
AREAS
Poverty alleviation; democratic governance; energy & environment; and crisis prevention & recovery
Independent Evaluation Office
Influencing the evaluand at all levels
IEO evaluations have influenced the new Strategic Plan of UNDP – 2014-2016 – thus all of UNDP work
90% uptake of IEO recommendations Management reflects in its documents how it has
addressed recommendations & updates public website
All new programmes must be preceded by and include response to an evaluation
4
Independent Evaluation Office
Recent Evaluations from IEO
5
Independent Evaluation Office
Recent Evaluations from IEO
6
Visit IEO website @ www.undp.org/evaluation
Independent Evaluation Office
The critical “how question”
Professional evaluation, what does it mean in practice
Principles, approaches and ensuring that the end is in sight
7
Independent Evaluation Office
Importance of the ‘Evaluation Triad’
Utility
Credibility
Independence
8
Independent Evaluation Office
Independent Evaluation Office in UNDP
PROCEDURESPOLICY
IEO work is guided by the Board approved
Evaluation policy
Evaluation Policy periodically reviewed
Management engagement
from inception to conclusion
Structurally Independent
Reports directly to UNDP Executive Boardwho appoints/renews/
dismisses Director
STRUCTURE
9
Independent Evaluation Office
Independence
10
Quality Assurance Policy review
Evaluation Advisory Panel
Peer Reviews
Reporting Line
Director reports to Executive Board to
prevent censorship and enhance transparency
INTERNALLY @ IEO Staff protected from
occupational detriment Director takes full
accountability
Independent Evaluation Office
Credibility
11
Credibility
Methodology
Guidance
Institutional Linkage and
Capacity
Professional Forums & Networks
Independent Evaluation Office
Utility
12
Uptake of recommendations• Management responses to evaluations and key actions publicly available in Evaluation Resource
Centre in IEO website• UNDP strategies and policies are informed by evaluations
(UNDP strategic plan 2014-2017)
Stakeholder engagement• Management engagement throughout the evaluation process• Stakeholder workshop organized for all evaluations to share its findings and recommendations.
Government, civil societies and all other stakeholders participate at these workshops
Organizational accountability & learning• Implementation of evaluation recommendation and associated key actions reported to UNDP
Executive Board in Annual Report on Evaluation• Assessment of Development Results(ADR) provides substantive in put for development of new
UNDP Country Programmes• All new country offices are required to submit fully costed evaluation plan with their new country
programme documents (CPDs)
Independent Evaluation Office
UN Evaluation Group (UNEG)
13
The United Nations Evaluation Group (UNEG) is a unique interagency professional network in the UN system that aims to advance the effectiveness, efficiency, impact and sustainability of the UN system’s work by promoting and strengthening evaluation. UNEG brings together units responsible for evaluation in the UN system, including UN departments, specialized agencies, funds and programmes, and affiliated organizations. It currently has 45 such members and three observers.
UNEG’s mission is to: promote the independence, credibility and usefulness of the evaluation function and evaluation across the UN system; advocate for the importance of evaluation for learning, decision-making and accountability; support the evaluation community in the UN system and beyond.
Independent Evaluation Office
Professionalization
14
Evaluation is emerging as a mature profession.
For evaluation to be taken seriously it needs to distinguish itself as a profession which requires policies, strategies and practices that bring changes at the agency and the staff level.
UNEG advances the professionalization of evaluation within the UN system and promotes adherence to the norms and standards through the external review processes of evaluation functions, the development of relevant guidance materials, as well as the development of a professional competency framework for UN evaluators.
Independent Evaluation Office
UNEG Strategic Objectives 2014-2019
15
(UNEG Strategy 2014-2019 available at: www.unevaluation.org/uneg_strategy)
Strategic Objectives
1
Evaluation functions and products of UN
entities meet the UNEG Norms and Standards
for evaluation;
Strategic Objectives
2
UN entities and partners use evaluation
in support of accountability and
programme learning;
Strategic Objectives
3
Evaluation informs UN system-wide initiatives
and emerging demands;
Strategic Objectives
4
UNEG benefits from and contributes to an
enhanced global evaluation profession.
Independent Evaluation Office
Engaging with governments
The National Evaluation Capacity development initiatives …
16
Independent Evaluation Office
National Evaluation Capacity
17
Independent Evaluation Office
National Evaluation Capacity
18
Independence
Credibility
Utility
- Rebeca Grynspan, Associate Administrator UNDP (2010-2014)
‘National evaluation capacities are increasingly critical to countries’ overall
ability to capture and demonstrate results to a variety of constituencies and to
promoting learning and enabling decision makers to make informed decisions and
plan strategically.’
Independent Evaluation Office
2015 – Year of Evaluation
Declared by EvalPartners and currently working on a UN resolution to have 2015 declared Year of Evaluation
Motto: Empowering nations and people through evaluation
19
Independent Evaluation Office
EvalYear
EvalYear, a catalyst to promote
demand and use of evaluation in policy-making
Inauguration of EvalYear at country level
Evaluation policy process at
country level
Promote national evaluation capacity building
Strengthening work with
parliamentarians and policy
makers (enabling
environment)
Strengthening stakeholders and
facilitate collaboration/ partnerships among them
20
Independent Evaluation Office
Concluding thoughts..
It is necessary to drive evaluations at many levels & ensure that all of the complexity and context is addressed.
Evaluation principles should not be lost in this pursuit Working across sectors and with partners helps build
support Evaluation should ultimately help to improve the lives of
people through ensuring development effectiveness. Professional and well capacitated evaluation functions
remain critical for this, and training such as IPDET is invaluable.
21
Thank you
United Nations Development Programme
Independent Evaluation Office220 East 42nd StreetNew York, NY 10017www.undp.org/evaluation