OECD DAC Network on

Development EvaluationDevelopment Evaluation

Eun-Pyo HONG and Hans LUNDGREN

OECD

November 4 2009

Contents

• Overview

– OECD, OECD/DAC

– DAC Network on development evaluation

– Nature and Placement of evaluation– Nature and Placement of evaluation

• DAC Network

– Priorities

– Current major work streams

Organisation for Economic Co-operation and

Development (OECD)

• The OECD was created in 1961 with mission to:

– support sustainable economic growth

– boost employment and living standards

– assist other countries’ economic development – assist other countries’ economic development

– contribute to growth in world trade

• Made up of 30 member countries sharing a

commitment to democratic government and the

market economy

• Members work in topic-specific committees on a range

of public policy arenas

Development Assistance Committee (DAC)

• A neutral forum of OECD countries (major bilateral

donors) and multilateral organisations in support of

sustainable development

• Changing behaviour in the aid system to make aid more • Changing behaviour in the aid system to make aid more

effective and produce better development results

• Good practice in development co-operation

• Statistics, peer reviews and knowledge networks

• Home to the Network on Development Evaluation

DAC Network on Development Evaluation

• Works to increase the development effectiveness of aid by

promoting high quality, independent evaluation

• 30 Network members include heads of evaluation and experts

from all DAC member countries and seven multilateral

institutions/banks.

• OECD countries not yet members of the DAC attend as observers. • OECD countries not yet members of the DAC attend as observers.

Partner countries and evaluation associations often participate

in workshops and meetings.

Evaluating development assistance

• Definition: Evaluation is a systematic and objective

assessment of an on-going or completed project,

programme or policy; and its design, implementation

and results.

• Aim: to determine the relevance and fulfilment of • Aim: to determine the relevance and fulfilment of

objectives, development efficiency, effectiveness,

impact and sustainability (DAC criteria)

• Outcome: provide information that is credible and

useful, enabling the incorporation of lessons learned

into the decision making process of both partners and

donors.

Placement of evaluation in development

context

PD (2005) & AAA(2008): Changing the ways development

partners address global challenges

• Evaluation has a key role in formulating policy decisions

and helping to hold all development partners mutually and helping to hold all development partners mutually

accountable for development results

• To meet the evaluation needs of all partners: evaluation

must reflect this new context more harmonised, better

aligned and increasingly country-led.

Network priorities

• Improving the quality of evaluation and its use for

learning and accountability

• Strengthening individual members’ evaluation systems

• Facilitating harmonisation and coordination of • Facilitating harmonisation and coordination of

evaluation

• Improving dissemination, communication and feedback

of evaluation results and guidance, within and outside

the DAC evaluation community

• Promoting partnerships with civil society and other

partner country stakeholders

Current major work streams

• Facilitate and support joint evaluations

• Develop evaluation norms and standards to guide

policy and practice

• Study of member evaluation systems and resources • Study of member evaluation systems and resources

• Support evaluation capacity development in

Partner countries

• Share knowledge, including through the online DAC

Evaluation Resource Centre “DEReC”

Joint evaluation

• A “collaborative approach to evaluation involving more

than one partner country or development agency.”

• Of growing importance in the context of commitments

to improve coordination, harmonisation and to improve coordination, harmonisation and

ownership

• Example: Joint evaluation of the international

response to the Indian Ocean involving over 40

humanitarian and development cooperation agencies.

Joint evaluation (Network supports)

The Network supports joint evaluation by:

• Harmonising evaluation norms and practices

• Sharing evaluation plans on a new online

platform to encourage co-ordinationplatform to encourage co-ordination

• Developing specific guidance on joint evaluations

• Facilitating and participating major joint

evaluations, such as the joint evaluation of the

implementation of the Paris Declaration

Joint evaluation (of the Paris Declaration)

• The evaluation is an integral part of

the Paris Declaration itself

• Involves development agencies and

partner countries

• Phase I focused on ways of improving

and enhancing implementation.(HLF3)

• Phase II will provide information

about the impacts and effects of

increased aid effectiveness (for HLF4)

12

Norms and standards (Evolution)

• DAC Principles for Evaluation of Development Assistance

– Adopted in 1991, Heart of the Evaluation Network

– For management and institutional arrangement

– reviewed in 1998 found the Principles to be highly

relevant but need relevant but need

=> DAC Evaluation criteria for evaluating Development

Assistance

• Evaluation system and use: a working tool for PER reviews

and assessments (Developed in 2006)

• Definition of evaluation quality standards (2009/2010)

DAC principles for evaluation (elements)

Purpose of evaluation: learning and

accountability

Impartiality and independence

Participation of donors and recipients

Credibility

Design and implementation

of evaluation

Collaboration and joint

evaluation

Usefulness, Dissemination and Feedback

DAC evaluation criteria

effectiveness

Guide of measure to assess development work

relevance and fulfilment of

development efficiency

sustainabilityimpact

fulfilment of objectives

Evaluation system and use

To strengthen evaluation function, to promote transparency and accountability in development agency

Impartiality, transparency and

independence

Evaluation uses, Evaluation policy

independence

Resources and staff, quality

Evaluation partnership and

capacity building

Planning, coordination,

harmonisation

Dissemination, Feedback, knowledge

management, learning

Evaluation policy

Norms and standards(Quality Standards)

• The DAC Quality Standards for Development Evaluation:

– complement the Principles by describing the key

pillars of a quality evaluation process and product.

– The Standards were developed through a consultative

process with members and other partners.process with members and other partners.

– approved in 2006 for a 3-year test phase, applied

widely by range of development partners (one

objection, 95% approved).

– have been revised based on these experiences and are

being finalised (by end 2009)

Key pillars of quality standards

Rationale, purpose, objective of evaluation

Evaluation methodologyEvaluation ethics,

Independence

Evaluation scope, Context

Information sourceRelevance of evaluation results

Quality assurance, Completeness

Norms and standards (summary)

Evaluation Principles Criteria Systems and use Quality standards

Available 1991 1998? 2006 2009/10

Purpose Management and

institutional

arrangement

Guide to assess

development

work

Working tool for PEER

reviews

Quality evaluation

process and product

Use Benchmark used in

the peer reviews

specific guidance

for more

harmonised

evaluation

Reviewers need

questions to ask as they

aren’t specialists. To

promote transparency

and accountability

the report or other

dissemination

mechanisms

Key 1. Central 1. Relevance 1. Policy: role, duty & 1. Rationales, Key

elements

1. Central

message

2. Purpose

3. Impartiality,

independence

4. Credibility

5. Usefulness

6. Participation of

donors/recipients

7. Donor co-

operation

8. Programming

9. Design,

implementation

10. Reporting,

dissemination,

feedback

1. Relevance

2. Effectiveness

3. Efficiency

4. Impact

5. Sustainability

1. Policy: role, duty &

objectives of

evaluation units

2. Impartiality,

transparency &

independence

3. Resource, staff

4. Partnerships,

capacity building

5. Quality

6. Planning,

coordination,

harmonisation

7. Dissemination,

feedback, knowledge

management

8. Evaluation use

1. Rationales,

purpose, objective

2. Scope

3. Context

4. Methodology

5. Information

sources

6. Independence

7. Ethics

8. Quality

9. Relevance of

results

10. Completeness

Key norms and standards (Publication)

Publication includes:

• DAC Principles for Evaluation of

Development Assistance

• The Review of the DAC Principles• The Review of the DAC Principles

• Elaboration of the DAC evaluation

criteria

• Evaluation systems and use

“working tool” for peer reviews

• (draft)Quality Standards

Study of member evaluation systems

• Study to examine by Secretariat “state of development

evaluation of Network members,” covering key areas

of evaluation systems and resources:

• Financial and human resources

• Evaluation policy

• Quality

• Coordination • Evaluation policy

• Independence

• Aims to build a current and comprehensive picture of

evaluation policy and practice among members

• Goal: To inform ongoing efforts to strengthen the

contribution of evaluation to improving development

results and mutual accountability.21

• Coordination

• Use and impact

Evaluation capacity development in partner

countries

• Evaluation capacity development: unleashing,

strengthening and maintaining evaluation capacity.

• Covers different “levels” of capacity: from setting the

agenda to technical skills, demand for evaluation,

design, feedback and learning, etc. within the design, feedback and learning, etc. within the

evaluation system

22

Individual

Institutional

Accountability Environment

Moving the capacity agenda forward

• Need to learn more about

the effects of donor

evaluation policies and

practice on partner

capacity.capacity.

• Work on meeting both

partner and donor

evaluation needs (learning

and accountability) in joint

processes

23

Sharing evaluation knowledge

• DAC Evaluation Resource Centre: an open and free public database

containing more than 2,100 evaluation reports and guidelines

• Continuously updated and regularly undergoing further development

• Offers rapid and easy access to a large choice of publications

• For use by researchers, evaluators, development agencies, partners, • For use by researchers, evaluators, development agencies, partners,

NGOs, and the international development evaluation community

24 www.oecd.org/DAC/EvaluationNetwork/DEReC

DEReC “Front Page”

• New feature launched in 2009 to highlight evaluation findings

relevant to current policy topics and world events

• Targeted to a wider audience to promote use of evaluation

outputs

First “edition” of the DEReC Front Page (February 2009)

Olivier Laban/ Getty Images

Donors respond to Gaza Destruction

The recent war destroyed some 14,000 homes,

219 factories and 240 schools in the Gaza Strip.

The Palestinian Authority is seeking $2.8 billion

in aid to help rebuild.

What can evaluation tell us about donors’ past

efforts in Palestine and strategies for emergency

relief and reconstruction in conflict affected

regions?

Other Network publications

• Guidance for Managing Joint

Evaluations

• Glossary (in 14 languages) of Key

Terms in Evaluation and Results

Based ManagementBased Management

• Draft Guidance for Evaluating

Conflict Prevention and

Peacebuilding Activities

• Factsheets

Websites

www.oecd.org/dac/evaluationnetwork

www.oecd.org/dac/evaluationnetwork/derec