Uganda Evaluation

Week 2015

The World of Evaluation:

Towards a global evaluation community

11.03.2015 1

PD Dr. Wolfgang MeyerSaarland University

P.O. Box 151150

D – 66041 Saarbrücken

www.ceval.dew.meyer@mx.uni-saarland.de

Uganda Evaluation

Week 2015

� How to measure the degree of professionalization on a global scale

� The State of the Art – the World of Evaluation TodayEvaluation Today

� Africa – the Future of Evaluation?

11.03.2015 2PD Dr. Wolfgang Meyer

Uganda Evaluation

Week 2015

Measuring Professionalism

11.03.2015 3PD Dr. Wolfgang Meyer

Measuring Professionalism

Uganda Evaluation

Week 2015

System(Social Institutions)

Standardized

Training

Social

Perception

Interest

Organization

Supplier’s

Monopoly

Social

Endorsement

To measure professionalization

Theory

Expertise Awareness Regulation Market Rewards

Actor(Exchange and Interaction)

Specific

Skills

Professional

Identity

Shared

Values

Need for

Professionalism

Autonomy

PD Dr. Wolfgang Meyer11.03.2015 4

Uganda Evaluation

Week 2015

� University-based Study Courses

� Scientific Journals

To measure professionalization

Indicators

Expertise

Awareness

Standardized

Training

Social

Perception

� National Evaluation Societies (VOPEs)

� Evaluation Standards

� National Evaluation Policy

11.03.2015 5

Market

RegulationInterest

Organization

Shared

Values

Need for

Professionalism

PD Dr. Wolfgang Meyer

Uganda Evaluation

Week 2015

University-based Study Courses

� USA: Several empirical studies since 1980s (LaVelle & Donaldson 2010; actualised by Meyer 2015)

� Canada: Empirical Study in 2009 (Devine 2009)

To measure professionalization

Database Expertise

� Canada: Empirical Study in 2009 (Devine 2009)

� Europe: Two empirical studies 2005 and 2012(Beywl & Harich 2007; Friedrichs 2015)

� Africa, Asia and South America: No systematic studies available yet

11.03.2015 6PD Dr. Wolfgang Meyer

About 20 Programs in 17 countries collected

Uganda Evaluation

Week 2015

Scientific Journals

� English Language Journals (USA/CAN/AUS/UK): Listed in several databases, especially in VOPEs

� Chinese, French, German, Spanish Language Journals: Systematic research done by Wolfgang Meyer and Christoph

To measure professionalization

Database Awareness

Systematic research done by Wolfgang Meyer and ChristophMueller at CEval (ongoing)

� Other Languages (e.g. Arabian, Italian, Russian etc.): still in process.

11.03.2015 7PD Dr. Wolfgang Meyer

About 40 Journals in 15 countries collected

Uganda Evaluation

Week 2015

VOPEs (Voluntary Organizations for Professional Evaluation)

� All VOPEs can register themselves for free at the global

association Internationl Organization for

To measure professionalization

Database Regulation

association Internationl Organization for

Cooperation in Evaluation (IOCE). They are asked

to answer a survey including questions about the

number and kind of members, the year of foundation,

the activites etc.

11.03.2015 8PD Dr. Wolfgang Meyer

Almost 100 National Organizations collected

Uganda Evaluation

Week 2015

Evaluation Standards

� One of the IOCE Questions is about the existence of

Standards (answered by the VOPEs)

To measure professionalization

Database Regulation

11.03.2015 9PD Dr. Wolfgang Meyer

36 countries endorsed standards

Uganda Evaluation

Week 2015

National Evaluation Policies

� On behalf of the Parliamentarians Forum on Development Evaluation in South Asia and in cooperation with EvalPartners and IOCE, Rosenstein (2013) mapped 115 countries for National Evaluation Policies in 2013

To measure professionalization

Database Market

Policies in 2013

11.03.2015 10PD Dr. Wolfgang Meyer

20 countries have written and legalized policies

Uganda Evaluation

Week 2015

Results: The State of the Art

11.03.2015 11PD Dr. Wolfgang Meyer

Results: The State of the Art

Uganda Evaluation

Week 2015

67

4448

5050

60

70

80

University-based Study-Programs in USA 1980 – 2014

Expertise

Results: University Programs

11.03.2015 12PD Dr. Wolfgang Meyer

44

38

27

0

10

20

30

40

1980 1986 1994 2006 2008 2014

Quelle: LaVelle / Donaldson 2010: 15; aktualisiert von Meyer 2015

Uganda Evaluation

Week 2015

University-based Study-Programs in Europe 2005 and 2012

Expertise

Results: University Programs

11.03.2015 13PD Dr. Wolfgang Meyer

Uganda Evaluation

Week 2015

Study Courses on Evaluation at Universities

• Building up single courses and training programs

• Developing Master of Evaluation Programs

Expertise: The State of the Art

Master

Expertise

• Developing PhD Programs

11.03.2015 14PD Dr. Wolfgang Meyer

But only a few Universities established full Master of Evaluation programs

There are no standards, the courses arevery different and part of diverse disciplines

High fluctuate single courses are thedominant form

Uganda Evaluation

Week 2015Results: Evaluation Scientific Term

Evaluation as term in Scientific Journals at SAGE 1900-2013 (annually)

14000

16000

18000

20000

Awareness

11.03.2015 15PD Dr. Wolfgang Meyer

0

2000

4000

6000

8000

10000

12000

14000

19

00

19

05

19

10

19

15

19

20

19

25

19

30

19

35

19

40

19

45

19

50

19

55

19

60

19

65

19

70

19

75

19

80

19

85

19

90

19

95

20

00

20

05

20

10

absolut

Uganda Evaluation

Week 2015Results: Evaluation Journals

30

35

40

Cumulative Development of Evaluation Journals 1973 – 2014

Awareness

11.03.2015 16PD Dr. Wolfgang Meyer

0

5

10

15

20

25

1973

1975

1977

1979

1981

1983

1985

1987

1989

1991

1993

1995

1997

1999

2001

2003

2005

2007

2009

2011

2013

Uganda Evaluation

Week 2015

Evaluation Journals

• Creating a platform for scientific exchange

• Enhance the quality of Evaluation research

• Integrating practice reports from the Societies

Awareness: The State of the Art

Journal

Awareness

• Integrating practice reports from the Societies

11.03.2015 17PD Dr. Wolfgang Meyer

The term „evaluation“ started its career during the second World War (first used for reviews, thenfor technical and medicine issues, then for education).Its use is still increasing.

There are about a dozen Journals on Evaluation in the World merely in English, but also inChinese, French, Italian, Spanish and German

Uganda Evaluation

Week 2015Results: National VOPEs

Foundation of important Evaluation Societies 1980 - 2014

8

10

12

Regulation

11.03.2015 18PD Dr. Wolfgang Meyer

0

2

4

6

8

1980

1981

1982

1983

1984

1985

1986

1987

1988

1989

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

Uganda Evaluation

Week 2015

Building an Evaluation Society

• Discussion Forum and Exchange Platform

• Organization of regular Meetings and Conferences

• Capacity Building and Training Standards

• Evaluation Standards and Codes of Conduct

Regulation: The State of the Art

Society

Regulation

• Evaluation Standards and Codes of Conduct

11.03.2015 19PD Dr. Wolfgang Meyer

There are Evaluation Societies in almosthalf of the States in the World today.

The dynamic increased in the 21st century, especially in the Global South

Additionally, many smaller states are coveredby Regional Evaluation Societies (like AfrEA,EES, ReLAC, AES and the new build Asia-Pacific Evaluation Association)

Uganda Evaluation

Week 2015Results: Public Governance

The Need for Evaluation – Improving Public Governance

Worldwide Governance Indicators (WGI) 2012Market

11.03.2015 20PD Dr. Wolfgang Meyer

Uganda Evaluation

Week 2015Results: National Evaluation Policy

Market

11.03.2015 21PD Dr. Wolfgang Meyer

Uganda Evaluation

Week 2015

Public Governance and Evaluation

• Setting rules how evaluations should be done

• Institutionalization of Monitoring and Evaluation

• Defining bureaucratic procedures for handling

Evaluation

Market: The State of the Art

Policies

Market

Evaluation

11.03.2015 22PD Dr. Wolfgang Meyer

Evaluation is established as a new instrumentfor evidence based policy in the most advancedand best governed countries

Some other countries (like Uganda) are about to follow this way. There is an increasing demandfor appropriate Monitoring and Evaluation tools

Uganda Evaluation

Week 2015

CEval Professional Evaluation Index (CPEI) 2014

� University-based Study Courses 2014 (Master courses with more than 7 courses and Evaluation in Title)

� Scientific Journals 2014(Journals with „Evaluation“ or „Assessment“ in Title)

� National Evaluation Societies 2012

Composition of CPEI

� National Evaluation Societies 2012(Registered national VOPEs at IOCE)

� Evaluation Standards 2012(Reported Standards in IOCE questionnaire)

� National Evaluation Policies 2013(as mapped by Barbara Rosenstein)

11.03.2015 23PD Dr. Wolfgang Meyer

Additative Index from „0“ (none of them) to „5“ (all of them)

Uganda Evaluation

Week 2015CPEI: The State of the Art

Professional Evaluation all over the World

CEval Professional Evaluation Index (CPEI) 2014

11.03.2015 24PD Dr. Wolfgang Meyer

Uganda Evaluation

Week 2015

� Five Groups of Evalation

• The Forerunners (12 countries)e.g. Australia, Canada, South Africa, USA(Germany and Uganda also belong to them!)

• The Catch-Ups (24 countries)e.g. Argentine, Costa Rica, Finland, Kenya, Malaysia

Results: Where we are

Society Standards

Master Journal

Malaysia

• The Strugglers (24 countries)e.g. Brazil, Guatemala, Cameroon, Italy, Pakistan

• The Stay Behinds (37 countries)e.g. Bolivia, Nicaragua, Slovenia, Niger, India

• The „No-Gos“ (Rest of the World)e.g. Guyana, Panama, Algeria, Mocambique,Saudi Arabia, North Korea

11.03.2015 25PD Dr. Wolfgang Meyer

Uganda Evaluation

Week 2015The Future of Evaluation

11.03.2015 26PD Dr. Wolfgang Meyer

Editors: Reinhard Stockmann and Wolfgang Meyer

Published in Spring 2015

Uganda Evaluation

Week 2015

Africa – the Future of Evaluation?

11.03.2015 27PD Dr. Wolfgang Meyer

Evaluation?

Uganda Evaluation

Week 2015

„Evaluation“ Regions

�South Africa

� East Africa

�West Africa

Regions in Africa

11.03.2015 28PD Dr. Wolfgang Meyer

Blanks:

� The Arabian World

� South-West Africa

Uganda Evaluation

Week 2015

Strength

= Institutions= Global Support

= Incentives

Opportunities

= Professionalism= Fast Development

= Better Careers

Africa – the Future of Evaluation?

Weakness

= Poor Governance

= Dependence= Missing National Structures

Threats

= Abuse of Evaluation

= Development Barriers

= Dominant Internationals

11.03.2015 29PD Dr. Wolfgang Meyer

Uganda Evaluation

Week 2015

� Uganda as a regional hub for training and education

� Uganda as a center for evaluation research

The Role of Uganda

� Uganda as a vanguard for evidence based policy in East Africa

11.03.2015 30PD Dr. Wolfgang Meyer

Uganda Evaluation

Week 2015

� Beywl, W.; Harich, K. (2007): University-based continuing education in evaluation: The baseline in Europe. Evaluation, 13(1):121-134.

� Devine, H. (2009): Consortium of Universities for Evaluation Education (CUEE) Project: Research on Evaluation Education at the Graduate Level in Canadian Universities, Victoria.

� Friedrichs, V. (2015): European university-based study programs in Evaluation. In: Stockmann, R. & Meyer, W. (eds.): The Future of Evaluation. Basingstoke: Palgrave (forthcoming)

� LaVelle, J. M. & Donaldson, S. L. (2010): University-Based

References

� LaVelle, J. M. & Donaldson, S. L. (2010): University-Based Evaluation Training Programs in the United States 1980-2008: An Empirical Examination. American Journal of Evaluation 31(1): 9-23.

� Meyer, W. (2015). Towards Professionalization? The Contribution of University-based Training Programs in Pioneer Countries. In: Stockmann, R. & Meyer, W. (eds.): The Future of Evaluation. Basingstoke: Palgrave (forthcoming)

� Rosenstein, B. (2013): Mapping the Status of National Evaluation Policies. Commissioned by Parliamentarians Forum on Development Evaluation in South Asia jointly with EvalPartners, IOCE/EvalPartners: Renfrew Canada.

11.03.2015 31PD Dr. Wolfgang Meyer