Theme: Evaluation Use, Communication and Reach.

Use of Evaluation results to enhance organizational effectiveness: Do evaluation findings improve

organisational effectiveness?

Innocent Karugota MuhumuzaPlanning, Monitoring and Evaluation Officer.

Caritas Switzerland-Uganda

B.A, Dip Disaster Mgt, PGD M&E, Masters M&E (In-progress)

September 28, 2016

Maastricht, Netherlands

Content

Defining Monitoring and Evaluation (M&E)

Why Conduct Evaluations andWhat is Use

Typical Use of Evaluation (Case Study)

Factors influencing Use

Conclusions

Defining Monitoring and Evaluation

EvaluationMonitoring A continuing function that uses systematic

collection of data on specified indicatorsto provide management and the main

stakeholders of an ongoing developmentintervention with indications of the extentof progress and achievement of objectivesand progress in the use of allocated funds-

OECD, 2002

A time-bound exercise that attempts to assess

systematically and objectivelythe relevance, performance and

success of ongoing and completed programs and

projects- UNDP, 2002

Why Conduct Evaluations?

According to Development Assistance Committee (DAC),

Evaluations are conducted to assess:

• Effectiveness How well objectives are met

• Efficiency How well resources have been used

• Relevance How appropriate the intervention is/was to meet the needs of the beneficiaries

• Impact The intended and uninteded benefits of the intervention

• Sustainability The ability/potential of interventions’ impact tocontinue long after the end of the funding or involvementof the donor

What is Evaluation Use/ Utilisation?

Numerous definitions of Evaluation Use exist

Evaluation Use and Evaluation Utilisation used interchangeably to mean:

• The application of evaluation processes, products or findings to produce an effect (Johnson, Greenseid et al, 2009).

• How real people in the real world apply evaluation findings and experience and learn from the evaluation process (Patton, 2013).

Types of use

Many types of use exist, including;

• Instrumental- when decision makers use the evaluation findings to modify the object of the evaluation (Shulha & Cousins, 1997)

• Conceptual- An evaluation helps program staffs understand the program in a new way-(Weiss, 1979)

• Symbolic- When the organisation or individuals use the mere existence of evaluations and not any aspects of the results to persuade or to convince

Process use incorporates features from instrumental use, enlightment

and conceptual useTypes of use, Cont’d

• Legitimisation-the evaluation is used to justify current views, interests, policies or actions

• Enlightment- evaluation findings add knowledge to the field and so may be used by any one (Weiss, 1979)

• Process-....cognitive, behavioural, program and organisational changes resulting from engagement in the evaluation process and thinking evaluatively (Patton, 2003). Influences thinking and behaviour and program/organisations’ procedures and culture

Evaluation use in practice Acase ofAdvance Afrika (AA)-Uganda

The project; Youth Entrepreneurship Enhancement Project (YEEP)

• Was a pilot for one year (March 2004-Feb 2015)

• Focus: Reformation, Rehabilitation and Re-integration

• Covered prisons in only 3 district of Northern Uganda

• Implemented by AA staff, Staff of Uganda Prisons Service, Students of GuluUniversity-Uganda

The Evaluation:

• Involved; AA staff, Staff of Uganda Prisons Service (UPS), Students of Gulu University and Ex-convicts (beneficiaries)

• Facilitated by an external Evaluator

• 8 recommendations were made:• Conduct refresher training for all UPS trainers

• Strengthen Advocacy• improve training manual

• Develop a standard reporting guide for the project

• UPS should be involved more in following up Ex-Convicts• Re-define the project target beneficiaries (geographical boundaries)

• involve more (number of) UPS staff for wider support to the project

• Build capacity of AA staff and co-implementers

Evaluation use in practice, Cont’d

From the recommendations:

• Two full-fledged projects were developed

1. Advancing Youth Entrepreneurship (AYE)-(2015-16)

2. Social Re-integration and Economic Empowerment (SREE)-(2016-18)

• Project covers 26 prisons in Acholi and Lango sub-regions of Nothern Uganda

Factors influencing Use

Quality External Factors

Organisational Factors

UtilisationStaff & Organisation

• Culture• Structure• Systems• Policies• Budgets

• Purpose• Design • Planning and timing• Dissemination and• Credibility of evidence

• Professional Bodies/Associations

• Project Beneficiaries• Donors• Protection of funding and

reputation

EnvironmentTechnology

Legal

Conclusions

Quality factors.

It is key that, the design of the evaluation as much as possible involves keystakeholders not as mere repondents but as active participants in the process. This helps to enhance credibility of the evidence and ownership of the results.

Organisational Factors.

Even when it is good to have structures and policies in the organisation, care must be paid on how this affects learning and relations.

External Factors

Pressure from the external influencers e.g donors must be kept in check andorganisations be granted opportunity to learn and improve from failure and also consolidate the gains made. This means that the external actors ought to supportorganisations to improve rather than instilling fear of the ‘‘unknown’’

AsanteSana!