Impact evaluation
-
Upload
carlo-magno -
Category
Education
-
view
435 -
download
2
description
Transcript of Impact evaluation
![Page 1: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/1.jpg)
IMPACT EVALUATION
![Page 2: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/2.jpg)
Purpose of Impact Evaluation
• Impact evaluation is predicated on whether programs are making a difference.
![Page 3: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/3.jpg)
Nature of Impact Evaluation
• IE has a strong summative emphasis in that it provides findings from which a judgment of the worth of the program can be made.
• IE are retrospective in that it logically occur at an end-point.
![Page 4: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/4.jpg)
Nature of Impact Evaluation
• IE are undertaken on programs which are in a mature or settled stage and have had sufficient time to have an effect.
• IE can also be administered to immature programs but the evaluator must give caution.
![Page 5: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/5.jpg)
IE is concerned with………………
• Determining the range and extent of outcomes of a program.
• Determining whether the program has been implemented as planned and how implementation has affected outcomes.
![Page 6: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/6.jpg)
• Providing evidence to funders about the extent to which resources allocated to a program have been spent wisely.
IE is concerned with………………
• Informing decisions about replication or extension of the program.
![Page 7: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/7.jpg)
Major concern of IE…………
• Outcomes are major concerns of IE not output.
• What are outcomes? - are benefits for participants during or after their involvement with a program. - outcomes relate to knowledge, skills, attitudes, values, behavior, condition or status.
![Page 8: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/8.jpg)
Major concern of IE…………
• What are outputs? - outputs are products of the program’s activities. Example: number of meals provided, classes taught etc.
• IE is concerned with establishing what works and why.
![Page 9: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/9.jpg)
IE is beneficial to the following…
• Stakeholders of the program• Scholars• Policy-makers
![Page 10: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/10.jpg)
Approaches to IE…….
• In order to determine if a program has impact, the following must be done:
- selection of key variables- setting standards- evidence
* It is the most practised form of evaluation
![Page 11: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/11.jpg)
Approaches to IE……….
• Objectives-based;• Needs-based;• Process-product studies; and• Performance audit
![Page 12: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/12.jpg)
OBJECTIVES - BASED
• Based on the judgment of whether the stated goals or objectives of the program have been met.
• These are often expressed in terms of gains in attainment of program participants
![Page 13: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/13.jpg)
OBJECTIVES - BASED
Main Task in setting up Objectives-based
• Determine the issue of the stakeholders.• Determine the real objectives of the program.• Determine the most appropriate way to check
if the program has led to the attainment of the goals.
![Page 14: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/14.jpg)
OBJECTIVES – BASED
• Select an appropriate measuring instrument, or write and trial a new instrument.
• Identify the sources of evidence.• Collect and analyze the evidence.• Draw conclusions, make judgments and
recommendation and report findings.
![Page 15: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/15.jpg)
NEEDS – BASED
• Determine the impact if the program meets an identifiable need.
![Page 16: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/16.jpg)
GOAL - FREE
• The purpose is to examine all program effects, rather than limiting the investigation to outcomes which reflect program objectives.
• Goal-free evaluations are rare.• In goal-free, one examinees both the intended
and unintended outcomes.
![Page 17: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/17.jpg)
PROCESS – OUTCOME
• The focus is on the actual program implementation in order to explain the process of the outcome.
• The outcome = DV; the process = IV.
![Page 18: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/18.jpg)
PROCESS – OUTCOME
• Standard procedure is to develop instruments to measure outcomes.
• Determine the degree to which implementation action is consistent with the intentions of the program.
• Preferred – observations by trained personnel.
![Page 19: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/19.jpg)
Implementation as DV
• This is done when it is more important to look at and evaluate the implementation rather than the outcome.
![Page 20: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/20.jpg)
PERFORMANCE AUDIT
• A custom crafted analysis of program efficiency and effectiveness. It involves the determination of the economy, efficiency and effectiveness of government organizations, activities and functions, in addition to their compliance with laws and regulations.
![Page 21: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/21.jpg)
PERFORMANCE AUDIT
• The actual practice of PA include:- a systematic process;- objectively obtaining and evaluating evidence;- ascertaining the degree of correspondence between assertion and established criteria.- communicating the results to interested users.
![Page 22: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/22.jpg)
Summary of IE…….
• Orientation- What is the worth of the program?- Justification of decisions to mount the program?- Accountability to functions and other
stakeholders.
![Page 23: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/23.jpg)
Summary of IE……….
• Typical Issues- has the program been implemented as
planned?- have the stated goals of the program
beenachieved?
- have the needs of those served by the program been met?
![Page 24: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/24.jpg)
Summary of IE………
• Typical Issues- what are the unintended outcomes?- does the implementation strategy lead to
the intended outcomes?- how do differences in the implementation
affect the program outcomes?- what are the benefits of the program given
the costs?
![Page 25: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/25.jpg)
Summary of IE……….
• State of program- settled
• Major focus- delivery and outcomes
• Timing - after the program has been completed
![Page 26: Impact evaluation](https://reader033.fdocuments.net/reader033/viewer/2022061205/5481023bb4af9f27748b4995/html5/thumbnails/26.jpg)
Summary of IE………
• Key approaches- objectives based- needs based- goal free- process outcome- performance audit