Program Evaluation Overview

Definitions of Program Evaluation

systematic collection of information abut the activities, characteristics, and outcome of programs

for use by specific people to reduce uncertainties, improve effectiveness, and make decisions with regard to what those programs are doing and affecting

(Patton, 1986). 

Definitions of Program Evaluation

establish and provide useful information for judging decision alternatives;

assist an audience to judge and improve the worth of some educational program or object;

assist the improvement of policies and programs.

Stufflebeam

Purpose of Program Evaluation

Improve

Maintain

Stop

Objectives of Program Evaluation

Discover whether and how effectively the objectives are being met

Discover unplanned and unexpected consequences

Determine underlying policies and related activities that contribute to success or failure in particular areas

Provide continuous feedback

Context, Input, Process, and ProductEvaluation

The CIPP Model

Aspect of evaluation

Type of decision Kind of question answered

Context Planning decisions

What should we do?

Input Structuring decisions

How should we do it?

Process Implementing decisions

Are we doing it as planned? And if not, why not?

Product Recycling decisions

Did it work?

CIPP Model

Simple systems model applied to program evaluation.

context evaluation, input evaluation, process evaluation, and product evaluation.

These types are typically viewed as separate forms of evaluation, but

they can also be viewed as steps or stages in a comprehensive evaluation

Formative < --------- > Summative

With the CIPP model, there is a clear distinction between

formative evaluation (conducted for the purpose of improving the program) and

summative evaluation (conducted for the purpose of accountability which requires determining the overall effectiveness or merit of the program).

Context Evaluation

includes examining and describing the context of the

program being evaluated, conducting a needs and goals assessment, determining the objectives of the program, and determining whether the proposed objectives will be

sufficiently responsive to the identified needs. It helps in making program planning decisions.

Input Evaluation

includes activities such as a description of the program inputs andresources,

a comparison of how the program might perform compared to otherprograms,

a prospective benefit/cost assessment (i.e., decide whether you thinkthe benefits will outweigh the costs, before it is actually implemented),

an evaluation of the proposed design of the program, and an examination of what alternative strategies and procedures should

be considered and recommended. In short, this type of evaluation examines what the program plans on

doing. It helps in making program structuring decisions.

Process Evaluation

includes examining how a program is being implemented, monitoring how the program is performing, auditing the program to make sure it is following required legal and

ethical guidelines, and identifying defects in the procedural design or in the implementation of

the program. It is here that evaluators provide information about what is actually

occurring in the program. Evaluators typically provide this kind of feedback to program personnel

because it can be helpful in making formative evaluation decisions (i.e., decisions about how to modify or improve the program).

In general, process evaluation helps in making implementing decisions.

Product Evaluation

includes determining and examining the general and specific outcomes of the program (using impact or outcome assessment techniques),

measuring anticipated outcomes, attempting to identify unanticipated outcomes, assessing the merit of the program, conducting a retrospective benefit/cost assessment (to

establish actual worth or value), and/or conducting a cost effectiveness assessment (to determine if the program is cost effective compared to other similar programs).

Product evaluation is very helpful in making summative evaluation decisions (e.g., What is the merit and worth of theprogram? Should it be continued?)