EVAL 6000: Foundations of Evaluation

Dr. Chris L. S. CorynNick Saxton

Fall 2014

Agenda

• Course overview• Introductions• Questions and discussion• Activity 1 & 2

Course description

• This course is designed to provide an overview of the theory, method, and practice of evaluation– Comparative study of theory, research,

and practice perspectives– Analysis of core concepts and

definitions, rationale and uses, the field’s history and standards, alternative models and approaches, and emerging and enduring issues

Course website

• The website for this course is located at

http://www.wmich.edu/evalphd/courses/ eval-6000-foundations-of-evaluation/

• From this site you can access– The course syllabus– Required and supplementary readings– Weekly lecture notes– Other materials related to the course

Required textbooks

Learning objectives

1. A deep understanding of a wide array of evaluation theory and practice perspectives

2. An in-depth understanding of the origins and history of evaluation as well as its evolution toward an independent discipline

3. A clear understanding of key evaluation concepts/vocabulary/terminology

4. A clear understanding of the nature and purpose of evaluation, and the distinctions between evaluation, basic and applied research, and related terms such as assessment and diagnosis

Learning objectives

5. An ability to describe, distinguish among, and critically evaluate the usefulness and validity of selected models and approaches to evaluation, and to identify the conditions under which each should be used

6. A firm grasp of the fundamental logic and methodology of evaluation

7. A basic understanding of how to integrate traditional methodologies with evaluation-specific methodologies

8. An evaluative and critical thinking mindset, in general

By the end of this course you should have a clear understanding of evaluation’s…

history, standards, theories, methods, & practices

Secondary learning objectives

1. Conveying constructive criticism in a professional, balanced, and tactful manner

2. Facilitating discussion to engage others in dialogue about evaluation theory, method, and practice

3. Writing clearly and concisely for both academic and non-academic audiences

4. Giving high quality, professional oral presentations for both academic and non-academic audiences

Course components

• Attendance & class participation 10%• Critical readings papers

30%• Application paper

20%• Thought paper & presentation 20%• Final examination

20%

A note on course assignments

• All course assignments are due by 5:00 PM on the date indicated in the course schedule

• All assignments should be e-mailed to the instructor and teaching assistant with “EVAL 6000” in the subject line

• NO late assignments will be accepted, will not be graded, and will be assigned a grade of “0” (“F”)

Schedule of topics

• Fundamentals of evaluation (two parts)

• Evaluation approaches & models (three parts)

• Evaluation tasks, procedures, & tools (three parts)

• Metaevaluation & institutionalizing evaluation

Course structure

• If necessary, a question-and-answer session for prior week’s material

• Lecture– At the end of each lecture is a list of

entries from the Encyclopedia of Evaluation that you are expected to study (many will appear in the final examination)

• Discussion• Activity

Who are you?

• Why are you here?• What do you expect to learn?• What prior experiences do you have

with evaluation? Research? Design? Measurement? Statistics (or analysis in general)?

Who are we?

An abbreviated life history…that has significantly influenced my world view and evaluation practice

“…bad is bad and good is good and it is the job of evaluators to decide which is which”

— Scriven

Major influence on my thinking about evaluation

Other influences

Davidson

Cook

Shadish

Patton

Cronbach

Stufflebeam

Chelimsky

Hattie

Most profound influence on my evaluation practice

Who are we?

An abbreviated life history…that has significantly influenced my world view and evaluation practice

Major Influence on My Thinking About Evaluation

“A system of morality […] based on relative emotional values is […] a thoroughly vulgar conception, which has nothing sound in it and nothing true”

— Socrates

Activity 1

• Draw the first image that comes to mind when you hear the word ‘evaluation’

• In small groups share your images and identify similar and dissimilar themes

• Share your group’s thematic analysis

Activity 2

• Carefully review the ‘definitions’ of evaluation enumerated by major evaluation theorists on the following slides

• In small groups, discuss what you see as potential factors influencing these theorists’ definitions of evaluation and how their definitions might influence evaluation practice

• Share your group’s analysis

“…the act or process of determining the merit, worth, or significance of something or the product of that process”

— Scriven

“…the use of evaluation concepts and techniques to foster self-determination”

— Fetterman

“…the use of social science research procedures to systematically investigate the effectiveness of social intervention programs”

— Rossi, Lipsey, & Freeman

“…[to] describe and assess what was intended (goals and objectives), what happened that was unintended, what was actually implemented, and what outcomes and results were achieved”

— Patton

“…[to explain] how and why programs work, for whom, and under what conditions”

— Chen

Encyclopedia entries

• Assessment• Accountability• Auditing• Campbell, Donald T.• Cook, Thomas D.• Criteria• Evaluand• Evaluation• Evaluation Theory• External Evaluation• Formative Evaluation• History of Evaluation

• Independence• Logic of Evaluation• Objectivity• Scriven, Michael• Shadish, William R.• Standards• Summative Evaluation• Value-free Inquiry• Value Judgment• Values

“…just because you have a library card

doesn’t make you Yoda”

— Pitt