EFFECTIVE PROFESSIONAL DEVELOPMENT RESULTING

IN LONG-TERM STUDENT ACHIEVEMENT

Theories of Change and Logic Modeling:

Matching Means and Ends

NCLB/ITQ SymposiumOctober 10, 2008

IBHE Model for Evaluation

Theory of Action

Universal Theory of Action: Collaborative partnerships are the mechanism for action

Project Theories of Action: how each intervention is delivered train-the-trainers model on-line learning model direct instruction model cooperative learning model workshop series model mentor/mentee model

The Theory of Action describes HOW a project is delivered, usually based on the professional development literature

Limitations of Theories of Action

They describe HOW a project is delivered, not WHY it will work or what EFFECTS it will have.

They are therefore evaluated by outputs, not by outcomes/effects.

Output evaluation is necessary, but not

sufficient, to show value for the project.

Theory of Change: Three Elements

Element One: The Change Framework

A backward mapping process that begins with Outcomes to plan, implement, and continuously develop and improve programs

Element Two: Program Interventions

Designing interventions or activities intended to create desired results, and linking them systematically

Element Three: Program Assumptions

Carefully articulating assumptions about how change will occur and testing the assumptions systematically through evaluation

Theory of Change: Background

Evaluation is challenging because program assumptions are not well-articulated.

The more complex the problem, the more difficult it is to be clear about how things will change as a result of the program.

By designing a Pathway of Change, program developers/ implementers/ evaluators, can strengthen the case that an intervention caused a result.

Theory of Change: Process

Backward mapping from long, short, and mid-range goals or outcomes laid out in sequence

Link outcomes to program elementsPlan evaluation from the outset to track

achievement of the intended outcomesUse evaluation results for improvement and

accountability

Broad Benefits of Theories of Change

“Stakeholders value theories of change as part of program planning and evaluation because they create a commonly understood vision of the long-term goals, how they will be reached, and what will be used to measure progress along the way.”

ActKnowledge/The Aspen Roundtable www.theoryofchange.org

Theory of Change:Project Benefits

Makes explicit the basis for the project intervention

Consolidates the research that undergirds the project

Provides an overarching framework for project planning, implementation and evaluation

Theory of Change and Logic Modeling

A Logic Model is a graphic form of a Theory of Change

Takes the theory, which is selected on the basis of empirical evidence that it will work, and translates it into an operational plan.

Created by backward mapping.

The “IfThen” Connections

IF certain inputs are available…THEN certain activities/interventions can take

place;IF these interventions occur…THEN certain outputs will be seen; andIF the interventions are effective…THEN certain outcomes will be attained.

Finally, what we learn from the outputs and outcomes can be used to improve the intervention.

Building the Logic Model

Start with the Theory of Change:

“Companies do better when their employees are healthy.”

Cite empirical evidence for this theory.

Building the Logic Model

Do the backward mapping

Ask and answer the “How?” questions

Centers for Disease Control Healthier Worksite Initiative

http://www.cdc.gov/nccdphp/dnpa/hwi/program_design/logic_model.htm

Building The Logic Model

Lay out the steps in order from inputs to outcomes

Answer the “Why?” questions.

Centers for Disease Control Healthier Worksite Initiative

http://www.cdc.gov/nccdphp/dnpa/hwi/program_design/logic_model.htm

Building The Logic Model

Create the model with appropriate connections

•Funding• Staff

• Scientific Evidence

INPUTS

Building The Logic Model

Create the model with appropriate connections

•Funding• Staff

• Scientific Evidence

Workplace Walkability Program

INPUTS INTERVENTION

Building The Logic Model

Create the model with appropriate connections

•Funding• Staff

• Scientific Evidence

Workplace Walkability Program

Employees walk to

meetings more often

INPUTS INTERVENTION OUTPUT

Building The Logic Model

Create the model with appropriate connections

•Funding• Staff

• Scientific Evidence

Workplace Walkability Program

Employees walk to

meetings more often

Employee level of physical

activity and fitness

increases

Productivity Increases

INPUTS INTERVENTION OUTPUT OUTCOMES

Building The Logic Model

Create the model with appropriate connections

Identify your evaluation measures for outputs and outcomes

•Funding• Staff

• Scientific Evidence

Workplace Walkability Program

Employees walk to

meetings more often

Employee level of physical

activity and fitness

increases

Productivity Increases

INPUTS INTERVENTION OUTPUT OUTCOMES

Employee Logs

Employee Fitness Screenings

Company Data

Project Benefits of Logic Models

They make a stronger case for project investment – funders understand more clearly what you intend to do

They reflect group process and understanding among project staff and project participants

They indicate where and how to evaluate program effects

We Now Find These Everywhere

U.S. Department of Health & Human Services

Idaho State Board of Education

We Now Find These Everywhere

National Institute of Standards and Technology

processoutcome.org

Research Utilization Support & Help (RUSH) Canadian Department of Justice

NOT Recommended!O

hio

Dep

artm

ent

of

Ed

uca

tio

n (

OD

E)