LOGIC MODELS AND OUTCOME MEASUREMENT United Way Community
Investment Process Training
Slide 2
Workshop Overview I.Welcome and Introductions II.Basics of
Outcome Measurement III.Developing an Effective Logic Model
IV.Evaluating your Logic Model V.Wrap-Up
Slide 3
What are your other expectations?
Slide 4
Logic Models What difference are you making? How do you know
it? What is the value of your program? Theory of Change Program
Matrix Logical Framework
Slide 5
5 Logic Model Conceptual Chain Goal Theory Target Population
Inputs Activities Outputs Short-term Outcomes Mid-term Outcomes
Long-term Outcomes
Slide 6
Slide 7
Slide 8
Slide 9
What Exactly is An Outcome? Mid-Term Change in Behavior
Short-Term Increased Knowledge Long-Term Change in Condition
Slide 10
10 What is Outcome Measurement? Outcome measurement is the
regular, systematic measuring of progress toward intended outcomes.
Internal External
Slide 11
11 Measuring Program Outcomes: Eight Steps to Success 8. Use
Your Outcome Findings 7. Improve Your System 6. Analyze and Report
Your findings 5. Try Out your Outcome Measurement System 4. Prepare
to Collect Data on Your Indicators 3. Specify Indicators for Your
Outcomes 2. Choose the Outcomes You Want to Measure 1. Get
Ready
Slide 12
12 What happens with data that we are measuring?
Slide 13
Logic Model Process Questions What outcome do you want to
achieve? What activities will it take to get you there? What
resources (inputs) will you need to complete the activities? How
will you know when you have achieved your outcome? (How will you
measure?)
Slide 14
14 Logic Model Foundation GOAL Describe the Goal of your
program: The goal should be client- focused - broad statement of
intended change which identifies the target population. Broad
statement of desired condition Client focused Describe intended
change of target population Example: Working poor individuals and
families living in the Roanoke Valley will attain gainful
employment and become self-sufficient.
Slide 15
15 Logic Model Foundation TARGET POPULATION Describe the
population of clients that will be served. Age group of program
targets Description of population (i.e. gender, race, disability,
socio economic status, etc) Unique characteristics of the target
population (i.e. needs, condition, barriers, etc) Geographic
location or community of focus of program targets May include a
profile of a typical client
Slide 16
Inputs Human Financial Organizational Community If I have
access to these resources, I can conduct. Activities Processes
Events Programs If we conduct these activities, we should see.
Outputs Amount of Service delivered Number of Product If the target
population receives these services and products, they should
demonstrate Outcomes Indicates a measurable change to the target
Increased behavior Increased knowledge Increased skills If these
benefits are realized, then we have IMPACT!
Slide 17
17 Logic Model Components INPUTS Inputs are resources needed
for operation of the program, such as: Staff positions & staff
time Volunteers & volunteers time Facilities Equipment and
supplies Funding and funding sources Collaborative partners
Slide 18
18 Logic Model Components ACTIVITIES Activities, processes or
events undertaken with the inputs Services a program provides for
its participants to fulfill its purpose No quantities (percentages,
numbers of participants, numbers served, etc.) Examples: Feed and
shelter homeless families Provide job training Educate the public
about signs of child abuse Counsel pregnant women Create mentoring
relationships for youth
Slide 19
19 Logic Model Components OUTPUTS The direct results of program
strategies, activities, processes, and/or events. Usually described
in terms of size and scope of the services or products delivered or
produced by the program. Outputs indicate whether or not a program
was delivered to the intended audiences at the intended dose.
Example: the number of classes taught, meetings held, materials
distributed, counseling hours, program participation rates, or
total service delivery hours.
Slide 20
20 Logic Model Components OUTCOMES Outcomes are specific
statements of the desired changes in the lives of a particular
group (the target population). All of the outcomes relate to the
clients listed in the target population All of the outcomes relate
to the programs components listed in the strategies All of the
outcomes state significant changes in or desirable benefits to be
achieved by the target population All of the outcomes are stated
positively and are appropriate for the target population All of the
outcomes follow a logical progression (if.then)
Slide 21
21 Logic Model Components INDICATORS Indicators track a
programs success on outcomes. All of the indicators are clearly
linked to and measure progress toward the associated outcome. Each
indicator is stated with a target number and target percentage of
the total population that is expected to achieve the outcome. All
the indicators are based on the target population. All the
indicators are observable will measure some aspect of the outcomes
they are attached to.
Slide 22
22 Logic Model Components OUTCOMES vs. INDICATORS Outcome:
Participating students succeed in school. Indicator: The # and % of
participating students who pass all their core subjects. Outcome:
Job training graduates become gainfully employed. Indicator: The #
and % of graduates who secure full-time employment. Outcome:
Parents read to their preschoolers more often Indicator: The # and
% of parents who read to their preschoolers everyday for a minimum
of 30 minutes per day.
Slide 23
TEST YOUR UNDERSTANDING! Build the logic model for your
program
Slide 24
24 What are Outcomes? Outcomes are specific statements of the
desired changes in the lives of a particular group (the target
population). All of the outcomes relate to the clients All of the
outcomes flow logically from the strategies All of the outcomes
state significant changes in or desirable benefits to be achieved
by the target population All of the outcomes are stated positively
and are appropriate for the target population All of the outcomes
follow a logical progression (if.then)
Slide 25
25 Client-focused Outcomes vs. Process-focused Outcomes
OUTCOMES: Should be client-focused.
Slide 26
26 Outputs vs. Outcomes Program ExampleOutputs (amount of work)
might be Intended outcomes (results of the work) might be
Comprehensive child care Number of children registered Number of
days of care Children exhibit age- appropriate social skills
Children are ready for kindergarten Treatment for youth abusing
drugs and alcohol Number of counseling sessions conducted Number of
youth completing treatment Participating youth know the risks of
substance abuse Participants remain clean and sober for 6 months
Congregate meals for seniors Number of seniors enrolled Number of
meals served Seniors eat at least one nutritious meal each day
Seniors interact socially with their peers
Slide 27
27 Writing Effective Outcome Statements Some tips: Specifically
states what will be true of the clients/participants as the result
of your programs influence on their lives Outcomes focus on the
participants/beneficiaries, not on the program Statement contains
only one intended change or benefit Statement is very intentional
(specific) in the desired change or benefit, and not simply that
something is now improved, increased, decreased, enhanced, or
expanded
Slide 28
28 Developing Your Outcomes Questions to ask yourself What do
we want to be true of participants because of their involvement
with our program? What do we want to be able to say about them? If
we succeed with a participant (or dont), what has changed (or
hasnt)? If we carry out these strategies, then what do participants
believe, know, have, or do as a result? And what benefit or change
flows from that?
Slide 29
29 What are Indicators? INDICATORS Indicators track a programs
success on outcomes. All of the indicators are clearly linked to
and measure progress toward the associated outcome. Each indicator
is stated with a target number and target percentage of the total
population that is expected to achieve the outcome. All the
indicators are based on the target population. All the indicators
are observable will measure some aspect of the outcomes they are
attached to. Indicators are SMART: Specific, Measurable,
Attainable, Relevant, and Time Bound
Slide 30
30 Outcomes & Associated Indicators Outcome: Benefits for
participants due to their involvement with a program. Indicator:
The specific information collected to track a programs success on
outcomes. e.g., The number and percent of
Slide 31
31 Outcomes & Associated Indicators Outcome: Participating
students succeed in school. Indicator 1: 70 of the participating
students (78%) will pass all their core subjects. Indicator 2: 60
of the participating students (67%) will advance to the next grade
level. Outcome: Job training graduates become gainfully employed.
Indicator 1: 52 of the graduates (69%) will secure full-time
employment. Indicator2: 40 of the employed graduates (77%) will
remain employed for at least six months. Outcome: Parents read to
their preschoolers more often Indicator: 280 parents (80%) will
read to their preschoolers everyday for a minimum of 30 minutes per
day during the Fall 2007 semester.
Slide 32
32 Common Misconstructions when Writing Indicators Vague
Indicators e.g. Job Promotions 50 clients (64%) will receive job
promotions within 3-months of completing the course. Not Written
Measurably e.g. Able to converse better 42 ESL graduates (91%) will
demonstrate improved English language conversation skills by
graduation.
Slide 33
33 Common Misconstructions when Writing Indicators Listing
Measurement Documents e.g. Attendance Forms 33 students (67%) will
have perfect attendance during the Fall 2007 semester. Listing
Measurement Plans e.g. Review of Test Scores 75 students (85%) will
pass all of their core subject final exams.
Slide 34
34 Common Misconstructions when Writing Indicators Listing
Outputs or Strategies (process-focused) e.g. Clients attend all 10
workshops 45 clients (74%) who attended all 10 work shops will
demonstrate improved credit score ratings. e.g. Six trainings are
offered during the year 123 training participants (90%) will show
increased knowledge of the subject based on pre/post testing.
Slide 35
35 Common Misconstructions when Writing Indicators Listing
Outcomes e.g. Clients change behaviors and make healthier choices
58 clients (89%) show improved nutritional intake at the 3-month
re-assessment Not Associated to Outcome e.g. Homework assignment
completed (measuring self confidence) 12 students (72%) who
complete the leadership training course will self-report improved
self confidence.
Slide 36
36 Selecting Outcomes You Want to Measure Pick the fewest
number of outcomes that, as a group, will yield information for
three key purposes 1.Assure that you are achieving meaningful
benefits for the intended beneficiaries Which outcomes are the most
important for program participants to achieve? 2.Communicate the
value of your program to key audiences Which outcomes do key
audiences, including funders, care most about? Which best tell your
story? 3.Guide program managers and staff in increasing the
effectiveness of your program Which outcomes will be the most
helpful in identifying where the program is, or is not, being
successful?
Slide 37
Evaluating Your Logic Model for Improvements
Slide 38
Ask the following general questions: Are the outcomes really
outcomes? Do they describe changes or benefits for program
clients/participants? Is the logic logical? Does it make sense that
the strategies will lead to the initial outcomes, which will lead
to the next outcomes, etc.? Is the long-term outcome meaningful for
program participants? Have clients/participants experienced a
change or benefit that makes a real difference in their lives? Is
the long-term outcome reasonable? Is reasonable to think that the
program actually can influence this outcome to a reasonable degree?
Is the long-term outcome plausible? Will clients actually achieve
this outcome? Evaluating your Logic Model