Post on 08-Apr-2017
Evaluating ChangeKaren Horsch
Evaluation Consultant
We are experiencing epidemic rates of childhood obesity that have long-term consequences for health and well-being of individuals and society
It’s getting worse, not better
What we know:
Reversing this trend in a sustained way is going to take fundamental changes to the way we think and act: we cannot program our way out of this
Reversing this trend in a sustained way is going to require that we build a movement
What we know:
We need to shift perceptions such that physical activity and healthy eating are the social norm
We need to establish environments that make individuals’ default decisions healthy
What needs to happen:
They raise issues of urgency (identify problems)
They build a base of interested and motivated individuals and coalitions
They educate and raise awareness They advocate for positive changes in
policies and institutions They identify and promote promising
interventions and programs (show what’s possible)
What movements do….
How does evaluation fit in?
Data about the movement (is the movement moving?)
Data to build the movement
Two levels of evaluation when building a movement:
So how do we evaluate whether we are building a movement?
NOT: What did we accomplish?
BUT: Did we accomplish what we set out to accomplish? Why or why not?
Start with the right evaluation question:
The “theory of change” about how to build a movement
Laying out what we hope to accomplish:
The Strategies and Interventions
The Results we hope to achieve:
• Short-term
• Long-term
• Intermediate term
Strategies
• Coalition building
• Grassroots organizing/mobiliz-ation
• Leadership development
• Advocacy training
• Media advocacy
• Identification & replication of proven programs
Short-Term Outcomes
• Increased awareness about and prioritization of the issue
• Strong and diverse network of advocates
• Strengthened alliances
• Increased media attention to the issue
• Established relationships with key decision-makers
.
Intermediate-Term Outcomes
• Increased public and policymaker support for policies and actions that promote positive changes in the issue
• Actions supporting physical activity and healthy eating:
-- more funding-- more programs-- policy change-- changes in institutions
Long-Term Outcomes
• Behavior changes: increased physical activity and healthy eating and reduced screen time
• Reductions in obesity rates
• Improved health
• Who the decision makers are• Who the public is• Who partners might be
Being clear about:
• Information sent out and overall message framing
• Number and type of people who receive information
• Attendance at meetings/events• Website hits• Public opinion surveys
Increased awareness about and prioritization of the issue:
• Number and type of people trained in advocacy
• Number and type of people taking on leadership roles in advocacy
• Number of new advocates • High profile people who adopt the issue and
publicly advocate for it• Actions taken by advocates
Strong and diverse network of advocates:
Number and type of coalitions/partnerships established
Number of active members of coalition Health of coalitions Number and type of cross-organization and
cross-sector partnerships/collaborations Network analysis
Strengthened alliances:
Number of press releases Number of op-eds/letters to the editor
published Number of articles covering the issue Relationships with the press How the issue is framed in media
Increased media attention to the issue:
Visits to decision makers Number of decision makers attending
events related to issue % of phone calls returned by decision
makers
Established relationships with decision makers:
Number of decision makers and/or opinion leaders who express support for the issue
Policy makers’ press releases and comments in legislative hearings
Policy maker attendance at hearings related to the issue
Increasing support for:
New policy proposals New policies passed Policies expanded/or not reversed Implementation of policies
Improving Policies:
Balancing the data collection effort with the utility of the information collected
It’s not all countable: the importance of qualitative data
Data Considerations
So what data do we need to build the movement?
Obesity as connected to environments and not solely individual behavior
Role of government in this Importance of prevention rather than long-
term health consequences
Data that helps to reframe the issue:
Population-level data collection Community mapping and other visuals The cost of doing nothing
Data that points to the urgency of the issue and its causes:
What programs work and how well◦ How many they serve◦ The outcomes they achieve
What programs cost or save
Data about what works (Program Evaluation)
“Policy change occurs when community leaders receive credible and reliable data and research AND community members provide personal stories and advocate for change.”
-- Kay Monaco, New Mexico Voices for Children
Policymakers receive a lot of information Easy visuals: mapping, metrics, summaries Personal stories and testimonials matter Timeliness and credibility are critical
Policymakers as Audience
POLICY CHANGE
Organizing and Coalition
Building
Finding information/ Research
Making a Plan
Mobilization and
Communications
Climate ready
For change
Source: PolicyLink