Introduction to Monitoring and Evaluation –

Measuring Success

Cheryl Cichowski

Kira Rodriguez

Michelle Mitchell

Monitoring

Evaluation

Measuring success

2

The difference between monitoring and evaluation

3

Monitoring Evaluation

When is it done? Continuously Occasionally

What is measured Efficiency Effectiveness

Who is involved? Internal staff External contractors

What sources of information are used?

Internal documents Internal and external documents

Who uses the results? Managers and program staff

Mangers, staff, donors, beneficiaries, etc

How are the results used? To make minor changes To make major changes in policy, strategy, and future work

Group Exercise: Monitoring or Evaluation?

• Attendance at the District Coordinating Council meeting

• Completion of milestones• Policy and / or environmental changes in the

community• Correct use of funding• Representation of Advisory Council• Change in knowledge, attitudes, beliefs and

skills• Number of strategies implemented

4

Monitoring• A continuous process of reviewing information

gathered from program activities to compare the efficiency of the activities in meeting strategies.

• Monitoring helps focus the direction of program activities and facilitates minor changes.

Some Areas of State Monitoring

• Milestones

• Quarterly Reports

• Quarterly Counts

• Resources

• Success Stories

Why your Project Officer Monitors

• Compliance with contract

• Provide TA when needed

• State program needs to show work progress to federal

funders

• Learn from the grantees and share with others

• Facilitate a network of prevention providers

Everyday Evaluation• Labor Day Weekend activity

8

Evaluation• A systematic and objective assessment of

an ongoing or completed program, its design, implementation and results.

9

10

Engage Stakeholders

Describe the Program

Focus the Evaluation Design

Gather Credible Evidence

Justify Conclusions

Ensure use and lessons learned

Accuracy

UtilityFeasibility

Propriety

Overarching Evaluation Questions

• How well does the HMP Initiative help strengthen Maine’s public health infrastructure?

• How well is the HMP Initiative being implemented at community and state levels?

• What impact does the HMP Initiative have on the health status of Maine residents?

Data Collection

• Coalition Characteristics & Functioning Survey

• Objective Tracking

• Environmental Indicators Survey

• Secondary Surveillance Datasets

Coalition Characteristics Survey

13

Objective Tracking

14

Objective Database

Breathe Easy

Maine Tobacco-Free College

Network

Maine Tobacco-Free Hospital

Network

Department of Labor

Maine 2-1-1

KeepMEWellHMW

Environmental Indicator SurveysSample Results from 2011 Muni Survey

15

RESA: How familiar are you with HMP? (All Respondents)      Count %

$RESa Very Familiar 28 6%

  Somewhat Familiar 208 42%

  Somewhat Unfamiliar 110 22%

  Very Unfamiliar 145 30%

  DK/NO ANSWER 5 1%

  Total 490 100%

RESB: Do you know who your local HMP is? (All Respondents)     Count %

$RESb Yes 97 20%

  No 393 80%

  DK/NO ANSWER 7 1%

  Total 490 100%

RESC: Have you had any interaction with your local HMP in the past year? (All Respondents)     Count %

$RESc Yes 91 19%

  No 395 81%

  DK/NO ANSWER 11 2%

  Total 490 100%

Surveillance

16

Ensuring use of findings• Reports• Data sheets• Presentations• Website• Peer Journals• Annual Conferences• Display tables

18

5 CLICKS TO EVALUATION DOCUMENTS

www.healthymainepartnerships.org

HMP Grantee Resources

Resources and Documents

Useful Documents

Evaluation Resources

19

Contact Information

Please contact us with any questions

Cheryl: Cheryl.Cichowski@maine.gov Kira : krodriguez@une.edu

Michelle: mici.mitchell@gmail.com