CLEVER EVALUATION

Nikki Dettmar

Evaluation Librarian

NN/LM Outreach Evaluation

Resource CenterPresentation to the

NN/LM Pacific Southwest Region

February 19, 2014

Why don’t we exercise?

Evaluation is like exercise

It’s a good idea…and you’ll start doing it as soon as you have time

Advice for doing evaluation is the same as for exercise

Plan well Set realistic goals

Use efficient techniques

These strategies will get you across the finish line

Plan for

Evaluation

Know how you will use evaluation

Needs/Community

Process Results andOutcomes

Sta

rt

Fin

ish

Needs and community assessment can help you…

• Explore needs• Identify resources and

opportunities• Use• Design services/programs• Establish baseline info• Get stakeholder buy-in

Process assessment keeps you on the right track

• Monitor deliverables• Assess quality

• Use• Make midproject

adjustments• Report lessons learned

Outcomes evaluation is used to show results

• Report on planned • outcomes• Identify unplanned• outcomes

Use• Decision-making • Advocacy

Time Saver #1:

Time saver:

Community/needs assessment: Don’t assess what you (or someone else) already knows

Explore the collective knowledge of external experts

http://phpartners.org

http://phpartners.org/health_stats.html

For primary (“local”) information, think

“shoestring” methods

• Key informant interviews• Focus group interviews• Short surveys• Rapid response methods

Time Saver #2

For process and outcomes

assessment,

use a project plan

Think Through Activities, Outcomes and Indicators

Activities

If we do thesethings

Outcomes

We will get these results

Indicators

We will “see” this

if…

Think through activities, outcomes, and indicators

ActivitiesIndicators:• Deliverables• Job well done

OutcomesIndicators:• Evidence of results

Diffusion of Innovations theory: People adopt new

innovations in phases

Knowledge Persuasion DecisionImplemen-

tationConfirm-

ation

Design measures for the behavioral changes you are targeting

Knowledge Persuasion Decision Implemen-tation

Confirm-ation

AwarenessFavorable

AttitudeTrial use Application

Behavioral-level Outcomes

Think “small ball”POWER BALL: SMALL BALL:

Large number of VS. Winning games

runs in few by scoring

innings single runs in multiple innings

Communities adopt innovations in waves

Source: http://upload.wikimedia.org/wikipedia/en/4/45/DiffusionOfInnovation.png

Use different strategies for different parts of the adoption process

Knowledge Persuasion DecisionImplement-

ationConfir-ation

AwarenessFavorable

AttitudeAdoption Application

Behavioral-level Attitudes

Use to help others

Promote to others

Example 1: Train-the-trainer programs

Who are you targeting? • Opinion leaders

What are the planned outcomes? • Use for self• Use for others• Promotion to

othersHow would you measure them? • Post-training

attitude measures• Stories of use

Other “clever evaluation” time savers

Interviews and focus groups can be networking

opportunities

…And marketing opportunities

Evaluation techniques can support learning

Use tests to reinforce learning or practice skills

QUIZ

1. What is the “important warning” about the drug Warfarin? _______________

2. What usually are the first warning signs of pulmonary hypertension?_________

3. Name one health news article from November 5, 2008 _______________

Pose questions to help participants process information

1. “List one example of information you found today that helped you with a health concern”

2. “List two topics you plan to investigate using MedlinePlus”

3. “List 2 or 3 health topics that you plan to research at home this week using MedlinePlus”

Work with others as

a team

Don’t forget to ask for help with analysis, too

Free Support Resources

Planning and Evaluating Health Information Outreach Projects

• Available online http://nnlm.gov/evaluation/guides.html

• Want a print copy of the set? Email me!

OERC Tools and Resources http

://guides.nnlm.gov/oerc/tools

Nikki Dettmar, Evaluation Librarian

University of Washington, Seattle

206-543-3409

snydern@uw.edu

OERC Contact Information(nnlm.gov/evaluation)