Identifyingand Measuring

Outcomes inESC Work

Edward M. Haugh Jr.ESC Consultant

Outline of Presentation

III. Recommendations for Applying Outcomes Planning

to ESC

I. Introduction to Outcomes Planning

II. A Sample ESC Project Using Outcomes Planning

I. Introduction to outcomes planning

First

Promotes accountability

Helps us know if we are we really getting value for our money

Meets government, and funders demand for real tangible results

Why Focus on Outcomes?

Strengthen existing services

Target effective services for expansion

Identify staff and training needs

Develop and justify budgets

Prepare long-range plans

Focus board on programmatic issues

Outcomes help us and our Clients

Outcomes are what matter to stakeholders

What are your measurable outcomes?

Mark each category that applies to your grant measures. Increased

visibility/awareness of issue Effected policy change Engaged constituents &

beneficiaries Increased resources for

issue Educated constituents Improved condition/quality

of life in community Influenced stakeholders

ESC Must Demonstrate ResultsA Funder Asked ESC the Following:

Tangible benefits or changes for individuals or populations, during or after participating in a program or project

What is an outcome?

Improved (nutrition, morale)Increased (employment) Reduced (HIV transmission)Enhanced (knowledge)Maintained (vaccination coverage)

Outcomes – often expressed as verbs in past tense

Improved board governanceIncreased financial sustainabilityReduced volunteer turnoverEnhanced market recognitionImproved strategic focus

Typical ESC Project Outcomes –

Outcomes are the results

Outputs are the tangible products produced by an interventionUsually expressed as nounsCan be counted

Outcomes vs. Outputs

Classes taught Brochures distributed Clients served Contracts completed Board retreats completed Strategic plans prepared

Some Examples of Outputs?

Measure the quantity or quality of the outcome or output

Indicators

Outcomes sometimes confused with outcome indicators Outcome: girls participating in a

training program obtain formal employment

Outcome indicators: the # and % of participants employed within 3 months of completing the program

Outcome Indicators

Outcome: Participants in a job training program obtain a job in a timely manner

# (%) of participants employed within 6 weeks

# (%) of participants still employed 3 months after completing the program

Indicators

# (%) of participants employed within 6 weeks

# (%) of

participants still employed 3 months after completing the program

60% of participants will gain employment within 6 weeks

95% of participants who obtained employment will still be employed in 3 months

Indicators & Targets

Logic Model Helps Define Outcomes

Processes

OutcomesOutputsInputs Impact

Putting it all TogetherComponent Indicator Target Key

Assumptions

Outcome

Output

Program/Activity/Process

Input

Outcomes

Outputs

ProgramsActivitiesProcesses

Inputs

If

If

If

Key A

ssum

ptio

ns

Type I Failure – the model doesn’t work

Type II Failure – the model is not properly implemented

Logic Model is not a Panacea

Explain why an outcome is or is not being achieved

Still need to track inputs, activities, outputs

Prove that the outcome is the result of your program

Note that Measuring Outcomes will Not:

Outputs are often more tangible

Outcomes are less tangible

Its easier to count tangibles than to describe intangibles

Outcomes take longer to produce

Measuring outcomes is challenging

Measuring outcomes can be expensive and time consuming

So Why Don’t We Focus on Outcomes?

II. A sample ESC project using outcomes planning

Next

A good problem analysis or diagnosis

Good Planning

Producing Strong Outcomes

A typical ESC Problem Analysis

Lack Brd/Mgt financial oversight

Low operating efficiency and high per patient costs

ED and (staff) overwhelmed and poorly focused

Insufficient numbers of volunteer dentists and poor coordination

Lack financial information for Brd/Mgt

Lack budget and operating plan no plan to expand access

No financial sustainability

Lack of Access In spite of high demand – empty dental chairs

Poor oral health for clients and potential clients

High costs of health insurers and general public due to overuse of emergency rooms

Loss of productivity and economic potential in the County due to poor health

Start with a Problem Analysis

Inputs Processes OutputsShort-term Outcomes

Medium-term

Outcome

• Staff time• Volunteer

time• CRM Time• ESC

management time

• Board time• Funds from

scholarship

• Board & Mgt meeting with ESC Consultants

• Financial review

• Develop projections for expanding access

• Develop plan for access

• Gap analysis and ED training

• Progress reports

• Meetings completed

• Projections prepared and approved

• Expansion plan approved

• Gap analysis finalized

• Final report• ED training

completed

• Improved fiduciary management

• Improved financial performance

• Improved ED performance

Improved access to

oral health

Long-term

Outcome

ImprovedHealth

Logic model for the ESC Engagement

Deliverables

ImpactImproved health,

economic productivity and lowered emergency room costs

for the county

Applying outcomes planning to ESC’s work

Finally

28

ESC’s Business Logic Model

ESC Project planning

Project implementation/Monitoring

Program Evaluation/ensuring accountability

Communicating value to funders and stakeholders

Annual reporting

How we can use outcomes planning

Define desirable outcomes for key activities/programs

Develop indicators to determine if outcomes have been achieved

Identify the means of verifying the performance

Identify a SMART goal relating to the outcomes

30

How to IntroduceOutcome Measures for ESC

Outcomes • Improved governance - board operating more effectively

Indicator

• Regular board meetings• Meetings are more effective• Board attendance has increased• Board turnover has reduced• Board members more satisfied with their

volunteer service

Means of indicator • Discussions with board chair, board minutes, surveys, attendance records, etc.

Sample SMART goal

• Attendance at board meetings increased from 75% of members per meeting to 90% within 6 months;

• Board satisfaction (as measured by a survey) will be increased from “satisfied” to “highly satisfied” with 6 months

31

Board Development

Outcomes

• Improved understanding of board/staff roles

• Improved committee structure• Improved relationships among the

board members

Indicator

• Participants can distinguish between the roles of board/staff

• All necessary committees in place and staffed• Board members have increased knowledge of

each other’s backgrounds, skills, motivation for joining board

Means of indicator• Pre-and post-tests, interviews• Survey, interviews• Surveys, interviews, final evaluation

32

Board Retreat

33

Implications for ESC

Board and management “buy-in”

Consultant “buy-in”

Client “buy-in”

Staff training

Time and effort (financial)

Continuous learning cycle

Develop a set of model outcomes for each ESC activity

Clearly specify the intended outcome for engagements

Evaluate whether the outcome has been achieved (or is likely to be achieved within an appropriate timeframe)

Provide training and support to secure “buy in”

Collect data and report to our stakeholders

Help our clients measure their own outcomes34

Recommendations

Board Retreat

ESC facilitated x board retreats which the nonprofits reported improved relations among the board members, x developed improved committee structures and x reported improved board member understanding of the roles the board and staff.

Annual Outcomes Report

35

Board Development

ESC helped x no of nonprofits to improve their board governance; of these x filled all vacant board positions; x developed more effective committees and x reported more effective board meetings.

36

Sample Annual Outcomes Report

Thank you!