Reasonable ROI Tracking Measuring Internal and External Impact Created in Partnership between the...

Reasonable ROI TrackingMeasuring Internal and External Impact

Created in Partnership between the Forum of Regional Associations of Grantmakers and TCC Group

2

Outline of Presentation

I. Introduction

II. Key Issues in Corporate Community Involvement Measurement

III. Defining Measurement

IV. Approach to Measurement A. Resources for Measurement

B. Framework for Measurement

V. Reporting Results

Section 1Introduction

4

Thesis of the Presentation

Expectations for companies as responsible

corporate citizens are rising, and stakeholders are

demanding results. Clearly defined goals with

strong social impact makes for a more compelling

story to tell your stakeholders, resulting in greater

impact for the business.

5

The Key Questions

Why measure?

For whom?

What do they want to know?

What will you measure and how?

6

Telling a Compelling Story

The Typical Story

• Focuses only on the amount donated

• Does not highlight business benefits

• Targeted to “everyone”

The Compelling Story

• Tells the big picture and the specific goals/results

• Explains the business rationale to internal audiences

• Messages targeted to key audiences

“In the past 10 years, XYZ company has donated more than $10,000,000 to hundreds of non-

profit organizations that have enriched communities

worldwide.”

7

Examples of Compelling Stories

Procter & Gamble Programs

Pampers and UNICEFWorking Together for Healthy Babies:

45.5 million women and their babies have been protected against maternal and neonatal tetanus.

P&G and Hope SchoolsP&G and Hope Schools have benefited over 100,000 children across China in the last 12 years.

SafeguardBuilding a Great Wall of Hygiene in China and Beyond

The program’s goal is to educate 10 million children in the next eight years..

Focus on the Goals/Impact you are trying to make: Even with limited resources, the story can still be compelling!

Section 2Key Issues in Corporate Community Involvement Measurement

9

Increasing Expectations / Demand

Competitive Advantage

Continuous Improvement

Why Measure?

10

Barriers to Measurement

• Different expectations of “results”

• Lack of internal consensus on what to measure

• Limited budgets or support

• Limited internal knowledge/capacity

11

Who are Your Key Stakeholders?

Internal

• Employees • CEO• Senior Mgmt / Executive

Team• Corporate Board of

Directors• Foundation Board• Other Departments• Business Units• Domestic/International Field

Offices• Shareholders• Business Partners

External

• Key Opinion Leaders (Influencers)

• Community Partners• Grantees• Consumers• The “Field”• Academics• Watch Dog Groups• General Public

12

What do Your Key Stakeholders Care About?

Internal Stakeholders

• Employees • CEO• Senior Mgmt / Executive

Team• Corporate Board of Directors• Foundation Board• Other Departments• Business Units• Domestic/International Field

Offices• Shareholders• Business Partners

Expectations / Concerns

• Company Reputation• Employee Involvement/Pride• Employee Retention,

Recruitment, Morale• Business Alignment/CSR

Integration• Continuous Improvement,• Learning about Best

Practices• Social Impact• Increased sales / demand for

products or services

13

What do Your Key Stakeholders Care About?

External Stakeholders

• Key Opinion Leaders (Influencers)

• Community Partners• Grantees• Consumers• The “Field”• Academics• Watch Dog Groups• General Public

Expectations / Concerns

• Transparency• Real Social Impact• Improving the community• Sound environmental

practices• Treating employees well

Section 3Defining Measurement

15

OR, a method for finding out what happened & why…

What is Measurement?

“A systematic study conducted to assess how well a program is working...typically focused on achievement of program objectives.”

The General Accounting Office (GAO)

16

Comparisons of Different Types of Measurement

Documenting Research

Tracking investment and progress Determining causality

Minimum evaluation expertise required High level evaluation expertise required

Low cost High cost

Monitoring Performance measurement

Outcome measurement Impact study

17

What Can Be Measured?

18

Examples of What Can Be Measured

19

Monitoring

Business Value Social Value

• Company investment and employee giving (dollars)

• Employee participation (hours)

• Grantee spending

• Grantee project focus

• Grantee timeline

Performance measurement

Outcome measurement Impact studyMonitoringMonitoring

20

Performance Measurement


• Employee participation (hours)

• Employee giving (dollars)

• Employee satisfaction (w/ project)

• Lines of business/plant comparisons

• Number of activities (type, freq.)

• Number of individuals served

• Characteristics of who is served

• Alignment w/ proposed activities

• Quality of proposed activities

Monitoring Outcome measurement Impact study

Performance Performance measurementmeasurement

21

Outcome Measurement


• Improved leadership

• Improved management

• Improved team work

• Improved productivity

• Improved retention and loyalty

• Improved employee morale

• Individual change (positive)

• Improved quality of life

• Increased org capacity

• Changes in policy

Monitoring Performance measurement Impact study

Outcome Outcome measurementmeasurement

22

The Pathway to Change: Social Value

Individual Behavior Change (Learning / for Whom)• Awareness

• Knowledge

• Attitude

• Motivation

• Skills

• Opportunity

Organizational / Systems Change (Actions)• Behavior

• Practice

• Policies

• Decision-making

• Social Action

Community Change • Health

• Social

• Economic

• Environmental

Outcomes

Impact

23

The Pathway to Change: Business Value

Levels of Outcomes Internally Externally

Individual• Awareness• Knowledge• Attitude• Motivation• Skills• Opportunity• Behavior

Employee•Increased awareness of CI•Increase knowledge of community investment•Improved attitude toward company

Public / Consumer•Increased awareness of company•Increase knowledge of and attitude toward the brand•Increased loyalty/consumption

Organizational• Practice• Policy• Decision-making• Social Action

Company/ Business Unit•Improved leadership•Improved management•Improved team work •Improved productivity•Improved retention and loyalty•Improved perception of brand

Public / Consumers•Employers/nonprofits change purchasing decisions•Employers/nonprofits align w/ CSR of company•Employers/nonprofits partner with company

24

Impact Study


As a result of the CI program…

•Improved reputation, Improved brand recognition and perception of brand (among key stakeholders)

As a result of the CI program…

•Improved community

•Improved system

•Global change

Monitoring Performance measurement

Outcome measurement Impact study

25

Exercise: What types of measurementdo you currently do?

Answer the following:

• What types of measurement do you currently do?

• Where are you on the continuum?

• What did you learn from that measurement experience?

• What types of measurement would you like to do? Why?

26

Program ContentKey Stakeholder(s)

Reached

Signature Focused on one issue All

Major Issue or Focus Area Giving

Usually 2-3 issuesDomestic/International Offices; Community Partners; Employees

Local GivingBroad: based on local needs

Local community leaders, employees

Client-Driven GivingBroad: based on client requests

Clients; Business Partners

Disaster Relief GivingFocused on one region/issue

Potentially national/international audience; regional stakeholders

Employee Engagement (Volunteerism, Dollars for Doers, Matching Gifts, etc.)

Broad: based on employee interests and company priorities

Employees; Community Partners

Framework: Portfolio of Programs

27

Case Study (APC)

The American Product Company (APC)

• The Current SituationAPC’s reputation has diminished over time with key stakeholders due to some environmental accidents, layoffs due to the recession, and some recalls of its products. The company has a small number of sales people around the country, 500 employees, and is headquartered in Grand Ville, a depressed urban city.

• Current ProgramsThe company contributes less than $250,000 annually to organizations in Grand Ville, and has a small number of employee involvement programs. Greater detail on next slide.

• The Assignment• What should APC’s measurement approach be? • How should APC determine what to measure and at what level?

28

APC’s Portfolio of Programs

Program Content Resources

Signature Program: Teacher Training in Grand Ville

Targeted support of teachers in 3 public schools in poor communities

$100,000 annually

50 volunteers

Employee contributions

Major Focus Areas: Education, Social Services

Support of critical needs in education and social services in Grand Ville

$75,000 annually

Business / Client-DrivenSupport for events, organizations based on client requests to sales reps around the country

$25,000 annually


Support for matching gifts and dollars for doers

$50,000 annually

APC focuses primarily on its headquarters community and where its employees live and work

29


Reached

Signature


Local Giving

Client-Driven Giving

Disaster Relief Giving


Exercise: Your Portfolio of Programs

30

Higher Cost Strategies

•Develop a program model with clear and measurable goals

•Hire an external measurement expert to design data collection plans, systems and tools

•Provide PD for grantee’s internal staff around data collection and measurement

•Fund peer-to-peer exchanges re: measurement

Resource Choices

Low-Cost Strategies

•Develop a program model with clear and measurable goals

•Ensure that CI program vision, goals and objectives align

•Focus on measuring inputs and outputs

•Create a tracking system to regularly monitor and report progress (inputs and outputs)

Medium-Cost Strategies

•Develop a program model with clear and measurable goals•Create a measurement framework that lays out key measurement questions, indicators, data sources, data collection methods•Focus on measuring inputs, outputs and short-term outcomes•Utilize data collection tools that have been developed or could be developed easily•Create a tracking system to regularly monitor and report progress

31

Resources: As a Percent of Program Budget

Monitoring

1 – 2%

Performance measurement

3 – 5%

Outcome measurement

5 – 10%

Impact study

10% +

32

What resources do you currently haveor can you leverage?

Skills and resources(Also consider other departments)

• People (time and expertise)• Money• Technology• Partners• Leadership

Existing data/ tools(Also consider other departments)

• Marketing reports• Demographic analyses• Community data, reports

Company culture• HR data• Department data

Broader context

33

Grantee Data Collection / Reporting

Low-Cost Strategies

Question: Did you do what you said you would do?

Focus on measuring inputs and outputs through budgeting and program tracking mechanisms

•Report on resource allocation, grant expenditures, activities, etc. in alignment with program model

•Articulate alignment of activities and expenditures with vision, goals and objectives

Medium-Cost Strategies

Question: Did you do what you said you would do well?

Focus on measuring components and quality of processes and beginning to measure outcomes through easily gathered internal data (HR data collection), and interviews with external stakeholders•Create a measurement framework that lays out key measurement questions, indicators, data sources, data collection methods•Focus on measuring inputs, outputs and short-term outcomes

Higher-Cost Strategies

Question: What were the results of what you did?

Focus on measuring outcomes through research based tools and rigorous methods including surveys, discussion groups, interviews

•Hire an external measurement expert to design data collection plans, systems and tools

34

Exercise: Barriers and Challenges

Skills and resources

Existing data/ tools

Company culture

Broader context

Section 4Approach to Measurement

36

Building a Measurement Framework

1. How can measurement data be used?

2. What levels of information is needed? By whom?

3. What can and should be measured (at a portfolio level and a project level)?

4. What tool/system can be used to measure?• Data that already exist• New data that need to be collected

5. Who participates in measurement?

6. What is the estimated cost of measurement?

37

Measurement Questions

(What do we want to know?)

Indicators/ Measures

(How will we know?)

Information Source (From whom will we collect the data?)

Data Collection Methods (How will we collect the data?)

Business Impact

Question: To what extent

are employees performing

better?

Social Impact Question: To

what extent have teachers improved their

classroom effectiveness?

Outcome: Increased

perceptions, productivity, engagement, performance,

etc.

Asst Supt

C. Offc Prin Tch Stu Par Oth Intvw Obsvn Foc

GrpCase Study

Sur-vey

Doc Rvw Other

Outcome:Teacher’s

demonstrate increased capacity in program

related areas

A Sample Measurement Framework

38

Guiding Questions

1. How can measurement data be used?

2. What levels of information is needed? By whom?

3. What can and should be measured (at a portfolio level and a project level)?

4. What tool/system can be used to measure?• Data that already exist• New data that need to be collected

5. Who participates in measurement?

6. What is the estimated cost of measurement?

39

vs. ReactiveProactive

How can Measurement Data be Used?

40

Types of Evaluation

Formative• prospective• analyzing toward improvement• feedback

Summative• retrospective• document results• evidence

“When the cook tastes the soup, that’s formative; when the guests taste the soup, that’s summative.”

~ Robert Stake, Director, Center for Instructional Research and Curriculum Evaluation

41

How Measurement is Often Used in the Corporate Sector

• To identify business impacts

• To better plan their work

• To improve program implementation

• To monitor how well they are doing in meeting their goals

• To make a clear business case for community investment

42

Types of Measurement Needed by Types of Program

Signature

Major Issue or Focus Area

Giving

Business-Driven Giving

Employee Programs

Pro

gra

ms

Inputs Outputs Short-term Outcomes Long-term Outcomes Impact

43

Levels of Information Needed by Stakeholders


ReachedLevel of Information

Needed

Signature Focused All Impact; Long-term outcomes


Focused on several issues

Regional Stakeholders; Community Partners;

Employees

Outputs; short-term outcomes

Local Area GivingBroad: based on

local needsLocal community leaders,

employeesInputs and outputs, some

short-term outcomes

Client-Driven Giving

Broad: based on client requests

Clients, Business Partners Inputs

Employee Programs

Broad: based on employee

interests and company priorities

Employees, Community Partners

Varies by program; primarily inputs and outputs, some

short-term outcomes

44

Examples of What Can Be Measured

InputsInputs

All the resources needed to support a program:•People•Dollars•Marketing•Lines of business

The specific activities and/or programs that serve a particular target audience:•Signature programs•Focused giving•Employee engagement activities

OutputsOutputs

Affects on target audience:•Community awareness, perception of company•Grantee knowledge, attitudes, behaviors•Employee morale, retention, engagement, team work, leadership, productivity

OutcomesOutcomes

The long-term and aggregate effect of the program on overall target audience and the company:•Social impact•Business impact (Reputation among key stakeholders)

ImpactImpact

45

APC’s Portfolio of ProgramsWhich Programs Need to be Measured?

Program Content Resources



$100,000 annually

50 volunteers



Support of critical needs in education and social services in Grand Ville

$75,000 annually

Client-Driven

Support for events, organizations based on client requests to sales reps around the country

$25,000 annually



$50,000 annually

APC focuses primarily on its headquarters community and where its employees live and work

46

OutcomesOutcomesStrategiesStrategiesInputsInputs

The key “learning-based” question

Measurement Questions

Which resources were most important for providing “high-quality” service?

Which strategies (program qualities) were most

important for achieving the desired outcomes (including

business outcomes)?

What was the quantity and quality of the

resources we used to implement our

programs?

How much service did we provide and what was the quality of that service?

How were our employees involved (to what extent, in

what ways)?

Did our business or employees change?

Did our service recipients change?

If so, how much and in what ways?

The key “learning-based” question

47

Signature Program Major Focus Area

Resources

Strategies/

Outputs

Outcomes

Impact

Practice Writing Measurement Questions

48

Sample Data Collection Methods, Tools

In House:• Financial databases/reporting• HR surveys and data (recruitment,

retention, morale, productivity, performance)

• Gifts databases• Grantee reports• Line of Business systems and

reports• Existing data (consider marketing

and sales information)• Media Hits• Consumer data

External:• Document/File Review• Surveys• Interviews• Focus Groups• Core Capacity Assessment Tool

(CCAT)

49

Program Content ResourcesLevel of

MeasuresMeasurement

Tools



$100,000 annually

50 volunteers


Outcomes

• Teacher survey• Employee survey• Media hits


Support of critical needs in education & social services in HQ community

$75,000 annually Outputs

• Financial databases

• Gifts database• Grantee reports

Client-DrivenSupport for events, organizations based on client requests

$25,000 annually Inputs• Financial

databases



$50,000 annuallyOutputs - Outcomes

• Financial databases

• HR, survey• Media hits

How to Measure… An Example: APC

50

Measurement Question

What would you see? & What would you hear?

Information Source

Practice Creating Measurement Framework

51

Who Participates in Measurement?What role do they play?

MeasurementMeasurement

Community Stakeholders,

Grantees, Volunteers

C-Suite Executives

Senior Leaders and Managers

Community Involvement

Staff

Implementer / Coordinator

ReviewerReviewer

Recipients of Findings

52

Using SMART Criteria for Assessing Your Measurement Plan

• Specific: Define the target, focus efforts• Who, What

• Measurable: Define the criteria, obtain feedback• How much, How many, How will I know?

• Action-oriented : Define/align the desired change/movement

• Verb (Increase, Decrease, Improve, etc.)

• Relevant to Outcome level: Align to level of change…• Individual, Organization, System

• True to Program Mission: Align to…

53

APC Company SMART Example

Specific

Who (teachers) & What (knowledge, awareness, capacity)

Teachers knowledge

Measurable

Movement / change – more or better (e.g., increased, enhanced, built capacity, etc.)

Number / Percentage

Action Oriented Verb Increase

Relevant Aligned with outcome level Individual

True to mission Aligned with goal area Academic Excellence

54

Key Takeaways

• Maximize the resources you already have

• Leverage the resources of other departments

• Select your measurement level based on your resources, the program you want to measure, and the type of information you need for your targeted audiences

Section 5Reporting Results

56

Ready to Report

Now that you have:

1. Determined why you want to measure your results

2. Decided to whom you want to communicate

3. Decided which programs and what about them to measure

4. Created and implemented a measurement plan

You’re ready to report your results!

57

Planning for Reporting

How you report varies

• By the audience(s) you are trying to reach• By the type of program you are measuring / the results

you have• By the level of resources you can direct toward

reporting

58

Types of Reporting

ProgramWhat You Measured

Key Stakeholder(s) to Communicate to

Ways to Communicate

SignatureImpact; Long-

term outcomesAll

• Annual report• Company website• Sharing results with field through

conferences and publications


Outputs; short-term outcomes

Regional Stakeholders; Community Partners;

Employees; Senior Leadership

• Annual report• Company website• Local/regional press• Intranet articles• Report for Sr. Leaders

Client-Driven InputsClients, Business

Partners• Report for Sr. Leaders

Employee Programs

Primarily inputs and outputs; some short-

term outcomes

Employees; Community Partners;

Senior Leadership

• Annual report• Company website • Intranet articles• Local/regional press• Report for Sr. Leaders

59

Types of Reporting

60

Intranet Articles

Reports to Senior Leadership

Annual CSR Report

Fewer Resources Needed More Resource Intensive

Info on Company Website

Press Releases / Media Hits

Resources Needed for Reporting

For more resource intensive efforts, try to tap into the existing communications,

design, and marketing expertise in your company

61

Best Practices in Reporting

Tailor to audience

The right amount of info

Tap into 3rd party endorsements

Employees can be your most important group

62

Many Benefits of Reporting

Employees know about opportunities to engage

Senior Leaders realize the business value of your programs

Employees have greater pride in their company

Nonprofit organizations know what issues you

care about

Community Leaders hear how you are

improving their area

Compelling Compelling Story / Story / ResultsResults

General public views you as a good corporate

citizen

63

Examples of Effective Reporting:Cole Hardware’s Website

64

Examples of Effective Reporting:Sleep Train’s Website

65

Examples of Effective Reporting:CSR Reports

66

Exercise: How Do You Plan to Report Your Results?

Program/Activity

Who do you want to reach?

What do they need to know?

How will you communicate with them? How often?

Section 6 Resources

68

Resources

CEP website (foundations and strategy reports): www.effectivephilanthropy.org/index.php?page=publications

DHFS Evaluation Resource Guide. Wisconsin Department of Health and Family Services. Office of Strategic Finance Evaluation Section. Updated June 2006. http://dhs.wisconsin.gov/aboutDHS/OPIB/policyresearch/EvaluationResourceGuide06-06.pdf

Diversity in Philanthropy website: www.diversityinphilanthropy.org/news/case_studies/pdfs/DPP-Evaluation-Case-Study2.pdf

Davidson, Jane. ‘Evaluation Methodology Basics: The Nuts and Bolts of Sound Evaluation’: www.sagepub.com/authorDetails.nav?contribId=532276

GEO website (see report on Foundations and Evaluation and the Change Agent Project): www.geofunders.org/publications.aspx

Joint Committee on National Standards: www.jcsee.org Kellogg Foundation Program Evaluation and Logic Model Manuals:

www.wkkf.org/~/media/475A9C21974D416C90877A268DF38A15.ashx Mertens, Donna; and Ginsberg, Pauline. The Handbook of Social Research Ethics:

www.sagepub.com/refbooksProdDesc.nav?prodId=Book230293 RWJF Evaluation Series:

http://www.rwjf.org/files/research/50349.quality.checklist.final.pdf

Reasonable ROI Tracking Measuring Internal and External Impact Created in Partnership between the...

Documents

Transcript of Reasonable ROI Tracking Measuring Internal and External Impact Created in Partnership between the...