The Fiscal Policy Studies Institute Book - DVD Orders amazon.com resultsleadership.org...

The Fiscal Policy Studies Institutewww.raguide.org www.resultsaccountability.com

Book - DVD Ordersamazon.com

resultsleadership.org

Results-Based Accountability™ RBA

TRAINING FOR TRAINERS AND COACHES

DAY 1

RBA/OBA

Ends vs. MeansExercise

The Fiscal Policy Studies Institute

Santa Fe, New Mexico

Websitesraguide.org

resultsaccountability.com



INSTRUCTIONSOn a blank sheet of paper

Ends vs. Means Exercise

Step 2. Draw a line from the upper left point to the lower right point.

Step 3. Draw a line from the upper right point to the lower left point.

Step 4. Draw a horizontal line connecting the bottom two points.

Step 5. Mark another point approximately 1 inch below the middle of the horizontal line.Step 6. Connect the upper left point to this new point

Step 7. Connect the upper right point to this new point.

Like this.

Step 1. Draw 4 points that represent the 4 corners of a box or square, about two inches on a side.

What do you have?

A Star !!!

Question If I asked you to draw a star would you

need the step-by-step instructions?

Of course not !

So what’s the point?

And Start Seeing Stars !

Give people the ends and let them figure out

the means.

Bonus QuestionHow many stars were on

the previous page?

28How many guessed between 26 and 30?

RBA / OBAis about starting with ends

and working backward to means.

And the closing point….

So, for example,

Ask people to turn the curve on CHILD ABUSE RATES…..

Ask people to turn the curve on SCHOOL ATTENDANCE RATES…

Ask people to turn the curve on RATES OF AIR OR WATER POLLUTION….

AND LET THEM FIGURE OUT “HOW.”

THANKS !

’s

1. Why is this work important?

2. Population vs. Performance Accountability and Common Language

3. Population Accountability

4a. Performance Accountability

4b. Putting Population andPerformance Accountabilitytogether

Creating a Results Accountability Presentation

Simple, Common Sense, Plain Language, Minimum Paper, Useful (2a)

Pop vs. Perf Accountability (2b)Language Trap (3b)Definitions (4a)

Example of Results (7-9)(Leaking Roof (10)Results Acct. Talk to Action (11)One turn the curve story (15-16)

4 Quadrants – types of perf measures (27)

Perf. Acct. Talk to Action 7 Questions (28)

Relationship between Pop and Perf. Acct (30b). Every time you make a presentation (31b)

Training for Trainers and Coaches


TEACHING ROUND 1

WHY ARE WE HERE?WHY ARE YOU HERE?

WHY IS THIS WORK IMPORTANT?

ORWhat is the dissatisfaction driving people to consider new ways of working?

Why do you believe in this work enough to want to teach it to others?

TELL A STORYPages 2b, 32b

SIMPLE

COMMON SENSE

PLAIN LANGUAGE

MINIMUM PAPER

USEFUL

3 - kinds of performance measures. How much did we do? How well did we do it? Is anyone better off?

RBA in a Nutshell2 – 3 - 7

2 - kinds of accountability Population accountability Performance accountability

7 - questions from ends to means in less than an hour.

plus language discipline Results & Indicators Performance measures

Baselines & Turning the Curve


TEACHING ROUND 1

WHY ARE WE HERE?WHY ARE YOU HERE?

WHY IS THIS WORK IMPORTANT?

ORWhat is the dissatisfaction driving people to consider new ways of working?

Why do you believe in this work enough to want to teach it to others?

TELL A STORY

Fiscal Policy Studies Institute

Santa Fe, New Mexicowww.resultsaccountability.comwww.raguide.org


TEACHING ROUND2

Population vs. Performance Accountability

Common Language

Pages 2b, 3b, 4a

Results Accountabilityis made up of two parts:

Performance Accountabilityabout the well-being of

CLIENT POPULATIONSFor Programs – Agencies – and Service Systems

Population Accountabilityabout the well-being of

WHOLE POPULATIONSFor Communities – Cities – Counties – States - Nations

THE LANGUAGE TRAPToo many terms. Too few definitions. Too little discipline

Benchmark

Target

Indicator Goal

Result

Objective

Outcome

Measure

Modifiers Measurable Core Urgent Qualitative Priority Programmatic Targeted Performance Incremental Strategic Systemic

Lewis Carroll Center for Language DisordersMeasurable urgent systemic indicatorsCore qualitative strategic objectivesMake up your own jargon.

DEFINITIONS

Children born healthy, Children ready for school, Safe communities, Clean Environment, Prosperous Economy

Rate of low-birthweight babies, Percent ready at K entry, crime rate, air quality index, unemployment rate

1. How much did we do? 2. How well did we do it? 3. Is anyone better off?

RESULT or OUTCOME

INDICATOR or BENCHMARK

PERFORMANCE MEASURE

A condition of well-being for children, adults, families or communities.

A measure which helps quantify the achievement of a result.

A measure of how well a program, agency or service system is working. Three types:

Popu

latio

nPe

rform

ance

Children born healthy

Rate of low-birthweight babies Percent ready at K entry

Children ready for school

crime rate

Safe communities

air quality index

Clean Environment

unemployment rate

Prosperous Economy

INDICATOR

RESULT

PERFORMANCE MEASURE

Popu

latio

nPe

rform

ance

= Customer Results

1.

2.

3.

1. Safe Community

2. Crime Rate

3. Average Police Dept response time

4. Healthy People

5. Rate of drug-resistant tuberculosis

6. People have living wage jobs and income

7. % of people with living wage jobs and income

8. % of participants in job training who get living wage jobs

IS IT A RESULT, INDICATOR OR PERFORMANCE MEASURE?

RESULT

INDICATOR

PERF. MEASURE

RESULT

INDICATOR

RESULT

INDICATOR

PERF. MEASURE

Translation Guide/Rosetta StoneNot the Language Police

Ideas

1. A condition of well-being for children, adults, families & communities

2.

3.

etc.

Group 1 Group 2 Group 3 etc.

RESULT OUTCOME GOAL

TRANSLATION

Back to the Idea

3 - kinds of performance measures. How much did we do? How well did we do it? Is anyone better off?

RBA in a Nutshell2 – 3 - 7

2 - kinds of accountability Population accountability Performance accountability

7 - questions from ends to means in less than an hour.

plus language discipline Results & Indicators Performance measures

Baselines & Turning the Curve




TEACHING ROUND2

Population vs. Performance Accountability

Common Language

Pages 2b, 3b 4a




Lunch




SORTING EXERCISE




POPULATION ACCOUNTABILITY

TURN THE CURVE EXERCISE

Creating a Working Baselinefrom Group Knowledge

Now

Indicator or Performance Measure

75% Not OK?

BackcastingForecasting

Turn the Curve Exercise: Population Well-being

5 min: Starting Points - timekeeper and reporter - geographic area - two hats (yours plus partner’s)

10 min: Baseline - pick a result and a curve to turn - forecast (to 2016) – OK or not OK?

15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes

15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works

10 min: Report convert notes to one page

Two pointers to action

ONE PAGE Turn the Curve Report: Population

Result: _______________Indicator

(Lay Definition)IndicatorBaseline

Story behind the baseline --------------------------- --------------------------- (List as many as needed)

Partners --------------------------- --------------------------- (List as many as needed)

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost

SharpEdges

4. --------- Off the Wall


Turn the Curve Exercise – LessonsFrom talk to action in less than an hour

Look what you did! Talk to action in less than 1 hour!

• What did you learn? How was this different from other processes? What worked and what didn’t work?

• Why did we ask for:

a. Baseline with forecastb. Story before what worksc. No cost / low costd. Two hatse. Results before indicatorsf. Crazy idea

3. Do you think a lay audience could understand the reports?

4. How many think you could lead this exercise with a small group?

HOMEWORK:

1. Self Assessment (Page 53)

2. ReadingNext Generation ContractingRBA/OBA Facebook Selected EssaysCollective Impact Using RBApp 153-54 Appendix C other language

3. Hardest Question(s) "How do we….?"

4. Check out the RBA Facebook Discussion Group




TEACHING ROUND 3


Pages 2b, 10a, 11 or 12a plus 1 story

Community Outcomesfor Christchurch, NZ

● A Safe City

● A City of Inclusive and Diverse Communities

● A City of People who Value and Protect the Natural Environment

● A Well-Governed City

● A Prosperous City

● A Healthy City

● A City for Recreation, Fun and Creativity

● A City of Lifelong Learning

● An Attractive and Well-Designed City

Example ofresults for

all residents

Georgia Policy Councilfor Children and Families

RESULTS

● Healthy Children

● Children Ready for School

● Children Succeeding in School

● Strong Families

● Self Sufficient Families

1. All CT residents live in safe families and communities

2. All CT residents are economically secure

3. All CT residents are developmentally, physically, and mentally healthy across the life span

4. All CT residents who are elderly or have disabilities live engaged lives in supportive environments of their choosing

5. All CT residents succeed in education and are prepared for careers, citizenship, and life

6. All CT children grow up in a stable environment, safe, healthy, & ready to succeed.

7. All CT residents have a safe and efficient transportation system.

8. All CT residents have a clean and healthy environment.

9. All CT residents have a fair ethical and efficient government

10. All CT residents and visitors enjoy the state's rich artistic and cultural life.

Connecticut Population Results(Composite Draft)

VERMONTPopulation Outcomes (S.293)

1. Vermont has a prosperous economy.2. Vermonters are healthy.3. Vermont’s environment is clean and sustainable.4. Vermont’s communities are safe and supportive.5. Vermont’s families are safe, nurturing, stable, and supported.6. Vermont’s children and young people achieve their potential.7. Vermont’s elders and people with disabilities and people with mental conditions live with dignity and independence in settings they prefer.8. Vermont has open, effective, and inclusive government at the State and local levels.

OttawaPopulation Results

1. Ottawa has a prosperous economy.2. Ottawa residents are physically and mentally healthy.3. Ottawa’s environment is clean and sustainable.4. Ottawa’s communities are safe and welcoming.5. Ottawa’s families are safe, stable, and supportive.6. Ottawa’s children and young people achieve their potential.7. Ottawa’s elders and people with disabilities and live with dignity and independence in settings they prefer.8. Ottawa has open, effective, and inclusive government.

DRAFT

CARDIFF, WALESCommunity Outcomes

1. People in Cardiff are healthy

2. Cardiff has a clean attractive and sustainable environment

3. People in Cardiff are safe and feel safe

4. Cardiff has a thriving and prosperous economy

5. People in Cardiff achieve their full potential.

6. Cardiff is a great place to live, work and play.

7. Cardiff has a fair, just and inclusive society.

New Zealand Coventry, UK

Santa Cruz, CA Halton, Ontario, CA

Leaking Roof(Results thinking in everyday life)

Experience

Measure

Story behind the baseline (causes)

Partners

What Works

Action Plan

Inches of WaterBASELINE

? Fixed

Not OK

Turning the Curve

Action Plan # 2

Results / Outcomes

Indicators

Baseline & Story

1975

1980

1982

1990

2000

2005

0

5

10

15

20

25

30

Tho

usan

ds

Source 1982 to 2005: Actual data from the NHTSA Fatality Analysis Reporting System (FARS)Source 1975 to 1981: Estimate based on NHTSA data on % of fatality drivers with BAC of .10 or greater.

Alcohol-Related Traffic FatalitiesUnited States 1975 to 2005

45 people per day

75 people per day

2010

28 peopleper day

MADD

Rebound

no. of Tagged Sites visited

02000400060008000

1000012000140001600018000

2001

/02

2002

/03

2003

/04

2004

/05

2005

/06

2006

/07

2007

/08

2008

/09

2009

/10

year

no

. of

site

s

2001/02

2002/03

2003/04

2004/05

2005/06

2006/07

2007/08

2008/09

2009/10

Christchurch, New ZealandNumber of Graffiti Sites

FY 2002 to FY 2010

Hull, UK"Woundings"




TEACHING ROUND 3


Pages 2b, 10a, 11 or 12a plus 1 story

HOMEWORK:

1. Self Assessment ()

2. ReadingNext Generation ContractingRBA/OBA Facebook Selected EssaysCollective Impact Using RBApp 153-54 Appendix C other language

3. Hardest Question(s) "How do we….?"

4. Check out the RBA Facebook Discussion Group

THANKS !



Results Accountability

Fiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.orgwww.trafford.com


DAY 2

TOUGHEST QUESTIONS

Source: Center for Systemic Peace and Integrated Network for Social Conflict Research

The Simplest WayTO IMPLEMENT RBA

POPULATION ACCOUNTABILITY 1. Pick an important indicator curve 2. Run the Population Turn the Curve Exercise 3. Take action Repeat

PERFORMANCE ACCOUNTABILITY 1. Pick an important performance measure curve 2. Run the Performance Turn the Curve Exercise 3. Take action Repeat

Select 3 to 5 Performance MeasuresACROSS THE BOTTOM OF THE ORG CHART

3 - 5 3 - 5 3 - 5 3 - 5 3 - 5 3 - 53 - 5

3 - 5 3 - 5 3 - 5

3 - 5

20 – 60 – 20 Rule

The

20-60-20 RULE

RESULTS BASED BUDGETING

Do we need it?

Is it working?


PERFORMANCE ACCOUNTABILITY

1. Ask each program to assign someone to become the in-house RBA expert. (Treat this like a technical support function like an in-house computer expert.)

2. Have these people practice and get good at selecting performance measures and running the Turn the Curve Exercises.

3. Create a network of in-house experts so they can support each other, learn how to do joint RBA training and coaching.

4. Ask the in-house experts to review the RBA Self- Assessment Questionnaire (and create a score if useful) on a regular basis.

RBA expert in each unit

Alternative to Traditional Evaluation Methods:DEMONSTRATING a CONTRIBUTION

to complex change efforts… requires 3 elements:

A Curve to Turn

1 We tried a bunch of stuff that had a credible

chance of making a difference …

2 …and it had a timely relationship to….

3 …. a turn in the curve.

c FPSI

Next Generation ContractingContract Provisions

Provision 1. Specify the 3 to 5 most important performance measures (from the How well did we do it? and Is anyone better off? categories).

Provision 2. Specify that the contractor will use a continuous improvement process (the RBA 7 Questions).

Provision 3. Specify how the funder and contractor will work in partnership to maximize LR customer results (quarterly meetings using the 7 questions as the agenda).

Provision 4. Specify that the funder will work with the funding community to simplify and standardize contracting and performance reporting.

Provision 5: : Clear articulation of role in population/community well-being using the language of contribution not attribution.

Provision 6: 10% for quality management and administration.

Provision 7: Multi-year funding using 3 year rolling contracts

Provision 8: Use of targets that are fair and useful.

Provision 9: Fund flexibility and virtual funding pool: transfer of up to 10% across line items and program lines.

Provision 10: Request for Results: Getting past the sometimes negative effects of competitive RFP contracting or tendering.

Next Generation ContractingContract Provisions

Performance Accountability

For Programs, Agencies and Service Systems

Fiscal Policy Studies InstituteSanta Fe, New Mexicowww.resultsaccountability.comwww.raguide.org

“All performance measures

that have ever existed

for any program

in the history of the universe

involve answering two sets of

interlocking questions.”

HowMuchdid we do?

( # )

HowWell

did we do it?

( % )

Quantity Quality

Program Performance Measures

EffortHow hard did we try?

EffectIs anyone better off?


Effort

Effect

HowMuch

HowWell


How much service did we deliver?


How welldid we

deliver it?

How much change / effect

did we produce?

What quality of change / effect

did we produce?

Quantity Quality

Effe

ct

Effo

rt

O

utpu

t

In

put

How much did we do?


How welldid we do it?

Is anyonebetter off?

Quantity Quality

Effe

ct

Effo

rt

# %

How much did we do?

Education

How well did we do it?

Is anyone better off?

Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Number ofhigh schoolgraduates

(completionor certificate rate)

Percent ofhigh schoolgraduates

(completionor certificate rate)

How much did we do?

Education



Quantity Quality

Effe

ct

E

ffort Number of

studentsStudent-teacher

ratio

Percent of 9th graders whoenter college or

employment after graduation

Number of 9th graders whoenter college or

employment after graduation

How much did we do?

Pediatric Practice



Number ofchildrentreated

Percent ofpatients waiting

less than30 min in thewaiting room

Quantity Quality

Effe

ct

E

ffort

#children

fullyimmunized

(in the practice)

%children

fullyimmunized

(in the practice)

How much did we do?

Drug/Alcohol Treatment Program



Number ofpersonstreated

Percent ofstaff withtraining/

certification

Number of clientsoff of alcohol & drugs - at exit - 12 months after exit

Percent of clientsoff of alcohol & drugs - at exit - 12 months after exit

Quantity Quality

Effe

ct

E

ffort

How much did we do?

Fire Department



Number ofresponses

ResponseTime

Quantity Quality

Effe

ct

E

ffort

# of fireskept to

room of origin

% of fireskept to

room of origin

How much did we do?

General Motors



# of production hrs

# tons of steel

Employees pervehicle

produced

# of cars sold

$ Amount of Profit

$ Car value after 2 years

Quantity Quality

Effe

ct

E

ffort

Source: USA Today 9/28/98

% Market share

Profit per share

% Car value after 2 years

How much did we do?

Not All Performance Measures Are Created Equal



LeastImportant

Quantity Quality

Effe

ct

E

ffort

MostImportant

Least

Most

AlsoVery Important

Quality

How much did we do?

The Matter of Control



Quantity Quality

Effe

ct

E

ffort

LeastControl

PARTNERSHIPS

MostControl

How much did we do?

Performance AccountabilityTypes of Measures found in each Quadrant



# Clients/customers served

# Activities (by type of activity)

% Common measurese.g. client staff ratio, workload ratio, staffturnover rate, staff morale, % staff fully trained, % clients seen in their own language,worker safety, unit cost

% Skills / Knowledge (e.g. parenting skills)

#

% Attitude / Opinion (e.g. toward drugs)

#

% Behavior (e.g.school attendance)

#

% Circumstance (e.g. working, in stable housing)

#

% Activity-specific measures

e.g. % timely, % clients completing activity, % correct and complete, % meeting standard

Point in Timevs.

2 Point Comparison

# %

LR

UR

Primary v. Secondary

Direct v. Indirect

Internal v. External

Baseline & Story

Watermain Breaks per yearLondon Ontario

How much did we do?

Service: __________________________________



Primary customers# studentspatientspersons trained

Primary activity# hours of

instructiondiagnostic testsjob coursesalarms responded to

If your service works really well,how are your customer's better off?

# students who graduatepatients who fully recoverfires kept to room of origin

%

Unit cost

Workload ratio

% of ___x___ that happen on time

persons who get jobs

School Hospital Job Training Fire Department

customers served




SELECTING PERFORMANCE MEASURES

DEMONSTRATION & EXERCISE

How much did we do?

Choosing Headline Measures and the Data Development Agenda



Quantity Quality

E

ffect

Effo

rt

# Measure 1 ----------------------------

# Measure 2 ----------------------------

# Measure 3 ----------------------------

# Measure 4 ----------------------------

# Measure 5 ----------------------------

# Measure 6 ----------------------------

# Measure 7 ----------------------------

#1 Headline

#2 Headline

#3 Headline

#1 DDA

#2 DDA

#3 DDA% Measure 8 ----------------------------

% Measure 9 -----------------------------

% Measure 10 ---------------------------

% Measure 11 ---------------------------

% Measure 12 ---------------------------

% Measure 13 ---------------------------

% Measure 14 ---------------------------

# Measure 15 ----------------------------

# Measure 16 ----------------------------

# Measure 17 ----------------------------

# Measure 18 ----------------------------

# Measure 19 ----------------------------

# Measure 20 ----------------------------

# Measure 21 ----------------------------

% Measure 15 ----------------------------

% Measure 16 ----------------------------

% Measure 17 ----------------------------

% Measure 18 ----------------------------

% Measure 19 ----------------------------

% Measure 20 ----------------------------

% Measure 21 ----------------------------

How to Measure:- Capacity Building

- Vibrant Neighborhoods- Developed Countries

etc.

See pages 128- 129

Performance

measure

PopulationIndicator




USING PERFORMANCE MEASURES

Turn the Curve Exercise

Creating a Working Baselinefrom Group Knowledge

Now

Indicator or Performance Measure

75% Not OK?

BackcastingForecasting

Turn the Curve Exercise: Program Performance

5 min: Starting Points - timekeeper and reporter - identify a program to work on - two hats (yours plus partner’s)

10 min: Performance measure baseline - choose 1 measure to work on – from the lower right quadrant - forecast (to 2016) – OK or not OK?

15 min: Story behind the baseline - causes/forces at work - information & research agenda part 1 - causes

15 min: What works? (What would it take?) - what could work to do better? - each partners contribution - no-cost / low-cost ideas - information & research agenda part 2 – what works

10 min: Report convert notes to one page

Two pointers to action

Program: _______________Performance Measure

(Lay definition)PerformanceMeasureBaseline

Story behind the baseline --------------------------- --------------------------- (List as many as needed)

Partners --------------------------- --------------------------- (List as many as needed)

Three Best Ideas – What Works 1. --------------------------- 2. --------------------------- 3. ---------No-cost / low-cost

ONE PAGE Turn the Curve Report: Performance

SharpEdges



Turn the Curve Exercise – LessonsFrom talk to action in less than an hour

Look what you did! Talk to action in less than 1 hour!

• What did you learn? How was this different from other processes? What worked and what didn’t work?

• Why did we ask for:

a. Baseline with forecastb. Story before what worksc. No cost / low costd. Two hatse. Results before indicatorsf. Crazy idea

3. Do you think a lay audience could understand the reports?

4. How many think you could lead this exercise with a small group?


www.resultsaccountability.comwww.raguide.org


TEACHING ROUND 4


CONNECTING POPULATION & PERFORMANCE ACCOUNTABILITY

How much did we do?








#


#


#


#



Point in Timevs.

2 Point Comparison

LR

UR

Primary v. Secondary

Direct v. Indirect

Internal v. External

Baseline & Story

Contributionrelationship

Alignmentof measures

Appropriateresponsibility

THE LINKAGE Between POPULATION and PERFORMANCE


Healthy Births Rate of low birth-weight babiesChildren Ready for School Percent fully ready per K-entry assessmentSelf-sufficient Families Percent of parents earning a living wage

CUSTOMERRESULTS

# personsreceivingtraining

Unit costper person

trained

# who getliving wage jobs

% who getliving wage jobs


POPULATIONRESULTS

Job Training Program

How much did we do?








#


#


#


#



Point in Timevs.

2 Point Comparison

Next Steps:

1. What’s 1 thing I could personally do with what I learned these two days?

2. What’s 1 thing I would like to ask someone else or some organization to do to support this work?




NAME-MOTTO-MASCOT

LAST SHOT AT HARD QUESTIONS

NEXT STEPS

EVALUATION & CERTIFICATES

Resources



www.raguide.orgwww.resultsaccountability.com

RBA Facebook Group

THANK YOU !



The Fiscal Policy Studies Institute Book - DVD Orders amazon.com resultsleadership.org...

Documents

Transcript of The Fiscal Policy Studies Institute Book - DVD Orders amazon.com resultsleadership.org...