Making Performance Management Happen...Making the Cycle Work in Real Life •Steps may have to...
32
Making Performance Management Happen Isaac Castillo, Senior Research Scientist, Child Trends Leah Galvin, Technical Assistance and Implementation Project Manager, OMNI Institute Audio Broadcast or Toll Free Number 877-668-4490 | Event ID 714 666 502
Transcript of Making Performance Management Happen...Making the Cycle Work in Real Life •Steps may have to...
Making Performance
Management Happen
Isaac Castillo, Senior Research Scientist, Child Trends
Leah Galvin, Technical Assistance and Implementation
Project Manager, OMNI Institute
Audio Broadcast or Toll Free Number 877-668-4490 | Event ID 714 666 502
Asking Questions Using Chat
Icon: Chat Bubble
Speakers
Isaac Castillo
Senior Research Scientist
Child Trends
Leah Galvin
Technical Assistance and
Implementation Project Manager
OMNI Institute
Review of Prior Webinars
• Using What You Measure: An Introduction to
Performance Management on Aug. 14, 2012
• Taking the Next Step toward Performance
Management - How PerformWell Can Help on Sep.
20, 2012
• Creating a Performance Culture on Oct. 11, 2012
• Full webinar materials and recordings available at:
– http://www.performwell.org/index.php/webinars
Formal Definition of Performance Management
“The systematic process by which an agency
involves its employees, as individuals and
members of a group, in the accomplishment of
agency mission and goals.”
The US Office of Personnel Management
PerformWell Performance Management Definition
• Performance management is a dynamic
process that is designed to better understand
program operations, monitor outcomes, and
ultimately, help nonprofits become high
performing organizations which produce positive
outcomes.
• It involves regular, ongoing performance
measurement, reporting, analysis, and program
modification.
Performance Management Requires a Cultural
Change
• Literally need to change how the organization
operates.
• Reconceptualize how information can be
used and who should be using it.
• Information as a valuable and necessary
resource.
What Needs to Be in Place First
• Willingness to shift culture to become
learning organization
• Clear alignment between services and
outcomes
• Willingness to devote resources and time to
develop performance management capacity
Thinking of Performance Management
• Logic models, theories of change, and other
conceptualizations are often linear:
• However, performance management as a
process is better thought of as a circle, or as
a cycle
Inputs Activities or
Services Outputs Outcomes
Define Success for Program Participants
Identify / Modify / Develop Data
Collection Methods
Train Staff on Data Collection
Methods
Begin Program / Service and Data
Collection
Ongoing Quality Assurance
Checks
Provide Interim Reports to Staff
Complete Data Collection (and Data Cleaning)
Analyze Data
Prepare Reports (Internal and
External)
Share Outcomes and Reports with
Staff
Use Outcomes and Reports to Improve /
Change Program
Performance
Management
Cycle
Define Success for Participants
• Work with program or organizational staff to
define what success looks like for participants
• Be as specific as possible
– “Get better” is not good enough
– Encourage staff to share examples of what
success looks like
• These become your outcomes for the
program
• Try to limit to 3-4 outcomes to begin
Define Success for Program Participants
Identify Data Collection Methods
• Determine how you will actually measure progress towards your outcomes
• Could use variety or combination of methods
– Surveys or pre/post tests
– Focus groups or interviews
– Case notes
– Data collected by external sources
– Assessments
• Methods must provide you with ability to get data during an appropriate timeframe
Identify / Modify / Develop Data
Collection Methods
Train Staff on Data Collection Methods
• Detailed trainings on what staff need to do to
collect data
• Should include guidance on data entry
• Refresher trainings need to occur regularly
• Should also explain the importance of the
data collection process
Train Staff on Data Collection
Methods
Define Success for Program Participants
Identify / Modify / Develop Data
Collection Methods
Train Staff on Data Collection
Methods
Begin Program / Service and Data
Collection
Ongoing Quality Assurance
Checks
Provide Interim Reports to Staff
Complete Data Collection (and Data Cleaning)
Analyze Data
Prepare Reports (Internal and
External)
Share Outcomes and Reports with
Staff
Use Outcomes and Reports to Improve /
Change Program
Performance
Management
Cycle
Begin Program and Data Collection
• Complete intake / entry data collection
process
• Use first few weeks to quickly obtain missing
data from participants
• Look for quick fixes that can be implemented
fast to make process easier
Begin Program / Service and Data
Collection
Ongoing Quality Assurance Checks
• Build in time to conduct quick data quality
checks on regular basis
• Work to identify areas where data is missing,
incomplete, or inappropriate
• Important to think of checks as non-
judgmental – everyone make mistakes
Ongoing Quality Assurance
Checks
Create Interim Reports for Staff
• Generate interim reports for staff while the program is running
– Demographics
– Attendance
– Baseline data
– Short term outcomes / indicators
• Ask program staff questions using the reports
– Is this what you expected?
– Does anything here look too good to be true?
• Another form of quality control
Provide Interim Reports to Staff
Complete Data Collection
• Complete all data collection for as many
participants as possible
• Conduct final quality assurance checks
• Data cleaning
Complete Data Collection (and Data Cleaning)
Define Success for Program Participants
Identify / Modify / Develop Data
Collection Methods
Train Staff on Data Collection
Methods
Begin Program / Service and Data
Collection
Ongoing Quality Assurance
Checks
Provide Interim Reports to Staff
Complete Data Collection (and Data Cleaning)
Analyze Data
Prepare Reports (Internal and
External)
Share Outcomes and Reports with
Staff
Use Outcomes and Reports to Improve /
Change Program
Performance
Management
Cycle
Analyze Data
• Take time to properly analyze data
• Think about how demographic and
attendance data may affect outcomes
• Ask for assistance / guidance if you need it
• Consider (and document) limitations of your
data and your analysis
Analyze Data
Prepare Reports
• Prepare reports in useable formats
• Consider different types of reports for
different audiences
• Balance comprehensiveness, speed, and
types of findings to be shared
Prepare Reports (Internal and
External)
Define Success for Program Participants
Identify / Modify / Develop Data
Collection Methods
Train Staff on Data Collection
Methods
Begin Program / Service and Data
Collection
Ongoing Quality Assurance
Checks
Provide Interim Reports to Staff
Complete Data Collection (and Data Cleaning)
Analyze Data
Prepare Reports (Internal and
External)
Share Outcomes and Reports with
Staff
Use Outcomes and Reports to Improve /
Change Program
Performance
Management
Cycle
Share Outcomes with Staff
• Once you prepare the reports – remember to
share them with staff!
• Use the reports to start a discussion
– Do they have theories as to why some outcomes
happened (or didn’t happen)?
– Ask them to look at the reports and identify any
areas that seem interesting or troubling
• Should share reports with all levels of
program and organization
Share Outcomes and Reports with
Staff
Using Outcomes to Improve
Programs
• Work with staff to identify programmatic
changes that can be made to improve
outcomes
• Changes could be small or drastic – will
depend on the outcomes observed
• Also ask questions about the data collection
methods – they could be the reason for
outcomes as much as the programmatic
approach
Use Outcomes and Reports to Improve /
Change Program
Start the Process Again
• You have changed your program, does that
mean that your definition of success has
changed?
• Process never really ends – as you continue
to collect and use data, you change
programming, which changes data collection
Define Success for Program Participants
Identify / Modify / Develop Data
Collection Methods
Train Staff on Data Collection
Methods
Begin Program / Service and Data
Collection
Ongoing Quality Assurance
Checks
Provide Interim Reports to Staff
Complete Data Collection (and Data Cleaning)
Analyze Data
Prepare Reports (Internal and
External)
Share Outcomes and Reports with
Staff
Use Outcomes and Reports to Improve /
Change Program
Performance
Management
Cycle
Making the Cycle Work in Real Life
• Steps may have to happen out of order – but
important to make sure all steps are covered
• Start small – better to track one thing very
well than a dozen things poorly
• Explain what you are doing to staff –
important to get their buy-in from the start
• Recognize the additional burden you are
placing on staff – and do something about it
Contact Information
Isaac Castillo
E-mail:
Twitter: isaac_outcomes
www.childtrends.org
Contact PerformWell
Leah Galvin
E-mail:
Twitter: leahagalvin
www.omni.org
LIVES ON THE LINE: WHY AND HOW
MISSION-DRIVEN LEADERS ARE
EMBRACING PERFORMANCE MANAGEMENT
March 7, 2013 3:00-4:00pm EST
David Hunter, Ph.D., Principal, Hunter Consulting & author
of Working Hard – And Working Well
Mario Morino, Chairman, Venture Philanthropy Partners &
author of Leap of Reason
Sam Cobbs, CEO, First Place for Youth
Register at www.performwell.org
FUNDRAISING WITH PERFORMANCE
DATA
April 11, 2013 3:00-4:00pm EST
Isaac Castillo will be joined by Bridget Laird, CEO, Wings
for Kids
Register at www.performwell.org
Questions?
Helping practitioners deliver
more effective programs
Speakers
Isaac Castillo
Senior Research Scientist
Child Trends
Leah Galvin
Technical Assistance and
Implementation Project Manager
OMNI Institute
