Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State...
-
Upload
griselda-carter -
Category
Documents
-
view
224 -
download
3
Transcript of Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State...
![Page 1: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/1.jpg)
Performance Measurement: The nuts and bolts of
establishing effective measures
Mary Davis, DrPH, MSPH; NC Institute for Public Health
Kristin Adams, Ph.D., CHES; Indiana State Department of Health
![Page 2: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/2.jpg)
Performance Measurement Definitions
Performance measurement is the “regular collection and reporting of data to track work produced and results achieved”
Performance measure is “the specific quantitative representation of capacity, process, or outcome deemed relevant to the assessment of performance”
Performance measurement is NOT punishment
![Page 3: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/3.jpg)
Organizational Performance
Infrastructure Process Coordination Results
Leadership & Decision making
Boards of Health Policies &
Procedures Enforcement/
Investigation Protocols
Financial management
Information & reporting
Set of Core Indicators
Program performance goals/ objectives
Assessment processes
CD investigation case write-ups
EH enforcement action and case files
Education sessions
Work of community groups and coalitions
Communication mechanisms
Public and private work on access to services
Hand-offs between local health and state programs
Program evaluation results
Key indicator outcomes (CD/EH/PP)
Financial performance
Monitoring the results of performance review against PHAB Standards and Program Evaluation
Monitoring Indicators and Outcomes
![Page 4: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/4.jpg)
Mary Davis, UNCDr. Mary Davis is a Senior Investigator at the NC Institute for Public Health and Adjunct Assistant Professor in the Department of Health Behavior, both part of the UNC, Gillings School of Global Public Health. Since Dr. Davis conducts public health systems and services research and program evaluations using participatory approaches; specialty areas are accreditation, quality improvement, and public health partnerships.
Her experiences in these areas have been shaped by participating in the Multi-State Learning Collaborative, evaluation of the NC Local Health Department Accreditation program, and working with the NNPHI and CDC to evaluation the National Public Health Improvement Initiative. Dr. Davis earned her doctorate from the Johns Hopkins University School of Public Health and her masters in science in public health from the University of Alabama at Birmingham.
![Page 5: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/5.jpg)
Dr. Kristin Adams is the Director of the Office of Public Health Performance Management at the Indiana State Department of Health (ISDH). She oversees agency performance management, quality improvement, and future public health accreditation efforts. She was the Principal Investigator in Indiana for the Multi-State Learning Collaborative grant. She is currently the Principal Investigator for the National Public Health Improvement Initiative cooperative agreement from CDC.
Prior to joining ISDH, Dr. Adams worked for a national non-profit agency focusing on program evaluation and training for health programs targeted toward girls and young women. Dr. Adams has taught numerous health education courses and has presented at national and international conferences. Dr. Adams holds a BS and MA in School Health Education from Indiana State University, a Ph.D. in Health Education from Southern Illinois University Carbondale, and is a Certified Health Education Specialist.
Kristin Adams, ISDH
![Page 6: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/6.jpg)
Performance Measurement: Nuts and Bolts
Basics
Mary V. Davis, DrPH, MSPHOpen Forum for Quality Improvement
November 20, 2013
![Page 7: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/7.jpg)
Outline
• What is a performance measure?• Where are performance measures in a
performance management system?• What should be in place before establishing
measures?• What’s a system of work?• What are the characteristics of a useful process
measure?• How do you go about establishing useful
process measures?
![Page 8: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/8.jpg)
What is a Performance Measure?
• Performance Measurement is regular collection and reporting of data to track work produced and results achieved.
• Performance Measure is quantitative representation of – Capacity– Process– Outcome
![Page 9: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/9.jpg)
Where are Performance Measures in a Performance Management System?
YOU ARE HERE!
![Page 10: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/10.jpg)
Standards and Measures: PHAB Example
• Standard 9.1: Use a performance management system to monitor achievement of organizational objectives.– Measure 9.1.2 A Implement a performance
management system.– Use a process to determine and report on
achievement of goals, objectives, and measures set by the performance management system.
![Page 11: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/11.jpg)
What Should be in Place Before Measures?
![Page 12: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/12.jpg)
Quality Improvement Projectsand
Process ImprovementOpportunities
Resu
lts and
Cu
stom
er, Stak
eho
lder, P
artner F
eedb
ack
Vis
ion
Mis
sion
and
Val
ues
Goa
ls a
nd O
bjec
tives
Str
ateg
ies
Str
ateg
ic P
lan
nin
g
Perform
ance Measures
and Operational Indicators O
peratio
nal/
Bu
siness P
lans
Performance Monitoring and Analysis
Performance Monitoring and Analysis
Improvem
entO
pportunitiesIm
p ro vement
Op po rtun it i es
Washington State Department of Health Example:
Results help to inform the planning process
![Page 13: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/13.jpg)
Line of Sight Framework
Restaurant inspections conducted and reports produced
restaurants achieving acceptable sanitation grades can post grades
Public uses restaurants with acceptable sanitation grades
Reduction in food-borne illnesses and disease outbreaks
Work Process Measure
Process Outcome
Short term Outcome
Long term outcome
![Page 14: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/14.jpg)
What is a System of Work?
Factory: How?
Widget: What?
Customers: Who?
Outcomes: Why?
![Page 15: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/15.jpg)
Focus on the Widgets
• What’s a Widget?
![Page 16: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/16.jpg)
Widgets
• Are Things• Are Deliverables• Can be Counted• Are Specific
![Page 17: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/17.jpg)
Common Public Health Agency Widgets
• Environmental Permits, Notices of Violations• Rules and regulations• Vaccinations• Epidemiology Reports• Infrastructure
– Filled Vacancies– Contracts
![Page 18: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/18.jpg)
What Widgets Do You Produce?
• Take a minute: write down 5 widgets you produce
• What examples do you have to share?
![Page 19: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/19.jpg)
Measures: General Considerations
• Measures reveal our values• Measures drive behavior• Measures can inspire us• Measures can help us learn
![Page 20: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/20.jpg)
Useful Measures: Specifics
• Meaningful• Focused on customer
needs• Accurate• Valid• Reliable• Responsive• Functional
• Credible• Simple enough to be
understood• Available• Abuse-Proof• Cost effective to
collect and report• Comparable data
over time
![Page 21: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/21.jpg)
SMART Measures
• Specific• Measurable• Aggressive• Results-Oriented• Time-bound
![Page 22: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/22.jpg)
Developing Measures
• Talk with customers and those involved in the process
• Identify which process measures– Are of value to customers and producers– Will drive behavior– Will inspire– Will promote learning
• Determine how process measures fit into other measurement processes in the agency (performance standards, strategic plan)
![Page 23: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/23.jpg)
What to Include in a Measure
• Statement of the Measure• Target (population, process)• Numerator and denominator• Target goal• Who will collect the information
![Page 24: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/24.jpg)
What to Include in a Measure
• Form or tool to collect information• How often data will be collected• Who will conduct data analysis• How often will the data be reported
![Page 25: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/25.jpg)
Data Description FormPerformance measure:
Target population:
Numerator:
Denominator:
Which are you using—a target or benchmark?
What is the target/benchmark?
SMART objective:
Source of data:
Who will collect the information?
How often will the data be analyzed?
How often , by what mechanism and who reports these data and analysis?
Baseline measurement data and date(s):
Definitions, such as accuracy and validity, and other comments:
MarMason Consulting
![Page 26: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/26.jpg)
Resources
• “We Don’t Make Widgets” Ken Miller• Turning Point Resources: Guidebook for
Performance Measurement• Marni Mason tools and presentations• Washington State Department of Health QI
Plan
![Page 28: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/28.jpg)
Getting to the 500 foot level of Performance
MeasuresKristin A. Adams, Ph.D., CHES
![Page 29: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/29.jpg)
![Page 30: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/30.jpg)
Ask a series of questions:◦ Who is your customer◦ What do they care about◦ What is the purpose of your
funding/mission/vision of the program area◦ Does this goal and objective tie to the strategic
plan Does your program have a role in the key priorities of
the agency◦ Can you measure it
Is it meaningful???
Setting Goals/Objectives
![Page 31: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/31.jpg)
![Page 32: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/32.jpg)
Several areas of resources to determine measures:◦ State law/code◦ Program requirements ◦ Healthy People 2020◦ Historical data of the program◦ Benchmarks by national groups (i.e., March of
Dimes)
What should we measure?
![Page 33: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/33.jpg)
Develop the key outcome measure◦ Decrease Smoking Rates◦ Decrease Obesity Rates◦ Increase Immunization Rates◦ Decrease the late payment fees
Identify key activities to get you to making a change in the measure◦ Increase number of people receiving coaching from quitline◦ Increase the complete streets communities throughout the
state ◦ Reminder recalls--% of those who become compliant due to
the reminder notice◦ Decrease invoice wait time; increase invoice processing
time
How Do We Get There
![Page 34: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/34.jpg)
Development of a Dashboard
Sometimes it is all about Process
![Page 35: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/35.jpg)
![Page 36: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/36.jpg)
![Page 37: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/37.jpg)
![Page 38: Mary Davis, DrPH, MSPH; NC Institute for Public Health Kristin Adams, Ph.D., CHES; Indiana State Department of Health.](https://reader036.fdocuments.net/reader036/viewer/2022062421/56649cc55503460f9498fa38/html5/thumbnails/38.jpg)
Ensure your epidemiologists are involved Ensure the experts to the program are
involved If you are doing a “data pull” ensure the
IT/database manager is involved Involve Executive Leadership Do a “mass” education campaign
Lessons Learned