Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.
-
Upload
devon-mowles -
Category
Documents
-
view
214 -
download
0
Transcript of Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.
![Page 1: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/1.jpg)
Using Audit and Feedback to Improve Quality of Care
December 7-8, 2012
![Page 2: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/2.jpg)
What are we talking about here?
A definition of Audit and Feedback:
summary of clinical performance over a specified period of time and provision to health care provider(s)/organization(s)
NOT: reminders at the point of careNOT: relay of individual, patient-specific clinical data between providers
![Page 3: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/3.jpg)
Why Talk about Feedback?
“the limiting factor to Self-Directed Learning for practicing physicians may be their limited ability to accurately self-assess”
Davis et al. JAMA 2006 Sep 6;296(9):1094-1102.
“You can’t manage what you can’t measure”
![Page 4: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/4.jpg)
Feedback and Behaviour Change
Old behaviours
NewbehavioursPerformance
FeedbackPerformance
Feedback
![Page 5: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/5.jpg)
![Page 6: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/6.jpg)
ObjectiveTo develop a research agenda to improve the effectiveness of audit and feedback interventions to reliably improve quality of care.
Goals:1. Establish the foundation for a program of research aimed at improving
the effectiveness of AF interventions1a) prioritize elements of AF design that should be tested1b) identify settings that provide opportunities for testing AF design1c) utilize methodology to efficiently test and improve upon AF interventions
2. Develop a knowledge translation plan:2a) integrated collaboration with clinical, administrative, policy stakeholders2b) disseminating the results of the meeting 2c) developing a wiki-based platform to support ongoing collaboration 2d) establishing plans for new operating grants that will carry out the agenda developed in the meeting
INTRODUCTIONS
![Page 7: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/7.jpg)
Agenda
![Page 8: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/8.jpg)
Results from the Cochrane Review
• What do we know about the impact of AF?– Cochrane Review and Meta-regression– Cumulative Meta-analysis (not yet published)
![Page 9: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/9.jpg)
Findings of 2006 Cochrane Review
• 88 comparisons from 72 studies• dichotomous outcomes median adjusted risk difference of
compliance with desired practice = 0.05 (IQR 0.03 to 0.11)
• “Intensity of audit and feedback might also help to explain variation in the absolute effect (p = 0.04).”
• “Intensive”(individual recipients) AND ((verbal format)OR (a supervisor or senior colleague as the source)) AND (moderate or prolonged feedback)• “Non-intensive” ((group feedback) NOT (from a supervisor or senior colleague)) OR ((individual feedback) AND (writ-ten format) AND (containing information about costs or numbers of tests without personal incentives))• “Moderately intensive”(any other combination of characteristics than described in Intensive or Non-intensive group).
Jamtvedt G et al. Cochrane Database of Systematic Reviews 2006, Issue 2. Art. No.: CD000259. DOI: 10.1002/14651858.CD000259.pub2.
![Page 10: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/10.jpg)
Unclear how to “do” AF
“an unreliable approach to quality improvement until we learn how and when it works best”
Foy R. et al. BMC Health Services Research, 2005;5, 50.
![Page 11: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/11.jpg)
2012 Update: Changes in Review Methodology
ANALYSIS OF HETEROGENEITY
Meta-regression• Format (verbal; written; both; unclear)• Source (supervisor or senior colleague; review organization or employer;
investigators; unclear)• Frequency (weekly; monthly; less than monthly; one-time)• Instruction for improvement (explicit goal; action plan; both; neither)• Direction of change required (increase behaviour; decrease; mix/unclear)• Recipient (physician; other health professional)• Baseline performance (continuous)• Risk of bias (high; unclear; low)
Plus exploratory analyses:• Targeted behaviour (prescribing; test ordering; dm/cvd)
![Page 12: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/12.jpg)
Characteristic N % Characteristic N %Publication Year Classification of Intervention 2006-2010 32 23 AF alone 49 35 1996-2005 76 54 Multifaceted 91 65 1986-1995 20 14 case mgmt/team change 3 2 before 1986 12 9 clinician education (not outreach) 48 34Risk of Bias educational outreach 28 20 Low 45 32 clinician reminders, cdss 17 12 Unclear 70 50 patient intervention 8 6 High 25 18 continuous qi 9 6Number of Arms in Trial financial incentives 5 4 Two 98 70 Three 22 16 Targeted Behaviour(s) Four 20 14 DM/CVD mgmt 30 21Clinical Setting Laboratory testing/radiology 21 15 Outpatient 94 67 Prescribing 31 22 Inpatient 36 26 Other 50 41 Other/unclear 10 7 Targeted Health Professional(s) Medical Specialty(s) Physician 121 86 GP 84 60 Nurses 16 11 Internists 60 43 Pharmacists 5 4 Other 40 29 Other 3 2
2012 Update: Study Characteristics
![Page 13: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/13.jpg)
Characteristics of Included Studies:Features of Feedback Design
Characteristic N % Characteristic N %
Format
verbal only 13 9
written only 84 60
both 32 23 Instructions for Improvement unclear 11 8 Goal-setting 11 8Source Action planning 41 29 supervisor/colleague 13 9 Both 4 3 employer 15 11 Neither 84 60 investigators/unclear 112 80 Direction of Change RequiredFrequency Increase current behaviour 57 41 weekly 11 8 Decrease current behaviour 29 21 monthly 19 14 Mix or unclear 55 39 less than monthly 36 26
once only 68 49
![Page 14: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/14.jpg)
![Page 15: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/15.jpg)
Meta-RegressionCharacteristic Effect Characteristic EffectFormat of feedback p=0.020 Instructions for improvement p<0.001Verbal 3.4 Target/Goal 2.52Written 9.5 Action plan 9.57Both Verbal and Written 11.2 Both Goal and Action plan 11.09Not clear 5.3 Neither 6.2Source of feedback p<0.001 Direction of change required p<0.001A clinical supervisor or colleague 16.5 Increase current behaviour 4.34A ’PSRO' or employer 2.4 Decrease current behaviour 10.54The investigators 5.0 Chg behaviour/mix/unclear 7.16Not clear 5.5 Baseline performance p=0.007Frequency of feedback p<0.001 at 25% 9.11Frequent (up to weekly) 1.4 at 50% 7.07Moderate (up to monthly) 9.8 at 75% 5.03Infrequent (less than monthly) 4.8 Profession of recipient p=0.561Once only 2.6 Physician 7.9Unclear; 18.1 Non-physician 6.8
Risk of bias p=0.679Low risk of bias 7.68Unclear 7.02High risk of bias n/a
![Page 16: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/16.jpg)
Meta-Regression - Exploratory
Characteristic Effect
Type of professional practice P<0.001Diabetes/CVD 5.91Laboratory testing/radiology referrals 4.21Prescribing 11.11Other 4.71
Direction of change required P=0.525Increase current behaviour 6.64Decrease current behaviour 7.13Change behaviour or mix or unclear 5.7
![Page 17: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/17.jpg)
Summary
• AF is effectiveo One quarter of studies find relatively large effect (>16% aRD)
• AF may be more effective when:o baseline performance is low, o the source is a supervisor or senior colleague, o delivered both verbally and written, o provided more than once, o and includes both explicit targets and an action plan
• Targeted behavior plays an important role
![Page 18: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/18.jpg)
Growing Literature; Stagnant Science?
1985 1990 1995 2000 2005 20100
20
40
60
80
100
120
140
160
![Page 19: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/19.jpg)
Cumulative Meta-Analysis
![Page 20: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/20.jpg)
Feedback and Behaviour Change
Old behaviours
NewbehavioursPerformance
FeedbackPerformance
Feedback
![Page 22: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/22.jpg)
Agenda
![Page 23: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/23.jpg)
Role of Theory
• What can it offer?
![Page 24: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/24.jpg)
Illustrative Examples
• Good, bad, and ugly
![Page 25: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/25.jpg)
Moving forward
Tension between theory and practice?
![Page 26: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/26.jpg)
Agenda
![Page 27: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/27.jpg)
Small groups: part 1
• Building a complete list of potential effect modifiers
• What are the things to consider when thinking about doing audit and feedback interventions?
![Page 28: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/28.jpg)
Small groups: part 2
• Prioritizing effect modifiers for testing
• What are the top five things that we need to know in order to sort out how to make audit and feedback more effective?
![Page 29: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/29.jpg)
Agenda
![Page 30: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/30.jpg)
![Page 31: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/31.jpg)
Agenda
![Page 32: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/32.jpg)
Agenda
![Page 33: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/33.jpg)
• What is the impact of engaging the recipient in design of AF
• What is the impact of adding to AF
![Page 34: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/34.jpg)
5 Research Questions1. What is impact of engaging the recipient in design implementation vs.
engaged in none of them2. What is the impact of adding to AF:
• Incentives/penalties – financial, CME credit + licensing• Tools/practise aid – eg. Clinical decision tool…
• Practise redesign, coaches, facilitation, mentorship3. Skill of the person providing feedback make a significant difference4. Take the top quintile of AF studies and replicate their interventions5. The 5 most important aspects of AF to study are:
1. Frequency, individual vs. group, evidence based, in person vs electronic delivery, number of targets
2. Developing strategies for replicating and implementing successful intervention in other settings, looking at
![Page 35: Using Audit and Feedback to Improve Quality of Care December 7-8, 2012.](https://reader035.fdocuments.net/reader035/viewer/2022081602/55173852550346f5558b6054/html5/thumbnails/35.jpg)
Dinner 6.30pmEmpire Grill 47 Clarence Street (in Byward Market) (About 20
min walk)