Teaching for Quality (Te4Q)

January 12, 2014

University of California, San Francisco

Introduction & Background: why Te4Q?

Faculty

Dave Davis, MD

Senior Director, Continuing Education and Improvement, Association of American Medical Colleges, Washington, DC

Jennifer S. Myers, MD FACP FHM

Associate Professor of Medicine Director of Quality & Safety Education; Associate DIO for Quality, GME Perelman School of Medicine, University of Pennsylvania Philadelphia, PA

Participants & Projects: what’s brought you here?

Getting the ideaDeveloping an educational innovation in Quality

Improvement/Patient Safety

Participant & Project introductions..

Name Faculty Role Project Aim/Goal Learners

the patient’s perspective

The clinical care gap

The National view:Dartmouth Atlas 2010 - HbA1c data

The TeQ Report: Why Teachingfor Quality?

The Te4Q Vision:Quality

Improvement is core to what it means to be a

physician

aamc.org/te4q

Te4Q Recommendation

“Every academic health center will have a critical mass of faculty ready, able and willing to engage in, role model, and teach about patient safety and the improvement of health care”

AAMC’s response:The Te4Q Faculty Development Program

Moving ‘QI/PS savvy’ clinicians to expert QI educators

Clinical faculty as: Teachers Curriculum developers Evaluators, competency assessors Educational Quality Improvers Change Agents Leaders Role Models Mentors/peer advisers

The Te4Q Faculty Development Certificate Program

Pre-Requisite: some knowledge of QI/PS Self- & Organizational-Readiness Assessments Pre-reading Skill Building Workshop QI Educational Project w/presentation in 3 mo. Community of Practice Dissemination of Work—Presentation or

Publication Certificate

Te4Q Workshop Objectives

Address an identified gap in the education of students, residents, and/or practicing clinicians regarding quality improvement and patient safety

Design an educational innovation to fill that gap Effectively implement the initiative Assess the impact of the innovation on learners and

the larger community Enable and lead educational and organizational change

Workshop Agenda

Introductions

Adult Learning and Educational Principles

Identify Gaps

Learner Levels/Competencies

Educational Program Goals and Objectives

Interprofessional Education

Educational Design: effective formats for learning

Reflection & Feedback

Developing QI/PS Content (JM)

Frameworks for Teaching Quality

What to Teach

Teaching & Learning in the Clinical Environment

Workshop Agenda, cont’d Assessing the Impact I

Learner Assessment Formative vs Summative Feedback

Assessing the Impact II Program Evaluation

Reflection & Feedback 

Making the Case & Leading Change Culture & the Hidden Curriculum Creating a Strategy for producing change Developing and implementation strategy

Dissemination and Next steps

16

Thank You

Section I:Adult Learners:

setting goals, achieving competency in QI/PS

Knowing Your Learner: Who are they? What are the core and supportive competencies in QI/PS?

Assessing your own learning needs

In time…developing effective educational goals and learning objectives based on your learners needs

SOI Knowledge

Subject Matter

Knowledge – Clinical

Medicine

Subject Matter Knowledge: Knowledge basic to the things we do. Professional knowledge: our clinical, educational, and/or research expertise

Knowledge

of Education

Knowledge of education: Knowledge of adult learning principles, curriculum development, learner assessment, etc.

We Have a Lot of These People

We Have Some of These People

Knowledge of Education

Subject Matter Knowledge – Clinical Medicine

Great

Teach

ers

Knowledge of Education

We Have a Lot of These People

Subject Matter Knowledge – Clinical Medicine

SOI Knowledge

Science of Improvement: Knowledge of the theories of systems, variation, measurement, and psychology.

We Don’t Have a Lot of These People

SOI Knowledge

Subject Matter Knowledge

Improvement occurs when we combine subject matter knowledge and the science of improvement in creative ways to

develop effective ideas for change and then execute them using proven quality improvement methods.

Impro

vem

e

nt

We Have a Precious Few Number of These People

Creating Quality & Safety Educators

Subject Matter Knowledge – Clinical Medicine

Knowledge of Education

SOI Knowledge

Quality & Safety

EducatorsWe Do Not Have Enough of These

People

What Type of Faculty Are Interested in Teaching Quality & Safety?

Education Role QI/PS Role No Formal Role0

10

20

30

40

50

60

70

80

2012

2013

2014

* Quality and Safety Educators Academy – Attendee Roles

Concepts of Competence: what do we want our learners

to do? To ‘look like’?

Competency is defined as the observable behavior that combines knowledge, skills, values, and attitudes related to a

specific activity

Meet Dr Zadowski

Emily Z is a 3rd year Internal Medicine resident.

She meets you in Emerg to discuss Jim Halton, a 72 yr old male with congestive heart failure. He needs re-admission, has increasing symptoms of CHF (weight gain, increasing SOBOE, orthopnea) - the result of inadequate care in his last admission.

You and she review the notes from his admission and discharge last week – including nursing and other professional notes. You use the chart to stimulate her observations about quality this case and others like it.

Is she competent in Quality? What questions would you think to ask?

Medical Knowledge Patient Care Interpersonal Communication Professionalism Systems-based Practice Practice-based Learning and Improvement

Common competencies:the ACGME/ABMS Framework

In the Beginning…

The Mirror:Practice Based Learning & Improvement

The Village:Systems-Based Practice

What Kind of Doctor Does our Health System Need?

Frank, et al. Lancet. 2010. 376: 1923-57.

Old Model

New Model

Teamwork & Professionalism

Knowledge

Skills

Attitudes

Quality Safety

QI methods, Measurement,& Tools

Safety Tools: RCA, FMEA

Safety Principles Error Types Human Factors

Leadership & Change Management

Safety Culture

Error Reporting

Evidence-Based MedicineHealth Information Technology

IOM

SafeTimelyEfficientEffectivePt centeredEquitable

Stewardship

Value

Myers, Tess, et al. Am J Med Qual. 2013

The checklist: Te4Q QI/PS “Proficient” Competencies

Critically evaluate and apply current healthcare information and scientific evidence for patient care

Systematically analyze practice using quality improvement methods and demonstrate improvements in practice

Working effectively in health care delivery settings, including identifying system issues and improving them

Incorporate considerations of cost awareness and risk-benefit analysis in patient and/or population-based care

Participate in identifying system errors and implementing potential system solutions (patient safety)

Work in interprofessional teams to enhance patient safety and improve patient care quality

Dreyfus Model: Novice Expert Dreyfus and Dreyfus, 1982

Developmental Stages in QI/PS

Level Training Level Example

Novice Pre-Clinical medical student

• Introductory lectures, web exercises• Group work on case studies in QI/PS

Advanced Beginner

Clerkship Student or Sub-Intern

• Students apply concepts (e.g., RCA) in a “project” at the academic health center•Teacher is model and “coach”

Competent Residency & Fellowship

• Apply concepts to his or her own panel of patients in interprofessional team

Proficient Fellowship & Early practice

• Regularly review and improve care for patients

Expert Advanced practice/faculty

• Develop novel ways to understand and improve systems of care

Two Types of Faculty Development

*All Faculty are clinical teachers and role models for the residents

Quality and

Safety Educat

ors

Formal

Curriculum

Type 1: Expert Type 2: Proficient

Leading Experiential Activities or

Immersion Activities

Our Trainees Need Both

All Faculty*

Informal Curriculum

Faculty Learners: Skills We’d Like to Build

Proficient (Type 1) Expert (Type 2) MasterCore knowledge of QI/PS

Common language

Doing basic improvement in practice

Modeling QI/PS principles with learners

Prepared as a good improvement team member

Participating in MOC Part IV

Other attributes…..

Proficient, plus…

Increased experience in QI/PS projects (eg. lead)

Leader in education and curricular implementation

Able to create experiential and didactic learning activities for students, residents, others

Able to understand and measure learner progress

Expert, plus…

Curricular reform and/or clinical leadership roles related to QI/PS

Scholarship in QI/PS

Career focus in QI/PS

Te4Q

A Look at Your Self Assessments

• What is YOUR stage of development in QI/PS?• What are your strengths• Which skills need development?

Knowing the TeacherSelf-Reflection Exercise:

Proficient Expert MasterCore knowledge of QI/PS

Common language

Doing basic improvement in practice

Modeling w/learners

Prepared as good improvement team member

Participating in MOC Part IV

Proficient, plus…

Increased experience in QI/PS projects (eg. lead)

Leader in education and curricular implementation

Able to create experiential and didactic learning activities for students, residents, others

Able to understand and create metrics to assess learner progress

Expert, plus…

Curricular reform and/or clinical leadership roles related to QI/PS

Scholarship in QI/PS

Career focus in QI/PS

Think About Your Project

Your Primary Learner(Who Is Teaching? )

Your Secondary Learner(Who Is Learning?)

KnowledgeType

Subject Matter

Science of Improve-

ment

EducationalPrinciples

Subject Matter

Science of Improvement

UMEProficient

-> Expert

? ? Novice Novice

GME Proficient -> Expert

? ?Advanced Beginner -> Competent

Novice -> Advanced Beginner

CPD Proficient -> Expert ? ?

Proficient -> Expert ?

Knowing your LearnersTeam Think - Share• Who are your faculty learners?

• What is their stage(s) of learning in the science of improvement? in educational principles?

• Based on the above, what are their educational needs?

1. GOAL/AIM: What is the gap or problem you want to address through your educational program? What are the Quality Improvement/Patient Safety competencies you want your learners to exemplify at the completion of your educational program?  2. OBJECTIVES: What are the learning objectives? Use the term ‘performance expectation’ to guide you here. What do you want learners to do after the activity?

Think SMART: specific, measurable, actionable, relevant and time bound 3. EDUCATIONAL DESIGN: What learning methods/formats will you use? Develop 2-3 bullet points to outline your educational idea, project or innovation and what QI/PS content you want to include.  4. LEARNER ASSESSMENT: How will you assess your learners? What methods will you use to assess what your learners have accomplished?  5. PROGRAM EVALUATION: How will you evaluate your initiative? How will you know its impact and how it might be improved?  6. IMPLEMENTATION: How will you implement your project/innovation? Who will be your partners? What resources will you need? From whom will you need ‘buy-in”?   How confident are you that you will be able to complete your project?

7. DISSEMINATIONDissemination Plan: How will you report your project to a larger audience? eg. Poster, presentation, publication.

   

  

BREAKand evaluations

Adult learning Section II:

Creating effective educational program and innovations

the program planning cycle goals/objectives

effective educational interventions

Quality Cycle

Kern: Curriculum Development for Medical Education 2009

1. Problem Identification

& Needs Assessment

1. Problem Identification

& Needs Assessment

2. Goals & Objectives2. Goals & Objectives

3. Educational Strategies

4. Implementation

5. Evaluation & Feedback

Educational Planning Cycle

Comparing Educational Planning with Quality Improvement

Adapting problem and aim statement from QI/PS: Developing Goals & ObjectivesWe’ve identified our problem and level of learner, and now….

| 43

| 43

Creating a Problem Statement• Commonly used in both

Academic and Quality / Performance Improvement methodologies.

• Should meet the following criteria:

• Focused only on one problem• Only one or two sentences long• Should not suggest a solution• Unambiguous and devoid of

assumptions

| 43

Taking Aim

| 44

• Aim statements are very specific declarations of what a team will be focusing on as they strive to improve a process or a system.

• They should include a few elements:

• The system to be improved and the population

• A numerical goal (preferably an ambitious “stretch” goal)

• A timeframe

Your need/problem/goal

Goals/Objectives: Make them SMART

1. First, a stem…• After completing the seminar, residents will be able to…. • After this unit, the medical student will have . . .• By completing the Year 3 curriculum, the health professional

student will….. • At the conclusion of the course/unit/study the learner will . .

2. Next, add an action verb• Use verbs from Bloom’s taxonomy list • Determine the actual product, process, or outcome….

Creating Educational Objectives

http://www.educationoasis.com/curriculum/LP/LP_resources/lesson_objectives.htm

Interface of Learning Objectives and Teaching Strategiescase studies

projectsexercises

projects critiquesproblems simulations

case studies appraisalscreative exercises

problems develop plans EVALUATIONexercises constructs appraise

case studies simulations assessexercises critical incidents choosepractice discussion SYNTHESIS estimate

demonstrations questions arrange evaluatequestions projects test collect judgediscussion sketches compose measure

review simulations ANALYSIS construct ratetest role play analyze create revise

assessment microteach calculate design scorereports   compare formulate select

lecture learner APPLICATION contrast organize valuevisuals presentations apply criticize managevideo writing demonstrate debate planaudio     dramatize diagram prepare

examples COMPREHENSION employ differentiate proposeillustrations describe illustrate distinguish set upanalogies discuss interpret experiment

Cognitive Processing Dimension

    explain operate inspect→ KNOWLEDGE express practice inventory  define Identify perform question

list recognize schedule relate record restate shop test

Learning Objectives

repeat translate sketch  use

Adapted from: Bloom, B.S. (Ed.), Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. New York: David McKay.

Ellen F. Goldman, EdD 11-1-2010

Bloom’s Taxonomy

Goal/Aim: After the full QI/PS course, 25% of all Internal Medicine and Medicine-Pediatric residents will complete a longitudinal QI project with general internal medicine faculty

AVOID CHOOSE• Vague verbs

“Know how to …” Explain, describe, discuss• General terms Be as specific as possible

Example of Program/Activity Objective

Examples of Learner Objective

Not so good: At the end of this session, the resident will understand quality improvement methodologies….

Suggestions for Improvement:

Better…

After completion of this course, learners in that course will be able to:

define Root Cause Analysis* (RCA); recognize which clinical situations require RCA; explain why RCA is important; demonstrate mutual respect on interprofessional

teams; and have performed an interprofessional mock RCA.

**a determination of the factors, processes and problems leading to an error or care gap

Your Te4Q Educational InitiativeTeam Think – Share:• Define/refine your learning objectives for your QI/PS

Initiative.• Share one of your objectives with us, your “consultants”

Interface of Learning Objectives and Teaching Strategiescase studies

projectsexercises

projects critiquesproblems simulations

case studies appraisalscreative exercises

problems develop plans EVALUATIONexercises constructs appraise

case studies simulations assessexercises critical incidents choosepractice discussion SYNTHESIS estimate

demonstrations questions arrange evaluatequestions projects test collect judgediscussion sketches compose measure

review simulations ANALYSIS construct ratetest role play analyze create revise

assessment microteach calculate design scorereports   compare formulate select

lecture learner APPLICATION contrast organize valuevisuals presentations apply criticize managevideo writing demonstrate debate planaudio     dramatize diagram prepare

examples COMPREHENSION employ differentiate proposeillustrations describe illustrate distinguish set upanalogies discuss interpret experiment

Cognitive Processing Dimension

    explain operate inspect→ KNOWLEDGE express practice inventory  define Identify perform question

list recognize schedule relate record restate shop test

Learning Objectives

repeat translate sketch  use

Adapted from: Bloom, B.S. (Ed.), Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook 1: Cognitive domain. New York: David McKay.

Ellen F. Goldman, EdD 11-1-2010

Bloom’s Taxonomy

Developing Educational Strategies & Methods

Principles and Methods

What the theory of adult learning tells us..

Comfort/respect

Experience: (Kolb) build on learner’s

experience

Relevance: to current status, roles

Engagement: role play, simulations

Reflection: (Schon) on past

experience, errors,

prevention

What the research tells us

Physicians and others not self-aware: objective needs assessment, performance feedback important

Knowledge necessary but not sufficient for change; didactics lousy at changing performance

What works? Interactivity; sequencing; predisposing, enabling and reinforcing strategies

‘CPD’ > conferences; = practice-based tools (reminders, audit-feedback, protocols & training)

Docs pass through stages of learning: awareness, agreement, adoption to adherence

…Cochrane reviews, AHRQ/EB reviews, others

Large Group: Interactive Lecturing

Active participation: e.g.,

think-pair-share Lecturer=facilitator, docent,

group leader Widespread use of case, problems, vignettes NOITE: the Flipped classroom: reading and learning

before the session with application of knowledge at the session.

Generally 7-10 learners Uses case-based materials to stimulate discussion

• A patient safety issue, e.g., wrong site surgery and a subsequent RCA or fishbone exercise

• A communication error in a team• Other…your own

Requires clear learning objectives, expectations of full participation

May use a tutor (expert or non-expert), or be self-led

Very useful for team development

Small Group Problem-Based Learning

Experiential LearningReal-world experience

- Handoffs- M&M conferences- Rounds- Bedside/Clinic

Simulations- Role play- Standardized

patients- Simulation labs- Cases- Computerized/

games

LEAN thinking

TeamSTEPS Microsystem planning

Process mapping

Fishbone diagrams

The five why’s

Using an Tool BoxQuality Improvement Tools

Root cause analysis

Educational

http://patientsafetyed.duhs.duke.edu/module_b/module_overview.html

Other Siteshttps://depts.washington.edu/toolbox/errors

www.aamc.org/initiatives/cei/te4Q

Methods useful in Staging Learning – e.g., changing the culture of quality in the workpace Examples using the Pathman/PRECEED model

Stage/Method Awareness Agreement Adoption Adherence

Predisposing LectureNewsletterGrand roundsMeetings

Enabling ChampionsSmall group discussionLeadership buy-inPeer buy-in

WorkshopsTraining sessionsSimulationMentorship, coachingQuality projects

Reinforcing Role-playFeedback in practiceMentorship

RemindersAudit/FeedbackCelebration of achievements

Teach Dr Zadowski

Emily Z is a 3rd year IM resident.

She meets you in Emerg to discuss Jim Halton, a 72 yr old male with congestive heart failure. He needs re-admission. You and she review the notes from his admission and discharge last week – including nursing and other professional notes. You use the chart to stimulate her observations about QI/PS this case and others like it.

How would you prepare her and her classmates for practices which focus on quality improvement and patient safety? What educational programs, experiences and other education can you create?

Core Entrustable Professional Activities for Entering Residency (CEPARs)

• EPA 13: Identify system failures and contribute to a culture of safety and improvement

• EPA 8: Give or receive a patient handover to transition care responsibility

• EPA 9: Collaborate as a member of an interprofessional team

1. GOAL/AIM: What is the gap or problem you want to address through your educational program? What are the Quality Improvement/Patient Safety competencies you want your learners to exemplify at the completion of your educational program?  2. OBJECTIVES: What are the learning objectives? Use the term ‘performance expectation’ to guide you here. What do you want learners to do after the activity?

Think SMART: specific, measurable, actionable, relevant and time bound 3. EDUCATIONAL DESIGN: What learning methods/formats will you use? Develop 2-3 bullet points to outline your educational idea, project or innovation and what QI/PS content you want to include.  4. LEARNER ASSESSMENT: How will you assess your learners? What methods will you use to assess what your learners have accomplished?  5. PROGRAM EVALUATION: How will you evaluate your initiative? How will you know its impact and how it might be improved?  6. IMPLEMENTATION: How will you implement your project/innovation? Who will be your partners? What resources will you need? From whom will you need ‘buy-in”?   How confident are you that you will be able to complete your project?

7. DISSEMINATIONDissemination Plan: How will you report your project to a larger audience? eg. Poster, presentation, publication.

   

  

Section III:

Applying the Content of QI/PS to educational program and innovations

Teaching Quality Can Feel Like This

What to teach ?

Will the Residents “buy-in”?…

With what time ?

Who will teach ?

Will the Faculty “buy-in”? even “show up”?

How to teach ?

Where is the Curriculum Guide?

Outline

How Can This Content be Organized?• Educational Frameworks

What to Teach? • Quality & Safety Content

How to Teach?• Strategies for Effective Teaching of Quality & Safety

Our Reality: The “Middle Place”

Undergraduate & Graduate Medical

Education in quality and safety is still highly variable

Patien

t Saf

ety

Qua

lity Im

prov

emen

t

Inpa

tient

Han

doffs

Discha

rge

Trans

itions

High V

alue

Cost C

onsc

ious C

are

0

20

40

60

80

100

QSEA 2012 QSEA 2013 QSEA 2014 Penn Fall 2013

Progress Has Been Slow

Medical Knowledge Patient Care Interpersonal Communication Professionalism Systems-based Practice Practice-based Learning and Improvement

Common competencies:the ACGME/ABMS Framework

Some Content Outlines

http://www.aacn.nche.edu/education-resources/ipecreport.pdfhttp://qsen.org/http://www.ihi.org/education/ihiopenschool/Pages/default.aspxhttp://www.who.int/patientsafety/education/curriculum/EN_PSP_Education_Medical_Curriculum/en/

Quality & Safety in the ACGME Milestones

Good News:

…they mostly all say the same thing

Bad News:

…each specialty chose their own language to describe their target competencies in:

• Systems-Based Practice• Practice-Based Learning & Improvement• Interpersonal & Communication Skills • Professionalism

Example: Surgery Systems-Based Practice Milestone

This resident makes suggestions for changes in the healthcare system that may improve patient care.

This resident reports problems that could produce medical errors.

This resident participates in groups or PI teams designed to reduce errors & improve health outcomes

SBP Milestones for a Graduating ResidentMedicine Surgery Emergency Med

Efficiently coordinates activities of team members to optimize care

Coordinates activities of health care professional team to provide optimal care at time of discharge…

Coordinates system resources to optimize patient care for complicated medical situations.

Identifies and advocates for safe care and optimal systems

Makes suggestions for health system improvement

Leads team reflections such as de-briefings, RCAs, or M&M to improve team performance

Activates formal system resources to investigate or mitigate error

Reports problems with technology or processes that could result in medical error

Identifies situations when breakdowns in teamwork or communication could lead to medical error

Participates in PI teams designed to reduce errors and/or improve care outcomes

Participates in an institutional PI project to optimize ED practice

Advocates for cost-conscious utilization of resources & incorporates these principles

Understands appropriate use of standardized approaches to care & contributes to them

Practices cost effective care

The Milestones are mostly all at the “Does” Level

Miller’s Pyramid for Learner Assessment

Outline

How Can This Content be Organized?• Educational Frameworks

What to Teach? • Quality & Safety Content

How to Teach?• Strategies for Effective Teaching of Quality & Safety

Teamwork & Professionalism

Knowledge

Skills

Attitudes

Quality Safety

QI methods,Measurement,& Tools

Safety Tools: RCA, FMEA

Safety Principles Error Types Human Factors

Leadership & Change Management

Safety Culture

Error Reporting

Evidence-Based MedicineHealth Information Technology

IOM

SafeTimelyEfficientEffectivePt centeredEquitable

Stewardship

Value

Myers, Tess, et al. Am J Med. 2013

Teach Dr Zadowski

Emily Z is a 3rd year IM resident.

She meets you in Emerg to discuss Jim Halton, a 72 yr old male with congestive heart failure. He needs re-admission. You and she review the notes from his admission and discharge last week – including nursing and other professional notes. You use the chart to stimulate her observations about QI/PS this case and others like it.

How would you prepare her and her classmates for practices which focus on quality improvement and patient safety? What educational programs, experiences and other education can you create?

Safety & Quality Are Connected

Safety event

Quality Improvement Opportunity

Afferent Arm

Efferent Arm

One Safety Event or Quality Problem:Two Pathways For Teaching & Learning

Preventable Adverse Event, Near Miss, Quality Problem

Patient Safety

Methods

QI Methods

Retrospective Analysis Prospective Improvement

What Should We Review?Both? Neither?

Patient Safety Methods Model for human error Hierarchy of Improvement Plans Cause-Effect Diagramming Event Reporting/Safety Culture

QI Methods Framing & scoping the

problem FOCUS-PDSA Process Mapping &

Observation Root Cause Tools Aim statements Measuring for Quality

Only Two Strategies to Eliminate Errors

• Find and fix systems problems

• Plug up the “holes” in the swiss cheese

• Mitigate and absorb the natural human error rate

• Promote safety behaviors and culture

How Do Errors Happen?

EVENT ofHARM

Reason, Managing the Risks of Organizational Accidents, 1997

Multiple Barriers - technology, processes, and people - designed to stop active errors (our “defense in depth”)

BLUNT END

LATENT ERRORSAre inherent properties of the system that allow or “set up” the individual to fail

SHARP END

ACTIVE ERRORS are made by individuals

Blunt vs SharpActive vs Latent

Fishbone (Cause & Effect) Diagram

Why has roomturnover

time increased?

People Place

Policies Processes

EFFECT

CAUSES

• Structured team brainstorming

• Represents relationship between some effect and all of the possible causes influencing it

Five WhysEnvironmental Services Taking Longer to Clean rooms1st Why? New specialized room sterilization 2nd Why? High rates of resistant organisms3rd Why? Some staff less familiar with the process4th Why? No standardized training process for part-

time, temporary staff

Weak Actions: Signs

Intermediate Actions: Beeping

Strong Actions: Swipe card only

How do you prevent customers from leaving their ATM cards behind?

Hierarchy of Improvement Plans

Ranking the Effectiveness of Error-Reduction Strategies

Most Effective (Strong)

– Forcing functions and constraints– Automation and computerization– Standardization and protocols– Checklists and double-check systems– Rules and policies– Education and information– Exhortation: “Be more careful. Be vigilant.”

Least Effective (Weak)

Gosbee JW, Gosbee LL, eds. Human Factors Engineering to Improve Patient Safety.

Oakbrook IL: Joint Commission Resources 2005

One Safety Event or Quality Problem:Two Pathways For Teaching & Learning

Preventable Adverse Event, Near Miss, Quality Problem

Patient Safety

Methods

QI Methods

Retrospective Analysis Prospective Improvement

How I Used to Do Quality Improvement

Different Quality Improvement Methodologies: Similar Principles…All Foreign to Healthcare

A3 Diagram

FOCUS Before You Begin PDSA CyclesD E F I N E

F I N D

Problem Statement (from patient’s/customer’s viewpoint): (10 points)

Concise. Customer- focused. Addresses the business case. Shows why a change is needed. Background Provides background for the problem statement (1-3 paragraphs with a minimum of 3

references using APA reference format. Why is this problem important? Who is interested in it (nationally and/or locally) and why? Is there evidence behind this being a problem in healthcare? If so, provide a summary of that evidence. [note: this item will be handed in typewritten, attached to your paper A3]

Target Condition: (10 points)

Your target condition will move you towards delivering exactly what the customer wants, closer to IDEAL. You identify at least one process, outcome, and balancing measure for your proposed project. You have a clear aim statement, and your process and outcome metrics are expressed as S.M.A.R.T. goals.

P L A N A N D

D O

I M P R O V E

Propose and Test Countermeasures: (10 points)

Each countermeasure is clearly related to a root cause in the previous section. For each countermeasure, state how you will DO the tests - What, When and by Whom.

M E A S U R E

O R G A N I Z E

C L A R I F Y

Current Condition: (20 points)

There is evidence that you did your “Go and See”. The section is rich with data. Data is presented graphically – bar charts, run charts, or SPC charts are

preferred. You have at least one process map (Value-stream, flow chart, spaghetti diagram). The metric(s) you are trying to move are clearly identified. Baseline/pre-test metrics are presented here.

Must consider at least one potential process, outcome, and balancing metric You have completed a stakeholder analysis and attached it to your paper A3.

Metrics/Results: (10 points)

If a test of change has been implemented, the S.M.A.R.T. goals are restated and results presented as data. Data should be in a run chart or a statistical process control chart (note: this is not required for successful completion of the project for class; some students may not have this item completed yet; see next bullet)

If a test of change has not been implemented, describe which countermeasure you have chosen to test first and why.

Reflect on What You Have Done and Learned so Far in Improvement work: (10 points) Write a 1-3 paragraph reflection on what you learned by going through this process. What was most challenging and why? What surprised you most and why? What do you think is your biggest challenge going forward? [note: this will be handed in typewritten attached to your A3]

CH E CK

or

S T U D Y

C O N T R O L

A N A L Y Z E

U N D E R S T A N D

S E L E C T

Root Cause Analysis: (20 points)

You use a relevant root cause tool. (5 Whys, Fishbone, Pareto). Must use at least one tool, though you may need more than one for a deep analysis.

Your choice of which root cause(s) to tackle is supported by the data in the Current Condition section – i.e. the data validates that you have identified the true root cause(s) and have chosen the right one(s) to work on first.

Make it Standard Work (Implement Successful Countermeasures): While this step is not required for this class, be aware that failing to execute this step is extremely common in healthcare. Skipping this step is usually the root cause for failing to sustain improvements after attention has moved on to the next project. You should think about how you might implement a successful countermeasure and sustain any improvements.

A C T

Executive Sponsor Initial Approval (signature and date): Not required for this class – but never do a real project without this!

Executive Sponsor Final Approval (signature and date): Not required for this class – but never do a real project without this!

FOCUS Find – Problem StatementOrganize – Current ConditionClarify – Current ConditionUnderstand – Root Cause ToolsSelect - Root Cause Tools…beginning to select and think of countermeasures that match the root cause(s)

Propose and Test Countermeasures

PDSAPlan – Plan your 1st Test of ChangeDo – “Do” the ChangeStudy – Track your results (metrics) and

compare them to your predictionsAct – Reflect on the test and decide what to

do next…i.e. second test of change, 3rd, test of change, eventually implement

Outline

How Can This Content be Organized?• Educational Frameworks

What to Teach? • Quality & Safety Content

How to Teach?• Strategies for Effective Teaching of Quality &

Safety

Building Bridges: Linking Educational Activities With Health

System Improvement Efforts

Sound Familiar? “Our hospital is struggling with the over-utilization of labs and we have had 2 RCAs on medication errors at discharge. We just found out that the GME office is starting QI education at the program level and neither of these topics was prioritized. Why doesn’t anyone talk to each other around here?” - Chief Quality & Safety Officer

“We have an incident reporting system, but I am pretty sure the nurses are the only ones who use it. The

residents don’t know who reads them so are scared to report. We also don’t see the faculty reporting so why

would we bother? We are busy enough.”

- Resident

“We cannot get a departmental QI project off the ground because we can’t get the baseline data, let alone set up a process for ongoing measurement and evaluation.”.

-Associate Program Director

Aligning Our Missions

Residents as Common End Point

Operational Quality/Safety Goals:Trainees as Front line providers

Educational goals: Trainees as learners

Residents as Common End Point

Operational Quality/Safety Goals:Trainees as Front line providers

Educational goals: Trainees as learners

Barriers to Alignment and Capability are Deeply Entrenched

More Barriers

• Discontinuous trainee involvement

- Impedes the concept of ongoing system improvement

• Difficulties in scheduling and implementing essential team-based interprofessional learning and improvement

‒ Simulation the default

• Failure to rush provide real-time data for improvement to the front-line

• Lack of faculty trained in point-of-care experiential, inter-professional learning techniques and improvement science methods

• Tortuous and painful promotional pathway and reward structure for such faculty

Slide adapted with permission from Don Goldmann MD, IHI

Will Require Shared Responsibility:New Relationships, Roles, and Work

Trainees

Infrastructure, Shared Work Plan

OversightCentralized resources

Curriculum development,Teaching,Mentorship

Daily supervision, role modeling, & practice enforces local quality/safety culture

Core QI/PS

Faculty

Frontline Faculty & All

Staff

Tess, Vidyarthi, Yang, Myers [In Press]

“Top 5 List” for Healthcare Organizations That Wish to Achieve Excellence in Teaching for Quality

#1. Select Educational Content that aligns with health care system needs

“ Top-Down” vs “Bottom-Up” Approach to Project Selection

106

ENGAGEMENT Involve faculty and staff as partners with patients and families to achieve goals of care.

Penn Medicine will improve the health of our patients and assure safe care.

CONTINUITY Deliver seamlessly coordinated care across all settings and service lines.

VALUE Provide high quality, efficient care and the best outcomes for all patients.

Resident Project Selection Aligns with a Clinical Microsystem QI Need

Resident Unit QI Topic Career Path

I.L. Gen Med Discharge Med Rec Endocrine & CHIPS* fellow

M.P. MICU Post-Intubation Checklist RWJ Scholar

I.W. CCU FMEA Door to Balloon Time Cardiology Fellow

M.A. L & D Safety Culture and Team Debriefings Urogyn Fellow

H.G. Emerg Dept Communication: Neurology & ED Academic Emerg Med

S.G. Medicine Clinic Screening Colonoscopy Rates Primary Care

Patel, Brennan, Myers, et al. Acad Med. 2014

*Center for Healthcare Improvement & Patient Safety, UPenn

#2: The Education Must Be “Hands-On”

• Real cases for quality improvement & patient safety learning– Dissect a near miss or preventable adverse event using root cause tools– Use a current quality problem to:

• Define and scope the problem• Write a specific aim statement• Identify process, outcome, and balancing measures• Identify root causes and select countermeasures

• Real data for QI review– What type of data?

• Healthcare associated infections• Patient satisfaction scores• Readmission rates• Test utilization • Other?

Data Matters

You Need to Consider:How will I get these cases?

How/where will I get this data?

For this hands-on education, need to determine the relative value of these goals -- to improve care right now, to teach someone to improve care, or both?

Examples of Each from Penn

• Improve care right now:– On the general medicine teaching services, we have committed

to including the nurse in bedside rounds

• Teach someone to improve care– 3rd year Penn Medical, Nursing, and Pharmacy students reflect

upon and dissect a preventable adverse event together using root cause analysis tools

• Do Both– 2nd year Penn internal medicine residents each review one of

their own 30-day readmissions looking for opportunities for improvement to feed back to the Dept of Medicine Readmission task force

– Role Modeling

#3. The Educators Must…. Make Connections for the Trainees

One success factor for trainee QI projects is faculty mentors who have local system knowledge and deep

connections within the organization*

Ogrinc G, et al. Academic Medicine, 2014

What are we connecting the trainees to?

Residents & FellowsInfrastructure

Data

People in our Organization

Creating Quality & Safety Educators

Subject Matter Knowledge – Clinical Medicine

Knowledge of Education

SOI KnowledgeConnections

within their health system

Quality & Safety Educators

Assume trainees know nothing (and I mean nothing)

about managing a project longitudinally and running meetings

#4 Use a Common Language

• Make the “jargon” UCSF-branded jargon• Takes awhile, but begins to catch on• Helpful for teaching across the continuum and

sharing of educational materials

#5 Look for Every Opportunity to Make Teaching & Learning Interprofessional

• Teaching about Health care delivery and improvement is a natural IPE topics

• IP Quality & Safety Grand Rounds• By definition, QI projects are interprofessional

Break: Apply the 5 Tips to Your Project

Select Educational Content That Aligns With Organizational Health Care Needs

“Hands-On” EducationEducators Making Connections For TraineesUse a Common LanguageMake it Interprofessional Whenever Possible

1. GOAL/AIM: What is the gap or problem you want to address through your educational program? What are the Quality Improvement/Patient Safety competencies you want your learners to exemplify at the completion of your educational program?  2. OBJECTIVES: What are the learning objectives? Use the term ‘performance expectation’ to guide you here. What do you want learners to do after the activity?

Think SMART: specific, measurable, actionable, relevant and time bound 3. EDUCATIONAL DESIGN: What learning methods/formats will you use? Develop 2-3 bullet points to outline your educational idea, project or innovation and what QI/PS content you want to include.  4. LEARNER ASSESSMENT: How will you assess your learners? What methods will you use to assess what your learners have accomplished?  5. PROGRAM EVALUATION: How will you evaluate your initiative? How will you know its impact and how it might be improved?  6. IMPLEMENTATION: How will you implement your project/innovation? Who will be your partners? What resources will you need? From whom will you need ‘buy-in”?   How confident are you that you will be able to complete your project?

7. DISSEMINATIONDissemination Plan: How will you report your project to a larger audience? eg. Poster, presentation, publication.

   

  

#6: Remember that all formal curriculum has a parallel “hidden” curriculum…

…more on this tomorrow!

“QI”