The development of a conceptual guideline The development of a conceptual guideline implementabilityimplementability tool (GUIDEtool (GUIDE--IT): IT):

A qualitative study of guideline developer A qualitative study of guideline developer and endand end--user perceptionsuser perceptions

Monika Kastner, PhDMonika Kastner, PhDLi Ka Li Ka ShingShing Knowledge Institute of St. MichaelKnowledge Institute of St. Michael’’s Hospital, s Hospital,

University of TorontoUniversity of Toronto

Guidelines International Network (GIN) ConferenceGuidelines International Network (GIN) ConferenceChicago, August 27, 2010Chicago, August 27, 2010

•• Clinical practice guidelines (Clinical practice guidelines (CPGsCPGs) can ) can facilitate the implementation of evidence into facilitate the implementation of evidence into practice practice –– not consistently achievednot consistently achieved

•• The relationship between the perceived The relationship between the perceived characteristics of characteristics of CPGsCPGs and their uptake in and their uptake in practice is not clearly understoodpractice is not clearly understood•• Lack of usability testing of guidelines may Lack of usability testing of guidelines may

be one contributing factor to poor uptake be one contributing factor to poor uptake

BackgroundBackground

•• We conducted a realist review to determine:We conducted a realist review to determine:•• The intrinsic attributes of guidelines that The intrinsic attributes of guidelines that

impact uptakeimpact uptake•• Gain a multidisciplinary perspective of the Gain a multidisciplinary perspective of the

concept of concept of ““implementabilityimplementability””

Background Background

WordingWording(Simplicity, Clarity, (Simplicity, Clarity,

ActionabilityActionability, Framing), Framing)

Preliminary Framework of Preliminary Framework of ImplementabilityImplementability

EvidenceEvidence(Evidence(Evidence--use, Riskuse, Risk--benefit)benefit)

PresentationPresentation(Style, Format)(Style, Format)

ValuesValues(Flexibility)(Flexibility)

ContextContext(Feasibility)(Feasibility)

Guideline Guideline ImplementabilityImplementability

UptakeUptake

•• To validate the core set of guideline To validate the core set of guideline attributes found in the realist reviewattributes found in the realist review

•• To better understand the difference in To better understand the difference in perceptions between guideline developers perceptions between guideline developers and end users about guideline and end users about guideline implementabilityimplementability

•• To determine what components should be To determine what components should be included in a guideline included in a guideline implementabilityimplementability tool tool (GUIDE(GUIDE--IT)IT)

ObjectivesObjectives

Methods: Qualitative Study DesignMethods: Qualitative Study Design

PHASE 2: PHASE 2: Interviews Interviews and Focus and Focus GroupsGroups:: End End usersusers

PHASE 4:PHASE 4: Focus Groups:Focus Groups: Mixed group of Mixed group of developers and developers and endend--usersusers

PHASE 3:PHASE 3: Participant Participant observation and observation and Focus GroupsFocus Groups: : DevelopersDevelopers

PHASE 1:PHASE 1: Establish Establish collaboration with a collaboration with a guideline development guideline development group group

Formed a relationship with Formed a relationship with CC--CHANGE group, who CHANGE group, who provided draft provided draft recommendationsrecommendations

•• To reveal guideline attributes perceived as To reveal guideline attributes perceived as facilitators and barriers to implementationfacilitators and barriers to implementation

•• To elicit feedback from family physicians on To elicit feedback from family physicians on draft recommendations developed by Cdraft recommendations developed by C-- CHANGE CHANGE

•• To reveal possible components of a guideline To reveal possible components of a guideline implementabilityimplementability tool tool

Objectives: PHASE 2Objectives: PHASE 2

Sampling & PopulationSampling & Population•• Guideline developers:Guideline developers: Harmonized cardiovascular Harmonized cardiovascular

guidelines development group guidelines development group -- CC--CHANGECHANGE•• Guideline endGuideline end--users:users: Academic and community Academic and community

family physicians family physicians Interview & Focus Group Sessions Interview & Focus Group Sessions -- 3 parts:3 parts:•• SemiSemi--structured, openstructured, open--ended questions ended questions •• Feedback on contentious guideline Feedback on contentious guideline

recommendations provided by Crecommendations provided by C--CHANGECHANGE•• Ideas for building a guideline Ideas for building a guideline implementabilityimplementability tooltool

Methods Methods –– PHASE 2PHASE 2

AnalysisAnalysis•• Interviews and focus group sessions: audioInterviews and focus group sessions: audio--taped taped

and transcribed verbatimand transcribed verbatim•• Transcripts analyzed independently by 2 Transcripts analyzed independently by 2

investigators using investigators using NvivoNvivo 8, guided by grounded 8, guided by grounded theory methodologytheory methodology

Methods Methods –– PHASE 2PHASE 2

For persons with diabetes and normal urinary albumin excretion aFor persons with diabetes and normal urinary albumin excretion and nd without chronic kidney disease, with BP without chronic kidney disease, with BP ³³130/80 mm Hg, despite 130/80 mm Hg, despite lifestyle interventions: Any of the following medications (listlifestyle interventions: Any of the following medications (listed in ed in alphabetical order) is recommended, with special consideration alphabetical order) is recommended, with special consideration to to ACE inhibitors and ACE inhibitors and ARBsARBs given their additional renal benefits [Grade given their additional renal benefits [Grade D, Consensus, for the special consideration to ACE inhibitors anD, Consensus, for the special consideration to ACE inhibitors and d ARBsARBs]: ACE inhibitor [Grade A, Level 1A (19)]; ARB [Grade A, Level ]: ACE inhibitor [Grade A, Level 1A (19)]; ARB [Grade A, Level 1A (20); Grade B, Level 2, for non1A (20); Grade B, Level 2, for non--left ventricular hypertrophy (20)]; left ventricular hypertrophy (20)]; DHP CCB [Grade B, Level 2 (22)]; DHP CCB [Grade B, Level 2 (22)]; ThiazideThiazide--like diuretic [Grade A, like diuretic [Grade A, Level 1A (22)]; If the above drugs are contraindicated or cannotLevel 1A (22)]; If the above drugs are contraindicated or cannot be be tolerated, a tolerated, a cardioselectivecardioselective beta blocker [Grade B, Level 2 (21)] or beta blocker [Grade B, Level 2 (21)] or nonnon--DHP CCB [Grade B, Level 2 (23)] can be substituted; Additional DHP CCB [Grade B, Level 2 (23)] can be substituted; Additional antihypertensive drugs should be used if target BP levels are nantihypertensive drugs should be used if target BP levels are not ot achieved with standardachieved with standard--dose dose monotherapymonotherapy [Grade C, Level 3[Grade C, Level 3 (12,22)]; Add(12,22)]; Add--on drugs should be chosen from the firston drugs should be chosen from the first--line choices line choices listed above [Grade D, Consensus].listed above [Grade D, Consensus].

Example of a contentious Example of a contentious recommendationrecommendation

Reactions to example recommendation: Reactions to example recommendation: •• ““Confusing, wordy, unclearConfusing, wordy, unclear…””…””•• ““I need another degree to be able to understand thisI need another degree to be able to understand this””•• ““Second line and I am already stoppingSecond line and I am already stopping…”…”•• ““I donI don’’t know what they are actually suggestingt know what they are actually suggesting…”…”

Suggestions for improvement:Suggestions for improvement:•• Bullets for all sectionsBullets for all sections•• Flow diagramFlow diagram•• Put into logical orderPut into logical order•• Recommend only those that are based on strong Recommend only those that are based on strong

evidenceevidence

Preliminary Results Preliminary Results –– PHASE 2PHASE 2

3 main themes of guideline end user 3 main themes of guideline end user perceptions:perceptions:

1.1.Features that are important to include in Features that are important to include in guidelines guidelines

2.2.Facilitators and barriers to guideline useFacilitators and barriers to guideline use3.3.Suggested components to include in a Suggested components to include in a

guideline guideline implementabilityimplementability tooltool

Preliminary Results Preliminary Results –– PHASE 2PHASE 2

•• Clear statements on diagnosis (targets, Clear statements on diagnosis (targets, population), and management (clear decision population), and management (clear decision tree)tree)

•• Quickly accessibleQuickly accessible•• Structured approach Structured approach •• ComprehensiveComprehensive•• ““Boiled downBoiled down””•• ClearClear--cut instructions about what to docut instructions about what to do•• Summary linesSummary lines

Theme 1: Features that are important Theme 1: Features that are important to include in guidelinesto include in guidelines

Theme 2: Facilitators & barriers to Theme 2: Facilitators & barriers to guideline useguideline useTheme Facilitators BarriersWording • Simple, clear, well articulated,

boiled down• In the language of physicians• Provides exceptions and how to

deal with them

• Too many clauses, too long, too much information

• Open, narrative form

Evidence • States quality of evidence • If not indicated or doesn’t fit in with new evidence; what to do with mid- level evidence

Format • Bullet-point summaries• Easy to navigate

• User-interface

Feasibility • Practical and cost efficient • TimeDevelopment • Trustworthy

• Reviewed by front-liners• Written by people not involved in

primary care

Guideline as a whole

• Accessible, up-to-date• Flexible• Validated

• Out of date

•• Include endInclude end--users (i.e., family physicians) in the users (i.e., family physicians) in the development processdevelopment process

•• Provide guidance on dissecting and improving Provide guidance on dissecting and improving recommendationsrecommendations

•• How should endHow should end--users be engaged in the process? users be engaged in the process? •• Convene a working group of family physiciansConvene a working group of family physicians•• Provide an online/electronic platform to involve Provide an online/electronic platform to involve

individual family physiciansindividual family physicians•• Provide CME credits as incentive for involvementProvide CME credits as incentive for involvement•• Barrier would be time and resources of Barrier would be time and resources of

physiciansphysicians

Theme 3: Components to include in an Theme 3: Components to include in an implementabilityimplementability tooltool

ImplementabilityImplementability FrameworkFramework

Guideline Guideline ImplementabilityImplementability

SimplicitySimplicityClarityClarity

UptakeUptake

Guideline Guideline DevelopersDevelopers

PresentationPresentation

EvidenceEvidence use

FeasibilityFeasibility

ActionabilityActionability RiskRisk--benefitbenefit

Guideline Guideline EndEnd--usersusers

CommunicationCommunication

Conceptual design of the Guideline Conceptual design of the Guideline ImplementabilityImplementability Tool (GUIDETool (GUIDE--IT) IT) •• Engage endEngage end--users in the guideline users in the guideline

development process development process –– target both target both developers AND providersdevelopers AND providers

•• Use Use duringduring guideline development, at the guideline development, at the stage where recommendations have been stage where recommendations have been developed and evidencedeveloped and evidence--linked but not yet linked but not yet finalizedfinalized

•• Sequential componentsSequential components

Conceptual design of a guideline Conceptual design of a guideline implementabilityimplementability tool (GUIDEtool (GUIDE--IT) IT)

2.2. Facilitating the involvement Facilitating the involvement of guideline endof guideline end--usersusers

3.3. Tool for assessing the Tool for assessing the implementabilityimplementability of of recommendations and generating recommendations and generating suggestions for improvementsuggestions for improvement

4.4. Process for Process for determining how end determining how end user assessments in user assessments in Step 3 could be used Step 3 could be used to revise the final to revise the final recommendationsrecommendations

Target: Developers

1.1. Process for appraising and Process for appraising and improving recommendations improving recommendations

Target: Developers and End-users

Target: End users

Target: Developers

•• Revealed perceptions of guideline endRevealed perceptions of guideline end--users users about attributes of guideline about attributes of guideline implementabilityimplementability•• Guidelines recommendations need to be clear, quickly Guidelines recommendations need to be clear, quickly

accessible, accessible, ““boiled downboiled down”” and logically structuredand logically structured

•• Resulting attribute themes confirmed attribute Resulting attribute themes confirmed attribute clusters in our preliminary framework clusters in our preliminary framework •• Wording (simple, clear); Evidence (stated and linked); Wording (simple, clear); Evidence (stated and linked);

Format (structure), Feasibility Format (structure), Feasibility

•• Provided feedback to build a conceptual Provided feedback to build a conceptual design of GUIDEdesign of GUIDE--IT IT •• The importance of establishing a working relationship The importance of establishing a working relationship

between guideline developers and end users between guideline developers and end users

Conclusions of PHASE 2Conclusions of PHASE 2

•• Conducting the remaining 2 phases of the Conducting the remaining 2 phases of the qualitative study to build the GUIDEqualitative study to build the GUIDE--IT IT prototype prototype

•• Conduct a usability evaluation of the Conduct a usability evaluation of the prototype to ensure that it meets all end user prototype to ensure that it meets all end user needs and to determine:needs and to determine:•• Which guideline attributes are the most feasible Which guideline attributes are the most feasible

to change during the guideline development to change during the guideline development processprocess

•• Which attributes have the greatest potential for Which attributes have the greatest potential for improving recommendationsimproving recommendations

Next stepsNext steps

KT Canada Collaborators:KT Canada Collaborators:OnilOnil Bhattacharyya Bhattacharyya -- Li Ka Li Ka ShingShing Knowledge InstituteKnowledge InstituteMerrick Merrick ZwarensteinZwarenstein -- Sunnybrook Health SciencesSunnybrook Health Sciences

Sharon Straus Sharon Straus -- Li Ka Li Ka ShingShing Knowledge InstituteKnowledge InstituteJeremy Jeremy GrimshawGrimshaw -- Ottawa Health Research InstituteOttawa Health Research Institute

Ian Graham Ian Graham -- Ottawa Health Research InstituteOttawa Health Research InstituteAndreas Andreas LaupacisLaupacis -- Li Ka Li Ka ShingShing Knowledge InstituteKnowledge Institute

Elizabeth Elizabeth EsteyEstey -- Li Ka Li Ka ShingShing Knowledge InstituteKnowledge InstituteLaureLaure Perrier Perrier -- Li Ka Li Ka ShingShing Knowledge InstituteKnowledge Institute

Questions ??Questions ??

ACKNOWLEDGEMENTSACKNOWLEDGEMENTS