1. Objectives Describe ways in which the EPC program has engaged stakeholders to improve relevance...
-
Upload
alan-mcdowell -
Category
Documents
-
view
233 -
download
0
Transcript of 1. Objectives Describe ways in which the EPC program has engaged stakeholders to improve relevance...
1
Objectives
• Describe ways in which the EPC program has engaged stakeholders to improve relevance and applicability of its products
• Present methods used for stakeholder engagement
• Identify challenges in stakeholder engagement and strategies employed by the EPC program
Evidence-based Practice Centers (EPCs) 11 centers throughout the United States and Canada
Systematically and critically appraise existing research and synthesize knowledgeProduce comparative effectiveness and effectiveness reviews
Scientific Resource Center (SRC)Responsible for scientific methodological work of reviews and other research projects for EPCs and DEcIDES
EPCs & SRC
EPCs & SRCBlue Cross and Blue Shield Association, Technology Evaluation Center, Chicago, ILBrown University, Providence Rhode IslandECRI Institute, Plymouth Meeting, PAJohns Hopkins University, Baltimore, MDKaiser Foundation Research Institute, Oakland, CAOregon Health & Science University, Portland, ORRTI International – University of North Carolina at Chapel Hill, Chapel Hill, NCRAND Corporation , Santa Monica, CAUniversity of Alberta, Edmonton, Edmonton, AlbertaUniversity of Minnesota EPC, Minneapolis, MNVanderbilt University Medical Center, Nashville, TN
Scientific Resource Center (SRC) for EPCs and DEcIDES – Portland VA Research Foundation, Portland OR
Public
Nominate Review Topics
Technical Expert Panel
Key Research Questions
Peer and Public Review
Feedback on Summary Guides
Future Research
Disseminate Reports
Key Informants
Opportunities for Stakeholder Input
EHC Report on Engaging Stakeholders for Development and Prioritization of Future Research Needs
• O’Haire C, McPheeters M, Nakamoto E, LaBrant L, Most C , Lee K, Graham E, Cottrell E, Guise J-M. Engaging Stakeholders To Identify and Prioritize Future Research Needs. Rockville (MD): Agency for Healthcare Research and Quality (US); 2011 Jun. (Methods Future Research Needs Reports, No. 4.) Available from: http://www.ncbi.nlm.nih.gov/books/NBK62565/
Literature Synthesis
Phase III:EPC interviews
(Vanderbilt EPC)
Recruitment and Interview Guide Development
Analysis of Interviews
Key Informant Interviews
Phase II: Key Informant Interviews(Oregon EPC)
Phase I:Literature Summary
(Oregon EPC)
Literature Review
Literature Search
Cumulative Synthesis
EPC Protocol Review
Invitations and EPC Discussion Guide
EPC Discussions
Analysis of Discussions
Stakeholder Identification & Recruitment• Identify pertinent stakeholder groups
(intentional based upon stakeholder groupings)• Ensure a balance of stakeholder perspectives
(i.e. consumer, clinician, researcher, research funder, insurer/payer, manufacturer)
• The results of EPC FRN pilot projects suggest that engaging 6-10 stakeholders is appropriate
• OMB restricts the number of participants in surveys to no more than 9 non-federal employees
Research Needs Development
StakeholdersStakeholder Group Description
Consumer (public, patient, and caregiver)
An individual or advocacy group representing individuals who have a health condition, use health care services and/or who are members of the community
Clinician (Health professional)
Health care or public health provider (e.g., academic, rural/frontier, and community) and medical and/or public health organizations
Social Service Organizations An individual or organization that advances human welfare or social work (e.g. community or condition service providers, school-based programs, justice system)
Policymaker An individual or organization who is involved in health care policy (e.g. local, state, provincial and Federal legislators and staff)
Medical organizations Governmental organizations (e.g. VA, AHRQ, etc.)
Researcher An individual who conducts and/or facilitates research activities in:
Basic, translational, clinical sciences; research methodology, public health or health services, and systematic reviews
Research Funder A public or private organization that funds research (e.g. National Institute of Health, Susan G. Komen Foundation, and American Cancer Society)
Insurer/Payer An organization or agency that pays for health-related goods and services (e.g. Blue Cross Blue Shield, Medicaid, and Medicare) or a business group that pays for health insurance (e.g. employers and government)
Manufacturer A business group that produces health-related items (e.g. pharmaceuticals and medical devices)
Methods of Stakeholder Engagement
Engagement Most common: Focus Groups, Symposia/Conferences
Prioritization Most common: Priority Questionnaire, Consensus
Engagement & Prioritization Simultaneously Priority Survey Delphi
1:1 Meetings
1:1 Meetings (phone)
Focus Groups
GroupCalls Symposia
Consensus or Voting Survey Delphi
Short Timeframe • + – + – – + –
↓ Budget – + • + – – + –Geographic
Barriers • + – + – – + +Differing
Perspectives + + – – + – + +Multiple
Stakeholder Groups + + – – + – + +
↓Research Experience + + + – + – – •
Complex Program + + + – + – • –
+GOOD – FAIR • POOR
Topic Generation
Nominate topics via the web
Topic Development
Clarify nomination
intent
Frame clinical
context of nomination
Topic Refinement
Provide “real world” input
Provide input on clinical logic
Public comment via
Web
Review
Provide methodological & clinical
input
Provide input via peer review
Provide public
comment via Web
Research Needs
Development
Participate in discussions to describe research gaps &
prioritize research
needs
Report Translation & Dissemination
Participate in focus groups
Participate in individual interviews
Provide expert
review of summary
guides
EHC Program Research Reviews
Topics Evidence Review Research D&I
Topic Generation ApproachesNominal Group Process In-person Idea generating Intended to promote group participation
Our experience: In-person brainstorming with flip chart used to document topics Each state had 8 stickers to vote for priority topics:
#1 (2 red dots)#2 (2 green dots) #3 (2 blue dots), and #4 (2 yellow dots)
Weighting system applied to priorities: #1 - 4 points for each vote#2 - 3 points#3 – 2 points#4 – 1 point
Topic Generation Approaches
Medicaid Medical Directors
Analytic Process:Weighting system :
#1 - 4 points #2 - 3 points#3 – 2 points#4 – 1 point
Prevention in Women’s HealthModified Process:
Local Key InformantsThree web meetings:
May 11 – Framework & Initial Topics(Extranet & email topic generation)June 8 – Topic Generation(Evidence Scan)
13 – Final PrioritizationPrioritization
Survey – survey monkeyWeb whiteboard during 3rd meeting
Topic Generation Approaches
Stakeholders Engaged in:Development of CER key questions Refinement of topicReview of Draft
Methods used:Group or individual callsWeb-based forums with survey type format to provide
responses
Scoping CER
Preparation of Stakeholders is critical:
• Assume that all stakeholders need orientation to the CER process
• Very important for them to know where they fit in and what to expect from their feedback
• Familiarity with research methods likely to be very variable
• Conflicts of interest are to be expected with stakeholder groups, but must be disclosed.
Scoping CER
Topic Generation
Nominate topics via the web
Topic Development
Clarify nomination
intent
Frame clinical
context of nomination
Topic Refinement
Provide “real world” input
from provider,
patient, and policy
perspectives
Provide input on clinical
logic
Provide public
comment via Web
Research Review
Provide methodological & clinical input
as a TEP member
Provide input via peer review
Provide public comment via
Web
Research Needs
Development
Participate in
discussions to describe research gaps &
prioritize research
needs
Report Translation & Dissemination
Participate in focus groups
Participate in
individual interviews
Provide expert
review of summary
guides
EHC Program Research Reviews
Stakeholder Preparatory Materials
Where We Arein the Comparative
Effectiveness Review Process
Name of Panel EPC Role & What to Expect
Describe what has occurred to date
• Describe Intent of panel
• Numbers and types of individuals
• Numbers of times and ways they may be engaged
• List ways TEP is especially helpful to process
• The role of the Evidence-based Practice Center (EPC) is to …….
• List Key Principles• Reviews commissioned under
the Effective Healthcare Program (EHC) are posted publically at different stages of the review process, including at the stage of proposed Key Questions and the draft report stage.
Stakeholder Preparatory Materials
• Comparative Effectiveness Reviews (CERs) critically appraise and synthesize bodies of literature and identify areas where research is lacking or insufficient in either quantity or quality
• There are limitations in funds available to support research
• To fully actualize their potential, CERs should be utilized to inform and guide future research
• Future Research Needs (FRN) documents are designed to facilitate this process by engaging stakeholders to prioritize future research topics
Future Research Needs Development
Purpose
Future Research Needs Development
Step 1
•Research gaps from CER interviews
Step 2
•Identify & Recruit Stakeholders
Step 3
•Orient stakeholders to project
Step 4
•Solicit input on gaps, generate topics
Step 5
•Engage stakeholders to prioritize FRN (at least 2 rounds)
Step 6
•Categorize by PICOs
Step 7
•Search for ongoing Research & Reviews
Step 8
•Write up report
Step 9
•Elicit stakeholder feedback
Step
10
•Final report
Combined Methods:• Semi-structured interviews to identify topics• Modified Delphi process for prioritization
Semi-structured interviews Telephone Conference calls Webinars
Delphi#1 Survey
Topics organized by PICOsLikert Scale – 6-point scale of no, low, highest priority
#2 SurveyLikertRankingIndicate top 3 in order
Research Needs Development
Research Needs DevelopmentPriority Final Rank Weighted
Scorea
Studies to test clinical, institutional, or policy interventions to increase access to “safe” TOL 1 72
Research on barriers to providing safe TOL, including factors that limit the ability of hospitals to meet the “immediately available” requirement
2 51
Studies comparing outcomes for mother and infant in settings where physicians are “immediately available” vs. settings where physicians are “readily available”
3 46
Studies to understand best practice models based on institutions that are currently offering safe TOL
4 41
Development of standardized measures for short-and long-term maternal and infant outcomes
5 41
Surveillance to determine long-term clinical outcomes of TOL vs. ERCD 6 38
Research on how patients understand risk, how they respond to different ways of framing risk, and how best to communicate risks of TOL vs. ERCD
7 37
Clinical and policy relevant studies to address the threat of legal liability on practice patterns regarding TOL vs. ERCD
8 34
Development/utilization of a reliable model or tool to predict the probability of successful VBAC for individual women /tool to predict probability of successful VBAC
9 32
Studies to refine, validate, and implement informed consent templates that are informative, reliable, and able to be well documented
10 26
Stakeholder Involvement: Across Research PhasesRESEARCH CONTINUUM
TopicGeneration
StudyDesign Implementation Analysis/
Interpretation Research
PrioritizationDissemination
CliniciansConsumers
FundersInsurers
ResearchersPolicymakers
CliniciansConsumers
FundersInsurers
ManufacturersResearchers
PayersPolicymakers
CliniciansManufacturers
ResearchersPolicymakers
CliniciansConsumers
FundersInsurers
ManufacturersResearchers
PayersPolicymakers
CliniciansConsumersResearchers
Policymakers
CliniciansResearchers
EPC Stakeholder Engagement Expanded Scope of Future Research Needs Beyond CER
Research Needs Development
SummarySeveral options for methods to use when engaging stakeholdersMethod depends on several factors including
• Intent (identifying topics, prioritization)• Resources• Number and variety of stakeholders• What is already known in topic• Experience
Important to outline approach and intent a prioriRelationships with stakeholders are important
• Contact potential stakeholders through multiple venues• Consistent communication and follow-up in the form of emails or
phone calls is important to ensuring a balanced selection of stakeholders and a high response rate
Preparation of stakeholders is critical to define expectations and allow engagement