A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE...
-
Upload
tracey-dean -
Category
Documents
-
view
216 -
download
1
Transcript of A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE...
![Page 1: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/1.jpg)
A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT
Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,Robert Kunkle, M.A., Robert M. Plovnick, M.D., M.S.
Executive Committee for Practice Guidelines
American Psychiatric Association
![Page 2: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/2.jpg)
Disclosures• The authors are with the Practice Guidelines Project of
the American Psychiatric Association• Dr. Yager is Professor of Psychiatry, University of
Colorado School of Medicine• Dr. Fochtmann is Professor of Psychiatry, School of
Medicine, SUNY Stony Brook• Mr. Kunkle is Director, Practice Guidelines, APA• Dr. Plovnick is Director, Quality Care, APA
• None of the authors has any financial disclosures pertinent to this presentation
![Page 3: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/3.jpg)
Guideline First Edition Second Edition Third Edition
Alzheimer’s 1997 2007
ASD & PTSD 2004
Bipolar Disorder 1994 2002
Borderline Personality 2001
Delirium 1999
Eating Disorders 1993 2000 2006
HIV/AIDS 2000
MDD 1993 2000 2010
OCD 2007
Panic Disorder 1998 2009
Psych Evaluation 1995 2006
Schizophrenia 1997 2004
SUDs 1995 2006
Suicidal Behaviors 2003
Available APA Guidelines
![Page 4: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/4.jpg)
Major Themes• Positive aspects of rigorous systematic reviews
• Limitations, Challenges and Dangers of guidelines based on limited high quality evidence
• Opportunities for AHRQ to help professional associations to develop trustworthy guidelines
![Page 5: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/5.jpg)
Positive aspects of rigorous systematic reviews
• Where robust evidence bases exit, systematic reviews are worth the effort and cost.
• Such reviews will help develop trustworthy guidelines in the following ways:
![Page 6: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/6.jpg)
Rigorous systematic reviews can help • equilibrate ratings regarding strength of evidence and, in turn, strength of
recommendations, across guidelines
• differentiate strong from weak recommendations
• determine treatment priorities
• reduce vague, underspecified recommendations
• reduce biases resulting from informal group dynamics, differential power status of “experts”, and potential competing interests
• increase transparency regarding evidence-base to guideline users
• meet IOM standards
![Page 7: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/7.jpg)
But, let’s face it… relying primarily on systematic evidence-based reviews poses……
•Limitations
•Challenges
•Dangers…
![Page 8: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/8.jpg)
Limitations of Relying Primarily on Systematic Evidence-Based Reviews
for Guideline Development
• For many important clinical questions there’s not much high quality evidence out there.
• Example from eating disorders
![Page 9: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/9.jpg)
A Sidebar on Evidence-Based Medicine:Strength of Recommendations in
Eating Disorders Practice Guideline
• APA Guideline Recommendation
• Level [I] : 118• Level [II] : 57• Level [III] : 21
• British NICE Recommendation
• Grade A: 3• Grade B: 13• Grade C: 85
![Page 10: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/10.jpg)
National Institute for Clinical Excellence (NICE)Recommendations
![Page 11: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/11.jpg)
Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomized controlled trials
(BMJ 327 : 1459 Published 18 December 2003)
![Page 12: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/12.jpg)
Limitations• The more precise the PICO-TS question with regard to
patient population characteristics, comparative treatments, and meaningful outcomes, the less high quality evidence is available.
• Think co-occurring medical and psychiatric comorbidities, patient preferences, prior longitudinal illness course, and treatment experience.
![Page 13: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/13.jpg)
Evidence for PICO-TS questions?• For patients with paranoid schizophrenia who have developed
tardive dyskinesia on first generation antipsychotic medications and who suffer from obesity and type II diabetes mellitus, which of the following treatments (clozapine vs. aripiprazole vs. olanzapine vs. perphenazine) will result in the most desirable outcomes with respect to psychiatric symptoms, psychiatric impairment, medical morbidity and mortality?
• For currently depressed patients with bipolar disorder type I, with co-occurring alcohol and marijuana dependence, which of the following combinations of medications and psychosocial interventions will result in the most desirable outcomes with respect to psychiatric symptoms, psychiatric impairment, medical morbidity and mortality?
![Page 14: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/14.jpg)
Limitations• When you factor in patient/family preferences and access
to/costs of care, don’t expect to find much.
• There will always be lots of wiggle room in framing assumptions and interpreting results.
• Patients and practitioners seek clinical guidance for complex questions where randomized controlled trials are unlikely to ever be conducted. Are we supposed to ignore those needs?
![Page 15: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/15.jpg)
Limitations• Common clinical questions may vary depending upon the
level of generalist/specialist/subspecialist using the guidelines.
• AHRQ funded EBM reviews may not address situations where lack of access limits clinicians’ abilities to offer best practices.
![Page 16: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/16.jpg)
Limitations
• AHRQ reports may focus on particular treatments (e.g. second generation antipsychotics) rather than on comparisons needed for choosing a treatment approach best suited for an individual patient.
![Page 17: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/17.jpg)
Challenges of Relying Primarily on Systematic Evidence-Based Reviews for Guideline
Development
• AHRQ evidence reports may cover only a fraction of a professional societies' guideline development efforts. Do we have “trustworthy” guidelines and “good enough” guidelines?
• How are professional associations expected to pay for guideline development processes where AHRQ is not funding an EBC effort and/or where high quality evidence is not available?
![Page 18: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/18.jpg)
Challenges: Determining when the juice is worth the squeeze.
• What degree of pre-screening should be conducted and what type of evidence should be required before an EBC or PA heads off on detailed review?
• For example, should a minimum of two high quality RCTs that specifically address a PICO-TS question be required before embarking on a detailed search?
• How closely to the precise PICO questions should these studies be expected to adhere?
![Page 19: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/19.jpg)
Dangers of Relying Primarily on Systematic Evidence-Based Reviews for
Guideline Development
• Systematic reviews may ignore or minimize important systematic biases, e.g. clinical trials design biases.
• The IOM Systematic Review recommendations are admittedly riddled with strong recommendations for which there is no evidence base! A cynic might think that COI was at play.
![Page 20: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/20.jpg)
Dangers of Relying Primarily on Systematic Evidence-Based Reviews for
Guideline Development
• “Without an assessment of hard, irrefutable measures of clinical decision-making that include individual preferences for treatment, decisions about the appropriateness of clinical treatments and variations of care cannot be made.”
• Livingston EH, McNutt RA. The Hazards of Evidence-Based Medicine: Assessing Variations in Care. JAMA. 2011;306(7):762-763. doi: 10.1001/jama.2011.1181
![Page 21: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/21.jpg)
Opportunities for AHRQ to help Professional Associations Developing Practice Guidelines
• Establish and disseminate criteria for trustworthy pre-screening searches to determine potential value of full-scale systematic reviews.
• Fund grants to professional associations and other guideline developing groups for conducting pre-screening searches.
![Page 22: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/22.jpg)
Opportunities for AHRQ to help Professional Associations Developing Practice Guidelines
• When extensive systematic reviews are conducted:
• Organize and distribute evidence tables to enable guideline developers to use them to address clinically different questions than those for which they were initially created.
• Require EBC evidence tables to link to the PubMed abstracts for all related RCTs.
• Require EBC evidence tables to contain clinically relevant calculations, effect sizes, NNT, NNH, as well as proportion of individuals experiencing particular adverse effects and global outcomes.
![Page 23: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/23.jpg)
Opportunities for AHRQ to help Professional Associations Developing Practice Guidelines
• More broadly:
• Work with other agencies to require investigators to post data in a manner suitable for incorporation into formal medical decision analyses.
• Work with other agencies to require investigators to post individual subject or group descriptive data (means, standard deviations) split according to key factors such as age, sex, race/ethnicity even when the numbers are too small to analyze otherwise.
![Page 24: A PROFESSIONAL ASSOCIATION PERSPECTIVE ON SYSTEMATIC REVIEWS AND THEIR UTILITY FOR GUIDELINE DEVELOPMENT Joel Yager, M.D., Laura J. Fochtmann, M.D., M.S.,](https://reader035.fdocuments.net/reader035/viewer/2022062516/56649db35503460f94aa36b5/html5/thumbnails/24.jpg)
Opportunities for AHRQ to help Professional Associations and Other Groups Developing
Practice Guidelines
• For circumstances when only low-quality (or no) data is available:
• Help develop acceptable standardized methods for acquiring expert consensus
• Help develop policies and procedures authorizing the development of trustworthy guidelines based on available evidence plus expert consensus.