A Systematic Literature Review Assessing the Role of ... · Potential Role of Systematic Literature...
1
Introduction Systematic Literature Reviews • Systematic literature reviews (SLRs) are a key component in the evidence-based assessment of healthcare interventions. • In contrast to the traditional or narrative literature review, SLRs use a more rigorous and well-defined approach to reviewing the literature in a specific subject area. • Based on the utilization of a predefined structured methodology designed to identify all available research prior to the development of the study hypothesis, SLRs have become valuable resource in the critical impact analysis of therapeutic interventions across multiple research studies. • Existing as standalone evidence, SLRs are designed to address broader clinical questions than single empirical studies, enabling such reviews to be prioritized above other research designs in the “hierarchy of evidence” due to the potential to provide practical evidence-based conclusions. 1 • Specifically, SLRs employ a prespecified and transparent approach for searching and assessing the literature to provide greater clinical insights (Figure 1). Clinical Trial Protocols • Central to the implementation of clinical trials is the development of a clinical trial protocol, a document that describes the background, rationale, objectives, design, methodology, statistical considerations, and organization of a clinical research project. • The International Congress on Harmonization (ICH) Good Clinical Practice Guidelines identify specific topics that should generally be included in a clinical trial protocol, including background to the disease state, goals and objectives of the study, and a proposed study hypothesis. 2 • In a similar, but abbreviated manner, each clinical trial protocol must be approved by an institutional review board (IRB), independent ethics committee (IEC), ethical review board (ERB), or research ethics board (REB), who reviewing the study hypothesis and methods proposed for research to ensure there are no ethical violations. Potential Role of Systematic Literature Review in Clinical Trial Development • It has been proposed that one reason for the failure of clinical trials may be due to an inaccurate study hypothesis, or the development of a clinical trial protocol that is designed to answer a different question. 3-7 • Utilizing an SLR as part of the discovery phase may assist in identifying the full scope of scientific information available regarding a potential clinical trial design, providing valuable guidance to the development of the overall study hypothesis. • The current use of SLRs in the development of clinical trial protocols is unknown. Systemic Limitations for SLRs to Support Clinical Trials • The main source of information for the IRBs, IECs, ERBs, and REBs for approving a clinical trial protocol is the investigator’s brochure, which is a comprehensive document containing all of the preclinical and clinical information regarding an investigational product for the risk-benefit assessment to justify its use in humans. – For early-phase human trials (phase 1/2), the investigator’s brochure should provide a detailed summary of all the available evidence regarding the nonclinical pharmacology, toxicology, pharmacokinetics, and metabolism of an investigational product, as well as any corresponding evidence in humans, if available. 8 – The summary of nonclinical pharmacology should incorporate studies assessing potential therapeutic activity and safety, including preclinical pharmacodynamics studies that evaluate efficacy in animal models. 8 – Recent studies of investigator’s brochures, preclinical efficacy publications, and risk–benefit assessments have demonstrated significant deficits in the study design and reporting of preclinical efficacy studies, 9 as well as publication biases in both the investigator’s brochures and peer-reviewed journals. 10 – Issues with the conduct and reporting of preclinical efficacy may be a leading cause of the poor probably of success for clinical trials in many therapeutic fields. 11,12 Hypothesis • There is poor utilization of SLRs in the development of clinical trial protocols. Methods • Authors conducted a systematic literature search of all National Library of Medicine databases to determine if study investigators are conducting an SLR when designing the protocol for phase 2/3 clinical trials. Protocol • A brief overview of the protocol developed and used to identify SLRs conducted as part of trial protocol development can be found in Table 1. Table 1. Abridged SLR Protocol Literature Database National Library of Medicine Search Type Boolean-based AND/OR analysis Search Terms • Systematic Literature Review – Systematic Literature Review; Systematic Review • Clinical Trial – Clinical Trial; Clinical Trial, Phase II; Clinical Trial, Phase III; Comparative Study; Controlled Clinical Trial; Multicenter Study; Observational Study; Randomized Controlled Trial • Study Protocol – Protocol; Study Protocol; Trial Protocol MeSH Headings All Study Population All Age All Ethnicity All Species Human Language English Publication Type All Journal Category All Timeframe January 1, 1990, to December 31, 2017 Search Exclusions • Clinical Trial – Clinical Trial, Phase I; Preclinical • Publication Type – (Non-systematic) Literature Review; Case Studies; Secondary Analysis • Search Terms – None • Population – None Results • The flowchart for the results of the SLR is presented in Figure 2. Figure 2. SLR Flow Chart Literature Search Databases: Study Population: Age: Type: Timing National Library of Medicine, N=603,953 All Child (birth-18 years); Newborn (birth-1 month); Infant (birth-23 months); Preschool Child (2-5 years); Child (6-12 years); Adolescent (13-18 years); Adult (18-65 years); Elderly (65+ years) Boolean-based AND/OR analysis January 1, 1990, to December 31, 2017 Articles reporting the utilization of a systematic literature review in the development of a clinical trial protocol (n=16) Record includes “systematic literature review” as search term (n=16,856) Abstract body and/or title includes the search term “systematic literature review” (n=7,995) Records excluded for lack of an appropriate search term (n=587,097) Records excluded for not including search term in body/title (n=8,861) Reviewed Eligibility Screening Identification Records excluded for various reasons (n=356) • Includes discussion of systematic literature review, but not utilized in protocol development of clinical trial (n=211) • Includes discussion of systematic literature review results, but not utilized in the context of a clinical trial (n=85) • Scientific review article (n=29) • Systematic literature review (n=19) • Secondary analysis of clinical trial (n=7) • Other (=5) Oncology (n=3) Emergency Medicine, Cardiovascular Disease, Dermatology (n=2 each) Endocrinology, gastroenterology, hematology, internal medicine, mental health, nephrology, rheumatology (n=1 each) Full-text articles assessed for eligibility (n=370) • A total of 603,953 publications were identified under the protocol for the search terms “clinical trials” and “study protocols.” • With the addition of the “systematic literature review” term, the search identified a total of 16,856 publications. • Of the 16,856 publications, a total of 7,995 utilized the terms “systematic literature review” or “systematic review” in the title or abstract body. • Analysis of these 7,995 publications identified that only 370 discussed systematic evidence and clinical trial design within the context of the overall article. – Over time there has been an increase in these types of publications, though the overall total number is still relatively low (Figure 3). Figure 3. Publication Timeline for Peer-reviewed Publications Meeting Search Criteria (N=370) 1994 0 5 10 15 20 25 30 35 40 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 Publication Year Number of Publications 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 • Retrieval of 370 articles was conducted to challenge the study hypothesis that there is poor utilization of SLRs in the development of clinical trial protocols. • Of the 370 articles identified for in-depth review, only 16 publications reported the results of an SLR that was conducted as part of the design of an interventional clinical trial. – Oncology: n=3 – Emergency medicine, cardiovascular disease, dermatology: n=2 each – Endocrinology, gastroenterology, hematology, internal medicine, mental health, nephrology, rheumatology: n=1 each, with 7 focusing on a therapeutic intervention. • The primary purpose of the 16 SLRs was to justify the unmet need asked as part of the development of the clinical trial hypothesis. • For perspective, as of April 1, 2018, there are 14,684 phase 2 and 7,689 phase 3 clinical trials registered on the ClinicalTrials.gov database that are currently recruiting, enrolling by invitation, or active but not recruiting. Conclusions • This SLR supports the study hypothesis, indicating that SLRs are substantially underutilized in the development of clinical trial protocols, in line with other analyses. 13,14 • Despite the well-documented importance of SLRs in evidence-based medicine and clinical practice decision-making, there is limited evidence to support the hypothesis that SLRs are being utilized when designing clinical trial protocols. • A greater awareness of the potential role of SLRs in the design of clinical trial protocols is warranted. References 1. Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016;21(4):125-127. 2. European Medicines Agency. Guideline for good clinical practice E6(R2). 2016. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500002874. pdf. Accessed April 12, 2018. 3. Stein DG. Embracing failure: What the Phase III progesterone studies can teach about TBI clinical trials. Brain Inj. 2015;29(11):1259-1272. 4. Howard RB, Sayeed I, Stein DG. Suboptimal Dosing Parameters as Possible Factors in the Negative Phase III Clinical Trials of Progesterone for Traumatic Brain Injury. J Neurotrauma. 2017;34(11):1915-1918. 5. Singer W, Low PA. Optimizing clinical trial design for multiple system atrophy: lessons from the rifampicin study. Clin Auton Res. 2015;25(1):47-52. 6. Gheorghiade M, Vaduganathan M, Greene SJ, et al. Site selection in global clinical trials in patients hospitalized for heart failure: perceived problems and potential solutions. Heart Fail Rev. 2014;19(2):135-152. 7. Gelenberg AJ, Thase ME, Meyer RE, et al. The history and current state of antidepressant clinical trial design: a call to action for proof-of-concept studies. J Clin Psychiatry. 2008;69(10):1513-1528. 8. US Food and Drug Administration. E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1): Guidance for Industry. 2018. https://www.fda.gov/downloads/Drugs/Guidances/ UCM464506.pdf. Accessed April 12, 2018. 9. Wieschowski S, Chin WWL, Federico C, et al. Preclinical efficacy studies in investigator brochures: Do they enable risk-benefit assessment? PLoS Biol. 2018;16(4):e2004879. 10. Cohen D. Oxford TB vaccine study calls into question selective use of animal data. BMJ. 2018;360. 11. Wong CH, Siah KW, Lo AW. Estimation of clinical trial success rates and related parameters. Biostatistics (Oxford, England). 2018 12. Lowenstein PR, Castro MG. Uncertainty in the translation of preclinical experiments to clinical trials. Why do most phase III clinical trials fail? Curr Gene Ther. 2009;9(5):368-374. 13. Pandis N, Fleming PS, Koletsi D, Hopewell S. The citation of relevant systematic reviews and randomised trials in published reports of trial protocols. Trials. 2016;17(1):581. 14. Clarke M, Hopewell S, Chalmers I. Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting. Lancet. 2010;376(9734):20-21. Further Readings • Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Info Libr J. 2010;27(2):114-122. • Page D. Systematic literature searching and the bibliographic database haystack. Electronic Journal of Business Research Methods. 2008;6(2):171-179. • Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118-121. • Cronin P, Ryan F, Coughlan M. Undertaking a literature review: a step-by-step approach. Br J Nurs. 2008;17(1):38-43z • Cooper H, Hedges LV, Valentine JC. The Handbook of Research Synthesis and Meta-Analysis. NY, NY: Russell Sage Foundation; 2009. A Systematic Literature Review Assessing the Role of Systematic Literature Reviews in Clinical Trial Design Philip Sjostedt, Cindy Busch, and Nicole A. Coolbaugh The Medicine Group, LLC, New Hope, PA, USA 18938 [email protected] Poster 38 Figure 1. Process Map of Conducting a Systematic Literature Review Identify the issue and determine the research question Write a plan for the review (ie, protocol) Conduct search for studies Sift and select studies Extract data from the studies Assess the quality of the studies Combine the data (synthesis or meta-analysis) Discuss and conclude overall findings Systematic review Dissemination
Transcript of A Systematic Literature Review Assessing the Role of ... · Potential Role of Systematic Literature...
IntroductionSystematic Literature Reviews• Systematic literature reviews (SLRs) are a key component in the evidence-based assessment of healthcare interventions.
• In contrast to the traditional or narrative literature review, SLRs use a more rigorous and well-de� ned approach to reviewing the literature in a speci� c subject area.
• Based on the utilization of a prede� ned structured methodology designed to identify all available research prior to the development of the study hypothesis, SLRs have become valuable resource in the critical impact analysis of therapeutic interventions across multiple research studies.
• Existing as standalone evidence, SLRs are designed to address broader clinical questions than single empirical studies, enabling such reviews to be prioritized above other research designs in the “hierarchy of evidence” due to the potential to provide practical evidence-based conclusions.1
• Speci� cally, SLRs employ a prespeci� ed and transparent approach for searching and assessing the literature to provide greater clinical insights (Figure 1).
Clinical Trial Protocols• Central to the implementation of clinical trials
is the development of a clinical trial protocol, a document that describes the background, rationale, objectives, design, methodology, statistical considerations, and organization of a clinical research project.
• The International Congress on Harmonization (ICH) Good Clinical Practice Guidelines identify speci� c topics that should generally be included in a clinical trial protocol, including background to the disease state, goals and objectives of the study, and a proposed study hypothesis.2
• In a similar, but abbreviated manner, each clinical trial protocol must be approved by an institutional review board (IRB), independent ethics committee (IEC), ethical review board (ERB), or research ethics board (REB), who reviewing the study hypothesis and methods proposed for research to ensure there are no ethical violations.
Potential Role of Systematic Literature Review in Clinical Trial Development• It has been proposed that one reason for
the failure of clinical trials may be due to an inaccurate study hypothesis, or the development of a clinical trial protocol that is designed to answer a different question.3-7
• Utilizing an SLR as part of the discovery phase may assist in identifying the full scope of scienti� c information available regarding a potential clinical trial design, providing valuable guidance to the development of the overall study hypothesis.
• The current use of SLRs in the development of clinical trial protocols is unknown.
Systemic Limitations for SLRs to Support Clinical Trials• The main source of information for the IRBs, IECs, ERBs, and REBs for approving a clinical trial protocol is the investigator’s
brochure, which is a comprehensive document containing all of the preclinical and clinical information regarding an investigational product for the risk-bene� t assessment to justify its use in humans.
– For early-phase human trials (phase 1/2), the investigator’s brochure should provide a detailed summary of all the available evidence regarding the nonclinical pharmacology, toxicology, pharmacokinetics, and metabolism of an investigational product, as well as any corresponding evidence in humans, if available.8
– The summary of nonclinical pharmacology should incorporate studies assessing potential therapeutic activity and safety, including preclinical pharmacodynamics studies that evaluate ef� cacy in animal models.8
– Recent studies of investigator’s brochures, preclinical ef� cacy publications, and risk–bene� t assessments have demonstrated signi� cant de� cits in the study design and reporting of preclinical ef� cacy studies,9 as well as publication biases in both the investigator’s brochures and peer-reviewed journals.10
– Issues with the conduct and reporting of preclinical ef� cacy may be a leading cause of the poor probably of success for clinical trials in many therapeutic � elds.11,12
Hypothesis• There is poor utilization of SLRs in the development of clinical trial protocols.
Methods• Authors conducted a systematic literature search of all National Library of Medicine databases to determine if study investigators
are conducting an SLR when designing the protocol for phase 2/3 clinical trials.
Protocol• A brief overview of the protocol developed and used to identify SLRs conducted as part of trial protocol development can be
found in Table 1.
Table 1. Abridged SLR Protocol
Literature Database National Library of Medicine
Search Type Boolean-based AND/OR analysis
Search Terms • Systematic Literature Review – Systematic Literature Review; Systematic Review
• Clinical Trial – Clinical Trial; Clinical Trial, Phase II; Clinical Trial, Phase III; Comparative Study; Controlled Clinical Trial;
Multicenter Study; Observational Study; Randomized Controlled Trial• Study Protocol
– Protocol; Study Protocol; Trial Protocol
MeSH Headings All
Study Population All
Age All
Ethnicity All
Species Human
Language English
Publication Type All
Journal Category All
Timeframe January 1, 1990, to December 31, 2017
Search Exclusions • Clinical Trial – Clinical Trial, Phase I; Preclinical
• Publication Type – (Non-systematic) Literature Review; Case Studies; Secondary Analysis
• Search Terms – None
• Population – None
Results• The � owchart for the results of the SLR is presented in Figure 2.
Figure 2. SLR Flow Chart
Literature SearchDatabases:Study Population:Age:
Type:Timing
National Library of Medicine, N=603,953AllChild (birth-18 years); Newborn (birth-1 month); Infant (birth-23 months); Preschool Child (2-5 years); Child (6-12 years); Adolescent (13-18 years); Adult (18-65 years); Elderly (65+ years)
Boolean-based AND/OR analysisJanuary 1, 1990, to December 31, 2017
Articles reporting the utilization of a systematic literature review in the development of a clinical trial protocol(n=16)
Record includes “systematic literature review”as search term (n=16,856)
Abstract body and/or title includes the search term“systematic literature review” (n=7,995)
Records excluded for lack of an appropriate search term(n=587,097)
Records excluded for not including search term in body/title(n=8,861)
Rev
iew
edE
ligib
ility
Scr
eeni
ngId
enti�
catio
n
Records excluded for various reasons (n=356)• Includes discussion of systematic literature review, but not utilized in protocol
development of clinical trial (n=211)• Includes discussion of systematic literature review results, but not utilized
in the context of a clinical trial (n=85)• Scienti�c review article (n=29)• Systematic literature review (n=19)• Secondary analysis of clinical trial (n=7)• Other (=5)
Oncology(n=3)
Emergency Medicine, Cardiovascular Disease,Dermatology (n=2 each)
Endocrinology, gastroenterology, hematology, internal medicine,mental health, nephrology, rheumatology (n=1 each)
Full-text articles assessed for eligibility (n=370)
• A total of 603,953 publications were identi� ed under the protocol for the search terms “clinical trials” and “study protocols.”
• With the addition of the “systematic literature review” term, the search identi� ed a total of 16,856 publications.
• Of the 16,856 publications, a total of 7,995 utilized the terms “systematic literature review” or “systematic review” in the title or abstract body.
• Analysis of these 7,995 publications identi� ed that only 370 discussed systematic evidence and clinical trial design within the context of the overall article.
– Over time there has been an increase in these types of publications, though the overall total number is still relatively low (Figure 3).
Figure 3. Publication Timeline for Peer-reviewed Publications Meeting Search Criteria (N=370)
19940
5
10
15
20
25
30
35
40
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006
Publication Year
Num
ber
of
Pub
licat
ions
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017
• Retrieval of 370 articles was conducted to challenge the study hypothesis that there is poor utilization of SLRs in the development of clinical trial protocols.
• Of the 370 articles identi� ed for in-depth review, only 16 publications reported the results of an SLR that was conducted as part of the design of an interventional clinical trial.
– Oncology: n=3 – Emergency medicine, cardiovascular disease, dermatology: n=2 each – Endocrinology, gastroenterology, hematology, internal medicine, mental health, nephrology, rheumatology: n=1 each, with
7 focusing on a therapeutic intervention.
• The primary purpose of the 16 SLRs was to justify the unmet need asked as part of the development of the clinical trial hypothesis.
• For perspective, as of April 1, 2018, there are 14,684 phase 2 and 7,689 phase 3 clinical trials registered on the ClinicalTrials.gov database that are currently recruiting, enrolling by invitation, or active but not recruiting.
Conclusions• This SLR supports the study hypothesis, indicating that SLRs are substantially underutilized in the development of clinical trial
protocols, in line with other analyses.13,14
• Despite the well-documented importance of SLRs in evidence-based medicine and clinical practice decision-making, there is limited evidence to support the hypothesis that SLRs are being utilized when designing clinical trial protocols.
• A greater awareness of the potential role of SLRs in the design of clinical trial protocols is warranted.
References 1. Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016;21(4):125-127.2. European Medicines Agency. Guideline for good clinical practice E6(R2). 2016. http://www.ema.europa.eu/docs/en_GB/document_library/Scienti� c_guideline/2009/09/WC500002874.
pdf. Accessed April 12, 2018.3. Stein DG. Embracing failure: What the Phase III progesterone studies can teach about TBI clinical trials. Brain Inj. 2015;29(11):1259-1272.4. Howard RB, Sayeed I, Stein DG. Suboptimal Dosing Parameters as Possible Factors in the Negative Phase III Clinical Trials of Progesterone for Traumatic Brain Injury. J Neurotrauma.
2017;34(11):1915-1918.5. Singer W, Low PA. Optimizing clinical trial design for multiple system atrophy: lessons from the rifampicin study. Clin Auton Res. 2015;25(1):47-52.6. Gheorghiade M, Vaduganathan M, Greene SJ, et al. Site selection in global clinical trials in patients hospitalized for heart failure: perceived problems and potential solutions. Heart Fail
Rev. 2014;19(2):135-152.7. Gelenberg AJ, Thase ME, Meyer RE, et al. The history and current state of antidepressant clinical trial design: a call to action for proof-of-concept studies. J Clin Psychiatry.
2008;69(10):1513-1528.8. US Food and Drug Administration. E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1): Guidance for Industry. 2018. https://www.fda.gov/downloads/Drugs/Guidances/
UCM464506.pdf. Accessed April 12, 2018.9. Wieschowski S, Chin WWL, Federico C, et al. Preclinical ef� cacy studies in investigator brochures: Do they enable risk-bene� t assessment? PLoS Biol. 2018;16(4):e2004879.10. Cohen D. Oxford TB vaccine study calls into question selective use of animal data. BMJ. 2018;360.11. Wong CH, Siah KW, Lo AW. Estimation of clinical trial success rates and related parameters. Biostatistics (Oxford, England). 201812. Lowenstein PR, Castro MG. Uncertainty in the translation of preclinical experiments to clinical trials. Why do most phase III clinical trials fail? Curr Gene Ther. 2009;9(5):368-374.13. Pandis N, Fleming PS, Koletsi D, Hopewell S. The citation of relevant systematic reviews and randomised trials in published reports of trial protocols. Trials. 2016;17(1):581.14. Clarke M, Hopewell S, Chalmers I. Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting. Lancet. 2010;376(9734):20-21.
Further Readings• Papaioannou D, Sutton A, Carroll C, Booth A, Wong R. Literature searching for social science systematic reviews: consideration of a range of search techniques. Health Info Libr J.
2010;27(2):114-122.• Page D. Systematic literature searching and the bibliographic database haystack. Electronic Journal of Business Research Methods. 2008;6(2):171-179.• Khan KS, Kunz R, Kleijnen J, Antes G. Five steps to conducting a systematic review. J R Soc Med. 2003;96(3):118-121.• Cronin P, Ryan F, Coughlan M. Undertaking a literature review: a step-by-step approach. Br J Nurs. 2008;17(1):38-43z• Cooper H, Hedges LV, Valentine JC. The Handbook of Research Synthesis and Meta-Analysis. NY, NY: Russell Sage Foundation; 2009.
A Systematic Literature Review Assessing the Role of Systematic Literature Reviews in Clinical Trial DesignPhilip Sjostedt, Cindy Busch, and Nicole A. CoolbaughThe Medicine Group, LLC, New Hope, PA, USA 18938 [email protected]
Poster 38
Figure 1. Process Map of Conducting a Systematic Literature Review
Identify the issue and determine the research question
Write a plan for the review (ie, protocol)
Conduct search for studies
Sift and select studies
Extract data from the studies
Assess the quality of the studies
Combine the data(synthesis or meta-analysis)
Discuss and concludeoverall �ndings
Systematic review
Dissemination
