Supporting syntheses of the literature

SHINE

Quality – The Whole Picture

26th November 2004

Content• Definition and purpose• Types of syntheses• The role of the information specialist

– Identifying previous reviews, scoping searches, retrieval of primary studies

• Challenges of the role

Definition and purpose

• Definition– reduce large quantities of primary information into ‘palatable

pieces for digestion’ by decision makers (Mulrow, 1995)– produce reliable results on which to base decisions by using

robust processes to locate, appraise and synthesise research evidence (Song et al., 2000)

• Purpose– Quality improvement

Systematic reviews

• ‘…a review [that] has been prepared using some kind of systematic approach to minimising biases and random errors, and that the components of the approach will be documented in the materials and methods section.’ (Chalmers & Altman, 1995)

• Organisations undertaking systematic reviews– Cochrane Collaboration– Campbell Collaboration– Centre for Reviews and Dissemination

Health Technology Assessments

• ‘a multidisciplinary field of policy analysis that studies the medical, social, ethical, and economic implications of the development, diffusion and use of a health technology’ (INAHTA, 2000)

• Organisations undertaking HTAs– NHS QIS– NICE– INAHTA

Guidelines

• ‘Clinical guidelines are systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.’ (www.sign.ac.uk)

• Organisations preparing evidence-based guidelines– SIGN– NICE– GIN

One topic: three syntheses

• Systematic review– In alcohol dependence, is acamprosate or naltrexone more

effective for preventing relapse?

• HTA– Prevention of relapse in alcohol dependence

• Guideline– The management of harmful drinking and alcohol

dependence in primary care

The role of the information specialist

• To comprehensively identify studies to answer the review question(s)

• Stages– Identifying previous reviews– Undertaking scoping searches– Retrieving primary studies

Identifying previous reviews

• Preliminary evaluation of the evidence base– Has this question already been answered?

– If not, can an answerable question be formulated?

– Is there a reasonable evidence base?

• CDSR, DARE, HTA database, Guidelines Finder, National Guideline Clearing House

• Ongoing reviews• Bibliographic databases, journals and websites

Scoping searches

• ‘Scoping searches establish the feasibility of a review question and investigate how searching and appraisal can be operationalised.’ (Booth & Fry-Smith, 2003)

– establish the volume and quality of the primary literature– aid development of the review protocol– indicate how much the review might cost

Retrieval of primary studies

• Formulating clear search questions• Recognising the ‘best’ evidence to answer the

question(s)• Identifying the most likely sources to search• Searching information sources effectively

(Snowball, 2002)

Formulating clear search questions

• The product– The ‘well-built’ clinical question (Richardson et al., 1995)

– The ‘well-formulated’ question (Counsell, 1997)

– The ‘anatomy’ of a question (Sackett et al., 1997)

• The process– Focusing the question (Oxman et al., 1993)

– Formulating the question (Counsell, 1997)

– Formulating the problem (Cochrane Handbook)

Slide reproduced with kind permission from Andrew Booth

Formulating clear search questions

• ‘Ask a poor question and you will get a poor review. A clear question also helps the reader rapidly assess whether the review is relevant to his or her own clinical practice’ (Counsell, 1997).

• Clearly framed questions ‘guide much of the review process including strategies for locating and selecting studies or data, for critically appraising their relevance and validity, and for analysing variation among their results’ (Cochrane Handbook).

Slide reproduced with kind permission from Andrew Booth

Formulating clear search questions

• POPULATION/PATIENT – who? • INTERVENTION/EXPOSURE – what? • COMPARISON – compared with?• OUTCOME – with what result?

• PICO, PIOC or PECO

Slide reproduced with kind permission from Andrew Booth

Worked example 1• Population/Patients: are persons with alcohol

dependence

• Intervention: who are treated with acamprosate

• Comparison: compared with those treated with naltrexone

• Outcome(s): less likely to relapse (into heavy drinking and dependence)?

Worked example 2• Population/Patients: should pregnant women be

offered

• Intervention: first trimester ultrasound scanning (nuchal translucency measurement)

• Comparison: or second trimester serum screening

• Outcome(s): accuracy of risk assessment for trisomy 21 (Down’s Syndrome)?

Translating PICO into a search strategy

POPULATION

Alcoholics

OR

Alcoholism

And

INTERVENTION

Acamprosate

OR

Campral

And

COMPARISON

Naltrexone

OR

Trexan

And

OUTCOME

Temperance

OR

Sobriety

Slide reproduced with kind permission from Andrew Booth

Is PICO effective (1)?

• Statistically significant correlation between use of PICO-structured form and complexity of search strategy (P = 0.002) and between clinical requests handled by PICO-structured form and fewer items retrieved (P = 0.028). However, librarians preferred minimally structured forms to PICO-structured forms in every dimension except informativeness.

(Booth et al., 2000)

Slide reproduced with kind permission from Andrew Booth

Recognising the ‘best’ evidence to answer the question

• Therapy• Diagnosis, screening• Aetiology• Prognosis, natural history• Cost-effectiveness• Qualitative

(McKibbon, 1999)

Recognising the ‘best’ evidence to answer the question

• T: In alcohol dependence, is acamprosate or naltrexone more effective for preventing relapse?

• D: What is the accuracy of screening for alcohol problems in primary care?

• A: What are the environmental risk factors associated with binge drinking in college students?

• P: What is the long-term prognosis of patients with alcoholic cardiomyopathy following total abstinence?

• CE: Are brief interventions cost-effective for problem drinkers?

• Q: What are adolescents' perceptions of parental alcoholism?

Identifying the sources to search (1)

• Subject– General, eg: MEDLINE, EMBASE– Subject-specific, eg: AMED, CINAHL, BNI

• Type of question/type of evidence– Therapy, eg: Cochrane Database of Systematic Reviews

(CDSR); Cochrane Central Register of Controlled Trials (CENTRAL)

– Cost-effectiveness, eg: NHS Economic Evaluation Database (NHS EED); Health Economic Evaluations Database (HEED)

Identifying the sources to search (2)

• Databases– MEDLINE, EMBASE, CINAHL, PsycINFO, AMED, HMIC

• Searchable e-journal packages– Science Direct, Ingenta, Journals@Ovid

• Registers of Research– mRCT, National Research Register, cancer.gov

• Conference Proceedings– Index to Conference Proceedings, ISI Proceedings

• Dissertations– Digital Dissertations, Index to Theses

• Interested parties– Professional associations, voluntary groups

• Internet ….

Searching information sources effectively

• Translating the parts of the question (PICO) into search terms

• Combining search terms using Boolean logic• Sensitivity vs precision• Sensitivity maximising

– Synonyms– Subject headings and free-text– Truncation and wildcards ($, ?, *)– Proximity operators (adj, near, same)

• Search filters, quality filters

1. Alcoholism/

2. Alcohol drinking/

3. alcoholic?

4. alcoholism.tw

5. dipsomania$.tw

6. or/1-5

7. Taurine/

8. acamprosate.tw

9. campral$.tw.

10. or/7-9

A search strategy

11. Naltrexone/12. naltrexone.tw13. trexan.tw14. or/11-1315. Temperance/16. temperance.tw17. sobriety.tw18. ((control$ or reduc$ or

restrict$) adj2 drink$).tw19. or/15-1820. 6 and 10 and 14 and 19

P

Challenges of the role

• Publication bias• Time vs quality trade-off• Complexity of review questions• Widening perspectives

ReferencesAlderson P, Green S, Higgins JPT eds. Cochrane Reviewers’ handbook 4.2.2. [updated March 2004]. Available from <www.cochrane.org/resources/handbook/hbook.htm> [Accessed 11 October 2004].

Booth A and Fry-Smith A. 2003. Developing the research question. In: Etext on Health Technology Assessment (HTA) information resources, Available from Internet <URL: http://www.nlm.nih.gov/nichsr/ehta/chapter1.html>, [Accessed 14 March 2003].

Booth A, O'Rourke AJ, Ford NJ. 2000. Structuring the pre-search reference interview: a useful technique for handling clinical questions. Bull Med Libr Assoc 88(3):239-46.

Chalmers I and Altman DG eds. 1995. Systematic reviews. London, BMJ Publishing Group.

Counsell C. 1997. Formulating questions and locating primary studies for inclusion in systematic reviews. Annals of Internal Medicine, 127(5), 380-387.

ReferencesINAHTA. 2000. INAHTA: International Network of Agencies for Health Technology Assessment: global networking for effective healthcare, Stockholm, INAHTA.

McKibbon A. 1999. PDQ evidence-based principles and practice. Hamilton, Ontario: BC Decker Inc.

Mulrow C. 1995. Rationale for systematic reviews. In: Chalmers I and Altman DG eds. Systematic reviews. London, BMJ Publishing Group.

Oxman AD, Sackett DL and Guyatt GH. 1993. Users' guides to the medical literature. I. How to get started. The Evidence-Based Medicine Working Group. JAMA, 270(17), 2093-2095.

Richardson WS, Wilson MC, Nishikawa J and Hayward RS. 1995. The well-built clinical question: a key to evidence-based decisions. ACP Journal Club, 123(3), A12-A13.

ReferencesSacket DL, Richardson WS, Rosenbery W, Haynes RB. 1997. Evidence-based medicine. How to practice and teach EBM. 1st ed. London: Churchill Livingston, 1997.

Snowball R. 2002. 'Find the evidence'--reflections on an information skills course for community-based clinical health-care staff at the Cairns Library, Oxford. Health Information & Libraries Journal, 19(2), 109-112.

Song F, Eastwood AJ, Gilbody S, Duley L and Sutton AJ. 2000. Publication and related bias. Health Technol Assess, 4 (10).

Villanueva EV, Burrows EA, Fennessy PA, Rajendran M and Anderson JN. 2001. Improving question formulation for use in evidence appraisal in a tertiary care setting: a randomised controlled trial. BMC Medical Informatics and Decision Making, 1(1), 4.

 

Janette Boynton

Senior Health Information Scientist

0141 225 6982

jboynton@htbs.org.uk