EVALUATION OF RELEVANT SCIENTIFIC LITERATURE

Ahmed ZeeneldinAssociate Prof of Medical Oncology

Master of Clinical Research, Liverpool, UK

Why should we evaluate the literature?• PubMed

• 2011: ~ 1000 000 new articles added• Dec 2011: >100 000 new articles added• Anthracycline AND chemotherapy

AND breast cancer: >8000 articles

• Thus: • little time and huge literature volume• Prioritize• Evaluate what you read/hear

Acquiring information• The ‘push’ method

• lectures, seminars, • reading journals • no control over the content

• The ‘pull’ Method• You search for answers

for your questions

This medicine is based on what?

Based on Nothing?

Evidence 3-dimensions (apq)

• Author (a)• Known• Expert

• Publisher (p)• Peer-reviewed• Non-reviewed

• Quality of evidence (q)• On the fly: EBM guides• Comprehensive

Literature assessment/appraisal

• Pre-appraised sources • Up-to-date• Trip database• Rehab+ database• Risk lies on expert authority (a & P)

• Appraise it yourself (q)- Rapidly: EBM tools- Extensively: critical appraisal tools

Types of Information Sources• Primary Sources: original research

• Not appraised

• Secondary Sources: about primary• Appraised

• Tertiary Sources: about secondary• Appraised

Primary Sources • Original materials • Not been filtered through interpretation, condensation• Not evaluated by a second party; • Examples:

• Original journal articles• Theses

Secondary Sources • Sources about primary, or original, information, • Modified, selected, or rearranged for a specific purpose or

audience• Examples:

• Textbooks• Review articles

Tertiary Sources • Sources twice removed from original sources• Examples:

• Handbooks• Encyclopedias

Steps of reading literature• You may start by 3ry or 2ry information recourses

• Textbook• Review article• Encyclopedia• handbook

• To get overview of the topic• Then you use the bibliography

to locate 1ry sources• Or search directly for 1ry sources

Evidence-based medicine (EBM) tools

•EBM: is a process to systematically • FIND, • APPRAISE, and • APPLY• research findings to clinical decisions

OLD EBM CONCEPT

Evidence Pyramid• I - Controlled and randomized • II-I Controlled but not randomized • II-2 Cohort or case control • II-3 Multiple time series • III- Expert opinion or case study

2011 OXFORD CENTRE FOR EVIDENCE-BASED MEDICINE (OCEM) LEVELS OF EVIDENCE

EVIDENCE BOXES

2011 OCEBM Levels of Evidence (Evidence Boxes)

If you have limited time, where do you begin searching for evidence?

- hierarchy of the likely best evidence- for busy people (few minutes to few hours)

PubMed search for “ Anthracycline AND chemotherapy AND (breast cancer) ” plus some filters

Type Term used No. of articlesAll articles (no filter) 8066

SR Systematic review 32

RCT "random allocation" [MeSH] 120

DBRCT double blind method [mh] 58

Cohort "cohort studies" [MeSH] 1379

Case-control "Case-Control Studies"[Mesh] 521

Case report Case Reports [Publication Type] 568

Evidence according to question• Incidence

(How common is the problem?)• Local surveys with random sampling OR consensus

• Diagnosis(Is this diagnostic or monitoring test accurate?)• SR of cross sectional studies with reference standard

• Prognosis(What will happen if we do not add a therapy?)• SR of inception cohort studies

• Treatment Benefits(Does this intervention help or harm?)• SR of RT

• Screening(Is this (early detection) test worthwhile• SR of RT

Critical appraisal of study reports• Systematic Review• Diagnostics • Prognosis • RCT •

Systematic Review Appraisal

• What is the question (PICO)?• Were important and relevant studies missed?• Were criteria to select articles appropriate?• Were the quality of included articles assessed?• Were the results similar from study to study?• What were the results? and how they are presented?

Diagnostic test study appraisal• Was the diagnostic test evaluated in a representative

spectrum of patients?• Was the reference standard applied regardless of the

index test result?• Was there an independent, blind comparison between

the index test and an appropriate reference ('gold') standard of diagnosis?

• Are test characteristics presented?• Accuracy: sensitivity and specificity• Performance in population: PPV and NPV

• Details to replicate the test?

Critical appraisal of Prognostic studies • Were patients representative and assembled at a

common point?• Was patient follow-up sufficiently long and complete?• Were outcome criteria either objective or applied in a

‘blind’ fashion? • If subgroups with different prognoses are identified, did

adjustment for important prognostic factors take place?

• How likely are the outcomes over time?• How precise are the prognostic estimated?• Are the results applicable to my patients?

RCT appraisal• What question did the study ask (PICO)?• Was the assignment of patients to treatments

randomised?• Were the groups similar at the start of the trial?• Aside from the allocated treatment, were groups treated

equally?• Were all patients who entered the trial accounted for?

and were they analysed in the groups to which they were randomised?

RCT appraisal• Were measures objective or were the patients and

clinicians kept “blind” to which treatment was being received?

• How large was the treatment effect?• How precise was the estimate of the treatment effect?• Will the results help me in caring for my patient?

Steps of EBM

1. asking questions2. searching for answers3. critically appraising the results

_____________________________4. determining applicability to practice5. integrating data with experience6. involving patient in decisions

Asking questions1) what is the question about?

• Therapy, diagnosis, prognosis, aetiology/harm, screening etc2) what is the type of question?

- background Q (topic)- foreground Q (patient)

3) how can questions be structured to facilitate a search for precise answers?

PICO’ components:• Patient group/Population• Intervention • Comparison• Outcome

Examples of Focused Questions1 2 3 4

Patient or Problem

Intervention(a cause, prognostic

factor, treatment, etc.)

Comparison Intervention

(if necessary)

Outcomes

Med OncologyIn patients with MBC does FEC regimen Compared to CMF

regimenYield better OS rates?

Surgical Oncology

In patients with early BC Does BCS Compared to MRM Lead to higherrecurrences and lower OS?

RadiotherapyIn patients with operable BC

Does IO RT Compared to PO RT Yield higher recurrences?

Clinical Pathology

In patients with ovarian cancer

Does HEP4 Compared to CA125 Leads to earlier diagnosis of recurrence?

EpidemiologyIn normal people Does smoking Compared to no

smokingIncrease risk of lung cancer

How to Conduct a Computer Search 1. Specify the research problem2. Select the databases3. Select the key concepts,

operators, limits4. Conduct the search5. Increase or decrease your total

results 6. Review the citation list

1. Honey and cancer2. PubMed3. Honey AND cancer4. ENTER: 198 records5. Honey OR bees

AND cancer: 281Honey AND cancer AND treatment: 129

Determining the Level of Evidence • Quickly:

• Evidence box according to question

• Thoroughly (critically appraise according to study type): • Relevance (related):

• applicability of the evidence to my problem • Validity (correct):

• the extent to which a concept, conclusion or measurement is well-founded and corresponds accurately to the real world

TRANSLATING EVIDENCE INTO DECISIONThe integration of relevant evidence with clinical experience

Trial Validity • Internal: proper study

• Did the study measure what it is meant to?• Do we trust the results?

• External: generalizability of results

Making decisions• Define the population and intervention

• Patients’ characteristics• Intervention: price and availability, training

• Search for and understand the biases• Critical appraisal

• Interpret the findings and apply them to your patient• Sample size• Summary statistic: • Probablity (p) and confidence of the results (CI)

Conclusion• Pre-appraised literature may be the

main source of information for undergraduate students

• However, graduates and faculty staff should be able to appraise information both quickly (EBM tools) and thoroughly (critical appraisal tools)

• APQ 3-dimension concept

Recommendation• I suggest that principles of EBM be taught to all NCI

students• By whom?

• Concerned Departments? • Epidemiology/Biostatistics Department?