Critically Appraised Topic (CAT)
Paul Cronin B.A., M.B. B.Ch. B.A.O., M.S.,
M.R.C.P.I., .F.R.C.R.
Department of Radiology, Division of Cardiothoracic Radiology,
University of Michigan, Ann Arbor, Michigan
Introduction
• Difficult to publish Case Reports • IRB • Critically appraised topic - CAT
– Standardized summary of research evidence
– Organized around a clinical question – Providing a critique of the research – A statement of the clinical relevance
Critically Appraised Topic - CAT
• 275 – 17
• 4 – Seminars in Roentgenology – 2009 • 5 - Abdominal Imaging - 2008 • 8 - Canadian Association Radiology Journal
– 2001-2005
• Mini meta
Critically Appraised Topic
1. Asking a focused and answerable question that translates uncertainty to an answerable question
2. Searching for the best available evidence
3. Critically appraising the evidence for validity and clinical relevance
Critically Appraised Topic
4. Applying the results to clinical practice
5. Evaluation of performance
Step 1: Ask an Answerable Question
• Important to – The patient, clinician, and/or learners – The patients’ well-being – The clinicians’ knowledge needs – Likely to recur in clinical practice – Answerable in the time available
Step 1: Ask an Answerable Question
• PICO – P – Population / problem – I – Intervention / index test – C – Control / reference test – O – Outcome
CAT study question PIOC format question Patient or Problem Intervention Comparison
Intervention
Outcome
Whether low-dose CTC would
perform as well as optical
colonoscopy (OC) for screening
patients for CRC?
Is low-dose CT colonography
equivalent to optical
colonoscopy in identifying >5
mm colonic polyps?
Screening
population of
patients with polyps
CT colonography Optical
colonoscopy
Accuracy in
diagnosis of >5 mm
colonic polyps /
colorectal cancer
Whether ultrasound performs
better than MRI in the diagnosis
of rotator cuff tears?
In patients with rotator cuff tears,
how does US compare to MRI for
diagnosis?
Patients with rotator
cuff tear
Ultrasound MRI Accuracy in
diagnosis of rotator
cuff tears
Whether coronary CTA performs
comparably to invasive coronary
angiography for identifying
potentially or probably
hemodynamically significant
native coronary artery disease,
defined as luminal diameter
stenosis of at least 50%?
In patients with known or
suspected coronary artery
disease, how does coronary CTA
compare to invasive coronary
angiography for identifying ≥
50% luminal diameter coronary
artery stenosis?
Patients with known
or suspected
coronary artery
disease
Coronary CT
angiography
Catheter coronary
angiography
Accuracy in
diagnosis of ≥ 50%
luminal diameter
coronary artery
stenosis
How does computerized
tomography angiography (CTA)
performs in comparison with
magnetic resonance
angiography (MRA) in the
detection of symptomatic carotid
stenosis?
In patients with symptomatic
carotid stenosis, how does CTA
compare with MRA for
diagnosis?
Patients with
symptomatic
carotid artery
stenosis
Carotid CT
angiography
Carotid MR
angiography
Accuracy in
diagnosis of carotid
artery stenosis
What is the current role of RFA
and how does RFA compare to
surgical resection for treatment
of colorectal liver metastases?
In patients with colorectal liver
metastases how does
percutaneous radiofrequency
ablation compare with surgical
resection or other ablative
techniques?
Patients with
colorectal liver
metastasis
Percutaneous
radiofrequency
ablation
Surgical resection Annual recurrence
or mortality rate
Step 2- Search for the Best Current Evidence
1. Identify terms to fit the PICO question
2. Search for secondary sources 3. Search for primary sources
PIOC format question Patient or Problem Intervention Comparison Intervention Outcome
Is low-dose CT colonography
equivalent to optical
colonoscopy in identifying
clinically meaningful colonic
polyps?
Polyp
OR
Colorectal Neoplasm
OR
Colonic Neoplasm
AND Computed Tomography
Colonography
OR
Colonography, Computed
Tomography
AND
Low-Dose
AND Colonoscopy AND Diagnosis
OR
Sensitivity and Specificity
In patients with rotator cuff
tears, how does US compare to
MRI for diagnosis?
Rotator Cuff
OR
Shoulder
OR
Tendon
OR
Injury
OR
Pathology
AND Ultrasonography
OR
Ultrasound
AND Magnetic resonance Imaging
OR
MRI
OR
Contrast enhanced
MRI
AND Diagnosis
OR
Sensitivity and Specificity
In patients with known or
suspected coronary artery
disease, how does coronary
CTA compare to invasive
coronary angiography for
identifying ≥ 50% luminal
diameter coronary artery
stenosis?
Coronary artery disease AND Tomography, x-ray computed
OR
Tomography, x-ray
computed/methods
AND Angiography
OR
Coronary angiography
OR
Coronary angiography
methods
AND Coronary stenosis
OR
Diagnosis
OR
Sensitivity and specificity
In patients with symptomatic
carotid stenosis, how does CTA
compare with MRA for
diagnosis?
Carotid artery stenosis AND Tomography, x-ray computed
OR
Tomography, x-ray
computed/methods
AND MRI / MRA
OR
Magnetic resonance Imaging
OR
Contrast enhanced
MRI
AND Diagnosis
OR
Sensitivity and Specificity
In patients with colorectal liver
metastases how does
percutaneous
radiofrequency ablation
compare with surgical resection
or other ablative techniques?
“liver neoplasm”
OR
“Liver neoplasm/secondary”
AND catheter ablation AND liver neoplasm / surgery AND Efficacy
OR
Recurrence
OR
Mortality
Step 2- Search for the Best Current Evidence
• Primary Study Designs – Randomized double-blind clinical (or
controlled) trial - RCT – Observational studies
• Prospective cohort studies • Retrospective cohort studies • Case-control studies • Case series
Step 2- Search for the Best Current Evidence
• Levels of Evidence – The Oxford center for evidence based
medicine has graded study designs in a hierarchy of evidence
Level Etiology Diagnosis Intervention Prognosis
Ia Systematic Review with
homogeneity of Level Ib
studies
Systematic Review with
homogeneity of Level Ib
studies
Systematic Review with
homogeneity of Level Ib
studies
Systematic Review with
homogeneity of Level Ib
studies
Ib Randomized controlled trial Validating cohort study with
good reference standard
Randomized controlled trial Cohort study with good (>80%)
follow up
Ic Absolute SP-ins and SN-outs4
IIa Systematic Review with
homogeneity of Level IIb
studies
Systematic Review with
homogeneity of Level IIb
studies
Systematic Review with
homogeneity of Level IIb
studies
Systematic Review with
homogeneity of Level IIb
studies
IIb Cohort study (prospective) Exploratory cohort study with
good reference standard
Cohort study (prospective) Cohort study (retrospective)
IIc Outcomes research Outcomes research Outcomes research Outcomes research
IIIa Systematic Review with
homogeneity of non
consecutive cohort or case
control studies
Systematic Review with
homogeneity of Level IIIb
studies
Systematic Review with
homogeneity of case control
studies
Systematic Review with
homogeneity of case control
studies
IIIb Non consecutive cohort study
or case control study
Non consecutive studies; or
without consistently applied
reference standard
Case control study
IV Poor quality cohort study or
poor quality case control
study or case series
Case series or poor or non
independent reference
standard
Poor quality cohort study or
poor quality case control
study or case series
Poor quality cohort study or
poor quality case control study
or case series
V “expert” opinion without
critical appraisal
“expert” opinion without
critical appraisal
“expert” opinion without
critical appraisal
“expert” opinion without
critical appraisal
Step 2- Search for the Best Current Evidence
• Sources of Evidence for 1st Literature – Studies
• PubMed • MEDLINE • EMBASE • ISI Web of Knowledge • MD Consult • Google Scholar
Step 2- Search for the Best Current Evidence
• Sources of Evidence for 2nd Literature – Syntheses
• PubMed Clinical • Cochrane Library • Turning Research into Practice (TRIP) • National Institute of Clinical Excellence
(NICE) • SUMSearch
Step 2- Search for the Best Current Evidence
• Sources of Evidence for 2nd Literature – Syntheses
• Cumulative Index to Nursing and Allied Health Literature (CINAHL)
• National Guidelines Clearinghouse (NGC) • National Library for Health • Scottish Intercollegiate Guidelines Network
(SIGN)
Step 2- Search for the Best Current Evidence
• Sources of Evidence for 2nd Literature – Synopses
• The American College of Physicians (ACP) Journal Club
• Evidence Based Medicine Online – Systems
• Clinical Evidence • Up to Date
Section and Topic
Item
#
TITLE/ABSTRACT /
KEYWORDS
1 Identify the article as a study of diagnostic accuracy (recommend
MeSH heading ‘sensitivity and specificity’).
INTRODUCTION
2 State the research questions or study aims, such as estimating
diagnostic accuracy or comparing accuracy between tests or across
participant groups.
METHODS
Participants 3 Describe the study population: The inclusion and exclusion criteria,
setting and locations where the data were collected.
4 Participant recruitment: Was recruitment based on presenting
symptoms, results from previous tests, or the fact that the participants
had received the index tests or the reference standard?
5 Participant sampling: Was the study population a consecutive series
of participants defined by the selection criteria in item 3 and 4? If not,
specify how participants were further selected.
6 Data collection: Was data collection planned before the index test
and reference standard were performed (prospective study) or after
(retrospective study)?
Test methods 7 The reference standard and its rationale.
8 Technical specifications of material and methods involved including
how and when measurements were taken, and/or cite references for
index tests and reference standard.
9 Definition of and rationale for the units, cutoffs, and/or categories of
the results of the index tests and the reference standard.
10 The number, training, and expertise of the persons executing and
reading the index tests and the reference standard.
11 Whether or not the readers of the index tests and reference standard
were blind (masked) to the results of the other test and describe any
other clinical information available to the readers.
Statistical methods 12 Methods for calculating or comparing measures of diagnostic
accuracy, and the statistical methods used to quantify uncertainty
(e.g., 95% confidence intervals).
13 Methods for calculating test reproducibility, if done.
RESULTS
Participants 14 Report when study was done, including beginning and ending dates
of recruitment.
15 Clinical and demographic characteristics of the study population
(e.g., age, sex, spectrum of presenting symptoms, comorbidity,
current treatments, recruitment centers).
16 The number of participants satisfying the criteria for inclusion that did
or did not undergo the index tests and/or the reference standard;
describe why participants failed to receive either test (a flow diagram
is strongly recommended).
Test results
17 Time interval from the index tests to the reference standard, and any
treatment administered between.
18 Distribution of severity of disease (define criteria) in those with the
target condition; other diagnoses in participants without the target
condition.
19 A cross tabulation of the results of the index tests (including
indeterminate and missing results) by the results of the reference
standard; for continuous results, the distribution of the test results by
the results of the reference standard.
20 Any adverse events from performing the index tests or the reference
standard.
Estimates 21 Estimates of diagnostic accuracy and measures of statistical
uncertainty (e.g., 95% confidence intervals).
22 How indeterminate results, missing responses, and outliers of the
index tests were handled.
23 Estimates of variability of diagnostic accuracy between subgroups of
participants, readers or centers, if done.
24 Estimates of test reproducibility, if done.
DISCUSSION 25 Discuss the clinical applicability of the study findings.
STARD
Item Yes No Unclear
1. Was the spectrum of patients representative of the patients who will receive
the test in practice?
( ) ( ) ( )
2. Were selection criteria clearly described? ( ) ( ) ( )
3. Is the reference standard likely to correctly classify the target condition? ( ) ( ) ( )
4. Is the time period between reference standard and index test short enough to
be reasonably sure that the target condition did not change between the two
tests?
( ) ( ) ( )
5. Did the whole sample or a random selection of the sample, receive verification
using a reference standard of diagnosis?
( ) ( ) ( )
6. Did patients receive the same reference standard regardless of the index test
result?
( ) ( ) ( )
7. Was the reference standard independent of the index test (i.e. the index test
did not form part of the reference standard)?
( ) ( ) ( )
8. Was the execution of the index test described in sufficient detail to permit
replication of the test?
( ) ( ) ( )
9. Was the execution of the reference standard described in sufficient detail to
permit its replication?
( ) ( ) ( )
10. Were the index test results interpreted without knowledge of the results of the
reference standard?
( ) ( ) ( )
11. Were the reference standard results interpreted without knowledge of the
results of the index test?
( ) ( ) ( )
12. Were the same clinical data available when test results were interpreted as
would be available when the test is used in practice?
( ) ( ) ( )
13. Were uninterpretable/ intermediate test results reported? ( ) ( ) ( )
QUADAS
Step 3- Appraise the Literature • Interpreting Results in Diagnostic
Radiology – Sensitivity – Specificity – Positive predictive value – Negative predictive value – Positive (LR+) – Negative (LR-)
Measure Meaning Interpretation
Diagnostic Literature
Sensitivity Patients with the disease with positive test results
("true positives")/ patients with disease
Negative test result in a highly sensitive test
can rule out the disease if prevalence is
relatively low
Specificity Patients without the disease with negative results
("true negatives")/ well people (no disease)
Positive test result in a highly specific test
can rule in the disease if prevalence is
relative high
Positive predictive
value (PPV)
Patient with true positive test results / all patients
with positive test result (true positives + false
positives)
This value is affected by the prevalence in
the population studied. If a test has a high
positive predictive value (>95%) in your
population, one may confidently commence
treatment
Negative predictive
value (NPV)
Patients with true negative results / all patients with
negative test result (true negatives + false
negatives)
This value is affected by the prevalence in
the population studied. If a test has a high
negative predictive value (>95%) in your
population, one may be able to safely
withhold treatment
Likelihood ratio (LR) Likelihood of a given test result in patients with
disease / the likelihood of the same result in
patients without disease
LR=0 (negative LR): excludes the disease
LR= infinity (positive LR): confirms the
disease
LR=1 the test result is uninformative as the
result is equally likely in the two groups
Step 3- Appraise the Literature • Statistical Measures for Therapeutic
Studies – Hazard ratio - HR – Relative risk - RR – Attributable risk - AR – Number needed to treat / harm - NNT / NNH
Measure Meaning Interpretation
Therapeutic Literature
Relative risk (RR) Risk of developing disease (or outcome)
in the treatment group/ risk of developing
disease (or outcome) in the control group
RR=1: no difference between
groups
RR<1 treatment reduces the risk of
the disease
RR>1 treatment increases the risk
of the disease
Absolute risk (AR) Difference in the rates of events
calculated as the rate in the control group
minus the rate in the experimental group
AR =0 no difference between
groups
AR positive: treatment is beneficial
AR negative: treatment is harmful
Number needed to
treat (NNT)
The number of patients needed to be
treated for 1 of them to benefit from the
treatment
Number needed to
harm (NNH)
The number of patients needed to be
treated for 1 patient to experience an
Study Design Quality of Evidence Lower if Higher if
Randomized Trial → High Risk of bias
-1 Serious
-2 Very serious
Inconsistency
-1 Serious
-2 Very serious
Indirectness
-1 Serious
-2 Very serious
Imprecision
-1 Serious
-2 Very serious
Publication bias
-1 Likely
-2 Very likely
Large effect
+1 Large
+2 Very Large
Dose response
+1 Evidence of gradient
All plausible confounding
+1 Would reduce a
demonstrated effect or
+1 Would suggest a
spurious effect when
results show no effect
Moderate
Observational study → Low
Very low
GRADE
Section/topic # Checklist item Reported
on page #
TITLE
Title 1 Identify the report as a systematic review, meta-analysis, or both.
ABSTRACT
Structured summary 2 Provide a structured summary including, as applicable: background; objectives;
data sources; study eligibility criteria, participants, and interventions; study
appraisal and synthesis methods; results; limitations; conclusions and implications
of key findings; systematic review registration number.
INTRODUCTION
Rationale 3 Describe the rationale for the review in the context of what is already known.
Objectives 4 Provide an explicit statement of questions being addressed with reference to
participants, interventions, comparisons, outcomes, and study design (PICOS).
METHODS
Protocol and
registration
5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web
address), and, if available, provide registration information including registration
number.
Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report
characteristics (e.g., years considered, language, publication status) used as
criteria for eligibility, giving rationale.
Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact
with study authors to identify additional studies) in the search and date last
searched.
Search 8 Present full electronic search strategy for at least one database, including any
limits used, such that it could be repeated.
Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in
systematic review, and, if applicable, included in the meta-analysis).
Data collection
process
10 Describe method of data extraction from reports (e.g., piloted forms, independently,
in duplicate) and any processes for obtaining and confirming data from
investigators.
Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding
sources) and any assumptions and simplifications made.
Risk of bias in
individual studies
12 Describe methods used for assessing risk of bias of individual studies (including
specification of whether this was done at the study or outcome level), and how this
information is to be used in any data synthesis.
Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means).
Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done,
including measures of consistency (e.g., I2) for each meta-analysis.
Risk of bias across
studies
15 Specify any assessment of risk of bias that may affect the cumulative evidence
(e.g., publication bias, selective reporting within studies).
Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses,
meta-regression), if done, indicating which were pre-specified.
RESULTS
Study selection 1
7
Give numbers of studies screened, assessed for eligibility, and included in the
review, with reasons for exclusions at each stage, ideally with a flow diagram.
Study
characteristics
1
8
For each study, present characteristics for which data were extracted (e.g., study
size, PICOS, follow-up period) and provide the citations.
Risk of bias within
studies
1
9
Present data on risk of bias of each study and, if available, any outcome level
assessment (see item 12).
Results of individual
studies
2
0
For all outcomes considered (benefits or harms), present, for each study: (a)
simple summary data for each intervention group (b) effect estimates and
confidence intervals, ideally with a forest plot.
Synthesis of results 2
1
Present results of each meta-analysis done, including confidence intervals and
measures of consistency.
Risk of bias across
studies
2
2
Present results of any assessment of risk of bias across studies (see Item 15).
Additional analysis 2
3
Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses,
meta-regression [see Item 16]).
DISCUSSION
Summary of
evidence
2
4
Summarize the main findings including the strength of evidence for each main
outcome; consider their relevance to key groups (e.g., healthcare providers, users,
and policy makers).
Limitations 2
5
Discuss limitations at study and outcome level (e.g., risk of bias), and at review-
level (e.g., incomplete retrieval of identified research, reporting bias).
Conclusions 2
6
Provide a general interpretation of the results in the context of other evidence, and
implications for future research.
FUNDING
Funding 2
7
Describe sources of funding for the systematic review and other support (e.g.,
supply of data); role of funders for the systematic review.
PRISMA
1. Was an ‘a priori’ design provided?
The research question and inclusion criteria should be established
before the conduct of the review.
Yes
No
Can’t answer
Not applicable
2. Was there duplicate study selection and data extraction?
There should be at least two independent data extractors and a
consensus procedure for disagreements should be in place.
Yes
No
Can’t answer
Not applicable
3. Was a comprehensive literature search performed?
At least two electronic sources should be searched. The report must
include years and databases used (e.g. Central, EMBASE, and
MEDLINE). Key words and/or MESH terms must be stated and where
feasible the search strategy should be provided. All searches should
be supplemented by consulting current contents, reviews, textbooks,
specialized registers, or experts in the particular field of study, and by
reviewing the references in the studies found.
Yes
No
Can’t answer
Not applicable
4. Was the status of publication (i.e. grey literature) used as an
inclusion criterion?
The authors should state that they searched for reports regardless of
their publication type. The authors should state whether or not they
excluded any reports (from the systematic review), based on their
publication status, language etc.
Yes
No
Can’t answer
Not applicable
5. Was a list of studies (included and excluded) provided?
A list of included and excluded studies should be provided.
Yes
No
Can’t answer
Not applicable
6. Were the characteristics of the included studies provided?
In an aggregated form such as a table, data from the original studies
should be provided on the participants, interventions and outcomes.
The ranges of characteristics in all the studies analyzed e.g. age,
race, sex, relevant socioeconomic data, disease status, duration,
severity, or other diseases should be reported.
Yes
No
Can’t answer
Not applicable
7. Was the scientific quality of the included studies assessed and
documented?
‘A priori’ methods of assessment should be provided (e.g., for
effectiveness studies if the author(s) chose to include only
randomized, double-blind, placebo controlled studies, or allocation
concealment as inclusion criteria); for other types of studies
alternative items will be relevant.
Yes
No
Can’t answer
Not applicable
8. Was the scientific quality of the included studies used
appropriately in formulating conclusions?
The results of the methodological rigor and scientific quality should
be considered in the analysis and the conclusions of the review, and
explicitly stated in formulating recommendations.
Yes
No
Can’t answer
Not applicable
9. Were the methods used to combine the findings of studies
appropriate?
For the pooled results, a test should be done to ensure the studies
were combinable, to assess their homogeneity (i.e. Chi-squared test
for homogeneity, I²). If heterogeneity exists a random effects model
should be used and/or the clinical appropriateness of combining
should be taken into consideration (i.e. is it sensible to combine?).
Yes
No
Can’t answer
Not applicable
10. Was the likelihood of publication bias assessed?
An assessment of publication bias should include a combination of
graphical aids (e.g., funnel plot, other available tests) and/or
statistical tests (e.g., Egger regression test).
Yes
No
Can’t answer
Not applicable
11. Was the conflict of interest stated?
Potential sources of support should be clearly acknowledged in both
the systematic review and the included studies.
Yes
No
Can’t answer
Not applicable
AMSTAR
Original AGREE Item AGREE II Item
Domain 1. Scope and Purpose
1. The overall objective(s) of the guideline is (are)
specifically described.
The overall objective(s) of the guideline is (are) specifically
described.
2. The clinical question(s) covered by the guideline is (are)
specifically described.
The health question(s) covered by the guideline is (are)
specifically described.
3. The patients to whom the guideline is meant to apply are
specifically described.
The population (patients, public, etc.) to whom the guideline is
meant to apply is specifically described.
Domain 2. Stakeholder Involvement
4. The guideline development group includes individuals
from all the relevant professional groups.
The guideline development group includes individuals from all
the relevant professional groups.
5. The patients’ views and preferences have been sought.
The views and preferences of the target population (patients,
public, etc.) have been sought.
6. The target users of the guideline are clearly defined.
No change
7. The guideline has been piloted among end users.
Delete item. Incorporated into user guide description of item
19.
Domain 3. Rigour of Development
8. Systematic methods were used to search for evidence.
Systematic methods were used to search for evidence.
Renumber to 7.
9. The criteria for selecting the evidence are clearly
described.
The criteria for selecting the evidence are clearly described.
Renumber to 8.
NEW Item 9. The strengths and limitations of the body of
evidence are clearly described.
10. The methods for formulating the recommendations are
clearly described.
The methods for formulating the recommendations are clearly
described.
11. The health benefits, side effects, and risks have been
considered in formulating the recommendations.
The health benefits, side effects, and risks have been
considered in formulating the recommendations.
12. There is an explicit link between the recommendations
and the supporting evidence.
There is an explicit link between the recommendations and the
supporting evidence.
13. The guideline has been externally reviewed by experts
prior to its publication.
The guideline has been externally reviewed by experts prior to
its publication.
14. A procedure for updating the guideline is provided.
A procedure for updating the guideline is provided.
Domain 4. Clarity of Presentation
15. The recommendations are specific and unambiguous.
The recommendations are specific and unambiguous.
16. The different options for management of the condition
are clearly presented.
The different options for management of the condition or
health issue are clearly presented.
17. Key recommendations are easily identifiable.
Key recommendations are easily identifiable.
Domain 5. Applicability
18. The guideline is supported with tools for application.
The guideline provides advice and/or tools on how the
recommendations can be put into practice.
AND Change in domain (from Clarity of Presentation) AND
renumber to 19
19. The potential organizational barriers in applying the
recommendations have been discussed.
The guideline describes facilitators and barriers to its
application.
AND change in order – renumber to 18
20. The potential cost implications of applying the
recommendations have been considered.
The potential resource implications of applying the
recommendations have been considered.
21. The guideline presents key review criteria for
monitoring and/ or audit purposes.
The guideline presents monitoring and/ or auditing criteria.
Domain 6. Editorial Independence
22. The guideline is editorially independent from the funding
body.
The views of the funding body have not influenced the content
of the guideline.
23. Conflicts of interest of guideline development members
have been recorded.
Competing interests of guideline development group
members have been recorded and addressed.
AGREE
Steps Primary research Secondary research Critical appraisal issue
(Systematic review /
Meta-analysis)
Q Formulate a question Formulate a question Does the research ask a
(PI CO) (PI CO) clearly focused question
(PI CO) and use it to direct
the search and select
articles for inclusion?
F Find the best evidence Find the best evidence Did the search find all the
best evidence?
A Appraise the included Appraise the included Have the studies been
studies studies critically appraised?
S Synthesise the results Have the results been
(summary tables and synthesized with
plots) appropriate summary
tables and plots?
Apply the results Apply the evidence
Step 3- Appraise the Literature
• Results of a Meta-analysis – Have the studies been critically
appraised (using appropriate quality criteria)?
– Have the results been synthesized with appropriate summary tables and plots?
Step 3- Appraise the Literature
• Summarizing the Results of your Literature Review – http://www.radiography.com/pub/
Section A To calculate diagnostic accuracy parameters from a 2x2 table Enter numbers of patients in the highlighted cells: Test Positive Negative Disease + 85 11 96 Disease - 22 37 59
n = 155
Calculated Diagnostic Accuracy Parameters:
Sample size = 155
Prevalence = 0.619 Lower CI Limit Upper CI Limit
Sensitivity = 0.885 95% CI(Sens)= 0.822 0.949
Specificity = 0.627 95% CI(Spec)= 0.504 0.751 PPV = 0.794 NPV = 0.771
LR + result*= 2.375 *Interpreting Likelihood Ratios:
LR - result*= 0.183 0 excludes disease,infinity excludes normality LR > 10 or < 0.1 = large change in likelihood LR 5-10 or 0.1-0.2 = moderate change LR 2-5 or 0.2-0.5 = small change LR of 1 = no change
Section B
To calculate the predictive values of this test in local population: Enter local disease prevalence (0-1) 0.15 2*2 Table recalculated (new prevalence, n =100) Test Positive Negative Disease + 13 2 Disease - 32 53 Local PPV = 0.295 Local NPV = 0.969
Section C Bayesian Analysis (for any test with known sensitivity and specificity): 3 Variables required - enter data in the highlighted cells: Sensitivity (0-1) 0.885 Specificity (0-1) 0.627 Pretest prob. (0-1) 0.15
Posttest prob. pos result= 0.295
Posttest prob. neg result= 0.031
Section D
Estimation of 95% CI; known sample size, disease prevalence, sensitivity and specificity* 4 variables required - enter data in highlighted cells: Total sample size 100 Prevalence (0-1) 0.5 Lower Limit Upper Limit Sensitivity (0-1) 0.95 95% CI 0.890 1.010 Specificity (0-1) 0.8 95% CI 0.689 0.911
*NB: Read limitations of CI estimations for small sample sizes and sensitivity /specificity >0.9 or <0.1
Section E Calculation of Likelihood Ratios from known Sensitivity and Specificity 2 variables required - enter data in highlighted cells: Sensitivity (0-1) 0.885 Specificity (0-1) 0.627 LR +ve test = 2.373 LR -ve test = 0.183
Step 4: Apply
• Test result means to an individual patient – Transform sensitivity/specificity to
likelihood of having the disease – Pre-test probability → Post test
probability – Fagan’s nomogram – Graph of conditional probabilities
Step 5: Evaluate
• Evaluate - applied to or alter clinical practice – Ongoing developments in technology – Evaluate technical parameters /
generations of equipment used • Limitations of available literature
– Need for better quality studies / secondary studies
Name Link
PubMed. US National Library of Medicine and National Institutes of Health. http://www.ncbi.nlm.nih.gov/sites/entrez.
Medline / Ovid SP. http://gateway.ovid.com/
EMBASE http://www.embase.com/.
Google Scholar http://scholar.google.com/.
ISI Web of Knowledge http://apps.isiknowledge.com/UA_GeneralSearch_input.do?product=UA&search_mode=General
Search&SID=4BKm4mOcfDnofbKjHBI&preferencesSaved=.
MD Consult http://www.mdconsult.com/php/120885574-2/homepage.
Cochrane Collaboration http://www.cochrane.org.
The Cumulative Index to Nursing and Allied Health Literature http://www.ebscohost.com/cinahl/.
National Institute of Clinical Excellence http://www.nice.org.uk/.
Scottish Intercollegiate Guidelines Network http://www.sign.ac.uk/.
SUMSearch search engine http://sumsearch.uthscsa.edu/.
The National Library for Health http://www.library.nhs.uk/Default.aspx.
National Guidelines Clearinghouse http://www.guideline.gov/.
PubMed Clinical Queries http://www.ncbi.nlm.nih.gov/entrez/query/static/clinical.shtml.
The American College of Physicians Journal Club http://www.acpjc.org/.
BMJ Evidence Based Medicine http://ebm.bmj.com/.
Bandolier electronic journal. http://www.medicine.ox.ac.uk/bandolier/aboutus.html.
The Turning Research Into Practice Database http://www.tripdatabase.com/index.html.
UpToDate http://www.uptodate.com/home/index.html.
Dynamed Clinical Reference Tool http://www.ebscohost.com/dynamed/.
Physicians Information and Education Resource http://pier.acponline.org/index.html.
BMJ Clinical Evidence http://clinicalevidence.bmj.com/ceweb/index.jsp.
The Database of Abstracts of Reviews of Effects http://mrw.interscience.wiley.com/cochrane/cochrane_cldare_articles_fs.html.
The Cochrane Central Register of Controlled Clinical Trials (CENTRAL) http://www.mrw.interscience.wiley.com/cochrane/cochrane_clcentral_articles_fs.html.
The National Library of Medicine http://www.nlm.nih.gov/.
Netting the Evidence http://www.shef.ac.uk/scharr/ir/netting/.
The Center for Evidence Based Radiology at the Brigham and Women’s` hospital http://www.brighamandwomens.org/cebi/default.aspx
Centre for Reviews and Dissemination at the University of York http://www.york.ac.uk/inst/crd/.
The Centre for Health Evidence at the University of Alberta http://www.cche.net/.
The Centre for Evidence Based Medicine at Oxford http://www.cebm.net/.
The Centre for Evidence Based Medicine at the University Health Network, Toronto http://www.cebm.utoronto.ca/practise/evaluate/index.htm#top
The Blue Cross and Blue Shield Association Technology Evaluation Center http://www.bcbs.com/blueresources/tec/.
Critically Appraised Topic (CAT)
Paul Cronin B.A., M.B. B.Ch. B.A.O., M.S.,
M.R.C.P.I., .F.R.C.R.
Department of Radiology, Division of Cardiothoracic Radiology,
University of Michigan, Ann Arbor, Michigan
Top Related