Nur3052 ch5

Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins

Chapter 5: Critically Appraising Quantitative Evidence for Clinical Decision Making


Critically Appraising Quantitative Studies

The process focuses on three questions

1. Are the results of the study valid? (Validity)

2. What are the results? (Reliability)

3. Will the results help me in caring for my patients? (Applicability)

Interpretation of results requires consideration of the clinical significance of the study findings and the statistical significance of the results


The Question of Validity

Bias - anything that distorts study findings in a systematic way and arises from the study methodology

Selection bias

Knowledge of who is or is not receiving an intervention

Measurement bias

Recall bias

Contamination


Question

Tell whether the following statement is true or false.

The best way to prevent selection bias is to randomly assign study participants to groups.


Answer

True

Rationale: Selection bias in quantitative studies is best controlled by assigning participants to groups on a random basis. Other systematic and deliberate methods of assignment normally increase the chance of selection bias.


The Question of Validity (cont’d)

Confounded Study Results

A study’s results may be confounded when a relationship between two variables is actually due to a third, either known or unknown variable

Often encountered in studies about lifestyle and health

Frequently a result of participant history


Reliability Evaluate whether the sum of all n values equals the

original N

Magnitude of the effect

How strong is the difference between groups?

Tables

Statistical tests


Reliability (cont’d)

Strength of association

Absolute risk reduction (ARR)

Absolute risk increase (ARI)

Relative risk (RR)

Relative risk reduction (RRR)

Odds ratio (OR)

Number needed to treat/harm (NNT/NNH)


Question

The findings of a quantitative study testing a high school-based sexual health program reveal that for every 140 female students who take the program, one pregnancy is prevented. This conclusion indicates the:

a. OR

b. NNT

c. NNH

d. ARR


Answer

b. NNT

Rationale: The number needed to treat (NNT) represents the number of people who would need to receive the therapy or intervention (the educational program) to prevent one bad outcome (teenage pregnancy).



Random error

Variations that occur purely by chance

The extent to which random error may influence a measurement can be reported using statistical significance (or p values) or by confidence intervals.



Statistical significance

The aim of statistical analysis is to determine whether an observed effect arises from the study intervention or has occurred by chance

Study hypothesis and null hypothesis

The smaller the p value, the less likely the null hypothesis is true

Confidence interval - describes the range in which the true effect lies with a given degree of certainty


Confidence Intervals


Questions to Ask in a Critical Appraisal

Why was the study done?

What is the sample size?

Are measurements reliable and valid?

How were the data analyzed?

Did any untoward event happen during the study?

How do the findings fit with previous research?

What does this mean for practice?


Appraising Case Studies

Historically ranked lower in the hierarchy of evidence

Must be used with caution to inform practice, and any application requires careful evaluation of the outcomes


Appraising Case Control Studies

These investigate why certain people develop a specific illness, have an adverse event with a particular treatment, or behave in a particular way

Questions to ask

How were the cases obtained?

Were appropriate controls selected?

Were data collection methods the same for the cases and controls?


Appraising Cohort Studies

These are used for investigating the course of a disease or the unintended consequences of a treatment

Questions to ask

Was there a representative and well-defined sample of individuals at a similar point in the course of the disease?

Was follow up sufficiently long and complete?

Were objective and unbiased outcome criteria used?


Appraising Cohort Studies (cont’d)

Did the analysis adjust for important prognostic risk factors and confounding variables?

What is the magnitude of the relationship between predictors (i.e., prognostic indicators) and targeted outcome?

How likely is the outcome event(s) in a specified period of time?

How precise are the study estimates?


Question

A team of researchers have received a grant to investigate the potential links between diet and the development of stomach cancer. What methodology is most likely to answer the researchers’ clinical question?

a. Case control

b. Case study

c. Randomized controlled trial (RCT)

d. Qualitative study


Answer

a. Case control

Rationale: A case control study often selects individuals with a particular disease (e.g., stomach cancer) and looks back to identify factors that may underlie that disease (e.g., diet). Neither a case study nor a qualitative study would inform this relationship and an RCT would be unethical and impractical.


Appraising Randomized Controlled Trials (RCTs) These can accurately establish cause and effect and can

inform the efficacy and effectiveness of interventions

Questions related to validity of RCTs

Were the subjects randomly assigned to the experimental and control groups?

Was random assignment concealed from the individuals who were first enrolling subjects into the study?

Were the subjects and providers kept blind to study group?


Appraising the Validity of RCTs

Were reasons given to explain why subjects did not complete the study?

Were the follow-up assessments long enough to fully study the effects of the intervention?

Were the subjects analyzed in the group to which they were randomly assigned?

Was the control group appropriate?

Were instruments valid and reliable?

Were the subjects in each of the groups similar on demographic and baseline clinical variables?


Appraising Systematic Reviews

Systematic reviews are a compilation of similar studies that address a specific clinical question

Not the same as a literature review or narrative review

The process the was used to conduce the review should be explicit

Look for a detailed description of the databases accessed, the search strategies, and the search terms

Clinicians should be able to clearly see which studies were included and which were excluded and how selected studies were assessed


Appraising Systematic Reviews (cont’d) The report should inform clinicians about how data were

extracted from the individual studies

It should provide an overview of the evaluation of the included studies

Meta-analysis

The statistical approach to synthesizing the results of two or more studies

A relatively new methodology that has become a hallmark of EBP

As with all methodologies, applicability must be considered


Evaluation and Synthesis of Quantitative Studies Once studies have been selected, they should be melded

together into a synthesis upon which to base practice and standards

Individual studies should be evaluated using an evaluation table - see Table 5-8

Synthesis is not reporting the findings of study after study; it is combining, contrasting, and interpreting a body of evidence to reach a conclusion


Question

Tell whether the following statement is true or false.

Meta-analysis results in evidence that is applicable to a larger patient population than individual studies.


Answer

False

Rationale: In meta-analyses, combining the results of several studies produces a larger sample size and thus greater power to accurately determine the magnitude of the effect. This does not, however, increase the size of the relevant patient population.

Nur3052 ch5

Technology

Transcript of Nur3052 ch5