CREATING THE FOUNDATION FOR EVIDENCE BASED PRACTICE

Evidence Based Research

What is an Evidence Based Guideline?

"Guidelines are recommendations intended to assist providers and recipients of health care and other stakeholders to make informed decisions. Recommendations may relate to clinical interventions, public health activities, or government policies."

- WHO 2003, 2007

Grade Approach to Clinical Recommendations

The Grade Approach is one tool that organizations can use to distill a body of research to make a clinical claim.

An “evidenced based treatment” is a treatment that has research evidence supporting it.

Research evidence can be of high, moderate, low or very low quality.

Based on the quality of the research a clinician can make a claim about an intervention.

Further Assessing the Quality of Evidence

Randomized trials = high quality evidence.

Observational studies (i.e., case-control studies and controlled prospective or retrospective cohort studies) = low quality evidence.

Unsystematic observations (i.e., case series, case reports, clinical experience) = very low quality evidence.

Quality of Research = Quality of Evidence

The “quality of evidence” is the confidence that you have that the direction and the magnitude of the estimated effect are correct.

Quality of evidence Suggested implicationsHigh Further research is unlikely to change the confidence in an

estimated effect; we are confident that we can expect very similar effect in a population for which the recommendation is intended

Moderate Further research is likely to have an important impact on the confidence in an estimated effect and may change that estimate

Low Further research is very likely to have an important impact on the confidence in an estimated effect and is likely to change that estimate

Very low Any estimate of an effect is very uncertain

Quality of Evidence = Strength of Claim

The higher the quality of the evidence, the stronger the claim one can make about a treatment.

As a high quality study is published other researchers can begin to gain interest in the SE.

With increased publications it becomes easier to get funding for research.

With increased publications it becomes easier to get organizations to provide SE.

Levels of Evidence – Levels of Recommendations

The Image on the Right

Explores how a review of sleep treatments classified the level of evidence to support a given treatment.

They might for instance find Level I evidence for CBT but a much lower level of evidence for other interventions. Morgenthaler, T., Kramer, M., Alessi, C., Friedman, L., Boehlecke, B., Brown, T., ... & Swick, T. (2006). Practice parameters

for the psychological and behavioral treatment of insomnia: an update. An American Academy of Sleep Medicine report. SLEEP-NEW YORK THEN WESTCHESTER-, 29(11), 1415.

Levels of Evidence = Levels of Recommendations

This image explores recommendations a clinician could make to his clients or patients based on evidence.

For instance if the evidence is high we would say it is an intervention is a “standard of practice”

If the evidence is low but positive we might call the treatment an “option for care”

Morgenthaler, T., Kramer, M., Alessi, C., Friedman, L., Boehlecke, B., Brown, T., ... & Swick, T. (2006). Practice parameters for the psychological and behavioral treatment of insomnia: an update. An American Academy of Sleep Medicine report. SLEEP-NEW YORK THEN WESTCHESTER-, 29(11), 1415.

Grade Approach to Clinical Recommendations

Below is an example of a clinical claim about sleep treatment that was made using a similar approach to the Grade approach.

Recommendation: Psychological and behavioral interventions are effective and recommended in the treatment of primary chronic insomnia .

Recommendation Level: This is a clinical standard

Review of Evidence: This is a new recommendation implied but not specifically stated in the prior practice parameters. There were 17 studies identified in the current review that evaluated the effects of treatment for primary insomnia, including 5 randomized control trials with level I evidence that demonstrated the effectiveness of psychological and behavioral interventions.

Plain Language Translation: Psychological and Behavioral treatmentsare effective in treating chronic insomnia and should be offered to all patients with insomnia as a standard of care.

Morgenthaler, T., Kramer, M., Alessi, C., Friedman, L., Boehlecke, B., Brown, T., ... & Swick, T. (2006). Practice parameters for the psychological and behavioral treatment of insomnia: an update. An American Academy of Sleep Medicine report. SLEEP-NEW YORK THEN WESTCHESTER-, 29(11), 1415.

Risk of Bias – Blinding

Risk of Bias – General

Blinding: Keeps the people administering the study “blind” to the which group they are in or working with. It reduces the risk of unconscious and conscious bias.

Patient Blinded: If patients are aware of receiving the treatment it can impact outcomes. This is often inevitable in psychological research.

Caregiver blinded: The provider offering treatment can impact the outcomes with unconscious behaviors.

Assessor blinded: The person administering the tests or assessments can unconsciously or consciously impact the outcomes of a study with their knowledge of what group a participant is in.

Base Line Differences: Happen when there is a difference between the control group and treatment group at start of study.

Selection Bias: When the subjects chosen could impact the outcomes of a study.

Attrition: People leaving the study during the study.

Timing: Stopping study prior to timing expected to find benefit.

Important Factors in Study Design

It is important to assess the quality of a study…

Reasons to downgrade the quality of evidence (i.e. factors that lower your confidence in estimated effect): Risk of bias – This is the risk that the researcher could

consciously or indirectly influence the outcome of a study. Inconsistency – This occurs when there is conflicting data in

different studies e.g. one study implies a treatment is effective another implies less efficacy.

Indirectness – Does the study compare groups or interventions that are similar.

Imprecision – This happens when there is a small sample size (number of participants in a study) or the statistics have a wide variation in them. Particularly when the confidence interval is wide.

Reporting bias – The tendency of journals to publish positive results, profit motive, or

Evidence GRADE: A,B,C,D

References

Morgenthaler, T., Kramer, M., Alessi, C., Friedman, L., Boehlecke, B., Brown, T., & Swick, T. (2006). Practice parameters for the psychological and behavioral treatment of insomnia: an update. An American Academy of Sleep Medicine report. SLEEP-NEW YORK THEN WESTCHESTER-, 29(11), 1415.

Schunemann, H., Ahmed, F., & Morgan., R. (2009) Guideline Developmetn Using GRADE. McMaster University and Center for Disease Control. Pulled from web: http://www.cdc.gov/vaccines/acip/recs/GRADE/downloads/guide-dev-grade.pdf