Comparative Effectiveness:Completing the Process to Improving Health

Daniel E. Ford, MD, MPHVice Dean for Clinical Investigation

Johns Hopkins UniversitySchool of Medicine

How much better are the outcomes? Were the right outcomes measured? What was the comparison group in the study? Were the patients in the study representative of the

broader population of patients for which the new therapy is targeted?

Questions to Ask After the First Randomized Clinical Trial Demonstrates Efficacy of a New Therapy

What kind of training would be required to provide the therapy described in the study?

How much of an investment is required to “try out” the new therapy?

How much does the new therapy cost? Will payers cover the costs? What are the

out-of-pocket costs? Will providers adhere to the study protocol

in the clinical trial?

Questions to Ask After the First RCT Demonstrates Efficacy of a New Therapy

Will patients adhere to the study protocol in the clinical trial?

After more widespread use, is the safety of the new therapy confirmed?

After more widespread use, is the effectiveness of the new therapy confirmed?

Questions to Ask After the First RCT Demonstrates Efficacy of a New Therapy

Warfarin has been used for years –does decrease cardiovascular events but needs monitoring, frequent dose adjustments, and has relatively high rate of bleeding complications

Dabigatran (Pradaxa) no need for monitoring and does not have a narrow window for therapeutic utility

Dabigatran As New Anticoagulant

Figure 1 Country distribution of mean time in therapeutic range in the RE-LY trial <ce:link id="celink10" locator="gr1"/>

Lars Wallentin , Salim Yusuf , Michael D Ezekowitz , Marco Alings , Marcus Flather , Maria Grazia Franzosi , Prem ...

Efficacy and safety of dabigatran compared with warfarin at different levels of international normalised ratio control for stroke prevention in atrial fibrillation: an analysis of the RE-LY trial

The Lancet Volume 376, Issue 9745 2010 975 - 983

http://dx.doi.org/10.1016/S0140-6736(10)61194-4

TRANSLATIONAL RESEARCH

Efficacy: Does an intervention improve outcomes under ideal conditions?

Effectiveness: Does the intervention improve outcomes in routine or usual clinical care?

Efficacy and Effectiveness

Table 1. Domains of the pragmatic–explanatory continuum indicator summary [20].

Chalkidou K et al. Clin Trials 2012;9:436-446

Copyright © by The Society for Clinical Trials

Definition from the IOM Report on Priorities for CER

 “The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.”

Comparative Effectiveness Research (CER)

Clinical trials in everyday care settings◦ Practice-based research

networks◦ Compare two active

treatments and not placebo

Observational assessments◦ Clinical research

registries◦ Administrative data (ex.

CMS) Evidence-synthesis

Comparative Effectiveness Research

Patients in ideal trial are different than usual care◦ Less comorbidity◦ More willing to accept side effects◦ Not paying for the treatment◦ More rigorous follow up to continue with treatment

Providers are different◦ Better training◦ Willing to follow protocol

Why Does Efficacy Not Always Translate to Effectiveness?

Clinical trials not that common Using observational methods

◦ Large datasets with less depth◦ Large sample needed to assess safety◦ Most vexing issue is how to control for case-mix? Patient

differences in who gets one intervention/treatment as compared to another Propensity scores Instrumental variables

Evaluating Effectiveness

Randomized 1663 patients with CHF to receive spironolactone or placebo

As part of study excluded all patients at high risk for elevated potassium

Study stopped early because found 30% reduction in risk of death

After study released, widely used in broader population of patients

Dramatic rise in rate of hyperkalemic-associated hospital admissions and deaths

Randomized Aldactone Evaluation Study RALES

Compare cardiac computed tomography (CCTA) to functional tests like exercise stress tests

150 centers and usual physicians interpreting results

Treatment based on tests is not protocol driven but based on clinicians judgement

Endpoints are death, MI, procedural complications, unstable angina, quality of life, resource use and cost effectiveness

PROMISE trial – PROspective Multicenter Imaging Study for Evaluation of Chest Pain

Figure 1. Number of RCTs and percentage of pRCTs from 1990 to 2010.RCT: randomized controlled trial; pRCT: pragmatic randomized controlled trial.

Chalkidou K et al. Clin Trials 2012;9:436-446

Copyright © by The Society for Clinical Trials

Efficacy to Effectiveness Comparative Effectiveness

◦ Deciding on best approach in usual care settings Knowledge Implementation or Knowledge

Transfer◦ More quickly moving evidence-based approaches to all

practice settings

Moving to Ultimate Implementation

Everett Rogers Agricultural Cooperative Agents Diffusion depends on:

◦ Relative advantage◦ Compatibility◦ Complexity◦ Trialability◦ Observability

Diffusion Theory

Copyright ©2008 BMJ Publishing Group Ltd.

Pronovost, P. J et al. BMJ 2008;337:a1714

Strategy for translating evidence into practice

Costs◦ Direct◦ Indirect

Effectiveness◦ Quality adjusted life years

Cost-effectivenessCosts(new) – Costs(old) /Effectiveness(new)-Effectiveness(old)

Cost-Effectiveness

Support clinicians who want to learn by reviewing the scientific literature and evaluate how they treat patients

Make clinical data available for analysis◦ Best data are electronic and comprehensive

Provide access to biostatisticians Encourage clinicians to participate in multicenter

studies Support RESEARCH on quality improvement

Creating an Environment to Promote Comparative Effectiveness Research

Online free courses◦ http://ocw.jhsph.edu/◦ Epidemiology, Biostatistics, Health Economics

MPH with Certificate in Comparative Effectiveness Research

Institute for Clinical and Translational Research 2 week intensive course

JHM Comparative Effectiveness Research Resources

Goal of medical research does not stop when treatment/diagnostic test evaluated in academic centers

Need to always consider ultimate customers of research

A learning health system has to include comparative effectiveness research

Conclusions