8/3/2019 2-RCT

1/29

Understanding articles aboutUnderstanding articles about

treatmenttreatment

Dan Merenstein, MDDan Merenstein, MDAssistant Professor and Director of Research Programs,Assistant Professor and Director of Research Programs,

Family MedicineFamily Medicine

Georgetown UniversityGeorgetown University

8/3/2019 2-RCT

2/29

ObjectivesObjectives

Explain benefits ofExplain benefits ofRCTsRCTs

Internal and External ValidityInternal and External Validity

Allocation ConcealmentAllocation Concealment BlindingBlinding

LimitationsLimitations

8/3/2019 2-RCT

3/29

ProbioticsProbiotics

Live microorganisms when administered in

adequate amounts confer a beneficial health

effect on the host- WHO definition

All yogurts have at least two probiotics

(Lactobacillus bulgaricus and

Streptococcus thermophilus)

8/3/2019 2-RCT

4/29

CaseCase

You are seeing a 34 y/o woman who has 3You are seeing a 34 y/o woman who has 3

kids and has heard a lot about these newkids and has heard a lot about these newmiracle products called probiotics. Shemiracle products called probiotics. She

wants to know if it will keep her kidswants to know if it will keep her kids

healthy and if so what specifically shouldhealthy and if so what specifically should

she give them.she give them.

8/3/2019 2-RCT

5/29

Study MethodsStudy Methods

to Answer This Questionto Answer This Question

EpidemiologyEpidemiology: Countries that consume more: Countries that consume more

probiotics have less episodes of ear infectionsprobiotics have less episodes of ear infections

BiologyBiology: LGG ointment increases T cells in rats: LGG ointment increases T cells in rats

Case reportCase report: It worked on one patient: It worked on one patient

Case-seriesCase-series: It worked on a bunch of patients: It worked on a bunch of patients

Randomized controlled trialRandomized controlled trial: 1/2 get probiotic, 1/2: 1/2 get probiotic, 1/2placebo. No one knows who until the end who tookplacebo. No one knows who until the end who took

whatwhat

8/3/2019 2-RCT

6/29

Randomization is theRandomization is the

best protection againstbest protection againstbeing misleadbeing mislead

8/3/2019 2-RCT

7/29

Levels of Evidence (LOE)Levels of Evidence (LOE)

Centre for Evidence-Based Medicine,Centre for Evidence-Based Medicine,

OxfordOxford

Expert Opinion: LOE = 5Expert Opinion: LOE = 5 Case Series: LOE = 4Case Series: LOE = 4

Case Control: LOE = 3bCase Control: LOE = 3b

RCT: LOE = 1bRCT: LOE = 1b

Systematic review with homogeneity = 1aSystematic review with homogeneity = 1a http://www.http://www.cebmcebm..net/levels_of_evidencenet/levels_of_evidence.asp.asp

8/3/2019 2-RCT

8/29

Relative Risk

Reduction

Case fatality

rate

Historical

control 42% 38.3%

Randomized

controlled trial 31% 19.6%Concealed

Allocation 18% 12.1%

32 controlled trials of anticoagulation in acute32 controlled trials of anticoagulation in acuteMIMI

Results by type of study:Results by type of study:

The value of randomizationThe value of randomization

8/3/2019 2-RCT

9/29

Come Up Front for SnackCome Up Front for Snack There are 10 drinksThere are 10 drinks

Some are regular yogurt with two starterSome are regular yogurt with two starterculturescultures

Others are identical except we add BB-12 atOthers are identical except we add BB-12 atvery high dosevery high dose

Taste, smell, compare next to each otherTaste, smell, compare next to each other

Look at the bottles and consistencyLook at the bottles and consistency

8/3/2019 2-RCT

10/29

Strawberry YogurtStrawberry Yogurt

Can the research assistant tell which is theCan the research assistant tell which is theactive or placebo-allocation and blindingactive or placebo-allocation and blinding

Can participants tell which is active orCan participants tell which is active orplacebo-blindingplacebo-blinding

Is this something you can see yourself orIs this something you can see yourself orpatient drinking-external validitypatient drinking-external validity

Is the amount necessary to consumeIs the amount necessary to consume

reasonable-external validityreasonable-external validity What are the side effects and cost-externalWhat are the side effects and cost-external

validityvalidity

8/3/2019 2-RCT

11/29

Patient Oriented Evidence thatPatient Oriented Evidence that

Matters (POEM)Matters (POEM)

Disease Oriented Evidence (DOE)Disease Oriented Evidence (DOE)

Generally are RCT and meta-analysis butGenerally are RCT and meta-analysis but

dondont have to bet have to be

CAST ExampleCAST Example

8/3/2019 2-RCT

12/29

ValidityValidity

Internal validityInternal validity: How well was the study: How well was the study

done? Do the results reflect thedone? Do the results reflect the truthtruth??

External validityExternal validity: can I apply these results: can I apply these results

toto MYMY patients?patients?

8/3/2019 2-RCT

13/29

Determining ValidityDetermining Validity Read the methods sectionRead the methods section

What is the primary outcomeWhat is the primary outcome Study design flaws areStudy design flaws are commoncommon, but are, but are

theythey fatalfatal??

8/3/2019 2-RCT

14/29

Potential Subjects

Conducting a StudyConducting a Study

Actual

Subjects

A BRandomization

Blinding, etc

Trial starts

Concealed

Allocation

8/3/2019 2-RCT

15/29

Was allocation assignmentWas allocation assignment

concealedconcealed??

Concealed allocationConcealed allocation blindingblinding

Blinding can occur without concealment andBlinding can occur without concealment and

concealment can occur without blindingconcealment can occur without blinding

UVA example- surfactant in the NICUUVA example- surfactant in the NICU

Allocation can be concealed in an unblindedAllocation can be concealed in an unblinded

studystudy

PT vs surgery for knee DJDPT vs surgery for knee DJDMoseley JB, O'Malley K, Petersen NJ, et al. N Engl J Med 2002; 347:81-8Moseley JB, O'Malley K, Petersen NJ, et al. N Engl J Med 2002; 347:81-8 ..

8/3/2019 2-RCT

16/29

Was studyWas study double-blindeddouble-blinded??

Did theDid thepatientspatients know to which group they wereknow to which group they were

assigned?assigned?

Did theDid the treatingtreatingphysician know?physician know?

Triple blinding-Did investigatorsTriple blinding-Did investigators assessingassessing

outcomes knowoutcomes know

8/3/2019 2-RCT

17/29

Were intervention and controlWere intervention and control

groups similar?groups similar?

Table 1 of most studiesTable 1 of most studies

Randomization is best way to avoid bias,Randomization is best way to avoid bias,though imbalances still can occur (especiallythough imbalances still can occur (especially

if allocation was not concealed)if allocation was not concealed)

Review table 1 for both clinical and statisticalReview table 1 for both clinical and statistical

differencesdifferences

Small differences sometimes are importantSmall differences sometimes are important

8/3/2019 2-RCT

18/29

Participant DemographicsParticipant DemographicsNo.No. SubjectsSubjects Active=314Active=314 Placebo=324Placebo=324

GenderGender

--Male--Male 157 ( 50.0%)157 ( 50.0%) 172 ( 53.1%)172 ( 53.1%)----FemaleFemale 157 ( 50.0%)157 ( 50.0%) 152 ( 46.9%)152 ( 46.9%)

AgeAge

--MEAN (--MEAN (sdsd)) 4.86 (+1.12)4.86 (+1.12) 4.94 (+1.13)4.94 (+1.13)

HoursHours//weekweekinin schoolschool

----LessLess thanthan 1515 hourshours 6 ( 1.9%)6 ( 1.9%) 5 ( 1.5%)5 ( 1.5%)

--15-20--15-20 hourshours 33 ( 10.5%)33 ( 10.5%) 44 ( 13.6%)44 ( 13.6%)--21-30--21-30 hourshours 62 ( 19.7%)62 ( 19.7%) 47 ( 14.5%)47 ( 14.5%)

--More--More thanthan 4040 hourshours 143 ( 45.5%)143 ( 45.5%) 159 ( 49.1%)159 ( 49.1%)

--31-40--31-40 hourshours 70 ( 22.3%)70 ( 22.3%) 69 ( 21.3%)69 ( 21.3%)

IncomeIncome

----LessLess thanthan $15,000$15,000 13 ( 4.3%)13 ( 4.3%) 29 ( 9.1%)29 ( 9.1%)

--$15,000-$30,000--$15,000-$30,000 41 ( 13.7%)41 ( 13.7%) 40 ( 12.6%)40 ( 12.6%)

--$30,001-$50,000--$30,001-$50,000 28 ( 9.4%)28 ( 9.4%) 26 ( 8.2%)26 ( 8.2%)

--$50,001-$75,000--$50,001-$75,000 38 ( 12.7%)38 ( 12.7%) 38 ( 11.9%)38 ( 11.9%)

--$75,001-$100,000--$75,001-$100,000 55 ( 18.4%)55 ( 18.4%) 43 ( 13.5%)43 ( 13.5%)

--More--More thanthan $100,000$100,000 124 ( 41.5%)124 ( 41.5%) 142 ( 44.7%)142 ( 44.7%)

8/3/2019 2-RCT

19/29

Are the study patients similar toAre the study patients similar to

yours?yours? Addresses generalizability of results toAddresses generalizability of results to

your practice (external validity)your practice (external validity)

Examples:Examples:

Probiotic supplementsProbiotic supplements

Very high dose probioticsVery high dose probiotics

8/3/2019 2-RCT

20/29

Were all the patients properly accountedWere all the patients properly accounted

for at its conclusion?for at its conclusion?

Complete follow-up?Complete follow-up?

Intention to treatIntention to treat analysis?analysis?

Patients are analyzed in the groups to which they are assignedPatients are analyzed in the groups to which they are assigned

Attempts to reflectAttempts to reflect real worldreal world clinical situations in which not allclinical situations in which not all

patients are compliantpatients are compliant

Watch when they compare compliers and exclude non-compliersWatch when they compare compliers and exclude non-compliers

Compliant subjects always do better overallCompliant subjects always do better overall

8/3/2019 2-RCT

21/29

Participant Flow ChartParticipant Flow Chart

324 Control

681 Assessed for Eligibility

106 Not Eligible

638 Randomized

745 Screened

314 Intervention

8/3/2019 2-RCT

22/29

NonfebrileNonfebrile Seizure IncidenceSeizure Incidence

8/3/2019 2-RCT

23/29

Diabetes Complications andDiabetes Complications and

Control TrialControl Trial Patients:Patients:

Type 1 diabetes, 13-39 years oldType 1 diabetes, 13-39 years old

No HTN, cholesterol or diabetic complicationsNo HTN, cholesterol or diabetic complications Willing to check BS QID, inject insulin 3-4 times/dayWilling to check BS QID, inject insulin 3-4 times/day

Monthly visits for 6.5 yearsMonthly visits for 6.5 years

Twice weekly phone follow-up x 6.5 yearsTwice weekly phone follow-up x 6.5 years

No co-morbid conditionsNo co-morbid conditions

Are these patients representative of type 2Are these patients representative of type 2diabetics seen in most offices?diabetics seen in most offices?

ADA uses these results to support tight glucose controlADA uses these results to support tight glucose controlin type 2 DMin type 2 DM

8/3/2019 2-RCT

24/29

Ethical DilemmaEthical Dilemma

A nonrandomized study showed that MVIA nonrandomized study showed that MVI

prevented neural tube defectsprevented neural tube defects

Does treating neonates with high levels ofDoes treating neonates with high levels of

oxygen prevent deathoxygen prevent death

Determination if PSA tests prevent prostateDetermination if PSA tests prevent prostate

cancer deathcancer death

Determination if Pap Smears preventDetermination if Pap Smears preventcervical deathcervical death

8/3/2019 2-RCT

25/29

New PSA StudiesNew PSA Studies Two studies, total N=253,000 menTwo studies, total N=253,000 men

Take home pointsTake home points

Seven fewer cancers for every 10,000 menSeven fewer cancers for every 10,000 men

screened and followed for 9 yearsscreened and followed for 9 years ExampleExample

One man is screened, + test leads to biopsyOne man is screened, + test leads to biopsy

There is a 1 in 50 chance that he was sparedThere is a 1 in 50 chance that he was spared

death in next 10 yearsdeath in next 10 yearsHowever 49/50 chance he was treatedHowever 49/50 chance he was treated

unnecessarily for a cancer that would have nounnecessarily for a cancer that would have noimpact on his lifeimpact on his life

8/3/2019 2-RCT

26/29

Types ofTypes ofRCTsRCTs

Efficacy- ideal or laboratory circumstancesEfficacy- ideal or laboratory circumstances

Effectiveness-ordinary or real life situationsEffectiveness-ordinary or real life situations

Practical Clinical Trials-designedPractical Clinical Trials-designedspecifically to answer questions by decisionspecifically to answer questions by decision

makers. Generally have more than onemakers. Generally have more than one

outcomeoutcome

8/3/2019 2-RCT

27/29

RCT ShortcomingsRCT Shortcomings

External ValidityExternal Validity

AdherenceAdherence

RandomizationRandomization ExpenseExpense

BiasBias

8/3/2019 2-RCT

28/29

EvidenceEvidence

OBJECTIVES: To determine whether parachutes are effective inOBJECTIVES: To determine whether parachutes are effective inpreventing major trauma related to gravitational challenge. DESIGN:preventing major trauma related to gravitational challenge. DESIGN:Systematic review ofSystematic review ofrandomisedrandomised controlled trials. DATA SOURCES:controlled trials. DATA SOURCES:Medline, Web of Science,Medline, Web of Science, EmbaseEmbase, and the Cochrane Library, and the Cochrane Librarydatabases; appropriate internet sites and citation lists. STUDYdatabases; appropriate internet sites and citation lists. STUDYSELECTION: Studies showing the effects of using a parachute duringSELECTION: Studies showing the effects of using a parachute duringfree fall. MAIN OUTCOME MEASURE: Death or major trauma,free fall. MAIN OUTCOME MEASURE: Death or major trauma,

defined as an injury severity score > 15. RESULTS: We were unabledefined as an injury severity score > 15. RESULTS: We were unableto identify anyto identify any randomisedrandomised controlled trials of parachute intervention.controlled trials of parachute intervention.CONCLUSIONS: As with many interventions intended to prevent illCONCLUSIONS: As with many interventions intended to prevent illhealth, the effectiveness of parachutes has not been subjected tohealth, the effectiveness of parachutes has not been subjected torigorous evaluation by usingrigorous evaluation by using randomisedrandomised controlled trials. Advocatescontrolled trials. Advocatesof evidence based medicine haveof evidence based medicine have criticisedcriticised the adoption ofthe adoption ofinterventions evaluated by using only observational data. We think thatinterventions evaluated by using only observational data. We think thateveryone might benefit if the most radical protagonists of evidenceeveryone might benefit if the most radical protagonists of evidencebased medicinebased medicine organisedorganised and participated in a double blind,and participated in a double blind,randomisedrandomised, placebo controlled, crossover trial of the parachute., placebo controlled, crossover trial of the parachute.

8/3/2019 2-RCT

29/29

SummarySummary

Determine RelevanceDetermine Relevance

Is it a common POEM that willIs it a common POEM that will requirerequire you toyou to changechange

your practice?your practice?

Determine ValidityDetermine Validity

Proper allocation and blindingProper allocation and blinding

Intention-to-treat analysisIntention-to-treat analysis

Weigh fatal vs. nonfatal flawsWeigh fatal vs. nonfatal flaws

Ideal RCT: study of patients similar to yoursIdeal RCT: study of patients similar to yours