JOURNAL APPRAISALRandomized Controlled Trial (RCT)
Ann Meredith U. Garcia, MD, DPCP
RANDOMIZED CONTROLLED TRIAL (RCT) Type of scientific (often medical) experiment, where the people being
studied are randomly allocated to the different treatments under study
Gold standard for a clinical trial Often used to test the efficacy or effectiveness of various types of
medical interventions and may provide information about adverse effects, such as drug reactions
RANDOMIZED CONTROLLED TRIAL (RCT) Contains control groups, in which groups receiving the experimental
treatment are compared with control groups receiving: No treatment (a placebo-controlled study) Previously tested treatment (a positive-control study)
RANDOMIZED CONTROLLED TRIAL (RCT) Random assignment of intervention is done after subjects have
been assessed for eligibility and recruited, but before the intervention to be studied begins.
After randomization, the two (or more) groups of subjects are followed in exactly the same way. The only differences between the care they receive, for example, in terms
of procedures, tests, outpatient visits, and follow-up calls, should be those intrinsic to the treatments being compared.
The most important advantage of proper randomization is that it minimizes allocation bias, balancing both known and unknown prognostic factors, in the assignment of treatments.
CLASSIFICATION OF RCTs By study design
Parallel-group – each participant is randomly assigned to a group, and all the participants in the group receive (or do not receive) an intervention
Crossover – over time, each participant receives (or does not receive) an intervention in a random sequence
Cluster – pre-existing groups of participants are randomly selected to receive (or not receive) an intervention
Factorial – each participant is randomly assigned to a group that receives a particular combination of interventions or non-interventions
CLASSIFICATION OF RCTs By outcome of interest (efficacy vs. effectiveness)
Explanatory RCTs – test efficacy in a research setting with highly selected participants and under highly controlled conditions
Pragmatic RCTs – test effectiveness in everyday practice with relatively unselected participants and under flexible conditions
By hypothesis (superiority vs. noninferiority vs. equivalence) Superiority trials – one intervention is hypothesized to be superior
to another in a statistically significant way Noninferiority trials – to determine whether a new treatment is no
worse than a reference treatment Equivalence trials – hypothesize that two interventions are
indistinguishable from each other
RANDOMIZATION Advantages of proper randomization:
Eliminates bias in treatment assignment, specifically selection bias and confounding
Facilitates blinding of the identity of treatments from investigators, participants, and assessors
Permits the use of probability theory to express the likelihood that any difference in outcome between treatment groups merely indicates chance
RANDOMIZATION Two processes:
Randomization procedure – to generate an unpredictable sequence of allocations
Allocation concealment – stringent precautions taken to ensure that the group assignment of patients are not revealed prior to definitively allocating them to their respective groups Sequentially numbered, opaque, sealed envelopes (SNOSE) Sequentially numbered containers Pharmacy controlled randomization Central randomization
SAMPLE SIZE Affects reliability
As the sample size increases, a trial may be able to demonstrate a significant effect of the treatment, even if this effect is small.
BLINDING Procedures that prevent study participants, caregivers, or
outcome assessors from knowing which intervention was received
Sometimes inappropriate or impossible to perform in an RCT (e.g., if an RCT involves a treatment in which active participation of the patient is necessary)
IMPORTANT CONSIDERATIONS IN THE ANALYSIS OF RCT DATA “Intention-to-treat analysis” – the extent to which the groups
can be analyzed exactly as they existed upon randomization “Pure" intention-to-treat analysis – possible only when complete
outcome data are available for all randomized subjects When some outcome data are missing, options include analyzing
only cases with known outcomes and using imputed data. The more that analyses can include all participants in the groups to
which they were randomized, the less bias that an RCT will be subject to.
Subgroup analysis – often discouraged because multiple comparisons may produce false positive findings that cannot be confirmed by other studies
LIMITATIONS OF EXTERNAL VALIDITY Factors that can affect RCTs' external validity include:
Where the RCT was performed Characteristics of the patients Study procedures Outcome measures Incomplete reporting of adverse effects of interventions
CLINICAL SCENARIO A 56/F ECOG 1 patient comes to your clinic for second
opinion after being diagnosed with unresectable Stage III pancreatic cancer. Her first doctor advised her to undergo chemotherapy with gemcitabine alone, but she is reluctant to proceed because she has an acquaintance with the same condition who died shortly after receiving a few cycles of the same regimen. She found out on Google that gemcitabine monotherapy is indeed the current standard of care for advanced pancreatic cancer, but she is still interested to know if there are other new treatment options on top of this that may improve survival in patients like her.
CLINICAL QUESTIONAmong adult patients with unresectable pancreatic cancer, are there other treatment options which have been shown to improve survival compared to gemcitabine alone? P: Adult patients with unresectable pancreatic cancer I: Other treatment options vs. gemcitabine alone O: Survival M: RCT
APPRAISING DIRECTNESSClinical Question Research Question
P Adult patients with unresectable pancreatic cancer
Adult patients with locally advanced or metastatic pancreatic adenocarcinoma
I Other treatment options vs. gemcitabine alone
Gemcitabine plus erlotinib (100 or 150 mg/d orally) vs. gemcitabine plus placebo
O Survival Overall survival
M RCT RCT
APPRAISING VALIDITY1. Were patients randomly
assigned to treatment groups?
YES
*Tip: To check if randomization was performed, look for the words ‘randomization’, ‘randomized’ or ‘randomly allocated/assigned’ in the title, abstract or methodology.
APPRAISING VALIDITY2. Was allocation
concealed?
NOT STATED
*Tip: To ascertain allocation concealment, look for a phrase or paragraph stating one of the following: that a clinician recruiting a patient
must contact a remote center to obtain a patient treatment assignment;
that envelopes containing the allocation sequence were sealed and numbered; or
that vehicles or packages containing the medications were indistinguishable.
APPRAISING VALIDITY3. Were baseline
characteristics similar at the start of the trial?
YES
APPRAISING VALIDITY4. Were patients blinded to
treatment assignment?
YES
5. Were caregivers blinded to treatment assignment?
YES
*Tip: To decide if caregivers were blinded in the conduct of the trial, look for use of identical preparations.
APPRAISING VALIDITY6. Were outcome
assessors blinded to treatment assignment?
YES
*Outcome assessors can be the patients themselves, or their caregivers.
*Sometimes, the outcome assessors are study personnel), responsible for deciding how a patient responded to therapy.
APPRAISING VALIDITY7. Were all patients analyzed in the
groups to which they were originally randomized?
YES
*Analyzed in their original treatment group (intention-to-treat analysis), or excluded from the study (censored analysis)
*Tip: To make sure that investigators did not censor non-compliant patients from the analysis, look for the term ‘intention- to-treat’ under the analysis section of the article. If not stated explicitly, look for some indication that patients retained their original assignment even if they were non-compliant. If this is not stated as well, a last resort would be to check that the number of patients randomized is equal to the number of patients analyzed at the end of the study.
APPRAISING VALIDITY8. Was follow-up rate
adequate?
YES
N = 285 N = 284
239 / 282 247 / 280
3 4
(239 + 3) / (282 + 3) = 242 / 285(84.9%)
(247 + 0) / (280 + 4) = 247 / 284(87%)
(239 + 0) / (282 + 3) = 239 / 285(83.9%)
(247 + 4) / (280 + 4) = 251 / 284(88.4%)
APPRAISING VALIDITY
Were patients randomly assigned to treatment groups?
Was allocation concealed? ☐
Were baseline characteristics similar at the start of the trial?
Were patients blinded to treatment assignment?
Were caregivers blinded to treatment assignment?
Were outcome assessors blinded to treatment assignment?
Were all patients analyzed in the groups to which they were originally randomized?
Was follow-up rate adequate?
APPRAISING THE RESULTS
APPRAISING THE RESULTS1. How large was the effect of treatment?
“The final analysis was conducted after 486 deaths (239 on erlotinib and gemcitabine and 247 on placebo and gemcitabine). Overall survival was significantly longer in the erlotinib and gemcitabine arm with an estimated HR of 0.82 (95% CI, 0.69 to 0.99; P = .038; log-rank test stratified for performance status, extent of disease, and pain score at baseline; Fig 1A). Median survival times were 6.24 months versus 5.91 months for the erlotinib and gemcitabine versus placebo and gemcitabine groups with 1-year survival rates of 23% (95% CI, 18% to 28%) and 17% (95% CI, 12% to 21%), respectively (P = .023). A multivariate Cox regression analysis showed that erlotinib treatment (HR, 0.82; 95% CI, 0.69 to 0.99; P = .04) and female sex (P = .03) were significantly associated with longer overall survival. While there was an imbalance in male:female ratio between the arms, the treatment effect remains significant when adjusted for sex. Progression-free survival was significantly longer in the erlotinib and gemcitabine arm than the placebo and gemcitabine arm with an estimated HR of 0.77 (95% CI, 0.64 to 0.92; P = .004; log-rank test stratified for performance status, extent of disease, and pain score at baseline; median, 3.75 months v 3.55 months; Fig 1B).“
`
APPRAISING THE RESULTS2. How precise was the estimate of the treatment effect?
`
`
`
ASSESSING APPLICABILITYSCRAPS (Sex, Co-morbidities, Race, Age,
Pathology, Socioeconomic issues)
INDIVIDUALIZING THE RESULTS
83%(1-year survival rate = 17%)
0.82
INDIVIDUALIZING THE RESULTS
83% X 0.82 = 68.06%
83% – 68.06% = 14.94%
100 / 14.94 = 7