Methods in Clinical Cancer Research:Introduction

Elizabeth Garrett-Mayer, PhDProfessor of BiostatisticsHollings Cancer Center

Medical University of South Carolina

Lecture 1January 6, 2015

Methods in Clinical Cancer Research

• Course website: http://people.musc.edu/~elg26/teaching/MCCR2015/MCCR2015.htm

• Tuesdays and Thursdays, 1:30-3pm• BSB302

• Primary Instructor email: garrettm@musc.edu

Course Objectives

• At the end of the course, students should be able to:– Understand the key components required for designing,

activating and implementing a cancer clinical trial. – Write a proposal for a cancer clinical trial, including

objectives, endpoints, trial design, patient population selection, and have some understanding of the required sample size and analytic techniques used to analyze the data at the end of the trial.

– Effectively review and critique clinical trial protocols and published cancer clinical trials research.

Course Description• Didactic lectures will cover the following areas:

(1) clinical and statistical design of phase I, II and III trials; (2) incorporation of correlative and biomarkers in clinical trials, (3) considerations in chemotherapy, surgery, radiation and multimodality trials, (4) quality of life and other patient reported outcomes in cancer research, (5) the protocol review and IRB process, (6) informed consent, (7) data collection, trial monitoring and investigator responsibilities, (8) grants, cooperative groups in oncology and pharma.

• Other topics are incorporated as well, (e.g., disparities research). In addition to the didactic portions of the training, each trainee will have a clinical research proposal which will be developed into a “letter of intent” (LOI) for a clinical trial.

• In addition to the didactic sessions , contact hours will take the form of a journal club where clinical research papers from journals such as Clinical Cancer Research or Journal of Clinical Oncology are discussed, and protocols that are being undertaken at HCC are reviewed and discussed.

• Trainees will also be required to attend and take part in the HCC Protocol Review Committee’s monthly meetings (meetings occur every 3 weeks). This will allow the trainees to be exposed to a variety of studies ranging from Phase I to III cancer trials, in addition to observational, translational and qualitative research studies.

• Trainees will also be encouraged to attend one or more of the HCC Data Safety and Monitoring Board meetings to gain exposure to issues of trial review and monitoring.

Texts and Assessments• Recommended Texts for Reference:

– Clinical Trials: A Methodologic Perspective (Piantadosi)– Oncology Clinical Trials (Kelly & Halabi)– Principles of Anti-Cancer Drug Development (Hidalgo, Eckhardt, Garrett-Mayer, Clendennin)

• Assessment of Students: – Students will be graded based on the following components where each assignment is given

numeric score, according to the Merit Grades for the MUSC grading system. – Written reviews of protocols, given as assignments. There will be 3-4 protocols assigned and

the review will be structured with particular questions about appropriateness of study design, clarity of the study aims, incorporation of early stopping rules in the trial design, etc. (45% of grade)

– Oral presentation of journal article presenting results of a cancer clinical trial. The article will be selected by the student and Dr. Garrett-Mayer. The student will present to the class an overall summary of the trial and provide a critique of the methods employed. (25% of grade)

– Submitted LOI: The LOI will be submitted twice. First, a draft will be submitted about two-thirds through the course. Dr. Garrett-Mayer will provide feedback. This first draft will constitute 15% of the total grade. The final LOI will be submitted as the ‘final’ and will also count for 15% of the course grade. Total: 30% of grade

Assignments policy

• Homeworks are due by 5pm on the due date. All homeworks should be emailed to the primary instructor (garrettm@musc.edu) or turned in at lecture time. Asking for extensions on homeworks is strongly discouraged. However, it is expected that, on occasion, extenuating circumstances may arise. Therefore, the policy is that each student may request an extension on homework twice and the extension is to be no more than 2 days. You must notify the primary instructor that you are requesting an extension before the time the assignment is due. After using two extensions, no more extensions will be granted except with a medical note.

Classroom Etiquette• Attention to material: Laptops are permitted in class, but it is

expected that if they are used, it is to follow along with the lecture. Email and web browsers should not be visited during class time. Checking phones during lecture is not acceptable. The instructors are giving their time and expertise. Be respectful and give them your attention.

• Classroom disruptions: Some of us have patients, clinic staff, family members and others who we need to be able to be in contact with during lectures. It is acceptable to bring pagers or cell phones to class. Please be sure they are on silent mode. If you need to leave during lecture to take a phone call, or make a phone call, please do so. However, this should be a relatively rare occurrence. Texting and emailing during lecture time is not acceptable.

Schedule (work in progress)

Lecture number

Date Topic Instructor

1 Tu Jan 6 Introduction EGM2 Th Jan 8 Phase I trials: practical considerations EGM3 Tu Jan 13 Phase I trial designs Britten?4 Th Jan 15 Phase I in practice: current

topics/controversiesBritten?

5 Tu Jan 20 Phase II trials: practical considerations EGM6 Th Jan 22 Phase II trial designs EGM7 Tu Jan 27 Phase II in practice: current

topics/controversiesEGM

8 Th Jan 29 Phase III trials: practical considerations EGM9 Tu Feb 3 Phase III trial designs EGM

10 Th Feb 5 Phase III trials: practical considerations EGM11 Tu Feb 10 Observational studies 12 Th Feb 12 Biomarker clinical trial designs EGM13 Tu Feb 17 14 Th Feb 19 Endpoint selection issues EGM15 Tu Feb 24 Power calculations EGM

Tentative Lecture Schedule:

Schedule (work in progress)Tentative Lecture Schedule:

16 Th Feb 26 Correlative studies EGM17 Tu Mar 3 Imaging endpoints Ravenel18 Th Mar 5 Quality of Life and patient reported outcomes Sterba19 Tu Mar 10 Spring break (no class) Th Mar 12 Spring break (no class) Tu Mar 17 Data collection and privacy issues EGM

20 Th Mar 19 Informed consent Stuart21 Tu Mar 24 Protocol review and IRB process EGM22 Th Mar 26 Data safety and monitoring EGM23 Tu Mar 31 Disparities research Ford24 Th Apr 2 Prevention and Control studies Wallace25 Tu Apr 7 Data safety and monitoring EGM26 Th Apr 9 Grants, cooperative groups and pharma EGM27 Tu Apr 14 Radiation and Multimodality Trials Graham Warren28 Th Apr 16 Tobacco Cessation in Clinical Cancer Trials Graham Warren29 Tu Apr 21 Prediction vs. Association 30 Th Apr 23 Intention to Treat and Compliance 31 Tu Apr 28

Lecture Notes

• Every lecturer will have his/her own style• Notes may be – prepared ahead of time and posted– Prepared and posted after the lecture– Nonexistent

• Lecture notes will NOT be printed by the instructors prior to lecture.

• If they are available and you would like a paper copy, it is your responsibility to print them out.

Heterogeneous Population

• K-12 Training Program

• Biostats & Epi grad students

Questions?

Types of Research Studies in Cancer

• Basic Science• Translational• Clinical– Exploratory/Pilot/Correlative– Phase I– Phase II– Phase III– Other: e.g. prevention, survivorship

• Epidemiological

Phases of Drug Development

• Phase I– Dose finding– Usually designed to find the highest safe dose.– 12-30 patients.– 2014 update: 12 - 700

• Phase II– Preliminary efficacy and safety– Generally not ‘head to head’ comparison– 20-80 patients

• Phase III– Definitive comparative trial against the standard of care– Usually hundreds or thousands of patients

Clinical Trials: the beginning

• Write a clinical trial protocol• Usually 70-180 pages• Not like writing a grant• Every detail spelled out: no page limit!• There are standard templates that can/should

be used.

Imagine….• You are principal investigator (PI) of a clinical trial• In the middle of the trial, you change careers• You are now an astronaut and fly to the moon• Meanwhile, a new patient is enrolled.• The new PI needs to know:

– How should the patient be assigned to a dose?– How should dose modifications occur?– What measurements should be taken and when?– What are the definition of the primary and 2ndary outcomes?– Who and how are the data to be reviewed for safety and

efficacy?

Statistical design and development of clinical trials

• Statistical considerations permeate the design and analytic plan

• Requires interaction with your statistician – Early interaction! Before the design (and budget)

are set in stone.– bad: “i have almost finished writing the protocol,

and then i will send to you to insert a statistical plan”

Where are the statisticians?• Academic cancer centers have biostatistics cores or

biostatistics shared resources• It is the role of these biostatisticians to help design clinical

trials• Find them!• Other places:

– University settings usually have biostatistics departments or divisions

– Pharma will have biostatisticians on site or have biostatistical consultants available• Small pharma/biotech will contract from “CROs”.

– Cooperative groups have statistical teams familiar with the group and its trials.

Statistical Considerations: 5 part process

I. Stating research aims/objectives

II. Determining your outcome measures

III. Choosing the experimental design

IV. The analytic plan

V. Sample size justification

(note: there are MANY other parts to the protocol! These are just the stats considerations).

Motivating Example

• Randomized Phase II study evaluating two administration schedules of flavopiridol given in timed sequential combination with cytosine arabinoside (ara-C) and mitoxantrone for adults with newly diagnosed, previously untreated, poor-risk acute myelogenous leukemias (AML)*

• Principal Investigator: Judy Karp • Two different administration schedules: – bolus – “hybrid bolus-infusion”

*Haematologica. 2012 Nov;97(11):1736-42

1. Stating research Aims• Authors devised a protocol, beginning with research aims (objectives)• Aims should be concrete and include measurable outcomes• Bad examples:

– To evaluate the effect of flavopiridol on cancer.– To see if flavopiridol improves cancer outcomes– To determine the safety of flavopiridol

• What is wrong with these aims?– what does “effect” mean? what kind of cancer, in what patients?– “Improves” compared to what? what is the outcome of interest?– How is “safe” defined?

• Think about how you are going to determine if this treatment works or not

1. Stating research aims

• Better examples: – To evaluate the efficacy of flavopiridol administered by

two different schedules followed by ara-C and mitoxantrone in adults with newly diagnosed AML with poor-risk features

– To describe the toxicities of flavopiridol administered by two different schedules followed by ara-C and mitoxantrone in adults with newly diagnosed AML with poor-risk features

• Keywords for aims:– determine, estimate, evaluate, describe, identify, compare– efficacy, safety, toxicity, survival

Devising your aims• Generally, there is ONE primary aim and your study is designed to

address the primary aim

• Very often:– Phase I: primary aim is finding the “recommended” dose– Phase II: primary aim is determining if there is sufficient efficacy

• Secondary aims:– Important, but do not drive the design– Examples in Phase I:

• describe pharmacokinetics• describe pharmacodynamic outcomes (e.g., methylation)• describe clinical responses

– Examples in Phase II:• determine/describe/compare overall survival • describe safety • evaluate changes in biomarkers• Assess/compare quality of life

Aims and Hypotheses

• Aims are often accompanied by hypotheses.• Stating the hypothesis to be tested can be a useful

guide for the analytic plan:• Examples of clinical research hypotheses: – The complete remission rate of patients in the bolus

infusion arm will be at least 55%– The complete remission rate of patients in the hybrid-

bolus infusion arm will be at least 55%– The median disease-free survival time across both

arms will be at least 14 months.

Q1: Complete the following with the best answer:The primary objective of this study is to______ the

_____of paclitaxel as second-line therapy in endometrial adenocarcinoma.

1. study…..effect2. observe….consequences3. evaluate….toxicity4. determine….endpoints5. show….financial benefits

II. Determining your outcome measures

• What is an endpoint or outcome?– patient-level measure of “effect” of interest– An outcome is measured on each patient in the study– it is QUANTIFIABLE

• Aim ≠ endpoint

• Example 1:– Aim: To evaluate the toxicity of paclitaxel as second-line therapy in endometrial

adenocarcinoma.– Outcome: grade 1, 2, 3 & 4 toxicities as defined by CTCAE v4

• Example 2:– Aim: To compare overall survival in hepatocellular carcinoma patients treated

with sorafenib vs. bevacizumab + erlotinib as first-line treatment.– Outcome: overall survival defined as the time from enrollment to death.

Aims and outcomes and parameters

• Your primary outcome is measured to address your primary aim

• The outcome measure will depend on the parameter of interest

• Examples of common parameters of interest in phase I:– the grade 3 & 4 toxicity rate– the maximum tolerated dose

• Examples of common parameters of interest in phase II: – response rate or complete remission rate– median progression-free survival– 6 month progression-free survival rate

• Example of common parameter of interest in phase III:– hazard ratio for overall survival

Parameter of interest vs. outcome

Parameter of interest Outcome

Response rate: proportion of patients with CR or PR

Response (Complete Response or Partial Response)

6 month overall survival Time from enrollment to death (or last follow-up)

Hazard ratio for overall survival Time from enrollment to death (or last follow-up)

Mean change in quality of life Difference in quality of life scores from baseline to follow-up

Q2: An investigator wrote a trial with the following primary aim:– To estimate the median progression-free survival (PFS) of

the combination of gemcitabine plus pazopanib (G+P) and to estimate the median PFS of the combination of gemcitabine plus docetaxel (G+T) in patients with previously treated, metastatic and/or locally advanced or recurrent soft tissue sarcoma

The primary outcome is:1. median progression-free survival2. progression-free survival, defined as the time from treatment

initiation to progression or death3. the hazard ratio comparing PFS in the G+T arm and the G+P

arm4. date of progression5. To estimate the median PFS in each arm

Q3: An investigator wrote a trial with the following primary aim:– To estimate the median progression-free survival (PFS) of the

combination of gemcitabine plus pazopanib (G+P) and to estimate the median PFS of the combination of gemcitabine plus docetaxel (G+T) in patients with previously treated, metastatic and/or locally advanced or recurrent soft tissue sarcoma

The parameter of interest is:1. median progression-free survival2. progression-free survival, defined as the time from treatment

initiation to progression or death3. the hazard ratio comparing PFS in the G+T arm and the G+P arm4. date of progression5. To estimate the median PFS in each arm

Q2 & Q3: Right and Wrong

1. median progression-free survival

2. progression-free survival, defined as the time from treatment initiation to progression or death

3. the hazard ratio comparing PFS in the G+T arm and the G+P arm

4. date of progression

5. To estimate the median PFS in each arm

The following are NOT endpoints• These are estimates of parameters:– response rate– median survival– AE rate– safety profile

• These describe the time course of the study in some way (don’t let the term ‘endpoint’ confuse you):

– length of time of treatment– time until patient goes off-study– length of study

III. Choosing the experimental design

• Based on the aims and the outcome, a design can be identified.

• Other considerations– patient population– accrual limitations– previous experience with the treatment of interest

in this or other populations– results from earlier phase studies

III. Choosing the experimental design

• There are common approaches within each phase of drug development

• However, there are often many options and seemingly small details that can make big differences.

• Two common ‘philosophies”– Frequentist– Bayesian

• Buzzword: “adaptive”

III. Choosing the experimental design

• Phase I:– how many dose levels and why?– combination or single agent?– one or multiple disease types?– is expansion at MTD feasible?– should a model-based design be used?

• Phase II– what is historical control rate?– is a “reference arm” needed because the historical control rate is not well-

defined (randomized phase II)?– is there more than one schedule being considered? (randomized phase II?)?– how well is safety profile defined?– is primary interest safety or efficacy (or both)?

AML Flavopiridol Trial• “This is a randomized Phase II study to evaluate two different

schedules of flavopiridol administration in combination with Ara-C and Mitoxantrone for response and toxicities. The primary outcome is complete remission.”

• The goals:– identify if either schedule is sufficiently efficacious– if both efficacious, to choose the better schedule.

• Study design:– Simon’s two-stage designs will be used in each arm which will allow an

arm to stop early if there is strong early evidence of futility (i.e., lack of efficacy). If both arms proceed through to the second stage and reject the null hypothesis, the schedule with the higher response rate will be selected for further study.

– A “pick the winner” approach will be used which has a 90% probability of selecting the best schedule if the true difference in CR rates is at least 15%.

IV. Analytic Plan

• Do you want to compare?• Do you want to estimate?• Do you want to test a hypothesis?• These questions, in regards to your stated

aims, will determine your analytic plan• Recall primary aim: To evaluate the efficacy of

two schedules of flavopiridol administration• Recall primary endpoint: complete remission

IV. Analytic Plan

• The analytic plan for the primary outcome usually involves two things:– estimating a parameter of interest– testing that the parameter is different than in another setting (e.g.,

different treatment)

• Estimation: a point estimate and some measure of precision• Example: “The CR rate in each arm will be estimated with its

confidence interval.”– this provides us with an estimate of the CR rate– it also provides us with a measure of precision about the estimate

Recall the 95% confidence interval• An interval that contains the true value of the parameter of interest 95% of the time.• “we are 95% confident that the true CR lies in this interval” • Example: below shows examples where the observed CR rate is 0.40.

95% confidence interval width depends on the sample size• Depending on the sample size, we have greater or less precision in our

estimate

IV. The Analytic Plan

• Hypothesis testing: Determining if the treatment is worthy of further study.

• Recall our hypotheses:– The complete remission rate of patients in the bolus

infusion arm will be at least 55%– The complete remission rate of patients in the hybrid-

bolus infusion arm will be at least 55%• What is a sufficiently LOW CR rate that we are not

interested in further pursuit?• Based on Dr. Karp’s experience, a CR rate of 30% is

too low to warrant further study.

IV. The Analytic Plan

• In each arm, we perform a hypothesis test:– Ho: p = 0.30 (null)– Ha: p = 0.55 (alternative)

• This test is performed using an exact binomial procedure or a chi-square test.

• The result is a p-value that provides “evidence” to either reject or fail to reject the null hypothesis

• In our a randomized phase II example: – the test is performed in each arm– the arms are not directly compared to one another (that

would require a different test)

Recall the p-value• p-value: the probability of observing a result as or more

extreme than we saw in our study if the null hypothesis is true.

• Small p-value: evidence that the null is not true (“significant result”)

• Large p-value: not sufficient evidence to reject the null (“not signficant”)

• Threshold for significance? The alpha level.– we usually think of 0.05, – but in phase II, often use 0.10.

P-value depends on the sample size

• For the same observed CR rate, a larger sample size will lead to a smaller p-value

• Important point: a large p-value does not always mean that “the null is true”. It may mean that the sample size was not large enough to reject the null.

• Another important point: a small p-value does not always mean that a clinically meaningful difference has been observed. It may be due to a rather large sample size and the effect is actually small.

→ Never interpret a p-value without also considering the clinical effect size.

IV. Analytic Plan

• Depends on the design and the goals• Example is a Phase II trial– single arm approach to analysis– compare to historical CR rate (e.g., 0.30)

• Phase I studies– often the analysis plan is descriptive– rare to see hypothesis testing (for primary aim)

• Phase III studies– head to head comparison of two groups– more common to see overall survival as the outcome of

interest. (time to event methods are required)

Q4: It is important to have an analytic plan written for each aim of your protocol because

1. it shows that you have considered how you are going to address each aim of the study

2. if you don’t, the protocol will not pass scientific peer-review at many institutions

3. it provides job security for the statisticians at your institution4. 1 and 2

V. Sample size justification• Two basic approaches

– power (most common)– precision

• Recall: – Limit number of participants treated at sub-therapeutic doses– Limit number of participants treated with ineffective therapy or exposed to

toxicity

• But, also we need to enroll enough patients to achieve our aims• Balancing act:

– Too few patients: you cannot answer the question– Too many patients: you have wasted resources and potentially exposed

patients to an ineffective treatment unnecessarily

• Most commonly motivate sample size by a hypothesis testing approach

Refresher of Alpha, Beta and Power

Type I error

Alpha = α = probability of Type I error (level of significance)Beta = β = probability of Type II errorPower = 1 - β

Accept Ho

Reject Ho

Ho is True Ho is NOT True

Type II error

V. Sample size justification• Usual motivation: hypothesis testing• Power = the probability of rejecting the null if it is false• Alpha = the probability of rejecting the null if it is true

• If a study is “underpowered”, it is too small to detect a clinically meaningful difference

• Example: Ho: p=0.30 vs. Ha: p=0.55– this is the assumed “clinically meaningful” difference– we chose power of 0.90 (beta = 0.10)– alpha was chosen to be 0.10

• Other design issues– we wanted to allow early stopping in each arm – chose Simon two-stage approach

“Plug and Chug”• With alpha = beta = 10% and Ho and Ha, a Simon two-stage design is

selected.

• The sample size per arm will be 15 patients or 35 patients (depending on early stopping)

• Total study size will be – N = 30 if both arms stop early– N = 50 if one arm stops early and one continues– N = 70 if both arms continue to the 2nd stage

The Simon’s two stage design we will use is defined as follows. Our null hypothesis is that the response rate is 30% and our alternative hypothesis is that the response rate is 55%. At the first stage we will enroll 15 patients in each arm. We will close accrual to an arm if < 4 responses are seen in that arm in the first stage. If 5 or more CRs are observed, then the arm(s) will remain open for an additional 20 patients per arm. An arm will be considered promising if the CR rate is >42% (i.e., at least 15 responses in 35 patients). This study is designed with power of 90% and a one-sided alpha of 10% for each arm.

V. Sample size justification

• Hypothesis testing is not always the way to go• Sometimes estimation is sufficient (but not always!

it is not an ‘escape route’)• In that case, sample size can be justified by precision• Example: with 45 patients, we will be able to

estimate the CR rate with a 95% confidence interval with half-width no greater than 0.15.

• Difficult part: is 0.15 half-width sufficiently precise? how to rationalize that?

Q5: Sample size is generally chosen based on

1. budget2. expected accrual3. the clinical effect size of interest4. type I and type II errors5. 3 and 46. all of the above

Feedback loop

• The process is actually not completely linear as stated

• Examples:– Design issues may cause you to change your

outcome or restate your aim– Accrual limitations may cause you to change the

design• “Dynamic process”

Additional aims (correlatives, etc.)• VERY important aims! • Same principles apply for stating aims, determining

outcomes, writing analytic plan• Usually power/sample size is less of a concern for

secondary aims• Design is generally DRIVEN by the primary aim– Changing design to accommodate 2ndary aims?– Suggests maybe that aim is not a 2ndary aim.

• “correlative” does not mean you can be vague! – these need to be well-conceived– often on biopsy tissue, pre post design– will you really learn anything?

Early Stopping Rules and Interim Analyses

• As a general rule, consider incorporating early stopping rules

• Why? Ethics and resources• Lots of reasons for stopping• Example: phase II designs– Early stopping for safety (one or more arms)– Early stopping for futility– Early stopping for harm

Implications of early/interim looks

• Early looks can/will affect– Type I error– Type II error

• Consequences? They need to be “built” into study design and power calculations

• Misconception: The DSMB will stop the study early if needed for safety or harm so there is no need to account for early looks.

• Having ‘independent’ review does not mean that the interim looks should not be built in to design.

Some good text books on trials • General Trials:

– Clinical Trials: A Methodologic Perspective (Piantadosi)– Clinical Trials (Meinert)

• Specific to Cancer:– Classic:

• Clinical Trials in Oncology (Green, Crowley, Benedetti and Smith)

– More recent• Principles of Anti-Cancer Drug Development (Hidalgo, Eckhardt, Garrett-Mayer,

Clendenin)• Oncology Clinical Trials: Successful Design, Conduct, and Analysis (Kelly, Halabi, Schilsky)

• Other references:– LoRusso et al. (2010) Clin Cancer Res; 16(6); 1710-8.– Seymour et al. (2010) Clin Cancer Res; 16(6); 1764-9.– Sullivan (2004) The Lancet; 5; Dec 2004 759 - 763