Lost Opportunities for Design Theory in Drug Development
-
Upload
samuel-mejia -
Category
Documents
-
view
45 -
download
4
description
Transcript of Lost Opportunities for Design Theory in Drug Development
(C)Stephen Senn 1
Lost Opportunities for Design Theory in Drug Development
Stephen Senn
(C)Stephen Senn 2
Basic Thesis
• Design theory has great potential in drug development
• But this potential is unrealised• Those working in so-called optimal design are so
ignorant of application realities that where their influence is not zero it is harmful
• On the other hand the understanding of design theory by biostatisticians is pitifully inadequate
• We must cooperate properly to cure this parlous state of affairs
(C)Stephen Senn 3
Outline
• Quick tutorial on cross-over trials• I shall then give two introductory examples
of nonsense– By leading design theoreticians– By leading biostatisticians
• I shall then consider ‘design nonsense’ further
• Some conclusions• After lunch a case-study
(C)Stephen Senn 4
Warning
• I am a biostatistician
• We are used to thinking of data matrices with rows as subjects and columns as measurements
• That means that we write sequences for designs with rows representing subjects and columns representing periods
(C)Stephen Senn 5
Cross-over Trials
Definition: A cross-over trial is one in which subjects are given sequences with the object of studying differences between individual treatments.
(C)Stephen Senn 6
An Example of an AB/BA cross-over in asthma
Sequence Period 1 Wash-out Period 2
for/sal formoterol salbutamol
sal/for salbutamol formoterol
(C)Stephen Senn 7
An Example from Rheumatism:2 doses of diclofenac and placebo
Period 1 Period 2 Period 3
D1 D2 PP D1 D2D2 P D1D2 D1 PP D2 D1D1 P D2
(C)Stephen Senn 8
Carry-over
Definition: Carry-over is the persistence (whether physically or in terms of effect) of a treatment applied in one period in a subsequent period of treatment.
If carry-over applies in a cross-over trial we shall, at some stage, observe the simultaneous effects of two or more treatments on given patients.
We may, however, not be aware that this is what we are observing and this ignorance may lead us to make errors in interpretation.
(C)Stephen Senn 9
Simple Carry-over
• Carry-over lasts for exactly one period
• It depends only on the engendering treatment and is unmodified by the perturbed treatment
• There is a huge literature proposing ‘optimal’ designs for this model
• There is no empirical evidence that any of this has been useful
(C)Stephen Senn 10
Three Period Bioequivalence Designs
• Three formulation designs in six sequences common.
• Subjects randomised in equal numbers to six possible sequences. – For example, 18 subjects, three on each of the
sequences ABC, ACB, BAC, BCA, CAB, CBA. – A = test formulation under fasting conditions, – B = test formulation under fed conditions – C = reference formulation under fed conditions.
(C)Stephen Senn 11
Period
Sequence 1 2 3
ABC A 0 B 1/6 C -1/6
ACB A 0 C -1/6 B 1/6
BAC B 1/6 A 0 C -1/6
BCA B 1/6 C -1/6 A 0
CBA C -1/6 A 0 B 1/6
CAB C -1/6 B 1/6 A 0
Weights for the Three Period Design:
not Adjusting for Carry-over
(C)Stephen Senn 12
Properties of these weights
• Sum 0 in any column, – eliminates the period effect.
• Sum 0 in any row – eliminates patient effect
• Sum 0 over cells labelled A– A has no part in definition of contrast
• Sum to 1 over the cells labelled B and to -1 over the cells labelled C– Estimate contrast B-C
(C)Stephen Senn 13
Period
Sequence 1 2 3
ABC A -1/24 Ba 4/24 Cb -3/24
ACB A 1/24 Ca -4/24 Bc 3/24
BAC B 4/24 Ab 2/24 Ca -6/24
BCA B 5/24 Cb -2/24 Ac -3/24
CBA C -4/24 Ac -2/24 Ba 6/24
CAB C -5/24 Bc 2/24 Ab 3/24
Weights for the Three Period Design:
Adjusting for Carry-over
(C)Stephen Senn 14
Period
Sequence 1 2 3
ABC A -1/24 Ba 4/24 Cb -3/24
ACB A 1/24 Ca -4/24 Bc 3/24
BAC B 4/24 Ab 2/24 Ca -6/24
BCA B 5/24 Cb -2/24 Ac -3/24
CBA C -4/24 Ac -2/24 Ba 6/24
CAB C -5/24 Bc 2/24 Ab 3/24
Weights for the Three Period Design:
Adjusting for Carry-over
(C)Stephen Senn 15
Properties of These Weights
• As before – Estimates B-C contrast– Eliminates, period and patient effect– Eliminates A
• Sum to zero over cells labelled a,b, and c– Eliminate simple carry-over
(C)Stephen Senn 16
Have We Got Something for Nothing?
• Sum of squares weights of first scheme is 1/3 (or 4/12)
• Sum of squares of weights of second scheme is 5/12
• Given independent homoscedastic within- patient errors, there is thus a 25% increase in variance
• Penalty for adjusting is loss of efficiency
(C)Stephen Senn 17
First ExampleSome Design Theory Nonsense
John, J. A., Russell, K. G., and Whitaker, D. (2004), "Crossover: An Algorithm for the Construction of Efficient Cross-over Designs," Statistics in Medicine, 23, 2645 - 2658.A cross-over experiment involves the application of sequences of treatments to several subjects over a number of time periods. It is thought that the observation made on each subject at the end of a time period may depend on the direct effect of the treatment applied in the current period, and the carry-over effects of the treatments applied in one or more previous periods. Various models have been proposed to explain the nature of the carry-over effects. An experimental design that is optimal under one model may not be optimal if a different model is the appropriate one. In this paper an algorithm is described to construct efficient cross-over designs for a range of models that involve the direct effects of the treatments and various functions of their carry-over effects. The effectiveness and flexibility of the algorithm are demonstrated by assessing its performance against numerous designs and models given in the literature.
(C)Stephen Senn 18
“Sometimes in a clinical trial it may be necessary to modify or extend an ongoing trial. For example, suppose that after a few periods have been completed one of the treatments is dropped from the trial. It will then be necessary to re-allocate the other treatments to the remaining periods of the trial.” (John et al, 2004 p. 2653)
“Jones and Donev [17] also consider augmenting a design to account for the removal of a treatment. The initial trial to compare four treatments A, B, C, and D in five periods using four groups of subjects used a Williams square [3] with the fourth period repeated. After the first two periods had been completed it was decided to drop treatment D from the remainderof the trial.”
What’s wrong here?
(C)Stephen Senn 19
(C)Stephen Senn 20
The reality
..in a single-dose cross-over trial in asthma in 12 patients reported by Palmqvistet al. [5] patients were treated in the first period of a cross-over on dates ranging from 5 May to 12 November 1987 [4]. They were treated in a second period on dates ranging from 18 May to 26 November 1987. Eleven of the patients had completed period two of treatment before the 12th patient was recruited. (Senn, 2005, p3675.)
A basic fact of clinical trials
You treat patients when they fall ill
(C)Stephen Senn 21
Multi-Story
Aspect Single-Dose Multi-Dose
When? Phase I/II Phase II/III
Why? PD, Dose Therapeutic
Primitive Constraints
Number of periods
Length of treatment
Carry-over Not a problem Potential Problem
(C)Stephen Senn 22
Conclusion
• Multi-dose trials real scope for design theory.
• These will employ active wash-out
• Design problem is trade-off between exploiting correlation and eliminating carry-over.
• Short vs long active wash-out periods
(C)Stephen Senn 23
Second ExampleSome Biostatistics Nonsense
Chow, S. C., and Liu, J. P. (2000), Design and Analysis of Bioavailability and Bioequivalence Studies (2nd ed.), New York: Marcel Dekker.
Have several discussions of efficiency of designs in their book which are completely beside the point. They compare designs in terms of residual degrees of freedom!
They conclude that Balaam’s design, which uses sequencesTR/RT/TT/RR
Is similar in efficiency to the more conventional TR/RT design.
They write “The degrees of freedom for the intrasubject residuals for the 2 × 2 and 4 × 2 design are 22 and 21 respectively. Therefore there is little difference in testing power.”
This is nonsense
(C)Stephen Senn 24
Second ExampleSome Biostatistics Nonsense
.Design
Source 2×2 4×2 2×3 2×4 4×4
Between 23 23 23 23
Seq 1 3 1 1
Res 22 20 22 22
Within 24 24 48 72
Period 1 1 2 3
Form 1 1 1 1
Carry 1 1 1
Res 22 21 44 67
Total 47 47 71 95
(C)Stephen Senn 25
What is wrong 1. It’s not correct design theory
• As any design expert knows residual degrees of freedom are (nearly) irrelevant to efficiency
• It is the impact of adjustment on the degree of orthogonality of the design matrix that is important
(C)Stephen Senn 26
What is wrong 2.It’s not realistic biostatistics
• In fact as any biostatistician who has had to think about it will know from a practical point of view far from being optimal Balaam’s design is simply inadmissible
• The reasons is that only half of the resources are devoted to actually measuring the treatment
• The rest are devoted to providing an adjustment for a form of carry-over that is itself implausible
(C)Stephen Senn 27
Allocation of patients for two designs
Sequence AB/BA Balaam
AB n/2 n/4
BA n/2 n/4
AA 0 n/4
BB 0 n/4
(C)Stephen Senn 28
Investigation of the real efficiency of Balaam’s design
Mathcad 2001 Program to compare the efficiency of Balaam'sdesign correcting for simple carry-over and the AB/BA design notcorrecting for carry-over.
Take simplest comparable case of four patients: one allocated to each of the sequences ofBalaam's design or two to each of the sequences of the AB/BA cross-over.
Set up elements of design matrix
Patient and period dummies common to both
Designs: AB/AB/BA/BA (standard design) or AB/BA/AA/BB (Balaam's design)Design matrix rows in order patient 1 period 1, patient 1 period 2, patient 2, period 1 etc.
(C)Stephen Senn 29
Five Reasons why the Simple Carry-over Model is not Useful
• If it applies then the investigator can design a trial which eliminates it . (double the periods)
• Implausible given pk/pd theory. (obvious)• Leads to inefficient estimators. (see investigation
to follow)• Can lead to poor designs. (ditto)• The models which incorporate it are self-
contradicting. (example: factorial X-overs)
(C)Stephen Senn 30
0 2 4
50
100
0.5125
Concentration/EC50
Response
0
Dose response: the pharmacokineticist’s version
(C)Stephen Senn 31
Dose Response:The Statistician’s Version
Dose
Response
This is what the simple carry-over model implies
(C)Stephen Senn 32
The Models Which use Simple Carry-over are Inconsistent
Consider a factorial cross-over in four periods comparing A, B and the combination of A and B to placebo. We can represent the four treatments by: **, A*, *B and AB.
Suppose we consider a patient who has received the sequence AB ** A* *B. A standard parameterisation for treatment and carry-over would be as in the following table.
(C)Stephen Senn 33
I II III IV
TreatmentCombination
AB ** A* *B
TreatmentParameters
A, B,
AB
A
B
Carry-overParameters
A, B,
AB
A
Paramaterisation of a factorial cross-over
Period
(C)Stephen Senn 34
The Rhinoceros
The rhinoceros has a kind heart, if you doubt it here’s the proof
That thing on his nose is for taking stones out of a horse’s hoof
He seldom ever meets a horse, it is this that makes him sad
When he does, then it hasn’t a stone in its hoof
But he would, if he did and it had
Flanders and Swann
(C)Stephen Senn 35
The Phoenix Bioequivalence Trials
• Analysed by D’Angelo & Potvin
• 20 drug classes
• 1989-1999
• 12 or more subjects
• 96 three period designs
• 324 two period designs
(C)Stephen Senn 36
AUC Cmax
0 : 115567899 1 : 01458999 2 : 01225568999 3 : 011335577 4 : 24688 5 : 35667788 6 : 00336667888 7 : 14444566999 8 : 011233468888 9 : 13335667899
0 : 223557888 1 : 4677799 2 : 000124566899 3 : 011124689 4 : 01223455799 5 : 00045599 6 : 000166667778 7 : 0345566779 8 : 2345779 9 : 13444556889
Three Treatment Designs
P-Values for Carry-Over
(C)Stephen Senn 37
AUC Cmax
0 : 00111111222222234444 0 : 5666777777789999 1 : 00000112222223333 1 : 5556667777899999 2 : 0011112223344444 2 : 555666788899999 3 : 00001112233344 3 : 5556666666777778888899999 4 : 001111112222223334 4 : 55666666777777788999 5 : 00000111222333344444 5 : 566677888899 6 : 000001134 6 : 55666667777888889999 7 : 111233333344 7 : 555556777888899 8 : 0000112234444 8 : 55666778888999 9 : 00011112233334444 9 : 555567777788999
0 : 00122222344 0 : 55555556666677999999 1 : 0001122233333344444444 1 : 55566667778888899 2 : 00011111122344 2 : 566667788889999 3 : 111112222233444444 3 : 555566666777778888999 4 : 000001112222333334444 4 : 5557888889999 5 : 00001122233 5 : 5555666678999 6 : 0000111222233334 6 : 55555566677788889999 7 : 000000112223344 7 : 6666777777889 8 : 0122233444 8 : 55666677888899 9 : 1111111222333444 9 : 555555556666677778889999
Two Treatment Designs
(C)Stephen Senn 38
StudyDesign
Variable Totalnumber
of studies
KSstatistic
p-value*
2-way AUC0-t 324 0.0645 0.1354Cmax 324 0.0496 0.4040
3-way AUC0-t 96 0.1048 0.2424Cmax 96 0.0542 0.9407
*H0: true cdf U[0,1] vs. H1: true cdf NOT U[0,1]
Test of Uniformity of P-Values
(C)Stephen Senn 39
Conclusions
• Distribution of P-values uniform– no evidence of carry-over
• Carry-over a priori implausible– presence testable by assay
• No point is testing for it– leads to bias
• Or adjusting for it– increased variance
(C)Stephen Senn 40
Do Bayesians do Better?
• In principle the Bayesian approach ought to allow us to be more flexible about nuisance parameters such as carry-over
• However, the Bayesian track record is not impressive here
• Realistic models have not been employed
(C)Stephen Senn 41
Hills and Armitage Eneuresis Data
10
8
14
2
12
6 1210
6
4
2
0
40 8
Dry nights placebo
Dry n
ights d
rug
Line of equality
Sequence Drug PlaceboSequence placebo drug
Cross-over trial in Eneuresis
Two treatment periods of 14 days each
Treatment effect significant if carry-over not fitted
2.037 ( 0.768, 3.306)
Treatment effect not significant if carry-over fitted
0.451 (-2.272, 3.174)
1. Hills, M, Armitage, P. The two-period cross-over clinical trial, British Journal of Clinical Pharmacology 1979; 8: 7-20.
(C)Stephen Senn 42
Identical ‘uninformative’ prior placed on carry-over as for treatment
NB Parameterisation here means that values of need to be doubled to compare to conventional contrasts
(C)Stephen Senn 43
Identical Priors for Treatment and Carryover?
• Patients treated repeatedly during trial• Fourteen day treatment period• Average time to last treatment plausibly 4 hours• Average time to previous treatment seven days• Saying that it is just as likely that carry-over
could be greater than treatment is not coherent• In any case the two cannot be independent• Is negative carry-over as likely as positive carry-
over?
(C)Stephen Senn 44
So What are Acceptable Models for Carry-over?
• Ignoring carry-over altogether (not allowing for it because one believes one has taken adequate steps to eliminate it)– This is always a reasonable strategy
• Using an integrated pharmacokinetic pharmacodynamic model (Sheiner et al, 1991)– This may work for dose-finding trials– Very difficult to implement where more than one
molecule is involved
(C)Stephen Senn 45
The Sheiner model
max
50
1
1
i
i
i j li
ijij
ij
jk T Ttk
ij ill
E dDE
D d
d D e e
PD dose response
PK model for dose-concentration as a consequence of previous dosing history
Steady state concentration for patient i in period l
(C)Stephen Senn 46
0 5 100
0.5
1
Pharmacodynamic model
Dose
Res
pons
e0.909
0
0.5
DE d 1( )
DE d 2( )
DE d 3( )
100
1 3
d
(C)Stephen Senn 47
1 2 3 40
1
2
3
4
5
Seq 1 ActualSeq 1 TheorySeq 2 ActualSeq 2 TheorySeq 3 ActualSeq 3 TheorySeq 4 ActualSeq 4 Theory
Dosing levels achieved
Period
Dos
e
Possible set of sequences for a design. These follow aWilliams square. NB This is probably not a good idea D
1
2
3
4
2
4
1
3
3
1
4
2
4
3
2
1
(C)Stephen Senn 48
1 2 3 40.2
0.1
0
0.1
Seq 1Seq 2Seq 3Seq 4Carry from 4
Carry-over by sequence
Period
Car
ry-o
ver
0
(C)Stephen Senn 49
The difference between mathematical and applied statistics is that the former is full of lemmas whereas the latter is full of dilemmas
(C)Stephen Senn 50
Advice for Design-Theoreticians
• Resist the temptation to give advice if you are unfamiliar with the application area
• Seek collaborators
• Ground your models in pharmacology
• Remember that the goal is good medicine not elegant mathematics
• Don’t defend the indefensible
(C)Stephen Senn 51
Advice for biostatisticians
• Remember that design theoreticians have many powerful results
• It’s just conceivable that some of them may even be useful
(C)Stephen Senn 52
(C)Stephen Senn 53
References
1. Senn, S.J., Is the 'simple carry-over' model useful? [published erratum appears in Statistics in Medicine 1992 Sep 15;11(12):1619]. Statistics in Medicine, 1992. 11(6): p. 715-26.2. Senn, S.J., The AB/BA cross-over: how to perform the two-stage analysis if you can't be persuaded that you shouldn't., in Liber Amicorum Roel van Strik, B. Hansen and M. de Ridder, Editors. 1996, Erasmus University: Rotterdam. p. 93-100.3. Senn, S.J., Cross-over Trials in Clinical Research. Second ed. 2002, Chichester:
Wiley.4. Senn, S.J., Statistical Issues in Drug Development. Statistics in Practice, ed. V.
Barnett. 2007, Chichester: John Wiley.