Sebastian Ueckert1, Elodie L. Plan1, Kaori Ito2, Mats O. Karlsson1, Brian W. Corrigan2, Andrew C. Hooker1

Improving cognitive testing with IRT and optimal design

1. Pharmacometrics Research Group, Uppsala University, Uppsala, Sweden 2. Global Clinical Pharmacology, Pfizer Inc, Groton, CT, USA

PODE 2013 - POPULATION OPTIMUM DESIGN OF EXPERIMENTS

2

ALZHEIMER’S DISEASE • Fatal disease • Most common form of dementia • Prevalence is 13% for age > 65

(US) • Increasing problem in aging

societies

• Cause remains vastly unknown • Only symptomatic treatment • No cure

Why no/few treatments?

Diagnosis

3

Clinical Routine

Observations

MMSE

Clinical Trials

ADAS-cog

ADCS-CGIC

CDR-SB

Alzheimer’s Disease

Fasting Plasma Glucose

Clinical Routine

Clinical Trials

Diabetes

ADAS-cog Score

4

• Cognitive subscale of Alzheimer’s Disease Assessment Scale • Cognitive assessment including broad range of sub-tests e.g.,

?Naming Objects

Commands

Construction

Ideational Praxis

7 June2012

?Orientation

Ability to Speak

Ability to Understand

Remembering Words LAKECLOCKFORESTANIMAL

5 7 8

Rosen et al. 1984

ADAS-cog Score

TASK BASED RATER ASSESSED

ADAS-cog Score Model

𝑦𝑖𝑖𝑖 Response of (category assigned to) subject 𝑖 for item 𝑗 at time 𝑖𝑘 𝑀 Number of test items in ADAS-cog assessment 𝑦�𝑖𝑖 ADAS-cog score

𝑦�𝑖𝑖 ≈�𝑦𝑖𝑖𝑖

𝑀

𝑖=1

Commonly used pharmacometric model:

𝑦�𝑖𝑖 = 𝜃0 + 𝜂0𝑖 + 𝜃1 + 𝜂1𝑖 ⋅ 𝑡𝑖𝑖 + 𝜀𝑖𝑖 𝜂0𝑖 , 𝜂1𝑖 ∼ Normal 0,Ω

𝜀𝑖𝑖 ∼ Normal 0,𝜎

5

ADAS-cog IRT Model

From now: consider only data from 1 time point (drop 𝑘 index)

𝐷𝑖 Latent variable “cognitive disability” of patient 𝑖 f𝑖 Response function for test 𝑗

P 𝑦𝑖𝑖 = 𝑥 = f 𝑖 𝑥,𝐷𝑖

𝐷𝑖 ∼ Normal 0,1

Instead of summary score, calculate

𝐷�𝑖 = arg max𝐷

� log f 𝑖 𝑦𝑖 ,𝐷𝑀

𝑖=0

+ log p 𝐷; 0,1

6

Item Response Theory

ADAS-cog IRT Model

7

?

7 June 2012

?

LAKE CLOCK FOREST ANIMAL

COGNITIVE DISABILITY

?

ADAS-cog IRT Model Latent variable model:

𝐷𝑖 ∼ Normal 0,1 Response Models:

g 𝜿,𝐷 = 𝜅1 + (𝜅2 − 𝜅1)𝑒𝜅3 𝐷−𝜅4

1 + 𝑒𝜅3 𝐷−𝜅4

𝜿 = 𝜅1, 𝜅2, 𝜅3, 𝜅4

P 𝑦𝑖𝑖 = 1 = g 𝜿𝑖 ,𝐷𝑖 P 𝑦𝑖𝑖 = 𝑙 =

𝑛𝑙

g 𝜿𝑖 ,𝐷𝑖𝑙 1 − g 𝜿𝑖 ,𝐷𝑖

𝑛−𝑙

P 𝑦𝑖𝑖 = 𝑙 =g 𝜿𝑖 ,𝐷𝑖 g 𝜿𝑖 ,𝐷𝑖 + 𝛿𝑙 𝑙−1𝑒−g 𝜿𝑗,𝐷𝑖 −𝛿𝑙

𝑙!

P 𝑦𝑖𝑖 ≥ 𝑙 = g 𝜿𝑖 ,𝐷𝑖 P 𝑦𝑖𝑖 = 𝑙 = P 𝑦𝑖𝑖 ≥ 𝑙 − P 𝑦𝑖𝑖 ≥ 𝑙 + 1

8

Binary:

Count (binomial):

Count (generalized Poisson):

Ordered categorical:

Parameter Estimation

• Test specific parameters 𝚱 = (𝜿1, … ,𝜿𝑀)(167 in total) estimated from clinical trial databases

– ADNI: • Observational study with normal, mild cognitively impaired (MCI) and mild AD

subjects • Baseline ADAS-cog data from 819 subjects

– CAMD: • Database with placebo arm data from clinical trials • First visit ADAS-cog data from 6 studies (1832 subjects)

• Estimation: – NONMEM 7.3 beta – LAPLACE method

9

Estimated Response Functions

10

Item Information

Which test item is most informative with respect to cognitive disability? Calculate item information function:

ℐ𝑖 𝐷𝑖 = −E𝜕2

𝜕𝐷𝑖2log f𝑖 𝑦 𝐷𝑖

11

Item Information - Binary Response

g 𝜿𝑖 ,𝐷 = 𝜅𝑖1 + 𝜅𝑖2 − 𝜅𝑖1𝑒𝜅𝑗3 𝐷−𝜅𝑗4

1 + 𝑒𝜅𝑗3 𝐷−𝜅𝑗4

P 𝑦𝑖𝑖 = 1 = g 𝜿𝑖 ,𝐷𝑖 P 𝑦𝑖𝑖 = 0 = 1 − g 𝜿𝑖 ,𝐷𝑖

l 𝜿𝒋,𝐷𝑖 =𝜕𝜕𝐷𝑖2

log g 𝜿𝒋,𝐷𝑖

= −𝜅𝑖32 𝑒𝜅𝑗3 𝐷𝑖−𝜅𝑗4 𝜅𝑖1 − 𝜅𝑖2 𝜅𝑖1 − 𝜅𝑖2𝑒2𝜅𝑗3 𝐷𝑖−𝜅𝑗4

𝑒𝜅𝑗3 𝐷𝑖−𝜅𝑗4 + 12𝜅𝑖1 + 𝜅𝑖2𝑒𝜅𝑗3 𝐷𝑖−𝜅𝑗4

2

h 𝜿𝒋,𝐷𝑖 =𝜕𝜕𝐷𝑖2

log 1 − g 𝜿𝒋,𝐷𝑖

= −𝜅𝑖32 𝑒𝜅𝑗3 𝐷𝑖−𝜅𝑗4 𝜅𝑖1 − 𝜅𝑖2 𝜅𝑖1 + 𝑒2𝜅𝑗3 𝐷𝑖−𝜅𝑗4 − 𝜅𝑖2𝑒2𝜅𝑗3 𝐷𝑖−𝜅𝑗4 − 1

𝑒𝜅𝑗3 𝐷𝑖−𝜅𝑗4 + 12𝜅𝑖1 − 𝑒2𝜅𝑗3 𝐷𝑖−𝜅𝑗4 + 𝜅𝑖2𝑒2𝜅𝑗3 𝐷𝑖−𝜅𝑗4 − 1

2

ℐ𝑖 𝜿𝒋,𝐷𝑖 = −g 𝜿𝒋,𝐷𝑖 l 𝜿𝒋,𝐷𝑖 − 1 − g 𝜿𝒋,𝐷𝑖 h 𝜿𝒋,𝐷𝑖

12

Item Information Functions

13

Population Information

Subjects disability is unknown prior to assessment can’t calculate individual information value calculate expected information for a population to be studied

ℐ�̅� = E ℐ𝑖 𝐷𝑖

= � p 𝐷𝑖; 𝜇pop,𝜎pop2 ℐ𝑖(𝐷𝑖)∞

−∞

𝑑𝐷𝑖

p(𝑥; 𝜇pop,𝜎pop2 )…Probability density function for normal distribution with mean 𝜇pop and variance 𝜎pop2

𝜇pop and 𝜎pop2 are estimated from ADNI data

14

Population Information

15

0

20

40

60

-2 0 2Cognitive Disability

Cou

nt

Population Normal MCI Mild AD

Disability Distribution in ADNI

0

20

40

60

-2 0 2Cognitive Disability

Cou

nt

Item Information

Average Information in Population: 3.81

Population Information

16

Component Ranking

17

MCI Population Component Information %

1 Delayed Word Rec 4.61 29.9 2 Word Recall 3.82 24.8 3 Word Recognition 1.98 12.8 4 Orientation 1.98 12.8 5 Naming Obj&Fing 1.06 6.9

mAD Population Component Information %

1 Orientation 4.92 22.7 2 Word Recall 3.79 17.5 3 Delayed Word Rec 3.26 15.0 4 Naming Obj&Fing 2.83 13.0 5 Number Cancellation 1.48 6.8

87% of total

Adaptive Assessment 𝕋 Set of available cognitive tests 𝑇𝑖 Set of already performed cognitive tests up to step 𝑘 𝑦𝑖 Subject response at step 𝑘 𝐷�𝑖 Disability estimate of subject after step 𝑘 𝜎𝑖2 Associated variance of the estimate Algorithm

18

𝐷�0 = 1,𝜎02 = 1,𝑘 = 0 while 𝜎𝑖2 > 𝑡𝑡𝑙 {

𝜏∗ = arg max𝑖∈𝕋\𝑇𝑘

� p 𝑥;𝐷�𝑖 ,𝜎𝑖2 ℐ𝑖(𝑥)∞

−∞

𝑑𝑥

𝑦𝑖 = patient response to τ∗ 𝑇𝑖 = 𝑇𝑖 ∪ 𝜏∗,𝑘 = 𝑘 + 1

𝐷�𝑖 = arg max� log f 𝑖 𝑦𝑖 ,𝐷𝑖

𝑖=0

+ log p 𝐷; 0,1

𝜎𝑖2 =𝑑2

𝑑𝐷2� log f 𝑖 𝑦𝑖 ,𝐷𝑖

𝑖=0

+ log p 𝐷; 0,1

}

AD i.d.e.a.- Alzheimer’s Disease integrated dynamic electronic assessment

Aim: Implement adaptive testing algorithm in a web application Architecture:

19

Algorithm Operation

20

Benefits of AD i.d.e.a.

Simplified testing procedure Decreased duration Increased sensitivity Assessment history Unique underlying scale

21

AD i.d.e.a. Clinical benefits

Routine cognitive testing Improved diagnosis More precise tracking Integration of multiple existing assessments

Algorithm Performance

22

Summary

23

Improved Diagnosis &

Clinical Trials

New Treatment Options

Acknowledgements • DDMoRe initiative

• Colleagues in Uppsala, Sweden • Pfizer colleagues in Groton, USA

24

The research leading to these results has received support from the Innovative Medicines Initiative Joint Undertaking under grant agreement n° 115156, resources of which are composed of financial contributions from the European Union's Seventh Framework Programme (FP7/2007-2013) and EFPIA companies’ in kind contribution. The DDMoRe project is also supported by financial contribution from Academic and SME partners. This work does not necessarily represent the view of all DDMoRe partners.