Multiple comparison correction

Methods & models for fMRI data analysis29 October 2008

Klaas Enno Stephan

Branco Weiss Laboratory (BWL)Institute for Empirical Research in EconomicsUniversity of Zurich

Functional Imaging Laboratory (FIL)Wellcome Trust Centre for NeuroimagingUniversity College London

With many thanks for slides & images to:

FIL Methods group

Overview of SPM

RealignmentRealignment SmoothingSmoothing

NormalisationNormalisation

General linear modelGeneral linear model

Statistical parametric map (SPM)Statistical parametric map (SPM)Image time-seriesImage time-series

Parameter estimatesParameter estimates

Design matrixDesign matrix

TemplateTemplate

KernelKernel

Gaussian Gaussian field theoryfield theory

p <0.05p <0.05

StatisticalStatisticalinferenceinference

Time

BOLD signalTim

e

single voxeltime series

single voxeltime series

Voxel-wise time series analysis

modelspecification

modelspecification

parameterestimationparameterestimation

hypothesishypothesis

statisticstatistic

SPMSPM

Inference at a single voxel

= p(t > u | H0)

NULL hypothesisH0: activation is zero

u

t-distribution

p-value: probability of getting a value of t at least as extreme as u.

If is small we reject the null hypothesis.

We can choose u to ensure a voxel-wise significance level of .

t =

contrast ofestimated

parameters

varianceestimate pN

TT

T

T

T

tcXXc

c

cdts

ct

~ˆ

ˆ

)ˆ(ˆ

ˆ

12

pN

TT

T

T

T

tcXXc

c

cdts

ct

~ˆ

ˆ

)ˆ(ˆ

ˆ

12

t-contrast: a simple example

Q: activation during listening ?

Q: activation during listening ?

cT = [ 1 0 ]

Null hypothesis:Null hypothesis: 01

)ˆ(

ˆ

T

T

cStd

ct

)ˆ(

ˆ

T

T

cStd

ct

Passive word listening versus rest

SPMresults:Height threshold T = 3.2057 {p<0.001}

Statistics: p-values adjusted for search volume

set-levelc p

cluster-levelp corrected p uncorrectedk E

voxel-levelp FWE-corr p FDR-corr p uncorrectedT (Z

)

mm mm mm

0.000 10 0.000 520 0.000 0.000 0.000 13.94 Inf 0.000 -63 -27 150.000 0.000 12.04 Inf 0.000 -48 -33 120.000 0.000 11.82 Inf 0.000 -66 -21 6

0.000 426 0.000 0.000 0.000 13.72 Inf 0.000 57 -21 120.000 0.000 12.29 Inf 0.000 63 -12 -30.000 0.000 9.89 7.83 0.000 57 -39 6

0.000 35 0.000 0.000 0.000 7.39 6.36 0.000 36 -30 -150.000 9 0.000 0.000 0.000 6.84 5.99 0.000 51 0 480.002 3 0.024 0.001 0.000 6.36 5.65 0.000 -63 -54 -30.000 8 0.001 0.001 0.000 6.19 5.53 0.000 -30 -33 -180.000 9 0.000 0.003 0.000 5.96 5.36 0.000 36 -27 90.005 2 0.058 0.004 0.000 5.84 5.27 0.000 -45 42 90.015 1 0.166 0.022 0.000 5.44 4.97 0.000 48 27 240.015 1 0.166 0.036 0.000 5.32 4.87 0.000 36 -27 42

Design matrix

0.5 1 1.5 2 2.5

10

20

30

40

50

60

70

80

1

X voxel-levelp uncorrectedT ( Z) mm mm mm

13.94 Inf 0.000 -63 -27 15 12.04 Inf 0.000 -48 -33 12 11.82 Inf 0.000 -66 -21 6 13.72 Inf 0.000 57 -21 12 12.29 Inf 0.000 63 -12 -3 9.89 7.83 0.000 57 -39 6 7.39 6.36 0.000 36 -30 -15 6.84 5.99 0.000 51 0 48 6.36 5.65 0.000 -63 -54 -3 6.19 5.53 0.000 -30 -33 -18 5.96 5.36 0.000 36 -27 9 5.84 5.27 0.000 -45 42 9 5.44 4.97 0.000 48 27 24 5.32 4.87 0.000 36 -27 42)0ˆ|( Tcyp )0ˆ|( Tcyp

Inference on images

Signal

Signal+Noise

Noise

11.3% 11.3% 12.5% 10.8% 11.5% 10.0% 10.7% 11.2% 10.2% 9.5%

Use of ‘uncorrected’ p-value, =0.1

Percentage of Null Pixels that are False Positives

Using an ‘uncorrected’ p-value of 0.1 will lead us to conclude on average that 10% of voxels are active when they are not.

This is clearly undesirable. To correct for this we can define a null hypothesis for images of statistics.

Family-wise null hypothesis

FAMILY-WISE NULL HYPOTHESIS:Activation is zero everywhere.

If we reject a voxel null hypothesisat any voxel, we reject the family-wisenull hypothesis

A false-positive anywhere in the imagegives a Family Wise Error (FWE).

Family-Wise Error (FWE) rate = ‘corrected’ p-value

Use of ‘uncorrected’ p-value, =0.1

FWE

Use of ‘corrected’ p-value, =0.1

The Bonferroni correction

The family-wise error rate (FWE), The family-wise error rate (FWE), ,, for a family of N a family of N independentindependent voxels is voxels is α = Nv

where v is the voxel-wise error rate.

Therefore, to ensure a particular FWE set

v = α / N

BUT ...

The Bonferroni correction

Independent voxels Spatially correlated voxels

Bonferroni is too conservative for smooth brain images !

Smoothness

• intrinsic smoothness– some vascular effects have extended spatial support

• extrinsic smoothness– resampling during preprocessing– matched filter theorem

deliberate additional smoothing to increase SNR

• described in resolution elements: "resels"

• resel = size of image part that corresponds to the FWHM (full width half maximum) of the Gaussian convolution kernel that would have produced the observed image when applied to independent voxel values

• # resels is similar, but not identical to the number of independent observations

• can be computed from spatial derivatives of the residuals

• critical for corrected p-values based on Gaussian random field theory

Random Field Theory

• Consider a statistic image as a discretisation of a continuous underlying random field

• Use results from continuous random field theory

Discretisation(“lattice

approximation”)

Euler Characteristic (EC)

Topological measure– threshold an image

at u

- EC = # blobs

- at high u:

p (blob) = E [EC]

therefore

FWE, = E [EC]

Euler Characteristic (EC)

)5.0exp()2)(2log4(ECE 22/3TT ZZR

R = number of reselsZT = Z value threshold

We can determine that Z threshold for which E[EC] = 0.05.

At this threshold, every remaining voxel represents a significant activation, corrected for multiple comparisons across the search volume.

Voxel level test:intensity of a voxel

Cluster level test:spatial extent above u

Set level test:number of clusters above u

Sensitivity

Regional specificity

Voxel, cluster and set level tests

Small volume correction (SVC)

• When we have a good hypothesis about where an activation should be, we can reduce the search volume:– mask defined by (probabilistic) anatomical atlases– mask defined by functional localisers– mask defined by orthogonal contrasts– spherical search volume around known coordinates

• Computing EC is relatively insensitive to the shape of the search volume.

Thank you