SnPM:SnPM:Statistical nonParametric MappingStatistical nonParametric Mapping

A Permutation Test for PET &A Permutation Test for PET &Second Level fMRISecond Level fMRI

Thomas Nichols, University of MichiganThomas Nichols, University of MichiganAndrew Holmes, University of GlasgowAndrew Holmes, University of Glasgow

SnPM:SnPM:Statistical nonParametric MappingStatistical nonParametric Mapping

A Permutation Test for PET &A Permutation Test for PET &Second Level fMRISecond Level fMRI

Thomas Nichols, University of MichiganThomas Nichols, University of MichiganAndrew Holmes, University of GlasgowAndrew Holmes, University of Glasgow

B B B B B BA A A A A A

B B B B B BA A A A AA

B B B B B BA A A A A AB B B B B BA A A A A AB B B B B BA A A A A AB B B B B BA A A A A AB B B B B BA A A A A A

B B B B B BA A A A AAB B B B B BA A A A AAB B B B B BA A A A AAB B B B B BA A A A AAB B B B B BA A A A AA

t-statisticvariancemean difference

=

ran

dom

ise

12 subjects

6 B

A…

6 A

B…

differenceV5 PET activation experiment…

i

123456

789101112

……exampleexample……exampleexample

HH00:: scan would have been samescan would have been same

whatever the conditionwhatever the condition

– labelling as labelling as activeactive or or baselinebaseline arbitrary arbitrary

– re-label scans re-label scans equally likely statistic equally likely statistic imageimage• consider all possible relabellings consider all possible relabellings (exchangability)(exchangability)

permutation distributionpermutation distribution• of of eacheach voxel statistic voxel statistic ??

• of maximal voxel statisticof maximal voxel statistic

mean difference

smoothed variance

t-statistic“pseudo” t-statistic

variance

mean difference

SnPM with “pseudo” t-statistic

SPM with standard t-statisic

permutation distribution

SnPM with standard t-statisic? – similar!

SnPMSnPMSnPMSnPM

• SnPM:SnPM:

+ minimal assumptionsminimal assumptions• guaranteed validguaranteed valid

+ intuitive, flexible, powerfulintuitive, flexible, powerful

+ any statistic: voxel / summaryany statistic: voxel / summary

+ any summary statisticany summary statistic• maximum maximum pseudo tpseudo t – restricted – restricted

volume – cluster size / height / volume – cluster size / height / mass – omnibus testsmass – omnibus tests

– computational burdencomputational burden

– need sufficient relabellingsneed sufficient relabellings

• UsesUses• low dflow df• dodgy parametricdodgy parametric• no parametric resultsno parametric results

• SnPM:SnPM:

+ minimal assumptionsminimal assumptions• guaranteed validguaranteed valid

+ intuitive, flexible, powerfulintuitive, flexible, powerful

+ any statistic: voxel / summaryany statistic: voxel / summary

+ any summary statisticany summary statistic• maximum maximum pseudo tpseudo t – restricted – restricted

volume – cluster size / height / volume – cluster size / height / mass – omnibus testsmass – omnibus tests

– computational burdencomputational burden

– need sufficient relabellingsneed sufficient relabellings

• UsesUses• low dflow df• dodgy parametricdodgy parametric• no parametric resultsno parametric results

Non-parametric tests in Non-parametric tests in ffNI…NI…Non-parametric tests in Non-parametric tests in ffNI…NI…

• Classic testsClassic tests• Wilcoxon rank sum testWilcoxon rank sum test

• Kolmogorov-Smirnov testKolmogorov-Smirnov test

• Permutation testsPermutation tests• Holmes, Arndt Holmes, Arndt (PET)(PET)

• Bullmore, Locascio Bullmore, Locascio ((ffMRI)MRI)noise whitening, permutationnoise whitening, permutation

• Nichols & Holmes (Nichols & Holmes (ffMRI)MRI)label (re)-randomisationlabel (re)-randomisation

weak distributional assumptions

don’t assume normality replace data by ranks

lose information exchangeability

independence – fMRI minimal assumptions

exchangeabilityvalid often exactmultiple comparisons

via maximal statisticsflexible computational burden sufficient permutations additional power at low d.f.

via “pseudo” t-statistics

SnPM SnPM (standard (standard t)t)SnPM SnPM (standard (standard t)t)

• 12 scans12 scans

• 221212 permutations permutations

• All 2048/2 computedAll 2048/2 computed

• pp=1/2048=1/2048

• = 0.05 critical threshold:= 0.05 critical threshold: uu = 7.9248 = 7.9248

• Bonferoni critical threshold: Bonferoni critical threshold: uu = 9.0717 = 9.0717

• 30 min on Sparc Ultra 1030 min on Sparc Ultra 10

• 12 scans12 scans

• 221212 permutations permutations

• All 2048/2 computedAll 2048/2 computed

• pp=1/2048=1/2048

• = 0.05 critical threshold:= 0.05 critical threshold: uu = 7.9248 = 7.9248

• Bonferoni critical threshold: Bonferoni critical threshold: uu = 9.0717 = 9.0717

• 30 min on Sparc Ultra 1030 min on Sparc Ultra 10

SnPM SnPM (pseudo (pseudo tt))SnPM SnPM (pseudo (pseudo tt))

• 12 scans12 scans

• 221212 permutations permutations

• All 2048/2 computedAll 2048/2 computed

• p = p = 1/20481/2048

• = 0.05 critical threshold:= 0.05 critical threshold: uu = 5.120 = 5.120

• 40 min on a Sparc Ultra1040 min on a Sparc Ultra10

• 12 scans12 scans

• 221212 permutations permutations

• All 2048/2 computedAll 2048/2 computed

• p = p = 1/20481/2048

• = 0.05 critical threshold:= 0.05 critical threshold: uu = 5.120 = 5.120

• 40 min on a Sparc Ultra1040 min on a Sparc Ultra10

SnPM vs Parametric RF SnPM vs Parametric RF SnPM vs Parametric RF SnPM vs Parametric RF

Corrected Significance of ThresholdPermutation

RT Theory

Holmes AP, Blair RC, Watson JDG, Ford I (1996)“Non-Parametric Analysis of Statistic Images from Functional Mapping Experiments”Journal of Cerebral Blood Flow and Metabolism 16:7-22

Arndt S, Cizadlo T, Andreasen NC, Heckel D, Gold S, O'Leary DS (1996)“Tests for comparing images based on randomization and permutation methods”Journal of Cerebral Blood Flow and Metabolism 16:1271-1279

Nichols TE, Holmes AP (2002)“Nonparametric permutation tests for functional neuroimaging experiments: A primer with examples” Human Brain Mapping 15:1-25

Bullmore ET, Brammer M, Williams SCR, Rabe-Hesketh S, Janot N, David A, Mellers J, Howard R, Sham P (1995)“Statistical Methods of Estimation and Inference for Functional MR Image Analysis”Magnetic Resonance in Medicine 35:261-277

Locascio JJ, Jennings PJ, Moore CI, Corkin S (1997)“Time series analysis in the time domain and resampling methods for studies of functional magnetic resonance brain imaging” Human Brain Mapping 5:168-193

Raz J, Zheng H, Turetsky B (1999)“Statistical Tests for fMRI based on experimental randomisation” (ENAR Conference Proceedings)

Marchini JL, Ripley BD (2000)“A new statistical approach to detecting significant activation in functional MRI” NeuroImage

Holmes AP, Blair RC, Watson JDG, Ford I (1996)“Non-Parametric Analysis of Statistic Images from Functional Mapping Experiments”Journal of Cerebral Blood Flow and Metabolism 16:7-22

Arndt S, Cizadlo T, Andreasen NC, Heckel D, Gold S, O'Leary DS (1996)“Tests for comparing images based on randomization and permutation methods”Journal of Cerebral Blood Flow and Metabolism 16:1271-1279

Nichols TE, Holmes AP (2002)“Nonparametric permutation tests for functional neuroimaging experiments: A primer with examples” Human Brain Mapping 15:1-25

Bullmore ET, Brammer M, Williams SCR, Rabe-Hesketh S, Janot N, David A, Mellers J, Howard R, Sham P (1995)“Statistical Methods of Estimation and Inference for Functional MR Image Analysis”Magnetic Resonance in Medicine 35:261-277

Locascio JJ, Jennings PJ, Moore CI, Corkin S (1997)“Time series analysis in the time domain and resampling methods for studies of functional magnetic resonance brain imaging” Human Brain Mapping 5:168-193

Raz J, Zheng H, Turetsky B (1999)“Statistical Tests for fMRI based on experimental randomisation” (ENAR Conference Proceedings)

Marchini JL, Ripley BD (2000)“A new statistical approach to detecting significant activation in functional MRI” NeuroImage

Nonparametric approaches…Nonparametric approaches…Nonparametric approaches…Nonparametric approaches…