Alexandre Gramforthttp://alexandre.gramfort.net

http://scikit-learn.org

“Lire dans les pensées avec Scikit-Learn”

“Mind Reading with Scikit-Learn”

Paris Machine Learning Meetup - Sept. 2013

Alexandre Gramfort Mind Reading with the Scikit-Learn

Basics of Functional MRI (fMRI)

2

Oxy. Hb

Deoxy. Hb

Neurons

3D volumes(1 every 1 or 2s)

High spatialresolution

(vox ⋍ 2mm)

Scanner

NuclearMagnetic

Resonance

�������������������

����� ���������������������� �����

����������������������������������

!�����������

courtesy of Gael Varoquauxhttp://www.youtube.com/watch?v=uhCF-zlk0jY

Alexandre Gramfort Mind Reading with the Scikit-Learn

Learning from fMRI

4

Image,sound, task

fMRI volumes

Challenge: Learn and Predict from the fMRI data

scanningMachine Learningstim

Any variable:healthy?

Alexandre Gramfort Mind Reading with the Scikit-Learn

Result from Miyawaki et al. Neuron 2008

5

http://www.youtube.com/watch?v=h1Gu1YSoDaY

Alexandre Gramfort Mind Reading with the Scikit-Learn

Result from Miyawaki et al. Neuron 2008

6

• Some details about the data:

• 2h of scanning

• 1 image for 12s then 12s of rest

• 800MB of raw data (200MB compressed)

• 5,000 good voxels

Alexandre Gramfort Mind Reading with the Scikit-Learn

Result from Nishimoto et al. 2011

7

http://www.youtube.com/watch?v=nsjDnYxJ0bo

Alexandre Gramfort Mind Reading with the Scikit-Learn

Result from Nishimoto et al. 2011

8

• Some details about the data:

• 30GB of stimuli (15 frames/s in .png for 3h)

• about 4,000 volumes

• about 10GB of raw data

• 30,000 “good” voxels

• > 3h in the scanner

Alexandre Gramfort Mind Reading with the Scikit-Learn

Classification example with fMRI

9

!!"#$%&'()*+,-#./0123(%45678*###############################3(%45678*-#9:#;+*"#/:9:#

<=+))8>8&+?85*#@#748*&87=()

67+&(#5>#?'(#A4+8*#>(+?%4()#BC%=?8D+48+?(E

!"#$$%&'()*'+)#,-.FG4(H8&?#%*)((*#B?()?E#8C+I(@######54F<5C7+4(#74(H8&?(H#=+A(=#J8?'#?4%(#?+4I(?

K

L8D(*#+#?4+8*8*I#H+?+#)(?#@#7+84)5>#B>(+?%4()-#=+A(=E-#(/'",#?'(#&'+4+&?(48)?8&#5>#(+&'#&+?(I54,#8*#?'(#>(+?%4(#)7+&(@

F.*#?'8)#&+)(#74(H8&?(H#M#?4%(#

FN(7(+?#>54#+==#)+C7=()FOD(4+I(

!!"#$%&'()*+,-#./0123(%45678*###############################3(%45678*-#9:#;+*"#/:9:#

<=+))8>8&+?85*#@#748*&87=()

67+&(#5>#?'(#A4+8*#>(+?%4()#BC%=?8D+48+?(E

!"#$$%&'()*'+)#,-.FG4(H8&?#%*)((*#B?()?E#8C+I(@######54F<5C7+4(#74(H8&?(H#=+A(=#J8?'#?4%(#?+4I(?

K

L8D(*#+#?4+8*8*I#H+?+#)(?#@#7+84)5>#B>(+?%4()-#=+A(=E-#(/'",#?'(#&'+4+&?(48)?8&#5>#(+&'#&+?(I54,#8*#?'(#>(+?%4(#)7+&(@

F.*#?'8)#&+)(#74(H8&?(H#M#?4%(#

FN(7(+?#>54#+==#)+C7=()FOD(4+I(

The objective is to be ableto predict

given an fMRI volume

!!"#$%&'()*+,-#./0123(%45678*###############################3(%45678*-#9:#;+*"#/:9:#

<=+))8>8&+?85*#@#748*&87=()

67+&(#5>#?'(#A4+8*#>(+?%4()#BC%=?8D+48+?(E

!"#$$%&'()*'+)#,-.FG4(H8&?#%*)((*#B?()?E#8C+I(@######54F<5C7+4(#74(H8&?(H#=+A(=#J8?'#?4%(#?+4I(?

K

L8D(*#+#?4+8*8*I#H+?+#)(?#@#7+84)5>#B>(+?%4()-#=+A(=E-#(/'",#?'(#&'+4+&?(48)?8&#5>#(+&'#&+?(I54,#8*#?'(#>(+?%4(#)7+&(@

F.*#?'8)#&+)(#74(H8&?(H#M#?4%(#

FN(7(+?#>54#+==#)+C7=()FOD(4+I(

ie.

objective: Predict given y = {�1, 1} x 2 Rp

y = {�1, 1}!!"#$%&'()*+,-#./0123(%45678*###############################3(%45678*-#9:#;+*"#/:9:#

<=+))8>8&+?85*#@#748*&87=()

67+&(#5>#?'(#A4+8*#>(+?%4()#BC%=?8D+48+?(E

!"#$$%&'()*'+)#,-.FG4(H8&?#%*)((*#B?()?E#8C+I(@######54F<5C7+4(#74(H8&?(H#=+A(=#J8?'#?4%(#?+4I(?

K

L8D(*#+#?4+8*8*I#H+?+#)(?#@#7+84)5>#B>(+?%4()-#=+A(=E-#(/'",#?'(#&'+4+&?(48)?8&#5>#(+&'#&+?(I54,#8*#?'(#>(+?%4(#)7+&(@

F.*#?'8)#&+)(#74(H8&?(H#M#?4%(#

FN(7(+?#>54#+==#)+C7=()FOD(4+I(

Patient Controlsvs.Faces Housesvs.

... ...vs.1 -1vs.

Demo onHaxby et al. Science 2001

Challenge: Predict the object category viewed

Sample stimuli:

Face House Chair Shoe

Alexandre Gramfort Mind Reading with the Scikit-Learn

Miyawaki et al. 2008 with Scikit-Learn

11

< 250 Lines of codes

Alexandre Gramfort

alexandre.gramfort@telecom-paristech.fr

http://alexandre.gramfort.net

http://www.github.com/agramfort

@agramfort

Contact: