Incorporating higher dimensions in joint decomposition of EEG-fMRI

Wout Swinnen, BIOMED KU Leuven

Introduction: EEG and fMRI

EEG fMRI

Introduction: EEG and fMRI

EEG fMRI

• Problems: • Difficult to interpret• EEG bad spatial resolution, fMRI bad temporal resolution• Lots of data:

• EEG: SubjectsChannelsTime (3rd order tensor)• fMRI: SubjectsXYZTime (5th order tensor)

• Solution: • Combine modalities and extract joint components

• In a data-driven fashion (BSS) JointICA

Introduction: EEG and fMRI

EEG fMRI

• Problems: • Difficult to interpret• EEG bad spatial resolution, fMRI bad temporal resolution• Lots of data:

• EEG: SubjectsChannelsTime (3rd order tensor)• fMRI: SubjectsXYZTime (5th order tensor)

• Solution: • Combine modalities and extract joint components

• In a data-driven fashion (BSS) JointICA

JointICA• Assume: ERP activity (= average EEG over trials) and fMRI response

are generated by same neuronal activity (Stronger ERP peaks lead to stronger fMRI response)

• If correct:

Allows ICA formulation with common mixing matrix for EEG and fMRI,

is EEG channel readings in matrix, is fMRI readings in matrix.

JointICA will decompose problem into components where fMRI sources show regions that participated in ERP source activity

JointICA• Assume: ERP activity (= average EEG over trials) and fMRI response

are generated by same neuronal activity (Stronger ERP peaks lead to stronger fMRI response)

• If correct:

Allows ICA formulation with common mixing matrix for EEG and fMRI,

is EEG channel readings in matrix, is fMRI readings in matrix.

JointICA will decompose problem into components where fMRI sources show regions that participated in ERP source activity

JointICA

(B. Mijovic, 2013) Proved that assumption is correct

But one EEG channel only

Incorporate multiple channels in JointICA1. Concatenate EEG channels in subject dimension: sJointICA

Regards all channels as one common virtual channel,with higher number of subjects

2. Concatenate EEG channels in time dimension: tJointICA

fMRI activity is linked to a pattern of ERP peaks in multiple electrodes, all generated by the same neuronal activity

Incorporate multiple channels in JointICA1. Concatenate EEG channels in subject dimension: sJointICA

Regards all channels as one common virtual channel,with higher number of subjects

2. Concatenate EEG channels in time dimension: tJointICA

fMRI activity is linked to a pattern of ERP peaks in multiple electrodes, all generated by the same neuronal activity

Incorporate multiple channels in JointICA1. Concatenate EEG channels in subject dimension: sJointICA

Regards all channels as one common virtual channel,with higher number of subjects

2. Concatenate EEG channels in time dimension: tJointICA

fMRI activity is linked to a pattern of ERP peaks over multiple electrodes, all generated by the same neuronal activity

Materials• Visual detection task

(Mijovic, 2013)

• Down-left visual stimulusPress of button

• 18 subjects, fMRI and EEGread non-simultaneously

• Preprocessing:o EEG ERP’s

Averaged and interpolated

o fMRI PSC mapsUsing SPM software and contrastingfMRI signal after stimulus vs background

Results: JointICA•

For electrode PO8, 18 ICs extracted (ICASSO)First 5 ICs shown

• Results:

fMRI regions corresponding to ERP peaks coincide with sourceregions described in literature

Example: IC with Late N1 ERP (d,e)(d) Activations in somatosensory and motor areas (BA 1,2,3,4,6)

(e) And in visual areas (BA 19)

Results: JointICA•

For electrode PO8, 18 ICs extracted (ICASSO)First 5 shown

• Conclusions:

Meaningful decomposition revealing underlyingphysiological mechanisms

But one channel only!

Results: sJointICA

Analysis for electrode sets:(a) [O2, PO8]

(b) [Oz, O2, PO8]

(c) [PO7, Oz, PO8]

(c) [PO7, O1, Oz, O2, PO8]

54 components extracted, first one shown

Compare to single channel jointICA

• Results:The IC’s produced describe:-fMRI areas that are more hard to interpret-less stable/natural ERP phenomenon

Results: sJointICA

Analysis for electrode sets:(a) [O2, PO8]

(b) [Oz, O2, PO8]

(c) [PO7, Oz, PO8] Look at other IC’s

(c) [PO7, O1, Oz, O2, PO8]

54 components extracted, first one shown

Compare to single channel jointICA

• Results:The IC’s produced describe:-fMRI areas that are more hard to interpret-less stable/natural ERP phenomenon

Results: sJointICAFirst 18 IC’s of 54 for electrode set [PO7,Oz,PO8]

Results:ERP parts of the IC’s make up smaller timeresolution

But ERP phenomenondescribed is less natural (narrow peak)and more hard tointerpret

Results: tJointICA

Analysis for electrode sets:

(a) [Oz, PO8]

(b) [PO7, Oz, PO8]

(c) [PO7, O1, Oz, O2, PO8]

18 IC’s extracted, First IC shown

Compare to jointICA

• Results:The IC’s produced show increasingly pronounced/robust the areas thatfunction as sources for the patternof N1 activity in the different electrodes

Results: tJointICA

Analysis for electrode sets:

(a) [Oz, PO8]

(b) [PO7, Oz, PO8] Look at other IC’s

(c) [PO7, O1, Oz, O2, PO8]

18 IC’s extracted, First IC shown

Compare to jointICA

• Results:The IC’s produced show increasingly pronounced/robust the areas thatfunction as sources for the patternof N1 activity in the different electrodes

Results: tJointICA

All 18 IC’s forelectrode set[PO7,Oz,PO8]

Results:Only strongestERP characteristicsdescribed(such as N1)

No good IC’sfor weaker ERPcharacteristics(such as P1,...)

Conclusions

• JointICA: Meaningful decomposition showing underlying physiological mechanisms, but only 1 channel

• sJointICA:o Results are more difficult to interpret

• tJointICA:o IC’s show more pronounced and robust fMRI sources for certain

patterns of ERP activityo More information on strong ERP characteristics at expense of weak

ERP characteristics

Questions?