Brain Connectivity Analysis: from Unimodal to Multimodal · Unimodal Connectivity Analysis: EEG...
Transcript of Brain Connectivity Analysis: from Unimodal to Multimodal · Unimodal Connectivity Analysis: EEG...
Brain Connectivity Analysis:from Unimodal to Multimodal
Sergey M. Plis
joint work with Rene Huster, Terran Lane, Vince Clark, Michael Weisend and Vince D. Calhoun
Contents
1 Definitions and Tools
2 Unimodal Connectivity Analysis: EEG
3 Contrasting modalities
4 Data Sharing Fusion
5 Validation through Interventions
1 / 27
Definitions and Tools
Outline
1 Definitions and Tools
2 Unimodal Connectivity Analysis: EEG
3 Contrasting modalities
4 Data Sharing Fusion
5 Validation through Interventions
1 / 27
Definitions and Tools
Introduction
Neuroimaging studies brain function
Advanced techniques produce immense amounts of data
Each with their strength and weaknesses
Our goal: causal relations among brain networks
2 / 27
Definitions and Tools
Functional Neuroimaging (fMRI)
functional Magnetic ResonanceImaging (fMRI)
Blood Oxygenation Level Dependent(BOLD) response
4D data (3D volumes evolving in time)
Advantage: Relatively well localizedDisadvantage: Slow sampling rate
3 / 27
Definitions and Tools
Functional Neuroimaging (MEG)
Magneto-EncephaloGraphy (MEG)
Electromagnetic phenomenonAdvantage: Instant reflection of theunderlying activity (ms resolution)Disadvantage: Uncertain spatiallocalization
4 / 27
Definitions and Tools
Common Underlying Phenomenon
Inverse problem
Functional connectivity
5 / 27
Definitions and Tools
Data Fusion: Source Analysis
6 / 27
Definitions and Tools
Single modality: Connectivity Analysis
MEG fMRI
7 / 27
Definitions and Tools
Connectivity Inference: Bayesian Networks
Pθ(X ) =
n∏
i=1
P(Xi |Pa(Xi); θ)
hippocam
pus
amygdala
hypothalamus
amygdala hypothalamus
hippocampus
child
parents
8 / 27
Definitions and Tools
Comparing the Results: Graph Characterization1
in-degree
out-degree
degree centrality
maximum degree
diameter
density
average path length
1Rubinov, M. et al. Neuroimage 52, 1059–1069 (2010).
9 / 27
Unimodal Connectivity Analysis: EEG
Outline
1 Definitions and Tools
2 Unimodal Connectivity Analysis: EEG
3 Contrasting modalities
4 Data Sharing Fusion
5 Validation through Interventions
9 / 27
Unimodal Connectivity Analysis: EEG
Stop-Go task
10 / 27
Unimodal Connectivity Analysis: EEG
What are the nodes?
11 / 27
Unimodal Connectivity Analysis: EEG
Contrasted sliding graphs metrics2
Higher clustering coefficient for the stop task:network consolidates for processing?
Shorter characteristic path-length for the stop task:network becomes more efficient?
2Grzegorczyk, M. et al. Machine Learning 71, 265–305 (2008).
12 / 27
Contrasting modalities
Outline
1 Definitions and Tools
2 Unimodal Connectivity Analysis: EEG
3 Contrasting modalities
4 Data Sharing Fusion
5 Validation through Interventions
12 / 27
Contrasting modalities Neuroimaging
Comparison Pipeline3
singleparadigmsubjects
+
MEG
fMRI
voxelcurrents
ROIaverages
ROIaverages
quantize structuresearch
analyzestructures
afnideconvolve
MNE projectto source space
average
compare
voxelHRFs
measured data trigger locked data ROI data discrete data BN graphs metric distributions
collect modalities: same subjects same paradigm
process modalities: place data in the same ROIs
pre-process data: align sampling rates and quantize
infer effective connectivity: Bayesian Nets
compare results: aggregate metric
3Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
13 / 27
Contrasting modalities Neuroimaging
Comparison Pipeline3
singleparadigmsubjects
+
MEG
fMRI
voxelcurrents
ROIaverages
ROIaverages
quantize structuresearch
analyzestructures
afnideconvolve
MNE projectto source space
average
compare
voxelHRFs
measured data trigger locked data ROI data discrete data BN graphs metric distributions
collect modalities: same subjects same paradigm
process modalities: place data in the same ROIs
pre-process data: align sampling rates and quantize
infer effective connectivity: Bayesian Nets
compare results: aggregate metric
3Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
13 / 27
Contrasting modalities Neuroimaging
Comparison Pipeline3
singleparadigmsubjects
+
MEG
fMRI
voxelcurrents
ROIaverages
ROIaverages
quantize structuresearch
analyzestructures
afnideconvolve
MNE projectto source space
average
compare
voxelHRFs
measured data trigger locked data ROI data discrete data BN graphs metric distributions
collect modalities: same subjects same paradigm
process modalities: place data in the same ROIs
pre-process data: align sampling rates and quantize
infer effective connectivity: Bayesian Nets
compare results: aggregate metric
3Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
13 / 27
Contrasting modalities Neuroimaging
Comparison Pipeline3
singleparadigmsubjects
+
MEG
fMRI
voxelcurrents
ROIaverages
ROIaverages
quantize structuresearch
analyzestructures
afnideconvolve
MNE projectto source space
average
compare
voxelHRFs
measured data trigger locked data ROI data discrete data BN graphs metric distributions
collect modalities: same subjects same paradigm
process modalities: place data in the same ROIs
pre-process data: align sampling rates and quantize
infer effective connectivity: Bayesian Nets
compare results: aggregate metric
3Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
13 / 27
Contrasting modalities Neuroimaging
Comparison Pipeline3
singleparadigmsubjects
+
MEG
fMRI
voxelcurrents
ROIaverages
ROIaverages
quantize structuresearch
analyzestructures
afnideconvolve
MNE projectto source space
average
compare
voxelHRFs
measured data trigger locked data ROI data discrete data BN graphs metric distributions
collect modalities: same subjects same paradigm
process modalities: place data in the same ROIs
pre-process data: align sampling rates and quantize
infer effective connectivity: Bayesian Nets
compare results: aggregate metric
3Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
13 / 27
Contrasting modalities Neuroimaging
Processing
novel target
ME
GfM
RI
novel
targ
et
0
67
17
34
51
ROIs
0
67
17
34
51
ROIs
1 2 3 4 5 61 2 3 4 5 6
MEG fMRI
subjects
time (epochs) time (epochs)
quantization levels
14 / 27
Contrasting modalities Neuroimaging
Resulting ConnectivityR
OIs
novel
ME
G
target
RO
Is
ROIs ROIs
fMR
I
nove
ltarg
et
fMRI MEG
left hemisphere right hemisphere
marginal distributions of edges highest scoring networksin transverse view
15 / 27
Contrasting modalities Neuroimaging
Comparing the ResultsGraph Metrics Distributions
0.1
0.2
0.3
0 5 10 15 20 25
0.1
0.2
0.3
0 5 10 15 20 25
novel
# of children
MEG
pro
babi
lity
fMRI
pro
babi
lity
# of children
target
node-degree distribution
16 / 27
Contrasting modalities Neuroimaging
Comparing the ResultsGraph Metrics Distributions
0.1
0.2
0.3
0 5 10 15 20 25
0.1
0.2
0.3
0 5 10 15 20 25
novel
# of children
MEG
pro
babi
lity
fMRI
pro
babi
lity
# of children
target
noveltarget
max
degr
eecl
uste
ring
coeffi
cien
tav
erag
e pa
thle
ngth
MEG fMRI
graph metrics distributionsnode-degree distribution
16 / 27
Data Sharing Fusion
Outline
1 Definitions and Tools
2 Unimodal Connectivity Analysis: EEG
3 Contrasting modalities
4 Data Sharing Fusion
5 Validation through Interventions
16 / 27
Data Sharing Fusion
Why do fusion in dynamical settings?
temporal resolution affects causality4
fusion helps to avoid temporal inverse problem5
4Daunizeau, J. et al. Neuroimage 36, 69–87 (May 2007).
5Riera, J. J. et al. Neuroimage 21, 547–567 (Feb. 2004).
17 / 27
Data Sharing Fusion
Why do fusion in dynamical settings?
temporal resolution affects causality4
fusion helps to avoid temporal inverse problem5
4Daunizeau, J. et al. Neuroimage 36, 69–87 (May 2007).
5Riera, J. J. et al. Neuroimage 21, 547–567 (Feb. 2004).
17 / 27
Data Sharing Fusion Neuroimaging
Dynamic Bayesian Networks6
P(
Rt0:tTR,Mt0:tTR
,Bt0,tTR
)
=P(
Rt0
)
P(
Bt0 |Rt0
)
P(
BtTR|RtTR
)
TR∏
i=1
P(
Rti |Rti−1
)
TR∏
i=0
P(
Mti |Rti
)
m
0.065
0.032
-0.00021
-0.033
-0.066
m
0.081
0.053
0.026
-0.0019
-0.029
m
-0.089
-0.047
-0.005
0.037
0.079
circles - hidden
squares - observed6Murphy, K. PhD thesis (UC Berkeley, 2002).
18 / 27
Data Sharing Fusion Neuroimaging
Dynamic Bayesian Networks transition model
P(
Rt0:tTR,Mt0:tTR
,Bt0,tTR
)
=P(
Rt0
)
P(
Bt0 |Rt0
)
P(
BtTR|RtTR
)
TR∏
i=1
P(
Rti |Rti−1
)
TR∏
i=0
P(
Mti |Rti
)
Rt = kRt−1 + σRηt
circles - hidden
squares - observed
18 / 27
Data Sharing Fusion Neuroimaging
Dynamic Bayesian Networks MEG forward model6
P(
Rt0:tTR,Mt0:tTR
,Bt0,tTR
)
=P(
Rt0
)
P(
Bt0 |Rt0
)
P(
BtTR|RtTR
)
TR∏
i=1
P(
Rti |Rti−1
)
TR∏
i=0
P(
Mti |Rti
)
Mt = MFM(Rt) + σMηt ,
circles - hidden
squares - observed6Sarvas, J. eng. Phys Med Biol 32, 11–22 (1987).
18 / 27
Data Sharing Fusion Neuroimaging
Dynamic Bayesian Networks fMRI forward model6
P(
Rt0:tTR,Mt0:tTR
,Bt0,tTR
)
=P(
Rt0
)
P(
Bt0 |Rt0
)
P(
BtTR|RtTR
)
TR∏
i=1
P(
Rti |Rti−1
)
TR∏
i=0
P(
Mti |Rti
)
Bt = HFM(Rt) + σBηt
circles - hidden
squares - observed6Friston, K. J. et al. Neuroimage 12, 466–477 (2000).
18 / 27
Data Sharing Fusion Neuroimaging
Dynamic Bayesian Networks inference6
P(
Rt0:tTR,Mt0:tTR
,Bt0,tTR
)
=P(
Rt0
)
P(
Bt0 |Rt0
)
P(
BtTR|RtTR
)
TR∏
i=1
P(
Rti |Rti−1
)
TR∏
i=0
P(
Mti |Rti
)
Particle Filtering
circles - hidden
squares - observed
6(eds Doucet, A. et al.) (Springer-Verlag, Berlin, 2001).
18 / 27
Data Sharing Fusion Neuroimaging
Demonstration7
fMRI only MEG only fMRI+MEG
7Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
19 / 27
Data Sharing Fusion Neuroimaging
Comparison: fMRI vs. fMRI+MEG
from sparseto constant activity
1000 runs per point
E =
1000∑
i=1
‖Ti − Mi‖2
‖Ti‖2
Event Related studies
fusion yields:
lower errors
stabler estimates
20 / 27
Data Sharing Fusion Neuroimaging
Comparison: fMRI vs. fMRI+MEG
from sparseto constant activity
1000 runs per point
E =
1000∑
i=1
‖Ti − Mi‖2
‖Ti‖2
Event Related studies
fusion yields:
lower errors
stabler estimates
20 / 27
Data Sharing Fusion Neuroimaging
Comparison: fMRI vs. fMRI+MEG
from sparseto constant activity
1000 runs per point
E =
1000∑
i=1
‖Ti − Mi‖2
‖Ti‖2
Event Related studies
fusion yields:
lower errors
stabler estimates
20 / 27
Data Sharing Fusion Neuroimaging
Comparison: fMRI vs. fMRI+MEG
from sparseto constant activity
1000 runs per point
E =
1000∑
i=1
‖Ti − Mi‖2
‖Ti‖2
Event Related studies
fusion yields:
lower errors
stabler estimates
20 / 27
Data Sharing Fusion Neuroimaging
Speed up and stability
BOLD (% of change)
21 / 27
Data Sharing Fusion Neuroimaging
Speed up and stability
BOLD (% of change) neural activity (a.u.)
fusion yields: ◦ lower variance and ◦ faster computation
21 / 27
Data Sharing Fusion Neuroimaging
Real data
same paradigm for fMRI and MEG
120 trials of an 8 Hz checkerboard reversal
MEG
1200 Hz
averaged
fMRI
interpolated to 1200 Hz
averaged
22 / 27
Data Sharing Fusion Neuroimaging
Real data results
fMRI only
23 / 27
Data Sharing Fusion Neuroimaging
Real data results
fMRI only MEG only
23 / 27
Data Sharing Fusion Neuroimaging
Real data results
fMRI only MEG only fMRI+MEG
23 / 27
Validation through Interventions
Outline
1 Definitions and Tools
2 Unimodal Connectivity Analysis: EEG
3 Contrasting modalities
4 Data Sharing Fusion
5 Validation through Interventions
23 / 27
Validation through Interventions
Does inferred connectivity reflect brain function?
Manipulation principle: learn by breaking parts of the system!
How to alter brain function without subjects complaining too loud?
Transcranial Direct Current Stimulation (tDCS): a noninvasive lowcurrent technique affecting firing thresholds of cortical neurons.
24 / 27
Validation through Interventions
Does inferred connectivity reflect brain function?
Manipulation principle: learn by breaking parts of the system!
How to alter brain function without subjects complaining too loud?
Transcranial Direct Current Stimulation (tDCS): a noninvasive lowcurrent technique affecting firing thresholds of cortical neurons.
24 / 27
Validation through Interventions
Does inferred connectivity reflect brain function?
Manipulation principle: learn by breaking parts of the system!
How to alter brain function without subjects complaining too loud?
Transcranial Direct Current Stimulation (tDCS): a noninvasive lowcurrent technique affecting firing thresholds of cortical neurons.
24 / 27
Validation through Interventions
Preliminary Results
0.1
0.2
0.3
0 5 10 15 20 25
0.1
0.2
0.3
0 5 10 15 20 25
novel
# of children
MEG
pro
babi
lity
fMRI
pro
babi
lity
# of children
target
25 / 27
Validation through Interventions
Preliminary Results
0.1
0.20.3
run1
pro
babili
ty(logscale
)
# of children
run2
# of children
run3
# of children
run4
# of children # of children
run5
0 5 10 15 20 25 30 35 40
# of children
0 5 10 15 20 25 30 35 40
# of children
0 5 10 15 20 25 30 35 40
# of children
0 5 10 15 20 25 30 35 40
# of children
distribution of edges in likely graphs (right hand stimulation)
0 5 10 15 20 25 30 35 40
0.1
0.20.3
# of children
pro
bability
(logscale
)sh
amfull
< 20 children
>= 20 children
25 / 27
Validation through Interventions
Summary
Goal: Combine modalities to infer function-induced networks
Results so far:Demonstrated useful results of sliding window treatmentDemonstrated pitfalls of single-modality connectivity estimation8
Demonstrated fMRI+MEG fusion in the DBN framework9
Future work:Causal structure fusionWhole brain DBN fusion frameworktDCS-based analysis framework for validation
8Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
9Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
26 / 27
Validation through Interventions
Summary
Goal: Combine modalities to infer function-induced networks
Results so far:Demonstrated useful results of sliding window treatmentDemonstrated pitfalls of single-modality connectivity estimation8
Demonstrated fMRI+MEG fusion in the DBN framework9
Future work:Causal structure fusionWhole brain DBN fusion frameworktDCS-based analysis framework for validation
8Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
9Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
26 / 27
Validation through Interventions
Summary
Goal: Combine modalities to infer function-induced networks
Results so far:Demonstrated useful results of sliding window treatmentDemonstrated pitfalls of single-modality connectivity estimation8
Demonstrated fMRI+MEG fusion in the DBN framework9
Future work:Causal structure fusionWhole brain DBN fusion frameworktDCS-based analysis framework for validation
8Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
9Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
26 / 27
Validation through Interventions
Summary
Goal: Combine modalities to infer function-induced networks
Results so far:Demonstrated useful results of sliding window treatmentDemonstrated pitfalls of single-modality connectivity estimation8
Demonstrated fMRI+MEG fusion in the DBN framework9
Future work:Causal structure fusionWhole brain DBN fusion frameworktDCS-based analysis framework for validation
8Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
9Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
26 / 27
Validation through Interventions
Summary
Goal: Combine modalities to infer function-induced networks
Results so far:Demonstrated useful results of sliding window treatmentDemonstrated pitfalls of single-modality connectivity estimation8
Demonstrated fMRI+MEG fusion in the DBN framework9
Future work:Causal structure fusionWhole brain DBN fusion frameworktDCS-based analysis framework for validation
8Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
9Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
26 / 27
Validation through Interventions
Summary
Goal: Combine modalities to infer function-induced networks
Results so far:Demonstrated useful results of sliding window treatmentDemonstrated pitfalls of single-modality connectivity estimation8
Demonstrated fMRI+MEG fusion in the DBN framework9
Future work:Causal structure fusionWhole brain DBN fusion frameworktDCS-based analysis framework for validation
8Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
9Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
26 / 27
Validation through Interventions
Summary
Goal: Combine modalities to infer function-induced networks
Results so far:Demonstrated useful results of sliding window treatmentDemonstrated pitfalls of single-modality connectivity estimation8
Demonstrated fMRI+MEG fusion in the DBN framework9
Future work:Causal structure fusionWhole brain DBN fusion frameworktDCS-based analysis framework for validation
8Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
9Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
26 / 27
Validation through Interventions
Summary
Goal: Combine modalities to infer function-induced networks
Results so far:Demonstrated useful results of sliding window treatmentDemonstrated pitfalls of single-modality connectivity estimation8
Demonstrated fMRI+MEG fusion in the DBN framework9
Future work:Causal structure fusionWhole brain DBN fusion frameworktDCS-based analysis framework for validation
8Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
9Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
26 / 27
Validation through Interventions
Summary
Goal: Combine modalities to infer function-induced networks
Results so far:Demonstrated useful results of sliding window treatmentDemonstrated pitfalls of single-modality connectivity estimation8
Demonstrated fMRI+MEG fusion in the DBN framework9
Future work:Causal structure fusionWhole brain DBN fusion frameworktDCS-based analysis framework for validation
8Plis, S. M. et al. Computers in Biology and Medicine 41, 1156–1165 (2011).
9Plis, S. M. et al. Frontiers in Neuroinformatics 4, 12 (2010).
26 / 27
Validation through Interventions
Thank you!
27 / 27