Computational Neuroimaging - TNU · Acetylcholine Noradrenaline. Application of the HGF: Two types...
Transcript of Computational Neuroimaging - TNU · Acetylcholine Noradrenaline. Application of the HGF: Two types...
What is it all about?
Why do we use functional magnetic resonance imaging? To measure brain activity
When does the brain become active? When it learns
i.e., when its predictions have to be adjusted
Where do these predictions come from? A model
2017-12-06 2Methods & Models: Computational Neuroimaging
2017-12-06 3
How to build a model
Sensory Input
Predictions
Inferred Hidden States
TrueHidden States
Methods & Models: Computational Neuroimaging
“Prediction Error”
Computational Neuroimaging
Iglesias et al., 2016
2017-12-06 4Methods & Models: Computational Neuroimaging
Advantages of computational neuroimaging
Computational neuroimaging permits us to:
Infer the computational mechanisms underlying brain function
Localize such mechanisms
Compare different models
2017-12-06 5Methods & Models: Computational Neuroimaging
Explanatory Gap
Biological
• Molecular
• Neurochemical
Cognitive
• Computational
• “cognitive/
• computational phenotyping”
Phenomenological
• Performance Accuracy
• Reaction Time
• Choices, preferences
Computational Models
2017-12-06 6Methods & Models: Computational Neuroimaging
Computational Level: predictions, prediction errors
Algorithmic Level: reinforcement learning, hierarchical
Bayesian inference, predictive coding
Implementational Level: Brain activity, neuromodulation
3 ingredients:
1. Experimental paradigm:
2. Computational model of learning:
3. Model-based fMRI analysis:
David Marr, 1982
Three Levels of Inference
2017-12-06 3Methods & Models: Computational Neuroimaging
Outline
2017-12-06 8Methods & Models: Computational Neuroimaging
1. Computational
2. Algorithmic
4. Application to Psychiatry
3. Implementational
Example of a simple model
2017-12-06 9Methods & Models: Computational Neuroimaging
Rescorla-Wagner Learning:
𝜇(𝑘) = 𝜇(𝑘−1) + 𝛼(𝑢 𝑘 − 𝜇 𝑘−1 )
Inferred States New Input
Prediction Error
Example of a simple model
2017-12-06 10Methods & Models: Computational Neuroimaging
Rescorla-Wagner Learning:
Δ𝜇(𝑘) ∝ 𝜶𝛿Belief Update
Learning Rate
2017-12-06 11
Computational Variables
Sensory Input
Predictions
Inferred Hidden States
TrueHidden States
Methods & Models: Computational Neuroimaging
Prediction Error
Learning Rate
Example of a simple model
2017-12-06 12Methods & Models: Computational Neuroimaging
Rescorla-Wagner Learning:
Δ𝜇(𝑘) ∝ 𝜶𝛿Belief Update
Learning Rate
Example of a hierarchical model
2017-12-06 13Methods & Models: Computational Neuroimaging
Hierarchical Gaussian Filter :
Δ𝜇(𝑘) ∝Ƹ𝜋𝑖−1
(𝑘)
𝜋𝑖(𝑘)
𝛿Belief Update
Weight
= ℎ𝑜𝑤 𝑚𝑢𝑐ℎ 𝑤𝑒′𝑟𝑒 𝑙𝑒𝑎𝑟𝑛𝑖𝑛𝑔 ℎ𝑒𝑟𝑒
ℎ𝑜𝑤 𝑚𝑢𝑐ℎ 𝑤𝑒 𝑎𝑙𝑟𝑒𝑎𝑑𝑦 𝑘𝑛𝑜𝑤
2017-12-06 Methods & Models: Computational Neuroimaging 14
Inference is Hierarchical
Prediction Errors
PredictionsBeliefPrecision
SensoryPrecision
• Learning rate as precision-weighted prediction errors
Δ𝜇𝑖(𝑘)
∝ො𝜋i−1(𝑘)
𝜋𝑖(𝑘)
𝛿𝑖−1(𝑘)
SensoryPrecision
BeliefPrecision
BeliefUpdate
PE
Mathys et al., Frontiers Human Neurosci 2011
2017-12-06 Methods & Models: Computational Neuroimaging 15
Inference is Hierarchical
Prediction Errors
PredictionsBeliefPrecision
SensoryPrecision
• Learning rate as precision-weighted prediction errors
Δ𝜇𝑖(𝑘)
∝ො𝜋i−1(𝑘)
𝜋𝑖(𝑘)
𝛿𝑖−1(𝑘)
SensoryPrecision
BeliefPrecision
BeliefUpdate
PE
Mathys et al., Frontiers Human Neurosci 2011
Outline
2017-12-06 16Methods & Models: Computational Neuroimaging
1. Computational
2. Algorithmic
4. Application to Psychiatry
3. Implementational
Perception (learning) via hierarchical
interactions
Top-down;Predictions
Bottom-up;Sensations/Prediction Error
Filter
Update
2017-12-06 17Methods & Models: Computational Neuroimaging
From perception to action
2017-12-06 19Methods & Models: Computational Neuroimaging
Inversion of Perceptual Model
Generative Model
Inversion of Response Model
From perception to action to observation
2017-12-06 20Methods & Models: Computational Neuroimaging
Inversion of Perceptual Model
Generative Model
Inversion of Response Model
Experimenter
Observing the observer
Daunizeau et al., PONE, 2011
• The observer obtains input from the world via the sensory systems
• He/she has prior beliefs about the state of the world and how it is changing.
• Based on these prior beliefs and the sensory inputs, he makes predictions.
2017-12-06 21Methods & Models: Computational Neuroimaging
Observing the observer
• The observer obtains input from the world via the sensory systems
• He/she has prior beliefs about the state of the world and how it is changing.
• Based on these prior beliefs and the sensory inputs, he makes predictions.
• As the experimenter, we want to infer on what the observer is thinking …
• But all we can observe is his/her behaviour.
• We invert the observer’s beliefs from his/her behaviour: computational model
2017-12-06 22Methods & Models: Computational Neuroimaging
Daunizeau et al., PONE, 2011
2017-12-06 Methods & Models: Computational Neuroimaging 23
Bayesian Models
𝑝 Θ 𝑦,𝑚 =𝑝 Θ|𝑚 𝑝(𝑦|Θ,𝑚)
𝑝 Θ|𝑚 𝑝 𝑦 Θ,𝑚 𝑑Θ
Sensory DataPrediction Errors
Predictions
The Bayesian Brain
• The brain is an inference machine• Conceptualise beliefs as probability
distributions• Updates via Bayes’ rule:
Posterior Belief
Evidence
PriorBelief
2017-12-06 Methods & Models: Computational Neuroimaging 24
Hierarchical Gaussian Filter
Prediction Errors
PredictionsBeliefPrecision
SensoryPrecision
Hierarchy
• Updates as precision-weighted prediction errors
Δ𝜇𝑖(𝑘)
∝ො𝜋i−1(𝑘)
𝜋𝑖(𝑘)
𝛿𝑖−1(𝑘)
SensoryPrecision
BeliefPrecision
BeliefUpdate
PE
Mathys et al., Frontiers Human Neurosci 2011Mathys et al., Frontiers Human Neurosci 2014
The hierarchical Gaussian filter (HGF): a computationally tractable model for individual learning under uncertainty
Mathys et al., Front Hum Neurosci, 2011
2017-12-06 26Methods & Models: Computational Neuroimaging
Level 1: Stimulus category
𝑝 𝑥1 = 1 =1
1 + 𝑒−𝑥2
Level 3: Phasic volatility
𝑝 𝑥3(𝑘)
~𝒩(𝑥3𝑘−1
, 𝜗)
Level 2: Tendency towards category 1
𝑝 𝑥2(𝑘)
~𝒩(𝑥2𝑘−1
, 𝑒(𝜅𝑥3𝑘−1
+𝜔))
The hierarchical Gaussian filter (HGF): a computationally tractable model for individual learning under uncertainty
2017-12-06 27Methods & Models: Computational Neuroimaging
Mathys et al., Front Hum Neurosci, 2011
HGF: Variational inversion and update equations
Inversion proceeds by introducing a mean field approximation and fitting quadratic approximations to the resulting variational energies.
This leads to simple one-step update equations for the sufficient statistics (mean and precision) of the approximate Gaussian posteriors of the hidden states 𝑥𝑖.
The updates of the means have the same structure as value updates in Rescorla-Wagner learning:
2017-12-06 28Methods & Models: Computational Neuroimaging
Δ𝜇𝑖 =ො𝜋i−1𝜋𝑖
𝛿𝑖−1
Prediction Error
Precisions determine the learning rate
From perception to action
2017-12-06 30Methods & Models: Computational Neuroimaging
• In behavioural tasks, we observe actions 𝑎
• How do we use them to infer on beliefs 𝜆?
• Answer: we invert (estimate) a response model
Example of a simple response model
Options A, B and C have values: 𝑣𝐴 = 8, 𝑣𝐵 = 4, 𝑣𝐶 = 2
We translate these values into action probabilities via a Softmax
function:
2017-12-06 31Methods & Models: Computational Neuroimaging
𝑝 𝑎 = 𝐴 =𝑒𝛽𝑣𝐴
𝑒𝛽𝑣𝐴 + 𝑒𝛽𝑣𝐵 + 𝑒𝛽𝑣𝐶
Parameter 𝛽 determines sensitivity to value differences:
low 𝛽 high 𝛽
All the necessary ingredients
Perceptual model (updates based on prediction errors)
Value function (inferred state to action value)
Response model (action value to response probability)
2017-12-06 32Methods & Models: Computational Neuroimaging
Outline
2017-12-06 33Methods & Models: Computational Neuroimaging
1. Computational
2. Algorithmic
4. Application to Psychiatry
3. Implementational
Computational fMRI: The advantage
The question event-related/block designs answer:
- Where in the brain do particular experimental conditions elicit BOLD responses?
The question model-based fMRI answers:
- How (i.e., by activation of which areas) does the brain implement a particular cognitive process?
It is able to do so because its regressors correspond to particular cognitive processes instead of experimental conditions.
2017-12-06 34Methods & Models: Computational Neuroimaging
Computational fMRI analyses
of neuromodulation
Iglesias et al., 2016
2017-12-06 35Methods & Models: Computational Neuroimaging
Computational fMRI analyses
of neuromodulation
Iglesias et al., 2016
2017-12-06 36Methods & Models: Computational Neuroimaging
Dopamine
Serotonin
Acetylcholine
Noradrenaline
Application of the HGF: Two types of PEs
1. Outcome PE
2. Cue-Outcome Contingency PE
2017-12-06 37Methods & Models: Computational Neuroimaging
𝝐𝟐
𝝐𝟑
Iglesias et al., Neuron, 2013
Application of the HGF: Sensory Learning
0 50 100 150 200 250 3000
0.5
1
prediction800/1000/1200 ms
target150/300 ms
cue300 ms
or
ITI2000 ± 500 ms
time
Changes in cue strength (black), and posterior expectation of visual category (red)
2017-12-06 38Methods & Models: Computational Neuroimaging
Iglesias et al., Neuron, 2013
Application of the HGF: Representation of precision-weighted PEs
Iglesias et al., Neuron, 2013
1. Outcome PE
z = -18
• right VTA
Dopamine
• left basal forebrain
Acetycholine
2. Probability PE
2017-12-06 39Methods & Models: Computational Neuroimaging
Application of the HGF: Social Learning
2017-12-06 40Methods & Models: Computational Neuroimaging
Changes in advice accuracy (black), and posterior expectation of adviser fidelity (red)
0 20 40 60 80 100 120 140 160 180
0.2
0.4
0.8
1
Trials
Diaconescu et al., SCAN, 2017
Representation of precision-weighted PEs in the social domain
1. Outcome PE
2017-12-06 41Methods & Models: Computational Neuroimaging
A Bfirst fMRI study
C
4
second fMRI study conjunction
x = -4, y = -12, z = -10
4
x = -4, y = -12, z = -10
4
x = -4, y = -12, z = -10
Diaconescu et al., SCAN, 2017
Representation of precision-weighted PEs in the social domain
2. Probability PE
2017-12-06 42Methods & Models: Computational Neuroimaging
Diaconescu et al., SCAN, 2017
x = -5, y = 12, z = -7
A. first fMRI study B. second fMRI study C. conjunction4
x = -5, y = 12, z = -7
3
x = -5, y = 12, z = -7
3
Computational hierarchy & its neural signature
x = -6 z = -11y = -14
x = -8 z = 5y = 6
x = -8 z = 1y = -14
x = -8 z = -11y = 10
x = 5 z = 23y = 36
x = -3 z = 34y = 2
F(46)
50
50
40
40
30
40
Hierarchy
A. B.
Hierarchy
iii. Outcome PE
ii. Advice PE
v. Volatility PE
iv. Advice Precision
vi. Volatility Precision0.6
0.8
− 1
01
2
1
1.5
0
0.5
1
0 5 0 100 15 0 200Trial number
0
0.5
1
− 1
0
1
0.4
0.5
0.6
0.8
− 1
01
2
1
1.5
0
0.5
1
0 5 0 100 15 0 200Trial number
0
0.5
1
− 1
0
1
0.4
0.5
� �( � )
� �( � )
� �( � )
� �( � )
� �( � )
� ��
Diaconescu et al., Under Revision
2017-12-06 Methods & Models: Computational Neuroimaging 43
Analogy to electrophysiological data
3. parameter estimates —SPM
2. specify regressors —general linear model
1. data volume —3d
x
z
y
mm
tim
e (
ms)
mm
x
y
t
Temporal series EEG
(t)bin time course
amplitude
voxel time courseTemporal
series fMRI
amplitude
fMRI
EEG
2017-12-06 Methods & Models: Computational Neuroimaging 44
Diaconescu et al., Under Revision
Temporal hierarchy
posterior
anterior
50 550258 534166
right
left
Time (ms)
Scalp
Location
Scalp
Location
iii. Outcome
PE
ii. Advice
PE
vi. Volatility
Precision
iv. Advice
Precision
v. Volatility
PE
352 398
2017-12-06 Methods & Models: Computational Neuroimaging 45
Diaconescu et al., Under Revision
Outline
2017-12-06 46Methods & Models: Computational Neuroimaging
1. Computational
2. Algorithmic
4. Application to Psychiatry
3. Implementational
2017-12-06 47Methods & Models: Computational Neuroimaging
Hierarchical Gaussian Filter
Prediction Errors
PredictionsBeliefPrecision
SensoryPrecision
Δ𝜇𝑖(𝑘)
∝ො𝜋i−1(𝑘)
𝜋𝑖(𝑘)
𝛿𝑖−1(𝑘)
SensoryPrecision
BeliefPrecision
BeliefUpdate
PE
Mathys et al., Frontiers Human Neurosci 2011Mathys et al., Frontiers Human Neurosci 2014
Autism
2017-12-06 48Methods & Models: Computational Neuroimaging
Lawson et al., Nature Neuroscience, 2017
Psychosis & Hallucinations
2017-12-06 49Methods & Models: Computational Neuroimaging
Powers et al., Science, 2017
Psychosis & Hallucinations
2017-12-06 50Methods & Models: Computational Neuroimaging
Powers et al., Science, 2017
Conclusion
2017-12-06 51Methods & Models: Computational Neuroimaging
Δ𝜇𝑖(𝑘)
∝ො𝜋i−1(𝑘)
𝜋𝑖(𝑘)
𝛿𝑖−1(𝑘)
SensoryPrecision
BeliefPrecision
BeliefUpdate
PE Δ𝜇𝑖(𝑘)
∝ො𝜋i−1(𝑘)
𝝅𝒊(𝒌)
𝛿𝑖−1(𝑘)
SensoryPrecision
BeliefPrecision
BeliefUpdate
PE
𝝅𝒊(𝒌)
Autism Psychosis
3rd level of the hierarchy 2nd level of the hierarchy
Take-Home Message
2017-12-06 52Methods & Models: Computational Neuroimaging
Stephan et al., Frontiers Human Neurosci 2016
References1. Daunizeau, J., den Ouden, H.E.M., Pessiglione, M., Kiebel, S.J., Stephan, K.E., and Friston, K.J. (2010).
Observing the Observer (I): Meta-Bayesian Models of Learning and Decision-Making. PLoS One 5.
2. Diaconescu, A.O., Mathys, C., Weber, L.A.E., Kasper, L., Mauer, J., and Stephan, K.E. (2017).
Hierarchical prediction errors in midbrain and septum during social learning. Soc. Cogn. Affect.
Neurosci. 12, 618–634.
3. Iglesias, S., Mathys, C., Brodersen, K.H., Kasper, L., Piccirelli, M., den Ouden, H.E.M., and Stephan,
K.E. (2013). Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning.
Neuron 80, 519–530.
4. Iglesias, S., Tomiello, S., Schneebeli, M., and Stephan, K.E. (2016). Models of neuromodulation for
computational psychiatry. Wiley Interdiscip. Rev. Cogn. Sci.
5. Lawson, R.P., Mathys, C., and Rees, G. (2017). Adults with autism overestimate the volatility of the
sensory environment. Nat. Neurosci. advance online publication.
6. Mathys, C., Daunizeau, J., Friston, K.J., and Stephan, K.E. (2011). A Bayesian foundation for individual
learning under uncertainty. Front Hum Neurosci 5.
7. Mathys, C.D., Lomakina, E.I., Daunizeau, J., Iglesias, S., Brodersen, K.H., Friston, K.J., and Stephan,
K.E. (2014). Uncertainty in perception and the Hierarchical Gaussian Filter. Front. Hum. Neurosci. 8.
8. Powers, A.R., Mathys, C., and Corlett, P.R. (2017). Pavlovian conditioning–induced hallucinations result
from overweighting of perceptual priors. Science 357, 596–600.
9. Stephan, K.E., Manjaly, Z.M., Mathys, C.D., Weber, L.A.E., Paliwal, S., Gard, T., Tittgemeyer, M.,
Fleming, S.M., Haker, H., Seth, A.K., et al. (2016). Allostatic Self-efficacy: A Metacognitive Theory of
Dyshomeostasis-Induced Fatigue and Depression. Front. Hum. Neurosci. 10.
2017-12-06 53Methods & Models: Computational Neuroimaging
How do we construct regressors that correspond to cognitive processes and use them in SPM?
1. Pass individual subject trial history into SPM:
2017-12-06 54Methods & Models: Computational Neuroimaging
How do we construct regressors that
correspond to cognitive processes and
use them in SPM?
2. Estimated subject-by-subject model parameters: Model Inversion:
2017-12-06 55Methods & Models: Computational Neuroimaging
How do we construct regressors that
correspond to cognitive processes and
use them in SPM?
3. Generate model-based time-series:
3. Convolve them with HRF:
Adapted from O’Doherty et al., 20072017-12-06 56Methods & Models: Computational Neuroimaging
How do we construct
regressors that correspond to
cognitive processes and use
them in SPM?
5. Construct your GLM:
Adapted from Behrens et al., 20102017-12-06 57Methods & Models: Computational Neuroimaging
Estimate: single subject
6. First-level analysis: Load your regressors:
2017-12-06 58Methods & Models: Computational Neuroimaging
Estimate: single subject
6. First-level analysis: Open SPM: Specify first
level analysis
2017-12-06 59Methods & Models: Computational Neuroimaging
Estimate: single subject
6. First-level analysis: Load Design matrix into
Batch editor
2017-12-06 60Methods & Models: Computational Neuroimaging