- Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of...
Transcript of - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of...
![Page 1: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/1.jpg)
Biological Modeling
of Neural Networks:
Week 14 – Dynamics and Plasticity
Wulfram Gerstner
EPFL, Lausanne, Switzerland
14.1 Reservoir computing
- Review:Random Networks
- Computing with rich dynamics
14.2 Random Networks - stationary state
- chaos
14.3 Stability optimized circuits - application to motor data
14.4. Synaptic plasticity
- Hebbian
- Reward-modulated
14.5. Helping Humans - oscillations
- network states
Week 14 – Dynamics and Plasticity
![Page 2: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/2.jpg)
Neuronal Dynamics – Brain dynamics is complex
motor
cortex
frontal
cortex
to motor
output
10 000 neurons
3 km wire 1mm
Week 14-part 1: Review: The brain is complex
![Page 3: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/3.jpg)
Neuronal Dynamics – Brain dynamics is complex
motor
cortex
frontal
cortex
to motor
output
Week 14-part 1: Review: The brain is complex
-Complex internal dynamics
-Memory
-Response to inputs
-Decision making
-Non-stationary
-Movement planning
-More than one ‘activity’ value
![Page 4: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/4.jpg)
Liquid Computing/Reservoir Computing:
exploit rich brain dynamics
Readout 1
Readout 2
Maass et al. 2002,
Jaeger and Haas, 2004
Review:
Maass and Buonomano,
Stream of
sensory inputs
Week 14-part 1: Reservoir computing
![Page 5: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/5.jpg)
See Maass et al. 2007
Week 14-part 1: Reservoir computing
-‘calculcate’ 3 4
- if-condition on 1
![Page 6: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/6.jpg)
Modeling Hennequin et al. 2014,
See also:
Maass et al. 2002,
Sussillo and Abbott, 2009
Laje and Buonomano, 2012
Shenoy et al., 2011
Experiments of Churchland et al. 2010
Churchland et al. 2012
See also:
Shenoy et al. 2011
Week 14-part 1: Rich dynamics Rich neuronal dynamics
![Page 7: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/7.jpg)
-Long transients
-Reliable (non-chaotic)
-Rich dynamics (non-trivial)
-Compatible with neural data (excitation/inhibition)
-Plausible plasticity rules
Week 14-part 1: Rich neuronal dynamics: a wish list
![Page 8: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/8.jpg)
Biological Modeling
of Neural Networks:
Week 14 – Dynamics and Plasticity
Wulfram Gerstner
EPFL, Lausanne, Switzerland
14.1 Reservoir computing
- Review:Random Networks
- Computing with rich dynamics
14.2 Random Networks - rate model
- stationary state and chaos
14.3 Hebbian Plasticity - excitatory synapses
- inhibitory synapses
14.4. Reward-modulated plasticity
- free solution
14.5. Helping Humans - time dependent activity
- network states
Week 14 – Dynamics and Plasticity
![Page 9: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/9.jpg)
I(t)
)(tAn
Week 14-part 2: Review: microscopic vs. macroscopic
![Page 10: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/10.jpg)
Homogeneous
network: -each neuron receives input
from k neurons in network
-each neuron receives the same
(mean) external input
Week 14-part 2: Review: Random coupling
excitation
inhibition
![Page 11: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/11.jpg)
Stochastic spike arrival:
excitation, total rate Re
inhibition, total rate Ri
u
0u
EPSC IPSC
Synaptic current pulses
)()()( ttIRuuudt
d meaneq
Langevin equation,
Ornstein Uhlenbeck process
Fokker-Planck equation
)()()(
','
''
,
fk
fki
fk
fkeeq ttqttqRuuu
dt
d
Week 14-part 2: Review: integrate-and-fire/stochastic spike arrival
Firing times:
Threshold crossing
![Page 12: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/12.jpg)
( )i i ij j
j
dr r F w r
dt
Fixed point with F(0)=0
0ir
stable unstable
Suppose:
1 '(0) ( 0)d
F F xdx
( )d
x x F w xdt
Suppose 1 dimension
Week 14-part 2: Dynamics in Rate Networks F-I curve
of rate neuron
Slope 1
![Page 13: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/13.jpg)
Exercise 1: Stability of fixed point
Next lecture:
9h43 ( )d
x x F w xdt
Fixed point with F(0)=0
0x Suppose:
1 '(0) ( 0)d
F F xdx
( )d
x x F w xdt
Suppose 1 dimension
Calculate stability,
take w as parameter
![Page 14: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/14.jpg)
( )i i ij j
j
dr r F w r
dt
Fixed point with F(0)=0
0ir
stable unstable
Suppose:
1 '(0) ( 0)d
F F xdx
( )d
x x F w xdt
w<1 w>1
Suppose 1 dimension
Week 14-part 2: Dynamics in Rate Networks
Blackboard:
Two dimensions!
![Page 15: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/15.jpg)
( )i i ij j
j
dr r F w r
dt
Fixed point:
0ir
stable unstable
Chaotic dynamics:
Sompolinksy et al. 1988
(and many others:
Amari, ...
1 '(0) ( 0)d
F F xdx
Random,
10 percent connectivity
Re( ) 1 Re( ) 1
Week 14-part 2: Dynamics in RANDOM Rate Networks
![Page 16: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/16.jpg)
Rajan and Abbott, 2006
Image: Ostojic, Nat.Neurosci, 2014
( ) ( )i i ij j i
j
dr r F w r t
dt
Unstable dynamics and Chaos
Image:
Hennequin et al. Neuron, 2014
chaos
![Page 17: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/17.jpg)
Image: Ostojic, Nat.Neurosci, 2014
( )f
i i ij j
j f
du u w t t
dt
Firing times:
Threshold crossing
Week 14-part 2: Dynamics in Random SPIKING Networks
![Page 18: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/18.jpg)
Ostojic,
Nat.Neurosci, 2014
Week 14-part 2: Stationary activity: two different regimes
Switching/bursts
long autocorrelations:
Rate chaos
Re( ) 1
Stable rate fixed point,
microscopic chaos
![Page 19: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/19.jpg)
-Long transients
-Reliable (non-chaotic)
-Rich dynamics (non-trivial)
-Compatible with neural data (excitation/inhibition)
-Plausible plasticity rules
Week 14-part 2: Rich neuronal dynamics: a wish list
![Page 20: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/20.jpg)
Biological Modeling
of Neural Networks:
Week 14 – Dynamics and Plasticity
Wulfram Gerstner
EPFL, Lausanne, Switzerland
14.1 Reservoir computing
- Review:Random Networks
- Computing with rich dynamics
14.2 Random Networks - stationary state
- chaos
14.3 Stability optimized circuits - application to motor data
14.4. Synaptic plasticity
- Hebbian
- Reward-modulated
14.5. Helping Humans - time dependent activity
- network states
Week 14 – Dynamics and Plasticity
![Page 21: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/21.jpg)
Image:
Hennequin et al. Neuron, 2014
Re( ) 1 Re( ) 1
Week 14-part 3: Plasticity-optimized circuit
![Page 22: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/22.jpg)
Optimal control of transient dynamics in balanced networks
supports generation of complex movements
Hennequin et al. 2014,
![Page 23: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/23.jpg)
Random stability-optimized circuit (SOC)
Random
connectivity
![Page 24: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/24.jpg)
Random
Week 14-part 3: Random Plasticity-optimized circuit
![Page 25: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/25.jpg)
study
duration
of transients
F-I curve
of rate neuron
slope 1/linear theory
slope 1/linear theory
a1= slowest
= most amplified
Week 14-part 3: Random Plasticity-optimized circuit
![Page 26: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/26.jpg)
slope 1/linear theory F-I curve
of rate neuron
a1= slowest
= most amplified
Week 14-part 3: Random Plasticity-optimized circuit
![Page 27: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/27.jpg)
Optimal control of transient dynamics in balanced networks
supports generation of complex movements
Hennequin et al.
NEURON 2014,
![Page 28: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/28.jpg)
Churchland et al. 2010/2012 Hennequin et al. 2014
Week 14-part 3: Application to motor cortex: data and model
![Page 29: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/29.jpg)
Comparison: weak random
Hennequin et al. 2014,
Week 14-part 3: Random Plasticity-optimized circuit
![Page 30: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/30.jpg)
Random connections, fast
‘distal’ connections, slow,
(Branco&Hausser, 2011)
structured
12000 excitatory LIF = 200 pools of 60 neurons
3000 inhibitory LIF = 200 pools of 15 neurons
Classic sparse random connectivity (Brunel 2000)
Stabilizy-optimized random connections
Overall:
20% connectivity
Fast AMPA
slow NMDA
Week 14-part 3: Stability optimized SPIKING network
![Page 31: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/31.jpg)
Neuron 1 Neuron 2 Neuron 3
Spontaneous
firing rate
Single neuron
different initial conditions
Hennequin et al. 2014
Classic sparse random connectivity (Brunel 2000)
Week 14-part 3: Stability optimized SPIKING network
![Page 32: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/32.jpg)
Hennequin et al. 2014
Classic sparse random connectivity (Brunel 2000)
Week 14-part 3: Stability optimized SPIKING network
![Page 33: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/33.jpg)
-Long transients
-Reliable (non-chaotic)
-Rich dynamics (non-trivial)
-Compatible with neural data (excitation/inhibition)
-Plausible plasticity rules
Week 14-part 3: Rich neuronal dynamics: a result list
![Page 34: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/34.jpg)
Biological Modeling
of Neural Networks:
Week 14 – Dynamics and Plasticity
Wulfram Gerstner
EPFL, Lausanne, Switzerland
14.1 Reservoir computing
- Review:Random Networks
- Computing with rich dynamics
14.2 Random Networks - stationary state
- chaos
14.3 Stability optimized circuits - application to motor data
14.4. Synaptic plasticity
- Hebbian
- Reward-modulated
14.5. Helping Humans - time dependent activity
- network states
Week 14 – Dynamics and Plasticity
![Page 35: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/35.jpg)
Hebbian Learning
= all inputs/all times are equal
),( postpreFwij
pre post
i j
ijw
![Page 36: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/36.jpg)
Week 14-part 4: STDP = spike-based Hebbian learning
Pre-before post: potentiation of synapse
Pre-after-post: depression of synapse
![Page 37: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/37.jpg)
Modulation of Learning
= Hebb+ confirmation confirmation
( , , )ijw F pre post CONFIRM
local global
Functional Postulate Useful for learning the important stuff
Many models (and experiments) of synaptic plasticity
do not take into account Neuromodulators.
Except: e.g. Schultz et al. 1997, Wickens 2002, Izhikevich, 2007; Reymann+Frey 2007;
Moncada 2007, Pawlak+Kerr 2008; Pawlak et al. 2010
![Page 38: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/38.jpg)
Consolidation of Learning
Success/reward Confirmation
-Novel
-Interesting
-Rewarding
-Surprising
Neuromodulators
dopmaine/serotonin/Ach
‘write now’ to long-term memory’
Crow (1968),
Fregnac et al (2010),
Izhikevich (2007)
![Page 39: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/39.jpg)
Plasticity
BUT - replace by inhibitory plasticity
avoids chaotic blow-up of network
avoids blow-up of single neuron (detailed balance)
yields stability optimized circuits
Vogels et al.,
Science 2011
- here: algorithmically tuned
Stability-optimized curcuits
![Page 40: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/40.jpg)
Plasticity
BUT
- replace by 3-factor plasticity rules
- here: algorithmically tuned
Readout
Izhikevich, 2007
Fremaux et al.
2012 Success signal
![Page 41: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/41.jpg)
Week 14-part 4: Plasticity modulated by reward
Dopamine encodes success=
reward – expected reward
Dopamine-emitting neurons:
Schultz et al., 1997
Izhikevich, 2007
Fremaux et al.
2012 Success signal
![Page 42: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/42.jpg)
Week 14-part 4: Plasticity modulated by reward
![Page 43: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/43.jpg)
Week 14-part 4: STDP = spike-based Hebbian learning
Pre-before post: potentiation of synapse
![Page 44: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/44.jpg)
Week 14-part 4: Plasticity modulated by reward
STDP with pre-before post: potentiation of synapse
![Page 45: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/45.jpg)
Week 14-part 4: from spikes to movement
How can the
readouts
encode movement?
![Page 46: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/46.jpg)
Population vector coding of movements
Schwartz et al.
1988
Week 14-part 5: Population vector coding
![Page 47: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/47.jpg)
• 70’000 synapses
• 1 trial =1 second
• Output to trajectories via population vector coding
• Single reward at the END of each trial based on
similarity with a target trajectory
Population vector coding of movements
Fremaux et al., J. Neurosci. 2010
Week 14-part 4: Learning movement trajectories
![Page 48: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/48.jpg)
Perf
orm
ance
Fremaux et al. J. Neurosci.
2010
QuickTime™ and a decompressor
are needed to see this picture.
R-STDP LTP-
only
Week 14-part 4: Learning movement trajectories
![Page 49: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/49.jpg)
Reward-modulated STDP for movement learning
- Readout connections, tuned
by 3-factor plasticity rule
Fremaux et al.
2012 Success signal
Hebbian STDP
- inhibitory connections, tuned
by 2-factor STDP, for
stabilization
Week 14-part 4: Plasticity can tune the network and readout
Vogels et al.
2011
![Page 50: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/50.jpg)
Exam: - written exam 17. 06. 2014 from 16:15-19:00
- miniprojects counts 1/3 towards final grade
For written exam: -bring 1 page A5 of own notes/summary
-HANDWRITTEN!
Last Lecture TODAY
![Page 51: - Review:Random Networks - Computing with rich dynamics …lcn€¦ · Biological Modeling of Neural Networks: Week 14 – Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne,](https://reader033.fdocuments.net/reader033/viewer/2022043004/5f84fad45a0d03454054460d/html5/thumbnails/51.jpg)
Nearly the end:
what can I improve for the students next year?
Integrated exercises?
Miniproject?
Overall workload ?(4 credit course = 6hrs per week)
Background/Prerequisites?
-Physics students
-SV students
-Math students
Quizzes?
Slides? videos?