The BCM theory of synaptic plasticity. The BCM … BCM theory of synaptic plasticity. The BCM theory...
Transcript of The BCM theory of synaptic plasticity. The BCM … BCM theory of synaptic plasticity. The BCM theory...
The BCM theory of synaptic plasticity.
The BCM theory of cortical plasticity
BCM stands for Bienestock Cooper and Munro, it dates back to 1982. It was designed in order to account for experiments which demonstrated that the development of orientation selective cells depends on rearing in a patterned environment.
BCM Theory (Bienenstock, Cooper, Munro 1982; Intrator, Cooper 1992)
LTD LTP
€
dw j
dt=ηx jφ(y,θm )
€
φ(y,θm )
€
θm
€
y
€
y
w1 w2 w3 w4 w5
€
x1
€
x2
€
x3
€
x4
€
x5€
y = w jj∑ x j
For simplicity – linear neuron
1) The learning rule:
BCM Theory
• Bidirectional synaptic modification LTP/LTD • Sliding modification threshold • The fixed points depend on the environment, and in a patterned environment only selective fixed points are stable.
LTD LTP
Requires
€
dw j
dt=ηx jφ(y,θm )
€
φ(y,θm )
€
θm ∝ E y 2[ ] =1τ
y 2−∞
t
∫ ( ′ t )e−( t− ′ t ) /τd ′ t
€
θm
€
y
2) The sliding threshold
1)
Is equivalent to this differential form: The integral form of the average:
Note, it is essential that θm is a superlinear function of the history of C, that is:
with p>1
Note also that in the original BCM formulation (1982) rather then
€
θm ∝ E y 2[ ] =1τ
y 2−∞
t
∫ ( ′ t )e−( t− ′ t ) /τd ′ t
€
dθmdt
=1τ(y 2 −θm )
€
dθmdt
=1τ(y p −θm )
€
θm ∝ E y[ ]2
€
θm ∝ E y 2[ ]
What is the outcome of the BCM theory?
Assume a neuron with N inputs (N synapses), and an environment composed of N different input vectors.
An N=2 example:
What are the stable fixed points of W in this case?
x1
x2 €
x1 =1.00.2
x2 =
0.10.9
(Notation: )
What are the fixed points? What are the stable fixed points?
Note:"Every time a new input is presented, W changes, and so does θm"
x1
x2
€
yi = wT ⋅ xi
(Show matlab)
Two examples with N= 5
Note: The stable FP is such that for one pattern yi=wTxi=θm while for the others"y(i≠j)=0.
(note: here c=y)
Show movie
BCM Theory Stability
• One dimension
• Quadratic form
• Instantaneous limit
€
dwdt
=ηy y −θM( )x
€
dwdt
= ηy y − y 2( )x= ηy 2(1− y)x
BCM Theory Selectivity
• Two dimensions
• Two patterns
• Quadratic form
• Averaged threshold
€
dwdt
=η yk yk −θM( )x k
€
θM = E y 2[ ]patterns
= pk (yk )2
k=1
2
∑
,
• Fixed points
BCM Theory: Selectivity
• Learning Equation
• Four possible fixed points , , , , (unselective)
(unselective) (Selective)* (Selective)
• Threshold*
€
dwdt
=ηyk yk −θM( )x k
Stability of Fixed point
Assume w* is a fixed point:
Assume that Δw is the deviation from this fixed point. w=w*+Δw.
Then:
If then €
dwdt
=dw*
dt+dΔwdt
=dΔwdt
€
dwdt
= F(w)
€
dΔwdt
= F(w* + Δw) ≈ F w*( ) + Δw ∂F∂W w*
€
F(w*) = 0
Consider a selective F.P (w1) where:
and
So that
for a small pertubation from the F.P such that
The two inputs result in: €
θm1 = E[y 2] =
12(w1 ⋅ x1)2 + (w1 ⋅ x 2)2[ ] =
12θm1[ ]
2
So that
€
θm =12(w1 ⋅ x1 + Δw ⋅ x1)2 + (w1 ⋅ x 2 + Δw ⋅ x 2 )2{ }
≈θm* (1+ Δw ⋅ x1) +O(Δw2) = θm
* + 2Δw ⋅ x1
€
ηΦ ≈ −ε2y
€
ηΦ ≈ ε1 y −θm( )
At y≈0 and at y≈θm we make a linear approximation
In order to examine whether a fixed point is stable we examine if the average norm of the perturbation ||Δw|| increases or decreases.
Decrease ≡ Stable Increase ≡ Unstable
Remember:
For the preferred input x1:
Similarly, for the non preferred input x2:
€
θm = θm* + 2Δw ⋅ x1
€
dΔwdt
= ε1(y −θm )x1 = ε1(θm
* + Δw ⋅ x1 −θm )x1
ε1(θm* + Δw ⋅ x1 −θm
* − 2Δw ⋅ x1)x = −ε1(Δw ⋅ x1)x1
€
Δw•
= −ε2y ⋅ x2 = −ε2(Δw ⋅ x
2)x 2
Use trick:
And average over two input patterns
Insert previous result to show that:
For the non selective F.P we get:
€
ddt
Δw[ ]2 =ddt
Δw ⋅ Δw[ ] = 2Δw ⋅ Δw•
€
E ddt
Δw[ ]2
=12Δw ⋅ Δw
•
x1
+12Δw ⋅ Δw
•
x 2
€
E ddt
Δw[ ]2
= − ε1(Δw ⋅ x
1)2 + ε2(Δw ⋅ x2)2[ ] < 0
€
E ddt
Δw[ ]2
≥ 0
Phase plane analysis of BCM in 1D
Previous analysis assumed that θm=E[y2] exactly. If we use instead the dynamical equation
Will the stability be altered?
Look at 1D example
€
dθmdt
=1τ(y 2 −θm )
Phase plane analysis of BCM in 1D
Assume x=1 and therefore y=w. Get the two BCM equations:
There are two fixed points y=0, θm=0, and y=1, θm=1. The previous analysis shows that the second one is stable, what would be the case here? How can we do this? (supplementary homework problem)
0 0.5 1 θm
y 1
0.5
0
?
€
dydt
=ηy(y −θm )
dθm
dt=1τ(y 2 −θm )
Summary
• The BCM rule is based on two differential equations, what are they?
• When there are two linearly independent inputs, what will be the BCM stable fixed points? What will θ be?
• When there are K independent inputs, what are the stable fixed points? What will θ be?
Homework 2: due on Feb 1
1. Code a single BCM neuron, apply to case with 2 linearly independent inputs with equal probability
2. Apply to 2 inputs with different probabilities, what is different?
3. Apply to 4 linearly indep. Inputs with same prob.
Extra credit 25 pt 4. a. Analyze the f.p in 1D case, what are the stable
f.p as a function of the systems parameters. b. Use simulations to plot dynamics of y(t), θ(t) and their trajectories in the m θ plane for different parameters. Compare stability to analytical results (Key parameters, η τ)
Natural Images, Noise, and Learning
image retinal activity
• present patches
• update weights
• Patches from retinal activity image
• Patches from noise
Retina
LGN
Cortex
BCM neurons can develop both orientation selectivity and varying degrees of Ocular Dominance
Shouval et. al., Neural Computation, 1996
Left Eye
Right Eye
Right Synapses Left
Synapses
0 50 100
0 50 100
0 20 40
1 2 3 4 5 0 50 100
Bin
No. o
f Cel
ls
The distinction between homosynaptic and heterosynaptic models
Examples:
BCM – homosynaptic
PCA (Oja) - heterosynaptic €
dw j
dt=ηx jφ(y,θm )
Noise Dependence of MD Two families of synaptic plasticity rules
Hom
osyn
aptic
He
tero
syna
ptic
Blais, Shouval, Cooper. PNAS, 1999
QBCM K1 S1
Noise std Noise std Noise std
S2 K2 PCA
Noise std Noise std Noise std
Nor
mali
zed
Tim
e
N
orm
alize
d T
ime
I n t r a o c u l a r i n j e c t i o n o f T T X r e d u c e s a c t i v i t y o f t h e " d e p r i v e d - e y e " L G N i n p u t s t o c o r t e x
0 5 1 0 1 5 2 0 2 5 3 0
0 2 1
E y e o p e n
0 5 1 0 1 5
0 2 1
S p i k e s / s e c
0 5 1 0 1 5
0 2 1
L i d c l o s e d
T i m e ( s e c )
i LGN
TTX
Experiment design
Blind injection of TTX and lid- suture (P49-61)
Dark rearing to allow TTX to wear off
Quantitative measurements of ocular dominance
Rittenhouse, Shouval, Paradiso, Bear - Nature 1999
MS= Monocular lid suture MI= Monocular inactivation (TTX)
Cumulative distribution Of OD
Why is Binocular Deprivation slower than Monocular Deprivation?
Monocular Deprivation Binocular Deprivation