Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele &...
-
Upload
emily-wood -
Category
Documents
-
view
218 -
download
0
description
Transcript of Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele &...
![Page 1: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/1.jpg)
Several strategies for simple cells to learn orientation and
direction selectivityMichael Eisele & Kenneth D. Miller
Columbia University
![Page 2: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/2.jpg)
ONOFF
illustration by de Angelis et al. 99
Orientation and Direction SelectivityOrientation Selectivity
(OS)
Direction Selectivity
(DS)
![Page 3: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/3.jpg)
spacesp
ace
spac
e
space
no OS OSorientation-selective?
spac
e
timesp
ace
time
DSno DSdirection-selective?
orientation-selective?
Lampl et al 01
Priebe & Ferster 05
![Page 4: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/4.jpg)
Selected models• Simple Hebbian learning rule produces OS (Miller
94), but not DS (Wimbauer et al 97) for unstructured input.
• Nonlinear Hebbian learning rules produce DS, but only for structured input (Feidler et al 97, Blais et al 00).
• More general principles (sparse coding, ICA, blind source separation) can explain occurence of OS (Olshausen & Field 96; Bell & Sejnowski 97) and DS (van Hateren & Ruderman 98), if applied to input from natural scenes.
![Page 5: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/5.jpg)
Some OS and DS develops early
(kittens at time of eye opening; Albus & Wolf 84)
awake ferret P27 (before eye opening)Chiu & Weliky 01
Early spontaneous activity
Ferret P30-32 correlations decay over a few 100 ms and several mm cortex (Fiser et al 04)
![Page 6: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/6.jpg)
•Find rule that robustly produces DS, using only unstructured input.
•Identify underlying principle.
Goal
Blind source separationmixing
![Page 7: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/7.jpg)
sources unmixed sources
mixing unmixing
Blind source separation (BSS)
sensors
![Page 8: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/8.jpg)
sources sensors
random mixing
Blind source separationof random, spontaneous activity
unmixing
more even mixing
mixing
?
![Page 9: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/9.jpg)
Motivation for blind source mixing (BSM)
DS responses to all positions
⇒
no responseto some positions
no DS no DSresponses
to all positions
Hebbian learning
![Page 10: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/10.jpg)
Combining BSM and Hebbian learning
![Page 11: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/11.jpg)
Δw = η⋅(x⋅y + ε⋅x⋅y3) - λ⋅w
w = weightΔw = weight-change η = learning ratex = inputy = outputλ = multiplicative constraint
linear Hebbian
ε>0: blind source separationε<0: blind source mixing
based on bottom-up approach to blind-source separation; see “Independent Component Analysis” Hyvärinen, Karhunen, Oja 2001
Combined learning rule
![Page 12: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/12.jpg)
•spatial correlations: Mexican hat
•distribution of input amplitudes: long tails
•upper weight limits: none
•temporal input filters: diverse
Important factors
4 week old kittensCai et al 97
![Page 13: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/13.jpg)
•single neuron learning
•rate-coded
•only feedforward input
•arbor function
•linear neuron model
Simplifications
![Page 14: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/14.jpg)
•Whitened input ⇒ BSM can perfectly mix sources.
•Gradient principle ⇒ convergence
A few analytical results
![Page 15: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/15.jpg)
preferred orientations of 100 receptive fields
other choice ofinitial weights:
Dependence on initial conditions
rotationON ⇔ OFF
ε = −0.25
![Page 16: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/16.jpg)
ε = 0(Hebb)
ε = −0.15
ε = −0.5
ε = −0.15
ε = −0.2
ε = −0.5
Robustness against parameter changes Δw = η⋅(x⋅y + ε⋅x⋅y3) - λ⋅w
OS and DS develop robustly under BSM + Hebb
![Page 17: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/17.jpg)
Limitations
special initial conditions input = drifting gratings
input amplitudes = subgaussian distributionlarge negative ε: BSM dominates
![Page 18: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/18.jpg)
response amplitudenum
ber o
f res
pons
esComparision of response distributions
![Page 19: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/19.jpg)
Other strategies:BSS with structured inputBSM with subgaussian input
Hebb with hard upper w-limitHebb with soft upper w-limit
hybrid with unstructured inputhybrid with structured input hybrid = BSS and
Hebb with upper
weight limit
Any rule that produces OS and DS for structured and unstructured input?
![Page 20: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/20.jpg)
Linear Hebbian rule + upper weight limitMiller 94
![Page 21: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/21.jpg)
•Blind source mixing (BSM) is designed to produce an output that responds evenly to many sources.
•BSM and and Hebbian learning can be combined to a simple synaptic learning rule.
•This rule robustly produces OS and DS while the input is unstructured.
Conclusions
BSS +➧BSM +Hebb +OS, DS ➧OS, DS
known: new:
![Page 22: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/22.jpg)
Speculationexternal world internal network neuron
unlearn correlations that are produced internally: BSM
learn correlations that are produced externally: BSS
Unlearning of higher-order correlations.Compare Crick & Mitchison 83:unlearning of any-order correlations.
➡ ➡
![Page 23: Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University.](https://reader035.fdocuments.net/reader035/viewer/2022062412/5a4d1afb7f8b9ab059983fc5/html5/thumbnails/23.jpg)
supported by the Swartz Foundation and theHuman Frontiers Science Program