Post on 15-Jan-2016
1
Learning induced improvement of internal representations in motor
cortex
Eilon Vaadia
2
Outline
Introduction: Voluntary movement and its Neural representation
Neuronal correlates of learning in the monkey brain.
Epilog: Implication for future applications
Introduction: Voluntary movement and its Neural representation
3
Action is Essential for Sensory Perception
4
Visually Guided movements
Sensory perception and motor action are embedded in one tight sensorimotor loop
During visually reaching movement, visual representation of the target location must be transformed into appropriate movement coordinates
5
Forward Dynamic model
The sensorimotor loop
Forward sensory model
Inverse model
Previous state
New State
Neuronal representations
context
context
Wolpert and Ghahramani 2000 (review)
context
Desired Kinematic
s
Motor command
6
Internal model in a simplistic definition:
The neuronal process that computes a desired action given the sensory
inputs and their context
7
The ingenuity of our brain
Ability to adapt, to a wide variety of perturbations
Predict the effect of the motor output on the inputs (von Helmholtz, 1867)
How does the brain perform these tasks?
8
Testing properties of internal modelsExample: Psychophysics of reaching movements
“standard mapping”
9
Standard Mapping
10
Standard Mapping
11
Standard Mapping
12
Standard Mapping
13
Standard Mapping
14
Standard Mapping
15
Standard Mapping
16
Reaching with Visuomotor rotation
17
Reaching with Visuomotor rotation
18
Reaching with Visuomotor rotation
19
Reaching with Visuomotor rotation
20
Reaching with Visuomotor rotation
21
Reaching with Visuomotor rotation
22
Reaching with Visuomotor rotation
23
Reaching with Visuomotor rotation
24
At the end of learning
25
Return to default; the aftereffect tells us that an internal model was formed
26
Return to default; the aftereffect tells us that an internal model was formed
27
Return to default; the aftereffect tells us that an internal model was formed
28
Return to default; the aftereffect tells us that an internal model was formed
29
Return to default; the aftereffect tells us that an internal model was formed
30
Return to default
Return to default; the aftereffect tells us that an internal model was formed
31
Testing for non-learned directions:Is the aftereffect local ?
32
Testing for non-learned directions:Is the aftereffect local ?
33
Testing for non-learned directions:Is the aftereffect local ?
34
Testing for non-learned directions:Is the aftereffect local ?
35
Testing for non-learned directions:Is the aftereffect local ?
36
YES!Learning is Local: No Generalization
Testing for non-learned directions:Is the aftereffect local ?
37
Human and monkeys learn to perform this task rapidly (few trials up to 100).
38
Humans
Krakauer et al J. Neuroscience, 20(23):8916–8924 2000 (Ghez lab.)
39
What is the neural basis of such local learning?
Localized internal model suggests:The internal model uses neuronal elements with localized spatial fields
Interestingly.…
40
The motor cortex
MI
41
Directional Tuning of MI Cells
Movement onset
(Georgopoulos et al 1982)
P.D.
42
Cosine tuning
PD
Direction of movement
Spi
kes/
sec
43
Population of neurons in MI Accurately Represent the Direction of Movements
44
The population of cells in MI tell the external observes quite accurately what is the movement direction
Each of the cells in this population may serve as the “local elements” we need…
End of part 1
45
Part II
Introduction: Voluntary movement and its Neural representation
Neuronal correlates of learning in the monkey brain*
Implication for future applications
* R. Paz, C. Nathan, T. Boraud, H. Bergman and E. Vaadia, Nature Neurosci. 2003
46
Objectives
• Teach the animal (the brain) to generate a new mapping between visual instruction and motor output
• Search for a for the neuronal representation of localized internal models
47
Monkeys and video games…
48
The behavioral task
Default (pre) (8-target task)
“Preparatory” “Movement”
Go!
CueStart trial Movement
Default (post)(8-target task)
Transformation (one-target task)
“learned direction”
CursorHand
49
Direction
norm
aliz
ed d
evia
tion
Pre-learning
Post-learning:
12-34-56-7
0.4
0
0.2
315 0 45 90 135 180 225 270
1. The Behavioral Aftereffect is local
Results
50
2. Single neurons show dynamic adaptation during learning
Tri
al n
umbe
r
Target onset MVT onset
Time Timecounts counts0 750 15 0 750 15
Preparatory Movement
Note: The adaptation of rate is evident only during the preparatory period
51
3. Firing-rates and behaviors improvement
show similar dynamics
Trial number
Nor
mal
ized
rat
e / N
orm
aliz
ed e
rror
Preparatory Movement-related
Actual ActivityExpectedBehavior
52
4. Directional tuning of some neurons is modified
(Post vs. Pre)
PD
Distance from learned direction0 180-180
30S
pik
es/
sec
Pre learning:Post learning:
53
5. Population Tuning: Only cells with PD near the learned direction show increased activity
Pos
t-P
re d
iffe
renc
e
Distance of Preferred direction from learned direction
Learned
±30
54
6. Learning-induced enhancement: Almost all Cells with PD near the learned direction!
55
OK… So what do we have till now?
1. The behavioral effect is local (aftereffect is local)2. The effect on neuronal activity is local
a. Firing rate increases only near the learned directionb. Only cells with PD near the learned direction show
the effect
3. The effect occurs during movement preparation4. The effect persists in post learning, even after
return to default movements.
56
How do these changes help the brain?
57
-90 -45 0 45 90
Err
or (
Deg
)35
20
10
The PV error is reduced
Learned direction
58
The Signal to Noise Increases
Non-learned directions
SN
R: M
ean/
S.D
impr
ovem
ent
Learned directionRepeated
*
59
Conclusion
After learning: The brain (and external observers) can better predict the direction of the learned movement.
60
How was the representation improved?
61
Mutual information (rate and direction)
r
d r
drPdrPdPrPrPI )|(log)|()()(log)( 22
i
iid ddrdpN
d i )]())(log([))((log(1
maxargˆ
Cover and Thomas 1991
Rolls, Treves et al 1997
1. All directions
2. A specific direction
3. Predicting a direction
I – Mutual information r =rate d= direction
σi - the mean firing rate of cell i in direction d ri - the firing rate in randomly drawn trial Sanger, 1996
r rP
drPdrPdI
)(
)|(log)|()( 2
62
Change in Mutual Information after Learning
PDs distribution Cells with increased information (p>0.95)
All cells
Information
p-value
Num
ber
of
cells
Mutual information (bits)
63
The improvement is associated with…
Δ- Fano-factor
Δ - slope
Δ-I
nfor
mat
ion
Δ-I
nfor
mat
ion2. Changed Variability?
1. Changed Slope?
64
Tuning slope is locally increased
Δ (
post
-pre
) sl
ope
Distance from learned direction
Cells with significant increased information
Other cells
Learned
65
Three ways to affect the slope
Peak-rate
Learned
1 .PD shift Narrowing
Learned
Change of PD
-In
form
atio
n C=0.05
Change of Width(at half height)
C=0.18
Change of rate (at learned direction)
C=0.56
Learned
66
Summary of Results
Learning induces highly specific and subtle changes in single neurons’ properties.
The neuronal changes are maintained after learning
Learning-induced changes specifically improve the neuronal representation of the learned movement
67
Conclusions (speculations…)
The motor motor cortex learns Local learning may be useful – The
representation of the learned direction is improved without affecting representations of other directions.
The internal model is maintained without interfering with default behavior.
End of part 2
68
Epilog
Introduction: Voluntary movement and its Neural representation
Neuronal correlates of learning in the monkey brain
Implication for future applications
69
Towards Neural prosthesis
Miguel Nicolelis 2001
70
Improving movement reconstruction
Learning helps in shaping brain activity. Thus, A smaller number of cells may be used more efficiently to predict movements
Future advances Improve data (Schwartz et al)
• Appropriate training• Appropriate recordings
Add signals (LFPs) (Aertsen et al) Improve algorithms (Shpigelman et al)
71
Linear Regression
)()()()( tututMu
Xab
The end point M at time t where X is the matrix
of units activity and a is a set of impulse response functions (weights).
72
“Spikernel” - Motivation
a “spike trains” kernel that maps similar activity patterns to nearby areas of the feature space.
)()()( tutMu
ab )( ut X )]([ utX
ttk
btktM
ii
iiii
xxxx
xx
,
,VectorSupport
*
73
“Spikernel” Performance
“standard” Kernels
Spi
kern
el
74
Reconstruction of movement Velocity(open loop)
(30 units)
75
Towards Neural prosthesis
It’s a long way to go… …But it seems the landscapes on
the way are at least as exciting as the destination
76
With Contributions by:Hagai Bergman, ICNCNaftali Tishby ICNCYoram Singer CS
Gal Chechick ICNCAmir Globerson ICNC
Acknowledgments
Rony Paz (Learning Experiments)
Chen Nathan (Learning Experiments)Thomas Boraud (Learning Experiments)
Lavi Shpigelman (spikernel)
77
78
Possible ways to improve the representation
Distance (degrees) from learned direction
79
The Increased information is best correlated with local enhancement of firing rate
C=0.56P=0.183
C=0.189P=0.276
C=0.057P=0.419
80
81
Conclusions
1. The firing rate of some cells is enhanced during learning.
2. A population of selected cells – show a similar enhancement
3. The enhancement takes similar time course as the learning.
4. The enhancement occurs only during movement preparation.
Selected cells? Who are these
cells?
selected
85A.B. Schwartz, Pittsburg, USA
86A.B. Schwartz, Pittsburg, USA
87A.B. Schwartz, Pittsburg, USA
88
Monkeys Visual cortex
Improved long term neuronal performance resulted from changes in the slope of orientation-tuning of individual neurons.
The slope increased only for the subgroup of cells (with PO near the trained orientation)
No modifications of the tuning curve were observed for untrained orientations
Schoups, Vogels, Qian & Orban 2001
89
5. Population Tuning: Only cells with PD near the learned direction show increased activity
Distance of Preferred direction from learned direction
Avg
. nor
mal
ized
rat
es
Pre-learningPost-learning
90
“Local elements Exist !”
-42 -17 0 +16 +37
P.O.Cell 1
Cell 3Cell 2 Cell 4 Cell 5
Schoups, Vogels, Qian & Orban 2001
91
Epilog 1
Similar cortical basis for learning in visual and motor function?
Monkeys Visual cortex Improved long term neuronal performance resulted from
changes in the slope of orientation-tuning of individual neurons.
only subgroup of cells with PO close to the trained orientation increased the slope.
No modifications were observed for untrained orientations
Schoups, Vogels, Qian & Orban 2001
92
The future
You are the future… Students who will become
scientists that combine practice and theory
93
Theory is when you understand everything but nothing works
Practice is when everything works but no one knows why
In our lab we combine theory and practice:
Nothing Works and no one knows why…
94
“Spikernel” - Motivation
Assumptions: 1. A cortical population may display specific temporal
patterns that represent specific information. 2. The actual firing rate may vary with the same stimuli.
(inherent noise, stability, unmonitored behavior like changing context).
3. Similar patterns may also be distorted in time through non-linearly shifting.
4. Patterns that are associated with identical values of an external stimulus at time t may be similar at that time but different at t +