1 Learning induced improvement of internal representations in motor cortex Eilon Vaadia.

1

Learning induced improvement of internal representations in motor

cortex

Eilon Vaadia

2

Outline

Introduction: Voluntary movement and its Neural representation

Neuronal correlates of learning in the monkey brain.

Epilog: Implication for future applications


3

Action is Essential for Sensory Perception

4

Visually Guided movements

Sensory perception and motor action are embedded in one tight sensorimotor loop

During visually reaching movement, visual representation of the target location must be transformed into appropriate movement coordinates

5

Forward Dynamic model

The sensorimotor loop

Forward sensory model

Inverse model

Previous state

New State

Neuronal representations

context

context

Wolpert and Ghahramani 2000 (review)

context

Desired Kinematic

s

Motor command

6

Internal model in a simplistic definition:

The neuronal process that computes a desired action given the sensory

inputs and their context

7

The ingenuity of our brain

Ability to adapt, to a wide variety of perturbations

Predict the effect of the motor output on the inputs (von Helmholtz, 1867)

How does the brain perform these tasks?

8

Testing properties of internal modelsExample: Psychophysics of reaching movements

“standard mapping”

9

Standard Mapping

10

Standard Mapping

11

Standard Mapping

12

Standard Mapping

13

Standard Mapping

14

Standard Mapping

15

Standard Mapping

16

Reaching with Visuomotor rotation

17


18


19


20


21


22


23


24

At the end of learning

25

Return to default; the aftereffect tells us that an internal model was formed

26


27


28


29


30

Return to default


31

Testing for non-learned directions:Is the aftereffect local ?

32


33


34


35


36

YES!Learning is Local: No Generalization


37

Human and monkeys learn to perform this task rapidly (few trials up to 100).

38

Humans

Krakauer et al J. Neuroscience, 20(23):8916–8924 2000 (Ghez lab.)

Eilon Vaadia

h

39

What is the neural basis of such local learning?

Localized internal model suggests:The internal model uses neuronal elements with localized spatial fields

Interestingly.…

40

The motor cortex

MI

41

Directional Tuning of MI Cells

Movement onset

(Georgopoulos et al 1982)

P.D.

42

Cosine tuning

PD

Direction of movement

Spi

kes/

sec

43

Population of neurons in MI Accurately Represent the Direction of Movements

44

The population of cells in MI tell the external observes quite accurately what is the movement direction

Each of the cells in this population may serve as the “local elements” we need…

End of part 1

45

Part II


Neuronal correlates of learning in the monkey brain*

Implication for future applications

* R. Paz, C. Nathan, T. Boraud, H. Bergman and E. Vaadia, Nature Neurosci. 2003

46

Objectives

• Teach the animal (the brain) to generate a new mapping between visual instruction and motor output

• Search for a for the neuronal representation of localized internal models

47

Monkeys and video games…

48

The behavioral task

Default (pre) (8-target task)

“Preparatory” “Movement”

Go!

CueStart trial Movement

Default (post)(8-target task)

Transformation (one-target task)

“learned direction”

CursorHand

49

Direction

norm

aliz

ed d

evia

tion

Pre-learning

Post-learning:

12-34-56-7

0.4

0

0.2

315 0 45 90 135 180 225 270

1. The Behavioral Aftereffect is local

Results

50

2. Single neurons show dynamic adaptation during learning

Tri

al n

umbe

r

Target onset MVT onset

Time Timecounts counts0 750 15 0 750 15

Preparatory Movement

Note: The adaptation of rate is evident only during the preparatory period

51

3. Firing-rates and behaviors improvement

show similar dynamics

Trial number

Nor

mal

ized

rat

e / N

orm

aliz

ed e

rror

Preparatory Movement-related

Actual ActivityExpectedBehavior

52

4. Directional tuning of some neurons is modified

(Post vs. Pre)

PD

Distance from learned direction0 180-180

30S

pik

es/

sec

Pre learning:Post learning:

53

5. Population Tuning: Only cells with PD near the learned direction show increased activity

Pos

t-P

re d

iffe

renc

e

Distance of Preferred direction from learned direction

Learned

±30

54

6. Learning-induced enhancement: Almost all Cells with PD near the learned direction!

55

OK… So what do we have till now?

1. The behavioral effect is local (aftereffect is local)2. The effect on neuronal activity is local

a. Firing rate increases only near the learned directionb. Only cells with PD near the learned direction show

the effect

3. The effect occurs during movement preparation4. The effect persists in post learning, even after

return to default movements.

56

How do these changes help the brain?

57

-90 -45 0 45 90

Err

or (

Deg

)35

20

10

The PV error is reduced

Learned direction

58

The Signal to Noise Increases

Non-learned directions

SN

R: M

ean/

S.D

impr

ovem

ent

Learned directionRepeated

*

59

Conclusion

After learning: The brain (and external observers) can better predict the direction of the learned movement.

60

How was the representation improved?

61

Mutual information (rate and direction)

r

d r

drPdrPdPrPrPI )|(log)|()()(log)( 22

i

iid ddrdpN

d i )]())(log([))((log(1

maxargˆ

Cover and Thomas 1991

Rolls, Treves et al 1997

1. All directions

2. A specific direction

3. Predicting a direction

I – Mutual information r =rate d= direction

σi - the mean firing rate of cell i in direction d ri - the firing rate in randomly drawn trial Sanger, 1996

r rP

drPdrPdI

)(

)|(log)|()( 2

62

Change in Mutual Information after Learning

PDs distribution Cells with increased information (p>0.95)

All cells

Information

p-value

Num

ber

of

cells

Mutual information (bits)

63

The improvement is associated with…

Δ- Fano-factor

Δ - slope

Δ-I

nfor

mat

ion

Δ-I

nfor

mat

ion2. Changed Variability?

1. Changed Slope?

64

Tuning slope is locally increased

Δ (

post

-pre

) sl

ope

Distance from learned direction

Cells with significant increased information

Other cells

Learned

65

Three ways to affect the slope

Peak-rate

Learned

1 .PD shift Narrowing

Learned

Change of PD

-In

form

atio

n C=0.05

Change of Width(at half height)

C=0.18

Change of rate (at learned direction)

C=0.56

Learned

66

Summary of Results

Learning induces highly specific and subtle changes in single neurons’ properties.

The neuronal changes are maintained after learning

Learning-induced changes specifically improve the neuronal representation of the learned movement

67

Conclusions (speculations…)

The motor motor cortex learns Local learning may be useful – The

representation of the learned direction is improved without affecting representations of other directions.

The internal model is maintained without interfering with default behavior.

End of part 2

68

Epilog


Neuronal correlates of learning in the monkey brain

Implication for future applications

69

Towards Neural prosthesis

Miguel Nicolelis 2001

70

Improving movement reconstruction

Learning helps in shaping brain activity. Thus, A smaller number of cells may be used more efficiently to predict movements

Future advances Improve data (Schwartz et al)

• Appropriate training• Appropriate recordings

Add signals (LFPs) (Aertsen et al) Improve algorithms (Shpigelman et al)

71

Linear Regression

)()()()( tututMu

Xab

The end point M at time t where X is the matrix

of units activity and a is a set of impulse response functions (weights).

72

“Spikernel” - Motivation

a “spike trains” kernel that maps similar activity patterns to nearby areas of the feature space.

)()()( tutMu

ab )( ut X )]([ utX

ttk

btktM

ii

iiii

xxxx

xx

,

,VectorSupport

*

73

“Spikernel” Performance

“standard” Kernels

Spi

kern

el

74

Reconstruction of movement Velocity(open loop)

(30 units)

75

Towards Neural prosthesis

It’s a long way to go… …But it seems the landscapes on

the way are at least as exciting as the destination

76

With Contributions by:Hagai Bergman, ICNCNaftali Tishby ICNCYoram Singer CS

Gal Chechick ICNCAmir Globerson ICNC

Acknowledgments

Rony Paz (Learning Experiments)

Chen Nathan (Learning Experiments)Thomas Boraud (Learning Experiments)

Lavi Shpigelman (spikernel)

78

Possible ways to improve the representation

Distance (degrees) from learned direction

79

The Increased information is best correlated with local enhancement of firing rate

C=0.56P=0.183

C=0.189P=0.276

C=0.057P=0.419

81

Conclusions

1. The firing rate of some cells is enhanced during learning.

2. A population of selected cells – show a similar enhancement

3. The enhancement takes similar time course as the learning.

4. The enhancement occurs only during movement preparation.

Selected cells? Who are these

cells?

selected

85A.B. Schwartz, Pittsburg, USA

88

Monkeys Visual cortex

Improved long term neuronal performance resulted from changes in the slope of orientation-tuning of individual neurons.

The slope increased only for the subgroup of cells (with PO near the trained orientation)

No modifications of the tuning curve were observed for untrained orientations

Schoups, Vogels, Qian & Orban 2001

89

5. Population Tuning: Only cells with PD near the learned direction show increased activity

Distance of Preferred direction from learned direction

Avg

. nor

mal

ized

rat

es

Pre-learningPost-learning

90

“Local elements Exist !”

-42 -17 0 +16 +37

P.O.Cell 1

Cell 3Cell 2 Cell 4 Cell 5


91

Epilog 1

Similar cortical basis for learning in visual and motor function?

Monkeys Visual cortex Improved long term neuronal performance resulted from

changes in the slope of orientation-tuning of individual neurons.

only subgroup of cells with PO close to the trained orientation increased the slope.

No modifications were observed for untrained orientations


92

The future

You are the future… Students who will become

scientists that combine practice and theory

93

Theory is when you understand everything but nothing works

Practice is when everything works but no one knows why

In our lab we combine theory and practice:

Nothing Works and no one knows why…

94

“Spikernel” - Motivation

Assumptions: 1. A cortical population may display specific temporal

patterns that represent specific information. 2. The actual firing rate may vary with the same stimuli.

(inherent noise, stability, unmonitored behavior like changing context).

3. Similar patterns may also be distorted in time through non-linearly shifting.

4. Patterns that are associated with identical values of an external stimulus at time t may be similar at that time but different at t +

1 Learning induced improvement of internal representations in motor cortex Eilon Vaadia.

Documents

Transcript of 1 Learning induced improvement of internal representations in motor cortex Eilon Vaadia.