Learning and Remapping in Learning and Remapping in the Sensory Motor Systemthe Sensory Motor System

F.A. Mussa-Ivaldi Northwestern University

In natural arm movements, complex In natural arm movements, complex motions of the joints lead to simple motions of the joints lead to simple hand kinematicshand kinematics

From Morasso 1981

Is the shape of trajectories the byproduct of dynamic optimization?

(Uno, Kawato and Suzuky, 1989)

Kinematic AdaptationKinematic Adaptation

Flanagan and Rao (1995)

Dynamic AdaptationDynamic Adaptation

Unperturbed movements

Shadmehr & Mussa-Ivaldi, 1994

Perturbing fieldPerturbing field

rF

1.112.11

2.111.10

-1.5 -1 -0.5 0 0.5 1 1.5

-1

-0.5

0

0.5

1

Note: there is no force at rest 0)r(

AdaptationAdaptation

After-effects

Transport of a spring-mass Transport of a spring-mass objectobject

Dingwell, Mah and Mussa-Ivaldi (2004)

Post-adaptation kinematics

Baseline

Initial response

The shape of trajectories is not a by-product of dynamic optimization.

It reflects and is consistent with the Euclidean properties of ordinary space

What is space ?What is space ? Intuitive Concept

Geometry, from a physical standpoint is the totality of the laws according to which rigid bodies mutually at rest can be placed with respect to each other… “Space” in this interpretation is in principle an infinite rigid body – or skeleton - to which the position of all other bodies is related.”

A. Einstein, 1949 Less Intuitive

ConceptsSignals spaces, configuration spaces, state spaces, feature spaces, etc...

Ordinary SpaceOrdinary Space

Euclid

Pythagoras

Two equivalent statements

?180o

x

yd 222 yxd

Euclidean SymmetryEuclidean SymmetryAmong all possible norm only the L2 (Euclidean) norm is invariant under rigid transformations (e.g. rotations, translations).

Size does not depend upon position and orientation.

xxx T 2Euclidean (L2) Norm

1AATOrthogonal transformation

Axy

21

2

xxxAxAx

AxAxyyy

TT

TTT

Signal spacesSignal spaces

Neither the visual nor the motor signals spaces are Euclidean.

Yet, we perceive Euclidean symmetry and we coordinate motor commands to control independently movement amplitude and direction

•It is a common misconception that the visual system provides the primal information about this symmetry.

•In fact it is likely for the opposite to be true: we learn the properties of space through active perception

Is the bias toward Euclidean symmetry a guiding principle for the emergence of new motor programs?

The motor system can reassign The motor system can reassign degrees of freedom to novel tasksdegrees of freedom to novel tasks

Glove Talk. Sidney Fels and Geoffrey Hinton, 1990

Vocabulary Sam-I-am

The properties of space are captured by the motor system when forming novel motor programs whose purpose is to control the motion of an endpoint

19 SIGNALS (Hand Configuration)

2 Screen Coordinates

F.A. Mussa-Ivaldi, S. Acosta, R. Scheidt, K.M. Mosier

Remapping Remapping SpaceSpace

The taskThe task The subject must place the cursor

corresponding to the hand inside a circular target

Experiment begins with the cursor inside a target.

New target is presented – Cursor is suppressed.

The subject must try to make a single rapid movement of the fingers so as to place the (invisible) cursor inside the new target and stop.

As the hand is at rest after this initial movement, the cursor reappears

The subject corrects by moving the cursor under visual guidance inside the target

Only the initial open loop movement is analyzed

-3 -2 -1 0 1 2 3-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

-3 -2 -1 0 1 2 3

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

-1 0 1 2 30

10

20

30

40

50

t (sec)

v (

cm

/se

c)

-1 0 1 2 30

10

20

30

40

50

Part 1 Part 2A B

C D

-3 -2 -1 0 1 2 3

-3

-2

-1

0

1

2

-3 -2 -1 0 1 2 3-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

-3 -2 -1 0 1 2 3

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

-3 -2 -1 0 1 2 3

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

-3 -2 -1 0 1 2 3

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

-3 -2 -1 0 1 2 3

-2

-1

0

1

2

3

Some examplesSome examples

Weird task. Cannot be represented before the experiment

Body and task endpoint are physically uncoupled

Only feedback pathway is vision Dimensional imbalance Metric imbalance

Some observationsSome observations

1 4 7 101.4

1.6

1.8

2

2.2

2.4

2.6

2.8

3 Ensemble

1 4 7 100.5

1

1.5

2

2.5

3

3.5

Trial number

Err

or

(cm

)

Single Subject

Endpoint errorEndpoint error

LinearityLinearity

S

E

A

B

Aspect-Ratio: AB/SE

1 4 7 100.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Trial number

Asp

ect

Ratio

1 4 7 100.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65 EnsembleSingle Subject

•Cursor movements DO NOT become straighter !

Extended training Extended training 4 consecutive days 2 groups of subjects No-Vision subjects (P2) just engage in the

task for 4 days Vision subjects (P3) train with continuous

visual feedback trials. But they are tested in the same no-vision condition as the other group.

The two groups receive the same amount of training

Extended Training ResultsExtended Training Results Both group have the same learning trend. Continuous visual feedback of endpoint movements

facilitates the formation of straighter open-loop trajectories

1 2 3 41

1.5

2

2.5

Day

Err

or (

cm)

P2 (No Vision)P3 (Vision)

1 2 3 4

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Day

Asp

ect R

atio

•Error reduction and straightening of motion are independent processes

•Straightening of motion is facilitated when vision and movement execution are concurrent

The subjects learn to solve an ill-posed inverse problem (from cursor location to hand gesture) and in doing so, they learn the geometrical structure of the map, i.e. a specific partition of degrees of freedom.

Decomposition of Glove Decomposition of Glove SignalsSignals

CFBCBF

PQA

BF

)()(

)(

)( 1

ATIAN

AAAT

AAAA

n

TT

0

Component" Space-Task")(

Component" Space-Null")(

TNT

TN

T

N

hh

hhh

hATh

hANh

1 2 3 4140

160

180

200

220

240

Day

Nu

ll M

otio

n (

G.S

.U.)

1 2 3 460

70

80

90

100

110

120

Day

Ta

sk M

otio

n (

G.S

.U.)

P2 (No Vision)P3 (Vision)

Task-space and Null-space Task-space and Null-space MotionMotion

↓19%↓24%

↓20% ↓23%

Importing a metric Importing a metric structurestructure

•Identify Residual Control Signals (Movements, EEG, EMG, Multiunit…)

•Form a first “vocabulary” of commands (faster, slower, right, left, MIDI, etc…)

•Machine learns to respond appropriately

•Patient practice and leans new commands

Practical Applications:Practical Applications:Body-Machine Interfaces Body-Machine Interfaces (BMI)(BMI)

Interface

DUAL LEARNING

Sensing garments (Smartex -Pisa)

•52 smeared sensors on lycra

•No metallic wires

•External connection realized by metallic press stud

Calibration: Find optimally Calibration: Find optimally controlled DOFs (synergies) controlled DOFs (synergies) by dimensionality reductionby dimensionality reduction

Natural Synergies -> Control Natural Synergies -> Control ParametersParameters

A bad map A better map

Tonal SpacesTonal Spaces

The interaction of human and machine learning: Co-Adaptation by Least Mean Squares

The interactive LMS procedure - The interactive LMS procedure - 1 1

1) Set an arbitrary starting map:

hAx

Endpoint vector dim(x) =2 “Body” vector dim(h)=20

The interactive LMS procedure - The interactive LMS procedure - 2 2

2) Collect movements toward a set of K targets:

Kxxx ˆ,,ˆ,ˆ 21

Kxxx ,,, 21

Khhh ,,, 21

Targets

Movements (endpoint coordinates)

Movements (body coordinates)

The interactive LMS procedure - The interactive LMS procedure - 3 3

3) Calculate the error (in endpoint coordinates)

iii xAxe ˆ)(

K

i

iTi eeE1

2 )(

For each movement

For a whole test epoch

The interactive LMS procedure - The interactive LMS procedure - 4 4

4) Calculate the gradient of the error (w.r.t. A)

k

i

Tii heA

E

1

2

)(2

5) Update the A-matrix in the direction of the gradient

A

EAA

2

4) Repeat from step 2 until convergence.

An ExampleAn Example

1 2 3 4 5 6 70

0.5

1

1.5

Training Epoch #

Nor

mal

ized

End

poin

t E

rror

Per

Mov

emen

t

Updated mappings do not occuruntil after the dashed linebetween epochs 5 and 6

ConclusionConclusion

Sensory-Motor Cognition IS NOT

an oxymoron !