Encoding & Decoding of Neuronal Ensembles in Motor Cortex
Nicholas HatsopoulosDept. of Organismal Biology & Anatomy
Committees on Computational Neuroscience & NeurobiologyUniversity of Chicago
Co-founder & board member of Cyberkinetics, Inc.
Encoding Problem
trial 1trial 2trial 3trial 4trial 5
Decoding Problem
Behavior
Multi-trial averaging
Single-trial prediction
“The motor cortex appears to be par excellence a synthetic organ for motor acts… the motor cortex seems to possess, or to be in touch with, the small localized movements as separable units, and to supply great numbers of connecting processes between these, so as to associate them together in extremely varied combinations. The acquirement of skilled movements, though certainly a process involving far wider areas (cf. V. Monakow) of the cortex than the excitable zone itself, may be presumed to find in the motor cortex an organ whose synthetic properties are part of the physiological basis that renders that acquirement possible.”
Leyton & Sherrington (1917)
The two components of language
• Words or elementary primitives of meaning
• Rules or grammar by which the primitives are combined
Pinker (1999)
The language of motor action in motor cortex
• Motor primitives: position, velocity, direction, trajectory
• Motor grammar: addition
Center-Out Task
Directional Tuning
time (s)
0°
45°
90°
135°
180°
225°
270°
315°
20 40
frequency (Hz)150
0-0.5 1.00
Behavioral Apparatus
Random-walk task
B0 20 40 60 80 100 120
140
160
180
200
220
240
260
Utah/Bionic Technologies ProbeRichard Normann U Utah
Chronic Multi-electrode array
Primary Motor Cortex (MI)
LegLeg
FaceFace
ArmArm
5 mm5 mm
MIMI
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0 100 200 300 400 500 600 700 800 900 1000
DAYS POSTIMPLANT
AR
RA
Y Y
IEL
D
subject 1 (tom)subject 2 (coco)subject 3 (buddy)subject 4 (radley)
Long-term Reliability & Stability
Many Neurons Every Day(19 tests over 110 days)Blue - no recordingRed - best recordings
Why view motor cortical encoding astime-dependent?
• Trajectory-selective activity in motor cortex (Hocherman & Wise, 1990, 1991)
• Preferred directions shift in time (Mason et al., 1998; Sergio et al., 2005; Sergio & Kalaska, 1998)
-200 0 200
0 200 400 600
-200 0 200
Center-out taskShifts versus time
-200 0 200
0 200 400 600
-200 0 200
Center-out task Random-walk taskShifts versus time Shifts versus lead/lag time
movement onset
lag lead
Temporal tuning (information theory)
Lead/lag (s)
The Encoding Model
A class of general linear models (e.g. logistic regression) that estimates the probability of a spike given a particular movement trajectory:
γvke)v|tP(spike
)()...(),(),...1(),(),1(),...,( tvtvtvtvtvtvv yxxxxx
k
= preferred velocity trajectory
= preferred trajectory and path (“pathlet”)k
integrated
γycxbsav
vk
e)y,x,s,v|tP(spike
)(
X position (mm)
20 40 60 80 10 0 1 201 40
1 60
1 80
2 00
2 20
2 40
2 60
0
0
t = to
γycxbsav
vk
model input =
model input
pro
ba
bili
ty o
f a
sp
ike
lead/lag (ms)
-3.5 -3 -2.5
0.040.06
0.080.1
0.12
-4 -3 -2
0.05
0.1
0.15
0.2
-6 -4
0.02
0.04
0.06
-100 0 100200300
00.2
0.4
0.6
0.8
-100 0 100 200300-0.5
0
0.5
1
-100 0 100200 300
0
0.5
1
X po s itio n-2 0 2 4
-2
0
2
4
-10 -5 0
0
5
10
-5 0 5 10 15-15
-10
-5
0
5
Neuron 1 Neuron 3 Neuron 6 Neuron 8 Neuron 9 Neuron 15 Neuron 19 Neuron 20 Neuron 27
Neuron 29 Neuron 30 Neuron 31 Neuron 32 Neuron 36 Neuron 37 Neuron 38 Neuron 39 Neuron 40
Neuron 41 Neuron 45 Neuron 47 Neuron 48 Neuron 49 Neuron 50 Neuron 51 Neuron 52 Neuron 53
Temporal stability of pathlet representation
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False alarm probability
Hit
prob
abili
ty
red +/-50 msblue +/-150 msmagenta +/-300 mscyan +/- 500 ms
ROC analysis to quantify the performance of the encoding model
Lead/lag time (ms)-300 -200 -100 0 100 200 300
0.37
0.38
0.39
0.4
0.41
0.42
Encoding Performance as a function of trajectory length
-300 -200 -100 0 100 200 30035
40
45
50
55
60
65
Lead/lag time (ms)
Decoding Performance as a function of trajectory length
C audal
R ostral
X po s itio n
rs041130; MI
Map of Pathlets
0 0.5 1 1.5 2 2.5 3-0.05
0
0.05
0.1
0.15
Inter-electrode distance (mm)
Horizontal connectivity in motor cortex
1 mm Huntley & Jones (1991)
Rule for combining pathlets: Additive rule
Assuming conditional independence,
)|()|(& 2121 vspPvspP)v|spP(sp
22111212 vkvkγvk eee
2112 kkk
-4 -2 00
0.2
0.4
model input
spik
e p
rob
ab
ility neuron 1
0 5 10-10
-5
0
x-position
y-p
osi
tion
integrated k1
-3 -2 -1 00
0.2
0.4
neuron 2
-5 0 5
-10
-5
0
integrated k2
-6 -4 -20
0.05
0.1
0.15joint spiking model
-5 0 5 10 15-20
-10
0
integrated k12
-6 -4 -20
0.05
0.1
product of singleneuron models
-5 0 5 10 15-20
-10
0
integrated (k1+k
2)
r1031206; MI; 47&48:+/-5 ms
-8 -6 -4 -20
0.05
0.1
0.15neuron 1
model input
spik
e p
rob
ab
ility
-30 -20 -10 0-20
0
20
x-position
y-p
osi
tion
integrated k1
-3 -2 -1 00
0.2
0.4
neuron 2
-5 0 5
-10
-5
0
integrated k2
-10 -50
0.01
0.02joint spiking model
-30 -20 -10 0-30
-20
-10
0
10
integrated k12
-10 -50
0.01
0.02
product of singleneuron models
-30 -20 -10 0-30
-20
-10
0
10
integrated (k1+k
2)
r1031206; MI; 15&48;+/-5ms
-4 -3 -2 -10
0.1
0.2neuron 1
model input
spik
e p
rob
ab
ility
-5 0 5-5
0
5
x-position
y-p
osi
tion
integrated k1
-3 -2 -1 00
0.2
0.4
neuron 2
-5 0 5
-10
-5
0
integrated k2
-6 -5 -4 -30
0.02
0.04joint spiking model
-5 0 5-10
-5
0
integrated k12
-6 -5 -4 -30
0.02
0.04
product of singleneuron models
-5 0 5-10
-5
0
integrated (k1+k
2)
r1031206; MI; 2 vs. 48;+/-5ms
neuron 39 vs. neuron 51
-0.4 -0.2 0 0.2 0.4
20
40
60
80
100
neuron 40 vs. neuron 41
-0.4 -0.2 0 0.2 0.4
20
60
100
140
180
220
260
Cou
nts/
bin1 ms bin
Case #1 Case #2
Potential violations of conditional independence
Spike Jitter Method
neuron 1(reference)
+J-J
+/-w
2 parameters:
w, time resolution of synchrony J, the jitter window
neuron 2(target)
2 spikejitteredtrains
10% of all cell pairs (N=1431) show significant synchrony at a resolution of +/-5 ms, p<0.05
Case #1 Case #2 1000 jitters
Potential violations of conditional independenceCase #1
Case #2
-20 -10 00
0.02
0.04
0.06
-14 -12 -10 -8 -6 -4 -20
0.05
0.1
0.15
-1 2 -1 0 -8 -6 -4 -21 0
1 0
1 0
-1 4 -12 -1 0 -8 -6 -4 -21 0
1 0
1 0
21212121 )()(21 )&( vkkoffsetvkk egespspP
Conclusion: Synchronization preserves additive rule but increases the sensitivity of the tuning function by increasing the gain
When conditional independence appears to be violated
Computer
Neuro-motor prosthetic system
1) Multi-electrode array implant
2) Decoding of neural signals
3) Output interface
Personal computer:• mouse• keyboard
Assistive Robotics:• robotic arm• mechanized prosthetic arm
Biological interface:• muscles • peripheral nerve • spinal cord
Sensor
Cable
Cart
BrainGateTM Pilot Device
BrainGateTM Sensor Implantation and Post-Op Recovery as Planned
• Surgery as planned
• Post-op recovery unremarkable
• Wound healing around pedestal complete
Arrayon Cortex
Insertion
2 months post implant
Binary Modulation-Imagined Opening/Closing of Hand
BrainGate Signal Detection and Analysis;SCI Able to Modulate Neural Output
XRRRf T1T
Warland et al. (1997)
2. Algorithmic level:Optimal Linear Filter Reconstruction
RfX(t)ˆ
Estimated position of hand in time
Response of neural ensemble in time
filter coefficients
Two Dimensional Cursor control
Hatsopoulos LabQingqing XuWei Wu, PhD
Sunday Francis Zach HagaJignesh JoshiJohn O’LearyDawn PaulsenJake ReimerJonathan KoJoana Pellerano
Richard Penn, MDMatt FellowsYali AmitCyberkinetics, Inc.
Cross-Validated Performance as a function of pathlet length
Cross-validated Pathlet Population Vector Decoding
Top Related