Neuromorphic Computing and Learning:A Stochastic Signal Processing Perspective
Osvaldo SimeoneJoint work with Hyeryung Jang (KCL)
King’s College London
Osvaldo Simeone Neuromorphic Computing 1 / 74
Overview
Motivation and Introduction
Applications
Models
Learning Algorithms
Examples
Osvaldo Simeone Neuromorphic Computing 2 / 74
Overview
Motivation and Introduction
Applications
Models
Learning Algorithms
Examples
Osvaldo Simeone Neuromorphic Computing 3 / 74
Machine Learning Today
[Rajendran [íô�
Breakthroughs in ML using (deep) Artificial Neural Networks (ANNs)have come at the expense of massive memory, energy, and timerequirements...
Osvaldo Simeone Neuromorphic Computing 4 / 74
Resource-Constrained Machine LearningHow to implement ML (inference and learning) on mobile orembedded devices with limited energy and memory resources?[Welling ’18]
mobile personal assistants medical and health wearables
IoT mobile or embedded devices neural prosthetics
Osvaldo Simeone Neuromorphic Computing 5 / 74
Machine Learning at the EdgeA solution is mobile edge or cloud computing: offload computationsto an edge or cloud server.
Possible privacy and latency issues
Osvaldo Simeone Neuromorphic Computing 6 / 74
Machine Learning at the EdgeA solution is mobile edge or cloud computing: offload computationsto an edge or cloud server.
Possible privacy and latency issues
Osvaldo Simeone Neuromorphic Computing 6 / 74
Machine Learning on Mobile DevicesAnother solution is to scale down energy and memory requirements ofANNs via tailored hardware implementations for mobile devices.Active field with established players and start-upsTrade-offs between accuracy and complexityMostly limited to inference
Osvaldo Simeone Neuromorphic Computing 7 / 74
Machine Learning on Mobile DevicesAnother solution is to scale down energy and memory requirements ofANNs via tailored hardware implementations for mobile devices.Active field with established players and start-upsTrade-offs between accuracy and complexityMostly limited to inference
Osvaldo Simeone Neuromorphic Computing 7 / 74
Neuromorphic ComputingBeyond ANN...
13 Million Watts5600 sq. ft. & 340 tons
∼ 1010 ops/J
∼ 20 Watts2 sq. ft. & 1.4 Kg∼ 1015 ops/J
Source: https://www.olcf.ornl.gov, Google Images
Osvaldo Simeone Neuromorphic Computing 8 / 74
Neuromorphic ComputingNeurons in the brain sense, process, and communicate over time usingsparse binary signals (spikes or action potentials).This results in a dynamic, sparse, and event-driven learning andinference.Spiking signals minimize energy per bit [Verdu ’11].
[Gerstner]
Osvaldo Simeone Neuromorphic Computing 9 / 74
Neuromorphic ComputingNeurons in the brain sense, process, and communicate over time usingsparse binary signals (spikes or action potentials).This results in a dynamic, sparse, and event-driven learning andinference.Spiking signals minimize energy per bit [Verdu ’11].
[Gerstner]
Osvaldo Simeone Neuromorphic Computing 9 / 74
Spiking Neural Networks
Spiking Neural Networks (SNNs) are networks of spiking neurons[Maas ’97].
Unlike ANNs, in SNNs neurons operate sequentially over time byprocessing and communicating via spikes in an event-driven fashion.
x2
x1
xn
y
Artificial Neural Network
(ANN)
Spiking Neural Network (SNN)
Time
Time
w1
wn
w2
...
w1
wn
w2
...
x2(t)
x1(t)
xn(t)
y(t)
Osvaldo Simeone Neuromorphic Computing 10 / 74
Spiking Neural Networks
Topic at the intersection of computational neuroscience (focused onbiological plausibility) [Gerstner and Kistler ’02] [Pillow et al ’08],machine learning, and hardware design (focused on accuracy andefficiency) [Rezende et al ’11] [Jang et al ’18] [Rajendran et al ’19].
Osvaldo Simeone Neuromorphic Computing 11 / 74
Spiking Neural NetworksProof-of-concept and commercial hardware implementations of SNNshave demonstrated significant energy savings as compared to ANNs[Rajendran et al ’19].
Osvaldo Simeone Neuromorphic Computing 12 / 74
Spiking Neural Networks
Increasing press coverage and positive market predictions...
Osvaldo Simeone Neuromorphic Computing 13 / 74
Overview
Motivation and Introduction
Applications
Models
Learning Algorithms
Examples
Osvaldo Simeone Neuromorphic Computing 14 / 74
I/O Interfaces
SNN
Osvaldo Simeone Neuromorphic Computing 15 / 74
I/O Interfaces
SNNNeuromorphic
sensor
Neuromorphic
actuator
Osvaldo Simeone Neuromorphic Computing 16 / 74
I/O Interfaces
iniVation’s Dynamic Vision Sensor (DVS), iniVation and AiCTX’sSpeck, Prophesee
[Prophesee]
Osvaldo Simeone Neuromorphic Computing 17 / 74
I/O Interfaces
From IBM’s DVS128 Gesture Dataset...
Osvaldo Simeone Neuromorphic Computing 18 / 74
I/O Interfaces
SNNencoder decoder
source actuator
5
Osvaldo Simeone Neuromorphic Computing 19 / 74
I/O InterfacesConversion from natural signals to spike signals – theory [Lazar ’06]Practice: Rate encoding, time encoding, population encoding; ratedecoding, first-to-spike decoding,...
Osvaldo Simeone Neuromorphic Computing 20 / 74
Applications
SNNs can performI Inference/ control: classification, regression, prediction, ...
SNN?
(a) inference/ control
Osvaldo Simeone Neuromorphic Computing 21 / 74
Applications
SNNs can performI Inference/ control: classification, regression, prediction, ...I Learning: supervised, unsupervised, reinforced
SNN?
(a) inference/ control
SNN
(b) learning
?
feedback/ reward
Osvaldo Simeone Neuromorphic Computing 22 / 74
Applications
SNNs can performI Inference/ control: classification, regression, prediction, ...I Learning: supervised, unsupervised, reinforced
SNN?
(a) inference/ control
SNN
(b) learning
?
feedback/ reward
Data presentation through I/O interfaces:I Frame-based (or batch)
Osvaldo Simeone Neuromorphic Computing 23 / 74
ApplicationsSNNs can perform
I Inference/ control: classification, regression, prediction, ...I Learning: supervised, unsupervised, reinforced
SNN?
(a) inference/ control
SNN
(b) learning
?
feedback/ reward
Data presentation through I/O interfaces:I Frame-based (or batch)I Streaming (or online)
Osvaldo Simeone Neuromorphic Computing 24 / 74
Overview
Motivation and Introduction
Applications
Models
Learning Algorithms
Examples
Osvaldo Simeone Neuromorphic Computing 25 / 74
SNN Models
SNNs are networks of spiking neurons.
Their operation is defined by:I topology (connectivity)I neuron model
x2
x1
xn
y
Artificial Neural Network
(ANN)
Spiking Neural Network (SNN)
Time
Time
w1
wn
w2
...
w1
wn
w2
...
x2(t)
x1(t)
xn(t)
y(t)
Osvaldo Simeone Neuromorphic Computing 26 / 74
Topology
Arbitrary directed graph with directed links representing synapticconnections
“Parent”, or pre-synaptic, neuron affects causally spiking behavior of“child”, or post-synaptic, neuron
Enables recurrent connectivity (directed loops), including self-loops
Osvaldo Simeone Neuromorphic Computing 27 / 74
Topology
Arbitrary directed graph with directed links representing synapticconnections
“Parent”, or pre-synaptic, neuron affects causally spiking behavior of“child”, or post-synaptic, neuron
Enables recurrent connectivity (directed loops), including self-loops
Osvaldo Simeone Neuromorphic Computing 27 / 74
Topology
Discrete (algorithmic) time, as in many practical implementations(e.g., Intel’s Loihi) –
Osvaldo Simeone Neuromorphic Computing 28 / 74
Neuron Model
Each neuron is characterized by an internal state known as membranepotential [Gerstner and Kistler ’02].
Generally, a higher membrane potentially entails a larger propensityfor spiking.
The membrane potential evolves over time as a function of the pastbehavior of pre-synaptic neurons and of the neuron itself.
Osvaldo Simeone Neuromorphic Computing 29 / 74
Neuron Model
Each neuron is characterized by an internal state known as membranepotential [Gerstner and Kistler ’02].
Generally, a higher membrane potentially entails a larger propensityfor spiking.
The membrane potential evolves over time as a function of the pastbehavior of pre-synaptic neurons and of the neuron itself.
Osvaldo Simeone Neuromorphic Computing 29 / 74
Membrane Potential
ui ,t =∑j∈Pi
wj ,i
(at ∗ sj ,t
)+ wi
(bt ∗ si ,t
)+ γi
Osvaldo Simeone Neuromorphic Computing 30 / 74
Membrane Potential
ui ,t =∑j∈Pi
wj ,i
(at ∗ sj ,t
)+ wi
(bt ∗ si ,t
)+ γi
Feedforward filter (kernel) at with learnable synaptic weight wj ,i
Osvaldo Simeone Neuromorphic Computing 31 / 74
Membrane Potential
ui ,t =∑j∈Pi
wj ,i
(at ∗ sj ,t
)+ wi
(bt ∗ si ,t
)+ γi
Feedback filter (kernel) bt with learnable wi (e.g., refractory period)Bias (threshold) γi
Osvaldo Simeone Neuromorphic Computing 32 / 74
Membrane Potential
Kernels can more generally be parameterized via multiple basisfunctions and learnable weights [Pillow et al ’08].
This allows learning of the temporal “receptive fields” of the neurons,e.g., by adapting synaptic delays.
Osvaldo Simeone Neuromorphic Computing 33 / 74
Deterministic SNN Models
The most common model is leaky integrate-and-fire (LIF) [Gerstnerand Kistler ’02]: Spike when membrane potential is positive
0
0.5
1
1.5s pr
e(t
)
0 10 20 30 40 50 60 70 80 90 100t
0 10 20 30 40 50 60 70 80 90 100t
0
1
2
upo
st(t
)-po
st
0
0.5
1
s post
(t)
0 10 20 30 40 50 60 70 80 90 100t
Osvaldo Simeone Neuromorphic Computing 34 / 74
Deterministic SNN Models
LIF-based SNNs can approximate operation of feedforward andrecurrent ANNs: spiking rates in SNN ≈ neuron outputs in ANN
Often used for inference by converting a pre-trained ANN [Rueckaueret al ’17]
Direct training is required to leverage temporal information processingand learning
This is made difficult by output non-differentiability with respect tomodel parameters
Heuristic training algorithms based on approximations, such assurrogate gradient [Neftci ’18] [Anwani and Rajendran ’18]
As for ANNs, these require backpropagation
Osvaldo Simeone Neuromorphic Computing 35 / 74
Deterministic SNN Models
LIF-based SNNs can approximate operation of feedforward andrecurrent ANNs: spiking rates in SNN ≈ neuron outputs in ANN
Often used for inference by converting a pre-trained ANN [Rueckaueret al ’17]
Direct training is required to leverage temporal information processingand learning
This is made difficult by output non-differentiability with respect tomodel parameters
Heuristic training algorithms based on approximations, such assurrogate gradient [Neftci ’18] [Anwani and Rajendran ’18]
As for ANNs, these require backpropagation
Osvaldo Simeone Neuromorphic Computing 35 / 74
Deterministic SNN Models
LIF-based SNNs can approximate operation of feedforward andrecurrent ANNs: spiking rates in SNN ≈ neuron outputs in ANN
Often used for inference by converting a pre-trained ANN [Rueckaueret al ’17]
Direct training is required to leverage temporal information processingand learning
This is made difficult by output non-differentiability with respect tomodel parameters
Heuristic training algorithms based on approximations, such assurrogate gradient [Neftci ’18] [Anwani and Rajendran ’18]
As for ANNs, these require backpropagation
Osvaldo Simeone Neuromorphic Computing 35 / 74
Probabilistic SNN Models
Probabilistic Generalized Linear Model (GLM): Conditional spikingprobability [Pillow et al ’08]
p(si ,t = 1|s≤t−1, s≤t−1) = σ(ui ,t)
𝜎(𝑥)
𝑥
Osvaldo Simeone Neuromorphic Computing 36 / 74
Probabilistic SNN Models
Probabilistic Generalized Linear Model (GLM): Conditional spikingprobability [Pillow et al ’08]
p(si ,t = 1|s≤t−1, s≤t−1) = σ(ui ,t)
0
0.5
1
1.5
s pre(t
)
0 10 20 30 40 50 60 70 80 90 100t
0 10 20 30 40 50 60 70 80 90 100t
-1
0
1
2
upo
st(t
)-po
st
0
0.5
1
s post
(t)
0 10 20 30 40 50 60 70 80 90 100t
Osvaldo Simeone Neuromorphic Computing 37 / 74
Probabilistic SNN Models
GLM SNN models are dynamic generalization of belief networks [Neal’92].
They enable direct training of spike-based statistical criteria.
Osvaldo Simeone Neuromorphic Computing 38 / 74
Overview
Motivation and Introduction
Applications
Models
Learning Algorithms
Examples
Osvaldo Simeone Neuromorphic Computing 39 / 74
Training SNNs
SNNs can be trained using supervised, unsupervised, andreinforcement learning.
To fix the ideas, we focus here on supervised learning.
Supervised learning:
training data = {(input, output)} → generalization
Batch or online
Osvaldo Simeone Neuromorphic Computing 40 / 74
Training SNNs
SNNs can be trained using supervised, unsupervised, andreinforcement learning.
To fix the ideas, we focus here on supervised learning.
Supervised learning:
training data = {(input, output)} → generalization
Batch or online
Osvaldo Simeone Neuromorphic Computing 40 / 74
Training SNNs
SNNs can be trained using supervised, unsupervised, andreinforcement learning.
To fix the ideas, we focus here on supervised learning.
Supervised learning:
training data = {(input, output)} → generalization
Batch or online
Osvaldo Simeone Neuromorphic Computing 40 / 74
Training SNNs
Input neurons, clamped to input data
Output neurons, clamped to output data
Latent neurons
SNN
input outputlatent
input dataoutput data
Osvaldo Simeone Neuromorphic Computing 41 / 74
Learning Rules
Probabilistic SNNs can be trained using standard statistical criteria,such as maximum likelihood, maximum mutual information, ormaximum average return.
Maximum likelihood for supervised learning:
max Edata[ln p(output | input)]
Regularization terms are typically added, e.g., to limit spiking rate(and hence energy consumption) – bounded rationality [Leibfried andBraun ’15]
Osvaldo Simeone Neuromorphic Computing 42 / 74
Learning Rules
Probabilistic SNNs can be trained using standard statistical criteria,such as maximum likelihood, maximum mutual information, ormaximum average return.
Maximum likelihood for supervised learning:
max Edata[ln p(output | input)]
Regularization terms are typically added, e.g., to limit spiking rate(and hence energy consumption) – bounded rationality [Leibfried andBraun ’15]
Osvaldo Simeone Neuromorphic Computing 42 / 74
Learning Rules
Stochastic Gradient Descent (SGD) updates for synaptic weights:
wj ,i ← wj ,i + η × `× pre-synj × post-syni
The learning signal ` is a global feedback signal (akin toneuromodulator in neuroscience).
Pre-synaptic and post-synaptic terms are local to each synapse.
Osvaldo Simeone Neuromorphic Computing 43 / 74
Learning Rules
Stochastic Gradient Descent (SGD) updates for synaptic weights:
wj ,i ← wj ,i + η × `× pre-synj × post-syni
The learning signal ` is a global feedback signal (akin toneuromodulator in neuroscience).
Pre-synaptic and post-synaptic terms are local to each synapse.
Osvaldo Simeone Neuromorphic Computing 43 / 74
Learning Rules
Stochastic Gradient Descent (SGD) updates for synaptic weights:
wj ,i ← wj ,i + η × `× pre-synj × post-syni
The learning signal ` is a global feedback signal (akin toneuromodulator in neuroscience).
Pre-synaptic and post-synaptic terms are local to each synapse.
Osvaldo Simeone Neuromorphic Computing 43 / 74
Learning Rules
Stochastic Gradient Descent (SGD) updates for synaptic weights:
wj ,i ← wj ,i + η × `× pre-synj × post-syni
The learning signal ` is derived using variational learning [Rezende etal ’11] [Osogami ’17] [Jang et al ’18]:
`t = 1 for observed (input and output) neurons,
`t =∑
i∈outputlog p(si ,t |ui ,t) for latent neurons
Positive feedback to hidden neurons if desired behavior has largeprobability
Osvaldo Simeone Neuromorphic Computing 44 / 74
Learning Rules
Stochastic Gradient Descent (SGD) updates for synaptic weights:
wj ,i ← wj ,i + η × `× pre-synj × post-syni
pre-synj = at ∗ sj ,t = pre-synaptic trace
Large if previous behavior of pre-synaptic neuron is consistent withsynaptic receptive field (e.g., if recent spiking)
Osvaldo Simeone Neuromorphic Computing 45 / 74
Learning Rules
Stochastic Gradient Descent (SGD) updates for synaptic weights:
wj ,i ← wj ,i + η × `× pre-synj × post-syni
post-syni = si ,t − σ(ui ,t)
= post-synaptic error
Post-synaptic error = desired/ observed behavior - model averagedbehavior [Bienenstock et al ’82]
No backpropagation
Osvaldo Simeone Neuromorphic Computing 46 / 74
Learning Rules
Stochastic Gradient Descent (SGD) updates for synaptic weights:
wj ,i ← wj ,i + η × `× pre-synj × post-syni
post-syni = si ,t − σ(ui ,t)
= post-synaptic error
Post-synaptic error = desired/ observed behavior - model averagedbehavior [Bienenstock et al ’82]
No backpropagation
Osvaldo Simeone Neuromorphic Computing 46 / 74
Learning Rules
The learning rules simplifies to Spike-Timing-Dependent Plasticity(STDP) [Markram ’97].
Further simplifications yield Hebbian learning: “Neurons that firetogether, wire together” [Hebb ’49].
Osvaldo Simeone Neuromorphic Computing 47 / 74
Learning Rules
The learning rules simplifies to Spike-Timing-Dependent Plasticity(STDP) [Markram ’97].
Further simplifications yield Hebbian learning: “Neurons that firetogether, wire together” [Hebb ’49].
Osvaldo Simeone Neuromorphic Computing 47 / 74
Overview
Motivation and Introduction
Applications
Models
Learning Algorithms
Examples
Osvaldo Simeone Neuromorphic Computing 48 / 74
Batch Training
encoder
source
decoder
5
Osvaldo Simeone Neuromorphic Computing 49 / 74
Batch Training
Graceful trade-off complexity/ delay vs accuracy [Jang et al ’18-2]
Osvaldo Simeone Neuromorphic Computing 50 / 74
Online TrainingOnline training for predictionFully connected (recurrent) SNN topology with hidden neurons
encoder
decoder
source input/ output
latent
Osvaldo Simeone Neuromorphic Computing 51 / 74
Online Training
Rate encoding
Osvaldo Simeone Neuromorphic Computing 52 / 74
Online TrainingRate encoding
Osvaldo Simeone Neuromorphic Computing 53 / 74
Online Training
Rate encoding
Osvaldo Simeone Neuromorphic Computing 54 / 74
Online Training
Time encoding
Osvaldo Simeone Neuromorphic Computing 55 / 74
Online Training
Rate vs time encoding
Osvaldo Simeone Neuromorphic Computing 56 / 74
Concluding Remarks
Statistical signal processing review of neuromorphic computing viaSpiking Neural Networks
Additional topics:I more general energy-based probabilistic SNN models [Osogami ’17]
[Jang ’19]I recurrent SNNs for long-term memory [Maas ’11]I neural sampling: information encoded in steady-state behavior [Buesing
et al ’11]I Bayesian learning via Langevin dynamics [Pecevski et al ’11] [Kappel et
al ’15]
Some open problems:I meta-learning, life-long learning, transfer learning [Bellec et al ’18]I training I/O interfaces [Lazar and Toth ’03]
Osvaldo Simeone Neuromorphic Computing 57 / 74
For More...
H. Jang, O. Simeone, B. Gardner and A. Gruning, “Spiking neuralnetworks: A stochastic signal processing perspective,”arXiv:1812.03929.
Osvaldo Simeone Neuromorphic Computing 58 / 74
Acknowledgements
This work has received funding from the European Research Council(ERC) under the European Union’s Horizon 2020 research and innovation
programme (grant agreement No. 725731) and from the US NationalScience Foundation (NSF) under grant ECCS 1710009.
Osvaldo Simeone Neuromorphic Computing 59 / 74
References
[Gerstner and Kistler ’02] W. Gerstner and W. M. Kistler, Spiking neuronmodels: Single neurons, populations, plasticity. Cambridge UniversityPress, 2002.[Pillow et al ’08] J.W. Pillow, J. Shlens, L. Paninski, A. Sher, A. M. Litke,E. Chichilnisky, and E. P. Simoncelli, “Spatio-temporal correlations andvisual signalling in a complete neuronal population,” Nature, vol. 454, no.7207, p. 995, 2008.[Osogami ’17] T. Osogami, “Boltzmann machines for time-series,” arXivpreprint arXiv:1708.06004, 2017.[Ibnkahla ’00] Ibnkahla M. Applications of neural networks to digitalcommunications–a survey. Signal processing. 2000 Jul 1;80(7):1185-215.[Koller and Friedman ’09] Koller D, Friedman N, Bach F. Probabilisticgraphical models: principles and techniques. MIT press; 2009.
Osvaldo Simeone Neuromorphic Computing 60 / 74
References
[Fremaux et al ‘08] N. Fremaux and W. Gerstner, “Neuromodulatedspike-timing-dependent plasticity, and theory of three-factor learningrules,” Frontiers in neural circuits, vol. 9, p. 85, 2016.[Jang et al ’18] H. Jang, O. Simeone, B. Gardnerm and A. Gruning,“Spiking neural networks: A stochastic signal processing perspective,”arXiv:1812.03929. ...[Rezende et al ’11] Rezende DJ, Wierstra D, Gerstner W. “Variationallearning for recurrent spiking networks”, In Advances in Neural InformationProcessing Systems, 2011 (pp. 136-144).[Brea et al ’13] J. Brea, W. Senn, and J.-P. Pfister, “Matching recall andstorage in sequence learning with spiking neural networks,” Journal ofNeuroscience, vol. 33, no. 23, pp. 9565–9575, 2013.[Hebb ’49] D. Hebb, The Organization of Behavior. New York: Wiley andSons, Nov. 1949.
Osvaldo Simeone Neuromorphic Computing 61 / 74
References
[Bienenstock et al ’82] E. L. Bienenstock, L. N. Cooper, and P. W. Munro,“Theory for the development of neuron selectivity: orientation specificityand binocular interaction in visual cortex,” Journal of Neuroscience, vol. 2,no. 1, pp. 32–48, 1982.[Pecevski et al ’11] D. Pecevski, L. Buesing, and W. Maass, “Probabilisticinference in general graphical models through sampling in stochasticnetworks of spiking neurons,” PLOS Computational Biology, vol. 7, no.12, pp. 1–25, 12 2011.[Rosenfeld et al ’18] Rosenfeld B, Simeone O, Rajendran B. LearningFirst-to-Spike Policies for Neuromorphic Control Using Policy Gradients.arXiv preprint arXiv:1810.09977. 2018 Oct 23.[Jang et al ’19] Jang H, Simeone O. Training Dynamic Exponential FamilyModels with Examples and Lateral Dependencies for GeneralizedNeuromorphic Computing”, in Proc. IEEE ICASSP 2019.
Osvaldo Simeone Neuromorphic Computing 62 / 74
References
[Bellec et al ’18] G. Bellec, D. Salaj, A. Subramoney, R. Legenstein, andW. Maass, “Long short-term memory and learning-to-learn in networks ofspiking neurons,” arXiv preprint arXiv:1803.09574, 2018.[Kappel et al ’15] Kappel D, Habenschuss S, Legenstein R, Maass W.Synaptic sampling: a Bayesian approach to neural network plasticity andrewiring. InAdvances in Neural Information Processing Systems 2015 (pp.370-378).[Buesing et al ’11] Buesing L, Bill J, Nessler B, Maass W. Neural dynamicsas sampling: a model for stochastic computation in recurrent networks ofspiking neurons. PLoS computational biology. 2011 Nov3;7(11):e1002211.[Lazar and Toth ’03] Lazar, Aurel A., and Laszlo T. Toth. "Time encodingand perfect recovery of bandlimited signals." ICASSP (6). 2003.[Maas ’11] Maass W. Liquid state machines: motivation, theory, andapplications. InComputability in context: computation and logic in thereal world 2011 (pp. 275-296).
Osvaldo Simeone Neuromorphic Computing 63 / 74
References
[Neftci ’18] Neftci EO. Data and power efficient intelligence withneuromorphic learning machines. iScience. 2018 Jul 27;5:52.[Anwani and Rajendran ’18] N. Anwani and B. Rajendran, “TrainingMultilayer Spiking Neural Networks using NormAD based Spatio-TemporalError Backpropagation,” arXiv:1811.10678.[Blouw et al ’18] Peter Blouw, Xuan Choo, Eric Hunsberger, ChrisEliasmith, “Benchmarking Keyword Spotting Efficiency on NeuromorphicHardware,” arXiv:1812.01739.[Binas et al ’17] J Binas, D Neil, SC Liu, T Delbruck, “DDD17:End-to-end DAVIS driving dataset,” arXiv:1711.01458, 2017.[Rajendran et al ’19] B. Rajendran, et al, “Low-Power NeuromorphicHardware for Signal Processing Applications”, arXiv:1901.03690.
Osvaldo Simeone Neuromorphic Computing 64 / 74
References
[Welling ’18] M. Welling, “Intelligence per kilowatt-hour”, plenary talk,ICML 2018, https://youtu.be/7QhkvG4MUbk.[Rueckauer ’17] Rueckauer B, Lungu IA, Hu Y, Pfeiffer M, Liu SC.Conversion of continuous-valued deep networks to efficient event-drivennetworks for image classification. Frontiers in neuroscience. 2017 Dec7;11:682.[Neal ’92] Neal RM. Connectionist learning of belief networks. Artificialintelligence. 1992 Jul 1;56(1):71-113.[Leibfried and Braun ’15] Leibfried, Felix, and Daniel A. Braun. ”Areward-maximizing spiking neuron as a bounded rational decision maker.”Neural computation 27.8 (2015): 1686-1720.
Osvaldo Simeone Neuromorphic Computing 65 / 74
Reinforcement Learning
encoder decoder
actuator
up
Osvaldo Simeone Neuromorphic Computing 66 / 74
Reinforcement Learning
From [Rosenfeld et al ’18]
GLM SNNIF SNNANN
GLM SNNIF SNN
Osvaldo Simeone Neuromorphic Computing 67 / 74
Applications
Example: Keyword spotting based on audio streaming; accuracycomparable to ANN.
[Blouw et al '18]
Osvaldo Simeone Neuromorphic Computing 68 / 74
Variational Inference (VI)
With latent variables z , Maximum Likelihood requires themaximization of
maxθ
ln p(x |θ) = ln
(∑z
p(x , z |θ)
)= ln
(Ez∼p(z|θ)[p(z|x , θ)]
)Key issue: Need to marginalize over latent variables, whosedistribution is to be learned.
Osvaldo Simeone Neuromorphic Computing 69 / 74
Variational Inference (VI)
VI tackles this problem by substituting the expectation for anoptimization over a predictive, or variational, distribution q(z |x , ϕ)
Optimization over both model parameters θ and variationalparameters ϕ is done by maximizing the Evidence Lower BOund(ELBO) or (negative) free energy
L(θ, ϕ) =Ez∼q(z|x ,ϕ)[ln p(x , z|θ)− ln q(z|x , ϕ)︸ ︷︷ ︸learning signal `(x ,z|θ,ϕ)
]
Osvaldo Simeone Neuromorphic Computing 70 / 74
Variational Inference (VI)
VI tackles this problem by substituting the expectation for anoptimization over a predictive, or variational, distribution q(z |x , ϕ)
Optimization over both model parameters θ and variationalparameters ϕ is done by maximizing the Evidence Lower BOund(ELBO) or (negative) free energy
L(θ, ϕ) =Ez∼q(z|x ,ϕ)[ln p(x , z|θ)− ln q(z|x , ϕ)︸ ︷︷ ︸learning signal `(x ,z|θ,ϕ)
]
Osvaldo Simeone Neuromorphic Computing 70 / 74
Variational Inference (VI)
The ELBO is a global lower bound on the log-likelihood (LL) function
ln p(x |θ) ≥ L(θ, ϕ),
Equality holds at a value θ0 if and only if the distribution q(z |x , ϕ) isthe posterior of z given x (i.e., optimal Bayesian estimate)
q(z |x , ϕ) = p(z |x , θ0).
LL
0
Osvaldo Simeone Neuromorphic Computing 71 / 74
Variational Inference (VI)
VI approximates the EM algorithm by using SGD over both variableswith gradients
∇θL(θ, ϕ) ≈ ∇θ log p(x , z),
∇ϕL(θ, ϕ) ≈ `(x , z |θ, ϕ) · ∇ϕ log q(z |x , ϕ)
with z ∼ q(z |x , ϕ)
Osvaldo Simeone Neuromorphic Computing 72 / 74
Variational Inference (VI) for SNNs
In order to simplify the evaluation of the learning signal and thelearning rule, a typical choice for GLM SNNs is
q(zi ,t |x≤T , z≤t−1) = p(zi ,t |ui ,t , θ)
This leads to a simplified learning signal
`t =∑i
log p(xi ,t |ui ,t)
where the sum is over the observed spike trains.
The learning signal measure the likelihood of the observed spike trains
Osvaldo Simeone Neuromorphic Computing 73 / 74
Pseudocode for Online Learning
For each time tI Sampling: Each hidden neuron i emits a spike with probability σ(ui,t)I Global feedback (1): A central processor updates the learning signal
`t = κ`t−1 + (1− κ)∑i
log p(xi,t |ui,t),
where the sum is over the observed neurons, which is fed back to alllatent neurons
I Global feedback (2): The central processor feeds back to all neuronsany reward signal rt (if reinforcement learning)
I Local parameter update: Three-factor rule with Mt = `t × rt for hiddenneurons and Mt = rt for the observed neurons
Osvaldo Simeone Neuromorphic Computing 74 / 74
Top Related