USAGEE INC. / KAZUYA GOKITA

BACKPROPAGATION

Backpropagation(BP)DeepLearning BP

CNN BP, LSTM BP(BPTT)

( )BP

Deep Learning

Deep BP

:

↓

OVERVIEW

≠ BPBP ( =DeepLearning )

┗ BP

┗ BP

BP

┗ Deep Learning BP

┗ Deep Learning BP

┗

BP

BP …

forward, backward

forward, backward

(STDP

BPTT

BP

Target Propagation

Difference Target Propagation

OK 1 OK

Feedback Alignment

Direct Feedback Alignment

Decoupled Neural Interfaces using Synthetic Gradients

≠ BP

BP

Reservoir Computing (ESN, LSM, FORCE…)

An Analysis of Single-Layer Networks in Unsupervised Feature Learning

Topographic Independent Component Analysis (TICA)

Contrastive Hebbian Learning

Slow Feature Analysis

(BP, DL)

Target Propagation…

(

→ Approximate Inverse

TP Difference Target Propagation

Approximate Inverse target

Difference Target Propagation https://arxiv.org/abs/1412.7525

Feedback Alignment1 …

Direct Feedback Alignment Provides Learning in Deep Neural Networks https://arxiv.org/abs/1609.01596

Reservoir ComputingReservoir (LSM, ESN, FORCE )

V1

Fully analogue photonic reservoir computer An overview of reservoir computing: theory, applications, and implementations

Reservoir Computing

, . . 2008 9

FORCE

Reservoir

Transferring Learning from External to Internal Weights in Echo-State Networks with Sparse Connectivity Generating coherent patterns of activity from chaotic neural networks

Product Reservoir Computing: Time-Series Computation with Multiplicative Neurons

Slow Feature Analysis…

V1

STDP (Slowness: An Objective for Spike-Timing-Dependent Plasticity?)

Wiskott, L. and Sejnowski, T. Slow feature analysis: unsupervised learning of invariances. Neural Computation, 14(4):715--770, 2002.

Slow Feature Analysis

Franzius M, Sprekeler H, Wiskott L. Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLoS Comput Biol. 2007 Aug;3(8):e166.

Boccara, C. N., Sargolini, F., Thoresen, V. H. et al.: Grid cells in pre- and parasubiculum. Nat. Neurosci., 13, 987-994, 2010

Backpropagation

AI