Neural Networks and Deep Learning

Anastasiia Kornilova

● What is Neural Networks and Deep Learning

● How to train Neural Network● Unsupervised Feature Learning● Building Handwritten Digits Classifier● Tips and Tricks

Agenda

Inspired by human brainBest suitable for human brain tasks:● speech recognition● object recognition

NN and Deep Learning

How brain works?

How neural networks work?

Activation functions:

Feedforwarding

Error function

Stochastic Gradient Descent

Backpropagation

Backpopagation

1. Perform a feedforward pass, computing the activations for layers L2, L3, and so on up to the output layer .

2. For each output unit i in layer nl (the output layer), set

3. For

For each node i in layer l, set

4. Compute the desired partial derivatives, which are given as:

Autoencoder

Stacked autoencoder

1) Data visualization2) Train autoencoder3) Build classifier4 ) Enjoy!

NN for digits recognition

Optimization tips● Linear algebra libraries● Minibatch● More optimizations methods

(activation functions, dropout, dropconnect, automatization learning rates)

● GPU computing

Links● Machine Learning by Andrew Ng● Unsupervised Feature Learning and D

eep Learning tutorial● Neural Networks for Machine Learnin

g by Geoffrey Hinton● Neural Networks and Deep Learning -

free online book● Pylearn2 - framework for deep learnin

g