Konvolun neuronov stMichal Hradi

http://www.fit.vutbr.cz/~ihradis/

Obsah

Motivace highlights industry success stories

Cesta k hlubokmu uen x run navren pznaky

Co to je neuronov s?

Jak se u neuronov st?

Co jsou to konvolun st?

Co jde dlat s konvolunmi stmi?

Nstroje

Zklady Caffe

Vyhledvn fotek

Google photo search search your photo collection

https://photos.google.com/search

"man in black shirt

is playing guitar."

Andrej Karpathy, Li Fei-Fei: Deep Visual-Semantic Alignments for Generating Image Descriptions. CVPR 2015

http://cs.stanford.edu/people/karpathy/deepimagesent/

"girl in pink dress

is jumping in air."

Automatick popis fotek

"construction worker in orange

safety vest is working on road."

"black cat is sitting

on top of suitcase.""boy is doing

backflip on wakeboard."

"construction worker in orange

safety vest is working on road."

Andrej Karpathy, Li Fei-Fei: Deep Visual-Semantic Alignments for Generating Image Descriptions. CVPR 2015

http://cs.stanford.edu/people/karpathy/deepimagesent/

"a cat is sitting on a couch

with a remote control."

Automatick popis fotek

"a woman holding a teddy

bear in front of a mirror."

"a horse is standing

in the middle of a road."

Andrej Karpathy, Li Fei-Fei: Deep Visual-Semantic Alignments for Generating Image Descriptions. CVPR 2015

http://cs.stanford.edu/people/karpathy/deepimagesent/

"a young boy is

holding a baseball bat."

Automatick popis fotek

Facebook - DeepFace

Deep Dreams

Thorne Brandt

Style transfer

Leon A. Gatys, Alexander S. Ecker, Matthias Bethge: A Neural Algorithm of Artistic Style

Style transfer

Style transfer https://vimeo.com/139123754

https://vimeo.com/139123754

13

Co je na obrzku?

Intelligent

stuff

outdoors yes

indoors no

sport yes

person yes

water yes

trees yes

politician no

singing no

dog no

cat no

dancing no

cars no

walking no

running no

swimming no

mountains no

Tags

Kategorie

slovn zsoba

aktivn 3k 10k

pasivn 50k

Obecn kategorie

Objekty, innosti, prosted, podmnky, nlada, pocity, druh

fotografie/zbru, nr

Specifick (pojmenovan)

msta, konkrtn osoby, znaky aut,

Strojov uen

Intelligent

stuff

Flag

Flag

Flag

AK-47

AK-47

Zklad - podobnost obraz

16

Bag of Words

Bag of Words

Bag of Words

19

Image representations

Fixed engineered features followed by simple trainable

classifier

Human-driven

academic evolution

Learning algorithm

Traditional approach

Feature ExtractorSimple Classifier

(lin./kernel SVM)

9

2

0

1

x 0.7 = 6.3

x 0.4 = 0.8

x -0.3 = 0

x -0.2 = -0.2

SUM = 7.1

> 0 GODDES

< 0 Something

else

Navreno lovkem

(pracovit Ph.D. stud.) Uc algoritmus

Hlubok uen deep learning

Hlubok uen

Extrakce pznak

Jednoduch

klasifiktor

(lin./kernel SVM)

Uc algoritmus

Extrakce

pznakklasifiktor

Hierarchy of features

Low-level

features

Mid-level

features

High-level

features

Co je to neuronov s?

0.1

0.2

0.5

0.6

0.7

0.8

0.4

0.2

0.6

function with

trainable parameters

0

1

0.5

1.2

3

2

1

2

0

Numerical

input vectorNumerical

output vector

Neural Network

Pklad neuronov st

= +

Strojov uen

Intelligent

stuff

Flag

Flag

Flag

AK-47

AK-47

Pklad neuronov st

= +

argmin,

(,)

2

Generalizace uen

Generalizace uen

Neuronov st

Klasick dopedn neuronov st

St mou eit velkou klu loh podle vlastnost neuron

ve vstupn vrstv a objektivn funkce

Multi-class, multi-label, regrese,

Hlub neuronov st

Source: UFLDL tutorial, http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial

Hlub neuronov st

Source: UFLDL tutorial, http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial

Hlub neuronov st

Source: UFLDL tutorial, http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial

x

Zklad neuronu

1

2

3

()

1

2

3

= 11 +22 +3 3

Neuron s ReLU aktivan funkc

1

2

3

()

1

2

3

= max(11 +22 +3 3, 0)

Hlub neuronov st

Source: UFLDL tutorial, http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial

Vpoet

2 = 11, 0

1 2 3 4

3 = 22, 0

4 = 33, 0

Uen neuronovch st

Zptn en chyby

derivace objektivn vzhledem ke vem parametrm st

vyuit pravidla pro efektivn derivace sloen funkce (chain rule)

Pklad chybov fce. half squared error

()

=()

()

()

Source: UFLDL tutorial, http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial

= ( )

Zptn en chyby

1. Dopedn prchod st, spotat a uloit aktivace vech

neuron

2. Spotat parciln derivace vech vstupnch neuron

3. Zptn prchod rekurzivn spotat parciln derivace

vech neuron v sti vdy z parcilnch derivac neuron v

nsledujc vrstv

4. Spotat derivace vah a bias z aktivac pedel vstvy a

derivac nsledujc vrstvy

Stochastic Gradient Descent

Jak minimalizovat hodnotu objektivn funkce na datasetu?

Gradient

Parciln derivace dohromady tvo gradient

Pohyb ve smru gradientu zvyuje hodnotu funkce (zhoruje

vsledky st) a pohyb proti smru gradientu sniuje hodnotu funkce

(zlepuje s)

Stochastic gradient se pot na podmnoinch trnovac

sady

Pravidlo pro uen v Stochastic Gradient Descent

Stochastic Gradient Descent

Historie Konvolunch st

1980 Kunihiko Fukushima Neocognitron: A Self-

organizing Neural Network Model for a Mechanism of

Pattern Recognition Unaffected by Shift in Position

1998 LeCun et al.: Gradient-Based Learning Applied to

Document Recognition

2012 Krizhevsky et al.: ImageNet Classification with Deep

Convolutional Neural Networks

2013-dnes exploze aplikac

Konvolun neuronov st

Jsou to pln normln dopedn neuronov st

Bvaj hlubok

Vrstvy maj pevnou strukturu propojen KONVOLUCE

vhy neuronovch st = konvolun jdra konvolunch st

43

Konvoluce

Konvolun vrstva

Konvolun vrstva - kanly

Pooling vrstva

Podvzorkovn

pixely z okol se agreguj do jedn hodnoty

zmenen rozmru vrstvy

pouvaj se operace MAX/MEAN

MAX pmo zvyuje invarianci vi posunut

Architektura st

Vlastnosti konvolunch st

Vhody

Klasifikace obrzk je velmi rychl. >100 fps na rychl GPU

Problmy

Pro trnovn je poteba rozumt, co se dje uvnit. asto st

nedlaj, co maj, a je poteba zjistit pro.

Trnovn je vpoetn nron. Velk st se na nejnovjch GPU

trnuj klidn msc.

Jak trnovat st?

Koupit rychlou hern grafiku

GTX 970/980 a podobn

Vybrat nstroj a nainstalovat

Caffe, Lasagne, Keras, OpenDeep

Definovat problm

Jak vstupy, jak vstupy

Sehnat data

Sthnout existujc pedtrnovanou s

Model Zoo https://github.com/BVLC/caffe/wiki/Model-Zoo

Podle poteby vymnit vstupn vrstvu st

Dotrnovat s na vlastnm datasetu

https://github.com/BVLC/caffe/wiki/Model-Zoo

CNN features

Take pre-trained network and use activation of a late layer

as features

https://github.com/BVLC/caffe/wiki/Model-Zoo

E.g. learned on ImageNet

Use any classifier for your problem

Fine-tuning on your problem gives better results.

Donahue et al.:DeCAF: A Deep Convolutional Activation Feature for Generic Visual

Recognition

CNN features

5

Donahue et al.: DeCAF: A Deep Convolutional Activation Feature for Generic Visual

Recognition. ICML 2014

Cross-domain image search

Crowley and Zissrman: In Search of Art, Workshop on Computer Vision for Art Analysis,

ECCV, 2014

Feature visualization Layer 2

Zeiler, Fergus: Visualizing and Understanding Convolutional Networks, ECCV 2014.

Feature visualization Layer 5

Zeiler, Fergus: Visualizing and Understanding Convolutional Networks, ECCV 2014.

Layers and network size

5

Krizhevsky, A., Sutskever, I. and Hinton, G. E.: ImageNet Classification with Deep

Convolutional Neural Networks, NIPS 2012.

Start small

low resolution, ~4 hidden layers, not too many channels

run fast experiments

Scale-up until overfitting takes over

Fix overfitting and scale-up again

Very Deep Convolutional Networks

5

Karen Simonyan, Andrew Zisserman: Very Deep Convolutional Networks for Large-Scale

Image Recognition,