딥러닝의이해와응용

(Deep Machine LearningKOSS Lab. 2기

Mario Cho (조만석)

hephaex@gmail.com

Contents

• 기계학습 (NNs)

• Neural Networks

• 딥러닝기술

• 활용방안

Mario ChoDevelopment Experience◆ Image Recognition using Neural Network◆ Bio-Medical Data Processing◆ Human Brain Mapping on High Performance

Computing◆ Medical Image Reconstruction

(Computer Tomography) ◆Enterprise System◆Open Source Software Developer

Open Source Software Developer◆ OPNFV (NFV&SDN) & OpenStack◆ Machine Learning (TensorFlow)

Book◆ Unix V6 Kernel Korea Open Source Software Lab.

Mario Chohephaex@gmail.com

Today’s information

* http://www.cray.com/Assets/Images/urika/edge/analytics-infographic.html

The Future of Jobs

“The Fourth Industrial Revolution, which includes developments in previously disjointed fields such as artificial intelligence & machine-learning, robotics, nanotechnology, 3-D printing, and genetics & biotechnology, will cause widespread disruption not only to business models but also to labor market over the next five years, with enormous change predicted in the skill sets needed to thrive in the new landscape.”

What is the Machine Learning ?• Field of Computer Science that evolved from the study of pattern recognition and computational learning theory into Artificial Intelligence.

• Its goal is to give computers the ability to learn without being explicitly programmed.

• For this purpose, Machine Learning uses mathematical / statistical techniques to construct models from a set of observed data rather than have specific set of instructions entered by the user that define the model for that set of data.

Required of New type of Computing understand information, to learn, to reason, and act upon it

Human Intelligence

Brain Map

Neuron

hippocampus

Neural network vs Learning networkNeural Network Deep Learning Network

Neural Network

W1

W2

W3

f(x)

1.4

-2.5

-0.06

Neural Network

2.7

-8.6

0.002

f(x)

1.4

-2.5

-0.06

x = -0.06×2.7 + 2.5×8.6 + 1.4×0.002 = 21.34

Neural Network : x1 XNOR x2

+1

x1

x2

+1

a1(2)

a2(2)

a1(3)

-30

20

20

10

-20

-10

-1010

-20x1 x2 a1(2) a2(2) a1(3)

0 0 0 1 10 1 0 0 01 0 0 0 01 1 1 0 0

Output10

Multi-layer Neural Networks

Train this layer first

Multi-layer Neural Networks

Train this layer firstthen this layer

then this layerthen this layer

finally this layer

Deep learning - CNN

Traditional learning vs Deep Machine Learning

Eiffel Tower

Eiffel Tower

RAW data

RAW data

Deep Learning Network

FeatureExtraction

Vectored Classification

Traditional Learning

Deep Learning

Image recognition in Google Map

* Source: Oriol Vinyals – Research Scientist at Google Brain

Deep Learning Hello World == MNIST

MNIST (predict number of image)

CNN (convolution neural network) training

MNIST

Hierarchical Representation of Deep Learning

* Source: : Honglak Lee and colleagues (2011) as published in “Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks”.

* Source: : Honglak Lee and colleagues (2011) as published in “Unsupervised Learning of Hierarchical Representations with Convolutional Deep Belief Networks”.

Deep Learning (object parts)

Face extraction method

Face recognition data- sets?

Human-Level Face Recognition

• Convolutional neural networks based face recognition system is dominant

• 99.15% face verification accuracy on LFW dataset in DeepID2 (2014)� Beyond human-level recognition

Source: Taigman et al. DeepFace: Closing the Gap to Human-Level Performance in Face Verification, CVPR’14

Image Recognition

* Source: Oriol Vinyals – Research Scientist at Google Brain

Object Classification and Detection

Scene Parsing

[Farabet et al. ICML 2012, PAMI 2013]

How to the Object recognition ?

ILSVRC (Image-net Large Scale Visual Recognition Challenge)

Language Generating

* Source: Oriol Vinyals – Research Scientist at Google Brain

Image Caption Generation

Human-Level Object Recognition

• ImageNet• Large-Scale Visual Recognition Challenge�Image Classification / Localization�1.2M labeled images, 1000 classes�Convolutional Neural Networks (CNNs)has been dominating the contest since..� 2012 non-CNN: 26.2% (top-5 error)� 2012: (Hinton, AlexNet)15.3%� 2013: (Clarifai) 11.2%� 2014: (Google, GoogLeNet) 6.7%� 2015: (Google) 4.9%� Beyond human-level performance

Scene Parsing

[Farabet et al. ICML 2012, PAMI 2013]

Auto pilot car

Tic Tac Toc

AlphaGo

RAW data

Layer1Policy network & value network

Selection

Expansions

Evaluation

Backup

~ Layer13

Automatic Colorization of Black and White Images

Generate sounds on old-movies

Automatic Machine Translation• Automatic Translation of Text.

• Automatic Translation of Images.

Automatic Handwriting Generation

Text Generation

create stylized images from rough sketches.

Inspirer Humanity

Thanks you!

Q&A