Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn...
Transcript of Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn...
![Page 1: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/1.jpg)
Deep Learning 소개 - 현재 딥러닝 기술 수준
강원대학교 IT대학
이창기
![Page 2: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/2.jpg)
Object Recognition
https://www.youtube.com/watch?v=n5uP_LP9SmM
![Page 3: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/3.jpg)
Semantic Segmentation
https://youtu.be/ZJMtDRbqH40
![Page 4: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/4.jpg)
Semantic Segmentation
VGGNet + Deconvolution network
![Page 6: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/6.jpg)
Neural Art
• Artistic style transfer using CNN
![Page 7: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/7.jpg)
Hand Writing by Machine
Input: recurrent neural network handwriting generation demo
Style:
http://www.cs.toronto.edu/~graves/handwriting.html
LSTM RNN:
![Page 8: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/8.jpg)
Music Composition
https://highnoongmt.wordpress.com/2015/05/22/lisls-stis-recurrent-neural-networks-for-folk-music-generation/
![Page 9: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/9.jpg)
Image Caption Generation
![Page 10: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/10.jpg)
Visual Question Answering
Facebook: Visual Q&A
![Page 11: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/11.jpg)
Play Game
![Page 12: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/12.jpg)
Word Analogy
King – Man + Woman ≈ Queen
http://deeplearner.fz-qqq.net/
![Page 14: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/14.jpg)
Neural Conversation Model
![Page 15: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/15.jpg)
Abstractive Text Summarization
로드킬로 숨진 친구의 곁을 지키는 길고양이의 모습이 포착되었다.
RNN_search+input_feeding+CopyNet
![Page 16: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/16.jpg)
Learning to Execute LSTM RNN
![Page 17: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/17.jpg)
Learning Approximate Solutions • Travelling Salesman Problem: NP-hard • Pointer Network can learn approximate solutions: O(n^2)
![Page 18: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/18.jpg)
Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique: hand-designed update rules learned update rules
SGD:
Momentum:
Adagrad:
Adadelta or RMSprop:
Adam:
![Page 19: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/19.jpg)
One Shot Learning • Learning from a few examples • Matching Nets use attention and memory
a(x1,x2) is a attention kernel
![Page 20: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/20.jpg)
Binarized Neural Networks BNN: neural networks with binary weights (i.e., 1 or -1) and activations at run-time 7 times faster
Binary weight filters
![Page 21: Deep Learning 소개 - Kangwonleeck/ML/deep_learning_intro.pdf · 2016-09-27 · Learning to Learn with RNN • Deep Learning: hand-designed features learned features • This technique:](https://reader034.fdocuments.net/reader034/viewer/2022042303/5ecead11e0b45c00a410da10/html5/thumbnails/21.jpg)
Interpretable Predictive Model in Healthcare