© 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**)...
Transcript of © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**)...
![Page 1: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/1.jpg)
1© 2015 The MathWorks, Inc.
![Page 2: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/2.jpg)
2© 2015 The MathWorks, Inc.
딥러닝을 활용한 영상 인식응용프로그램 개발 워크플로우
송완빈Application Engineer
MathWorks Korea
![Page 3: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/3.jpg)
3
Agenda
▪ Deep Learning Application for object recognition
▪ What is Deep Learning?
▪ Object Recognition using Deep Learning
– Convolutional Neural Network
– Regions with Convolutional Neural Network
– Semantic Segmentation
▪ Key Takeaways
![Page 4: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/4.jpg)
4
Users with Mobile Devices
Smart Phone Application with Deep Learning
Mobile Internet Browser Mobile Camera
![Page 5: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/5.jpg)
5
Image processing algorithm Machine Learning Approach
What is a scene where machine learning should be used?
Computer
Program
‘Face’
‘Helmet’
Hand Written Program Formula or Equation
If brightness > 0.5
then ‘Helmet’
If (edge_density < 4 and
major_axis > 5)
then …
𝑌𝐶𝑙𝑎𝑠𝑠𝑖𝑓𝑦= 𝛽1𝑋edge + 𝛽2𝑌𝑏right+ 𝛽3𝑍in𝑡𝑒𝑛𝑠𝑖𝑡𝑦 +
…
𝑚𝑜𝑑𝑒𝑙 = <𝑴𝒂𝒄𝒉𝒊𝒏𝒆𝑳𝒆𝒂𝒓𝒏𝒊𝒏𝒈𝑨𝒍𝒈𝒐𝒓𝒊𝒕𝒉𝒎
>(images, label)
𝑚𝑜𝑑𝑒𝑙 ∶ Inputs → Outputs
• Specify condition by numerical value and carve
• When clear division is possible
• Learn classifiers using image data
• Flexible separation can be done if it works
![Page 6: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/6.jpg)
6
Image processing algorithm Machine Learning Approach
What is a scene where machine learning should be used?
Computer
Program
‘Face’
‘Helmet’
Hand Written Program Formula or Equation
If brightness > 0.5
then ‘Helmet’
If (edge_density < 4 and
major_axis > 5)
then …
𝑌𝐶𝑙𝑎𝑠𝑠𝑖𝑓𝑦= 𝛽1𝑋edge + 𝛽2𝑌𝑏right+ 𝛽3𝑍in𝑡𝑒𝑛𝑠𝑖𝑡𝑦 +
…
Deep Learning
• Specify condition by numerical value and carve
• When clear division is possible
• Learn classifiers using image data
• Flexible separation can be done if it works
![Page 7: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/7.jpg)
7
Machine learning vs Deep learning
Deep learning performs end-to-end learning by learning features, representations and tasks directly
from images, text and sound
Deep learning algorithms also scale with data – traditional machine learning saturates
Machine Learning
Deep Learning
![Page 8: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/8.jpg)
8
MATLAB deep network in a nutshell
• A MATLAB deep network (**) is a MATLAB object that contains
an array of trained layer objects.
• Layers array can be created, imported, edited, plotted in
MATLAB
• Layers are trained with a lot of data andnet = trainNetwork(…,layers), most of the time.
• MATLAB deep networks have different usages.
layers = [ ...
imageInputLayer([28 28 1])
convolution2dLayer(12,25)
…
reluLayer
fullyConnectedLayer(1)
regressionLayer ]
Object detection (*)
car
dog
✓
Classification
32
Regression Semantic segmentation (*)
sky
car
road
YPred = classify(net,X); YPred = predict(net,X); C = semanticseg(I,net); [bboxes,scores] = detect(net,I);
Data=? Data=?
(**) Neural Network Toolbox (*) Computer Vision Syst. Toolbox
![Page 9: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/9.jpg)
9
Object Recognition using Deep Learning
Front of Car Stop Sign
R-CNN / Fast R-CNN / Faster R-CNN
Object detection and recognition
Road Vehicle
SegNet / FCN
Object recognition
(in pixels)
CNN (Convolutional Neural Network)
Cat
Dog
ProbabilityImage
Object recognition
(whole image)
![Page 10: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/10.jpg)
10
Object Recognition using Deep Learning
Front of Car Stop Sign
R-CNN / Fast R-CNN / Faster R-CNN
Object detection and recognition
Road Vehicle
SegNet / FCN
Object recognition
(in pixels)
CNN (Convolutional Neural Network)
Cat
Dog
ProbabilityImage
Object recognition
(whole image)
![Page 11: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/11.jpg)
11
▪ Train “deep” neural networks on structured data (e.g. images, signals, text)
▪ Implements Feature Learning: Removes need for “hand crafted” features
▪ Be trained using GPUs for performance
Convolutional Neural Networks
Convolution +
ReLu PoolingInput
Convolution +
ReLu Pooling
… …
Flatten Fully
ConnectedSoftmax
cartruck
bicycle
…
van
… …
Feature Learning Classification
![Page 12: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/12.jpg)
12
Two approaches for Deep learning
Approach 1. Train a Deep Neural Network from Scratch
…𝟗𝟓%𝟑%
𝟐%
Truck
Car ✓
Bicycle
Convolutional Neural Network (CNN)
Learned features
Lots of data
Approach 2. Fine-tune a pre-trained model (Transfer learning)
New Task
Fine-tune network weights
Truck
Car ✓Pre-trained CNN
Medium amounts of data
![Page 13: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/13.jpg)
13
…
𝟓%𝟏%
𝟖𝟐%
Boy
Cow
Bicycle✓
Learned features
SVM,
Logistic,
etc.
Classification
(Machine Learning)(Deep Learning)
Cat
Dog ✓Traditional Feature Extraction
CNN Feature Extractor
Training data 100s to 1000s of labeled images (small)
Computation Moderate computation (GPU optional)
Training Time Seconds to minutes
Model accuracy Good, depends on the pre-trained CNN model
Recommended when:the accuracy is not high enough using transfer learning
Another Deep Learning Workflows: Feature ExtractionUse a pretrained CNN as an automatic feature extractor
![Page 14: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/14.jpg)
14
Deep Learning Application Approaches
Train a Deep Neural Network from Scratch
Fine-tune a pre-trained model (Transfer learning)
Use a pretrained CNN as an automatic feature extractor
![Page 15: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/15.jpg)
15
Deep Learning Application Approaches
Train a Deep Neural Network from Scratch
Fine-tune a pre-trained model (Transfer learning)
Use a pretrained CNN as an automatic feature extractor
![Page 16: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/16.jpg)
16
Preprocess Images
Define Layers in CNN
Set training options
Train the network
Test trained network
Deploy trained network
Repeat these steps until
network reaches desired
level of accuracy
Deep Learning Workflow from Scratch
![Page 17: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/17.jpg)
17
Example : MNIST, The “Hello, World!” of computer vision
What? A set of handwritten digits from 0-9
Why?An easy task for machine learning
beginners
How many?60,000 training images
10,000 test images
Best results? 99.79% accuracy = 7
Sources: http://yann.lecun.com/exdb/mnist/https://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results
![Page 18: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/18.jpg)
18
1
9
8
2
28 x 28 pixel
Handwritten CharacterNumber (0-9)
Co
nv. L
aye
r
No
rma
liza
tion
Ma
x P
oo
ling
Co
nv. L
aye
r
No
rma
liza
tion
Ma
x P
oo
ling
fully
Co
nn
ecte
d
So
ftma
xL
aye
r
Cla
ssific
atio
n
• Typical CNN Architecture
• Learning is performed by error back propagation method
Example : MNIST, The “Hello, World!” of computer vision
![Page 19: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/19.jpg)
19
Train a Deep Neural Network from Scratch
▪ Example of network construction with 28x28 sized image
layers = [ ...
imageInputLayer([28 28 1], 'Normalization', 'none');
convolution2dLayer(5, 20);
reluLayer();
maxPooling2dLayer(2, 'Stride', 2);
fullyConnectedLayer(10);
softmaxLayer();
classificationLayer()];
opts = trainingOptions('sgdm', 'MaxEpochs', 50);
net = trainNetwork(XTrain, TTrain, layers, opts);
Define Layers as in order
Define learning rate and maximum iteration number
Call learning functions
Automatically judge the presence or absence of GPU. If there is GPU, learning with CPU if not.
![Page 20: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/20.jpg)
20
Demo
![Page 21: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/21.jpg)
21
Hurdle to create a network from scratch
▪ What kind of learning does the network with high precision do?
▪ A hurdle when creating a network with scratch
– Knowledge of network construction
– Large number of image sets
– Extensive calculation cost
AlexNet
■ 5 to 6 days of learning with 2 NVIDIA® GeForce ® GTX 580
VGG Net
■ 2 to 3 weeks study with NVIDIA® GeForce ® TITAN Black 4 aircraft
AlexNet
VGG
Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton "ImageNet Classification with Deep Convolutional Neural Networks" In NIPS, pp.1106-1114, 2012
K. Simonyan, A. Zisserman "Very Deep Convolutional Networks for Large-Scale Image Recognition" arXiv technical report, 2014
Source: ILSVRC Top-5 Error on ImageNet
![Page 22: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/22.jpg)
22
Deep Learning Application Approaches
Train a Deep Neural Network from Scratch
Fine-tune a pre-trained model (Transfer learning)
Use a pretrained CNN as an automatic feature extractor
![Page 23: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/23.jpg)
23
Deep Learning Application Approaches
Train a Deep Neural Network from Scratch
Fine-tune a pre-trained model (Transfer learning)
Use a pretrained CNN as an automatic feature extractor
![Page 24: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/24.jpg)
24
Convolution +
ReLu PoolingInput
Convolution +
ReLu Pooling
…
Feature Learning
…
Fully
ConnectedSoftmax
cartruck
bicycle
…
van
… …
Classification
Transfer Learning in CNN: Replace final layers
Fully
Connected Softmaxfood
tree…
New Classification
![Page 25: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/25.jpg)
25
Transfer Learning Workflow
Early layers learn low-
level features (edges,
blobs, colors)
Last layers
learn task-
specific features
1 million images
1000s classes
Load pretrained network
![Page 26: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/26.jpg)
26
Transfer Learning Workflow
Early layers that learned
low-level features
(edges, blobs, colors)
Last layers that
learned task
specific features
1 million images
1000s classes
Load pretrained network
Fewer classesLearn faster
New layers learn
features specific
to your data
Replace final layers
![Page 27: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/27.jpg)
27
Transfer Learning Workflow
Early layers that learned
low-level features
(edges, blobs, colors)
Last layers that
learned task
specific features
1 million images
1000s classes
Load pretrained network
Fewer classes
Learn faster
New layers to learn
features specific
to your data
Replace final layers
100s images
10s classes
Training images
Training options
Train network
![Page 28: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/28.jpg)
28
Fewer classes
Learn faster
New layers learn
features specific
to your data
Replace final layers
Transfer Learning Workflow
Early layers that learned
low-level features
(edges, blobs, colors)
Last layers that
learned task
specific features
1 million images
1000s classes
Load pretrained network
Fewer classes
Learn faster
New layers to learn
features specific
to your data
Replace final layers
100s images
10s classes
Training images
Training options
Train network
Test images
New Trained
Network
Predict and assess
network accuracy
![Page 29: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/29.jpg)
29
Transfer Learning Workflow
Early layers that learned
low-level features
(edges, blobs, colors)
Last layers that
learned task
specific features
1 million images
1000s classes
Load pretrained network
Fewer classes
Learn faster
New layers to learn
features specific
to your data
Replace final layers
100s images
10s classes
Training images
Training options
Train network
Test images
Trained Network
Predict and assess
network accuracy
Test images
Trained Network
Predict and assess
network accuracyProbability
Boat
Plane
Car
Train
Deploy results
![Page 30: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/30.jpg)
30
Transfer Learning Workflow
Probability
Boat
Plane
Car
Train
Deploy results
Early layers that learned
low-level features
(edges, blobs, colors)
Last layers that
learned task
specific features
1 million images
1000s classes
Load pretrained network
Fewer classes
Learn faster
New layers to learn
features specific
to your data
Replace final layers
100s images
10s classes
Training images
Training options
Train network
Test images
New Trained
Network
Predict and assess
network accuracy
![Page 31: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/31.jpg)
31
Example: Transfer Learning
Validation Accuracy : 100%
MathWorks Logo Laptop
ClipSmart Phone
![Page 32: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/32.jpg)
32
Deep Learning Application Approaches
Train a Deep Neural Network from Scratch
Fine-tune a pre-trained model (Transfer learning)
Use a pretrained CNN as an automatic feature extractor
![Page 33: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/33.jpg)
33
Deep Learning Application Approaches
Train a Deep Neural Network from Scratch
Fine-tune a pre-trained model (Transfer learning)
Use a pretrained CNN as an automatic feature extractor
![Page 34: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/34.jpg)
34
Convolution +
ReLu PoolingInput
Convolution +
ReLu Pooling
…
Feature Learning
…
Fully
ConnectedSoftmax
cartruck
bicycle
…
van
… …
Classification
Transfer Learning: Feature extraction
food
tree
Machine learning
SVM,
Logistic,
etc.
…
![Page 35: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/35.jpg)
35
What is Feature Extraction ?
Image Pixels ColorHistogram of Oriented
GradientsEdges
Dense Sparse
![Page 36: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/36.jpg)
36
Example : Image anomaly Detection Task
4 Defective units
100 hexagon nutsTask: Find 4 defective units in 100 test images.
▪ Challenge
– Number of defective units is very small.
– Difficult to apply supervised learning to this task.
![Page 37: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/37.jpg)
37
Example : Image anomaly Detection Task
Early layers that learned
low-level features
(edges, blobs, colors)
Last layers that
learned task
specific features
Load pretrained network
Training
images
Extract features
and train 1-class SVM
Test images
Predict with Test images
・Extract features with pre-trained
model(AlexNet)・Train 1-class SVM with 100
images.
・Unsupervised training
・Predict
・List sorted by predicted score
・Find 4 defective units from 100
test images
1-class
SVM
1-class
SVM
・Load pretrained AlexNet
![Page 38: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/38.jpg)
38
Demo
![Page 39: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/39.jpg)
39
Object Recognition using Deep Learning
Front of Car Stop Sign
R-CNN / Fast R-CNN / Faster R-CNN
Object detection and recognition
Road Vehicle
SegNet / FCN
Object recognition
(in pixels)
CNN (Convolutional Neural Network)
Cat
Dog
ProbabilityImage
Object recognition
(whole image)
![Page 40: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/40.jpg)
40
R-CNNs : Regions with Convolutional Neural Networks
▪ Object detection and identification method combining CNN with computer
vision method
Detection example when R-CNN learns car front and stop signs
Front of Car
Stop Sign
![Page 41: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/41.jpg)
41
R-CNNs : Regions with Convolutional Neural Networks
▪ Take a neural network trained for image classification and modify it for object
detection.
1. For a given input image, region proposals (ROIs) are generated
2. Each region proposal is then independently send through the CNN to compute features
3. The neural network makes a classification for each region proposal
The differences between the R-CNN methods occur at the region proposal stage.
Convolutional
Neural Network
Person? No
Alligator? Yes
Truck? No
![Page 42: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/42.jpg)
42
R-CNNs Learning
▪ >> detector = trainRCNNObjectDetector(groundTruth, network, options)
▪ >> [bbox, score, label] = detect(detector, image);
Ground Truth Series Network or Array of layers
FastRCNNFasterRCNN
![Page 43: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/43.jpg)
43
Data
Ground truth Labeling
“How do I label
my data?”
New App for
Ground Truth
Labeling
Label pixels
and regions for
semantic
segmentation
![Page 44: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/44.jpg)
44
Example : Detect and Classify hexagon nut using Faster RCNN
▪ Detect and classify hexagon nut into Top/Bottom using Faster R-CNN.
• Features of Top / bottom are Same shape, Same color, Same size and Tiny
difference in texture
![Page 45: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/45.jpg)
45
Demo
![Page 46: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/46.jpg)
46
Example : Detect and Classify hexagon nut using Faster RCNN
▪ Control manipulator robot using this object recognition.
![Page 47: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/47.jpg)
47
Object Recognition using Deep Learning
Front of Car Stop Sign
R-CNN / Fast R-CNN / Faster R-CNN
Object detection and recognition
Road Vehicle
SegNet / FCN
Object recognition
(in pixels)
CNN (Convolutional Neural Network)
Cat
Dog
ProbabilityImage
Object recognition
(whole image)
![Page 48: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/48.jpg)
48
Original Image
ROI detection
Pixel classification
Semantic Segmentation
![Page 49: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/49.jpg)
49
Semantic Segmentation Network
Boat
Airplane
Other classes
![Page 50: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/50.jpg)
50
Semantic Segmentation Network
![Page 51: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/51.jpg)
51
Semantic Segmentation
CamVid Dataset1. Segmentation and Recognition Using Structure from Motion Point Clouds, ECCV 2008
2. Semantic Object Classes in Video: A High-Definition Ground Truth Database ,Pattern Recognition Letters
![Page 52: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/52.jpg)
52
Semantic Segmentation
▪ A method of categorizing each pixels based on its meaning.
• Distinguish between sidewalks and roadways
• It is not just looking at colors
![Page 53: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/53.jpg)
53
Example : Semantic Segmentation for Free road detection
▪ Use Semantic Segmentation to detect the free space on the road as well as
lanes and pavements
![Page 54: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/54.jpg)
54
Demo
![Page 55: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/55.jpg)
56
MATLAB deep network in a nutshell
• A MATLAB deep network (**) is a MATLAB object that contains
an array of trained layer objects.
• Layers array can be created, imported, edited, plotted in
MATLAB
• Layers are trained with a lot of data andnet = trainNetwork(…,layers), most of the time.
• MATLAB deep networks have different usages.
layers = [ ...
imageInputLayer([28 28 1])
convolution2dLayer(12,25)
…
reluLayer
fullyConnectedLayer(1)
regressionLayer ]
Object detection (*)
car
dog
✓
Classification
32
Regression Semantic segmentation (*)
sky
car
road
YPred = classify(net,X); YPred = predict(net,X); C = semanticseg(I,net); [bboxes,scores] = detect(net,I);
Data=? Data=?
(**) Neural Network Toolbox (*) Computer Vision Syst. Toolbox
![Page 56: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/56.jpg)
57
MathWorks can help you do Deep Learning
▪ Guided evaluations with a
MathWorks deep learning
engineer
▪ Proof-of-concept projects
▪ Deep learning hands-on
workshop
▪ Seminars and technical deep
dives
▪ Deep learning onramp course
▪ Consulting services
▪ Training courses
▪ Technical support
▪ Advanced customer support
▪ Installation, enterprise, and cloud
deployment
▪ MATLAB for Deep Learning
Free resources More options
![Page 57: © 2015 The MathWorks, Inc. · 8 MATLAB deep network in a nutshell • A MATLAB deep network (**) is a MATLAB object that contains an array of trained layer objects. • Layers array](https://reader036.fdocuments.net/reader036/viewer/2022070213/610c3558f2ae8664ed4e1903/html5/thumbnails/57.jpg)
58
© 2018 The MathWorks, Inc. MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See
www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand names may be
trademarks or registered trademarks of their respective holders.