머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow...
Transcript of 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow...
![Page 1: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/1.jpg)
머신러닝/딥러닝 시스템
![Page 2: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/2.jpg)
MEMOML Software Stack
![Page 3: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/3.jpg)
MEMO
High-level Structure
ML Framework
•
•
•
![Page 4: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/4.jpg)
MEMO
High-level Structure
ML Framework
•
![Page 5: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/5.jpg)
MEMO
Training
ML Framework
Forward pass: logits computationMini-batch
Backward pass: gradients computation
Input pipelineCompute
loss
![Page 6: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/6.jpg)
MEMO
Auto-Differentiation
ML Framework
•
•
•
![Page 7: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/7.jpg)
MEMO
Reverse-mode Autodiff
ML Framework
![Page 8: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/8.jpg)
MEMO
Auto-differentiation
ML Framework
![Page 9: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/9.jpg)
MEMOML Framework Categories
Build a graphExecute the graph
Directly execute statements
TensorFlow, Caffe2, MXNet, CNTK
PyTorch, TensorFlow Eager, MXNet Imperative
* Python: De-facto deep learning programming language
![Page 10: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/10.jpg)
MEMOML Framework Categories
Build a graphExecute the graph
Directly execute statements
TensorFlow, Caffe2, MXNet, CNTK
PyTorch, TensorFlow Eager, MXNet Imperative
Easy to optimize and deploy
Hard to program and debug
Easy to program and debug
Little room for optimization
![Page 11: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/11.jpg)
MEMO
Google TensorFlow
ML Framework Example
•
•
•
![Page 12: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/12.jpg)
MEMO
Facebook PyTorch
ML Framework Example
•
•
•
![Page 13: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/13.jpg)
MEMO
Open Neural Network Exchange (ONNX)
Interoperability between Frameworks
![Page 14: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/14.jpg)
MEMO
ML frameworks
Symbolic ML frameworks and imperative ML frameworks
ML framework examples
Summary
![Page 15: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/15.jpg)
텐서플로우
![Page 16: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/16.jpg)
MEMOTensorFlow
TensorFlow 1.x default mode
•
Express numerical computation as a computation graph
•
•
Tensor: N-dimensional array
•
•
•
Tensors flow through the graph → TensorFlow
![Page 17: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/17.jpg)
MEMOGoogle TensorFlow
The graph’s compiled to CPU / GPU / AIPU code
Salient features of TensorFlow graphs
![Page 18: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/18.jpg)
MEMOTensorFlow Programming Model
Graph: model computation
Session: runs a graph
•
![Page 19: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/19.jpg)
MEMOTensorFlow Programming Model
Symbolic Graph Style
•
•
•
•
![Page 20: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/20.jpg)
MEMOTensorFlow Eager Mode
Statements directly executed without the separation of graph definition and execution
Write code that you can easily execute in a REPL
![Page 21: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/21.jpg)
MEMOTensorFlow Programming Example
ReLU
Add
MatMul
![Page 22: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/22.jpg)
MEMOTensorFlow Programming Example
ReLU
Add
MatMul
![Page 23: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/23.jpg)
MEMOTensorFlow Programming Example
ReLU
Add
MatMul
![Page 24: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/24.jpg)
MEMOTensorFlow Programming Example
ReLU
Add
MatMul
![Page 25: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/25.jpg)
MEMOTensorFlow Programming Example
We can deploy the graph with a session
![Page 26: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/26.jpg)
MEMOTensorFlow Eager Mode
Enabling eager execution requires two lines of code
Lets you write code that you can easily execute in a REPL
![Page 27: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/27.jpg)
MEMOTensorFlow Example
Linear Regression
![Page 28: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/28.jpg)
MEMOTensorFlow Example
Linear Regression
![Page 29: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/29.jpg)
MEMOTensorFlow Example
Linear Regression
![Page 30: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/30.jpg)
MEMOInput Pipeline with TensorFlow Dataset
![Page 31: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/31.jpg)
MEMOInput Pipeline with TensorFlow Dataset
Shuffle, repeat, batch your data
Map each element of your dataset to transform it in a specific way to create a new dataset
![Page 32: 머신러닝 딥러닝시스템SNUk+SNU051_011k+2019_T1+type@asset... · 2019. 4. 9. · TensorFlow Programming Example MEMO ReLU Add MatMul. TensorFlow Programming Example MEMO We](https://reader034.fdocuments.net/reader034/viewer/2022051910/5fff3af519ccee2b9b27e1b8/html5/thumbnails/32.jpg)
MEMOSummary
TensorFlow programming model
TensorFlow basic program example
TensorFlow linear regression example
TensorFlow Dataset