TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core...
Transcript of TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core...
![Page 1: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/1.jpg)
TensorFlow: what and why?
Konstantin Shmelkov
Grenoble Data Science Meetup
14 Sep 2016
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 1 / 20
![Page 2: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/2.jpg)
What is TensorFlow?
From the whitepaper: “TensorFlow is an interface for expressing machinelearning algorithms, and an implementation for executing such algorithms”.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 2 / 20
![Page 3: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/3.jpg)
Data flow graph
Pictures are from colah.github.io
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 3 / 20
![Page 4: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/4.jpg)
Data flow graph
Pictures are from colah.github.io
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 3 / 20
![Page 5: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/5.jpg)
Data flow graph
Pictures are from colah.github.io
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 3 / 20
![Page 6: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/6.jpg)
Back in the past: Theano (2009)
Framework mostly developed in LISA group at the University of Montreal.
Features:
symbolic differentiation,
transparent use of GPU,
dynamic code generation (C and CUDA),
everything in Python!
Main challenges of deep learning framework: flexible, distributed,easy-to-use.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 4 / 20
![Page 7: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/7.jpg)
Back in the past: Theano (2009)
Framework mostly developed in LISA group at the University of Montreal.
Features:
symbolic differentiation,
transparent use of GPU,
dynamic code generation (C and CUDA),
everything in Python!
Main challenges of deep learning framework: flexible, distributed,easy-to-use.
Theano: flexible, distributed, easy-to-use.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 4 / 20
![Page 8: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/8.jpg)
TensorFlow features
distributed on all levels: real multithreading, multi-GPU, multiplecluster nodes,
symbolic differentiation,
freshly designed API,
Python API, C++ core (Eigen),
powerful visualization with TensorBoard,
model deployment with TensorFlow Serving,
integration with Google Cloud Platform.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 5 / 20
![Page 9: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/9.jpg)
TensorFlow features
distributed on all levels: real multithreading, multi-GPU, multiplecluster nodes,
symbolic differentiation,
freshly designed API,
Python API, C++ core (Eigen),
powerful visualization with TensorBoard,
model deployment with TensorFlow Serving,
integration with Google Cloud Platform.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 5 / 20
![Page 10: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/10.jpg)
TensorFlow features
distributed on all levels: real multithreading, multi-GPU, multiplecluster nodes,
symbolic differentiation,
freshly designed API,
Python API, C++ core (Eigen),
powerful visualization with TensorBoard,
model deployment with TensorFlow Serving,
integration with Google Cloud Platform.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 5 / 20
![Page 11: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/11.jpg)
TensorFlow features
distributed on all levels: real multithreading, multi-GPU, multiplecluster nodes,
symbolic differentiation,
freshly designed API,
Python API, C++ core (Eigen),
powerful visualization with TensorBoard,
model deployment with TensorFlow Serving,
integration with Google Cloud Platform.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 5 / 20
![Page 12: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/12.jpg)
Core abstractions
This is a data flow graph.
x is a Placeholder.W and b are Variables.Everything else are intermediate Tensors.
import tensorflow as tf
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.nn.relu(tf.matmul(x, W) + b)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 6 / 20
![Page 13: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/13.jpg)
Core abstractions
This is a data flow graph.x is a Placeholder.
W and b are Variables.Everything else are intermediate Tensors.
import tensorflow as tf
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.nn.relu(tf.matmul(x, W) + b)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 6 / 20
![Page 14: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/14.jpg)
Core abstractions
This is a data flow graph.x is a Placeholder.W and b are Variables.
Everything else are intermediate Tensors.
import tensorflow as tf
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.nn.relu(tf.matmul(x, W) + b)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 6 / 20
![Page 15: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/15.jpg)
Core abstractions
This is a data flow graph.x is a Placeholder.W and b are Variables.Everything else are intermediate Tensors.
import tensorflow as tf
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.nn.relu(tf.matmul(x, W) + b)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 6 / 20
![Page 16: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/16.jpg)
Core abstractions
This is a data flow graph.x is a Placeholder.W and b are Variables.Everything else are intermediate Tensors.
import tensorflow as tf
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.nn.relu(tf.matmul(x, W) + b)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 6 / 20
![Page 17: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/17.jpg)
Core abstractions
This is a data flow graph.x is a Placeholder.W and b are Variables.Everything else are intermediate Tensors.
import tensorflow as tf
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.nn.relu(tf.matmul(x, W) + b)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 6 / 20
![Page 18: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/18.jpg)
Let’s compute that!
import numpy as np
batch = np.random.randn (128, 784)
with tf.Session () as sess:
sess.run(tf.initialize_all_variables ())
val = sess.run(C, feed_dict ={x: batch})
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 7 / 20
![Page 19: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/19.jpg)
Let’s compute that!
import numpy as np
batch = np.random.randn (128, 784)
with tf.Session () as sess:
sess.run(tf.initialize_all_variables ())
val = sess.run(C, feed_dict ={x: batch})
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 7 / 20
![Page 20: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/20.jpg)
Simple network example
import tensorflow as tf
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.matmul(x, W) + b
# C = tf.nn.relu(tf.matmul(x, W) + b)
y = tf.placeholder(tf.int64 , shape=[None])
xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(C, y)
loss = tf.reduce_mean(xentropy)
optimizer = tf.train.GradientDescentOptimizer (1e-3)
train_op = optimizer.minimize(loss)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 8 / 20
![Page 21: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/21.jpg)
Simple network example
import tensorflow as tf
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.matmul(x, W) + b
# C = tf.nn.relu(tf.matmul(x, W) + b)
y = tf.placeholder(tf.int64 , shape=[None])
xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(C, y)
loss = tf.reduce_mean(xentropy)
optimizer = tf.train.GradientDescentOptimizer (1e-3)
train_op = optimizer.minimize(loss)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 8 / 20
![Page 22: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/22.jpg)
Let’s optimize that!
import numpy as np
batch = np.random.randn (128, 784)
labels = np.random.randint (10, size =128)
with tf.Session () as sess:
sess.run(tf.initialize_all_variables ())
train_loss , _ = sess.run([loss , train_op],
feed_dict ={x: batch , y=labels })
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 9 / 20
![Page 23: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/23.jpg)
ConvNet example in TF Slim
import tensorflow.contrib.slim as slim
with slim.arg_scope ([slim.conv2d , slim.fully_connected],
activation_fn=tf.nn.relu ,
weights_initializer=tf.truncated_normal_initializer (0.0, 0.01),
weights_regularizer=slim.l2_regularizer (0.0005)):
net = slim.conv2d(net , 64, [3, 3], scope=’conv1’)
net = slim.max_pool2d(net , [2, 2], scope=’pool1’)
net = slim.conv2d(net , 128, [3, 3], scope=’conv2’)
net = slim.max_pool2d(net , [2, 2], scope=’pool2’)
net = slim.conv2d(net , 256, [3, 3], scope=’conv3’)
net = slim.max_pool2d(net , [2, 2], scope=’pool3’)
net = slim.fully_connected(net , 1024, scope=’fc1’)
net = slim.dropout(net , 0.5, scope=’dropout1 ’)
net = slim.fully_connected(net , 10, activation_fn=None ,
scope=’fc2’)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 10 / 20
![Page 24: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/24.jpg)
Too many GPUs?
import tensorflow as tf
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.nn.relu(tf.matmul(x, W) + b)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 11 / 20
![Page 25: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/25.jpg)
How to stop worrying and start using multi-GPU?
Model parallelism
import tensorflow as tf
with tf.device(’/gpu:1’):
x = tf.placeholder(tf.float32 ,
shape=[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
C = tf.nn.relu(tf.matmul(x, W) + b)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 12 / 20
![Page 26: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/26.jpg)
Data parallelism scheme
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 13 / 20
![Page 27: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/27.jpg)
Data parallelism code example
Data parallelism
import tensorflow as tf
with tf.device(’/cpu:0’):
x1 = tf.placeholder(tf.float32 ,
shape =[None , 784])
x2 = tf.placeholder(tf.float32 ,
shape =[None , 784])
W = tf.Variable(tf.random_normal ([784 , 10]))
b = tf.Variable(tf.zeros ([10]))
with tf.device(’/gpu:0’):
C1 = tf.nn.relu(tf.matmul(x1, W) + b)
with tf.device(’/gpu:1’):
C2 = tf.nn.relu(tf.matmul(x2, W) + b)
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 14 / 20
![Page 28: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/28.jpg)
Scopes in TF
Flexible system of hierarchical structures
arg scope — redefine default arguments for enclosed functions.
name scope — group intermediate Tensors together.
variable scope — facilitate variable reuse to build complicated graphswith tied weights (implies a name scope).
Examples of variable scope:
/myNetwork/convLayer2/weights
/myNetwork/convLayer2/BatchNorm/gamma
/vgg16/fc7/biases
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 15 / 20
![Page 29: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/29.jpg)
Scopes in TF
Flexible system of hierarchical structures
arg scope — redefine default arguments for enclosed functions.
name scope — group intermediate Tensors together.
variable scope — facilitate variable reuse to build complicated graphswith tied weights (implies a name scope).
Examples of variable scope:
/myNetwork/convLayer2/weights
/myNetwork/convLayer2/BatchNorm/gamma
/vgg16/fc7/biases
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 15 / 20
![Page 30: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/30.jpg)
Scopes in TF
Flexible system of hierarchical structures
arg scope — redefine default arguments for enclosed functions.
name scope — group intermediate Tensors together.
variable scope — facilitate variable reuse to build complicated graphswith tied weights (implies a name scope).
Examples of variable scope:
/myNetwork/convLayer2/weights
/myNetwork/convLayer2/BatchNorm/gamma
/vgg16/fc7/biases
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 15 / 20
![Page 31: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/31.jpg)
Scopes in TF
Flexible system of hierarchical structures
arg scope — redefine default arguments for enclosed functions.
name scope — group intermediate Tensors together.
variable scope — facilitate variable reuse to build complicated graphswith tied weights (implies a name scope).
Examples of variable scope:
/myNetwork/convLayer2/weights
/myNetwork/convLayer2/BatchNorm/gamma
/vgg16/fc7/biases
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 15 / 20
![Page 32: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/32.jpg)
Tensorboard demo
LIVEFasten your seat belts and such
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 16 / 20
![Page 33: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/33.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 34: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/34.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 35: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/35.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 36: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/36.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 37: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/37.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 38: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/38.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 39: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/39.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 40: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/40.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 41: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/41.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 42: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/42.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 43: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/43.jpg)
Other goodies
Queues and preprocessing: data loading, data augmentation, batchshuffling can be easily offloaded to TF threads.
Checkpoints: computational graph and variables can be easily saved ortransferred across the network.
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 17 / 20
![Page 44: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/44.jpg)
TensorFlow Serving
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 18 / 20
![Page 45: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/45.jpg)
TensorFlow Serving
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 18 / 20
![Page 46: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/46.jpg)
TensorFlow Serving
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 18 / 20
![Page 47: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/47.jpg)
Comparison
Caffe Theano Torch DL4J TensorFlow
RNN kind of Yes Yes Yes Yes
multi-GPU C++ only Yes Yes Yes Yes
multi-node No No kind of Yes Yes
API C++, Python Python Lua Java C++,Matlab Python
autodiff No Yes Recently No Yes
extensibility No Yes Yes ? Yes
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 19 / 20
![Page 48: TensorFlow: what and why? - Meetupfiles.meetup.com/18200471/meetup3_4.pdf · Python API, C++ core (Eigen), powerful visualization with TensorBoard, model deployment with TensorFlow](https://reader033.fdocuments.net/reader033/viewer/2022052010/6020b38241ec5b439b339672/html5/thumbnails/48.jpg)
Thank you!
Thank you for your time!Any questions?
Contact info: [email protected]
Konstantin Shmelkov (Grenoble Data Science Meetup)TensorFlow: what and why? 14 Sep 2016 20 / 20