Lab 11 CNN

Sung Kim <hunkim+ml@gmail.com>http://hunkim.github.io/ml/

https://github.com/nlintz/TensorFlow-Tutorials

CNN

http://parse.ele.tue.nl/cluster/2/CNNArchitecture.jpg

MNIST 28x28x1 image

32 filters (3x3x1)

Convolutional layers

28x28x1 image

32 filters (3x3x1)w=tf.Variable(tf.random_normal([3,3,1,32], stddev=0.01))

Convolutional layers

28x28x1 image

32 filters (3x3x1)w=tf.Variable(tf.random_normal([3,3,1,32], stddev=0.01))

Convolutional layers

28x28x1 image

32 filters (3x3x1)activation maps

(?, ?, 32)

Convolutional layers

ReLU

Pooling layer (sampling)

conv layer

l1 = tf.nn.max_pool(c1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

Lecture 7 - 27 Jan 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 - 27 Jan 201655

1 1 2 4

5 6 7 8

3 2 1 0

1 2 3 4

Single depth slice

x

y

max pool with 2x2 filters and stride 2 6 8

3 4

MAX POOLING

Pooling layer (sampling)

conv layer

l1 = tf.nn.max_pool(c1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

28x28x32

? x ? x 32

Tensor("MaxPool:0", shape=(?, 14, 14, 32), dtype=float32)

Shape not sure? Print tensor

Tensor("Conv2D:0", shape=(?, 28, 28, 32), dtype=float32)

X = trX.reshape(-1, 28, 28, 1)

X = trX.reshape(-1, 28, 28, 1)

dropout

CNN

http://parse.ele.tue.nl/cluster/2/CNNArchitecture.jpg

Fully connected net

cost and optimization

https://www.tensorflow.org/versions/r0.8/api_docs/python/train.html#RMSPropOptimizer

Other TF optimizers

https://www.tensorflow.org/versions/r0.8/api_docs/python/train.html#RMSPropOptimizer

Train and testing

Train and testing 0 128

128 256 256 384 384 512

Train and testing

https://github.com/nlintz/TensorFlow-Tutorials