ML with Tensorflow-new - GitHub Pages › ml › lec12.pdf · 2017-10-02 · Fei-Fei Li & Andrej...

Lecture 12 RNN

Sung Kim <[email protected]>http://hunkim.github.io/ml/

Sequence data

• We don’t understand one word only

• We understand based on the previous words + this word. (time series)

• NN/CNN cannot do this

Sequence data

• We don’t understand one word only

• We understand based on the previous words + this word. (time series)

• NN/CNN cannot do this

http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Lecture 10 - 8 Feb 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 10 - 8 Feb 201613

Recurrent Neural Network

x

RNN



x

RNN

yusually want to predict a vector at some time steps



x

RNN

yWe can process a sequence of vectors x by applying a recurrence formula at every time step:

new state old state input vector at some time step

some functionwith parameters W



x

RNN

yWe can process a sequence of vectors x by applying a recurrence formula at every time step:

Notice: the same function and the same set of parameters are used at every time step.

/HFWXUH�� )HE��)HL�)HL�/L��$QGUHM�.DUSDWK\��-XVWLQ�-RKQVRQ)HL�)HL�/L��$QGUHM�.DUSDWK\��-XVWLQ�-RKQVRQ /HFWXUH�� )HE��

�9DQLOOD��5HFXUUHQW�1HXUDO�1HWZRUN

[

511

\

7KH�VWDWH�FRQVLVWV�RI�D�VLQJOH�³KLGGHQ´�YHFWRU�K�


Character-levellanguage modelexample

Vocabulary:[h,e,l,o]

Example trainingsequence:“hello”

x

RNN

y

RNN applications

• Language Modeling

• Speech Recognition

• Machine Translation

• Conversation Modeling/Question Answering

• Image/Video Captioning

• Image/Music/Dance Generation

http://jiwonkim.org/awesome-rnn/

https://github.com/TensorFlowKR/awesome_tensorflow_implementations


Recurrent Networks offer a lot of flexibility:

Vanilla Neural Networks



e.g. Image Captioningimage -> sequence of words



e.g. Sentiment Classificationsequence of words -> sentiment



e.g. Machine Translationseq of words -> seq of words



e.g. Video classification on frame level

Multi-Layer RNN

Training RNNs is challenging

• Several advanced models- Long Short Term Memory (LSTM)- GRU by Cho et al. 2014

Next

RNN in TensorFlow

ML with Tensorflow-new - GitHub Pages › ml › lec12.pdf · 2017-10-02 · Fei-Fei Li & Andrej...

Documents

Transcript of ML with Tensorflow-new - GitHub Pages › ml › lec12.pdf · 2017-10-02 · Fei-Fei Li & Andrej...

ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab12.pdfFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 10 -21 8 Feb 2016 Character-level language model example Vocabulary:

Lecture 8: Spatial Localization and Detectioncs231n.stanford.edu/slides/2016/winter1516_lecture8.pdf · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 8 - 1 1 Feb 2016 Lecture

Lecture 11 - Stanford Universitycs231n.stanford.edu/slides/2016/winter1516_lecture11.pdf · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 11 - 47 17 Feb 2016 The power of

Lecture 2: Image Classification pipeline - Artificial Intelligencevision.stanford.edu/teaching/cs231n/slides/2015/lecture2.pdf · Fei-Fei Li & Andrej Karpathy Lecture 2 - 32 7 Jan

DenseCap: Fully Convolutional ... - cv-foundation.org · DenseCap: Fully Convolutional Localization Networks for Dense Captioning Justin Johnson∗ Andrej Karpathy∗ Li Fei-Fei Department

Deep Fragment Embeddings for Bidirectional Image Sentence ... · Deep Fragment Embeddings for Bidirectional Image Sentence Mapping Andrej Karpathy Armand Joulin Li Fei-Fei Department

CS60010: Deep Learningsudeshna/courses/DL18/... · Deep Visual -Semantic Alignments for Generating Image Descriptions, Karpathy and Fei -Fei Show and Tell: A Neural Image Caption

Linear Classification Lecture 3 - Artificial Intelligencevision.stanford.edu/teaching/cs231n/slides/2015/lecture3.pdf · Fei-Fei Li & Andrej Karpathy Lecture 2 - 20 7 Jan 2015 2.

Lecture 12 - Stanford University CS231n: Convolutional ...cs231n.stanford.edu/slides/2016/winter1516_lecture12.pdf · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 12 - 22

Abstract - arXiv · 2020. 4. 16. · 2016; Karpathy & Fei-Fei, 2015; Mao, Xu, Yang, Wang, Huang, & Yuille, 2014). However, attending to ﬁne-grained and important fragments of visual

Lecture 12 - Stanford Universityvision.stanford.edu/.../cs231n/slides/2016/winter1516_lecture12.pdf · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 12 ... Fei-Fei Li & Andrej

DenseCap: Fully Convolutional Localization Networks for ... · DenseCap: Fully Convolutional Localization Networks for Dense Captioning Justin Johnson Andrej Karpathy Li Fei-Fei Department

Obj2Text: Generating Visually Descriptive Language from ...text. bitrary images has received considerable atten-tion (Vinyals et al.,2015;Karpathy and Fei-Fei, 2015;Mao et al.,2015).

Convolutional Neural Networks - Alan Ritteraritter.github.io/courses/5523_slides/cnn.pdfConvolutional Neural Networks (First without the brain stuff) Fei-Fei Li & Andrej Karpathy &

Case Study: LeNet-5€¦ · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -60 27 Jan 2016 Case Study: LeNet-5 [LeCun et al., 1998] Conv filters were 5x5, applied at stride

Recurrent Neural Networks - Artificial Intelligencevision.stanford.edu/teaching/cs231n/slides/2016/... · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 10 - 38 8 Feb 2016

Lecture 3: Loss functions and Optimizationcs231n.stanford.edu/slides/2016/winter1516_lecture3.pdf · · 2016-01-12Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 3 - 2 11

Lecture 4: Backpropagation and Neural Networks part 1vision.stanford.edu/teaching/cs231n/slides/2016/winter1516_lecture… · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture

Classification and Semantic Segmentationyboykov/Courses/cs898/Lectures/lec5... · Semantic Segmentation Most slides are from Fei-Fei Li, Justin Johnson, Andrej Karpathy, SerenaYeung,

ML with Tensorflow-new - GitHub Pageshunkim.github.io/ml/lec12.pdf · e.g. Machine Translation seq of words -> seq of words. Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture