ML with Tensorflow-new - GitHub Pageshunkim.github.io/ml/lec12.pdf · e.g. Machine Translation seq...

Lecture 12 RNN

Sung Kim <hunkim+mr@gmail.com>http://hunkim.github.io/ml/

Sequence data

• We don’t understand one word only

• We understand based on the previous words + this word. (time series)

• NN/CNN cannot do this

Sequence data

• We don’t understand one word only

• We understand based on the previous words + this word. (time series)

• NN/CNN cannot do this

http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Lecture 10 - 8 Feb 2016Fei-Fei Li & Andrej Karpathy & Justin JohnsonFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 10 - 8 Feb 201613

Recurrent Neural Network

yusually want to predict a vector at some time steps

yWe can process a sequence of vectors x by applying a recurrence formula at every time step:

new state old state input vector at some time step

some functionwith parameters W

yWe can process a sequence of vectors x by applying a recurrence formula at every time step:

Notice: the same function and the same set of parameters are used at every time step.

/HFWXUH�� )HE��)HL�)HL�/L��$QGUHM�.DUSDWK\��-XVWLQ�-RKQVRQ)HL�)HL�/L��$QGUHM�.DUSDWK\��-XVWLQ�-RKQVRQ /HFWXUH�� )HE��

�9DQLOOD��5HFXUUHQW�1HXUDO�1HWZRUN

7KH�VWDWH�FRQVLVWV�RI�D�VLQJOH�³KLGGHQ´�YHFWRU�K�

http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Character-levellanguage modelexample

Vocabulary:[h,e,l,o]

Example trainingsequence:“hello”

RNN applications

• Language Modeling

• Speech Recognition

• Machine Translation

• Conversation Modeling/Question Answering

• Image/Video Captioning

• Image/Music/Dance Generation

http://jiwonkim.org/awesome-rnn/

https://github.com/TensorFlowKR/awesome_tensorflow_implementations

Recurrent Networks offer a lot of flexibility:

Vanilla Neural Networks

e.g. Image Captioningimage -> sequence of words

e.g. Sentiment Classificationsequence of words -> sentiment

e.g. Machine Translationseq of words -> seq of words

e.g. Video classification on frame level

Multi-Layer RNN

Training RNNs is challenging

• Several advanced models- Long Short Term Memory (LSTM)- GRU by Cho et al. 2014

RNN in TensorFlow

ML with Tensorflow-new - GitHub Pageshunkim.github.io/ml/lec12.pdf · e.g. Machine Translation seq...

Documents

Transcript of ML with Tensorflow-new - GitHub Pageshunkim.github.io/ml/lec12.pdf · e.g. Machine Translation seq...

FEI -8 FEI 0.5 - 10 14.0 16.0 8.0 10.0 [RoHS2 (AWG) (22-20) · 14.0 16.0 8.0 10.0 [rohs2 (awg) (22-20) 5- (20-19) (18) (16) fei o. fei fei fei fei 75-10 16.0 14.0 16.0 14.0 16.0 10.0

crowdsourcing, benchmarking & other cool thingsimage-net.org/papers/ImageNet_2010.pdf · crowdsourcing, benchmarking & other cool things Fei-Fei Li (publish under L. Fei-Fei) Computer

DECISION of the FEI TRIBUNAL dated 3 June 2020 FEI ... · effective 1 January 2016. Veterinary Regulations (“VRs”), 13th edition, effective 1 January 2016, Art. 1046 and seq.

Defense-Fei Fei

FEI Prohibited Substances List - FEI Clean Sport

Lecture3 xing fei-fei

Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 2 ...cs231n.stanford.edu/slides/2017/cs231n_2017_lecture2.pdf · All pixels change when ... Fei-Fei Li & Justin Johnson & Serena

ML with Tensorflow-new - GitHub Pageshunkim.github.io/ml/lec11.pdf · 2017-10-02 · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -68 27 Jan 2016 Case Study: AlexNet [Krizhevsky

Fei endurance

Grile Fei

FEI Brochure

RNA-seq analysis worksop - · PDF fileRNA-seq analysis worksop Zhangjun Fei Boyce Thompson Institute for Plant Research USDA Robert W. Holley Center for Agriculture and Health Cornell

Fei-Fei Li & Justin Johnson & Serena Yeungcs231n.stanford.edu/slides/2019/cs231n_2019_lecture07.pdf · 2019-04-23 · Fei-Fei Li & Justin Johnson & Serena Yeung Lecture 7 - 4 April

FEI Presentation

ML with Tensorflow-new - GitHub Pages · Fei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 7 -68 27 Jan 2016 Case Study: AlexNet [Krizhevsky et al. 2012] Full (simplified) AlexNet

ML with Tensorflow lab - GitHub Pageshunkim.github.io/ml/lab12.pdfFei-Fei Li & Andrej Karpathy & Justin Johnson Lecture 10 -21 8 Feb 2016 Character-level language model example Vocabulary:

FEI Schedule

NTEGROVANDÝ OPRASVYN SÍTÉMO LOMOUCKÉHOK RAJE … · ml ml ml ml ml ml ml ml ml ml ml ml ml ml ml ml ml ml m ml ml ml ml lm ml ml ml ml lm ml ml ml ml ml ml lm ml ml ml lm ml lm

Cos 429: Face Detection (Part 2) Viola-Jones and AdaBoost Guest Instructor: Andras Ferencz (Your Regular Instructor: Fei-Fei Li) Thanks to Fei-Fei Li,

Descriptors II - courses.cs.washington.edu• Fei-Fei & Perona, 2005 • Interest point detector • Csurka et al. 2004 • Fei-Fei & Perona, 2005 • Sivic et al. 2005 • Other methods