A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX...

34
A Convolutional Attention Network for Extreme Summarization of Source Code

Transcript of A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX...

Page 1: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple

A Convolutional Attention Network for Extreme Summarization of Source

Code

Page 2: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 3: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 4: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 5: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 6: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 7: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 8: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 9: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 10: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 11: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 12: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 13: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 14: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 15: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple

ATTENTION MECHANISM

➤ “Attention: Withdrawal from some things in order to deal effectively with others” ~William James

Page 16: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 17: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 18: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 19: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 20: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 21: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 22: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 23: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 24: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 25: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple

LIBGDX

➤ Cross-platform game and visualization development framework

➤ Write code once and use it in multiple platforms

➤ As low level as you want

Page 26: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple

GHTORRENT PROJECT

➤ Monitors Github event time lines

➤ Gets the content and their dependencies

➤ Stores raw JSON to MONGODB

➤ Distributed

Page 27: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple

POS2VECThe "Pos2Vec" component is a Deep Belief Network that consists of 4 stacked autoencoders, that are trained layer by layer, unsupervised

Page 28: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple

SPEARMINTSpearmint is a package to perform Bayesian optimization

Page 29: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple

➤ Sample weight vectors with the first probability

Page 30: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 31: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 32: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 33: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple
Page 34: A Convolutional Attention Network for Extreme ...cs762/slides/RNN_Amini.pdf · LIBGDX Cross-platform game and visualization development framework Write code once and use it in multiple

THANK YOU➤ Sources:

➤ Lectures of Andrew NG

➤ Lectures of Geoffry Hinton

Neural Machine Translation by Jointly Learning to Align and Translate