Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet
-
Upload
artifacia -
Category
Technology
-
view
282 -
download
0
Transcript of Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet
![Page 1: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/1.jpg)
Introduction to Recurrent Neural Networks with Application to Sentiment Analysis
by Rajarshee Mitra, Research Engineer (NLP) , Artifacia(@rajarshee_mitra)
November 19, 2016
![Page 2: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/2.jpg)
AI Meet|
Agenda
1. What is AI ? 2. What is NLU ? Why is it hard ?3. Introduction to Neural Networks.4. Introduction to Recurrent Neural Networks (RNN).5. Application of RNN models.6. Variants of RNN.7. Sequence to Sequence Learning.8. Sentiment Analysis - an application.9. Food for Thought.
![Page 3: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/3.jpg)
AI Meet|
What is AI
AI is the ability of software to mimic human brain and perform human-like abilities such as understanding emotions and meanings from text, handling ambiguities, recognizing objects etc.
![Page 4: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/4.jpg)
AI Meet|
What is NLU? Why is it hard ?
Natural Language Understanding is the ability to process, understand and generate human languages (to create some action or intent).
● Language contains ambiguities.“I am looking at the elephant in white pyajamas”
● Modification of a single word (insertion, deletion) changes the meaning of the whole sentence.
● Context plays a serious role in language understanding.
![Page 5: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/5.jpg)
AI Meet|
Neural Network - An introduction
![Page 6: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/6.jpg)
AI Meet|
Neural Network - An introduction
INPUT OUTPUT TARGET
am He, running I, going
looked I, am I, at
sofa The, is The, is
Neural Language Modelling - skip gram model
![Page 7: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/7.jpg)
AI Meet|
Neural Network - An introduction
Loss Functions:
1. Absolute Difference2. Root Mean Square3. Cross Entropy or Log Loss
![Page 8: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/8.jpg)
AI Meet|
Recurrent Neural Nets
![Page 9: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/9.jpg)
AI Meet|
Application of RNN Models
1. Sentiment Analysis2. Language Modelling3. Translation4. Conversational Agents5. Language Generation6. Image Captioning7. Text Summarization
![Page 10: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/10.jpg)
AI Meet|
Some Interesting Variants of Neural Network
1. Long Short Term Memory Networks
2. Gated Recurrent Unit.
3. End to End networks - Sequence to Sequence Learning
4. Memory Networks.
![Page 11: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/11.jpg)
AI Meet|
Sequence to Sequence Learning
![Page 12: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/12.jpg)
AI Meet|
Sentiment Analysis - An ApplicationThe process of computationally identifying and categorizing opinions expressed in a piece of text, especially in order to determine whether the writer's attitude towards a particular topic, product, etc. is positive, negative, or neutral.
![Page 13: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/13.jpg)
AI Meet|
Sentiment Analysis - An Application
1. We can treat the last output vector as our predicted sentiment.
2. We calculate loss between our output and target vector which contains the actual sentiment.
3. We update our model accordingly to minimize the loss.
4. A successfully learnt model will automatically predict sentiments of unseen sentences.
![Page 14: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/14.jpg)
AI Meet|
Food for Thought
As we are more approaching towards linking concepts of neuroscience with mathematical concepts of Deep Learning, I imagine a system which might have following components :
1. A processor or generator - An RNN that process sentence word by word or generates sentence in the same way.
2. A memory that will facilitate read / write operations.3. A some variant of feed forward neural network that will act
between the processor and the memory and will determine which neurons to activate, what to read and write.
![Page 15: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/15.jpg)
AI Meet|
Further Reading
1. http://karpathy.github.io/2015/05/21/rnn-effectiveness/
2. http://colah.github.io/posts/2015-08-Understanding-LSTMs/
3. http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf
From Our Blog:- http://research.artifacia.com/learn-deep-learning-the-hard-way
![Page 16: Introduction to Recurrent Neural Network with Application to Sentiment Analysis - Artifacia AI Meet](https://reader034.fdocuments.net/reader034/viewer/2022052609/586f89921a28ab54768b606f/html5/thumbnails/16.jpg)
THANK YOUJoin
meetup.com/Artifacia-AI-Meet/