Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks
-
Upload
amazon-web-services -
Category
Documents
-
view
530 -
download
0
Transcript of Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks
![Page 1: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/1.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Vikram Madan, Sr. Product Manager
November 14, 2017
Sentiment Analysis Using
Apache MXNet and Gluon
![Page 2: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/2.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Sentiment analysis
Sentiment analysis is a natural language processing (NLP) task for classifying
text or speech as some specified sentiment
Brand Sentiment Political Sentiment Customer Satisfaction
![Page 3: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/3.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Performing sentiment analysis
Labelled Data
for Training
Word
Embedding
Train Deep
Neural Network
Deploy Model
for Prediction
Focus of Today’s Discussion
![Page 4: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/4.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
IMBD dataset for movie reviews
Positive
“The impetuous pace of the film
is at one with its moral
shamelessness, and, without
thinking, we sign up for both”
“What a magical movie”
“It is a film so awe-inspiringly
wooden that it’s basically a fire-risk”
“An explosion in a stupid factory”
“Love him, hated Hur”
Negative
25,000 movie reviews: 12,500 positive & 12,500 negative
![Page 5: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/5.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Converting words to vectors
Two Major Approaches: word2vec & GloVe
![Page 6: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/6.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Converting words to vectors
Training Co-Occurrence Data Output Vectors
![Page 7: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/7.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Converting words to vectors
Example: 3-Dimensional Word Vectors
![Page 8: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/8.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
GloVe: Global Vectors for Word Representation
![Page 9: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/9.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Deep Neural Networks
![Page 10: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/10.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Recurrent Neural Networks (RNNs)
![Page 11: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/11.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Long Short-Term Memory (LSTM) networks
![Page 12: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/12.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Illustrative Example (Part 1 of 2)
Edwin Chen Blog (http://blog.echen.me/)
![Page 13: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/13.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Illustrative Example (Part 2 of 2)
Edwin Chen Blog (http://blog.echen.me/)
![Page 14: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/14.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Gluon API Specification
Gluon is a technical specification providing a programming
interface that: (1) simplifies development of deep learning models,
(2) provides greater flexibility in building neural networks, and (3)
offers high performance
Simplicity Flexibility Performance
![Page 15: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/15.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Gluon API is Available in Apache MXNet
Others…
Development of the Gluon API specification was
a collaboration between AWS and Microsoft
Available Now
Coming Soon
Gluon API is implemented in Apache
MXNet and soon will be in CNTK
![Page 16: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/16.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Advantages of the Gluon API
Simple, Easy-to-
Understand Code
Flexible, Imperative
Structure
Dynamic Graphs
High Performance
Neural networks can be defined using simple, clear, concise code
Plug-and-play neural network building blocks – including predefined layers,
optimizers, and initializers
Eliminates rigidity of neural network model definition and brings together
the model with the training algorithm
Intuitive, easy-to-debug, familiar code
Neural networks can change in shape or size during the training process to
address advanced use cases where the size of data fed is variable
Important area of innovation in Natural Language Processing (NLP)
There is no sacrifice with respect to training speed
When it is time to move from prototyping to production, easily cache neural
networks for high performance and a reduced memory footprint
![Page 17: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/17.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Simple, Easy-to-Understand Code
Use plug-and-play neural network building blocks, including
predefined layers, optimizers, and initializers
# First step is to initialize your model
net = gluon.nn.Sequential()
# Then, define your model architecture
with net.name_scope():
net.add(gluon.nn.Dense(256, activation="relu")) # 1st layer (256 nodes)
net.add(gluon.nn.Dense(256, activation="relu")) # 2nd hidden layer
net.add(gluon.nn.Dense(num_outputs)) # Output layer
![Page 18: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/18.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Flexible, Imperative Structure
Prototype, build, and debug neural networks more easily with a
fully imperative interface
epochs = 10
for e in range(epochs):
for i, batch in enumerate(train_data):
with autograd.record(): # Start recording the derivatives
output = net(data) # the forward iteration
loss = softmax_cross_entropy(output, label)
loss.backward()
trainer.step(data.shape[0])
![Page 19: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/19.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Dynamic Graphs
Build neural networks on the fly for use cases where neural
networks must change in size and shape during model training
def forward(self, F, inputs, tree):
children_outputs = [self.forward(F, inputs, child)
for child in tree.children]
#Recursively builds the neural network based on each input sentence’s
#syntactic structure during the model definition and training process
![Page 20: Sentiment Analysis Using Apache MXNet and Gluon - AWS Online Tech Talks](https://reader033.fdocuments.net/reader033/viewer/2022051521/5a6478bf7f8b9a4c568b45dd/html5/thumbnails/20.jpg)
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
High Performance
Easily cache the neural network to achieve high performance
when speed becomes more important than flexibility
net = nn.HybridSequential()
with net.name_scope():
net.add(nn.Dense(256, activation="relu"))
net.add(nn.Dense(128, activation="relu"))
net.add(nn.Dense(2))
net.hybridize()