Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural...
Transcript of Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural...
![Page 1: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/1.jpg)
1
Fine-Grained Opinion Mining: Current Trend and Cutting-Edge Dimensions
Wenya Wang, Jianfei Yu, Sinno Jialin Pan and Jing JIang
Nanyang Technological University and
Singapore Management University
![Page 2: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/2.jpg)
1
Part III
Target-Oriented Sentiment Classification
![Page 3: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/3.jpg)
2
Outline
§ Background
§ Methodology
§ Summary
![Page 4: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/4.jpg)
2
Background § Sentence/Document-Level Sentiment Classification
– Input • A sentence or document
– Output • Overall sentiment polarity
– Positive, Negative, Neutral
– Example
The movie was fabulous, and the characters are quite engaging!
The restaurant was horrible, and their service was also poor!
![Page 5: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/5.jpg)
2
Background § Target-oriented Sentiment Classification (TSC)
– Input • A sentence or document • An opinion target
– 1. Aspect Term (Aspect-Level Sentiment Classification)
– 2. Aspect Category (Aspect Category-Based Sentiment Classification)
– 3. Target Entity (Entity-Level Sentiment Classification)
– Output • Sentiment polarity towards the opinion target
– Positive, Negative, Neutral
![Page 6: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/6.jpg)
Background
2
§ Examples (Product Review) – Aspect-Level Sentiment Classification
The [fish] was rather over cooked, but the [staff] was quite nice!
Ø sentiment over fish: negative
Ø sentiment over staff: positive
![Page 7: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/7.jpg)
Background
2
§ Examples (Product Review) – Aspect Category-Based Sentiment Classification
The [fish] was rather over cooked, but the [staff] was quite nice!
Ø sentiment over food: negative Ø sentiment over service: positive Ø sentiment over ambience: N.A Ø sentiment over price: N.A Ø sentiment over miscellaneous: N.A
![Page 8: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/8.jpg)
Background
2
§ Examples (Tweet) – Entity-Level Sentiment Classification
Ø sentiment over 400m T37: neutral Ø sentiment over Georgina Hermitage: positive
[Georgina Hermitage] is a #one2watch since she broke the [400m T37] WR!
![Page 9: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/9.jpg)
2
Outline
§ Background
§ Methodology
§ Summary
![Page 10: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/10.jpg)
Methodology – Big Picture
Use opinion targets as queries
to MemNet
Supervised Machine Learning Methods
RNN
Manually create features related to
opinion targets
5
Deep Learning Feature Engineering
Linear Classifier CNN MemNet Recursive NN
Adapt standard model architectures to be sensitive to opinion targets
BERT
![Page 11: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/11.jpg)
2
Outline § Background § Methodology
– Linear Classifier – Recursive Neural Network – Memory Network – CNN-based Methods – RNN-based Methods – BERT-based Methods
§ Summary
![Page 12: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/12.jpg)
Linear Classifier § Extract various features
Methodology
8
Duy-Tin Vo, and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. In Proceedings of IJCAI .
![Page 13: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/13.jpg)
Linear Classifier § Extract various features
– Target-dependent features from words filtered by sentiment lexicon
Methodology
8
Duy-Tin Vo, and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. In Proceedings of IJCAI .
![Page 14: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/14.jpg)
Linear Classifier § Extract various features
– Target-dependent features from the left context, right context, and target, respectively
Methodology
8
Duy-Tin Vo, and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. In Proceedings of IJCAI .
![Page 15: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/15.jpg)
Linear Classifier § Extract various features
– Full tweet features
Methodology
8
Duy-Tin Vo, and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. In Proceedings of IJCAI .
![Page 16: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/16.jpg)
Linear Classifier § Extract various features
– Feed the concatenated features to a discriminative classifier
• SVM
Methodology
8
Duy-Tin Vo, and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. In Proceedings of IJCAI.
![Page 17: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/17.jpg)
2
Outline § Background § Methodology
– Linear Classifier – Recursive Neural Network – Memory Network – CNN-based Methods – RNN-based Methods – BERT-based Methods
§ Summary
![Page 18: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/18.jpg)
Recursive Neural Network § Dependency Tree-based Approach
– AdaRNN • propagate sentiment information to the target node in a bottom-up manner
Methodology
8
Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive Recursive Neural Network for Target-dependent TwitterSentiment Classification. In Proceedings of ACL, 49-54.
![Page 19: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/19.jpg)
Recursive Neural Network § Dependency Tree-based Approach
– AdaRNN • propagate sentiment information to the target node in a bottom-up manner
Methodology
8
Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive Recursive Neural Network for Target-dependent TwitterSentiment Classification. In Proceedings of ACL, 49-54.
![Page 20: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/20.jpg)
Recursive Neural Network § Dependency + Constituent tree-based Approach
– PhraseRNN
Methodology
8
Thien Hai Nguyen, and Kiyoaki Shirai. 2015. PhraseRNN: Phrase Recursive Neural Network for Aspect-based Sentiment Analysis. In Proceedings of EMNLP, 2509-2514.
![Page 21: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/21.jpg)
Recursive Neural Network § Dependency + Constituent tree-based Approach
– PhraseRNN
Methodology
8
Thien Hai Nguyen, and Kiyoaki Shirai. 2015. PhraseRNN: Phrase Recursive Neural Network for Aspect-based Sentiment Analysis. In Proceedings of EMNLP, 2509-2514.
![Page 22: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/22.jpg)
2
Outline § Background § Methodology
– Linear Classifier – Recursive Neural Network – Memory Network – CNN-based Methods – RNN-based Methods – BERT-based Methods
§ Summary
![Page 23: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/23.jpg)
Memory Network § MemNet
– Word embedding of target words as queries to MemNet
Methodology
8
Duyu Tang, Bing Qin, Ting Liu, et al. Aspect level sentiment classification with deep memory network. In EMNLP , 2016.
The Model Architecture of MemNet
![Page 24: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/24.jpg)
2
Outline § Background § Methodology
– Linear Classifier – Recursive Neural Network – Memory Network – CNN-based Methods – RNN-based Methods – BERT-based Methods
§ Summary
![Page 25: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/25.jpg)
CNN-based Methods § GCN (Gated Convolutional Networks)
– Incorporate gate mechanism to be sensitive to be opinion targets
Methodology
8
Xue, Wei, and Tao Li. "Aspect Based Sentiment Analysis with Gated Convolutional Networks.” In Proceedings of ACL 2018.
Model I. GCN for Aspect Category-based Sentiment Classification
![Page 26: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/26.jpg)
CNN-based Methods § GCN (Gated Convolutional Networks)
– Incorporate gating mechanism to be sensitive to be opinion targets
Methodology
8
Xue, Wei, and Tao Li. "Aspect Based Sentiment Analysis with Gated Convolutional Networks.” In Proceedings of ACL 2018.
Model II. GCN for Aspect-Level Sentiment Classification
![Page 27: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/27.jpg)
2
Outline § Background § Methodology
– Linear Classifier – Recursive Neural Network – Memory Network – CNN-based Methods – RNN-based Methods – BERT-based Methods
§ Summary
![Page 28: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/28.jpg)
RNN-based Methods § GRU
– Gating Mechanism
Methodology
8
Meishan Zhang, Yue Zhang, and Duy-Tin Vo. Gated Neural Networks for Targeted Sentiment Analysis. In Proceedings of AAAI 2016.
![Page 29: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/29.jpg)
RNN-based Methods § LSTM
– Sentence Encoding
Methodology
8
Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. Effective LSTMs for target-dependent sentiment classification. In COLING, 2016.
The Model Architecture of TD-LSTM and TC-LSTM
![Page 30: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/30.jpg)
RNN-based Methods § LSTM
– Attention Mechanism
Methodology
8
Yequan Wang, Minlie Huang, Li Zhao, et al. Attention-based LSTM for aspect-level sentiment classification. In EMNLP, 2016.
The Model Architecture of AE-LSTM
![Page 31: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/31.jpg)
RNN-based Methods § LSTM
– IAN • Interactive Attention Mechanism
Methodology
8
Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. Interactive attention networks for aspect-level sentiment classification. In IJCAI, 2017.
The Model Architecture of IAN
![Page 32: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/32.jpg)
RNN-based Methods § LSTM
– RAM • Position-based Weighting Strategy • Multi-Hop Attention Mechanism
Methodology
8
Peng Chen, Zhongqian Sun, Lidong Bing, and Wei Yang. Recurrent attention network on memory for aspect sentiment analysis. In EMNLP, 2017.
The Model Architecture of RAM
![Page 33: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/33.jpg)
RNN-based Methods § LSTM
– TNet
Methodology
8
Xin Li, Lidong Bing, Wai Lam, and Bei Shi. Transformation networks for target-oriented sentiment classification. In ACL, 2018.
Generating contextualized word representations
Refine word-level representations with the input target and the
contextual information
Introduce position information to detect the most salient features more accurately
The Model Architecture of TNet
![Page 34: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/34.jpg)
2
Outline § Background § Methodology
– Linear Classifier – Recursive Neural Network – Memory Network – CNN-based Methods – RNN-based Methods – BERT-based Methods
§ Summary
![Page 35: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/35.jpg)
BERT-based Methods Methodology
8
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI, 2019.
§ Feed the transformed sentence to BERT
Input Embedding
Self Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Sent
ence
Enc
oder
LS×
Pooling & Linear & Softmax
POSITIONAL ENCODING
positive
[CLS] $T$ is a #one2watch since she broke the 400m T37 WR. [SEP] Georgina Hermitage [SEP]
Input Embedding
Self Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Sent
ence
Enc
oder
LS×
Pooling & Linear & Softmax
POSITIONAL ENCODING
neutral
[CLS] Georgina Hermitage is a #one2watch since she broke the $T$ WR. [SEP] 400m T37 [SEP]
[Georgina Hermitage] is a #one2watch since she broke the [400m T37] WR!
![Page 36: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/36.jpg)
2
Outline
§ Background
§ Methodology
§ Summary
![Page 37: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/37.jpg)
Summary
5
§ Three Benchmark Datasets
– Laptop, Restaurant are from SemEval-2014 – Twitter-2014 from (Dong et al. ACL 2014) – Another two Restaurant datasets from SemEval-2015,
SemEval-2016
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI, 2019.
Data Set #Training Samples #Test Samples POS NEG NEU POS NEG NEU
Laptop 980 858 454 340 128 171
Restaurant 2159 800 623 730 195 196
Twitter-2014 1567 1563 3127 147 147 346
![Page 38: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/38.jpg)
Summary
5
§ Experimental Results on Three Benchmark Datasets
Method Laptop Restaurant Twitter-2014
Accuracy Macro-F1 Accuracy Macro-F1 Accuracy Macro-F1
SVM 70.49 - 80.16 - 63.40 63.30
AE-LSTM 68.90 - 76.60 - - -
IAN 72.10 - 78.60 - - -
TD-LSTM 71.83 68.43 78.00 66.73 66.62 64.01
MemNet 70.33 64.09 78.16 65.83 68.50 66.91
RAM 75.01 70.51 79.79 68.86 71.88 70.33
TNet-LF 76.01 71.47 80.79 70.84 74.68 73.36
TNet-AS 76.54 71.75 80.69 71.27 74.97 73.60
MGAN 75.39 72.47 81.25 71.94 72.54 70.81
BERT 76.96 73.67 84.29 77.22 75.14 74.15
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI, 2019.
![Page 39: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/39.jpg)
Summary
5
Supervised Machine Learning Methods
RNN
Deep Learning Feature Engineering
Linear Classifier CNN Recursive NN MemNet BERT
Pros: Simple and fast to train Cons: Many feature engineering efforts
Before 2014
Pros: make use of syntactic information Cons: 1. Noisy syntactic tree; 2. Gradient Vanishing
2014-2015
Pros: Good performance Comparison 1. Training/Test Time: MemNet ≈ CNN < RNN 2. Performance: RNN > CNN ≈ MemNet
2016-now 2019
Best results
Pros: Better results Cons: Lack of good
Explanation
![Page 40: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/40.jpg)
1
Part V
Cutting-Edge Dimensions of Fine-Grained Opinion Mining
![Page 41: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/41.jpg)
2
Outline
§ Transfer Learning
§ Multi-Task Learning
§ Multimodal Learning
§ Summary
![Page 42: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/42.jpg)
2
Outline
§ Transfer Learning – Cross-Domain – Cross-Lingual – Short Summary
§ Multi-Task Learning § Multimodal Learning § Summary
![Page 43: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/43.jpg)
Cross-Domain § Background
– Popular Methods for Fine-Grained Opinion Mining • Supervised Machine Learning (NN)
Transfer Learning
8
Machine Learning System
Large amount of training data
Classifier
![Page 44: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/44.jpg)
Cross-Domain § Background
– Real Scenario • Limited or no Labeled Data for many domains
Transfer Learning
8
Machine Learning System
Weak Classifier
Machine Learning System
Knowledge Transfer
Strong Classifier
Ø Domain Adaptation: recognize and apply knowledge and skills learned in previous domain to novel domains
Target domain
Source domain
![Page 45: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/45.jpg)
Cross-Domain § Background
– Challenge of Domain Adaptation
Transfer Learning
8
Training Data
Test Data
Movie
Movie
78%
Opinion Target Extraction
Model
![Page 46: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/46.jpg)
Cross-Domain § Background
– Challenge of Domain Adaptation
Transfer Learning
8
Training Data
Test Data
Movie
Movie
78%
Digital Device
45%
(Source Domain)
(Target Domain)
Opinion Target Extraction
Model
![Page 47: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/47.jpg)
Cross-Domain § Background
– Reasons behind performance drop
Transfer Learning
8
Movie (source domain) Digital Device (target domain)
The [movie] is great. The [camera] is excellent. I really like his [characters]. I highly recommend this [laptop].
The [plot] is quite dull. The [Mac OS] is quite fast.
Ø Opinion targets in the source domain movie, characters, plot
Ø Opinion targets in the target domain camera, laptop, Mac OS
![Page 48: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/48.jpg)
Cross-Domain § Background
– General Solution • Learn a shared representation across domains
Transfer Learning
8
Domain-Independent Auxiliary Tasks
Label/Unlabeled
Source
Unlabeled Target
Learn Mapping
Training Set of Source Domain
Test Set of Target Domain
![Page 49: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/49.jpg)
Cross-Domain § Cross-Domain Opinion Target Extraction
– Domain-Independent Auxiliary Task • Syntactic structures are shared across domains.
Transfer Learning
8
Ying Ding, Jianfei Yu, and Jing Jiang. Recurrent neural networks with auxiliary labels for crossdomain opinion target extraction. In AAAI 2017.
(Source Domain) (Target Domain)
![Page 50: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/50.jpg)
Cross-Domain § Cross-Domain Opinion Target Extraction
– The same task as our Auxiliary Tasks • Unsupervised Extraction Method
Transfer Learning
8
Ying Ding, Jianfei Yu, and Jing Jiang. Recurrent neural networks with auxiliary labels for crossdomain opinion target extraction. In AAAI 2017.
great bad …
Sentiment Word List
Step 1 Extract Target/Aspect
Words
Step 1 Extract More Opinion
Words
Step 1 Extract Target/Aspect
Words
Step 1 Extract More Opinion
Words
(Source Domain)
(Target Domain)
Syntactic rule
Syntactic rule
Syntactic rule
Syntactic rule
![Page 51: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/51.jpg)
Cross-Domain § Cross-Domain Opinion Target Extraction
– RNN with Auxiliary Tasks (AuxRNN)
Transfer Learning
8
Ying Ding, Jianfei Yu, and Jing Jiang. Recurrent neural networks with auxiliary labels for crossdomain opinion target extraction. In AAAI 2017.
Auxiliary Tasks
![Page 52: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/52.jpg)
Cross-Domain § Cross-Domain Aspect and Opinion Terms Co-Extraction
– Recursive Neural Structural Correspondence Network (RNSCN)
Transfer Learning
8
Wenya Wang and Sinno Jialin Pan. Recursive Neural Structural Correspondence Network for Cross-domain Aspect and Opinion Co-Extraction. In Proc. ACL, 2018.
![Page 53: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/53.jpg)
2
Outline
§ Transfer Learning – Cross-Domain – Cross-Lingual – Short Summary
§ Multi-Task Learning § Multimodal Learning § Summary
![Page 54: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/54.jpg)
Cross-Lingual § Cross-Lingual Aspect Term Extraction
– Transition-based Adversarial Network (TAN)
Transfer Learning
8
Wenya Wang and Sinno Jialin Pan. Transition-based Adversarial Network for Cross-lingual Aspect Extraction. In Proc. IJCAI, 2018.
![Page 55: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/55.jpg)
2
Outline
§ Transfer Learning – Cross-Domain – Cross-Lingual – Short Summary
§ Multi-Task Learning § Multimodal Learning § Summary
![Page 56: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/56.jpg)
Short Summary § Key to Cross-Domain/Lingual
– Step 1: Identify shared knowledge across domains or languages
• General Sentiment Words like good, bad, etc • Syntactic Structure • Domain/Language Discriminator • Auto-encoder (reconstruction of the input)
– Step 2: Design auxiliary tasks based on these shared knowledge
Transfer Learning
8
![Page 57: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/57.jpg)
Short Summary § Benchmark Datasets for Cross-Domain Aspect and
Opinion Terms Co-Extraction
– Laptop from SemEval-2014 – Restaurant from SemEval-2014, 2015 – Digital Device from (Hu and Liu, KDD2004)
Transfer Learning
8
Data Set #Sentences Train Test Laptop 3,845 2,884 961
Restaurant 5,841 4,381 1,460
Digital Device 3,836 2,877 959
![Page 58: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/58.jpg)
Short Summary § Results on Benchmark Datasets
• Hier-Joint: (Ding, Yu and Jiang, AAAI 2017) • RNSCN: (Wang and Sinno, ACL 2018)
Transfer Learning
8
Ø Incorporating domain-independent auxiliary tasks can indeed significantly outperform the baseline approach.
![Page 59: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/59.jpg)
Short Summary § Benchmark Datasets for Cross-Lingual Aspect Term
Extraction
– All from SemEval-2016 Task 5
Transfer Learning
8
Data Set #Sentences Train Test English 2,676 2,000 676
French 2,429 1,733 696
Spanish 2,951 2,070 881
![Page 60: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/60.jpg)
Short Summary § Results on Benchmark Datasets
• CL-DSCL: (Ding, Yu and Jiang, AAAI 2017) • TAN: (Wang and Sinno, IJCAI 2018)
Transfer Learning
8
Ø Incorporating language-independent auxiliary tasks can indeed significantly outperform the baseline approach.
![Page 61: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/61.jpg)
2
Outline
§ Transfer Learning § Multi-Task Learning
– Aspect and Opinion Terms Co-Extraction – End to End ABSA
• Aspect Term Extraction + Aspect-Level Sentiment Classification
§ Multimodal Learning § Summary
![Page 62: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/62.jpg)
2
Background § Aspect and Opinion Terms Co-extraction
– Input • A sentence or document
– Output • Aspect Term • Opinion Term
– Example
– Sequence Labeling Problems
Mul4-‐Task Learning
The fish was rather over cooked, but the staff was quite nice!
Ø Aspect Term: fish, staff Ø Opinion Term: over cooked, nice
![Page 63: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/63.jpg)
2
Outline
§ Transfer Learning § Multi-Task Learning
– Aspect and Opinion Terms Co-Extraction – End to End ABSA
• Aspect Term Extraction + Aspect-Level Sentiment Classification
§ Multimodal Learning § Summary
![Page 64: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/64.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Input • A sentence or document
– Output • Aspect Term • Sentiment polarity towards the aspect term
– Positive, Negative, Neutral
– Example
Mul4-‐Task Learning
The fish was rather over cooked, but the staff was quite nice!
Ø (fish, negative), (staff, positive)
![Page 65: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/65.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Neural CRF • Method 1: pipeline
Mul4-‐Task Learning
Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2015. Neural networks for open domain targeted sentiment. In Proceedings of EMNLP 2015.
Two Sequence Labeling Tasks
Pipeline Model
![Page 66: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/66.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Neural CRF • Method 2: joint
Mul4-‐Task Learning
Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2015. Neural networks for open domain targeted sentiment. In Proceedings of EMNLP 2015.
Two Sequence Labeling Tasks
Joint Model
![Page 67: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/67.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Neural CRF • Method 3: collapsed
Mul4-‐Task Learning
Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2015. Neural networks for open domain targeted sentiment. In Proceedings of EMNLP 2015.
One Sequence Labeling Task
Collapsed Model
![Page 68: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/68.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Neural CRF • Comparison
Mul4-‐Task Learning
Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2015. Neural networks for open domain targeted sentiment. In Proceedings of EMNLP 2015.
![Page 69: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/69.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Unified Solution
Mul4-‐Task Learning
Xin Li, Lidong Bing, Piji Li and Wai Lam. A Unified Model for Opinion Target Extraction and Target Sentiment Prediction. In AAAI 2019.
Two Sequence Labeling Tasks
![Page 70: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/70.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Unified Solution
• Two LSTMs for the target boundary detection task (auxiliary) and the complete TBSA task (primary).
• BG component: exploiting boundary information • SC component: maintaining sentiment consistency • OE component: improving the quality of the boundary information
Mul4-‐Task Learning
Xin Li, Lidong Bing, Piji Li and Wai Lam. A Unified Model for Opinion Target Extraction and Target Sentiment Prediction. In AAAI 2019.
![Page 71: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/71.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Span Extraction-based approach
Mul4-‐Task Learning
Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification. Minghao Hu, Yuxing Peng, Zhen Huang, Dongsheng Li, Yiwei Lv. In Proceedings of ACL. 2019.
![Page 72: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/72.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Span Extraction-based approach • BERT as encoder
Mul4-‐Task Learning
Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification. Minghao Hu, Yuxing Peng, Zhen Huang, Dongsheng Li, Yiwei Lv. In Proceedings of ACL. 2019.
Ø The last block’s hidden states are used to propose one or multiple candidate targets based on the probabilities of the start and end positions
Ø Predict the sentiment polarity using the span representation of the given target
![Page 73: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/73.jpg)
5
– Laptop, Restaurant are from SemEval-2014 – Twitter-2014 from (Dong et al. ACL 2014) – Another two Restaurant datasets from SemEval-2015,
SemEval-2016
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI, 2019.
Data Set #Training Samples #Test Samples POS NEG NEU POS NEG NEU
Laptop 980 858 454 340 128 171
Restaurant 2159 800 623 730 195 196
Twitter-2014 1567 1563 3127 147 147 346
Background § End to End Aspect-Based Sentiment Analysis
– Comparison of previous three approaches • Benchmark Datasets
Mul4-‐Task Learning
![Page 74: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/74.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Comparison of previous three approaches • Unified Approach vs LSTM-based Methods
Mul4-‐Task Learning
Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification. Minghao Hu, Yuxing Peng, Zhen Huang, Dongsheng Li, Yiwei Lv. In Proceedings of ACL. 2019.
![Page 75: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/75.jpg)
2
Background § End to End Aspect-Based Sentiment Analysis
– Comparison of previous three approaches • BERT-based Methods vs Unified Approach
Mul4-‐Task Learning
Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification. Minghao Hu, Yuxing Peng, Zhen Huang, Dongsheng Li, Yiwei Lv. In Proceedings of ACL. 2019.
![Page 76: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/76.jpg)
2
Outline
§ Transfer Learning § Multi-Task Learning § Multimodal Learning
– Target-Oriented Multimodal Sentiment Classification
§ Summary
![Page 77: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/77.jpg)
Recall
§ Target-oriented Sentiment Classification (TSC) – Input
• A sentence or document • An opinion target
– Output • Sentiment polarity towards the opinion target
2
§ Examples The fish was rather over cooked, but the
chicken was fine!
Ø sentiment over fish: negative Ø sentiment over chicken: positive
Mul4modal Learning
![Page 78: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/78.jpg)
Motivation
3
§ Limitation of TSC – Ineffective for multimodal social media posts
• Incomplete Textual Contents
Mul4modal Learning
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
![Page 79: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/79.jpg)
Motivation
3
§ Limitation of TSC – Ineffective for multimodal social media posts
• Incomplete Textual Contents
W/O Image Rihanna: neutral ×
Mul4modal Learning
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
![Page 80: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/80.jpg)
Motivation
3
§ Limitation of TSC – Ineffective for multimodal social media posts
• Incomplete Textual Contents
With Image
Rihanna: positive √
Mul4modal Learning
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
![Page 81: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/81.jpg)
Motivation
3
§ Limitation of TSC – Ineffective for multimodal social media posts
• Incomplete Textual Contents
• Irregular Expressions
W/O Image Georgina Hermitage: neutral ×
Mul4modal Learning
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
![Page 82: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/82.jpg)
Motivation
3
§ Limitation of TSC – Ineffective for multimodal social media posts
• Incomplete Textual Contents
• Irregular Expressions
With Image
Georgina Hermitage : positive √
Mul4modal Learning
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
![Page 83: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/83.jpg)
6
§ Target-oriented Multimodal Sentiment Classification (TMSC) – Input
• A sentence or document • An opinion target • An associated image
– Output • Sentiment polarity towards the opinion target
Mul4modal Learning
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
Motivation
![Page 84: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/84.jpg)
§ Base model with BERT – Input Transformation
• Context as the first sentence • Opinion Target as the second sentence
– Example
10
[Georgina Hermitage]positive is a #one2watch since she broke the [400m T37]neutral WR!
Opinion Target BERT Input Label
Georgina Hermitage
[CLS] $T$ is a #one2watch since she broke the 400m T37 WR. [SEP] Georgina Hermitage [SEP] Positive
400m T37 [CLS] Georgina Hermitage is a #one2watch since she broke the $T$ WR. [SEP] 400m T37 [SEP] Neutral
Methodology -- BERT
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
Mul4modal Learning
![Page 85: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/85.jpg)
§ Apply BERT to TSC – Feed the transformed sentence to BERT
10
Input Embedding
Self Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Sent
ence
Enc
oder
LS×
Pooling & Linear & Softmax
(a). Georgina Hermitage
POSITIONAL ENCODING
positive
[CLS] $T$ is a #one2watch since she broke the 400m T37 WR. [SEP] Georgina Hermitage [SEP]
Input Embedding
Self Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Sent
ence
Enc
oder
LS×
Pooling & Linear & Softmax
(b). 400m T37
POSITIONAL ENCODING
neutral
[CLS] Georgina Hermitage is a #one2watch since she broke the $T$ WR. [SEP] 400m T37 [SEP]
Methodology -- BERT
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
Mul4modal Learning
![Page 86: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/86.jpg)
10
Input Embedding
Self Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Sent
ence
Enc
oder
LS×
POSITIONAL ENCODING
positive
[CLS] $T$ is a #one2watch since she broke the 400m T37 WR. [SEP] Georgina Hermitage [SEP] Image
Multimodal Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Mul
timod
al E
ncod
er
Lm×
CONCAT
Pooling & Linear & Softmax
ResNet
Imag
e En
code
r
§ Limitation – Image features are not
sensitive to opinion targets
• Georgina Hermitage
• 400m T37
Methodology -- multimodal BERT (mBERT) Mul4modal Learning
![Page 87: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/87.jpg)
Methodology -- Target-oriented mBERT (TomBERT)
10
Target Attention
K V Q
Target Embedding
Feed Forward
ResNet
Add & Norm
Add & Norm
Self Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Targ
et E
ncod
er
Imag
e En
code
r
Targ
et-Im
age
Mat
chin
g
Lc×
Lt×
[CLS] Georgina Hermitage [SEP]
§ Target Attention – Target as queries, images as keys and values
Mul4modal Learning
![Page 88: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/88.jpg)
10
positive
[CLS] $T$ is a #one2watch since she broke the 400m T37 WR. [SEP] Georgina Hermitage [SEP]
Target Attention
K V Q
Input Embedding
Feed Forward
ResNet
CONCAT
Add & Norm
Add & Norm
POSITIONAL ENCODING
Self Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Targ
et E
ncod
er
Sent
ence
Enc
oder
Imag
e En
code
r
Multimodal Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Mul
timod
al E
ncod
er
Targ
et-Im
age
Mat
chin
g
Lc× Ls×
Lt× Lm×
Pooling & Linear & Softmax
Self Attention
K V Q
Feed Forward
Add & Norm
Add & Norm
Target Embedding
§ Full Model Mul4modal Learning
Methodology -- Target-oriented mBERT (TomBERT)
![Page 89: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/89.jpg)
Experiments
16
§ Two Multimodal Datasets
– The two multimodal Twitter datasets are based on two
public multimodal Named Entity Recognition (NER) datasets
Modality Data Set #Training Samples #Dev Samples #Test Samples
POS NEG NEU POS NEG NEU POS NEG NEU
Text+Image Twitter-2015 928 368 1883 303 149 670 317 113 607
Twitter-2017 1508 416 1638 515 144 517 493 168 573
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
![Page 90: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/90.jpg)
Experimental Results
17
§ Results on the Two Multimodal Datasets Modality Method
Twitter-2015 Twitter-2017 Accuracy Macro-F1 Accuracy Macro-F1
Visual Res-Target 59.88 46.48 58.59 53.98
Text
AE-LSTM 70.30 63.43 61.67 57.97
MemNet 70.11 61.76 64.18 60.90
RAM 70.68 63.05 64.42 61.01
MGAN 71.17 64.21 64.75 61.46
BERT 74.15 68.86 68.15 65.23
BERT+BL 74.25 70.04 68.88 66.12
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
![Page 91: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/91.jpg)
Experimental Results
17
§ Results on the Two Multimodal Datasets Modality Method
Twitter-2015 Twitter-2017 Accuracy Macro-F1 Accuracy Macro-F1
Visual Res-Target 59.88 46.48 58.59 53.98
Text
AE-LSTM 70.30 63.43 61.67 57.97
MemNet 70.11 61.76 64.18 60.90
RAM 70.68 63.05 64.42 61.01
MGAN 71.17 64.21 64.75 61.46
BERT 74.15 68.86 68.15 65.23
BERT+BL 74.25 70.04 68.88 66.12
Text + Visual
Res-MGAN 71.65 63.88 66.37 63.04
Res-MGAN-TFN 70.30 64.14 64.10 59.13
Res-BERT+BL 75.02 69.21 69.20 66.48
Res-BERT+BL-TFN 73.58 68.74 67.18 64.29
mBERT 75.31 70.18 69.61 67.12 TomBERT 77.15 71.75 70.34 68.03
Jianfei Yu and Jing Jiang. Adapting BERT for Target-Oriented Multimodal Sentiment Classification. In IJCAI 2019.
![Page 92: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/92.jpg)
2
Outline
§ Transfer Learning § Multi-Task Learning § Multimodal Learning § Summary
![Page 93: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/93.jpg)
Summary
5
Cutting-Edge Dimensions
Multi-task Learning Transfer Learning
Cross-Domain ABSA Cross-
Lingual Aspects/Opinions
Co-Extraction ABSC
Multimodal Learning
State-of-the-art Methods: Attention-based LSTM models
State-of-the-art Methods: BERT-based models
![Page 94: Fine-Grained Opinion Mining: Current Trend and Cutting-Edge … · 2019-08-12 · Recursive Neural Network ! Dependency Tree-based Approach – AdaRNN • propagate sentiment information](https://reader033.fdocuments.net/reader033/viewer/2022060402/5f0e5f407e708231d43eed08/html5/thumbnails/94.jpg)
Thank you !