Handling tree structures — recursive SPs, nested sets, recursive CTEs
Recursive Structure - speech.ee.ntu.edu.tw
Transcript of Recursive Structure - speech.ee.ntu.edu.tw
![Page 1: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/1.jpg)
Recursive Structure
![Page 2: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/2.jpg)
x1
h0 f h1
x2
f
x3
h2 f
x1
h1
f
x2 x3
Application: Sentiment Analysis
x4
h3 f g
x4
f
h2
f
h3 gWord Sequence
sentiment
Recurrent Structure
Recursive Structure
Special case of recursive structure
How to stack function f is already determined
same dimension
word vector
![Page 3: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/3.jpg)
Recursive Model
not very good
not very good
syntactic structure
word sequence:
How to do it is out of the scope
![Page 4: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/4.jpg)
Recursive Model
f
By composing the two meaning, what should the meaning be.
Meaning of “very good”V(“very good”)
not very good
syntactic structure
not very good
V(“not”) V(“very”) V(“good”)
Dimension of word vector = |Z|
Input: 2 X |Z|, output: |Z|
![Page 5: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/5.jpg)
Recursive Model
not very good
V(“not”) V(“very”) V(“good”)
f
not very good
syntactic structureV(wA wB) ≠ V(wA) + V(wB)
“good”: positive
“not”: neutral
“not good”: negative Meaning of “very good”V(“very good”)
![Page 6: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/6.jpg)
Recursive Model
V(wA wB) ≠ V(wA) + V(wB)
“棒”: positive
“好棒”: positive
“好棒棒”: negative
not very good
V(“not”) V(“very”) V(“good”)
f
not very good
syntactic structure
Meaning of “very good”V(“very good”)
network
![Page 7: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/7.jpg)
Recursive Model
not very good
V(“not”) V(“very”) V(“good”)
f
not very good
syntactic structure
f
“not” “good”
f
“not” “bad”
“not bad”“not good”
“not”: “reverse” another input
Meaning of “very good”V(“very good”)
![Page 8: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/8.jpg)
Recursive Model
not very good
V(“not”) V(“very”) V(“good”)
f
not very good
syntactic structure
f
“very” “good”
f
“very” “bad”
“very bad”“very good”
“very”: “emphasize” another input
Meaning of “very good”V(“very good”)
![Page 9: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/9.jpg)
f
f
V(“not”) V(“very”) V(“good”)
g
-
output5 classes
( -- , - , 0 , + , ++ )
not very good
ref
f
g
Train both …V(“not very good”)
![Page 10: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/10.jpg)
Recursive Neural Tensor Network
f
𝑎 𝑏
𝑝
W= 𝜎
Little interaction between a and b
+= 𝜎
x
xT
W
𝑖,𝑗
𝑊𝑖𝑗𝑥𝑖𝑥𝑗
![Page 11: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/11.jpg)
Experiments
5-class sentiment classification ( -- , - , 0 , + , ++ )
Demo: http://nlp.stanford.edu:8080/sentiment/rntnDemo.html
![Page 12: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/12.jpg)
Experiments
Socher, Richard, et al. "Recursive deep models for semantic compositionality
over a sentiment treebank." Proceedings of the conference on empirical
methods in natural language processing (EMNLP). Vol. 1631. 2013.
![Page 13: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/13.jpg)
f
Matrix-Vector Recursive Network
inherent meaning
how it changes the others
not good
𝑎 𝐴 𝑏 𝐵Zero? Identity?
Informative?−1 00 −1
?
=
=W𝜎 = WM
=
![Page 14: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/14.jpg)
Tree LSTMht-1
xt
LSTM
mt-1
ht
mt
h1 m1 h2m2
LSTM
h3 m3
Typical LSTM
Tree LSTM
![Page 15: Recursive Structure - speech.ee.ntu.edu.tw](https://reader031.fdocuments.net/reader031/viewer/2022013001/61ca528a7c88890688018d4c/html5/thumbnails/15.jpg)
More Applications
• Sentence relatedness
NN
Sentence 1 Sentence 2
Recursive Neural
Network
Recursive Neural
Network
Tai, Kai Sheng, Richard Socher, and
Christopher D. Manning. "Improved
semantic representations from tree-
structured long short-term memory
networks." arXiv preprint
arXiv:1503.00075 (2015).