Parsing of natural language sentences to syntactic and ... · arg1 arg1 arg3 Reentrantnodes:...
Transcript of Parsing of natural language sentences to syntactic and ... · arg1 arg1 arg3 Reentrantnodes:...
Parsing of natural language sentences tosyntactic and semantic graph representationsAbschlussvortrag zum Forschungsprojekt
Pius MeinertApril 13, 2018
Overview
Graph Representations
Corpora
Parsing Techniques
Parser
1
Overview
Graph Representations
Corpora
Parsing Techniques
Parser
Semantic: AMR, UCCA, depen-dency graphsSyntactic: Constituency treederived, Use of syntactic infor-mation
1
Overview
Graph Representations
Corpora
Parsing Techniques
Parser
AMR, UCCA, SemEval-2014/-2015:dependency graphs, Penn Tree-bank, TIGER Corpus
1
Overview
Graph Representations
Corpora
Parsing Techniques
Parser
Maximum SubgraphTransition-BasedSynchronous HRG
1
Overview
Graph Representations
Corpora
Parsing Techniques
Parser
1
Abstract Meaning Representation (AMR) [Ban+13]
2
Abstract Meaning Representation (AMR) [Ban+13]
contrast
possible
say
YouI
person
dream
include
only —
arg0-of
arg1
arg2 arg3
2
Abstract Meaning Representation (AMR) [Ban+13]
rooted, directed,edge-labeled andleaf-labeled
contrast
possible
say
YouI
person
dream
include
only —
arg0-of
arg1
arg2 arg3
2
Tree Definition
A tree is a directed graph G = (V,A) that has a vertex r, namedroot, such that every vertex v ∈ V is reachable from r via aunique directed path. [KJ15; KO16]
3
Dependency Graph [KJ15]
The gene thus can prevent a plant from fertilizing itselfbv arg1 arg1 bv arg2
arg2arg1
arg1
arg3
4
Dependency Graph [KJ15]
unconnected
The gene thus can prevent a plant from fertilizing itselfbv arg1 arg1 bv arg2
arg2arg1
arg1
arg3
Connectedness:There exists an undirected path between every two pairs ofvertices. Nodes with in- and out-degree zero are calledsingletons. [KO16]
4
Dependency Graph [KJ15]
unconnected, multi-rooted
The gene thus can prevent a plant from fertilizing itselfbv arg1 arg1 bv arg2
arg2arg1
arg1
arg3
Top nodes:Nodes of in-degree zero, a graph’s equivalent to the uniqueroot in a tree. [KO16]
4
Dependency Graph [KJ15]
unconnected, multi-rooted, reentrancy
The gene thus can prevent a plant from fertilizing itselfbv arg1 arg1 bv arg2
arg2arg1
arg1
arg3
Reentrant nodes:Nodes with in-degree greater than one. [WXP15; DCS17; BB17]
4
Dependency Graph - Noncrossing [KJ15]
The gene thus can prevent a plant from fertilizing itselfbv arg1 arg1 bv arg2
arg2arg1
arg1
arg3
5
Dependency Graph - Noncrossing [KJ15]
Coverage ranges from 48% to 78% for various graph banks(CCGbank, Prage Semantic Dependencies, etc.). [KJ15; SCW17]
The gene thus can prevent a plant from fertilizing itselfbv arg1 arg1 bv arg2
arg2arg1
arg1
arg3
5
Dependency Graph - 1-Endpoint-Crossing [PKM13]
das mer em Hans es huus hälfed aastriche
6
Dependency Graph - 1-Endpoint-Crossing [PKM13]
das mer em Hans es huus hälfed aastriche
6
Dependency Graph - 1-Endpoint-Crossing [PKM13]
Account for 95.7− 97.7% of the dependency structures that areused in [Cao+17].
das mer em Hans es huus hälfed aastriche
6
Dependency Graph - Book Embedding [SCW17]
The company that Mark wants to buy
arg1 arg1arg2
arg1
arg1arg1
arg2arg2
arg2
7
Dependency Graph - Book Embedding [SCW17]
The company that Mark wants to buy
arg1 arg1arg2
arg1
arg1arg1
arg2arg2
arg2
7
Dependency Graph - Book Embedding [SCW17]
The company that Mark wants to buy
arg1 arg1
arg2
arg1
arg1arg1
arg2arg2
arg2
7
Dependency Graph - Book Embedding [SCW17]
Coverage with respect to different page numbers:PN coverage1 48− 78%2 20− 49%3 0.3− 1.7%
The company that Mark wants to buy
arg1 arg1
arg2
arg1
arg1arg1
arg2arg2
arg2
7
Evaluation Metric - AMR Representations
AMR graph
go–1
boy
want-01
ARG0 ARG1
ARG0 instance
instance
instance
8
Evaluation Metric - AMR Representations
AMR graph
go–1
boy
want-01
ARG0 ARG1
ARG0 instance
instance
instance
PENMAN notation
(w / want-01:arg0 (b / boy):arg1 (g / go-01)
:arg0 b)
8
Evaluation Metric - AMR Representations
AMR graph
go–1
boy
want-01
ARG0 ARG1
ARG0 instance
instance
instance
logic format
instance(a, want-01) ∧instance(b, boy) ∧instance(c, go-01) ∧ARG0(a, b) ∧ARG1(a, c) ∧ARG0(c, b)
8
Evaluation Metric - Smatch [CK13]
The boy wants the football
instance(x, want-01) ∧instance(y, boy) ∧instance(z, football) ∧ARG0(x, y) ∧ARG1(x, z)
The boy wants to go
instance(a, want-01) ∧instance(b, boy) ∧instance(c, go-01) ∧ARG0(a, b) ∧ARG1(a, c) ∧ARG0(c, b)
9
Evaluation Metric - Smatch [CK13]
The boy wants the football
instance(x, want-01) ∧instance(y, boy) ∧instance(z, football) ∧ARG0(x, y) ∧ARG1(x, z)
The boy wants to go
instance(a, want-01) ∧instance(b, boy) ∧instance(c, go-01) ∧ARG0(a, b) ∧ARG1(a, c) ∧ARG0(c, b)
inter-annotator agreement study:Smatch score ranges from 0.79 to 0.83.
9
Graph Parsing Techniques
Maximum Subgraph“all pairs” approach [BM06] - Consider all possible (weighted)arcs and find the maximum spanning connected subgraph.
Transition-based“stepwise” approach [BM06] - Build the graph step by step byapplying transitions to the current configuration.
Synchronous Hyperedge Replacement Grammar (SHRG)HRGs as “an intuitive generalization of context free grammars(CFGs) from strings to hypergraphs.” [Jon+12; Hab92]
10
Maximum Subgraph - Problem Definition[SCW17]
Input directed, weighted graph G = (V,A) (complete)
Implicit sentence s, class of graphs G
Output subgraph G′ = (V,A′ ⊆ A) with maximum totalweight such that G′ belongs to G
G′(s) = argmaxH∈G(s,G)∑
p∈H ScorePart(s,p)
Example if class of graphs G is the class of all trees,Maximum Subgraph = Maximum Spanning Tree
11
Maximum Subgraph - Learning and Features [MN07]
G′(s) = argmaxH∈G(s,G)∑
p∈H ScorePart(s,p)
12
Maximum Subgraph - Learning and Features [MN07]
Globallearning
Optimize entire graph score, not only single arcattachments.
G′(s) = argmaxH∈G(s,G)∑
p∈H ScorePart(s,p)
12
Maximum Subgraph - Learning and Features [MN07]
Globallearning
Optimize entire graph score, not only single arcattachments.
G′(s) = argmaxH∈G(s,G)∑
p∈H ScorePart(s,p)
Localfeatures
Restricted to a limited number of arcs (to keepinference and learning tractable).
12
JAMR [Fla+14]
First published AMR parser. It solves the task by means of twophases:
Concept identification: Match spans of words to concept graphfragments.
Relation identification: Find the maximum spanning connectedsubgraph over those graph fragments.
13
JAMR - Concept Identification Phase [Fla+14]
The boy wants to visit New York City
∅ boy want-01 ∅ visit-01 name
city“New”
“York”
“City”
nameop1
op2
op3
14
JAMR - Concept Identification Phase [Fla+14]
The boy wants to visit New York City
c:
w:b:k = 6
b0 b1 b2 b3 b4 b5 b6
∅ boy want-01 ∅ visit-01 name
city“New”
“York”
“City”
nameop1
op2
op3
score(b, c; θ) =∑k
i=1 θ>f(wbi−1:bi ,bi−1,bi, ci)
Solve by dynamic programming: O(n2).
14
JAMR - Relation Identification Phase [Fla+14]
1. Initialization:Include all edges and vertices given by the conceptidentification phase.
2. Pre-processing:Reduce the set of edges considered to one edge per pair ofnodes: Either the edge given by the first phase or the highestscoring one.
3. Core algorithm:First, add all positive edges and then greedily add the leastnegative edge that connects two components until the graph isconnected.
15
Transition-Based - Transition System [WXP15]
A transition system for parsing is a tuple S = (S, T, s0, St) where
• S is a set of parsing states (configurations).• T is a set of parsing actions (transitions), each of which isa function t : S→ S.
• s0 is an initialization function, mapping each inputsentence w to an initial state.
• St ⊆ S is a set of terminal states.
16
Transition-Based - Parsing Algorithm [WXP15]
Input: sentence w = w0...wnOutput: parsed graph G1: s← s0(w)2: while s /∈ St do3: T ← all possible actions according to s4: bestT ← argmaxt∈T score(t, s)5: s← apply bestT to s6: end while7: return G
17
Transition-Based - Learning and Features [MN07]
bestT ← argmaxt∈T score(t, s)
18
Transition-Based - Learning and Features [MN07]
Locallearning
Optimization only for single transitions, nottransition sequences.
bestT ← argmaxt∈T score(t, s)
18
Transition-Based - Learning and Features [MN07]
Locallearning
Optimization only for single transitions, nottransition sequences.
bestT ← argmaxt∈T score(t, s)
Globalfeatures
Features may be based on whole graph built sofar/entire transition history.
18
Transition-Based AMR Parser [WXP15]
Idea: Use similarities between an AMR and the dependencystructure of a sentence.
Two-stage framework:
1) dependency parser to generate dependency tree for thesentence
2) transition-based algorithm to transform dependency treeto an AMR graph
The dependency parser can be trained on a much larger dataset.
19
Transition-Based AMR Parser - Transition Actions [WXP15]
REENTRANCE action
want
police arrest
reentrance
⇒want
police arrestARG0
REPLACE-HEAD action
live
in
Singapore
⇒live
Singapore
20
Towards a Catalogue of Linguistic Graph Banks [KO16]
• graph structures are of growing relevance to much NLPresearch
• provide common terminology and transparent statisticsfor different (collections of) graphs
• propose to establish shared community resource:https://aclweb.org/aclwiki/Graph_Parsing_(State_of_the_Art)
21
References i
Laura Banarescu, Claire Bonial, Shu Cai,Madalina Georgescu, Kira Griffitt, Ulf Hermjakob,Kevin Knight, Philipp Koehn, Martha Palmer, andNathan Schneider. “Abstract Meaning Representationfor Sembanking”. In: Proceedings of the 7thLinguistic Annotation Workshop and Interoperabilitywith Discourse, LAW-ID@ACL 2013, August 8-9, 2013,Sofia, Bulgaria. 2013, pp. 178–186. url:http://aclweb.org/anthology/W/W13/W13-2322.pdf.
References ii
Jan Buys and Phil Blunsom. “Robust IncrementalNeural Semantic Graph Parsing”. In: Proceedings ofthe 55th Annual Meeting of the Association forComputational Linguistics, ACL 2017, Vancouver,Canada, July 30 - August 4, Volume 1: Long Papers.2017, pp. 1215–1226. doi: 10.18653/v1/P17-1112.url:https://doi.org/10.18653/v1/P17-1112.
References iii
Sabine Buchholz and Erwin Marsi. “CoNLL-X SharedTask on Multilingual Dependency Parsing”. In:Proceedings of the Tenth Conference onComputational Natural Language Learning, CoNLL2006, New York City, USA, June 8-9, 2006. 2006,pp. 149–164. url:http://aclweb.org/anthology/W/W06/W06-2920.pdf.
References iv
Junjie Cao, Sheng Huang, Weiwei Sun, andXiaojun Wan. “Parsing to 1-Endpoint-Crossing,Pagenumber-2 Graphs”. In: Proceedings of the 55thAnnual Meeting of the Association for ComputationalLinguistics, ACL 2017, Vancouver, Canada, July 30 -August 4, Volume 1: Long Papers. 2017, pp. 2110–2120.doi: 10.18653/v1/P17-1193. url:https://doi.org/10.18653/v1/P17-1193.
References v
Shu Cai and Kevin Knight. “Smatch: an EvaluationMetric for Semantic Feature Structures”. In:Proceedings of the 51st Annual Meeting of theAssociation for Computational Linguistics, ACL 2013,4-9 August 2013, Sofia, Bulgaria, Volume 2: ShortPapers. 2013, pp. 748–752. url:http://aclweb.org/anthology/P/P13/P13-2131.pdf.
References vi
Marco Damonte, Shay B. Cohen, and Giorgio Satta.“An Incremental Parser for Abstract MeaningRepresentation”. In: Proceedings of the 15thConference of the European Chapter of theAssociation for Computational Linguistics, EACL 2017,Valencia, Spain, April 3-7, 2017, Volume 1: LongPapers. 2017, pp. 536–546. url:https://aclanthology.info/papers/E17-1051/e17-1051.
References vii
Jeffrey Flanigan, Sam Thomson, Jaime G. Carbonell,Chris Dyer, and Noah A. Smith. “A DiscriminativeGraph-Based Parser for the Abstract MeaningRepresentation”. In: Proceedings of the 52nd AnnualMeeting of the Association for ComputationalLinguistics, ACL 2014, June 22-27, 2014, Baltimore, MD,USA, Volume 1: Long Papers. 2014, pp. 1426–1436. url:http://aclweb.org/anthology/P/P14/P14-1134.pdf.
Annegret Habel. Hyperedge Replacement: Grammarsand Languages. Vol. 643. Lecture Notes in ComputerScience. Springer, 1992.
References viii
Bevan Jones, Jacob Andreas, Daniel Bauer,Karl Moritz Hermann, and Kevin Knight.“Semantics-Based Machine Translation withHyperedge Replacement Grammars”. In: COLING 2012,24th International Conference on ComputationalLinguistics, Proceedings of the Conference: TechnicalPapers, 8-15 December 2012, Mumbai, India. 2012,pp. 1359–1376. url:http://aclweb.org/anthology/C/C12/C12-1083.pdf.
References ix
Marco Kuhlmann and Peter Jonsson. “Parsing toNoncrossing Dependency Graphs”. In: TACL 3 (2015),pp. 559–570. url:https://tacl2013.cs.columbia.edu/ojs/index.php/tacl/article/view/709.
Marco Kuhlmann and Stephan Oepen. “Towards aCatalogue of Linguistic Graph Banks”. In:Computational Linguistics 42.4 (2016), pp. 819–827.doi: 10.1162/COLI_a_00268. url:https://doi.org/10.1162/COLI_a_00268.
References x
Ryan T. McDonald and Joakim Nivre. “Characterizingthe Errors of Data-Driven Dependency ParsingModels”. In: EMNLP-CoNLL 2007, Proceedings of the2007 Joint Conference on Empirical Methods inNatural Language Processing and ComputationalNatural Language Learning, June 28-30, 2007, Prague,Czech Republic. 2007, pp. 122–131. url: http://www.aclweb.org/anthology/D07-1013.
Emily Pitler, Sampath Kannan, and Mitchell Marcus.“Finding Optimal 1-Endpoint-Crossing Trees”. In: TACL1 (2013), pp. 13–24. url:https://tacl2013.cs.columbia.edu/ojs/index.php/tacl/article/view/23.
References xi
Weiwei Sun, Junjie Cao, and Xiaojun Wan. “SemanticDependency Parsing via Book Embedding”. In:Proceedings of the 55th Annual Meeting of theAssociation for Computational Linguistics, ACL 2017,Vancouver, Canada, July 30 - August 4, Volume 1:Long Papers. 2017, pp. 828–838. doi:10.18653/v1/P17-1077. url:https://doi.org/10.18653/v1/P17-1077.
References xii
Chuan Wang, Nianwen Xue, and Sameer Pradhan. “ATransition-based Algorithm for AMR Parsing”. In:NAACL HLT 2015, The 2015 Conference of the NorthAmerican Chapter of the Association forComputational Linguistics: Human LanguageTechnologies, Denver, Colorado, USA, May 31 - June 5,2015. 2015, pp. 366–375. url:http://aclweb.org/anthology/N/N15/N15-1040.pdf.