Different Semantic Perspectives for Question Answering Systems

NLP & Semantic Computing Group

N L P

Different Semantic Perspectives forHybrid Question Answering Systems

Andre FreitasUniversity of Passau

OKBQA, Jeju, 2016


http://www.slideshare.net/andrenfreitas

These slides:


Outline Multiple Perspectives of Semantic

Representation Lightweight Semantic Representation Knowledge Graph Extraction from Text Answering Queries with

Knowledge Graphs Reasoning Take-away Message


Multiple Perspectives of Semantic Representation


QA & Semantics

• Question Answering is about managing semantic representation, extraction, selection trade-offs.

• And it is about integrating multiple components in a complex approach.

•Semantic best-effort, systems tolerant to noisy, inconsistent, vague, data.

5


“Most semantic models have dealt with particular types of constructions, and have been carried out under very simplifying assumptions, in true lab conditions.”

“If these idealizations are removed it is not clear at all that modern semantics can give a full account of all but the simplest models/statements.”

Formal World Real World

Baroni et al. 2013

Semantics for a Complex World

6


Why Not RDF?•Follows a more “database-type” of

representation perspective.

•Gap towards representing text.


Choices of Semantic Representation• Logical

• Frames: verbs | nouns

• Binary relations: binary | n-ary

• Named entities

• Language Models

• Syntactic structures

• Bag-of-words

Concept-level representation

Background knowledge

Extraction complexity

8


Information Extraction• Logical



• Named entities

• Syntactic Structures & LMs

• Bag-of-words

• Semantic parsing

• Semantic role labeling

• Relation extraction: – closed/open

• Named entity recognition

• Syntactic/N-gram Parsing

• Indexing

9


Information Extraction• Logical



• Named entities

• Syntactic Structures & LMs

• Bag-of-words

• Semantic parsing

• Semantic role labeling

• Relation extraction: – closed/open

• Named entity recognition

• Syntactic/N-gram Parsing

• Indexing

Use all of them!


Representation focal points•Types of knowledge to focus at the

representation: Facts vs Definitions vs Opinions Temporality Spatiality Modality Polarity Rhetorical structures …

11


Lightweight Semantic Representation


Objective•Provide a lightweight knowledge representation model which: Can represent textual discourse

information.• Maximizes the capture of textual information.

Is convenient to extract from text. Is convenient to access (query and

browse).13


Lightweight Semantic Representation

Representing Texts as Contextualized Entity-Centric Linked Data Graphs, WebS 2013


Representation Assumptions• Data integration:

Named entities (instances) Abstract classes (unary predicates)

• Rich taxonomical structures.

• Context representation as a first class citizen.

• Open vocabulary.

• Word instead of sense/concept.15


Representation of Complex RelationsGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

17


Data Integration pointsGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

Named entities are lower entropy integration points Pivot

points18



Named entities are also low entropy entry points for answering queries Pivot

points19



Also abstract classes … Pivot

points20



They are also a very convenient way to represent. Pivot

points21


Taxonomy Extraction Are predicates with more complex compositional patterns

which describe sets.

Parsing complex nominals.

American multinational conglomerate corporation

On the Semantic Representation and Extraction of Complex Category Descriptors, NLDB 2014

multinational conglomerate corporation

corporation

conglomerate corporation

is a

is a

is a

Pivot points


Context RepresentationGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

Reification as a first class representation element

25


Context RepresentationGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

Temporality, spatiality, modality, rhetorical relations …

26


Rhetorical Structures using Reification• cause:

e.g. “because scraping the bottom with a metal utensil will scratch the surface.”

• circumstance e.g. “After completing your operating system reinstallation,”

• concession e.g. “Although the hotel is situated adjacent to a beach,”

• condition e.g. “If you can break the $ 1000 dollar investment range,”

• contrast e.g. “but you can do better with 2.4ghz or 900mhz phones.”

• purpose e.g.“in order for the rear passengers to get in the vehicle.”

• …27


Open VocabularyGeneral Electric Company, or GE , is an American multinational conglomerate corporation incorporated in Schenectady , New York

Temporality, spatiality, modality, rhetorical relations …

29


Open Vocabulary

•Easier to extract but difficult to consume.

•We pay the price at query time.

•How to operate over a large-scale semantically heterogeneous knowledge-graphs?

30


Words instead of Senses•Motivation: Disambiguation is a tough

problem.

•Sense granularity can be, at many situations, arbitrary (too context dependent).

•We treat a word as a superposition of senses, almost in a “quantum mechanical sense”. 32


Sens

e Su

perp

ositi

on

Coecke et al. (2010): Category theory and Lambek calculus.


Revisited RDF (for Representing Texts)• Data Model Types: Instance, Class, Property…

• RDFS: Taxonomic representation.

• Reification for contextual relations (subordinations).

• Blank nodes for n-ary relations.

• Triple.

• Labels over URIs.34


Abstract Meaning Representations – AMR,Maximal Use of PropBank Frame Files

Alternative Representations


Distributional Semantics


Distributional Semantic Models Semantic Model with low acquisition effort

(automatically built from text)

Simplification of the representation

Enables the construction of comprehensive commonsense/semantic KBs

What is the cost?

Some level of noise(semantic best-effort)

Limited semantic model37


Distributional Semantics as Commonsense Knowledge

Commonsense is here

θ

car

dog

cat

bark

run

leashSemantic Approximation is

here

38


I find it rather odd that people are already trying to tie the Commission's hands in relation to the proposal for a directive, while at the same calling on it to present a Green Paper on the current situation with regard to optional and supplementary health insurance schemes.

I find it a little strange to now obliging the Commission to a motion for a resolution and to ask him at the same time to draw up a Green Paper on the current state of voluntary insurance and supplementary sickness insurance.

=?

Beyond Single Word Vector Models: Compositionality


Compositional Semantics Can we extend DS to account for the

meaning of phrases and sentences? Compositionality: The meaning of a

complex expression is a function of the meaning of its constituent parts.


Compositional Semantics

Words in which the meaning is directly determined by their distributional behaviour (e.g., nouns).

Words that act as functions transforming the distributional profile of other words (e.g., verbs, adjectives, …).

dogs

old

41


Compositional-Distributional Semantics

NLP & Semantic Computing Group Recursive Neural Networks for Structure Prediction

43


New Model: Recursive Neural Tensor Network•Goal: Function that composes two vectors.•More expressive than any other RNN so far.

44 Socher et al.


Socher et al.


Compositional-distributional model for Categories

46


Embedding Knowledge Graphs

47


The vector space is segmented48

Dimensional reduction mechanism!

A Distributional Structured Semantic Space for Querying RDF Graph Data, IJSC 2012


Compositional-distributional model for paraphrases

A Compositional-Distributional Semantic Model for Searching Complex Entity Categories, *SEM (2016)


Knowledge Graph Extraction from Text


Graphene


Graph Extraction Pipeline

Text Transformati

on

N-ary Relation Extractio

nText Simplificatio

n GraphSerializatio

n

Taxonomy

Extraction

Storage

RST Classificati

on

ML-based

Rule-based

Rule-based

ML-based

52


Minimalistic Text Transformations

Text Transformati

on


nText Simplificatio

n GraphSerializatio

n

Taxonomy

Extraction

Storage

RST Classificati

on

ML-based

Rule-based

Rule-based

ML-based

53


Minimalistic Text Transformations

•Co-reference Resolution Pronominal co-references.

•Passive We have been approached by the investment

banker. The investment banker approached us.

•Genitive modifier Malaysia's crude palm oil output is estimated

to have risen. The crude palm oil output of Malasia is

estimated to have risen.54


Text Simplification

Text Transformati

on


nText Simplificatio

n GraphSerializatio

n

Taxonomy

Extraction

Storage

RST Classificati

on

ML-based

Rule-based

Rule-based

ML-based

55


Text Simplification for KG Extraction“Defeating Republican nominee Mitt Romney, Obama, who was the first African American to hold the office, was reelected president in November 2012.”

relations are spread across clauses relations are presented in non-canonical form

56


Text Simplification for KG Extraction

•Insertion of a text simplification step

Obama was reelected president in November 2012.

Obama was the first African American to hold the office.

Obama was defeating Mitt Romney. Mitt Romney was Republican nominee.

57


Syntax-driven sentence simplification approachTask:

• Reduce the linguistic complexity of a text while retaining the original information/meaning using a set of syntax-based rewrite operations (deletion, insertion, reordering, sentence splitting).

Idea:• Simplify a sentence by separating out components

that supply only secondary information into simpler stand-alone context sentences, thus yielding one or more reduced core sentences.


Approach• Linguistic analysis of sentences from the English Wikipedia to identify constructs which provide only secondary information:

• non-restrictive relative clauses• non-restrictive and restrictive appositive phrases• participial phrases offset by commas• adjective and adverb phrases delimited by punctuation• particular prepositional phrases• lead noun phrases• intra-sentential attributions• parentheticals• conjoined clauses with specific features• particular punctuation

•Rule-based simplification rules.

Improving Relation Extraction by Syntax-based Sentence Simplification (2016)


N-ary Relation Extraction

Text Transformati

on


nText Simplificatio

n GraphSerializatio

n

Taxonomy

Extraction

Storage

RST Classificati

on

Rule-based

Rule-based

ML-based

60

OpenIE, University of Washington


Taxonomy Extraction

Text Transformati

on


nText Simplificatio

n GraphSerializatio

n

Taxonomy

Extraction

Storage

RST Classificati

on

Rule-based

Rule-based

ML-based

61

Representation and Extraction of Complex Category Descriptors, NLDB 2014


RST Classification

Text Transformati

on


nText Simplificatio

n GraphSerializatio

n

Taxonomy

Extraction

Storage

RST Classificati

on

Rule-based

Rule-based

ML-based

62


Rhetorical Structure Extraction

63

TEXT-LEVEL RST-STYLE DISCOURSE PARSER (Feng and Hirst, 2012)

Structure classification

Relation classification


Answering Queries with Knowledge Graphs


Now our graph supports semantic approximations as a first-class operation


Approach Overview

Query Planner

Ƭ-Space(embedding

graphs)

WikipediaCommonsense

knowledge

RDF

Explicit Semantic Analysis

Core semantic approximation &

composition operations

Query AnalysisQuery Query Features

Query Plan

66


Core Principles Minimize the impact of Ambiguity, Vagueness,

Synonymy. Address the simplest matchings first (semantic

pivoting).

Semantic Relatedness as a primitive operation.

Distributional semantics models as commonsense knowledge representation.

Lightweight syntactic constraints.67


•Step 2: Query NER Rules-based: POS Tag + IDF

Who is the daughter of Bill Clinton married to?(PROBABLY AN INSTANCE)

Query Pre-Processing (Question Analysis)

68


•Step 3: Determine answer type Rules-based.

Who is the daughter of Bill Clinton married to? (PERSON)


69


•Transform natural language queries into a pseudo-logical form.

“Who is the daughter of Bill Clinton married to?”


70



Bill Clinton

daughter married to

(INSTANCE)

Person

ANSWER TYPE

QUESTION FOCUS71

• Step 5: Determine the query pattern Rules based.• Remove stop words.• Merge words into entities.• Reorder structure from core entity position.


• Step 5: Determine the query pattern Rules based.• Remove stop words.• Merge words into entities.• Reorder structure from core entity position.


Bill Clinton

daughter married to

(INSTANCE)

Person

(PREDICATE) (PREDICATE) Query Features

72


• Map query features into a query plan.• A query plan contains a sequence of:

Search operations. Navigation operations.

Query Planning

(INSTANCE) (PREDICATE) (PREDICATE) Query Features

(1) INSTANCE SEARCH (Bill Clinton) (2) DISAMBIGUATE ENTITY TYPE (3) GENERATE ENTITY FACETS (4) p1 <- SEARCH RELATED PREDICATE (Bill Clintion, daughter) (5) e1 <- GET ASSOCIATED ENTITIES (Bill Clintion, p1) (6) p2 <- SEARCH RELATED PREDICATE (e1, married to) (7) e2 <- GET ASSOCIATED ENTITIES (e1, p2) (8) POST PROCESS (Bill Clintion, e1, p1, e2, p2)

Query Plan

73


Core Entity SearchBill Clinton

daughter married to Person

:Bill_Clinton

Query:

KB:

Entity search

74


Distributional Semantic SearchBill Clinton


:Bill_Clinton

Query:

:Chelsea_Clinton

:child

:Baptists:religion

:Yale_Law_School:almaMater...

(PIVOT ENTITY)

(ASSOCIATED TRIPLES)

75

KB:




:Bill_Clinton

Query:

:Chelsea_Clinton

:child

:Baptists:religion

:Yale_Law_School:almaMater...

sem_rel(daughter,child)=0.054

sem_rel(daughter,child)=0.004

sem_rel(daughter,alma mater)=0.001

Which properties are semantically related to ‘daughter’?

76

KB:




:Bill_Clinton

Query:

:Chelsea_Clinton

:child

77

KB:




:Bill_Clinton

Query:

:Chelsea_Clinton

:child

(PIVOT ENTITY)

78

KB:




:Bill_Clinton

Query:

:Chelsea_Clinton

:child:Mark_Mezvinsky

:spouse

79

KB:

Note the lazy disambiguation


80


What is the highest mountain?Second Query Example

(CLASS) (OPERATOR) Query Features

mountain - highest PODS

81


Entity SearchMountain highest

:Mountain

Query:

:typeOf

(PIVOT ENTITY)

82

KB:


Extensional ExpansionMountain highest

:Mountain

Query:

:Everest:typeOf

(PIVOT ENTITY)

:K2:typeOf

...

83

KB:


Distributional Semantic MatchingMountain highest

:Mountain

Query:

:Everest:typeOf

(PIVOT ENTITY)

:K2:typeOf

...

:elevation

:location...:deathPlaceOf

84

KB:


Get all numerical valuesMountain highest

:Mountain

Query:

:Everest:typeOf

(PIVOT ENTITY)

:K2:typeOf

...

:elevation

:elevation

8848 m

8611 m

85

KB:


Apply operator functional definitionMountain highest

:Mountain

Query:

:Everest:typeOf

(PIVOT ENTITY)

:K2:typeOf

...

:elevation

:elevation

8848 m

8611 m

SORTTOP_MOST

86

KB:


Results

87


StarGraph•Open source NoSQL platform for building

and interacting with large and sparse knowledge graphs.

•Semantic approximation as a built-in operation.

•Scalable query execution performance.

88


Heuristics for the selection of the semantic pivot is critical!•Discussed here just superficially:

Information-theoretical justification.

How hard is the Query? Measuring the Semantic Complexity of Schema-Agnostic Queries, IWCS (2015).

Schema-agnositc queries over large-schema databases: a distributional semantics approach, PhD Thesis (2015).

On the Semantic Mapping of Schema-agnostic Queries: A Preliminary Study, NLIWoD (2015).

89


Indra Multilingual platform for experimentation

with different word vector models.

"Indra's net" is the net of the Vedic god Indra, whose net hangs over his palace on Mount Meru, the axis mundi of Hindu cosmology and Hindu mythology. Indra's net has a multifaceted jewel at each vertex, and each jewel is reflected in all of the other jewels.

In the Avatamsaka Sutra, the image of "Indra's net" is used to describe the interconnectedness of the universe.


Indra


Bridging Structured & Unstructured Data•NER + Text + Passage Retrieval Ranking

Simple and powerful QA basis.

•Lazy disambiguation.

94


Treo Answers Jeopardy Queries (Video)


Ranking Candidate Answers•But what if there are multiple candidate answers!

Q: Who was Queen Victoria’s second son?•Answer Type: Person

• Passage:The Marie biscuit is named after Marie Alexandrovna, the daughter of Czar Alexander II of Russia and wife of Alfred, the second son of Queen Victoria and Prince Albert

96Dan Jurafky’s slides


Ranking Candidate Answers•But what if there are multiple candidate answers!

Q: Who was Queen Victoria’s second son?•Answer Type: Person

• Passage:The Marie biscuit is named after Marie Alexandrovna, the daughter of Czar Alexander II of Russia and wife of Alfred, the second son of Queen Victoria and Prince Albert

97Dan Jurafky’s slides


Feature Engineering

The Marie biscuit is named after Marie Alexandrovna, the daughter of Czar Alexander II of Russia and wife of Alfred, the second son of Queen Victoria and Prince Albert

followed by a ‘,’ followed by an apposition

Who was Queen Victoria’s second son?

contains an entity in the query

has a four-word overlap

type = PERSON

matches AnswerType

98


Propositionalisation

e0 followedBy(,) followedByAppositionContainingQueryEntities() answer …

Alfred true true true… … …

passage

entity (e0)

entity (en)

…

The Marie biscuit is named after Marie Alexandrovna, the daughter of Czar Alexander II of Russia and wife of Alfred, the second son of Queen Victoria and Prince Albert

answer

99


Reasoning for Text Entailment


Beyond Word Vector Models

give birth mother

car

θ

Distributional semantics can give us a hint about the concepts’ semantic proximity...

...but it still can’t tell us what exactly the relationship between them is

give birth

mother???


Beyond Word Vector Models

give birth

mother???

give birth

mother???


Beyond Word Vector Models: Intensional Reasoning

Representing structured intensional-level knowledge.

Creation of an intensional-level reasoning model.

104


Commonsense Reasoning

Selective (focussed) reasoning - Selecting the relevant facts in the

context of the inference

Reducing the search space.Scalability


Extended WordNet (XWN)

NLP & Semantic Computing Group Commonsense Data (ConceptNet)

http://conceptnet5.media.mit.edu/

107


Distributional semantic relatedness as a Selectivity Heuristics

Distributional heuristics

108

target

source answer




109

target

source answer




110

target

source answer


John Smith

EngineerInstance-level

occupation

Does John Smith have a degree?


A Distributional Semantics Approach for Selective Reasoning on Commonsense Graph Knowledge Bases, NLDB (2015).


Bringing it into the Real World


Semeval 2017


Take-away Message• Choosing the sweet-spot in terms of semantic

representation is critical for the construction of robust QA systems.

Work at a word-based representation instead of a sense representation.

Text simplification/clausal disembedding critical for relation extraction.

Need for a standardized semantic representation for relations extracted from texts.


Take-away Message•Text entailment:

Intensional-level reasoning. Natural logic. Distributional semantics.

•Distributional semantics: Robust, language-agnostic semantic

matching. Selective reasoning over commonsense

KBs.

Different Semantic Perspectives for Question Answering Systems

Science

Transcript of Different Semantic Perspectives for Question Answering Systems