Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata...

67
Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results Iterating Semantic Automata Shane Steinert-Threlkeld (Joint work with Thomas Icard) May 21, 2013

Transcript of Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata...

Page 1: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Semantic Automata

Shane Steinert-Threlkeld(Joint work with Thomas Icard)

May 21, 2013

Page 2: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Outline

1 Generalized QuantifiersIntroduction to GQTIterated Quantifiers

2 Semantic AutomataBasic DefinitionsGeneralized Quantifiers and Regular LanguagesGeneralized Quantifiers and Context-Free Languages

3 Iterating Semantic AutomataIterating Two DFAsIterating With One or More PDAs

4 Experimental ResultsExperiments for Single GQsExperiments for Nested GQsAutomata and LinguisticsReferences

Page 3: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Introduction to GQT

Definition of GQ

Definition ([Mos57], [Lin66])

A generalized quantifier Q of type 〈n1, . . . , nk〉 is a class of modelsM = 〈M,R1, . . . ,Rk〉 closed under isomorphism, where eachRi ⊆ Mni . A generalized quantifier is monadic if ni = 1 for all i ,and polyadic otherwise.

We write QM R1 . . . Rk as shorthand for 〈M,R1, . . . ,Rk〉 ∈ Q.Usually the subscripted M is omitted for readability. Thus, e.g., fortype 〈1, 1〉 we write Q A B , where A and B are sets. 1

1For more complete introductions to the theory of generalized quantifiers,see [BC81, B86, Wes89, Kee96, KW11, PW06].

Page 4: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Introduction to GQT

Examples of GQs

This connects with the more familiar definition given in linguisticsemantics as can be seen by the following examples:

all = {〈M,A,B〉 | A ⊆ B}

some = {〈M,A,B〉 | A ∩ B 6= ∅}

more than four = {〈M,A,B〉 | |A ∩ B | ≥ 4}

The isomorphism closure condition (partially) captures theintuition that quantifiers are sensitive only to the size of therelevant subsets of M and not the identity of any particularelements or the order in which they are presented.

Page 5: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Introduction to GQT

Definability of GQs

Definition

A generalized quantifier Q of type 〈1, 1〉 is first-order definable ifand only if there is a first-order language L and an L-sentence ϕwhose non-logical vocabulary contains only two unary predicatesymbols A and B such that for any model M = 〈M,A,B〉,

QMAB ⇔ 〈M,A,B〉 |= ϕ.

The generalization to higher-order definability (and other typesthan 〈1, 1〉) is obvious.

Page 6: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Introduction to GQT

Definability of GQs

Definition

A generalized quantifier Q of type 〈1, 1〉 is first-order definable ifand only if there is a first-order language L and an L-sentence ϕwhose non-logical vocabulary contains only two unary predicatesymbols A and B such that for any model M = 〈M,A,B〉,

QMAB ⇔ 〈M,A,B〉 |= ϕ.

The generalization to higher-order definability (and other typesthan 〈1, 1〉) is obvious.

First-order: some, all, more than n, between m and n, . . .

Higher-order: an even/odd number of, most, less than m/n

Page 7: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Introduction to GQT

Constraints on GQs

CONS QMAB iff QMA(A ∩ B).

EXT QMAB iff QM′AB for every M ⊆ M ′.

Lemma

A quantifier Q satisfies CONS + EXT iff, for all M = 〈M,A,B〉:

QMAB ⇔ QAA(A ∩ B).

Page 8: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Introduction to GQT

Constraints on GQs

CONS QMAB iff QMA(A ∩ B).

EXT QMAB iff QM′AB for every M ⊆ M ′.

Lemma

A quantifier Q satisfies CONS + EXT iff, for all M = 〈M,A,B〉:

QMAB ⇔ QAA(A ∩ B).

Theorem

A quantifier Q satisfies CONS and EXT if and only if for everyM,M ′ and A,B ⊆ M, A′,B ′ ⊆ M ′, if |A− B | = |A′ − B ′| and|A ∩ B | = |A′ ∩ B ′|, then QMAB ⇔ QM′A′B ′.

Page 9: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Introduction to GQT

GQs as Binary Relations on N

In other words, quantifiers that satisfy CONS and EXT can besummarized succinctly as binary relations on natural numbers.Given Q we define:

QcMxy ⇔ QMAB and |A− B | = x , |A ∩ B | = y .

Standard generalized quantifiers can thus be seen as particularsimple cases.

Page 10: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Introduction to GQT

GQs as Binary Relations on N

In other words, quantifiers that satisfy CONS and EXT can besummarized succinctly as binary relations on natural numbers.Given Q we define:

QcMxy ⇔ QMAB and |A− B | = x , |A ∩ B | = y .

Standard generalized quantifiers can thus be seen as particularsimple cases.

every cM xy ⇔ x = 0

somecM xy ⇔ y > 0

at least threecM xy ⇔ y ≥ 3

mostcM xy ⇔ y > x

an even number of cM xy ⇔ y = 2n for some n ∈ N

Page 11: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterated Quantifiers

Embedded Quantification in Natural Language

To handle sentences such as

(1) (a) One of our neighbors stole all but four of the sunflowers.(b) Three explorers discovered most of the islands.

in which quantified phrases appear both in object position andsubject position, we need to look at so-called polyadic lifts ofmonadic quantifiers. Intuitively, these sentences express complexproperties of the respective transitive verbs. Since these verbs taketwo arguments, it will be impossible to give truth-conditions usingmonadic predicates.

Page 12: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterated Quantifiers

Iterating Type 〈1, 1〉 GQs

If Q1 and Q2 are type 〈1, 1〉, then It(Q1,Q2) will be of type〈1, 1, 2〉, defined:

It(Q1,Q2) A B R ⇔ Q1 A {x | Q2 B Rx}

where Rx = {y | Rxy}.We will sometimes use the alternative notation Q1 · Q2 forIt(Q1,Q2).

2

2This is a special case of a general definition for iterating quantifiers. Fordetails, see Chapter 10 of [PW06].

Page 13: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Outline

1 Generalized QuantifiersIntroduction to GQTIterated Quantifiers

2 Semantic AutomataBasic DefinitionsGeneralized Quantifiers and Regular LanguagesGeneralized Quantifiers and Context-Free Languages

3 Iterating Semantic AutomataIterating Two DFAsIterating With One or More PDAs

4 Experimental ResultsExperiments for Single GQsExperiments for Nested GQsAutomata and LinguisticsReferences

Page 14: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Basic Definitions

Main Idea

We will associate each GQ with a formal language in {0, 1}∗. First,every finite model will get mapped to a string s by

Starting with a model M = 〈M,A,B〉

Enumerating A as ~a

Writing a 0 for each element of A \ B and a 1 for eachelement of A ∩ B

From the earlier results, we have that

M ∈ Q ⇔ 〈#0(s),#1(s)〉 ∈ Qc ,

Page 15: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Basic Definitions

More Formally Defined

Definition

Let M = 〈M,A,B〉 be a model, ~a an enumeration of A, andn = |A|. We define τ (~a,B) ∈ {0, 1}n by

(τ (~a,B))i =

{

0 ai ∈ A \ B

1 ai ∈ A ∩ B

Thus, τ defines the string corresponding to a particular finitemodel.

Page 16: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Basic Definitions

More Formally Defined

Definition

Let M = 〈M,A,B〉 be a model, ~a an enumeration of A, andn = |A|. We define τ (~a,B) ∈ {0, 1}n by

(τ (~a,B))i =

{

0 ai ∈ A \ B

1 ai ∈ A ∩ B

Thus, τ defines the string corresponding to a particular finitemodel.

Definition

For a type 〈1, 1〉 quantifier Q, define the language of Q as

LQ = {s ∈ {0, 1}∗ | 〈#0(s),#1(s)〉 ∈ Qc}

Page 17: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Finite State Automata

The simplest class of automata are the (deterministic) finite-stateautomata:3

Definition

A deterministic finite-state automaton (DFA) is a tuple〈Q,Σ, δ, q0,F 〉:

Q a finite set of states

δ : Q × Σ → Σ a transition function

F ⊆ Q the set of accepting states

Σ a finite set of input symbols

q0 the start state

We denote the components of a DFA M by Q (M), Σ (M), etc.

3For a canonical reference on automata theory, see [HMU01].

Page 18: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Examples of GQ Automata

0

1

0

1

Figure : A finite state automaton for every.

Page 19: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Examples of GQ Automata

0

1

0

1

Figure : A finite state automaton for every.

10

0

1

0

1

Figure : A finite state automaton for at least two.

Page 20: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

First-Order Definability

Not all quantifiers whose languages are accepted by DFAs are FOdefinable:

0 01

1

Figure : A cyclic finite state automaton for an even number of.

Page 21: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Characterizing First-Order Definable GQs

Theorem ([B86], p.156-157)

A quantifier Q is first-order definable iff LQ can be recognized by apermutation-invariant acyclic finite state automaton.

Page 22: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Characterizing First-Order Definable GQs

Theorem ([B86], p.156-157)

A quantifier Q is first-order definable iff LQ can be recognized by apermutation-invariant acyclic finite state automaton.

Lemma (Ehrenfeucht-Fraısse)

Q is FO-definable iff ∃k s.t.

M ≡k M′ ⇒ M ∈ Q iff M′ ∈ Q

where ≡k means Dup has k-round winning strategy in E-F game.

In our case:

A ∼k B ⇔ |A| = |B | = n < k or |A|, |B | ≥ k

and M ≡k M′ iff A ∩ B ,A \ B ,B \ A,M \ (B ∪ A) all bear ∼k totheir primed counterparts.

Page 23: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Proof of Definability Theorem

Proof.⇒:

Interpret E-F “threshold” in “Tree of Numbers”

Page 24: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Proof of Definability Theorem

Proof.⇒:

Interpret E-F “threshold” in “Tree of Numbers”

Transform Tree of Numbers into FSA

Page 25: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Proof of Definability Theorem

Proof.⇒:

Interpret E-F “threshold” in “Tree of Numbers”

Transform Tree of Numbers into FSA

By previous remarks, accepts the right language

By inspection, acyclic and permutation-invariant

Page 26: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Proof of Definability Theorem

Proof.⇒:

Interpret E-F “threshold” in “Tree of Numbers”

Transform Tree of Numbers into FSA

By previous remarks, accepts the right language

By inspection, acyclic and permutation-invariant

⇐: a p.i. acyclic FSA has a finite number of accepting states,each of which has a finite number of paths leading to it, each ofone of the forms: =i 1/ =j 0,=i 1/ ≥j 0,≥i 1/ =j 0,≥i 1/ ≥j 0.Each of these conditions is FO-definable. Take disjunction for allsuch paths for each accepting state. Then take disjunction overthe accepting states.

Page 27: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Regular Languages

Characterizing GQs with Regular Languages

The type 〈1〉 divisibility quantifier Dn is defined:

〈M,A〉 ∈ Dn iff |A| is divisible by n.

Theorem ([Mos91])

Finite state automata accept exactly the class of quantifiers of type〈1, . . . , 1〉 definable in first-order logic augmented with Dn for all n.

Page 28: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Context-Free Languages

Pushdown Automata

Definition

A (non-deterministic) pushdown automaton (PDA) is a tuple〈Q,Σ, Γ, δ, q0,Z0,F 〉:

Q is a finite set of states

Γ is a finite stack alphabet

q0 is the start state

Z0 is the start symbol

Σ is a finite set of input symbols

δ : Q × (Σ ∪ {ǫ})× Γ → P (Q × Γ∗) is a transition function

F is the set of accepting states

Page 29: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Context-Free Languages

Example: Less Than Half

ǫ, 0/ǫ

0, ǫ/01, ǫ/11, 0/ǫ0, 1/ǫ0, 0/001, 1/11

ǫ,Z0/Z0

ǫ, 0/ǫ

Figure : A pushdown automaton for less than half.

Page 30: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Generalized Quantifiers and Context-Free Languages

Characterizing the PDA Quantifiers

Definition

A quantifier Q is first-order additively definable if there is aformula ϕ in the first-order language with equality and an additionsymbol + such that

QcMab ⇔ 〈N,+, a, b〉 |= ϕ(a, b)

Theorem ([B86], p.163-165)

LQ is computable by a pushdown automaton if and only if Q isfirst-order additively definable.

Page 31: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Outline

1 Generalized QuantifiersIntroduction to GQTIterated Quantifiers

2 Semantic AutomataBasic DefinitionsGeneralized Quantifiers and Regular LanguagesGeneralized Quantifiers and Context-Free Languages

3 Iterating Semantic AutomataIterating Two DFAsIterating With One or More PDAs

4 Experimental ResultsExperiments for Single GQsExperiments for Nested GQsAutomata and LinguisticsReferences

Page 32: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Translation of Models

Definition

Let M = 〈M,A,B ,R〉 be a model, ~a and ~b enumerations of A andB , with n = |A|, m = |B |. We overload notation (see Definition 5)by allowing τ to take a relation as an extra argument:

τ(

~a, ~b,R)

=(

τ(

~b,Rai

)

)

i≤n

where τ(

~b,Rai

)

is the translation given in the single-quantifier

case. The operation (·)i≤n concatenates instances of (·) for0, . . . , n. The � functions as a separator symbol in a way that willshortly be made precise.

Page 33: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Example

Example

To see this translation in a concrete example, consider a model〈M,A,B ,R〉 where M = {x , y , z}, A = {x , y}, B = M, and

R = {〈x , y〉, 〈y , x〉, 〈y , y〉, 〈y , z〉}

Let the enumerations ~a and ~b be given alphabetically. Then

τ(

~a, ~b,R)

will be

010 � 111� (1)

Page 34: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Example

Example

To see this translation in a concrete example, consider a model〈M,A,B ,R〉 where M = {x , y , z}, A = {x , y}, B = M, and

R = {〈x , y〉, 〈y , x〉, 〈y , y〉, 〈y , z〉}

Let the enumerations ~a and ~b be given alphabetically. Then

τ(

~a, ~b,R)

will be

010 � 111� (1)

Proposition

Let M = 〈M,A,B ,R〉 be a model and Q1,Q2 quantifiers of type〈1, 1〉. Then for any enumerations ~a and ~b of A and B,

τ(

~a, ~b,R)

∈ LQ1·Q2⇔ 〈M,A,B ,R〉 ∈ It(Q1,Q2)

Page 35: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Iterated Languages

Definition

Let Q1 and Q2 be quantifiers of type 〈1, 1〉. We define thelanguage of Q1 ·Q2 by

LQ1·Q2= {w ∈ (wi�)∗ | i ≤ n, wi ∈ {0, 1}∗ and

card ({wi | wi /∈ LQ2}) , card ({wi | wi ∈ LQ2

})⟩

∈ Qc1}

For w ∈ (wi�)∗, we write numsep (w) for the number of �

symbols in w .

Page 36: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Examples of Iterated Languages

Example

Here are a few examples using some, every, and most. We omitsome of the conditions (e.g., that i ≤ n) to enhance readability.

w ∈ Levery ·some ⇔⟨

card ({wi | wi ∈ Lsome}) ,

card ({wi | wi /∈ Lsome})⟩

∈ every c

⇔ card ({wi | wi /∈ Lsome}) = 0

⇔ card ({wi | #(1) = 0}) = 0

Page 37: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Examples of Iterated Languages

Example

Here are a few examples using some, every, and most. We omitsome of the conditions (e.g., that i ≤ n) to enhance readability.

w ∈ Levery ·some ⇔⟨

card ({wi | wi ∈ Lsome}) ,

card ({wi | wi /∈ Lsome})⟩

∈ every c

⇔ card ({wi | wi /∈ Lsome}) = 0

⇔ card ({wi | #(1) = 0}) = 0

w ∈ Lsome·every ⇔ card ({wi | #(0) = 0}) > 0

One can see that the translation given in (1) will be in Lsome·every .

Page 38: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Examples of Iterated Languages

Example

Here are a few examples using some, every, and most. We omitsome of the conditions (e.g., that i ≤ n) to enhance readability.

w ∈ Levery ·some ⇔⟨

card ({wi | wi ∈ Lsome}) ,

card ({wi | wi /∈ Lsome})⟩

∈ every c

⇔ card ({wi | wi /∈ Lsome}) = 0

⇔ card ({wi | #(1) = 0}) = 0

w ∈ Lsome·every ⇔ card ({wi | #(0) = 0}) > 0

w ∈ Lmost·some ⇔ card ({wi | #(1) > 0}) > card ({wi | #(1) = 0})

One can see that the translation given in (1) will be in Lsome·every .

Page 39: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Preliminary Definitions

Definition

Let M be a DFA. The pushdown reader of M, Mp is defined by

Q (Mp) = Q (M), q0 (Mp) = q0, F (Mp) = F (M) ;

Σ (Mp) = ∅ ;

Γ (Mp) = Σ (M) ;

δ (Mp) = {〈q1, ǫ, r , ǫ, q2〉 | 〈q1, r , q2〉 ∈ δ (M)}.

Page 40: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Preliminary Definitions

Definition

Let M be a DFA. The pushdown reader of M, Mp is defined by

Q (Mp) = Q (M), q0 (Mp) = q0, F (Mp) = F (M) ;

Σ (Mp) = ∅ ;

Γ (Mp) = Σ (M) ;

δ (Mp) = {〈q1, ǫ, r , ǫ, q2〉 | 〈q1, r , q2〉 ∈ δ (M)}.

For an automaton M, let the sign of q ∈ Q (M) be given by

sgn (q) =

{

1 q ∈ F

0 q /∈ F

We define the sign of M as

sgn (M) = sgn (q0 (M))

Page 41: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Central Definition

Definition (Iteration Automaton)

Let Q1 and Q2 be two DFAs accepting LQ1and LQ2

, respectively.The PDA It (Q1,Q2) is given by:

Q = {qI} ∪ Q(

Qp1

)

∪ Q (Q2)

Σ = {0, 1,�}

Γ = {0, 1}

q0 = qI

F = F(

Qp1

)

Page 42: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Central Definition

Definition (Iteration Automaton, cont.)

Transition function:

δ = δ(

Qp1

)

∪ {〈qI , ε, x , sgn (Q2) x , q0 (Q2)〉 | i ≤ n}

∪ {〈q1, 1, x , x , q2〉 | 〈q1, 1, q2〉 ∈ δ (Q2) and sgn (q1) = sgn (q2)}

∪ {〈q1, 0, x , x , q2〉 | 〈q1, 0, q2〉 ∈ δ (Q2) and sgn (q1) = sgn (q2)}

∪ {〈q1, 1, x , xc , q2〉 | 〈q1, 1, q2〉 ∈ δ (Q2) and sgn (q1) 6= sgn (q2)}

∪ {〈q1, 0, x , xc , q2〉 | 〈q1, 0, q2〉 ∈ δ (Q2) and sgn (q1) 6= sgn (q2)}

∪ {〈q,�, x , x , qI 〉 | q ∈ Q (Q2)}

∪{

〈qI , ε, x , x , q0(

Qp1

)

〉}

Page 43: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Example of Iterated Automaton

Example

Here is a PDA for computing some · every .

qI

0, 1/01, x/x

1, 0, x/x

ǫ, x/1x

�, x/x

�, x/x

ǫ, x/x ǫ, 1/ǫ

ǫ, 0/ǫ

Q2

Qp1

Figure : A pushdown automaton for some A R every B.

Page 44: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Key Lemma

Lemma

Let Q1 and Q2 be quantifiers corresponding to regular languagesand wi ∈ {0, 1}∗. Abbreviate It (Q1,Q2) by M.

(1) If wi ∈ LQ2, then (qI ,wi � w ,X ) ⊢∗

M (qI ,w , 1X ) for anyX ∈ Γ∗, w ∈ Σ∗.

(2) If wi /∈ LQ2, then (qI ,wi � w ,X ) ⊢∗

M (qI ,w , 0X ) for anyX ∈ Γ∗, w ∈ Σ∗.

In other words, for any string wi , there is a wi� path through theiterated PDA such that a 1 or 0 is pushed onto the stack accordingto whether or not wi ∈ LQ2

.

Proof.

By induction on the length of wi .

Page 45: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Iterated Machines Accept Iterated Languages

Theorem

Let Q1 and Q2 be quantifiers corresponding to regular languages.The language accepted by It (Q1,Q2) is LQ1·Q2

.

Proof.

⊆: by induction on numsep(w) for w ∈ LQ1·Q2, show that

(qI ,w ,Z0) ⊢∗M (qI , ǫ,X ) where

card ({wi | wi ∈ LQ2}) = #1 (X )

card ({wi | wi /∈ LQ2}) = #0 (X )

⊇: numsep(w) applications of previous lemma and the definitionof iterated PDA

Page 46: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Are PDAs necessary?

Proposition

If LQ1and LQ2

are regular, then so too is LQ1·Q2.

Proof.

Basically closure of regular languages under substitution.

Page 47: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating Two DFAs

Are PDAs necessary?

Proposition

If LQ1and LQ2

are regular, then so too is LQ1·Q2.

Proof.

Basically closure of regular languages under substitution.

Proposition

If LQ1and LQ2

are star-free, then so too is LQ1·Q2.

Proof.

The star-free langauges are exactly those definable in FO[<].a Usethe � separator to change quantifiers over characters to quantifiersover words.

aSee [DG07].

Page 48: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating With One or More PDAs

Some Redefining

Definition

Let M be a PDA. The pushdown reader of M, Mp, is a two-stackpushdown automaton defined by

Q (Mp) = Q (M), q0 (Mp) = q0, F (Mp) = F (M)

Σ (Mp) = ∅

Γ (Mp) = Σ (M)

δ (Mp) = {〈q1, ǫ,X , r , q2, γ, ǫ〉 | 〈q1, r ,X , q2, γ〉 ∈ δ (M)}

The central definition goes exactly as before.

Page 49: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating With One or More PDAs

Example of Iterating With a PDA

Example

Here is a two-stack PDA accepting Lnmost·some .

qI

1, ǫ/ǫ, 0/10, ǫ/ǫ

1, 0, x/x , x/x

ǫ, ǫ/ǫ, x/0x

�, ǫ/ǫ, x/x

ǫ ǫ, 1/ǫ, x/x

ǫ, ǫ/0, 0/ǫǫ, ǫ/1, 1/ǫǫ, 0/ǫ, 1/ǫǫ, 1/ǫ, 0/ǫǫ, 0/00, 0/ǫǫ, 1/11, 1/ǫ

ǫ,Z0/Z0, x/x

ǫ, 1/ǫ, x/x

Page 50: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Iterating With One or More PDAs

Do We Need Two Stacks

Proposition

If LQ1and LQ2

are context-free, then so too is LQ1·Q2.

Proof.

Basically closure of context-free languages under substitution.

Page 51: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Outline

1 Generalized QuantifiersIntroduction to GQTIterated Quantifiers

2 Semantic AutomataBasic DefinitionsGeneralized Quantifiers and Regular LanguagesGeneralized Quantifiers and Context-Free Languages

3 Iterating Semantic AutomataIterating Two DFAsIterating With One or More PDAs

4 Experimental ResultsExperiments for Single GQsExperiments for Nested GQsAutomata and LinguisticsReferences

Page 52: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Experiments for Single GQs

Primary Experimental Results

That some quantifiers are recognizable by FSAs while othersrequire PDAs suggests a difference in cognitive demand: the lattermay require more executive resources, such as working memory.

Page 53: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Experiments for Single GQs

Primary Experimental Results

That some quantifiers are recognizable by FSAs while othersrequire PDAs suggests a difference in cognitive demand: the lattermay require more executive resources, such as working memory.[MCM+05] test this:

12 volunteers at UPenn

120 grammatical sentences, of two types, half of each true

at least three, all, someless than half, an odd/even number

Measured accuracy and did fMRI

Page 54: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Experiments for Single GQs

Primary Experimental Results

Main findings:

Behavioral: Higher-order significantly less accurate thanfirst-order

Imaging: significant differential activation in dorsolateralprefontal and inferior frontal cortices bilaterally.

These areas previously found to be involved in workingmemory.

Page 55: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Experiments for Single GQs

Primary Experimental Results

Main findings:

Behavioral: Higher-order significantly less accurate thanfirst-order

Imaging: significant differential activation in dorsolateralprefontal and inferior frontal cortices bilaterally.

These areas previously found to be involved in workingmemory.

[SZ10a, SZ10b, SZ11] corroborate these results (using behavioralmethods) and also tease apart higher-order quantiers from thoseneeding PDA.

Page 56: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Experiments for Nested GQs

Preliminary Findings for Iterated Quantifiers

Can these techniques be used to tease apart FSA from PDArepresentations of iterated quantifiers? In [SSTZI13], we presentthe first experimental results.

Page 57: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Experiments for Nested GQs

Preliminary Findings for Iterated Quantifiers

Can these techniques be used to tease apart FSA from PDArepresentations of iterated quantifiers? In [SSTZI13], we presentthe first experimental results.

21

Figure : Examples of stimuli used in the study. Sentence ’Every circle isconnected with some square’ is true in situation 1. Sentence ’Some circleis connected with every square’ is true in situation 2.

Page 58: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Experiments for Nested GQs

Preliminary Findings

Table : Means (SD) of all variables and correlations between iterationtask and other cognitive measures

Control Memory Prop15 acc Prop17 acc Mean (SD)

Every-some -.12 -.04 -.14 .02 5019 (2520)readSome-every -.05 -.01 -.15 -.07 4738 (2140)readEvery-some .11 .11 -.01 .01 2506 (1129)verSome-every .10 -.13 -.02 -.10 2079 (867)verEvery-some -.06 .02 .10 -.05 6.48 (1.76)accSome-every -.38** .29* .32** .45** 7.61 (.92)accControl -.26* -.33** -.29* 95.68 (38.22)

Memory .25* .30** 20.89 (2.15)

Prop15 acc .49** 6.86 (1.22)

Prop17 acc 6.88 (1.24)

* p < 0.05** p < 0.01

Note Read - reading time; ver - verification time; acc - accuracy; prop15 - proportional

quantifiers presented with 15 objects, prop17 - proportional quantifiers presented with

17 objects.

Page 59: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Experiments for Nested GQs

Preliminary Findings, cont.

Table : Means (SD) of iterated sentences in true and false conditions,and their correlations with other cognitive measures

Control Memory Prop15 acc Prop17 acc Mean (SD)

Every-some .12 .05 -.02 .05 2231 (870)ver falseEvery-some .09 .20 -.02 -.06 2781 (1569)ver trueSome-every .03 -.12 .10 .05 2468 (1315)ver falseSome-every .16 -.20 -.21 -.19 1690 (752)ver trueEvery-some -.06 .18 .03 .03 3.5 (0.80)acc falseEvery-some -.05 -.09 .05 -.11 2.96 (1.35)acc trueSome-every -.30** .24* .23* .41** 3.72 (0.62)acc falseSome-every -.38** .28* .31** .36** 3.90 (0.42)acc true

Page 60: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Automata and Linguistics

How to Understand the Role of Semantic Automata

In [STI13], we propose understanding semantic automata in termsof Marr’s [Mar82] levels of explanation for theories incomputational psychology:

(1) Computational

(2) Algorithmic

(3) Implementation

Automata can be seen as existing at a level 1.5, specifying abstractcontrol structure of algorithms without particular details aboutprocessing order, saliency features, and the like. This also invites asemantic competence/performance distinction.

Page 61: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Automata and Linguistics

General Conclusion

We hope to have shown that there can still be very fruitfulinteraction between

logic

language

computation

cognition

Page 62: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Automata and Linguistics

References I

[BC81] Jon Barwise and Robin Cooper. Generalized Quantifiersand Natural Language. Linguistics and Philosophy,4(2):159–219, 1981.

[B86] Johan van Benthem. Essays in Logical Semantics. D. ReidelPublishing Company, Dordrecht, 1986.

[DG07] Volker Diekert and Paul Gastin. First-order definablelanguages. In Jorg Flum, Erich Gradel, and Thomas Wilke,editors, Logic and Automata: History and Perspectives,Texts in Logic and Games, pages 261–306. AmsterdamUniversity Press, Amsterdam, 2007.

[HMU01] John E. Hopcroft, Rajeev Motwani, and Jeffrey D.Ullman. Introduction to Automata Theory, Languages, andComputation. Addison Wesley, Boston, 2nd edition, 2001.

Page 63: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Automata and Linguistics

References II

[Kee96] Edward L. Keenan. The Semantics of Determiners. InShalom Lappin, editor, The Handbook of ContermporarySemantic Theory, number 1989, chapter 2, pages 41–63.Blackwell, Oxford, 1996.

[KW11] Edward L. Keenan and Dag Westerstahl. GeneralizedQuantifiers in Linguistics and Logic. In Johan van Benthemand Alice ter Meulen, editors, Handbook of Logic andLanguage, pages 859–910. Elsevier, second edition, 2011.

[Lin66] Per Lindstrom. First order predicate logic with generalizedquantifiers. Theoria, 32:186–195, December 1966.

[Mar82] David Marr. Vision. Freeman, San Francisco, 1982.

Page 64: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Automata and Linguistics

References III

[MCM+05] Corey T. McMillan, Robin Clark, Peachie Moore,Christian Devita, and Murray Grossman. Neural basis forgeneralized quantifier comprehension. Neuropsychologia,43(12):1729–1737, January 2005.

[Mos57] Andrzej Mostowski. On a generalization of quantifiers.Fundamenta Mathematicae, 44:12–36, 1957.

[Mos91] Marcin Mostowski. Divisibility Quantifiers. Bulletin of theSection of Logic, 20(2):67–70, 1991.

[PW06] Stanley Peters and Dag Westerstahl. Quantifiers inLanguage and Logic. Clarendon Press, Oxford, 2006.

[SSTZI13] Jakub Szymanik, Shane Steinert-Threlkeld, MarcinZajenkowski, and Thomas F. Icard III. Automata andComplexity in Multiple-Quantifier Sentence Verification. InProceedings of the 12th International Conference onCognitive Modeling, 2013.

Page 65: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Automata and Linguistics

References IV

[STI13] Shane Steinert-Threlkeld and Thomas F. Icard III.Iterating Semantic Automata. Linguistics and Philosophy,2013.

[SZ10a] Jakub Szymanik and Marcin Zajenkowski. Comprehensionof simple quantifiers: empirical evaluation of acomputational model. Cognitive Science, 34(3):521–532,April 2010.

[SZ10b] Jakub Szymanik and Marcin Zajenkowski. Quantifiers andWorking Memory. Lecture Notes in Artificial Intelligence,6042:456–464, 2010.

[SZ11] Jakub Szymanik and Marcin Zajenkowski. Contribution ofworking memory in parity and proportional judgments.Belgian Journal of Linguistics, 25(1):176–194, January 2011.

Page 66: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Automata and Linguistics

References V

[Wes89] Dag Westerstahl. Quantifiers in Formal and NaturalLanguages. In D Gabbay and F Guenthner, editors,Handbook of Philosophical Logic (vol. 4), pages 1–132. D.Reidel Publishing Company, Dordrecht, 1989.

Page 67: Iterating Semantic Automata - Stanford University · Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results ... The generalization to higher-order

Generalized Quantifiers Semantic Automata Iterating Semantic Automata Experimental Results

Automata and Linguistics

Thank You

Questions?