Language Mind and Brain: The Unification Problem Colin Phillips Cognitive Neuroscience of Language...

Language Mind and Brain:The Unification Problem

Colin PhillipsCognitive Neuroscience of Language Laboratory

Department of LinguisticsUniversity of Maryland

• Objective is to bridge gap between linguistic models, real-time models of mental processes, and brain-level models, i.e., focus is on the Unification Problem

• Cognitive Neuroscience of Language necessarily draws on disparate areas - not just looking at pictures of brains!

Unification Problem

• No longer concerned with questions of whether linguistics is a natural science

• Language is clearly a remarkable specialization of human neurobiology (e.g., a sophisticated symbolic, recursive system, with fixed and parameterized aspects)

• What is it about human psychology & neurobiology that allows it to support the things that we know about language?

• In addressing this question, are we faced with problems or with mysteries?

Unification Problem

• ...And if there are mismatches between theories at different levels, whose problem is this?

Linguists?

Psychologists?

Neuroscientists?

Unification Problem

On mismatches between cognitive and neural models

“The more we learn about the brain,the greater the disanalogy becomes.”

(A philosophy of (neuro-)science talk, October 2001)

Unification Problem

On mismatches between cognitive and neural models

“If language is unlike anything else in the biological world,…then too bad for biology!”

(linguist, often accompanied by story about chemistry & physics)

Encoding & Computation

• Two main issues– How are linguistic representations encoded?– How are linguistic representations computed?

Sensory Maps

Internal representations of the outside world. Cellular neuroscience has discovered a great deal in this area.

Notions of sensory maps may be applicable to human phonetic representations…

…although attempts to find them have had little success to date.

Vowel Space

Encoding of Symbols: Abstraction

• But most areas of linguistics (phonology, morphology, syntax, semantics) are concerned with symbolic, abstract representations,

...which do not involve internal representations of dimensions of the outside world.

…hence, the notion of sensory maps does not get us very far into language

Computation: Discrete Infinity

• In neuroscience there are many findings about long-term storage of object representations (e.g., edges, faces, grandmothers, toothbrushes, …).

• Such representations are always finite in number, which can be retrieved from long-term memory

• BUT, much of what interests us in linguistics is infinite

• If representations are drawn from an infinite set, they cannot be retrieved from long-term memory; they must be constructed on-line, as needed - this poses a different kind of challenge …

Overview of Talks

1. The Unification Problem

Overview of Talks

2. Building Syntactic Relations

In-situ

1 2 3 4 5 6 7 8

Region

Reading Time

どの生徒に…

Overview of Talks

3. Abstraction: Sounds to Symbols

In-situ

1 2 3 4 5 6 7 8

Region

Reading Time

どの生徒に…

Overview of Talks

3. Abstraction: Sounds to Symbols

4. Linguistics and Learning

In-situ

1 2 3 4 5 6 7 8

Region

Reading Time

どの生徒に…

with help from ...University of Maryland

Shani AbadaSachiko Aoshima

Daniel Garcia-PedrosaAna Gouvea

Nina KazaninaMoti LiebermanLeticia PablosDavid PoeppelBeth RabbinSilke Urban

Carol Whitney

University of Delaware

Evniki EdgarBowen HuiBaris KabakTom Pellathy

Dave SchneiderKaia Wong

Alec Marantz, MITElron Yellin, MIT

National Science FoundationJames S. McDonnell Foundation

Human Frontiers Science ProgramJapan Science & Technology Program

Kanazawa Institute of Technology

Outline

• The Challenge• Real-time Grammar• Accurate Parsing• Incremental Parsing• Mapping onto the Brain: Electrophysiology• Encoding• Outlook

Outline

Discrete Infinity

• Linguistic CreativityAbility to make infinite use of finite means

• The finite and infinite aspects of this system present rather different challenges for explicit linking hypotheses

Discrete Infinity

• Lexical entries, Argument Structure templates• Widespread assumption: same representations

accessed in comprehension, production, acceptability ratings, etc.

• Learner’s task is to construct a single lexical entry that covers all of these tasks

Discrete Infinity

• Sentence structures• Structures must be assembled, cannot simply be

retrieved from memory• Widespread assumption: multiple different systems

responsible for structure assembly in comprehension, production, acceptability, etc.

• Learner must master a number of different systems

Discrete Infinity

• Time-independent vs. Time-dependent systems

Standard View

324697+ ?

217 x 32 = ?

arithmetic

Standard View

324697+ ?

217 x 32 = ?

specialized algorithmspecialized algorithm

arithmetic

Standard View

324697+ ?

217 x 32 = ?

something deeper

arithmetic

Standard View

speaking understanding

grammaticalknowledge,competencelanguage

Standard View

grammaticalknowledge,competence

precisebut ill-adapted toreal-time operationlanguage

Standard View

grammaticalknowledge,competence

well-adapted toreal-time operationbut maybe inaccuratelanguage

If speaking and understanding involve different systems, there must be an additional store of knowledge that encodes what is shared between speaking and understanding.

As soon as we assume constructs such as ‘parsing strategies’, we are adopting task-specific mechanisms, and endorsing something like the standard architecture.

Standard View

grammaticalknowledge,competencelanguage

speaking,understanding,grammaticality

Analysis-by-Synthesis

• Sentences are understood by internally generating a representation that matches the input

• No separate time-independent system of knowledge

• We know that humans have a real-time system for linguistic computation - issue is whether that’s all there is

But wait a minute...

• Wasn’t this all shown to be wrong long ago?

(Fodor, Bever & Garrett 1974; Levelt 1974; Fillenbaum 1971; etc., etc.)?

• And a recent commentary:

“In this desert of ignorance there have been attempts to resurrect earlier claims that the grammar and the parser are one and the same thing. … The enterprise is misconceived … probably incoherent.” (Smith, 1999)

Motivation for Standard Architecture

• How to constrain hypothesis generation• Grammars are not incremental real-time systems• Evidence for input/output-specific strategies• Analysis-by-synthesis implies active generation,

ahead of input• Reputation of performance systems as fast but

inaccurate• Debunking of ‘Derivational Theory of Complexity’

• Time-dependent system of computation makes it feasible to generate testable linking hypotheses

Motivation for Alternative Architecture

Outline

Linear Order and Constituency

Linguistic Inquiry, 2003

Incremental Structure Building

• Evidence that sentence structures can only be assembled in a left-to-right derivation.

ate the entire pizza

Constituents

• Many tools used to diagnose groupings of words:

– coordination

– deletion

– interpretation (coreference)

– movement, focus, topicalization

– etc.

• There are many cases where the tools converge on the same result

• There are also many cases where the tools yield conflicting results

(Phillips 2003)

A B constituent

(Phillips 2003)

B Cconstituent is destroyed by addition of new material

(Phillips 2003)

B Cconstituent

(Phillips 2003)

C Dconstituent is destroyed by addition of new material

(Phillips 2003)

the cat

(Phillips 2003)

the cat sat

(Phillips 2003)

the cat

sat on

(Phillips 2003)

the cat

on the rug

(Phillips 2003)

the cat

sat on

(Phillips 2003)

the cat

on the rug

(Phillips 2003)

the cat

on the rug

[sat on] is a temporary constituent, which is destroyed as soon as the NP [the rug] is added.

(Phillips 2003)

Conflicting Constituency Tests

Verb + Preposition sequences can undergo coordination…

(1) The cat sat on and slept under the rug.

…but cannot undergo pseudogapping (Baltin & Postal, 1996)

(2) *The cat sat on the rug and the dog did the chair.

(Phillips 2003)

the cat

sat on

(Phillips 2003)

the cat

sat on slept under

(Phillips 2003)

the cat

sat on slept under

coordination applies early, before the V+P constituent is destroyed.

(Phillips 2003)

the cat

sat on

(Phillips 2003)

the cat

on the rug

(Phillips 2003)

the cat

on the rug

and the dog did

(Phillips 2003)

the cat

on the rug

and the dog did

pseudogapping applies too late, after the V+P constituent is destroyed.

(Phillips 2003)

• Constituency ProblemDifferent diagnostics of constituency frequently yield conflicting results

• Incrementality Hypothesis(a) Syntactic processes see a ‘snapshot’ of a derivation - they target constituents that are present when the process applies(b) Conflicts reflect the fact that different processes have different linear properties

Ellipsis blocks Scope/Binding

Bill read all the books in a week (ambiguous: collective/distributive scope)

…and Sue did in a month (unambiguous: collective scope only)

Bill read as many books as Sue did in a week. (ambiguous)

Bill read as many books in a week as Sue did in a month. (unambiguous)

Ellipsis blocks Scope/Binding

John gave books to the children on Tuesday

…and Mary did on Thursday

John gave books to the children on each other’s birthdays

*…and Mary did on each other’s first day of school

Japanese

to the children

on each other’s birthdays

all the books

in a week

to the children

all the books

in a week

to the children

on each other’s birthdaysVP

to the children

to the childrenV

to the children

to the childrenV

on each other’s first day of schoolVP

Movement & Binding

a. John gave books to them on each other’s birthdays.

Movement & Binding

to them

(Pesetsky 1995)

Movement & Binding

to them

(Pesetsky 1995)

Movement & Binding

b. …and [give books to them] he did ___ on each other’s birthdays

(Pesetsky 1995)

to them

he did

to them

he did

to them

constituentmovement

to them

constituentmovement

to them

constituentmovement

binding underc-command

Interim Conclusion

• By building syntactic structures from left-to-right we can explain a number of otherwise mysterious constituency phenomena (see Phillips 1996, 2003 for more examples; see Richards 1999, 2002 for some applications to Japanese)

• We knew independently that humans have a left-to-right structure-building system (i.e. parser, producer)

• Possibility arises that the incremental left-to-right system is the only structure-building system that humans have

• Other arguments leading to related conclusions about grammar, in widely varying formalisms: Kempson et al. (2001), Steedman (2000), Kempen (1999), Milward (1992, 1994)

Outline

Grammatical Accuracy

• It is not enough to show that syntactic structure-building looks like a real-time process

• If the real-time system is the only system, then it should also show the syntactic sophistication normally associated with the grammar - the parser cannot be ‘dumb’

• Question: does the parser access only grammatically legal structural analyses?

Beyond Ambiguity

• Much of parsing literature focuses on issues of ambiguity,

i.e. when 2 structures are grammatically possible, how to choose the right one?

• More basic question: grammatical search

i.e. how do we figure out if a sequence of words has any grammatical analyses?

Example: Argument Structure

Dative and Double-Object Constructions

Alternator Verb: give

The millionaire gave the painting to the museum.

The millionaire gave the museum the painting.

Non-alternator Verb: donate

The millionaire donated the painting to the museum.

*The millionaire donated the museum the painting.

(Phillips, Edgar & Kabak, 2000)

A Severe Garden-Path Sentence

The man gave the boy the dog bit a cookie

The man gave [the boy [the dog bit]] a cookie

availability of double-object parseleads to difficulty at embedded verb

Non-Alternator Verb: donate

The man donated [the boy [the dog bit …

availability of double-object parseleads to difficulty at embedded verb

difficulty should ariseearlier in the sentence

Results (partial)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Region

G alternators, ambig

B alternators, unambig

NP1 NP2 verb

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Region

C datives, ambig

H datives, unambig

NP1 NP2 verb

Relative to unambiguous control sentence, readers get intodifficulty at V with alternator verbs, at NP2 with non-alternators.--> Argument structure constraint immediately active on-line

Alternators (e.g. give) Non-Alternators (e.g. donate)

Example: Movement Constraints

Grammatical Accuracy in Parsing

Wh-Questions

Englishmen cook wonderful dinners.

Wh-Questions

Englishmen cook wonderful dinners.

Wh-Questions

Englishmen cook what

Wh-Questions

Englishmen cook what

Wh-Questions

What do Englishmen cook

Wh-Questions

What do Englishmen cook gap

Wh-Questions

What do Englishmen cook gap

Long-distance Wh-Questions

Few people think that anybody realizes that Englishmen cook wonderful dinners

Few people think that anybody realizes that Englishmen cook what

What do few people think that anybody realizes that Englishmen cook gap

The plan to remove the equipment ultimately destroyed the building.

Direct Object NP Direct Object NP

Main Clause

Subject NP

Embedded Clause

What did the plan to remove the equipment ultimately destroy

What did the plan to remove the equipment ultimately destroy gap

What did the plan to remove ultimately destroy the building

What did the plan to remove gap ultimately destroy the building

Subject

Island Constraint

A wh-phrase cannot be moved out of a subject.

Question…

• Do people respect island constraints on movement immediately on-line?

(tomorrow’s talk)

Outline

Incrementality

• Question: is structure building immediate?

Does it operate on a word-by-word level?

…Even in a language where this may be hard?

Incremental Application of Binding Constraints in Japanese

Sachiko AoshimaColin Phillips

Amy Weinberg

Structure-building in Japanese

• NP-wa NP-ni [NP-ga NP-o V] V

• Head-driven Parsing (e.g. Pritchett 1991, Mulders 2002)Structure-building is delayed until verbs are processed– explains how parsing is possible in Japanese

– accounts for flexibility, limited garden-paths in Japanese

• Incremental ParsingStructure-building occurs immediately– accounts for native-speaker intuition of continuous comprehension

– hard to demonstrate experimentally

John-ga …

(Mazuka & Itoh 1995)

John-ga Mary-ni …

John-ga Mary-ni ringo-o …

John-ga Mary-ni ringo-o tabeta …

John-ga Mary-ni ringo-o tabeta inu-o ageta

Verb Surprise Effects

• Surprise effect appears at verb

Can be explained by both head-driven and incremental theories (Schneider 1999, Mulders 2002)

• Better: evidence of structure-building that precedes the verb

e.g. immediate application of grammatical constraints

English

To which of his children did the man give a gift

(Aoshima, Phillips & Weinberg 2002)

English

To which of his children did the man give a gift

Which of his children gave the man a gift?

Japanese

which of his children (DAT) the man (NOM) …

which of his children (NOM) the man (DAT) …

Japanese pronoun and its antecedent

which of his children (DAT) the man (NOM) … his

which of his children (NOM) the man (DAT) …**??

which of his children (DAT)

Gender Mismatch

the woman

Gender Mismatch

which of his children (DAT) the man (NOM) … his

which of his children (NOM) the man (DAT) …**??

which of his children (DAT)

the womanGender mismatch

the womanGender mismatch irrelevant

Conditionsa. Scrambled - Gender Mismatch

Adverb / [his / which NP]-dat / Adverb / NP FEMALE-nom / Adverb / NP-acc /

verb-Q / NPMALE-top / verb

b. Scrambled - Gender Match

Adverb / [his / which NP]-dat / Adverb / NP MALE-nom / Adverb / NP-acc /

verb-Q / NPFEMALE-top / verb

c. Non-scrambled - Gender Mismatch

Adverb / [his / which NP]-nom / Adverb / NP FEMALE-dat / Adverb / NP-acc /

verb-Q / NPMALE-top / verb

d. Non-scrambled - Gender Match

Adverb / [his / which NP]-nom / Adverb / NP MALE-dat / Adverb / NP-acc /

verb-Q / NPMALE-top / verb.

Examples

a. 台所で　彼の　どの子供に　朝食後　叔母が　急いで　お弁当を　渡したか　父親は　覚えていた。

b. 台所で　彼の　どの子供に　朝食後　叔父が　急いで　お弁当を　渡したか　叔母は　覚えていた。

c. 台所で　彼の　どの子供が　朝食後　叔母に　急いで　お弁当を　渡したか　父親は　覚えていた。

d. 台所で　彼の　どの子供が　朝食後　叔父に　急いで　お弁当を　渡したか　父親は　覚えていた。

Design & Procedure

• 2 X 2 factorial design• 4 lists were created by distributing 24 items in a

Latin Square design• 56 filler sentences• Comprehension questions: matching a subject

with a predicate• Self-paced reading task -Moving Window -• 40 native speakers of Japanese

Self-paced reading task

----- --- --- ---- ---- --- ------ -------

どの子供に --- --- ---- ---- --- ------ -------

----- 叔母は --- ---- ---- --- ------ -------

----- --- 母親が ---- ---- --- ------ -------

----- --- --- ケーキを ---- --- ------ -------

----- --- --- ---- 焼いたと --- ------ -------

----- --- --- ---- ---- 台所で ------ -------

----- --- --- ---- ---- --- お手伝いさんに -------

----- --- --- ---- ---- --- ------ 知らせましたか。

Results: Scrambled conditions

Slowdown at mismatching NP is observed.

1 2 3 4 5 6 7 8 9 10

scramble,match

scramble,mismatch

his/her

± Match F1(1, 39) = 8.6, p<.01; F2(1,23)=7.4, p<.01

Results: Non-scrambled conditions

Slowdown at mismatching NP only when NP is possible antecedent.

1 2 3 4 5 6 7 8 9 10

unscr,match

unscr,mismatch

his/her

± Match FS<1

Summary: Experiment 3

NP-nom

HIS-WH

Binding constraint application takes place in advance of the verb.

Wh-gap is posited in a simple clause.

Experiment 3 (off-line):Grammatical judgment test

Experiment 3 (off-line): Stimuli

a. Non-wh, Non-scrambled

[His children]-nom Adv the man-dat

b. Non-wh, Scrambled

[His children]- dat Adv the man-nom

c. Wh, Non-scrambled

[Which of his children]-nom Adv the man-dat

d. Wh, Scrambled

[Which of his children]-dat Adv the man-nom

Experiment 3 (off-line):Design & Procedure

• 4 lists were created by distributing 32 items in a Latin Square design

• 16 items: same materials from online test, and 16 items: different from those in online test.

• 32 filler sentences

• Anaphoric relation judgment task

• 40 native speakers of Japanese, same individuals as the online test

Experiment 3 (off-line):Results

• Backwards anaphora are more allowed in scrambled conditions.

• It confirms that the binding facts underlying in the online test are correct.

Non-wh, Unscr Non-wh, Scr Wh, Unscr Wh, Scr

Grammatical Search and Reanalysis

David SchneiderColin Phillips

Journal of Memory & Language, 2001

• Previous study showed incremental application of grammatical constraints,

i.e. derivations operate on a word-by-word time-scale

• Next study: do real-time derivations show consistency,

i.e does structure-building keep to a single derivation?how does is grammar searched to find possible analyses?

Grammatical Search

the man

knows the womanNP

(Schneider & Phillips, 2001)

Grammatical Search

the man

knows the womanNP

likesV

Grammatical Search

the man

the womanNP

likesV

Grammatical Search

the man

the womanNP

likesV

It’s clear that this is the right conclusion, but it’s less clear how the system reaches this conclusion.

Grammatical Search

the man

knows the womanNP

likesV

Option 1: combine with a local NP, ignoring existing status of the NP.

Grammatical Search

the man

knows the womanNP

likesV

Option 1: combine with a local NP, ignoring existing status of the NP.

Grammatical Search

the man

knows the womanNP

likesV

Option 2: search the structure for an NP subject that currently lacks a -role,i.e., focused search.

Grammatical Search

the man

knows the womanNP

likesV

Grammatical Search

the man

knows the womanNP

likesV

This fails, so reanalysis is needed, … but only as a last resort operation.

the man

knows the womanNP

likesV

Test CaseIf there is a higher NP, currently lacking a -role, a focused search will find it.

the man

knows the womanNP

likesV

the man

knows the womanNP

likesV

STest CaseIf there is a higher NP, currently lacking a -role, a focused search will find it.

the man

knows the womanNP

An unconstrained search will not be affected by the presence of the higher NP.

likesV

the man

the womanNP

likesV

the man

the womanNP

likesV

ProbeAntecedents for reflexives.

the man

the womanNP

likesV

...the recipe herself

...the recipe himself

the man

knows the womanNP

likesV

...the recipe herself

...the recipe himself

Grammatical Search

Relative to its unambiguous control, high attached reflexives pose no difficulty.

Grammatical Search

Relative to its unambiguous control, high attached reflexives pose no difficulty.

…but low attached reflexives present great difficulty.

Therefore...High attachment is chosen

…despite well-known local-attachment biases.

the man

knows the womanNP

likesV

Grammatical search is focused, constrained by existing commitments

Therefore...High attachment is chosen

…despite well-known local-attachment biases.

Grammatical search is focused, constrained by existing commitments

knows the womanNP

likesV

the manNP

This helps us to understand how grammatical search proceeds in simple cases.

Conclusions

• Local attachment is easy• The local reanalysis is easy (cf. Sturt et al. 2000)• So why is local attachment avoided?

Reanalysis is a Last Resort (even if it’s easy)

Reversal of results…

1 3 4 5 6 7 8 9 10 11 12 13 14

Sentence Region

H High Ambiguous

B High Unambiguous

C Low Ambiguous

G Low Unambiguous

verb reflexive

H H H HH

B B BB

1 3 4 5 6 7 8 9 10 11 12 13 14

Sentence Region

H High Ambiguous

B High Unambiguous

C Low Ambiguous

G Low Unambiguous

verb reflexive

NP-biased

hear, warn,understand

S-biased

claim, believe,suspect

Outline

Time Resolution

• Syntax: phrase-by-phrase time scale

• Reading-time studies: word-by-word time scale

• Brain recordings: millisecond time scale

Event-Related Potentials (ERPs)

s1 s2 s3

John is laughing.

Evolving understanding of ERP components associated with language…

Semantically unexpected input

• She spread the warm bread with socks.(Kutas & Hillyard,

• She was stung by a fly.(Kutas, et al., 1984;

Federmeier & Kutas, 1999)

(Slide from Kaan (2001)

• Negative polarity• peaking at around 400 ms• central scalp distribution

ERP Sentence Processing

• Developing understanding of N400 is informative

• Response to ‘violations’

I drink my coffee with cream and sugarI drink my coffee with cream and socks

Kutas & Hillyard (1980)

Vervet Monkeys

• Many predators: leopard, lionhyena, jackal, eagle, etc. etc.

• Distinct alarm calls for different predators

ERP Sentence Processing

• Developing understanding of N400 is informative

• Response to normal sentences

Fully CongruentMost new drugs are tested onwhite lab rats.

Van Petten & Kutas (1991)

ERP Sentence ProcessingN400

• N400 to semantic anomalies is a special case of a much more general phenomenon

• All words elicit N400-like response, timing and amplitude proportional to congruency, frequency, etc.

• More detailed understanding is contingent on more detailed models of semantic interpretation

Morpho-Syntactic violations

Every Monday he mows the lawn.

Every Monday he *mow the lawn.

The plane brought us to paradise.

The plane brought *we to paradise.(Coulson et al., 1998)

he mowshe *mow

Left Anterior Negativity (LAN)

ERP Sentence ProcessingLAN, P600

Sie bereist dasneuter Landneuter …Sie bereist denmasculine Landneuter …she travels the land ... Gunter et al. (2000)

Kaan et al. (2000)

ERP Sentence ProcessingP600

Emily wondered who the performer in the concert had imitated for the audience’s amusement.Emily wondered whether the performer in the concert had imitated a pop star for the audience’s amusement.

• P600 reflects normal structure-building processes.

Electrophysiology of Wh-movement

Colin Phillips, Nina Kazanina, Shani Abada, Daniel Garcia-Pedrosa

[revision of work done at UDel in 2000]

Experiment DesignMaterials

a. The actress wished that the producers knew that the witty host would tell the jokes during the party.

b. The actress wished that the producers knew which jokes the witty host would tell __ during the party.

c. The producers knew that the actress wished that the witty host would tell the jokes during the party.

d. The producers knew which jokes the actress wished that the witty host would tell __ during the party.

b. The actress wished that the producers knew which jokes the witty host would tell during the party.

d. The producers knew which jokes the actress wished that the witty host would tell during the party.

Short wh-dependency

b. The actress wished that the producers knew which jokes the witty host would tell during the party.

d. The producers knew which jokes the actress wished that the witty host would tell during the party.

Long wh-dependency

-100 0 100 200 300 400 500 600 700 800 900 1000

Electrode PZ Short Conditionsn=20

Effect of wh-movement significant (p<.01) from 300-400ms onwards

-100 0 100 200 300 400 500 600 700 800 900 1000

Electrode PZ Long Conditionsn=20

Effect of wh-movement significant (p<.01) from 300-400ms onwards

How fast is Structural Computation?

Silke UrbanColin Phillips

Background

Early Left Anterior Negativity

(Angela Friederici, Anja Hahne, et al.)

Neville et al., 1991

The scientist criticized a proof of the theorem.

The scientist criticized Max’s of proof the theorem.

500ms/word

Hahne & Friederici, 1999

Das Baby wurde gefüttertThe baby was fed

Das Baby wurde im gefüttertThe baby was in-the fed

Question: are the brain responses to violations automatic?

• Very fast: 150-250ms• Automatic• Left anterior (= frontal) scalp distribution• Elicited by a subclass of syntactic violations

“Phase 1 (100–300 ms) represents the time window in which the initial syntactic structure is formed on the basis of information about the word category.”

(Friederici 2002)

Questions about ELAN

• How plausible is it that ELAN reflects syntactic structure building?

– Speed: 150ms is faster than lexical access!

– Generality: ELAN is not elicited by most violations; almost all studies on ELAN involve one construction (for each of German, English)

– Localization…

Brodmann Areas

160 SQUIDwhole-headarray

pickup coil & SQUIDassembly

Magnetoencephalography (MEG)

(Friederici et al. 2000)

Two regions of interest

(Friederici et al. 2000)

Anterior Temporal Lobe?

• Why is anterior temporal lobe so important in ELAN?

• How does it differ from Broca’s area (BA 44 etc.) that are implicated so often in other studies?

• Friederici: both responsible for ‘structure building’; BA44 also responsible for ‘syntactic working memory’; ‘the inferior portion of BA44 is selectively activated when syntactic processes are in focus.’

• Anterior temporal lobe associated with– lexical information

– activated in fMRI by comparisons of sentences with word lists

Alternative Interpretation

• ELAN reflects violation/suppression of automatic lexical prediction– accounts for localization to anterior temporal lobe

– accounts for very early timing

– might account for automaticity

– accounts for very limited distribution

Max’s N

Das Baby wurde gefüttertThe baby was fed

Das Baby wurde im gefüttertThe baby was in-the fed

Prediction

• If ELAN reflects violation of lexical prediction, rather than syntactic structure-building, then…

– change lexical predictions

– keep syntactic violation the same

– should ‘turn off’ ELAN brain response

Max’s N

Possible to block the automatic prediction of an N following a possessor: ellipsis

Ellipsis

• Possessors may appear alone in ellipsis contexts:

Although I like Mary’s theory, I don’t like John’s.

Experimental Conditions

Although Erica kissed Mary’s mother, she did not kiss the daughter of the bride.

Although Erica kissed Mary’s mother, she did not kiss Dana’s of the bride.

Although the bridesmaid kissed Mary, she did not kiss the daughter of the bride.

Although the bridesmaid kissed Mary, she did not kiss Dana’s of the bride.

Experimental Conditions

Although Erica kissed Mary’s mother, she did not kiss the daughter of the bride.

Although Erica kissed Mary’s mother, she did not kiss Dana’s of the bride.

Although the bridesmaid kissed Mary, she did not kiss the daughter of the bride.

Although the bridesmaid kissed Mary, she did not kiss Dana’s of the bride.

ellipsis possible

ellipsis impossible

Experimental Design

• 384 sentences per session– 128 targets (drawn from 128 sets of 4 conditions)

– 64 items designed to elicit ‘agreement violation’ LAN

– 192 filler items, designed to hide violations and promote ellipsis

• Procedure– RSVP (Rapid Serial Visual Presentation), 500ms/word

– Grammaticality judgment task

• Recording: 32-electrode montage

• 22 subjects (so far)

Although

kissed

Mary’s

mother,

Dana’s

bride.

good bad

Preliminary Results

a. Although … Mary’s mother … Dana’s of …b. Although … Mary … Dana’s of …

Interim Conclusion

• Preliminary results lend support to our interpretation of the (E)LAN - the anterior negativity is reduced in an ellipsis context

– structural violation is identical in both conditions

– obligatory lexical prediction of N following possessor (e.g. Mary’s…) is absent in ellipsis context

• Structure-building may begin ~250-300ms after a word is presented

Outline

The Binding Problem

• Discrete infinity

Individual neurons (or groups of neurons) can store finite information about objects, words, etc.

But sentences are infinite in number!

• Representing structure: can’t just activate all words

e.g. THE + MAN + ATE + PIZZA

• Must create & discard structures quickly: 100s of msec

Temporal Binding

phaselocked

~25ms, 40Hz

Evidence from Animals & Humans

• Direct recordings (cat) • EEG recordings (human)

(Singer 1999) (Tallon-Baudry & Bertrand 1999)

Limitations…

• No evidence yet of role in human syntax• Limited capacity - ~7 bindings• 1-level of hierarchy only

• Interesting hypothesis (Whitney & Weinberg 2002)– temporal binding is the neural representation used for the

‘syntactic workspace’

– additional neural encoding mechanism used for long-term representation and storage

Outline

Outlook• Overview

– challenge for unification: real-time hypotheses

• Real-time Grammar– syntactic derivations look like real-time derivations

• Accurate Parsing– real-time derivations have the sophistication that is needed

• Incremental Parsing– real-time interpretation is time-locked to incoming words

• Mapping onto the Brain: Electrophysiology– extreme time-precision of EEG/MEG can be linked to detailed

linguistic constructs

• Encoding– plausible models of neural encoding of structure are emerging

• Unification: Problem or Mystery…

www.ling.umd.edu/colin

colin@umd.edu

Language Mind and Brain: The Unification Problem Colin Phillips Cognitive Neuroscience of Language...

Documents

Transcript of Language Mind and Brain: The Unification Problem Colin Phillips Cognitive Neuroscience of Language...

Language Unification, Labor and Ideology · integration. Similarly, the Italian spoken today was known to less than 3% of the population when Italian unification began in 1861. Long

Unification Theory

Italian unification

Left-Corner Unification-Based Natural Language Processing

Electroweak Unification

Japanese Unification

Unification Grammars

Language and the Brain - Linguistics at Marylandling.umd.edu/~colin/research/papers/phillips_sakai2005.pdf · Language and the Brain Colin Phillips, University of Maryland & Kuniyoshi

No language unification without neural feedback: How ...

Structure Building and Unificationling.umd.edu/~colin/research/papers/phillips_cunygc2001.pdf · 2008-01-08 · Colin Phillips, Supper @ CUNY, 3/27/01 1 Structure Building and Unification

Unification Benefit

Italian Unification. Map of Italy before Unification.

ITALIAN AND GERMAN UNIFICATION 1852-1871. ITALIAN UNIFICATION PROCESS.

Unification: A Multidisciplinary Survey7. UNIFICATION AND FEATURE STRUCTURES 8. UNIFICATION AND NATURAL LANGUAGE PROCESSING 8.1 Parsing with a Unification-Based Grammar 8.2 Research

Germany Unification

Boolean Unification

UNIFICATION THOUGHT

German unification

The role of neural feedback in language unification: How ...

Unification Review