1

Functional decomposition of spoken word production and its control

Ardi Roelofs

F.C. Donders Centre for Cognitive Neuroimaging

Max Planck Institutefor Psycholinguistics

Nijmegen Institute for Cognition and Information

Collège de France, Paris, Jeudi 11 Mai 2006

We produce about 2 to 5 words per second.

About 1 error per 1,000 words.

Vocabulary of more than 30,000 words.

A few numbers

With 1 hour talking per day, one produces

about 4.000.000 words per year,

about 200.000.000 words in half a century,

i.e., each word, on average, about 6500 times.

POIRE

2

“I can resist everything except temptation.”

Oscar Wilde

“Une distraction, à l’instant où l’irritationagit, est toujours punie par uneprolongation de durée de l’acte.”

F. C. Donders (1868)

(Archives néerlandaises des sciences exactes et naturelles, 1868)

Subtraction method

“[One] discovers in comparing the incoming and outflowing blood that oxygen has been consumed [by the brain].”

F. C. Donders (1868)

1868 2006

(2004)

Left-lateralized perisylvian network

3

J. Ridley Stroop(1935) CAFÉ

CLÉTANGO

BLEUVERT

ROUGE

Stroop asymmetry

J. Ridley Stroop(1935)

BLEU ROUGE XXXTask: Name ink color!

J. Ridley Stroop(1935)

VERTTask: Read word aloud!

BLEU ROUGE

4

Two accounts of Stroop asymmetry

Pathway strength (GRAIN)1

Functional architecture (WEAVER++)2

Prefrontal-parietal control network

Anterior Cingulate Cortex

Two accounts of ACC function

Conflict detection (GRAIN)1

Top-down regulation (WEAVER++)2

Stroop asymmetry and its time course.

2

Outline of the remainder

Control processes.3

Structure of lexical memory.

1

Stroop asymmetry and its time course.

2

Outline of the remainder

Control processes.3

Structure of lexical memory.

1

5

Slips of the tongue

Rudolf Meringer& Karl MayerVersprechen und Verlesen (1895)

Form

Meaning

“une contusion à la chambre”jambe

“Il faut faire grossir les gros obèses”maigrir

“Franche Dimance”c ch

Slips of the tongue

Syntax“sur la femme de leurs genoux”

les genoux de leur femme

Rossi & Defare (Int. J. Psycholing., 1995)

WEAVER++ Levelt, Roelofs, & Meyer (BBS, 1999)

Meaning

pommier

MEANING

masculine NUMBER singularGENDER

plural

Syntax

Form

Conceptualizing

Lemma retrieval

Form encoding

FORM FORMFORM

<pommier> <-z>

/p/ /o/ /m/ /j/ /e/

1 2 3 4

[pom] [je]

ONSET

<le>

/l/

1 2

/@/

NUCLEUS CODA

5

ONSETNUCLEUS

APPLE TREE(X)

STEM(X)

PLANT(X)

HAS-A

IS-A

Conceptualizing

Lemma retrieval

Form encoding

Articulation

POMMIER

“pommier”

ConceptualizingLemma retrieval

Form encoding

Articulation

POMMIER

“pommier”

Indefrey & Levelt (Cognition, 2004)

6

Stroop asymmetry and its time course.

2

Outline of the remainder

Control processes.3

Structure of lexical memory.

1

Stroop asymmetry and its time course.

2

Outline of the remainder

Control processes.3

Structure of lexical memory.

1

“Hi, I amColin MacLeod”

“Hi, I amColin MacLeod”

Stroop asymmetry

J. M. Cattell (1886)

“Reading is faster than color naming.”

ColorWord

7

-300 -200 -100 0 100 200 300

SOA (in milliseconds)

COLOR FIRST COLOR SECOND

VERT

-300 -200 -100 0 100 200 300

SOA (in milliseconds)

WORD FIRST WORD SECOND

VERT

RT E

FFEC

T (

mse

c)

-100

-50

0

50

100

150

-400

-200

-300

-100

400

1000

300

200

Color naming

SOA (msec)

-100

-50

0

50

100

150Word reading

-400

-200

-300

-100

400

1000

300

200

incongruentcongruent

WORD FIRST WORD SECOND COLOR FIRST COLOR SECOND

Glaser & Glaser (JEP:HPP, 1982)

Melara & Algom (Psych. Rev., 2003)

Melara & Mounts (Mem. & Cogn., 1993)

VERTVERT

(hear: bleu) incongruent

(hear: rouge) congruent

control task: keypress

Roelofs (Mem. & Cogn., 2005)

Repeating is as fast as color naming

400

425

450

475

500

525

550

-300 -200 -100 0 100 200 300

SOA(in milliseconds)

COLOR NAMING LATENCY (in milliseconds)

incongruentcongruent

8

-300 -200 -100 0 100 200 300

300

350

400

450

500

550

600incongruentcongruent

SOA(in milliseconds)

WORD REPETITION LATENCY (in milliseconds)

Two accounts of Stroop asymmetry

Pathway strength (GRAIN)1

Functional architecture (WEAVER++)2

Cohen et al. (1990, Psych. Rev.), Botvinick et al. (2001, Psych. Rev.)

WORD INPUTROUGE VERT

word reading

colornaming

VOCAL RESPONSE“rouge" “vert"

COLOR INPUT

TASK

GRAINincongruentcongruent

GRAIN

Cohen & Huston (1994, Att. & Perf.)

300

350

400

450

500

550

SOA (msec)

-300 -200 -100 0 100 200 300

COLOR FIRST COLOR SECOND

RT

(mse

c)

Word reading

-300 -200 -100 0 100 200 300

WORD FIRST WORD SECOND

500

550

600

650

700

750

SOA (msec)

Color naming

(b)

(a)

colorperception

word-formperception

word-formencoding

lemmaretrieval

conceptualidentification

articulation

COLOR

WORD

OVERT SPEECH

WEAVER++ model of spoken word production

Levelt et al. (BBS, 1999), Roelofs (Cognition, 1992, 1997; Psych. Rev., 2003, 2004)

9

IF task is color namingTHEN select concept for colorand enhance its activation

IF concept is selectedTHEN select lemma

COLOR NAMING(X)

TASKTASK

Roelofs (Psych. Rev., 2003)

COLOR(X)

RED(X)

GREEN(X)

BLACK(X)

WHITE(X)

GRAY(X)

BROWN(X)

YELLOW(X)

ORANGE(X)

PURPLE(X)

PINK(X)

BLUE(X)

OLIVE(X)

VIOLET(X)

couleur

rouge vert

noir blancgris

brun jaune

orangepourpre

rose

bleu

olive

violet

<rouge>

/r/ /u/ /G/

[ruG]

adj.

COLOR INPUT

WORD INPUT

ROUGE

CONCEPTUALSTRATUM

WORD-FORMSTRATUM

SYNTACTICSTRATUM

Colin MacLeod’s (1991) top-16

incongruency, congruency, reverse Stroop, response set, semantic gradient, time course, stimulus, spatial, multiple task, manual, bilingual, training, age,

and pathological effects.

3 free parameters taking 2 values each.r(model-data) = .98

WEAVER++ explains 96 percent of the variance.

250 data points from 16 classic studies

OBSERVED

RT E

FFEC

T (

mse

c)

SOA (msec)

-100

-50

0

50

100

150

-100

-50

0

50

100

150

-400

-200

-300

-100

400

1000

300

200

-400

-200

-300

-100

400

1000

300

200

-100

-50

0

50

100

150

-100

-50

0

50

100

150

-400

-200

-300

-100

400

1000

300

200

-400

-200

-300

-100

400

1000

300

200

Color naming

Word reading

WEAVER++

0

2.0

4.0

6.0

8.0

10.0

0 50 100 150 200 250

ACTI

VATI

ON

TIME (msec)

targetSOA = -100 msec

distractor

SOA = 0 msecdistractor

(b)

(a)

colorperception

word-formperception

word-formencoding

lemmaretrieval

conceptualidentification

articulation

COLOR

WORD

OVERT SPEECH

10

(b)

(a)

colorperception

word-formperception

word-formencoding

lemmaretrieval

conceptualidentification

articulation

COLOR

WORD

OVERT SPEECH

(b)

(a)

colorperception

word-formperception

word-formencoding

lemmaretrieval

conceptualidentification

articulation

COLOR

WORD

OVERT SPEECH

0 100 200

SOA (msec)

-80

-60

-40

-20

0

20

40

LATE

NCY

EFF

ECT

(mse

c)

color namingword reading (after task switch)

mediated (e.g., VESTE)direct (e.g., ROUTE)

VESTE

ROUTE

mediated

direct

Roelofs (Psych. Rev., 2003)

(b)

(a)

object-formperception

word-formperception

word-formencoding

lemmaretrieval

conceptualidentification

articulation

OBJECT

WORD

OVERT SPEECH

-400 -300 -200 -100 0 100 200 300 400

SOA (msec)

CHIEN

OBSERVEDWEAVER++

LATE

NCY

EFFE

CT

(mse

c)

SOA (msec)

-100

-50

0

50

100

150

-100

-50

0

50

100

150

-400

-200

-300

-100

400

1000

300

200

-400

-200

-300

-100

400

1000

300

200

-100

-50

0

50

100

150

-100

-50

0

50

100

150

-400

-200

-300

-100 400

1000

300

200

-400

-200

-300

-100

400

1000

300

200

Word reading

Picture naming

(Glaser & Düngelhoff, JEP:HPP, 1984)

11

LATE

NCY

EFF

ECT

(m

sec)

-100

-50

0

50

100

word categorizing

semantic unrelated

DISTRACTOR

-25

0

25

50

word reading with determiner

OBSERVEDWEAVER++

-25

0

25

word reading

TASK

Picture distractor

CHIEN

Say “chien"

CHIEN

Say “le chien"

Say “animal"

CHIEN

Roelofs (Psych. Rev., 2003)

Stroop asymmetry and its time course.

2

Outline of the remainder

Control processes.3

Structure of lexical memory.

1

Stroop asymmetry and its time course.

2

Outline of the remainder

Control processes.3

Structure of lexical memory.

1 Prefrontal-parietal

control network

Posner & Raichle (Images of Mind, 1994)

anterior cingulate cortex ACC

12

0.0

0.1

0.2

0.3

0.4

SCAN (2.5 sec)

1 2 3 4 5

incongruentcongruent

BOLD RESPONSE (% CHANGE)

MacDonald et al. (Science, 2000)

Two accounts of ACC function

Conflict detection (GRAIN)1

Top-down regulation (WEAVER++)2

Cohen et al. (Psych. Rev., 1990), Botvinick et al. (Psych. Rev., 2001)

WORD INPUTROUGE VERT

word reading

colornaming

VOCAL RESPONSE“rouge" “vert"

conflict detection (ACC)

COLOR INPUT

TASK (DLPF)

GRAIN

Botvinick et al. (Psych. Rev., 2001)

GRAIN

0.0

1.5

0 500

congruentincongruent

250

1.0

0.5

0.0

1.5

0 500

congruentincongruent

250

1.0

0.5

ENERGY

TIME (msec)

art

wfp

executingcontrol

wordplanning

frontallobe

COLORWORD

cpci/lr

wfe

inputcontrol

goalcontrol

centralsulcus

lateralfissure

anteriorcingulate

frontallobe

temporallobe

occipitallobe

VERBALRESPONSE

Roelofs (Psych. Rev., 2003)

WEAVER++

0.0

0.1

0.2

0.3

0.4fMRI observed

(ACC)WEAVER++

1 2 3 4 5

0.0

0.1

0.2

0.3

0.4

1 2 3 4 5

Roelofs & Hagoort(Cogn. Brain Res., 2002)

MacDonald et al. (Science, 2000)

BOLD RESPONSE (% CHANGE)

SCAN (2.5 sec)

incongruentcongruentincongruentcongruent

IF task is color namingTHEN select concept for colorand enhance its activation

13

VERTROUGEXXXX

incongruent

congruent

neutral

Stroop subtractions in neuroimaging

?

GAUCHE

RIGHT(X,(Y))

LEFT(X,(Y))

adj.WORD

CLASS

WORD

CLASSgauche droit

<droit>

/o/ /r//d/

<gauche>

/g/

[drwA][gosh]

1 1 2

on nu

FORMFORM

SENSESENSE

concepts

lemmas

forms

/sh/

2 3

co

IS-NOT

GAUCHE

vocalresponse

middlefinger

indexfinger

manual response

/w/ /A/

3

on1 on2 on3 nu

4

WEAVER++

manual

0

20

40

60

Arrow Word

RT

effe

ct (

mse

c)

Target

0

20

40

60

Arrow Word

RT

effe

ct (

mse

c)

Target

vocal

0

20

40

60

Arrow Word

RT

effe

ct (

mse

c)Target

0

20

40

60

Arrow Word

RT

effe

ct (

mse

c)

Target

observed WEAVER++

Turken & Swick (Nat. Neurosc., 1999)

Roelofs, Van Turennout, & Coles (submitted)

Manually

Respond to the arrow LEFT

LEFT

incongruent

LEFT

LEFT

RIGHT

RIGHT

congruent

RIGHT

RIGHT

neutral

XXXX

RIGHTRespond to the word

Respond to the arrow

incongruent

GAUCHE

incongruent

DROIT

DROIT

congruentcongruent neutral

XXXX

DROIT

XXXX

Respond to the wordGAUCHE

Incongruent vs. congruent in word task

14

Mean RT(ms)

1.2 2.8

arrow task word task475

525

575

625

0.9 4.6 1.2 6.3%E

-1.5-1

-0.50

0.51

1.5

Meanregressioncoefficient

arrow task word task

WEAVER++

475

525

575

625

arrow task word task

-1.5-1

-0.50

0.51

1.5

arrow task word task

congruent

incongruentneutral

Botvinick et al. (Psych. Rev., 2001)

GRAIN

incongruent

congruentneutral

selection

selection

ENER

GY

CYCLE

Summary

Stroop asymmetry and its time course:Strength vs. architecture

2

Control processes:ACC: conflict detection vs. regulation

3

Structure of lexical memory:Concepts, lemmas, forms

1

http://www.nici.ru.nl/~ardiroel/weaver++.htm

http://www.nici.ru.nl/~ardiroel/stroop.htm

Merci beaucoup!