Computer-supported intervention for children with language and

literacy problems

Ludo Verhoeven

In collaboration with Eliane Segers and Rosemarie Irausquin

University of Nijmegen

Possibilities computer

• Connecting speech, orthography, semantics• Virtual reality • Hypertekst • Communication• Adaptivity• Responsivity• Motivational value

Digital learning environments

• Computer-assisted instruction– Drills & practice– Games

• Open learning environments– (Automatic) feedback– Communication

Computer-supported intervention

• Computer-assisted instruction– Early language intervention (kindergarten)– Intervention for poor readers

• Open learning environment – Language in the content areas

Early intervention softwareSchatkist met de muis

• Storybook reading– Story telling– Story illustrations– Semantic meaning– Word blending

• Language play– Multimedial songs– Book making– Postcard writing– Rhyme and word segmenting– Word making

Overview of a CDROM (bold arrows show adaptivity)

Postcard writing

Rhyme and word synthesis: b-el•Attention to word sounds•Isolating beginning sound•Word blending

Grapheme booklet: computer keeps track of individual progress

Word maker

Effects

• Positive effects– Vocabulary– Text comprehension– Phonological awareness

• Effects equally strong for L1 and L2 learners• Effects for children with SLI

– Rhyme– Phonemic awareness– Role of speech manipulation???

Research Tallal, Merzenich & associates

• Children with SLI:– Problems with fast formant transitions in synthetic speech

(1974, 1981)– Lengthening signal > better discrimination (1973)– Lengthening formant transition > better discrimination

(1975, 1980)– New algorithm: lengthening whole signal + amplifying

fast formant transitions up to 20dB– Commercial program: Fast Forword (1996: Effectful after

100 hours in four weeks of training)

Difficulties with fast formant transitions

year authors subject group speech material conclusion

1974 Tallal & Piercy 12 aphasic children12 controlsage 6.9-9.3 years old

/ba/ - /da/; formant transition 43 msec

aphasics have more problems than controls

1980a Tallal, Stark, Kallmann & Mellits

35 developmental dysphasics

38 controls average age: 6.8 years

old

/ba/ - /da/ as in Tallal and Piercy 1974.

Synthetic copies of natural utterances /ba/ - /be/ /bi/ - /dae/, /de/ - /di/; formant transition varying naturally

dysphasics have more problems than controls

1981 Tallal & Stark same as above /ba/ - /da/, /da/ - /ta/, /e/ - /ae/, /dab/ - /daeb/, /sa/ - /sta/, /sa/ - /fa/

dysphasics have more problems than controls in /ba/-/da/, /da/-/ta/ and also /sa/-/fa/.

1989 Reed 23 reading disabled23 'normals'average age: 8.9

/ba/ - /da/, /e/ - /ae/, reading disabled have problems in /ba/ - /da/ as opposed to controls.

1992 Leonard, McGregor & Allen

8 SLI, 8 normals. 4.5-5.6 years old

/ba/ - /da/ , /dab/ - /daeb/, /i/ - /u/, /dab-i-ba/ - /dab-u-ba/, /das/ - /daf/

SLI's have more problems than controls not in /dab/-/daeb/ and /i/-/u/

Benefits of speech manipulation1975 Tallal & Piercy 12 aphasic children

12 controls (6.8-9.3)/ba/ - /da/; formant transition extended from 40 to 80msec

positive effects

1980 Frumkin & Rapin 20 dysphasic children9 control childrenaverage age 9.6

/ba/ - /da/ and /a/ - /u/Synthetic speech, 2nd and 3rd formant 40msec or 80 msec; total length 250 msec

positive effects for subgroup

1980b Tallal, Stark, Kallman & Mellits

14 developmental dysphasics23 normally developingage 5-9

/ba/ - /da/ Synthetic copies of natural utterances /be/ - /be/, /bi/ - /dae/, /d/ - /di/; formant transition varying naturally

results aphasics dependent on syllable speed and ISI time

1982 Alexander & Frost 24 children with language/speech problems (7.2-11.6)

/ba/ - /da/. Formant transitions: 80, 70, 60 and 40 msec.

positive effects

1984 Blumstein, Tartter, Nigro & Statlender

16 aphasics, 6 controls 2 synthetic /ba/ - /da/ - /ga/ continua with formant transition 65 and 85 msec.

no effects

1985 Riedel & Studdert-Kennedy

12 adult aphasics.average age 55

/ba/ - /da/ Formant transition 30 and 82 msec.

no effects

1985 Tallal, Stark & Mellits

26 developmental dysphasics

same as Tallal and Stark 1981 unclear

1996 Stark & Heinz 11 children with output disorders21 children with SLI 22 controls (6-10 yrs)

/ba/ /da/ Klatt synthesis. Formant transition 30 to 80 msec. in steps of 10.

positive effect for children with expressive and receptive problems

1999 Bradlow et al. 32 children with learning problems72 controls (6 –16 yrs)

two /da/ - /ga/ continua40 msec and 80 msec formant transition

no effects

Training studies using speech manipulation

1996 Tallal et al. 22 SLIaverage age 7.4

Computer games with or without speech manipulation (Fast Forword)

positive effects

1999 Habib, et al. 12 dyslexicsage 10-12

Listening exercises with or without speech manipulation

positive effects

2001 Gillam et al. 4 SLIaverage age 7 ;3

Fast Forword vs Laureate Learning software

no differential effects

2003 Troia & Whitney

37 LD: 25 FFW/12 Caverage age 9;4

Fast Forword overall effect on expressive language only

Study 1: Speech manipulation

• Participants– 21 children with SLI vs 24 NLA controls (5 yrs)

• Stimulus set– Five contrasts: b-p, d-t, v-w, h-g, b-d– 60 word pairs, e.g., buik-duik (12x60 items)

• Speech manipulation:– Normal speech– Amplifying fast formant transitions– Slowing down speech signal– Amplification + slowing down

Oscillogram normal vs amplified speech (enhanced fast transitional elements): /buik/ - /duik/

Study 1 (continued)

• Results– Normal children perform better than SLI children– +/- voice is more complex than place contrasts– No effect of speech manipulation

• Conclusion– No replication of effects reported by Tallal &

Piercy

Study 2: Training with natural speech manipulation

• Participants (kindergarten)– 36 SLI children: 24 experimental vs 12 control

• Training: rhyme and word blending• Normal speech (N1=12)• Manipulated speech (N2=12)

• Procedure– Pretest-posttest-retention test

Results study 2

-0,3

-0,2

-0,1

0

0,1

0,2

0,3

control group normal speech manipulatedspeech

progress atposttest

progress atretention

Conclusions Study 2

• Program is effective for children with SLI

• Significant retention effects

• No effect for speech manipulation

Study 3: Training with synthetic speech manipulation

• Participants (kindergarten)– 19 SLI children and 24 NLA children

• Stimulus set: minimal word pairs– Normal– Slowing down entire speech signal (like in FFW)– Slowing down just formant transitions

Results study 3

40

50

60

70

80

90

100

NLA SLI

normal speech

slowing downentire speechsignalslowing downformant transitions

Conclusions Study 3

• Program is effective for NLA and SLI

• Positive effect speech manipulation for SLI

• No difference between slowing down entire signal or formant transitions

General conclusions

• SLI children have difficulties in phonological tasks

• Phonological training is helpful for SLI children

• Role of speech manipulation is inconclusive

• Limitations: small N, limited hours of training

Dyslexia

• Phonological deficit– Phonological memory– Phonemic segmentation– Access to phonology

• Sensory defects– Processing brief sensory cues– Processing rapidly changing sequences– Limited use of temporal information

LEESLADDER: Adaptive computer program for poor readers

• cd-rom 1: alphabetic principle, phonological awareness, decoding of CVC words

• cd-rom 2: automatisation of reading and spelling of simple word structures + basic reading comprehension

• cd-rom 3: reading/spelling monosyllabic words with consonant clusters and specific orthographic patterns (-eeuw, -ooi, -ng, etc.) + reading comprehension

• cd-rom 4: decoding multisyllabic words + advanced reading comprehension

Pupildatabase

Pupil IDLearning history

Teacher part

Parameters

Planningabilities

Play types

(Sub)abilities

Sourcesdatabase

graphemeswordsSentences, etc..

Pupil part

Adaptive navigationmodule

Basic 1

Basic 2

Reading and spelling 1

Reading and spelling 2

Reading and spelling 3

Grapheme discovery

Sound discovery

Grapheme- and phoneme knowledge

Type & copy

Word closure

Auditory synthesis

Letter flashing

Grapheme order correct

Simple spelling Word reading correct

Grapheme order fast

Auditory word dictation

Picture-word

Word-picture

Word rows 1

Word reading fast

Visual word dictation

Word rows 2 Flash words

Basic 1

Basic 2

Reading and spelling 1

Reading and spelling 2

Reading and spelling 3

Grapheme discovery

Phoneme discovery

Grapheme and phoneme knowledge

Type and copy

Word closure

Auditory synthesis

Flash letters

Grapheme order correct

Elementary spelling

Word reading correct

Grapheme order fast

Auditory word dictation

Picture-word

Word-picture

Word rows 1

Worden reading fast

Visual word dictation Word rows 2 Flash words

Letter test

Effects on phonological awareness

0

1

2

3

4

5

6

7

8

9

10

Pretest Posttest

Me

an

Od

d-o

ne

-ou

t a

llit

era

tio

n s

co

re

TrainingControl

Effects on word decoding

0

2

4

6

8

10

12

14

16

Pretest Posttest

Me

an

Sc

ore

on

Ps

eu

do

wo

rd T

es

t

TrainingControl

Correlations between training intensity and reading gains

Exercises per week Days per week

CVC word reading test .53* .56*

Word reading test .54* .58*

Pseudoword reading test .62* .59*

Speed vs comprehension training

0

5

10

15

20

25

30

35

40

45

50

Speed1 Speed2 Compreh1 Compreh2

CVC

CCVCC

Bisyllabic

General discussion

• Age: plasticity of the brain

• Contents of training

• Synthetic vs natural speech

• N of hours of training (minimum 100?)

• Effects: phonology vs information processing

• Necessity of control groups

Open Learning Environment: facilitating language and knowledge construction

Communication

Text sample

Communication: reaction types

0

5

10

15

20

25

30

1 2 3 4 5 6 7

reaction type

Reaction types: (1) (dis)approval, (2) question about the text, (3) personal, (4) reasoning , (5) suggestion (6) hello/goodbye, (7) rest category

Online spelling feedback

0

2

4

6

8

10

12

14

H T C S

number oferrors

Conditions: Handwriting (H), Typing (T), Spellingchecker (C), Spelling suggestions (S)

Perspective

• Children need to learn how to communicate in open learning environment

• Spelling development can be supported by means of online feedback

• Children with learning problems equally benefit from the environment

• CIA can be integrated in open learning environments