Optimizing computer-based second language vocabulary learning

Optimizing computer-based second language vocabulary learning

Tatsuya NakataKansai University, Japan

[email protected]

This research was supported in part by Grant-in-Aid for Young Scientists (A) #16H05943 and Grant-in-Aid for Research

Activity Start-up (#15H06746) from MEXT.

mailto:[email protected]

Abstract• This presentation will examine how we can optimize

computer-based vocabulary learning in a second or foreign language (L2). Specifically, we will discuss empirical studies examining the effects of the following factors on vocabulary learning: studying vs. testing, part vs. whole learning, massed vs. spaced learning, equal vs. expanding spacing, immediate vs. delayed feedback, and single vs. repeated retrievals. These studies can recommend to us the optimal way to learn vocabulary from computer software.

• Flashcard (word card) learning: L2 on one side and meaning (L1 translation or definition) on the other.

Initial presentation

Retrieval

Feedback (Correct answer)

Outline

1. Effects of studying and testing (Karpicke & Roediger, 2008)

2. Effects of equal and expanding spacing (Nakata, 2015)

3. Effects of part and whole learning (Nakata & Webb, 2016)

4. Effects of single and repeated retrievals (Nakata, in press)

5. Q&A

Effects of studying and testing

Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of ＸＸＸ for learning. Science, 319, 966-968.

Method

• Participants: 40 American undergraduate students • Materials: 40 Swahili-English word pairs (e.g.,

chakura- food)• Independent variable: Number of study and test

episodes　 Study ： chakura = food　 Test ： chakura = ??? * No correct answer (e.g., food) was provided as

feedback after each test.

4 Conditions

* No correct answer (e.g., chakura = food) was provided as feedback after each test.

# of Study episodes

# of Test episodes

1. Control 1.94 1.942. Repeated study 4 2.083. Repeated test 1.92 44. Repeated study + Repeated test 4 4

Method

• Dependent variable: One-week delayed meaning recall test (e.g., chakura = ???)

• Please rank the following four conditions in order of effectiveness (meaning recall 1 week after learning):

# of Study episodes

# of Test episodes

1. Control 1.94 1.942. Repeated study 4 2.083. Repeated test 1.92 44. Repeated study + Repeated test 4 4

* No correct answer (e.g., chakura = food) was provided as feedback after each test.

Repeated study + Repeated test

ControlRepeated test Repeated study

Adapted from Roediger 2009

Discussion• Number of testing, not studying, affects long-term

retention.• Increasing the number of studying from two to four

does not significantly increase retention 1 week after learning （ 33 36%⇒ ）

• Increasing the number of testing from two to four, in contrast, significantly increases retention 1 week after learning （ 33 81%⇒ ）

• When the number of testing is four, increasing the number of studying from two to four does not affect retention （ 81% 81% ⇒ ）

Discussion• Vocabulary learning is a function of the

number of testing, not studying.• People learn more from testing than studying

(testing effect).• Gates, A.I. (1917). Recitation as a factor in

memorizing. Archives of Psychology, 6(40).• Karpicke, J. D., & Roediger, H. L. (2008). The

critical importance of retrieval for learning. Science, 319, 966-968.

• NY Times Jan. 20, 2011

Pedagogical implications

• Vocabulary quizzes are good!• It doesn’t necessarily mean that we have to

give “tests” in its traditional sense.• Self-testing also counts as a test.

(e.g.,) activities involving self-testing such as flashcards, fill-in-the-blanks, matching exercises, or translation exercises are helpful.

• Computer-based flashcards are also useful.

Outline





5. Q&A

• Nakata, T. (2015). Effects of expanding and equal spacing on second language vocabulary learning: Does gradually increasing spacing increase vocabulary learning? Studies in Second Language Acquisition. 37, 677-711. doi:10.1017/S0272263114000825

Background

• Karpicke and Roediger (2008) showed that testing facilitates vocabulary learning.

• How should the tests be distributed to maximize learning?

• Two practice schedules: equal and expanding spacing

Equal spacing vs. Expanding spacing• Equal spacing: The intervals between

encounters of a given item are held constant.

Note. A = a given target word.

Equal spacing vs. Expanding spacing

• Expanding spacing: The intervals between encounters of a given item are gradually increased as learning proceeds.

Note. A = a given target word.

Massed schedule (no spacing)• Target items are encountered several

times in a row.

E.g.,)A A A A B B B B C C C C D D D D …

• Used as the control treatment.

• Which is more effective for vocabulary learning, equal or expanding spacing?

• Expanding spacing is often regarded as the most effective practice schedule (Baddeley, 1997; Ellis, 1995; Hulstijn, 2001; Landauer & Bjork, 1978; Nation, 2001; Pimsleur, 1967; Schmitt, 2000).

• A common feature among existing flashcard software (e.g., iKnow, Word Engine, SuperMemo; Nakata, 2011)

Expanding spacing is …

http://www.wordengine.jp/vflash

Theoretical background

1. Retrieval practice effect (Karpicke & Roediger, 2010; Logan & Balota, 2008)

2. Retrieval effort hypothesis (Karpicke & Roediger, 2010; Logan & Balota, 2008; Pyc & Rawson, 2009)

Empirical evidence• Expanding > Equal (7 studies)

Cull et al., 1996; Dobson, 2011; Karpicke & Roediger, 2007; Landauer & Bjork, 1978; Maddox et al., 2011, Exp 2; Logan & Balota, 2008; Storm, Bjork, & Storm, 2010

• Expanding = Equal (9 studies) Balota et al., 2006; Carpenter & DeLosh, 2005; Cull, 2000; Kang et al., 2014; Karpicke & Bauernschmidt, 2011; Karpicke & Roediger, 2010; Maddox et al., 2011, Exp 2; Pyc & Rawson, 2007; Shaughnessy & Zechmeister, 1992

• Expanding < Equal (3 studies)Cull, 2000; Karpicke & Roediger, 2007; Logan & Balota, 2008

Empirical evidence

• Meta-analysis by Cepeda et al.(2006): Mixed results

• Why inconsistent results?The effects of expanding spacing may be affected by at least four factors … (Balota et al., 2007; Dobson, 2012; Karpicke & Roediger, 2007; Logan & Balota, 2008; Maddox et al., 2011; Roediger & Karpicke, 2010; Storm, Bjork, & Storm, 2010)

Empirical evidence• Expanding spacing tends to be effective

when … (1) the task difficulty is high, (2) feedback is not provided after retrieval, (3) the treatment involves a large amount of

spacing, and / or (4) the retention interval is shorter than 24 hours (i.e., posttest is given within 24 hours after the treatment).

Limitations• None of the existing L2 vocabulary

studies manipulated these four factors.(Kang et al., 2014; Karpicke & Bauernschmidt, 2011; Pyc & Rawson, 2007)

• This study will consider the effects of (a) the amount of spacing and (b) retention interval (i.e., timing of posttest).

Research questions1. Do (a) the amount of spacing and (b)

retention interval influence the effects of equal and expanding spacing on L2 vocabulary learning?

2. Which has a larger effect on L2 vocabulary learning: the type of spacing (equal or expanding) or the amount of spacing?

METHOD

• Participants: 128 Japanese college students

• Target items: 20 low frequency English wordse.g.,) apparition, billow, cadge, citadel, and dally

Independent variables• 1) Amount of spacing (between-participant)

Massed (No spacing) AAAA

Short (1 minute) A_ _A_ _A_ _A

Medium (2 minutes) A_ _ _ _A_ _ _ _A_ _ _ _A

Long (6 minutes) A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A

Independent variables

• 2) Type of spacing (within-participant)

• 3) Retention interval (within-participant)Immediate posttest 1-week delayed posttest

Equal A_ _ _ _ _ _ _ _ _ _ _ _A _ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A

Expanding A_ _ _ _ _ _ A_ _ _ _ _ _ _ _ _ _ _ _ A_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _A

Amount of spacing

Type of spacing

Short Equal A_ _A_ _A_ _A

Expanding A_A_ _A_ _ _A

MediumEqual A_ _ _ _A_ _ _ _A_ _ _ _A

Expanding A_ _ A_ _ _ _A_ _ _ _ _ _A

Long Equal A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A

Expanding A_ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _A

Massed (No spacing)

AAAA

7 Conditions

Procedure

Week 1

ExplanationPretest (1. Productive 2. Receptive)Treatment Filler taskImmediate posttest (1. Productive 2. Receptive)Questionnaire

Week 2 Delayed posttest (1. Productive 2. Receptive)

1st Encounter: Initial presentation (Studying)

2nd, 3rd, and 4th Encounters: Productive retrieval (Test)

Feedback (Correct response)

• (1) Productive test: Translate from Japanese (L1) to English (L2).

Pre / Posttests

• (2) Receptive test: Translate from English (L2) to Japanese (L1).

Pre / Posttests

RESULTS (1): Type of spacingIs there any difference between equal and expanding spacing?

Dependent variables:• 1) Learning phase performance• 2) Immediate posttest scores• 3) 1-week delayed posttest scores

Equal A_ _ _ _ _ _ _ _ _ _ _ _A _ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A

Expanding A_ _ _ _ _ _ A_ _ _ _ _ _ _ _ _ _ _ _ A_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _A

0.0

0.2

0.4

0.6

0.8

1.0

Learning phase performance: Short spacing group

Pro

porti

on o

f cor

rect

resp

onse

s

Retrieval 1 Retrieval 2 Retrieval 3

Equal

Note. Brackets enclose +1 SE.

0.0

0.2

0.4

0.6

0.8

1.0

Learning phase performance: Short spacing group

Pro

porti

on o

f cor

rect

resp

onse

s


Equal

Expanding

n.s. d < 0.07

(small effect)

Expanding > Equal p = .002, d = 0.41 (small effect)


0.0

0.2

0.4

0.6

0.8

1.0

Learning phase performance: Medium spacing group

Pro

porti

on o

f cor

rect

resp

onse

s


Equal

Expanding

n.s. d < 0.23

(small effect)


0.0

0.2

0.4

0.6

0.8

1.0

Learning phase performance: Long spacing group

Pro

porti

on o

f cor

rect

resp

onse

s


Equal

Expanding

n.s. d < 0.15

(small effect)


n.s. d < 0.12

(small effect)


n.s. d < 0.21

(small effect)


Summary (1): Type of spacing• 1) Learning phase: equal = expanding • 2) Immediate posttest: equal = expanding• 3) Delayed posttest: equal = expanding

• Previous studies: Expanding spacing may be effective depending on (a) the amount of spacing and (b) retention interval.

• Not supported in this study.

Massed (No spacing) AAAA

Short (1 minute) A_ _A_ _A_ _A

Medium (2 minutes) A_ _ _ _A_ _ _ _A_ _ _ _A

Long (6 minutes) A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A_ _ _ _ _ _ _ _ _ _ _ _A

RESULTS (2): Amount of spacing• Is there any difference among the massed,

short, medium, and long spacing groups?

Dependent variables• 1) Learning phase performance• 2) Immediate posttest scores• 3) 1-week delayed posttest scores

0.0

0.2

0.4

0.6

0.8

1.0

Learning phase performance

Pro

porti

on o

f cor

rect

resp

onse

s


Massed



0.0

0.2

0.4

0.6

0.8

1.0


Pro

porti

on o

f cor

rect

resp

onse

s


Massed

Short


0.0

0.2

0.4

0.6

0.8

1.0


Pro

porti

on o

f cor

rect

resp

onse

s


Massed

Short

Medium

0.0

0.2

0.4

0.6

0.8

1.0


Pro

porti

on o

f cor

rect

resp

onse

s


Massed

Short

Medium

Long

Massed > Short = Medium > Longp < .004, 0.78 < d < 4.75 (medium to large effect)


Mas

sed

Sho

rt

Med

ium

Long

Mas

sed

Sho

rt

Med

ium

Long

Productive posttest

Pro

porti

on o

f cor

rect

resp

onse

s

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0Immediate Delayed

Short = Medium = Long > Massedp < .004, 0.88 < d < 1.40 (large effect)

Mas

sed

Sho

rt

Med

ium

Long

Mas

sed

Sho

rt

Med

ium

Long

Productive posttest

Pro

porti

on o

f cor

rect

resp

onse

s

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9


Short = Massed n.s. d = 0.39 (small effect)

Medium = Long > Massedp < .014, 0.80 < d < 0.85 (large effect)

Short = Medium = Long > Massedp < .004, 0.88 < d < 1.40 (large effect)

Mas

sed

Sho

rt

Med

ium

Long

Mas

sed

Sho

rt

Med

ium

Long

Productive posttest

Pro

porti

on o

f cor

rect

resp

onse

s

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9


Spacing effect

0.0

0.2

0.4

0.6

0.8

1.0


Pro

porti

on o

f cor

rect

resp

onse

s


Massed

Short

Medium

Long

Mas

sed

Sho

rt

Med

ium

Long

Mas

sed

Sho

rt

Med

ium

Long

Productive posttest

Pro

porti

on o

f cor

rect

resp

onse

s

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9


Short > Massedp < .001, d = 1.07 (large effect)

Short = Massed n.s. d = 0.39 (small effect)

Spacing-retention interval

interaction

Mas

sed

Sho

rt

Med

ium

Long

Mas

sed

Sho

rt

Med

ium

Long

Receptive posttest

Pro

porti

on o

f cor

rect

resp

onse

s

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9


SUMMARY

• 1) Very little difference between equal and expanding spacing.

• 2) Expanding spacing was no more effective than equal spacing regardless of (a) the amount of spacing or (b) retention interval.> Expanding spacing may not be the most effective practice schedule.

SUMMARY

• 3) The massed schedule (no spacing) was the least effective in both short and long term.> The finding underscores the importance of spacing in L2 vocabulary learning (spacing effect).

Summary (2): Amount of spacing• 4) Medium and long spacing were the most

effective in both short and long term.• 5) Short spacing was effective in the short term,

but not in the long term.> Introducing a large amount of spacing may be effective, especially for improving long-term retention (spacing-retention interval interaction).>> The amount of spacing may have a larger effect on learning than the type of spacing.

PEDAGOGICAL IMPLICATIONS• The desirable difficulty framework (e.g., Bjork,

1994, 1999)Slow and effortful learning = good retentionFast and easy learning = poor retention(e.g.,) Massed vs. medium / long spacing

• Do not discourage learners from making errors• Make a distinction between learning phase

performance and long-term retention • We should not be concerned too much about

learning phase performance

DISCUSSION• Why no difference between equal and

expanding spacing?1. Primary / long-term memory (Karpicke &

Roediger, 2007; Roediger & Karpicke, 2010)

2. The retrieval effort hypothesis (Karpicke & Bauernschmidt, 2011)

LIMITATIONS1. (a) Task difficulty and (b) feedback were

not manipulated. 2. Short duration of the treatment (approx

25 minutes). Kang, Lindsey, Mozer, & Pashler (2014): Treatment conducted over 4 weeks. No difference between equal and expanding spacing.

Outline





5. Q&A

• Nakata, T., & Webb, S. (2016). Does studying vocabulary in smaller sets increase learning? The effects of part and whole learning on second language vocabulary acquisition. Studies in Second Language Acquisition, 38, 523-552. doi: 10.1017/S0272263115000236

Part vs. whole learning

• Whole learning: The materials to be learnt are repeated in one large block.

• Part learning: The materials are divided into smaller blocks and repeated.


• When memorising a poem, would it be more effective to read the whole poem several times (whole learning) or would it be more effective to read part by part for the same number of times (part learning)?

• When learning to play music, would it be effective to practise part by part (part learning) before trying to play the whole tune (whole learning)?


• When learning 20 words… • Whole learning: 20 words are repeated in one

large block• Part learning: 20 words are divided into

smaller blocks and repeated

Whole learning: 20 items are repeated in one large block

Part learning: 20 items are repeated in four blocks of 5 items

Question

• Which do you think is more effective for L2 vocabulary learning, whole learning or part learning?

Background

• Researchers, learners, teachers, and materials developers: part learning > whole learning (e.g., Kornell, 2009; Salisbury & Klein, 1988; Wissman, Rawson, & Pyc, 2012; Woodworth & Schlosberg, 1954).

• The retrieval practice effect (Baddeley, 1997; Ellis, 1995) and list-length effect (Gillund & Shiffrin, 1984; Van Bussel, 1994): part learning > whole learning

Empirical evidence

• Empirical studies: whole learning > part learning (Brown, 1924; Crothers & Suppes, 1967, Experiments 8 & 9; Kornell, 2009, Experiments 1-3; McGeoch, 1931, Experiments 1 & 3; Seibert, 1932).

Limitation of previous research• In existing studies, the part-whole distinction

and spacing have been confounded. (i.e., whole learning always had longer spacing than part learning).

• Spacing effect: Large spacing > short spacing (e.g., Bahrick & Phelps, 1987; Cepeda et al., 2008, 2009; Pashler et al., 2003)

• The results of the earlier studies: due to spacing or the part-whole distinction?

Whole learning

Whole learningSpacing of 19 other items

Part learning

Part learningSpacing of 4 other items

Whole learning: Spacing of 19 items⇒ 　 Large spacingPart learning: Spacing of 4 items⇒ 　 Short spacing

Limitation of previous research

• Spacing effect: Large spacing > short spacing.• The results of the earlier studies: due to

spacing or the part-whole distinction?

Present study

• Purpose: investigate the effects of part and whole learning on L2 vocabulary learning while isolating the effects of the part-whole distinction and spacing.

Experiment 1

• Purpose: To investigate the effects of part and whole learning that were matched in spacing.

• RQ: Is whole learning more effective than part learning for L2 vocabulary learning when spacing is equivalent?

METHOD


• Target items: 20 low frequency English wordse.g.,) apparition, billow, cadge, citadel, dally

Independent variable

• Unlike previous studies, part and whole learning were matched in spacing (approximately 4 min).

Part / whole distinction Block size

(1) 4-item part Part 4(2) 10-item part Part 10(3) Whole Whole 20

Procedure

Week 1

ExplanationPretest (1. Productive 2. Receptive)Treatment Filler taskImmediate posttest (1. Productive 2. Receptive)Questionnaire

Week 2 Delayed posttest (1. Productive 2. Receptive)

1st Encounter: Initial presentation

2nd - 5th Encounters: Productive retrieval

Feedback (Correct response)


Pre / Posttests

RESULTS

F (2, 88) = 0.14, p = .867, ηp

2 < .01

F (2, 88) = 0.41, p = .664, ηp

2 = .01

Discussion• Previous studies: Whole > part learning• This study: Whole part learning≒• As long as spacing is equivalent, the part-

whole distinction has little effect on learning.• The results of the earlier studies may have

been due to spacing rather than the part-whole distinction?

Limitation of Experiment 1

• We cannot necessarily attribute the lack of statistical significance to the fact that part and whole learning were controlled for spacing (e.g., small sample size, Type II error).

Experiment 2• Purpose: Test the three hypotheses below:• H1: When spacing is equivalent, whole learning does

not outperform part learning• H2: When whole learning has longer spacing than part

learning, whole learning outperforms part learning.• H3: Part learning with longer spacing outperforms part

learning with shorter spacing. • If all three hypotheses are supported, the lack of

statistical significance in Experiment 1 was perhaps because part and whole learning had equivalent spacing.

METHOD


• Target items: 20 low frequency English wordse.g.,) apparition, billow, cadge, citadel, dally


* (2) “4-item part” and (3) “Whole” learning will be collectively referred to as the experimental treatments.

Part / whole distinction Block size

Spacing(Approx.)

(1) Control Part 4 30 sec(2) 4-item part Part 4 3 min(3) Whole Whole 20 3 min

n.s. d < 0.49

RESULTSPart = Whole > Control

0.87 < d < 1.02

n.s. d < 0.49

Productive posttest

• Immediate: Experimental = Control • Delayed: Experimental > Control • Spacing-by-retention interval interaction:

- When the posttest is given after a short delay, short spacing tends to be effective.- When the posttest is given after a long delay, long spacing tends to be effective. (e.g., Cepeda et al., 2006, 2008, 2009; Pashler et al., 2007; Rohrer & Pashler, 2007)

n.s. d < 0.60

Receptive posttest

• Immediate: Experimental = Control • Delayed: Experimental = Control • At each test administration, the productive

test was given prior to the receptive test.• This perhaps affected performance on the

receptive test and reduced a potential difference among the three groups on the receptive posttest?

DISCUSSION• All three hypotheses were supported:• H1: When spacing is equivalent, whole learning

does not outperform part learning (whole 4-≒item part).

• H2: When whole learning has longer spacing than part learning, whole learning outperforms part learning (whole > control).

• H3: Part learning with longer spacing outperforms part learning with shorter spacing (4-item part > control).

Experiments 1 & 2

• As long as spacing is equivalent, the part-whole distinction has little effect on learning

• Spacing has a larger effect on learning than the part-whole distinction


1. As long as the treatment involves long spacing, learners may study with either part or whole learning without significantly affecting learning outcomes.

2. It is useful to pay more attention to spacing rather than the part-whole distinction. The amount of spacing has a larger effect on learning than the part-whole distinction.

Limitations

• 1. Short duration of the treatment (< 20 minutes).

• 2. Learning was restricted to flashcard learning.

Outline





5. Q&A

• Nakata, T. (in press). Does repeated practice make perfect? The effects of within-session repeated retrieval on second language vocabulary learning. Studies in Second Language Acquisition.

Background• Repeated input increases L2 vocabulary learning

(e.g., Bisson et al., 2014; Chen & Truscott, 2010; Horst, Cobb, & Meara, 1998; Pigada & Schmitt, 2006; Rott, 1999, 2007; Webb, 2007; Waring & Takaki, 2003; Zahar et al., 2001).

• Retrieval > input (e.g., Barcroft, 2007, 2013; Karpicke & Roediger, 2008; Royer, 1973).

• Would repeated retrieval increase learning?(1) repeated retrieval within one learning session (“within-session repeated retrieval”)(2) repeated retrieval over multiple learning sessions

Background• Repeated input increases L2 vocabulary learning (e.g.,

Bisson et al., 2014; Chen & Truscott, 2010; Horst, Cobb, & Meara, 1998; Pigada & Schmitt, 2006; Rott, 1999, 2007; Webb, 2007; Waring & Takaki, 2003; Zahar et al., 2001).

• Retrieval (= test) > input (= study) (e.g., Barcroft, 2007, 2013; Karpicke & Roediger, 2008; Royer, 1973).

• Would repeated retrieval increase learning?(1) repeated retrieval within one learning session (“within-session repeated retrieval”)(2) repeated retrieval over multiple learning sessions

Arguments in favor of within-session repeated retrieval

• Usage-based models of language acquisition (e.g., Goldberg, 2006; Tomasello, 2003): frequency is one of the major factors affecting acquisition

• Within-session repeated retrieval increases learning.

Arguments against within-session repeated retrieval

• Spacing effect: long spacing between retrievals> short spacing between retrievals (= within-session repeated retrieval)

• The benefits of within-session repeated retrieval are short-lived? (Rohrer et al., 2005; van Zeeland & Schmitt, 2013)


• van Zeeland & Schmitt (2013)Immediately: repeated > less input2-week delay: repeated = less input* repeated input, not repeated retrieval

• Rohrer et al. (2005)1-week delay: repeated > fewer retrievals

3-9 week delay: repeated = fewer retrievals * non-L2 vocab


• Retrieval effort hypothesis: concerned with the issue of efficiency (i.e., whether repeated retrieval allows learners to learn a large number of words when time-on-task is controlled).

• Difficult and effortful retrievals > easy and effortless retrievals

• In within-session repeated retrieval, retrievals will be progressively easier (e.g.,) 1st < 2nd < 3rd

Retrieval effort hypothesis

• The retrieval effort hypothesis: easy retrievals < difficult retrievals (e.g.,) 1st < 2nd < 3rd

• Most learning occurs in the first few retrieval attempts, and practicing retrieval beyond a certain frequency level may not provide a benefit that justifies its cost (i.e., increased study time).

• Within-session repeated retrieval is not necessarily efficient?

Previous research

• Barcroft (2013); Folse (2006)immediately: repeated > fewer retrievals

• Peters (2012, 2014)immediately & 1-2 weeks: repeated > fewer retrievals

Present study

1. Examine both the short and long term effects of within-session repeated retrieval.Posttests were given not only immediately and 1 week, but also 4 weeks after the treatment (Rohrer et al., 2005).

2. Examine not only effectiveness (posttest scores) but also efficiency (posttest scores / study time).

Research questions

• RQ1: In L2 vocabulary learning, does increasing within-session retrieval frequency increase effectiveness (posttest scores)?

• RQ2: In L2 vocabulary learning, does increasing within-session retrieval frequency increase efficiency (posttest scores / study time)?

Hypotheses

• Hypothesis 1: Within-session repeated retrieval is more effective than fewer retrievals immediately and 1 week after learning, but not 4 weeks (spacing effect).

• Hypothesis 2: Within-session repeated retrieval is more efficient than fewer retrievals regardless of the posttest timing (retrieval effort hypothesis).

METHOD


• Target items: 16 low frequency English wordse.g.,) apparition, billow, citadel, dally, gouge


• (1) Retrieval frequency (between-participant): 1, 3, 5, or 7 retrievals

• (2) Posttest timing (within-participant): immediately, 1 week, and 4 weeks after learning

Procedure

Week 1Pretest (a. Productive b. Receptive)Treatment Filler taskImmediate posttest (a. Productive b. Receptive)

Week 2 Delayed posttest 1 (a. Productive b. Receptive)

Week 5 Delayed posttest 2 (a. Productive b. Receptive)

1st Encounter: Initial presentation

Retrieval opportunities (1, 3, 5 or 7 times)

Feedback (Correct answer)


Pre / Posttests

1 3 5 7 1 3 5 7 1 3 5 7

Effectiveness: Productive posttest

Pro

porti

on o

f cor

rect

resp

onse

s

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0 Immediate 1-week Delayed 4-week Delayed

Retrieval 5 = 7 > 1 = 3(0.88 < d < 2.04)

1 3 5 7 1 3 5 7 1 3 5 7

Effectiveness: Receptive posttest

Pro

porti

on o

f cor

rect

resp

onse

s

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9


Retrieval 5 = 7 > 1 = 3(0.98 < d < 1.37)

Summary

• Effectiveness: Retrieval 1 = 3 < 5 = 7(regardless of posttest timing)

• Efficiency: Efficiency score = posttest score / study time (min.)

1 3 5 7 1 3 5 7 1 3 5 7

Efficiency: Productive posttest

Pro

porti

on o

f cor

rect

resp

onse

s

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

Immediate 1-week Delayed 4-week Delayed

Retrieval 1 > 3 = 5 = 7(0.36 < d < 1.88)

1 3 5 7 1 3 5 7 1 3 5 7

Efficiency: Receptive posttest

Pro

porti

on o

f cor

rect

resp

onse

s

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6


Retrieval 1 > 3 = 5 = 7(1.40 < d < 3.10)

Summary

• Effectiveness Retrieval 1 = 3 < 5 = 7(regardless of posttest timing)

• EfficiencyRetrieval 1 > 3 = 5 = 7(regardless of posttest timing)

Discussion: Effectiveness

• Hypothesis 1: benefits of within-session repeated retrievals are short-lived

• This study: repeated retrievals > fewer retrievals regardless of posttest timing

• The negative effects of massing practice were outweighed by the positive effects of increased frequency?

• Underscore the importance of repetition in L2 vocabulary learning.

Discussion: Efficiency

• Single retrieval was the most efficient regardless of the posttest timing, supporting Hypothesis 2.

• The findings are consistent with the retrieval effort hypothesis, according to which the positive effects of repeated retrieval decrease as a function of the retrieval frequency.

• Most learning occurred in the first retrieval attempt, and practicing retrieval more than once did not provide a benefit that justified its cost (i.e., increased study time).


• If efficiency is not an issue, increasing retrieval frequency within one learning session is preferable.

• If learners do not have enough time, it may be desirable to practice retrieval only once.

Limitations

1. The posttest timing was manipulated within participants.

2. The treatment in this study was restricted to flashcard learning.

SUMMARY

• Karpicke and Roediger (2008)• People learn more from testing than studying

(testing effect).• How should the tests be distributed to

maximize learning?• Two practice schedules: equal and expanding

spacing

Equal and expanding spacing

• 1) Very little difference between equal and expanding spacing.

• 2) The massed schedule was the least effective in both short and long term (spacing effect).

• 3) Medium and long spacing were the most effective in both short and long term.

• 4) Short spacing was effective in the short term, but not in the long term (spacing-retention interval interaction).

Whole and Part Learning

• As long as spacing is equivalent, the part-whole distinction has little effect on learning.

• Spacing has a larger effect on learning than the part-whole distinction.

• It is useful to pay more attention to spacing rather than the part-whole distinction.

Repeated retrieval• If efficiency is not an issue, increasing retrieval

frequency within one learning session is preferable.

• If learners do not have enough time, it may be desirable to practice retrieval only once.

• I am very grateful to Stuart Webb, Paul Nation, Jan Hulstijn, Rod Ellis, Stuart McLean, Atsushi Mizumoto, and Tomohiro Tsuchiya.

• Tatsuya Nakata (Kansai University)[email protected]

Optimizing computer-based second language vocabulary learning

Education

Transcript of Optimizing computer-based second language vocabulary learning