The effects of captioning videosused for foreign language listening activities

Paula Winke, Susan Gass, & Tetyana Sydorenko@ Michigan State University

Language Learning & Technology 14, Feb 2010

Many of us intuit that captions are helpful:1. The language data is presented multi-modally, which

may promote reader engagement.• Beatty (2010, p. 55) and Felix (2008, p. 154) support this notion.

2. Integrates 3 of the 4 skills• Garrett (2009, p. 720) implicitly supports this notion.

And indeed the research on captioning shows this:

Baltova (1999); Danan (1992); Garza (1991); Markham (1993, 1999); Neuman & Koskinen (1992)

Winke, Gass, & Syderenko (2010): “The general consensus [is] that captioning leads to superior performance on subsequent comprehension and vocabulary tests,” (p. 66).

But that doesn’t mean that use of captions is totally understood.

• Learner Level? Guillory (1998) found captions helpful for early learners, whereas Taylor (2005) did not.

• Target Language? Very little research into the use of captions for learners encountering new scripts (specifically, learners from orthographically Latinate languages studying non-Latinate scripts)

• Always Helpful? Taylor (2005) interviewed students who suggested that captions are distracting, and Pujola (2002) suggested that captions can be overused (context unclear).

Research Questions:“1. Do captioned videos result in better comprehension of video content and learning of vocabulary than noncaptioned ones? (Spanish)

2. When a video is viewed twice, is captioning more effective (as measured by comprehension tests and vocabulary learning tests) when the first viewing is with captions or when the second viewing is with captions? (all languages)

3. Are there different benefits derived from captioning order depending on the target language? (all languages)

4. Do proficiency differences affect the benefits of captioning derived from captioning order? (Russian and Spanish)” (p. 68)

Method• 154 language learners in American Midwestern

university. All except one were native English speakers, though some had previously studied other languages.

• Second-year learners of Arabic (N = 29)• Second-year learners of Chinese (N=13)• Second-year learners of Spanish (N = 47)• Fourth-year learners of Spanish (N=20)• Second-year learners of Russian (N= 24)• Fourth-year learners of Russian (N = 11)

Method (cont.)• 3 short nature videos dubbed into TLs by native speakers.

There were captioned (TL) and non-captioned versions for each language and video. Topics = salmon, bears, dolphins

• Everyone watched 2 versions of each video

• One group of 2nd year Spanish learners saw everything captioned; another saw all non-captioned

• All other learner groups were split into A and B; group A saw captions on the first run of every video, but not the second, and group B did the reverse

Method (cont. 2)• Prior knowledge for all learners used test used as

control

• Vocab test half written, half auditory, A and B form distributed randomly but 50-50 in all language groups.

• Multiple-choice main-points comprehension test in English also distributed.

• Volunteers (N=26) then asked some general questions about their use of captions

FindingsR1) T-tests show that for Spanish group of all captioned vs. all non-captioned videos, scores for all captions beat scores for no captions on written test, aural test, and comprehension test.

R2) T-test shows that using the captions first created a statistically significant improvement for performance on the aural post-test, but not the written test or comprehension test.

R3) T-tests suggest that an ordering difference across languages is not statistically significant, though insignificant trends did emerge.

R4) Proficiency showed no effect on benefits of caption ordering.

Note: Findings presented below are statistically insignificant.

Interview Findings“1. Learners have a need for multiple input modalities.

2. Captions reinforce and confirm what is taken in aurally.

3. Captions affect learners’ attention to the input.

4. Captions aid with the decomposition and/or analysis of language.

5. Captions are sometimes viewed as crutches” (p. 77-78).

Findings Compared to Previous Research• Agree that captions help overall• Findings that captioning first viewing helps aurally, but

not on written tests, appears to be new• Findings that language has no significant effect on

captioning benefits appears to be new• Finding that proficiency level does not have a

statistically significant effect on ordering is said to agree with Guillory (1998) but not Taylor (2005). Our authors suggest: “In light of our study, we suggest that the lower-level learners in Taylor’s study who reported that it was more difficult to attend to captions than upper-level students were perhaps having a harder time with the content of the video,” (p. 80-81).

Connection to Weekly Readings• Chappelle and Jamieson (2008) do support the use of video

for listening activities generally, as “Visual information helps to call up background information and schemata,” (p. 139).

• However, Chappelle and Jamieson (2008) recommend basing listening activities on videos chosen at appropriate difficulty level, showing authentic usage, on topics that are relevant and interesting for students (p. 131-5). These nature videos do not seem to satisfy the first two criteria.

• Also, for teachers intending to make use of Winke, Gass, and Syderenko’s (2010) findings, it’s worth noting that Chappelle and Jamieson (2008) also recommend checking comprehension DURING the video and giving students the ability to pause and rewind as they please.

My Evaluation• All saw same videos, despite proficiency differences,

even though these authors implicitly criticize Taylor (2005) for doing the same.

• What about curricular differences between languages? The authors cite Markham (2001) as showing that background knowledge affects usefulness of captions.

• Does it make sense to do written tests for listening exercises?

• Prior knowledge test AFTER the videos, “so as not to cause the subjects to pay special attention to these words,” which could logically affect reliability of prior knowledge findings (priming effect).

My Evaluation (2)• Next, process of norming findings by dividing raw score by #

of unfamiliar words to control for prior knowledge had a reference (Smith, 2004) but no justification.

• Why wasn’t proficiency as a total effect on Y/N captions investigated? Findings on ordering don’t relate directly to prior research.

• Despite use of only one language without Latinate orthography (Russian), and statistically insignificant findings, the authors press forward that, “We suggest that learners of a language whose orthography is closer to that of the target language are better able to use the written modality as an initial source of information,” (p. 80).

Questions? Comments?