Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne...
-
Upload
wade-storm -
Category
Documents
-
view
221 -
download
1
Transcript of Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne...
![Page 1: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/1.jpg)
Automated user-centered task selection and input modification
Rintse van der Werf
Geke Hootsen
Anne Vermeer
MASLA project
Tilburg University
![Page 2: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/2.jpg)
Outline• Background
• Research
• Discussion and future research
![Page 3: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/3.jpg)
User-centered learning
• Approaches in educational research– Authentic– User initiated– Motivating– Individual needs
![Page 4: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/4.jpg)
MASLA project
• Models of Adaptive Second Language Acquisition
• Combination of Computer Science and Second Language Acquisition
• Goal: building a model for personalized digital language learning web based applications
• How can learning materials automatically be adapted to fit the characteristics and preferences of the language learner?
• Criterion is learning effect.
![Page 5: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/5.jpg)
Requirements for adaptivity
• Annotated learning material– domain model
• Knowledge about learner characteristics– user model
• User model + domain model -> adaptation model (rules)
(Dexter model, 1990; AHAM model (De Bra, 2000))
![Page 6: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/6.jpg)
MASLA Framework
Graphical User Interface
Curriculum
L2 - proficiencies
Learning contents
Learning styles
Learner backgrounds
![Page 7: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/7.jpg)
Task: Vocabulary learning through reading
• Incidental vocabulary learning (side effect of reading for comprehension)
• ZOPD (Vygotsky, 1962); Comprehensible input (Krashen, 1987)
– Assessing learner proficiency
– Assessing text difficulty
based on frequency information from corpora
=> combined in text coverage
![Page 8: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/8.jpg)
Text Coverage
70
75
80
85
90
95
100
1 2 3 4 5 6 7 8 9
learner profficiency (x1000 lemmas)
lem
ma c
overa
ge (
%)
more difficult
easier
![Page 9: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/9.jpg)
Interpreting text coverage
• Hazenberg, 1994; Laufer, 1989; Vermeer, 1998
• Lemma Coverage:
– 85%: Global understanding– 90%: Good understanding– 95%: Almost complete understanding
![Page 10: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/10.jpg)
Effective Instruction
• Comprehensible but challenging
• Lemma coverage 85% - 92%
• Support from input modification– Dictionary/glossary (see Hulstijn et al., 1996; Plass et al.,
1998; Watanabe, 1997)– User initiated “focus on form”
![Page 11: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/11.jpg)
Text Coverage
70
75
80
85
90
95
100
1 2 3 4 5 6 7 8 9
learner profficiency (x1000 lemmas)
lem
ma c
overa
ge (
%)
Top criterion
Bottom criterion
![Page 12: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/12.jpg)
Summary of research background
• Web based tool for automatic adaptive selection of the appropriate text for a specific user.
• Automated analysis of text difficulty.
• User proficiency calculation from score on vocabulary test.
• User gets text that is comprehensible but challenging and has input modification for unknown words to support for understanding the text.
![Page 13: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/13.jpg)
Research questions
• A. Adaptive selection of texts leads to:
• A learning effect for all users• No difference between learners with different proficiency
levels
• B. Using input modification:
• There is a relation between noticing and retention• (There is no difference in this relation for different
proficiency levels)
![Page 14: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/14.jpg)
Method (1)
• Subjects (N=32)
• Reading Texts (16)– 4 clusters
• Input modification
![Page 15: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/15.jpg)
Text coverage for selected texts
60
65
70
75
80
85
90
95
100
1 2 3 5 8
Almost complete comprehension
Global comprehension
![Page 16: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/16.jpg)
Mean text coverage per cluster
60
65
70
75
80
85
90
95
100
1 2 3 5 8
Almost complete comprehension
Global comprehension
![Page 17: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/17.jpg)
Method (1)
• Subjects (N=32)
• Reading Texts (16)– 4 clusters
• Input modification
![Page 18: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/18.jpg)
![Page 19: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/19.jpg)
Method (2)
• Data collection:
– User logging and tracking
– Testing material• Vocabulary proficiency test• Text specific vocabulary tests• Comprehension questions
• Procedure
![Page 20: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/20.jpg)
Learning gains
Learning gains
Learning gains
Learning gains
Procedure
![Page 21: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/21.jpg)
Results (1)
• A mean learning effect occurred for all clusters– 5% learning gains
• No significant difference between groups– both pre and posttest scores– learning gains
![Page 22: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/22.jpg)
Results (2)
• Correlation between noticing and retention– Mean Φ correlation for subjects: .28– Mean Φ correlation for items: .50
• in general, the use of the dictionary was limited – No significant difference between proficiency groups
• In lookup behavior• In correlation
![Page 23: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/23.jpg)
Conclusion
• Automated assessment of texts based on corpora information is a useful indication of text (task?) difficulty.
• Adaptive selection of texts based on vocabulary proficiency works.
• Open, web based learning environment provides flexibility in the curriculum and opportunities for individualized tasks.
![Page 24: Automated user-centered task selection and input modification Rintse van der Werf Geke Hootsen Anne Vermeer MASLA project Tilburg University.](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649c725503460f94924792/html5/thumbnails/24.jpg)
Discussion and future work
• Increase learning gains– More adaptivity in text selection
• Increase exposure to target words• Based on observed behavior
• Increase usability of input modification– Individualize annotation
• Based on observed behavior• More focus on form
• Use different corpus for text coverage– Now children’s corpus, future Celex/CGN– Unknown lemmas – Multiword expressions