Exercise-Enhanced Sequential Modeling for Student...
Transcript of Exercise-Enhanced Sequential Modeling for Student...
![Page 1: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/1.jpg)
Exercise-Enhanced Sequential Modeling for Student Performance Prediction
The 38st Association for the Advancement of Artificial Intelligence (AAAI'17)
2018/02/02 - 02/07, New Orleans, Louisiana
![Page 2: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/2.jpg)
Outline
�
�
�
�
�
![Page 3: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/3.jpg)
!
Background- Online Education SystemOnline education systems provide students with open access for self-learning (e.g., learning remedy suggestion and personalized exercise recommendation).
![Page 4: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/4.jpg)
)
Background- Predict Student Performance
How can we say an education system understand a student?
(
)
![Page 5: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/5.jpg)
Research Probleml Urgent issue: Predict Student Performance(PSP)
l How to automatically predict student performance without manual intervention?
l This work regards score as performance.
l Opportunityl Exercises records of studentsl Text materials of exercise
![Page 6: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/6.jpg)
2
Challenge 1 for PSP
l Diverse expressions of exercises
l Need a unified way to understand and represent them automatically.
2 1 1
![Page 7: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/7.jpg)
Challenge 2 for PSP
l Long-term historical exercising
!" !# !$ !%
!%&"
![Page 8: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/8.jpg)
Challenge 3 for PSP
l Cold start problem
![Page 9: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/9.jpg)
Related Work for PSPl Education Psychology
l IRT (Item Response Theory)
l models student exercising records by a logistic-like function
l BKT (Bayesian Knowledge Tracing)
l traces them with a kind of hidden Markov model
l Machine Learning and Data Miningl PMF
l projects students and exercises into latent factors
l Deep Learning l DKT
l deep learning method uses RNN to model student exercising process for prediction .
![Page 10: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/10.jpg)
Outline
�
�
�
�
�
![Page 11: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/11.jpg)
Problem Definition• Given: the exercising records of each student and the text
descriptions of each exercise from 1 to T�• Goal: train a unified model M, predict the scores on the next exercise !"#$ of each specific student.
![Page 12: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/12.jpg)
Outline
�
�
�
�
�
![Page 13: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/13.jpg)
Exercise-Enhanced Recurrent Neural Network(EERNN)
![Page 14: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/14.jpg)
Step 1: Word Embedding
l Main procedures:l Word split
l Latex to feature
l Word to vector
l Goal: learn word representations from semantic perspective in math exercise.
![Page 15: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/15.jpg)
Step 2: Exercise Embedding• Goal: Exercise Embedding learns the semantic representation of each exercise
!" from its text input #" automatically.
![Page 16: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/16.jpg)
Step 3: Student Embeddingl Goal: Student Embedding aims at modeling the whole student exercising
process and learning the hidden representations of students.
!"0000…000000
!" 0000…000000
Combine the score and embedding %&
![Page 17: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/17.jpg)
Step 4: Prediction (Two Strategies)• Goal: Predicting her performance on exercise !"#$ at step T +1.
![Page 18: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/18.jpg)
Step 4: Prediction (Two Strategies)
����
������
��
��
��
��
……
��
��
��
��
Predicting her score on exercise"#$% is &̃#$%
![Page 19: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/19.jpg)
Step 4: Prediction (Two Strategies)• Goal: predicting her performance on exercise !"#$ at step T +1.
![Page 20: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/20.jpg)
Outline
�
�
�
�
�
![Page 21: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/21.jpg)
Experimentsl Experiments dataset
l Supplied by Zhixue, IFLYTEK
l a widely-used online learning system, which provides senior high school students with a large exercise resources for exercising.
![Page 22: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/22.jpg)
Experimentsl Baseline methods
l Variants of EERNN: LSTMM, LSTMAl just utilize knowledge-specific representationsl To validate the importance to incorporate exercise texts for the prediction in
EERNN
l Education Psychology: IRT, BKTl Machine Learning and Data Mining: PMFl deep learning method: DKT
l The most similar method to ours
l Evaluation metricsl Regression perspective: RMSEl classification perspective: ACC, AUC
![Page 23: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/23.jpg)
Experiments
![Page 24: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/24.jpg)
Experiments
Information can be used by EERNNM
More Information can be used by EERNNM
Old Information forget by EERNNMFocus useful informationby EERNNA
![Page 25: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/25.jpg)
Experiments
![Page 26: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/26.jpg)
Outline
�
�
�
�
�
![Page 27: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/27.jpg)
Conclusionl Proposed a novel EERNN framework to predict student future
performance.
l EERNN integrated two critical components, BiLSTM to extract exercise semantic representations from texts, LSTM architecture to trace student states.
l Proposed two strategies for prediction : EERNNM with Markov property and EERNNA with Attention mechanism.
l Experiments on real-world dataset demonstrated the effectiveness(specially cold start problem) of EERNN .
![Page 28: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/28.jpg)
Future Workl Different exercise types (e.g., the subjective exercises
with continuous scores)
l Incorporate more informationl Knowledge concepts
l The time cost on exercises
l Integrate some educational theoriesl learning and forgetting curves
l Guess and slip
![Page 29: Exercise-Enhanced Sequential Modeling for Student ...home.ustc.edu.cn/~huangzhy/files/slides/YuSu-AAAI2018-slide.pdfl Supplied by Zhixue, IFLYTEK l a widely-used online learning system,](https://reader035.fdocuments.net/reader035/viewer/2022071210/60213372504d9566102742fc/html5/thumbnails/29.jpg)
Q & A