Grupo: Miriã Suellén Poliana Cristina Rebeka Viviane Cristina.
Cristina Conati Department of Computer Science University of British Columbia
description
Transcript of Cristina Conati Department of Computer Science University of British Columbia
![Page 1: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/1.jpg)
Cristina ConatiDepartment of Computer ScienceUniversity of British Columbia
Beyond Problem-solving: Student-adaptive Interactive Simulations
for Math and Science
![Page 2: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/2.jpg)
Overview
Motivations Challenges of devising student-adaptive simulations Two examples of how we target these challenges
– ACE: interactive simulation for mathematical functions– CSP Applet: interactive simulation for AI algorithm
Conclusions and Future work
![Page 3: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/3.jpg)
Intelligent Tutoring Systems (ITS)
Create computer-based tools that support individual learners By autonomously and intelligently adapting to their specific needs
StudentModel
Tutor
DomainModel
Adaptive Interventi
ons
![Page 4: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/4.jpg)
ITS Achievements
In the last 20 years, there have been many successful initiatives in devising Intelligent Tutoring Systems
(Woolf 2009, Building Intelligent Interactive Tutors, Morgan Kaufman)
Mainly ITS that provide individualized support to problem solving through tutor-lead interaction (coached problem solving)– Well defined problem solutions => guidance on problem solving
steps– Clear definition of correctness => basis for feedback
![Page 5: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/5.jpg)
Beyond Coached Problem Solving
Coached problem solving is a very important component of learning
Other forms of instruction, however, can help learners acquire the target skills and abilities– At different stages of the learning process– For learners with specific needs and preferences
Our Goal: Extend ITS to other learning activities that support student initiative and engagement: – Interactive Simulations– Educational Games
![Page 6: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/6.jpg)
Overview
Motivations Challenges of devising student-adaptive simulations Two examples of how we target these challenges
– ACE: interactive simulation for mathematical functions– CSP Applet: interactive simulation for AI algorithm
Conclusions and Future work
![Page 7: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/7.jpg)
Challenges Activities more open-ended and less well-defined than
pure problem solving
– No clear definition of correct/successful behavior Different user states to be captured (meta-cognitive,
affective) in order to provide good tutorial interventions– difficult to assess unobtrusively from interaction events
How to model what the student is doing? How to provide feedback that fosters learning while
maintaining student initiative and engagement?
![Page 8: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/8.jpg)
Our Approach
Student models based on formal methods for probabilistic reasoning and machine learning
Increase information available to student model through innovative input devices:– e.g. eye-tracking and physiological sensors
Iterative model design and evaluation
![Page 9: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/9.jpg)
Overview
Motivations Challenges of devising student-adaptive simulations Two examples of how we target these challenges
– ACE: interactive simulation for mathematical functions– CSP Applet: interactive simulation for AI algorithm
Conclusions and Future work
![Page 10: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/10.jpg)
ACE: Adaptive Coach for Exploration
Activities organized into units to explore mathematical functions (e.g. input/ouput, equation/plot)
Probabilistic student model that captures student exploratory behavior and other relevant traits
Tutoring agent that generates tailored suggestions to improve student exploration/learning when necessary
(Bunt, Conati, Hugget, Muldner, AIED 2001)
![Page 11: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/11.jpg)
Adaptive Coach for Exploration
EDM 2010 11
![Page 12: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/12.jpg)
12
Adaptive Coach for Exploration
![Page 13: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/13.jpg)
13
Adaptive Coach for Exploration
Before you leave this exercise, why don’t you try scaling the function by a large negative value?Think about how this will affect the plot
![Page 14: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/14.jpg)
ACE Student Model(Bunt and Conati 2002)
Knowledge
Individual Exploration Cases
Exploration of Exercises
Exploration Categories
Exploration of Units
Iterative process of design and evaluation Probabilistic model of how individual exploration actions
influence exploration and understanding of exercises and concepts
e.g. (in Plot unit) • positive/negative slope• positive/negative intercept• large/small, positive/negative exponents…
![Page 15: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/15.jpg)
Modeling Student Exploration
Our first attempt (Bunt and Conati, 2002)
Learning
Student ModelNumber and Coverage of Exploratory Actions, e.g.• Positive/negative Y-Intercept• Odd/Even, Positive Negative Exponent....
Interface Actions
![Page 16: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/16.jpg)
Preliminary Evaluation
Quasi-experimental design with 13 participants using ACE (Bunt and Conati 2002)– The more exercises were effectively explored according to
the student model, the more the students improved
– The more hints students followed, the more they learned
Because the model only considers coverage of student actions, it can overestimate student exploration
Need to consider whether the student is reasoning about the effects of his/her actions– Self-explanation meta-cognitive skill:
![Page 17: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/17.jpg)
Revised User Model (Bunt, Muldner and Conati, ITS2004; Merten and Conati, Knowledge Based Systems 2007)
Learning
Student Model
Number and coverage of student actions Self-explanation of action outcomes
Time between actions Gaze Shifts in Plot Unit
Interface ActionsInput from eye-tracker
![Page 18: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/18.jpg)
Results on Accuracy
We evaluated the complete model against– The original model with no self-explanation– A model that uses only time in between actions as evidence of self-
explanation
Accuracy on SE Accuracy on Learning
50
60
70
80
No SE
SE (Time)
SE (Time + Gaze)
![Page 19: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/19.jpg)
What’s Next (1)
Test adaptive interventions to trigger self-explanation (Conati 2011)
![Page 20: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/20.jpg)
Discussion
ACE work provided evidence that• It is possible to track more “open ended” students’
behaviors than structured problem solving• eye-tracking can support the process
However, hand-coding the relevant behaviors, as we did for ACE (knowledge-based approach)• is time consuming
• likely to miss other, less intuitive patterns of interaction related to learning (or lack thereof)
![Page 21: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/21.jpg)
Alternative Approach (Amershi and Conati 2009, Kardan and Conati 2011)
Behavior Discovery Via Data Mining
Association Rules
MiningClustering
Actions LogsOther Data
Fe
atu
re
Ve
cto
rs
Vector of Interaction Features- Frequency Of
Actions- Latency Between Actions……………
Extract rules describing distinguishing patterns in each cluster
Groups together students that have similar interaction behaviors
Interpret in terms oflearning
• Experts• Performance
Measure(s)
![Page 22: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/22.jpg)
Overview
Motivations Challenges of devising student-adaptive simulations Two examples of how we target these challenges
– ACE: interactive simulation for mathematical functions– CSP Applet: interactive simulation for AI algorithm
Conclusions and Future work
![Page 23: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/23.jpg)
Tested with AI Space CSP applet
AISpace (Amershi et al., 2007)– set of applets implementing interactive simulations of
common Artificial Intelligence algorithms– Used regularly in our AI courses– Google “AISpace” if you want to try it out
Applet for Constraint Satisfaction problems (CSP), visualizes the working of the AC3 algorithm
![Page 24: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/24.jpg)
27
AISpace CSP Applet
Direct Arc Clicking
![Page 25: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/25.jpg)
User Study (Kardan and Conati 2011)
65 subjects– Read intro material on the AC-3 algorithm– Pre test– Use CSP applet on two problems– Post test
13,078 actions More than 17 hours of interaction
![Page 26: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/26.jpg)
Dataset
Features:– frequencies of use for each action– pause duration between actions (Mean and SD)– 7 actions 21 features
Performance measure for validation– Learning Gain from pretest to posttest
Feature vectors Clustering
Behavior Discovery
Rule Mining
![Page 27: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/27.jpg)
Found 2 clusters
Statistically significant difference in Learning Gains (LG)– High Learners (HL) and
Low Learners (LL) clusters
32
Feature vectors Clustering
Behavior Discovery
Rule MiningClustering
![Page 28: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/28.jpg)
Usefulness:Sample Rules
HL members: Use Direct Arc Click action very frequently (R1).
HL cluster: R1: Direct Arc Click frequency = Highest (Conf =100%, Class Cov = 100%)
LL cluster:R2: Direct Arc Click Pause Avg = Lowest (Conf =100%, Class Cov = 100%)
R3: Direct Arc Click frequency = Lowest (Conf = 93%, Class Cov=93.5%)
33
LL members: Use Direct Arc Click sparsely (R3) Leave little time between a Direct Arc Click and the next action
(R2)
Feature vectors Clustering
Behavior Discovery
Rule Mining
![Page 29: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/29.jpg)
Great, but what do we do with this?
We can use the learned clusters and rules to classify a new student based on her behaviors
Use detected behaviours for adaptive support– Promoting the behaviours conducive of learning– Discouraging/preventing detrimental behaviours
34
![Page 30: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/30.jpg)
The User Modeling Framework
35
Association Rules
MiningClustering
Feature Vector
Calculation
OnlineClassifier
Adaptive Interventions
Behavior Discovery
User Classification
Actions LogsOther Data
F
e
at
u
re
New user’s
Actions
Vector of Interaction Features
If user is a LL and uses Direct Arc Click very infrequently (R3)
Then prompt this action
If user is a LL and pauses very briefly after a Direct Arc Click (R2)
Then take action to slow her down
![Page 31: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/31.jpg)
Classifier Evaluation
Leave-one-out Cross Validation on dataset of 64 users For each user u in dataset
1. Remove user u2. do Behaviour Discovery on the remaining 633. for each of u’s actions:
» Calculate the feature vector uv
» Classify uv » Compare with u’s original label
![Page 32: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/32.jpg)
Accuracy as a function of observed actions
![Page 33: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/33.jpg)
Discussion
User modeling framework for open-ended and unstructured interactions– Relevant behaviours are discovered via data mining
techniques instead being hand-crafted Very encouraging results with CSP applet
– Detected clusters represent groups with different learning gains
– Online classifier: good accuracy soon enough to generate adaptive interventions
– These interventions can be derived from the generated rules
![Page 34: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/34.jpg)
Current WorkApplying the discovered rules to generate the adaptive
version of the CSP appletAdding eye-tracking input to the dataset
![Page 35: Cristina Conati Department of Computer Science University of British Columbia](https://reader033.fdocuments.net/reader033/viewer/2022051317/56816388550346895dd477a7/html5/thumbnails/35.jpg)
Conclusions
Research on devising student-adaptive didactic support for exploratory activities beyond problem solvingInteractive simulations
Challenges in modeling interactions with no clear structure or definition of correctness
Student modeling approaches based on probabilistic techniques and unsupervised machine learning very promising results
Shown how eye-tracking can help!We are also exploring it in relation to assessing engagement
and attention in educational games (Muir and Conati 2011)