1 Introducing Bayesian Nets in AgenaRisk An example based on Software Defect Prediction.
Bayesian Nets in Student Modeling ITS- Sept 30, 2004.
-
Upload
barbara-goodwin -
Category
Documents
-
view
216 -
download
0
Transcript of Bayesian Nets in Student Modeling ITS- Sept 30, 2004.
Bayesian Nets in Student ModelingBayesian Nets in Student Modeling
ITS- Sept 30, 2004ITS- Sept 30, 2004
Sources of UncertaintySources of Uncertainty• Incomplete and/or incorrect
knowledge• Slips and/or guesses• Multiple derivations• Invisible inferences• Not showing all work• Help messages• Self-explaining ahead
• Incomplete and/or incorrect knowledge
• Slips and/or guesses• Multiple derivations• Invisible inferences• Not showing all work• Help messages• Self-explaining ahead
Andes student modelAndes student model
• Knowledge tracing• Plan recognition• 1st to use student’s domain
knowledge
• Action prediction• Andes first to support all three
• Knowledge tracing• Plan recognition• 1st to use student’s domain
knowledge
• Action prediction• Andes first to support all three
Goals of AndesGoals of Andes• Students work as much as possible
alone• React to student’s incorrect action,
signal error, explain• React to student’s impasse, provide
procedural help• Assure student understands examples,
prompt self-explaining
• Students work as much as possible alone
• React to student’s incorrect action, signal error, explain
• React to student’s impasse, provide procedural help
• Assure student understands examples, prompt self-explaining
Types of helpTypes of help
• Error help• Procedural help (ask for hints)• Unsolicited help (for non-
physics errors)• Different levels of hints ‘til
“bottom-out hint”
• Error help• Procedural help (ask for hints)• Unsolicited help (for non-
physics errors)• Different levels of hints ‘til
“bottom-out hint”
Usage of student modelUsage of student model
• Plan recognition: recognize and support goals (requires prediction)
• Asses knowledge: help presentation (reminder v. minilesson)
• Assess mastery level: prompt self-explanation or not
• Plan recognition: recognize and support goals (requires prediction)
• Asses knowledge: help presentation (reminder v. minilesson)
• Assess mastery level: prompt self-explanation or not
Self-Explaining CoachSelf-Explaining Coach
• Step correctness (domain)• Rule Browser• E.g.: using force or acceleration
• Step utility (role in solution plan)• Plan Browser• Recognize goals
• Step correctness (domain)• Rule Browser• E.g.: using force or acceleration
• Step utility (role in solution plan)• Plan Browser• Recognize goals
Bayesian networkBayesian network
• Solution graph: map of all solutions with no variables (propositional)
• Solution graph: map of all solutions with no variables (propositional)
Types of nodesTypes of nodes• Domain-general: rules• 2 values indicating mastery
• Task-specific: • facts, goals, rule apps, strategy
nodes• Doable (done already or knows all
needed) • Not-doable
• Domain-general: rules• 2 values indicating mastery
• Task-specific: • facts, goals, rule apps, strategy
nodes• Doable (done already or knows all
needed) • Not-doable
Knowledge evolutionKnowledge evolution
• Dynamic Bayesian network• Analyze each exercise alone• Roll-up: prior probabilities set to
marginal probabilities for previous• Improvements: could model
dependencies & knowledge decay
• Dynamic Bayesian network• Analyze each exercise alone• Roll-up: prior probabilities set to
marginal probabilities for previous• Improvements: could model
dependencies & knowledge decay
Intention or ability?Intention or ability?
• Probability that student can and IS implementing a certain goal
• Decision-theoretic tutor keeps probabilites of “focus of attention”
• Probability that student can and IS implementing a certain goal
• Decision-theoretic tutor keeps probabilites of “focus of attention”
Problem creationProblem creation
• Givens• Goals• Problem-solver applies rules,
generating subgoals until done• Solution graph created
• Givens• Goals• Problem-solver applies rules,
generating subgoals until done• Solution graph created
Andes assessorAndes assessor
• Dynamic belief network for domain-general nodes
• Rules - priors set by test scores• Context-Rules• P(CR=true|R=true)=1• P(CR=true|R=false)=difficulty• One context changes, adjust rest
• Dynamic belief network for domain-general nodes
• Rules - priors set by test scores• Context-Rules• P(CR=true|R=true)=1• P(CR=true|R=false)=difficulty• One context changes, adjust rest
Task-specific nodesTask-specific nodes
• Fact, goal, rule application, strategy
• Context-Rule nodes link task-specific to domain-general rules
• Fact, goal, rule application, strategy
• Context-Rule nodes link task-specific to domain-general rules
Fact & Goal NodesFact & Goal Nodes
• A.k.a. Propositional Nodes• 1 parent for each way to derive• Leaky-OR: T if 1 parent T, also
sometimes true if not• Reasons: guessing, analogy, etc
• A.k.a. Propositional Nodes• 1 parent for each way to derive• Leaky-OR: T if 1 parent T, also
sometimes true if not• Reasons: guessing, analogy, etc
Rule-Application NodesRule-Application Nodes
• Connect CR,Strategy & Prop nodes to new derived Prop nodes
• Doable or not-doable• Parents: 1 CR, pre-condition Props,
sometimes one Strategy node• Noisy-AND: T if ALL parents T, but
sometimes not, 1-alpha
• Connect CR,Strategy & Prop nodes to new derived Prop nodes
• Doable or not-doable• Parents: 1 CR, pre-condition Props,
sometimes one Strategy node• Noisy-AND: T if ALL parents T, but
sometimes not, 1-alpha
Strategy NodesStrategy Nodes
• Used when >1 way to reach a Goal• Paired with a Goal Node• Values are mutually exclusive• No parents in network• Priors=freq. students use this strat.
• Used when >1 way to reach a Goal• Paired with a Goal Node• Values are mutually exclusive• No parents in network• Priors=freq. students use this strat.
Compare FiguresCompare Figures
• Figure 9 before observing A-is-body
• Figure 10 after observing A-is-body
• Figure 9 before observing A-is-body
• Figure 10 after observing A-is-body
HintsHints
• Add a new parent to a Prop node
• Accounts for guessing
• Add a new parent to a Prop node
• Accounts for guessing
SE-CoachSE-Coach• Adds nodes for Read• Link these to Prop nodes• Longer read time, higher prob knows
Prop (p 26)
• Adds nodes for plan selection• Link these to Context-Rules
• Rule Application node prob T if knows CR & all preconditions=Noisy-AND
• Adds nodes for Read• Link these to Prop nodes• Longer read time, higher prob knows
Prop (p 26)
• Adds nodes for plan selection• Link these to Context-Rules
• Rule Application node prob T if knows CR & all preconditions=Noisy-AND
EvaluationEvaluation
• Simulated students, 65% correct for rule mastery
• 95% if no “invis inferences” and has to “show all work”
• Post-test shows significant learning• Voluntary acceptance?• Accuracy of plan recognition
• Simulated students, 65% correct for rule mastery
• 95% if no “invis inferences” and has to “show all work”
• Post-test shows significant learning• Voluntary acceptance?• Accuracy of plan recognition