Improving a Mathematical Intelligent Tutoring System

Experiments & Equation Solver Improvements July 27th, 2012

Jennifer Ferris-Glick & Hee Seung Lee

Mathematical Intelligent Tutoring Systems

• How do we improve out dated textbook methods of learning and expand on intelligent tutoring systems?

Objective of Study

• To investigate the role of verbal explanation - When does it help learning?

• To Explore the degree to which we can teach without verbal instruction - Is verbal direction necessary? - How can we help students discover/learn underlying problem structure and induce correct rules for problem solving without explicit verbal directions?

5

+

5 -

29

*

+

29

*4

5

Linearize task: solving algebra problems using data-flow diagrams: allows to study algebra learning anew in college population

Findings from Linearize study

• Showing intermediate cognitive steps• Preventing superficial analogy from an example to

problem• Building appropriate problem representations• Avoiding excessive floundering• Giving a chance to reflect on solved problems

Showing Intermediate Cognitive Steps

• Students still have to fill in gaps between steps when students study a worked example

• Students learned better when intermediate cognitive steps were clearly revealed than when students had to determine hidden steps.

Force students to study step-by-step solutions

Preventing Superficial Analogy

• Given a highly similar example, students tend to draw a superficial analogy from the example to problem. – When superficial solution is available, students fail to

abstract deep problem structure.– When examples look too dissimilar from a target

problem, students sometimes simply ignore or do not spend enough time to study provided examples.

Examples should be dissimilar, and structurally similar to a target problem.

Building Appropriate Problem Representations

• Revealing hidden structure helped transfer - Making connections to Algebra activated prior knowledge and helped understand the problem structure

• Highlighting relevant features (e.g., color coding)

Highlight features to direct student’s attention and reveal hidden problem structure

Restricting Floundering & Supporting Reflection Activity

• When students show too much floundering, they may not remember solution paths. In order to reduce floundering:– Immediate feedback on wrong attempt– Providing hints when students show floundering– Auto-solving with a time limit

• Giving students an opportunity to make sense of already solved problems Allow only a certain number of off-track

activities. When a certain step was solved automatically, show corresponding verbal explanations for a while.

General Improvements to Carnegie Learning Cognitive Tutor

(all conditions)

Showing Intermediate Cognitive Steps– Force step-by-step guided

example before solving• Maybe show previously

solved equations– Hint improvements

• Some changes but not full review

Preventing superficial analogy– Present side-by-side example – Similarity of example

• Randomly selected from a pool of problems in each section (not too similar, not too dissimilar)

Appropriate Problem Representation– Aligning equations on the equals

sign– Coloring variable sides of

equation

Restrict Floundering– Reduce off-path steps

• either allow 0 or 1 off-path step

Reflection Time– Provide side-by-side already

worked out problems

Target MaterialOne-step linear equations

• Solving with addition and subtraction (no type in)– o w + 6 = -1

• Solving with addition and subtraction (type in)– o -4 = 1 + z

• Solving with multiplication and division (no type in)– o 2x = -1

• Solving with multiplication and division (type in)– o 8w = 5

Target MaterialTwo-step linear equations

• Solving One-step equations (Type in)– o Mix of types from unit 5

• Solving with multiplication (no type in)– o -2 = 5x + 10

• Solving with multiplication (type in)– o -2 = 5x + 10

• Solving with division (no type in)– o x/6 + 7 = 8

• Solving with division (type in)– o x/6 + 7 = 8

• Solving with a variable in the denominator (no type in)– o 10 = -2/x

• Solving with a variable in the denominator (type in)– o 2/z = -4

Target MaterialLinear equations with similar terms

• Solving two-step equations (type in)– o Mix of types from unit 6

• Combining like variable terms and a constant with integers (no type in)– o 2 – 9x + 6x = 2

• Combining like variable terms and a constant with integers (type in)– o -6x + 2 + 5x = -6

Combining like variable terms with decimals (no type in)– o 9 = 9.7x – 2x

• Combining like variable terms with decimals (type in)– o -2.1z – 8z = 2

• Combining like variable terms and a constant with decimals (no type in)– o 8.1 + 7.4x + 3.7x = 4.6

• Combining like variable terms and a constant with decimals (type in)– o -6.6z – 2.8 – 6.7z = 2.8

Target MaterialLinear equations and the distributive property

• Solving Linear Equations (Type In)– o 7 + 2x = -3

• Using Multiplication and Integers (No Type In)– o 4(z - 3) = 3

• Using Multiplication and Integers (Type In)– o -6(z – 6) = 10

• Using Multiplication and Decimals (No Type In)– o -4.5 = -2.6(z + 2.9)

• Using Multiplication and Decimals (Type In)– o 8.3(z – 5.9) = 6.8

• Using Multiplication and Large Decimals (No Type In)– o -43.54 = -16.63(z + 24.90)

• Using Multiplication and Large Decimals (Type In)– o -51.78(w + 97.21) = -11.63

• Using Division (No Type In)

Experimental Conditions

Kanawha County School District

3 Conditions divided by class (4 classes for each condition):Discovery, Instruction, Standard

Presented in line with curriculum.

Experimental ConditionsDiscovery Direct Instruction Current tutor

Hints “How” hints only (on general interface)

“How” and “why” hints available

“How” and “why”, no changes to 2011 std release

Just in Time (JITs) Messages

None Normal Normal

Step by Step Guided Problem

“How” hints onlyForce step-by-step in the beginning of new section

“How” and “why” Force step-by-step in the beginning of new section

Optional

Side by Side Worked Examples

Side-by-side worked example with history tags

Side-by-side worked example with history tags

None

Example Types• Step-by-Step Guided Problem

One per section – selected based on sections “new” material

Students simply follow directions in a step-by-step manner to solve the problem

• Side-by-Side Worked ExamplesAlready solved problems appear next to a

target problem

w + 6 = -1

Instruction condition •Provide “why” and “how” hints•Directly tell students what to do

To isolate w, subtract 6 from both sides. Select “subtract both sides” and enter 6.

Step-by-Step Guided ProblemInstruction

w + 6 = -1

Discovery condition •Provide “how” hints•Provide instruction on general interface

Select an item from Transform menu and enter a number.

Step-by-Step Guided ProblemDiscovery

w + 6 -6

w + 6 – 6

=

=

-1-6 subtract 6 from both sides-1 – 6

Both conditions •Provide guided problem•Align equation on the equal sign •Coloring variable side of the equation •Highlighting changes between steps

Color-Coding

w + 6 +6

w + 6 + 6

=

=

-1+6 add 6 from both sides-1 + 6

Wrong Attempts

This is not the best way to solve the problem. Click “undo” and ask for hint if you need help.

Both conditions•Reduce off-path steps using “undo” button.

Both conditions•Provide already solved problems (screen-shot)•Selected from the pool of problems in the same section

Side by Side Worked ExampleInstruction & Discovery

w + 6 +6

.

w + 6 + 6

w

=

.

=

=

-1+6 add 6 from both sides

-1 + 6 combine like terms

5

Example Problem

8 = x - 2

Post-Test

2 types of test questions:(paper-based, within subject)

-Plain equations -Equations with solver interface scaffolding