FACT (Formative Assessment with
Computational Technology)
Math Assessment Project
Hugh Burkhardt (Nottingham)
Malcolm Swan (Nottingham)
Daniel Pead (Nottingham)
Alan Schoenfeld (Berkeley)
David Foster (SVMI)
Arizona State University
1
Kurt VanLehn (PI)
Salman Cheema (leader)
14 grad students
ASU Center for Assured and Scalable Data
Engineering
CASCADE-IUCRC
Industry/University Collaborative Research
Center (I/UCRC) * NSF I/UCRC planning grant (NSF-IIP1464579)
CASCADE Mission
Mission: to support the innovation of data
architectures and tools that can support timely and
assured decision making to generate value.
Validate & Interpret
Act & Adapt
Sense & Integrate
Simulate & Predict
Can teachers circulate among student
groups with the computer helping?
4
Tablet
GroupGroup
For individual work, current intelligent
tutoring systems + teachers suffice
Tablet
When students work collaboratively in
small groups on a joint product, how
can tablets help the teacher?
Teacher
The FACT research problem
Given
– Successful formative assessment based on paper
– Unfortunately, the required analyses are difficult for
teachers to do, especially in real-time
Develop
– Analysis methods
– Media system: Input from students; Output to teachers
Evaluate
– Impact on student and teacher learning
– Impact on classroom processes
7
Describe the research problem
Describe FACT
Outline
8
Next
Classroom Challenges
Developed over 10 years by Mathematics
Assessment Project (map.mathshell.org)
Addresses CCSS-M
Approximately 100 available; grades 6 through 10
Widely used despite lack of publisher
Existing evaluation (Herman et al., 2015)
– Raised state test scores
– Little change in mathematical misconceptions/thinking
9
A Classroom Challenge is spread
over 3 days
10
Solve individually15 minutes of Day 1
5 minutes of Day 2 for feedback
Solve in small groups
55 minutes of Day 2
Teacher constantly analyses work-
in-progress and gives formative
feedback.
Discuss as whole class
Critique sample solutions in
groups
Solve individually 15 minutes of Day 3
A high school problem: Determine if “2000
descendants in 18 months” is realistic.
Length of pregnancy: 2 months
Age at which a female cat can first get pregnant: 4 months
Average number of litters a cat can have in one year: 3
Number of kittens per litter: 4 to 6
Age at which cat no longer has kittens: 10 years
11
What should a teacher say?
12
Length of pregnancy: 2 months
Age at which a female cat can first get pregnant: 4 months
Average number of litters a cat can have in one year: 3
Number of kittens per litter: 4 to 6
What should a teacher say?
13
Length of pregnancy: 2 months
Age at which a female cat can first get pregnant: 4 months
Average number of litters a cat can have in one year: 3
Number of kittens per litter: 4 to 6
What should a teacher say?
14
Length of pregnancy: 2 months
Age at which a female cat can first get pregnant: 4 months
Average number of litters a cat can have in one year: 3
Number of kittens per litter: 4 to 6
What should a teacher say?
15
Length of pregnancy: 2 months
Age at which a female cat can first get pregnant: 4 months
Average number of litters a cat can have in one year: 3
Number of kittens per litter: 4 to 6
What should a teacher say?
16
Length of pregnancy: 2 months
Age at which a female cat can first get pregnant: 4 months
Average number of litters a cat can have in one year: 3
Number of kittens per litter: 4 to 6
Advice from the teacher’s
manual.
17
Will these kittens grow up and have kittens?
Are they all males? Please write your assumptions
Some beautiful math, worth
pointing out to the whole class
18
Length of pregnancy: 2 months
Age at which a female cat can first get pregnant: 4 months
Average number of litters a cat can have in one year: 3
Number of kittens per litter: 4 to 6
Teacher formative assessments
decisions
What should I say to help this
student/group engage the math?
Is this something I can use with
the whole class?
19
Such decisions must be made very
rapidly and constantly
What should a teacher say?Hint: The numbers mentioned in the problem statement are: 2, 4, 3, 6 and 10.
20
What should a teacher say?
21
What should a teacher say?
22
Can the computer perceive kids’
derivations?
23
Can teachers circulate among students
with the computer helping?
24
Describe the research problem
Describe FACT
Outline
25
Next
Centaurs: Computers & humans
as nearly equal partners
Chess centaur
– Both agents propose & analyze possible moves
– Humans make the final decision
– Freestyle (fast) and correspondence (slow)
Classroom centaur
– Both agents propose & analyze possible advice to
students
– Human teacher makes the final decision
– During class (fast) and during evenings (slow)
26
FACT Media system replaces
paper with one tablet per student
27
Every student has a tablet.
Every tablet in a group edits the same poster.
28
One student
Another
student
Teachers’ Organize screen shows
students & groups. Tap to show poster.
29
Tap “TALK” button to pause student
tablets
30
TALK button
Tap “PROJECT” button to display
teacher’s screen to whole class
31
PROJECT button
Teacher’s Activity screen. Tap
row to select activity.
32
FACT Analysis system alerts
teacher to learning opportunities
33
Current FACT workflow for giving
advice to a single student/group
1. Alert
2. Peek
3. Preview
Send?
4. Sent
34
1. Alert
2. Peek
3. Preview
Send?
4. Sent
35
36
1. Alert
2. Peek
3. Preview
Send?
4. Sent
37
1. Alert
2. Peek
3. Preview
Send?
4. Sent
38
1. Alert
2. Peek
3. Preview
Send?
4. Sent
Current FACT workflow for
whole-class learning alert
39
Current status of FACT Software
– Android version stable but requires obsolete tablet
– Web app ready by end of July
Analytic power
– Product detectors for card-moving lessons – working well now
– Product detectors for handwriting lessons – next year
– Process detectors – next year
Policies for alerting
– Teachers: 2 policies explored, just starting
– Students: no policies explored yet
Evaluations
– Formative: 30 classroom trials so far; 10+ next year
– Summative: Applying for more funding
40
Take home points
Problem
– Complex, open-ended, collaborative reasoning is coming to
classrooms.
– Teachers cannot easily analyze such reasoning.
Research questions: Centaurs vs. mere tools/media
– Can computers perceive the students’ reasoning without disrupting
it or the classroom?
– Can computers can do formative analyses of complex, handwritten
problem solving?
– Can this assist rather than overwhelm the teachers?
41
Top Related