Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota,...
-
Upload
sophia-todd -
Category
Documents
-
view
219 -
download
0
Transcript of Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota,...
![Page 1: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/1.jpg)
Andrew Mason, Ph.D.Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin CitiesMAAPT Spring 2011 MeetingSt. Mary’s University, Winona, MN4/30/2011
Supported by NSF DUE #0230830 and DUE #0715615 and by the University of Minnesota
![Page 2: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/2.jpg)
Background Demonstration Preliminary findings
▪ Initial calibration data▪ Currently: Scoring using Rubric for Problem
Solving (Docktor 2009) Methodology for further study
![Page 3: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/3.jpg)
Use coaches in a calculus-based physics course for scientists and engineers Students are asked to volunteer; volunteers
are paid for their participation
Assign students into 2 matched groups Variables for matching: background information, e.g.
HS physics & math level, FCI/CLASS/math pretests
![Page 4: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/4.jpg)
Study #1 (Fall 2010) ~40 students, 1 lecture session of intro
calculus-based class (~20 for each group)
Subset of coaches available – energy (8), momentum (7) – done over 4 weeks (~4 per week)
![Page 5: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/5.jpg)
Retention rate: high in fall (18 of 21) for 15 coaches 2 others completed 12 of 15 All students found them useful
▪ Does this hold for other sections?▪ What will happen with a larger set of coaches?
![Page 6: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/6.jpg)
Student preference for each type Faculty tend to disagree with students
(found type 1 tedious) 1 student initially preferred type 1 but
switched to type 3 after gaining familiarity with physics
Most useful 2nd most useful
Least useful
Type 1 13 3 1
Type 2 0 9 9
Type 3 4 5 8
![Page 7: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/7.jpg)
• Average time range to complete between coaches: between 20 and 40 minutes
Percentage of correct answers seems to correlate with background info
2 sampled “A” students: 80-90% 2 sampled “C” students: 60-70% Students tend to stay on task
Only 2 of 18 had at least one break of more than 5 minutes for a question
![Page 8: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/8.jpg)
Time between questions(one type 1 coach)
Distribution suggests students are taking the tutors seriously Median time = 4.58
s
0
100
200
300
400
500
0 5 10 15 20 25 30F
requ
ency
Time (seconds)
![Page 9: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/9.jpg)
Rubric developed to evaluate student problem solutions Validity, reliability have been tested 2 raters use, discuss with each other until >90%
agreement Five rubric categories (established by
research on expert and novice problem solvers) Useful Description Physics Approach Specific Application of Physics Mathematical Procedures Logical Progression
![Page 10: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/10.jpg)
Study #2 (Spring 2011) 9 students, 1 lecture session of intro calculus-based
class Coaches available for 4 segments: kinematics (3),
dynamics (4), COE (8), COM (7)
Good time to establish baseline of rubric scores for general class Eventual comparison with computer coach
users 2 expert raters – PER researchers 23 students (3 tiers according to pretests);
eventually will expand to ~40 13 problems (8 from quizzes, 5 from final)
![Page 11: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/11.jpg)
![Page 12: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/12.jpg)
![Page 13: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/13.jpg)
![Page 14: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/14.jpg)
- Standard error bars are on the order of +/- 1 to +/- 2 out of 5
![Page 15: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/15.jpg)
Raters: 2 experts/PER researchers
![Page 16: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/16.jpg)
![Page 17: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/17.jpg)
~90 students, 1 lecture session of intro calculus-based class (~45 for each group) Comparison group given equal face time
with problems used in coaches
All coaches available (kinematics, dynamics, energy, momentum, rotational motion) 8 x 5 = 40 total
![Page 18: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/18.jpg)
Short-term questions: Will students use coaches? How will students use them? (keystroke
function) Do they improve students’ problem solving
skills with respect to baseline scores? (rubric scoring of quizzes)
![Page 19: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/19.jpg)
Longer-term questions: Are they adaptable to be used in teaching
other physics courses? Possible software/AI development – refine
beta versions; make it more able to follow student preferences
Can this software be modified by instructors to fit a different problem solving framework?
![Page 20: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/20.jpg)
Summary Initial results seem to have much to say; need to be
expounded upon Currently examining a baseline of exam
performance to compare to future data from computer coach users
Website: http://groups.physics.umn.edu/physed Can look at research, previous talks and publications Try out the coaches! Give us feedback!http://groups.physics.umn.edu/physed/prototypes.html
![Page 21: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/21.jpg)
Treatment and Comparison groups• Treatment group – computer coaching (on Web,
outside of class), 4 problems per week • Comparison group – normal class setting
Data collection• Diagnostic pretests and posttests• Written solutions on quizzes & final exam
• 2×4+5=13 for each student • Problem-solving interviews with students
![Page 22: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/22.jpg)
Can look at individual time on each question for each student
A few questions take some time regardless of student Entering answer into calculator
Can look for patterns in other questions
![Page 23: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/23.jpg)
Chi, Feltovich and Glaser, “Categorization and representation of physics problems by experts and novices,” 1981.
Collins, Brown and Newman, “Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics,” 1989.
Docktor and K. Heller, “Robust assessment instrument for student problem solving,” 2009; also see J. Docktor, dissertation, 2009.
P. Heller and K. Heller, “The Competent Problem Solver: A Strategy for Solving Problems in Physics,” 1995.
P. Heller and Hollabaugh, “Teaching Problem Solving Through Cooperative Grouping. Part 2: Designing Problems and Structuring Groups,” 1992.
Hsu, Heller, Mason, and Xu, Summer AAPT presentation, Portland, OR, 2010.
![Page 24: Andrew Mason, Ph.D. Kenneth Heller, Leon Hsu, Anne Loyle-Langholz, Qing Xu University of Minnesota, Twin Cities MAAPT Spring 2011 Meeting St. Mary’s University,](https://reader034.fdocuments.net/reader034/viewer/2022051401/56649ed05503460f94bddf6a/html5/thumbnails/24.jpg)
Larkin, McDermott, D. Simon and H. Simon, “Expert and Novice Performance in Solving Physics Problems,” 1980.
Newell and Simon, “Human Problem Solving,” 1972.
Palincsar and Brown, “Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities,” 1984.
Polya, “How to Solve It,” 1945; 1957. Polya, “Mathematical Discovery,” 1962. Reif and Scott, “Teaching Scientific Thinking
Skills: Students and Computers Coaching Each Other,” 1999; also see L. Scott dissertation, 2001.