Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger...
-
Upload
charleen-hopkins -
Category
Documents
-
view
214 -
download
0
Transcript of Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger...
![Page 1: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/1.jpg)
Experimental Assessment of Learning Outcomes Using a Text Book and Interactive
Package
Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian Schoner, Kari Stachuk and Stacey
Vojtek
University of [email protected]
![Page 2: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/2.jpg)
Sample Course evaluation
“I open the text and my mind goes blank”
![Page 3: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/3.jpg)
What we’ll address:
• An instructional bottleneck/anchor
• A distributed learning approach
• Evaluating alternative approaches
![Page 4: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/4.jpg)
The Instructional Environment
• University of Waterloo: – Professional and technical focus
• Faculty of Environmental Studies– Environment and Business– Environment and Resource Studies– Geography– School of Planning
![Page 5: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/5.jpg)
Course Context
• “Field Ecology”
• 0.5 Credits, 2nd year, compulsory
• Non-scientists
• Applied focus
• 2 h class + 3 h lab/field trip
![Page 6: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/6.jpg)
Technical Material as Instructional bottleneck
• Math-happy and Math-phobic students
• Text book “dry”
• Poor learning outcomes
![Page 7: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/7.jpg)
Life Tables:
• Accounting system for population dynamics
• Understanding comes only by doing
![Page 8: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/8.jpg)
Solution:A Life Tables Interactive Package
• Engage the learner
• Non-threatening “walk through” computation
• Oblige student to “do” life tables
• Emphasise applications
• Link to Field Work Videos
![Page 9: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/9.jpg)
The Life Tables Interactive Package (IP)
![Page 10: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/10.jpg)
Instructional Presentation
• Sampler in class
• Material on server
• Used in 5 course offerings
![Page 11: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/11.jpg)
What we don’t know:
• IP is fun, but do they learn more?
• Is text plus IP better?
• Retention of text vs IP?
![Page 12: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/12.jpg)
Big Picture
• Is CD in back of a text effective?
• Supplement distributed learning with text?
• Abandon text? (Publishing implications!)
![Page 13: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/13.jpg)
What others say:
• Students learn more with computers, faster, liked work more, with better attitudes (Kulic 1994).
• However:– “We need to go beyond generic conclusions
and make statements about the effectiveness of specific types of computer based instruction”
![Page 14: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/14.jpg)
What others say:
• Striking need for evaluation (Baker and O’Neil 1994,Anon 1998)
• Little evaluation of durability of learning effect with technologies (Lesgold 1995)
• Exceptions:– Hiltz (1990)
– Active Learning 8 (1998)
![Page 15: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/15.jpg)
Research Questions
• Different outcomes using IP and text+IP?
• Different retention for IP and text+IP?
• Does text/IP order influence:
– outcomes?– retention?
![Page 16: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/16.jpg)
Experimental design
Treatment Baseline test
1st study
Period
Test IP
vs. Text
2nd Study
Period
Test order of IP and Text
Test retention at 3 mo
Text-Text Text Text-IP
Text IP
IP-Text Text IP-IP
IP IP
![Page 17: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/17.jpg)
Experimental design
Treatment Baseline test
1st study
Period
Test IP
vs. Text
2nd Study
Period
Test order of IP and Text
Test retention at 3 mo
Text-Text Text Text-IP
Text IP
IP-Text Text IP-IP
IP IP
![Page 18: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/18.jpg)
Experimental design
Treatment Baseline test
1st study
Period
Test IP
vs. Text
2nd Study
Period
Test order of IP and Text
Test retention at 3 mo
Text-Text Text Text-IP
Text IP
IP-Text Text IP-IP
IP IP
![Page 19: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/19.jpg)
Experimental design
Treatment Baseline test
1st study
Period
Test IP
vs. Text
2nd Study
Period
Test order of IP and Text
Test retention at 3 mo
Text-Text Text Text-IP
Text IP
IP-Text Text IP-IP
IP IP
![Page 20: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/20.jpg)
Experimental design
Treatment Baseline test
1st study
Period
Test IP
vs. Text
2nd Study
Period
Test order of IP and Text
Test retention at 3 mo
Text-Text Text Text-IP
Text IP
IP-Text Text IP-IP
IP IP
![Page 21: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/21.jpg)
Experimental design
Treatment Baseline test
1st study
Period
Test IP
vs. Text
2nd Study
Period
Test order of IP and Text
Test retention at 3 mo
Text-Text Text Text-IP
Text IP
IP-Text Text IP-IP
IP IP
![Page 22: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/22.jpg)
Evaluation of Learning
• Before, 30 multiple choice
• At 45 min. 10 of above
• At 90 min. 10 of above
• At 3 months 10 of above
• 48 subjects in 4 treatments (Have 40)
![Page 23: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/23.jpg)
Results
(Tentative)
![Page 24: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/24.jpg)
Statistical Evaluation
• MANOVA on scores
• MANOVA on change in scores
![Page 25: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/25.jpg)
Mean scores
-2
0
2
4
6
8
10
Pre Study After 1period
After 2periods
After 3Months
Book/CD Book/Book CD/Book CD/CD
![Page 26: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/26.jpg)
Change in Mean Score from Pre-Study
-2
0
2
4
6
8
After 1 period After 2 periods After 3 Months
Book/Book Book/CD CD/Book CD/CD
Same except book/IP
At 3 mo all same except Ip/Book
![Page 27: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/27.jpg)
Technical Conclusions
• All approaches work (somewhat!)
• Book/IP is less effective?
• Decay in performance is profound!
• At 3-month retention not radically different
![Page 28: Experimental Assessment of Learning Outcomes Using a Text Book and Interactive Package Roger Suffling Kevin Harrigan, Pascale Proulx, Diane Salter, Vivian.](https://reader035.fdocuments.net/reader035/viewer/2022070400/56649ead5503460f94bb40ca/html5/thumbnails/28.jpg)
General Conclusion
All methods tried work about the same.
Value of interactive approach must lie in long-term motivation, not in the individual knowledge.
The long term effects on student progress need evaluation