Non-binary constraints Dual & Hidden Variable Encodings REFORMULATION.
Assessment as the “Hidden Variable” in Achievement
description
Transcript of Assessment as the “Hidden Variable” in Achievement
![Page 1: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/1.jpg)
Assessment as the
“Hidden Variable” in Achievement
Michael ZeilikDept. of Physics & AstronomyThe University of New Mexico
(NSF DUE 99-85511)
![Page 2: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/2.jpg)
What Works in Higher Ed? What Works in Higher Ed? 20th Century Gain Results20th Century Gain Results
Active Learning, Mastery Learning (SD ≥ 0.5SD ≥ 0.5, PSI, discussion, debates, games, role playing)
Cooperative Learning (SDSD≥ 0.5≥ 0.5, cognitive and affective; century of research, all disciplines)
One-on-One Tutoring (SD = 2 = 2,, with trained tutors)
![Page 3: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/3.jpg)
Lectures:Reinforce memorization; promote illusion of competence; educational seduction (Dr. Fox)!
Still not better than lectures: Computer-based, instructional
television, Web-based
What Does What Does NotNot Work? Work? Standard Models (Standard Models (SDSD < 0.3< 0.3))
Everything Else!Everything Else!
![Page 4: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/4.jpg)
Evidence-Based ReformEvidence-Based ReformActive learning
+ Classroom assessment=
Achievement: Deep conceptual understanding
(“sticky learning”)
![Page 5: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/5.jpg)
What is What is ActiveActive Learning?Learning?
Instructional activities that engage students’ minds (the “sticky” factor)
Interactive lectures, discussion, cooperative learning teams, role playing, peer teaching, minute papers, concept mapping, games, cooperative tests, controversies
Require alternative assessments
![Page 6: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/6.jpg)
What Works in Context: What Works in Context: Physics & Astronomy Physics & Astronomy (PAER)(PAER)
Disciplinary education research, “tools of the trade” for experiments
Empirical robust knowledge base about learning in physics & astronomy
Probe initial state (prior knowledge), final state (learning outcomes)
Measure change: Assessment! Field-tested Learning Assessment Guide (FLAG: www.flaguide.org)
![Page 7: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/7.jpg)
FLAG CATsFLAG CATs Attitudinal Surveys: E.
Seymour, E. Lewis Concept Tests:
A. Ellis Concept Maps:
M. Zeilik Conceptual Diagnostic
Tests: M. Zeilik
Interviews: M. Smith, S. A. Southerland
Performance Assessments: T. Slater
Portfolios: T. Slater
Scoring Rubrics: D. Ebert-May
Student Assessment of Learning Gains: E. Seymour
Weekly Reports: E. Etkina
Mathematical Thinking: M. Swan,
J. Rideway Multiple-Choice Tests: J. Parkes Minute Papers:
M. Zeilik
![Page 8: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/8.jpg)
Conceptual Diagnostic TestsConceptual Diagnostic Tests
Assess prior knowledge of students“Misconceptions” research,
“think aloud” interviews; “natural language”
Measure pre/post conceptual gains (normalized gain, <g>)
Force Concept Inventory (FCI); Astronomy Diagnostic Test: “Astro 101”, nationally normed, reliable & valid
![Page 9: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/9.jpg)
Pre/post: ADT National Project/UNMPre/post: ADT National Project/UNM<g> = (post% - pre%)/(100 - pre)<g> = (post% - pre%)/(100 - pre)
UNM
UNM
Standard errors plotted
![Page 10: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/10.jpg)
From Quarks to CosmosFrom Quarks to Cosmos(Committee on the Physics (Committee on the Physics
of the Universe; NRC)of the Universe; NRC)
11 interface areas (21st century!) UNM Physics 102 (Conceptual Physics) poll, top 5:
1. What is dark matter?2. What is the nature of
dark energy?3. How did the universe begin?4. Are there additional spacetime dimensions?5. What are the masses of neutrinos?
![Page 11: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/11.jpg)
Student Assessment of Student Assessment of Learning Gains (SALG)Learning Gains (SALG)
Probes learning gains students perceive for specific aspects
Avoids performance critiques (“Dr. Fox”) of instructor (not correlated with conceptual gain)
Easily customized (15 min); access through FLAG (hosted by University of Colorado/Boulder)
![Page 12: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/12.jpg)
UNM P102 SALG Spring 2003(Mean = 3.38)
3.68
3.52
3.64
3.71
4.15
3.75
3.99
1.00 2.00 3.00 4.00 5.00
Instructor talk
Discussions
Team work
Activities
Coop quizzes
CT/CRS
Videos
Rating (scale 1 - 5)
![Page 13: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/13.jpg)
P102 UNM Spring 2003 Gender Matters—Sometimes!
1.00 2.00 3.00 4.00 5.00
Math Self-Aptitude
Science Self-Aptitude
Last math
Pretestconfidence
PosttestConfidence
Score (1 = very poor, 5 = very good)
Female Male All
NS!
NS!
p < 0.002
p < 0.0001
p < 0.002
![Page 14: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/14.jpg)
Cooperative Quiz GainsCooperative Quiz Gainsby Genderby Gender
P102 UNM Spring 2003 Quizzes
0.00
0.20
0.40
0.60
0.80
1.00
Practice Quiz #1 Quiz #2 Quiz #3 Mean
<g> Female Effect size (F) <g> Male Effect size (M)
![Page 15: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/15.jpg)
Does it stick? Does it stick? (Coop quizzes/Test)(Coop quizzes/Test)
![Page 16: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/16.jpg)
Grade ExpectationsGrade Expectations
![Page 17: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/17.jpg)
![Page 18: Assessment as the “Hidden Variable” in Achievement](https://reader035.fdocuments.net/reader035/viewer/2022062803/5681473a550346895db478d5/html5/thumbnails/18.jpg)
Assessment “C” LessonsAssessment “C” Lessons(Gender (Gender SometimesSometimes Matters!) Matters!)
Content matters! Concepts matter!
Connections matter! Context matters!
Confidence? Competence?
Assessment probes all & increases achievement by 0.5 SD!