Practical Strategies for Assessing Student Learning in Departments and Programs: A...
-
date post
20-Dec-2015 -
Category
Documents
-
view
221 -
download
0
Transcript of Practical Strategies for Assessing Student Learning in Departments and Programs: A...
![Page 1: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/1.jpg)
Practical Strategiesfor Assessing Student Learningin Departments and Programs:
A Utilization-Focused Approach
Jo BeldSt. Olaf College
![Page 2: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/2.jpg)
Guiding questions
• Who are the users of the evidence?
• How might the evidence be used?
• How will you gather the evidence?
• How will you animate the evidence?
![Page 3: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/3.jpg)
Theoretical perspectives
Utilization-focused assessment
(Patton, 2008):
Focus on intended uses
by intended users
![Page 4: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/4.jpg)
Theoretical perspectives
Backward design
(Wiggins & McTighe, 2005):
“Beginning with the end in mind”
![Page 5: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/5.jpg)
Theoretical perspectives
Traditional instructional design:
Choose texts
Develop classroom activities
Make up tests
![Page 6: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/6.jpg)
Theoretical perspectives
Backward instructional design:
Identify intended learning outcomes
Determine appropriate evidence
Plan instruction and practice
![Page 7: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/7.jpg)
Theoretical perspectives
Traditional assessment design:
Choose an assessment instrument
Gather and summarize evidence
Send a report to someone
![Page 8: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/8.jpg)
Theoretical perspectives
Backward assessment design:
Identify intended users and uses
Define and locate the learning
Choose assessment instrument
![Page 9: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/9.jpg)
Who are the users?
Faculty roles, commitments, and disciplinary identities offer both incentives and disincentives to engage assessment
![Page 10: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/10.jpg)
What are the uses?
What kind of outcome do you want to investigate…
–Knowledge (“Understanding of…”)
–Proficiencies (“The ability to…”)
–Practices (“The habit of…”)
–Values or attitudes (“A concern for…”)
![Page 11: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/11.jpg)
What are the uses?…and why?
• Affirming current practices• Tweaking the content of key courses• Extending a specific pedagogy• Enhancing “scaffolding”• Piloting innovations• Supporting grant applications• Setting future assessment agendas
![Page 12: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/12.jpg)
How will you gather evidence?
Assessment is like any other kind of investigation; use strategies that fit the questions you are trying to answer
![Page 13: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/13.jpg)
How will you gather evidence?
“Direct” Assessment”
“Direct” Assessment
Evidence of what students actually know, can do, or care about
“Indirect”Assessment
Evidence of learning-relatedexperiences or
perceptions
![Page 14: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/14.jpg)
How will you gather evidence?
Common direct assessment “artifacts”
Papers, essays, abstracts
Presentations and posters
Oral or written examination items
Analytic journals
Responses to survey or interview questions that ask for examples of knowledge, practice, or value
![Page 15: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/15.jpg)
How will you gather evidence?
Common indirect assessment “artifacts”
Course-taking patterns or transcript analysis
Responses to survey or interview questions about experiences, perceptions, self-reported progress, or impact of program experiences
Reflective journals
![Page 16: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/16.jpg)
How will you gather evidence?
But wait!! Aren’t we observing student work all the time anyway? What’s the difference between grading and assessment?
![Page 17: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/17.jpg)
How will you gather evidence?
Gradingsummarizes
many outcomes
for
one student
Assessmentsummarizes
one outcome
for
many students
![Page 18: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/18.jpg)
How will you gather evidence?
The purpose of an assessment instrument is to provide systematic, summarized information about the
extent to which a group of students has realized one or more intended learning
outcomes
![Page 19: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/19.jpg)
How will you gather evidence?
Options to consider:
Use an instrument developed by someone else
Adapt an existing instrument
Add to something you’re already doing
Connect to institutional-level evidence
Invent something new
![Page 20: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/20.jpg)
How will you gather evidence?
Where possible, pair indirect observations of processes and perceptions with direct observations of outcomes (pp. 16-17).
![Page 21: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/21.jpg)
How will you gather evidence?
The dual goal of sampling:
Representativenessand
Manageability
![Page 22: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/22.jpg)
How will you gather evidence?
Examples involving comprehensive sampling:
Survey of all senior majors
Application of rubric to all research abstracts in all seminars
Application of rubric to all work submitted for senior art show
![Page 23: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/23.jpg)
How will you gather evidence?
Examples involving selective sampling:
Application of rubric to randomly-selected subset of final papers in capstone course
Pre/post administration of locally-developed quiz in required sophomore methods course
End-of-course survey in one introductory and one senior-level course
Aggregation of results on selected items in an evaluation form for student work
![Page 24: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/24.jpg)
How will you animate the evidence?
Evidence never speaks for itself; be intentional in summarizing, interpreting, and responding to the findings
![Page 25: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/25.jpg)
How will you animate the evidence?
Plan intentionally for summarizing and interpreting findings:
• Delegate and distribute• Enlist help from staff and students• Use technology to save time• Balance individual and collective work• Dedicate time for discussion
![Page 26: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/26.jpg)
How will you animate the evidence?
Plan intentionally for responding to the findings:
• Borrow strategies from past successes in collective departmental action
• Focus reporting on planned actions, not on the evidence itself
• Weight Watchers trumps The Biggest Loser
• Dedicate resources for action
![Page 27: Practical Strategies for Assessing Student Learning in Departments and Programs: A Utilization-Focused Approach Jo Beld St. Olaf College.](https://reader031.fdocuments.net/reader031/viewer/2022012916/56649d495503460f94a25075/html5/thumbnails/27.jpg)
A final thought….
Less really is more!