Education Abroad and Formative Outcomes Assessment: What Professors Think They're Teaching and What...
Transcript of Education Abroad and Formative Outcomes Assessment: What Professors Think They're Teaching and What...
Formative Assessment: What STEM professors think they’re teaching & what
students think they’re learning. Katherine Yngve, Purdue University
Elizabeth Brewer, Beloit College
Types of Assessment
Summative Purpose is to gather feedback which will be helpful to both student & instructor in the learning process.
• Learning Style Inventories • Quiz • Homework • Reflection Essay or Activity • Interview/Observation w/ Feedback • Portfolio • Rubric
Formative
Purpose is to measure the level of proficiency which has been gained due to an instructional unit or intervention.
• Final Exam
• Pre/Post Survey
• Capstone Project
• Term Paper; Senior Recital, etc.
• Portfolio
• Rubric
Session Overview
Beloit College What the Curriculum & Students Say
• Life-long Learning – Study Abroad Learning Goals
• Sequential, Iterative Assessment
• Essay Prompts
• What the Research Shows
• Hands-On Activity: Assessing Essays
Purdue University What Profs Think They’re Teaching
• Embedded Learning & the UG Core Curriculum
• Intercultural Learning Rubric
• What the Research Shows
• Hands-On Activity: Assessment Tools to Move the Learning Needle
Debriefs & Discussion as time allows!
Beloit Curriculum: Life-Long Learning Skills
Experiences outside the classroom that allow students to:
“connect, adapt, and apply knowledge and skills gained in one setting to new contexts”
Preparation for Life-Long Learning Beloit First Year Seminars
Encounter and Exploration – campus, mission, curriculum
Agency – putting students in charge of their path through college
Higher-Order Communication Skills – writing and speaking
Social Identities – self, others, communities
Reflection
Life-Long Learning: Beloit Study Abroad Learning Goals
new perspectives on studies
intercultural competencies and communication skills
engage with situations and questions that challenge assumptions and values,
articulate cultural experience
learn about and from host environments
First-Year seminar
Study Abroad
Post-SA Essays
Study Abroad Plan
1. Students Develop Agency & Communication Skills
2. Students Identify Learning Goals &
Connections to Studies
3. Students Execute & Adjust Plans 4.
Students Reflect: What, Why, How?
So What? What Next?
Stages in
Formative Assessment
Rubrics used by
• Applicants: To prepare a study abroad plan
• Faculty: To evaluate the plans
• Faculty/staff/student teams: To assess the development of life-long learning skills as evidenced in post-study abroad reflective essays
Study Abroad Application Prompt 1:
Learning Goals
A. State three to five things you hope to learn abroad.
B. How will your choice of university/program and location help you achieve your goals?
C. How might differences in educational philosophy and/or delivery affect your ability to achieve your goals?
Study Abroad Application Prompt 2:
Connection to Studies
Connection to Studies
How will the study abroad you propose build on your education to date as well as connect to your education when you return to campus?
Note: There are four prompts. The others focus on preparation and social identities
What Research Shows
• Life-long learning skills advance more during study abroad than intercultural competency
Analysis of post-study abroad reflective essays
• Asking SA applicants to identify their own learning goals and make connections to studies has helped them make stronger connections between study abroad and their studies on campus
Analysis of presentations on study abroad experiences in an annual international symposium
• Students are better able to articulate their learning if given chances post-study abroad for reflection and dialogue
Analysis of reflective writing
Impacts of Formative Assessment
International Symposium content
Connections to Studies
Cultural / Intercultural Learning
Questioning own Assumptions / Values
Host Country / Site
2014 96% 60% 36% 92%
2002 32% 68% 4% 96%
Analysis of 387 abstracts
Impacts of Formative Assessment
International Symposium content sources
Field / Independent Research or Course Project
Informal Observation
Internship
2014 68% 28% 8%
2002 44% 48% 8%
Group-work Task A 1. Read and score application essays using the rubric
labeled “original.” What feedback would you give the student?
2. Compare the original and new prompts and rubrics. Do the changes make sense? Are the new prompts clearer?
3. Brainstorm ideas for how your organization/institution could help students exercise agency around study abroad and articulate their plans.
Purdue Core Curriculum (2012 - 2015)
Assessing Faculty-Led Programs
What will your short-term program teach?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
AG (11) CLA (18) EDU (8) ENG (10) HHS (5) MGMT (5) TECH (12)
World
Comm
Curious
Open
Self-Aware
Empathy
What will your short-term program teach?
College of Agriculture Teamwork Civic Engagement
College of Liberal Arts Comfort with Difference Knowledge of Intercultural
Theory Social Justice Orientation
College of Education Social Justice Orientation Comfort with Difference
College of Engineering Teamwork Knowledge of Intercultural
Theory Civic Engagement
College of Health & Human Sciences Teamwork
College of Technology Teamwork
College of Management Teamwork
Selected Formative Assessment Tools
MAXSA Handbook
• Language (Learning) Strategy Survey
• Culture (Learning) Strategy Survey
• Culture Mapping Survey
• Stress Factors Inventory
• You as a Culturally Diverse Person
Other sources
• Kolb Learning Style Inventory
• Cross-Cultural Adaptability Index
• Intercultural Effectiveness Scale
• Intercultural Conflict Style Inventory
• Hofstede Centre Culture Compass
• My Cultural Awareness Profile (for teachers; by NAFSA)
• Rubin’s Behavioral Observation Rubric
• Foreign Language Classroom Anxiety Scale
• Liebowitz’ Social Anxiety Scale
• Multi-factor Leadership Questionnaire
• Scale of Ethno-cultural Empathy
Group-Work Task B
1) Select a program description & review the 6
elements of both AAC&U rubrics in packet.
2) Identify 2-3 rubric items that this particular course
addresses (e.g. “Curiosity” or “Action & Reflection”)
3) Review the formative assessment tools on offer
4) Jointly select at least two assessment tools that
your group believes will move the students forward on at least one objective
5) Be prepared to defend your selection of the tools in
open forum debrief!