Using&Rubrics&for&GE&and&Major&...
Transcript of Using&Rubrics&for&GE&and&Major&...
Using Rubrics for GE and Major Assessment Repor9ng
Caroline Breitenberger Larry Krissek
2015 Campus-‐Wide Conference on Assessment
Broad learning goal: Par9cipants will learn how to develop and use rubrics for evalua9ng and repor9ng data about course and program learning outcomes.
Specific learning outcomes: Par9cipants will be able to:
Define “rubric” Design a basic rubric Explain how rubrics may be useful for repor9ng assessment results and for program improvement Describe a range of different types of rubrics
Uses of Assessment Results
Outcomes Assessment Results
Course improvement
Program modifica4on
Accredita4on
Inform stakeholders
Improve student learning Feedback to
students
Undergraduate Program Assessment in the College of Arts and Sciences at Ohio State • General Educa9on program – Assigned to ASC Curriculum CommiUee – Handled by ASCC’s Assessment Panel – Historically done by course reports; some department-‐level reports now used
– Reports submiUed when requested; data should be collected regularly
• Major program – Assigned to ASC college-‐level administrators – Reports submiUed annually
Case: Frustrated Department Chair
Now the college has asked me to prepare a report about how our students meet their program learning outcomes. How can I get our faculty to do this?
Our courses obviously address the program learning outcomes! Why can’t we just report student grades?
hUp://z.about.com/d/learningdisabili9es/1/0/x/0/-‐/-‐/frustrated_teacher.jpg
What is a Rubric? • You may have used a rubric for grading, like this one, for
grading a physics lab notebook:
• Or this one, for grading a history paper:
hUp://www.uwstout.edu/soe/profdev/upload/Krubric.pdf; hUp://www.cmu.edu/teaching/designteach/teach/rubrics.html
What is a Rubric? • A rubric is a descrip4on of specific performance standards that can be used to provide consistency in evalua4ng … almost anything
– Student papers – Presenta9ons – Facili9es – Adherence to safety guidelines – etc.
Developing a Rubric for Assessment Repor4ng (GE Educa9on Abroad Rubric as an example)
• Faculty establish expected learning outcomes (ELOs), and outline a rubric aligned to the ELOs – Other stakeholders may provide input (e.g. Office of Interna9onal
Educa9on, Service Learning Ini9a9ve, accredi9ng bodies) Expected Learning Outcomes: 1. Students recognize and describe similari4es, differences, and interconnec4ons between their host country/countries and the U.S. 2. Students func4on effec4vely within their host country/countries. 3. Students ar9culate how their 4me abroad has enriched their academic experience.
Capstone (4) Milestone (3) Milestone (2) Benchmark (1) (ELO1) Knowledge of host country and US: Culture and worldview frameworks (ELO2) Skills for effec4ve func4oning: a) Verbal and
nonverbal communica4on
b) Problem solving
(ELO3) Enrichment of academic experience: a) Knowledge
b) Skills
c) ATtudes/ Perspec4ves
Developing a Rubric for Assessment Repor4ng (GE Educa9on Abroad Rubric as an example)
• Faculty establish expected learning outcomes (ELOs), and outline a rubric aligned to the ELOs
• Define standards for student achievement for each ELO
hUp://www.assessment.uconn.edu/primer/taxonomies1.html Describe, define, iden4fy, recognize…
Compare, discuss, interpret, restate…
Apply, demonstrate, model, solve…
Analyze, categorize, priori4ze, test…
Compile, design, integrate, predict…
Assess, cri4cize, recommend, validate…
Examples of verbs that might be used at various levels of Bloom’s taxonomy:
Developing a Rubric for Assessment Repor4ng (GE Educa9on Abroad Rubric as an example)
• Faculty establish expected learning outcomes (ELOs), and outline a rubric aligned to the ELOs
• Define standards for student achievement for each ELO, and desired level of achievement
Capstone (4) Milestone (3) Milestone (2) Benchmark (1)
(ELO1) Knowledge of host country and US: Culture and worldview frameworks
Ar4culates sophis4cated understanding of differences, similari9es, and interconnec9ons between cultural rules and prac9ces of host country and US.
Demonstrates deeper understanding of differences, similari9es, and interconnec9ons between cultural rules and prac9ces of host country and US.
Describes similari9es and differences and recognizes interconnec9ons between cultural rules and prac9ces of host country and US.
Recognizes similari9es and differences in cultural rules and prac9ces between host country and US.
Capstone (4) Milestone (3) Milestone (2) Benchmark (1)
(ELO2) Skills for effec4ve func4oning: (a) Verbal and nonverbal communica4on (b) Problem solving
Ar4culates a complex understanding of cultural differences in verbal and nonverbal communica9on between host country and US. Is able to skillfully nego4ate a shared understanding based on those differences. Navigates host country and overcomes obstacles with confidence and ingenuity.
Shows higher level understanding of cultural differences in verbal and nonverbal communica9on between host country and US. Begins to nego4ate a shared understanding based on those differences. Navigates host country and overcomes obstacles comfortably.
Shows basic level understanding of cultural differences in verbal and nonverbal communica9on between host country and US. Shows awareness that misunderstandings across cultures can occur. Navigates host country and overcomes obstacles at basic level.
Shows minimal level understanding of cultural differences in verbal and nonverbal communica9on between host country and US. Struggles to navigate host country at basic level and to overcome obstacles.
Developing a Rubric for Assessment Repor4ng (GE Educa9on Abroad Rubric as an example)
• Faculty establish expected learning outcomes (ELOs), and outline a rubric aligned to the ELOs
• Define standards for student achievement for each ELO
• Decide on data to be collected – Direct measures: standardized tests, classroom assignments (pre-‐ and post-‐tests, embedded ques9ons), evaluated body of work (porjolio, capstone project)
– Indirect measures: surveys and interviews, syllabus review, grade review
Suggested prompt for an end-‐of-‐course student reflec9on paper:
Please write a thoughjul four-‐page (double-‐spaced, typed) reflec9on paper that considers the following aspects of your Educa9on Abroad experience: 1. What are the most striking similari9es, differences, and interconnec9ons you have encountered between your host country/countries and the U.S.?
2. How have you developed an ability to func9on effec9vely within your host country/countries? (Think of the ways you communicated verbally and non-‐verbally in your host country and how you overcame challenges.) 3. Overall, how has your Educa9on Abroad experience enriched your academic experience? Use concrete examples.
• required for assessment purposes • the instructor may (or may not) choose to include this assignment as one of the
assignments a student completes for his/her final grade.
GE Educa4on Abroad Category requires an end-‐of-‐course assignment which can take many forms (paper, video, blog, etc.)
Addi4onal Notes Developing a Rubric for Assessment Repor9ng
• Faculty establish expected learning outcomes (ELOs), and outline a rubric aligned to the ELOs (other stakeholders may provide input)
• Define standards for student achievement for each ELO • Decide on data (direct and indirect measures) to be collected
• UCAT consultants can facilitate process • Some kind of standardiza4on among evaluators may be valuable
Report examples
• Educa9on Abroad category • One course is targeted to freshmen, who go to England for one week; other is a capstone course for students pursuing a specific minor, who go to Europe for 3 weeks
• Both courses used the Educa9on Abroad assessment rubric to report assessment results
Capstone (4) Milestone (3) Milestone (2) Benchmark (1) (ELO1)
Students recognize and describe similari4es, differences, and interconnec4ons between their host country/countries and the U.S.
6
19
(ELO2)
Students func4on effec4vely within their host country/countries.
13
12
(ELO3)
Students ar4culate how their 4me abroad has enriched their academic experience.
2
7
16
Freshmen – one week in London (25 students)
Capstone (4) Milestone (3) Milestone (2) Benchmark (1) (ELO1)
Students recognize and describe similari4es, differences, and interconnec4ons between their host country/countries and the U.S.
1
6 11
19 1
1
(ELO2)
Students func4on effec4vely within their host country/countries.
8
13 3
12 3
(ELO3)
Students ar4culate how their 4me abroad has enriched their academic experience.
2 6
7 8
16
Freshmen – one week in London (25 students) Capstone course – three weeks in Europe (14 students)
Instructors’ narra9ves “The second theme was that English memory [of WWII] … concentrated on the local and quo9dian experiences of the war, rather than on high-‐blown ideological or poli9cal elements …”
Proposed course changes to provide “… greater congruence between pre-‐departure course materials and planned experiences in London.” Freshmen – one week in London (25 students) Capstone course – three weeks in Europe (14 students)
Major program assessment -‐ WGSS Women’s Gender and Sexuality Studies learning goals:
1. Student will be able to think cri9cally about women’s issues, gender and sexuality
2. Student will be able to pursue interdisciplinary inquiries about gender and sexuality and to pose ques9ons that cut across disciplinary fields
3. Student will be able to perform women’s, gender and sexuality studies research through developing an argument, organizing data and evidence for that argument, and expressing ideas in wri9ng
• Sample of student papers from capstone course: WGSS 4575, Senior Seminar
• Three faculty members who serve on undergraduate studies commiUee review these papers using rubric
(1) No
(2) Basic
(3) Intermediate
(4) Advanced
Student will be able to think cri4cally about women’s issues, gender, and sexuality
Shows no comprehension of concepts; liUle
knowledge of gender and sexuality
Shows comprehension of concepts related to
gender and sexuality and some applica9on of ideas
Shows comprehension of concepts related to gender and sexuality,
applies and analyzes ideas
Shows comprehension of concepts related to gender and sexuality; applies, analyzes and
evaluates ideas Student will be able to pursue interdisciplinary inquiries about gender and sexuality and to pose ques4ons that cut across disciplinary fields
Does not connect various disciplines or understand gender and sexuality as
interdisciplinary
Recognizes various disciplines with a focus on gender and sexuality but does not make clear
connec9ons
Connects various disciplines with an ar9culated focus on gender and sexuality
Synthesizes various disciplines through an
ar9culated and analy9cal focus on gender and
sexuality Student will be able to perform women’s, gender and sexuality studies research through developing an argument, organizing data, displaying evidence for that argument, and expressing ideas in wri4ng
Does not provide clear arguments or suppor9ve
evidence; does not express ideas clearly
Describes arguments and organizes data but lacks suppor9ve evidence and clear wriUen expression
Demonstrates an ability to develop arguments
supported with evidence and clear wriUen
expression
Ar9culates sophis9cated arguments, analyzes
interconnected evidence and develops clear discussion of ideas
Examples of Repor9ng Rubrics • AAC&U VALUE (Valid Assessment of Learning in
Undergraduate Educa9on) Intercultural Knowledge and Competence rubric
• PULSE (Partnership for Undergraduate Life Sciences Educa9on) Curriculum Alignment rubric
• Ohio State University GE Service Learning assessment rubric
• Rutgers – 21st Century Challenges rubric • Maryland – Scholarship in Prac9ce rubric
In your group, pick one rubric per person, and discuss the rubrics with each other; considering: What sort of program could be assessed using this rubric?
Case: Frustrated Department Chair
• Faculty ar9culate learning outcomes and differen9ate various levels of student achievement of expected learning outcomes
• Communicate shared expecta9ons for students to demonstrate that they achieve expected learning outcomes
• Align student learning ac9vi9es with expected learning outcomes • Establish common metrics which allow flexibility of course design
and student work • Opportunity for faculty, students, and administrators to discuss
progress toward and achievement of expected learning outcomes • Opportunity for informed course and program improvement
hUp://z.about.com/d/learningdisabili9es/1/0/x/0/-‐/-‐/frustrated_teacher.jpg