Measuring a Culture of Evidence: Social Justice and Inclusive Assessment Practices

Dr. Melissa Brown, OASI Dr. Janelle Coleman, Teaching & Learning Innovation

Presenters

Dr. Melissa Brown,Office of Assessment & Strategic Initiatives

Dr. Janelle Coleman, Teaching & Learning Innovation

By the end of this session, participants will...

• discuss the value of assessment as it pertains to the ongoing improvement of programs and services.

• articulate the impact that a social justice lens and inclusivity can have on the interpretation and utilization of assessment data.

• practice implementing the last four phases of the assessment cycle.

Assessment

“Assessment is any effort to gather, analyze, and interpret evidence which describes the institutional, departmental, divisional, or agency effectiveness” (Upcraft & Schuh, 1996, p. 18)

What Do We Mean By "Social Justice"*?• Social justice (SJ) is a paradigm that:

• advocates for spaces in which everyone is physically and psychologically safe and secure

• is inclusive and affirming of human agency and capacity for working collaboratively with others

• The goals of SJ are:• to interrupt and change oppressive patterns and

behaviors• to ensure that resources are equitably distributed.

(*Definition provided by Social Justice and Equity Committee, Long Beach State University, 2019).

What is "Cultural Humility"?• Perspective by which we

understand the process of developing cultural competency.

• "ability to maintain an interpersonal stance that is other-oriented (or open to the other) in relation to aspects of cultural identity that are most important to the [person]" (Hook, Davis, Owen, Worthington and Utsey, 2013).

Assumptions Surrounding Cultural Humility and Social Justice

• Depth of culture.• Existence of systemic

barriers / inequities.• Privilege due to systemic

inequities.• Recognition that

everyone cannot be viewed through the same lens. This Photo by Unknown author is licensed under CC BY-NC.

Test Your Knowledge!

Please use your smartphone with any browser and enter:

srs.campuslabs.com

Connect ID: 16938

Please use your preferred fake name:)

.

.

.

.

.

Assessment CycleWrite/Revisit

Outcomes

Identify Evidence(Surveys, KPIs etc.)

CollectEvidence

Analyze/InterpretEvidence

Use Datafor

Improvement

Types of Data Used for Assessment

• Large-scale campus surveys (e.g., campus climate, general education data, NSSE)

• Embedded Coursework• Portfolios• Student Life surveys• External consulting surveys• Program/course surveys• Focus groups• Interviews• Guided reflections• 2-minute papers

Transparency & Sharing Data

1) How and when do we use the data?2) Is the data enough for change?3) How do we share with the training

team without criticizing the team/work?

Focus of this Activity• The activity will focus on

coding student responses to a student engagement survey.

• We will walk through the last four stages of the assessment cycle.

• We will draft a plan (e.g., new set of goals) based on data.

Activity Background

Data Source•2018 StudentEngagement Survey

Survey Question:• What can UT do

to enhance your sense of belonging on campus?

Practice Session:Coding

Qualitative Data

Instructions

•Code the short answer question provided using the list of codes on the handout (Social Justice)

•Work individually

•Use the numbers as codes

•You may use more than 1 code per data point

We want all students to know that they matter and belong on our campus. What can UT do

to enhance your sense of belonging on campus?

1.Discourse2.Equality3.Equity4.Democracy5.Justice6.Integrity7.Participation8.Inclusive

9. Multiculturalism10. Diversity11. Individuality12. Respect13. Social good14. Responsibility to society15. Representation16. Belonging

Group Reflection• Are there observations about the

data beyond the codes?

• What are the “blind spots” or gaps as indicated by the data?

• What are one to two things that can be done to address those blind spots?

Activity: Action Plan

• In pairs: write 2-3 observations about what students said they wanted from their engagement experiences at UTK.

• Write 2-3 ideas for how you might address each need.

• Jot down a new outcome and a possible assessment to measure progress.

Debrief

• Share outs!

• One person can’t be responsible for analyzing the results

• Need to have a conversation to see multiple voices/support

• Reach out across campus too

Lessons Learned

• Assessment is not a destination but a process.

• Good assessment cannot be done in a silo.

• We must be cognizant of the lens (bias) we approach the data with.

Thank you!

Janelle Coleman633 Greve Hall

jcolema1@utk.edu

Melissa Brown413 Student

Services Bldg.mabrown@utk.edu