Student Learning andProgram Outcomes

Applicable to Disability Offices?2015 AHEAD Conference

July 15, St. Paul, Minnesota

Presenters• Jean Ashmore

• Past President AHEAD• current AHEAD representative to CAS• Rice University Disability Director Emerita

• Karen Agee• Past President College Reading & Learning Association• Univ. of Northern Iowa, Reading & Learning Coord., Emerita

• Paul Harwell• Texas A&M University

• Tom Thompson• Past AHEAD Board Member• Harper College Disability Director Emeritus

Lunch & Learn Session Topics

• Introducing CAS student learning and development outcomes

• Linking student outcomes and program standards

• Developing systematic practices for assessing program outcomes

• Managing outcome data

• Demonstrating continuous improvement and accreditation requirements

Why is DS self-assessment critically important?

• Gathering formative and summative data (metrics) is a primary means of:- Understanding what is working well

- Zeroing in on how to improve services- Justifying additional resources

• Demonstrating the impact of DS on student life, growth and success- Answering how do these supports impact engagement, persistence and completion

CAS - Counsel for Advancement of Standards in Higher Education

• www.cas.edu

• 47 sets of Standards & Guidelines for DRS and other functions usually within Student Affairs

• Disability Resources & Services standards revised in 2013 (Karen Agee, chair; Jean Ashmore, expert)

• Mission and Program the first of 12 areas of self-assessment

• CAS member associations include AHEAD, CRLA, NACADA, and 38 other professional organizations

Program Outcomes

• Defined strategically to be measurable

• Linked to DS mission, program of services, and resources – all locally developed

• Applicable– Within the DS office, e.g., tech training, testing,

alternative format production– Across campus, e.g., trainings, outreach, lectures

CAS Student Learning & Development Outcomes

• Adopted by CAS in 2008 (in current form):http://www.cas.edu/learningoutcomes• Based on Learning Reconsidered 2: Implementing

a Campus-Wide Focus on the Student Experience (Keeling, 2006)

• Applicable, in part or whole, to all 47 areas of CAS standards

• Expected to be defined locally and to be measured directly and indirectly

SL&D Outcomes

• Student Outcome Domains1. Knowledge acquisition, construction, integration

and application

2. Cognitive complexity

3. Intrapersonal development

4. Interpersonal competence

5. Humanitarianism and civic engagement

6. Practical competence

Dimensions

• Each of the 6 Outcome Domains has Dimensions:– Example – SLO domain of Interpersonal Competence has

dimensions Meaningful Relationships and Interdependence

– Example – SLO domain of Practical Competence has dimensions Pursuing goals, Communicating effectively, Technological competence, Managing personal affairs,

Managing career development, Demonstrating professionalism, Maintaining health & wellness, Living a purposeful & satisfying life

• Not all domains will apply to each campus entity

Domains and Dimensionshttp://www.cas.edu/learningoutcomes

• Domain - Practical CompetenceDimensions

Communicating effectively -Disclosing & articulating about one’s use of

accommodationsTechnical competence - Using technology

effectivelyManaging personal affairs - Balancing personal

life and college demandsManaging career development - Preparing for

transition to employment or grad school – issues of disclosure & using accommodations

How can SLOs be used by DS?

• Step 1: Define one’s DS mission and program of activities

• Step 2: Review SLOs critically, looking for links to program services & activities

• Step 3: Write specific student outcomes that are measurable, directly & indirectly

• Step 4: Collect outcome data regularly, and analyze, report, and adjust program services accordingly

How can SLOs be used by DS?

• Assessing SLOs formalizes what every DS professional does all the time anyway, to answer questions like these:– How well did that session go?– What did those students think our session accomplished?– Are we providing good services?– Do student employees know appropriate strategies?

• Assessing program outcomes formalizes the bigger questions we hope to affect:o Are we retaining our clients at this institution?o Are our clients academically successful here?

Methods of Assessing SLOs?Maki, P. L. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus.

Direct methods• Students demonstrate learning via some form of standardized test focusing on

aspects of student learningExamples: CAAP, CLA, ETS Proficiency Profile (formerly MAPP), GRE subject tests, PRAXIS exams

Indirect methods• Students report perceptions of their learning and the educational environment

that supports that learningExamples: CSEQ, NSSE, NSLLP

Performance-based methods• Students represent learning in response to assignments or projects that are

embedded in their educational experiences• Examples: Successful solution of a math problem not previously understood,

demonstrated writing skill, panic replaced by confidenceBut we DS professionals can create more direct methods for assessing learning than

standardized tests!

Examples of SLOs within DS

Domain 1: Knowledge Dimension: Relating knowledge to daily life

• Students who complete a DS intake workshop will

demonstrate an increase of at least 50% in knowledge

of the responsibilities of students and DS as

demonstrated by scores on pre- and post-workshop

quiz

Examples of SLOs within DS

Domain 1: KnowledgeDimension: Relating knowledge to daily life

• Students participating in at least three individual DS

sessions in the semester will demonstrate 50% improved

knowledge of accommodations appropriate for their

disability or disabilities as measured by their score on the

assessment checklist administered by DS staff

Examples of SLOs within DS

Domain 2: Cognitive complexityDimension: Critical thinking

• Students who work with a DS staff member to identify and acquire accommodations for a disability will, at the end of the session, be able to articulate understanding of the nature of his disability barriers, appropriate accommodations agreed upon, and steps the student has decided to take to acquire those accommodations

Examples of SLOs within DS

Domain 6: Practical Competence Dimension: Demonstrating professionalism

• By the 10th hour of training, student staff will

demonstrate development of professionalism as

measured by the program’s professionalism rubric

created during the 2nd hour of training

Examples of SLOs within DS

Domain 6: Practical Competence Dimensions: Communication Competence Pursuing Goals

• Students in group DS coaching sessions will demonstrate communication and goal-setting skills learned in sessions #2-3 by writing and emailing to the DS session leader and student members of the group a statement of academic goals with no more than 3 errors of spelling, grammar, and punctuation

Examples of SLOs within DS

Domain 6: Practical Competence Dimension: Managing personal affairs

• Students will demonstrate use of a time management system on Day 8 of the 12-session Effective Study Strategies course as measured by instructor analysis of in-class presentations, documented by artifacts [such as calendar, electronic schedule, and completed assignments]

Examples of SLOs within DS

Domain 6: Practical CompetenceDimensions: Pursuing goals

Managing personal affairs

Students who work with a DS staff member on

strategies for self-advocacy will document successful

attempts to achieve agreed-upon goals at the end-

of-semester session.

Why SLOs in DS?

• The purpose of setting and measuring student

learning & development outcomes in DS is

to build excellent programs

• Growth in DS #s = DS expansion = new

resources

• All driven by outcome data

DS Program Outcomes• Concept: Assess and evaluate overall program

of disability resources & services• Foundation: Have a strategic plan for DS with

identified goals and an operational plan with specific short range goals

• Regularly collect input from constituencies:– Students– Staff– Faculty– Institutional partners

Examples of Program Outcomes

• DS example Strategic Goal: “Promote systemic improvement in campus accessibility for facilities, digital systems and campus procedures”– FY Outcome – Build alt format production into the

course development process and train 12 faculty• DS example Strategic Goal: “Provide consultation

and training to faculty, staff and students that results in greater accessibility”– FY Outcome – Create a course substitution process

with support of the faculty, administration and DS.

Examples of Program Outcomes merged with SLOs

• Program Outcome – Interpreter and transcription services will be used appropriately and be cost effective

• SLOs – Practical Competence & Interpersonal Competence -students using interpreters or transcriptionists will articulate the guidelines for these services and follow required notification timelines with 95% compliance

Sample Assessment Schedule for a DS Program

• See Appendix A

Managing Outcome Data

• Develop comprehensive data collection practices

• Develop routine reporting cycles and distribution

• Use data as evidence of program’s budget needs– Key point here – doing assessment and evaluation

takes time and skill. In smaller offices, if a Director is to do this work, other staff are needed to see students

Continuous Improvement & Accreditation

• 360 degree thinking – data leads to action plan = continuous assessment = continuous improvement

• Accreditation agencies expect

– Evidence of effectiveness from both academic and co-curricular programs

– Strategic plans with goals and objectives and program assessment for continuous improvement

– Demonstration by disability offices of value to students

Conclusion: Disability programs

• must think strategically!

• need to develop measurable student learning outcomes!

• must incorporate program assessment into all elements of DS activity on campus as good business practice!

• need to go in this direction before being forced to by those who do not understand our work!

Contact Information

• Karen Agee– karen.agee@uni.edu

• Jean Ashmore– jean@ahead.org

• Paul Harwell– paulh@disability.tamu.edu

• Tom Thompson– tthompso51@gmail.com