CAS Standards

Program Review and Division-wide Assessment

With thanks to the generosity ofDeborah A. GarrettVice Chancellor for Student AffairsArkansas State University - Beebe

Carole S. Henry, ACUHO-I CAS LiaisonExecutive Director of Housing & Residence Life Old Dominion University

Doug Lange, CAS SecretaryDean of Student AffairsAi The Art Institute of Tennessee-Nashville

Agenda

Overview of CAS and program review

Preparation for a review

Self Study and Process

Developing an Action Plan

Assessing Student Learning Outcomes

CAS Resources

CAS Overview

Founded in 1979 (ACUHO-I charter member) in response to efforts to establish specialized accreditation for student affairs preparation programs

CAS is a consortium of prof. association members (100,000+) in higher education who work collaboratively to promulgate standards & guidelines

CAS Overview

Focus on quality programs & services for students

Promulgate standards and guidelines for practice and preparation

Promote assessment in educational practice

Promote quality assurance within higher education

WHAT IS PROGRAM REVIEW NOT?

it is not an evaluation of an individual staff member

an evaluation of an individual studenta vehicle for punishment

WHAT IS PROGRAM REVIEW?

Formal, collaborative process for systematic review and evaluation

Tied closely to mission, goals, and specified outcomes of a given department Provides a snapshot of where a department is now Identifies progress toward department priorities &

university strategic planningAssesses strengths and weaknessesMakes recommendations for program

improvement

ELEMENTS OF PROGRAM REVIEW

A. Collect, analyze data/evidence about a program

B. Compare results to some standard for purpose of evaluating services provided: effectiveness efficiency impact quality

C. Tied to continuous improvement

CAS Self Assessment Components

mission program leadership organization and management human resources financial resources facilities, technology, and equipment legal responsibilities equity and access campus and external relations diversity ethics assessment and evaluation

CAS Self Assessment Guides Purpose and Organization

Outcomes relevant to student learning and development that must be identified and assessed: knowledge acquisition cognitive complexity intrapersonal development interpersonal competence humanitarianism and civic engagement’ practical competence

Key Questions for CAS Components

Mission How does the unit uniquely contribute to and advance the

Division’s goals, objectives, & strategic initiatives?

Program What are the Unit’s core programs and services? Describe any key collaborative relationships (within and

beyond Student Affairs) in which the department is involved. Please explain how these partnerships:

What lines of communication exist for students’ input to be included in the mission/purpose of the dept.?

 

Key Questions for CAS Components

Leadership What leadership practices are used to establish and

promote the unit’s values, short and long term directions, and performance expectations?

How does unit leadership promote a work culture that encourages staff empowerment, innovation, risk taking to effectively meet needs of students?

Philosophy of decision-making? How does planning take place?

 

Key Questions continued

Organization and Management Explain the unit’s organizational structure, including

reporting relationships, distribution of responsibilities/authority, lines of communication, and use of management and/or work teams.

What criterion does the unit use to accurately determine the correct number of personnel that are needed to effectively fulfill the mission and meet the needs of those it serves?

Key Questions continued

Human Resources What strategies are used to recruit, hire, and retain staff? How are staff performance expectations established,

effectively communicated, and understood? How is staff performance evaluated? How is staff being trained and developed?

Financial Resources Please provide a financial overview of the unit, What is the funding model for the unit? What resource

changes have occurred over the past five years and what are the anticipated changes for the future?

How are budget allocation/reallocation decisions made in the unit?

Key Questions continued

Facilities, Technology, and Equipment How effectively do current facilities/space and

equipment support the work of the unit? How has technology been integrated into the program,

service, and operating functions of the unit? How has the unit kept pace with the development of hardware, software, maintenance, and training support? What are the projected technology needs for the future?

How effectively do the unit’s current facilities, space, and equipment support its operations?

Key Questions continued

Legal Responsibilities How is staff trained to ensure that they are knowledgeable

about the laws and regulations, and professional ethics that apply to their respective job responsibilities?

What processes are in place for managing risks

Equity and Access How is non-discriminatory, fair, and equitable treatment of

staff and constituents ensured?

Campus and External Relationships What collaborative relationships exist with campus

colleagues?

What short and long-term goals have been established to strengthen these types of relationships?

Key Questions continued

Diversity How does the unit advance the University and Division’s

diversity agenda specific to: program/service offerings recruiting, hiring, retaining, and promoting staff student learning and development outreach and engagement activities campus and work environments

Ethics What is the unit’s policy/strategy for managing student/client

and staff member confidentiality issues? How are they informed of these practices?

Key Questions continued

Assessment and Evaluation Provide a description of the unit’s assessment and evaluation

activities. How are student needs assessed and considered in the

development and delivery of programs and services? How is benchmark data used to stay current with peers?

Basic Steps:Self-Assessment Process

A. Establish the process, identify a team and train them.

B. Understand CAS Standards and Guidelines.

C. Determine data sources, compile and review evidence. Gather & analyze pertinent qualitative and

quantitative data

D. Judge performance.

E. Complete the assessment process.

Step B: Understand CAS Standards and Guidelines

CAS functional area standards and guidelines – basic statements that should be achievable by a program

STANDARD (Bold type; Verbs MUST or SHALL)

Sample: HLRP staff members must provide a variety of educational opportunities that promote academic success and the achievement of learning and student development outcomes.

GUIDELINE (Regular type; Verbs SHOULD or MAY)

Sample: HLRP should provide an environment that assists residents to remain in good academic standing, earn higher GPA’s, and be retained.

Step C: Compile and ReviewDocumentary Evidence for Your Area

Recruitment and marketing materials Participation policies and procedures Results reports; participant evaluation summaries

Program documents brochures, catalogs

institutional administrative documents organization charts financial statements

Research, assessment, evaluation data needs assessments, follow-up studies assessment outcomes reports

Staff activity reports Student activity reports

Step D: Judging Performance

Assessment criterion measures specific to your part of the review

ND 1 2 3 4 NRNot Done Not met Minimally Well Fully

Met Met Met

Using this scale, consider each criterion statement and decide the extent to which each criterion measure has been met by the program or service

Step D: Judging Performance continued

Each criterion measure focuses on a particular aspect of the standard, allowing raters to express more detailed and specific judgments

In most instances, there are multiple criteria for each standard

Judging Performance Exercise

Page 31: Review must statements

Diversity Criterion Measures

Rate 11.1, 11.2, 11.3, 11.4 independently

Review in group.

Formulating an Action Plan

1. Answer overview questions

2. Identify areas of program strength (Work form A)

3. Identify areas of program weakness (Work form A)

4. Describe practices requiring follow up (Work form A)

5. Summarize actions required for the program to meet the standards (Work form B)

6. Summarize program enhancement activities (Work form C)

7. Write program action plan

CAS: Work Form AAssessment, Ratings, and Significant Items

CASWork Form A

Assessment, Ratings, and Significant ItemsINSTRUCTIONS:This work form should be completed following individual ratings of the participants. For each of the 13 Parts, identify (circle) the criterion measure item number(s) in the column labeled for which there is a substantial rating discrepancy (two or more ratings apart). Items not circled should reflect consensus among judges that practice in that area is satisfactory. Items where judgment variance occurs need to be discussed thoroughly by team members. Follow this action by determining which practices (criterion measures) can be designated as “excellent” or “unsatisfactory” and record them in the Step One column. In Step Two, list the items requiring follow-up action including any criterion measure rated as being unsatisfactory by any reviewer.

Step One

Part Items Excellent Unsatisfactory

1. Mission 1.1a1.4

1.1b1.5

1.1c1.6

1.2 1.3

2. Program 2.1 2.4.3 2.4.8 2.4.132.6 2.8d2.11

2.2 2.4.4 2.4.9 2.4.142.7 2.8e2.12

2.32.4.52.4.102.4.152.8a2.8f2.13

2.4.12.4.62.4.112.4.162.8b2.9

2.4.22.4.72.4.122.52.8c2.10

CAS: Work Form A continued

Step Two: List item number(s) for each Part determined to merit follow-up and describe the practice weaknesses that require attention

1.

2.

3.

4.

5.

CAS Work Form B: Follow Up Actions

INSTRUCTIONS:

The purpose of this work form is to begin the planning for action to be taken on practices judged to merit follow-up (See Step 1, Work Form A). In Step Three, transfer short descriptions of the practices requiring follow-up and detail these items using the table format provided.

Step Three: Describe the current practice that requires change and actions to initiate the change

Practice Description

Corrective Action Sought

Task Assigned To Timeline Due Dates

CAS Work Form C:Summary Action Plan

Step Four: This form concludes the self-assessment process and calls for action to be taken as a consequence of study results. Write a brief action plan statement in the spaces below for each Part in which action is required.

Part 1: Mission

Part 2: Program

Part 3: Leadership

Part 4: Human Resources

Part 5: Ethics

Student Outcomes Assessment

What is the effect of our work on students?

How are they different as a result of interacting with our programs and services?

How do we demonstrate their learning? How do we know? How can we show that the outcome was due to

our programs, etc.?

Plan for Assessing Student Learning Outcomes

Review/revise missionIdentify major services, programs, activitiesSpecify desired student outcomesIdentify information needed to show student

learningIdentify existing and new data neededDevelop process for measuring outcomesIdentify ways to use findings to

change/improve

CAS Resources

www.cas.edu

CAS Statement of Shared Ethical Principles

CAS Characteristics of Individual Excellence

CAS Professional Standards, 2009 edition Prologue & Context with “How to” information

Self-Assessment Guides SAG’s- tools to help conduct self-assessment activities

Frameworks for Assessing Learning and Development Outcomes (FALDO’s) –contain information for measuring learning outcomes

CD of Self Assessment Guides includes all 38 functional area SAG’s and Standards; also includes an e-learning course on conducting self assessment & a power point presentation that can be modified by user.

Program Review Timeline

Self Study (January – April)

Program review committees studentsfacultystaff

Determine adequacy of evidence Collect more data? Conduct focus group?

Timeline (continued)

Report Findings Report of findings and recommendations

w/emphasis on how unit can improve and contribute more fully.

Timeline (continued)

External Review/Site Visit (October) Two or three external reviewers Write preliminary report based on Self Study Campus visit External Review Team Report

Timeline (continued)

Writing the Unit Program Review Report (November/December)

Prepare final report (generally same individuals as self study team)

Share with unit director (ifs/he is not on team) to correct any factual errors

Report to VP, Division Program Review Team, Unit Director, Supervisor

Timeline (continued)

Develop the Unit Action Plan (November – December)

Proposed actions Implementation strategies Person(s) responsible Appropriate timelines and identify resources Include in annual goal-setting