Assessment 101

An Introduction for Administrative and Support Units

Presented by Horace D. (“H.D.”) Stearman, Ph.D.

Director of Institutional Effectiveness

August 14, 2008

2

Module One: Overview

What Is Assessment?

Why Do We Do Assessment?

What The Accreditors Say

Assessment Misconceptions

The Assessment Cycle Step 1: Mission Statement and Core Functions/Services

Step 2: Expected Outcomes

Step 3: Measures

Step 4: Target Levels

Step 5: Findings

Step 6: Action Plans

Potential Assessment Pitfalls

Relation of Assessment to Planning

3

What Is Assessment?

Assessment is a systematic, ongoing process of collecting, reviewing, and using data about an institution’s academic, administrative, and support units to improve the effectiveness of these units.

We carry out assessment of administrative and support units by Creating a mission statement for our unit that briefly describes our purpose;

Identifying the core functions and services we provide that enable us to meet our mission;

Determining expected outcomes to describe the results we expect to see in the way we perform our core functions and services;

Selecting measures that will quantify the achievement of our expected outcomes;

Setting target levels of performance on the expected outcomes;

Interpreting the levels of performance that we attain on our expected outcomes to judge how well our actual performance matches our target levels; and

Using the results to document, explain, and improve the effectiveness of our unit in accomplishing our mission.

4

Why Do We Do Assessment?

To improve what we do To inform decision-making, budgeting, and planning To focus our efforts To show students, alumni, employers, and others that we’re doing a good job To satisfy the expectations of our accreditors

5

What The Accreditors Say

The institution engages in ongoing planning to achieve its missions and goals. It also evaluates how well, and in what ways, it is accomplishing its mission and goals and uses the results for broad-based, continuous planning and evaluation. Through its planning process, the institution asks questions, seeks answers, analyzes itself, and revises its goals, policies, procedures, and resource allocation.

Northwest Commission on Colleges and Universities

“Standard 1.B – Planning and Effectiveness”

6

Assessment Misconceptions

It’s A Passing Fad

It’s Not My Job

You Can’t Assess What I Do

I Don’t Have Time For Assessment

I Don’t Have The Expertise To Do Assessment

Assessment Might Reveal Shortcomings Or Failures

7

The Assessment Cycle

Outcomes

Plans for Improvement Assessment of Outcomes

8

Step 1: Mission Statement and Core Functions/Services Write a brief mission statement that describes your unit’s purpose. Identify the four to seven core functions or services that enable your unit to meet its

mission. Taken together, the mission statement and the list of core functions or services

answer the questions “What do we do and how do we support the RMUoHP mission?”

9

Some examples of mission statements and core functions

The Department of Career Services counsels students and alumni as they explore career directions and equips them with the information they need to make good decisions.

Our core functions include1. Provide resume-writing workshops, mock interviews, and related training to students as

they prepare for job searches

2. Coordinate on-campus visits by corporate recruiters and on-campus job fairs

3. Maintain a database of job postings and a library of employment-related books and DVDs for student use

4. Counsel students and alumni in best practices for job searches

The Office of the Registrar maintains and processes student academic

records accurately and efficiently.

Our core functions include1. Maintain student satisfaction with registration functions

2. Process degree audits for graduating students

3. Fill transcript requests

10

Step 2: Expected Outcomes

Write three to five expected outcomes that describe the results you expect to see in the way you perform your core functions and/or deliver your key services over the coming year.

Expected Outcomes should be related to something under the control of your unit

worded in terms of what your unit will accomplish or what your clients will think, know, or be able to do following the provision of services (“X will happen”)

measurable, meaningful, manageable

11

Some examples of expected outcomes

The Office of Institutional Research will collect and disseminate campus data for internal and external reporting purposes.

The Office of the Registrar will provide resources to enable students to use on-line registration successfully.

The Office of Career Services will engage students in development early and often through various outreach efforts.

The Office of Information Technology Support will provide its customers with prompt and effective assistance in resolving their technical problems related to systems, networks, and desktop applications.

12

Step 3: Measures

Select at least one method of measurement—preferably more—for each expected outcome that you created in Step 2.

Measures should be observable and, in most cases, yield quantitative data.

Types of measures include

counts/rates/percentages

questionnaires and surveys

minutes of meetings and focus groups

deadlines met

13

Step 4: Target Levels

For each of the measures selected in Step 3, set a specific goal for the “score” that you hope to achieve.

Other terms that are often used for target levels are indicators or criteria.

The target level that you set for each measure should appear reasonably attainable

take into account your past performance in the function or service being assessed

provide the potential for showing improvement

14

Some examples of measures and target levels

Average attendance at library orientations will be 80% of all first-year students.

90% of fifty randomly selected financial aid files will be complete and accurate when audited.

Average scores on the Student Satisfaction Survey will be 4.2 on a five-point scale.

100% of scholarship applications will be processed within sixty days of receipt.

By December 2008, faculty using TechLink will be surveyed to determine what they see as the three most critical areas of need in upgrading the service.

15

Step 5: Findings

Briefly summarize the findings of your assessment by including for each expected outcome the target level that you set

the actual level that you attained

a brief analysis of why you did or did not meet the target level

16

Some examples of findings

In the annual survey of faculty/staff satisfaction, 36% of our customers expressed frustration with the wait time for help desk requests. Our target level was no more than 15% for this measure. Analysis of survey data showed that 85% of those respondents expressing frustration with the wait time for help desk requests were located in the Office of Financial Affairs.

Average routing time for personnel action forms (PAF) was decreased by 21%. Our target level was 30% for this measure. Analysis of the routing pathways for a random sample of thirty PAFs showed that forms for administrative personnel required seven more hours in transit than those for support staff because of several additional sign-offs being necessary.

17

Step 6: Action Plans

Your action plan should be a brief description of what steps you will take to address any expected outcomes where you did not meet your target levels.

You also may wish to review the outcomes in which you met your target levels to see if other improvements or activities are needed in the future—especially if you anticipate conditions changing so as to pose a threat.

Action plans should be included as expected outcomes in your next assessment cycle.

Unless you “close the loop” by taking actions based on your findings, you’re not really doing assessment!

18

Some examples of action plans

To address the 15% decline in annual giving by alumni, a personal contact, mail, and email campaign emphasizing to new graduates the benefits of annual giving to the institution will be developed and implemented in July 2009.

The 20% increase in wait time for level-one response will be addressed by the hiring of four new employees for the help desk during fiscal 2009.

The 17% increase in the number of books not returned to the library on time will be addressed by increasing the fines for overdue books to $3.00 per day starting in fall semester 2009.

19

Potential Assessment Pitfalls

Taking measurements that are not related to expected outcomes Waiting until the plan is perfect before beginning Collecting results for so many measures that results are overwhelming Spending more time on measuring things than on doing things

20

Relationship of Assessment to Planning

21

Module Two: Overview

Tour of Assessment Folders on R Drive Template for Creating Assessment Plans Sample Assessment Plan Practice Exercises Summary Assignments: Complete by September 5

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

Template for Creating Assessment Plans

37

Template for Creating Assessment Plans

38

Template for Creating Assessment Plans

39

Sample Assessment Plan

40

Sample Assessment Plan

41

Practice Exercises: Generate Mission Statement and Core Functions or Services Divide into groups of 3 Choose 1 group member (#1) for whose unit you will write a mission statement and

four core functions or services Member #2 will ask questions, #3 will take notes on responses 5 minutes: questions and notes 5 minutes: all three members write mission statement and four core functions or

services 5 minutes: discussion of selected mission statements and core functions/services by

all workshop members

42

Practice Exercises: Generate Expected Outcomes Remaining in the same groups as in the previous exercise and retaining your focus

on the same unit for which you created a mission statement and core functions/services, use the same questioning and note-taking process again to generate three expected outcomes for the unit

5 minutes: questions and notes 5 minutes: all three members write three expected outcomes 5 minutes: discussion of selected expected outcomes by all workshop members

43

Practice Exercises: Generate Measures and Target Levels Remaining in the same groups as in the previous exercise and retaining your focus

on the same unit for which you created three expected outcomes, use the same questioning and note-taking process again to generate measures and target levels for each of the three expected outcomes

5 minutes: questions and notes 5 minutes: all three members write measures and target levels for the three expected

outcomes 5 minutes: discussion of selected measures and target levels by all workshop

members

44

Summary

Six Steps of the Assessment Cycle Mission Statement and Core Functions or Services Expected Outcomes Measures Target Levels Findings Action Plans

Why We Do Assessment To improve what we do To inform decision-making, budgeting, and planning To focus our efforts To show students, alumni, employers, and others that we’re doing a good job To satisfy the expectations of our accreditors

45

Assignments: Complete by September 5

Download a copy of the assessment template for administrative and support units from the R:\Assessment Folder

Write a first draft of your unit’s 2008-2009 assessment plan that includes Mission statement

Four core functions or services

Three to five expected outcomes

Measures and target levels for each expected outcome

Set up an appointment with the DIE to review your draft assessment plan during the period September 8 – September 19

46

Assignments: Complete by September 5

Assessment Plans Should be Created by These Individuals and Their Support Staff1.Academic Dean

2.Director of Institutional Effectiveness

3.Director of Institutional Marketing

4.Director of Research

5.Executive Vice President

6.Human Resources Manager

7.IRB Manager

8.Medical Librarian

9.Marketing Manager

10.President

11.Registrar

12.Vice President of Academic Affairs

13.Vice President of Finance

14.Vice President of Student Services