1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the...

18
1 Assessment: Navigating the Shoals Without Running Aground Tidewater Community College Norfolk, VA Wednesday, June 18, 2003 by Douglas Eder Undergraduate Assessment and Program Review & The Undergraduate Research Academy Southern Illinois University Edwardsville Assessment works best when it is embedded in teaching and learning and when it provides feedback that is useful. Without these properties, assessment interferes with survival. This interactive presentation aims to use good thinking, humor, and feedback to harness the honest power of good assessment. And by easing the administrative burdens of assessment, a faculty may revive, survive, and thrive. SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected] Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Transcript of 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the...

Page 1: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

1

Assessment:

Navigating the Shoals Without Running Aground

Tidewater Community College Norfolk, VA

Wednesday, June 18, 2003

by

Douglas Eder

Undergraduate Assessment and Program Review &

The Undergraduate Research Academy

Southern Illinois University Edwardsville

Assessment works best when it is embedded in teaching and learning and when it provides feedback that is useful. Without these properties, assessment interferes

with survival. This interactive presentation aims to use good thinking, humor, and feedback to harness the honest power of good assessment. And by easing the

administrative burdens of assessment, a faculty may revive, survive, and thrive.

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 2: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

2

Some Assessment Ways and Means

• Case Studies • Classroom assessments • Completion and retention rates • Content analysis • Debates • Direct observations • Focus groups • Graduate success • Internships and service learning • Interviews (including videotapes) • Exams for certification and licensure • Matrices • Performances • Portfolios of several kinds • Projects (Primary Trait Analysis) • Questionnaires and surveys

(Direct and telephone; employer, alumni, and student attitude and satisfaction)

• Reflection essays • Study and activity logs • Tests

(Locally-developed and standardized) • Transcript analysis

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 3: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

3

ASSESSMENT: Academic Year, 2002-03 SENIOR ASSIGNMENT REPORT (Due in Assessment Office, Box 1300 by August 31, 2003)

Department Name__________________________Signature______________________________________ Chairperson/Undergraduate Program Director A. Who filled out this report?_____________________________________________________________ B. Who else in the department has read it?___________________________________________________ C. Who else in the department can discuss it?________________________________________________ ______________________________________________________________________________________ D. How many students completed Senior Assignments during AY2002-03 (Su02, Fa02, Sp03)?________ E. How many students, if any, did your department officially excuse, and therefore did NOT complete a

Senior Assignment, during the 2002-03 academic year?______________________________________ F. Please send via e-mail the name and Senior Assignment title of each student who completed your

departmental Senior Assignment during AY 2002-2003. Format: First name <space> Initial (optional) <space> Last Name <Single Tab> Senior Assignment Title <Carriage Return>. Please do not insert extra punctuation, spaces, or characters. We will load what you have written into your departmental portion of the Assessment web page (please refer to---http://www.siue.edu/~deder/assass/sra.html). If you cannot send via email, please use the same format on a separate sheet of paper.

G. The traits listed on the next page(s) are those identified by your department as the baccalaureate goals

for your students (please confirm that we have supplied your current page). For each trait listed, please enter the requested information.

H. Are you using Primary Trait Analysis to assess your SRAs? If so, please include your PTA form. If

not, how did you acquire the assessment data you are reporting?

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 4: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

4

Dep

t. or

Pro

gram

Fo

llow

-up

Non

e re

quire

d Fo

llow

-up

com

plet

ed o

n da

te__

___

W

ill re

-exa

min

e by

da

te__

___

Non

e re

quire

d Fo

llow

-up

com

plet

ed o

n da

te__

___

W

ill re

-exa

min

e by

da

te__

___

Non

e re

quire

d Fo

llow

-up

com

plet

ed o

n da

te__

___

W

ill re

-exa

min

e by

da

te__

___

Non

e re

quire

d Fo

llow

-up

com

plet

ed o

n da

te__

___

W

ill re

-exa

min

e by

da

te__

___

Out

com

e of

A

naly

sis

Obj

ectiv

e w

holly

sa

tisfie

d O

bjec

tive

not

who

lly sa

tisfie

d.

Follo

w-u

p st

rate

gy

is:_

____

____

____

_ __

____

____

____

__

Obj

ectiv

e w

holly

sa

tisfie

d O

bjec

tive

not

who

lly sa

tisfie

d.

Follo

w-u

p st

rate

gy

is:_

____

____

____

_ __

____

____

____

__

Obj

ectiv

e w

holly

sa

tisfie

d O

bjec

tive

not

who

lly sa

tisfie

d.

Follo

w-u

p st

rate

gy

is:_

____

____

____

_ __

____

____

____

__

Obj

ectiv

e w

holly

sa

tisfie

d O

bjec

tive

not

who

lly sa

tisfie

d.

Follo

w-u

p st

rate

gy

is:_

____

____

____

_ __

____

____

____

__

Whe

n an

d B

y W

hom

Wer

e R

esul

ts A

naly

zed?

Dep

t. m

eetin

g da

te

____

____

____

____

or

Indi

vidu

al a

naly

sis

(des

crib

e)?

____

____

____

____

____

____

____

____

____

____

____

____

Dep

t. m

eetin

g da

te

____

____

____

____

or

Indi

vidu

al a

naly

sis

(des

crib

e)?

____

____

____

____

____

____

____

____

Dep

t. m

eetin

g da

te

____

____

____

____

or

Indi

vidu

al a

naly

sis

(des

crib

e)?

____

____

____

____

____

____

____

____

Dep

t. m

eetin

g da

te

____

____

____

____

or

Indi

vidu

al a

naly

sis

(des

crib

e)?

____

____

____

____

____

____

____

____

Obs

erva

tions

of

Stud

ent

Perf

orm

ance

How

man

y st

uden

ts

____

_exc

eede

d __

___m

et

____

_did

not

mee

t

exp

ecta

tions

? __

___e

xem

pted

__

___T

OTA

L

How

man

y st

uden

ts

____

_exc

eede

d __

___m

et

____

_did

not

mee

t

exp

ecta

tions

? __

___e

xem

pted

__

___T

OTA

L

How

man

y st

uden

ts

____

_exc

eede

d __

___m

et

____

_did

not

mee

t

exp

ecta

tions

? __

___e

xem

pted

__

___T

OTA

L

How

man

y st

uden

ts

____

_exc

eede

d __

___m

et

____

_did

not

mee

t

exp

ecta

tions

? __

___e

xem

pted

__

___T

OTA

L

Expe

ctat

ion

for

Satis

fact

ory

Perf

orm

ance

[e.g

.] C

lear

incl

usio

n of

scie

ntifi

c m

etho

d;

expo

sitio

n of

test

able

hy

poth

esis

, sta

tistic

ally

co

mpe

tent

ana

lysi

s of

resu

lts, d

iscu

ssio

n of

w

ork

in li

tera

ture

.

[e.g

.] (a

& b

) C

ompe

tent

te

chni

cal

expl

anat

ion

such

th

at la

y as

sess

or

mat

ches

at l

east

4

out o

f 5 m

ain

poin

ts

on st

uden

t’s o

utlin

e.

[e.g

.] G

rade

at l

east

C

on S

enio

r Sem

inar

pa

per t

hat a

pplie

s pr

inci

ples

of g

en. E

d.

Pape

r to

disc

ussi

on

topi

c of

the

Seni

or

Sem

inar

subm

itted

w

ith S

RA.

[e.g

.] Al

l gra

phs

prep

ared

via

com

pute

r gr

aphi

cs; a

xes,

scal

es,

title

pre

sent

; gra

phic

st

yle

(his

to, b

ar, p

ie)

appr

opri

ate

to d

ata;

re

adab

le fr

om 1

met

er

away

.

Whe

re, W

hen,

and

H

ow M

onito

red

[e.g

.] W

ritte

n an

d or

ally

def

ende

d Se

nior

Ass

ignm

ent

each

Spr

ing

sem

este

r; m

ajor

ity

of fa

culty

ass

esse

s th

roug

h Pr

imar

y Tr

ait A

naly

sis.

[e.g

.] (a

) Ora

l de

fens

e of

SRA

(see

ab

ove)

; (b)

requ

ired

40

0-le

vel s

emin

ar

cour

se w

ith o

ral

repo

rt re

quir

emen

t

[e.g

.] D

iscu

ssio

n in

Se

nior

Sem

inar

of

own

wri

tten

wor

k fr

om a

ny e

thic

s co

urse

take

n in

SI

UE

Gen

eral

Ed

ucat

ion

prog

ram

.

[e.g

.] Se

nior

As

sign

men

t

SIU

E U

nder

grad

uate

Ass

essm

ent a

nd P

rogr

am R

evie

w

SAM

PLE

– S

AM

PLE

– S

AM

PLE

D

epar

tmen

t of

D

emo

Prog

ram

– In

itiat

ed

Goa

l or O

bjec

tives

[For

exa

mpl

e]

Bacc

alau

reat

e kn

owle

dge

of

[maj

or] d

isci

plin

e

[For

exa

mpl

e] O

ral

com

mun

icat

ion

[For

exa

mpl

e]

Sens

e of

eth

ics a

nd

ethi

cal i

mpa

ct o

n so

ciet

y of

maj

or

disc

over

ies w

ithin

th

e [m

ajor

] di

scip

line.

[For

exa

mpl

e]

Com

pute

r co

mpe

tenc

e

Oth

er o

bser

vatio

ns?

Une

xpec

ted

findi

ngs?

___

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

___

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

_ __

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

____

___

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 5: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

5

SIUE Undergraduate Assessment and Program Review – Department of The Senior Assignment (SRA) is a scholarly inquiry between students and dedicated faculty members that results in a product. Because the product is visible, it and the curriculum that produced it can be assessed. A faculty can reflect upon the evidence of student learning revealed by the SRA and come to some conclusions about how its students learn. Upon reflecting on the evidence, what has your department discovered about student learning under its direction? What are you going to do about what you have discovered? How is the information about student learning going to feed back into your curriculum so that teaching and learning improve to the “Next Level”? What, if any, departmental assessment- or SRA-related highlights regarding student achievement from this past year do you wish to share?

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 6: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

6

Goal Ranking and Matching

What do you hope to get out of this seminar/workshop? What goals or expectations do you wish to satisfy? This Classroom Assessment Technique (CAT) is designed to help make your goals and expectations visible to yourself and to assist you in discussing them with others, including the presenter. Part of this CAT also reveals the presenter’s goals to you so you can see how well they match your own. 1. On the lines below, please write 2-5 goals you hope to achieve ---specific things you want to learn--- by participating in this seminar/workshop. What you want to learn: Your ranking: Do they match

the presenter’s?

_______________________ ___________ Yes No _______________________ ___________ Yes No _______________________ ___________ Yes No _______________________ ___________ Yes No _______________________ ___________ Yes No 2. Use the middle column to rank your goals in terms of their relative importance to you. The most important goal is ranked #1, then next most important #2, etc. 3. When you hear the presenter’s goals, circle Yes in the right hand column next to each of your matching goals, regardless of rank. If you have goals left over at the end, circle No in the right hand column next to them. 4. Prepare to talk or ask questions about any of your important goals that are not also in the presenter’s list.

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 7: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

7

Approximate Analogy, Virtually Speaking INSTRUCTIONS: Complete one analogy below. 1. Assessment is to teaching as: ___________________________ is to ___________________________. Explain. _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ 2. F2F classroom teaching is to technology-assisted education as: ___________________________ is to ___________________________. Explain. _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 8: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

8

Tidewater Community College, June 18, 2003 Navigating Assessment Shoals © 2003 Douglas J. Eder, SIUE

In Scholarship Assessed, Ernest Boyer’s principles of reflective practice are advocated as desirable habits. Application of these principles endows assessment with elegant properties that can be used to satisfy criteria for classroom assessment, formal program review, institutional accreditation, and individual promotion/merit/tenure portfolios as well. Thus, assessment merits our attention because it supports honest, reflective, scholarly practice. Principle #1: Clear Goals: Does the scholar state the basic principles of his or her work clearly? Does the scholar define objectives that are realistic and achievable? Does the scholar identify important questions in the field?

Picture a domain --- assignment, topic, course, or program --- for which you have responsibility. What Big Rocks related to student learning do you, as a professor, want to "go in first"? That is, what major academic goals do you wish your students to achieve under your guidance? ________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

Principle #2: Adequate Preparation: Does the scholar show an understanding of existing scholarship in the field? Does the scholar bring the necessary skills to his or her work? Does the scholar bring together the resources necessary to move the project forward? Ideally and briefly, what would you need to see (what evidence needs to be present, what specific behavior needs to be visible) in order for a skeptic to see that your baccalaureate students are achieving the student-related Big Rocks of Assessment #1? Identify those that require a disciplinary specialist for interpretation and those that would be meaningful to a disciplinary layperson.

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________ SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 9: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

9

Stating Academic Objectives

----after Lion Gardiner, Rutgers University

Conclusions from a variety of audits suggest that perhaps 30-40% of the money spent in the US on higher education is not used effectively. Because taxpayers, donors, students, and their parents provide money for higher education, it's natural that they want to know what we are doing. Statements of educational goals and objectives communicate expectations and standards to these audiences. They are promises about actions and results.

To be useful, goals must be visible and they must be used. They are not statements about institutional structure, departmental hopes, or things that faculty members are working on. Objectives are indicators of goals. Here's a draft list of objectives for consideration. A critique is available at the end.

Draft objectives for students in Department XXXXXX:

1. In agreement with the University mission statement, this department's first priority is excellence in undergraduate teaching.

2. Students will be able to think, write, and speak clearly. 3. Students will understand current issues in this academic discipline and engage in

life-long learning. 4. Students will be aware of problems in the discipline and be familiar with the

knowledge and methods necessary to solve those problems. 5. Students will analyze available data on one such problem and construct a

workable solution that is deemed satisfactory by an expert in the field. 6. Students will cultivate ethical standards and aesthetic appreciation. 7. The department will foster tolerance and an ability for students to work in diverse

groups. 8. Faculty development activities will be available in order to improve classroom

teaching. 9. Advisors will be punctual, approachable, and understanding and will encourage

students to investigate opportunities that elevate them to their full potential. 10. Before leaving the office, each student will have correctly filled out lines #2 and

#6 of the FAFSA and recited out loud future deadlines for turning in fully completed forms.

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 10: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

10

“In each case the portfolio assessment project was mounted not to assess students’ individual achievements but rather to gain an overall picture of student performance in order to examine systematically the effectiveness of the curriculum…

“The most crucial feature of portfolio assessment for program evaluation is the assessment framework, that is, the set of criteria for assessing the portfolios. …The criteria should reflect a consensus among lecturers about what the course really values or aims to achieve. Ideally, the portfolios are assessed against the explicitly stated learning goals of the course. The extent to which these goals have been defined from one course to another may vary greatly. The crucial issue, however, is that in the absence of explicitly stated learning goals, portfolio assessment provides an opportunity for lecturers in a particular course to make explicit the expectations of their discipline.

“…Portfolio assessment that has as its principle aim to assess individual students’ achievements will consider the extent to which a student’s portfolio shows evidence of, for example, critical reflection, conceptual understanding, and creativity. However, portfolio assessment for program evaluation is less interested in individual students’ performance and more interested in the overall performance picture for the various tasks. In addition to assessing students’ development form task to task, assessors are interested in assessing students’ performance across a single task. This perspective reveals which tasks have prompted the desired responses (for example, critical, analytical thinking) and why. …In the case of each project, we believe that the portfolio assessment process has resulted (or will result) in improvements at the level of course design and delivery.”

--from Suellen Shay, Portfolio Assessment for program evaluation. Assessment Update, March-April, 1997: 9(2): 8-13.

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 11: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

11

Annotated Word Journal

Read the assigned text and write one word that captures the essence of what you’ve read and summarizes your response to it.

___________________________ One Word Summary

Explain why you chose that word and how it provides, in a capsule, your summary of the reading. _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________ _____________________________________________________________

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 12: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

12

Principle #3: Appropriate Methods: Does the scholar use methods appropriate to the goals? Does the scholar apply effectively the methods selected? Does the scholar modify procedures in response to changing circumstances?

A. What venues exist where assessment activities might take place to gather evidence of the sort described in Assessment #2?

B. What resources need to be maintained in your environment in order for you to collect this evidence? Which of these are the most important?

C. What obstacles, if any, are preventing you from collecting the evidence described in Principle #2 and achieving your student-related Big Rocks?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

Principle #4: Significant Results: Does the scholar achieve the goals? Does the scholar's work add consequentially to the field? Does the scholar's work open additional areas for further exploration?

Subdivide one item of evidence, activity, or behavior into component parts and construct a rubric representing a 3- or 4-point assessment of quality.

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 13: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

13

Principle #5: Effective Presentation: Does the scholar use a suitable style and effective organization to present his or her work? Does the scholar use appropriate forums for communicating work to its intended audiences? Does the scholar present his or her message with clarity and integrity?

Incoming Student Curriculum Teaching & Learning Processes Outcome Outgoing Student

To what forums and audiences will assessment reports in your chosen domain be addressed? Who will write the reports? Who will read them? What constructive feedback mechanisms are active? How will reports be used and what will happen as a consequence?

The Four Components of Complete Assessment What did you learn?

How do you feel? Who noticed? Who cares?

Three Questions Appearing in Every External Assessment Inquiry

What have you learned about your students’ learning? What are you going to do about it?

When, where, and how are you going to do it?

Principle #6: Reflective Critique: Does the scholar critically evaluate his or her own work? Does the scholar bring an appropriate breadth of evidence to his or her critique? Does the scholar use evaluation to improve the quality of future work?

Assessment? --- Standardized Test

Who: The student What: Takes a standardized test When: Just prior to graduation Why: To provide an assessment of academic achievement at the completion of

the degree program. Description: All degree-seeking students are asked to take a standardized, norm-based

field test, to be scored by a testing service, prior to receipt of the degree. In order to take the test the student must have a 2.0 GPA and have completed a specified number of credit hours in the program.

Data: Well above average, above average, average, below average, well below average

---after Patricia D. Murphy

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 14: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

14

Assessment? --- Comprehensive Exam Who: The student What: Takes a comprehensive exam When: Just prior to graduation Why: To provide a summary assessment of academic achievement at the

completion of the degree program. Description: All degree-seeking students are asked to take a comprehensive exam in the

discipline prior to receipt of the degree. The exam will be developed, administered, and read locally. In order to sit for the exam the student must have a 2.0 GPA and have completed a specified number of credit hours in the program.

Data: Exceptional, satisfactory, contingency, unsatisfactory ----after Patricia D. Murphy

Assessment? --- Thesis/Final Project Who: The student What: Undertakes a scholarly, [semi-] independent project under faculty

supervision When: During the final year of study Why: To assess the student's scholarship, critical thinking, creativity, research

skills, written and oral communication. Description: The thesis or final project requires the student to plan, implement, and

defend research, an action plan, or a project which is the culmination of a program of studies. Each thesis or written project will be read by a committee of the faculty. This assessment is a synthesis of knowledge and skills developed throughout the program and is a demonstration of the student's scholarship, creative and analytical skills, communication, and understanding of the content area.

Data: Satisfactory, unsatisfactory ----after Patricia D. Murphy

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 15: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

15

Plan for Systematic Evaluation, Department of XXXXXXXX

The Department of XXXXXXXXX is an undergraduate element in the YYYYYYY College curriculum with a component that bears national professional accreditation through ZZZZZZZ. The department believes that assessment is integral, even essential, to the educational process and will engage in a continuous, vigorous process of appraisal in order to form the basis for judging all aspects of its programs and for making improvements. Our objectives are to (a) examine whether our philosophy aligns with current best practices, standards, and trends in the field, (b) assure that all faculty members participate as involved scholars, (c) determine whether the program provides the means for students to excel, (d) improve teacher preparedness, and (e) unify the departmental curriculum. For this purpose, the departmental assessment program will systematically evaluate pedagogical goals, curricular structure, administrative support, faculty and staff credentials, educational facilities, financial resources, and students services.

Elements of the plan (a) Alignment of philosophy: We will examine our catalog, viewbook, and similar publications

for content and clarity of our statements. This formal process will occur every two years and will separately involve a committee of the faculty, the council of deans, the senate curriculum committee, and a broad of external reviewers. This will assure that public statements regarding our practices are aligned and up to date with both community college and professional standards.

(b) Faculty involvement: We will review faculty vitae and faculty and staff performance evaluations for publications, career progress, student satisfaction, and professional development as appropriate. This process, to be conducted every two years, will be undertaken by department chairpersons, deans, and an advisory committee comprised of elected faculty and staff. Formal biennial review will assure that our faculty and staff are of adequate size and scope and that all members of the community college are involved and committed to continuous improvement through continuing education and training.

(c & d) Curricular excellence and teacher performance: We will study course syllabi, examinations, assignments, tutoring services, and academic advising in order to confirm that the agreed-upon departmental philosophy penetrates the curriculum and to gather evidence that our course efforts and content harmonize with that philosophy. We will sponsor faculty and staff development activities, hold informative departmental seminars, and catalog the new pedagogies we have acquired and used. We wish to assure that diverse elements are in place so that students can excel in the competencies set for them. This study will recur every three years and will involve an elected committee within the department, two representatives from outside the department, the dean, and two external evaluators from similar departments at other universities.

(e) Unified, supported curriculum: We will conduct student evaluations, exit interviews, focus groups, alumni surveys, and employer surveys in order to ascertain stakeholder satisfaction with our educational quality and scope. We will examine budgets and facilities. This undertaking will recur every five years and will involve our entire faculty, the dean, the academic vice president, and one representative from the professional certifying association that accredits our program.

A. What are the programs’ intended student learning outcomes? ________________________________________________________________________ ________________________________________________________________________ B. What evidence of learning is being collected? ________________________________________________________________________ ________________________________________________________________________

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 16: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

16

Some Useful References on Academic Assessment

Angelo, T.A., and K.P. Cross, 1993. Classroom assessment techniques: A handbook for college teachers, 2nd ed. San Francisco: Jossey-Bass. Astin, A.W., 1993. What matters in College? Four critical years revisited. San Francisco: Jossey-Bass. Banta, T.W., J.P. Lund, K.E. Black, and F.W. Oblander, 1995. Assessment in Practice: Putting principles to work in college campuses. San Francisco: Jossey-Bass. Eder, D.J., 2001. Accredited programs and authentic assessment. In Banta, T.W., and C.A. Palomba (eds.), Assessing Student Competence in Accrredited Disciplines. Sterling, VA: Stylus. Ewell, P.T., and D.P. Jones, 1996. Indicators of “Good Practice” in undergraduate education: A handbook for development and implementation. Boulder, CO: National Center for Higher Education Management Systems (NCHEMS). Ewell, P.T., and R.P. Lisensky, 1988. Assessing institutional effectiveness. Washington, DC: Consortium for the Advancement of Private Higher Education. Fong, B., 1994. Assessing the departmental major. In Stark, J.S., and A. Thomas (eds.), Assessment and Program Evaluation. ASHE Reader Series. Needham Hts., MA: Simon and Schuster, p. 413-21. Chickering, A.W., and Z.F. Gamson, 1991. Applying the Seven Principles for Good Practice in undergraduate education. San Francisco: Jossey-Bass. Glassick, C.E., M.T. Huber, and G.I. Maeroff, 1997. Scholarship assessed. San Francisco: Jossey-Bass. Mosteller, F., 1989. The “Muddiest Point in the Lecture” as a feedback device. In On teaching and learning: The Journal of the Harvard-Danforth Center. 3(April): 10-21. Palomba, C.A., and T.W. Banta, 1999. Assessment essentials. San Francisco: Jossey-Bass. Pascarella, E.T., and P.T. Terenzini, 1991. How college affects students: Findings and insights from twenty years of research. San Francisco: Jossey-Bass. Santanello, C., and D.J. Eder, 2001. Stopping to listen: Classroom assessment techniques. NEA Higher Education Advocate. Smith, D., and D.J. Eder, 2000. Assessment and program review: Linking two processes. In A Collection of Papers on Self-Study and Institutional Improvement. Chicago: North Central Association, p. 186-88. Walvoords, B.E., and V. Anderson, 1988. Effective grading: A tool for learning and assessment. San Francisco: Jossey-Bass. Wiggins, G., 1993. Assessment: Authenticity, context, and validity. Phi Delta Kappan (Nov.): 200-14.

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 17: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

17

Tidewater Community College, June 18, 2003 Navigating Assessment Shoals

THREE-STEP ASSESSMENT

Assessment #1: What is the most useful thing you’ve learned today from this Assessment Seminar/Workshop?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

Assessment #2: What one major question arises in your mind or is still unanswered? That is, what is the least clear to you and that you would like to have addressed?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

Assessment #3: Imagine that, as you walk across campus during the next three weeks, a member of your Board of Trustees stops you and asks what 2-3 main things you learned about assessment today and what specific behaviors you’re going to change as a result. You’ll have this person’s attention for about 45 seconds. In four sentences maximum, what will you say?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Page 18: 1 Assessment: Navigating the Shoals Without Running · PDF fileAssessment: Navigating the Shoals Without Running Aground ... fi c m e tho d; ... required 400-level seminar course with

18

SIUE Undergraduate Assessment and Program Review -&- The Undergraduate Research Academy Box 1300 voice: (618) 650-2640 FAX: (618) 650-3633 e-mail: [email protected]

Edwardsville, IL 62026-1300 USA http://www/siue.edu/~deder/assess/index.html

Tidewater Community College, June 18, 2003 Navigating Assessment Shoals

Seminar/Workshop Evaluation

Assessment is the monitoring of learning. It is not a judgment of teaching. In contrast, evaluation does imply judgment of instructional quality. What you were asked on the previous page focused on what you learned. The questions on this page focus on the qualities and procedures of what was taught.

Evaluation #1. How did this seminar/workshop experience stack up in terms of facilitator enthusiasm and presentation, material usefulness, session organization, and your commitment? Please briefly describe your observations and degrees of satisfaction.

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

Evaluation #2. What do you consider to be the strongest and most successful aspects of this seminar/workshop?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________

Evaluation #3. What do you consider to be the weakest aspects and what would you eliminate or do to improve them?

________________________________________________________________________

________________________________________________________________________

________________________________________________________________________