Making Good Use of Existing Information about Student Learning to Inform and Support Curricular Reform

Dr. Diane M. EnersonProfessor of Psychology & Director of the Center for

Teaching Excellence

Dr. Heidi Northwood Professor of Philosophy & Director of the Core

Curriculum

The Old Core

• Had become calcified into a set of seemingly unconnected requirements

• Politics…

• But no comprehensive data analysis-- dissatisfaction with old Core based on faculty perceptions.

The Union of Insufficiencies

“What we need therefore is a union of insufficiencies, a marriage of complements,

in which the flaws of individual approaches to assessment are offset by the virtues of their

fellows.” Shulman, L.S. (1988)

The New Core

• Oversight• Faculty endorsed Core SLOs (Skills & Whole

Person)• Intentionally integrated curriculum (Enduring

Questions, Multidisciplinary exploration of central topic, Experiential Learning, etc.)

• Emphasis on student responsibility & ownership of learning

• BUT… how do we know if it’s any better?

Our SLOsSKILLS

• Read and write with clarity and precision• Speak with clarity and precision and actively listen to

formal and informal discourse• Demonstrate ability to discern the formal, historical and

cultural aspects of fine arts• Effectively confront and solve problems using

quantitative and qualitative methods• Identify, evaluate and engage in scientific and empirical

modes of inquiry• Communicate information symbolically, visually,

numerically, and verbally• Rationally and objectively apply criteria in order to

evaluate and question information or ideas• Individually and collaboratively articulate multiple

solutions to problems or questions, and evaluate those solutions in both disciplinary and multidisciplinary contexts

• Identify, access, evaluate, manipulate & use information effectively from a variety of sources

• Demonstrate the technological skills necessary for personal and scholarly activities

• Describe and evaluate, from various perspectives, the historical construction of cultures, including one’s own

• Interact effectively in various social and cultural settings

WHOLE PERSON• Describe and reflect on the learning process • Identify, reflect on, and compare different worldviews,

philosophies and spiritualities including one’s own• Evaluate the ethical implications of decisions • Achieve an appreciation of and ability to analyze

aesthetic qualities of fine arts• Explore, evaluate and articulate personal values • Integrate and apply academic skills to understanding

practical experiences and problems found in our world• Demonstrate knowledge of different disciplinary

perspectives used to understand and explain the human experience

• Identify and pursue life choices that promote physical and psychological health and well-being.

Other Things We Thought About

• Mission of Institution• Middle States • College of Arts & Science SLOs• The data that had been collecting

dust on our shelves… NSSE, HERI, LAST and IDEA.

The result…

Our SLO Mapping DocumentCore Skills SLOs Middle States SLOs College of Arts &

Sciences SLOsPossible Rubric Direct in Core

EARLYDirect in Core

LATEIndirect Assessments

1. Read & Write Written Communication

Communicate eff. through writing

AAC&U Written Communication

ENGW courses Core Milestone Exp, LAST (partial)

HERI Q14, NSSE Q1c, NSSE Q11c, IDEA Q28

2. Speak & Listen Oral Communication

Communicate eff. though oral presentation skills

AAC&U Oral Communication

First Year Seminar & Academic College Success (ACS)

CME NSSE NAZ Q11a, NSSE Q1b, NSSE Q11d, IDEA Q28

3. Fine Arts discern Our ‘V/P Arts’ THA, ART, MUS

4. Quan/Qual Quantitative Reasoning

Develop quant. abilities

AAC&U Quantitative Lit

MTH, SCI LAST (partial)Some Capstones

NSSE Q11f

5. Scientific Scientific Reasoning

Our ‘Scientific Method’

BIO, CHM, SCI LAST (partial)Some Capstones

6. Information Com AAC&U Quan Lit MTH LAST (partial)

7. Eval ideas Critical Anaysis/ Reasoning

Develop analyt abil and crit thinking

Our ‘Crit Thinking’ PHL, PSC, ENG, PSY CMESome Capstones

HERI Q1, NSSE Q2b, NSSE Q2d, NSSE Q11e, IDEA Q31

8. Problem Solving Critical Anaysis/ Reasoning

Problem solving abilities

Our ‘Crit Thinking’? All P-EQ courses Some Capstones NSSE Q1g, NSSE Qih, NSSE Q1i, NSSE Q2c, NSSE Q11h, XEEW Q11m, IDEA Q23

9. Use Information Critical Anaysis/ Reasoning & Info Literacy

Communicate eff through critical reading skills

Our ‘Crit Thinking’? HIS, ECO CME, LAST (partial)

HERI Q15, NSSE 1d, NSSE Q11g

10. Tech Skills Technological Competence

To be developed CME NSSE Q1l, NSSE Q10g, NSSE Q11g

11. Hist construct Cultural & Global Awareness & Sens

Cultural compet & civic engaged

AAC&U Intercul Kn & Competence

HIS, RES, SOC, ANT, ENG

NSSE Q1e

12. Interact social Prep for Enlightened Judg.

Cultural compet & civic engaged

AAC&U Intercul Kn & Competence

ACS HERI Q10, HERI Q16, HERI Q17, NSSE Q6e, NSSE Q10c, NSSE Q11l

Nazareth College Core Baseline Report

INTRODUCTIONThe following is a compilation of the various indirect assessment data (HERI, NSSE, IDEA) that are related to the new Core Curriculum’s Student Learning

Outcomes. Each question on each survey was judged to be either relevant or not as indirect evidence of an SLO, and further, whether it captured students’ beliefs about their own progress towards a particular SLO or indicated a behavior or activity that was judged to be relevant to the SLO (indicated below as ‘Related Activities’). Data for each question were grouped under each Core SLO (both Core Skills SLOs and Core Whole Person SLOs). In addition, there is a ‘Highlights’ section for each Core SLO. For the most part, these highlights are only descriptive.

The intent of this document is to provide a ‘baseline’ set of data that can be compared to answers to the same questions on the same survey instruments regularly during the implementation process of the new core, and once it is fully implemented. Starting Fall 2011 we will also have (in addition to the LAST data discussed below) direct measures for some of these SLOs through the use of AAC&U rubrics in our pilots of both Perspective-Enduring Questions courses and First Year Seminar.

P.S. The NSSE results for 2011 is very different from that of 2004 and 2007. We need to study this.==============================================================================================================================CORE SKILLS SLO#1: READ AND WRITE WITH CLARITY AND PRECISION(HERI Q14, NSSE Q11c, IDEA Q28, NSSE Q1c; LAST (Written Analytic Expression))Highlights:• 95% of faculty think that it is essential or very important to promote students’ ability to write effectively. (HERI)• By the time they are seniors, 77% (2004) and 76% (2007) and 83% (2011) of students report that their experience at Nazareth has promoted their

writing clarity and effectiveness ‘very much’ or ‘quite a bit’. If one adds ‘some’ to this, the percentages are 92% (2004) and 90% (2007) and 98% (2011). But first year and seniors reports are surprisingly similar, or, in the cases of both 2007 and 2011, a higher percentage of first year students than seniors report that Nazareth has promoted this skill). (NSSE)

• At the individual course level, the percentages of students who report that they have made substantial or exceptional progress towards developing skill in oral or written expression is much lower (59%-63% range, 2007-2010). (IDEA)

• There are significant differences between 2004/2007 and 2011 with regard to student reports about how often they write multiple drafts of a paper or assignment between turning it in. The main difference is that in 2011, students report engaging in this activity a lot in their first year but not so much in their senior year (74% of first year students report that they do this ‘often’ or ‘very often’, whereas 43% of seniors say the same). The opposite trend is found in both 2004 and 2007. In their senior year, approximately 60% of seniors (in both 2004 and 2007) report that they ‘often’ or ‘very often’ write multiple drafts of a paper or assignment before turning it in (as opposed to 40% of first year students in 2004, and 33% in 2007; only 17% of seniors in 2007 reported that they ‘never’ did so. (NSSE Q1c) Also, while the data are not included below, there is a clear trend to shorter paper assignments between 2004 and 2007 (see NSSE Q3c-e). The significance and consequences of this should be more fully studied.

• Of the five sub-tests in LAST, Written Analytic Expression had the lowest success rate (86%) (cf. Science, Math & Technology: 98%). ______________________________________________________________________________________________________________________________…and so on…

Where We Are Now

• Continuing to collect Indirect Assessment Data• Faculty beginning to use standard rubrics in

new core pilot courses• Much discussion & tweaking of rubrics• Much discussion & tweaking of who is

responsible for what (SLOs, rubrics, etc.)• Assessment of pilots (IDEA, rubrics, student

evaluation, faculty focus groups)

Your Task Today

1) Select a Student Learning Outcome2) Choose an approach (i.e., forwards vs. backwards)3) Using the IDEA data from Utopian U, answer the

following: a) What information do we have that might apply?b) How can we use it?c) What inferences can we draw?d) What else do we need to know?

Take Aways

DIANE:• Persistence pays• Together blind men CAN

describe an elephant

HEIDI:• Stop and smell the data• Rome was not built in a

day