Assessment of Adult Programs: Lessons Learned and New Approaches

Laurie Dodge Associate Vice Chancellor of Institutional Assessment and PlanningBrandman University

ALFI at Brandman University

Brandman University has conducted assessment of Adult Learning Focused Principles (ALFI) through a paper-based portfolio system for the past seven years at twenty-six campuses. In this session, information will be presented on the successful launch of an e-portfolio system for assessing ALFI Principles that aligns with university key strategic indicators and meets individual campus cultures and needs. In addition, the process for developing an e-portfolio system and the rubric for evaluation will be shared.

Intervention

AssessmentAnalysis

Why Is Assessment Important

Accountability• Show Evidence• Data demonstrating

strengths and weaknesses• Show-case accomplishments

Continuous improvement• Data-driven strategic

approach to address areas of improvement

ALFI at Brandman University

• 8 ALFI Principles + 1 BU Customized Principle (Mission)

• Principles are aligned to WASC Standards• Assess two principles each year

2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014

Outreach *Development & Calibration

Transitions Life/Career Planning

Assessment of Learning

Outcomes

Teaching/Learning Process

Student Support Systems

Technology Learning Community

Strategic Partnership

Technology

ALFI at Brandman University

• Standardize– Avoid “scrapbook” submissions– Targeted evidence gathering– Focused goal setting

• Flexibility– Able to create own goals– Choose evidence

ALFI at Brandman University

• Use existing key performance indicators– Advising survey– Student satisfaction survey– Alumni survey– Work environment survey– Course evaluations

ePortfolio

• Ease of submission• Transparency

Process

• Standardized approach with templates– Complete Assessment Report

Template– Upload two pieces of evidence– Establish Goals using Template

• Goals assessed after 6 months• Rubric applied to Assessment Report,

Submitted evidence, and Goal Attainment

Assessment Report

Evidence

Goals

Goals Assessed

Rubric

Content Areas Exemplary (4)

Proficient (3)

Developing (2)

Emerging (1)

Evidence (Uploaded evidence)

Have two pieces of innovative evidence for the assigned ALFI principle that reflects its intent.

Have two pieces of evidence for the assigned ALFI principle that reflects its intent.

Have one or more piece of evidence for the assigned ALFI principle that does not clearly reflect its intent.

Lacking adequate evidence or multitude of evidence that is not targeted to the principle or the evidence.

Quality of Evidence and Reflection Included in Assessment Report (AR)

Summary of evidence and reflection is thorough and a direct indication that the campus exceeds the subprinciple.

Summary of evidence and reflection is thorough and a direct indication that the campus meets the subprinciple.

Summary of evidence and reflection is complete and an indication that the campus somewhat meets the subprinciple.

Summary of evidence and reflection was lacking and there is no indication that the campus meets the subprinciple.

Reflection of Goal Attainment (Revised Goals Document)

Reflection of goal attainment for each ALFI principle is insightful, focused and aligned to the evidence.

Reflection of goal attainment for each ALFI principle is focused and aligned to the evidence.

Reflection of goal attainment is somewhat focused and most of the content was aligned to the evidence.

Reflection of goal attainment is vague, ambiguous, and/or not aligned to the evidence.

Goal Attainment (Revised Goals Document)

75% or more of the goals are attained and suggestions for continuous improvement were noted.

75% of the goals are attained.

50% of the goals are attained.

Less than 50% of the goals are attained.

Rubric Applied

Closing the Loop

• Feedback provided to Campus Directors– Letter– Follow-up at monthly meeting– Sharing rubric– Bridge with Student Affairs

Closing the Loop

Lessons Learned

• Give campus directors a full year to assess their goals

• Allow more than two pieces of evidence• Apply rubric scores more strictly (no use of

decimals)

Questions?