Integrated Learning Experiential Assessment Program (I-LEAP)

Julie Burdick Director of Academic Planning & Assessment

julie.burdick@uc.edu513-556-4317

www.uc.edu/icl/ILEAP.html

What is I-LEAP?

I-LEAP is an innovative experiential learning assessment instrument which gathers external reviewers’ observations of student learning outcomes and skills as demonstrated in contextual learning environments such as:

• Faculty mentor assessment of students’ undergraduate research

• Community partner assessment of students’ service learning projects

• Employer assessment of student internships, practicum, clinicals, and student teaching placements

• Faculty assessment of students’ study abroad experiences

I-LEAP History

• In 1999, the US Department of Education Fund for the Improvement of Postsecondary Education [FIPSE] awarded UC’s Professional Practice a grant to develop a Corporate Feedback System for Use in Curricular Reform.

• The objective of the grant is to build a closed-loop system that measures student performance, including General Education learning outcomes, while on co-op and directs this feedback into curricular development.

• The survey instrument was developed in partnership with 1,500 co-op employers and UC faculty and reviewed by the National Association for Cooperative Education.

• Data from this project (400,000 data points per year) are used as evidence of student learning outcomes on page 5 of the UC’s College Portrait, part of the Voluntary System of Accountability (http://www.uc.edu/institutionalresearch/CoopLO.htm).

Co-op Employer Data found on UC’s College Portrait

A 2-3 year rolling average is used to increase student numbers for data validity and take into account assessor bias

Co-op Employer Data Findings

College of Business: found that students scored low in teamwork related questions. A comprehensive First Year Experience was added to the curriculum emphasizing teamwork, leadership, and communication skills. Teamwork scores have improved significantly.

Design Architecture Art & Planning (DAAP): found that students entered with high scores and left co-op with high (or slightly lower) scores indicating a “ceiling effect.”

College of Engineering (Computer Engineering): found that students’ writing ability deteriorated over the co-op years. Computer Engineering will work with the English department to reconsider the content and placement of technical writing in the curriculum.

I-LEAP Timeline

September 2008: Pilot the Service Learning I-LEAP using SurveyMonkey

January – August 2009: Design and development of I-LEAP software

March 2009: Pilot the Study Abroad I-LEAP (SurveyMonkey)

June 2009: Pilot the Undergraduate Research (SurveyMonkey). Pilot Learning Community Peer Leader I-LEAP

July - August 2009: Software testing

September 2009: I-LEAP survey launch. (A select number of I-LEAP surveys will include student self-assessment)

I-LEAP Service LearningUniversity of CincinnatiI-LEAP (Integrated Learning Experiential Assessment Program)Community Partner Assessment of Student PerformanceService Learning & Community Engagement Learning OutcomesFall 2008

3.99

3.97

4.22

4.16

4.22

4.22

3.8 3.85 3.9 3.95 4 4.05 4.1 4.15 4.2 4.25

Awareness of social issues

Ability to discuss social issues

Acknowledges the value ofdiverse opinions, talents, and

backgrounds

Understands and works withinthe culture of the group

Respects diversity

Recognizes political and socialimplications of actions

Le

arn

ing

Ou

tco

me

Score

Scale: 5= Excellent 4= Good 3= Satisfactory 2= Unsatisfactory 1= Poor N/A= not applicable

Note: The data in this chart includes the assessment of 68 undergraduate (multi-disciplinary) students who participated in course-related service learning projects with 5 community partners (local and international).

I-LEAP Study AbroadUniversity of CincinnatiI-LEAP: Integrated Learning Experiential Assessment ProgramFaculty Assessment of Students' Study Abroad ExperienceGlobal Competence Learning OutcomesSpring 2009

4.10

4.06

4.47

4.83

4.04

4.17

4.11

4.10

4.47

4.13

1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Awareness of global social issues

Discuss cultural issues from an informed perspective

Acknowledge value of diverse opinions

Accommodate cultural differences while interacting w/others

Acceptance of a different culture as a result of this experience

Diverse cultural frame of reference

Understand own culture within global context

Understand local culture through interactions and observations

Respects diversity

Recognize political and social implications of actions

Le

arn

ing

Ou

tco

me

s

Mean ScoreThis survey is a pilot program for I-LEAP. The data on this chart represents 34 students (14 Business, 20 College Conservatory of Music/Multi) who took part in a week-long study abroad experience. Business: European study of opportunities & challenges of doing business by exploring business/politics/culture. CCM: Study tour of Florence, Italy including CCM & multi-discipline honor students.

Scale:5= Excellent4= Above Average3= Satisfactory2= Unsatisfactory1= PoorN/A= not applicable

Challenges & Lessons Learned• With the goal of individual student learning outcome data, we will probably need to shorten

the survey. (i.e. 20 individual surveys would take over 3 hours to complete)

• While university “broad brush” assessment data (rolling averages) maybe best for data validity, the program and course level information is essential for continuous curricular improvement

• Training, assessor “buy-in”, and university mission support of engaged learning are keys to success

• Assessors want to choose their questions (from a pool of questions) instead of selecting “n/a” for several questions

• The “overall performance score” of the Study Abroad I-LEAP is lower than any of the other learning outcomes assessed. The question becomes “Are there any questions that we have missed?”

• The Service Learning students scored higher in acknowledging diverse opinions yet lower scores in the ability to discuss social issues. (When the faculty focus group met in fall 2008 to define the modifications to the I-LEAP survey, they had mentioned this challenge of teaching the student to articulate/think critically concerning social responsibility issues)

Promises

• The ability to increase our sources of internal and external validation of student achievement of General Education baccalaureate competencies or specific skills

• The ability to track individual student progress in different disciplines and types of experiential learning

• The ability to analyze “touch points” (FY, Mid-Collegiate, Capstone) for validation of value-added learning and for continuous curricular improvement

• The ability to authentically assess the “big picture” of UC’s Integrated Core Learning and its impact on students’ growth and learning transformation