Using Data to Improve Student SuccessFaculty Development Model - Competency-Based Education
-
Upload
becky-lopanec -
Category
Education
-
view
67 -
download
0
Transcript of Using Data to Improve Student SuccessFaculty Development Model - Competency-Based Education
Using Data to Improve Student Success
Thursday, June 4, 2015 Jason Levin, WGU
Bruce Clayton & Chris@ Amato, Sinclair Community College Dena Laney, Bellevue College
This product was funded in part by a grant awarded by the U.S. Department of Labor’s Employment and Training Administra@on. The product was created by the grantee and does not necessarily reflect the official posi@on of the U.S. Department of Labor. The Department of Labor makes no guarantees, warran@es, or assurances of any kind,
express or implied, with respect to such informa@on, including any informa@on on linked sites and including, but not limited to, accuracy of the informa@on or its completeness, @meliness, usefulness, adequacy, con@nued availability, or ownership.
Western Governors University Salt Lake City, Utah
Data Feedback Cycles at WGU Ins@tu@onal Research, 06.04.2015
About WGU The principal mission of Western Governors University is to improve quality and expand access to post-‐secondary educa;onal opportuni;es by providing a means for individuals to learn independent of ;me and place and to earn competency-‐based degrees and other creden;als that are credible to both academic ins;tu;ons and employers.
• 59,000 students, growing at 20% annually • Online, non-‐profit • Regionally and na@onally accredited • 76% of students from underserved category (Rural, Minority, Low Income, First
Genera@on College Student) • All students are full @me • 24% Masters, 76% Bachelors • Four colleges:
• Business (35%) • Health Professionals (19%) • Informa@on Technology (17%) • Teachers College (29%) ** Only NCATE accredited online college **
• Six month terms, new term star@ng on the first of every month • Tui@on is $2,890 per term • Competency based, self-‐paced, asynchronous
Provide differen@al treatments to students based on risk classifica@on
Create proac;ve vs. reac;ve protocols Use all available data about a student at any given @me to classify student risk based on the probability making term OTP Classifica@on of students into high, medium, and low risk categories is dynamic and influenced by recent behavior
Student – Faculty Feedback Student Risk Classifica@on and Interven@on
5
Ins@tu@onal Feedback Loop Harris Interac@ve Survey of Alumni
WGU (A)
National (B)
Public 4 Year (C)
Current employment status % % % Total employed 89%
B 84% 86%
Employed full-‐time 78% 74% 77% Employed part-‐time 8% 6% 5% Self-‐employed 3% 4% 4% Total not employed 11% 16% 14% Not employed, but looking for work 7% 8% 7% Not employed and not looking for work/retired/homemaker/student 4% 8%
A 7%
A
Employed in same field as degree earned (among currently employed) 83% 81% 83%
Graduates 2006-‐2009 Graduates 2010-‐2012
WGU (A) National (B)
Public 4 year (C) WGU (D) National (E)
Public 4 year (F)
NET CHANGE in income from prior to earning degree to present
+ $18,600 + $12,400 + $16,600 + $9,000 + $8,500 + $8,400
Source: WGU Harris Study, 2012
12
CBE4CC
Ins@tu@onal & Programma@c Context
• Large size. Sinclair serves approximately 20,000 students per fall and spring term.
• Significant non-‐tradi3onal student popula3on.
• CBE program: self-‐paced and online.
CBE4CC
Challenges
• Suppor@ng large numbers of students with limited student-‐support resources
• Providing adequate academic support to students who have been away from the classroom and who have compe@ng demands on their @me
• Compensa@ng for an instruc@onal model that limits interac@ons between students and faculty and access to on-‐campus student-‐support services
CBE4CC
Student Performance Repor@ng: Essen@al Requirements • Caseload based • Sensi@ve to known predic@ve indicators • Real-‐@me, ac@onable course-‐performance informa@on • At-‐a-‐glance usability • Fully automatable genera@on and delivery
CBE4CC
Performance Repor@ng Within LMS
• Basic course-‐sec@on-‐level repor@ng • Course-‐sec@on specific • Informa@on segmented across mul@ple reports • Restricted access • Poten@ally insensi@ve to key performance metrics
• Enhanced repor@ng through LMS analy@cs packages • More flexible, but ojen limited to pre-‐defined report elements
• Ojen available as an LMS add-‐on at addi@onal cost
CBE4CC
Moving Beyond the Standard End-‐User Experience • Establish the structure of a useful student-‐performance report • Repor@ng “wish list”
• Iden@fy the data systems and sources that contain the relevant informa@on • Student informa@on system/ERP • Case management system • Learning management system
• Leverage available resources to assemble and deliver final report • Office of Research, Analy@cs & Repor@ng, staffed by team of report writers, data analysts, researchers, and database administrators
• Data warehouse and sophis@cated data management sojware
CBE4CC
Student Informa3on Course Informa3on Risk Indicators
Last Name
First Name
Tartan ID Course Term Enrollment
Status Week Term Progress
Course Progress
Mid-‐Term Exam Status
Average Grade
Final Grade
Unsuccessful ALempt On Record
Mul3ple Unsuccessful ALempts On Record
Logins Last 7 Days
Submissions Last 7 Days
Risk Level Comments
Hilton Davis 0652665 BIS-‐1120 15/SP Enrolled 7 35% 0% N/A . . No No 5 0 High No Submissions In Week 1
Rasor Robert 0564225 MAT-‐2170 15/SP Enrolled 7 35% 0% N/A 71.00% . No No 1 0 High Very Slow Progress
Raston Eric 0133647 CIS-‐1510 15/SP Enrolled 7 35% 52% Pass 98.78% . No No 2 0 Moderate Low Ac@vity
Raston Eric 0133647 CIS-‐2640 15/SP Enrolled 7 35% 36% . 96.90% . No No 1 0 Moderate Low Ac@vity
Varney Lisa 0369845 MAT-‐2170 15/SP Enrolled 7 35% 0% N/A . . No No 1 0 High No Submissions In Week 1
Vinton Samantha 0975632 BIS-‐1120 15/SP Enrolled 4 21% 0% N/A . . No No 0 0 High No Submissions In Week 1
Vinton Samantha 0975632 CIS-‐1111 15/SP Enrolled 4 21% 3% . 100.00% . No No 0 0 Moderate Low Ac@vity
Vinton Samantha 0975632 CIS-‐1130 15/SP Enrolled 4 21% 5% . 90.00% . No No 0 0 Moderate Low Ac@vity
Vinton Samantha 0975632 CIS-‐1107 15/SP Enrolled 4 21% 13% . 100.00% . No No 2 2 Low
Aston Noah 0512897 CIS-‐1140 15/SP Enrolled 7 35% 18% N/A 90.00% . No No 6 0 Moderate Low Ac@vity
Barton Emma 0221554 CIS-‐1510 15/SP Enrolled 7 35% 66% Pass 96.82% . No No 0 0 Moderate Low Ac@vity
Bosco James 0398521 CIS-‐1140 15/SP Enrolled 7 35% 18% N/A 107.50% . No No 1 0 Moderate Low Ac@vity
Bosco James 0398521 CIS-‐2416 15/SP Enrolled 7 35% 70% N/A 99.50% . No No 13 14 Low
Brunswik Sylvia 0021056 CIS-‐2426 15/SP Enrolled 7 35% 37% N/A 99.43% . No No 2 0 Moderate Low Ac@vity
Clancy Thomas 0648875 CIS-‐1411 15/SP Enrolled 7 35% 24% N/A 99.62% . No No 0 0 Moderate Low Ac@vity
Clancy Thomas 0648875 CIS-‐1140 15/SP Enrolled 7 35% 73% N/A 95.31% . No No 7 4 Low
Colton Daniel 0021557 CIS-‐1107 15/SP Enrolled 7 35% 25% . 100.00% . No No 0 0 Moderate Low Ac@vity
Davis Benjamin 0364987 CIS-‐2212 15/SP Enrolled 7 35% 44% . 92.53% . No No 1 0 Moderate Low Ac@vity
Hanson Holly 0864135 CIS-‐1111 15/SP Enrolled 4 21% 24% . 100.00% . No No 1 0 Moderate Low Ac@vity
Hanks Kendra 0215477 CIS-‐2640 15/SP Enrolled 4 21% 17% . 93.31% . No No 0 0 Moderate Low Ac@vity
Harris Stephanie 0677954 CIS-‐1510 15/SP Enrolled . . 100% . 94.81% A . . . . . .
Accelerate IT Caseload Performance Report Academic Coach: Chris@na Amato
CBE4CC
Lessons Learned
• Develop a full partnership between student-‐support and IR/IT personnel
• Maximize the capabili@es of exis@ng data systems in ways that enhance and transcend standard end-‐user applica@ons
• Secure faculty buy-‐in • Know and engage other impacted stakeholders
CBE4CC
About Bellevue’s Competency Based Program • Launched 6 month pilot project in January 2014 for our Business Sojware Specialist cer@ficate program • Consists of 6 courses + 1 required orienta@on course completed in a quarter system • Accepts both majors and non-‐majors who need the course as part of another program. • Program con@nued ajer pilot project – averaging about 100 student enrollments per quarter and growing
CBE4CC
Navigator Student Support
• Handles enrollment interviews • Provides entry codes for our students to register for courses
• Teaches the orienta@on course • Provides support and checks in with students throughout the quarter
• Collects lots of data stored in many different places • SmarterMeasure Test Results • Interview Notes • Entry Codes/Interest Lists • Progress Reports from Students • Progress Updates from Course Learning Management System/Instructor
CBE4CC
Faculty Student Support
• Routes inquiries to Navigator for enrollment/interview • Works with Navigator to issue entry codes • Provides support and checks in with students throughout the quarter
• Collects lots of data stored in many different places • Canvas Learning Management System (par@cipa@on, page views, assignment comple@on, interac@on with Instructor, last logged in)
• Other course sojware such as MyITLab progress • Feedback from Student Navigator
CBE4CC
The Problem: Managing our Data
Ajer the first quarter of our pilot project, we knew we had to get a handle on the enormous amount of data that we were collec@ng to ensure our students were on track and would be successful comple@ng the course.
CBE4CC
Navigator Data? Many files in OneDrive
*this is only about 1/3 of the files (mostly
excel) from the past 17 months!
CBE4CC
Faculty Solu@on: Lots of hand wriUen notes! My first idea to track progress was to keep lots of notes which quickly became a problem.
They weren’t very prety. Or very func@onal. But they helped me iden@fy the students that were not on track.
CBE4CC
Learning Management Analy@cs A tradi@onal online course analy@cs – Note my Sunday deadlines
A competency based course with varying start dates – no official deadlines
CBE4CC
Communica@on Breakdown
There was a communica@on breakdown between: • Faculty and Student • Navigator and Student • Faculty and Navigator • Everyone and data
Too much @me was spent trying to answer simple ques@ons – • “Is this student enrolled in mul@ple competency based courses?” • If so, are they performing well in those courses?
CBE4CC
Implemen@ng a Solu@on
We needed a solu@on that helped us track all of this data • Without spending hours trying to locate the spreadsheet or progress report on the OneDrive • Without trying to locate my hand writen notes or my own self created progress spreadsheets • Without having to email constantly back and forth with the navigator to check in on a student • Allows for a quick lookup, progress and note tracking
CBE4CC
Other Data Tracked • Interested Student Contact Informa@on & Notes • File Upload for SmarterMeasure test results & Orienta@on assessment • Progress Report Web Based Form to allow students to quickly update us • Automated Progress Report Notes from Student
CBE4CC
Wish List/Pending Development
• Import data from Course Learning Management Sojware • Import data from external tools (MyITLab, Cer@fica@on Prac@ce Exams) • Student Portal for feedback, entry code request & course road map view • Extensive analy@cs/averages per class to iden@fy problems • Automated student alert system for progress updates/concerns
CBE4CC
What we have learned • Managing student data is a MUST for a CBE program to support the faculty, student navigator and the student’s overall success! • As the program grows, it is very challenging to keep track of which students are coming and going • Having a tool in place from the beginning will save a lot of data entry! • Being able to u@lize the tool saves a lot of @me searching for informa@on and follow up between Faculty and Student Navigator and Student. • Allows faculty to be informed when communica@ng with students and see the big picture. • Everyone can be on the same page!
CBE4CC
Considera@ons • What problem or problems can data help your ins@tu@on solve?
• Who at your ins@tu@on needs to be involved in formula@ng solu@ons from data?