Evaluation
24
Evaluation Evaluation EnAct Campuses Sonoma, Humboldt, Chico, San Francisco, San Jose, Fresno, Bakersfield, Pomona Presentation July 11, 2007 Public Works, Inc. Mikala L. Rahn, PhD Patty O’Driscoll, MPA
description
Evaluation. EnAct Campuses Sonoma, Humboldt, Chico, San Francisco, San Jose, Fresno, Bakersfield, Pomona Presentation July 11, 2007 Public Works , Inc. Mikala L. Rahn, PhD Patty O’Driscoll, MPA. Who Are We?. - PowerPoint PPT Presentation
Transcript of Evaluation
EvaluationEvaluation
EnAct CampusesSonoma, Humboldt, Chico, San Francisco,
San Jose, Fresno, Bakersfield, Pomona
PresentationJuly 11, 2007
Public Works, Inc.Mikala L. Rahn, PhDPatty O’Driscoll, MPA
Who Are We?Who Are We?
Public Works, Inc. a nonprofit in Pasadena dedicated to working with communities, government, schools and parents by providing services and resources to educate and inform children, youth and families. Our work is in three areas:
• Education Reform• Workforce Development• Intervention/Prevention
Evaluation GoalsEvaluation Goals
Measure the success of participating faculty and their students with disabilities overtime as well as compared to students without disabilities on a number of measures
Monitor the project based on federal requirements and offer program improvement strategies
Improve the consistency of measurement across the institutions
Research QuestionsResearch Questions
1) Which of the 14 elements of the Universal Design for Learning (UDL) are most important or are they all equal?
2) How does faculty training in UDL impact student resilience (persistence/graduation)? Are there more appropriate courses or faculty to target to affect resilience?
Research QuestionsResearch Questions
3)Why do faculty participate? 4)How is the use of online and other
technologies cost-effective, beneficial, and scalable for rural and remote regions serving the target populations?
5)Did the project reach the target segments of the CSU system with information about research and developments in effective teaching of students with disabilities?
Balanced Evaluation Balanced Evaluation ApproachApproach
Process measures focus on how a program is implemented and the outcome measures focus on the results of the program or intervention.
Public Works, Inc. is using both quantitative and qualitative data collection methods
MethodologyMethodology
Baseline data from participating institutions; annual data collection
Attendance data at workshops and institutes
Preliminary Assessment of Faculty Implementation of UDL
UDL Training EvaluationFaculty Activity Survey/Assessment Course ArtifactsWeb user statistics
Federal Grant Performance Report Federal Grant Performance Report (GPR)(GPR)
Objectives…Measures...Targets...Performance Data
• The difference between the rate at which students with documented disabilities complete courses taught by faculty trained in project activities, and the rate at which other students complete those courses.
• The percentage of faculty trained in project activities that incorporate elements of training into their classroom teaching.
Federal Grant Performance Report Federal Grant Performance Report (GPR)(GPR)
Objectives…Measures...Targets...Performance Data
• Collaboration to Ensure Access to a Quality Postsecondary Education
• Technology to Ensure Access to a Quality Postsecondary Education
• Dissemination of EnACT Content and Processes
Program ImprovementProgram Improvement
How do we better serve students with disabilities?
How do we better support faculty?
What do we need to know and be able to do?
Overall Evaluation GoalsOverall Evaluation Goals
• Keep the grant by meeting the requirements
• Learn lessons from each other
• Support faculty and students
Summary of ResultsSummary of ResultsAnnual Performance ReportAnnual Performance Report
Goal 1: Collaboration• EnACT Leadership Team, Advisory
Committee, and campus-based Communities of Practice established
• CSU Fresno, Bakersfield and Pomona to join current collaborative
• CSU ATI Initiative partnership at 2007 Institute
• UDL workshops attended by 194 attendees-target was 100
Summary of ResultsSummary of ResultsAnnual Performance ReportAnnual Performance Report
Goal 2: Technology to ensure access
• Enhancement of AIM resource specifications for authoring
• 13 AIM resources proposed or published to date
Summary of ResultsSummary of ResultsAnnual Performance ReportAnnual Performance Report
Goal 3: Dissemination of EnACT• Enhancements of Web site• Dissemination at 11 professional
conferences or Webcasts
Summary of ResultsSummary of ResultsAnnual Performance Report Annual Performance Report HighlightsHighlights
Project Specific Performance Measures: Course Completion Rate
• Students with disabilities completed at 100% (increase of 7% from baseline) n=9
• Students without disabilities completed at 96% n=328
• GPA for students with disabilities was 3.22 (increase of .22 from baseline)
• GPA for students without disabilities was 3.26
Summary of ResultsSummary of ResultsAnnual Performance Report Annual Performance Report HighlightsHighlights
Project Specific Performance Measures: Faculty participation
• Faculty participation across campuses totaled 34, above target of 25
• 73% of faculty agreed they would “more likely make changes to their courses” after exposure to UDL workshops (target was 60%)
• All participating faculty reported making changes to their courses as a result of participation
Summary of ResultsSummary of ResultsFaculty Interview HighlightsFaculty Interview Highlights
Changes to Courses• Changes the result of the combination of
training they received, not individual components
• Changes ranged from revisions to teaching strategies to more technologically complex (multi-media, Web CT)
• Prompted by desire to make course accessible to all students rather than for accommodations to individual students
Summary of ResultsSummary of ResultsFaculty Interview HighlightsFaculty Interview Highlights
Sample of course changes• Students acting out lessons or concepts• Improvements to posting on the
Internet• Revisions to Syllabi using EnACT info• Revising PowerPoint to be more
accessible• Study guides, guided lecture notes,
assigning all students to be note takers• Using rubrics and adding new
assessment strategies
Summary of ResultsSummary of ResultsFaculty Interview HighlightsFaculty Interview Highlights
Faculty Feedback about EnACT• Each component received positive
feedback: overview from UDL I/II, specific information from Institute, and collaboration in Faculty Learning Community
• Changes to courses had positive impact on teaching and learning; support from EnACT prepared them to continue to make changes
Summary of ResultsSummary of ResultsFaculty Interview HighlightsFaculty Interview Highlights
Faculty Feedback about EnACT• Would like support in implementation of
technology• Would like continued opportunities for
collaboration that occurred during FLC
Evaluation RecommendationsEvaluation Recommendations
• Continue to examine ways to implement EnACT to impact larger numbers of students with disabilities
• Use lessons learned from EnACT to implement ATI initiative and expand impact on campus
• Examine the impact, use and usefulness of AIMs
Evaluation Next StepsEvaluation Next Steps
• Orient new campuses to evaluation requirements at Leadership meeting
• Work with campuses to continue to streamline data collection processes
• Survey students in Fall 2007• Collect Cohort II implementation data in
Spring 2008 (Fall 2007 grades)• Collect Cohort III baseline data in Spring
2008 (Fall 2007 grades)
Contact UsContact Us
Mikala L. Rahn, PhDPresidentPublic Works, Inc. 90 N. Daisy AvenuePasadena, CA 91017(626) 564-9890(626) [email protected]
Contact UsContact Us
Patty O’DriscollConsultantPublic Works, Inc. 1191 Loma Ct. Sonoma, CA 95476(707) 933-8219(707) 996-8726 [email protected]
