The STARS Alliance Students & Technology in Academics, Research, and Service STARS Alliance...

download The STARS Alliance Students & Technology in Academics, Research, and Service STARS Alliance Evaluation Tiffany Barnes Evaluation Team Kim Buch, Audrey

of 40

  • date post

    27-Mar-2015
  • Category

    Documents

  • view

    214
  • download

    0

Embed Size (px)

Transcript of The STARS Alliance Students & Technology in Academics, Research, and Service STARS Alliance...

  • Slide 1

The STARS Alliance Students & Technology in Academics, Research, and Service STARS Alliance Evaluation Tiffany Barnes Evaluation Team Kim Buch, Audrey Rorrer Web: Anthony Chow Qualitative & PP: Sally Berenson Slide 2 The STARS Alliance Students & Technology in Academics, Research, and Service 2 Barnes, Eval Team NSF Site Visit 2007 Evaluation Successes: Refined approach to evaluate Organizational, program, individual level outcomes Instrument development Combined qualitative and quantitative measures for: SLC Alliance effectiveness Articulated new goals and measures (STARS/Computing Identity) Challenges: Changing team New research questions arise all the time Comprehensive program has many goals - too many surveys for same people Data collection from different people/institutions Small team Slide 3 The STARS Alliance Students & Technology in Academics, Research, and Service 3 Barnes, Eval Team NSF Site Visit 2007 Impact on pipeline Q1: The STARS Alliance features a broad scope (middle school to graduate school) across three categories of population (gender, ethnic, disability), with a variety of outreach, research, mentoring, and career counseling components. Characterize, quantitatively if possible, what has been attempted within this scope of activity and what has been achieved to date. To what extent have each of the target groups been reached? Each of the pipeline stages? Approximately 110 college SLC students each year About 40% returning in Year 2 Over 500 K-12 students at outreach events Over 500 parents Over 100 teachers and counselors > 5 major community events (AU, FSU, UNCC) Slide 4 The STARS Alliance Students & Technology in Academics, Research, and Service 4 Barnes, Eval Team NSF Site Visit 2007 Measures Q3: What new measures of impact/outcome have been developed to capture what the Alliance is seeking to achieve, i.e., beyond traditional pre- post test measures appropriately disaggregated (by gender, ethnicity, disability, and cohort) such as GPA, year-to-year retention, and degree attainment? For example, how have the observations in students' electronic journals been used by the Alliance institutions and Advisory Boards? Selected interviews each semester Journaling currently not being analyzed Climate survey to measure impact of STARS on computing culture at schools New outreach evaluation instruments Slide 5 The STARS Alliance Students & Technology in Academics, Research, and Service 5 Barnes, Eval Team NSF Site Visit 2007 Comparison Q7: What data is the Alliance using for comparison purposes? Are they national sources, local institutional information on students, some combination, or other? Explain. Comparison of applicants, enrollments across years Comparison with NSF and Taulbee survey data Slide 6 The STARS Alliance Students & Technology in Academics, Research, and Service 6 Barnes, Eval Team NSF Site Visit 2007 Celebration impacts Q4: Describe the benefits of the summer workshop to different categories of participant. Faculty- networking, project collaborations, information exchange, e.g. o Junior Faculty Round Table o Implementing Pair Programming Slide 7 The STARS Alliance Students & Technology in Academics, Research, and Service 7 Barnes, Eval Team NSF Site Visit 2007 STARS Celebrations The STARS Celebration emphasizes STARS core values of excellence, leadership, civic engagement, service, and community values intended to foster student success. 2007 at UNC Charlotte 113 students Over 40 faculty Over 50 partners and community 2006 at Georgia Tech 105 students 23 faculty/staff Slide 8 The STARS Alliance Students & Technology in Academics, Research, and Service 8 Barnes, Eval Team NSF Site Visit 2007 STARS Celebration Results Students felt the Celebration provided opportunities for community building. (2006: 97%, 2007: 92% ) Students learned how computing professionals can improve quality of life. (2006: 83%, 2007: 89%) Students learned about disparity of representation in IT: by gender and race (2006: 98%, 2007: 88%) by persons with disabilities (2006: 86%, 2007: 69%) Students had ample time to select leadership assignments to fit their professional needs (2006: 80%, 2007: 88%) Slide 9 The STARS Alliance Students & Technology in Academics, Research, and Service 9 Barnes, Eval Team NSF Site Visit 2007 Demonstration Projects The STARS Alliance serves as an incubator for new demonstration projects and the scaling and replicating of best practices among the diverse alliance institutions. STARS Leadership Corps Pair Programming Tiered mentoring model Teaching Math to the Visually Impaired African American Researchers in Computer Science Culturally Situated Design Tools Slide 10 The STARS Alliance Students & Technology in Academics, Research, and Service 10 Barnes, Eval Team NSF Site Visit 2007 STARS Leadership Corps Model Slide 11 The STARS Alliance Students & Technology in Academics, Research, and Service 11 Barnes, Eval Team NSF Site Visit 2007 STARS Tiered Mentoring and Role Models High school student Middle school student Professional or Grad student Undergrad student Regional Partnerships - A computing community from Kindergarten to the Workforce Slide 12 The STARS Alliance Students & Technology in Academics, Research, and Service 12 Barnes, Eval Team NSF Site Visit 2007 Evaluation Three-Tiered Outcomes Model: Organizational Outcomes (e.g., Alliance) Program Level Outcomes (e.g., Demonstration Projects) Individual Level Outcomes (e.g., Students) Evaluation model: CIPP Context (setting of project) Input (project resources) Process (activities & strategies) Product (ultimate results) Slide 13 The STARS Alliance Students & Technology in Academics, Research, and Service 13 Barnes, Eval Team NSF Site Visit 2007 Alliance Outcomes Increased student enrollment in computing and IT programs Increased student awareness about computing and IT Increased student readiness to enter computing and IT graduate school and workforce Increased participation of undergraduates entering computing and/or IT graduate school or workforce Increased graduate rates of computing and IT students Increased persistence and declaration of students majoring in computing or IT Increased college adjustment and GPA for students in computing or IT Slide 14 The STARS Alliance Students & Technology in Academics, Research, and Service 14 Barnes, Eval Team NSF Site Visit 2007 Alliance Outcomes Sustained Alliance Efficacy Institutionalize Alliance Partnerships Increased national awareness of effective practices for Alliance development Serve as a model and repository for BPC Slide 15 The STARS Alliance Students & Technology in Academics, Research, and Service 15 Barnes, Eval Team NSF Site Visit 2007 Year 1 Accomplishments Steering Committee: Jan. 2006-2007, & Aug. 2006-2007 Regular teleconferences for planning & update STARS Celebration Successful Annual Conference held in August 06-07 2006: 105 students, 23 faculty, 2007: 113 students, >40 faculty 2007: New evening for partners, >50 additional attendees Student Leadership Corps 2 universities implemented in Summer 2006 All implemented in Fall 2006, Spring 2007, Fall 2007 Workshops Pair Programming, Mentoring, IBPC: Jan & Aug 2007 CSDT at Celebration Aug 2007 Qualitative methods workshops Aug 06-07, April 2007 Slide 16 The STARS Alliance Students & Technology in Academics, Research, and Service 16 Barnes, Eval Team NSF Site Visit 2007 Accomplishments Web portal: StarsAlliance.org Evaluation Instrument development IRB exploration: applying for waivers for K-12 outreach evaluation Interviews & Training Coordination with UNC Charlotte REU Site Faculty recruiting at UNC Charlotte! Slide 17 The STARS Alliance Students & Technology in Academics, Research, and Service Results Slide 18 The STARS Alliance Students & Technology in Academics, Research, and Service 18 Barnes, Eval Team NSF Site Visit 2007 Institutional Enrollment Slide 19 The STARS Alliance Students & Technology in Academics, Research, and Service 19 Barnes, Eval Team NSF Site Visit 2007 SLC 06-07 Participants SLC Participants FemalesMalesTotal African American 20%23%43% Asian 7% Caucasian 29%19%48% Total 58%42%100% Slide 20 The STARS Alliance Students & Technology in Academics, Research, and Service 20 Barnes, Eval Team NSF Site Visit 2007 2006-2007 SLC 107 students participated in 2006-2007 STARS Leadership Corps Slide 21 The STARS Alliance Students & Technology in Academics, Research, and Service 21 Barnes, Eval Team NSF Site Visit 2007 SLC Survey Results Retention of SLC students: Over 95% are forming meaningful peer & faculty relationships 88% more committed to computing majors 94% interested in graduate education 83% felt the SLC helped them develop computing skills 91% would recommend the SLC to others Recruiting: 90% are helping others understand the value of computing Hundreds of K-12 students have participated in outreach events Building a like community: 88% felt the SLC gave them opportunities to work with people like themselves Slide 22 The STARS Alliance Students & Technology in Academics, Research, and Service 22 Barnes, Eval Team NSF Site Visit 2007 Who is SLC? NOT the average UG student in Computing Older, average age= 24 years Higher GPA in Computing, M=3.3 Higher SES 85% middle class 64% had parents with at least some college; 53% were college graduates Much more diverse Women 51% Under-represented minorities 62% Physical disability 14% Extremely confident that they will obtain UG degree (4.7 on 5-pt scale) Very likely to attend graduate school 66% believe they will obtain Masters 50% believe they will obtain PhD Slide 23 The STARS Alliance Students & Technology in Academics, Research, and Service 23 Barnes, Eval Team NSF Site Visit 2007 SLC & Computing 93% are confident that Computing is the right major 87% plan to st