PEEF Interim Evaluation Report - SFUSD: Home Immediate Outcomes ... Arts and Music ... PEEF 2011-12...

Public Education Enrichment Fund (PEEF)

2011–12 Interim Evaluation Report

Issued by: PEEF Office, SFUSD December 20, 2012

Prepared by:

Monica E. Lopez, PhD, Program Analyst, Public Education Enrichment Fund

Steve Newton, PhD, Senior Statistician, Research, Planning, and Accountability Office

Kathleen Fleming, Supervisor, Public Education Enrichment Fund

Christopher Armentrout, Director of Development and Local Government Relations

PEEF Final 2010-11 Evaluation Report, December 20, 2012 2

Contents List of Figures .............................................................................................................................................. iii

List of Tables ............................................................................................................................................... iii

Executive Summary ..................................................................................................................................... iv

PEEF Outputs............................................................................................................................................ v

PEEF Immediate Outcomes ..................................................................................................................... vi

PEEF Long-Term Outcome: Academic Achievement ............................................................................. vi

Future Steps ............................................................................................................................................ vii

Report Overview ........................................................................................................................................... 1

Public Education Enrichment Fund Background .......................................................................................... 1

SRI International Recommendations ............................................................................................................ 3

Methods ........................................................................................................................................................ 9

Inputs: What does the PEEF initiative provide? ......................................................................................... 10

Outputs: What are the levels of Access and Equity related to the PEEF initiative? ................................... 13

Outputs 1: Professional and Instructional Capacity and Quality ............................................................ 13

Outputs 2: School Climate ...................................................................................................................... 16

Outputs 3: School Engagement ............................................................................................................... 17

Outputs 4: Student Physical and Mental Health ..................................................................................... 19

Outputs 5: Academic Achievement ........................................................................................................ 22

Outcomes: What are the outcomes associated with the PEEF initiative? ................................................... 24

Immediate Outcomes 1: Professional and Instructional Capacity and Quality ....................................... 24

Immediate Outcomes 2: School Climate ................................................................................................ 28

Immediate Outcomes 3: School Engagement ......................................................................................... 30

Immediate Outcomes 4: Student Physical and Mental Health ................................................................ 32

Long-Term Outcomes: Academic Achievement ........................................................................................ 35

Propensity Score Matching Study: Effects of Middle School and High School Athletics on Attendance, GPA and Suspension ................................................................................................................................... 38

Conclusion .................................................................................................................................................. 41

PEEF Outputs.......................................................................................................................................... 41

PEEF Immediate Outcomes .................................................................................................................... 41

PEEF 2011-12 Interim Evaluation Report, December 20, 2012 ii

PEEF Long-Term Outcome: Academic Achievement ............................................................................ 42

Future Steps ............................................................................................................................................ 43

Appendix A: City Charter Section 16.123 1-10 .......................................................................................... 44

Appendix B: 2011-12 PEEF Initiative Programs Descriptions and Activities ........................................... 52

Physical Education .................................................................................................................................. 52

Athletics .................................................................................................................................................. 52

Libraries .................................................................................................................................................. 53

Arts and Music ........................................................................................................................................ 54

Student Support Professionals ................................................................................................................ 55

Peer Resources ........................................................................................................................................ 56

Wellness Initiative .................................................................................................................................. 56

Restorative Practices ............................................................................................................................... 56

Career Technical Education .................................................................................................................... 57

Teacher Academy ................................................................................................................................... 57

Formative Assessment System ............................................................................................................... 58

Translation and Interpretations Services ................................................................................................. 58

Custodial Services ................................................................................................................................... 58

Human Capital Support (formerly, Teacher Recruitment) ..................................................................... 58

Appendix C: PEEF Initiative Data Charts .................................................................................................. 59

Outputs 1: Professional and Instructional Capacity and Quality ............................................................ 60

Outputs 2: School Climate ...................................................................................................................... 63

Outputs 3: School Engagement ............................................................................................................... 64

Outputs 4: Student Physical and Mental Health ..................................................................................... 67

Outputs 5: Academic Achievement ........................................................................................................ 71

Outcomes 1: Professional and Instructional Capacity and Quality ......................................................... 73

Outcomes 2: School Climate .................................................................................................................. 75

Outcomes 3: School Engagement ........................................................................................................... 76

Outcomes 4: Student Physical and Mental Health .................................................................................. 77

Outcomes 5: Academic Achievement ..................................................................................................... 78

Effects of Middle School and High School Athletics on Attendance, GPA and Suspension ................. 80

PEEF 2011-12 Interim Evaluation Report, December 20, 2012 iii

List of Figures 1. PEEF Initiative Funding Allocations ...................................................................................................... 2

2. Public Education Enrichment Fund (PEEF) Initiative Logic Model ...................................................... 8

List of Tables 1. Sports, Libraries, Arts and Music Budget 2010–11 ............................................................................. 11

2. Other General Uses Budget 2010–11 ................................................................................................... 12

3. PEEF Initiative Outputs Related to Professional and Instructional Capacity and Quality: Change Over Time ............................................................................................................................... 14

4. PEEF Initiative Outputs Related to School Climate: Change Over Time ............................................ 16

5. PEEF Initiative Outputs Related to School Engagement: Change Over Time ..................................... 18

6. PEEF Initiative Outputs Related to Student Physical and Mental Health: Change Over Time ........... 20

7. PEEF Initiative Outputs Relevant to Academic Achievement: Change Over Time ............................ 23

8. PEEF Initiative Outcomes Related to Professional and Instructional Capacity and Quality: Change Over Time ............................................................................................................................... 25

9. PEEF Initiative Outcomes Related to Professional and Instructional Capacity and Quality: Snapshots ......................................................................................................................... 27

10. PEEF Initiative Outcomes Related to School Climate: Change Over Time e ...................................... 28

11. PEEF Initiative Outcomes Related to School Climate: Snapshots ....................................................... 29

12. PEEF Initiative Outcomes Related to School Engagement: Change Over Time ................................. 30

13. PEEF Initiative Outcomes Related to School Engagement: Snapshots ................................................ 31

14. PEEF Initiative Outcomes Related to Student Physical and Mental Health: Change Over Time ........ 33

15. PEEF Initiative Outcomes Related to Student Physical and Mental Health: Snapshots ...................... 34

16. PEEF Initiative Outcomes Related to Academic Achievement: Change Over Time ........................... 36

17. PEEF Initiative Outcomes Related to Academic Achievement: Snapshots ......................................... 37

18. Attendance, GPA, and Suspensions for High School Athletes and Matched Non-Athletes ................ 39

19. Attendance, GPA, and Suspensions for Middle School Athletes and Matched Non-Athletes ............. 40

PEEF 2011-12 Interim Evaluation Report, December 20, 2012 iv

Executive Summary In March of 2004, San Francisco voters approved the ballot initiative Proposition H. The passage of Proposition H established the Public Education Enrichment Fund (PEEF) as law within the City Charter, Section 16.123.1-10 (Appendix A). The passage of Proposition H came at a time when district resources had significantly diminished due to state and federal funding.

PEEF provides funding to improve the quality of education for the youth of San Francisco and is shared by both First 5 San Francisco and SFUSD. One third of the total PEEF fund is allocated to First 5 San Francisco for preschool support; another third supports SFUSD sports, libraries, the arts and music (SLAM); and the last third funds programs such as Student Support Professionals, Wellness Centers, Peer Resources, Restorative Practices, and Translation Services.

This report presents an evaluation of the PEEF initiative using data through the 2011–12 school year.

Given the complexity of the programs supported by the PEEF initiative, the PEEF Office consulted with SRI International (SRI), for guidance on the assessment and communication processes. The specific work with SRI focused on streamlining and aligning current data collection activities so that data collected are organized in a way that can be effectively analyzed and communicated to PEEF stakeholders. The overarching goal of this report is to link the investment of PEEF funding to student outcomes to demonstrate how PEEF impacts schools and students.

SRI recommended using a district-wide logic model approach that considers the PEEF initiative as a district-wide portfolio rather than 14 distinct and separate programs. The PEEF district-wide logic model provides the comprehensive coordination to link (1) the funding; (2) the program activities designed to promote access and equity; (3) immediate outcomes, such as creating a positive school climate and increasing student engagement; and (4) the long-term outcomes such as student achievement and post-graduation success. This model provides a conceptual roadmap for understanding how PEEF activities relate to outcomes, and a way to analyze data to assess PEEF’s impact on students.

SRI’s recommendations included significant changes from the way PEEF evaluation data had been considered and presented in the past. The overarching recommendation from SRI entailed a shift in evaluation from considering programs as single and distinct entities to viewing the district-wide PEEF initiative in a comprehensive and holistic manner. To provide the rationale for this new direction in the PEEF initiative evaluation, SRI’s description of and justification for the logic model as it was presented in the final SRI report received on December 9, 2011 is included below.

Recommendations from SRI report:

Achieving the PEEF evaluation goal of measuring the impact of PEEF as a district-wide investment initiative requires an evaluation design that can measure the synergistic effect within the district of the 14 PEEF-funded programs as a whole. As an appropriate and feasible alternative to creating comparison groups through random assignment or propensity scoring, SRI recommends that PEEF use a logic model approach to assess the results of the overall initiative and the programs its funding helps to support and to examine the relationship of program outcomes to activities and investments.

PEEF 2011-12 Interim Evaluation Report, December 20, 2012 v

Following SRI’s recommendations, the PEEF office employed a district-wide logic model approach that considered the PEEF initiative as a district-wide portfolio rather than 14 distinct and separate programs. This report presents an analysis of the impact of PEEF as described by the PEEF logic model. Moving left to right, the logic model provides a framework for understanding how PEEF funding can contribute to increases in student outcomes. The analysis examines trends over time across each of the major categories in the model in order to provide a picture of how PEEF funds have been allocated to the district (inputs), how the funds have been used to increase access to, and equitable distribution of, enrichment programs (outputs), intermediate student outcomes, and long-term student outcomes. The analysis thus tests whether the programs funded by PEEF have led to positive student outcomes as anticipated in the design of the initiative. That is, upward or downward trends in each section are logically linked with upward or downward trends in the next section of the logic model. For example, greater student engagement is expected to lead to greater achievement. The analysis in this report will summarize the extent to which data either confirms, or negates, the anticipated effects of the PEEF initiative.

PEEF is a district-wide investment in a diverse, yet strategic portfolio of SFUSD programs that has contributed to increases in access to services and programs and student achievement. The evaluation of PEEF and this report was guided by the PEEF district-wide logic model, which depicts a chain of reasoning of “if... then” statements that explain how each of the components in the model (inputs, outputs, immediate outcomes and long-term outcomes) are connected and sequential. Findings related to each of the logic model components were organized by PEEF’s goals of improving professional and instructional capacity and quality, school climate, school engagement, student physical and mental health, and ultimately, student academic achievement. Below is a summary of PEEF outputs, outcomes, and impact.

PEEF Outputs PEEF-funded programs increased access and equity by increasing levels of offerings and participation related to the following areas:

1. Professional and instructional capacity and quality (e.g., qualified staff, number of physical education and arts classes offered to students, student access to libraries, and professional development for staff)

2. School climate (e.g., security at athletic events, Restorative Practices implementation at schools, additional custodians at elementary schools)

3. School engagement (e.g., arts instruction for elementary students, students receiving mentoring, interpretation for ELL parents)

4. Student health (e.g., participation in athletic teams, students served at Wellness Centers, student access to Student Support Professionals)

5. Academic achievement (e.g., student participation in tutoring programs, City College courses offered to SFUSD students, teachers aides)

As presented in the outputs section of the report, 73 out of 83 measures (88%) show a positive trend, indicating that PEEF has contributed to increasing access and equity in SFUSD.

PEEF 2011-12 Interim Evaluation Report, December 20, 2012 vi

PEEF Immediate Outcomes With these increased outputs, immediate outcomes are expected to improve. As evidenced by the data included in this report, positive trends were manifested in the following areas:

1. Professional and instructional capacity and quality (e.g., number of highly qualified teachers hired in SFUSD, diversity of new hires, low number of classroom vacancies on the first day of school)

2. School climate (decrease in suspension rates for middle school students, increase in students’ perceptions of school safety and positive school environment)

3. School engagement (improved attendance rates, students’ perceptions of caring adults at the school)

4. Student physical and mental health (e.g., aerobic capacity and increased social support of students)

This report has presented data for 93 immediate outcome measures, including:

• 31 measures of change over time, between the implementation year of a PEEF program and the 2011-2012 school year

• 44 ”data snapshot" measures based on responses from 2011-12 district and programs surveys

• 18 measures that compare school suspensions and enrollment of PEEF participants to non-PEEF participants in the middle school and high school Athletics

Of the 46 measures showing change over time, 43 (93%) demonstrate a positive trend. The 44 data snapshot measures present an overall picture of positive staff, parent, and student perceptions related to professional and instructional capacity and climate, school climate and engagement, and student’s physical and mental health. In addition 94% of the measures that compare participants and non-participants in middle and high school Athletics indicated that participation in Athletics plays a significant role in improving student engagement in schooling, for students overall, and particularly for those traditionally underserved (African-American and Latino students). These combined findings suggest that PEEF has contributed to improving SFUSD professional and instructional capacity and quality, improving school climate, increasing school engagement, and improving student physical and mental health.

PEEF Long-Term Outcome: Academic Achievement With the progress made toward these immediate outcomes it is reasonable to expect increases in student achievement. When student achievement was examined, it was found that GPA and CST ELA and CST math scores have improved district-wide. Analysis that compared participants and non-participants provides strong evidence that participation in middle- and high-school athletics helps improved student achievement. Overall, athletes had higher GPAs than matched non-athletes who closely resembled them before beginning high school or middle school.

This report presented data for 31 academic achievement measures, 23 of which show change over time, nine of which compare participants with non-participants, and eight of which show 2011–12 data snapshots. All 23 measures showing change over time display a positive trend. Nine measures compare the achievement (GPAs) of participants and nonparticipants in middle and high school athletics, after

PEEF 2011-12 Interim Evaluation Report, December 20, 2012 vii

using a statistical technique (propensity score modeling) to help ensure that athletes and their comparison groups shared similar characteristics before entering middle and high school. Of these nine measures, 100% indicated higher GPAs for participants than nonparticipants, for students overall, and for those who are African-American and Latino. In addition, student survey respondents frequently reported that participation in Wellness Center programs helped them improve their academic performance. SFUSD school staff reported confidence in their ability to prepare students for college and/or careers. Parents had similar sentiments, with 81% of respondents to the SFUSD Family Satisfaction Survey indicating that they strongly agreed/agreed that teachers at the children’s schools had the skills and knowledge to prepare them for college and/or career.

Across all of the PEEF immediate and long-term outcomes, a considerable majority of the outcomes (88%) show a positive trend. District-wide data indicates that SFUSD is making gains; over the course of the PEEF years many of the key district-wide indicators of student success have improved. Since PEEF funding was initiated in 2004–05, attendance rates have increased, GPAs have increased, and CST ELA and math scores have improved for all school levels. Continued investment in PEEF programing is likely to increase outcomes in the impact areas outlined in this report and result in greater gains in student achievement for all students at SFUSD.

This pattern of positive increases in both outputs and immediate outcomes indicate that the PEEF has contributed to the improvement of long-term outcomes for students in SFUSD. PEEF funding has been vital during a period of state budget cuts and decreases in other funding. Data suggests that PEEF has been essential to the continuation of programing that not only contributes to increased academic achievement at the SFUSD but that also provides students with the tools and skills necessary for continued success after graduation.

Future Steps This report builds on the PEEF logic model approach to evaluating the PEEF initiative. To test the pathways in the PEEF logic model, the PEEF Office will continue to collect and report relevant and meaningful measures of outputs and outcomes. The PEEF Office began this process by following recommendations from SRI on how to enhance PEEF data. The PEEF Office will continue to focus on collecting data to show the changes from the year prior to PEEF implementation to the current year, and, when possible, examine the differences between students served by PEEF and those not served by PEEF.

We also expect that in coming years our measures will better allow PEEF to disaggregate outputs and outcomes by student and school characteristics, such as student’s race/ethnicity, gender, ELL status, school level (e.g., elementary, middle, high school), school area/zone. In addition, the PEEF Office will work with PEEF program managers to strengthen program capacity to collect high-quality data, as well as develop a PEEF data management system. Such collaborative efforts projected for the 2012-13 year include a case study on the implementation of the Restorative Practices program and an exploration of the impact of services provided by Student Support Professionals.

PEEF 2011-12 Interim Evaluation Report, December 20, 2012 1

Report Overview This report presents an evaluation of the Public Education Enrichment Fund (PEEF) initiative using data through the 2011–12 school year.

Given the complexity of the programs supported by the PEEF initiative, the PEEF Office consulted with SRI International (SRI), for guidance on the assessment and communication processes. The specific work with SRI focused on streamlining and aligning current data collection activities so that the data collected is organized in a way that can be effectively analyzed and communicated to PEEF stakeholders. The overarching goal of this report is to link the investment of PEEF funding to student outcomes which ultimately show how those outcomes impact school sites and student achievement.

SRI recommended using a district-wide logic model approach that considers the PEEF initiative as a district-wide portfolio rather than 14 distinct and separate programs. The PEEF district-wide logic model provides the comprehensive coordination to link (1) the funding; (2) the program activities designed to promote access and equity; (3) immediate outcomes, such as creating a positive school climate and increasing student engagement; and (4) the long-term outcomes such as student achievement and post-graduation success. This model provides a conceptual roadmap for understanding how PEEF activities relate to outcomes, and a way to analyze data to assess PEEF’s impact on students.

Public Education Enrichment Fund Background In March of 2004, San Francisco voters, with approval of over 70% of the electorate, passed the ballot initiative Proposition H. The passage of Proposition H established the Public Education Enrichment Fund (PEEF) as law within the City Charter, Section 16.123.1-10 (Appendix A). The passage of Proposition H came at a time when district resources had significantly diminished due to state and federal funding. The preamble of SEC. 16.123-1 includes the rationale for creating PEEF:

(a) The people of the City and County of San Francisco find and declare that:

1. Quality public education is highly correlated with higher earnings potential, reduced crime, lower rates of teen pregnancy and substance abuse, and greater self-esteem;

2. Urban public schools have the greatest need for comprehensive educational programs-including preschool programs, arts and music programs, sports activities, and after school programs-but often have the fewest resources to provide them;

PEEF provides funding to improve the quality of education for the youth of San Francisco and is shared by both First 5 San Francisco and SFUSD. One third of the total PEEF fund is allocated to First 5 San Francisco for preschool support; another third supports SFUSD sports, libraries, the arts and music (SLAM); and the last third funds programs such as Student Support Professionals, Wellness Centers, Peer Resources, Restorative Practices, and Translation Services.


Figure 1: PEEF Initiative Funding Allocations

*Amount refers to financial support plus the cash value of any in-kind support services. **Amount reflects submitted budget (March 1, 2011) Note. 2008–09 through 2011–12 allocations reduced 25% due to City’s projected $100 million or more budgetary shortfall. Contributions in years 2010–11 through 2014–15 are adjusted based on the estimated percentage increase or decrease in the City’s discretionary General Fund revenues in each year.

Amount*

Fiscal Year(s) First 5 (Pre-K)

SFUSD SLAM SFUSD Other Total

Amount Received by

SFUSD

2005–06 $3,333,333 $3,333,333 $3,333,333 $10 million $6,666,666

2006–07 $6,666,667 $6,666,667 $6,666,667 $20 million $13,333,334

2007–08 $10,000,000 $10,000,000 $10,000,000 $30 million $20,000,000

2008–09 $15,000,000 $15,000,000 $15,000,000 $45 million $22,500,000

2009–10 $20,000,000 $20,000,000 $20,000,000 $60 million $30,000,000

2010–11 $20,000,000 $20,000,000 $20,000,000 $60 million $29,307,000

2011-12 $20,000,000 $20,000,000 $20,000,000 $60 million $29,900,000**

2012–15 $20,000,000 $20,000,000 $20,000,000 $60 million TBD


SRI International Recommendations In the fall of 2011, SFUSD contracted with SRI International (SRI) to provide technical assistance in reviewing current PEEF data collection activities. SRI provided direct assistance in refining and streamlining current data collection activities in order to effectively communicate PEEF activities and results. SFUSD received SRI’s final report, San Francisco Unified School District’s Public Education Enrichment Fund: Evaluation Technical Assistance Report on December 9, 2011. The report provided recommendations based on current PEEF performance measures and data, program materials, data collection tools, and information gathered through meetings and interviews with the PEEF program administrators and managers. The most significant recommendation adopted was to use a district-wide logic model approach to assess the success of this initiative.

This report presents the PEEF initiative logic model, including the rationale for utilizing it to evaluate the PEEF initiative, and data organized along the themes outlined in the model. Future steps in the PEEF evaluation process are outlined in the final section of this report.

The recommendations from SRI entail significant changes from the way that PEEF evaluation data has been presented and considered in the past. The overarching recommendation from SRI entails a shift in evaluation from considering programs as single and distinct entities to viewing the district-wide PEEF initiative as an interconnected system. To provide the rationale for this new direction in the PEEF evaluation, SRI’s description of and justification for the logic model as it was presented in the final SRI report received on December 9, 2011 is included below.

SRI Report Overview

The Logic Model Approach1

“Stakeholders in investment programs such as PEEF frequently want to know, “Did our investments cause improvements in outcomes of participants in the programs we fund?” To make a causal statement, one must rule out all other possible explanations for a change in outcomes, which requires rigorous and often expensive research approaches. The gold standard for measuring causality is a randomized control trial (RCT) in which service recipients are randomly assigned to intervention and nonintervention groups. This approach generates comparison groups that are similar on all other factors except

1 Newman, L., Fabrikant, N., and Peterson, D. (2011). San Francisco Unified School District’s Public Education Enrichment Fund: Evaluation Technical Assistance Report. Menlo Park, CA: SRI International.

their participation in the intervention being evaluated. Although an RCT is the most scientifically rigorous design for ruling out alternative explanations for impacts observed (Shadish, Cook, & Campbell, 2002), it is expensive, complex, difficult to implement, and most importantly, not feasible for evaluating complex initiatives like PEEF for multiple reasons:

• As an investment portfolio, PEEF has no clearly-defined, separately funded “intervention” to which changes in outcomes can be linked, a requirement of a RCT. PEEF funds mingle with funds from other sources to support a set of widely diverse programs that are intended to work together collectively to improve a wide range of student and staff outcomes district-wide.


• PEEF-funded services often are voluntary and sometimes are mandatory (e.g., physical education), so participants cannot be randomly assigned to receive or not receive PEEF-funded services. The motivations for participants to self-select into a PEEF-funded program make them inherently different from nonparticipants, introducing possible explanations for differences in outcomes other than program participation.

• PEEF explicitly intends to have impacts that spread from individual program participants to others in their schools and district. For example, certain PEEF investments intend to improve overall school climate by providing specific students with peer supports or through training specific teachers in restorative practices. This intended spread of impacts beyond individual participants means that an uncontaminated comparison group whose members are not impacted by any PEEF-funded programs cannot be identified.

A statistical analysis approach that can simulate analyses of data derived from an RCT, propensity scoring, adjusts for differences in program participants and nonparticipants and estimates effects that approximate what might be obtained in an RCT (Cepeda, Boston, Farrar, & Strom, 2003; Rosenbaum & Rubin, 1983, 1985). To use propensity scoring methods, programs need to collect individual, unduplicated data for participants. Currently, only a few PEEF-funded programs (e.g., athletics and the wellness initiative) identify individual program participants. Further, although propensity

scoring reduces the bias resulting from self-selection of participants, it may not eliminate it completely, especially when factors such as the spread of effects from program participants to others in a school confounds the differences between participants and nonparticipants. Therefore, propensity scoring also cannot capture overall impact of PEEF investments. This type of analysis was conducted on the effects of participation in athletics, but was not considered appropriate for any other PEEF program at this time. Fundamentally, standard program evaluation designs cannot meet the needs of a PEEF evaluation because it is not a program, but an investment portfolio.

Achieving the PEEF evaluation goal of measuring the impact of PEEF as a district-wide investment initiative requires an evaluation design that can measure the synergistic effect within the district of the 14 PEEF-funded programs as a whole. As an appropriate and feasible alternative to creating comparison groups through random assignment or propensity scoring, SRI recommends that PEEF use a logic model approach to assess the results of the overall initiative and the programs its funding helps to support and to examine the relationship of program outcomes to activities and investments.

The evaluation field promotes a logic model approach for evaluating complex, multifaceted school- or community-wide initiatives because it not only produces useful and credible evaluation results, but also a valuable program or initiative planning tool that is a foundation for monitoring implementation. In fact, PEEF program managers found that the newly-created district-wide logic model helped clarify what the investment initiative intends to accomplish and the model encouraged them to reflect on how the initiative can be strengthened.


A logic model depicts the ways in which various aspects of an initiative connect to one another and how they can be expected to produce the desired outcomes. The recommended approach to constructing a logic model is to first reach agreement with stakeholders as to the outcomes an initiative is expected to achieve and how they can be measured. “Backward mapping” is then used to identify what has to happen—what activities must occur (e.g., a specific program is funded, staff are trained to deliver it, students participate)—if the outcomes can reasonably be expected to occur. Continuing to backup from these “outputs,” the resources, or “inputs” needed to produce the outputs are identified. With these three components identified—resources needed to support activities, which must occur for outcomes to be achieved, and the outcomes desired—stakeholders can monitor both the implementation and outcomes of initiative and their relationship to each other.

Although the logic model approach cannot eliminate all alternative explanations for the outcomes measured in an evaluation, tracking data on each of the logic model components provides credible evidence regarding the actual resources committed, the level of implementation achieved, and results attained. The logic model approach for establishing impact “draws on tried and true scientific traditions of testing hypotheses about cause and effect relationships, including methods used in physical, biological, and other social sciences” (Connell & Kubisch, 1998, p. 34). Logic models have been used by many other programs and evaluations, including the U.S. Department of Education’s Promise Neighborhood’s effort, Ready Schools Miami, Kellogg Youth Initiative Partnerships, Devolution, ENLACE (Engaging Latino communities for Education), the Native American Higher Education Initiative, and the

Build Initiative (Chen & Rossie, 1992; Connell & Kubisch, 1995; Golan et al., 2011; Julian, 1997; Lambur & Mayeske, 2000; Scheirer, 1999; Smith, 2011; W. K. Kellogg Foundation, 2001; Weiss, 1995; Westmoreland, Lopez, & Rosenberg, 2009; Yampolskaya, Nesman, Hernandez, & Koch, 2004), In fact, as part of its effort to improve educational outcomes, the U.S. Department of Education requires successful applicants for many of its grant-making efforts to include a logic model in their proposals and funds the Center for Evaluation and Education Policy to provide technical assistance to applicants in support of logic-model development (http://www.tadnet.org/model_and_performance).

PEEF District-Wide Logic Model

Key PEEF-initiative components and the relationships among them are graphically represented in the PEEF logic model (Figure 2). Because PEEF is a district-wide initiative, this model measures the synergistic effect on the district of the 14 PEEF-funded programs as a whole rather than focusing on each program separately.

The PEEF logic model describes a chain of reasoning or "If… then…" statements that explain how PEEF-funded programs and activities are linked to SFUSD school and student outcomes. There are five components in the model (inputs, outputs, immediate outcomes, and two long-term outcomes), each of which is connected to the next component and is dependent on the component before it occurring.

Beginning at the left of the logic model, the PEEF logic model depicts the 14 PEEF-funded programs (inputs)—the 4 SLAM programs and the 10 Third-Third programs. If these program inputs are funded, then we would expect increases in access and equity to be produced as outputs of what those programs do—their

http://www.tadnet.org/model_and_performance


offerings and activities, including for example, the number and types of services, events, and professional development trainings; and the numbers and types of course offerings and equipment provided; and the number and types of program participants (students, parents, and staff).

If outputs are provided and students and staff participate in them, then a set of immediate outcomes would be expected to improve. The four types of immediate outcomes specified in the logic model include: 1) increased professional and instructional capacity and quality (e.g., staff qualifications, skills, and knowledge); 2) improved school climate (e.g., greater safety and cleanliness, reduced behavioral incidents); 3) increased school engagement (e.g., increased student attendance, motivation, joy in learning, and leadership skills, and greater parent participation); and 4) improved student health (e.g., self-confidence, self-efficacy, level of stress, fitness and health knowledge and behavior).

If progress is made toward these Immediate outcomes—schools deliver higher quality instruction and have more highly qualified staff, more students feel safe in clean, orderly, and well-maintained schools, more students and parents are engaged and connected to schools, and more students are in good physical and mental health—then it is reasonable to expect that student academic achievement will improve. Extensive research has demonstrated significant links between these four types of immediate outcomes and increased academic achievement. For example, increased teacher qualifications and professional capacity have been related in multiple studies to improved student reading and math achievement levels, higher scores on high school graduation tests, and reduced gaps in student learning frequently associated with differences in

household income and race/ethnicity (Boyd, Lankford, Loeb, Rockoff, & Wyckoff, 2008; Heck, 2007; Powers, 2003; Rivkin, Hanushek, & Kain, 2005; Siegrist, Weeks, Pate, & Monetti, 2009). Aspects of school climate, including students’ feeling safe at school, fewer school-wide behavioral incidents, and overall cleanliness have been associated with increased achievement, including higher grades, lower absenteeism, stronger performance on standardized achievement tests, and high school completion rates (Brand et al., 2008; Brand, Felner, Shim, Seitsinger, & Dumas, 2003; Duran-Narucki, 2008; McEvoy & Welker, 2000; Mendel & Heath, 2004; Samdal, Nutbeam, Wold, & Kannas, 1998). Students’ school engagement, school connectedness, and motivation toward school also have been demonstrated to have significant impacts on academic achievement, particularly for at-risk urban students (Bond et al., 2007; Fenzel & O'Brennan, 2007; Malecki & Demaray, 2006; Samdal, et al., 1998). Improved student social-emotional and physical health and fitness have been found to be positively related to academic outcomes, including grade performance and high school persistence (Efrat, 2011; Multon, Brown, & Lent, 1991; Murray-Harvey, 2010).

Finally, if increases in academic achievement are attained, then we can expect that students are well on their way to becoming successful adults, ready for college and careers, and equipped with the skills, capacities, and dispositions necessary for 21st century success—SFUSD's Vision for Student Success. The combination of” if-then” relationships and measures presented in the PEEF logic model collectively provides strong and compelling evidence about the success of PEEF. If positive changes in immediate outcomes and academic achievement are documented and these improvements are shown to coincide with


increases (or in some cases maintenance in the face of budget cuts from other funding sources) in the number and types of PEEF-funded activities and students and staff served, then it is reasonable to conclude that PEEF played an important role in that impact.”


Figure 2. Public Education Enrichment Fund (PEEF) Initiative Logic Model*

*Updated by PEEF Office, December, 2012


Methods The following sections of the report analyze a broad range of data related to PEEF’s activities and effects, organized by the logic model developed by SRI. The logic model provides a framework for understanding how PEEF can lead to better student outcomes. The analysis summarizes trends over time across each of the major categories in the model to provide a picture of how PEEF funds have been allocated to the district (inputs), how the funds have been used to increase access to, and equitable distribution of programs (outputs), intermediate student outcomes, and long-term student outcomes. Taken together, these trends describe the pattern of implementation and student outcomes that have actually occurred, and thus portray the extent to which PEEF is having, or not having, the intended positive effects on students, as anticipated in the design of the initiative.

In addition to the presentation of trend data, this report includes a propensity score matching study of the effects of participation in athletics, which uses a statistical matching approach to compare attendance, grades, and suspension rates of athletes with matched non-athletes. Although as explained previously, this method is not feasible for assessing the whole initiative, athletics data, which identifies individual program participants, provides an opportunity to rigorously test this program’s effects, and also suggests the kinds of effects that PEEF-funded enrichment programs can have on students.

The PEEF Office and PEEF program managers gathered the most complete data available for measures included in this report from the year before PEEF (2004–05) through the last complete year (2011–12). In all, 207 measures are included in this analysis. In addition, when a large number of measures related to the same program had similar content, only a few representative measures were included in this report. Detailed information about the data included in this report (e.g., data sources and trends for additional years) can be found in Appendix C.


Inputs: What does the PEEF initiative provide? The first box in the PEEF district-wide logic model depicts the funds received by the PEEF program, which is considered inputs. The PEEF budget for the 2011–12 school year was $29,900,000 (budget submission to the City, March 1, 2011). Included in that amount was:

• $14,950,000 for sports, libraries, the arts and music (SLAM) and $14,950,000 for Other General Uses (the Third-Third). The Other General Uses funding included: $7,601,172 for Ongoing Programs in the areas of Learning Support Services (Student Support Professionals, Wellness Initiative, Restorative Practices, and Peer Resources), Academic Support (Career Technical Education, Teacher Academy, and Formative Assessment System), Family Support (Translation and Interpretation Services), Safe and Clean Schools (Custodial Services), and PEEF Infrastructure (Teacher Recruitment and General Infrastructure).

• 5,031,578 for Reserve Funds: Allocations to the SFUSD General Operating fund (weighted student formula) to address the State budget shortfall. The Weighted Student Formula provides basic needs to school sites such as classroom teachers, school administration, and instructional materials.

• $2,317,250 for In-Kind Services (this amount is 7.75% of the total SFUSD allocation which is consistent with previous years): services received from the Department of Children, Youth and their Families; Department of the Environment; and the Public Utilities Commission.

Appendix B contains a narrative description of the 14 PEEF–funded programs as well as a description of PEEF-funded activities. Tables 1 and 2 display the amount of PEEF funding budgeted for each program in 2011–12.


Table 1: Sports, Libraries, Arts and Music Budget 2011-12

SPORTSSupport for Physical Education21.0 FTE: 15.0 FTE ES PE Specialists to provide instruction at 30 elementary schools, 2.0 FTE Elementary Implementation Specialists, 1.0 FTE Middle School Content Specialist to support 22 middle schools, 1.0 FTE High School Content Specialist to support 19 high schools, 1.0 FTE Program Administrator, and 1.0 FTE Clerk $1,768,431Elementary School PE Site Coordinators 30 coordinator stipends for 30 sites $18,000Allocations for secondary sites: $16.00 per student for 22 middle and 19 high schools $444,112Allocations for secondary County/Community/Court and Charter sites: $16.00 per student $39,376Instructional Supplies and equipment for 36 elementary and secondary school sites (Increased to $71,748 in 3-01-11 revised budget) $71,748Professional Development for staff and teachers approximately 150 workshops for K-12 teachers $150,000Physical Education Total $2,491,667

Athletics Office AllocationCoaches Pay / Benefits - Approximately 150 coaches at $28.89/hr. + benefits $752,909Medical Supplies for 13 high school training rooms and athletic teams $35,000Medical Personnel and Security Officials - Including Athletic Trainers for high schools, Emergency Medical Technicians and doctors for football games, game officials, security guards, and SFPD Police Officers $510,000Bus Transportation for athletes - approx. 1200 trips to local athletic contests, regional, and state playoffs $400,000

School Athletic Facility Repair/Maintenance/Construction - renovation of athletic facilities at two high schools (Increased to $288,758 in 3-01-11 revised budget) $288,758Non-School Site Facilities Rental (for league, playoff, and championship athletic events not held at SFUSD school sites) $10,000Athletic Equipment (e.g. scoreboards, track equipment, football equipment, and other athletic equipment) $40,000Professional Development for approximately 100 coaches $10,000Site Based Allocations for 21 middle and 13 high schools $445,000Athletics Total $2,491,667Sports Total $4,983,334

LIBRARIES47.0 FTE (43.5 FTE K - 8 Librarians assigned to all elementary and middle schools (86 sites) 2-3 days per week, 1.0 FTE Program Administrator, 0.5 FTE Libraries Supervisor, 1.0 FTE Teacher Librarian on Special Assignment, and 1.0 FTE Library Technician) $4,318,333

Site allocations for all K-8 schools and 5 small high school at $5 per student for library materials, technology, or other library related resources $190,000

Large high school partial staffing support of $11,166 per site for 12 high schools (Increased to $12,287 per site in 3-01-11 revised budget) $147,450Allocations for County/Community/Court and 9 Charter Schools at $5 per student for library materials, technology, or other library related resources. Includes allocation for 0.2 FTE for 2 K-8 charter sites. $47,000

Library research and reference databases for K-12 schools –Netrekker, Facts On File, Teaching Books and support for catalog and circulation software $160,000Technology upgrade (computers and printers for 20 schools and 3 LCD projectors for 6 sites) $25,000Professional Development for 65 librarians $9,0001.0 FTE Librarian for five small high schools (additional 1.0 FTE in amount of $86,500 in 3-01-11 revised budget) $86,550Libraries Total $4,983,333

ARTS AND MUSIC14.4 FTE Generalist Visual and Performing Arts Teachers - 0.2 FTE for 72 elementary site (includes K-8 sites) $1,197,919

. 18.6 FTE Middle School Arts Teachers 1-5 days per site for 21 sites (includes K-8 sites) $1,587,4916.4 FTE High School Arts Teachers 1-3 days for 18 sites $514,9962.0 FTE Arts Teacher for County/Community/Court schools $135,9194.6 FTE District-Wide Support and Administration: 0.5 FTE VAPA Supervisor, 0.5 FTE Artistic Director, 1.0 FTE Arts Education Master Plan Implementation Manager, 1.6 FTE Teacher on Special Assignment, and 1.0 FTE clerk $483,1701.0 FTE District-Wide Piano Technician $93,70998 Elementary, Middle, and High School Arts Coordinators stipends of $1000 plus benefits (increase stipend to $1,500 in March 1, 2011 budget)

$181,598Site allocations for 72 ES, 14 MS, and 18 HS (includes K-8 sites) for arts supplies/materials, field trips, artists-in-residence and/or credentialed arts specialists ($5.00 for elementary and $10.00 for middle and high schools) $406,550Site allocations for County/Community/Court & 9 Charter Schools for arts supplies/materials, field trips, artists-in-residence and/or credentialed arts specialists ($5.00 for elementary and $10.00 for middle and high schools), and allocations to support 2.4 FTE Arts Teachers

$234,057Supplies/Materials/Equipment for Generalist Arts Program to support 72 elementary sites (Increased to $27,200 in 3-01-11 revised budget) $27,200District-Wide Music Instrument Repair & Supplies $30,000Professional Development for all principals, Arts Coordinators, and Arts and Classroom Teachers $20,000Visual and Performing Arts Implementation Support - Consultants, Content Specialist Extended Hours $3,000VAPA Parent Outreach and Assessment costs $2,724Young At Art (K-12) festival production $65,000Arts and Music Total $4,983,333

$14,950,000SLAM Total

Public Education Enrichment FundSFUSD 2011-12 Detailed Budget - Revised 3-01-11

Uses - Sports, Libraries, Arts and Music


Table 2: Other General Uses Budget 2011–12

Student Support Professionals

40.0 FTE includes: 37.0FTE Learning Support Professionals and School District Nurses serving 74 E.S., K-8, & M.S. sites, 1.0 FTE Mentor Student Support Professional to provide coaching, site support and professional development, 1.0 FTE Program Administrator, and 1.0 FTE Senior Clerk Typist $3,315,107Professional Development includes: Clinical supervision groups, instructional supplies and materials, purchase of evidence-based health and mental health curricula, and travel and conferences $37,500Program Evaluation $60,000Supplies, materials and operating costs $24,260

Student Support Professionals Total $3,436,867Peer Resources - Approximately 5.35 FTE Peer Resource Teachers $467,471Wellness Initiative9.2 FTE includes: 4.0 FTE Wellness Coordinators, 3.6 Community Health Outreach Workers, and 1.6 District Nurses $707,552Stipends for Youth Outreach Coordinator and Youth Outreach Workers (Leadership and youth development at 3 sites) $18,000

Wellness Initiative Total $725,552

Restorative Justice - 4.5 FTE coordinator and site coaches, professional development, data base support, reference and library books, and supplies $664,763

Career Technical Education - 1.0 FTE to provide district wide support for 8 high schools $79,879Teacher Academy - Stipends for 43 students, extended hours for teachers and classroom supplies $65,000Formative Assessment System - 1.0 FTE Researcher, 0.2 FTE Program Administrator, and Data Director database $284,750

Translation and Interpretation Services - 4.5 FTE, contracts and extended hours for translation, equipment upgrades and replacements $604,000

Custodial Services - 9.5 FTE (Approx. 8.5 Custodians & 1.0 Assistant Supervisor) $695,773

Teacher Recruitment - 1.2 FTE (Approx. 1.0 FTE Recruitment Analyst + Approx. 0.2 FTE Education Credential Technician) $109,238General Infrastructure1.0 FTE Program Administrator $116,049Program Evaluation $200,0001.0 Grant Writer $143,830Supplies and Materials $5,000Community Advisory Committee Support $3,000General Infrastructure Total $467,879

Funds in Reserve for Weighted Student Formula to Address State Budget Shortfall (Increased to $5,031,578 in 3-01-11 revised budget) $5,031,578

$12,632,750

Sustainability/Environmental Initiative Director, Department of the Environment, and the Public Utilities Commission $150,000SF Promise, Department of Children, Youth and Their Families $180,000Center for Academic Recovery and Empowerment - Truancy Prevention, Department of Children, Youth and Their Families $150,000Salad Bars - Fruits in Schools, Department of Children, Youth and Their Families $234,000Out of School (OTS) - School Based, Department of Children, Youth and Their Families $946,656Youth Leadership, Empowerment & Development, Department of Children, Youth and Their Families $610,094Additional services to be identified November 2011 (Increase of $46,500 in 3-01-11 revised budget) $46,500In-Kind Services Total $2,317,250

$14,950,000

$29,900,000

Reserve Funds

General Uses Total

Family Support

The in kind services total represents 7.75 % of the total PEEF allocation, which is consistent with previous years. List of recognized services will be finalized in November 2011.

PEEF Grand TotalRevised 3-01-11 budget is based on 2011-12 allocation estimate from Controller's Office as of January 26, 2011

Safe and Clean Schools

General Uses Total

In Kind Services

PEEF Infrastructure

Public Education Enrichment Fund

Academic Support

Learning Support Services

SFUSD 2011-2012 Detailed Budget - Revised 3-01-11 Uses - Other General


Outputs: What are the levels of Access and Equity related to the PEEF initiative? Outputs are what PEEF-funded programs “do”—their offerings and activities including: 1) staffing; 2) course offerings, instructional equipment, services, and program sponsored events ; 3) professional development and trainings ; and 4) number of students, parents, and staff served.

The second box in the PEEF district-wide logic model depicts PEEF outputs. If PEEF programs are funded (as described in the inputs section), then increases would be expected in access and equity to be produced as outputs of what those programs do.

PEEF outputs are organized by their primary outcome theme from the logic model, including the four immediate outcomes: (1) increased professional and instructional capacity and quality; (2) improved school climate; (3) increased school engagement; and (4) improved student physical and mental health, as well as the long-term outcome of increased academic achievement. Although these outputs are presented by their primary theme, readers should be aware that many outputs are related to multiple outcomes. Research has demonstrated significant links between the four types of immediate outcomes and the long-term outcome of academic achievement.

Outputs 1: Professional and Instructional Capacity and Quality What are the levels of offerings and participation related to professional and instructional capacity

and quality, and what is their distribution?

Program outputs impacting professional and instructional capacity and quality include staff allocations, program offerings, and professional developments and trainings. Twenty four out of 25 measures reported show positive trends over time (Table 3). Continuing investment in these outputs is likely to increase access to quality programming and services for students in the SFUSD and thus, positively influence academic achievement. Table 3 and subsequent tables are marked with a blue upward pointing arrow to indicate a desired trend or an orange downward pointing arrow to indicate an undesired trend (e.g., if suspension rate data decreases over time an upward pointing blue arrow is used to indicate a desired trend).

The measures presented show how PEEF funding has increased the number of qualified staff in SFUSD, including , number of certified arts teachers, teacher librarians, and physical education specialists, and thereby increased student access to quality instruction. PEEF funding has contributed to an increase in the number of athletic coaches in SFUSD. In 2004-05 there were 458 athletic coaches staffed by the Athletics department, in 2011-12 there were 557 coaches, thirty one percent of whom were PEEF funded. The Libraries program was also able to increase the number of FTE teacher librarians with PEEF funds. This in turn, made libraries more accessible to students. Prior to PEEF funding, only 17% of SFUSD schools had a teacher librarian. Through PEEF-funded facility improvements and the addition of 48 FTE certified staff, 103 out of 104 schools had access to a library staffed with a certified teacher librarian in 2011-12. Through PEEF funding, Physical Education department staffed 15 Physical Education specialists at elementary schools to deliver physical education to students and to support elementary school teachers providing Physical Education instruction. In 2011-12, Physical Education provided


Program Support Option 1 (credentialed Physical Education specialist, professional development, and equipment) to 47% of elementary schools (36 of 73).

The Visual and Performing Arts program staffed 43.6 FTE arts teachers and 103 arts coordinators at elementary schools in 2011-12. Prior to PEEF there were few credentialed art teachers at the elementary school level or arts coordinators, now nearly all schools (99%) have an arts coordinator at their site and more elementary school students have access to a credentialed arts teacher (12,626 in 2011-12). The Visual and Performing Arts (VAPA) program offered 509 arts classes to elementary school students, 433 art courses to middle school students and 832 arts courses to high school students. The decrease in the number of arts courses at the high school level is a result of increased course requirements (A-G and language pathways) and/or site scheduling/budgets.

In addition to increasing quality of instruction and professional capacity in SFUSD by funding the hiring of highly qualified staff, PEEF is also funding professional development trainings for staff. In 2011-12 the Physical Education department offered 70 trainings for elementary classroom teachers and 38 trainings for Physical Education teachers and specialists; the Library department offered 32 trainings for teacher librarians, and the Restorative Practices program offered 58 development trainings for staff. To summarize, all measures show that PEEF has had a positive impact on the offerings and participation related to professional and instructional capacity and quality.

2011-12

Year N (%) N (%)

1 Number and percent of athletic coaches funded by PEEF 2005-06 2004-05 0 172 (31%)

2Number of PEEF-funded full-time equivalent (FTE) certified teacher librarians in SFUSD 2005-06 2004-05 0 48

3Number of PEEF-funded full-time equivalent (FTE) elementary school PE specialist 2005-06 2004-05 0 15

4Number of PEEF funded full-time equivalent (FTE) arts teachers at school sites (all grade levels) 2005-06 2004-05 0 43.6

5Number and percent of schools with arts coordinators (all PEEF-funded) 2005-06 2004-05 0 103 (100%)

6 Number and percent of schools with a teacher librarian 2005-06 2004-05 20.3 (17%) 103 (98%)

7 Number and percent of elementary/K-8 schools receiving PEEF-funded Program Support Option 1

2005-06 2004-05 0 37 (47%)

8 Number of PE teachers trained to implement Polar Fitness of Fitness for Life equipment

2005-06 2006-07 0 90

9 Number of SFUSD staff trained to be Restorative Practices trainers

2010-11 2009-10 0 29

**Blue upward pointing arrows represent desirable trends, orange downward pointing arrows represent undesirable trends.*Baseline data represents year prior to PEEF funding or 1st year of available data.

Table 3. PEEF Initiative Outputs Related to Professional and Instructional Capacity and Quality: Change Over Time

Outputs: Professional and Instructional Capacity and Quality

1st year of PEEF

Baseline*Trend**


2011-12

Year N (%) N (%)

10 Number of professional development trainings provided to K-12 teacher librarians

2005-06 2004-05 4 32

11 Number of PE professional development trainings offered to elementary school classroom teachers

2005-06 2004-05 0 70

12 Number of professional development trainings for PE teachers and PE specialists

2005-06 2004-05 3 38

13 Number of PEEF funded professional development arts workshops to SFUSD staff

2005-06 2004-05 0 30

14Number of Restorative Practices introductory presentations at school sites 2010-11 2009-10 0 44

15Number of Restorative Practices professional development trainings offered district wide 2010-11 2009-10 0 58

16Number of Restorative Practices full-day trainnings on Restorative Practices 2010-11 2009-10 0 17

17Number and percent of fully officiated athletic contests (games, matches, meets, and tournaments) supported by PEEF funds 2005-06 2006-07 764 (56%) 1,401(93)

18Number and percent of students with access to a library at their school site that is staffed with a teacher librarian 2005-06 2004-05 22316 (39%) 53,634(99%)

19Number of arts classes offered to ES students through the PEEF funded Visual and Performing Arts Generalist program 2005-06 2004-05 0 509

20Number of Visual and Performing Arts classes offered to MS students 2005-06 2004-05 347 433

21 Number of Visual and Performing Arts classes offered to HS students

2005-06 2004-05 877 832

22Number of athletic coaches participating in at least one professional development training (PEEF-funded) 2005-06 2004-05 0 84

23Number of school site and centralized staff receiving introductory presentations in Restorative Practices as alternative methods of discipline

2010-11 2009-10 0 963

24Number of staff participating in Restorative Practices professional development trainings 2010-11 2009-10 0 1,415

25Number of staff participating in full-day trainings on Restorative practices 2010-11 2009-10 0 754


Table 3 Continued. PEEF Initiative Outputs Related to Professional and Instructional Capacity and Quality: Change Over Time

Outputs: Professional and Instructional Capacity and Quality

1st year of PEEF

Baseline*Trend


Outputs 2: School Climate What are the levels of offerings and participation related to school climate, and what is their

distribution?

PEEF funding has contributed to improving school climate by increasing availability and access to programs and services geared at improving the quality and character of school life for SFUSD students and staff. This is exemplified by positive trends in output measures associated with school climate over time (Table 4). Of the measures that have been categorized as relevant to school climate, six out of seven show a positive trend, demonstrating that PEEF has contributed to increasing access to programming and services that affect school climate.

PEEF funding has allowed the Athletics Department to increase safety at athletic events by providing additional private security guards and San Francisco Police Department (SFPD) Security Resource Officers (SRO). PEEF funding for security at athletic events began in 2007-08. In 2011-12, 435 events were covered by private security guards and SROs. PEEF funding has also been allocated, through the Custodial Services Department, to provide annual deep cleaning to all 37 SFUSD Early Education Department Centers. Additionally, 11 elementary schools received a 0.5 FTE custodian, which resulted in these schools staying open for additional 4 hours daily and additional resources for maintaining the cleanliness and safety of these sites. The Peer Resources program provides conflict mediation via Peer Leaders. The number of students participating in conflict mediation has declined from 2006-07 levels. This reduction corresponds to a reduction in funding from both PEEF and other sources over the past several years.

2011-12

Year N (%) N (%)

1Number of athletic events covered by PEEF funded private security guards and SFPD resources officers 2005-06 2005-06 0 435

2Number of elementary schools receiving an additional 0.5 FTE PEEF funded custodian 2007-08 2006-07 0 11

3Number and percent of SFUSD Early Education Department centers receiving deep cleaning at least once a year 2007-08 2006-07 0 37

4Number of SFUSD schools going through a 2 year program of full school implementation of restorative practices (run by the International Institute for Restorative Practices, IIRP)

2010-11 2009-10 0 3

5 Number of Restorative Practices Conferences 2010-11 2009-10 0 30

6Number of Restorative Practices planning meetings at school sites 2010-11 2009-10 0 34

7Number of students participating in conflict mediation through Peer Resources 2005-06 2006-07 835 329

**Blue upward pointing arrows represent desirable trends, orange downward pointing arrows represent undesirable trends.

Table 4. PEEF Initiative Outputs Related to School Climate: Change Over Time

Outputs: School Climate 1st year of PEEF

Baseline*Trend**

*Baseline data represents year prior to PEEF funding or 1st year of available data.


Outputs 3: School Engagement What are the levels of offerings and participation related to school engagement, and what is their

distribution?

School engagement refers to students’ level of engagement in different aspects of schooling, including motivation, joy in learning, and school connectedness. Table 5 displays the data relating to how PEEF is impacting the offerings, participation, and distribution related to school engagement. Of the 16 measures, 10 show a positive trend, five show a downward trend.

In 2011-12, the Athletics department provided 1,113 bus trips to athletes. Since PEEF funding began, translation and interpretation services have shown positive trends. The number of district-wide events interpreted increased from 34 in 2006–07 (pre-PEEF) to 544 in 2011–12. The number of translation requests from schools also increased from 260 in 2005-06 (pre-PEEF) to 924 in 2011-12. The numbers of pages translated into other languages also increased from 1,489 in 2006-07 to 4,180 in 2011-12. Through the increase in documents translated and events interpreted, English Language Learner parents have increased opportunities to engage in their children’s education.

Through the Visual and Performing Arts program, PEEF funding has increased program offerings in the arts and consequently the participation of students in arts courses and events. Participation in arts courses in K-12 increased from 14,789 students in 2004-05 to 26,660 students in 2011-12. Enrollment at the high school level has remained relatively flat due to increased course requirements (A-G and language pathways) and/or site scheduling/budgets.

Participation in Peer Resources mentoring and tutoring programs has increased since the beginning of PEEF. The number of students participating in Peer Resources mentoring increased from 680 students in 2005-06 to 1,110 students in 2011-12. The number of students participating in tutoring services increased from 542 in 2005-06 to 983 students in 2011-12. Declines in the number of Peer mentors/leaders and participation in support groups and other program offerings has been a result of many factors including decreases in program funding and the number of schools with Peer Resources programs and a shift in the focus of the program at different sites.


2011-12

Year N (%) N (%)

1 Number and percent of bus trips for athletic teams funded by PEEF

2005-06 2004-05 0 1,113 (100%)

2 Number of SFUSD middle schools with a Peer Resources program

2005-06 2004-05 5 5

3 Number of SFUSD high schools with a Peer Resources program 2005-06 2004-05 12 9

4Number of SFUSD students receiving mentoring from a Peer Resources program 2005-06 2006-07 680 1,110

5Number of SFUSD students receiving tutoring from a Peer Resources program 2005-06 2006-07 542 983

6 Number of Wellness Youth Outreach Workers 2007-08 2007-08 0 117

7 Number of school and district-wide events that were interpreted by the SFUSD's Translation and Interpretation Unit

2007-08 2006-07 34 544

8 Number of translation requests from school sites and central office fulfilled

2007-08 2006-07 177 924

9Number of pages translated (Chinese, Spanish, Tagalog, Russian, Vietnamese, Arabic & Samoan) 2007-08 2006-07 1,489 4,180

10Number and percent of ES students receiving arts instruction through the PEEF funded Visual and Performing Arts Generalist program

2005-06 2004-05 0 12,626 (56%)

11Number and percent of middle school students enrolled in Visual and Performing Arts classes 2005-06 2004-05 6,313 (50%) 6,686 (61%)

12Number and percent of high school students enrolled in Visual and Performing Arts classes 2005-06 2004-05 8,476 (44%) 7,348 (43%)

13 Number of students participating in mentoring programs at school sites through Student Support Professionals

2005-06 2006-07 0 334

14 Number of students who are peer mentors/peer leaders 2005-06 2006-07 819 760

15 Number of students participating in support groups through Peer Resources

2005-06 2006-07 510 55

16Number of students participating in site-based workshops, peer education, and special projects through Peer Resources 2005-06 2006-07 26,101 13,921


Table 5. PEEF Initiative Outputs Related to School Engagement: Change Over Time

Outputs: Student Engagement 1st year of PEEF

Baseline*Trend**



Outputs 4: Student Physical and Mental Health What are the levels of offerings and participation related to student physical and mental health, and

what is their distribution?

Student physical and mental health refers to students’ physical fitness, and health knowledge and behaviors as well as their socio-emotional health, levels of stress, sense of self-efficacy and confidence. Table 6 presents the data relating to how PEEF is impacting the levels of offerings, participation, and distribution that concern student physical and mental health. Of the 27 measures, 25 show a desirable trend over time, demonstrating that PEEF has increased student access to mental and physical health services and offerings.

With PEEF funding, the athletics department has staffed athletic trainers to teach students about injury prevention and to address sports injuries. In 2006-07, PEEF-funded athletic trainers provided preventative or injury treatment to 177 students, 5 percent of all high school athletes. In 2011-12, athletic trainers provided preventative or injury treatment to 705 students, 18 percent of high school athletes.

Student Support Professionals (SSP) have been found to function as a school resource, working together with other school programs and services to positively influence students’ academic achievement (Stone, Shields, Hilinski, & Sanford, 2012). In 2011-12, PEEF funded 37 student support professionals (including school social workers and School District Nurses) at K-5, K-8 and middle school sites. The number of students receiving individual and/or group services from PEEF funded SSPs increased from 1,742 in 2007-08 to 6,253 in 2011-12. There were also 5,075 teacher consultations, 2,282 student behavioral plans, and 2,183 students referred to Student Assistance Programs or Student Success Teams. Declines in these SSP services and referrals may be attributable to a change in the way students are referred to SAP, and to the fact that SSP has identified and provided individual and group services to students at-risk, therefore reducing the need for referrals. Declines may also be a function of a decrease in program funding and reallocation of staff resources to other projects.

With PEEF funding, the Wellness Initiative increased its services to SFUSD students by expanding to 4 additional schools in 2007-08. Prior to PEEF funding, Wellness centers provided 33,263 of direct services hours to students, in 2011-12 that number has increased to 50,442. The number of students receiving individual and/or group services at Wellness centers also increased from 4,831 in 2006-07 (pre-PEEF) to 7,487 in 2011-12. The most commonly utilized Wellness services are medical care, case management, behavioral and general counseling, and health education. The decrease in utilization of health education services, which are typically provided through partnerships with community based organizations, is primarily the result of decreases in funding from outside funding sources.


2011-12

Year N (%) N (%)

1 Number of athletic trainers serving high school athletic teams 2005-06 2004-05 0 5

2 Number and percent of high school athletes receiving preventative or injury treatment from an athletic trainer

2005-06 2004-05 0 705 (18%)

3 Number of athletic facility improvements supported by PEEF funds (cumulative)

2005-06 2004-05 0 10

4 Number of elementary schools receiving instructional equipment that compliments Physical Education curriculum

2005-06 2004-05 0 35

5Number of middle and high schools receiving a Fitness Lab (cummulative) 2005-06 2004-05 0 18

6 Number of Student Support Professionals that are PEEF funded 2005-06 2007-08 33 37

7 Number of teacher consultations provided by Student Support Professionals

2005-06 2007-08 6,563 5,075

8 Number of student behavioral plans developed by Student Support Professionals and teachers

2005-06 2007-08 4,364 2,282

9Number of students referred to funded Student Assistance Programs or Student Success Teams by Student Support Professionals

2005-06 2007-08 2,995 2,183

10Number of classroom presentations made by Student Support Professionals related to health and mental health promotion and other topics

2005-06 2007-08 734 853

11Number of student referred to Student Intervention Team for mental health services (including as part of their IEP) by Student Support Professionals

2005-06 2007-08 361 182

12 Number of outreach calls and meetings with parents with Student Support Professionals

2005-06 2007-08 5,963 6,920

13 Number of student/families referred to funded community agencies for services

2005-06 2007-08 3,541 1,698

14 Number of parent presentations offered by Student Support Professionals

2005-06 2007-08 58 68

15 Number of direct services hours provided to students at Wellness centers

2007-08 2006-07 33,263 50,442

16 Average number of direct services hours per student at Wellness centers

2007-08 2006-07 6.2 6.7

17 Number of YOW health education classroom presentations 2007-08 2006-07 0 201


Table 6. PEEF Initiative Outputs Related to Student Physical and Mental Health: Change Over Time

Outputs: Student Physical and Mental Health 1st year of PEEF

Baseline*Trend**


2011-12

Year N (%) N (%)

18 Number and percent of middle school students participating in athletic teams

2005-06 2004-05 2,811 (25%) 2,655 (25%)

19 Number and percent of high school students participating in athletic teams

2005-06 2004-05 3,827 (20%) 3,871(23%)

20Number and percent of elementary, K-8, and middle school students receiving individual and/or group health and mental health services through Student Support Professionals

2005-06 2007-08 1,742 (4.8%) 6,253 (17%)

21 Number of students receiving individual and/or group services at Wellness Centers

2007-08 2006-07 4,825 7,487

22 Number of students receiving 5 or more counseling sessions at Wellness centers

2007-08 2006-07 1,144 1,944

23 Number of SFUSD students receiving medical services at Wellness Centers

2007-08 2006-07 2,112 3,844

24 Number of SFUSD students receiving behavioral counseling services at Wellness Centers

2007-08 2006-07 1,448 2,188

25 Number of SFUSD students receiving general counseling services at Wellness Centers

2007-08 2006-07 1,852 1,965

26 Number of SFUSD students receiving case management services at Wellness Centers

2007-08 2006-07 1,479 2,886

27 Number of SFUSD students receiving health education services at Wellness Centers

2007-08 2006-07 989 739

Table 6 Continued. PEEF Initiative Outputs Related to Student Physical and Mental Health: Change Over Time

Outputs: Student Physical and Mental Health 1st year of PEEF

Baseline*Trend



Outputs 5: Academic Achievement What are the levels of offerings and participation related to academic achievement, and what is their

distribution?

The PEEF-initiative logic model depicts a chain of reasoning that links all PEEF-related outputs to the long-term outcome of academic achievement. These linkages, however, are not always direct and some outputs are linked to academic achievement indirectly, through immediate outcomes. Outputs described in of the preceding four sections of this report are linked to academic achievement through the immediate outcomes of increased professional and instructional capacity and quality, school climate, school engagement, and student physical and mental health.

This section describes the outputs that are considered to be directly connected to academic achievement, such as tutoring, enrollment in community college courses, and involvement as Teacher Academy Aides. Table 7 presents trends in academic achievement outputs impacted by PEEF programs. All eight measures presented show positive trends over time providing evidence that PEEF is offering SFUSD students increased access to opportunities that enhance academic achievement.

Since 2007–08, PEEF has funded a 1.0 FTE Teacher on Special Assignment within the Career Technical Education (CTE) program whose main task has been to facilitate collaboration between City College of San Francisco and SFUSD and internship opportunities for CTE students. Since PEEF began funding for CTE, the number of Career Technical Education City College courses for students has increased from two in 2006-07 to 80 in 2011-12. The number of high school seniors completing at least two Community College of San Francisco (CCSF) courses has also increased as has the number of students completing internships through CTE .

The Teacher Academy program has provided students opportunities to gain teaching experience by providing tutoring services as teacher’s aides in classrooms and to earn transferrable college credits through their dual enrollment program with CCSF. Since PEEF funding began, the number of Teacher Academy aides has increased from 83 in 2007-08 to 102 in 2011-12. The number of classes with Teacher Academy Aides also increased from 28 in 2007-08 to 50 in 2011-12. Consequently, the number of students tutored by Teacher Academy aides nearly doubled from 850 to 1500 within that same time period.


2011-12

Year N (%) N (%)

1 Number of city college of SF course offering for 11th and 12th grade students organized by Career Technical Education

2007-08 2006-07 2 80

2 Number of students completing an internship through Career Technical Education

2007-08 2006-07 155 224

3 Percent of Teacher Academy Aides graduating with 6 college units

2007-08 2009-10 20% 98%

4 Number of classes with Teacher Academy Aides 2007-08 2007-08 28 50

5 Number of students tutored by Teacher Academy Aides 2007-08 2007-08 850 1,500

6 Number of high school seniors completing at least two Community College of San Francisco courses

2007-08 2009-10 84 102

7 Number of Teacher Academy Aides 2007-08 2007-08 83 102

8 Percent of Teacher Academy Aides graduating with 2 or more college units

2007-08 2009-10 80% 98%

*Baseline data represents year prior to PEEF funding or 1st year of available data.**Blue upward pointing arrows represent desirable trends, orange downward pointing arrows represent undesirable trends.

Table 7. PEEF Initiative Outputs Relevant to Academic Achievement: Change Over Time

Outputs: Academic Achievement 1st year of PEEF

Baseline*Trend**


Outcomes: What are the outcomes associated with the PEEF initiative? The third and fourth boxes in the PEEF district-wide logic model depict immediate and long-term outcomes. If the outputs described in the prior section are provided and students and staff participate in them, then a set of immediate outcomes would be expected to improve. If progress is made toward these immediate outcomes—schools deliver higher quality instruction and have more highly qualified staff, more students feel safe in clean, orderly, and well-maintained schools, more students and parents are engaged and connected to schools, and more students are in good physical and mental health—then it is reasonable to expect that student academic achievement will improve.

This section begins with a description of PEEF-initiative immediate outcomes, presented by the four theme areas of professional and instructional capacity and quality , school climate, school engagement, and student physical and mental health, followed by outcomes directly related to academic achievement.

Immediate Outcomes 1: Professional and Instructional Capacity and Quality Has professional and instructional capacity and quality and increased since the advent of PEEF?

Staff qualifications, skills, and knowledge are central to the functioning of a strong school district. Increased professional and instructional capacity and quality has been shown to be associated to increased student academic achievement, including higher reading and math levels, and reduced gaps in student learning frequently associated with differences in household income and race/ethnicity. Table 8 displays data depicting changes in measures of professional and instructional capacity and quality since the advent of PEEF. Positive trends can be observed for 13 of the 16 measures presented. Table 9 reports student, teacher, principal, and parent satisfaction data from program and district-wide surveys administered in 2011–12.

PEEF is impacting professional and instructional capacity and quality in SFUSD in a number of ways. The Human Capital Support program (formerly known as the Teacher Recruitment program), is responsible for recruiting, selecting, and placing new teachers in SFUSD schools. Since PEEF funding for this program began in 2007-08, more newly hired teachers meet the No Child Left Behind standards for highly qualified; 98% of newly hired teachers met the standards in 2011-12. The diversity of newly hired teachers has also improved. African American and Latino teachers make up 5% and 11% respectively, of the total teacher population in the SFUSD. In 2007–08 only 3 percent of newly hired were African Americans, in 2011-12 that percentage increased to 5; for Latinos the percentage increased from 7% to 14%. In 2011-12, 6 percent of teaching job applications came from African Americans and 12 percent from Latinos. In 2007-2008, there were 46.4 classroom vacancies on the first day of school, in 2011-12 that number decreased to 2. There were also nine teachers hired as part of the Teacher Residency Program in 2011-12, 7 of whom were placed in hard to staff schools.

The Formative Assessment program facilitates the creation of Common Learning Assessments (CLA) and SFUSD teachers have been increasingly using these assessments to inform their instruction. In 2010-11, 33,000 SFUSD students participated in English Language Arts (ELA) CLA and 35,000 participated in the Math CLA, this was an increase from the prior year. The number of teachers who used at least one CLAs to assess student achievement jumped from 10% (300) to 95% (2,192) in 2011-12. Use of OARS/Data Director to access student data has also increased at SFUSD. In 2007-08, there were 750 staff with at least 1 log-in to OARS/Data Director, in 2011-12, there were 3,500 staff with at least 1 log-in to OARS/Data Director. The average number of log-ins per user in 2011-12 was 35. The Formative


Assessment program also provides staff with training and support on the use of assessment data to inform instruction. In 2011-12 there were 1,154 participants at 63 trainings held. Declines in number of trainings from 2009-10 are due to a shift in focus of the program. As teachers across the district have increased their general understanding of formative assessment practices, are widely participating in district interim assessments, and regularly access student data, the Formative Assessment program has shifted much of its support focus to providing teachers with formative assessment tools and resources to inform instruction and planning.

2011-12

Year N (%) N (%)

1Number and percent of newly hired teachers who meet the No Child Left Behind standards 2007-08 2007-08 300 (90%) 346 (97%)

2 Number and percent of SFUSD teachers who are African American 2007-08 2008-09 175 (5%) 175 (5%)

3 Number and percent of SFUSD teachers who are Latino 2007-08 2008-09 350 (10%) 385 (11%)

4 Number and percent of newly hired teachers who are African American

2007-08 2007-08 10 (3%) 17 (5%)

5 Number and percent of newly hired teachers who are Latino 2007-08 2007-08 20 (7%) 50 (14%)

6Number of teaching job applications received by SFUSD from African American applicants 2007-08 2010-11 90 122

7Number of teaching job applications received by SFUSD from Latino applicants 2007-08 2010-11 235 245

8 Number of classroom vacancies on the first day of school (in FTEs) 2007-08 2007-08 46.4 2


Table 8. PEEF Initiative Outcomes Related to Professional and Instructional Capacity and Quality: Change Over Time

Outcomes: Professional and Instructional Capacity and Quality 1st year of PEEF

Baseline*Trend**


Table 9 reports measures of student, teacher, principal, and parent satisfaction with PEEF programming affecting professional and instructional capacity and quality as well as other measures of program impact.

With PEEF funding, the library department has been able to increase the number of teacher librarians and professional development trainings. In the 2011-12 annual survey of teacher librarians administered by the Library Services department, 88% of teachers librarians (N:60) strongly agreed or agreed that the professional development trainings offered by library services had helped them become a more effective librarian.

Physical Education support at the elementary school level has focused not only on improving student access to quality Physical Education but also on promoting opportunities for students to engage in physical activities and improve their physical skills. To this end the Physical Education Department has allocated physical education specialists, equipment, and professional development offerings to school sites. In the 2011-12 Elementary school survey administered by the Physical Education department, 84% of the elementary classroom teachers (N: 161) at schools receiving physical education specialist support and 83% of secondary school PE teachers (N:56) agreed or strongly agreed that the professional development provided by the PE department helped them become better teachers of Physical Education.

The Formative Assessment program also administers annual surveys to SFUSD teachers to receive feedback on their program services and assessment tools. In their 2011-12 annual survey, 73% of teachers surveyed (N: 386) strongly agreed or agreed that they knew how to access CLA results from Data Director and make meaning out of them.

2011-12

Year N (%) N (%)

9 Number of students participating in ELA common assessments 2007-08 2010-11 17,817 33,000

10 Number of students participating in Math common assessments 2007-08 2010-11 26,717 35,000

11Number of CCSS-aligned performance assessments given to assess student achievement 2007-08 2010-11 0 27,132

12Number and percent of teachers using at least one of the district's common assessments to assess student achievement 2007-08 2005-06 300(10%) 2,192 (95%)

13 Number of professional development on the use of assessment data to inform instruction

2007-08 2009-10 172 63

14 Number of SFUSD staff provided with training and support on the use of assessment data to inform instruction

2007-08 2009-10 3,573 1,154

15 Number of users with at least 1 log-in to Data Director/OARS 2007-08 2007-08 750 3,500

16 Average number of Data Director/OARS log-ins per user 2007-08 2008-09 4.5 35


Table 8 Continued: PEEF Initiative Outcomes Related to Professional and Instructional Capacity and Quality: Change Over Time

Outcomes: Professional and Instructional Capacity and Quality 1st year of PEEF

Baseline*Trend**



1 84%

2 91%

3 90%

4 83%

5 88%

6 73%

7 57%

8 99%

9 99%

10 89%

Percent of secondary PE teachers who strongly agree/agree that the professional development provided by the SFUSD's PE department has helped them become a better teacher of physical education. (PE Department Secondary PE Teacher Survey, N:56)

Table 9: PEEF Initiative Outcomes Related to Professional and Instructional Capacity and Quality

Instructional and professional capacity and quality: 2011-12 Snapshots

Percent of ES teachers at schools at schools with PE program support option 1 who strongly agree/agree that the professional development provided by the site-based PE specialist has helped them become a better teacher of physical education. (PE Department ElementaryTeacher Survey; N:161)

Percent of ES teachers at schools with PE program support option 1 who strongly agree/agree receiving program support from PE department has helped them improve their ability to teach students physical skills such as kicking, swinging, and throwing. (PE Department ElementaryTeacher Survey; N:161)

Percent of ES teachers at schools with PE program support option 1 who strongly agree/agree that receiving program support from PE department has helped them improve their ability to teach students how to maintain and increase a level of physical fitness to improve their health. (PE Department ElementaryTeacher Survey; N:161)

Percent of SFUSD teachers who strongly agree/agree that teachers at their school work together to improve their instructional practice (Staff Satisfaction Survey; N: 1,695)

Percent of Teacher Librarians who strongly agree or agree that professional development trainings offered by Library Services helped them become a more effective librarian. (Library ServicesTeacher Librarian Survey; N: 60)

Percent of teachers who strongly agree/agree that they know how to access Common Learning Assessment (CLA) results on Data Director and to make meaning out of the results. (Formative Assessment Teacher Survey, N:386)

Percent of teachers who strongly agree or agree that they used Common Learning Assessment (CLA) results to inform instruction and planning. (Formative Assessment Teacher Survey; N:386)

Percent of SFUSD principals who were satisfied with the quality of the human capital workshop sponsored by the Teacher Recruitment department. (Human Capital Support Satisfaction Survey; N: 84)

Percent of SFUSD principals who were satisfied with the overall customer services provided by their Human Capital Specialist (Human Capital Support Satisfaction survey; N:108)


Immediate Outcomes 2: School Climate Has school climate improved since the advent of PEEF?

Aspects of school climate, including students’ feeling safe at school, fewer school-wide behavioral incidents, and overall cleanliness have been associated with increased achievement, including higher grades, lower absenteeism, stronger performance on standardized achievement tests, and high school completion rates. Several outcome measures related to school climate are presented in this section, including student suspension rates, principals’ perceptions of school cleanliness, staff, parents’, and students’ perceptions of feeling safe and respected at school. Table 10 shows the data relating to the question of whether school climate has improved since the advent of PEEF. Of the four measures in table 10, two display a desirable trend and one measure remains flat.

A review of SFUSD suspension data from the year prior to the implementation of PEEF funding and in 2011–12, shows a decrease in the percent of students who are suspended (3.1% in 2004–05 and 2.6% in 2011–12). The suspension rate of elementary school students however increased slightly (from 0.7% in 2004–05 to 0.8% in 2011-12). The suspension rates of middle school students show an overall stable trend (6.1% in both 2004–05 and 2011-12). The suspension rate of high school students decreased from 4.4% in 2004-05 to 3.1% in 2011-12.

Table 11 includes 2011–12 snapshots of additional data related to School Climate. In 2011–12, 9 out of 10 Elementary school principals at schools with PEEF funded custodians and 14 out of 15 Early Education Department Center Site managers who completed the Custodial Services survey (N: 25) reported being either very satisfied or satisfied with the overall cleanliness of their school site. The SFUSD administers annual satisfaction surveys to staff, parents, and students to gain feedback on how the school district is adhering to its strategic plan. In 2011-12, 87% of parents who participated in the SFUSD Family Satisfaction survey (N: 13,654), strongly agreed or agreed that their child’s school was a safe place. Students were also asked about their feelings of safety at school. In the 2011-12 Student Satisfaction survey, 75% of 5th graders (N: 3,260), 60% of 8th graders (N: 2,639), and 73% of 11th graders (N: 2,595) reported feeling safe at school. The quality of interpersonal relationships and exchanges between teachers and students and among students is also a measure of school climate. In the 2011-12 Student Satisfaction survey, 87% of 5th graders (N: 3,260), 72% of 8th graders (N: 2,639), and 80% of 11th

2011-12

Year % %

1 Percent of students suspended district wide 2004-05* 3.1% 2.6%

2 Percent of SFUSD ES students suspended 2004-05* 0.7% 0.8%

3 Percent of SFUSD MS students suspended 2004-05* 6.1% 6.1%

4 Percent of SFUSD HS students suspended 2004-05* 4.4% 3.1%


Table 10. PEEF Initiative Outcomes Related to School Climate: Change Over Time

Outcomes: School Climate Baseline*

Trend**



graders (N: 2,595) reported that their teachers treated students with respect. Peer-to-peer relationships, however, were perceived to be less than optimal; only 48% of 5th graders (N: 3,260), 44% of 8th graders (N: 2,639), and 63% of 11th graders (N: 2,595) strongly agreed or agreed that students at their school treated each other with respect.

Since 2005–06, PEEF has funded programs that contribute to improving school climate. The positive trends in indicators of school climate included in this report, e.g., drop in school suspension rates, as well as the number of students reporting feeling safe and respected at school provide evidence of PEEF’s contribution to the improvement of school climate, although with the understanding that work continues to need to be done, particularly related to peer-to-peer relationships.

1 92%

2 84%

3 86%

4 75%

5 60%

6 73%

7 87%

8 72%

9 80%

10 87%

Percent of SFUSD 8th graders who agree and strongly agree that they always feel safe at school. (SFUSD Student Satisfaction Survey; N:2639)

Table 11: PEEF Initiative Outcomes Related to School ClimateSchool Climate: 2011-12 Snapshots

Percent of SFUSD staff who agree and strongly agree that their school is a safe place for teaching from early in the morning until late at night. (SFUSD Staff Satisfaction Survey;N: 2679)

Percent of SFUSD parents who agree and strongly agree that their child's school is a safe place from early in the morning to late at night. (SFUSD Family Satisfaction Survey;N:13654)


Percent of principals and Early Education Department Center Site managers who reported being either very satisfied/ satisfied with the overall cleanliness of their school site. (Custodial Services Survey; N: 25)

Percent of SFUSD staff who agree and strongly agree that school staff values and builds on studetns' language, cultures, and lived experiences. (SFUSD Staff Satisfaction Survey;N: 2679)


Percent of SFUSD 5th graders who agree and strongly agree that teachers treat students with respect. (SFUSD Student Satisfaction Survey; N:3260)




Immediate Outcomes 3: School Engagement Has school engagement increased?

This section addresses the question of whether students’ school engagement has improved since the advent of PEEF. Students’ school engagement, school connectedness, and motivation toward school have been demonstrated to have significant impacts on academic achievement, particularly for at-risk urban students. Table 12 presents trend data and table 13 presents 2011-2012 data from program and district surveys related to school engagement. All five measures included in table 12 demonstrate a positive trend.

A strong indicator of student engagement is attendance rates. Since the advent of PEEF, district-wide attendance rates have increased from 92% in 2004-05 to 94% in 2011-12. Positive trends can be observed at all school levels, with the greatest increase in attendance at the high school level (86% in 2004-05 to 89% in 2011-12). Students at the SFUSD also participated in other aspects of schooling that fostered engagement, including the Annual SFUSD Arts Festival, sponsored by the Visual and Performing Arts program. In 2011-12, 9,693 students participated in this event.

Table 13 includes 2011–12 snapshots from program and district surveys related to school engagement. Survey results show high levels of student and parent engagement in school. Students reported high levels of satisfaction with academic programing at their schools; seventy five percent of 5th graders, sixty four percent of 8th graders, and seventy seven percent of 11th graders either strongly agreed or agreed that the courses they were taking were engaging and challenging. High levels of social engagement were also noted for both students and parents. The majority of students surveyed felt that their teachers cared about their success; 86 % of 5th graders, 79% of 8th graders and 77% of 11th graders either strongly agreed or agreed that school staff cared about the success of all the students at their school. Ninety two percent of parents also reported feeling like their children’s’ school provided a caring, inclusive, and engaging environment for learning. Communication between school and home is essential for parental involvement and an important aspect of school engagement. PEEF funds have been essential in providing resources through the Translation and Interpretation department to facilitate communication between SFUSD and

2011-12

Year N (%) N (%)

1Number of students participating in the SFUSD Annual Arts Festival 2005-06 2006-07 6,097 11,306

2 2004-05 92% 94%

3 2004-05 95% 96%

4 2004-05 94% 95%

5 2004-05 86% 89%

ES attendance rate

MS attendance rate

HS attendance rate


District wide attendance rate

Table 12. PEEF Initiative Outcomes Related to School Engagement: Change over Time

Outcomes: School Engagement 1st year of PEEF

Baseline*Trend**


non-English speaking parents. Ninety-three percent of parents who have used translation and/or interpretation services report being very satisfied/satisfied with these services. Additionally, eighty-five percent of parents who participated in the 2011-12 SFUSD Family Satisfaction survey strongly agreed/agreed that communication between home and school was frequent, useful, and in their home language. The impact of programs on the academic and social engagement of students was also noted from survey data of student participants. Ninety three percent of Wellness Center clients surveyed (N: 198) reported that there was an adult in the Wellness program who really cared about them.

1 86%

2 70%

3 93%

4 85%

5 75%

6 64%

7 77%

8 92%

9 86%

10 79%

11 80%

12 80%

Percent of SFUSD parents who strongly agree/agree that communication between home and school is frequent and useful and in the home language. (SFUSD, Family Satisfaction Survey;N:13,654)

Table 13. PEEF Initiative Outcomes Related to School Engagement: Snapshots

Outcomes: School Engagement - 2011-12 Snapshots

Percent of SFUSD parents who strongly agree/agree that schools offer a wide variety of high-quality of courses and activities during the school day that their children enjoy. (SFUSD, Family Satisfaction Survey; N:13654)

Percent of SFUSD parents who strongly agree/agree that they attended and actively participated in relgularly scheduled meetings, events, and adult educational opportunities. (SFUSD, Family Satisfaction Survey;N:13,654)

Percent of parents who are either very satisfied or satisfied with interpretation services (Translation and Interpretation Services, Parent Satisfaction Survey; N: 421)

Percent of 11th grade students who strongly agree/agree that their teachers and other school staff care about the success of all the students at their school. (SFUSD Student Satisfaction Survey; N:2,595)

Percent of 5th grade students who strongly agree/agree that the courses that they are taking are engaging and challenging. (SFUSD Student Satisfaction Survey; N:3,260)



Percent of SFUSD parents who strongly agree/agree that their children's schools provide a caring, inclusive, and engaging environment for learning. (SFUSD, Family Satisfaction Survey; N:13,654)



Percent of Wellness clients reporting that there is an adult in the Wellness program that really cares about them (Wellness participant survey; N: 198)


Immediate Outcomes 4: Student Physical and Mental Health Has student physical and mental health improved?

Improved students’ physical and mental health has been positively related to academic outcomes, including improved grade performance and increased high school persistence. Table 14 shows change data relating to the question of whether student physical and mental health has improved since the advent of PEEF and table 15 presents 2011-2012 responses from student surveys related to physical and mental health Of the six measures included in table 14, four show a positive trend.

As outlined in the physical and mental health output section of this report, PEEF is increasing student access to a variety of services and offerings that promote physical and mental health (e.g., Physical Education, Athletics, Student Support Professionals and Wellness Centers).

Through funding in Physical Education and Athletics, PEEF has increased students’ access to quality Physical Education instruction and sports which has translated into increased opportunities for students to engage in physical activities. We expect that through increased engagement in physical activities, students’ level of fitness will increase. The California Physical Fitness Test is a measure of physical fitness that is administered to students in the SFUSD. It is comprised of six different tests: Aerobic Capacity, Body Composition, Abdominal Strength and Endurance, Trunk Extensor Strength and Flexibility, Flexibility, and Upper Body Strength and Endurance. In order to pass the California Fitness Test, students must score in the Healthy Fitness Zone in five out of six tests.

The percentage of 9th graders who have passed the California Physical Fitness Test has increased since 2004–05 (from 53% in 2004–05 to 58% in 2011–12).There was an overall increase in the percent of students scoring in the Healthy Fitness Zone for Aerobic Capacity from 2004-05 to 2011-12. The percent of 5th grade students scoring in the Healthy Fitness Zone increased by 8% from 64% in 2004-05 to 72% in 2011-12; the percent of 7th grade students scoring in the Healthy Fitness Zone increased by 9% (from 66% to 75%); the percent of 9th grade students scoring in the Healthy Fitness Zone increased by 26% (from 49% to 75%). The greatest gains in overall physical fitness were seen at the high school level among 9th graders.


Table 15 shows snapshots of survey data from students collected in 2011-12. These data provide a glimpse of how PEEF funding has impacted the physical and emotional health of SFUSD students.

Children’s attitudes about physical fitness and health have been impacted by PE support at the elementary school level. In the secondary student survey administered by the PE department, 66% of students (N:4834) reported that participation in physical education helped them make healthier life choices. Eighty-one percent of elementary school students at schools with PE Program Support Option 1 (N: 1093) felt that PE class helped them improve their physical skills.

Student’s socio-emotional health has also been impacted by PEEF-funding for Wellness Centers and Peer Resources programs. In 2011-12, the Wellness program surveyed a sample of students receiving five or more counseling sessions at their Wellness Centers. Survey participants (N:202) reported feeling better about themselves after receiving services (74%) and improved coping skills (70%). They also reported learning information about ways to improve their health from Wellness Centers (80%), including ways to reduce stress in their lives (73%), and information about the effects of alcohol, tobacco and drugs (61%). Additionally, Wellness clients reported being better informed about community resources available to them for their health and mental health needs (75%).

2011-12

Year % %

1Percent of 5th grade students who pass the CA Physical Fitness Test (pass 5/6 or 6/6 fitness standards) 2004-05 58% 46%



4Percent of 5th grade students meeting Aerobic Endurance Standard 2004-05 64% 72%




Table 14. PEEF Initiative Outcomes Related to Student Physical and Mental Health: Change Over Time

Outcomes: Student Physical and Mental HealthBaseline*

Trend**


1 76%

2 66%

3 81%

4 74%

5 70%

6 80%

7 91%

8 73%

9 61%

10 63%

11 77%

12 75%

Percent of Wellness clients (students who receive 5 or more counseling sessions) who agreed/strongly agreed that they learned new information about the effects of using tobacco, alcohol and other drugs. (Wellness Center Client Survey; N:202)

Of Wellness clients who learned new information about the effects of tobacco, drugs and alcohol (n:122), percent who strongly agreed/agreed that they learned new ways to reduce their use. (Wellness Center Client Survey; N:202)

Of Wellness clients who learned information about ways to reduce their use of tobacco, alcohol and other drugs (n:75), percent who strongly agreed/agreed that they attempted to reduce their use. (Wellness Center Client Survey; N:202)

Percent of Wellness clients (students who receive 5 or more counseling sessions) who agree or strongly agree that they learned about organizations/clinics in the community that they could go to for help if they needed it. (Wellness Center Client Survey; N:202)

Percent of Wellness clients (students who received 5 or more counseling sessions) who strongly agreed/agreed that they were able to cope when things go wrong after receiving services from the Wellness Center at their school. (Wellness Center Client Survey; N:202)

Percent of Wellness clients (students who received 5 or more counseling sessions) who strongly agreed/agreed that they learned information about how to improve their health and well-being. (Wellness Center Client Survey; N:202)

Among the Wellness clients who strongly agreed/agreed that they learned information about how to improve their health and well-being (n: 159), percent who strongly agreed or agreed that they planned to take steps to improve their health. (Wellness Center Client Survey)

Percent of Wellness clients (students who receive 5 or more counseling sessions) who strongly agreed/agreed that they learned new ways to reduce stress in their lives through direct services from the Wellness Center at their school. (Wellness Center Client Survey; N:202)

Table 15. PEEF Initiative Outcomes Related to Student Physical and Mental Health

Student Physical and Mental Health: 2011-12 Snapshots

Percent of middle and high school students who strongly agreed/agreed that participation in physical education activities helped to improve their physical fitness (PE Department Secondary Student Survey; N:4834)

Percent of middle and high school students who strongly agreed/agreed that because of what they have learned in physical education class, they made healthier choices in their lives (PE Department Secondary Student Survey; N:4834)

Percent of ES students who strongly agreed/agreed that their PE class helped them improve their physical skills, such as kicking, swinging, running, throwing, and catching (PE Department Elementary Student Survey; N:1093)

Percent of Wellness clients (students who received 5 or more counseling sessions)who strongly agreed/agreed that they felt better about themselves after receiving direct services from the Wellness Center at their school (Wellness Center Client Survey; N:202)


Long-Term Outcomes: Academic Achievement Has academic achievement increased?

As described in the logic model section of this report and supported by the research literature, each of the four immediate outcomes presented in the previous sections—increased professional and instructional capacity and quality, school climate, school engagement, and student health—has been shown to increase student academic achievement. Following the logic of the PEEF district-wide logic model, if increases are seen in the immediate outcomes, it is expected that increases will be seen in the academic achievement of SFUSD students.

Table 16 presents data relating to the question of whether students’ academic achievement has improved since the advent of PEEF, focusing on three aspects of achievement: library book circulation ratios, GPAs, and performance on California Standardized Tests (CSTs). In addition, this section presents responses to program and district survey questions related to academic achievement (table 17).

Of the 14 measures included in table 16, 13 show a positive trend. Since PEEF began funding libraries in 2005–06, overall book circulation in SFUSD has increased from 6.5 books per student 2004–05 (pre-PEEF) to 16.5 books per student in 2011–12. This is an indication that students are reading more. Higher book circulation rates have been positively associated with improved academic achievement (Greaney, 1980) (Anderson, Wilson, & Fielding, 1988). Book circulation rates increased in elementary schools (8.1 vs. 29 books per student), K-8 schools (19.2 vs. 8.2) and middle schools (7.3 vs. 6.8). Only high schools did not demonstrate increased book circulation. High school book circulation rates were much lower than elementary school rates (3.1 vs. 29.0, respectively). It is possible that high school students prefer to utilize digital media rather than print media to read and conduct academic research. For example, online database usage at the high school level in 2011-12 was much higher than at the elementary school level, (3.5 compared to 1.0 sessions per student).

The average district-wide GPA for grades 6th to 12th increased from 2.6 in 2004-05 to 2.9 in 2011-12. The percentage of students scoring at or above the proficiency level in CST English Language Arts (ELA) and in mathematics increased district-wide and across grade levels between 2004-05 and 2011-12. District-wide, the percentage of students (grades 2-11) scoring at or above proficiency level increased from 46% to 61% in ELA, and from 48% to 58% in math. At the elementary school level, the percentage scoring at or above proficiency increased from 45% to 62% in ELA and from 57% to 70% in math. Increases also were evident at the middle school (ELA: 49% to 63, math: 50% to 56%) and high school levels (ELA: 46% to 57%, math: 34% to 40%).


Table 17 includes 2011-12 snapshots from program and district surveys related to academic achievement. Wellness clients (students who received 5 or more counseling sessions) report that participation in Wellness Center programs has helped them improve their academic performance. In the 2011-12 Wellness client survey, 65% of Wellness clients (N: 202) strongly agreed or agreed that they did better in school after receiving services through their Wellness Center.

Study skills are essential for achieving good academic performance, and students at the SFUSD reported having learned these skills at school. In the 2011-12 Student Satisfaction Survey, 90% of 5th graders (N: 3,260), 65% of 8th graders (N: 2,639) and 63% of 11th graders (N: 2,595) strongly agreed/agreed that they learned good study skills and habits at school. Additionally, 59% of 8th graders (N: 2,639) and 67% of 11th graders (N: 2,595) strongly agreed/agreed that their teachers and school staff prepared them well for college and a career.

SFUSD staff also report confidence in their ability to prepare students for college and/or careers. In the 2011-12 SFUSD Staff Satisfaction Survey, 84% percent of staff (N: 2,679) reported that they strongly agreed/agreed that teachers at their schools had the skills and knowledge to prepare students for post-

2011-12

Year N (%) N (%)

1 Overall book/student circulation ratio in SFUSD 2004-05 6.5 16.5

2 Book/student circulation ratio in elementary school 2004-05 8.1 29

3 Book/student circulation ratio in K-8 2004-05 8.2 19.2

4 Book/student circulation ratio in middle school 2004-05 6.8 7.3

5 Book/student circulation ratio in high school 2004-05 4.2 3.1

6 District wide GPA 2004-05 2.6 2.9

7Percent of SFUSD students (grades 2-11) scoring at or above proficiency level on the CST English Language Arts test 2005-06 48% 61%

8Percent of SFUSD students (grades 2-11) scoring at or above proficiency level on the CST math test 2005-06 48% 58%

9Percent of SFUSD elementary school students scoring at or above proficiency level on the CST English Language Arts test 2004-05 45% 62%

10Percent of SFUSD middle school students scoring at or above proficiency level on the CST English Language Arts test 2004-05 49% 63%

11Percent of SFUSD high school students scoring at or above proficiency level on the CST English Language Arts test 2004-05 46% 57%

12Percent of SFUSD elementary school students scoring at or above proficiency level on the CST math test 2004-05 57% 70%

13Percent of SFUSD middle school students scoring at or above proficiency level on the CST math test 2004-05 50% 56%

14Percent of SFUSD high school students scoring at or above proficiency level on the CST math test 2004-05 34% 40%


Table 16. PEEF Initiative Outcomes Related to Academic Achievement: Change Over Time

Outcomes: Academic Achievement Baseline*

Trend**



secondary education and careers in a high-skills economy. Parents had similar sentiments, 81% of parents who responded to the SFUSD Family Satisfaction Survey (N: 13,654) said that they strongly agreed/agreed that teachers at their children’s schools had the skills and knowledge to prepare them for college or a career.

1 65%

2 90%

3 65%

4 63%

5 59%

6 67%

7 84%

8 81%

Percent of 11th grade students who strongly agree/agree that they learned good study skills and habits at school (SFUSD Student Satisfaction Survey; N: 2595)

Table 17. PEEF Initiative Outcomes Related to Academic Achievement

Academic Achievement: 2011-12 Snapshots

Percent of 11th grade students who strongly agree/agree that their teachers ad school staff prepare them well for college and a career. (SFUSD Student Satisfaction Survey; N: 2595)Percent of SFUSD staff who strongly agree/agree that teachers at their schools have the skills and knowledge to prepare students for post-secondary education and for a high-skills economy. (SFUSD Staff Satisfaction Survey; N: 2679)

Percent of SFUSD parents who strongly agree/agree that teachers at their children's schools have the skills and knowledge to prepare their children for college or a career. (SFUSD Family Satisfaction Survey; N: 13,654)

Percent of Wellness Center clients who strongly agree/agree that they are doing better in school as a result of after receiving services through the wellness center (Wellness Client Survey; N: 202)Percent of 5th grade students who strongly agree/agree that they learned good study skills and habits at school (SFUSD Student Satisfaction Survey; N: 3260)

Percent of 8th grade students who strongly agree/agree that they learned good study skills and habits at school (SFUSD Student Satisfaction Survey; N: 2639)

Percent of 8th grade students who strongly agree/agree that their teachers and school staff prepare them well for college and a career. (SFUSD Student Satisfaction Survey; N: 2639)


Propensity Score Matching Study: Effects of Middle School and High School Athletics on Attendance, GPA and Suspension In Subsequent reports, PEEF will pursue student level data linked to academic achievement. Using available student level data for Athletics, this section of the report displays the type of analysis that can be conducted when programs have access to student level data.

The ideal way to measure the effects of a program or treatment would be to randomly assign participants to either a treatment or control condition. The effect of the treatment is then easily estimated because it is the difference in outcomes for the two groups. However, when participants become involved in a program through their own choice, rather than random assignment, there is no fully equivalent way to measure the effects of a treatment. In order to deal with these types of situations, researchers have developed a number of approaches that resemble randomly assigning participants, including creating matched treatment and control groups based on their propensity scores.

The nature of participation in Athletics makes it a program that be assessed for impact on its own by using matching. In order to study the effects of a program using matching, it is important to have student-level data, the program must be likely to be impactful on outcomes that are measured, and relevant student data must be available to match participants with a control group of equivalent non-participants. This study created a matched comparison groups composed of non-athletes that closely resembled high school and middle school athletes on a large number of demographic, academic, and school participation characteristics using a statistical technique called propensity score matching. For high school, two separate matching analyses were run, matching athletes and non-athletes based on their 5th grade (and then 8th grade) characteristics of ethnicity, poverty (free or reduced lunch), gender, special education, GATE, English learner status, attendance, GPA, and number of suspensions. When creating matched groups, the goal is to create two groups that are identical on measured characteristics immediately prior to participating in the program, in this case Athletics. Creating the match based on 8th grade characteristics allowed for groups that were the most similar just before they entered high school. However, matching on 8th grade characteristics meant that some students in each group would already have begun participating in middle school athletics and thus, while the match is done prior to high school athletics participation; it is not necessarily prior to all athletics participation. Because of this, a separate match was also done at 5th grade, which is prior to all formal school athletics participation. Matching over three years before high school, however, means that students may have changed somewhat before high school, in which case matching on 8th grade is preferable. So, results from both matches were used.

Matching means that athletes and their comparison group were quite similar before entering high school, thus allowing for a more accurate estimate of the effects of involvement in athletics than just comparing results based on one or two demographic characteristics (such as gender and race). The analysis looked at high school athletes and their matched comparisons in 2008-09, 2009-10, 2010-11, and 2011-12, with a total of 10,703 (of 10,884 possible instances) athletes compared with 39,935 matched non-athletes matched on 8th grade characteristics. The 5th grade analysis matched 6,349 athletes (of 6,371 possible instances) with 21,170 non-athletes.


As can be seen, athletes had better results than matched non-athletes on measures of attendance, GPA (not including PE), and number of suspensions, both for athletes of all races/ethnicities and for African-American and Latino students, specifically. These results were found whether the comparison group was matched on Grade 5 characteristics or Grade 8 characteristics. These differences were statistically significant (p<0.05) in all instances except for number of suspensions for African-Americans matched on Grade 8 characteristics. Keep in mind that at the time of matching, athletes had not only the same demographic characteristics as the matched group, but also averaged the same attendance, GPA, and number of suspensions. So, after participating in athletics, high school athletes demonstrated more engagement in school indicated by attendance (2.8% - 4.5% higher), higher grades (0.2 grade points higher), and received fewer suspensions (-0.02 suspensions/student). The positive results for participating in high school athletics were even stronger for African-American and Latino students than for athletes as a whole. African-American athletes had attendance rates that were 7.4%-7.8% higher than matched students, GPA 0.4-0.5 higher than matched students, and 0.05-0.08 fewer suspensions per student. Similarly, Latinos had attendance rates that were 4.2%-7.8% higher than matched students, GPA 0.3-0.5 higher than matched students, and 0.03-0.05 fewer suspensions per student. Thus, participant in athletics seemed to have an even larger effect on these traditionally underserved students. A similar analysis was conducted matching middle school athletes with matched non-athletes based on their 5th grade characteristics. The same factors as used to match high school athletes were included except for GPA, which is not available for elementary students in the district’s administrative database for elementary students. Again, athletes had positive results compared with matched non-athletes and all of

All 94.1% 89.6% 4.5% YES 0.001African Americans 87.2% 79.8% 7.4% YES 0.001Latinos 90.1% 82.3% 7.8% YES 0.001

All 3.0 2.8 0.2 YES 0.001African Americans 2.4 1.9 0.5 YES 0.001Latinos 2.6 2.1 0.5 YES 0.001

All 0.04 0.06 -0.02 YES 0.001African Americans 0.19 0.27 -0.08 YES 0.05Latinos 0.05 0.1 -0.05 YES 0.01

*Difference not significant.

Attendance

GPA

Number of Suspensions Per Student

Table 18: Attendance, GPA, and Suspensions for High School Athletes and Non-Athletes Matched on 5th grade characteristics(2008-09 to 2011-12)

Demographic Athletes Matched Non-Athletes Difference

Do Athletes out-perform their counterparts?

Significance


All 3.0 2.8 0.2 YES 0.001African Americans 2.3 1.9 0.4 YES 0.001Latinos 2.6 2.1 0.5 YES 0.001

All 0.04 0.06 -0.02 YES 0.001African Americans 0.21 0.26 -0.05 * *Latinos 0.06 0.08 -0.02 YES 0.05

Attendance

GPA


*Difference not significant.

Table 19: Attendance, GPA, and Suspensions for High School Athletes and Non-Athletes Matched on 8th grade characteristics(2008-09 to 2011-12)

Demographic Athletes Matched Non-Athletes Difference Do Athletes out-

perform their Significance


the differences were statistically significant. The sizes of the differences tended to be smaller than was found for high school athletes with regard to attendance and GPA but there were larger differences in numbers of suspensions. African-American middle school athletes again had the largest differences from their matched non-athletes in attendance, GPA, and lower suspension rates. Middle school athletes overall, and Latino athletes in particular, also had better results than matched students on attendance, GPA, and suspension rates, but the differences were smaller than those found for African-Americans. The analysis looked at middle school athletes and their matched comparisons in 2008-09, 2009-10, 2010-11, and 2011-12, with a total of 8,326 (of 8,336 possible instances) athletes compared with 26,494 matched non-athletes matched on 5th grade characteristics.

This analysis provides strong evidence that participation in middle school and high school athletics helps improve student engagement and achievement, and lowers suspension rates. Athletes had positive results across these areas compared with students who closely resembled them before beginning high school and middle school, so participation in athletics is the main factor that can explain these different results. Overall, middle school athletes tended to have better attendance, GPAs, and suspension rates compared with matching non-athletes, and these differences were largest for African-American students. The effects of participation in athletics appear to be even larger in high school where athletes tended to have much higher attendance, substantially higher grades, and somewhat lower suspension rates than non-athletes. The analysis indicates that participation in high school athletics helps increase student engagement and achievement. For traditionally underserved (African-American and Latino) students, the differences between athletes and matched non-athletes were very large. Athletics seems to play a significant role in improving student engagement and achievement for high school athletes from these groups. Those who resembled the athletes in 8th grade across a variety of measures, however, averaged low attendance and GPAs. It may be worth considering exploring ways to engage these students in high school in a way similar to how athletes were engaged, using activities that befit their interests and abilities.


All 3.2 3 0.2 YES 0.001African Americans 2.7 2.3 0.4 YES 0.001Latinos 2.9 2.6 0.3 YES 0.001

All 0.07 0.11 -0.04 YES 0.001African Americans 0.30 0.48 -0.18 YES 0.01Latinos 0.06 0.10 -0.04 YES 0.001

Attendance

GPA


Table 20: Attendance, GPA, and Suspensions for Middle School Athletes and Non-Athletes Matched on 5th grade (2008-09 to 2010-12)

Demographic AthletesMatched

Non-athletes

DifferenceDo Athletes out-

perform their counterparts?

Significance


Conclusion PEEF is a district-wide investment in a diverse, yet strategic portfolio of SFUSD programs that has contributed to improvements in student achievement and success. The evaluation of PEEF and this report was guided by the PEEF district-wide logic model, which depicts a chain of reasoning of “if... then” statements that explain how each of the components in the model (inputs, outputs, immediate outcomes and long-term outcomes) are connected and sequential. Findings related to each of the logic model components were organized by PEEF’s goals of improving professional and instructional capacity and quality, school climate, school engagement, student physical and mental health, and ultimately, student academic achievement. Below is a summary of PEEF outputs, outcomes, and impact.

PEEF Outputs PEEF-funded programs increased access and equity by increasing levels of offerings and participation related to the following areas:

6. Professional and instructional capacity and quality (e.g., qualified staff, number of physical education and arts classes offered to students, student access to libraries, and professional development for staff)

7. School climate (e.g., security at athletic events, Restorative Practices implementation at schools, additional custodians at elementary schools)

8. School engagement (e.g., arts instruction for elementary students, students receiving mentoring, interpretation for ELL parents)

9. Student health (e.g., participation in athletic teams, students served at Wellness Centers, student access to Student Support Professionals)

10. Academic achievement (e.g., student participation in tutoring programs, City College courses offered to SFUSD students, teachers aides)

As presented in the outputs section of the report, 73 out of 83 measures (88%) show a positive trend, indicating that PEEF has contributed to increasing access and equity in SFUSD.

PEEF Immediate Outcomes With these increased outputs, immediate outcomes are expected to improve. As evidenced by the data included in this report, positive trends were manifested in the following areas:

5. Professional and instructional capacity and quality (e.g., number of highly qualified teachers hired in SFUSD, diversity of new hires, low number of classroom vacancies on the first day of school)

6. School climate (decrease in suspension rates for middle school students, increase in students’ perceptions of school safety and positive school environment)

7. School engagement (improved attendance rates, students’ perceptions of caring adults at the school)

8. Student physical and mental health (e.g., aerobic capacity and increased social support of students)

This report has presented data for 93 immediate outcome measures, including:

• 31 measures of change over time, between the implementation year of a PEEF program and the 2011-2012 school year


• 44 ”data snapshot" measures based on responses from 2011-12 district and programs surveys

• 18 measures that compare school suspensions and enrollment of PEEF participants to non-PEEF participants in the middle school and high school Athletics

Of the 46 measures showing change over time, 43 (93%) demonstrate a positive trend. The 44 data snapshot measures present an overall picture of positive staff, parent, and student perceptions related to professional and instructional capacity and climate, school climate and engagement, and student’s physical and mental health. In addition 94% of the measures that compare participants and non-participants in middle and high school Athletics indicated that participation in Athletics plays a significant role in improving student engagement in schooling, for students overall, and particularly for those traditionally underserved (African-American and Latino students). These combined findings suggest that PEEF has contributed to improving SFUSD professional and instructional capacity and quality, improving school climate, increasing school engagement, and improving student physical and mental health.

PEEF Long-Term Outcome: Academic Achievement With the progress made toward these immediate outcomes it is reasonable to expect increases in student achievement. When student achievement was examined, it was found that GPA and CST ELA and CST math scores have improved district-wide. Analysis that compared participants and non-participants provides strong evidence that participation in middle- and high-school athletics helps improved student achievement. Overall, athletes had higher GPAs than matched non-athletes who closely resembled them before beginning high school or middle school.

This report presented data for 31 academic achievement measures, 23 of which show change over time, nine of which compare participants with non-participants, and eight of which show 2011–12 data snapshots. All 23 measures showing change over time display a positive trend. Nine measures compare the achievement (GPAs) of participants and nonparticipants in middle and high school athletics, after using a statistical technique (propensity score modeling) to help ensure that athletes and their comparison groups shared similar characteristics before entering middle and high school. Of these nine measures, 100% indicated higher GPAs for participants than nonparticipants, for students overall, and for those who are African-American and Latino. In addition, student survey respondents frequently reported that participation in Wellness Center programs helped them improve their academic performance. SFUSD school staff reported confidence in their ability to prepare students for college and/or careers. Parents had similar sentiments, with 81% of respondents to the SFUSD Family Satisfaction Survey indicating that they strongly agreed/agreed that teachers at the children’s schools had the skills and knowledge to prepare them for college and/or career.

Across all of the PEEF immediate and long-term outcomes, a considerable majority of the outcomes (88%) show a positive trend. District-wide data indicates that SFUSD is making gains; over the course of the PEEF years many of the key district-wide indicators of student success have improved. Since PEEF funding was initiated in 2004–05, attendance rates have increased, GPAs have increased, and CST ELA and math scores have improved for all school levels. Continued investment in PEEF programing is likely to increase outcomes in the impact areas outlined in this report and result in greater gains in student achievement for all students at SFUSD.


This pattern of positive increases in both outputs and immediate outcomes indicate that the PEEF has contributed to the improvement of long-term outcomes for students in SFUSD. PEEF funding has been vital during a period of state budget cuts and decreases in other funding. Data suggests that PEEF has been essential to the continuation of programing that not only contributes to increased academic achievement at the SFUSD but that also provides students with the tools and skills necessary for continued success after graduation.

Future Steps This report builds on the PEEF logic model approach to evaluating the PEEF initiative. To test the pathways in the PEEF logic model, the PEEF Office will continue to collect and report relevant and meaningful measures of outputs and outcomes. The PEEF Office began this process by following recommendations from SRI on how to enhance PEEF data. The PEEF Office will continue to focus on collecting data to show the changes from the year prior to PEEF implementation to the current year, and, when possible, examine the differences between students served by PEEF and those not served by PEEF.

We also expect that in coming years our measures will better allow PEEF to disaggregate outputs and outcomes by student and school characteristics, such as student’s race/ethnicity, gender, ELL status, school level (e.g., elementary, middle, high school), school area/zone. In addition, the PEEF Office will work with PEEF program managers to strengthen program capacity to collect high-quality data, as well as develop a PEEF data management system. Such collaborative efforts projected for the 2012-13 year include a case study on the implementation of the Restorative Practices program and an exploration of the impact of services provided by Student Support Professionals.

PEEF 2011-12 Interim Evaluation Report, December 20, 2012

44

Appendix A: City Charter Section 16.123 1-10 SEC. 16.123-1. PREAMBLE.

(a) The people of the City and County of San Francisco find and declare that:

1. Quality public education is highly correlated with higher earnings potential, reduced crime, lower rates of teen pregnancy and substance abuse, and greater self-esteem;

2. Urban public schools have the greatest need for comprehensive educational programs-including preschool programs, arts and music programs, sports activities, and after school programs-but often have the fewest resources to provide them;

3. While California once led the nation in public school spending and performance, investments have greatly declined. Despite its high cost of living, San Francisco per pupil spending ranks 34th among 43 comparable central City U.S. public school districts of similar size. As of 2001, adjusted for cost-of-living, teacher salaries for the San Francisco Unified School District (SFUSD) ranked 99th of 100 metropolitan areas;

4. SFUSD enrollment has dropped in recent years as families have left San Francisco in search of affordable neighborhoods with high-quality public schools;

5. The choices businesses make about where to locate include the quality of public services the City provides, including public safety, transportation and education;

6. Since 2000, the SFUSD has made strong improvements in achievement measures and financial management; and

7. As the economy begins to recover, now is the time to invest in our children's future, before further declines begin to erode the progress the SFUSD has made.

(b) This measure may be referred to as "The Arts, Music, Sports, and Pre-School for Every Child Amendment of 2003."

(Added March 2004)

SEC. 16.123-2. PUBLIC EDUCATION ENRICHMENT FUND.

(a) Creating the Fund. There shall be a Public Education Enrichment Fund. The City shall each year appropriate monies to the Public Education Enrichment Fund according to subsections (b), (c), and (d), below. In determining whether the City has met its annual obligation to the Fund, the Mayor and the Board of Supervisors may consider both direct financial support and the cash value of any in-kind support services, as described in Section 16.123-5, provided by the City to the San Francisco Unified School District and the Children and Families First Commission (hereinafter the "First Five Commission") or any successor agency, provided that at least two-thirds of the City's contribution to the Fund each year shall be


45

comprised of direct financial support necessary to meet the requirements of Sections 16.123-3 and 16.123-4 of this measure.

(b) Baseline Appropriations. The Fund shall be used exclusively to increase the aggregate City appropriations to and expenditures for the San Francisco Unified School District. To this end, the City shall not reduce the amount of such City appropriations (not including appropriations from the Fund and exclusive of expenditures mandated by state or federal law) in any of the eleven years during which funds are required to be set aside under this Section below the amount so appropriated for the fiscal year 2002-2003 ("the base year"). These baseline appropriations shall be separate from the City's annual contributions to the Public Education Enrichment Fund under subsection (c), and shall be appropriated by the City to the School District each year during the term of this measure for the same purposes and in the same relative proportions among those purposes as in the base year, as certified by the Controller.

The amount of the City's baseline appropriations to the School District shall be adjusted for each year after the base year by the Controller based on calculations consistent from year to year by the percentage increase or decrease in City and County discretionary General Fund revenues. In determining City and County discretionary General Fund revenues, the Controller shall only include revenues received by the City and County that are unrestricted and may be used at the option of the Mayor and the Board of Supervisors for any lawful City purpose. Errors in the Controller's estimate of discretionary revenues for a fiscal year shall be corrected by an adjustment in the next year's estimate. Using audited financial results for the prior fiscal year, the Controller shall calculate and publish the actual amount of City appropriations that would have been required under this baseline for the School District.

(c) Annual Contributions to the Fund--FY 2005-2006 through FY 2009-2010. In addition to the annual baseline appropriation provided above, the City shall, for years two through six of this measure, contribute the following amounts to the Public Education Enrichment Fund:

Fiscal Year 2005-06 $ 10 million

Fiscal Year 2006–07 $ 20 million




(d) Annual Contributions to the Fund-FY 2010–11 through FY 2014–15. For Fiscal Years 2010–11 through FY 2014–15, the City's annual contribution to the Public Education Enrichment Fund shall equal its total contribution for the prior year, beginning with Fiscal Year 2009–2010, adjusted for the estimated increase or decrease in discretionary General Fund revenues for the year.

(e) Audit Requirements. All disbursements from the Fund and from the baseline appropriations shall be subject to periodic audit by the Controller. The San Francisco Unified School District and the First Five Commission shall agree to such audits as a condition of receiving disbursements from the Fund.

(Added March 2004)


46

SEC. 16.123-3. ARTS, MUSIC, SPORTS, AND LIBRARY PROGRAMS.

Each year during the term of this measure, the City shall appropriate one-third of the money in the Public Education Enrichment Fund to the San Francisco Unified School District for arts, music, sports, and library programs in the schools.

(Added March 2004)

SEC. 16.123-4. UNIVERSAL ACCESS TO PRESCHOOL.

(a) Universal Access to Preschool. It shall be the policy of the City and County of San Francisco to provide all four-year-old children who are City residents the opportunity to attend preschool, and it shall be the goal of the people in adopting this measure to do so no later than September 1, 2009.

(b) Planning Process. No later than September 1, 2004, the First Five Commission, in consultation with the San Francisco Childcare Planning and Advisory Council, the San Francisco Unified School District, the San Francisco Department of Children, Youth and Their Families, and community stakeholders, shall submit to the Board of Supervisors a proposal for a universal preschool program for San Francisco. The Board of Supervisors shall approve the plan by resolution; if the Board does not approve the plan, it may refer the plan back to the First Five Commission for revision.

In preparing the plan, the First Five Commission shall develop universal preschool funding guidelines consistent with the Childcare Planning and Advisory Council's San Francisco Childcare Needs Assessment, including guidelines designed to meet neighborhood-specific needs, such as subsidies, new facility development, and provider support for both family childcare homes and childcare centers. Such funding guidelines also shall address the unmet need for universal preschool and childcare slots in specific City neighborhoods.

(c) Annual Disbursements. Each year during the term of this measure, the City shall appropriate one-third of the money in the Public Education Enrichment Fund to the First Five Commission for universal preschool programs administered by the Commission.

(Added March 2004)

SEC. 16.123-5. OTHER CITY SUPPORT FOR THE SAN FRANCISCO UNIFIED SCHOOL DISTRICT.

(a) In-Kind Support. Not later than one year after the effective date of this measure, the City and the School District shall identify areas of potential in-kind support that the City could provide to the School District free of charge or at substantially reduced rates. In-kind support, for these purposes, may include, but is not limited to:

Learning support services, including health, counseling, social work, and nutrition services;

Financial support services;


47

Telecommunication and information services;

Construction management services;

Utility services;

Transportation services;

Legal services; and

Public safety services.

(b) Planning Process. No later than six months after the effective date of this measure, the School District shall submit to the Board of Supervisors proposals for in-kind services that could be provided by the City to the District to further the educational goals and operations of the District. The Board shall distribute those proposals to all City departments having expertise in providing or capability to provide such in-kind services, and no later than nine months after the effective date of this measure, the departments will respond to the Board with proposals to provide such in-kind services to the District. The School District may use any direct financial support provided under this Section to hire consultants to help identify possible in-kind services. The Board of Supervisors may, by ordinance, provide for continuation of this planning process during the subsequent term of the measure.

(c) Annual Disbursements. Each year during the term of this measure, the City shall provide direct financial assistance from the Public Education Enrichment Fund to the San Francisco Unified School District, in an amount equal to one-third of the money in the Fund, or in-kind support services of equal value.

(d) Permissible Uses. The San Francisco Unified School District may expend funds provided as direct financial support under this Section for any educational or support purpose provided under law, including, but not limited to, gifted and talented programs, magnet programs, literacy programs, dual-language immersion programs, special education, employee compensation, career and college centers at high schools, teacher mentoring or master teacher programs, or other instructional purposes. The City recognizes that in providing such programs and services, a well-run school district requires both certificated and classified staff, and urges the San Francisco Unified School District to hire both certificated and classified staff to carry out the purposes of this measure.

(e) Within one year of the effective date of this measure, the School District, with the assistance of the City's Department of Public Health, Department of Human Services, and Department of Children, Youth, and Their Families, shall conduct an assessment of health, counseling, social work, and nutritional needs of pupils in the District, including problems related to asthma and other chronic diseases. The City may appropriate a specific portion of the disbursement under this Section through its annual appropriation process for these purposes, pursuant to recommendations from the School District.

(Added March 2004)

SEC. 16.123-6. EXPENDITURE PLANS.


48

(a) No later than February 1 of each year during the term of this measure, the San Francisco Unified School District and the First Five Commission shall each submit an expenditure plan for funding to be received from the Public Education Enrichment Fund for the upcoming fiscal year. The proposed expenditure plans must include prior year total budgeted and expended appropriations and Fund budgeted and expended appropriations by category, as well as average daily attendance information for the prior year and anticipated average daily attendance information for the plan year, to facilitate multi-year comparison.

(b) The Controller shall review the plans and transmit them, with his or her comments, to the Mayor and the Board of Supervisors for their review and comment.

(c) The plans shall include a budget for the expenditures, performance goals, target populations, hiring and recruitment plans for personnel, plans for matching or other additional funding, operating reserves, and any other matters that the District and Commission deem appropriate or the Mayor or the Board requests.

(d) The Mayor and the Board of Supervisors may request further explanation of items included in the plans, and the District and the Commission shall respond in a timely manner to such inquiries. The Board may place appropriations provided for under this measure on reserve until it has received adequate responses to its inquiries.

(Added March 2004)

SEC. 16.123-7. STRUCTURAL SAVINGS TO THE CITY'S BUDGET.

(a) Controller's and Budget Analyst's Recommendations. Not later than October 1 of each fiscal year from Fiscal Year 2005–06 through 2009–10, the Controller and the Board of Supervisors' Budget Analyst shall prepare and submit recommended cuts or other structural changes to reduce, on an ongoing basis, spending on City departmental operations, or identify new revenues, in an amount sufficient to meet each year's required funding for the Public Education Enrichment Fund.

(b) Board of Supervisors' Proposals. Not later than December 15 of each fiscal year from Fiscal Year 2005–06 through Fiscal Year 2009–10, the Board of Supervisors shall hold hearings on the recommendations made by the Controller and the Budget Analyst and shall forward its proposals to the Mayor.

(c) Budget Requirements. In his/her annual budget submission to the Board of Supervisors for each fiscal year from Fiscal Year 2005–06 through Fiscal Year 2009–10, the Mayor shall incorporate the Board of Supervisors' proposals, or identify alternative revenue or expenditure savings sufficient to appropriate funds to the Public Education Enrichment Fund according to the schedule set forth in Section 16.123-2 of this measure.

(Added March 2004)

SEC. 16.123-8. ADJUSTMENTS.

(a) In any year of this measure, if the joint budget report as prepared by the Controller, the Mayor's Budget Director and the Board of Supervisors' Budget Analyst projects a budgetary shortfall of $100


49

million dollars or more, the Mayor and the Board of Supervisors may reduce the City's contribution to the Public Education Enrichment Fund under Section 16.123-2, and its disbursements under Sections 16.123-3, 16.123-4, or 16.123-5, by up to 25 percent; provided, however, that the City must pay back the amount deferred within the period from June 30, 2015, the last day of the term of this measure, and June 30, 2018, a date three years later, unless the voters extend this measure beyond July 1, 2015 or authorize a substantially similar measure at that time.

(b) Audit Recommendations. The Mayor and the Board of Supervisors may suspend the City's disbursements from the baseline appropriations or the Public Education Enrichment Fund under Sections 16.123-3, 16.123-4, or 16.123-5 in whole or in part for any year where the Controller certifies that the San Francisco Unified School District or the First Five Commission has failed to adopt audit recommendations made by the Controller.

As part of the audit function, the Controller shall review performance and cost benchmarks developed by the School District and the First Five Commission in consultation with the Controller for programs funded under this measure. The Commission's performance and cost benchmarks shall be based on the same performance and cost benchmarks as are required for other City departments, and on comparisons with other cities, counties, and public agencies performing similar functions. The School District's performance and cost benchmarks shall be based on similar standards.

In particular, the Controller shall assess:

(1) Measures of workload addressing the level of service being provided or providing an assessment of need for a service;

(2) Measures of efficiency including cost per unit of service provided, cost per unit of output, or the units of service provided per full time equivalent position; and

(3) Measures of effectiveness including the quality of service provided, citizen perceptions of quality, and the extent a service meets the needs for which it was created.

The Controller's audits may address the extent to which the School District and the First Five Commission have met their respective performance and cost benchmarks.

(c) Reserve Policies. The Mayor and the Board of Supervisors may suspend the City's disbursements from the baseline appropriations or the Public Education Enrichment Fund under Sections 16.123-2, 16.123-4, or 16.123-5 in whole or in part for any year where the Controller certifies that the San Francisco Unified School District or the First Five Commission has failed to adopt reserve policies recommended by the Controller.

(d) Transfer and Use of Suspended Distributions. If the Mayor and the Board of Supervisors suspend City distributions from the baseline appropriations or the Public Education Enrichment Fund under subsections (b) or (c), the City shall transfer the amount that would otherwise be distributed from the baseline appropriations or the Public Education Enrichment Fund for that year to the Children's Fund established in Charter Section 16.108, for the provision of substantially equivalent services and programs.


50

(e) New Local Revenues. The Board of Supervisors may, by ordinance, proportionally reduce the contribution to the Public Education Enrichment Fund and the disbursements to the San Francisco Unified School District and the First Five Commission required by this measure if the voters of San Francisco adopt new, dedicated revenue sources for the School District or the Commission, and the offsetting reduction in disbursements is specifically authorized by the local revenue measure.

(f) New State Revenues. The Board of Supervisors may, by ordinance, proportionally reduce the contribution to the Public Education Enrichment Fund and the disbursements to the San Francisco Unified School District required by this measure if the percentage increase in per-pupil Revenue Limit funding provided by the State of California to the San Francisco Unified School District in any fiscal year exceeds the percentage increase in the City's cost of living during the previous fiscal year.

The Board of Supervisors may, by ordinance, proportionally reduce the contribution to the Public Education Enrichment Fund and the disbursements to the First Five Commission if the State of California provides funding to the City for universal preschool, provided that such disbursements are not required to match state and/or other funding.

(g) Eighteen months prior to the expiration of this measure, the Controller shall conduct a complete analysis of the outcomes of the programs funded through the Public Education Enrichment Fund. The Controller's study shall also address changes in the levels of state and federal funding for local schools, per-pupil spending in the San Francisco Unified School District compared to urban school districts of similar size. The Controller shall present the results of this analysis to the Mayor and the Board of Supervisors no later than nine months prior to the expiration of the measure.

(Added March 2004)

SEC. 16.123-9. STATE REDISTRIBUTION OF LOCAL EDUCATION REVENUES.

(a) The people of the City and County of San Francisco find and declare that major urban school districts, such as San Francisco, serve an ethnically and economically diverse student population that requires more resources than currently provided under state guidelines. In adopting this measure, the people of San Francisco choose to provide additional City resources to complement, and not supplant, state funding for the San Francisco Unified School District.

(b) Consistent with subsection (a), the people of the City and County of San Francisco specifically find that their contributions to and disbursements from the baseline appropriations and the Public Education Enrichment Fund are discretionary expenditures by the City for the direct benefit of the children of San Francisco, their families, and the community at large. In the event that the State attempts, directly or indirectly, to redistribute these expenditures to other jurisdictions or to offset or reduce State funding to the School District because of these expenditures, the City shall transfer said monies that would otherwise be distributed to the School District each year to the City's Children's Fund established in Charter Section 16.108, for the provision of substantially equivalent services and programs.

(Added March 2004)


51

SEC. 16.123-10. SUNSET.

The provisions of this measure shall expire in eleven years, at the end of Fiscal Year 2014-15, unless extended by the voters.

(Added March 2004)


52

Appendix B: 2011-12 PEEF Initiative Programs Descriptions and Activities The following section of the report includes a summary of budgeted program descriptions and approved activities for each PEEF funded program as listed in the SFUSD 2011-12 PEEF Expenditure Plan.

Physical Education On May 18th, 2009 the San Francisco Unified School Board unanimously approved the Physical Education Master Plan. The goal of the physical education program and the Physical Education Master Plan is to provide quality K-12 physical education for every child through participation in a comprehensive, sequential physical education system that promotes physical, mental, emotional, and social well-being.

The Physical Education’s Department’s expenditure proposal includes support for:

• 15.0 FTE Elementary School Physical Education Teachers to provide direct instruction to approximately 10,100 elementary students at approximately 30 sites, supporting approximately 500 elementary classroom teachers. Elementary Physical Education Specialists work closely with the classroom teacher to implement the district adopted physical education curriculum to all students.

• 2.0 FTE Elementary Physical Education Implementation Specialists to provide support to Elementary Physical Education Specialists, classroom teachers, and all elementary school sites.

• 2.0 FTE Secondary Physical Education Content Specialists to support all secondary school sites and approximately 140 physical education teachers.

• 1.0 FTE Physical Education Program Administrator and 1.0 FTE Physical Education Clerk to implement programming, support teachers at K-12 school sites, and to ensure alignment with standards and the District’s Physical Education Master Plan

• 30 stipends for elementary school teachers to serve as Physical Education Site Coordinators to support teachers and staff in implementation of Physical Education requirements at 30 sites.

• Instructional supplies and equipment for elementary and secondary school sites. • 39 SFUSD Secondary site allocations of $16.00 per student to purchase instructional materials to

support implementation of district-adopted curriculum (approximately 30,000 students). This includes support for County/Community/Court schools and nine charter sites.

• Professional development on content standards, physical education implementation, and physical fitness testing (approximately 150 workshops for 650 K-5 teachers).

Athletics The goal of the Athletics program is to provide accessible, fully coached, safe, and well supported athletic opportunities to students at all SFUSD middle and high schools. PEEF funding supports the Athletic Department in providing an interscholastic athletic program that provides student athletes opportunities to increase physical activity, develop skills and abilities in team building, leadership development, overcoming adversity and pressure, setting and achieving goals, and winning and losing with honor.

The Athletics Department’s expenditure proposal includes recommendations for:

• Funding for approximately 150 coaches at $28.89/hr. plus benefits. PEEF funding will enable SFUSD to maintain the current number of coaches and athletic directors at 21 middle schools (includes K-8 sites) and 13 high schools and will enable sites to continue supporting student athletes in both sports and academics.

• Funding to maintain sufficient training room medical supplies at 13 high schools and funds to purchase additional automatic external defibrillators for each high school’s athletic facility.


53

• Consultants for medical personnel, security personnel, and contest officials (to support approximately 360 teams across 34 league sports): including Athletic Trainers for 13 high schools, four Emergency Medical Technicians for freshman/sophomore football games, approximately 11 doctors for varsity football games, private security guards, and San Francisco Police Officers (approximately 350 contests require 1-5 guards and/or officers at each of the events), and officials for football, volleyball, wrestling, baseball, and softball games and matches (approximately 1,500 contests require 1-4 officials per event depending on the sport).

• Bus transportation for athletes to and from high school and middle school athletic competitions within San Francisco and throughout the Bay Area providing approximately 1,200 athletic team trips.

• School athletic facility repair, maintenance, and construction: including construction and repair of athletic field and track areas at SFUSD High School sites. In 2011-2012, funding is earmarked to renovate the deteriorating football field located at Lowell High School and the natural grass softball field located at Burton High School.

• Rent of non-SFUSD athletic facilities to provide fields and gymnasiums for use by schools that do not have athletic facilities, and for playoff and championship events.

• Non-capitalized equipment replacements and repairs to address conditions that limit student participation in athletics: athletic training equipment, scoreboards, batting cages, golf driving cages, volleyball standards, and basketball backboards.

• Professional development stipends (approximately 100) to provide leadership development opportunities for new and returning coaches to create better mentors and leaders for our students as mandated by the California Interscholastic Federation. Additionally, Cardio Pulmonary Resuscitation and First Aid certification trainings will be provided for all paid coaches and some veteran and volunteer coaches.

• Site allocations for 34 middle and high schools to purchase athletic equipment, uniforms, and stipends for athletic event personnel uniforms. Each allocation is based on each sites plan to establish teams and the approximate costs required per team.

Libraries The goal of Library Services is to provide high quality and effective library services that enhance the education and instruction for all students. In 2011-12, PEEF funded sites will receive teacher librarians, technology upgrades, and updated library collections that are current, relevant, engaging, and accessible to all students. The Library Services Department’s expenditure proposal includes recommendations for:

• 43.5 FTE Teacher Librarians assigned to 82 elementary and middle school sites, each receiving 2, 2.5 or 3 days of support, determined by school size.

• 1.0 FTE Program Administrator, 0.5 Supervisor, 1.0 FTE Library Technician and 1.0 FTE, Librarian on Special Assignment for program coordination and to provide centralized support.

• Allocations to 12 large High Schools of $11,166 per site for additional Teacher Librarian FTE, materials, technology, or other library-related resources.

• Allocations to 71 Elementary and Middle Schools (including K-8) of $5.00 per student for library books, materials and supplies.

• Allocations to five small high schools of $5.00 per student for library books, materials and supplies.


54

• Allocations to County/Community/Court sites and nine Charter Schools of $5.00 per student for library books, materials and supplies. Includes allocation for 0.2 FTE at 2 middle school charter sites.

• Library research and reference databases for K-12 schools: Netrekker, Facts on File, teaching books and support for catalog and circulation software.

• Continue replacement cycle of library computers and printers for book circulation, support circulation software, and presentation equipment; includes 20 computers and 20 printers for 20 schools and three LCD projectors for six schools.

• Professional development for 65 librarians including stipends for conferences, and workshops on design and development of new research units.

Arts and Music The goal of the Visual and Performing Arts program is to provide access and equity in arts education (both arts as a discrete discipline taught by certificated teachers and as arts-infused academics using San Francisco as the campus) for every SFUSD student, at every school, during the curricular day. Proposed spending for arts and music follows the Arts Education Master Plan and was approved by the Arts Education Master Plan Advisory Committee. All areas of funding support the Master Plan’s six areas of focus: administrative leadership, professional development, resources and staffing, curriculum and instruction, partnerships and collaborations, and assessment.

The Visual & Performing Arts Department’s expenditure proposal includes recommendations for:

• 14.4 FTE credentialed Generalist Visual and Performing Arts Teachers serving 72 elementary schools at 0.2 FTE for each elementary site (includes elementary grades at K-8 sites).

• 18.6 FTE Middle School Arts Teachers for 21 sites (includes middle school grades at K-8 sites). Each site receives 0.2 FTE - 1.6 FTE.

• 6.4 FTE High School Arts Teachers for 18 sites. Each site receives between 0.2 FTE - 0.6 FTE. • 2.0 FTE Arts Teacher for County/Community/Court middle and high schools. • Funds to support 2.4 FTE at charter schools based on student enrollment. Approximately 0.2 - 0.4

FTE for nine charter schools. • Site allocations (per pupil $5.00 for elementary and $10.00 for middle and high schools) for 72

elementary schools, 21 middle schools (includes K-8 sites), 18 high schools, County/Community/Court schools, and nine charter schools for arts supplies and materials, field trips, artists-in-residence and/or partial credentialed arts teacher.

• Supplies, materials, and equipment for 72 elementary sites to support Generalist Visual and Performing Arts Teachers of dance, drama, visual art, and music.

• 1.0 FTE Arts Education Master Plan Implementation Manager. • 1.6 FTE Visual and Performing Arts Teachers on Special Assignment to provide support for the

Young at Art Festival, assessment, and curriculum. • 1.0 FTE for Visual & Performing Arts district-wide support and administration: includes 0.5 FTE

Visual and Performing Arts Supervisor and 0.5 FTE Artistic Director. • 1.0 FTE clerical staff to support the Visual and Performing Arts Department. • 1.0 FTE district-wide piano technician. • District-wide musical instrument repair and supplies. • 98 elementary, middle and high school Arts Coordinators stipends (stipends of $1000 with

benefits) for 98 sites. • Professional development for Arts Coordinators, principals, arts teachers and classroom teachers.


55

• Visual and Performing Arts implementation support: consultants and content specialist extended hours to support sites, teachers, and program implementation.

• Visual and Performing Arts infrastructure costs including parent outreach and assessment. • Young at Art (K-12) festival production to support district-wide student and teacher achievement in

the arts.

Student Support Professionals Student Support Professionals are either Learning Support Professionals or School District Nurses who provide site-based services to improve the social and emotional learning of students as a crucial part of improving academic performance. Working with teachers and other designated school site staff, the essential function of the Learning Support Professionals is to provide support for students’ mental health and behavioral needs. School District Nurses focus on prevention, early detection, and management of health and behavioral concerns.

Support services provided by both Learning Support Professionals and School District Nurses include:

• Group counseling and classroom behavior support. • Crisis assistance and prevention planning. • Facilitation of effective student support structures (e.g. Student Assistance Programs, Student

Success Teams). • Case management and coordination of services. • Health education and personal social skill development. • Referral to a broad range of community resources. • Family outreach to support home involvement in schooling.

Recommendations include support for:

• 37.0 FTE Learning Support Professionals and School District Nurses serving 74 elementary, middle and K-8 sites.

• 1.0 FTE Mentor Student Support Professional to provide coaching, site support, and professional development to 37.0 FTE site based Learning Support Professionals and Nurses.

• 1.0 FTE Program Administrator to oversee all aspects of PEEF Student Support Professional program.

• 1.0 FTE Senior Clerk Typist to provide clerical support to PEEF Student Support Professional program.

• Professional Development for Student Support Professionals and extended hours for 2012-13 planning.

• Instructional supplies, materials, and purchase of evidence-based health and mental health curricula.

• Mileage reimbursement to Student Support Professionals for car travel incurred while performing job duties (e.g. home visits).

• Travel and conferences for California Association of School Social Work annual conference and other area conferences to learn best practices in the field of Student Support.

• Rental/lease of equipment including Xerox machine rental and service. • Consultant fees for comprehensive evaluation of services provided by Student Support

Professionals including analysis of student and teacher pre- and post-surveys. • Supplies, materials and operating costs.


56

Peer Resources The goal of Peer Resources is to empower students to be leaders in their own lives, in their schools, and in their communities. Students receive training to become Peer leaders and gain the skills and knowledge necessary to look critically at their schools and communities to take action to improve them. Peer Resources is committed to developing young leaders in San Francisco with the use of fun, creative, and interactive strategies that build safety, confidence, and skills. Core student-run delivery services includes conflict mediation, peer mentoring, peer tutoring, and peer education programs that focus on improving schools and schools communities, particularly for vulnerable student populations

Recommendations include support for 5.35 FTE Peer Resources Teachers assigned to 10 middle and high schools. Sites provide matching funds to increase FTEs.

Wellness Initiative The Wellness Initiative, in collaboration with school site administrators, manages and funds Wellness Programs in 15 SFUSD high schools. The mission of the Wellness Initiative is to improve the health, well-being and educational outcomes of high school students through the promotion of healthy lifestyles and the provision of coordinated prevention and early intervention health and wellness services at school sites. The Wellness Programs provide free, confidential services to students, including behavioral health counseling, nursing services, support and empowerment groups, reproductive health services, youth leadership programs, and linkages to health resources in the community.

PEEF funding for the San Francisco Wellness Initiative began in the 2007-08 school year, which allowed the Initiative to establish four new programs at June Jordan School for Equity, Newcomer, Wallenberg and Washington High Schools. Due to the closure of Newcomer High School, resources associated with Newcomer were reallocated in 2010-11 to support Newcomer students at six “Newcomer Pathway” high school sites: Lincoln, Washington, Galileo, Mission, Thurgood Marshall, and San Francisco International.

2011-12 PEEF funds will continue to support newcomer students and the Wellness Programs at June Jordan School for Equity, Wallenberg and Washington High Schools. The Wellness Initiative intends to transition the Newcomer Pathways interim program to the establishment of the standard Wellness Program model at San Francisco International High School (300 newcomer students are anticipated in the 2011-12 year).

Recommendations for funding supports:

• 9.6 FTE includes: 4.0 FTE Wellness Coordinators, 3.6 Community Health Outreach Workers, and 1.6 School District Nurses.

• Stipends for one Youth Outreach Coordinator and 6-12 Youth Outreach Workers (leadership and youth development program).

Restorative Practices Restorative Practices, when broadly and consistently implemented, will promote and strengthen positive school culture and enhance pro-social relationships within the school community. An improved sense of


57

community will significantly decrease the need for suspensions, expulsions and time that students are excluded from instruction due to behavior infractions. This shift in practice will result in a culture which is inclusive, builds fair process into decision-making practices, and facilitates students learning to address the impact of their actions through a restorative approach. Students will learn to accept accountability and to repair the harm their actions caused, recognize their role in maintaining a safe school environment, build upon their personal relationships in the school community and recognize their role as a positive contributing member of the school community. Ultimately, they will learn to make positive, productive, and effective choices in response to situations they may encounter in the future.


• 4.5 FTE includes: 1.0 Teacher on Special Assignment - Restorative Practices Coordinator, 3.0 Teachers on Special Assignment - Restorative Practices Site Coaches, and 0.5 Clerk.

• Professional development includes: substitute release days, extended hours and stipends and travel and conferences for trainers, site leaders and Restorative Practices Leadership Teams.

• Reference books, library books and professional libraries for sites. • Supplies. • Consultants– International Institute of Restorative Practices, Educators for Social Responsibility

and other local trainers to provide professional development as well as development and maintenance of system-wide data tracking and reporting tools.

Career Technical Education The Career Technical Education program provides a PEEF funded Instructional Specialist, responsible for expanding the program consistent with this program’s Strategic Plan. The goal of the program is to improve instructional programs within the career academies and pathways model, improve and expand career academies and pathways programs to underserved SFUSD students, and ensure that Career Technical Education students will demonstrate improved enrollment, attendance and completion of post-secondary programs.

The Career Technical Education program is allocated $79,879. Recommendations include support for a 1.0 FTE Career Technical Education Instructional Specialist who develops and improves technology and enhancement of all career academies and pathways programs in the eight high schools funded by the Career Technical Education Department.

Teacher Academy The Teacher Academy program addresses multiple goals including higher student achievement in mathematics, support for the broader school community, and an increase in the diversity of the District’s teaching pool. Teacher Academy high school students are provided paid internships as Algebra/Mathematics tutors and serve in SFUSD middle school math-intervention classrooms.


• Stipends for 43 students working as teaching aides • Extended hours for six Teacher Academy teachers • Classroom supplies


58

Formative Assessment System The goal of the district-wide assessment system is to provide multiple performance measures and tools for all teachers to inform instruction and to provide students with the means to demonstrate understanding and reflect upon their learning.

Recommendations include support for a 1.0 FTE Researcher, 0.2 FTE Program Administrator, reproduction, supplies, and database software.

Translation and Interpretations Services The Translation and Interpretation Unit’s primary goal is to provide translation and interpretation services to Limited English Proficiency parents with equal access to information and services by maintaining and expanding translation interpretation services at school sites and the central office.

Translation and Interpretations Services is allocated $604,000. Recommendations include support for:

• 2.0 FTE Chinese language translators/interpreters. • 2.0 FTE Spanish language translators/interpreters. • 0.5 FTE Assistant Manager. • Contract funds for translation and interpretation for additional languages. • Extended hours for additional interpretation services/pool of interpreters. • Translation and interpretation equipment upgrades and replacements.

Custodial Services Custodial Services supports a year-round crew for the Child Development Program to perform deep cleanings and additional FTE custodians assigned to approximately 11 understaffed elementary schools. Custodial Services is allocated $ 695,773.


• Approximately 8.5 FTE custodians • 1.0 FTE custodian assistant supervisor

Human Capital Support (formerly, Teacher Recruitment) High performing, properly credentialed teachers are directly linked to student achievement. Recruiting and retaining diverse, quality teachers to the San Francisco Unified School District is critical to our students’ success, particularly in lower performing schools. The program aims to increase student achievement by recruiting and selecting high quality teachers, and providing incoming and current teachers with credential advisement.


• Approximately 1.0 FTE Recruitment Analyst • Approximately 0.2 FTE Education Credential Technician


59

Appendix C: PEEF Initiative Data Charts This appendix includes data charts for every measure presented in the body of this report. To provide a baseline for pre-PEEF funding, trend data in this appendix are presented beginning with the year prior to a program initially receiving PEEF funding. For example, six programs first received funding in the 2005–06 school year.2 The first year of data included in the charts for those programs is for the 2004–05 school year. In addition to this trend data, the outcomes section of this appendix also includes data from several surveys. Data from these surveys are reported for the most recent survey administration. The survey dates are included with the charts depicting this type of data.

2 Six of the programs that are currently funded by PEEF started receiving PEEF funding in 2005–06: Physical Education, Athletics, Libraries, Arts and Music, Student Support Professionals, and Peer Resources. In 2007–08, seven additional programs that are currently receiving PEEF funding were included in the PEEF investment portfolio: Wellness Initiative, Career Technical Education, Formative Assessment, Translation and Interpretation, Custodial Services, Human Capital Support, and Teacher Academy. In 2010–11, PEEF funding was extended to the Restorative Practices program.


Outputs 1: Professional and Instructional Capacity and Quality


Outputs 2: School Climate


64

Outputs 3: School Engagement


65


66


67

Outputs 4: Student Physical and Mental Health


68


69


70


71

Outputs 5: Academic Achievement


72


73

Outcomes 1: Professional and Instructional Capacity and Quality


74


75

Outcomes 2: School Climate


76

Outcomes 3: School Engagement


77

Outcomes 4: Student Physical and Mental Health


78

Outcomes 5: Academic Achievement


79


80

Effects of Middle School and High School Athletics on Attendance, GPA and Suspension


81

PEEF Interim Evaluation Report - SFUSD: Home Immediate Outcomes ... Arts and Music ... PEEF 2011-12...

Documents

Transcript of PEEF Interim Evaluation Report - SFUSD: Home Immediate Outcomes ... Arts and Music ... PEEF 2011-12...