Using Data to Organize Team Meetings

Susan Barrettsbarrett@pbismaryland.org

www.pbis.org

Systems Perspective• Organization do not

“behave” …individuals behave

• “Organization is group of individuals who behave together to achieve a common goal”

• “Systems are needed to support collective use of best practices by individuals in an organization” (Horner, 2001)

Schools as SystemsGoal to create communities that for all its members have common

• Vision• Language, & • Experience

Biglan, 1995; Horner, 2002

Adopt systems perspective at a Building Level

Funding Visibility PolicyPoliticalSupport

Training Coaching Behavioral ExpertiseEvaluation

LEADERSHIP TEAM(Coordination)

Local School/District Implementation Demonstrations

Integrated Functions Across All Tiers of Support

Team approach

Progress monitoring

Data-based decisions

Evidence-based practices

Beh

avio

r Sup

port R

eading Support

Universal Screening

Should get easier for your school over time

• Handbook• Describes core features• Expectations and teaching matrix (rules for settings)• Teaching plans and teaching schedule• Acknowledgement system• Continuum of consequences for problem behavior

• Building Leadership Team• Regular meeting schedule and process• Regular schedule for annual planning and training

• Annual Calendar of Activities• On-going support for staff

What a Leadership Team does…

• Communicates common vision for schoolwide supports

• Works collaboratively to establish building capacity to support all students

• Commits resources to establish procedures for support

• Develops methods for evaluating progress towards measureable outcomes

• Action planning based on data

Identify Team Member Roles• Team Leader - starts the meeting, reviews the purpose of the meeting,

facilitates the meeting by keeping the team focused on each step• Recorder - taking notes, transcribing the team’s responses on flip chart

paper, transparency, etc• Timekeeper- monitors the amount of time available keeps the team

aware of time limits by giving “warnings” (i.e., “10 minutes left”)• Data Specialist- is trained in entering and accessing data from the SWIS

data system• Behavior Specialist- competent with behavioral principles and assists in

analyzing data• Administrator- actively encourages team efforts, provides planning

time, feedback, and support initiatives• Communications – acts as the point person for communication

between the team and staff regarding PBIS and behavior issues• PBIS Coach- district-level (external) or school-based (internal)

individual that facilitates the team through the process, becomes the school’s main contact

Were data collected?

YES NO

Are goals being met?

• Problem solve data collection—determine how to get data

• Collect data for 2 weeks and reconvene

• Celebrate and continue

• Have plan for fading

Is plan being implemented as

designed?

YES NO

• Modify intervention• Consider move to next

level

• Problem solve barriers to implementation

• Collect data and reconvene in 2 weeks

YES NO

Basic QuestionsWhat is the current status?

Are we satisfied?

How could improve ?

How would we know if “it” did improve?

Using various websites as a referencepbismaryland.org

• School Examples• Cost Benefit• Other tools, presentations, updated materials

• Updated Materials- How many team members and staff have gone to Formal New Team Training?

• Have you seen new training materials?• Tier 1 now includes Classroom Systems !• Workbook is posted on pbismaryland

Purpose of Systems MeasuresAssess Current Status

• BOQ Implementation Checklist– Evaluates status of Tier I Positive Behavior

Supports

– Submitted online at www.pbssurveys.org

• Self-Assessment Survey– Evaluates status of Schoolwide, Nonclassroom,

Classroom and Individual Student Supports

– Submitted online at www.pbssurveys.org

What is the Self-Assessment Survey?

• Self-assessment survey to assess the extent to which Positive Behavior Support practices and systems are in place within a school– School-wide (15 items)– Non-classroom (Specific Setting) (9 items)– Classroom (11 items)– Individual Student (8 items)

Who Completes the Self-Assessment Survey?

• Initially, the entire staff in a school completes the Survey.

• In subsequent years and as an on-going assessment and planning tool, the Survey can be completed in several ways:

– All staff at a staff meeting.

– Individuals from a representative group.

– Team member-led focus group.

Using the Self-Assessment Information for Decision Making

• Is a system in place?– “in place” > 66%

• Is there a need to focus on a system?– Current status of “in place” is < 66% and– Priority for improvement is “High” for > 50%

• Which system should receive focus first?– Always establish schoolwide as first priority

• Which features of the system need attention?• Combine survey outcomes with information on office

referrals, attendance, suspensions, vandalism, perceptions of staff/faculty

Individual Summary Charts

• Charts are provided for each system (school-wide, nonclassroom, classroom, and individual)

• Current status Charts

– Percentage of respondents who answered "In Place", "Partially In Place", and "Not In Place"

• Improvement Priority Charts

– Percentage of respondents who answered "High", "Medium", and "Low”

Example of PBS Self Assessment Survey Individual Summaries Chart

Analysis of Schoolwide System Chart

Shows a chart with bars for components of the schoolwide system

– Expectations defined (question 1)– Expectations taught (question 2)– Reward system (question 3)– Violations system (question 4-8) – Monitoring (question 10-12)– Management (question 9, 14-16)– District support (question 17-18)

Analysis of Schoolwide System Chart

Example of

PBS Self Assessment Survey

Individual Item Score

Schoolwide Component

White = In Place

Yellow = Partial In Place

Red = Not In Place

Why conduct Self-Assessment Survey in addition to Checklists?

• Checklists are conducted by team, all/most staff complete survey• Look for areas of convergence across tools

– Increases confidence of data• Look for areas of divergence across tools

– Decrease confidence of data?– Possible reasons for disparity…

• Lack of understanding of questions• Staff not fully aware of work of Building Leadership Team• Support component not fully “In Place”

Differences between the BOQ Implementation Checklist and the Self-Assessment Survey

BOQ Team Implementation Checklist

EBS Self-Assessment Survey

Purpose?

Evaluate on-going progress towards schoolwide PBS

Evaluate extent that all systems (schoolwide, nonclassroom, classroom, individual) are in place

When administered?

Quarterly Annually

Who completes?MiBLSi school team, completed as a team

All school staff (or representative sample) completed individually

Time involved?10-15 minutes 30-45 minutes

What Information do we need?Universal Screeners…

• Office Discipline Referrals– Rate, location, time, problem behavior , student, – Other? Ethnicity, gender grade, teacher, – 0-1, 2-5, 6 or more

• Attendance Rates (patterns)• Tardies• Benchmarks, CBM, Grades, Homework completion pattern• Nurse log• Counselor log• SW expectations- Burke et al

Start with Office Referral

• Purpose?

• Gather as much info as you can!!

Reviewing Student MeasuresAnswer the “Big Five” questions

1. How often are problem behavior events occurring?2. Where are they happening?3. What types of problem behaviors?4. When are the problems occurring?5. Who is contributing?

Using SWIS “Big Five” reports6. Major Discipline Referrals per Day per Month7. Major Discipline Referrals by Location8. Major Discipline Referrals by Problem Behavior9. Major Discipline Referrals by Time10.Major Discipline Referrals by Student

General Decision RulesIF…., FOCUS ON….

>40% of students received 1+ ODR >2.5 ODR/student School-wide System

>60% of referrals come from classroom >50% of ODR come from <10% of classrooms Classroom System

>35% of referrals come from non-classroom settings >15% of students referred from nonclassroom settings Non-Classroom Systems

>10-15 students receive >5 ODR Targeted Group Interventions

<10 students with >10 ODR <10 students continue rate of referrals after receiving

targeted group support Small # students destabilizing overall functioning of school

Individualized Action Team Systems

Problem Identification (look at Major Discipline Referral per Day per Month on next slide)

1. Is there a problem with the absolute standard?

4784.78 .344.78 1.63 ✔

PROBLEM- ODRs per day higher than national avg.

Langley Elementary School: 478 Students, Grades K-5

100

Langley Elementary SchoolReferrals per Day per Month

4784.78 .344.78 1.63 ✔

Problem Identification (look at Major Discipline Referral per Day per Month on next slide)

1. Is there a problem with the absolute standard?2. Are there trends or patterns?

PROBLEM- ODRs per day higher than national avg.TREND- 4 consecutive mos. of increasing trend

Langley Elementary School: 478 Students, Grades K-5

Langley Elementary

Langley Elementary

Langley Elementary

Langley Elementary

PROBLEM- ODRs per day higher than national avg.TREND- 4 consecutive mos. of increasing trend

Happening mostly on the playground

Tardiness a problemDisrespect also a problem

Happening during morning and lunch recess periods

About 3% of students with 2 or more ODRs, 12 students with 5 or more ODRs, 5 students with >30 ODRs

Using Data to Build Solutions• Prevention: How can we avoid the problem context?

– Who, When, Where– Schedule change, curriculum change, etc

• Teaching: How can we define, teach, and monitor what we want?– Teach appropriate behavior– Use problem behavior as negative example

• Recognition: How can we build in systematic reward for desired behavior?

• Extinction: How can we prevent problem behavior from being rewarded?

• Consequences: What are efficient, consistent consequences for problem behavior?

• How will we collect and use data to evaluate (a) implementation fidelity, and (b) impact on student outcomes?

What Information do we need?Universal Screeners…

• Office Discipline Referrals– Rate, location, time, problem behavior , student, – Other? Ethnicity, gender grade, teacher, – 0-1, 2-5, 6 or more

• Attendance Rates (patterns)• Tardies• Benchmarks, CBM, Grades, Homework completion pattern• Nurse log• Counselor log• SW expectations- Burke et al

Next Steps• Next year…

– Continue to gather and act on behavior and reading data

• Three data review sessions

• Focus on strengthening Universal Support Systems

– Training on Secondary and Intensive Level Supports

Decision Rules for Access to Advanced Tiers (and decision rules for prevention-if we can predict the trajectories , then

we can prevent it from happening)

• Youth has 2 Major ODRs• Youth has 1 Suspension• Youth experiences more than ? minutes out of instruction • Youth misses more than ? days unexcused absences• Youth drops GPA by more than ??• Youth – benchmark testing- McIntosh • Youth- incomplete class work/homework• Attendance (look at predictors for drop-out and school completion)• Admin Referral• Teacher/Staff Referral• Family Referral• Other: