Module 3: Data-Based Decision-Making Using Outcome Data Session III Team Training Presented by the MBI Consultants Implement Solution(s) with High Integrity Implement Solution(s) with High Integrity Establish Solution Goal(s) Establish Solution Goal(s) Identify Problem with Precision Monitor Impact of Solution(s) and Compare with Goal Monitor Impact of Solution(s) and Compare with Goal Evaluate Problem and Redirect Evaluate Problem and Redirect Meeting Foundations Team-Initiated Problem-Solving (TIPS II) Model Discuss and Select Solution(s) with Contextual Fit Discuss and Select Solution(s) with Contextual Fit Collect and Use Data Video REVIEW: DEFINING THE PROBLEM Using data the primary problem is the difference between expected/desired student behavior & current student behavior Identify Problem with Precision 4 Starting with Primary Problem Statements (Big Picture) Look at the Primary Problem, use data to refine the problem by moving to development of Precise Problem Statement(s) Move to Precise Problem Statements What, Where, When, Who, Why More Precision Is Required to Solve Identified Problems 2 (p. 11) How precise is precise? Gang-like behavior is increasing. Bullying (verbal and physical aggression) on the playground is increasing during first recess, is being done mostly by four 4 th grade boys, and seems to be maintained by social praise from the bystander peer group 4 Why? The hardest question Given a specific behavior in a specific context: What consequence is perceived as maintaining the problem behavior? Always assess motivation AFTER you have defined who, what, where? You always ask WHY the students misbehave in A SPECIFIC CONTEXT Look for the primary motivation if there are multiple possibilities. 7 Precision Problem Statement Planner # 1 Data Observations What patterns (if any) do we see in the Big 5 Reports? A. Per Day Per Month Overall: For a building our size the average = ___with a range of ___ to ___ Our average per day per month rate for the current month = ___ with a ___ trend across ___year(s) (stable/increasing/decreasing ) B. Problem Behavior: Top problem behavior this month? C. Location: What location are referrals coming from? D. Time of Day: Are there particular times of day that stand out t? E. Students Involved: How many students are involved this month? 0-1 referrals = ___ 2-5 referrals = ___ 6+ referrals = ___ 8 5 Example: Trevor Test School Middle School Grades 6, 7, and 8 565 students Referrals are equally distributed throughout each grade level Is there a problem? (p. 11) Cafeteria Class Commons Hall 11:45 Lang. Defiance Disruption Harass Skip Gathering Data 75 students with 1 ODR; 28 students with 2 ODRs This report is for 3 or more ODRs Gathering Data Perceived motivation for inappropriate language and disruption in the cafeteria (all students) Gathering Data Step 1: Developing a Precision Problem Statement for Trevor Test School # 1 Data Observations What patterns (if any) do we see in the Big 5 Reports? A. Per Day Per Month Overall: For a building our size the average = 5.6 with a range of 4 to 13 Our average per day per month rate for the current month = 13 with a DK trend across DK year(s) (stable/increasing/decreasing) (no comparison data available) B. Problem Behavior: Top problem behavior this month? Disruption, *(Inappropriate language, defiance) C. Location: What location are referrals coming from? Cafeteria *(hallway) D. Time of Day: Are there particular times of day that stand out? 11:45 (NOTE: Range from 11:30 to 12:15) E. Students Involved: How many students are involved this month? referrals = referrals = referrals = 6 (NOTE: 108 referrals occurred between 11:30 to 12:15) 5 A Precise Problem Statement for Trevor Test School During lunch from 11:30 to 12:15 (When) 108 students from all grade levels (Who) are engaging in disruption, inappropriate language and defiance (What) in the cafeteria and hallway (Where) and the behaviors appear to be maintained by peer attention (Why) 5 1 Implement Solution(s) with High Integrity Implement Solution(s) with High Integrity Establish Solution Goal(s) Identify Problem with Precision Identify Problem with Precision Monitor Impact of Solution(s) and Compare with Goal Monitor Impact of Solution(s) and Compare with Goal Evaluate Problem and Redirect Evaluate Problem and Redirect Meeting Foundations Team-Initiated Problem-Solving (TIPS II) Model Discuss and Select Solution(s) with Contextual Fit Discuss and Select Solution(s) with Contextual Fit Collect and Use Data Video Step 2: Goals Is it a SMART goal? S pecific: How do we want the problem to change? M easureable: Does it address the W questions? What evidence do we need to show that we have achieved the goal? A ttainable: Is the solution appropriate to your goal? R elevant: Is it achievable based on the context? T imely: What is the time frame we could expect this change to occur? 5 Examples of Measures for Goals Percent reduction from current (baseline) status of precisely-defined problem Absolute reduction in daily or monthly rate Satisfaction level by those involved Identify date which you expect (hope) to achieve Goal A Goal for Trevor Test School By April 1 during lunch period from 11:30 to 12:15 (when) all students (who) will show an overall reduction of ODRs from 108 to 52 (50%) (what) in the cafeteria and hallway (where) The behavior appears to be maintained by peer attention and a lack of sufficient supervision (why) 5 Implement Solution(s) with High Integrity Implement Solution(s) with High Integrity Establish Solution Goal(s) Establish Solution Goal(s) Identify Problem with Precision Identify Problem with Precision Monitor Impact of Solution(s) and Compare with Goal Monitor Impact of Solution(s) and Compare with Goal Evaluate Problem and Redirect Evaluate Problem and Redirect Meeting Foundations Team-Initiated Problem-Solving (TIPS II) Model Discuss and Select Solution(s) with Contextual Fit Discuss and Select Solution(s) with Contextual Fit Collect and Use Data Video Solutions with Contextual Fit Do the proposed solutions address or fit the goal? Is the solution appropriate for problem? Is the solution likely to produce desired change? Is the solution linked to assessment? Solution Action Elements Possible Generic Solution Actions Prevent What can we do to prevent the problem? Adjust physical environment. Define & document expectations and routines. Assure consistent & clear communication with all staff. Teach What do we need to teach to solve the problem? Explicit instruction linked to school wide expectations. Teach what to do, how to do it and when to do it. Model respect. Reward What can we do to reward appropriate behavior? Strengthen existing school wide rewards. Include student preferences. Use function-based reinforcers Extinguish What can we do to prevent the problem behavior from being rewarded? Use signal for asking person to stop. Teach others to ignore (turn away/look down) problem behavior. Correct What will we do to provide corrective feedback? Intervene early by using a neutral, respectful tone of voice. Label inappropriate behavior followed by what to do Follow SW discipline procedures Safety Do we need additional safety precautions? Separate student from others if he/she is unable to demonstrate self-control. Make sure adult supervision is available. 2 Prevent TriggerChange lunch schedule so fewer students are eating between 11:30 AM & 12:00 PM. Define and TeachReview established expectations to see if change is needed; reteach cafeteria and hall expectations in class periods occurring just prior to lunch; develop and post expectation signage in cafeteria Reward/ReinforceSet up Friday 5 (extra 5 mins. of lunch time on Friday, if no ODRs occur in cafeteria during lunch time) ExtinctionMake reward for problem behavior less likely; ensure staff dont argue back and forth with student if instance of disruption occurs (may be an inadvertent reward); remind students that paying attention to a disruptive student can mess up Friday 5 Corrective ConsequenceEnsure active supervision during lunch (add one supervisor between 11:30 AM and 12:00 PM?); ensure quick corrective consequence, per our handbook SafetyIncrease adult supervision by 50% Trevor Test Middle School Hypothesis - cafeteria and hallway overcrowded; attention from peers rewarding disruption p. 13 Action Planning Every solution/task needs an action plan Who will do the task? When will the task be completed? 4 For every solution action, define who will complete it with a specific date for completion Problem-Solving Action Plan for Trevor Test Middle School Precise Problem Statement Solution ActionsWho?When?Goal, Timeline, Rule & Updates Many - students are engaging in disruption, inappropriate language and harassment in cafeteria and hallway during lunch, and the behavior is maintained by peer attention Prevention: Maintain current lunch schedule, but shift classes to balance numbers Teach: Teach behavioral expectations in cafeteria Principal to adjust schedule and send to staff Teachers will take class to cafeteria; Cafeteria staff will teach the expectations Changes begin on Monday Rotating schedule on November 15 Goal: Reduce cafeteria ODRs by 50% per month (Currently 24 per month average) Measure: 1. SWIS ODRs 2. Brief fidelity survey Timeline: Review monthly Recognition: Establish Friday Five: Extra 5 min of lunch on Friday for five good days Extinction: Encourage all students to work School Counselor and Principal will create chart and staff extra recess Principal to give announce- ment on intercom on Monday Implement Solution(s) with High Integrity Implement Solution(s) with High Integrity Establish Solution Goal(s) Establish Solution Goal(s) Identify Problem with Precision Identify Problem with Precision Monitor Impact of Solution(s) and Compare with Goal Monitor Impact of Solution(s) and Compare with Goal Evaluate Problem and Redirect Evaluate Problem and Redirect Meeting Foundations Team-Initiated Problem-Solving (TIPS II) Model Discuss and Select Solution(s) with Contextual Fit Discuss and Select Solution(s) with Contextual Fit Collect and Use Data Video Fidelity of Implementation Measure the degree in which the intervention was implemented as defined/expected Use percent/absolute value/ rate/scale as metric Strive for 80% fidelity of implementation as measured weekly (bi-weekly) on scale of 1-5 Are we implementing the plan? No Yes Make easy for staff to record data Fidelity Check Board: X on number line Fist of five Fidelity check basket Direct observation Trevor Test Middle School Evaluation Plan How & With What?When?Who? Measuring Fidelity of Implementation Observe cafeteria supervision. Provide feedback on physical movement, use of names, use of specific positive feedback Once a weekPrincipal Team Teach Expectations to all grades. Keep records of dates and what was taught Fidelity of instruction rating 1-5 (did we do it?, did we do it well?, was it worth the time?) MondayTeachers & MBI Coach Trevor Test Middle School Fidelity Questions: How will we know if the solution was implemented with fidelity? How will we measure fidelity, how often, and who will administer fidelity measure? Implement Solution(s) with High Integrity Implement Solution(s) with High Integrity Establish Solution Goal(s) Establish Solution Goal(s) Identify Problem with Precision Identify Problem with Precision Monitor Impact of Solution(s) and Compare with Goal Monitor Impact of Solution(s) and Compare with Goal Meeting Foundations Team-Initiated Problem-Solving (TIPS II) Model Discuss and Select Solution(s) with Contextual Fit Discuss and Select Solution(s) with Contextual Fit Collect and Use Data Evaluate Problem and Redirect Evaluate Problem and Redirect Video Monitor Impact Effectiveness of Implementation Impact on students? Effectiveness of solution? Data analyst brings data summaries needed for an update of implemented solution outcomes Are we solving the problem? Is the desired goal being achieved? Frequency of Monitoring Consider your meeting schedule and monitor impact of solution by most recently-completed calendar month First of month Time since last meeting Time since solution was implemented or partially implemented 31 TIPS II Training Manual (2013) TIPS II Training Manual (2013)32 February 2, 2012 March 1, 2012 Monitoring the Impact of Solution Against Goal Implement Solution(s) with High Integrity Implement Solution(s) with High Integrity Establish Solution Goal(s) Establish Solution Goal(s) Identify Problem with Precision Identify Problem with Precision Evaluate Problem and Redirect Evaluate Problem and Redirect Meeting Foundations Team-Initiated Problem-Solving (TIPS II) Model Discuss and Select Solution(s) with Contextual Fit Discuss and Select Solution(s) with Contextual Fit Collect and Use Data Monitor Impact of Solution(s) and Compare with Goal Monitor Impact of Solution(s) and Compare with Goal Video Evaluation Plan Every problem needs evaluation and follow-up Assessment of plan implementation & effectiveness Data indicates response to the solution? Problem solved? Goal met? Adaptation? Continue or set a new goal? Revise or adjust solution? Change the solution? Activity: Using Your Data Define problems with precision Establish SMART goals Build solutions by asking the right questions Select actions that are effective and have strong contextual fit Build Solution Actions (who and when for selected solutions) Evaluate fidelity of implementation Monitor impact on student outcomes Evaluate the effectiveness of the plan