Katie Gray Unique Potential Consulting Catherine Griffith, Karen Harrington Fredrickson Center for...

THE IMPACT OF A CONSTRUCT-BASED APPROACH

IN ASSESSING STUDENT PROGRESS AT THE ELEMENTARY LEVEL

Katie Gray Unique Potential Consulting

Catherine Griffith, Karen Harrington Fredrickson Center for School Counseling Outcome Research and Evaluation

This presentation will outline how and why a research driven, construct based approach to collecting social emotional data via a “Protective Factor Index” [PFI] was developed as part of an ESSC Grant funded project

We will address why a construct based approach to collecting data on non-cognitive factors is important to facilitate meaningful data based decision making for school counselors as well as to enhance the process for other educators in the school.

The goal for such an approach is to allow data teams to determine not just WHAT a student has not learned but WHY he/she is not doing well academically. Once the why is established, a team can expand the menu of interventions for these children to include school counselor led programs and services designed to support the acquisition of social-emotional competencies that research says matter.

THE RANDOLPH PROJECT

3 year ESSC Grant awarded in 2012 Grant goals:

Implement the ASCA Model and link the work of school counselors to academic achievement

Institute a meaningful data-based decision-making model at the district’s 4 elementary schools

Create a Protective Factor Index {PFI} reporting system, using student level data to drive interventions and classroom guidance curriculum

THE RANDOLPH TEAM Unique Potential Consulting Fredrickson Center for School

Counseling Outcome Research and Evaluation

Sebastian Management 4 newly hired school counselors District superintendent and principals

SCHOOL COUNSELORS AND ACADEMIC ACHIEVEMENT Ed Trust, The ASCA Model and

AccountabilitySchool counselors must use data to

demonstrate the impact of their work and link it to academic achievement

School counselors should be leaders in data based decision making in their schools to support closing achievement gaps and other school improvement efforts

SCHOOL COUNSELORS AND DATA Theory Vs Practice

Counselors often not adequately trained to use data effectively

Most often the data that is available is academic data

Counselors operating in a social-emotional “data vacuum” [except for attendance and discipline data]

Little information available that reveals information about underlying issues that need to be addressed in order for academic achievement to improve

Not clear what socio-emotional data to collect Mechanisms to collect social-emotional data don’t

exist

DEVELOPING AND PILOTING A CONSTRUCT BASED APPROACH

TO SOCIO-EMOTIONAL DATA

Meeting the Challenges

WHAT SOCIO-EMOTIONAL DATA SHOULD BE

COLLECTED?

Challenge #1

THE RESEARCH Squier, Nailor and Carey[2014]

Comprehensive review of educational and developmental psychology literature to determine what abilities, capabilities and skills are strongly related to academic achievement and later success in lifew

Found 4 overarching constructs that explicitly link to student success: Motivation Self- knowledge Self-direction Relationships

THE RESEARCH [CONT.] Bass, Lee and Carey [2015]

These 4 constructs associate strongly with students’ academic achievement and well being

These 4 constructs are malleable and receptive to intervention

These 4 constructs are within the range of expertise of school counselors

GETTING STARTED The Grant Team created a “Protective

Factor Index” [PFI] that consists of a 13-item rating scale based on the Construct-Based Approach (CBA) to school counseling (Squire, Nailor & Carey, 2014), as a research-based mechanism for collecting these social emotional data.

Construct K-5 Standards

Motivation

Engages in class activities Demonstrates an eagerness to learn Demonstrates perseverance in completing tasks

Self-knowledge

Identifies personal feelings Identifies things he/she is interested in learning Identifies things he/she values

Self-direction

Demonstrates the ability to self-regulate feelings and actions Demonstrates resilience after setbacks Uses learning strategies effectively Maintains an organized work area; Produces neat and careful work

Relationships

Works collaboratively in groups of various sizes Respects and accepts authority Forms respectful, equitable relationships with peers

THE PFI

GETTING DISTRICT BUY-IN

Challenge #2

THE GRANT TEAM PROPOSAL Use the district’s existing Student

Information System Include the PFI on all report cards K-5Create a scoring rubric to accompany the

PFITrain classroom teachers to use the rubric

to determine whether a student was demonstrating expected age appropriate skills in each domain

Have classroom teachers complete the PFI each time they enter grades

CREATING A USER-FRIENDLY SYSTEM FOR COLLECTING THE SOCIO-EMOTIONAL DATA

Challenge #3

A SAMPLE REPORT CARD

School Counseling Program Protective Factors Protective Factors are characteristics and behaviors that reflect positive internal growth and development of students. These factors include core constructs such as positive values and identities, social competencies and commitment to learning. Research shows that development of protective factors increases likelihood of success in school. Rating: Student demonstrates the protective factor 1- Consistently [90-100% of the time] 2- Most of the time [75-90% of the time] 3- Some of the time [50-75% of the time] 4- Rarely or not at all [less than 50% of the time]

Protective Factors T1 T2 T3

Motivation

Actively participates in class activities

Demonstrate perseverance Takes pride in work performance Able to identify activities he/she enjoys when given choices

Self-Knowledge

Identifies personal feelings Identifies personal strengths Asks for help when appropriate Aware of personal boundaries

Self –Direction

Listens attentively Organizes workspace and materials Completes tasks in allotted time Demonstrates ability to self-regulate feelings and actions

Relationships

Cooperates with others Respectful to others Forms relationships with peers Respects teacher authority

Randolph Public Schools K-1 Report Card

School Year 2013-14

Draft 1 April, 2013

Student Name:

Teacher’s Name:

Attendance record

THE RUBRICRating Scale

4 (Meets Standard) Student consistently demonstrates this standard with independence 90-100% of the time

3 (Progressing towards Standard) Student often demonstrates this standard with occasional support from teacher 75–89% of the time

2 (Emerging) Student occasionally demonstrates this standard with frequent support from teacher 60–74% of the time

1 (Not meeting Standard) Student does not demonstrate this standard and needs ongoing encouragement and teacher support Less than 60% of the time

KINDERGARTEN AND FIRST GRADE

MOTIVATION

Meets Standard (4)

Progressing(3)

Emerging(2)

Not Meeting Standard (1)

Engages in class activities

Listens and contributes during class activities and discussions; Follows directions; Takes turns; Joins an activity when askedConsistently, independently is engaged 90-100% of the time

With occasional teacher supportis engaged 75-89% of the time

With frequent teacher supportis engaged 60-74% of the time

Needs ongoing teacher support and is often disengaged

Demonstrates an eagerness to learn

Volunteers to take on new challenges (e.g., trying a new activity); Asks questions to further understanding of a topic; Maintain a positive attitude when faced with challenges; Expresses enthusiasm when learning something newConsistently, independently is eager to learn 90-100% of the time

With occasional teacher supportis eager to learn 75-89% of the time

With frequent teacher supportis eager to learn 60-74% of the time

Needs ongoing teacher support and is often not eager to learn

Demonstrates perseverance in completing tasks

Uses strategies to work through challenges and complete work; Maintains a positive attitude when faced with a difficult or unfamiliar task; Recovers from setbacksConsistently, independently demonstrates perseverance90-100% of the time

With occasional teacher supportdemonstrates perseverance 75-89% of the time

With frequent teacher supportdemonstrates perseverance 60-74% of the time

Needs ongoing teacher support and often does not demonstrates perseverance

Exhibits positive and optimistic behavior

Maintains a positive attitude when engaged in class activities; Expresses enthusiasm; Believes in own capability as a learner; Shares excitement about learning with peers; Identifies aspects of school that are interesting and excitingConsistently, independently is positive and optimistic 90-100% of the time

With occasional teacher supportis positive and optimistic 75-89% of the time

With frequent teacher supportis positive and optimistic 60-74% of the time

Needs ongoing teacher support and is often is not negative and pessimistic

2ND AND 3RD GRADE

SELF-KNOWLEDGE

Meets Standard (4)

Progressing(3)

Emerging(2)


Identifies academic strengths and abilities

When asked, can name multiple strengths and abilities (e.g., “I am a good writer,” or “I have neat handwriting”Consistently, independently names academic strengths and abilities

With occasional teacher supportnames academic strengths and abilities

With frequent teacher supportnames academic strengths and abilities

Needs ongoing teacher support to name academic strengths and abilities

Identifies things he/she is interested in learning

When asked, can name multiple learning interests (e.g., “I like learning to read,” or “I want to learn more about butterflies”)Consistently, independently identifies learning interests

With occasional teacher supportidentifies learning interests

With frequent teacher supportidentifies learning interests

Needs ongoing teacher support to identify learning interests

2ND AND 3RD GRADE SELF-DIRECTION

Meets Standard (4)

Progressing(3)

Emerging(2)


Demonstrates the ability to self-regulate actions and emotions

Sustains attention during activities (such as centers) and routines (such as circle time); Resolves or avoids conflicts with others; Exhibits safe behavior across school settings, throughout the day; Refrains from emotional outbursts (e.g., temper tantrums or major upsets)

Consistently, independently self-regulates90-100% of the time

With occasional teacher supportself-regulates 75-89% of the time

With frequent teacher supportself-regulates 60-74% of the time

Needs ongoing teacher support and is often unable to self-regulate

Demonstrates resilience after setbacks

Recovers from (academic and social) disappointments and challenges without a lengthy disruption in engagement (e.g., being told that s/he completed an assignment incorrectly)Consistently, independently demonstrates resilience90-100% of the time

With occasional teacher supportdemonstrates resilience 75-89% of the time

With frequent teacher supportdemonstrates resilience 60-74% of the time

Needs ongoing teacher support and is often does not demonstrate resilience

Makes productive use of classroom time

Participates in activities as assigned; Stays focused on the task at hand; Completes assignments and activities on timeConsistently, independently is productive90-100% of the time

With occasional teacher supportis productive 75-89% of the time

With frequent teacher supportis productive 60-74% of the time

Needs ongoing teacher support and often not productive

4TH AND 5TH GRADERELATIONSHIPS

Meets Standard (4) Progressing

(3)Emerging(2)


Works collaboratively in groups of various sizes

Listens to peers; Participates actively (e.g., sharing verbally and/or listening); Works with classmates to complete a task in the time given; Demonstrates flexibility taking on different roles during group work (e.g., can follow and lead)

Consistently, independently works collaboratively in groups

With occasional teacher supportworks collaboratively in groups

With frequent teacher supportworks collaboratively in groups

Needs ongoing teacher support and often does not work collaboratively in groups

Seeks assistance when necessary

Uses classroom resources and protocols (e.g., asking a peer, using a reference sheet) before asking an adult for help; Actively seeks the appropriate adult/s who can provide the needed support

Consistently, independently seeks assistance when needed 90-100% of the time

With occasional teacher supportseeks assistance when needed 75-89% of the time

With frequent teacher supportseeks assistance when needed 60-74% of the time

Needs ongoing teacher support and is often unable to seek needed assistance

Respects and accepts authority

Follows directions provided by all adults at school; Communicates politely with adults; Responds to limit-setting without disruptive outbursts; Knows and follows school rules

Consistently, independently respects authority90-100% of the time

With occasional teacher supportrespects authority perseverance 75-89% of the time

With frequent teacher supportrespects authority perseverance 60-74% of the time

Needs ongoing teacher support and often disrespects authority figures

Forms respectful, equitable relationships with peers

Demonstrates care for classmates and has formed friendships with peers; Identifies peers by name; Cooperates with peers and can work with others towards a common goal (e.g., completing a group science project or playing in a soccer game at recess); Avoids or resolves interpersonal conflicts (during instructional and non-instruction times)

Consistently, independently forms respectful relationships with peers

With occasional teacher supportforms respectful relationships with peers

With frequent teacher supportforms respectful relationships with peers

Needs ongoing teacher support and often behaves disrespectfully towards peers

COUNSELORS LEADING WITH DATA VS. COLLECTING DATA The beauty of this system for data collection is that

students are rated by the teachers who are with them all day and have a picture of whether the student understands a construct AND whether he/she puts it into practice.

The counselor is the analyzer/interpreter of data trends, rather than the actual data collector.

The teacher is the beneficiary of additional data, interpreted by the school counselor, to help understand why a child may be struggling.

Together, teachers and counselors can have data

based discussions about how to intervene.

WHAT PROFESSIONAL DEVELOPMENT IS NECESSARY TO SUPPORT THIS APPROACH?

Challenge #4

TEACHER PD

Teachers needed to know WHY they were being asked to complete the PFI

They also needed to know HOW to complete it accurately

COUNSELOR PD Counselors needed to learn HOW TO

EXTRACT the PFI data from the SIS Counselors also needed to learn how to

understand/analyze and use the data they were seeing

Counselors needed training in EXCEL Counselors needed training in leading data

based conversations with teachers and families

Counselors needed help identifying evidence based interventions that are available to support needs identified by the data at the universal [tier1] as well as Tier 2 level

PD FORMATS Large group

How/Why for teachers, families

EmbeddedCounselors have been charged with

delivering on-going embedded professional development by presenting PFI data after each marking period and leading conversations about school wide trends at faculty meetings, grade level trends at Common Planning Time meetings and classroom and individual trends in consultation meetings with teachers.

PROMISING PRACTICES

#1: IT IS POSSIBLE TO COLLECT SOCIO-EMOTIONAL DATA VIA EXISTING SIS SYSTEMS

Teachers reported they do not feel overburdened by adding PFI information when they complete report cards

With training in exporting to EXCEL, as well as the filter, sort and chart making functions of EXCEL, counselors can use PFI data to examine district, school, grade level, classroom and individual trends relative to each of the factors on the PFI.

Counselors report it takes them 2-3 hours after each marking period to manipulate the data, analyze it and prepare charts

#2: THE RUBRIC IS A VALUABLE TOOL Each factor is described with concrete,

observable behaviors and ratings are quantified

Analysis of the data …..Karen add here about reliability

Important to share with parents Parental feedback has been very

positive; parents agree socio-emotional factors are important and they value getting this information on a regular basis

#3 PFI DATA USED TO DRIVE TIERED INTERVENTIONS

Analysis of baseline data indicated that self-regulation was problematic at every grade at every school

SELF REGULATION DATA FROM 4 ELEMENTARY SCHOOLS

DATA DRIVEN INTERVENTIONS Counselors revised their Tier 1 services to

include targeted instruction on self-regulation to all classes in every grade via the developmental guidance curriculum

Teachers’ summer reading was “The Zones of Regulation”

Counselors led a faculty in-service on “The Zones”

Additionally, data is used to identify students needing additional support [Tier 2] so they can be placed in small groups focusing specifically on self-regulation

#4 COUNSELORS ARE SEEN AS LEADERS AND DATA EXPERTS Counselors are taking on a leadership role

in helping their colleagues understand the connections between non-cognitive factors and academic achievement

Teachers and Administrators are having “ah-ha” moments where they see these connections and then are looking for strategies to help promote the acquisition of these factors

Counselors are collecting teacher led strategies to promote self-regulation and are publishing best practices for all.

A COUNSELOR DEVELOPED PPT FOR 4TH

GRADE TEACHERS’ COMMON PLANNING TIME

Protective Factor Index

2013-2014 Term 3 Data

What are Protective Factors?

• The protective factor index highlights a different set of skills than those traditionally focused on on a report card. These 13 skills have been statistically linked to success in children. Therefore, this data is very important in understanding areas where our students are thriving and areas they need to improve in. By looking at this data, it can help us to be more focused on the type of interventions we put into place.

What are we using this data for?

• Tier 1: Classroom lessons that target areas that a high number of students are struggling in

• Tier 2: Social/Emotional groups are formed based on this data

• Tier 3: Individual’s needs can be determined using this data and can lead appropriate interventions

Who produces this data?

• YOU! This data comes from the 2013-2014 report cards (specifically focusing on Term 3 for this power point). The 2014-2015 report cards will have a 4 point scale, instead of just “struggling” or “on target” to more specifically indicate student growth.

#1) Demonstrates an eagerness to learn

%Struggling % On-Target0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

0.16

0.84

Current Fourth Graders

Third

#2) Demonstrates perseverance in completing tasks


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

0.19

0.81


Third

#3) Demonstrates resilience after setbacks


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.12

0.88


Third

#4) Demonstrates the ability to self-regulate actions and emotions


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

0.15

0.85


Third

#5) Engages in class activities


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.13

0.87


Third

#6) Exhibits positive and optimistic behavior


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.13

0.87

Currrent Fourth Graders

Third

#7) Forms respectful, equitable relationships with peers


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.06

0.94


Third

#8) Identifies academic strengths and abilities


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.12

0.88


Third

#9) Identifies things he/she is interested in learning


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.06

0.94


Third

#10) Makes productive use of classroom time


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.25

0.75


Third

#11) Respects and accepts authority


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.06

0.94


Third

#12) Seeks assistance when necessary


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.12

0.88


Third

#13) Works collaboratively in groups of various sizes


0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

0.22

0.78


Third

Questions/Comments

• What did you notice?• Where are we struggling? • Where are we doing well?• What interventions do we already have in

place?• What interventions might be helpful for us to

put in place to increase our number of on target students?

#5: DISTRICT IS EXPLORING INCORPORATING PFI AT MS AND HS

Superintendent is prioritizing budget funds tosupport building more data collection

infrastructureTraining MS and HS counselors to be leaders

in data-driven conversations about the connections between non-cognitive factors and academic achievement

THE CRUCIAL NEXT STEPSOver time it will become clear whether intervening to support the development of non-cognitive factors results in improved academic achievement.

Katie Gray Unique Potential Consulting Catherine Griffith, Karen Harrington Fredrickson Center for...

Documents

Transcript of Katie Gray Unique Potential Consulting Catherine Griffith, Karen Harrington Fredrickson Center for...