02/29/2020

1

Utilizing Research Practice Partnerships to Advance

Implementation of Evidence-BasedPractices in EI/ECSE

Sondra M. Stegenga Ph.D., Sloan O. Storie, Ph.D., Erin Kinavey Wennerstrom, M.Ed., Jane Squires Ph.D.

Conference for Research Innovation in Early Intervention (CRIEI)San Diego, CA - February 29, 2020

Session Agenda for Today• Brief background to the problem + overview of implementation

science and research practice partnerships (RPPs)• Overview of three studies founded in partnership frameworks and

focused on EI/ECSE/ECE related settings/needs (focus of these overviews on partnership aspects/impacts)• Strengths• Challenges• Potential Solutions• Methodological Considerations• Discussion and Problem Solving

Background to the Problem• Use of evidence based practices (EBPs) is foundational to

improving long term outcomes (e.g. Odom & Cook, 2013)

• We are charged to use EBPs in early intervention and early childhood special education (EI/ECSE) through recommended practices and regulations (e.g. DEC, 2009; DEC, 2014; IDEIA, 2004)

• Yet, we know there is a well-cited research to practice gap in EI/ECSE and education with few EBPs moving in to practice or few making it to scale (e.g. Odom, 2009; Dunst, 2013)!

“The lack of attention to implementation methods has led to what some have termed the quality chasm”

- National Implementation Research Network (NIRN)

What can we do about it?• Increase focus on implementation related issues (moving beyond

fidelity)• Readiness• Implementation• Scale-up• Sustainability

• Authentic Research Practice Partnerships (RPPs)

What are Research Practice Partnerships (RPP)s?

Research Practice Partnerships (RPP)s in education are identified as “long-term collaborations between practitioners and researchers that

are organized to investigate problems of practice and solutions for improving schools and school districts and….”

– Coburn & Penuel (2016)Mutual ownership over

the project

Community partners are involved in all

aspects of the research from initiation

of the idea, to data collection, analysis,

results, and resulting systems/practice

change

Mutual benefit

Data is often organically driven (e.g. already gathered such as administrative

data or partners may gather the data themselves or be directly involved in data

processes (may require more sophisticated analyses due to data

imperfections)

02/29/2020

2

Looks Like/Doesn’t Look Like for RPPsLooks Like

• EACH member is considered an expert and feels equal in the partnership• All opinions are considered and

purposefully elicited to ensure each and every voice is heard• Research decisions and ideas stem

from collaborative discussion• Requires trust building over time• Aims to solve a real-world pressing

issue/problem of practice as determined by the community partners

Doesn’t Look Like• Researcher as expert• Researcher always leading and the main voice

in meetings• Research decisions and ideas are formed by the

researcher possibly with approval from the community after the ideas have been established (e.g. obtains letters of support from partners but they did not assist in the design or innovation of the idea)• Researcher does not have long standing

relationship with the community partner(s)• Issue being addressed may be a gap in the

research/literature but not necessarily a highly rated priority related to practice issues

Why Do We Need RPPs?• Works to amplify community and stakeholder voices• Communities know best• Democratization of data and research• Move from doing to to collaborating with from the very beginning and

throughout• Our ethical obligation through CEC – DEC to ensure we are using EBPs – if

they are not being implemented we really are not using the EBPs and reaching those who need them most!• Will never know what works, for whom, and under what conditions

without partnering in real world settings under authentic conditions (there remains a lot of unaccounted for variance in our models – hence we must continue to work to identify what this might be!)

Three Case Examples of RPP Research in EI/ECSE/ECE Related Settings

Maximizing Pilot Phase Measures to Inform Quality Improvement in

Early Intervention Systems

Sondra M. Stegenga Ph.D., M.S., OTR/LAssistant Professor – Department of Special Education

sondra.stegenga@utah.edu

Journey to the Research… History and Development of the RPP

• Project to Improve

Identification and Service

Coordination for

Infants/Toddlers with Feeding

Difficulties

• Resulted in local guidance

through collaboration with

district leadership

• Requests for sharing

throughout the state

• Role: PI and EI Provider (OT)

collaborating with local

systems leaders

2009-2011

• Project to Improve Service

Access for Infants/Toddlers

with Feeding Difficulties

• Identified that service areas

with higher levels of

collaboration also had lower

reported rates of denials for

feeding supports

• Role: PI and Local Part C

Leader collaborating with

CSPD, state, and local leaders

(stemmed from 2009-11

project)

2012-2015

• Project to Improve service

delivery in Part C EI service Areas

• Resulted in self reports of service

providers using Primary Service

Provider approach

• Resulted in service delivery

improvements (e.g. formalized

meeting times, foundational

understanding of PSP in team

structures)

• Role: EI Program Leader

collaborating with State CSPD and

state leaders

2013-2017

• Project focused on use of

EBPs related to SSIP C11

indicator - social emotional

• Role: PI collaborating with

CSPD and state leaders

2018-2019

Less Formal Collaboration (e.g. no written guidelines)

Formal Collaboration(Data sharing and partnership agreements in writing)

02/29/2020

3

Theoretical Foundations and Background• The premises of partnership underlie all I do in my work as an

educator, researcher, and partner in the community• Overarching goal is to improve use of evidence-based practices in our

EI/ECSE systems• This requires a heavy focus on integrating implementation,

improvement, and partnership frameworks into processes from the beginning of the design of the intervention to the implementation and scale up of interventions and EBPs

Recent Project: Examining Implementation Related to the New Part C Indicator C11(State Identified)

• State identified focus on social emotional development• Stemmed from the State Systemic Improvement Planning (SSIP)

process• Focused on aspects my partners wanted to address and better

understand but did not have time or money in the current plan (meeting a self-identified need)• State partners were in a pilot phase aiming to do full state scale-up

based on findings from the initial pilot sites

Big Picture• What can we learn from this pilot phase prior to full state

scale up???• Plan Do Study Act (PDSA) to avoid implementation

failure!• Requires an intentional and measured approach (Taylor et al.,

2014)

• Get creative! Begin harnessing newly available data sources for measures (Stegenga, Munger, Squires, Anderson, 2018)

• Develop research practice partnerships (RPP) and community based participatory research (Romano & Schnurr, 2020)

For more information on PDSA cycles and other implementation resources: https://nirn.fpg.unc.edu/resources/activity-L61-apply-pdsa-cycle-your-work

Proposed Theory of Change

Theory of Change is a drill down from the SSIP Theory of Action!

What I thought my research was

going to look like & the reality after following the RPP

journey!

Both delicious and beautiful just not the same…and in fact, I

argue the end result is much better (sprinkles on top and all!)

Aim #1 - FeasibilityExamine feasibility, acceptability, and appropriateness

of initial implementation of an evidence based practice (early social emotional assessment use)

within early intervention/Part C systems in the pilot phase

02/29/2020

4

Aim #2 - ImpactExamine usage rates/uptake of the evidence based

practice (early social emotional assessment use) relative to key drivers and determinants (supports

and barriers)

Methods & ParticipantsSequential mixed methods design (quantitative – qualitative)

Participants: • Part C early intervention program leaders and providers serving

infants and toddlers with disabilities birth to age three and their caregivers. • Four service areas in a Midwest state

Methods, Data, & Analyses (brief)• Short comparative interrupted time series• Data – already existing aggregated and de-identified state data of usage

rates of the electronic social emotional assessment• Feasibility, Acceptability, and Appropriateness – validated measures (FIM,

AIM, IAM) (Weiner et al., 2017)• Data - gathered via electronic survey/Qualtrics

• Semi-structured interviews on determinants (supports and barriers to use), qualitative content analysis w/ rapid review format for implementation work• Data – gathered via phone interviews to best meet the needs and

schedules of participants• Stages of implementation completion for further understanding of

determinants• Information gathered through semi-structured interviews with the four

program coordinators according to the Universal Stages of Implementation Completion (U-SIC) in collaboration with the developer, Dr. Lisa Saldana, for this adaptation

Results – Mixed Methods

Notable Findings (brief)• The social emotional assessment (e-DECA) was rated overall

as being feasible, appropriate, and acceptable for use in Part C/early intervention [means of 4.08 (FIM), 4.00 (AIM), and 4.03 (IAM)]

• Initial training had a significant impact on usage rates but one time follow-up leader coaching did not !2 = .36, F (12, 119) = 5.51, p < .01 and accounted for 36% of the variance. Training (p = 0.000), follow-up leader coaching (p =0.148).

• Stages of Implementation Completion (SIC) results identified some gaps in the stage of readiness and also the stage of fidelity monitoring

“We do have access to [expert coach] and then we had access to a state appointed kind of point person to work with us on this project – local admin

“We don’t have written

guidelines yet” – Local admin

“So we have different directives each year” – Provider

Our area was using it pretty highly in the beginning, I don’t know if that continues to be the case” – Local Admin____________________“I haven’t even done an e-DECA this school year” (2019) – Provider

Recommendations (brief)• Establish clear written guidelines with descriptive explanations for use• Manualize trainings for replicability to ensure similar results across trainers in the

future• Mechanize data collection for ongoing ability to conduct objective data analyses

for understanding determinants to use • Use measures and criteria for understanding levels of readiness prior to scale up• Provide local leader training and coaching on systems change and

implementation science to maximize leverage in assisting with systems change

“I mean they have talked about implementation science but I’m like, okay sister, I just want a plan, you know. I don’t want a bunch of charts. I just want a real plan with

what I’m doing” – Local Part C Early Intervention Leader

02/29/2020

5

Overview of the Project

• Monthly meetings with established partnership team and other stakeholders as needed for particular decision-making• Meetings continued 6 months after my dissertation to ensure

dissemination of information and translation into work with the systems• Some hiccups along the way…

• Took 6 months to establish formalized agreements (formalized collaboration & data sharing)

• Established formal roles/responsibilities including identifying what benefits each partner would receive as well as resources dedicated/outcomes expected by each collaborator

Challenges/Offsets of Partnership Work

Timelines delayed due to…

The Polar Vortex of 2019!

What makes it all worthwhile? Real world change and knowing amazing, passionate, individuals truly dedicated to improving the lives of young children and their

families! (and random emails) J

Shared Visions: An Exploratory Approach to Understanding the

Opioid Treatment of MothersSloan Storie, Ph.D.

Assistant Professor – Department of Special Education and Child Development

sstorie1@uncc.edu

Community Partnership Background: Partnerships

Gaps between research and

practice require participatory

approaches (M inkler,

Salvatore, & Chang, 2018)

Pragmatic research aims to understand the context of the problem (Glasgow, 2013)

02/29/2020

6

Community-based Participatory Research Continuum of engagement in CBPR (Balazs & M orello-Frosch, 2013)

Background: Community Needs

• Data to inform decision making • Community voice • Understanding of the strengths

and needs of the community• Dissemination of information

“How are we going to shift community

perception?”

Perceived barriers are still barriers

Community Voice

Context of the Problem

• 90 people die each day from an opioid related overdose in the United States (Center for Behavioral Health Statistics

and Quality, 2016).

• 74% of adults who are misusing drugs are also parents (Newport &

W ilke, 2013); of those parents, 70% are women (Niccols et al., 2012).

• 1/2 of children with a parent experiencing opioid use disorder remain in the care of that parent (Taplin & M attick, 2015).

• Since 2010, Caucasian women from rural areas have been the most representative sample of individuals entering treatment programs (Rodriguez & Sm ith, 2018).

Methods & Participants

• Phenomenological approach• Development of guiding

questions• Purposive sampling• Semi-structured interviews• Face-to-face and phone

• Professional transcription

• 18 participants between March and April, 2019• 11 Mothers in recovery

• 18-60 years of age• Child(ren) ≤ 12 years of age• In recovery from OUD

• 7 Community providers working with mothers in recovery• 18-60 years of age• Working with mothers in recovery from OUD• Work in the community

02/29/2020

7

Methods & Analysis

• Social constructivism• Content analysis• Deductive and inductive coding• Secondary coding• Consensus• Member checking

Notable Findings & Limitations

• Importance of the individual, relationships, and community• Need for accessibility,

communication/collaboration• Transition plans in place

• Sample size• Phone interview vs. face-to-

face• Short time frame• Mothers not in recovery • Tribal representation• Generalizability

Dissemination

• Coos County Planning Summit on Substance Misuse• May 8th, 2019

• Oregon Conference on Opioids & Other Drugs, Pain and Addiction Treatment• May 31st, 2019

• Division for Early Childhood• October 2nd, 2019

Future Directions: Research and Practice

• Focus on early supports• Head Start

• Home visiting programs

• Transition Programs• Jail to safe housing/treatment center• Reunification to family-centered practices

• Mixed-method approaches• “How can we shift community perception?”• Evaluation of partnerships

Future Directions: Partnerships and Policy

• Ongoing community-academic partnerships • bridging factors

• Evaluation of partnerships (e.g., trust)• Building collaborative partnerships within the community

• contextual factors• Improve communication systems

• Using community voice to compel change in the community• Connection to community leaders and decision makers to promote

change

Takeaways

• Focus on supports early• Early Intervention• Head Start• Home visiting

• Build and maintain community-academic partnerships• Get everyone at the table

02/29/2020

8

Community Partner UPDATE!

LADPC is implementing a series of advocacy training workshops to train “everyone who cares” on civic organizing methods. The end result- a resolution that elected officials can sign declaring addiction a public health crisis in Coos County.

Compassion fatigue, exclusionary practices and

teacher beliefs about discipline

Erin Kinavey Wennerstrom M.Ed., IMH-E®

community

Alaska ranks 46 among all other states in total road miles.

Age 0-5 by Number of ACES in Alaska

Zero ACEs

59.8%

One ACE

24.7%

Two or More ACEs

15.5%

S o u rce : N atio n a l S u rve y o f C h ild re n ’s

H e a lth 2 0 1 1 /2 0 1 2 , G rap h ic cre ate d b y th e

A laska M e n ta l H e a lth B o ard /A d viso ry B o ard

o n A lco h o lism an d D ru g A b u se S ta ff.

2018AK study shows preliminary prevalence of exclusionary practices

Small team recommends use of Expulsion Policy Strategy Tool, formation of steering team to drive the work and looking into teacher stress and wellness

2010- 2013 Formal TACSEI Effort led by Part C/619. Demonstration sites, reduction in challenging behaviors and a focus on teacher supports

2016 (fall) ACF Releases guidance memo on exclusionary practices 2017 (summer)Alaska brings team to

2019Second AK study shows consistent prevalence of exclusionary practices and begins to look at compassion fatigue, secondary stress, recommends monitoring

Expulsion Policy Strategy Tool assessment

Steering team forms to charter and guide the work.

Develop action plan for coming 12-18 months

Take initial steps to address implicit bias

Support workforce development

2020Another round of AK study of compassion fatigue exclusionary practices planned- Spring

Develop data system to monitor and improve rates of exclusionary practices

Form on-going governance structure and process

Fall 2017Informal RPP begins stakeholder group formed in AK to develop assessment/ survey

RPP development over time

02/29/2020

9

Suspension and ExpulsionDefined preventexpulsion.org

In-school suspensions – Practices that involve removing or excluding the child from the classroom.

Out-of-school suspensions – Practices that involve temporarily removing the child from the program.

Expulsions – Permanent removal or dismissal from the program.

Soft-expulsions – Practices that make it so that the program is not a viable or welcoming care arrangement for the family and leaves the family with little choice but to withdraw their child.

PDSA

2017-18• Initial literature review

and survey design

• M any “hom e-cooked”

survey questions

• Perform ed interviews

with a handful of

program s to identify

causes of stress

• Analysis challenges due

to question design and

two survey groups –

teachers and

adm inistrators

• Decision to use

validated tools and

stop interview

2018-19• Adoption of ProQOL

and STSS instrum ents

• Focus on lead teachers

• Added incentives for

com pletion

• Concurrent with

survey, inter-agency

leadership team

com pleted self-

assessm ent on state of

im plem entation of

policies related to

exclusionary practices

• After analysis, decision

to stop using STSS

instrum ent

2019-20• Addition of BIM S

(behavioral questions)

• M ulti-state survey to

increase responses (N)

• Increased chance of

w inning incentives to

increase response rate

(considering telephone

outreach to drive

survey com pletion)

Methods 2018 Driver Diagram

Driver Diagram, revisited

Com passion Fatigue

Com passion Satisfaction)

W ork Environm ent

Teacher Environm ent

Personal Environm ent

Traumatized by work

Secondary Exposure (ProQOL

STS)

Primary Exposure

Frustration Anger

ExhaustionDepressed by

Work Environment

(ProQOLBurnout)

Years experienceEducationSelf care routinesFinancial

well being

Type of programRurality

SupportsHours per

dayFull staff

Adapted from Stamm, 2009

Exclusionary practices

Teacher beliefs about discipline

02/29/2020

10

Methods and Results (brief)• Data gathered via secure

electronic survey platform (SurveyMonkey Gold)• Participants• 104• Lead teachers in Alaska in a

licensed child care setting (9.6% family child care)

• Descriptive analyses to identify the foundational need and meet the identified aims

• 46% of teachers reported moderate levels of burnout• 35% of teachers moderate levels

of STS• Differences between education

levels and burnout and STSS. • Small n problem• Led to multi state approach

Interesting but not quite there yet…

Expulsion Policy Strategy Tool• Published by Child Care State Capacity Building Center – Administration

for Children & Families• Designed to address the complexity of preventing exclusionary practices• Promotes collaborative and comprehensive inter-agency approach• Used to identify areas of strengths and priorities for action, rating the

extent of implementation of policies across 6 strategies• Clear Goals and Progress Monitoring• Fair and Appropriate Policies• Strong Family Partnerships• Universal Developmental and Behavioral Screening• Highly Skilled Workforce• Access to Specialized Consultation

• Links to resources to address opportunity areas

Assessment of the policy areas in

Expulsion Policy Strategy Tool …

Integrated stage-based framework

• Form team(s)• Define the work• Assess community

needs• Evaluate possible

ways to meet those needs

• Judge feasibility• Decide on plan of

action

• Establish individual and organizational competencies

• Ensure resource availability

• Prepare for deployment

• Services or new program or practices are first made available to providers, families, children (small scale)

• Promote continuous improvement for increasing quality

• Services made available across sites (scale-up to full scale)

Future Research• Impact of training (Implicit bias training)• Mental health consultation and other supports• Early Childhood Educator well being

02/29/2020

11

Bringing It all Together

Strengths to RPP Work• Focused on real community/state/country needs and issues for

increased use of the research and findings• Improved potential for change• Amplifies community voices• Empowers independence in systems change• Creates real world change• Makes a truly impactful difference in our systems and in the lives of

young children and their families!

Challenges to RPP WorkIt Takes TIME• To develop relationships• To establish trust (consistency over time)• To complete an authentic needs assessment to determine research questions

and aims of research• To ensure all necessary partners are involved both in the research (e.g.

methodological expertise) and for understanding the context and issue (e.g. who else in the community needs to be at the table)?

• Can be cost intensive• You may not (and likely will not) get to do the research exactly

according to your initial vision

Potential Solutions to RPP Challenges• Work with partners and the field as a whole to make data collection

and data systems conducive to partnership research• Clear definitions• Standardized protocols for gathering data• Less restrictive processes for aggregated and de-identified data• Procedures in place to allow letters of support• Policies that support partnership research

• Clarify clarify clarify (jargon, meaning, roles, etc.) on a regular basis! We can think we are talking about the same thing but actually are seeing/hearing/thinking very different things!

Remember the Dress? Team Yellow/White or Team Blue/Black – Same Dress but Seeing Very

Different Things!Potential Solutions to RPP Challenges• Ensure that RPP and partnership is not just added piece but rather a

foundational layer to all of your research• Roles and expectations are clearly discussed, agreed upon, and put in writing• Research questions and aims should stem from collaborative work together

• Plan extra time to decrease frustration and ensure that needs are met of all partners• Be willing to follow the journey of the RPP and that reality will look

different than you initially imagine…

02/29/2020

12

Other Collaboration Considerations• Check your data sharing agreements closely! There may be

unreasonable and costly elements that are protocol but not necessarily required pending the type of data you plan to use…• Obtain a formal collaboration agreement (if possible) that outlines

roles of both researcher and community partners• Plan extra time! (Natural disasters, polar vortexes, change in

leadership or staff, illness, change in rules, changes in political parties or leadership with sudden changes in funding and staff, etc etc)• Multiple IRB boards

Other Collaboration Considerations• Time is a resource – how much are you asking

of your partners? Is it planned ahead?• Identify how the partnership will benefit both

the community partner and researcher (symbiotic mutual benefit)• Make sure the purpose for partnership and

expectations are clear (don’t oversell what you can do or it could erode trust)

Methods Considerations and Lessons Learned• Research Practice Partnership is a foundational layer and underpins

all methodological decisions based on the research questions determined collaboratively with partner. Is this regularly being represented in journal articles as part of reporting?• Time is needed for coalition building, needs assessment (formal or

informal). How do we build this in to timelines and IRB?• Real world existing data is often messy – help systems begin to

structure and align data systems so it is easier for their use and collaboration with research partners. What work is occurring related to data alignment for easier collaboration with research? • Studies that are less tightly controlled, as can be in partnership work,

makes it harder to publish – how can we overcome this? (e.g. perhaps adding standards for RPP related studies?)

Thoughts/Considerations/Needs for the Field• Ensure we have reviewers in journals who understand the

foundational concepts of partnership work• Create calls for articles and special journal topics that

focus on partnerships in EI/ECE/ECSE• Use of hybrid designs (Type I, II, III) for

implementation/efficacy to speed translation of research to practice• Tenure policies – should there be considerations for

researchers who engage in practice based collaborative research, such as RPP work, for the additional time it takes to due this work authentically?

Discussion• What are the methodological considerations related to RPPs?• How do we establish formal RPPs?• What types of research funding are available for RPPs and

Implementation Focused Research?• What experiences have you had with RPPs? How have you overcome

some of the barriers?• What impacts have you seen from RPPs? Where are more RPPs

needed?