Intervention Implementation Integrity Within Conjoint Behavioral Consultation: Strategies to Maximize Outcomes

S. Andrew Garbacz, MASusan M. Sheridan, PhDMichelle S. Swanger-Gagne, MAAmanda L. Witte, MA

University of Nebraska-Lincoln

The Nebraska Center for Research on Children, Youth, Families, and Schools (CYFS; http://cyfs.unl.edu)

The project is funded by the Institute of Educational Sciences Grant Award Number: R305F050284

School-Based Consultation

School-based behavioral consultation is an indirect form of service delivery wherein a specialist with expertise in social-behavioral and academic interventions works collaboratively with a treatment agent (i.e., consultee, such as a teacher or parent).

School-based consultation may focus on behavioral, socio-emotional, or academic concerns, which interfere with learning.

Conjoint Behavioral Consultation Conjoint behavioral consultation (CBC; Sheridan,

Kratochwill, & Bergan, 1996; Sheridan & Kratochwill, 2008) is an extension of traditional behavioral consultation services.

CBC is, by definition, an indirect model of service delivery intended to enhance the skills of natural treatment agents (i.e., teachers, parents) who assume responsibility for implementing intervention plans.

Goals of CBC focus on addressing the specific needs of the child, while working collaboratively with the child’s teacher and parent to strengthen family-school partnerships.- Including enhancing the skills of natural treatment agents (i.e.,

consultees).

Implementation Integrity

Intervention implementation integrity refers to the extent to which an intervention plan is implemented as intended.

An important consideration in CBC is the ability of individuals to deliver treatment plans accurately and efficiently.

Thus, the issue of implementation integrity is central to the utility of CBC in addressing students’ behavioral needs.

Implementation Integrity

Little systematic attention is afforded to the specific strategies that consultants can use to ensure accurate and reliable implementation of treatment plan elements in natural settings (Sanetti & Kratochwill, 2008).

Strategies to support families in implementing interventions within a problem-solving framework have not received adequate attention.

Socioeconomic Conditions

A significant correlate with treatment dropout and poor outcomes following parent training interventions is socioeconomic hardship (Henggeler et al., 1992; Kazdin, 1990; McMahon et al., 1981).

Families living in low socioeconomic conditions face numerous stressors (e.g., poverty, lack of social support) that may interfere with their ability to implement parenting interventions, thereby compromising treatment effects.

Purpose

1) Describe the degree to which families implement interventions as designed in a consultation study to promote positive behavioral outcomes for children experiencing behavioral difficulties.

2) Describe and illustrate multiple methods of assessing integrity.

3) Discuss strategies used to maximize intervention implementation integrity of treatment plans by all families to increase positive outcomes.

4) Present methods used by consultants to support families with diverse challenges.

Methods - Participants

Child Participants: 62 children 75% Male 67% White, non-Hispanic Average age = 6.72 (range: 5 – 9)

All students were referred to the study by their teachers who indicated a need for further intervention due to behavioral concerns.

Methods - Participants

Parent Participants: 62 Parents 83% White, non-Hispanic Average age = 35 88% Female 78% received some college or post-secondary education 26 parents met the criteria to be included in a demographically

diverse sub-sample

Data used in this investigation represent part of a larger, four-year study evaluating the efficacy of CBC as an intervention that addresses concerns of kindergarten through third grade students whose disruptive behaviors place them at risk for academic failure.

Methods - Setting

Cases were conducted in 17 kindergarten, first, second, and third grade classrooms in a moderately-sized Midwestern city and surrounding area, including public and parochial schools.

Interventions were implemented across home and school settings for students who were the focus of consultation.

Methods - Selection

Students were selected via teacher ratings:o Systematic Screening for Behavior

Disorders (SSBD) multiple-gate screening procedure (Walker et al., 1990).

o Screening instrument with ratings for severity of externalizing behaviors; frequency of externalizing behaviors; and degree of need for additional intervention.

Methods – Data Procedures

Cases were separated into two groups: General mainstream group (N=36) Demographically diverse group (N=26)

Relative to mainstream group, demographically diverse was defined as low-income status, racially diverse, linguistically diverse, living in a single parent home, or less than high school diploma.

Consultation Procedures

Consultants (i.e., trained graduate students) met with the teacher and two to three parents for approximately four to five conjoint consultation sessions over approximately eight weeks via three phases.1) Needs Identification and Needs Analysis2) Intervention Development and Implementation3) Intervention Evaluation

Behavioral interventions consisted of 3 components.1) Communication component2) Motivation component3) Functional component

Implementation Support

Strategies to promote high integrity in the home setting were developed as part of the larger study

Sources:1) Literature review that informed consultant training.

2) Narrative information reported by trained consultants

See Handout 1

Implementation Support

General Strategies:1) Follow the partnership model during intervention

development phase (Sheridan & Kratochwill, 2008).

2) Provide a rationale for collecting integrity data.

3) Script and package intervention implementation integrity forms (Watson, 2004).

4) Train and educate the family on intervention delivery with various techniques.

See handout 1

Implementation Support

Additional strategies for families requiring more support:

1) Consider culturally-sensitive procedures (Sheridan & Kratochwill, 2008).

2) Focus on strategies that build trust between families and educators. 3) Enhance communication between consultant and families.4) Implement family-centered approach throughout consultant

process. 5) Gather information about the home setting and family system. 6) Facilitate regular contacts and provide collective support. 7) Adjust integrity data collection as to improve ease of collection.

See handout 2

Case Study

6 year old female in 1st grade was the focus of consultation.

Biological parents and 4 siblings moved to the United States from Sudan 7 years prior as refugees.

The family currently meets the 2008 HHS criteria for a family living in poverty.

Case Study - Preconsultation

Conducted an initial meeting to introduce consultation.

Validated parent concerns (e.g., acculturating to the United States, feedback from classroom teachers).

Met with a cultural liaison about roles, structure, and expectations.

Conducted a home visit to discuss roles and expectations.

Case Study – Needs Identification and Analysis

Strengths: Strong verbal skills Independent

Concerns: Reading skills Ability to stay on-task; disruptive behaviors

Functional assessment revealed escape motivated behavior, and grade level academic skills.

Case Study – Needs Identificationand Analysis

Intervention development:

School: “chunking” reading work, appropriate escape, self-monitoring.

Home: structured homework routine that included entire family, home-school note, activity checklist with visual cues. Family checklist to increase integrity. Additional support (e.g., meeting reminders,

frequent communication).

Measurement of Treatment Integrity

Parent Self-Report Form

1) Completed daily by parents and used to assess intervention implementation integrity at home.

2) Listed all the steps of the behavioral intervention plan clearly and objectively.

3) Example includes intervention plan steps that the parent in this case completed

See handout 3

Measurement of Treatment Integrity Permanent Product Report Form

1) Completed by parents daily for the duration of intervention (i.e., at least 4 weeks of intervention).

2) Charts were collected from parents on which evidence (e.g., stickers, notes, marks, checks) demonstrating that s/he implemented steps of the intervention.

3) Trained research assistants and consultants reviewed permanent products and completed the permanent product report forms to reflect parents’ delivery of plan components as reported on permanent products.

See handouts 4 and 5

Activity Checklist

Case Study – Strategies to Maximize Integrity

Integrity information was scripted and standardized

Intervention steps were simple and consistent Opportunity to have forms translated to Arabic;

use of visual cues. Home visits for modeling, observation,

performance feedback, and discussion.

The family reported adhering to 100% of intervention steps over 4 weeks of implementation.

Case Study – Intervention Evaluation

School Increase from an average of 45% accuracy prior

to the intervention to 80% accuracy after implementation.

Home Decrease in the number of problem behaviors

exhibited from an average of 4 to an average of 1.

Parents indicated improved behavior management strategies, communication with the school, and praising children for attaining goals.

Assessment of Implementation

Table 3Summary Statistics of Intervention Implementation Integrity across Sample

Parent (self)a report

Permanent Products

Mainstream Group

Percent .77 .79

Standard Deviation .26 .24

Sample Size 26 28

Diverse Group

Percent .81 .91

Standard Deviation .17 .18

Sample Size 19 21a Participants completed self-report report measure, permanent product measure, or both measures

Discussion

High levels of intervention implementation integrity were noted in the home setting across multiple sources (i.e., families via parent-report and coders of permanent product data) Families seem to be responsive to tactics used by

consultants to maximize treatment integrity.

Families also appear to adhere to the steps of behavioral interventions regardless of the integrity measure used, with families reporting slightly higher integrity when recording integrity on a permanent product.

Discussion

Permanent products are a feasible and useful way of measuring treatment integrity because families naturally use the products as they implement interventions (Sanetti & Kratochwill, 2008).

High integrity levels reported on permanent products may suggest that parents are not over-reporting on self-report measures.

Families “at-risk” reported implementing intervention with higher integrity than the mainstream group of families.

Implications

To maximize outcomes, educators may find it helpful to use a collaborative consultation model to design and implement behavioral interventions with families.

In CBC, training, supporting, and partnering with teachers and families provides a consistent and continuous approach to service delivery.

Future Research Directions

Examine integrity and the variables that may influence levels of treatment integrity experimentally. Specifically, future research could attempt to measure the

impact of disadvantaged social settings on the degree to which families follow a behavioral intervention protocol.

Explore resources that are needed and specific procedures that are critical to intervention implementation. The cost-effectiveness of such practices needs to be assessed in future studies.

Future Research Directions

Measure and evaluate treatment integrity in other ways. For example, the dosage, quality of, and

responsiveness to implementation may be important in understanding the quality of services students are receiving (Dane & Schneider, 1998).

Standardized measures of consultant

strategies for maximizing integrity should be developed and used in future research.

Future Research Directions

Psychometric properties of future measures of integrity and strategies used by consultants to improve integrity should be examined (e.g., validity, reliability).

Similar issues with regard to measurement and evaluation should be explored at school.

The link between home and school treatment integrity and its effects on student performance requires examination.

Contact Information

Susan M. Sheridan: ssheridan2@unl.edu Andy Garbacz: agarbacz2@unl.edu Amanda Witte: awitte2@unl.edu

http://cyfs.unl.edu

References

Dane, A. V. & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23-45.

Henggeler, S. W., Melton, G. B., & Smith, L. A. (1992). Family preservation using Multisystemic Therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting and Clinical Psychology, 60, 953-961.

Kazdin, A. E. (1990). Premature termination from treatment among children referred for antisocial behavior. Journal of Child Psychology and Psychiatry, 31, 415-425.

McMahon, R. J., Forehand, R., Griest, D. L., & Wells, K. C. (1981). Who drops out of treatment during parent behavioral training? Behavioral Counseling Quarterly, 1, 79-85.

Sanetti, L. M. H., & Kratochwill, T. R. (2008). Treatment integrity in behavioral consultation: Measurement, promotion, and outcomes. International Journal of Behavioral Consultation and Therapy, 4, 95-113.

References

Sheridan, S. M., & Kratochwill, T. R. (2008). Conjoint behavioral consultation: Promoting family-school connections and interventions (2nd ed.). New York, NY: Springer Publishing.

Sheridan, S. M., Kratochwill, T. R., & Bergan, J. R. (1996). Conjoint behavioral consultation: A procedural manual. New York: Plenum.

Walker, H.M. & Severson, H. H., Todis, B.J., Pedego, A.E. (1990). Systematic screening for behavior disorders (SSBD): Further validation, replication, and normative data. RASE: Remedial & Special Education, 11, 32-46.

Watson, T. S. (2004). Treatment integrity. In T. S. Watson & C. H. Skinner (Eds.), Encyclopedia of school psychology (pp. 356-358). New York, NY: Springer Publishing.