Individual differences in response to intervention: An application of Integrative Data Analysis in...
-
Upload
oswald-berry -
Category
Documents
-
view
220 -
download
0
Transcript of Individual differences in response to intervention: An application of Integrative Data Analysis in...
![Page 1: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/1.jpg)
Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS
Sara A. HartFlorida State University
[email protected]@saraannhart
& Chris Schatschneider, Carol Connor & Stephanie Al Otaiba
![Page 2: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/2.jpg)
Expanding our search for moderators of intervention
• A little about me– Behavioral genetics background– Interested to move these findings to schools
• Even with modest effect sizes, individual differences in intervention response
• Bioecological model (Bronfenbrenner & Ceci, 1994)
– Provides framework for differentiating students based on non-intervention related traits• Lets follow individualized medicine
![Page 3: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/3.jpg)
Integrative Data Analysis (IDA)
• Item-level pooled data (Curran & Hussong, 2009)
• Capitalizes on cumulative knowledge – Longer developmental time span– Increased statistical power– Increased absolute numbers in tails
• Controls for heterogeneity – Sampling, age/grade, cohort, geographical, design,
measurement
![Page 4: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/4.jpg)
Project KIDS
• Expanded definition of moderators of response to intervention– Cognitive, psychosocial, environmental, familial/genetic
risk• IDA across 9 completed intervention projects – Approximately 5600 kids
• Data entry of item level data common across at least 2 projects– ~30 different assessments
• Questionnaire data collection
![Page 5: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/5.jpg)
Proof of Concept
• Behavior problems and achievement are associated
• More behavior problems are typically seen in LD populations
• Is adequate vs inadequate response status differentiated by behavior problems?
![Page 6: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/6.jpg)
Method
• Participants– 2007-2008 ISI intervention through FL LDRC (Al Otaiba et al., 2011)
• RCT: 23 treatment, 21 contrast teachers• 556 kindergarteners • A2i recommendations vs enhanced standard practice
– 2009-2010 RTI Intervention through FL LDRC (Al Otaiba et al., 2014)
• RCT: 34 classrooms, kids randomized • 522 1st graders• regular RTI vs dynamic RTI
– 2005-2006 ISI intervention project (Connor et al., 2007)
• RCTish : 22 treatment, 25 contrast teachers, 3 pilot• 821 first graders• A2i recommendations vs standard practice
![Page 7: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/7.jpg)
Method• Measures– WJ Tests of Achievement Letter-Word
Identification (LWID)• Pre- and post-intervention testing periods
– Social Skills Rating Scale: Behavioral Problems subscale • Teacher completed during intervention year
07/08 K ISI LDRCMean (SD)
09/10 1st RTI LDRCMean (SD)
05/06 1 ISIMean (SD)
WJ LWID Fall 12.03 (5.50) 26.66 (9.03) 24.28 (7.97)
WJ LWID Spring 21.75 (7.09) 38.73 (8.07) 36.54 (7.39)SSRS .50 (.44) .34 (.39) .53 (.44)
![Page 8: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/8.jpg)
r=.89 r=-.23
r=-.26
![Page 9: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/9.jpg)
Results: Calibration LWID IDA
• Randomly selected 1 time point/child/project to form “calibration sample” for LWID
• Decision to include only items > 5% endorsement rate
• Reduced item sample from 75 38 – Items 11 to 49
![Page 10: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/10.jpg)
Results: Calibration LWID IDA
Hahahaahaaa.
![Page 11: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/11.jpg)
Results: Calibration LWID IDA
• Non-linear factor analysis– Multiple group analysis to arrive at single factor
for LWID– Using measurement (in)variance principles across
the 3 projects, using modindices in Mplus to adjust for project variant differences • Key decision: We wanted to constrain based on
meaningful differences – Chi-square difference value set to equal Cohen’s d = .20
![Page 12: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/12.jpg)
![Page 13: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/13.jpg)
Metric (in)variance-localized misfit for items 32, 35, 36, 37, 39, & 46-49
![Page 14: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/14.jpg)
Scaler (in)variance-localized misfit for thresholds of items 46, 47 & 49
![Page 15: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/15.jpg)
Residual variance (in)variance-all could be constrained to equal
![Page 16: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/16.jpg)
Factor variance (in)variance-All could be constrained to be equal
![Page 17: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/17.jpg)
Final Model-using all data, set parameters based on final factor model, exported factor scores
![Page 18: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/18.jpg)
Results: SSRS
![Page 19: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/19.jpg)
Results
r = .97 r = .98
![Page 20: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/20.jpg)
Results
r = .96
![Page 21: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/21.jpg)
Now I have equivalent data!!!
• It’s a lot of steps to get a regular old data set• Now I can answer content questions, but with
more kids in a more generalizable sample
• So, are behavior problems bad for treatment response?
![Page 22: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/22.jpg)
Results: Response
• Proc mixed: covariance adjusted LWID score– 1712 children
![Page 23: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/23.jpg)
Results: Response
• 818 treatment children
![Page 24: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/24.jpg)
Results: Response
• 818 treatment children
UnresponsiveCutoff < 20%
N=164!
![Page 25: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/25.jpg)
Results
• Logistic regression– SSRS behavior problems significant predictor of
response status (OR = 1.58, CI = 1.30-1.90)• average behavior problems = 41% probability of being
“unresponsive”• greater than average behavior problems(+ 1SD) = 50%
probability of being “unresponsive”• Less than average behavior problems (-1SD) = 34%
probability of being “unresponsive”
![Page 26: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/26.jpg)
Conclusions
• Response status is differentiated by behavior problems– Mo’ behavior problems, mo’ (reading) problems!
![Page 27: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/27.jpg)
Overall IDA conclusions
• IDA is a “cheap” way to get more power, more n at tails, and show more generalizable effects
• Given how similar many of our projects are, consider doing item-level data entry – Easy potential to combine data– Can you do factor analysis and IRT? You can do IDA*!
• These data are more useful together than apart– IRT within and between samples?– Treatment effectiveness across samples?– Characteristics of lowest responders?
![Page 28: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/28.jpg)
Project KIDS goals
• Intervention response status is known• Can we predict responders/non-responders with questionnaire
data?– Family history
• 1st degree vs 2nd degree, dosage
– Cognitive correlates• Executive functioning, ADHD
– Behavior• Comorbid behavior issues
– Environments• Home literacy environment vs neighborhood vs school
• Individualize instruction based on child traits– No cheek swab needed
![Page 29: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/29.jpg)
Acknowledgements • Stephanie Al Otaiba • Carol Connor• Chris Schatschneider• Great staff & grad students, and many wonderful data
enterers
NICHD grant HD072286
![Page 30: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/30.jpg)
r = .95
![Page 31: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/31.jpg)
r = .78
r = .88
![Page 32: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/32.jpg)
What about without DIF?r = .97
r = .99
![Page 33: Individual differences in response to intervention: An application of Integrative Data Analysis in Project KIDS Sara A. Hart Florida State University shart@fcrr.org.](https://reader035.fdocuments.net/reader035/viewer/2022062805/5697bfc71a28abf838ca7935/html5/thumbnails/33.jpg)