Differential Item Functioning by Ethnicity in the Inhibitory Control Subscale of the Children’s...

1
Differential Item Functioning by Ethnicity in the Inhibitory Control Subscale of the Children’s Behavior Questionnaire (CBQ) Kristin L. Moilanen, Chad B. Henry, Daniel S. Shaw, Thomas J. Dishion, Frances Gardner, & Melvin N. Wilson Acknowledgements This research was supported by grant 5 R01 DA16110 from the National Institutes of Health to the third and fourth authors. We gratefully thank the Early Steps and the Child and Family Center staff for their assistance, and all the families who participated in this project. Abstract In this study, the 13-item inhibitory control subscale of the Children’s Behavior Questionnaire (CBQ) was examined for evidence of differential item functioning (DIF) by ethnicity. As part of a larger study, 720 primary caregivers completed this subscale when their children were 4 years old. Nested ordinal regression analyses indicated that 7 items functioned equivalently across ethnic groups. Four of these items favored White children, and two favored children of Black and Hispanic/Other ethnicities. Further attention to this subscale’s item properties is necessary in order to advance research on the development of inhibitory control in diverse populations of children. Study Purpose The purpose of the current study was to explore ethnicity as a source of differential item functioning (DIF) in the inhibitory control subscale of the Children’s Behavior Questionnaire (CBQ; Rothbart, Ahadi, Hershey, & Fisher, 2001). This questionnaire was initially developed using samples of European American children, and has since been empirically validated in many nations around the world. Yet, it is problematic that potential item-level differences between ethnic groups have not yet been fully examined, because different ethnic groups may define children’s self-regulation in unique ways which may not be accurately represented in the current scale (i.e., presumed ethnic differences could actually be measurement bias). Methods •720 families enrolled in the Early Steps Multisite Project, a multi-site prevention program for young children at increased risk for clinically-elevated levels of conduct problems. •Child demographics: •Sex: 50% male •Race/ethnicity: 46% White, 26% Black, 28% Hispanic/Other •Site: 25% Charlottesville, VA, 38% Eugene, OR, & 37% Pittsburgh, PA •Primary caregivers (97% mothers) completed the inhibitory control subscale of the CBQ about their child at the age 4 assessment. •Inhibitory control : 7-point response scale, ranging from 1 (extremely untrue of child) to 7 (extremely true of child). Six items demonstrated DIF by ethnicity. Four of these items favored White children, and two favored both Black and Hispanic/Other children. Three of the four items that favored White children over Black children concerned compliance with instructions. •3. Has a hard time following instructions. (REV) •9. Is good at following instructions. •10. Approaches places s/he has been told are dangerous slowly and cautiously. Mothers of Black children may have higher expectations for their preschoolers’ compliance than the mothers of White children. White children were also favored over Hispanic/Other children on item 13 (“Is usually able to resist temptation when told s/he is not supposed to do something”). This is the only item that includes the modifier “usually.” White and Hispanic/Other mothers may have different standards for what it means for their children to “usually” resist temptation. Black and Hispanic/Other children were favored over EA children on two items that did not share a common theme. •1. Can lower his/her voice when asked to do so. •2. Is good at games like “Simon Says,” “Mother, May I?” and “Red Light, Green Light.” Parents of White children may have higher expectations for children’s speaking volume. Although item 2 makes theoretical sense, it is possible that mothers respond about their child’s overall game-playing skills, not their ability to play these particular games thought to involve inhibitory control. Black children were also favored over Hispanic/Other children on item 10 (see text above). The features and accessibility of “dangerous places” may vary for urban African American mothers and suburban/rural Hispanic/Other mothers. Conclusions & Future Directions •Nearly half of the items in this subscale do not function equivalently across ethnic groups. •No one group was consistently favored over the others by the items that showed DIF. •DIF may be reduced by sharpening item wording and Table 1 Descriptive Statistics White Black Hispanic/ Other Cohen’s d W/B Cohen’s d W/HO Cohen’s d B/HO N 291 166 171 Cronbach’s α .79 .64 .74 Scale M (SD) 4.43 (.82) 4.36 (.77) 4.55 (.82) .09 -.15 -.24 Item M(SD) 1 Can lower voice 5.18 (1.48) O 5.38 (1.58) 5.53 (1.35) W -.13 -.25 -.10 2 Is good at games 5.22 (1.40) O 5.40 (1.50) 5.65 (1.36) W -.12 -.31 -.17 3 Hard time following instructions 4.10 (1.54) 3.77 (1.66) 4.12 (1.68) .21 -.01 -.21 4 Prepares for trips 4.54 (1.70) 4.63 (1.62) 4.86 (1.57) -.05 -.20 -.14 5 Wait before starting activities 4.82 (1.42) O 4.73 (1.64) O 5.11 (1.41) W, B .06 -.20 -.25 6 Difficulty waiting in line 3.71 (1.61) 3.62 (1.77) 3.75 (1.73) .05 -.02 -.07 7 Trouble sitting still 3.43 (1.69) 3.19 (1.69) 3.40 (1.77) .14 .02 -.12 8 Able to resist laughing 3.90 (1.53) 3.97 (1.78) 3.86 (1.81) -.04 .02 .06 9 Good at following instructions 5.17 (1.26) B 4.83 (1.62) W, O 5.19 (1.33) B .23 -.01 -.24 10 Approaches slowly 4.61 (1.91) B 4.16 (1.99) W, O 4.74 (1.93) B .23 -.06 -.29 11 Not very careful 4.32 (1.93) 4.49 (2.02) 4.22 (2.01) -.08 .05 .13 12 Can easily stop 4.35 (1.50) 4.39 (1.65) 4.52 (1.68) -.03 -.10 -.08 13 Resist temptation 4.39 (1.47) 4.25 (1.63) 4.29 (1.65) .09 .06 -.02 Note. Bold values indicate significant mean differences. DIF Analyses DIF was detected by comparing nested ordinal regression models (Zumbo, 1999). Seven items did not show DIF by ethnicity. These items are relatively unambiguous and correspond to the “core elements” of inhibitory control during preschool years. •4. Prepares for trips and outings by planning things s/he will need. •5. Can wait before entering new activities if s/he is asked to do so. •6. Has difficulty waiting in line for something. (REV) •7. Has trouble sitting still when s/he is told to (at movies, church, etc.). (REV) •8. Is able to resist laughing or smiling when it isn’t appropriate. •11. Is not very careful and cautious in crossing streets. (REV) •12. Can easily stop an activity when s/he is told “no.” -3 -2 -1 0 1 2 3 -0.1 0.1 0.3 0.5 0.7 0.9 Item 7 - Does not show DIF W1 W7 B1 B7 H1 H7 Standard Deviations Above/Below the Mean Total Inhibitory Control Score Response Probability -3 -2 -1 0 1 2 3 -0.1 0.1 0.3 0.5 0.7 0.9 Item 10 - Shows DIF W1 W4 W7 B1 B4 B7 Standard Deviations Above/Below the Mean Total Inhibitory Control Score Response Probability

Transcript of Differential Item Functioning by Ethnicity in the Inhibitory Control Subscale of the Children’s...

Page 1: Differential Item Functioning by Ethnicity in the Inhibitory Control Subscale of the Children’s Behavior Questionnaire (CBQ) Kristin L. Moilanen, Chad.

Differential Item Functioning by Ethnicity in the Inhibitory Control Subscale of the Children’s Behavior Questionnaire (CBQ)Kristin L. Moilanen, Chad B. Henry, Daniel S. Shaw, Thomas J. Dishion, Frances Gardner, & Melvin N. Wilson

AcknowledgementsThis research was supported by grant 5 R01 DA16110 from the National Institutes of Health to the third and fourth authors. We gratefully thank the Early Steps and the Child and Family Center staff for their assistance, and all the families who participated in this project.

AbstractIn this study, the 13-item inhibitory control subscale of the Children’s Behavior Questionnaire (CBQ) was examined for evidence of differential item functioning (DIF) by ethnicity. As part of a larger study, 720 primary caregivers completed this subscale when their children were 4 years old. Nested ordinal regression analyses indicated that 7 items functioned equivalently across ethnic groups. Four of these items favored White children, and two favored children of Black and Hispanic/Other ethnicities. Further attention to this subscale’s item properties is necessary in order to advance research on the development of inhibitory control in diverse populations of children.

Study PurposeThe purpose of the current study was to explore ethnicity as a source of differential item functioning (DIF) in the inhibitory control subscale of the Children’s Behavior Questionnaire (CBQ; Rothbart, Ahadi, Hershey, & Fisher, 2001). This questionnaire was initially developed using samples of European American children, and has since been empirically validated in many nations around the world. Yet, it is problematic that potential item-level differences between ethnic groups have not yet been fully examined, because different ethnic groups may define children’s self-regulation in unique ways which may not be accurately represented in the current scale (i.e., presumed ethnic differences could actually be measurement bias).

Methods•720 families enrolled in the Early Steps Multisite Project, a multi-site prevention program for young children at increased risk for clinically-elevated levels of conduct problems.•Child demographics:

•Sex: 50% male•Race/ethnicity: 46% White, 26% Black, 28% Hispanic/Other•Site: 25% Charlottesville, VA, 38% Eugene, OR, & 37% Pittsburgh, PA

•Primary caregivers (97% mothers) completed the inhibitory control subscale of the CBQ about their child at the age 4 assessment. •Inhibitory control : 7-point response scale, ranging from 1 (extremely untrue of child) to 7 (extremely true of child). Mothers could also indicate whether any items were not applicable to their child. Scale scores were computed by averaging all numeric responses.

Results & Discussion•Group differences in scale total and item means are presented in Table 1. •There were no ethnic differences in overall inhibitory control, but there were mean level differences by ethnic group for 5 of the 13 scale items. •Effect sizes were not constant across items, which is a rough indicator that DIF is present (Smith, 2002).

Six items demonstrated DIF by ethnicity. Four of these items favored White children, and two favored both Black and Hispanic/Other children.

Three of the four items that favored White children over Black children concerned compliance with instructions.

•3. Has a hard time following instructions. (REV)•9. Is good at following instructions.•10. Approaches places s/he has been told are dangerous slowly and

cautiously.Mothers of Black children may have higher expectations for their preschoolers’ compliance than the mothers of White children.

White children were also favored over Hispanic/Other children on item 13 (“Is usually able to resist temptation when told s/he is not supposed to do something”). This is the only item that includes the modifier “usually.” White and Hispanic/Other mothers may have different standards for what it means for their children to “usually” resist temptation.

Black and Hispanic/Other children were favored over EA children on two items that did not share a common theme.

•1. Can lower his/her voice when asked to do so.•2. Is good at games like “Simon Says,” “Mother, May I?” and “Red Light,

Green Light.”Parents of White children may have higher expectations for children’s speaking volume. Although item 2 makes theoretical sense, it is possible that mothers respond about their child’s overall game-playing skills, not their ability to play these particular games thought to involve inhibitory control.

Black children were also favored over Hispanic/Other children on item 10 (see text above). The features and accessibility of “dangerous places” may vary for urban African American mothers and suburban/rural Hispanic/Other mothers.

Conclusions & Future Directions•Nearly half of the items in this subscale do not function equivalently across ethnic groups.•No one group was consistently favored over the others by the items that showed DIF.•DIF may be reduced by sharpening item wording and using more concrete scale anchors. •As at least three items reference compliance to instructions, it might be beneficial to eliminate at least one of these items so compliance isn’t over-represented in the scale. •More work is needed to determine if parents of different ethnicities have divergent expectations for children’s “controlled” behavior and emotions, and different perceptions of what “control” looks like.

Table 1Descriptive Statistics

White Black Hispanic/Other Cohen’s dW/B

Cohen’s d W/HO

Cohen’s d B/HO

N 291 166 171

Cronbach’s α .79 .64 .74

Scale M (SD) 4.43 (.82) 4.36 (.77) 4.55 (.82) .09 -.15 -.24

Item M(SD)

1 Can lower voice 5.18 (1.48) O 5.38 (1.58) 5.53 (1.35) W -.13 -.25 -.10

2 Is good at games 5.22 (1.40) O 5.40 (1.50) 5.65 (1.36) W -.12 -.31 -.17

3 Hard time following instructions 4.10 (1.54) 3.77 (1.66) 4.12 (1.68) .21 -.01 -.21

4 Prepares for trips 4.54 (1.70) 4.63 (1.62) 4.86 (1.57) -.05 -.20 -.14

5 Wait before starting activities 4.82 (1.42) O 4.73 (1.64) O 5.11 (1.41) W, B .06 -.20 -.25

6 Difficulty waiting in line 3.71 (1.61) 3.62 (1.77) 3.75 (1.73) .05 -.02 -.07

7 Trouble sitting still 3.43 (1.69) 3.19 (1.69) 3.40 (1.77) .14 .02 -.12

8 Able to resist laughing 3.90 (1.53) 3.97 (1.78) 3.86 (1.81) -.04 .02 .06

9 Good at following instructions 5.17 (1.26) B 4.83 (1.62) W, O 5.19 (1.33) B .23 -.01 -.24

10 Approaches slowly 4.61 (1.91) B 4.16 (1.99) W, O 4.74 (1.93) B .23 -.06 -.29

11 Not very careful 4.32 (1.93) 4.49 (2.02) 4.22 (2.01) -.08 .05 .13

12 Can easily stop 4.35 (1.50) 4.39 (1.65) 4.52 (1.68) -.03 -.10 -.08

13 Resist temptation 4.39 (1.47) 4.25 (1.63) 4.29 (1.65) .09 .06 -.02

Note. Bold values indicate significant mean differences.

DIF AnalysesDIF was detected by comparing nested ordinal regression models (Zumbo, 1999).Seven items did not show DIF by ethnicity. These items are relatively unambiguous and correspond to the “core elements” of inhibitory control during preschool years.•4. Prepares for trips and outings by planning things s/he will need.•5. Can wait before entering new activities if s/he is asked to do so.•6. Has difficulty waiting in line for something. (REV)•7. Has trouble sitting still when s/he is told to (at movies, church, etc.). (REV)•8. Is able to resist laughing or smiling when it isn’t appropriate.•11. Is not very careful and cautious in crossing streets. (REV)•12. Can easily stop an activity when s/he is told “no.”

-3 -2 -1 0 1 2 30

0.10.20.30.40.50.60.70.80.9

Item 7 - Does not show DIF

W1W7B1B7H1H7

Standard Deviations Above/Below the Mean Total Inhibitory Control Score

Resp

onse

Pro

babi

lity

-3 -2 -1 0 1 2 30

0.10.20.30.40.50.60.70.80.9

Item 10 - Shows DIF

W1W4W7B1B4B7

Standard Deviations Above/Below the Mean Total Inhibitory Control Score

Resp

onse

Pro

babi

lity