Explanatory Secondary Dimension Modeling of Latent Different Item

FunctioningPaul De Boeck, Sun-Joo Cho, and

Mark Wilson

DIF

• Conditional on person latent trait, different manifest groups have different probabilities to answer correctly/endorse an item. (manifest DIF)

• DIF as the consequence of neglecting the “nuisance dimensions”. (secondary dimensions)

This study

• Latent DIF: DIF between latent classes, i.e., a non-DIF vs a DIF latent class

group membership not observed or unobservableThe observed membership not reliable or not valid

• Mixed dimensionality: the secondary dimension in the DIF latent class, but not in the non-DIF latent class

Multidimensional model for DIF (MMD) (Roussos & Stout, 1996)

One dimensionality approach: Multiple-indicator multiple-cause (MIMIC) approach

• MMD: a secondary dimension in both manifest groups

• MIMIC: neither of the classes has a secondary dimension but a direct effect of the group variable on the primary dimension (exceptional: Glockner-Rist & Hoijtink, 2003)

• Mixed dimensionality: a secondary dimension but only in the DIF latent class (but may have some problems)

Measurement Invariance Notions

• Scalar variance: difference in item difficulties (intercept in factor model) and implies uniform DIF

• Metric variance: difference in item discriminations (loadings in factor model) and implies nonuniform DIF

• Relate the idea of mixed dimensionality to measurement invariance: With secondary dimension, scalar invariance (equivalence of difficulties) are regain (M2a vs M1a).

Outline of the Study

• Aim: to show the implication and application of a mixed dimensionality model into the latent DIF

• (1) Models specification• (2) Model Estimation and Evaluation• (3) Model application (Speededness &

Arithmetic operation)

Models

• Non-DIF One-Dimensional Mixture Models (1d-non-DIF; Model 0)

• αi and i are the same in both groups;• , (model identification)

• 2I + 3 para (I difficulties, I discriminations, a mean and variance of DIF latent class, and a mixing probability

ijgi βθlogit (Mixture 2PL model)

12 DIFnon 0 DIFnon

• One dimensional difficulty DIF mixture model (1d-dif; Model 1a)

• i are different between groups• Lack of scalar invariance, uniform DIF• 2I+D+3 para (D items showing latent DIF)

ifjfi βθlogit

• One dimensional difficulty and discrimination DIF mixture model (1d-dif; Model 1b)

• αi and i are different between groups• Lack of scalar and metric invariance, uniform and nonuniform DIF • 2(I+D)+3 para (D items showing latent DIF)

ifjfif βθlogit

• Mixture of one-dimensional and two-dimensional model(1&2d-DIF, Model 2a)

• For DIF latent class, (f1,f2)’ ~ BVN(,).

( )• 2I+D+4 free paras• One discrimination for a DIF items in dimension 2

of DIF latent class is fixed to 1.00. (over constrain)

ijfijfi βθθlogit 2211

122 f

• Features of Model 2a: • (1) no difference in difficulty between two latent

traits• (2) metric invariance of primary dimension & scalar

invariance• Implication of Model 2a:• (1) Respondents from the DIF latent class differ with

respect to how much DIF they show (pf2);

• All DIF is assumed to rely on the secondary dimension

• Mixture of one-dimensional and two-dimensional model with discrimination DIF (1&2d-dif, Model 2b)

• 2(I+D)+3 free parameters• Scalar invariance• Cov f12 is fixed to zero (over constrained)

ijfijfif βθθlogit 2211

• Two-dimensional mixture models(2d-DIF and 2d-DIF/ Models 3a and 3b): introduce the second dimension to the non-DIF latent class

• Model 3a: Measurement-invariance• Model 3b: scalar invariance, configural

invariance

Relationships between dimensional latent DIF mmodels

Aim of Applications

• Preliminary question: whether any DIF occurs at the level of latent classes (M0 vs M1a & 1b)

• Whether a secondary dimension in the DIF latent class can explain the DIF? (M1a vs M2a; or M1b vs M2b)

• Is the DIF of a kind that the discrimination on the primary dimension is affected? (M1a vs M1b or M2a vs M2b)

• Is the secondary dimension limited to the DIF latent class or does it also apply to the non-DIF latent class? (M2a vs M3a, or M2b vs M3b)

Model Estimation & Evaluation

• LatentGOLD 4.5 Syntax Cluster module (Vermunt & Magidson, 2007)

• Two major problems in mixture model: (1) Label switch; (2) multiple local maxima

• Model specification in LatentGOLD

• LRT (for nested models), AIC, BIC

Results of Speededness Data

First 23 items were assumed no speededness while last 8 items has (item location).

Speeded vs no speeded latent class

• M1a and 1b fit much better than M0, suggesting that DIF exist at the level of latent class;

• First horizontal comparison (M2a> M1a; M2b>M1b) suggests the mixed dimensionality approach is favored.

• Vertical comparison (M1b>M1a;M2b>M2a) suggests discrimination DIF is related to the primary dimension. Thus, M2b is chose.

• Second comparison (M3b is not significantly better than M2b) suggests secondary dimension not need to extend to the non-DIF latent class.

• =0.311, speeded class is a minority class;• Nonzero discriminations on the second dimension

for the DIF items in the speeded class; the discrimination of items on the primary dimension differ between two latent classes.

• For DIF class, mean=(-0.634,-0.987)’,var-cov=(0.787, 0; 0, 1), suggesting that the speeded class has a lower ability and being speeded at the end of the test seems not to help.

• Validation check for the DIF items

• Confirmatory and exploratory two-dimensional models without latent classes were not supported by AIC and BIC.

Results of Arithmetic Operations Data

• Class 1 (Poorer on the division items) vs class 2 (proficient)

• DIF exist between two latent classes;• Secondary dimension helps;• The primary dimension discrimination DIF is

not strongly supported. Hence, M2a is favored.

• No needed to extend the second dimension to the non-DIF latent class.

• For DIF class, mean=(-1.150,-2.945)’,var-cov=(1.034, 0.353*sqrt(1.034); 0.353*sqrt(1.034), 1).

• =0.292• AIC supported exploratory two-dimensional

model without latent class but BIC did not.

Conclusions

• Mixed dimensionality approach are supported by the generally better goodness of fit.

• It was sufficient to include the secondary dimension in only one of the latent class.

• Mixed dimensionality approach has some merits (for latent DIF, explaining DIF…)

• Mixed dimensionality approach should primarily be used for confirmatory and explanatory purposes, but not for exploratory or detection purpose. (problematic)

Discussions & Future Studies

• The mixed dimensionality approach have some problems (not usual; local dependence).

• To define item properties which are not available as the clue of the suspected DIF before the secondary dimension approach can be implemented.

• Applied mixed dimensionality approach to manifest groups, to study cognitive development, mutiple strategies…

• Applied the mixture approach within the cognitive development theoretical framework

• Whether individual differences in DIF is proper?