Measurement Bias Detection Through Factor Analysis

Barendse, M. T., Oort, F. J. Werner, C. S., Ligtvoet, R., Schermelleh-Engel, K.

Defining measurement bias

• Violation of measurement invariance

Where V is violator• If V is grouping variable, then MGFA is suitableIntercepts – uniform biasFactor loadings – non-uniform bias (vary with

t)

Restricted Factor Analysis (RFA)

• Advantages of RFA over MGFA:V can be continuous or discrete, observed or

latentInvestigate measurement bias with multiple Vs.More precise parameter estimates and larger

power• Disadvantage of RFA:Not suited for nonuniform bias (interaction term)

Approaches for non-uniform bias

• RFA with latent moderated structural equations (LMS)

---- Simulation (categorical V) showed at least as good as MGFA

• RFA with random regression coefficients in structural equation modeling (RSP)

---- performance unknown

This paper…• Compared methods:MGFA RFA with LMSRFA with RSP• Measurement biasUniformNonuniform• ViolatorDichotomousContinous

Data generation (RFA)

• True model:

• Uniform bias: . Nonuniform bias: • T and v are bivariate standard normal

distributed with correlation r• e is standard normal distributed• u is null vector

0b 0c

Simulation Design

For continuous V:• Type of bias (only on item 1): No bias (b=c=0), uniform bias(b=0.3,c=0), nonuniform bias (b=0,c=0.3), mixed bias (b=c=0.3)• Relationship between T and V Independent (r=0), dependent (r=0.5)

Simulation Design

For dichotomous V:• V=-1 for group 1 and v=1 for group 2• Model can be rewritten into

• Relationship between T and V: Correlation varies!

.)()(

,)()()2(

)1(

detcabux

detcabux

)1,4.0(~

)1,4.0(~)2(

)1(

NT

NT

The MGFA method

• When v is dichotomous, regular MGFA• When v is continuous, dichotomize x by V• Using chi-square difference test with df=2Uniform : interceptsNonuniform: loadings

The RFA/LMS method

• V is modeled as latent variable:Single indicatorFix residual variance (0.01)Fix factor loading• Three-factor model: T, V, T*V• Robust ML estimation• Chi-square test with S-B correction: : uniform bias : nonuniform bias

0b0c

RFA/RSP method

• Replacing with , where is a random slope.

• Robust ML estimation• Chi-square test with S-B correction: : uniform bias : nonuniform bias

0b

0c

Single & iterative procedures• Single run procedure: test once for each item• Iterative procedure: 1)Locate the item with the largest chi-square

difference2)Free constrains on intercepts and factor

loadings for this item and test others3)Locate the item with the largest chi-sqaure

difference 4)…5)Stops when no significant results exist or half

are detected as biased

Results of MGFA – single run

• Shown in Table 2.• Conclusion:1.better with dichotomous than with

continuous V; 2.non-uniform bias is more difficult to detect

than uniform bias; 3.Type I error inflated.

Results of MGFA – iterative run

• Shown in Table 3.• Conclusion:1.Iterative procedure produces close power as

single run does.2.Iterative procedure produces better

controlled Type I error rate.

Results of RFA/LMS & RFA/RSP - single run

• Shown in Table 4 and Table 5.• Conclusion:1.LMS and RSP produce almost equivalent

results. 2. larger power than MGFA with continuous V.3.More severely inflated Type I error rates

Results of RFA/LMS & RFA/RSP - iterative run

• Shown in Table 6.• Conclusion:1.Power is close to the single run2.Type I error rates are improved

Results of estimation bias - MGFA

• Shown in Table 7.• Conclusion:1.Bias in estimates is small2.Bias in SD is non-ignorable3.Smaller bias in estimates for dichotomous V

(dependent T&V)

Results of estimation bias - RFA

• Shown in Table 8 & 9• Conclusion:1.Similar results for LMS and RSP2.Small bias in estimates3.Non-ignorable bias in SD4.Smaller SE than MGFA5.Smaller bias in estimates than MGFA with

dependent T&V, continuous V.

Discussion

• Nonconvergence occurs with RFA/LMS

Non-convergence• Summary: