Variability Indicators in Structural Equation Models
description
Transcript of Variability Indicators in Structural Equation Models
Variability Indicators in Structural Equation Models
Michael BidermanUniversity of Tennessee at Chattanooga
www.utc.edu/Michael-Biderman
For the past few years I’ve investigated the utility of a structural equation model of faking of personality questionnaires, specifically the Big Five.
The model is a CFA containing
1) latent variables representing personality dimensions, and
2) one or more latent variables representing amount of response distortion, i.e., faking.
Background: Modeling Faking of Personality Tests
The Basic Faking Model
F
FE3FE2FE1
FE4FE5
FA3FA2FA1
FA4FA5
FC3FC2FC1
FC4FC5
FS3FS2FS1
FS4FS5
FI3FI2FI1
FI4FI5
E HE3HE2HE1
HE4HE5
A HA3HA2HA1
HA4HA5
C HC3HC2HC1
HC4HC5
S HS3HS2HS1
HS4HS5
I HO3HO2HO1
HO4HO5
E-H
A-H
C-H
S-H
O-H
E-D
A-D
C-D
S-D
O-D
E-I
A-I
C-I
S-I
I-I
E
A
C
S
I
FP
FA
Applied to two-condition data Applied to three-condition data
Beyond the Basic Model:
The basic model represents changes in central tendency associated with faking fairly well.
Is that all there is?
Last year, during manual data entry of Mike Clark’s thesis data (Yes, UTC is a full-service university) . . .I noticed that some participants seemed to be targeting specific responses, e.g., 6 6 6 6 6 6
Since such targeting results in low variability this suggested the possibility that variability of responding might be an indicator of faking.
The following describes an attempt to model variability.
5
Other studies of variability
Traitedness studies (Britt, 1993; Dwight, Wolf & Golden, 2002; Hershberger, Plomin, & Pedersen, 1995).
A person is highly traited on a dimension if the variability of responses to items from the dimension is small.
Extreme response style (Greenleaf, 1992).
Stability of responses to the same scale over time (Eid & Diener, 1999; Kernis, 2005).
No studies of variability of responses in faking situations. None of variability and the Big Five.
1: Biderman & Nguyen, 2004. N=2032: Wrensen & Biderman, 2005. N=166
Two-condition data: Honest and Fake Good50 item IPIP Big Five questionnaire given twice 2-item parcels analyzed
3. Clark & Biderman, 2006. N=166
Three-conditions: Honest, Incentive, Instructed fakingThree 30-item IPIP Questionnaires given.Whole-scale scores analyzed.
Wonderlic Personnel Test (WPT) was given to all participants prior to the first condition.
Three used datasets.
7
Measuring Variability
To represent variability of responses within dimensions,
I computed the standard deviation of responses within each Big Five dimension for each participant for each condition
I added the standard deviations as observed variables to the data to which the faking model had previous been applied
8
Datasets 1 and 2 with Standard Deviations added
F
FE3FE2FE1
FE4FE5
FA3FA2FA1
FA4FA5
FC3FC2FC1
FC4FC5
FS3FS2FS1
FS4FS5
FI3FI2FI1
FI4FI5
E HE3HE2HE1
HE4HE5
A HA3HA2HA1
HA4HA5
C HC3HC2HC1
HC4HC5
S HS3HS2HS1
HS4HS5
I HO3HO2HO1
HO4HO5
SdHE
SdHA
SdHC
SdHS
SdHI
SdFE
SdFA
SdFC
SdFS
SdFI
9
Dataset 3 with standard deviations added
E-H
A-H
C-H
S-H
O-H
E-D
A-D
C-D
S-D
O-D
E-I
A-I
C-I
S-I
I-I
E
A
C
S
I
FP
FA
SdE-H
SdA-H
SdC-H
SdS-H
SdO-H
SdE-D
SdA-D
SdC-D
SdS-D
SdO-D
SdE-I
SdA-I
SdC-I
SdS-I
SdO-I
10
Modeling Standard Deviations - 1
Faking leads to elevated central tendency, often resulting in ceiling effects.
Ceiling effects lead to lower variability.
So the standard deviations were connected to central tendency via regression links.
Specifically, standard deviations were regressed onto parcel or scale scores.
11
Modeling Ceiling Effects in Datasets 1 and 2:Standard deviations were regressed onto parcel scores
F
FE3FE2FE1
FE4FE5
FA3FA2FA1
FA4FA5
FC3FC2FC1
FC4FC5
FS3FS2FS1
FS4FS5
FI3FI2FI1
FI4FI5
E HE3HE2HE1
HE4HE5
A HA3HA2HA1
HA4HA5
C HC3HC2HC1
HC4HC5
S HS3HS2HS1
HS4HS5
I HO3HO2HO1
HO4HO5
SdHE
SdHA
SdHC
SdHS
SdHI
SdFE
SdFA
SdFC
SdFS
SdFI
12
Modeling Ceiling Effects in Dataset 3: Standard Deviations regressed onto scale scores
E-H
A-H
C-H
S-H
O-H
E-D
A-D
C-D
S-D
O-D
E-I
A-I
C-I
S-I
I-I
E
A
C
S
I
FP
FA
SdE-H
SdA-H
SdC-H
SdS-H
SdO-H
SdE-D
SdA-D
SdC-D
SdS-D
SdO-D
SdE-I
SdA-I
SdC-I
SdS-I
SdO-I
13
Modeling Standard Deviations - 2
The assumption/hope? was that there are individual differences in variability of responding within dimensions
So a latent variable representing such individual differences was added to the model.
14
Modeling variability in Dataset 1 & 2: Adding a “Variability” latent variable
F
FE3FE2FE1
FE4FE5
FA3FA2FA1
FA4FA5
FC3FC2FC1
FC4FC5
FS3FS2FS1
FS4FS5
FI3FI2FI1
FI4FI5
.
.
.
.
.
E HE3HE2HE1
HE4HE5
A HA3HA2HA1
HA4HA5
C HC3HC2HC1
HC4HC5
S HS3HS2HS1
HS4HS5
I HO3HO2HO1
HO4HO5
.
.
.
.
.
SdHE
SdHA
SdHC
SdHS
SdHI
SdFE
SdFA
SdFC
SdFS
SdFI
V
15
Modeling variability in Dataset 3: Adding a “Variability” latent variable
E-H
A-H
C-H
S-H
O-H
E-D
A-D
C-D
S-D
O-D
E-I
A-I
C-I
S-I
I-I
E
A
C
S
I
FP
FA
SdE-H
SdA-H
SdC-H
SdS-H
SdO-H
SdE-D
SdA-D
SdC-D
SdS-D
SdO-D
SdE-I
SdA-I
SdC-I
SdS-I
SdI-I
V
16
Results
Did the regression links significantly improve goodness of fit?
Are there individual differences in variability of responding that are captured by the V latent variable?
17
Results of application of Variability model to Dataset 1Model Χ2(1539)=2501.249; p<.001; CFI=.871; RMSEA=.055
Both the regression links and the V latent variable improved model fit.
F
.50
.60
.62
.44
.50
FE3FE2FE1
FE4FE5
FA3FA2FA1
FA4FA5
FC3FC2FC1
FC4FC5
FS3FS2FS1
FS4FS5
FI3FI2FI1
FI4FI5
.37
.30
.27
.31
.22
E HE3HE2HE1
HE4HE5
A HA3HA2HA1
HA4HA5
C HC3HC2HC1
HC4HC5
S HS3HS2HS1
HS4HS5
I HI3HI2HI1
HI4HI5
.26
.37
.31
.27
.50
.10
.16
.28
.34
.38
.81
.69
.72
.79
.60
SHE-.06
.08
.35
.08
.30.17
SHA-.14
SHC-.14
SHS-.07
SHI-.11
SFE-.07
SFA-.16
SFC-.16
SFS-.09
SFI-.09
V
.31
.52
.47
.36
.38
.27
.42
.34
.34
.45
ΔΧ2(31)=597.537p<.001
Chi-square difference test of regression links: Χ2(50)=625.685p<.001
Chi-square difference test of V latent variable:Χ2(16)=218.041p<.001
18
Results of application of Variability model to Dataset 2Model Χ2(1539)=2666.874; p<.001; CFI=.816; RMSEA=.066
Chi-square difference test of V latent variable:ΔΧ2(16)=274.468p<.001
ΔΧ2(31)=608.423p<.001
F
.56
.63
.67
.51
.50
FE3FE2FE1
FE4FE5
FA3FA2FA1
FA4FA5
FC3FC2FC1
FC4FC5
FS3FS2FS1
FS4FS5
FO3FO2FO1
FO4FO5
.26
.20
.19
.26
.25
E HE3HE2HE1
HE4HE5
A HA3HA2HA1
HA4HA5
C HC3HC2HC1
HC4HC5
S HS3HS2HS1
HS4HS5
I HO3HO2HO1
HO4HO5
.14
.23
.23
.26
.20
-.07
.01
.02
.28
.31
.75
.67
.70
.75
.63
SHE-.08
.02
.00
.05
-.01.08
SHA-.16
SHC-.14
SHS-.06
SHO-.14
SFE-.13
SFA-.20
SFC-.19
SFS-.09
SFO-.18
V
.52
.39
.35
.38
.42
.56
.36
.40
.25
.46
Chi-square difference test of regression links:ΔΧ2(50)=747.249p<.001
Again, both the regression links and the V latent variable improve model fit.
19
Results of application of Variability model to Dataset 3Model Χ2(352)=532.552; p<.001; CFI=.883; RMSEA=.056
E-H
A-H
C-H
S-H
O-H
E-D
A-D
C-D
S-D
O-D
E-I
A-I
C-I
S-I
I-I
E
A
C
S
I
FP
FA
SdE-H
SdA-H
SdC-H
SdS-H
SdO-H
SdE-D
SdA-D
SdC-D
SdS-D
SdO-D
SdE-I
SdA-I
SdC-I
SdS-I
SdI-I
V
43
.26
.11
.26
.19
.35
.16
.26
.31
.57
.73
.62
.29
.38
-.09
-.24
-.43
.47
.52
.41
.67
Chi-square difference test of V latent variable:ΔΧ2(22)=405.333p<.001
ΔΧ2(12)=37.910p<.05
ΔΧ2(17)=260.217p<.001(FP correlations with Big 5 LVs set at 0 for this test.)
Chi-square difference test of regression links:ΔΧ2(15)=268.367p<.001
20
Tentative Conclusions regarding Variability Model
1) Ceiling effects seem to be successfully modeled by the regressions of standard deviations onto parcels or scale scores.
2) Individual differences in variability of responding to items within dimensions seem to be captured by the V latent variable.
Some persons consistently exhibited little variability in responding to items within questionnaire scales.
Others exhibit greater variability in responding.
21
What about V and faking?1) Loadings on V might be larger in faking conditions – magnifying individual differences in variability because some people are targeting while others are not.
Mean Standardized Loadings of Standard Deviation indicators on V
Dataset 1 Honest Incentive to fake Instructed to fakeMean loading .406 .366
Dataset 2 Honest Incentive to fake Instructed to fakeMean loading .411 .496
Dataset 3 Honest Incentive to fake Instructed to fakeMean loading .468 .521 .413
Tentative Conclusion: Loadings on V are approximately equal in honest and faking conditions.
22
2) V might be related to the faking latent variables.
Dataset 1: Correlation of V with F: .04 NS
Dataset 2: Correlation of V with F: .02 NS
Dataset 3: Correlation of V with FP: .16 NS
Correlation of V with FA: .10 NS
It appears from these preliminary analyses that variability of responding is not related to faking
However, other scenarios and models should be explored
What about V and faking?
23
What is V?1) Perhaps V is related to personality characteristics
It appears that V has discriminant validity with respect to the Big Five.
24
What is V?2) Perhaps V is related to cognitive ability
These results suggest that persons with higher CA exhibit less variability of responding.
25
Uses of VHow about using it to extract cognitive ability from the Big Five?
FA
V
I
WPT
e Dataset 1: Multiple R = .57
Dataset 2: Multiple R = .35
Dataset 3: Multiple R = .50
The structural model suggests that there is information on cognitive ability embedded in “noncognitive” personality tests.
26
Conclusions
1) V appears to be an individual difference variable that cuts across personality dimensions.
2) V appears to be unrelated to faking
3) V appears to be independent of the Big Five dimensions.
4) V appears to be related to cognitive ability – persons higher in cognitive ability have lower variability of responding
27
Implications
Don’t throw away old datasets.
You never know what constructs may be hidden in them.