MEASUREMENT MODELS
BASIC EQUATION
• x = + e
• x = observed score = true (latent) score: represents
the score that would be obtained over many independent administrations of the same item or test
• e = error: difference between y and
ASSUMPTIONS and e are independent
(uncorrelated)
• The equation can hold for an individual or a group at one occasion or across occasions:
• xijk = ijk + eijk (individual)
• x*** = *** + e*** (group)
• combinations (individual across time)
x x
e
RELIABILITY
• Reliability is a proportion of variance measure (squared variable)
• Defined as the proportion of observed score (x) variance due to true score ( ) variance:
2x = xx’
• = 2 / 2
x
Var()
Var(x)
Var(e)
reliability
Reliability: parallel forms
• x1 = + e1 , x2 = + e2
(x1 ,x2 ) = reliability
• = xx’
• = correlation between parallel forms
x1 x
e
x2
e
x
xx’ = x * x
ASSUMPTIONS and e are independent
(uncorrelated)
• The equation can hold for an individual or a group at one occasion or across occasions:
• xijk = ijk + eijk (individual)
• x*** = *** + e*** (group)
• combinations (individual across time)
Reliability: Spearman-Brown
• Can show the reliability of the composite is
kk’ = [k xx’]/[1 + (k-1) xx’ ]
• k = # times test is lengthened
• example: test score has rel=.7
• doubling length produces rel = 2(.7)/[1+.7] = .824
Reliability: parallel forms
• For 3 or more items xi, same general form holds
• reliability of any pair is the correlation between them
• Reliability of the composite (sum of items) is based on the average inter-item correlation: stepped-up reliability, Spearman-Brown formula
RELIABILITY
Generalizability d - coefficients ANOVA
g - coefficients
Cronbach’s alpha
test-retest internal consistency
inter-rater
parallel form Hoyt
dichotomous split halfscoring
KR-20 SpearmanKR-21 Brown
averageinter-itemcorrelation
COMPOSITES AND FACTOR STRUCTURE
• 3 MANIFEST VARIABLES REQUIRED FOR A UNIQUE IDENTIFICATION OF A SINGLE FACTOR
• PARALLEL FORMS REQUIRES:– EQUAL FACTOR LOADINGS– EQUAL ERROR VARIANCES– INDEPENDENCE OF ERRORS
x1
x
e
x2
e
x
xx’ = xi * xj
x3
e
x
RELIABILITY FROM SEM• TRUE SCORE VARIANCE OF THE
COMPOSITE IS OBTAINABLE FROM THE LOADINGS:
K = 2
i i=1
K = # items or subtests
• = K2x
Hancock’s Formula
• Hj = 1/ [ 1 + {1 / (Σ[l2ij/(1- l2
ij )] ) }
• Ex. l1 = .7, l2= .8, l3 = .6
• H = 1 / [ 1 +1/( .49/.51 + .64/.36 + .36/.64 )]
= 1 / [ 1 + 1/ ( .98 +1.67 + .56 ) ]
= 1/ (1 + 1/3.21)
= .76
Hancock’s Formula Explained
Hj = 1/ [ 1 + {1 / (Σ[l2ij/(1- l2
ij )] ) }
now assume strict parallelism: then l2ij= 2
xt
thus Hj = 1/ [ 1 + {1 / (Σ[2xt /(1- 2
xt)] ) }
= k 2xt / [1 + (k-1) 2
xt ]
= Spearman-Brown formula
RELIABILITY FROM SEM
• RELIABILITY OF THE COMPOSITE IS OBTAINABLE FROM THE LOADINGS:
= K/(K-1)[1 - 1/ ]
• example 2x = .8 , K=11
= 11/(10)[1 - 1/8.8 ] = .975
SEM MODELING OF PARALLEL FORMS
• PROC CALIS COV CORR MOD;
• LINEQS
• X1 = L1 F1 + E1,
• X2 = L1 F1 + E1,
• …
• X10 = L1 F1 + E1;
• STD E1=THE1, F1= 1.0;
TAU EQUIVALENCE
• ITEM TRUE SCORES DIFFER BY A CONSTANT:
i = j + k
• ERROR STRUCTURE UNCHANGED AS TO EQUAL VARIANCES, INDEPENDENCE
TESTING TAU EQUIVALENCE
• ANOVA: TREAT AS A REPEATED MEASURES SUBJECT X ITEM DESIGN:
• PROC VARCOMP;CLASS ID ITEM;
• MODEL SCORE = ID ITEM;
• LOW VARIANCE ESTIMATE CAN BE TAKEN AS EVIDENCE FOR PARALLELISM (UNLIKELY TO BE EXACTLY ZERO
CONGENERIC MODEL
• LESS RESTRICTIVE THAN PARALLEL FORMS OR TAU EQUIVALENCE:– LOADINGS MAY DIFFER– ERROR VARIANCES MAY DIFFER
• MOST COMPLEX COMPOSITES ARE CONGENERIC:– WAIS, WISC-III, K-ABC, MMPI, etc.
x1
x1
e1
x2
e2
x2
(x1 , x2 )= x1 * x2
x3
e3
x3
COEFFICIENT ALPHA
xx’ = 1 - 2E /2
X
• = 1 - [2i (1 - ii )]/2
X ,
• since errors are uncorrelated = K/(K-1)[1 - (s2
i )/ s2X ]
• where X = xi (composite score)
s2i = variance of subtest xi
sX = variance of composite
• Does not assume knowledge of subtest ii
COEFFICIENT ALPHA- NUNNALLY’S COEFFICIENT
• IF WE KNOW RELIABILITIES OF EACH SUBTEST, i
N = K/(K-1)[s2i (1- rii )/ s2
X ]
• where rii = coefficient alpha of each subtest
• Willson (1996) showed N
SEM MODELING OF CONGENERIC FORMS
MPLUS EXAMPLE: this is an example of a CFA
DATA: FILE IS ex5.1.dat;
VARIABLE: NAMES ARE y1-y6;
MODEL: f1 BY y1-y3;
f2 BY y4-y6;
OUTPUT: SAMPSTAT MOD STAND;
x1
x1
e1
x2
e2
x2
XiXi = 2xi + s2
i
x3
e3
x3
s1
NUNNALLY’S RELIABILITY CASE
s2
s3
x1
x1
e1
x2
e2
x2
Specificities can be
misinterpreted as a correlated
error model if they are
correlated or a second factor
x3
e3
x3
s
CORRELATED ERROR PROBLEMS
s3
x1
x1
e1
x2
e2
x2
Specificieties can be
misinterpreted as a
correlated error model
if specificities are
correlated or are a
second factor
x3
e3
x3
CORRELATED ERROR PROBLEMS
s3
SEM MODELING OF CONGENERIC FORMS- CORRELATED ERRORS
MPLUS EXAMPLE: this is an example of a CFA
DATA: FILE IS ex5.1.dat;
VARIABLE: NAMES ARE y1-y6;
MODEL: f1 BY y1-y3;
f2 BY y4-y6;
y4 with y5;
OUTPUT: SAMPSTAT MOD STAND;
specifies residuals of previous model, correlates them
MULTIFACTOR STRUCTURE
• Measurement Model: Does it hold for each factor?– PARALLEL VS. TAU-EQUIVALENT VS.
CONGENERIC
• How are factors related?
• What does reliability mean in the context of multifactor structure?
SIMPLE STRUCTURE
• PSYCHOLOGICAL CONCEPT:
• Maximize loading of a manifest variable on one factor ( IDEAL = 1.0 )
• Minimize loadings of the manifest variables on all other factors ( IDEAL = 0 )
SIMPLE STRUCTURE
Example
SUBTEST FACTOR1 FACTOR2 FACTOR3
A 1 0 0
B 1 0 0
C 0 1 0
D 0 1 0
E 0 0 1
F 0 0 1
MULTIFACTOR ANALYSIS
• Exploratory: determine number, composition of factors from empirical sampled data– # factors # eigenvalues > 1.0 (using squared
multiple correlation of each item/subtest i with
the rest as a variance estimate for 2xi
– empirical loadings determine structure
MULTIFACTOR ANALYSISTITLE:this is an example of an exploratory
factor analysis with continuous factor
indicators
DATA: FILE IS ex4.1.dat;
VARIABLE:NAMES ARE y1-y12;
ANALYSIS:TYPE = EFA 1 4;
MULTIFACTOR MODEL WITH THEORETICAL
PARAMETERS
MPLUS EXAMPLE: this is an example of a CFA
DATA: FILE IS ex5.1.dat;
VARIABLE: NAMES ARE y1-y6;
MODEL: f1 BY [email protected] [email protected] [email protected];
f2 BY [email protected] [email protected] [email protected];
f1 with [email protected];
OUTPUT: SAMPSTAT MOD STAND;
1
x1
x11
e1
x2
e2
x22
x3
e3
x31
MINIMAL CORRELATED FACTOR STRUCTURE
2
x4e4
x42
12
FACTOR RELIABILITY• Reliability for Factor 1:
= 2(x11 * x31 ) / (1 + x11 * x31 )(Spearman-Brown for Factor 1 reliability
based on the average inter-item correlation
• Reliability for Factor 2:
= 2(x22 * x42 ) / (1 + x22 * x42 )
FACTOR RELIABILITY• Generalizes to any factors- reliability is
simply the measurement model reliability for the scores for that factor
• This has not been well-discussed in the literature– problem has been exploratory analyses produce
successively smaller eigenvalues for factors due to the extraction process
– second factor will in general be less reliable using loadings to estimate interitem r’s
FACTOR RELIABILITY• Theoretically, each factor’s reliability should be
independent of any other’s, regardless of the covariance between factors
• Thus, the order of factor extraction should be independent of factor structure and reliability, since it produces maximum sample eigenvalues (and sample loadings) in an extraction process.
• Composite is a misnomer in testing if the factors are treated as independent constructs rather than subtests for a more global composite score (separate scores rather than one score created by summing subscale scores)
CONSTRAINED FACTOR MODELS
• If reliabilities for scales are known independent of the current data (estimated from items comprising scales, for example), error variance can be constrained:
• s2ei = s[1 - i ]
x1
x1
e1
x2
e2
x2
x3
e3
x3
CONSTRAINED SEM- KNOWN RELIABILITY
sx3 [1- 3 ]1/2
sx1 [1- 1 ]1/2 sx2 [1- 2 ]1/2
CONSTRAINED SEM-KNOWN RELIABILITY
MPLUS EXAMPLE: this is an example of a CFA with known error unreliabilities
DATA: FILE IS ex5.1.dat;
VARIABLE: NAMES ARE y1-y6;
MODEL: f1 BY y1-y3;
f2 BY y4-y6;
OUTPUT: SAMPSTAT MOD STAND;
similar statement for each item
SEM Measurement Procedures
• 1. Evaluate the theoretical measurement model for ALL factors (not single indicator variables included)
• Demonstrate discriminant validity by showing the factors are separate constructs
• Revise factors as needed to demonstrate- drop some manifest variables if necessary and not theoretically damaging
• Ref: Anderson & Gerbing (1988)
Top Related