Have I been criticized?
-
Upload
lamar-mathis -
Category
Documents
-
view
21 -
download
0
description
Transcript of Have I been criticized?
Have I been criticized?
Campbell’s (1982) comments as outgoing editor of the Journal of Applied Psychology (JAP):
…with perhaps one or two exceptions there has been very little opportunity to exercise any professional biases [as editor of JAP]. One possible exception pertains to the use of a self-report questionnaire to measure all the variables in a study. If there is no evident construct validity for the questionnaire measure or no variables that are measured independently of the questionnaire, I am biased against the study and believe that it contributes very little. (p. 692)
What are reviewers’ concerns about MV?
One possibility: compromised construct validity:
From CTT: Xij = Ti + Mj + Eij,
X T M Eij i j ij
2 2 2 2 Xij
Ti Mj
Eij
What are reviewers’ concerns about MV?
Second possibility: covariance distortion due to CMV:Again, from CTT:
Perhaps interestingly:
ijjMijiTijij EMTX
jiijiijiijjiij MMTTTTXXr ''''
jiij
jiijjiij
ii
TT
MMXX
TT
r
'
''
'
Xij Xi’j
Eij Ei’j
Ti Ti’Mj
What are reviewers’ complaints about MV?
Third possibility: effects of related methods:
This formula is for a heterotrait-heteromethod correlation in a MTMM design
Covariance distortion can still incur with different methods as long as the methods’ effects s) are not inconsequential and methods are related (MjMj’)
'''''''' jjjiijiijiijjiij MMMMTTTTXX
What are best analytic approaches to demonstrating lack of CMV?
CFA of MTMM data – is this the gold standard?
Williams and Anderson’s (1994) approach using substantive method-related variables
Lindell and Whitney’s (2001) partial correlation approach
Need research that compares these approaches to highlight advantages of each
Discussions and disputes on CMV?
Do not make the default assumption that “if it ain’t trait it must be method."
1 2 . . . . . M1
2.
. . N
Subjects
Traits
T
12.....
Methods
Discussions and disputes on CMV?
Very unfortunate consequences for at least two bodies of literatureRater sources in 360o ratings are not mere methodsAssessment center exercises are not mere methods
Occasions of measurement and state-trait-occasion models
Locate one’s measurement design within a broader multidimensional data array reminiscent of Cattell’s (1966) “data box”
Prototype multidimensional data array
Persons (or groups or collectivities)Focal constructsOccasions of measurementSituationsObservers or recordersResponse modalities
Example – 360o as persons x constructs x observersExample – AC as persons x constructs x situations