MEASUREMENT: VALIDITY
description
Transcript of MEASUREMENT: VALIDITY
![Page 1: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/1.jpg)
MEASUREMENT: VALIDITY
Lu Ann Aday, Ph.D.The University of Texas School of Public Health
![Page 2: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/2.jpg)
VALIDITY: Definition Extent of systematic departure
in answers to questions in relationship to the meaning of the concept itself (content), equivalent measures of the same concept (criterion), or hypothesized associations with measures of similar or different concepts (construct).
![Page 3: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/3.jpg)
VALIDITY: Types
Content validityCriterion validityConstruct validity
![Page 4: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/4.jpg)
VALIDITY: Computation Requires analyzing the
association between survey measures and documented definitions of the underlying concept (content); an equivalent “gold standard” of the survey measures (criterion); or theoretical associations with other abstract concepts (construct).
![Page 5: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/5.jpg)
VALIDITY: Content
Definition: extent to which the survey measures adequately represent the concept of interest
![Page 6: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/6.jpg)
VALIDITY: Content
Factors affecting: Failure to thoroughly conduct background research on the definition & measurement of the study concept
Failure to clearly define the study concept
![Page 7: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/7.jpg)
VALIDITY: Content Computation:
1. Identify domain or universe of interest through a thorough review of the literature.
2. Select or construct questions based on the review.
3. Ask subject matter experts to review the items you have selected or constructed.
![Page 8: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/8.jpg)
VALIDITY: Content Validity criteria:
1. Is there at least one item to represent the concept?
2. Are there enough items? 3. Are the items well accepted, based on the literature & expert opinion?
4. Are they reliable?
![Page 9: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/9.jpg)
VALIDITY: Criterion
Definition: extent to which the survey measure predicts (predictive) or agrees with (concurrent) a criterion indicator of the concept of interest
![Page 10: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/10.jpg)
VALIDITY: Criterion Factors affecting:
Underreporting due to problems with recalling non-salient or frequently occurring events
Overreporting due to problems with telescoping salient events
![Page 11: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/11.jpg)
VALIDITY: Criterion- Predictive: Correlation Analysis
Computation: Compute correlation coefficient between answers to survey question and future criterion source value for same concept :Respondent Survey Question MD Observation1 ADLs=3 ADLs=3 2 ADLs=2 ADLs=2 3 ADLs=1 ADLs=1 44 ADLs=0ADLs=0 ADLs=3ADLs=35 ADLs=0 ADLs=0
![Page 12: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/12.jpg)
VALIDITY: Criterion- Concurrent: Correlation Analysis
Computation: Compute correlation coefficient between answers to survey question and current criterion source value for same concept:
Respondent Survey Question Clinic Record1 BP=140/90 BP=140/90 2 BP=150/80 BP=150/80 3 BP=145/95 BP=145/95
44 BP=145/95BP=145/95 BP=120/80BP=120/805 BP=140/90 BP=140/90
![Page 13: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/13.jpg)
VALIDITY: Criterion-Concurrent: Sensitivity-Specificity Analysis
CRITERION /SURVEY
IS (+)Hypertensive
IS NOT (-)Hypertensive
IS (+)Hypertensive
a = true +Sensitivity =a/a + c
b = false +
IS NOT (-)Hypertensive
c = false - d = true –Specificity =d/b + d
![Page 14: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/14.jpg)
VALIDITY: Criterion-Concurrent: Sensitivity-Specificity Analysis
CRITERION /SURVEY
IS (+)Hypertensive
IS NOT (-)Hypertensive
IS (+)Hypertensive
a = 130Sensitivity =130/180= 72.2%
b = 1,000
IS NOT (-)Hypertensive
c = 50 d = 64,000Specificity =64,000/65,000=98.5%
![Page 15: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/15.jpg)
VALIDITY: Construct Definition: extent to which
relationships between measures agree with relationships predicted by theories or hypotheses Convergent – Ho: Related Discriminant – Ho: Not Related
![Page 16: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/16.jpg)
VALIDITY: Construct
Factors affecting: Inadequate conceptualization or measurement of study constructs
Inadequate theory to guide specification of hypothesized relationships
![Page 17: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/17.jpg)
VALIDITY: Construct: Correlation Analysis
Computation: Compute correlation coefficient between survey question or scales measuring construct and related (or unrelated) constructs :Respondent Physical Mental Overall
1 Good FairFair Good2 Good FairFair Good3 Good FairFair Good44 Good FairFair Good5 Good FairFair Good
![Page 18: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/18.jpg)
VALIDITY: Construct- Known Groups Validity Analysis (McHorney, et al., 1993, Table 3, p. 255)
COMPARISON GROUPS/MOS-SF 36 SUMMARY SCALES
MINOR MEDICAL
SERIOUS MEDICAL
PSYCHIATRIC ONLY
PSYCHIATRIC& SERIOUSMEDICAL
PhysicalFunctioning
80.53 57.35 80.62 46.37
MentalHealth
82.49 77.59 52.75 56.90
![Page 19: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/19.jpg)
VALIDITY: CONSTRUCT-Factor Analysis(McHorney, et al., 1993, Table 1, p. 251)
FACTOR LOADINGS (HYPOTHESES)/SUBSCALES
Physical Mental
Physical functioning 0.88 (+) 0.04
Role-physical 0.78 (+) 0.30
Bodily pain 0.77 (+) 0.24
Mental health 0.12 0.90 (+)
Role-emotional 0.19 0.81 (+)
Social Functioning 0.44 (~) 0.71 (+)
Vitality 0.59 (~) 0.57 (~)
General health perceptions
0.68 (~) 0.32 (~)
![Page 20: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/20.jpg)
VALIDITY: Construct—Multi-Trait, Multi-Method
Trait X Method
A B C A B C A B C
A (.90)
B (.85)
C (.80)
A .75 (.80)
B .55 (.75)
C .35 (.80)
A .80 .85 (.95)
B .55 .55 (.90)
C .30 .30 (.65)
![Page 21: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/21.jpg)
VALIDITY: Effect of Reliability
Theoretical Variable can be no more valid than it is reliable.
Empirical Rt = theoretical correlation between study &
criterion variables (unobservable) Rx = reliability of study variable (computed) Ry = reliability of criterion variable (computed) Rt * sqrt [ Rx * Ry] = Max. achievable
correlation 1.00 * sqrt [.60 * .80] = .69
![Page 22: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/22.jpg)
WHEN TO UNDERTAKE VALIDITY ANALYSIS
VALIDITY/DIMENSIONS
CONTENT CRITERION CONSTRUCT
QUESTIONS All questions Factual questions for which “gold standard” is available
Summary scales of attitudes or other abstract concepts
STUDIES All types of surveys
Esp. used in behavioral surveys
Esp. used in attitudinal surveys
STAGES Question development
Pretest or final study
Pretest or final study
![Page 23: MEASUREMENT: VALIDITY](https://reader035.fdocuments.net/reader035/viewer/2022081501/568150af550346895dbecc68/html5/thumbnails/23.jpg)
REFERENCES DeVellis, Robert F. (2003). Scale Development:
Theory and Applications. Second Edition. Thousand Oaks, CA: Sage.
Gandek, B., & Ware, J.E., Jr., for the IQOLA Project (1998). Methods for validating and norming translations of health status questionnaires: The IQOLA Project Approach. J. Clinical Epidemiology, 51 (11), 953-959.
McHorney, C.A., Ware, J.E., Jr., & Raczek, A.E. (1993). The MOS 36-item short-form health survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Medical Care, 31(3), 247-263. [Note: Tables 1 and 3 adapted and reprinted with the permission of Lippincott, Williams & Wilkins, Publishers.]