Chapter 8 Construct and External Validity in Experimental Research ♣ ♣ Construct Validity ...
-
Upload
kimberly-stevens -
Category
Documents
-
view
248 -
download
3
Transcript of Chapter 8 Construct and External Validity in Experimental Research ♣ ♣ Construct Validity ...
Chapter 8Construct and External Validity in Experimental Research ♣
Construct Validity
External Validity
Cautions in Evaluating the External Validity of Experiments
Relationship between Internal and External Validity
Back to Brief Contents
8.1 Construct Validity -1
Definition—extent to which we can
infer higher-order constructs for our operationsFig 8.1
Constructs are used for Research participants Independent variable Dependent variable Experimental setting
Back to Chapter Contents
8.1 Construct Validity -2
Assessing Construct Validity
Threats to Construct Validity Reactivity to the Experimental Situation
Experimenter Effect
Back to Chapter Contents
Assessing Construct Validity -1
Have a clear definition of the construct of interest Problem is identifying prototypical features of
the constructs
Affects ability to identify the concrete operations used to represent the construct
Back to Chapter Contents
Assessing Construct Validity -2(end)
Assess the match between the constructs and the operations used to represent them Ways to assess
Content validity
Criterion-related validity
— Predictive
— concurrent
Any other source of evidence
Back to Chapter Contents
Threats to Construct Validity
Inadequate explanation of the construct Tab 8.1 Construct confounding Mono-operation bias Mono-method bias Confounding constructs with level of constructs Treatment-sensitive factorial structure Reactive self-report changes Reactivity to the experimental situation Experimenter effects Novelty and disruption effects Compensatory equalization Compensatory rivalry Treatment diffusion
Back to Chapter Contents
Threats to Construct Validity
Inadequate explanation of the constructIf a construct is not adequately explained and analyzed,
it can lead to a set of operations that do not represent the construct adequately
Construct confoundingthe operations used in a study represent more than one construct
Mono-operation biasa study uses only one operationalization of a construct
This typically results in an underrepresentation of the construct and lowers construct validity
Back to Chapter Contents
Threats to Construct Validity
Mono-method biasa study uses only one method (e.g., physiological recording) to operationalize a construct
The method used may influence the results
Confounding constructs with level of constructsa study investigates only a few levels of a construct (e.g., three doses of a drug), but makes inferences about the overall construct (e.g., the overall effect of the dose)
Treatment-sensitive factorial structurean instrumentation change that occurs because of the experimental treatment
Back to Chapter Contents
Threats to Construct Validity
Reactive self-report changeschanges that a research participant may make on self-report measures as a result of a motivational shift after being included in the experimental study
Reactivity to the experimental situationresearch participants’ perceptions and motives can affect the responses they make to the dependent variable
and these responses can be interpreted as part of the treatment construct being tested
Back to Chapter Contents
Threats to Construct Validity
Experimenter effectsthe experimenter’s attributes and expectancies can influence the responses made by the research participants
and these responses can be interpreted as part of the treatment construct being tested
Novelty and disruption effectsResearch participants usually respond better to a new and novel situation and poorly to one that disrupts their routine
These effects are part of the overall treatment effect
Back to Chapter Contents
Threats to Construct Validity
Compensatory equalizationindividuals try to provide the same benefits or services to the control group that are received by the experimental group
Compensatory rivalryindividuals resent being assigned to the control group and respond more negatively than would be expected, because of the resentment they feel
Treatment diffusionindividuals in one treatment group receive some or all of another group’s treatment
Back to Chapter Contents
Reactivity to the Experimental Situation 1/3
Refers to research participants’ motives and perceptions influencing the response to the DV
Participant Effect Demand characteristics
Any of the cues available in an exp, such as
instructions, experimenter, rumors, experimental settings
Primary motive — positive self-presentation
Back to Chapter Contents
Reactivity to the Experimental Situation 2/3
Conditions producing a positive self-presentation motive Tedeschi, Schlenker, & Bonoma (1971)
P believe that others view their behavior as indicative of their true intentions, beliefs, or feelings
Exp. constructed P believe that others think their behavior is externally determined
Back to Chapter Contents
Reactivity to the Experimental Situation 3/3
Implication for research
Intertreatment interaction
Perception by P in different treatment groups that they can fulfill the positive self-presentation motive by responding in different ways
Intratreatment interaction
Perception by P in the same treatment group that they can fulfill the positive self-presentation motive by responding in different ways
Back to Chapter Contents
Experimenter Effect 1/3
Experimenter Effect Fig 8.2
Experimenter has motive of supporting the study hypothesis
Can unintentionally lead to recording errors
Back to Chapter Contents
Experimenter Effect 2/3
Ways experimenter may bias the study Experimenter attributes
Biosocial attributes
(e.g.) age, sex, race Psychosocial attributes
psychometrically determined characteristics
(e.g.) anxiety level, hostility, authoritarianism, intelligence, dominance, warmth
Situational factors
(e.g.) prior contact of P, naive or experienced
Experimenter expectancies (cont.)
Back to Chapter Contents
Experimenter Effect 3/3
Ways experimenter may bias the study Experimenter attributes Experimenter expectancies
Effect on experimenter
(e.g.) recording errors (biased, but low error rate) Effect on research participant Tab 8.2
Mediation of expectancies
— Handling in animal research
— Nonverbal communication in human studies Magnitude of expectancies—can exist in animal and
human research and can be greater than the IV Fig 8.3
Back to Chapter Contents
8.2 External Validity 1/4
Definition—generalizing across people, settings, treatment variations, outcomes and times
Represents a test of interactions
Threats to external validity Population validity
Ecological validity
Temporal validity
Treatment variation validity
Outcome validity
Back to Chapter Contents
8.2 External Validity 2/4
Population ValidityThe extent to which the results of a study can be generalized to the larger population Two-step inferential process Fig 8.4 Tab 8.3
Sample Experimentally accessible population
Target population College students as P
less emotionally and impulsively in laboratory studies Selection by Treatment Interaction
if interaction exists
cannot generalize to the target population
Back to Chapter Contents
8.2 External Validity 3/4
Ecological ValidityThe extent to which the results of a study can be generalized across settings or environmental conditions
Temporal ValidityThe extent to which the results of an experiment can be generalized across time Seasonal variation
A variation that occurs at regular time intervals Cyclical variation
A regular variation that occurs within people and other organisms
(ex) circadian rhythm, MC
Back to Chapter Contents
8.2 External Validity 4/4
Treatment Variation Validity (IV)The generalizability of results across variation of the treatment
Outcome Validity (DV)The generalizability of results across different but related dependent variables
Back to Chapter Contents
8.3 Cautions in Evaluating the External Validity of Experiments
Some studies conducted to increase knowledge and not to generalize to a real-life situation
Some studies assess a theoretical process
Moving out of the laboratory does not insure generalization
Back to Chapter Contents
8.4 Relationship between Internal and External Validity
Relationship between internal and external validity—often an inverse relationship
Back to Chapter Contents