Chapter 6 class version b
description
Transcript of Chapter 6 class version b
Chapter 6: Research Validity
Continued10/2/2012
Road Map
• Turn in Reflection Assignment #1• Announcement about lab• Quick Review• Finish Chapter 6
About Lab
• Go to class• Turn in assignments• Starting group project this week—you can’t
contribute fully if you aren’t there!• Remember: can be dropped after 4 absences• If you fail lab, you fail the course• Pay close attention to detail
Quick Review
• Statistical Conclusion validity• Construct Validity– Threats: participant reactivity
experimenter effects (will discuss next)
Construct Validity
Criterion-Related Validity
Predictive Concurrent
DiscriminantConvergent
Content-Related Internal Structure
Threats to Construct Validity
• Factors that impact how well our operationalizations actually represent constructs
• Pg 171, Table 6.2 – long list of threats
• We will focus on two major ones:– Participant reactivity to the experimental situation– Experimenter effects
Participant Reactivity to the Experimental Situation
• Participants’ motives and perceptions
• Demand characteristics
• Positive self-presentation
Instruction set #1
We want to see how well you are able to learn the following sets of letters. Letters will appear in groups of 3 to 7, and each letter will appear on the screen for 1 second. Following the presentation of the letters, …
Instruction Set #2
In the following task, you will be presented with groups of letters ranging from 3 to 7 letters. Each letter will appear on the screen for one second. Your task is to…
Experimenter effects
• Researcher actions and characteristics that influence the responses made by the research participant
• Expectancies– Clever Hans
• Attributes– Biosocial– Psychosocial– Situational factors
Clever Hans
I Math
Internal Validity
• The extent to which we can accurately infer that the independent and dependent variables are causally related
Observed Effect (DV)
Independent Variable
Causally Related
Independent Variable
Observed Effect (DV)
Cause must precede effect
Cause and effect are related (covary)
No other explanation is plausibleNo other explanation is plausible
Special Considerations
• Extraneous variables
• Confounding extraneous variables
• Control
Threats to Internal Validity• History • Maturation • Instrumentation • Testing • Regression Artifact • Attrition • Selection • Additive and Interactive Effects
History
• Any event occurring after the study begins that could produce the observed outcome
• Differential history: only one group experiences history event
• Note what history is NOT
Maturation
• Changes in biological and psychological conditions that occur with the passage of time – Factors within the individual
• Example: Head Start program and achievement over a school year
Instrumentation
• Changes in the assessment/measurement of the dependent variable
• Example: multiple observers and interviewers
Testing
• Changes in a person’s score on the second administration of a test are a result of previously having taken the test
• Example: pre-test and post-test on memory task
Regression Artifact
• a.k.a. regression toward the mean
• The tendency for extreme scores to become less extreme on a second assessment
Attrition
• Participant drop-out– Don’t show up for appointment– Decide to discontinue study
• Differential attrition is especially problematic
Selection
• The choice of participants for the various treatment groups based on different criteria – NOT random assignment
Additive & Interactive Effects
• The combined effect of several threats to internal validity
External Validity
• The extent to which the results of the experiment can be generalized across variations in:– people– Settings– Treatments– Outcomes– Time
What Limits External Validity?
• Lack of random selection/sampling
• Chance factors
• IV-DV relationship varies across levels of another variable
5 main categories of external validity
• Population Validity• Ecological Validity• Treatment Variation Validity• Outcome Validity• Time Validity
Population Validity
• The extent to which the results of a study can be generalized to the larger population.
• Target population: the population to which we want to generalize the results
• Accessible population: the population of research participants available to the investigator
Ecological Validity• The extent to which the results of a study can be
generalized across settings or environmental conditions
• Which research setting receives the most criticism here?– laboratory
Temporal Validity• The extent to which the results of an experiment can be
generalized across time
Treatment Variation Validity
• The generalizability of results across variation of the treatment
Outcome Validity
• The generalizability of results across different but related dependent variables.
Balancing Internal and External Validity
• Inverse relationship
• The conditions that increase internal validity (control) decrease external validity