Experimental Design

Threats to Internal Validity

1. No Control Group• Known as a “one-shot case study”

X O(IV) (DV)

Threats to Internal Validity

Example of One-Shot Case Study

Participants Brush with Crest Ask preference

Can’t tell if there was any effect of toothpaste type

X O

Concepts Basic Experiment

An IV with at least 2 levels Experimental Control Random Assignment Strengthens the Internal Validity - We can

tell if the IV caused a change in the DV!

Concepts Confound

When an uncontrolled variable is present in your experiment

You cannot identify whether the IV or the uncontrolled variable is causing the change in the DV

Weakens internal validity

Exercise Identify the confound!

Improving Internal Validity What can we do??? Ensure all aspects of the experiment

are equal except for the IV manipulation Add a good equivalent control group

(before the manipulation!) Any differences between groups can be

attributed to your manipulation

Improving Internal Validity Basic Control group design

Why does the control group have to be equivalent?

X OO

Threats to Internal Validity Nonequivalent Control group design

Selection differences - When participants who form the groups come from existing natural groups; a confound!

X OO

Overweight Volunteers

Traditional Dieters

Well Designed Experiments Posttest Only Design

X OO

Participantsrandom

random

Benefits: Ensures control and experimental groups are equal Limitation: Can’t demonstrate equality for sure; differences in mortality rate

Well Designed Experiments Pretest-Posttest Design

O X OO O

Participantsrandom

random

Benefits: You can see if mortality rate was due to any preexisting condition

Limitations: You might sensitize participants to your hypothesis

Design Variations

1. Independent Groups design aka Between Groups design 2 (or more) different groups determined

by Simple random assignment Matched Pairs random assignment

Used when you need to ENSURE equality on some measure

Matched Pairs Assignment Measure groups of

control variable of interest (e.g., IQ)

Arrange highest to lowest

Randomly assign 1st pair to each group; repeat for each pair

110109107107104103103101

9898

G1 G2

110107104103

98

109107103101

98

WholeSample

Means 104.4 103.6

Design Variations

2. Repeated Measures design aka Within Groups design Each person acts as their own control, so

fewer subjects needed Very sensitive to small differences since

both groups are identical on everything Problems???

Repeated Measures Design Order Effects

When the order in which the levels of the IV are presented affect the DV (threatens internal validity)

Practice Fatigue Contrast

Repeated Measures Design Overcome by

Increasing time interval between conditions counterbalancing

Randomly divide the sample into groups and administer the levels of the IV in reverse order

analyze all groups together

Repeated Measures Design Counterbalancing

1st 2nd

Sample

Group A

Group B

Alcohol Sober

Sober Alcohol

Repeated Measures Design Counterbalancing Problems:

The number of possible conditions dramatically increases the number of orders

2 conditions = 2 orders (2 x 1) 3 conditions = 6 orders (3 x 2 x 1) 5 conditions = 120 orders! At 30 Ss per condition, you need a LOT of

subjects

Repeated Measures Design Overcoming Counterbalancing

Problems: Latin Square Design

Special procedure for ensuring that each condition occurs at every position (1st, 2nd, etc.) and that each condition occurs before and after every other condition at least once.

Latin Square

1st 2nd 3rd 4th

Order 1 A B D C

Order 2 B C A D

Order 3 C D B A

Order 4 D A C B

Between Groups vs Repeated Measures Repeated measures advantages

Requires fewer participants Reduces differences between groups -

better able to detect small differences Between Groups advantage

No order effects

Between Groups vs Repeated Measures Also consider

Generalization - sometimes we experience in the real world variables alone, but sometimes together - choose the design that mirrors the outside world

Conditions with permanent changes don’t lend themselves to repeated measures - the sample is “spoiled” in the first condition

Conclusions True experiments improve internal validity

Use equivalent control groups Random assignment; matched pairs

assignment Between Subjects vs Repeated Measures

designs Counterbalancing controls for order effects

in repeated measures designs