Chapter 9Expanding on Experimental Designs:

Repeated Measures and Quasi-Experiments

Repeated Measures DesignsAdvantages to Repeated Measures Designs

Increasing efficiency in data collection

Increasing validity of data

Reducing error in measurements

Finding enough participants

Repeated Measures Designs

Repeated measures designs are sometimes not optimal?

Questions• How big is 9? • How big is 221?• Is 9 larger than 221?

Repeated Measures DesignsMethodIndependent Groups Design:Some participants responded to the question

How big is 9 on a scale of 1 to 10?Other participants responded to the question

How big is 221 on a scale of 1 to 10?

Repeated Measures Designs• Result: The mean rating of 9 was higher than

the mean rating of 221.

Source: Birnbaum, M. H. (1999). How to show that 9 > 221: Collect judgments in a between-subjects design. Psychological Methods, 4, 243-249. Copyright American Psychological Association. Used with permission.

Repeated Measures DesignsConclusion

• Participants compared 9 to small numbers, so it seemed large, relatively speaking.

• Participants compared 221 to very large numbers, so it seemed small, relatively speaking.

• A repeated measures design would avoid this type of result.

Repeated Measures Designs

Limitations to Repeated Measures Designs

Some research does not lend itself to repeated measures, like measuring young and old adults.

Studies using subject (participant) variables, like political affiliation, cannot make use of repeated measures because a person can realistically be in only one condition.

Sequence and order effects can be a problem if early treatments affect later treatments.

Repeated Measures DesignsOvercoming Sequence or Order Effects

Counterbalancing –Changing the order of conditions from one participant to avoid the problems of sequence and order effectsComplete counterbalancing—using all possible orders across participants (e.g., for conditions A, B, and C: ABC, ACB, BAC, BCA, CAB, CBA)Partial counterbalancing—using a subset of all possible orders across participants

Repeated Measures DesignsAdditional Problems with Repeated Measures

DesignsTransfer—participants change because they learn from one situation the next.

Symmetric Transfer—participants’ behaviors change equally with repeated measurements, no matter what treatments come first

Asymmetric transfer—participants’ behaviors change differently depending on what treatments come first

Data Analysis with Repeated Measures Designs

• When we compare two groups in a repeated measures design, we typically use a dependent measures t test (also called repeated measures t test or correlated groups t test)

• With multiple groups or multiple IVs, we typically use the Analysis of Variance

Quasi-Experimental Designs• When researchers create groups to compare

but cannot randomly assign participants to those groups, the design is called quasi-experimental.

• Quasi-experimental designs look like true experiments, but there is no random assignment or true manipulation of the IV.

• Without random assignment and manipulation of the IV, we cannot determine cause-effect relationships.

Quasi-Experimental DesignsThreats to Internal Validity Due to Participants

Selection threat—Groups may differ in important ways before the study even starts.

Maturation threat—Short- or long-term changes in participants may change their behaviors (e.g., boredom, physical development)

Attrition—People may drop out of the study; those who do may differ from those who remain

History threat—Some minor or major event may occur during a study that changes the participants’ behaviors.

Quasi-Experimental DesignsThreats to Internal Validity Due to Measurement

Instrumentation threat—Changes in the way the DV is measured due to changes in apparatus or in the way observers code data.

Testing threat—Changes in participants’ behaviors because they have undergone testing.

Statistical regression threat—Changes in participants’ scores on repeated testing because extreme scorers were included and their subsequent scores move toward the mean

Controversy: Sex and Death• Question: Are sex and death are psychologically

related, which Terror Management Theory suggests.

Controversy: Sex and DeathMethod

– Researchers measured people to see if they were low or high in neuroticism (a quasi-experimental variable, subject variable).

– They primed participants to think either of the romantic or of the physical aspects of sex (a true independent variable).

– The study involved a 2 x 2 design, with one quasi-experimental IV (level of neuroticism) and one true IV (type of prime)

Controversy: Sex and Death• Participants then completed word stems (e.g.,

COFF_ _ or SK_LL) to see if participants completed them with death-related words (COFFIN versus COFFEE or SKULL versus SKILL)

• Result: There was an interaction between the neuroticism level and type of prime.

– People high in neuroticism were more likely to generate death-related words when physically primed compared to people low in neuroticism and less likely to generate death-related words when romantically primed.

Controversy: Sex and Death

Conclusion• Sex and death are related among those high in

neuroticism.

Types of Quasi-Experimental Designs• One-Group Pretest-Posttest Design—Quasi-

experimental design in which a single group is tested, administered a treatment, then tested again.

Single group of participants Observation Treatment Observation

Types of Quasi-Experimental Designs

• Static-Group Comparison Design—Quasi-experimental design in which two groups of nonrandomly assigned participants differing in some pre-existing way are compared, with one receiving a treatment and the other no treatment

Nonrandom placement in Group 1 Treatment Observation

Nonrandom placement in Group 2 No Treatment Observation

Types of Quasi-Experimental Designs• Nonequivalent Control Group Design—Quasi-

experimental design in which two groups with pre-existing differences are compared in a repeated measures situation.

Nonrandom placement in Group 1 Observation Treatment Observation

Nonrandom placement in Group 2 Observation No Treatment Observation

Types of Quasi-Experimental Designs• Potential problem with Nonequivalent Control

Group Design• Participants may have pre-existing differences

such that they could show different patterns across time that you might not spot with simply two measurements.

Types of Quasi-Experimental Designs• Time Series Design—Quasi-experimental

design in which a researcher makes multiple observations over time, trying to discern patterns of behavior

Group of participants Observation Observation Observation

Types of Quasi-Experimental Designs• Interrupted Time Series Design—Quasi-

experimental design in which a researcher makes multiple observations over time, with a treatment at some point along the measurement continuum; there can be many observations before or after a treatment

Group Observation Observation Treatment Observation Observation

Types of Quasi-Experimental Designs• Replicated Interrupted Time Series Design—

Quasi-experimental design in which a researcher identifies two groups and makes multiple observations over time, with a treatment at different times for the two groups; there can be many observations before or after a treatment

Group 1 Observation Treatment Observation Observation Observation

Group 2 Observation Observation Observation Treatment Observation

Replicated Interrupted Time Series Design

• Question: Does antisocial behavior in the media lead to antisocial behavior in life?– In the early 1950s, Congress passed a law barring

the creation of new television stations in cities that did not already have them.

– Crime dramas were popular even in the early days of TV.

– So some cities had access to TV crime shows showing larceny, but other cities didn’t.

Replicated Interrupted Time Series Design

• Method: Collect data on the incidence of antisocial behavior (e.g., larceny) when cities did or did not have access to television that portrays antisocial behavior.– The researchers used a replicated interrupted

time series design.

Replicated Interrupted Time Series Design• Result: When cities did not have access to

crime shows on TV, the crime rate was lower compared to cities that had such access. When TV was later allowed in those cities, the crime rate rose.

0

2

4

6

8

10

Inci

denc

e of

Lar

ceny

46 48 50 52 54 56Year

Prefreeze

Postfreeze

Arrows indicate year of arrival of TV in the prefreeze and postfreeze cities.

Source: Hennigan, K. M., Del Rosario, M. L., Heath, L. Cook, T. D., Wharton, J. D., & Calder, B. M. (1982). Impact of the introduction of television on crime in the United States: Empirical findings and theoretical implications. Journal of Personality and Social Psychology, 42, 461-577. Copyright American Psychological Association. Used with permission.

Type of Quasi-Experimental Designs

• Conclusion: The arrival of media that portrays antisocial behavior is associated with more crime.

• The fact that crime rates rose twice, after each introduction of television, suggests that antisocial behavior like larceny (i.e., theft) results from exposure to that behavior on TV.

• The design is quasi-experimental, so we cannot determine causation with absolute confidence, although this design is very suggestive.