Chapter 12: Quasi-Experimental Designs

• When researchers can not manipulate the independent variable, rather it is a grouping variable (gender, age, disability) and equivalence between the groups can not ensured

• Researchers can not randomly assign participants to groups thus lack control over extraneous variables

• Quasi-independent variable: is not a true independent variable but usually occurs naturally or can not be manipulated. • Researchers still look for effect of the quasi-

independent variable.

• Quasi-experimental designs usually have lower internal validity than true experiments.

Types of Quasi-Experimental DesignsPretest-Postest designs • Test participants before an after the quasi-

independent variableOne group: measure participants before and after the

quasi-independent variable. Only have one group of participants (those that experienced the quasi-independent variable)• Test reading before children at school X start reading

program and then test their reading after they finish the reading program.

• O1 X O2

Threats to internal validity• Maturation: students may have matured over the

reading program. They may be better at reading just because of time and not due to the program.

• History Effects: something other than the independent variable may have occurred between the pretest and posttest.

• Pretest sensitization: taking the pretest may change the participants reaction to the posttest.

• Regression to the mean: Tendency for extreme scores on pretest to regress (move towards) the mean on a subsequent test (posttest).

• If participants are selected because they have extreme scores on the pretest (e.g. select a school with very poor reading ability) there may be other factors due to measurement error that resulted in such low scores at the pretest (tired, bad day etc.) that may have slightly deflated their scores.

• Measurement error causes extreme scores to be biased in the extreme direction (away form the mean).

• So when you test them a second time it is unlikely that you will have those same factors that may have deflated their scores and their scores will increase a bit and make it look like the program is have an effect.

Reading Skills Before and After New Reading Program

0

10

20

30

40

50

60

Before After

Me

an

Re

ad

ing

Sc

ore

Nonequivalent Control Group Design: • We cannot randomly assign participants to control and

study group, so we select a control group that is similar to the group that gets the quasi-independent variable.

Posttest-only: measure both groups after one group has received the treatment.• Measure reading in School A and School B after School A

has participated in the reading program.

Quasi-experimental group X O

Nonequivalent control group -- O

• Selection bias: we do not know whether the two groups were similar before the intervention

Pretest-Posttest design: Test both groups before one group gets the intervention, then test both groups again after one group gets the intervention (quasi-independent variable)

Quasi-experimental group O1 X O2Nonequivalent control group O1 -- O2• Allows researchers to see if the two groups

scored similarly on the dependent variable before the introduction of the treatment.

• To determine if the quasi-independent variable had an effect you want scores to change between pretest and posttest ONLY for the Quasi-experimental group and NOT for the Nonequivalent control group.

Time Series Designs• Measure the dependent variable many times

before and after the quasi-independent variable is introduced.

Simple interrupted time series design• Researchers make a series of observations of the

dependent variable before and after the treatment is introduced.

O1 O2 O3 O4 X O5 O6 O7 O8• Evidence for a treatment effect occurs when there

are abrupt changes in the time-series data at the time the treatment was implemented.

0

5

10

15

20

25

30

1970 1971

Percentage of cavities after the introduction of fluoride into toothpaste in 1970

0

5

10

15

20

25

30

35

1967 1968 1969 1970 1971 1972 1973

Percentage of cavities after the introduction of fluoride into toothpaste in 1970

0

5

10

15

20

25

30

35

1967 1968 1969 1970 1971 1972 1973

Percentage of cavities after the introduction of fluoride into toothpaste in 1970

• This design helps to distinguish changes due to maturation from the quasi-independent variable

• Contemporary History: Observed effect could still be due to another event that occurred at the same time as the quasi-independent variable

• Perhaps the electric toothbrush was introduced in 1970, or there was a major TV add campaign that promoted brushing teeth.

Interrupted time series with a reversal• Researchers measures the dependent variable

before and after the treatment is introduced and then again after the treatment is removed

O1 O2 O3 O4 X O5 O6 O7 O8 -X O9 O10 O11

• We can see what happens to the dependent variable after the quasi independent variable is introduced and then again after it is removed.

• If the quasi-independent variable was really having an effect we would expect performance to change back to normal after it is removed

0

5

10

15

20

25

30

35

1968 1969 1970 1971 1972 1973

Percentage of cavities after the introduction of fluoride into toothpaste in 1970

Limitations:• Researchers may not have the ability to remove the

quasi-independent variable• remove fluoride from toothpaste, remove a seatbelt

law

• Some effects of the quasi-independent variable may remain even after it is removed • If you did a time series study before and after introduction

of reading program, and then removed program, reading may not decrease, the children may not regress because they did learn to read.

• Removal of the quasi-independent variable may produce effects that are not due to the quasi-independent variable

Control Group Interrupted time series• Measure more than one group on several

occasions, but only one group receives the quasi-independent variable.

O1 O2 O3 O4 X O5 O6 O7 O8

O1 O2 O3 O4 -- O5 O6 O7 O8

• Helps to rule out history effects, and we can be more certain the a change was due to X rather than an outside influence.

• Could still have a local history effect.

Longitudinal Designs• Time serves as the quasi-independent variable• Commonly used in developmental research• Allows researchers to eliminate generational

effects (when effects differ depending on the era in which people grew up). In longitudinal research you are studying people of the same generation over time.

O1 O2 O3 O4 O5 O6 O7 O8

• Allows researchers to examine how individuals change with age (not just group differences).

Limitations:

• Difficult to obtain samples who are willing to be repeatedly tested over time.

• Difficult to keep track of participants over time (attrition may occur).

• Takes a lot of time and money.

Program Evaluation• Used to assess effectiveness of interventions (or

programs) and provide feedback to the administrators

• Assess needs, process, outcome, and efficiency of social services.

• Considered applied research.

Evaluating Quasi-Experimental Designs

• The presumed cause usually precedes the presumed effect.

• These designs do allow researchers to determine if the two variables covary together.

• BUT they can not eliminate effects of extraneous variables and ensure randomization.

• Examine individual participants rather than a group of participants.

• Idiographic approach: describe, analyze, and compare individual behavior

• Nomothetic approach: Describe, analyze, and compare behavior across individuals and make broad generalizations to a group.

• Many research areas started in single case-research (Ebbinhgaus’s memory research on himself, Skinner and Pavlov research)

Chapter 13: Single-Case Research

Three main criticisms of group designs

Error variance: error variance in group data does not always reflect variability in behavior (rather is due to the design).

• Group designs examine inter-participant variance which is across or between participants (individual differences)

• Rather single-case researchers emphasize intra-participant variance which is variability in an individual’s behavior.

Generalizability: single case researchers suggest that group designs have limited generalizability.

• Group designs usually reflect an average of the participants behavior which may not represent the response of any particular participant.• Average number of children adults have 2.1• Average anxiety score is 10 (but most people may be at

either high or low end 3-4 and 17-18).

Reliability: group designs may test an effect once, but do not always replicate it to see if the effect holds up and is reliable.

• Single-case researchers often test an effect in the same participant a few times (intraparticipant replication) or determine whether the same effect is found in a few other participants (interparticipant replication)

Single-Case Experimental Designs

ABA design: observe participants in absence of independent variable, baseline (A), then introduce independent variable, experimental period (B), then remove independent variable and observe behavior (A).

• Sometimes called a reversal design• Difficult to determine if some other event

occurring at the same time as the independent variable resulted in the effect

• The independent variable may produce permanent changes in a participants behavior, so it may not go back to baseline.

Multiple-I Designs• Present varying nonzero levels of the independent variable• ABC design: baseline (A), introduce IV (B), then remove this

IV and introduce another level of the IV (C).• ABACA: have baseline condition between each level of the

independent variable• Multiple Baseline Designs: two or more behaviors are

examined simultaneously. • After the baseline data, the researcher examines the effect of the

independent variable on both behaviors. Usually test to see if independent variable affects the hypothesized behavior.

Data Analysis• Graphic analysis: display all of a participants data

points (before and after independent variable) on a graph. Visually examine the graph to determine whether it looks like the independent variable produces an effect.

Uses of single-case designs:• Conditioning research (reinforcement and

punishment effect)• Behavior modification techniques (phobias)• Demonstrational purposes.

Critiques of single-case research

• Not necessarily generalizable because the participant is not usually chosen at random

• Difficult to study interactions among variables

• Ethical issues in ABA designs when the researcher may remove a very helpful treatment

Case Study Research• An intensive description and analysis of a single

individual, or sometimes a group• Usually gather a narrative description of

information about the individual(s). • Common when describing rare phenomena (rare

brain injuries or disorders, prodigies, assassins)

• Psychobiography: use psychological theories to understand lived of famous people (study Nixon)

• Used to make anecdotes and to illustrate general principles.

Limitations of Case Studies

• Very difficult to control extraneous variables. Usually unable to asses and rule out alternative explanations.

• Observer Biases