Experimental Design

True Experimental Designs

Random assignment Two comparison groups Controls threats to internal validity Strongest evidence for cause-effect

Quasi Experimental Designs

Lacks randomization or Lacks control group Accommodates for natural setting

Group Assignment

Completely randomized (between subjects) Randomized block design Repeated measures design (within subjects)

Selecting a Design

How many IV are being tested? How many levels does each IV have, and

are these levels experimental or control conditions?

How many groups of subjects are being tested?

Selecting a Design

How will subjects be selected, and how will they be assigned to groups?

How often will observations of responses be made?

What is the temporal sequence of interventions and measurements?

True Experimental Designs

Single factor designs for independent groups

One way design

Design 1A

Pretest-Posttest control group design R O1 X O2 O1 O2

Standard in Clinical research for cause -effect

Randomized controlled trial (RCT)

Design 1B

Two group pretest-posttest design O1 X1 O2 R O1 X2 O2

New vs. old

Difference between 2 treatments

Design 1C

Multigroup pretest-posttest control group design

O1 X1 O2 R O1 X2 O2

O1 O2 Threat to internal validity is attrition Threat to external validity is the interaction of

treatment and testing

Analysis of Design 1

Utilize change scores Interval-ratio data 2 groups unpaired t-test 3 groups ANOVA Ordinal data 2 groups Mann-Whitney U 3 groups Kruskal-Wallis analysis of

variance

Design 2

Posttest-only design R X1 O2

O2

True experimental design Internal validity is strong External validity is strong Analysis is identical to Design 1

Multifactor Designs for Independent Groups Design 3 Factorial

Design 4 Randomized Block

Design 5 Nested

Design 3 Factorial Design

2 or more Independent variables Has factor and levels 2 x 2 3 x 3 4 x 4

Two Way Factorial Design

Main effect Interaction effect Analysis is two way or

three way analysis of variance

Design 4: Randomized Block Design

Concern over an extraneous variable Attributed variable is crossed with an active

IV Analysis is 2 way analysis of variance or

multiple regression analysis

Design 5: Nested Design

Some attribute variables cannot be crossed

Repeated Measures Design

Design 6 One Way Repeated Measures Design

Design 7 Crossover Design

Design 8 Latin Square Design

Design 6: One Way Repeated Measures Design

All subjects exposed to all levels of one treatment variable

X1 O1

X2 O2

X3 O3

Internal threat is carry over Analysis one-way ANOVA

Design 7: Cross Over Design

Minimizes bias of test sequence Counterbalances treatment conditions Washout period R O1 X1 O2 ... O3 X2 O4

O1 X2 O2 ... O3 X1 O4

Analysis: Paired T-test or 2 way ANOVA Ordinal data Wilcoxon signed ranks test

Design 8: Latin Square Design

Sequencing of treatment order is concern

Randomizes treatment order

3 x 3 or 4 x 4

Multifactor Designs for Repeated Measures Design 9 Two-way design with two

repeated measures

Design 10 Mixed design

Design 9: Two-way Design with Two Repeated Measures Analysis: Two way analysis of variance

with two repeated measures to measure main and interaction effects

Design 10: Mixed Design

Analysis: 2 way anova with one repeated measure for main effect two way design with one repeated factor for interaction effects

Quasi Experimental Designs

Do not use random assignments

Potential bias of sample

Strong control over design

Quasi Experimental Designs

Design 11 One-group pretest-posttest design

Design 12 Time series design Design 13 Nonequivalent pretest-posttest

control group design Design 14 Nonequivalent posttest-only

control group design

Design 11: One-group Pretest-posttest Design O1 X O2

Threat to internal validity-no comparison group

Threat to external validity-potential interactions with selection

Analysis: Paired t-test for ratio/internal Wilcoxon signed ranks test for ordinal

Multigroup Design

Design 12 Time series design O1 O2 O3 O4 X O5 O6 O7 O8

Effect of tx on physiological or psychological variables over time

IV is time Threat to internal validity is history Variations 2 or more groups, tx withdrawal

Design 13: Nonequivalent Pretest-posttest Control Design O1 X O2

O1 O2

Threat to internal validity is interaction of selection with history and maturation

Analysis: Unpairted T-test, anova-interval or ratio

Mann-Whitney U-test-Ordinal

Design 14: Nonequivalent Posttest-Only Control Group Design

X O O

Threat to internal validity is selection bias and attrition

Analysis: Regression