8. Multi-Factor Designs
Transcript of 8. Multi-Factor Designs
8. Multi-Factor Designs
Chapter 8. Experimental Design II: Factorial Designs
1
• Identify, describe and create multifactor (a.k.a. “factorial”) designs
• Identify and interpret main effects and interaction effects
• Calculate N for a given factorial design
Goals
2
• As experimental designs increase in complexity:
• More information can be obtained.
• Care in design becomes ever more important.
• Designs with multiple factors and levels:
• Allow detection of interaction effects
• Allow detection of non-linear effects
• Involve more complexity around potential sequence effects and equivalent groups problems
Complexity and Design
3
8.1 Describing Multi-Factor Designs
4
• Have more than one IV (or factor). a.k.a. “factorial design”
• Described by a numbering system that gives the number of levels of each IV Examples: “2 × 2” or “3 × 4 × 2” design
• Also described by factorial matrices
Multi-Factor Designs
5
• Number of digits = number of IVs:
• “3 × 3” or “5 × 2” means two IVs.
• “2 × 2 × 2” or “3 × 4 × 2” means three IVs.
• Value of each digit = # of levels in each IV:
• 3 × 3 means two IVs, each with three levels.
• 3 × 4 × 2 means three IVs with 3, 4 and 2 levels, respectively
Numbering System for Factorial Designs
6
2 x 2 Factorial Design
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None Control ProzacPsycho-therapy
CBT CBT Combined Therapy
7
2 x 3 Factorial Design
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None Control Prozac
Psycho-therapy
CBT CBT CBT + Prozac
Psycho-therapy
EFT EFT EFT + Prozac
8
Going 3D: 2 x 2 x 2 Factorial Design
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None♀
Control
♀ProzacPsycho-
therapyCBT
♀CBT
♀Combo
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None♂
Control
♂ProzacPsycho-
therapyCBT
♂
CBT
♂
Combo
MaleFemale
9
2 x 2 x 3 Factorial Design
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None♀
Control
♀Prozac
Psycho-therapy
CBT ♀ CBT♀CBT +
ProzacPsycho-therapy
EFT ♀ EFT♀ EFT +
Prozac
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None♂
Control
♂
Prozac
Psycho-therapy
CBT ♂ CBT♂ CBT +
ProzacPsycho-therapy
EFT ♂ EFT♂ EFT +
Prozac
MaleFemale
10
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None♀Intro
Control
♀Intro
Prozac Psycho-therapy
CBT♀Intro
CBT
♀Intro
Combo
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None♂ Intro
Control
♂Intro
Prozac Psycho-therapy
CBT♂Intro
CBT
♂ Intro
Combo
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None♀ Extro
Control
♀ Extro
ProzacPsycho-therapy
CBT♀Extro
CBT
♀ Extro
Combo
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None♂Extro
Control
♂Extro
Prozac Psycho-therapy
CBT♂Extro
CBT
♂Extro
Combo
MaleFemaleIn
trov
erts
Extr
over
ts
11
• Level: One level of one IV. A row or column in the Factorial Matrix. Also, for 3+ IVs, one of the sub-matrices
• Condition: A particular combination of one level of each IV. One cell in the Factorial Matrix.
• In single-factor designs: level = condition
Levels vs. Conditions
12
Placebo Level of Drug Therapy IV
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None Control Prozac Psycho-therapy
CBT CBT Combo
13
Prozac Level of Drug Therapy IV
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None Control Prozac Psycho-therapy
CBT CBT Combo
14
None Level of Psychotherapy IV
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None Control Prozac Psycho-therapy
CBT CBT Combo
15
CBT Level of Psychotherapy IV
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None Control Prozac Psycho-therapy
CBT CBT Combo
16
One-factor Designs
Study TimeStudy Time
2 Hours 5 Hours
Study TimeStudy TimeStudy TimeStudy Time
2 Hours
3 Hours
4 Hours
5 Hours
2-level
Multilevel
17
Discussion / Questions
• Why are the terms level and factor interchangeable in a single-factor design?
• How many IVs are there in a 3×2×2 design? How many levels of each IV? How many total conditions?
18
8.2 Interpreting Data From Multi-Factor Designs
19
• Two types of effects can emerge in multi-factorial designs:
• Main Effects: When one IV has an effect on its own. That is, the mean for some pair of levels of the IV differ significantly from one another.
• Interaction Effects: When the effect of one IV is different for different levels of another IV.
• These are NOT mutually exclusive
Interpreting Data from Factorial Designs
20
A Simple 2x2 Design
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None Control Prozac Psycho-therapy
CBT CBT Combo
21
Main Effect of Psychotherapy
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None (Control+ Prozac ) / 2
(Control+ Prozac ) / 2Psycho-
therapyCBT (CBT + Combo)
/ 2(CBT + Combo)
/ 2
We collapse across the levels of all other IVs to evaluate a main effect
22
Main Effect of Drug Therapy
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None(Control+
CBT ) /2
(Prozac + Combo)
/2
Psycho-therapy
CBT
(Control+ CBT )
/2
(Prozac + Combo)
/2
We collapse across the levels of all other IVs to evaluate a main effect
23
Numerical Example
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None 12 ± 2 18 ± 1Psycho-therapy
CBT 17 ± 1 23 ± 3
24
Main Effect of Psychotherapy?
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None (12+18)/2 = 15(12+18)/2 = 15Psycho-therapy
CBT (17+23)/2 = 20(17+23)/2 = 20
25
Main Effect of Drug Therapy?
Drug TherapyDrug Therapy
Placebo Prozac
Psycho-therapy
None 12+172
14.5
18+232
20.5
Psycho-therapy
CBT
12+172
14.5
18+232
20.5
26
Numerical ExampleDrug TherapyDrug Therapy
Placebo Prozac µ ∆
Psycho-therapy
None 12 ± 2 18 ± 1 15 -6Psycho-therapy
CBT 17 ± 1 23 ± 3 20 -6
µ 14.5 20.5
∆ -5 -5
27
Numerical ExampleDrug TherapyDrug Therapy
Placebo Prozac µ ∆
Psycho-therapy
None 12 ± 2 18 ± 1 15 -6Psycho-therapy
CBT 17 ± 1 30 ± 3 20 -13
µ 14.5 20.5
∆ -5 -12
28
Evidence of Interaction
Discussion / Questions
• In a 3x3x2 design, how many potential main effects are there? How many IVs would you collapse across to evaluate each main effect?
29
• Multi-factorial experiments manipulate several IVs to see if their effects interact
• Example Question: Does gender interact with psychotherapy in affecting depression?
• Two IVs:
• Gender. 2 Levels = male; female
• Psychotherapy. 2 levels: control (none); experimental (therapy)
• One DV: Depression (measure = BDI)
Example Multi-Factorial Experiment
30
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
31
Another 2-Factor Design, 3 Levels Per Factor
ArousalArousalArousal
Low Med High
Task Difficulty
Easy LowEasy
MedEasy
HighEasy
Task Difficulty
Average LowAverage
MedAverage
HighAverage
Task Difficulty
Hard LowHard
MedHard
HighHard
32
Another 2-Factor Design, 3 Levels Per Factor
ArousalArousalArousal
Low Med High µ ΔLM ΔMH ΔLH
Task Difficulty
Easy 40 40 40 40 0 0 0
Task Difficulty
Avrge 15 30 15 20 15 -15 0Task Difficulty
Hard 8 5 2 5 -3 -3 -6
µ 21 25 19
ΔEA -25 -10 -25
ΔAH -7 -25 -13
ΔEH -32 -35 -38
33
Results: 3x3 Design
!"
#!"
$!"
%!"
&!"
'!"
()*" +,-./0" 1.23"
!"#$%#&
'()"*
+#%,-'.*
4567"
89,:52,"
15:-"
34
3x3 Results: Main Effects, No Interaction
ArousalArousalArousal
Low Med High µ ΔL
M
ΔM
H
ΔLH
Task Difficult
y
Easy 30 40 50 40 10 10 20
Task Difficult
yAvrge 15 25 35 25 10 10 20
Task Difficult
y
Hard 6 16 26 16 10 10 20
µ 17 27 37
ΔEA -15 -15 -15
ΔAH -9 -9 -9
ΔEH -24 -24 -2435
3x3 Results: 2 Main Effects, No Interaction
!"
#!"
$!"
%!"
&!"
'!"
(!"
)*+" ,-./01" 2/34"
!"#$%#&
'()"*
+#%,-'.*
5678"
9:-;63-"
26;."
36
• If one IV has an effect--that is, there’s a significant effect of going from one level of that IV to another, while ignoring (“collapsing across”) all other IVs--then that IV is said to produce a “main effect”.
• If the effect of one IV differs depending on the level of another IV, there’s an interaction.
Interpreting Data from Factorial Designs
37
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'!"
()*+,)-"""""""""""""
.*)"+/0,1234"
5/0,123"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'38
The Importance of Interactions
• Interpretation of interaction fx overrides interpretation of main fx
• Example: What’s most important in these results: Main effect of gender? Main effect of therapy? Interaction of the two?
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
If the gender factor is ignored, the therapy seems to simply be effective for all people. But this is not true. It is effective for females only.
39
X-Way Interactions
• When there are 2 IVs, a 2-way interaction is possible,with 3 IVs, may have a 3-way interaction, etc.
• 3-way interaction means the 2-way interaction changes depending on a 3rd variable.
40
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
Introverts Extroverts
Introverts Extroverts
Introverts Extroverts
41
Discussion / Questions
42
8.3 Mixed Multi-Factor Designs
43
All Participants (N = 20)
Condition 1(n = 10)
Condition 2(n = 10)
Review: Between-Subjects Design
44
All Participants (N=10)
Level 1 (N = 10)
Level 2 (N = 10)
Review:Within-Subjects Design
45
• With multiple factors/IVs, one can mix different kinds of variables (within/between; subject/manipulated, etc.)
• If all IVs are within-subjects then the design is “fully within”
• If all IVs are between-subjects then the design is “fully between”
• Otherwise, it’s a “mixed” design
Within, Between & Mixed Multi-Factor Designs
46
All Participants (N = 20)
Condition A1B1 (n=5)
Condition A2B1 (n=5)
Condition A1B2 (n=5)
Condition A2B2 (n=5)
2x2 Fully Between Subjects Design
47
All Participants (n = 20)
Condition A1B1(n = 20)
Condition A1B2(n = 20)
Condition A2B1(n = 20)
Condition A2B2(n = 20)
2x2 Fully Within
Subjects Design
Note that orders are not shown, there would be 24 for a fully-counterbalanced design!
48
All Participants (20)Level B1 (10) Level B2 (10)
A1B1(10)
A2B1(10)
A1B2(10)
A2B2(10)
2x2 Mixed Design
49
• a.k.a., Repeated-measures factorial design.
• All subjects are run through all conditions (i.e., all cells of the factorial matrix).
• Same advantages/disadvantages as single-factor repeated measures design
Fully Within-Subjects Factorial Design
50
• Question: Is face recognition more impaired by inversion than object recognition?
• Method
• Subjects are 20 undergraduates
• Materials are pictures of 25 famous faces and 25 common objects, either inverted or not. (So 100 images in all).
Example Experiment 1:Fully Within-Subjects
51
• Design: 2x2 Fully within-subjects factorial, with factors being Type of Image (Face or Object) and View (upright or inverted).
• Procedure: All 20 subjects are shown all 100 images several times in random order and asked to identify each as quickly as possible. Repeated-measures factorial design.
• DV is reaction time to name picture.
Example Experiment 1
52
Image TypeImage Type
Face Object
View
Upright
View
Inverted
53
• Expected results: RT will be higher for inverted images than upright ones (main effect). But this effect will be greater for faces (interaction).
• Implications: Implies that there’s something different about how people process faces as compared to objects
Example Experiment 1
54
Possible Results
!"#!!"$!!"%!!"&!!"'!!"(!!")!!"
*+,-./0" 1234,045"
!"#$%&'%()*"+,%
-./)010*%23/"$.#./4$%
67849" :;<4809"
55
• Each subject run through only one condition (i.e., one cell of the factorial matrix)
• If all IVs are subject variables, you have a Nonequivalent groups factorial design
• If all IVs are manipulated, decide how equivalent groups are formed:
• Random assignment: Independent groups factorial design
• Matching: Matched groups factorial design
Fully Between-Subjects Factorial Designs
56
• Question: Same as before, “are faces more affected by inversion than objects?”
• Method
• Subjects are 80 undergraduates (note higher N than within-Ss design).
• Materials: Same as before, 25 pictures of faces, 25 pictures of objects, shown both upright and inverted.
Example Experiment 2: Fully Between-Subjects
57
• Design 2×2 fully between-subjects factorial design. Assign subjects randomly to one of four groups of 20. Independent groups factorial design.
• Procedure: Each group sees 25 pictures (upright faces, inverted face, upright objects, or inverted objects).
Example Experiment 2
58
Image TypeImage Type
Face Object
View
Upright
View
Inverted
59
Discussion / Questions
60
• At least one IV within-subjects and one between-subjects.
• Subjects run through all levels of some IVs, but only single level of other IVs. That is, each subject goes through one row or column of the factorial matrix.
• Random assignment, matching, counterbalancing can all be used.
Mixed Factorial Designs
61
• Question: Is face recognition more impaired by inversion than object recognition?
• Method
• Subjects are 40 undergraduates (note higher N than fully within, but lower than fully between).
• Materials are pictures of 25 famous faces and 25 objects, either inverted or not.
Example Experiment 3:Mixed Factorial Design
62
• Design: 2x2 Mixed factorial with factors being Type of Image (face or object, within) and View (upright or inverted, between)
• Procedure: 20 subjects are shown the 50 inverted images (25 faces and 25 objects), while 20 other subjects are shown the 50 upright images (25 faces, 25 objects).
Example Experiment 3
63
Image TypeImage Type
Face Object
View
Upright
View
Inverted
64
• “Person by Environment”
• Variety of fully-between or mixed factorial design
• At least one subject IV (person) and at least one manipulated IV (“environment”)
PxE Factorial Designs
65
• Question: Does the effect of assigned study style interact with preferred study style?
• Method
• Person IV: Ss assigned to groups based on preferred study style: Crammers or Distributers. This is a subject IV
• Enviro IV: Half of subjects in each above group are assigned to study by cramming or by distributing study. This is manipulated
Example Experiment 4: PxE Design
66
Possible Results
Preferred Style (subject)Preferred Style (subject)
Crammer Distributer
Assigned Style (manipulated)
Cramming 65 65
Assigned Style (manipulated)
Distributing 80 90
67
Possible Results
!"#
$"#
%"#
&"#
'"#
(""#
)*+,,-./# 0-12*-342-./#
!"#$%&'"
()*+,-,."/$*'0)%)*10"
)*+,,5*1# 0-12*-3425*1#
Assigned Study Style
68
• Cannot draw causal links for the subject variables, can draw causal links for the manipulated (”environment”) variable.
• So a causal link can be established for assigned style but not preferred style.
• Cannot draw causal links for interaction effects.
Interpreting Results From PxE Designs
69
Example 2x3x2 StudyCaspi et al., 2007, PNAS, 104 (47), 18860-18865
70
How Many Participants?
• If I need 50 participants per cell in a 2×2 factorial design, what is the total N?
• What if the design is fully within?
• What if the design is mixed?
• Answer the same questions for a 3×2×3 design with 10 participants per cell.
71
Analyzing Data From Multi-Factor Designs
• As for multi-level designs, multi-factor designs are generally analyzed via ANOVA procedures:
• Pre-tests for normality and other assumptions
• 2-way (or X-way) ANOVA/MANOVA/ANCOVA...
• Post-hoc tests to examine effects in greater detail
• Planned comparison techniques may also be involved
• Note that there are no well-established techniques for dealing with multi-factor ordinal-scale data
72
Discussion / Questions
73
8.4 Summary: Design Complexity
74
• Can’t detect non-linear effects.
• Can’t detect interactions.
• Involve only simple counter-balancing or simple equivalent groups problems.
Single-Factor, 2-Level Experimental Designs
75
• Can detect non-linear effects
• Can’t detect interactions
• May involve relatively complex counter-balancing or equivalent groups problems
Single-Factor, Multilevel Designs
76
• Multi-factor Designs
• Can detect interactions and main effects
• Can detect non-linear effects where IVs have ≥ 3 levels
• May involve both complex counter-balancing and equivalent groups problems.
Multi-Factor Designs
77
Conclusion: Experimental Design
• Experiments and quasi-experiments are just one way of doing research
• True experiments (not quasi) allow conclusions about causality
• Next we will turn to observational research, which is simpler in some ways
78