1

Handout Eight: Two-Way Between-Subjects Design

with Interaction- Assumptions, & Analyses

EPSE 592Experimental Designs and Analysis in

Educational ResearchInstructor: Dr. Amery Wu

Today, we will introduce the analysis and assumptions of a two-way between-Subjects designs.This design explains/predicts continuous data using two independent variables, each with two or more levels (i.e., groups) where subjects are independent across the groups.

Where We Are TodayTwo-way Between-Subjects Factorial Design

with InteractionMeasurement of Data

Continuous Categorical

Type ofthe

Inference

DescriptiveA B

InferentialC D

2

3

The Design A two-way between-subjects ANOVA is an

inferential statistical method includes two factors (independent variables), each factor has two or more levels (groups or

categories) subjects are independent across the levels for each

factor.Data Assumptions

Data assumptions are the same as those of one-way ANOVA, except that assumptions are made about the RxC sub-samples.

R is the # of levels of the first factor the C is the # of levels of the second factor.

Review of Two-way Between-Subjects Factorial Design

4

Two-way Between-Subjects Factorial DesignPartitioning the Total Sum of Squares by the 2

Factors

SSw-g

SSb-gA

SSb-gBSSb-gA*B

2 Factors

5

Statistical Model of Two-way Factorial ANOVAIn a two-way ANOVA, two independent variables, A and B,are involved, which are termed “factors” in ANOVAHowever, the data (DV) is being modeled (explained orpredicted) by three terms: A, B, and their interaction effect A*BData = Model + residual

Y = intercept + aA + bB + cA*B + residual

Questions: What is the “model” for a one-way ANOVA? Write the equation.

6

Variable A is treated as the primary cause, of which the effect on the Y (DV) is the major research interest. An analogy is the effect of a switch on the light.

Variable B is interpreted as a moderator that adjusts the effect of A on Y. An analogy is a dimmer adjusts the effect of the switch on the light. Namely, the effect of A on Y is dependent on the level (or amount) of B

The choice of which is the primary cause should be guided by the research question and the substantive theory. Interaction Effect

Interpreting the Results- Randomized Experimental Data

Two-way Between-Subjects Factorial ANOVA

7

The relationship between A and Y (DV) is dependent on the level (or amount) of B.

Variable B moderates the relationship between A and Y.

The choice of whether A or B should be considered as the moderator depends on the research question and the substantive theory.

Interpreting the Results - Observational Data

Two-way Between-Subjects Factorial ANOVA

Interaction Relationship

Question: what are the differences between this figure and the one on the last slide?

8

One Way ANOVA (One Independent Variable A)SStot = SSbgA + SSwg, or simplySStot = SSA + SSr

Two Way ANOVA (Two independent variables A & B)SStot = SSbgA + SSbgB + SsbgA*bgB + SSwg or simplySStot = SSA + SSB + SSAB+ SSr

Comparing One-Way and Two-Way Factorial ANOVAUsing Data = Model + Residual

Interpreting a Two-way Factorial ANOVA-- When the Interaction Effect is Non-significant

ANOVA TableSS df MS F p

Instruction 35 2 17.50 9.55 <0.05Sex 7.5 1 7.50 4.09 >0.05Instruction*Sex 5 2 2.50 1.36 >0.05Error (within) 44 24 1.83Total 91.5 29

When the interaction effect is not significant, run the post hoc paired-comparisons for the main effect(s) that are significantIf both main effects are significant; hence, more multiplepaired comparisons are involved, a more conservative post-hoc t-test should be considered to protect from Type-I error. (see the Word document: GLM post hoc comparison.doc)

Instruction by Gender

4

5

6

7

8

9

10

A B C

Instruction Method

Perfo

rman

ceFemale

Male

An example Plot for No Interaction Effect

When the two separate lines (here representing the moderator) are parallel, it signals that there is no interaction effect.However, one must NOT use such a plot to infer about thepopulation interaction effect as many textbooks suggest because it is descriptive and does not test whether the parallelism occurred only by chance.

Lab Activity: Post-hoc Comparison for Main effects

SPSS Output: Post-hoc Comparison for Main effects

Multiple Comparisons

Dependent Variable: performanceTukey HSD

-2.00* .606 .008 -3.51 -.49-2.50* .606 .001 -4.01 -.992.00* .606 .008 .49 3.51-.50 .606 .691 -2.01 1.01

2.50* .606 .001 .99 4.01.50 .606 .691 -1.01 2.01

(J) instruction231312

(I) instruction1

2

3

MeanDifference

(I-J) Std. Error Sig. Lower Bound Upper Bound95% Confidence Interval

Based on observed means.The mean difference is significant at the .05 level.*.

MeansInstruction Method

A B C Marsex

Sex Male 6 7 8 7Female 6 9 9 8

Marinstruction 6 8 8.5 7.5

Number of SubjectInstruction Method

A B C Marsex

Sex Male 15 15 15 45Female 15 15 15 45

Marinstruction 30 30 30 90

Using the same research context (IV: instruction and gender;DV: performance), all the means of the main effect and interaction cells are the same as before. However, number of subjects increased from 5 to 15 for each cell.

14

Students’ performance was rated by their clinical supervisors on a scale of 0-10. Mean ratings of the

groupsare tabulated below.

An Example to Contextualize Learning

Two-way Between-Subjects Factorial ANOVA

Research Questions:1.Does the instruction method have an effect on

students’ performance?2.Is there a sex difference in students’

performance?3.Does the effect of instruction method on the

students’ performance depend on the sex of the students?

Means    Instruction Method      A B C Marsex

Sex Male 6 7 8 7  Female 6 9 9 8

  Marinstruction 6 8 8.5 7.5

15

Computing The F Statistic and the F TestTwo-way Between-Subjects Factorial ANOVA

Lab Activity: See Excel File “Two Way ANOVA with Interaction.xls.”

16

Calculating the F Statistic - SS of Total

Calculating Total Sum of Squares

Subject ID Sex Instruction performance Deviation2

1 female 1 4 12.25

2 female 1 5 6.25

3 female 1 6 2.25

4 female 1 7 0.25⁞ ⁞ ⁞ ⁞ ⁞

87 male 3 7 0.25

88 male 3 8 0.25

89 male 3 9 2.25

90 male 3 10 6.25

SS Total       274.5

df   Total N-1   89

17

Calculating the F Statistic - SS for the Main effects of Instruction Method

Means    Instruction Method      A B C Marsex

Sex Male 6 7 8 7  Female 6 9 9 8

  Marinstruction 6 8 8.5 7.5

SS instruction 30*(6-7.5)(6-7.5)+30*(8-7.5)(8-7.5)+30*(8.5-7.5)(8.5-7.5) 105df = C-1 2

18

Calculating the F Statistic - SS for the Main effects of Sex

Means    Instruction Method      A B C Marsex

Sex Male 6 7 8 7  Female 6 9 9 8

  Marinstruction 6 8 8.5 7.5

SS Sex 45*(7-7.5)*(7-7.5)+45*(8-7.5)*(8-7.5) 22.5df = R-1 1

19

Testing the interaction effect is, in essence, testing whether there is a true difference in the difference.

Calculating the F Statistic- SS for “instruction by sex” interaction effect

    Instruction Method        A B C  

Sex Male 6 7 8    Female 6 9 9  

  sex difference 0 -2 -1  

Sex diffInstruction A 0Instruction B -2Instruction C -1Mean -1

SS difference=15*(0-(-1))*(0-(-1))+15*(-2-(-1))*(-2-(-1))+15*(-1-(-1))*(-1-(-1)) 30SS interaction = Half SS difference 15df = (R-1)*(C-1) 2

20

Calculating the F Statistic- SS for within Groups

Calculating SS Within Groups

Subject ID Sex Instruction Performance Deviation2

1 female 1 4 42 female 1 5 13 female 1 6 04 female 1 7 15 female 1 8 46 female 1 4 47 female 1 5 18 female 1 6 09 female 1 7 110 female 1 8 411 female 1 4 412 female 1 5 113 female 1 6 014 female 1 7 115 female 1 8 4

    mf1 6 30 SSf1

16 male 1 4 417 male 1 5 118 male 1 6 019 male 1 7 120 male 1 8 421 male 1 4 422 male 1 5 123 male 1 6 024 male 1 7 125 male 1 8 426 male 1 4 427 male 1 5 128 male 1 6 029 male 1 7 130 male 1 8 4

    mm1 6 30 mm1

31 female 2 8 132 female 2 9 033 female 2 9 034 female 2 9 035 female 2 10 136 female 2 8 137 female 2 9 038 female 2 9 039 female 2 9 040 female 2 10 141 female 2 8 142 female 2 9 043 female 2 9 044 female 2 9 045 female 2 10 1

    mf2 9 6 SSf2

46 male 2 5 447 male 2 6 148 male 2 7 049 male 2 8 150 male 2 9 451 male 2 5 452 male 2 6 153 male 2 7 054 male 2 8 155 male 2 9 456 male 2 5 457 male 2 6 158 male 2 7 059 male 2 8 160 male 2 9 4

    mm2 7 30 SSm2

61 female 3 8 162 female 3 9 063 female 3 9 064 female 3 9 065 female 3 10 166 female 3 8 167 female 3 9 068 female 3 9 069 female 3 9 070 female 3 10 171 female 3 8 172 female 3 9 073 female 3 9 074 female 3 9 075 female 3 10 1

    mf3 9 6 SSf3

76 male 3 6 477 male 3 7 178 male 3 8 079 male 3 9 180 male 3 10 481 male 3 6 482 male 3 7 183 male 3 8 084 male 3 9 185 male 3 10 486 male 3 6 487 male 3 7 188 male 3 8 089 male 3 9 190 male 3 10 4

    mm3 8 30 SSm3

SS Within       132

dfTotal N- # of cell groups     84

21

The ANOVA Summary TableTwo-way Between-Subjects Factorial ANOVA

ANOVA Summary F Table  SS df MS F p

Instruction 105.0 2 52.50 33.41 <0.001Sex 22.5 1 22.50 14.32 <0.001Instruction*Sex 15.0 2 7.50 4.77 0.011Error (within) 132.0 84 1.57  

Total 274.5 29      

22

Lab Activity: Two-Way Between-Subjects Factorial ANOVA in SPSS

23

Lab Activity: SPSS Outputs for Two-Way Between-Subjects Factorial ANOVA in SPSS

Tests of Between-Subjects Effects

Dependent Variable: performance

142.500a 5 28.500 18.136 .0005062.500 1 5062.500 3221.591 .000

105.000 2 52.500 33.409 .00022.500 1 22.500 14.318 .00015.000 2 7.500 4.773 .011

132.000 84 1.5715337.000 90

274.500 89

SourceCorrected ModelInterceptinstructionsexinstruction * sexErrorTotalCorrected Total

Type III Sumof Squares df Mean Square F Sig.

R Squared = .519 (Adjusted R Squared = .491)a.

Interpreting a Two-way Factorial ANOVA-- When the Interaction Effect is SignificantIncreasing the sample size increased the power, the interaction effect is now significant even though all the cell means were the same.

Interpreting a Two-way Factorial ANOVA- When the Interaction Effect is Significant

When SPSS “General Linear Model” (GLM) menu is used: If the interaction effect is significant, interpret the

interaction effect ONLY (even if the main effects are significant).

One should then conduct the post hoc paired comparisons of the cell mean differences - simple main effects. The simple main effect is the mean difference between the groups of factor B (i.e., sex) for each given group of factor A (i.e., instruction method).

Using SPSS GLM, interpreting the main effects is unnecessary because, the main effects are included in the estimate of cell means. Test of the simple main effects actually have incorporated the test of main effects.

Lab Activity: SPSS Syntax for Testing Simple Main Effects

UNIANOVA performance BY instruction sex /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /CRITERIA = ALPHA(.05) /DESIGN = instruction sex instruction*sex /EMMEANS = TABLES (instruction*sex) compare (sex) adj (Bonferroni).

Note that for post hoc simple main effect t-tests, only the methods of BONFERRONI, LSD, SIDAK tests are available in SPSS.

SPSS drop-down menu dose not provide function of testing simple main effects. Instead, you should use the following syntax of EMMEANS to test them.

For females, there was significant difference between instruction methods A and B, methods A and C (p= 0.005). However there is no difference between methods B and C, p= 1.000.

For males, there were not significant differences between any instruction methods.

Lab Activity: SPSS Output for Testing Simple Main Effects

Protecting from Type- I error

Primary IV

Moderator IV

Remember to hit “Add”

Lab Activity: Plotting Interaction Effect in SPSS

Lab Activity: SPSS Plot for Interaction Effect

321

instruction

9

8

7

6Est

imat

ed M

argi

nal M

eans

malefemale

sex

Estimated Marginal Means of performance

The difference between female and male students’ performance is different for each instruction method group. Namely, the gender difference in performance is dependent on (moderated by) which instruction method they receive.

The effect of instruction method on students’ performance is different for males and females. Namely, the effect of the instruction method is dependent on ( moderated by) sex.

30

Interpreting Models with More Than Three Factors

In general, for any ANOVA models, if the higher-order interaction effect is statistically significant, we do not interpret the lower-order effects directly, including the main effects or the interaction effects.

Lower-order effects will be interpreted in the context of the higher level interaction effect.

For example, if the 3-way interaction of an ANOVA including three factors is statistically significant, the 2-way interaction effects are interpreted in the context of the 3-way interaction. If a 2-way interaction effect is significant, its corresponding one-way main effects are interpreted in the context of the 2-way interaction effect.

31

Observed Effect SizeEta-Squared vs. Partial Eta-Squared

Eta-squared and partial eta-squared are effect size measures for the effect/difference of the factors included in a ANOVA.

Lab Activity:Calculate the eta-squared and partial-eta squared using the ANOVA table in the Excel file: Two Way ANOVA with Interaction.xls.Questions:Is there a difference between eta-squared and partial eta-squared for one-way ANOVA?Will partial eta-squared always be smaller or greater than the eta-squared?

Two common used effect size measures for ANOVA:h2 (eta-squared) = SSeffect / SStotalPartial h2 = SSeffect / (SSeffect + SSerror) or

= SSeffect / (Sstotal – SSother effects)

Given that the numerators of the two effect size measures are the same, Partial h2 will yield a bigger effect size than does the h2 because the denominator for partial h2 is smaller (by removing the SS other than the given effect.)

• However, h2 is conceptually easier to interpret; indicating the percentage of the total sum of square that is attributable to a given effect. Also, h2 can be used to compare the relative effects because all the effects have a common denominator.

Observed Effect SizeEta-Squared vs. Partial Eta-

Squared

Lab Activity: Effect Sizes for Two-way ANOVA in SPSS

Tests of Between-Subjects Effects

Dependent Variable: performance

142.500a 5 28.500 18.136 .000 .5195062.500 1 5062.500 3221.591 .000 .975

105.000 2 52.500 33.409 .000 .44322.500 1 22.500 14.318 .000 .14615.000 2 7.500 4.773 .011 .102

132.000 84 1.5715337.000 90

274.500 89

SourceCorrected ModelInterceptinstructionsexinstruction * sexErrorTotalCorrected Total

Type III Sumof Squares df Mean Square F Sig.

Partial EtaSquared

R Squared = .519 (Adjusted R Squared = .491)a.