factorial between-Ps ANOVA I: omnibus testsuqwloui1/stats/3010 for post... · 2018. 8. 2. · 3 5...
Transcript of factorial between-Ps ANOVA I: omnibus testsuqwloui1/stats/3010 for post... · 2018. 8. 2. · 3 5...
1
1
psyc3010 lecture 2
factorial between-Ps ANOVA I:omnibus tests
last lecture: introduction to factorial designsnext lecture: factorial between-Ps ANOVA II:
(effect sizes and follow-up tests)
22
general announcementsgeneral announcements
•• tutorials start this weektutorials start this weekattendance strongly recommended!attendance strongly recommended!
•• list of private tutors available on Blackboardlist of private tutors available on Blackboard
•• revisiting upcoming assessment :revisiting upcoming assessment :quiz 1quiz 1 on ANOVA in late Augon ANOVA in late Augassignment 1assignment 1 on ANOVA in early Septon ANOVA in early Sept
2
3
more about quiz 1
• multiple-choice and problem-solving questions
• assesses material on between-subjects ANOVA covered in Lectures 1, 2, 3, and 4
• Practice Questions and Answers posted onBlackboard next week
• Practice Quiz available on Blackboard in Week 4 (to get used to format)
ICEBREAKER!
44
last lecture this lecturein the last lecture, we introduced the concept of factorial designs, focusing on key terminology and concepts– factors / independent variables, dependent variables– crossed designs: A x B– cell means, marginal means, grand means– main effects, interaction effects, simple effects– how one factor qualifies or moderates the effect of
another factor– ordinal and disordinal interactions
in this lecture, we learn more about factorial between-participants ANOVA, focusing on the logic and computation of omnibus tests
3
55
todaytodayconceptual underpinnings of ANOVA
relationship between hypotheses, variance, graphical representations of data, and formulae(one-way and two-way analyses) research questions and hypothesesbuilding up to the F ratiothe F teststructural modelassumptions
conducting 2-way factorial ANOVAdata table and key termsresults in source/summary tablegraphing results
effect sizeswhat they are and why they are importantthree estimates: η2 , ω2 , partial η2
66
anova: conceptual underpinnings anova: conceptual underpinnings
like most statistical procedures we use, anova is like most statistical procedures we use, anova is all about partitioning varianceall about partitioning variancewe want to see if variation due to our we want to see if variation due to our experimental manipulations or groups of interest experimental manipulations or groups of interest is proportionally greater than the rest of the is proportionally greater than the rest of the variance variance (i.e., that is (i.e., that is notnot due to any manipulations etc)due to any manipulations etc)do participants’ scores (on some DV) differ from do participants’ scores (on some DV) differ from one another because they are in different groups one another because they are in different groups of our study, more so than they differ randomly of our study, more so than they differ randomly and due to unmeasured influences?and due to unmeasured influences?
4
77
'There are three kinds of lies: lies, damned 'There are three kinds of lies: lies, damned lies, and statistics.' lies, and statistics.' -- DisraeliDisraeli
88
review of review of oneone--way ANOVAway ANOVA
5
99
remember variance?remember variance?variance = dispersion or spread of scores around a point of central tendency – the meanthere is always some variability in a dataset: every score does not sit on the meanerror variance = can’t be explained (random or due to unmeasured influences)treatment variance = systematic differences due to our IV (e.g., experimental manipulation)
xxx x
xx x
0 100TEST SCORES
xx xx
x xx x xx
x xxxx
x
xx
xx
xxxxx x
x xx
x xx
x
x xxx x
xx x
0 100TEST SCORES
xx xx
x xx x xx
x xxxx
x
xx
xx
xxxxx x
x xx
x xx
x
x
1010
sources of variance in 1sources of variance in 1--way ANOVAway ANOVA
between-groups variance = systematic variance due to membership in different groups / treatments
within-groups variance = error variance (due to random chance or unmeasured influences)
Variance due to treatmentVariance due to treatment ErrorErrorbetweenbetween--groups variancegroups variance withinwithin--groups variancegroups variance
total variationtotal variation
6
1111
sources of variance in 1sources of variance in 1--way ANOVAway ANOVA
1111μ.
μ2μ1
1111
within-groups variance =distribution of individual DV scores around the group mean
between-groups variance =distribution of group means around the grand mean
μ1 = population mean for group 1μ2 = population mean for group 2μ. = population grand mean
1212
hypotheses in 1hypotheses in 1--way ANOVAway ANOVAdifferences between 2 meansdifferences between 2 means
HH00: : μμ11 = = μμ22–– null hypothesis: no differences between treatment meansnull hypothesis: no differences between treatment means
HH11:: μμ11 ≠ μμ22–– alternative hypothesis: difference between treatment meansalternative hypothesis: difference between treatment means
Or another way of putting it: Or another way of putting it: HH00: : μμjj = = μμ. . HH11:: μμjj ≠ μμ.. for at least one groupfor at least one group
μj = population mean of group j
μ. = grand mean
7
1313
hypotheses in 1hypotheses in 1--way ANOVAway ANOVAdifferences among 3+ meansdifferences among 3+ means
HH00: : μμ11 = = μμ2 2 = = μμ3 3 = = = = μμjj–– null hypothesis: no differences between any treatment meansnull hypothesis: no differences between any treatment means–– Could also write, Could also write, HH00: : μμjj = = μμ..
HH11:: null hypothesis is falsenull hypothesis is false–– alternative hypothesis: at least one difference between alternative hypothesis: at least one difference between
treatment meanstreatment means–– Could also write, Could also write, HH11: : μμjj ≠ ≠ μμ. . for at least one jfor at least one j
μj = population mean of group j
μ. = grand mean
1414
sources of variance in 1sources of variance in 1--way ANOVAway ANOVA
1414μ.
μ2μ1
1414
What line is hypothesized to be zero in our null hypothesis?
8
1515
16μ. μ2μ1
what happens in 1what happens in 1--way ANOVAway ANOVA
•• given that there is always given that there is always withinwithin--group “error” variance group “error” variance in the in the sample, we want to know whether there is more sample, we want to know whether there is more betweenbetween--groups groups variation than expected based on this error variancevariation than expected based on this error variance
•• want to be sure that differences between groups are not due want to be sure that differences between groups are not due simply to simply to errorerror
Ok, but it’s never really zero, right? Ok, but it’s never really zero, right? we compare the ratio of betweenwe compare the ratio of between--groups variance to groups variance to withinwithin--groups variancegroups variance
μ. μ2μ1
9
1717
Variance due to treatmentVariance due to treatment ErrorErrorBetweenBetween--groups variancegroups variance WithinWithin--groups variancegroups variance
Total VariationTotal Variationunivariate anova
n∑(X bar j – X bar dot)2 = people per group x sum of squared differences between group means and grand mean = estimate of between groups variability
∑ − 2. )( XXn j
∑(X ij – X bar j)2 = sum of squared differences between individual scores and group mean = estimate of within groups variability
∑ − 2)( jij XX
1818
the test statistic is the Fthe test statistic is the F--ratioratio
F = MStreat/MSerror
where where MSMStreattreat = index of variability among treatment means = index of variability among treatment means (SS(SSTRTR//dfdfTRTR) or () or (SSSSjj//dfdfjj))
and and MSMSerrorerror = index of variability among participants within a = index of variability among participants within a cell, i.e. pooled withincell, i.e. pooled within--cell variance (cell variance (SSSSErrorError//dfdfErrorError))
= average of s= average of s22 from each sample, a good estimate from each sample, a good estimate of of σσee
22 (population variance)(population variance)
–– if if MSMStreattreat is a good estimate of is a good estimate of error variance, error variance, F F = = MSMStreattreat//MSMSerrorerror = 1= 1–– if if MSMStreattreat is more than just error variance, is more than just error variance, F F = = MSMStreattreat//MSMSerrorerror > 1> 1–– larger values of F indicate that H0 is probably wronglarger values of F indicate that H0 is probably wrong
logic of univariate (onelogic of univariate (one--way) anovaway) anova
10
1919
1. Estimate of between1. Estimate of between--groups variabilitygroups variability2. Estimate of within2. Estimate of within--groups variabilitygroups variability3. Weight each variability estimate by # of observations 3. Weight each variability estimate by # of observations used to generate the estimate (“degrees of freedom”)used to generate the estimate (“degrees of freedom”)Compare ratioCompare ratio
[[n∑(X bar j – X bar dot)2] / (j -1) ][[∑(X ij – X bar j)2 ] / [ j (n-1)] ]
When the F ratio is > 1, the treatment effect (variability between groups) is bigger than the “error” variability (variability within groups). Or more specifically:
The sum of the squared differences between the group means and the grand mean x the number of people in each group, divided by the number of groups minus 1, is bigger than
the sum of the squared differences between the observations and the group means, weighted by the number of observations in each group minus 1 x the # of groups
So what is ANOVA ?So what is ANOVA ?
2020
building up to the building up to the FF ratioratio
if you want to know more about the underlying math, take a look online at the extra materials for Lecture 2
for a refresher on DFs:* Field (3rd ed) pg. 37* Field (2nd ed) pg. 319
11
2121
2222
partitioning variance in 1partitioning variance in 1--way ANOVAway ANOVA
errortreatment
12
2323
structural model of 1structural model of 1--way ANOVAway ANOVA
XXijij == μμ.. ++ ττjj ++ eeijij
for i cases and j treatments
Xij, any DV score is a combination of:
μ. the grand mean
τj the effect of the j-th treatment (μj - μ.)eij error for i person in j-th treatment
2424
XXijij = = μμ. + . + ττjj + + eeijij
μ. μ2μ1Xi2
structural model of 1structural model of 1--way ANOVAway ANOVA
13
2525
Derivation for oneDerivation for one--way ANOVA: way ANOVA: expected mean squaresexpected mean squares
an expected value of a statistic is defined as the ‘longan expected value of a statistic is defined as the ‘long--range range average’ of a sampling statisticaverage’ of a sampling statisticour our expected mean squaresexpected mean squares –– are:are:–– EE(MS(MSerrorerror) ) σσee
22
•• i.e., the long term average of the variances i.e., the long term average of the variances withinwithin each sample (Seach sample (S22) ) would be the population variance would be the population variance σσee
22
–– EE(MS(MStreattreat) ) σσee2 2 + n+ nσσττ
22
•• where where σσττ22 is the long term average of the variance is the long term average of the variance between sample between sample
meansmeans and n is the number of observations in each groupand n is the number of observations in each group•• i.e., the long term average of the variances i.e., the long term average of the variances withinwithin each sample each sample
PLUSPLUS any variance any variance betweenbetween each sampleeach sample•• Basically Basically -- if group means don’t vary then if group means don’t vary then nnσσττ
22 = 0, = 0, and so then and so then EE(MS(MStreattreat) ) = = σσee
2 2 + 0 = + 0 = σσee2 = 2 = EE(MS(MSerrorerror) ) = = σσee
22
See e.g., Howell (2007) p. 303See e.g., Howell (2007) p. 303
2626
twotwo--way factorial way factorial ANOVAANOVA
14
2727
research questionsresearch questionsoneone--way ANOVA:way ANOVA:-- is there a treatment effect (is there between group variability)?is there a treatment effect (is there between group variability)?
twotwo--way ANOVA:way ANOVA:–– is there a main effect of Factor A? is there a main effect of Factor A?
Is there variability between the levels of A, averaging over the Is there variability between the levels of A, averaging over the other factor? Do the A group means differ from each other? other factor? Do the A group means differ from each other? Do the marginal means of A differ from the grand mean?Do the marginal means of A differ from the grand mean?
–– is there a main effect of Factor B? is there a main effect of Factor B? Is there variability between the levels of B, averaging over the Is there variability between the levels of B, averaging over the other factor? Do the B group means differ from each other? other factor? Do the B group means differ from each other? Do the marginal means of B differ from the grand mean?Do the marginal means of B differ from the grand mean?
–– is there an A x B interaction? is there an A x B interaction? Does the simple effect of A change for different B groups? Does Does the simple effect of A change for different B groups? Does the simple effect of B change for different A groups? Does the the simple effect of B change for different A groups? Does the simple effect change across the levels of the other factor? simple effect change across the levels of the other factor? Do the cell means differ from the grand mean more than would Do the cell means differ from the grand mean more than would be expected given the effects of A and B?be expected given the effects of A and B?
2828
Variance due to treatmentVariance due to treatment ErrorErrorbetweenbetween--groups variancegroups variance withinwithin--groups variancegroups variance
total variationtotal variation
variance due to factor A
variance due to factor B
variance due to A X B
1-way ANOVA
2-way ANOVA
15
sources of variance: 2 x 2 ANOVA
μ.
good notes, little study
good notes, much study
bad notes, little study
bad notes, much study
Factor 1 = note quality (bad or good)Factor 2 = study time (little or much)
DV = exam score
4 cells
29
30
sources of variance: main effects of note quality & study time
μ.
Μgood notesΜbad notes
good notes, little study
good notes, much study
bad notes, little study
bad notes, much study
ΜmuchΜlittle
marginal means to assess main effects
16
31
μ.
good notes, little study
good notes, much study
bad notes, little study
bad notes, much study
simple effect of study time for bad notes (compares cell means for S at N1)
simple effect of study time for good notes (compares cell means for S at N2)
the interaction effect will be significant if the simple effects of one factor differ at the various levels of the other factor
sources of variance: simple effects of study time
32
μ.
good notes, little study
good notes, much study
bad notes, little study
bad notes, much study
simple effect of note quality for little study (compares cell means for N at S1)
simple effect of note quality for much study (compares cell means for N at S2)
The interaction effect will be significant if the simple effects of one factor differ at the various levels of the other factor
sources of variance: simple effects of note quality
17
3333
degrees of freedom summarydegrees of freedom summarydf total = N – 1
df factor = # of levels of the factor – 1
dfB = b – 1dfA = a – 1
df interaction = product of df for factors in the interaction
dfBA = (b – 1) x (a –1)
dferror = total # of observations – # of treatments= N – ba
= df for each cell x # of cells= (n – 1) ba
3434
Variance due to treatmentVariance due to treatment ErrorErrorbetweenbetween--groups dfgroups df
= ab = ab -- 11
withinwithin--groups groups dfdf
= N = N –– abab = = abab (n (n -- 1)1)
total df = N total df = N -- 11
df for factor A
= a – 1
df for factor B
= b - 1
df for A X B
= (a - 1) x (b - 1)2-way ANOVA
1-way ANOVA
18
3535
partitioning variancepartitioning variancein 2in 2--way ANOVAway ANOVA
main effect of Amain effect of BAxB interactionerror
3636
19
3737
main effects main effects (shown for an IV with 3 levels)–– HH00:: μμ11 = = μμ22 = = μμ33
•• no differences no differences among means across levels of the among means across levels of the factorfactor
–– HH11:: null is falsenull is false
interaction interaction (one possibility for a 2 x 3 design)(one possibility for a 2 x 3 design)–– HH00:: μμ1111 -- μμ2121 = = μμ12 12 -- μμ2222 = = μμ1313 -- μμ2323
•• if there are differences between particular factor if there are differences between particular factor means, they are means, they are constantconstant at each level of the at each level of the other factor (hence the parallel lines)other factor (hence the parallel lines)
•• the ‘difference of the differences’ is zerothe ‘difference of the differences’ is zero–– HH11:: null is falsenull is false
hypothesis testing in 2hypothesis testing in 2--way ANOVAway ANOVA
μμ1111 = mean of the following group:1st level of Factor A and 1st level of Factor B
3838
interaction interaction (other possibility for a 2 x 3 design)(other possibility for a 2 x 3 design)–– Or HOr H00:: μμ1k1k -- μμ.. = .. = μμ2k 2k -- μμ.. ..
•• differences between the means of factor B and the grand differences between the means of factor B and the grand mean are the same for the 2 levels of factor A (because B mean are the same for the 2 levels of factor A (because B has 3 means, can’t just put up pairs)has 3 means, can’t just put up pairs)
–– HH11:: μμ1k1k -- μμ.. ≠ .. ≠ μμ2k 2k -- μμ.. ..
hypothesis testing in 2hypothesis testing in 2--way ANOVAway ANOVA
20
3939
1. In 21. In 2--way design, estimate betweenway design, estimate between--groups variabilitygroups variability–– Due to main effect of first factorDue to main effect of first factor–– Due to main effect of second factorDue to main effect of second factor–– Due to interaction of two factorsDue to interaction of two factors
2. Estimate within2. Estimate within--groups variabilitygroups variability3. Weight each variability estimate by # of 3. Weight each variability estimate by # of observations used to generate the observations used to generate the estimate (“degrees of freedom”)estimate (“degrees of freedom”)Compare Compare ratios ofratios of–– (1) between(1) between--groups groups variability among levels of variability among levels of
A to A to error; (2) B error; (2) B to to error; error; and and (3) (3) ABcellsABcells(adjusted for main effects) to error(adjusted for main effects) to error
So what is factorial ANOVA ?So what is factorial ANOVA ?
oneone--way ANOVAway ANOVA
F = MStreat / MSerror
as for 1as for 1--way ANOVA, way ANOVA, MSMSerrorerror = pooled variance (average s= pooled variance (average s22) )
but now we have a separate but now we have a separate MSMStreattreat for each effect:for each effect:1) 1) MSMStreattreat for effect of factor A = for effect of factor A = MSMSAA (1st main effect)(1st main effect)2) 2) MSMStreattreat for effect of factor B = for effect of factor B = MSMSBB (2nd main effect)(2nd main effect)3) 3) MSMStreattreat for effect of factor AB = for effect of factor AB = MSMSABAB ((intxintx effect)effect)
4040
the the FF test in 2test in 2--way ANOVAway ANOVA
a ratio of the systematic variance of EACH EFFECT (i.e. of your experimental manipulations or treatments) to the unsystematic variance
21
2 way ANOVA2 way ANOVA
three omnibus teststhree omnibus testsMain effect of AMain effect of AMain effect of BMain effect of BInteractionInteraction
4141
4242
XXijkijk == μμ. . ++ ααjj ++ ββkk ++ αβαβjkjk ++ eeijkijk
for i cases, factor A with j treatments, factor B with k treatments, and the AxB interaction with jk treatments:
Xijk, any DV score is a combination of:μ. the grand meanαα j the effect of the j-th treatment of factor A (μAj - μ.)ββkk the effect of the k-th treatment of factor B (μBk - μ.)αβαβjkjk the effect of differences in factor A treatments at different levels of factor B treatments (μ. - μAj - μBk + μjk)eijk error for i person in j-th and k-th treatments
structural model of 2structural model of 2--way ANOVAway ANOVA
22
4343
assumptions of ANOVAassumptions of ANOVApopulation– treatment populations are normally distributed
(assumption of normality)– treatment populations have the same variance
(assumption of homogeneity of variance)sample– samples are independent – no two measures are drawn from the
same participant • c.f. repeated-measures ANOVA (more on that later in semester)
– each sample obtained by independent random sampling – within any particular sample, no choosing of respondents on any kind of systematic basis
– each sample has at least 2 observations and equal ndata (DV scores)– measured using a continuous scale (interval or ratio)– mathematical operations (calculations for means, variance, etc)
do not make sense for other kinds of scales
4444
23
4545
conducting a 2conducting a 2--waywaybetweenbetween--subjects subjects factorial ANOVAfactorial ANOVA
4646
application of betweenapplication of between--subjects subjects factorial ANOVAfactorial ANOVA
a psychological study of creativity in complex socioa psychological study of creativity in complex socio--chemical environments (Field, 2000)chemical environments (Field, 2000)
2 factors:2 factors:–– three groups of participants go to the pub and have:three groups of participants go to the pub and have:
•• no beerno beer, or , or 2 pints2 pints, or , or 4 pints4 pints–– half of the participants are half of the participants are distracted distracted and half are and half are
not distracted not distracted (controls)(controls)hence, a hence, a 2 x 3 between2 x 3 between--subjects factorial designsubjects factorial designDV:DV: creativitycreativity– unbiased 3rd parties rate the quality of limericks made
up by each of our participants
24
4747
application of betweenapplication of between--subjects subjects factorial ANOVAfactorial ANOVA
research questions:research questions:is there a is there a main effectmain effect of alcohol consumptionof alcohol consumption??–– does the quality of limerick you make up depend upon does the quality of limerick you make up depend upon
how many pints of beer you have had?how many pints of beer you have had?
is there a is there a main effectmain effect of distractionof distraction??–– does the quality of limerick you make up depend on does the quality of limerick you make up depend on
whether you were distracted or not?whether you were distracted or not?
is there a is there a consumption x distraction interactionconsumption x distraction interaction??–– does the effect of distraction upon creativity depend does the effect of distraction upon creativity depend
upon consumption? (Or, does the effect of upon consumption? (Or, does the effect of consumption upon creativity depend upon consumption upon creativity depend upon distraction?)distraction?)
4848
0 2 4
50 45 3055 60 3080 85 3065 65 55
Distraction 70 70 3575 70 2075 80 4565 60 40
Cell Totals 535 535 285 1355
65 70 5570 65 6560 60 70
Controls 60 70 5560 65 5555 60 6060 60 5055 50 50
Cell Totals 485 500 460 1445
Marginal Totals (A ) 1020 1035 745 2800
Distraction Marginal Totals (B)
Alcohol Consumption (pints)a combination of IV levels, e.g., 0 pints and distracted, is
called a cell
in most between-subjects factorial
designs there are nobservations per cell
the number of cells multiplied by n gives
you N, the total number of
observations
(abn = N)
25
4949
0 2 4
50 45 3055 60 3080 85 3065 65 55
Distraction 70 70 3575 70 2075 80 4565 60 40
Cell Totals 535 535 285 1355Cell Means 66.88 66.88 35.63 56.46
65 70 5570 65 6560 60 70
Controls 60 70 5560 65 5555 60 6060 60 5055 50 50
Cell Totals 485 500 460 1445Cell Means 60.63 62.50 57.50 60.21
Marginal Totals (A ) 1020 1035 745 2800
Means 63.75 64.69 46.56 58.33
Distraction Marginal Totals (B)
Alcohol Consumption (pints)cell means are
calculated – these are the average of the
n observations in each cell
marginal means for each level of each
factor are also calculated –
these are group means averaged over the
levels of the other factor
the grand mean is the average of all N
observations
Means
Summary Table
Source df SS MS F sig
A (cons) 2 3332.3 1666.15 20.07 .000
B (dist) 1 168.75 168.75 2.03 .161
AB 2 1978.12 989.06 11.91 .000
Error 42 3487.5 83.02
Total 47 8966.7
50
review of source / summary table
p value
MS = SS / df F = MStreat / MSerror
if you want to know more about how the SS values were obtained from the data, look at the extra materials for Lecture 2
26
51
Summary Table
Source df SS MS F sig
A (cons) 2 3332.3 1666.15 20.07 .000
B (dist) 1 168.75 168.75 2.03 .161
AB 2 1978.12 989.06 11.91 .000
Error 42 3487.5 83.02
Total 47 8966.7
the results of this ANOVA show:
a significant main effect of pints consumed
no main effect of distraction
a significant cons. x dist. interaction
5252
0
10
20
30
40
50
60
70
80
0 2 4
Alcohol consumed (pints)
rate
d cr
eativ
ity
DistractionControls
no main effect of distractionno main effect of distraction
27
5353
0
10
20
30
40
50
60
70
80
0 2 4
Alcohol consumed (pints)
rate
d cr
eativ
ity
DistractionControls
main effect of alcohol consumedmain effect of alcohol consumed
5454
0
10
20
30
40
50
60
70
80
0 2 4
Alcohol consumed (pints)
rate
d cr
eativ
ity
DistractionControls
disordinaldisordinal interactioninteraction
28
5555
effect sizeseffect sizes
5656
are significance tests so useful?researchers have uncritically focused on researchers have uncritically focused on pp values...values...
problems with significance testing as a way of problems with significance testing as a way of determining the importance of findings:determining the importance of findings:
–– use of an arbitrary acceptance criterion (use of an arbitrary acceptance criterion (αα) results in a ) results in a binary outcome (significant or nonbinary outcome (significant or non--significant)significant)
–– no information about the practical significance of findingsno information about the practical significance of findings
–– a large a large pp--value (nonvalue (non--significant) will eventually slip under significant) will eventually slip under the acceptance criterion as the sample size increasesthe acceptance criterion as the sample size increases
thethe magnitude of experimental effectmagnitude of experimental effect, or , or effect sizeeffect size, has been proposed as an , has been proposed as an accompaniment (if not an outright replacement) accompaniment (if not an outright replacement) to significance testing to significance testing
29
5757
magnitude of experimental effectsmagnitude of experimental effects
the effect size gives you another way of assessing the effect size gives you another way of assessing the reliability of the result the reliability of the result in terms of variancein terms of variance
can compare size of effects within a factorial design: can compare size of effects within a factorial design: how much variance explained by factor 1, factor 2, how much variance explained by factor 1, factor 2, their interaction, etc.their interaction, etc.
differentiating effect sizes (Cohen, 1973):differentiating effect sizes (Cohen, 1973):0.2 = small0.2 = small0.5 = medium0.5 = medium0.8 = large0.8 = large
5858
etaeta--squared squared ηη22
ηη22 = = SSSSeffecteffect
SSSStotaltotaldescribes the describes the proportion of proportion of variancevariance in the sample’ssample’sDVDV scoresscores that is accounted that is accounted for by the effectfor by the effectconsidered a considered a biasedbiased estimate estimate of the true magnitude of the of the true magnitude of the effect in the population (tends effect in the population (tends to be larger than in reality)to be larger than in reality)most commonly reported most commonly reported effect size measure because effect size measure because easily interpretable (like Reasily interpretable (like R2))
omegaomega--squared squared ϖϖ22
ϖϖ2 2 = = SSSSeffecteffect –– ((dfdfeffecteffect))MSMSerrorerror
SSSStotaltotal + + MSMSerrorerror
describes the describes the proportion of proportion of variancevariance in thein the population’spopulation’sDV scores that is accounted DV scores that is accounted for by the effectfor by the effecta a less biasedless biased estimate of the estimate of the effect sizeeffect sizea more a more conservativeconservative estimate estimate (smaller)(smaller)
2 main approaches to estimating 2 main approaches to estimating effect sizes in ANOVAeffect sizes in ANOVA
difference between the two estimates depends on sample sizesample size and error varianceerror variance
30
5959
Summary Table – from Slide 41
Source df SS MS F sig
C (cons) 2 3332.3 1666.15 20.07 .000
D (distr) 1 168.75 168.75 2.03 .161
C x D 2 1978.12 989.06 11.91 .000
Error 42 3487.5 83.02
Total 47 8966.7
etaeta--squared:squared:ηη22 = = SSSSeffecteffect
SSSStotaltotal
ConsumptionConsumption: : = 3332.3 / 8966.7= 3332.3 / 8966.7= .37 (37% var)= .37 (37% var)
number of pints consumednumber of pints consumedexplains 37% of variance in explains 37% of variance in DV scores (creativity ratings)DV scores (creativity ratings)
distractiondistraction explains 2% of explains 2% of variance in DV scoresvariance in DV scores
interactioninteraction (distraction X number (distraction X number of pints consumed) explains of pints consumed) explains 22% in DV scores22% in DV scores
DistractionDistraction: : = 168.75 / 8966.7= 168.75 / 8966.7= .02 (2% var)= .02 (2% var)C x DC x D: : = 1978.12 / 8966.7= 1978.12 / 8966.7= .22 (22% var)= .22 (22% var)
6060
Summary Table – from Slide 41
Source df SS MS F sig
C (cons) 2 3332.3 1666.15 20.07 .000
D (distr) 1 168.75 168.75 2.03 .161
C x D 2 1978.12 989.06 11.91 .000
Error 42 3487.5 83.02
Total 47 8966.7
omegaomega--squared:squared:ϖϖ 2 2 = = SSSSeffect effect –– (df(dfeffecteffect)MS)MSerrorerror
SSSStotaltotal + MS+ MSerrorerror
produces very similar but smaller produces very similar but smaller (more conservative)(more conservative) estimatesestimates
ConsumptionConsumption= [3332.3 = [3332.3 -- 2(83.02)] / (8966.7 + 83.02)2(83.02)] / (8966.7 + 83.02)= = .34 (.34 (34% 34% var)var) [compared to 37%][compared to 37%]
DistractionDistraction= [168.75 = [168.75 -- 1(83.02)] / (8966.7 + 83.02)1(83.02)] / (8966.7 + 83.02)= .01 (= .01 (1%1% var) var) [compared to 2%][compared to 2%]
C x DC x D= [1978.12 = [1978.12 –– 2(83.02)] / (8966.7 +83.02)2(83.02)] / (8966.7 +83.02)= .20 (= .20 (20%20% var) var) [compared to 22%][compared to 22%]
31
6161
what’s what’s partialpartial eta squared?eta squared?compare means command in SPSScompare means command in SPSS: : asking for an ANOVA with effect size asking for an ANOVA with effect size gives you gives you eta squaredeta squared –– proportion of proportion of total variance total variance accounted for by the effectaccounted for by the effect
total
effect2
SSSS
=η
erroreffect
effectp22
SSSSSS)(partial
+=ηη
residual variance = variance left over to be explained (i.e., not accounted for by any other IV in the model)
UNIANOVA, MANOVA, or GLMUNIANOVA, MANOVA, or GLM: : asking for effect size gives you asking for effect size gives you partial eta squaredpartial eta squared – proportion proportion of of residual varianceresidual variance accounted accounted for by the effectfor by the effect
total variance = SS for all effects + SSerror
6262
limitations of partial η2
1. in factorial ANOVA, [error + effect] is less than [total], so partial η2 is more liberal or inflated
SSSSeffect1effect1 = 4= 4SSSSeffect2effect2 = 4= 4SSSSintxintx = 4= 4SSSSerrorerror = 4= 4
SSSStotaltotal = 16= 16
ηη22 = 4 / 16 = = 4 / 16 = .25
partial η2 = 4 / (4 + 4) = 4 / 8 = .50
effect size for factor 1:
total
effect2
SSSS
=ηerroreffect
effectp22
SSSSSS)(partial
+=ηη
32
6363
limitations of partial limitations of partial ηη22
2. in factorial ANOVA, η2 adds up to a maximum of 100%, but partial η2 can add to > 100%
SSeffect1 = 4SSeffect2 = 4SSintx = 4SSerror = 4
SStotal = 16
η2 partial partial ηη22
effect 1 .25 .5effect 2 .25 .5
interaction .25 .5error .25 .5sum 1.001.00 2.00 (!)2.00 (!)
(calculations as inprevious slide)
hard to make meaningful comparisons with partial η2
6464
Tests of Between-Subjects Effects
Dependent Variable: I am dissatisfied with my performance recently
54.963a 14 3.926 1.822 .066 .367250.920 1 250.920 116.420 .000 .726
.070 1 .070 .032 .858 .00144.031 9 4.892 2.270 .035 .317
9.326 4 2.331 1.082 .377 .090
94.834 44 2.155821.000 59149.797 58
SourceCorrected ModelInterceptgendernumgpaestimategendernum *gpaestimateErrorTotalCorrected Total
Type III Sumof Squares df Mean Square F Sig.
Partial EtaSquared
R Squared = .367 (Adjusted R Squared = .165)a.
example with SPSS output
•• partial partial ηη22 : interaction between the two IVs accounts for 9% of the residual variance
•• ηη22 : interaction only accounts for 6% of the total variance6% of the total variance(9.326 / 149.797)
ω2 < η2 < partial η2
33
6565
summarymain effects and interactions are main effects and interactions are omnibusomnibus teststests–– result from preliminary partitioning of varianceresult from preliminary partitioning of variance–– look for any difference (anywhere) between the meanslook for any difference (anywhere) between the means
statistical significance is not the bestatistical significance is not the be--allall--andand--endend--all all –– useful to estimate the size of effects useful to estimate the size of effects –– gives us more gives us more
information than statistical significanceinformation than statistical significance–– two approaches two approaches –– etaeta--squaredsquared (biased, but common) (biased, but common)
and and omegaomega--squaredsquared (less biased, but uncommon)(less biased, but uncommon)–– partial etapartial eta--squaredsquared is an even more commonly reported is an even more commonly reported
effect size measure (output by SPSS) which is the effect size measure (output by SPSS) which is the portion of residual variance the effects account forportion of residual variance the effects account for
6666
next week: Effect sizes, simple effects and follow-upsReadings:
between-Ps ANOVA I: omnibus tests (this lecture)Field (3rd ed): Chapter 12 (pp 421-430), Chapter 3Field (2nd ed): Chapter 10 (pp 389-397), Chapter 2
between-Ps ANOVA II: follow-up tests (next lecture)Field (3rd ed): Chapter 12 (pp 430-449 and pp 454-456)Field (2nd ed): Chapter 10 (pp 397-420 and pp 425-426)Howell (all eds): Chapter 13 (pp 413-445)