Cal State Northridge 320 Andrew Ainsworth PhD
description
Transcript of Cal State Northridge 320 Andrew Ainsworth PhD
![Page 1: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/1.jpg)
Power and Effect SizeCal State Northridge
320Andrew Ainsworth PhD
![Page 2: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/2.jpg)
Psy 320 - Cal State Northridge 2
Major Points
ReviewWhat is power?What controls power?Effect sizePower for one sample tPower for related-samples tPower for two sample t
![Page 3: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/3.jpg)
Psy 320 - Cal State Northridge 3
Important Concepts
Concepts critical to hypothesis testing–Decision–Type I error–Type II error–Critical values–One- and two-tailed tests
![Page 4: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/4.jpg)
Psy 320 - Cal State Northridge 4
Decisions
When we test a hypothesis we draw a conclusion; either correct or incorrect.–Type I error
• Reject the null hypothesis when it is actually correct.
–Type II error• Retain the null hypothesis when it is actually
false.
![Page 5: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/5.jpg)
Psy 320 - Cal State Northridge 5
Type I Errors
Null Hypothesis really is trueWe conclude the null is false.This is a Type I error–Probability set at alpha ()
• usually at .05–Therefore, probability of Type I error = .05
![Page 6: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/6.jpg)
Psy 320 - Cal State Northridge 6
Type II ErrorsThe Alternative Hypothesis is trueWe conclude that the null is trueThis is also an error (Type II)–Probability denoted beta ()
• We can’t set beta easily.• We’ll talk about this issue later.
Power = (1 - ) = probability of correctly rejecting false null hypothesis.
![Page 7: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/7.jpg)
Psy 320 - Cal State Northridge 7
Confusion Matrix
Reality Reality H0 H1 H0 H1
“H0” 1 - α β “H0” .95 .16
You
r D
ecis
ion
“H1” α 1 - β
You
r D
ecis
ion
“H1” .05 .84
1.00 1.00 1.00 1.00
![Page 8: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/8.jpg)
Psy 320 - Cal State Northridge 8
Critical Values
These represent the point at which we decide to reject null hypothesis.e.g. We might decide to reject null when (p|null) < .05.– In the null distribution there is some
value with p = .05–We reject when we exceed that value.–That value is called the critical value.
![Page 9: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/9.jpg)
Psy 320 - Cal State Northridge 9
One- and Two-Tailed Tests
Two-tailed test rejects null when obtained value too extreme in either direction–Decide on this before collecting data.One-tailed test rejects null if obtained value is too low (or too high)–We only set aside one direction for
rejection.
![Page 10: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/10.jpg)
Psy 320 - Cal State Northridge 10
One- & Two-Tailed Example
One-tailed test–Reject null if IQPLUS group shows an
increase in IQ• Probably wouldn’t expect a reduction and
therefore no point guarding against it.
Two-tailed test–Reject null if IQPLUS group has a mean
that is substantially higher or lower.
![Page 11: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/11.jpg)
Psy 320 - Cal State Northridge 11
What Is Power?
Probability of rejecting a false H0
Probability that you’ll find difference that’s really there1 - , where = probability of Type II error
![Page 12: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/12.jpg)
Psy 320 - Cal State Northridge 12
What Controls Power?
The significance level ()True difference between null and alternative hypothesesm1 - m2
Sample sizePopulation varianceThe particular test being used
![Page 13: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/13.jpg)
Psy 320 - Cal State Northridge 13
Distributions Under m1 and m0
![Page 14: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/14.jpg)
Psy 320 - Cal State Northridge 14
Effect SizeThe degree to which the null is false–Depends on distance between m1 and m2
–Also depends on standard error (of mean or of difference between means)1 0 1 0
1 0
ˆ if we can assume
that , and are adequate estimators or their population counterparts.
X Xd or ds
X X s
m m
![Page 15: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/15.jpg)
Psy 320 - Cal State Northridge 15
What happened to n?
It doesn’t relate to how different the two population means are.It controls power, but not effect size.We will add it in later.
![Page 16: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/16.jpg)
Psy 320 - Cal State Northridge 16
Estimating Effect Size
Judge your effect size by:–Past research–What you consider important–Cohen’s conventions
Effect Size d Small .20 Medium .50 large .80
![Page 17: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/17.jpg)
Psy 320 - Cal State Northridge 17
Combining Effect Size and n
We put them together and then evaluate power from the result.General formula for Delta
–where f (n) is some function of n that will depend on the type of design
[ ( )]d f n
![Page 18: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/18.jpg)
Psy 320 - Cal State Northridge 18
Power for One-Sample or Related samples t
First calculate delta with:
– where n = size of sample, and and g as above
Look power up in table using and significance level ()
d̂ n
![Page 19: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/19.jpg)
Psy 320 - Cal State Northridge 19
Power for Single Sample IQPLUS Study
One sample z and t– Compared IQPLUS group with population mean =
100, sigma = 10– Used 25 subjects– We got a sample mean of 106 and s = 7.78
1 0
1 0
106 100 6 0.60 or 10 10
106 100 6ˆ 0.777.78 7.78
Xd
Xds
mm
![Page 20: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/20.jpg)
Psy 320 - Cal State Northridge 20
IQPLUS
Assuming we don’t know sigma – = 0.77 – n = 25– –We are testing at = .05–Use Appendix D.5
.77 25 .77*5 3.85
d̂
![Page 21: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/21.jpg)
Psy 320 - Cal State Northridge 21
Appendix D.5
This table is severely abbreviated.
Power for = 3.85, = .05
0.10 0.05 0.02 0.011.00 0.26 0.17 0.09 0.062.00 0.64 0.52 0.37 0.282.40 0.78 0.67 0.53 0.432.50 0.80 0.71 0.57 0.472.80 0.88 0.80 0.68 0.593.00 0.91 0.85 0.75 0.663.80 0.98 0.97 0.93 0.893.90 0.99 0.97 0.94 0.914.00 0.99 0.98 0.95 0.92
Alpha ()
![Page 22: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/22.jpg)
Psy 320 - Cal State Northridge 22
Conclusions
If we can trust our estimates in the IQPLUS study then if this study were run repeatedly, 97% of the time the result would be significant.
![Page 23: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/23.jpg)
Psy 320 - Cal State Northridge 23
How Many Subjects Do I Really Need (Single/Related Sample(s))?Run calculations backward–Start with anticipated effect size ( )–Determine required for power = .80.
• Why .80?–Calculate nWhat if we wanted to rerun the IQPLUS study, and wanted power = .80?
d̂
![Page 24: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/24.jpg)
Psy 320 - Cal State Northridge 24
Calculating n
We estimated = .77Complete Appendix D.5 shows we need = 2.80Calculations on next slide
d̂
![Page 25: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/25.jpg)
Psy 320 - Cal State Northridge 25
IQPLUS n
2
22
2 2
ˆ , ˆ
ˆ ˆ(divide both sides by )ˆ ˆ
(square both sides)ˆ
2.80 13.22 14 subjects (where we had 25)ˆ .77
Since d n nd
d n dd d
nd
nd
![Page 26: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/26.jpg)
Psy 320 - Cal State Northridge 26
Power for Two Independent Groups
What changes from preceding?–Effect size deals with two sample
means–Take into account both values of nEffect size
1 2 1 2ˆ pooled
X Xd or ds
m m
![Page 27: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/27.jpg)
Psy 320 - Cal State Northridge 27
Estimating d We could calculate d directly if we knew populations.We could estimate from previous data.Here we will calculate using Violent Video Games example
![Page 28: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/28.jpg)
Psy 320 - Cal State Northridge 28
Example: Violent Videos GamesTwo independent randomly selected/assigned groups– GTA (violent: 8 subjects) VS. NBA 2K7 (non-
violent: 10 subjects)– We want to compare mean number of
aggressive behaviors following game play– GTA had a mean of 10.25 and s = 1.669– NBA 2K7 had a mean of 8.4 and s = 1.647– s2
pooled = 2.745, so spooled = 1.657
![Page 29: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/29.jpg)
Psy 320 - Cal State Northridge 29
Two Independent Groups
Then calculate from effect size
ˆ2nd
Note: The above formula assumes that the 2 groups have equal n
d̂
![Page 30: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/30.jpg)
Psy 320 - Cal State Northridge 30
Two Independent GroupsOur data do not have equal n, but let’s pretend they do for a moment (both 10)For our data
10.25 8.4 1.85ˆ 1.1161.657 1.657
10ˆ 1.116 2.495 2.52 2
d
nd
![Page 31: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/31.jpg)
Psy 320 - Cal State Northridge 31
Appendix D.5
This table is severely abbreviated.
Power for = 2.5, = .05Estimate = .71
0.10 0.05 0.02 0.011.00 0.26 0.17 0.09 0.062.00 0.64 0.52 0.37 0.282.40 0.78 0.67 0.53 0.432.50 0.80 0.71 0.57 0.472.80 0.88 0.80 0.68 0.593.00 0.91 0.85 0.75 0.663.80 0.98 0.97 0.93 0.893.90 0.99 0.97 0.94 0.914.00 0.99 0.98 0.95 0.92
Alpha ()
![Page 32: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/32.jpg)
Psy 320 - Cal State Northridge 32
Conclusions
If we had equal n and we can trust our estimates in the violent video game study then if this study were run repeatedly, 71% of the time the result would be significant.
![Page 33: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/33.jpg)
Psy 320 - Cal State Northridge 33
Unequal Sample SizesWith unequal samples use harmonic mean of sample sizes
Where k is the number of groups (i.e. 2), ni is each group sizeStandard arithmetic average will work well if n ’s are close.
i
h
n
kX 1
![Page 34: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/34.jpg)
Psy 320 - Cal State Northridge 34
Two Independent Groups
Our data do not have equal n, so we need to find the harmonic meanFor our data
2 2 8.8891 1 1 .2258 10
h
i
kX
n
![Page 35: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/35.jpg)
Psy 320 - Cal State Northridge 35
Two Independent Groups
Our data do not have equal n, so…For our data
10.25 8.4 1.85ˆ 1.1161.657 1.657
8.889ˆ 1.116 2.353 2.42 2
h
d
Xd
![Page 36: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/36.jpg)
Psy 320 - Cal State Northridge 36
Appendix D.5
This table is severely abbreviated.
0.10 0.05 0.02 0.011.00 0.26 0.17 0.09 0.062.00 0.64 0.52 0.37 0.282.40 0.78 0.67 0.53 0.432.50 0.80 0.71 0.57 0.472.80 0.88 0.80 0.68 0.593.00 0.91 0.85 0.75 0.663.80 0.98 0.97 0.93 0.893.90 0.99 0.97 0.94 0.914.00 0.99 0.98 0.95 0.92
Alpha ()
Power for = 2.4, = .05Estimate = .67
![Page 37: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/37.jpg)
Psy 320 - Cal State Northridge 37
Conclusions
If we we can trust our estimates in the violent video game study then if this study were run repeatedly, 67% of the time the result would be significant.
![Page 38: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/38.jpg)
Psy 320 - Cal State Northridge 38
How Many Subjects Do I Really Need (Independent Samples)?
Run calculations backward–Start with anticipated effect size ( )–Determine required for power = .80.
• Why .80?–Calculate nWhat if we wanted to rerun the violent video game study, and wanted power = .80?
d̂
![Page 39: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/39.jpg)
Psy 320 - Cal State Northridge 39
Calculating n
We estimated = 1.116Complete Appendix E.5 shows we need = 2.80Calculations on next slide
d̂
![Page 40: Cal State Northridge 320 Andrew Ainsworth PhD](https://reader035.fdocuments.net/reader035/viewer/2022070421/56815ff3550346895dcef21b/html5/thumbnails/40.jpg)
Psy 320 - Cal State Northridge 40
Violent Video Games n2
22
2
2 2
ˆIf then 2* ˆ2
ˆ2 ˆ(divide both sides by )ˆ ˆ
(square both sides)ˆ 2
(multiply both sides by 2)ˆ 2
2.802* 2* 12.59 13 subjects/groˆ 1.116
nd nd
ndd
d d
nd
nd
nd
up