Sample Size, Precision and Power Calculations: A - OMICS Group
CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations
description
Transcript of CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations
![Page 1: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/1.jpg)
CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations
BUSI 6480Lecture 4
![Page 2: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/2.jpg)
Distinction between Type I and Type III Sum of Squares
This printout is from Proc GLM using 5 Factors with two levels each. The dependent variable is House Price. Notice the section with Type I SS and Type III SS.
A - F tests are sequential G – K tests
assume that other variables are in model
L - Q t-tests are based on Type III SS . Directional hypotheses can be tested.
Test A is the overall F test
![Page 3: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/3.jpg)
Type I and III Tests Type I tests are sequential tests based on the order of
variables inputted. It is assumed that those variables listed above the variable being tested are included in the model.
Type III tests assume that all of the other variables are included in the model. Usually Type III sums of squares are the values presented in published articles.
Note that for the F tests in tests F and K (see slide #2) are the same.
![Page 4: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/4.jpg)
t tests (slide #2, tests L through Q)
Note that the p-values for t-tests for the parameter estimates are the same as for the Type III F tests. This is because the t tests assume that all other variables are in the model.
Note that the square of the t statistic is equal to the Type III F statistic. The reason that the Parameter tests are provided is to display the regression coefficient and its standard error.
![Page 5: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/5.jpg)
For a Factorial Experiment with an Orthogonal Design, Type I = Type III
For an orthogonal design, the Type I sum of squares and the Type III sum of squares will be the same since the sums of squares are independent.
![Page 6: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/6.jpg)
Measures of Strength of Association
For a CRD, omega squared 2 for the fixed effects model and the intraclass correlation I for the random effects model are defined as follows:
2 / (
2 + 2)
This measure indicates the proportion of the population variance in the dependent variable that is accounted for by the treatment levels.
effect variance
error variance
![Page 7: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/7.jpg)
Cohen (1988) suggested guidelines for omega square
2 = .010 is a small association 2= .059 ( approx .06) is a medium
association 2= .138 ( approx .14) is a large association
![Page 8: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/8.jpg)
Formulas for 2
For Fixed Effects Models: 2= (SSB – (p-1)MSE ) / (SSTO + MSE) 2= (p-1)(F-1) / ((p-1)(F-1) + np)
For Random Effects Models: = (MSB – MSE ) / (MSB + (n-1)MSE) 2= (F-1) / (n-1 + F) is the intraclass correlation of the random
treatments
![Page 9: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/9.jpg)
Effect Size
Cohen (1988) popularized the term effect size. It is a standardized difference between the mean of a group and the overall mean.
d1 = ( – 1) /
The top graph has a larger effect size than the bottom graph.
Just a standardized difference.
![Page 10: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/10.jpg)
Formulas for Effect Size (represented by the letter f) First recall that E(MSB) =
2 + nj2/(p-1) for a fixed effects
model. f = sqrt(((j–)2/p)/
sqrt((j2/p)/
f = sqrt(2 / (1- 2)) Note that (j
2/p) can be estimated by
((p-1)/(np)) * (MSB – MSE). This comes from the E(MSB) above.
![Page 11: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/11.jpg)
Cohen (1988) suggested guidelines for the f measure of effect size
f = .10 is a small effect size f = .25 is a medium effect size f = .40 (or larger) is a large effect size
![Page 12: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/12.jpg)
Introduction to Calculation of Power
The F statistic has a noncentrality parameter = j
2/(nwhen the null hypothesis is false.
The Tang charts in TableE.12 can be used to compute the power. To use the charts, the value of must be entered:
sqrt(p) = sqrt( (j2/p)/(
n
Note that (j2/p) can be estimated by
((p-1)/(np)) * (MSB – MSE).
![Page 13: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/13.jpg)
How to Increase Power Power is related to the effect size (separation
of means), standard deviation of residuals (sigma), significance level (alpha), and sample size. In general, there are four ways to increase power: increase effect size decrease residual variance increase sample size increase alpha
![Page 14: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/14.jpg)
Example Using Tang’s Charts
Assume that the estimate of is 2.21 and that the numerator (1) and denominator degrees of freedom (2) for the F statistic are 3 and 28 respectively.
For an alpha value of .05, use the chart on page 818 and locate on the x-axis the tick mark after the 2 (that’s approx 2.21). Go up the chart vertically till you intersect the curve with 30 degrees of freedom (that’s the closest to 28 df). Approximate the power at .95.
![Page 15: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/15.jpg)
Estimating Sample Size for a Specified Value of Power
Use the Tang Charts by substituting different values of n into the formula
sqrt(n’) sqrt( (j2/p)/(
Use a pilot study to estimate
by MSE and
(j2/p) by ((p-1)/(np)) * (MSB – MSE).
![Page 16: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/16.jpg)
Two Other Ways to Estimate Sample Size Using Tang Charts
If there is no pilot study, figure out a desirable value for d (effect size) such as 1.5 then use
sqrt(n’) sqrt( d2/2p)
Or specify a value of 2 (strength of the relationship) and use
sqrt(n’) sqrt(2 / ( 1 - 2 ))
![Page 17: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/17.jpg)
Relationship of effect size f to noncentrality parameter
f = sqrt(np)) Note that np is the total number of observations.
SPSS provides the noncentrality parameter when an estimate of the effect size is requested. The actual effect size needs to be computed using the above formula.
![Page 18: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/18.jpg)
Test for Presence of Trend
If the Levels can be expressed quantitatively (in an order), then the mean of each level can be estimated by
j = o + c1j + 2c2j + 3c3j for a p = 4 level factor.
To check for Linear, Quadratic, and Cubic Trend components the following contrasts can be used.
Trend Component X1 X2 X3 X4Linear (c1j) -3 -1 1 3Quadratic (c2j) 1 -1 -1 1Cubic (c3j) -1 3 -3 1
![Page 19: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/19.jpg)
Computing Effect Size in SAS DM "Log;Clear;OUT;Clear;";
PROC IMPORT OUT= WORK.KirkPage171 /*******Read Data******/ DATAFILE= “D:\KirkPage171Data.xls" DBMS=EXCEL2000 REPLACE; RANGE="Sheet1$"; GETNAMES=Yes; RUN;
proc print data=KirkPage171;
proc glm data=KirkPage171; /******One Way ANOVA*****/ class Level; model Response = Level ; ods output overallanova = atable; run; quit;
Data compute_effect; /****Compute Omega and Effect Size***/ set atable end = eof; if _n_ = 1 then do; pminus1 = df; fv = fvalue -1; end; omega_sq = (pminus1)*fv/((pminus1)*fv + df + 1); effect_size = sqrt(omega_sq/(1-omega_sq)); if NOT eof then delete; Keep omega_sq effect_size; Retain pminus1 fv;
proc print data = compute_effect;
![Page 20: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/20.jpg)
To Get Power Estimate in SAS
data KirkPg171; input Level ResponseMean cellsize; datalines; 1 3 8 2 3.5 8 3 4.25 8 4 6.25 8 ; proc glmpower data= KirkPg171; class Level; model ResponseMean =Level; power alpha= .01 .05 stddev=1.476 ntotal=32 power=.;
![Page 21: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/21.jpg)
Output from SAS on Power
The GLMPOWER Procedure
Fixed Scenario Elements
Dependent Variable ResponseMean Source Level Error Standard Deviation 1.476 Total Sample Size 32 Test Degrees of Freedom 3 Error Degrees of Freedom 28
Computed Power
Index Alpha Power
1 0.01 0.880 2 0.05 0.972
![Page 22: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/22.jpg)
Power and Effect Size in SPSSClick on Analyze > General Linear Model > Univariate
![Page 23: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/23.jpg)
Click on Options and select Estimates of effect size and Observed Power
Power and Effect Size in SPSS
![Page 24: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/24.jpg)
Output from SPSS
Power is given. To estimate the f (effect size) convert the Noncentrality Parameter to f
![Page 25: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/25.jpg)
Estimate Trend Compoments Across Levels Using SAS
proc glm data=KirkPage171; class Level; model Response = Level /noint; estimate 'Linear' Level-3 -1 1 3; contrast 'Quad ' Level 1 -1 -1 1; estimate 'Cubic ' Level -1 3 -3 1; run; quit; proc glm data=KirkPage171; class Level; model Response = Level/noint; estimate 'Linear' Level -.671 -.224 .224 .671; contrast 'Quad ' Level .5 -.5 -.5 .5; estimate 'Cubic ' Level -.224 .671 -.671 .224; run; quit;
Note that the sum of the squared coefficients sum to 1 for the decimal coefficients.
![Page 26: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/26.jpg)
Estimating Trend Components Across Levels Using SPSSSelect Contrasts and Choose Polynomial. The menu does not allow for specific values of the Polynomial coefficients.
SPSS uses the following polynomial SPSS uses the following polynomial coefficients for 4 groups.coefficients for 4 groups.'Linear' Level -'Linear' Level -.671.671 - -.224.224 .224.224 .671.671;;'Quad ' Level 'Quad ' Level .5.5 - -.5.5 - -.5.5 .5.5;;'Cubic ' Level -'Cubic ' Level -.224.224 .671.671 - -.671.671 .224.224;;
![Page 27: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/27.jpg)
SAS Proc Insight – Use to Get Quick Look at Data
PROC IMPORT OUT= WORK.KirkPage171 DATAFILE= “D:\KirkPage171Data.xls" DBMS=EXCEL2000 REPLACE; RANGE="Sheet1$"; GETNAMES=Yes; RUN;
proc Insight; Open KirkPage171; Fit Response = Level; run; quit;
![Page 28: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/28.jpg)
Power Plot in SAS
proc glmpower data= KirkPg203Ex2; class Level; model ResponseMean =Level; power alpha=.01 .05 stddev= 2.6618 /*sqrt(MSE)*/ ntotal=30 power=.; plot x=n min=20 max=80;
Stddev of ANOVA and total sample size need to be specified.
![Page 29: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/29.jpg)
Plot for Alpha = .01 and .05
![Page 30: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/30.jpg)
Contrasts can be added to statements such as Proc Glmpower and Proc Multtest, but note the use of the word Level in both.
proc multtest data=myAnovaData Holm; class Level; /****Note don't type Level before coefficients as in Prog glmpower**/ contrast 'Grp1VersusGrp2' 1 -1 0; contrast 'Grp2VersusGrp3' 0 1 -1; contrast 'Grp1VersusGrp3' 1 0 -1;Test mean(resp); /*resp is the dependent variable*/
proc glmpower data= KirkPg203Ex2; class Level; model ResponseMean =Level; contrast 'Grp1VersusGrp2' Level 1 -1 0; /*Note the word Level*/ contrast 'Grp2VersusGrp3' Level 0 1 -1; contrast 'Grp1VersusGrp3' Level 1 0 -1; power alpha=.01 .05 stddev= 2.6618 /*sqrt(MSE)*/ ntotal=30 power=.; ods output output=mypoweroutput; run;
![Page 31: CRD, Strength of Association, Effect Size, Power, and Sample Size Calculations](https://reader036.fdocuments.net/reader036/viewer/2022062805/56814d40550346895dba65ed/html5/thumbnails/31.jpg)
Put a “.” for ntotal if you want the total sample size for a specified power
power alpha=.01 .05 stddev= 2.6618 /*sqrt(MSE)*/ ntotal= . power= .8;