Biostatistics in PracticePeter D. ChristensonBiostatistician

http://gcrc.LABioMed.org/BiostatSession 4: Study Size and Power

Readings for Session 4from StatisticalPractice.com Sample Size Calculations

Some underlying theory and some practical advice.

Controlled trials

Outline for this SessionExample from a current local protocol.Review statistical hypothesis testing.Formulate example as hypothesis test.Software for study size and power.Other issues.

Local Protocol ExampleBrief study outline:Subjects arrive at ER with TBI (traumatic brain injury). Those with low cortisol, indicating possible adrenal insufficiency and pituitary damage, may or may not recover better if given hydrocortisone (HC) injections.Subjects who consent are randomized to receive HC or placebo for 4 days. Changes in recovery status from pre to post injection periods are compared between HC and placebo groups.

Project #10038: Dan Kelly & Pejman CohanHypopituitarism after Moderate and Severe Head Injury

Local Protocol Example, ContdThe primary outcomes for the hydrocortisone trial are changes in mean MAP and vasopressor use from the 12 hours prior to initiation of randomized treatment to the 96 hours after initiation.Mean changes in placebo subjects will be compared with hydrocortisone subjects using a two sample t-test.Project #10038: Dan Kelly & Pejman CohanHypopituitarism after Moderate and Severe Head InjuryBefore examining the study size, lets first discuss how the results will be analyzed.

Recall Statistical (t) test From Last SessionSuppose results from the study are plotted as:Is large enough to claim that HC is more effective? Use t-test.HCPlaceboChange in MAPEach point is the change in MAP for an individual subject.[Of course, the real study will have many more subjects.]

Local Protocol Example: Analysis with t-testWe are testing:H0: HC- Placebo = 0 vs. HA: HC- Placebo 0where HC is the expected post-pre change in all potential TBI patients if HC therapy is applied as in this study. Our decision rule is: Choose HA if the estimate of HC- Placebo from our limited sample, i.e., the observed mean change under HC minus the observed mean change under placebo, call it , is too far from 0 (which is specified by H0). Too far is > tc*SE or < tc*SE, where tc is usually about 2. SE is SE(), calculated from the data, and is for larger N and smaller SD.In other words, choose HA if || > tc*SE, or |t|=|/SE| > tc.By following this rule, there is only a 5% probability of choosing HA if in fact H0 is true.

Potentially Underpowered StudiesFrom the previous slide:By following this rule, there is only a 5% probability of choosing HA if in fact H0 is true.So, the probability is small (5%) that our study will (incorrectly) recommend that TBI subjects receive HC if it is worthless.But, is it able to correctly recommend that TBI subjects receive HC if it is effective? The probability of this is called the power of the study.Actually, there is not a single value for power. The study may have, say, 59% power if the true mean HC effect is 3 mmHg in MAP, but will have more power if the true effect is 4, since the subjects are more likely to reflect this greater effectiveness.Lets go back to last sessions graph to see this.

Graphical Representation of PowerH0HAH0: true effect=0HA: true effect=3Effect in study=1.13\\\ = Probability of concluding HA if H0 is true./// = Probability of concluding H0 if HA is true. Power=100-41=59%Note greater power if larger N, and/or if true effect>3.41%5%Effect (HC change Placebo change)

P-ValueRecall that our decision rule is: Choose HA if || > tc*SE, or |t|=|/SE| > tc.By following this rule, there is only a 5% probability of choosing HA if in fact H0 is true.In practice, though, we do not just report our decision as HA or H0.The p-value is the probability , if H0 is correct, that we would observe a as far from 0 as actually eventually occurred in the study. Here, p=Prob(>1.13), which is the area under H0 to the right of the green line in the previous figure.Small p-values support HA. Choosing HA is equivalent to p

Summary: Factors that Determine Study SizeFive factors including power are inter-related. Fixing four of these specifies the fifth:

Study size, N.

Power (often 80% is desirable).

p-value (level of significance, e.g., 0.05).

Magnitude of treatment effect to be detected.

Heterogeneity among subjects (standard deviation, SD).The next slide shows how these factors (except SD) are typically presented in a study protocol.

Quote from Local Protocol ExampleThus, with a total of the planned 80 subjects, we are 80% sure to detect (p

Comments on the Table on Previous Slide Typically power=80% and almost always p

Software for Study Size Calculations Calculations depend on the specific statistical method. We are using the t-test as an example, but the same concepts apply for, say, comparing % subjects who respond to treatment using another method such as a chi-square test.

In software, you specify the method, and 4 of the 5 factors. The value of the fifth factor is calculated.

Two free sites for calculations:1. http://calculators.stat.ucla.edu/powercalc2. http://www.stat.uiowa.edu/~rlenth/Power

A Software Site for Study Size Calculations

Local Protocol Example: CalculationsPilot data: SD=8.16 for MAP in 36 subjects.For p-value

Summary: Study Size and Power Power analysis assures that effects of a specified magnitude can be detected.

Five factors including power are inter-related. Fixing four of these specifies the fifth.

For comparing means, need pilot or data from other studies on variability of subjects for the outcome measure. [E.g., Std dev from previous study.] Comparing rates (%s) does not require pilot variability data. Use if no pilot data is available for means.

Helps support the believability of (superiority) studies if the conclusions turn out to be negative.

To prove no effect (e.g., that a less invasive therapy is equally as effective as standard care), use an equivalency study design.

Self-Test Exercise #1Go to www.stat.uiowa.edu/~rlenth/Power.

Select Type of Analysis = Two sample t-test.

Reproduce the detectable difference between placebo and HC treatment in change in MAP of 6.0 mm when using 30+30=60 subjects in the local example table that appears 6 slides back. Note that SD for MAP change was 8.16 in the pilot study.

Notes: Sigma refers to SD. Check equal sigmas, which assumes that SD is the same for both HC and placebo groups. Do not check equivalence.

Self-Test Exercise #2Go to www.stat.uiowa.edu/~rlenth/Power.

Select Type of Analysis = Test comparing two proportions.

Suppose that the outcome for the local example is not mean magnitude of change in MAP or vasopressor use, but instead is the proportion of subjects with an MAP reduction of at least 5 mm. If true such proportions among all potential TBI subjects who do or dont receive HC therapy are 30% and 60%, how many subjects are needed to be 80% sure to declare a HC effect with p

Self-Test Exercise #3A study was powered to detect a 10 point mean reduction in LDL cholesterol. A colleague claims that this means that if the subjects decrease LDL cholesterol by a mean 10 points, then p

Self-Test Exercise #4True story:

A protocol was designed with 80% power to detect (p