Post on 26-Dec-2015
Center for Biofilm Engineering
Al Parker, BiostatisticianStandardized Biofilm Methods Research TeamMontana State University
The Importance of Statistical
Design and Analysis in the Laboratory
Feb, 2011
Standardized Biofilm Methods Laboratory
Darla GoeresAl Parker
Marty Hamilton
Diane Walker
Lindsey Lorenz
Paul Sturman
Kelli Buckingham-Meyer
What is statistical thinking?
Data (pixel intensity in an image? log(cfu) from viable plate counts?)
Experimental Design - controls - randomization- replication (How many coupons?
experiments? technicians? labs?)
Uncertainty and variability assessment
Why statistical thinking?
Anticipate criticism (design method and experiments accordingly)
Provide convincing results (establish statistical properties)
Increase efficiency (conduct the least number of experiments)
Improve communication
Attributes of a standard method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Ruggedness
Responsiveness
Reproducibility (inter-laboratory)
Attributes of a standard method: Seven R’s
Relevance
Reasonableness
Resemblance
Repeatability (intra-laboratory)
Ruggedness
Responsiveness
Reproducibility (inter-laboratory)
Resemblance of Controls
Independent repeats of the same experiment in the same laboratory produce nearly the same control data, as indicated by a small
repeatability standard deviation.
Statistical tool:
nested analysis of variance (ANOVA)
1 2 3 4 5 6 7 8 9 10 11 12
A 100 100 100 100 100 50:N N GC SC
B 50 50 50 50 50 50:N N GC SC
C 25 25 25 25 25 50:N N GC SC
D 12.5 12.5 12.5 12.5 12.5 50:N N GC
E 6.25 6.25 6.25 6.25 6.25 50:N N GC
F 3.125 3.125 3.125 3.125 3.125 50:N N GC
G 1.563 1.563 1.563 1.563 1.563 50:N N GC
H 0.781 0.781 0.781 0.781 0.781 50:N N GC
MBEC Challenge Plate
disinfectant neutralizer test control
Resemblance Example: MBEC
Mean LD= 5.55
Control Data: log10(cfu/mm2) from viable plate counts
row cfu/mm2 log(cfu/mm2)A 5.15 x 105 5.71B 9.01 x 105 5.95C 6.00 x 105 5.78D 3.00 x 105 5.48E 3.86 x 105 5.59F 2.14 x 105 5.33G 8.58 x 104 4.93H 4.29 x 105 5.63
Exp RowControl
LDMean
LD SD1 A 5.71
5.55 0.311 B 5.951 C 5.781 D 5.481 E 5.591 F 5.331 G 4.931 H 5.63
2 A 5.41
5.41 0.172 B 5.712 C 5.542 D 5.332 E 5.112 F 5.482 G 5.332 H 5.41
Resemblance Example: MBEC
Resemblance from experiment to experiment
Mean LD = 5.48
Sr = 0.26
the typical distance between a control well LD from an experiment and the true mean LD
Resemblance from experiment to experiment
The variance Sr2
can be partitioned:
2% due to between experiment sources
98% due to within experiment sources
S
nc • m
c2
+
Formula for the SE of the mean control LD, averaged over experiments
Sc = within-experiment variance of control LDs
SE = among-experiment variance of control LDs
nc = number of control replicates per experiment
m = number of experiments
2
2
S
m
E2
SE of mean control LD =
CI for the true mean control LD = mean LD ± tm-1 x SE
8 • 2
Formula for the SE of the mean control LD, averaged over experiments
Sc = 0.98 x (0.26)2 = 0.00124
SE = 0.02 x (0.26)2 = 0.06408
nc = 8
m = 2
2
2
2SE of mean control LD =
0.00124+
0.06408= 0.1792
95% CI for the true mean control LD = 5.48 ± 12.7 x 0.1792
= (3.20, 7.76)
Resemblance from technician to technician
Mean LD = 5.44
Sr = 0.36
the typical distance between a control well LD and the true mean LD
The variance Sr2
can be partitioned:
0% due to technician sources
24% due to between experiment sources
76% due to within experiment sources
Resemblance from technician to technician
Repeatability
Independent repeats of the same experiment in the same laboratory produce nearly the same data, as indicated by a small repeatability standard deviation.
Statistical tool: nested ANOVA
Exp RowControl
LDMean
LD SD1 A 5.71
5.55 0.311 B 5.951 C 5.781 D 5.481 E 5.591 F 5.331 G 4.931 H 5.63
2 A 5.41
5.41 0.172 B 5.712 C 5.542 D 5.332 E 5.112 F 5.482 G 5.332 H 5.41
Repeatability Example: MBEC
1 2 3 4 5 6 7 8 9 10 11 12A 100 100 100 100 100 50:N N GC SC
B 50 50 50 50 50 50:N N GC SC
C 25 25 25 25 25 50:N N GC SC
D 12.5 12.5 12.5 12.5 12.5 50:N N GC
E 6.25 6.25 6.25 6.25 6.25 50:N N GC
F 3.125 3.125 3.125 3.125 3.125 50:N N GC
G 1.563 1.563 1.563 1.563 1.563 50:N N GC
H 0.781 0.781 0.781 0.781 0.781 50:N N GC
Repeatability Example: MBEC
Mean LR = 1.63
Exp RowControl
LDControl
Mean LD ColDisinfected 6.25% LD
Disinfected Mean LD LR
1 A 5.71
5.55 4.51 1.04
1 B 5.95 1 4.671 C 5.78 2 4.411 D 5.48 3 4.331 E 5.59 4 4.591 F 5.33 5 4.541 G 4.931 H 5.63
2 A 5.41
5.41 3.20 2.21
2 B 5.71 1 4.782 C 5.54 2 2.712 D 5.33 3 3.482 E 5.11 4 3.232 F 5.48 5 1.822 G 5.332 H 5.41
Repeatability Example
Mean LR = 1.63
Sr = 0.83
the typical distance between a LR for an experiment and the true mean LR
S
nc • m
c2
+
Formula for the SE of the mean LR, averaged over experiments
Sc = within-experiment variance of control LDs
Sd = within-experiment variance of disinfected LDs
SE = among-experiment variance of LRs
nc = number of control replicates per experiment
nd = number of disinfected replicates per experiment
m = number of experiments
2
2
2
S
nd • m
d2
+S
m
E2
SE of mean LR =
Formula for the SE of the mean LR, averaged over experiments
Sc = within-experiment variance of control LDs
Sd = within-experiment variance of disinfected LDs
SE = among-experiment variance of LRs
nc = number of control replicates per experiment
nd = number of disinfected replicates per experiment
m = number of experiments
2
2
2
CI for the true mean LR = mean LR ± tm-1 x SE
Formula for the SE of the mean LR, averaged over experiments
Sc2 = 0.00124
Sd2 = 0.47950
SE2 = 0.59285
nc = 8, nd = 5, m = 2
SE of mean LR =
8 • 2 2
0.00124+
0.59285
5 • 2
0.47950+ = 0.5868
95% CI for the true mean LR = 1.63 ± 12.7 x 0.5868
= 1.63 ± 7.46
= (0.00, 9.09)
How many coupons? experiments?
nc • m m
0.00124+
0.59285
nd • m
0.47950+margin of error= tm-1 x
no. control coupons (nc): 2 3 5 8 12no. disinfected coupons (nd): 2 3 5 5 12
no. experiments (m) 2 8.20 7.80 7.46 7.46 7.163 2.27 2.15 2.06 2.06 1.974 1.45 1.38 1.32 1.32 1.276 0.96 0.91 0.87 0.87 0.84
10 0.65 0.62 0.59 0.59 0.57100 0.18 0.17 0.16 0.16 0.16
A method should be sensitive enough that it can detect important changes in parameters of interest.
Statistical tool: regression and t-tests
Responsiveness
disinfectant neutralizer test control
Responsiveness Example: MBEC
A: High Efficacy
H: Low Efficacy
1 2 3 4 5 6 7 8 9 10 11 12
A 100 100 100 100 100 50:N N GC SC
B 50 50 50 50 50 50:N N GC SC
C 25 25 25 25 25 50:N N GC SC
D 12.5 12.5 12.5 12.5 12.5 50:N N GC
E 6.25 6.25 6.25 6.25 6.25 50:N N GC
F 3.125 3.125 3.125 3.125 3.125 50:N N GC
G 1.563 1.563 1.563 1.563 1.563 50:N N GC
H 0.781 0.781 0.781 0.781 0.781 50:N N GC
Responsiveness Example: MBEC
This response curve indicates responsiveness to decreasing efficacy between rowsC, D, E and F
Responsiveness Example: MBEC
Responsiveness can be quantified with a regression line:
LR = 6.08 - 0.97row
For each step in the decrease of disinfectant efficacy, the LR decreases on average by 0.97.
Summary
Even though biofilms are complicated, it is feasible to develop biofilm methods that meet the “Seven R” criteria.
Good experiments use control data! Assess uncertainty by SEs and CIs.
When designing experiments, invest effort in more experiments versus more replicates (coupons or wells) within an experiment.