Center for Biofilm Engineering

Al Parker, BiostatisticianStandardized Biofilm Methods Research TeamMontana State University

The Importance of Statistical

Design and Analysis in the Laboratory

Feb, 2011

Standardized Biofilm Methods Laboratory

Darla GoeresAl Parker

Marty Hamilton

Diane Walker

Lindsey Lorenz

Paul Sturman

Kelli Buckingham-Meyer

What is statistical thinking?

Data

Experimental Design

Uncertainty and variability assessment

What is statistical thinking?

Data (pixel intensity in an image? log(cfu) from viable plate counts?)

Experimental Design - controls - randomization- replication (How many coupons?

experiments? technicians? labs?)

Uncertainty and variability assessment

Why statistical thinking?

Anticipate criticism (design method and experiments accordingly)

Provide convincing results (establish statistical properties)

Increase efficiency (conduct the least number of experiments)

Improve communication

Why statistical thinking?

Standardized Methods

Attributes of a standard method: Seven R’s

Relevance

Reasonableness

Resemblance

Repeatability (intra-laboratory)

Ruggedness

Responsiveness

Reproducibility (inter-laboratory)

Attributes of a standard method: Seven R’s

Relevance

Reasonableness

Resemblance

Repeatability (intra-laboratory)

Ruggedness

Responsiveness

Reproducibility (inter-laboratory)

Resemblance of Controls

Independent repeats of the same experiment in the same laboratory produce nearly the same control data, as indicated by a small

repeatability standard deviation.

Statistical tool:

nested analysis of variance (ANOVA)

• 86 mm x 128 mm plastic plate with 96 wells• Lid has 96 pegs

Resemblance Example: MBEC

1 2 3 4 5 6 7 8 9 10 11 12

A 100 100 100 100 100 50:N N GC SC

B 50 50 50 50 50 50:N N GC SC

C 25 25 25 25 25 50:N N GC SC

D 12.5 12.5 12.5 12.5 12.5 50:N N GC

E 6.25 6.25 6.25 6.25 6.25 50:N N GC

F 3.125 3.125 3.125 3.125 3.125 50:N N GC

G 1.563 1.563 1.563 1.563 1.563 50:N N GC

H 0.781 0.781 0.781 0.781 0.781 50:N N GC

MBEC Challenge Plate

disinfectant neutralizer test control

Resemblance Example: MBEC

Mean LD= 5.55

Control Data: log10(cfu/mm2) from viable plate counts

row cfu/mm2 log(cfu/mm2)A 5.15 x 105 5.71B 9.01 x 105 5.95C 6.00 x 105 5.78D 3.00 x 105 5.48E 3.86 x 105 5.59F 2.14 x 105 5.33G 8.58 x 104 4.93H 4.29 x 105 5.63

Exp RowControl

LDMean

LD SD1 A 5.71

5.55 0.311 B 5.951 C 5.781 D 5.481 E 5.591 F 5.331 G 4.931 H 5.63

2 A 5.41

5.41 0.172 B 5.712 C 5.542 D 5.332 E 5.112 F 5.482 G 5.332 H 5.41

Resemblance Example: MBEC

Resemblance from experiment to experiment

Mean LD = 5.48

Sr = 0.26

the typical distance between a control well LD from an experiment and the true mean LD

Resemblance from experiment to experiment

The variance Sr2

can be partitioned:

2% due to between experiment sources

98% due to within experiment sources

S

nc • m

c2

+

Formula for the SE of the mean control LD, averaged over experiments

Sc = within-experiment variance of control LDs

SE = among-experiment variance of control LDs

nc = number of control replicates per experiment

m = number of experiments

2

2

S

m

E2

SE of mean control LD =

CI for the true mean control LD = mean LD ± tm-1 x SE

8 • 2

Formula for the SE of the mean control LD, averaged over experiments

Sc = 0.98 x (0.26)2 = 0.00124

SE = 0.02 x (0.26)2 = 0.06408

nc = 8

m = 2

2

2

2SE of mean control LD =

0.00124+

0.06408= 0.1792

95% CI for the true mean control LD = 5.48 ± 12.7 x 0.1792

= (3.20, 7.76)

Resemblance from technician to technician

Mean LD = 5.44

Sr = 0.36

the typical distance between a control well LD and the true mean LD

The variance Sr2

can be partitioned:

0% due to technician sources

24% due to between experiment sources

76% due to within experiment sources

Resemblance from technician to technician

Repeatability

Independent repeats of the same experiment in the same laboratory produce nearly the same data, as indicated by a small repeatability standard deviation.

Statistical tool: nested ANOVA

Repeatability Example

Data: log reduction (LR)

LR = mean(control LDs) – mean(disinfected LDs)

Exp RowControl

LDMean

LD SD1 A 5.71

5.55 0.311 B 5.951 C 5.781 D 5.481 E 5.591 F 5.331 G 4.931 H 5.63

2 A 5.41

5.41 0.172 B 5.712 C 5.542 D 5.332 E 5.112 F 5.482 G 5.332 H 5.41

Repeatability Example: MBEC

1 2 3 4 5 6 7 8 9 10 11 12A 100 100 100 100 100 50:N N GC SC

B 50 50 50 50 50 50:N N GC SC

C 25 25 25 25 25 50:N N GC SC

D 12.5 12.5 12.5 12.5 12.5 50:N N GC

E 6.25 6.25 6.25 6.25 6.25 50:N N GC

F 3.125 3.125 3.125 3.125 3.125 50:N N GC

G 1.563 1.563 1.563 1.563 1.563 50:N N GC

H 0.781 0.781 0.781 0.781 0.781 50:N N GC

Repeatability Example: MBEC

Mean LR = 1.63

Exp RowControl

LDControl

Mean LD ColDisinfected 6.25% LD

Disinfected Mean LD LR

1 A 5.71

5.55 4.51 1.04

1 B 5.95 1 4.671 C 5.78 2 4.411 D 5.48 3 4.331 E 5.59 4 4.591 F 5.33 5 4.541 G 4.931 H 5.63

2 A 5.41

5.41 3.20 2.21

2 B 5.71 1 4.782 C 5.54 2 2.712 D 5.33 3 3.482 E 5.11 4 3.232 F 5.48 5 1.822 G 5.332 H 5.41

Repeatability Example

Mean LR = 1.63

Sr = 0.83

the typical distance between a LR for an experiment and the true mean LR

S

nc • m

c2

+

Formula for the SE of the mean LR, averaged over experiments

Sc = within-experiment variance of control LDs

Sd = within-experiment variance of disinfected LDs

SE = among-experiment variance of LRs

nc = number of control replicates per experiment

nd = number of disinfected replicates per experiment

m = number of experiments

2

2

2

S

nd • m

d2

+S

m

E2

SE of mean LR =

Formula for the SE of the mean LR, averaged over experiments

Sc = within-experiment variance of control LDs

Sd = within-experiment variance of disinfected LDs

SE = among-experiment variance of LRs

nc = number of control replicates per experiment

nd = number of disinfected replicates per experiment

m = number of experiments

2

2

2

CI for the true mean LR = mean LR ± tm-1 x SE

Formula for the SE of the mean LR, averaged over experiments

Sc2 = 0.00124

Sd2 = 0.47950

SE2 = 0.59285

nc = 8, nd = 5, m = 2

SE of mean LR =

8 • 2 2

0.00124+

0.59285

5 • 2

0.47950+ = 0.5868

95% CI for the true mean LR = 1.63 ± 12.7 x 0.5868

= 1.63 ± 7.46

= (0.00, 9.09)

How many coupons? experiments?

nc • m m

0.00124+

0.59285

nd • m

0.47950+margin of error= tm-1 x

no. control coupons (nc): 2 3 5 8 12no. disinfected coupons (nd): 2 3 5 5 12

no. experiments (m) 2 8.20 7.80 7.46 7.46 7.163 2.27 2.15 2.06 2.06 1.974 1.45 1.38 1.32 1.32 1.276 0.96 0.91 0.87 0.87 0.84

10 0.65 0.62 0.59 0.59 0.57100 0.18 0.17 0.16 0.16 0.16

A method should be sensitive enough that it can detect important changes in parameters of interest.

Statistical tool: regression and t-tests

Responsiveness

disinfectant neutralizer test control

Responsiveness Example: MBEC

A: High Efficacy

H: Low Efficacy

1 2 3 4 5 6 7 8 9 10 11 12

A 100 100 100 100 100 50:N N GC SC

B 50 50 50 50 50 50:N N GC SC

C 25 25 25 25 25 50:N N GC SC

D 12.5 12.5 12.5 12.5 12.5 50:N N GC

E 6.25 6.25 6.25 6.25 6.25 50:N N GC

F 3.125 3.125 3.125 3.125 3.125 50:N N GC

G 1.563 1.563 1.563 1.563 1.563 50:N N GC

H 0.781 0.781 0.781 0.781 0.781 50:N N GC

Responsiveness Example: MBEC

This response curve indicates responsiveness to decreasing efficacy between rowsC, D, E and F

Responsiveness Example: MBEC

Responsiveness can be quantified with a regression line:

LR = 6.08 - 0.97row

For each step in the decrease of disinfectant efficacy, the LR decreases on average by 0.97.

Summary

Even though biofilms are complicated, it is feasible to develop biofilm methods that meet the “Seven R” criteria.

Good experiments use control data! Assess uncertainty by SEs and CIs.

When designing experiments, invest effort in more experiments versus more replicates (coupons or wells) within an experiment.

Any questions?