Experimental Design. Experimental design is the part of statistics that happens before you carry out...
-
date post
20-Dec-2015 -
Category
Documents
-
view
213 -
download
0
Transcript of Experimental Design. Experimental design is the part of statistics that happens before you carry out...
![Page 1: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/1.jpg)
Experimental Design
![Page 2: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/2.jpg)
Experimental Design
• Experimental design is the part of statistics that happens before you carry out an experiment
• Proper planning can save many headaches
• You should design your experiments with a particular statistical test in mind
![Page 3: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/3.jpg)
Why do experiments?
• Contrast: observational study vs. experiments
• Example: – Observational studies show a positive
association between ice cream sales and levels of violent crime
– What does this mean?
![Page 4: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/4.jpg)
Why do experiments?
• Contrast: observational study vs. experiments
• Example: – Observational studies show a positive
association between ice cream sales and levels of violent crime
– What does this mean?
![Page 5: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/5.jpg)
Alternative explanation
Hot weather
Ice cream
Violentcrime
![Page 6: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/6.jpg)
Alternative explanation
Hot weather
Ice cream
Violentcrime
Correlation is not causation
![Page 7: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/7.jpg)
Why do experiments?
• Observational studies are prone to confounding variables: Variables that mask or distort the association between measured variables in a study– Example: hot weather
• In an experiment, you can use random assignments of treatments to individuals to avoid confounding variables
![Page 8: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/8.jpg)
Goals of Experimental Design
• Avoid experimental artifacts• Eliminate bias
1. Use a simultaneous control group2. Randomization3. Blinding
• Reduce sampling error1. Replication2. Balance3. Blocking
![Page 9: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/9.jpg)
Goals of Experimental Design
• Avoid experimental artifacts• Eliminate bias
1. Use a simultaneous control group2. Randomization3. Blinding
• Reduce sampling error1. Replication2. Balance3. Blocking
![Page 10: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/10.jpg)
Experimental Artifacts
• Experimental artifacts: a bias in a measurement produced by unintended consequences of experimental procedures
• Conduct your experiments under as natural of conditions as possible to avoid artifacts
![Page 11: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/11.jpg)
Goals of Experimental Design
• Avoid experimental artifacts• Eliminate bias
1. Use a simultaneous control group2. Randomization3. Blinding
• Reduce sampling error1. Replication2. Balance3. Blocking
![Page 12: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/12.jpg)
Control Group
• A control group is a group of subjects left untreated for the treatment of interest but otherwise experiencing the same conditions as the treated subjects
• Example: one group of patients is given an inert placebo
![Page 13: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/13.jpg)
The Placebo Effect
• Patients treated with placebos, including sugar pills, often report improvement
• Example: up to 40% of patients with chronic back pain report improvement when treated with a placebo
• Even “sham surgeries” can have a positive effect
• This is why you need a control group!
![Page 14: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/14.jpg)
Randomization
• Randomization is the random assignment of treatments to units in an experimental study
• Breaks the association between potential confounding variables and the explanatory variables
![Page 15: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/15.jpg)
Experimental unitsC
onfo
undi
ng v
aria
ble
![Page 16: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/16.jpg)
Experimental unitsC
onfo
undi
ng v
aria
ble
Treatments
![Page 17: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/17.jpg)
Experimental unitsC
onfo
undi
ng v
aria
ble
Treatments
Without randomization, the confounding variable differs among treatments
![Page 18: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/18.jpg)
Experimental unitsC
onfo
undi
ng v
aria
ble
Treatments
![Page 19: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/19.jpg)
Experimental unitsC
onfo
undi
ng v
aria
ble
Treatments
With randomization, the confounding variable does not differ among treatments
![Page 20: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/20.jpg)
Blinding
• Blinding is the concealment of information from the participants and/or researchers about which subjects are receiving which treatments
• Single blind: subjects are unaware of treatments
• Double blind: subjects and researchers are unaware of treatments
![Page 21: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/21.jpg)
Goals of Experimental Design
• Avoid experimental artifacts• Eliminate bias
1. Use a simultaneous control group2. Randomization3. Blinding
• Reduce sampling error1. Replication2. Balance3. Blocking
![Page 22: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/22.jpg)
Replication
• Experimental unit: the individual unit to which treatments are assigned
Experiment 1
Experiment 2
Experiment 3
Tank 1 Tank 2
All separate tanks
![Page 23: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/23.jpg)
Replication
• Experimental unit: the individual unit to which treatments are assigned
Experiment 1
Experiment 2
Experiment 3
Tank 1 Tank 2
All separate tanks
2 Experimental Units
2 Experimental Units
8 Experimental Units
![Page 24: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/24.jpg)
Replication
• Experimental unit: the individual unit to which treatments are assigned
Experiment 1
Experiment 2
Experiment 3
Tank 1 Tank 2
All separate tanks
2 Experimental Units
2 Experimental Units
8 Experimental Units
Pseudoreplication
![Page 25: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/25.jpg)
Why is pseudoreplication bad?
• problem with confounding and replication!
• Imagine that something strange happened, by chance, to tank 2 but not to tank 1
• Example: light burns out
• All four lizards in tank 2 would be smaller
• You might then think that the difference was due to the treatment, but it’s actually just random chance
Experiment 2
Tank 1 Tank 2
![Page 26: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/26.jpg)
Why is replication good?
• Consider the formula for standard error of the mean:
SEY
s
n
Larger n Smaller SE
![Page 27: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/27.jpg)
Balance
• In a balanced experimental design, all treatments have equal sample size
Better than
Balanced Unbalanced
![Page 28: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/28.jpg)
Balance
• In a balanced experimental design, all treatments have equal sample size
• This maximizes power
• Also makes tests more robust to violating assumptions
![Page 29: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/29.jpg)
Blocking
• Blocking is the grouping of experimental units that have similar properties
• Within each block, treatments are randomly assigned to experimental treatments
• Randomized block design
![Page 30: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/30.jpg)
Randomized Block Design
![Page 31: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/31.jpg)
Randomized Block Design
• Example: tanks in a field
![Page 32: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/32.jpg)
![Page 33: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/33.jpg)
Very sunny
Not So Sunny
![Page 34: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/34.jpg)
Block 1
Block 4
Block 2
Block 3
![Page 35: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/35.jpg)
What good is blocking?
• Blocking allows you to remove extraneous variation from the data
• Like replicating the whole experiment multiple times, once in each block
• Paired design is an example of blocking
![Page 36: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/36.jpg)
Experiments with 2 Factors
• Factorial design – investigates all treatment combinations of two or more variables
• Factorial design allows us to test for interactions between treatment variables
![Page 37: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/37.jpg)
Factorial Design
5.5 6.5 7.5
25 n=2 n=2 n=2
30 n=2 n=2 n=2
35 n=2 n=2 n=2
40 n=2 n=2 n=2
Tem
pera
ture
pH
![Page 38: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/38.jpg)
Interaction Effects
• An interaction between two (or more) explanatory variables means that the effect of one variable depends upon the state of the other variable
![Page 39: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/39.jpg)
Interpretations of 2-way ANOVA Terms
0
10
20
30
40
50
60
70
25 30 35 40
Temperature
Gro
wth
Rate
pH 5.5
pH 6.5
pH 7.5
Effect of pH and Temperature,No interaction
![Page 40: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/40.jpg)
0
5
10
15
20
25
30
35
40
45
25 30 35 40
Temperature
Gro
wth
Rate
pH 5.5
pH 6.5
pH 7.5
Interpretations of 2-way ANOVA Terms
Effect of pH and Temperature,with interaction
![Page 41: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/41.jpg)
Goals of Experimental Design
• Avoid experimental artifacts• Eliminate bias
1. Use a simultaneous control group2. Randomization3. Blinding
• Reduce sampling error1. Replication2. Balance3. Blocking
![Page 42: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/42.jpg)
What if you can’t do experiments?
• Sometimes you can’t do experiments
• One strategy:– Matching– Every individual in the treatment group is
matched to a control individual having the same or closely similar values for known confounding variables
![Page 43: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/43.jpg)
What if you can’t do experiments?
• Example: Do species on islands change their body size compared to species in mainland habitats?
• For each island species, identify a closely related species living on a nearby mainland area
![Page 44: Experimental Design. Experimental design is the part of statistics that happens before you carry out an experiment Proper planning can save many headaches.](https://reader031.fdocuments.net/reader031/viewer/2022032800/56649d4c5503460f94a2a2a3/html5/thumbnails/44.jpg)
Power Analysis
• Before carrying out an experiment you must choose a sample size
• Too small: no chance to detect treatment effect
• Too large: too expensive
• We can use power analysis to choose our sample size