1

Chapter 3: Experimental Design

2

Effect of Wine Consumption onHeart Disease Death Rate

R2 = 0.7103

0

50

100

150

200

250

300

350

0 2 4 6 8 10

Alcohol from wine (l/year)

Hea

rt d

isea

se d

eath

rat

ep

er 1

00,0

00 p

eop

le

**Each data point represents a different country.

3

What can we conclude?

• Is it safe to say that increasing wine consumption causes a decrease in heart disease death rate?

– Why or why not?

• What kind of study is this?

4

Experiment

• This calls for an experiment:

– We actively impose some treatment in order to observe the response.

– We need experimental control!

5

Example Experimental Design

• Randomly assign one group of students to hear music during study hall for the term. Another group does not get music.

– Did the music make a difference?

6

Example Experimental Design

• Explanatory variable —a variable we think explains or causes changes in a response variable.

• A specific experimental condition applied to the units is called a treatment.

– Example: music or no music

• Many times, we measure several response variables on the experimental units (called subjects when dealing with people).

7

Benefits of Using Experimental Design

• Experiments can give good evidence for causation.

• Experiments allow us to study the specific factors we are interested in, while controlling the effects of the lurking variables. They also allow us to study the combined effects of several factors, as we saw above.

• Experimental control!

– First basic principle of statistical design of experiments.

8

Placebo Effect

• Example 3.2, p. 137

– Placebo

– Control group

– Comparative experiment

• One track experiment … Bad!

9

HW

• Reading in section 3.1:

– pp. 133-149

10

Practice

• 3.1 and 3.2, p. 138

• 3.3 and 3.4, p. 139

11

Randomization

• Systematic differences among the groups of experimental units in a comparative experiment cause bias.

• Randomization is the statistician’s remedy against bias.

– Example 3.3, p. 139

12

Questions about Example 3.3

• Are the two groups guaranteed to be identical?

• Do you think it is wise to include many experimental units?

– We hope that we can have the effects of chance to average out with multiple experimental units.

– Replication!

13

Principles of Experimental Design

• Experimental control

• Randomize

• Replicate

14

Practice Exercise

• p. 143: 3.6

15

How to live with observational studies

• Read carefully pp. 145-146

– Comparative

– Matching

– Adjusting for effects of confounded variables

16

Statistical Significance

• Even if we have controlled properly, randomized, and replicated, we can still get differences among our experimental units on our response variables that are caused by chance.

• We use the laws of probability, which give mathematical descriptions of chance behavior, to learn if the treatment effects are larger than we would expect to see if only chance were operating.– If they are larger than to be expected by chance, we call

them statistically significant.

17

Practice and/or Homework

• Exercises:

– 3.8, p. 143

– 3.12, p. 147

• Reading:

– Through p. 150

• Goodly quiz on Section 3.1 on Monday.

18

Section 3.2: Experiments in the Real World

• Cautions about experimentation

• Double-blind experiments

• Matched pairs designs and block designs

19

Principles of Experimental Design

• Experimental control

• Randomize

• Replicate

20

Activity 3.2: The Mozart Effect

21

Confounding

• A lurking variable is a variable that has an important effect on the relationship among the variables in a study but is not one of the explanatory variables studied.

• Two variables are confounded when their effects on a response variable cannot be distinguished from each other.

22

Treating Each Subject Identically

• When we do a randomized comparative experiment we must take special care to see that each subject (or each experimental unit) is treated identically.

– Is there researcher influence? Did other things not in our control change during the course of the experiment (e.g., ambient conditions, machines, people, etc.)?

23

The Powerful Placebo

• Example 3.9, p. 155

• Double-blind experiments– Gold standard in medical research

– pp. 155-156

24

Problems in Experimental Research

• Non-adherers

• Dropouts

• Can we generalize?

– Lack of realism?

– Subjects being monitored/experimented on may or may not be like the population.

25

Experiments with MultipleResponse Variables

• Figure 3.4, p. 163

• A textile example…

Spindle Speed17,000 rpm

Spindle Speed19,000 rpm

CeramicRing

AlloyRing

CeramicRing

AlloyRing

CeramicTraveler

X X X X

AlloyTraveler

X X X X

26

Practice

• Exercises:

– 3.22 and 3.25. pp. 158-159

– 3.28 and 3.30, p. 164

• Make sure you have read all of section 3.2.

27

Experimental Control, Revisited

• What are some of the ways we have talked about that researchers use for experimental control?

• Definition of experimental control:

– Taking account of extraneous variables in the experimental design, most simply by the use of equivalent groups for comparison.

28

Matched Pairs Designs

• A matched pairs design helps in experimental control.

• Activity 3.2B, p. 165

• Exercise 3.31, p. 168

29

Block Designs

• Example 3.17, p. 167

• Blocks are a way of holding fixed an extraneous variable that would otherwise cause large variations in the experimental results.

– Benefits, bottom of p. 167

– Similar conceptually to stratified samples we discussed in observational studies (surveys).

• Exercise 3.32, p. 168

30

Practices Exercises

• Exercises:– 3.23, 3.25 p. 158

– 3.28, 3.30 p. 164

– 3.32, 3.33 p. 168

– 3.35, p. 169

• pp. 171-173:– 3.36

– 3.37

– 3.39

– 3.42