CHAPTER 5: PRODUCING DATA

DESIGNING EXPERIMENTS

Objectives: To understand and use the

statistical vocabulary involved in designing experiments.

To design experiments that come as close as possible to eliminating lurking variables and bias.

EXPERIMENTAL UNITS, SUBJECTS, TREATMENT

A study is an experiment when we actually do something to people, animals, or objects in order to observe the response.

The individuals on which the experiment is done are the experimental units.

When the units are human beings, they

are called subjects.

A specific experimental condition applied to the units is called a treatment.

ADDITIONAL VOCABULARY The explanatory variables in an experiment are often

called factors because we want to study the effects that these factors have on the response variable.

In many experiments we want to study the joint effects of several factors. In such an experiment, each treatment is formed

by combining a specific value (often called a level) of each of the factors.

See example 5.9 on p.290

EXPERIMENTS A placebo is a dummy pill that creates a situation

so the experimenter can see what the effect of the subject will be just by being treated.

In principle, experiments can give good evidence for causation.

Experiments have the advantage of allowing us to study the specific factors we are interested in, while controlling the effects of lurking variables. For example, the subjects in the Physicians’ Health

Study were all middle-aged male doctors and all followed the same schedule of medical checkups. These similarities reduce variation among the subjects and make any effects of aspirin or beta carotene easier to see.

Experiments also allow us to study the combined effects of several factors. The interaction of several factors can produce effects that

could not be predicted from looking at the effects of each factor alone.

For example, the Physicians’ Health Study tells us that aspirin helps prevent heart attacks, at least in middle-aged men, and that beta carotene taken with the aspirin neither helps nor hinders aspirin’s protective powers.

COMPARATIVE EXPERIMENTS

Placebo effect (a dummy treatment) – Many patients respond favorably to any treatment, even a sugar pill. See example 5.11 on p.292

Control group – The group that receives the sham treatment (placebo).

Often times we rely on the controlled environment of a laboratory to protect us from lurking variables.

Control is the first basic principle of statistical design of experiments. Comparison of several treatments in the same

environment is the simplest form of control.

RANDOMIZATION Experimenters often attempt to match the patients in

both groups (treatment and control) by similarities. For example, medical researchers testing the effects of a new

cancer treatment may try to match the patients in a “new drug” experimental group and a “standard drug” control group by age, sex, physical condition, smoker or not, and so on.

This is not adequate because there are too many lurking variables that may affect the outcome (such as how advanced a cancer patient’s disease is).

The statistician’s remedy is to rely on chance to make an assignment that does not depend on any characteristic of the experimental units and does not rely of the judgment of the experimenter in any way. See example 5.12 on p.295

If you assign many units to each group then the effects of chance will average out and there will be little difference in the response variable unless the treatment actually caused a difference.

Use enough experimental units to reduce chance variation.

COMPLETELY RANDOMIZED DESIGN

When all experimental units are allocated at random among all treatments, the experimental design is completely randomized.See example 5.13 on p.297

PRINCIPLES OF EXPERIMENTAL DESIGN

The basic principles of statistical design of experiments are:

1. Control the effects of lurking variables on the response, most simply by comparing two or more treatments.

2. Randomize – use impersonal chance to assign experimental units to treatments.

3. Replicate each treatment on many units to reduce chance variation in the results.

STATISTICAL SIGNIFICANCE An observed effect so large that it would

rarely occur by chance is called statistically significant.

You will see the phrase “ statistically significant” in reports of investigations in many fields of study. It tells you that the investigators found good evidence for the effect they were seeking.

When all experimental units are allocated

at random among all treatments the experimental design is completely randomized.

CAUTIONS ABOUT EXPERIMENTATION

The logic of a randomized comparative experiment depends on our ability to treat all the experimental units identically in every way except for the actual treatments being compared.

In a double-blind experiment neither the subjects nor the people who have contact with them know which treatment a subject received.

In a single-blind experiment only the subject does not know whether they are part of the treatment or control group.

The most serious potential weakness of experiments is lack of realism. The subjects or treatments or setting of an experiment may not realistically duplicate the conditions we really want to study.

MATCHED PAIRS DESIGN

Completely randomized designs are the simplest statistical designs for experiments. However, completely randomized designs are

often inferior to more elaborate statistical designs.

In particular, matching the subjects in various ways can produce more precise results than simple randomization.

Match pairs designs compare just two treatments. We choose blocks of two units that are as closely matched as possible. See example 5.16 on p.301

BLOCKING DESIGNS Matched pairs is an example of blocking design.

A block is a group of experimental units or subjects that

are known before the experiment to be similar in some way that is expected to affect the response to the treatment. In a block design, the random assignment of units to

treatments is carried out separately within each block.

Block designs can have blocks of any size

A block design combines the idea of creating equivalent treatment groups by matching with the principle of forming treatment groups at random.

See example 5.17 & 5.18 on p.302-303

Homework: P.293-307 #’s 32, 36, 38, 44, & 45