4.2C HOW TO EXPERIMENT WELL

HOW TO EXPERIMENT POORLY

A high school regularly offers a review course to prepare students for the SAT. This year, the budget cuts will allow the school to offer only an online version of the course. Over the past 10 years, the average SAT score of students in the classroom course was 1620. The online group gets an average score of 1780. That’s roughly 10% higher than the long-time average for those who took the classroom review course. Is the online course more effective?

HOW TO EXPERIMENT POORLY

This experiment had a very simple design Experimental Units Treatment Measure response

 • What are the experimental units, explanatory variable(s), and

response variable(s)? • What are possible confounding variables? • What if you found out that the GPAs of students in the online

course were higher, on average, than the GPAs of students in the classroom course? Would you still conclude that the online course was more effective at preparing students? Why or why not? 

• How might you change the experiment to find out if the online course really does prepare students better for the SAT?

HOW TO EXPERIMENT WELL

Most well-designed experiments compare two or more treatments.

Random Assignment: Experimental units are assigned to treatments at random, using some sort of chance process. This will eliminate bias. 

THE RANDOMIZED COMPARATIVE EXPERIMENT

Suppose you have a class of 30 students who volunteer to be subjects in an experiment measuring caffeine consumption and pulse rate. Explain how you would randomly assign 15 students to each of the two treatments (those who drink caffeinated cola and those who drink uncaffeinated cola) using the following assignment methods:

THE RANDOMIZED COMPARATIVE EXPERIMENT

Using 30 identical slips of paper: Write the names of the 30 students on identical slips of paper. Put all the papers into a hat and mix them well. Draw out papers one at a time until 15 are selected. These students will be assigned to the group drinking a caffeinated beverage. The reaming students are the group that gets uncaffeinated cola.

THE RANDOMIZED COMPARATIVE EXPERIMENT

Using the Random Digit Table: Assign each student a label from 01 to 30, alphabetically by last name. Select a starting line on the Random Digit Table and read two-digit numbers from left to right. Duplicate labels outside the range of 01 to 30 will be ignored. The first 15 unique numbers from 01 to 30 will be the subject assigned to the caffeine group. The remaining students will be the uncaffeinated cola group.

THE RANDOMIZED COMPARATIVE EXPERIMENT

Using technology Assign each subject a label from 01 to 30 alphabetically by last name. Using the calculator’s RandIntNoRep (01,30,15) function to produce fifteen unique numbers from 01 to 30, assign these subjects to the caffeine group. Remaining subjects will be the uncaffeinated cola group.

THE RANDOMIZED COMPARATIVE EXPERIMENT

What makes this experimental design comparative? The experiment is comparative because it is comparing the pulse rates of those who consume caffeine with those who don’t. What makes it randomized? It is randomized because subjects are randomly assigned to one of two treatments.

COMPLETELY RANDOMIZED DESIGN DIAGRAM

Subjects

Random assignmen

t

Group 1½ subjects

Group 2½ subjects

Treatment 1

Treatment 2

Compare Results

CONTROL GROUP

Used to provide a baseline for comparing the effects of other treatments.

• inactive treatment (often called a placebo)• active treatment—an example: in a study of a

new cancer drug, a control group is given an existing drug in order to compare the effectiveness of the existing drug with the new drug.

 Nota Bene: Not all studies include control groups.

PRINCIPLES OF EXPERIMENTAL DESIGN

1. Comparison: Use a design that compares two or more treatments

2. Control: Ensures that influences other than the experimental treatments operate equally on all groups (income, general health, GPA, etc.)

3. Random: Random assignment helps balance the effects of lurking variables.

4. Replication: Use enough experimental units in each group so that any differences in average response to the treatments can be distinguished from chance differences between the groups.

EXIT TICKET: DUELING DIETS

A health organization wants to know if a low-carb or a low-fat diet is more effective for long-term weight loss. The organization decides to conduct an experiment to compare these two diet plans with a control group that is only provided with a brochure about healthy eating. Ninety volunteers agree to participate in the study for one year.

• Is the control group an active or inactive treatment?• Diagram a completely randomized design for this experiment. •  Write a few sentences describing how you would implement your

design.•  How would you randomize the groups?• What would you record for each subject at the end of the year?• What would you compare for each group?

• Was a control group completely necessary?

HOMEWORK

• pg 260-261, #59-65 odd