SP 2015CP PROBABILITY & STATISTICS

Observational Studiesvs.

Experiments

Chapter 11

Observational Study vs. Experiment

Observational StudyObserve only, no manipulation of factors is used

ExperimentFactors are manipulated to create treatments and randomly assign subjects to the different treatments

Observational Studies

Simply observing regular occurrences

Researchers do not assign choices

Can be RETROSPECTIVE or PROSPECTIVE

Observational Studies

Useful for:Discovering trends and potential relationshipsUsed in public health and marketing

Observational StudiesDO NOT demonstrate cause-and-effect relationships

Retrospective Study

Not based on random samplesTypically used to estimate:

Differences between groups Associations between variables

Typically focus on small segment of entire population

Retrospective Study - STEPS

Steps:Select SubjectsDetermine their previous

conditions/behaviors Exam historic information

Pull records from data bases/sources (prior grades, classes, etc...)

Ask subjects questions in order to gather knowledge of past events

Retrospective Study - EXAMPLE

Example:Identify people with a diseaseLook at their history, heritage to

determine things which may be related to their condition.

Retrospective Study - WARNING

Data collected through asking subjects to recall past events tends to have errors!!!!!!

What did you eat last week, exactly?

Retrospective Study – WARNING 2

Lurking Variables

Prospective Study

Observe subjects over time

Identifying subjects in advanceCollecting data as events unfold

No treatments/conditions are intentionally controlled

Example

Experiments

Manipulate factors to create treatments

Can show cause-and-effect relationships

Requires random assignment of subjects to treatments

Subjects are also referred to as Participants/ Experimental Units

Compares results from different treatment groups

Experiments - BASICS

Requires a random assignment of subjects to treatments

Study the relationship between two or more variables

Must have at least one of each:Explanatory variable (Factor) to

manipulate&

Response variable to measure

Experimenter - ROLE

Actively and deliberately manipulates factors to control the details of the possible treatments

RANDOMLY assigns the subjects to the treatments

Observes the response variableCompares responses for different

groups of subjects who have been treated differently.

Experiments - WHO

Humans are commonly called subjects or participants

Other individuals (rats, days, petri dishes of bacteria) are commonly referred to by the more generic term experimental units.

Experiments - WHAT

Factor – Variable whose levels are manipulated by the experimenter

Levels - Specific values chosen for a factorex. In a sleep study, we might assign participants to sleep for 4, 6, or 8 hours

Treatment – Process applied to a group of subjects. Treatments are the different levels of the factor.

Example - Continued

Treatments: Typical portions of 2 dog foods (1 from original company, 1 we know is

safe)

Response: Veterinarian’s assessment of test animals health

Experimental Design - 3 Principles

ControlRandomizeReplicate

Experimental Design - 3 Principles - CONTROL

We control sources of variation other than the factors we are testing by making conditions as similar as possible for all treatment groups.

For human subjects, we try to treat them alike.

Controlling sources of variation makes it easier to detect any differences caused by the treatments.

Control - Methods

Control treatment - baseline measurement

Control group - experimental units to whom the control treatment is applied

Placebo, Blinding

Blinding

People are so good at picking up subtle cues about treatments that it’s important to keep anyone who could affect the outcome or the measurement of the response from knowing which subjects have been assigned to which treatments.

2 groups: People who can influence the results (interviewer,

subjects, test administrator) People who evaluate the results (researcher,

doctors, judges)

Blinding

Single Blinding – when every individual in 1 of the groups is blinded

Double Blinding – when every individual in both groups is blinded

Recall: 2 groups People who can influence the results People who evaluate the results

Placebo & Placebo Effect

Placebo - A “fake” treatment that looks just like the treatments being tested.

Placebos are the best way to blind subjects from knowing whether they are receiving the treatment or not.

Placebo effect - when subjects treated with a placebo improve.

It’s not unusual for 20% or more of subjects given a placebo to report reduction in pain, improved movement, or greater alertness, or even to demonstrate improved health or performance.

Placebo controls are so effective that you shoulduse them as an essential tool for blinding whenever possible.

Blocking

We use blocking when there are pre-existing differences between groups of experimental units.

Randomization is introduced when we randomly assign treatments within each block.

Experimental Design - 3 Principles - RANDOMIZE

Allows us to equalize the effects of unknown or uncontrollable sources of variation.

Note: Randomization cannot eliminate the effects of these sources, but it should spread them out fairly equally across the treatment levels

Experimental Design - 3 Principles - Replicate

2 Types of Replication

Type 1 - Within the experiment Apply each treatment to several subjects

Type 2 – Additional Experiment Replication of an entire experiment with different subjects

Example - Continued

In the experiment comparing animal food, how could you implement the principles of

control, randomization, and replication?•Control the portion sizes•Reduce possible variability from other factors:• (Standardize other aspects of their environments—housing the dogs

in similar pens and ensuring that each got the same amount of water, exercise, play, and sleep time.)

• Restrict the experiment to a single breed and age • Assign dogs to the two feed treatments randomly.

•To try and equalize traits, pre-existing conditions, and other unknown influences•Replicate by assigning more than one dog to each treatment to allow for variability among individual dogs.•If time and funding, possibly replicate the entire experiment using, for example, a different breed of dog.

Diagram of Experimental Procedure

Your Turn! – CW1

On the following slide, you will be asked to design an experiment. Follow these steps:

State what you want to know (in general)Specify the response variableSpecify the factors and treatments/levelsSpecify the Experimental UnitsDesign the experiment (Control, Replicate, Random)Create a diagram to show the processHow will you display resultsReflect – are the observations made, significant?

Your Turn! – CW1