# ME Confidence Intervals: Bootstrap Distribution · 2015-02-21 · Bootstrap Sample Statistic...

### Transcript of ME Confidence Intervals: Bootstrap Distribution · 2015-02-21 · Bootstrap Sample Statistic...

2/20/2015

1

Statistics: Unlocking the Power of Data Lock5

STAT 250 Dr. Kari Lock Morgan

Confidence Intervals: Bootstrap Distribution

SECTIONS 3.3, 3.4

• Bootstrap distribution (3.3)

• 95% CI using standard error (3.3)

• Percentile method (3.4)

Statistics: Unlocking the Power of Data Lock5

Confidence Intervals

Population Sample

Sample

Sample

Sample Sample Sample

. . .

Calculate statistic for each sample

Sampling Distribution

Standard Error (SE): standard deviation of sampling distribution

Margin of Error (ME) (95% CI: ME = 2×SE)

statistic ± ME

Statistics: Unlocking the Power of Data Lock5

Reality

One small problem…

… WE ONLY HAVE ONE SAMPLE!!!!

• How do we know how much sample statistics vary, if we only have one sample?!?

BOOTSTRAP! Statistics: Unlocking the Power of Data Lock5

Your best guess for the population

is lots of copies of the sample!

Sample repeatedly from this “population”

Statistics: Unlocking the Power of Data Lock5

• It’s impossible to sample repeatedly from the population…

• But we can sample repeatedly from the sample!

• To get statistics that vary, sample with replacement (each unit can be selected more than once)

Sampling with Replacement

Statistics: Unlocking the Power of Data Lock5

Suppose we have a random sample of 6 people:

2/20/2015

2

Statistics: Unlocking the Power of Data Lock5

Original Sample

A simulated “population” to sample from

Statistics: Unlocking the Power of Data Lock5

Bootstrap Sample: Sample with

replacement from the original sample, using the same sample size.

Original Sample

Bootstrap Sample

Remember: sample

size matters!

Statistics: Unlocking the Power of Data Lock5

• How would you take a bootstrap sample from your sample of Reese’s Pieces?

Reese’s Pieces

Statistics: Unlocking the Power of Data Lock5

Your original sample has data values

18, 19, 19, 20, 21 Is the following a possible bootstrap sample?

18, 19, 20, 21, 22

a) Yes b) No

Bootstrap Sample

Statistics: Unlocking the Power of Data Lock5

Your original sample has data values

18, 19, 19, 20, 21 Is the following a possible bootstrap sample?

18, 19, 20, 21

a) Yes b) No

Bootstrap Sample

Statistics: Unlocking the Power of Data Lock5

Your original sample has data values

18, 19, 19, 20, 21 Is the following a possible bootstrap sample?

18, 18, 19, 20, 21

a) Yes b) No

Bootstrap Sample

2/20/2015

3

Statistics: Unlocking the Power of Data Lock5

Original Sample

BootstrapSample

BootstrapSample

BootstrapSample

.

.

.

Bootstrap Statistic

Sample Statistic

Bootstrap Statistic

Bootstrap Statistic

.

.

.

Bootstrap Distribution

Statistics: Unlocking the Power of Data Lock5

• What is the average mercury level of fish (large mouth bass) in Florida lakes?

• Sample of size n = 53, with 𝑥 = 0.527 ppm. (FDA action level in the USA is 1 ppm, in Canada the limit is 0.5 ppm)

•Key Question: How much can statistics vary from sample to sample?

Mercury Levels in Fish

Lange, T., Royals, H. and Connor, L. (2004). Mercury accumulation in largemouth bass (Micropterus salmoides) in a Florida Lake. Archives of Environmental Contamination and Toxicology, 27(4), 466-471.

Statistics: Unlocking the Power of Data Lock5

Mercury Levels in Fish

Statistics: Unlocking the Power of Data Lock5

You have a sample of size n = 53. You sample with replacement 1000 times to get 1000 bootstrap samples. What is the sample size of each bootstrap sample? (a) 53 (b) 1000

Bootstrap Sample

Statistics: Unlocking the Power of Data Lock5

You have a sample of size n = 53. You sample with replacement 1000 times to get 1000 bootstrap samples. How many bootstrap statistics will you have? (a) 53 (b) 1000

Bootstrap Distribution

Statistics: Unlocking the Power of Data Lock5

“Pull yourself up by your bootstraps”

Why “bootstrap”?

• Lift yourself in the air simply by pulling up on the laces of your boots

• Metaphor for accomplishing an “impossible” task without any outside help

2/20/2015

4

Statistics: Unlocking the Power of Data Lock5

Sampling Distribution

Population

µ

BUT, in practice we don’t see the “tree” or all of the “seeds” – we only have ONE seed

Statistics: Unlocking the Power of Data Lock5

Bootstrap Distribution

Bootstrap “Population”

What can we do with just one seed?

𝑥

Estimate the distribution and variability (SE) of 𝑥 ’s from the bootstraps

µ

Statistics: Unlocking the Power of Data Lock5

Bootstrap statistics are to the original sample statistic

as

the original sample statistic is to the population parameter

Golden Rule of Bootstrapping

Statistics: Unlocking the Power of Data Lock5

Center

•The sampling distribution is centered around the population parameter

• The bootstrap distribution is centered around the

•Luckily, we don’t care about the center… we care about the variability!

a) population parameter b) sample statistic c) bootstrap statistic d) bootstrap parameter

Statistics: Unlocking the Power of Data Lock5

Standard Error

• The variability of the bootstrap statistics is similar to the variability of the sample statistics

• The standard error of a statistic can be estimated using the standard deviation of the bootstrap distribution!

Statistics: Unlocking the Power of Data Lock5

Confidence Intervals

Sample Bootstrap

Sample

. . .

Calculate statistic for each bootstrap sample

Bootstrap Distribution

Standard Error (SE): standard deviation of bootstrap distribution

Margin of Error (ME) (95% CI: ME = 2×SE)

statistic ± ME

Bootstrap Sample

Bootstrap Sample

Bootstrap Sample

Bootstrap Sample

2/20/2015

5

Statistics: Unlocking the Power of Data Lock5

• What is the average mercury level of fish (large mouth bass) in Florida lakes?

• Sample of size n = 53, with 𝑥 = 0.527 ppm. (FDA action level in the USA is 1 ppm, in Canada the limit is 0.5 ppm)

•Give a confidence interval for the true average.

Mercury Levels in Fish

Lange, T., Royals, H. and Connor, L. (2004). Mercury accumulation in largemouth bass (Micropterus salmoides) in a Florida Lake. Archives of Environmental Contamination and Toxicology, 27(4), 466-471.

Statistics: Unlocking the Power of Data Lock5

SE = 0.047

0.527 2 0.047 (0.433, 0.621)

Mercury Levels in Fish

We are 95% confident that average mercury level in fish in Florida lakes is between 0.433 and 0.621 ppm.

Statistics: Unlocking the Power of Data Lock5

Same process for every parameter! Estimate the standard error and/or a confidence interval for...

• proportion (𝑝)

• difference in means (µ1 − µ2 )

• difference in proportions (𝑝1 − 𝑝2 )

• standard deviation (𝜎)

• correlation (𝜌)

• ... Generate samples with replacement Calculate sample statistic Repeat...

Statistics: Unlocking the Power of Data Lock5

Hitchhiker Snails

A type of small snail is very widespread in Japan, and colonies of the snails that are genetically very similar have been found very far apart.

How could the snails travel such long distances?

Biologist Shinichiro Wada fed 174 live snails to birds, and found that 26 were excreted live out the other end. (The snails are apparently able to seal their shells shut to keep the digestive fluids from getting in).

What proportion of these snails ingested by birds survive?

Yong, E. “The Scatological Hitchhiker Snail,” Discover, October 2011, 13.

Statistics: Unlocking the Power of Data Lock5

Hitchhiker Snails Give a 95% confidence interval for the proportion of snails ingested by birds that survive.

a) (0.1, 0.2) b) (0.05, 0.25) c) (0.12, 0.18) d) (0.07, 0.18)

Statistics: Unlocking the Power of Data Lock5

Body Temperature What is the average body temperature of humans?

www.lock5stat.com/statkey

Shoemaker, What's Normal: Temperature, Gender and Heartrate, Journal of Statistics Education, Vol. 4, No. 2 (1996)

2/20/2015

6

Statistics: Unlocking the Power of Data Lock5

Other Levels of Confidence

• What if we want to be more than 95% confident?

• How might you produce a 99% confidence interval for the average body temperature?

Statistics: Unlocking the Power of Data Lock5

Percentile Method

• For a P% confidence interval, keep the middle P% of bootstrap statistics

• For a 99% confidence interval, keep the middle 99%, leaving 0.5% in each tail.

• The 99% confidence interval would be

(0.5th percentile, 99.5th percentile)

where the percentiles refer to the bootstrap distribution.

Statistics: Unlocking the Power of Data Lock5

Bootstrap Distribution Best Guess at Sampling Distribution

Statistic

2 3 4 5 6 7 8

Best Guess at Sampling Distribution

Statistic

2 3 4 5 6 7 8

Observed

Statistic

Best Guess at Sampling Distribution

Statistic

2 3 4 5 6 7 8

Observed

Statistic

P%

Best Guess at Sampling Distribution

Statistic

2 3 4 5 6 7 8

Observed

Statistic

P%P%P%

Upper

Bound

Upper

Bound

Lower

Bound

• For a P% confidence interval:

Statistics: Unlocking the Power of Data Lock5

www.lock5stat.com/statkey

Body Temperature

We are 99% sure that the average body temperature is between

98.00 and 98.58

Middle 99% of bootstrap statistics

Statistics: Unlocking the Power of Data Lock5

Level of Confidence

Which is wider, a 90% confidence interval or a 95% confidence interval? (a) 90% CI (b) 95% CI

Statistics: Unlocking the Power of Data Lock5

Mercury and pH in Lakes

Lange, Royals, and Connor, Transactions of the American Fisheries Society (1993)

• For Florida lakes, what is the correlation between average mercury level (ppm) in fish taken from a lake and acidity (pH) of the lake?

𝑟 = −0.575

Give a 90% CI for

2/20/2015

7

Statistics: Unlocking the Power of Data Lock5

www.lock5stat.com/statkey

Mercury and pH in Lakes

We are 90% confident that the true correlation between average mercury level and pH of Florida lakes is between -0.702 and -0.433.

Statistics: Unlocking the Power of Data Lock5

Bootstrap CI Option 1: Estimate the standard error of the statistic by computing the standard deviation of the bootstrap distribution, and then generate a 95% confidence interval by Option 2: Generate a P% confidence interval as the range for the middle P% of bootstrap statistics

2statisti Sc E

Statistics: Unlocking the Power of Data Lock5

SE = 0.047

0.527 2 0.047 (0.433, 0.621)

Middle 95% of bootstrap statistics

Mercury Levels in Fish

Statistics: Unlocking the Power of Data Lock5

Bootstrap Cautions

• These methods for creating a confidence interval only work if the bootstrap distribution is smooth and symmetric

• ALWAYS look at a plot of the bootstrap distribution!

• If the bootstrap distribution is skewed or looks “spiky” with gaps, you will need to go beyond intro stat to create a confidence interval

Statistics: Unlocking the Power of Data Lock5

Bootstrap Cautions

Statistics: Unlocking the Power of Data Lock5

Bootstrap Cautions

2/20/2015

8

Statistics: Unlocking the Power of Data Lock5

Number of Bootstrap Samples

• When using bootstrapping, you may get a slightly different confidence interval each time. This is fine!

• The more bootstrap samples you use, the more precise your answer will be.

• For the purposes of this class, 1000 bootstrap samples is fine. In real life, you probably want to take 10,000 or even 100,000 bootstrap samples

Statistics: Unlocking the Power of Data Lock5

Summary The standard error of a statistic is the standard

deviation of the sample statistic, which can be estimated from a bootstrap distribution

Confidence intervals can be created using the standard error or the percentiles of a bootstrap distribution

Increasing the number of bootstrap samples will not change the SE or interval (except for random fluctuation)

Confidence intervals can be created this way for any parameter, as long as the bootstrap distribution is approximately symmetric and continuous

Statistics: Unlocking the Power of Data Lock5

To Do

Read Sections 3.3, 3.4

Do HW 3.3, 3.4 (due Friday, 2/7)