Hypothesis Tests

• Objective decision-making tools

• Are used to answer very specific questions.

For example, “Are the two gum groups equivalent in terms of mean

change in DMFS?”

• Decision rules for possible answers to your question are based on how consistent the possible answers are with the observed data.

The Null Hypothesis, H0

• A hypothesis test focuses on a single question

• That question is formulated in terms of a “TRUE or FALSE” statement called the Null Hypothesis, also denoted by H0.

• The evidence from a hypothesis test is in the form of how sure we are that H0 is not true.

• Thus, the null hypothesis is formulated in such a way that it is the statement that we are expecting to disprove.

Example: Chewing Gum Study

• We are hoping to find evidence that the gums are different in terms of DMFS progression.

• Thus, we would phrase the null hypothesis:

• We hope to prove the gums are different by disproving H0

• Disproving H0 = “rejecting” the null hypothesis

H0: The mean change in DMFS in both groups is the same

Creating the Decision Rule

We will base our decision about H0 on a statistic that:

1. is a function of the observed data.

2. gives information about H0, the question of interest.

3. will be predictable (we know it’s probability distribution) when H0 is true

Example: change in facial height

• Study to determine whether facial height of

people aged 21-26 years changes over a ten

year period.

• Facial heights were measured from a sample

of graduate students.

• After ten years, the measurements were taken

again on 84 of the same students.

Example: change in facial height

The average change in facial height was �� = −1.7 mm

with s= 6.8 mm.

We define our null hypothesis to be:

H0: µ = 0,

where μ = mean change in facial height.

Evidence against H0 will mean evidence that facial

heights change.

The t statisticTo test H0: µ = 0 we use the t statistic

• If H0 is true, then we would expect the sample mean,

�, to be close to µ = 0.

• If H0 is not true, then we would expect the � to be

far from µ = 0.

• Thus, values of the t statistic far from zero will

indicate evidence against H0.

� = � − 0

��

The t statisticTo test H0: µ = 0 we use the t statistic

� =� − 0

��

Furthermore, If the null hypothesis is true, then we know that T has t distribution with n-1 degrees of freedom.

0

Making the Decision

• Compute the statistic from the observed data

• Calculate the probability of seeing the observed statistic if the null hypothesis were true

• If the observed statistic would have low probability of occurring if the null hypothesis were true this will be considered evidence against the null hypothesis

Example: change in facial height

The average change in facial height for the 84 former

grad students was X0 = -1.7 mm with s = 6.8 mm.

The t statistic is:

� =−1.7 − 0

6.8 84⁄= −2.29

Example: change in facial height

The probability of observing T = -2.29 or something

even more unlikely when H0 is true would be

P(t83 < -2.29) + P(t83 > 2.29) = 2 × 0.012 = 0.024*

t83distribution

-2.29 2.29

prob = 0.012prob = 0.012

* Computed using Excel function “T.DIST.2T”

Example: change in facial height

Because the probability p = 0.024 is fairly low (less than

1 in 40), we consider this good evidence against H0.

This probability, p = 0.024, is an example of a p-value

t83distribution

-2.29 2.29

prob = 0.012prob = 0.012

P-value• The final decision on whether or not we should believe the null

hypothesis is based upon the p-value.

• The p-value is defined as the probability that if one performed an experiment in which it was known that H0 was true, that we would still see data producing a statistic as “extreme” (or more extreme) as the one observed.

• By an “extreme” statistic we mean one that would be unlikely if H0were true. Or a statistic that appears to show evidence that H0 is not true.

• The smaller the p-value, the less likely our observed statistic would be if H0 were true.

• Since we know that the observed statistic is real, this means that small p-values are actually evidence against H0 .

Decision Rule•How small of a p-value do we require to decide that the null hypothesis is just too unlikely?

•We decide upon a threshold value a priori, before looking at the data.

•This threshold value is called the significance level and is usually denoted by α.

•If the p-value is less than α and we reject H0 this is often called “statistically significant”

•Standard choices for α are .05 or .01

Decision Rule

Reject H0 if the p-value < α

Example: change in facial height

Test the null hypothesis H0: µ = 0 at significance level α = 0.05.

• The p-value was p = 0.024. Since 0.024 is less than .05, we

REJECT H0.

“There is good evidence at the α = 0.05 significance level that

H0 is not true.”

• If the significance level was α = 0.01 instead, then we would

NOT REJECT H0.

“There is not good evidence at the α = 0.01 significance level

that H0 is not true.”

Computing the p-value

• If you have access to Excel, you can calculate the

probabilities from any t distribution

– http://courses.washington.edu/dphs568/course/excel-t.htm

• If not, you can use Table 4 in the course notes to tell

whether the p-value is below certain values

• Specifically, it will be good to know whether the p-

value is above or below α.

Using Table 4 to find out whether the p-

value is above or below α

• Look up tn-1, 1-α/2 , the 1-α/2th percentile of the tn-1

distribution, in Table 4.

• Compare T to tn-1, 1-α/2

• If |T| > tn-1, 1-α/2 , then p-value < α. REJECT H0

• If |T| < tn-1, 1-α/2 , then p-value > α. DO NOT REJECT H0

How does that work?

For the facial height data test

H0: µ = 0 at significance level α = 0.05.

t83,0.975 = 1.99, so

P(t83 < -1.99) + P(t83 > 1.99) = .05

Since |T|= 2.29 > 1.99, this means

P(t83 < -2.29) + P(t83 > 2.29)

< P(t83 < -1.99) + P(t83 > 1.99)

= .05

t83distribution

-1.99 1.99

prob = 0.05

t83distribution

-2.29 2.29

p-value

Example: chewing gum data

Do the data indicate at the α=0.01 significance level

that the mean DMFS has changed in Group A?

• Let µ = mean change in DMFS

• Test hypothesis H0: µ = 0

• Note: “µ = 0” would indicate “no change in DMFS”

• For Group A: n = 25, X0 = -0.72, and s = 5.37.

• The t statistic is:

� =−0.72 − 0

5.37 25⁄= −0.67

Example: chewing gum data

� =−0.72 − 0

5.37 25⁄= −0.67

• Since α=0.01, we look up the (1- α/2) = 99.5th

percentile of a t24 distribution.

• From Table 4, t24,0.995 = 2.80

• Since |T| = 0.67 < 2.80, DO NOT REJECT H0

• p-value = P(|t24| > .67) = 0.51t24distribution

-0.67 0.67

p-value

Properties of a hypothesis test

• The decision of a hypothesis test is based on the random sample.

• The decision of the test can be incorrect.

• There are two possible scenarios, and two possible test outcomes in each scenario:

H0 is true H0 is not true

test rejects H0

type Ierror

test rejects H0

OK

test does not reject

H0

OKtest does not reject

H0

type IIerror

•Two scenarios:•H0 is true, or H0 is not true

•Two test outcomes: •Reject H0, or Do not reject H0

Properties of a hypothesis testScenario #1 Scenario #2

Outcomes of hypothesis

test

H0 is true

test rejects H0

type Ierror

test does not reject

H0

OK

• If the test rejects H0, that is considered a type I error.

•The probability of making a type I error is equal to the significance level, α

•The significance level is chosen by the investigator

Scenario #1: H0 is true

H0 is not true

test rejects H0

OK

test does not reject

H0

type IIerror

Scenario #2: H0 is not true

• If the test does not reject H0, that is considered a type II error.

•The probability of being correct in scenario #2, of not making a type II error, is called the Power of the test

H0 is true H0 is not true

test rejects H0

type Ierror

test rejects H0

OK

test does not reject

H0

OKtest does not reject

H0

type IIerror

Test Rejects H0Scenario #1 Scenario #2

Test rejects

H0 is true H0 is not true

test rejects H0

type Ierror

test rejects H0

OK

• If we reject H0 with, say, α=.05, the implication is clear.

• When rejecting H0, our possible error would be a type I error.

• Because we set up our hypothesis test with a specified type I error probability of α=.05, we know that there is less than 5% chance that our decision is incorrect

Test Rejects H0Scenario #1 Scenario #2

Test rejects

H0 is true H0 is not true

test rejects H0

type Ierror

test rejects H0

OK

test does not reject

H0

OKtest does not reject

H0

type IIerror

Test does not reject H0Scenario #1 Scenario #2

Test doesnot reject

H0 is true H0 is not true

test does not reject

H0

OKtest does not reject

H0

type IIerror

Test does not reject H0

Test doesnot reject

• If we do not reject H0 then our possible error would be a type II error.

• The probability of type II error is not as easily established as the type I error.

• When failing to reject H0 , it is less clear how likely it is that our decision is correct or incorrect.

Recap

• A hypothesis test allows you to make a decision

• The null hypothesis is always formulated in such a way that it is the statement that we want to disprove.

• Reject H0 if the p-value < α (“statistically significant”)

• A hypothesis test only controls the type I error– If we reject H0 our evidence is clear– If we fail to reject H0 then it is not as clear