Introduction

Hypothesis Testing FrameworkFarrokh Alemi Ph.D.FrameworkObjectiveThe objective of this section is to teach you the framework of hypothesis testing. The objective of this section is to teach you the framework of hypothesis testing. 2Null Hypothesis

The null hypothesis (H0) often represents either a skeptical perspective or a claim to be tested. The alternative hypothesis (HA) represents an alternative claim under consideration and is often represented by a range of possible parameter values. 3Skeptic

The null hypothesis is shown as H sub zero. Meu sub 1 indicates the mean of the population 1 and meu sub 2 indicates mean of population 2. The null hypothesis is that the two populations have equal means. Skeptic will not reject the null hypothesis (H0), unless the evidence in favor of the alternative hypothesis (HA) is so strong that she rejects H0 in favor of HA.4Skeptics

In statistics we do not accept a hypothesis. We only reject some hypothesis without accepting another. The rejection of the null hypothesis is not necessarily acceptance of the alternative hypothesis as other hypothesis may be true. 5Double Negatives

In many statistical explanations, we use double negatives. For instance, we might say that the null hypothesis is not implausible or we failed to reject the null hypothesis. Double negatives are used to communicate that while we are not rejecting a position, we are also not saying it is correct. 6Decision Errors

Hypothesis tests are not awless. We can make a wrong decision in statistical hypothesis tests. We can quantify how often we make such errors. We organize our test of a hypothesis in ways that minimize probability of errors.7Decision Errors

There are two competing hypotheses: the null and the alternative. The null hypothesis is shown here as H sub zero. The alternative hypothesis is shown here as H sub A, where A stands for alternative. 8Decision Errors

We will nd it most useful if we always list the null hypothesis as an equality. To show the hypothesized mean of the population we use the Greek letter meu and write the null hypothesis as = 7. ) The alternative hypothesis always uses an inequality. For example, is not equal to 79Decision Errors

, is greater than 710Decision Errors

or meu is less than 7.11Decision Errors

There are four possible scenarios in a hypothesis test, which are summarized in this Table. A Type 1 Error is rejecting the null hypothesis when null hypothesis is actually true. 12Decision Errors

A Type 2 Error is failing to reject the null hypothesis when the alternative is actually true.13Decision Errors

Hypothesis testing is built around rejecting or failing to reject the null hypothesis. That is, we do not reject null hypothesis unless we have strong evidence. As a general rule of thumb, for those cases where the null hypothesis is actually true, we do not want to incorrectly reject the null hypothesis more than 5% of the time. This corresponds to a signicance level of 0.05. 14Decision Errors

We often write the signicance level using the Greek letter alpha. Alpha always indicates extent of type two errors.15Decision Errors

If we reduce one type of error, we generally increase another type of error. If we always reject the null hypothesis regardless of the data, then type 1 error will be at its maximum and type two errors at its minimum. The two types of errors are related. 16Significance Level

Choosing a signicance level for a test is important in many contexts, and the traditional level is 0.05. However, it is often helpful to adjust the signicance level based on theconsequences of any conclusions reached from the test. If making a Type 1 Error is dangerous or especially costly, we should choose a small signicance level (e.g. 0.01). Under this scenario we want to be very cautious about rejecting the null hypothesis, so we demand very strong evidence favoring alternative hypothesis before we would reject null hypothesis. If a Type 2 Error is relatively more dangerous or much more costly than a Type 1 Error, then we should choose a higher signicance level (e.g. 0.10). Here we want to be cautious about failing to reject null hypothesis when the null is actually false. 17Confidence Interval

Imagine a distribution around the hypothesized null value, where the hypothesized null value is the mean of the distribution. we can use a condence interval to test if the sample value can be observed within 95% confidence interval of the hypothesized null value. 18Confidence Interval

We will make an error whenever the point estimate is at least 1.96 standard errors away from the hypothesized null value. This happens about 5% of the time (2.5% in each tail). So confidence interval can be used to test a hypothesis. If the sample value falls outside of the confidence interval then the hypothesis is rejected.19One Sided Test

If the researchers are only interested in showing an increase or a decrease, but not both, use a one-sided test. If the researchers would be interested in any dierence from the null value an increase or decrease then the test should be two-sided. To test using a one sided test, the Z value changes because now 95% of the data should be below the z value.20P-value

Sometimes we want to know the probability of observing a particular value if the null hypothesis was true. This is referred to as the p-value and depends on how many standard deviations the value is further than the hypothesized null value of the population. Here the average of the sample is more than 5 standard deviation smaller than the hypothesized value. The p-value tells us the probability of observing a sample with such a small average. 21P-value

To calculate the P-value, we calculate the standard Z score associated with observing the sample average. 22P-value

Recall this is the sample average, shown here as x-bar,23P-value

minus the hypothesized null value24P-value

divided by the standard error of the sample average25P-value

Once Z is calculated, we can look up the probability of observing values less than the Z value in a standard normal distribution. 26P-value

To identify the p-value, the distribution of the sample mean is considered as if the null hypothesis was true. Then the p-value is defined and computed as the probability of the observed average or an average even more favorable to the alternative hypothesis under this distribution.27P-value

If the alpha level is larger than the p-value, we reject the null hypothesis. As the probability of observing a value that far away from the hypothesized mean is too small. Otherwise we fail to reject the null hypothesis. The p-value quantifies how strongly the data favor alternative hypothesis over the null hypothesis. A small p-value (usually less than 0.05) corresponds to sufficient evidence to reject null hypothesis in favor of alternative hypothesis.28P-value

Here is a review of the steps in the framework. First test assumptions, including independence and skewness. 29P-value

Then state the null hypothesis and decide if it is a one sided or two sided test. 30P-value

Next calculate the Z value from sample point estimates. 31P-value

Then look up the p value, paying attention to whether it is a one sided or two sided test. 32P-value

Finally reject or fail to reject the null hypothesis by comparing the p value to alpha, the extent of type two errors that we are willing to tolerate. 33Exercise

Suppose the sample distribution of hours of sleep for 110 college students is as indicated. What is the probability that our sample comes from a distribution of mean of 7. To begin we have to check assumptions. 34Exercise

Data are moderately skewed and we assume independent, therefore the central limit theory tells us that the mean of the sample has a normal distribution35Exercise

Once assumptions have been checked, we need to state the hypotheses. The null hypothesis is that the population has a mean of 7 hours of sleep.36Exercise

The alternative hypothesis that the population has a mean larger than 7 hours. Notice that this is a one sided test. 37Exercise

Next we calculate the Z value. The mean of the sample was 7.4238Exercise

the standard error was .17. 39Exercise

The Z value associated with this mean and standard error and null hypothesis is 2.47. 40Exercise

We compute the p-value by finding the area below the Z value in standard normal distribution. In this case we look for area below Z value of 2.47. The area below this z value is 0.993. The P-value is 1 minus the area, which is 0.007. This is the probability of type 2 error. This is the probability of rejecting the null hypothesis when it is true. 41Exercise

We reject the hypothesis if the alpha level set for our confidence interval is higher than the p-value. In this case it is. So with 95% confidence, we reject the null hypothesis that the observed sample came from a population with mean of 7.

42Two Sided Test with P-value

When conducting two sided test of a null hypothesis, we calculate the Z value as before. 43Two Sided Test with P-value

We look up the area under the left tail of Z value in a standard normal distribution. 44Two Sided Test with P-value

Now to calculate the P-value, we need to consider both the right and left tails. 45Two Sided Test with P-value

Since the normal distribution is symmetric, this is calculated as twice the value of the left tail. 46Five steps: test assumptions, state hypotheses, calculate Z under null hypothesis, look up p, reject or fail to reject Take Home LessonA framework was presented that included 5 steps: test assumptions, state hypotheses, calculate Z under null hypothesis, look up p, reject or fail to reject 47