Comparing Means from Two Data Sets
description
Transcript of Comparing Means from Two Data Sets
04/21/23 HK 396 - Dr. Sasho MacKenzie 1
Comparing Means from Comparing Means from Two Data SetsTwo Data Sets
The t-testThe t-test
04/21/23 HK 396 - Dr. Sasho MacKenzie 2
Research QuestionsResearch Questions• To improve muscular power, should an athlete To improve muscular power, should an athlete
perform heavy resistance exercises, or light perform heavy resistance exercises, or light plyometric exercises?plyometric exercises?
• Is it better to imagine the flight of the ball or the Is it better to imagine the flight of the ball or the actions of your swing prior to striking a golf ball?actions of your swing prior to striking a golf ball?
• Is running 5 km or walking 5 km better for Is running 5 km or walking 5 km better for burning calories?burning calories?
• Do golfers sink more putts if they focus on the Do golfers sink more putts if they focus on the hole or on the ball during a putt?hole or on the ball during a putt?
• Will squatting to a lower depth during a vertical Will squatting to a lower depth during a vertical jump improve performance?jump improve performance?
04/21/23 HK 396 - Dr. Sasho MacKenzie 3
The t-testThe t-test• All of the questions posed on the previous slide can All of the questions posed on the previous slide can
be statistically addressed using the be statistically addressed using the t-test.
• A t-test determines if two groups of data are A t-test determines if two groups of data are significantly different (not meaningfully different).significantly different (not meaningfully different).
• A t-test is the ratio of the actual difference between A t-test is the ratio of the actual difference between two means to the difference that is expected due to two means to the difference that is expected due to chance alone.chance alone.
• The bigger the actual difference is compared to the The bigger the actual difference is compared to the expected difference due to chance, the more expected difference due to chance, the more statistically significant the t-test.statistically significant the t-test.
04/21/23 HK 396 - Dr. Sasho MacKenzie 4
The t-testThe t-test
• A t-test calculation produces a value (t-statistic) A t-test calculation produces a value (t-statistic) that is similar to a that is similar to a z-score..
• The The t-distributions, are also very similar to the , are also very similar to the z-score distribution (normal distribution).z-score distribution (normal distribution).
• The t-distribution changes depending on the The t-distribution changes depending on the sample size.sample size.
04/21/23 HK 396 - Dr. Sasho MacKenzie 5
The t Distributions (3 The t Distributions (3 examples)examples)
-3-3 00-1-1-2-2 11 3322-4-4 44
tt
N = 60 (same as normal curve)N = 60 (same as normal curve)
N = 10N = 10
N = 3N = 3
E.g., Area beyond t=3 E.g., Area beyond t=3 increases as N decreasesincreases as N decreases
04/21/23 HK 396 - Dr. Sasho MacKenzie 6
Let’s use an ExampleLet’s use an Example• Question: Do HK students drink more or less Question: Do HK students drink more or less
alcohol than the average St.FX student?alcohol than the average St.FX student?• Assumptions:Assumptions:
– Every student on campus honestly completed a form Every student on campus honestly completed a form and the average drinks/week is known.and the average drinks/week is known.
– Therefore, we know the mean of the population.Therefore, we know the mean of the population.
• Methods:Methods:– Determine the drinks/week for a sample of HK Determine the drinks/week for a sample of HK
students.students.– Determine if the sample mean is “different” than the Determine if the sample mean is “different” than the
population mean (perform a t-test).population mean (perform a t-test).
04/21/23 HK 396 - Dr. Sasho MacKenzie 7
What is “different”?What is “different”?
• Before the t-test, we must set a standard for Before the t-test, we must set a standard for statistical significance.statistical significance.
• This means determining the chance of error we are This means determining the chance of error we are willing to have in our final decision.willing to have in our final decision.
• I.e., How confident do we want to be in our I.e., How confident do we want to be in our decision that HK students drink a different amount?decision that HK students drink a different amount?
• This decision is represented by alpha (This decision is represented by alpha (), which is ), which is typically set at .05 (5%). This value is arbitrary.typically set at .05 (5%). This value is arbitrary.
• Assume no difference and that the study is Assume no difference and that the study is repeated 100 times. On 5 occasions, due to repeated 100 times. On 5 occasions, due to chance, we would incorrectly find that HK students chance, we would incorrectly find that HK students drink more.drink more.
04/21/23 HK 396 - Dr. Sasho MacKenzie 8
One-sample t-testOne-sample t-test
• We will use what’s called a one-sample t-test.We will use what’s called a one-sample t-test.• This compares the mean of a sample to the This compares the mean of a sample to the
mean of a population.mean of a population.
SEM
Xt
XX = = sample meansample mean• = population mean= population mean• SEMSEM = standard error of the mean= standard error of the mean
Actual differenceActual difference
Expected difference due to chanceExpected difference due to chance
04/21/23 HK 396 - Dr. Sasho MacKenzie 9
The HypothesisThe Hypothesis
• In statistics you must clearly state a testable In statistics you must clearly state a testable hypothesis.hypothesis.
• Typically the hypothesis tested is opposite to Typically the hypothesis tested is opposite to what you expect and is referred to as the what you expect and is referred to as the null hypothesis..
• Our null hypothesis is that HK students do not Our null hypothesis is that HK students do not drink a different amount than the average drink a different amount than the average university student.university student.
XX = = or or XX - - = = 00
04/21/23 HK 396 - Dr. Sasho MacKenzie 10
The CalculationThe Calculation• University PopulationUniversity Population
– Average drinks per week = 10Average drinks per week = 10
• HK Sample of studentsHK Sample of students– Mean = 12, SD = 5, N = 30Mean = 12, SD = 5, N = 30
19.2305
1012
SEM
Xt
• The odds of getting a t stat this big, or bigger, The odds of getting a t stat this big, or bigger, due to chance would then be determined by due to chance would then be determined by calculating a p-value. calculating a p-value.
04/21/23 HK 396 - Dr. Sasho MacKenzie 11
The P-valueThe P-value
• In Excel, the function TDIST() can be used to In Excel, the function TDIST() can be used to calculate the p-value. calculate the p-value.
• The degrees of freedom are N-1.The degrees of freedom are N-1.• Our example is a two-tailed test because HK Our example is a two-tailed test because HK
students may drink more, or less, than the students may drink more, or less, than the average. I.e., the sample mean could be either average. I.e., the sample mean could be either more or less than the population.more or less than the population.
• Since we set alpha = .05, if the p-value is less Since we set alpha = .05, if the p-value is less than .05, we will state HK students are than .05, we will state HK students are statistically different.statistically different.
04/21/23 HK 396 - Dr. Sasho MacKenzie 12
Graphic of two-tailed one sample Graphic of two-tailed one sample t-testt-test
-3-3 00-1-1-2-2 11 3322-4-4 44
tt
t distribution for N = 30t distribution for N = 30
Combined area beyond Combined area beyond t=2.19 and t = -2.19 is .037t=2.19 and t = -2.19 is .037
From TDIST, p = .037
04/21/23 HK 396 - Dr. Sasho MacKenzie 13
ConclusionConclusion
• Since p=.037 is less than alpha = .05, we reject the null Since p=.037 is less than alpha = .05, we reject the null hypothesis and conclude that HK students consume hypothesis and conclude that HK students consume significantly more drinks per week.significantly more drinks per week.
• The following shows how this would be explained in a The following shows how this would be explained in a study.study.
• It was determined that the average number of alcoholic It was determined that the average number of alcoholic drinks consumed by HK students (12 drinks), per week, drinks consumed by HK students (12 drinks), per week, was significantly more than the typical university student was significantly more than the typical university student (10 drinks), t(29) = 2.19, p = .037.(10 drinks), t(29) = 2.19, p = .037.
04/21/23 HK 396 - Dr. Sasho MacKenzie 14
Independent t-testIndependent t-test
• Determines if two sample means are Determines if two sample means are statistically different.statistically different.
• The null hypothesis is that the means come The null hypothesis is that the means come from the same population, from the same population, XX11 - -XX22 = 0. = 0.
• The bottom part of the t-stat now reflects the The bottom part of the t-stat now reflects the SEM for both samples, but is still a measure SEM for both samples, but is still a measure of how much you could expect the means of of how much you could expect the means of two samples from the same population to two samples from the same population to differ due to chance.differ due to chance.
04/21/23 HK 396 - Dr. Sasho MacKenzie 15
The EquationThe Equation
• The stuff on the bottom of the equation is The stuff on the bottom of the equation is called the standard error of the difference.called the standard error of the difference.
22
21
21
SEMSEM
XXt
04/21/23 HK 396 - Dr. Sasho MacKenzie 16
Independent t-test Independent t-test exampleexample
• Do HK students drink more or less than Do HK students drink more or less than Chemistry students?Chemistry students?
• Null Hypothesis: HK students and Null Hypothesis: HK students and Chemistry students drink the same Chemistry students drink the same amount of alcohol per week.amount of alcohol per week.
04/21/23 HK 396 - Dr. Sasho MacKenzie 17
The CalculationThe Calculation• HK sample of studentsHK sample of students
– Mean = 12, SD = 5, N = 30Mean = 12, SD = 5, N = 30
• Chemistry sample of studentsChemistry sample of students– Mean = 10, SD = 3, N = 30Mean = 10, SD = 3, N = 30
• The odds of getting a t stat this big, or bigger, The odds of getting a t stat this big, or bigger, due to chance would then be determined by due to chance would then be determined by calculating a p-value. calculating a p-value.
88.1
55.91.
1012222
221
21
SEMSEM
XXt
04/21/23 HK 396 - Dr. Sasho MacKenzie 18
Graphic of two-tailed Graphic of two-tailed independent sample t-testindependent sample t-test
-3-3 00-1-1-2-2 11 3322-4-4 44
tt
t distribution for N = 60t distribution for N = 60From TDIST, p = .065
Combined area beyond Combined area beyond t=1.88 and t = -1.88 is .065t=1.88 and t = -1.88 is .065
04/21/23 HK 396 - Dr. Sasho MacKenzie 19
ConclusionConclusion
• Since p=.065 is greater than alpha = .05, we Since p=.065 is greater than alpha = .05, we cannot reject the null hypothesis. There is cannot reject the null hypothesis. There is not enough evidence to suggest HK not enough evidence to suggest HK students drink more or less than Chemistry students drink more or less than Chemistry studentsstudents
• In a study it would be written as:In a study it would be written as:• It was determined that the average number It was determined that the average number
of alcoholic drinks consumed by HK of alcoholic drinks consumed by HK students (12 drinks), per week, was not students (12 drinks), per week, was not significantly different than the Chemistry significantly different than the Chemistry students (10 drinks), t(58) = 1.88, p = .065.students (10 drinks), t(58) = 1.88, p = .065.
04/21/23 HK 396 - Dr. Sasho MacKenzie 20
Dependent (Paired) t-testDependent (Paired) t-test• Determines if two Determines if two correlatedcorrelated sample means are sample means are
statistically different.statistically different.• Required when the same subjects are Required when the same subjects are
measured twice. E.g., Pre-test, Post-test study.measured twice. E.g., Pre-test, Post-test study.• Adjustments are made in how the variability Adjustments are made in how the variability
(SD) in the sample data is calculated. This (SD) in the sample data is calculated. This reduces the denominator in the t-statistic and reduces the denominator in the t-statistic and therefore increases the t-statistic.therefore increases the t-statistic.
• This accounts for reduction in the t-statistic due This accounts for reduction in the t-statistic due to the fact that the same subjects measured to the fact that the same subjects measured twice will show a smaller mean difference than twice will show a smaller mean difference than two completely separate groups. two completely separate groups.
04/21/23 HK 396 - Dr. Sasho MacKenzie 21
The DifferenceThe Difference
• Before any group means or standard deviations Before any group means or standard deviations are calculated, the difference scores between the are calculated, the difference scores between the two measurement times is determined.two measurement times is determined.
• For example, if you have a column of pre-test For example, if you have a column of pre-test scores and a column of post-test scores, then scores and a column of post-test scores, then generate a third column of post minus pre scores.generate a third column of post minus pre scores.
• The t-statistic is then calculated using information The t-statistic is then calculated using information from the column of difference scores.from the column of difference scores.
04/21/23 HK 396 - Dr. Sasho MacKenzie 22
The EquationThe Equation
• The variables in this equations come from a The variables in this equations come from a single column of difference scores.single column of difference scores.
pairsDiff
Diff
NSD
Xt
/
04/21/23 HK 396 - Dr. Sasho MacKenzie 23
Dependent t-test exampleDependent t-test example
• Do HK students drink more alcohol on the Do HK students drink more alcohol on the Saturday prior to a Biomechanics midterm, or Saturday prior to a Biomechanics midterm, or on the following Saturday?on the following Saturday?
• Null Hypothesis: HK students drink the same Null Hypothesis: HK students drink the same amount or less on the following Saturday, amount or less on the following Saturday, compared to the Saturday preceding a compared to the Saturday preceding a Biomechanics midterm.Biomechanics midterm.
04/21/23 HK 396 - Dr. Sasho MacKenzie 24
The Data: Number of The Data: Number of DrinksDrinks
10NSubjectSubject Sat. BeforeSat. Before Sat. AfterSat. After DiffDiff
11 1616 2121 5.05.0
22 66 99 3.03.0
33 1818 2525 7.07.0
44 1111 1010 -1.0-1.0
55 77 77 0.00.0
66 77 44 -3.0-3.0
77 88 1212 4.04.0
88 88 1010 2.02.0
99 1414 1515 1.01.0
1010 1212 1515 3.03.0
1.2DiffX
0.3DiffSD
04/21/23 HK 396 - Dr. Sasho MacKenzie 25
The CalculationThe Calculation• Difference Scores (Before – After)Difference Scores (Before – After)
– Mean = 2.1, SD = 3.0, N = 10Mean = 2.1, SD = 3.0, N = 10
• The odds of getting a t stat this big, or bigger, The odds of getting a t stat this big, or bigger, due to chance would then be determined by due to chance would then be determined by calculating a p-value. calculating a p-value.
24.2100.3
1.2
pairsDiff
Diff
NSD
Xt
04/21/23 HK 396 - Dr. Sasho MacKenzie 26
The P-valueThe P-value
• In Excel, the function TDIST() can be used to In Excel, the function TDIST() can be used to calculate the p-value. calculate the p-value.
• The degrees of freedom are (NThe degrees of freedom are (Npairs pairs - 1).- 1).
• Our example is a one-tailed test because we are Our example is a one-tailed test because we are assuming HK students drink more following a assuming HK students drink more following a midterm. This may not be a good assumption, midterm. This may not be a good assumption, but I needed a one-tailed example.but I needed a one-tailed example.
• Set alpha = .05, if the p-value is less than .05, Set alpha = .05, if the p-value is less than .05, we will state HK students drink significantly we will state HK students drink significantly more following a biomechanics midterm.more following a biomechanics midterm.
04/21/23 HK 396 - Dr. Sasho MacKenzie 27
Graphic of one-tailed Graphic of one-tailed dependent sample t-testdependent sample t-test
-3-3 00-1-1-2-2 11 3322-4-4 44
tt
t distribution for N = 10t distribution for N = 10From TDIST, p = .0258
The area beyond The area beyond t = 2.24 is .0258t = 2.24 is .0258
04/21/23 HK 396 - Dr. Sasho MacKenzie 28
ConclusionConclusion• Since p=.0258 is less than alpha = .05, we Since p=.0258 is less than alpha = .05, we
reject the null hypothesis and conclude that HK reject the null hypothesis and conclude that HK students consume significantly more drinks on students consume significantly more drinks on the Saturday following a midterm.the Saturday following a midterm.
• The following shows how this would be The following shows how this would be explained in a study.explained in a study.
• It was determined that HK students consume It was determined that HK students consume significantly more alcoholic drinks (2.1 more) on significantly more alcoholic drinks (2.1 more) on a Saturday after a midterm than on a Saturday a Saturday after a midterm than on a Saturday before a midterm, t(9) = 2.24, p = .0258.before a midterm, t(9) = 2.24, p = .0258.
04/21/23 HK 396 - Dr. Sasho MacKenzie 29
What if the t-test was two-What if the t-test was two-tailed?tailed?
• The null hypothesis would The null hypothesis would notnot be: HK be: HK students drink students drink the same amount or lessthe same amount or less on on the following Saturday, compared to the the following Saturday, compared to the Saturday preceding a Biomechanics midterm.Saturday preceding a Biomechanics midterm.
• But rather it would be: HK students drink But rather it would be: HK students drink the the same amountsame amount on the following Saturday, on the following Saturday, compared to the Saturday preceding a compared to the Saturday preceding a Biomechanics midterm.Biomechanics midterm.
04/21/23 HK 396 - Dr. Sasho MacKenzie 30
Graphic of two-tailed Graphic of two-tailed dependent sample t-testdependent sample t-test
-3-3 00-1-1-2-2 11 3322-4-4 44
tt
t distribution for N = 10t distribution for N = 10From TDIST, p = .0516
Combined area beyond Combined area beyond t=2.24 and t = -2.24 is .0516t=2.24 and t = -2.24 is .0516
Whoa! We no longer have significance at
=.05
04/21/23 HK 396 - Dr. Sasho MacKenzie 31
Interpreting the P-valueInterpreting the P-value• In an experiment of this size, if the populations really
have the same mean, what is the probability of observing at least as large a difference between sample means as was, in fact, observed?
• There is a p% chance of observing a difference as large as you observed even if the two population means are identical (the null hypothesis is true).
• Random sampling from identical populations would lead to a difference smaller than you observed in 1-p% of experiments, and larger than you observed in p% of experiments.