Education 793 Class Notes

Presentation 10

Chi-Square Tests and One-Way ANOVA

2

Review: CrosstabsVIEW8612 MARRIED WOMEN BEST IN HOME by SEX86 STUDENT'S SEX   SEX86 Page 1 of 1 Count | Row Pct |MALE FEMALE Col Pct | Row | 1 | 2 | TotalVIEW8612 --------+--------+--------+ 1 | 762 | 1590 | 2352 DISAGREE STRONG | 32.4 | 67.6 | 55.1 | 36.4 | 73.1 | +--------+--------+ 2 | 889 | 343 | 1232 DISAGREE SOME | 72.2 | 27.8 | 28.9 | 42.5 | 15.8 | +--------+--------+ 3 | 343 | 170 | 513 AGREE SOME | 66.9 | 33.1 | 12.0 | 16.4 | 7.8 | +--------+--------+ 4 | 97 | 72 | 169 AGREE STRONG | 57.4 | 42.6 | 4.0 | 4.6 | 3.3 |  +--------+--------+ Column 2091 2175 4266 Total 49.0 51.0 100.0

How do we assess whether the relationship we observe in the table is statistically significant?

We need a sampling distribution

3

Chi-Square

• The Chi-Square statistic is used to assess whether or not the observed frequencies in the table are significantly different from the expected frequencies.

• It is used in the form of counts (not percents).

• Observations must be mutually exclusive, one individual can only fall into one cell

4

Chi-Square

• Chi square is a nonparametric test. It does not require the sample data to be more or less normally distributed (as parametric tests like z and t-tests do), although it relies on the assumption that the variable is normally distributed in the population from which the sample is drawn.

• But chi square, while forgiving, does have some requirements: – The sample must be randomly drawn from the population – Data must be reported in raw frequencies (not percentages)– Values/categories on independent and dependent variables

must be mutually exclusive and exhaustive

– Observed frequencies cannot be too small

5

Three Types of Tests

• Goodness of Fit: Often called a One-Way 2 test. Used for data in one row and =>2 columns

• Two-way test: Used for data in =>2 rows and =>2 columns– Test of Independence (One population)– Test of Homogeneity (Two or more

populations)

6

Lucky US!!

• All three tests are computed the exact same way.

7

Example

Academic MajorSex Physics Enginr English Design Total

Male 108 (A) 345 (B) 94 (C) 17 (D) 564

Female 8 (E) 12 (F) 253 (G) 60 (H) 333

Total 116 357 347 77 897

1. Compute the Expected Cell Frequencies

1. Example: A = (Row Total X Column Total) / Grand Total

= (564 X 116) / 897

8

Example

Academic MajorSex Physics Enginr English Design Total

Male 108 (A) 345 (B) 94 (C) 17 (D) 564

Female 8 (E) 12 (F) 253 (G) 60 (H) 333

Total 116 357 347 77 897

After we compute all expected cell counts: Use formula to compute Chi-Square statistic:

...)108()( 22

2

A

A

Exp

ExpObs

df=(R-1)(C-1)

9

Details: Sample Sizes

• In general, the greater the degrees of freedom (i.e., the more values/categories on the independent and dependent variables), the more lenient the minimum expected frequencies threshold

• Some recommend 5 or more per cell, some recommend more than 5, and others require 10 or more

• A common rule is 5 or more in all cells of a 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count

10

Unix Example: Married women at home

get file cirp8690.sav. crosstab view8612 by sex86 /cells=count row column /statistics=chisq.  

 

 VIEW8612 MARRIED WOMEN BEST IN HOME by SEX86 STUDENT'S SEX   SEX86 Page 1 of 1 Count | Row Pct |MALE FEMALE Col Pct | Row | 1 | 2 | TotalVIEW8612 --------+--------+--------+ 1 | 762 | 1590 | 2352 DISAGREE STRONG | 32.4 | 67.6 | 55.1 | 36.4 | 73.1 | +--------+--------+ 2 | 889 | 343 | 1232 DISAGREE SOME | 72.2 | 27.8 | 28.9 | 42.5 | 15.8 | +--------+--------+ 3 | 343 | 170 | 513 AGREE SOME | 66.9 | 33.1 | 12.0 | 16.4 | 7.8 | +--------+--------+ 4 | 97 | 72 | 169 AGREE STRONG | 57.4 | 42.6 | 4.0 | 4.6 | 3.3 |  +--------+--------+ Column 2091 2175 4266 Total 49.0 51.0 100.0 

11

Actual vs. Expected

crosstab view8612 by sex86 /cells=count expected /statistics=chisq.

VIEW8612 MARRIED WOMEN BEST IN HOME by SEX86 STUDENT'S SEX  SEX86 Page 1 of 1 Count | Exp Val |MALE FEMALE | Row | 1 | 2 | TotalVIEW8612 --------+--------+--------+ 1 | 762 | 1590 | 2352 DISAGREE STRONG |1152.8 |1199.2 | 55.1% +--------+--------+ 2 | 889 | 343 | 1232 DISAGREE SOME | 603.9 | 628.1 | 28.9% +--------+--------+ 3 | 343 | 170 | 513 AGREE SOME | 251.4 | 261.6 | 12.0% +--------+--------+ 4 | 97 | 72 | 169 AGREE STRONG | 82.8 | 86.2 | 4.0% +--------+--------+ Column 2091 2175 4266 Total 49.0% 51.0% 100.0%  Chi-Square Value DF Significance-------------------- ----------- ---- ------------ Pearson 594.08275 3 .00000

12

Analysis of Variance

• The most basic type of Analysis of Variance is one in which there is only one continuous, dependent variable and more than two treatment levels (One-Way); the more complex types use similar logic

• Unlike a t-test that can only be used to compare two means, ANOVA can be used to compare two or more means:

Null Hypothesis

13

ANOVA

• Example: We want to look at whether or not there is a statistically significant difference in SAT prep programs. One outcome (SAT score). Three levels in type of program: Kaplan, Sylvan, Mitron

• If the variability between groups is greater than the variability within groups, this is evidence of a treatment effect.

14

How it Works

• Analysis of Variance works by comparing two estimates of variance ( 2).

– One estimate (Mean Square Error or "MSE") is based on the variances within the samples. The MSE is an estimate of 2 whether or not the null hypothesis is true.

– The second estimate (Mean Square Between or "MSB" for short) is based on the variance of the sample means. The MSB is only a good estimate of 2 if the null hypothesis is true. If the null hypothesis is false then MSB estimates something larger than 2.

15

How it Works

• If the null hypothesis is true, then MSE and MSB should be about the same since they are both estimates of the same quantity ( 2). This would give a ratio of the two around 1.

• However, if the null hypothesis is false then MSB can be expected to be larger than MSE since MSB is estimating a quantity larger then . Thus, if MSB is sufficiently larger than MSE, then the ratio of the two would be greater than 1, suggesting evidence to reject the Null hypothesis that there is no difference between the groups.

16

Computations, Definitions, Distributions

F = MSB / MSE

Two degrees of freedom are involved:

dfnumerator = a - 1 dfdenominator = N - a

a is the number of groups N is the total number of subjects in all groups

                                 

17

An Interactive Demonstration

Simulation of ANOVA

18

Design Requirements

• There is one dependent variable with two or more treatment levels

• The levels of the dependent variables differ either quantitatively or qualitatively

• A subject may appear in only one group

19

Assumptions

• The subjects scores are independent

• The scores within each treatment (level) are normally distributed

• The variances across treatment groups (levels) are equal (homogeneity)

• When cell sizes are equal, ANOVA is robust to violation of the homogeneity

20

Other Details

• In this course we will consider only fixed effect ANOVA

• The F-test is an omnibus test– It tests only if there is A difference among the

means, we don’t know where the differences exist.

• In more advanced applications it is possible to do post-hoc comparisons to find out which groups means differ

21

Next Week

• Analysis of Covariance– Chapter 17 p. 504-516