Section 7.1 Hypothesis Testing: Hypothesis: Null Hypothesis (H 0 ): Alternative Hypothesis (H 1 ):
Hypothesis Testing. Outline The Null Hypothesis The Null Hypothesis Type I and Type II Error Type I...
-
Upload
lorraine-greer -
Category
Documents
-
view
235 -
download
8
Transcript of Hypothesis Testing. Outline The Null Hypothesis The Null Hypothesis Type I and Type II Error Type I...
Hypothesis Hypothesis TestingTesting
OutlineOutline
The Null HypothesisThe Null Hypothesis
Type I and Type II ErrorType I and Type II Error
Using Statistics to test the Null HypothesisUsing Statistics to test the Null Hypothesis
The Logic of Data AnalysisThe Logic of Data Analysis
Research Questions and Research Questions and HypothesesHypotheses
Research question:Research question: Non-directional:Non-directional:
No stated expectation about outcomeNo stated expectation about outcome Example:Example:
Do men and women differ in terms of Do men and women differ in terms of conversational memory?conversational memory?
Hypothesis:Hypothesis: Statement of expected relationshipStatement of expected relationship
Directionality of relationshipDirectionality of relationship Example:Example:
Women will have greater conversational memory Women will have greater conversational memory than menthan men
Grounding Hypotheses in Grounding Hypotheses in TheoryTheory
Hypotheses have an underlying rationale:Hypotheses have an underlying rationale: Logical reasoning behind the direction of the Logical reasoning behind the direction of the
hypotheses (theoretical rationale – hypotheses (theoretical rationale – explanation)explanation)
Why do we expect women to have better Why do we expect women to have better conversational memory? conversational memory?
Theoretical rationale based on:Theoretical rationale based on: 1. Past research1. Past research 2. Existing theory2. Existing theory 3. Logical reasoning3. Logical reasoning
The Null HypothesisThe Null Hypothesis
Null Hypothesis - the absence of a Null Hypothesis - the absence of a relationshiprelationship E..g., There is no difference between men’s and E..g., There is no difference between men’s and
women’s with regards to conversational women’s with regards to conversational memoriesmemories
Compare observed results to Null HypothesisCompare observed results to Null Hypothesis How different are the results from the null How different are the results from the null
hypothesis?hypothesis?
We do not propose a null hypothesis as We do not propose a null hypothesis as research hypothesis - need very large research hypothesis - need very large sample size / powersample size / power Used as point of contrast for testingUsed as point of contrast for testing
Hypotheses testingHypotheses testing
When we test observed results against null:When we test observed results against null: We can make two decisions:We can make two decisions:
1. Accept the null1. Accept the null No significant relationshipNo significant relationship Observed results similar to the Null HypothesisObserved results similar to the Null Hypothesis
2. Reject the null2. Reject the null Significant relationshipSignificant relationship Observed results different from the Null Observed results different from the Null
HypothesisHypothesis
Whichever decision, we risk making an errorWhichever decision, we risk making an error
Type I and Type II ErrorType I and Type II Error
1. Type I Error1. Type I Error Reality: No relationshipReality: No relationship Decision: Reject the nullDecision: Reject the null
Believe your research hypothesis have received Believe your research hypothesis have received support when in fact you should have disconfirmed support when in fact you should have disconfirmed itit
Analogy: Find an innocent man guilty of a crimeAnalogy: Find an innocent man guilty of a crime
2. Type II Error2. Type II Error Reality: RelationshipReality: Relationship Decision: Accept the nullDecision: Accept the null
Believe your research hypothesis has not received Believe your research hypothesis has not received support when in fact you should have rejected the support when in fact you should have rejected the null.null.
Analogy: Find a guilty man innocent of a crimeAnalogy: Find a guilty man innocent of a crime
Potential outcomes of Potential outcomes of testingtesting
DecisionDecision
Accept NullAccept Null Reject Reject NullNull
RR
EE No No
AA RelationshipRelationship
LL
II RelationshipRelationship
TT
YY
1 2
3 4
Potential outcomes of Potential outcomes of testingtesting
DecisionDecision
Accept NullAccept Null Reject Reject NullNull
RR
EE No No
AA RelationshipRelationship
LL
II RelationshipRelationship
TT
YY
Correct decision 2
3 4
Potential outcomes of Potential outcomes of testingtesting
DecisionDecision
Accept NullAccept Null Reject Reject NullNull
RR
EE No No
AA RelationshipRelationship
LL
II RelationshipRelationship
TT
YY
1 2
3 Correctdecision
Potential outcomes of Potential outcomes of testingtesting
DecisionDecision
Accept NullAccept Null Reject Reject NullNull
RR
EE No No
AA RelationshipRelationship
LL
II RelationshipRelationship
TT
YY
1 Type I Error
3 4
Potential outcomes of Potential outcomes of testingtesting
DecisionDecision
Accept NullAccept Null Reject Reject NullNull
RR
EE No No
AA RelationshipRelationship
LL
II RelationshipRelationship
TT
YY
1 2
Type II Error 4
Potential outcomes of Potential outcomes of testingtesting
DecisionDecision
Accept NullAccept Null Reject Reject NullNull
RR
EE No No
AA RelationshipRelationship
LL
II RelationshipRelationship
TT
YYType II Error Correct
decision
Type I Error Correctdecision
Function of Statistical Function of Statistical TestsTests
Statistical tests determine:Statistical tests determine: Accept or Reject the Null HypothesisAccept or Reject the Null Hypothesis
Based on probability of making a Type I Based on probability of making a Type I errorerror Observed results compared to the results Observed results compared to the results
expected by the Null Hypothesesexpected by the Null Hypotheses What is the probability of getting observed What is the probability of getting observed
results if Null Hypothesis were true?results if Null Hypothesis were true? If results would occur less than 5% of the If results would occur less than 5% of the
time by simple chance then we reject the time by simple chance then we reject the Null HypothesisNull Hypothesis
Start by setting level of Start by setting level of risk of making a Type I risk of making a Type I
ErrorError How dangerous is it to make a Type I Error:How dangerous is it to make a Type I Error:
What risk is acceptable?:What risk is acceptable?: 5%? 5%? 1%?1%? .1%? .1%?
Smaller percentages are more conservative in Smaller percentages are more conservative in guarding against a Type I Errorguarding against a Type I Error
Level of acceptable risk is called Level of acceptable risk is called “Significance level” :“Significance level” : Usually the cutoff - <.05Usually the cutoff - <.05
Conventional Significance Conventional Significance LevelsLevels
.05 level (5% chance of Type I Error).05 level (5% chance of Type I Error)
.01 level (1% chance of Type I Error).01 level (1% chance of Type I Error)
.001 level (.1% chance of Type I Error).001 level (.1% chance of Type I Error)
Rejecting the Null at the .05 level means:Rejecting the Null at the .05 level means: Taking a 5% risk of making a Type I ErrorTaking a 5% risk of making a Type I Error
Steps in Hypothesis Steps in Hypothesis TestingTesting
1. State research hypothesis1. State research hypothesis
2. State null hypothesis2. State null hypothesis
3.Set significance level (e.g., .05 level)3.Set significance level (e.g., .05 level)
4. Observe results4. Observe results
5. Statistics calculate probability of results 5. Statistics calculate probability of results if null hypothesis were trueif null hypothesis were true
6. If probability of observed results is less 6. If probability of observed results is less than significance level, then reject the nullthan significance level, then reject the null
Guarding against Type Guarding against Type I ErrorI Error
Significance level regulates Type I ErrorSignificance level regulates Type I Error
Conservative standards reduce Type I Conservative standards reduce Type I Error:Error: .01 instead of .05, especially with large .01 instead of .05, especially with large
samplesample
Reducing the probability of Type I Error:Reducing the probability of Type I Error: Increases the probability of Type II ErrorIncreases the probability of Type II Error
Sample size regulates Type II ErrorSample size regulates Type II Error The larger the sample, the lower the The larger the sample, the lower the
probability of Type II Error occurring in probability of Type II Error occurring in conservative testingconservative testing
Statistical PowerStatistical Power
The power to detect significant The power to detect significant relationshipsrelationships The larger the sample size, the more powerThe larger the sample size, the more power The larger the sample size, the lower the The larger the sample size, the lower the
probability of Type II Errorprobability of Type II Error
Power = 1 – probability of Type II ErrorPower = 1 – probability of Type II Error
Statistical AnalysisStatistical Analysis
Statistical analysis:Statistical analysis: Examines observed dataExamines observed data Calculates the probability that the results Calculates the probability that the results
could occur by chance (I.e., if Null was true)could occur by chance (I.e., if Null was true)
Choice of statistical test depends on:Choice of statistical test depends on: Level of measurement of the variables in Level of measurement of the variables in
question:question: Nominal, Ordinal, Interval or RatioNominal, Ordinal, Interval or Ratio
Logic of data analysisLogic of data analysis
Univariate analysisUnivariate analysis One variable at a time (descriptive)One variable at a time (descriptive)
Bivariate analysisBivariate analysis Two variables at a time (testing relationships)Two variables at a time (testing relationships)
Multivariate analysisMultivariate analysis More than two variables at a time (testing More than two variables at a time (testing
relationships and controlling for other relationships and controlling for other variables) variables)
VariablesVariables
Dependent variable:Dependent variable: What we are trying to predictWhat we are trying to predict E.g., Candidate preferenceE.g., Candidate preference
Independent variables:Independent variables: What we are using as predictorsWhat we are using as predictors E.g., Gender, Party affiliationE.g., Gender, Party affiliation
Testing hypothesis for Testing hypothesis for two nominal variablestwo nominal variables
VariablesVariables Null hypothesisNull hypothesisProcedureProcedure
GenderGender
Passing is notPassing is not Chi- Chi-squaresquare
related to genderrelated to gender
Pass/FailPass/Fail
Testing hypothesis for Testing hypothesis for one nominal and one one nominal and one
ratio variableratio variableVariablesVariables Null hypothesisNull hypothesis ProcedureProcedure
GenderGender
Score is notScore is not T-T-testtest
related to genderrelated to gender
Test scoreTest score
Testing hypothesis for Testing hypothesis for one nominal and one one nominal and one
ratio variableratio variableVariableVariable Null hypothesisNull hypothesis ProcedureProcedure
Year in schoolYear in school
Score is notScore is not
related to year inrelated to year inANOVAANOVA
schoolschool
Test scoreTest score
Can be used when nominal variable has more than two Can be used when nominal variable has more than two categories and can include more than one independent categories and can include more than one independent variablevariable
Testing hypothesis for Testing hypothesis for two ratio variablestwo ratio variables
VariableVariable Null hypothesisNull hypothesisProcedureProcedure
Hours spentHours spent
studyingstudying Score is notScore is not
related to hoursrelated to hours Correlation Correlation
spent studyingspent studying
Test scoreTest score
Testing hypothesis for Testing hypothesis for more than two ratio more than two ratio
variablesvariablesVariableVariable Null hypothesisNull hypothesis ProcedureProcedure
Hours spentHours spent
studying studying Score is positivelyScore is positively
related to hoursrelated to hours
Classes Classes spent studying andspent studying and Multiple Multiple
missed missed negatively relatednegatively related regression regression
to classes missedto classes missed
Test scoreTest score
Commonality across all Commonality across all statistical analysis statistical analysis
proceduresprocedures Set the significance level:Set the significance level:
E.g., .05 levelE.g., .05 level Means that we are willing to conclude that there Means that we are willing to conclude that there
is a relationship if:is a relationship if: Chance of Type I error is less than 5%Chance of Type I error is less than 5%
Statistical tests tell us whether:Statistical tests tell us whether: The observed relationship has less than a 5% The observed relationship has less than a 5%
chance of occurring by chancechance of occurring by chance
Summary of Statistical Summary of Statistical ProceduresProcedures
Variables Procedure
Nominal IV, Nominal DV Chi-square
Nominal IV, Ratio DV T-test
Multiple Nominal IVs, Ratio DV
ANOVA
Ratio IV, Ratio DV Pearson’s R
Multiple Nominal IVs, Ratio DV with ratio covariates
ANCOVA
Multiple ratio Multiple Regression