Multiple Regression

   

The Problem

• Using several predictors to predict the dependent variable

• Finding a measure of overall fit

• Weighting each predictor

An Example

• Study by Kliewer et al. (1998) on effect of violence on internalizing behavior Define internalizing behavior

• Predictors Degree of witnessing violence Measure of life stress Measure of social support

Violence and Internalizing

• Subjects are children 8-12 years Lived in high-violence areas Hypothesis: violence and stress lead to

internalizing behavior.

• Data available at www.uvm.edu/~dhowell/StatPages/

More_Stuff/Kliewer.dat

Intercorrelation Matrix

Preliminary Stuff

• Note that both Stress and Witnessing Violence are significantly correlated with Internalizing.

• Note that predictors are largely independent of each other.

Multiple Correlation

• Directly analogous to simple r

• Always capitalized (e.g. R)

• Always positive Correlation of with observed Y

• where is computed from regression equation

Often reported as R 2 instead of R

YY

R 2

Regression Coefficients

• Slopes and an intercept.

• Each variable adjusted for all others in the model.

• Just an extension of slope and intercept in simple regression

• SPSS output on next slide

Slopes and Intercept

Regression Equation

• A separate coefficient for each variable These are slopes

• An intercept (here called b0 instead of a)

517.0076.0272.0038.0

ˆ0332211

SocSuppStressWit

bXbXbXbY

Interpretation

• Note slope for Witness and Stress are positive, but slope for Social Support is negative. Does this make sense?

• If you had two subjects with identical Stress and SocSupp, a one unit increase in Witness would produce 0.038 unit increase in Internal.

Cont.

Interpretation--cont.

• The same holds true for other predictors.

• t test on two slopes are significant SocSupp not significant. Elaborate

• R 2 has same interpretation as r 2. 13.6% of variability in Internal

accounted for by variability in Witness, Stress, and SocSupp.

Interpretation--cont.

• Intercept usually not meaningful. Prediction when all predictors are 0.0

Predictions

• Assume Witness = 20, Stress = 5, and SocSupp = 35.

023.

517.0)35(076.)5(272.)20(038.

517.0*074.*273.*038.ˆ

SocSuppStressWitY

Hypothesis Testing

• Test on R 2 given in Analysis of Variance table

Cont.

Testing--cont.

• Tests on regression coefficients given along with the coefficients.

• See next slide

• Note tests on each coefficient.

Testing Slopes and Intercept

Raw vs. Standardized Betas

• Raw beta weights are in their original units (not standardized) Use for prediction

• Standardized beta weights are not in their original units Use for comparing predictors

Procedures for Entering Predictor Variables

• Simultaneous Enter all predictors simultaneously

• Forward Enter largest predictor first and test. If

sig. enter next largest partial r. Continue until you no more sig. predictors (sig. level can change after entered but predictor remains)

Procedures for Entering Predictor Variables

Continue• Backward

Enter all predictors first. Remove smallest nonsig. predictor and retest. Continue until model contains no nonsig predictors (sig. level can change after entered but predictor remains)

• Stepwise Combing forward and backward

Procedures for Entering Predictor Variables

Continued• Hierarchical

Enter predictors in pre-determined order as a means of hypothesis testing• E.g., does social support influence

internalizing above and beyond stress and witnessing violence?

Stepwise Results for Kliewer data

Potential Problems

• Multicollinearity – large correlations between predictors VIF (Variance Inflation Factor) < 10 Doesn’t qualify overall model but

estimates of individual predictors can be unstable

• Uncorrelated residuals Durban-Watson test (0-4): 1.5 – 2.5

generally acceptable