Post on 15-Apr-2017
Presentation on Regression Analysis
Presentation on Chapter 9
Presented by
Dr.J.P.VermaMSc (Statistics), PhD, MA(Psychology), Masters(Computer Application)
Professor(Statistics)Lakshmibai National Institute of Physical Education,
Gwalior, India(Deemed University)
Email: vermajprakash@gmail.com
2
What is Regression ?
To answer the questions like
Going back to the original
Why to use?
Can I predict the fat % on the basis of the skinfolds?
What will be the weight of the person if the height is 175 cms?
3
Purpose of Regression Analysis
Which has not occurred so far Which is difficult to measure in field situation Which should occur for a particular independent variable
To predict the phenomenon
4
Types of Regression Analysis
Simple Regression Multiple Regression
5
This Presentation is based on
Chapter 9 of the book
Sports Research with Analytical Solution Using SPSS
Published by Wiley, USA
Complete Presentation can be accessed on
Companion Website
of the Book
Request an Evaluation Copy For feedback write to vermajprakash@gmail.com
6
Simple Regression
Developing Regression equation With
One Dependent and one Independent variable
7
Mechanism of prediction in regression analysis
Develop an equation of line between Y(dependent)
and X(independent) variablesy
x
cbxy
c
Height
Weight
8
Application of Regression in Physical Education
Predicting Obesity Coronary Heart
Disease Risk Body mass index Fitness status
Projection ofWinning MedalsEstimating performanceRuns scored
In Physical Education In Sports
Efficient prediction enhances success in sports
9
Procedure for Developing Regression Equations
Deviation methods Least Square methods
How to find the regression line?
Y=bX+c
10
Deviation Method for Constructing Regression Equation
Regression equation of Y on X
)XX(rYYx
y
YXrXrYx
y
x
y
cbXY
Regression equation of X on Y
)YY(rXXy
x
b = regression coefficient x
yr
c= slope YXr
x
y
…………(1)
…………(2)
Computing coefficients
11
Can these Two Equations be Same ?Yes if the slopes of the two equations are same
)xx(r)yy(x
y
)xx(r
)yy(x
y
------(1)
------(2)
After solving )xx(r)yy(x
y
------(3)
)yy(r)xx(y
x
------(4)
Equation (3) and (4) would be same if x
y
x
y
rr
r2 = 1 or r = 1
Implication If the relationship between two variables is either perfectly positive or perfectly negative one can be estimated with the help of others with 100% accuracy, which is rarely the case.
12
Precaution in Regression Analysis
But association is a necessary prerequisite for inferring causation
The independent variable must precede the dependent variable in time.
The dependent and independent variables must be plausibly lined by a theory
Regression focuses on association and not causation
13
Least Square Method for Constructing Regression Equation
Uses the concept of differential calculus
For n population points (x1,y1), (x2,y2), …….(xN , yN) an aggregate trend line can be obtained = β0 + β1xy
where : the estimated value of y
β0 : the population intercept (regression constant) β1 : the population slope(regression coefficient)
y
i yi = β0 + β1x +
For a particular score yi
Almost always Regression lines are developed on the basis of sample data hence these β0 and β1 are estimated by the sample slope b0 and intercept b1
14
Regression Line with Least Square Method
Infinite number of trend lines can be developed by changing the slope b1 and intercept b0
yi = b0 + b1xi + i
For n sample data the aggregate regression line = b0 + b1x y
y
x
0b
xbby 10
15
Regression Line with Least Square Method To find the best line so that the sum of
squared deviations is minimized What the issue is?
For a particular point (x1,y1) in the scattergram
y
x
i
To get the best line needs to be minimized
2i10i
2ii
2i
2 )xbby()yy(S
- A least square method
iy
xbby 10
0b
ii yy
yy ii
or
y1 = b0 + b1x1 +
1
2i
2S
16
What we do in Least Square method?
Find the values of slope(b0) and intercept (b1) for which the S2 is minimized
This is done by using the differential calculus
210i
2ii
2 )xbby()yy(S
0bS
0)xbby(2
n
1ii10i
1bS
n
1ii10ii 0)xbby(x2
Solving we get normal equations
n
1ii0
n
1ii1 ynbxb
n
1iii
n
1ii0
n
1i
2i1 yxxbxb
22
2
0 )x(xnxyxxyb 221
xxnyxxyn
b
xbby 10 - A line of best fit
17
Assumptions in Regression Analysis
Data must be parametric There is no outliers in the data Variables are normally distributed(if not try log, square root, square, and inverse
transformation The regression model is linear in nature The errors are independent (no autocorrelation) The error terms are normally distributed There is no multicollinearity The error has a constant variance(assumption of homoscedasticity)
18
Regression Analysis -An Illustration
Athletes data_________________
Height LBWin cms (in lbs)(x) (y)_________________191 162.5186 136191.5 163.5188 154190 149188.5 140.5193 157.3190.5 154.5189 151.5192 160.5_________________
19
SPSS Commands for Regression Analysis
After selecting variables
Click the tag Statistics on the screen Check the box of
R squared change Descriptive Part and partial correlations
Press Continue
Click the Method option and select any one of the following option
Stepwise Enter Forward Backward
Press O.K for output
Analyze Regression Linear
20
Different Methods Used in Regression Analysis
Variables selected in a particular stage is tested for its significance at every stage
Stepwise
All variables are selected for developing regression equationEnter
Variables once selected in a particular stage is retained in the model in subsequent stages
Forward
All variables are used to develop the regression model and then the variables are
dropped one by one depending upon their low predictability.
Backward
21
Output of Regression Analysis in SPSS
Model summary ANOVA table showing F-values for all the
models Regression coefficients and their significance
22
SPSS output – Model Summary
Model Summaryb
_________________________________________________
Model R R Square Adjusted R Std. Error of Square the Estimate
1 .816 .666 .624 5.56___________________________________________a. Predictors: (Constant), Heightb. Dependent Variable: Body Weight
23
SPSS output Testing Regression coefficients
Regression analysis output for the Body weight example ______________________________________________________________________
Unstandardized Standardized Coefficients Coefficients t Sig.
___________________________________ Model B Std. Error Beta ______________________________________________________________________
1 (Constant) - 517.047 167.719 -3.083 .015Height 3.527 .883 .816 3.995(Click) .004
______________________________________________________________________ Dependent Variable: Body weight R = 0.816 R2 = 0.666 Adjusted R2 =0.624
Look at the value of t computed in the last slide and in the SPSS
output Y(Weight) = -517.047 + 3.527 ×(Height)
24
SPSS Output - ANOVA table
F = t2 = 3.9952 = 15.96
In simple regression significance of regression coefficient and model are same
Significance of the model is tested by F value in ANOVA
ANOVA table___________________________________________________________________ Model Sum of Squaresdf Mean Square F Sig.___________________________________________________________________
2 Regression 494.203 1 494.203 15.959.004Residual 247.738 8 30.967Total 741.941 9
___________________________________________________________________ a. Predictors: (Constant), Height
Back to R2
25
Testing Assumptions of Regression Analysis with Residuals
Let’s See what the residual is?
26
Analyzing Residuals
Table Computation of residuals ____________________________________________Height Body weightin cms (in lbs)x y ___________________________________________
191 162.5 156.61 5.89
186 136 138.975 -2.975
191.5 163.5 158.3735 5.1265
188 154 146.029 7.971190 149 153.083 -4.083
188.5 140.5 147.7925 -7.2925
193 157.3 163.664 -6.364
190.5 154.5 154.8465 -0.3465189 151.5 149.556 1.944
192 160.5 160.137 0.363____________________________________________
yResiduals are estimates of experimental errors
For instance, for x= 188, = -517.047 + 3.527×188 = 146.029y
Maximum error: 7.97 1 lbs for height =188 cms Minimum error: 0.3465 lbs for height = 190.5 cms.
Worst case
Best case
Useful in identifying the outliers
yy
27
Residual Plot
184 186 188 190 192 194
Res
idua
ls
-10-8-6-4-202468
Height in cms
Residual plot for the data on lean body mass and height
Obtained by plotting an ordered pair of (xi, y- )
y
Useful in testing the assumptions in the regression analysis
28
How to test linearity of regression model?
Independent variable x
0R
esid
uals
o
ooo
ooo
o
o
o
o
o
oo
o
o
o
o
o
oo
o
o
oo
o oo
oo oo o
oo o
o
o
o
o
o oo
o
o
oooo
oo
o
ooo
o
ooo
ooo
oo
oo
o
oo
A curvilinear regression model
Residual plot
For low and high values of x the residuals are positives
And for middle value it is negative
29
How to test the Independence of errors?
oo
oo
o
o
oo o
oo
ooo
ooo
oo
o
oo
ooo o
oo
o
o oo
ooo
ooo ooo
oo o
o
o
oo oo
ooo o o o o
o oo ooo
oooo
o
x
0
Res
idua
ls
oo o
o oo
o
ooo
o
o
o oo
o ooo o
o oo o
o
oo
ooo o oooo o
oo oo
oooooooo o
o ooo
o oo
oo
Independent variable
Showing that the errors are related
No serial correlation should occur between a given error
term and itself over various time intervals
What is the pattern? : small positive residual occurs next to a small positive residual and a larger positive residual occurs next to the large positive residual
30
How to test the normality of error terms?
Normal Q-Q plot of the residuals
Error to be normally distributed all the points should be very close to the straight line
31
How to test that errors have constant variance - A test of homoscedasticity
Independent variable
o
ooo
ooo
oo
o
o
ooo
oo
o
o
o
oo
oo
oo o
oo
oo oo
oo oo
o
o
oo
o oo
oo
oo
oo
ooo
oo
oo
oo
o ooo oo
oo
o o
oo o
ooo o o
oooo
ooooo
ooo o
oooo
ooooo o
o ooo oo
ooo ooo o
oo o o o
o oo ooo
o
oo
o
o
o
oooo
oooo
o
x
0R
esid
uals
o oo oo o
oo o
o o o
o oo o o oo ooo
o
Showing unequal error variance
For homoscedasticity assumption to holds true
variations among the error terms should be similar at different points of x.
Back
32
Healthy residual Plot
The regression model is linear in nature The errors are independent The error terms are normally distributed The error has a constant variance
Holds all the assumptions of regression analysis
Independent variable
oo
oo
ooo
oo o
o
ooo
o
oo o
o
oo
oo
oo o
o
o
ooo
oo
o ooo
o
oo
ooo
oo
oo ooo
oooo
o
oo
o ooo ooo o
o o
o
ooo o o
oooo
oooo
oo
oo o
oooo
oooo o
oo o
oo oo
ooo ooo o
oo o o o o oo o
oo o
o
oo
oo
o
o
o oo
o
o
o
x
0
Res
idua
ls
o oo oo o
oo ooo
oo oo
o
o oo
o o oo o o o
o oooo o o oo o
oo
oooo
oo
oo oo o
o o oo o
o o oo ooo oo
o o o
o oooooo
oo
oo
oo o
oo
oo
oo o
Figure 6.9 Healthy residual plot
33
Different ways of testing a regression model
Analyzing residuals Residual Plot Standard error of estimate Testing significance of slopes Testing the significance of overall model coefficient of determination(R2)
34
To buy the book
Sports Research With Analytical Solutions Using SPSS
and all associated presentations click Here
Complete presentation is available on companion website of the book
For feedback write to vermajprakash@gmail.comRequest an Evaluation Copy