Linear Regression Handout

8/6/2019 Linear Regression Handout

1/17

Linear Regression - Topics

Basics of Linear Regression

Variation in Linear Regression

Linear Regression Analysis Goodness of Fit

Standard Error terms for Linear Regression

Hypothesis testing


2/17

Regression - Types


3/17

Linear Regression

A statistical technique that uses a single,independent variable (X) to estimate a single

dependent variable (Y).

Based on the equation for a line:

Y = b + mX


4/17

Linear Regression - Model

iI

X

Y

Y XFF!Yi

Xi

?(the actual value ofYi)


5/17

Linear Regression - Model

ii iY F F I

Regression Coefficients for a . . .

Population

SampleY = b0 + b1Xi + e

Y = b0 + b1Xi


6/17

ANOVA D - Variati

SST

SSE

SST

SST = SST + SSE

3-hour 1-day 10-week

70 61 82

77 75 88

76 67 90

80 63 96

84 66 92

78 68 98

SST is a measure f the t talvariati f bservati s. Ameasure f the differe ces ibservati s.

Due t treatme ts.

and m/unexplained.


7/17

Linear Regression - Variation

X Y

Temperature Sales

63 1.52

70 1.68

73 1.875 2.05

80 2.36

82 2.25

85 2.68

88 2.9

90 3.14

91 3.06

92 3.24

75 1.92

98 3.4100 3.28

92 3.17

87 2.83

84 2.58

88 2.86

80 2.26

82 2.14

76 1.98

Ice Cream ExampleIce Cream Sales

0

0.5

1

1.5

2

2.5

3

3.5

4

0 20 40 60 80 100 120

Y

Y= 2.53


8/17

Linear egressi n - Variati n

X Y

Temperature Sales

63 1 52

70 1 68

73 1 875 2 05

80 2 36

82 2 25

85 2 68

88 2 9

90 3 14

91 3 06

92 3 24

75 1 92

98 3 4

100 3 28

92 3 17

87 2 83

84 2 58

88 2 86

80 2 26

82 2 14

76 1 98

Ice Cream Example Ice eam Sa es

0

0 5

1

1 5

2

2 5

3

3 5

4

0 20 40 60 80 100 120

0 1Y b b X ! ! Sample

Regression Line


9/17

Linear egressi n - Variati n

SST

SSE

SS

SST = SS + SSE

ue t regressi n.

and m/unexplained.


10/17

Linear Regression - Variation

Xi

Y

X

Y

SST= (Yi-Y)2

SSE=(Yi-Yi)2

SSR= (Yi-Y)2

_

_

_


11/17

Determining the RegressionLine/Model

Use Excel (or any other popular statisticalsoftware)

Select Tools, Data Analysis, Regression Provide the X range

Provide the Y range

Output the analysis to a new sheet

Manual Calculations

X Y

T

m

r r

S


12/17

Determining the RegressionLine/Model using Excel

SUMMARY OUTPUT

Regression Statistics

Multiple R 0.969534312

R Square 0.939996782

Adjusted R Square 0.936838718

Standard Error 0.1461076

Observations 21

ANOVA

df SS MS F

Regression 1 6.35405596 6.354056 297.6496823

Residual 19 0.405601183 0.021347

Total 20 6.759657143

Coefficients Standard Error t Stat P-value

Intercept -2.534985905 0.295223266 -8.58667 5.7673E-08

X Variable 1 0.060727986 0.003519947 17.25253 4.58812E-13


13/17

Determining the RegressionLine/Model Manual Calculations

SST= (Yi-Y)2SSE=(Yi-Yi)

2 SSR= (Yi-Y)2

__

SSx =(Xi- X )2_

SSy =(Yi- Y)2

_

SSxy =(Xi- X )(Xi- Y )_ _

b1=SSxy/SSx

b0 = Y b1X_ _

MSE = SSE / dfMSR = SSR / df

R2 = SSR/SST

YX

SSES

n-2!

t-test = b1/ Sb1


14/17

Measures of Model Goodness

1. R2 Coefficient ofDetermination

2. F-test > F-crit or p-value less than alpha

3. Standard Error

4. t-test


15/17

Hypothesis testing for

Testing to see if the linear relationshipbetween X and Y is significant at the

population level. t-test

Follow the 5-step process

H0

:

HA:

t-crit, alpha or alpha/2, n-2 df

F


16/17

Standard Error Terms in LinearRegression

Se (standard error of the estimate)A measure of variation around the regression line

If the Se is small

Standard deviation Of the Errors

Sb1 (standard error of the the sampling distribution of b1)

Standard deviation of the slopes

A measure of the variation of the slopes from differentsamplesIf the Sb1 is smallour b1 estimate is probably very accurate

Estimates of

b1

b1

b1


17/17

Linear Regression Example

Petfood, Estimate Sales based on Shelf Space

Two sets of samples, 12 observations each

Perform a Regression Analysis on both sets ofdata

Space Sales

5 1.

5 2.2

5 1.

10 1.10 2.

10 2.

15 2.

15 2.

15 2.

20 2.

20 2.

20 3.1

Space Sales

5 1.

5 1.

5 1.

10 210 2.

10 2.

15 3.5

15 3.2

15 3.3

20 4.2

20 4.

20 4.5

Sample1 Sample2

Linear Regression Handout

Documents

Transcript of Linear Regression Handout