8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 1/23
Correlation and Regression Analysis
• Many engineering design and analysis problems involve factors that are
interrelated and dependent. E.g., (1) runoff volume, rainfall; (2) evaporation,temperature, ind speed; (!) pea" discharge, drainage area, rainfall intensity;
(#) crop yield, irrigated ater, fertili$er.
• %ue to inherent comple&ity of system behaviors and lac" of full understanding
of the procedure involved, the relationship among the various relevant factors
or variables are established empirically or semi'empirically.
• egression analysis is a useful and idely used statistical tool dealing ith
investigation of the relationship beteen to or more variables related in a
non'deterministic fashion.
• f a variable * is related to several variables + 1, +2, , +- and their
relationships can be e&pressed, in general, as
* g(+1, +2, , +- )
here g(.) general e&pression for a function;
* %ependent (or response) variable;
+1, +2,, +- ndependent (or e&planatory) variables.
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 2/23
Correlation• /hen a problem involves to dependent random variables, the degree of
linear dependence beteen the to can be measured by the correlation
coefficient ρ(+,*), hich is defined as
here 0ov(+,*) is the covariance beteen random variables X and Y defined
as
here 0ov(+,*) and ≤ ρ(+,*) ≤ .
• arious correlation coefficients are developed in statistics for measuring the
degree of association beteen random variables. 3he one defined above is
called the Pearson product moment correlation coefficient or correlation
coefficient.
• f the to random variables X and Y are independent, then ρ(+,*)
0ov(+,*) . 4oever, the reverse statement is not necessarily true.
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 3/23
Cases of Correlation
5erfectly linearly
correlated in opposite
direction
6trongly 7 positively
correlated in
linear fashion
5erfectly correlated innonlinear fashion, but
uncorrelated linearly.
8ncorrelated in
linear fashion
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 4/23
Calculation of Correlation Coefficient
• 9iven a set of n paired sample observations of to random variables
(&i, yi), the sample correlation coefficient ( r) can be calculated as
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 5/23
Auto-correlation
• 0onsider folloing daily stream flos (in 1::: m!) in une 2::1 at 0hung Mei
8pper 6tation (<1: ha) located upstream of a river feeding to 5lover 0oveeservoir. %etermine its 1'day auto'correlation coefficient, i.e., ρ(=t, =t>1).
• 2? pairs@ A(=t, =t>1)B A(=1, =2), (=2, =!), , (=2?, =!:)B;
elevant sample statistics@ n2?
3he 1'day auto'correlation is :.#!?
Day (t) Flow Q(t) Day (t) Flow Q(t) Day (t) Flow Q(t)
1 8.35 11 313.89 21 20.06
2 6.78 12 480.88 22 17.52
3 6.32 13 151.28 23 116.13
4 17.36 14 83.92 24 68.255 191.62 15 44.58 25 280.22
6 82.33 16 36.58 26 347.53
7 524.45 17 33.65 27 771.30
8 196.77 18 26.39 28 124.20
9 785.09 19 22.98 29 58.00
10 562.05 20 21.92 30 44.08
111C<.22; 2!:.:<; 1CD.#E; 22?.1Dt t t Q t QQ S Q S
++= = = =
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 6/23
Chung Mei Upper Daily Flow
10 20 30
0
100
200
300
400
500
600
700
800
Day
F l o w ( 1 0 0 0 c u b i c m e t e r s )
1 2 3 4 5
1.0
0.8
0.6
0.4
0.2
0.0
0.20.4
0.6
0.8
1.0
F u t o c o r r e l a t i o n
Futocorrelation for une 2::1 %aily Glos at 0hung Mei 8pper, 4-
3ime lags (%ays)
:
1::
2::
!::
#::
::
<::
D::
C::
?::
: 2:: #:: <:: C:: 1:::
=(t), 1::: mH!
= ( t > 1 ) , 1 : : : m H !
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 7/23
Regression Models
• due to the presence of uncertainties a deterministic functional
relationship generally is not very appropriate or realistic.
• 3he deterministic model form can be modified to account for
uncertainties in the model as
* g(+1, +2, , +- ) > ε
here ε model error term ith E(ε):, ar(ε)σ2
.
• n engineering applications, functional forms commonly used for
establishing empirical relationships are
I Fdditive@ * β: > β1+1 > β2+2 > > β- +- >ε
I Multiplicative@ +ε.- 21 J
-
J
2
J
1: +...++J* =
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 8/23
Least Square Method
6uppose that there are n pairs of data, A(& i, yi)B, i1, 2,.. , n and a plot of
these data appears as
/hat is a plausible mathematical model describing & 7 y relationK
&
y
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 9/23
Least Square Method
0onsidering an arbitrary straight line, y β:>β1 &, is to be fitted through thesedata points. 3he Luestion is /hich line is the most representativeNK
1 β1
β:
&i
&
yi
yiH
y β:>β1 &H
ei yi I yi error (residual)H
y
1 β1
β:
&i
&
yi
yiHyiH
y β:>β1 &Hy β:>β1 &Hy β:>β1 &H
ei yi I yi error (residual)Hei yi I yi error (residual)H
y
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 10/23
Least Square Criterion
• /hat are the values of β: and β1 such that the resulting line bestN fits
the data pointsK
• Out, ait PPP /hat goodness'of'fit criterion to use to determine among
all possible combinations of β: and β1 K
• 3he least sLuares (Q6) criterion states that the sum of the sLuares of
errors (or residuals, deviations) is minimum. Mathematically, the Q6
criterion can be ritten as@
• Fny other criteria that can be usedK
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 11/23
Noral !quations for LS Criterion
• 3he necessary conditions for the minimum values of % are@
and
• E&panding the above eLuations
• Rormal eLuations@
::
=∂∂β D :
1
=∂∂β D
( )[ ]( )
( )[ ]( )
=−+−=∂∂
=−+−=∂∂
⇒
∑
∑
=
=
n
i
iii
n
i
ii
x x y D
x y D
1
1:
1
1
1:
:
:2
:12
β β β
β β β
[ ]
[ ]
=−−
=−−⇒
∑
∑
=10
=10
n
i
iii
n
i
ii
x y x
x y
1
1
:
:
β β
β β
=−−
=−−
=
1
=
0
=
=
10
=
:
1
2
11
11
n
i
i
n
i
i
n
i
ii
n
i
i
n
i
in
=
+
=
+
⇒
∑∑∑
∑∑
=1
=0
=
=1
=0
n
i
ii
n
i
i
n
i
i
n
i
i
n
i
i
y x x x
y xn
11
2
1
11
β β
β β
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 12/23
LS Solution "# Un$nowns%
−
−=
−
−=
−=
−
=
∑
∑
∑∑
∑∑∑
∑∑
=
=
==
===
1
11==
0
2
1
2
1
2
11
2
111
11
1
1
S
SS
n
n
n
n
nn
n
i
i
n
i
ii
n
i
i
n
i
i
n
i
i
n
i
i
n
i
ii
n
i
i
n
i
i
β
β β β
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 13/23
Fitting a &olynoial !q' (y LS Method
ni x x x y i
k
ik iii ,,2,1,2
2 ⋅⋅⋅=++⋅⋅⋅+++=
10 ε β β β β
Q6 criterion@
minimi$e % ( )[ ]∑=
210 +⋅⋅⋅+++−n
i
k
iiii x x x y1
22
κ β β β β
κ β β ,, ⋅⋅⋅0
6et k j for D
j,,2,1,:,: ⋅⋅⋅==∂
∂β
⇒ Rormal ELuations are@
=
+⋅⋅⋅+
+
=
=
+⋅⋅⋅+
+
=
+⋅⋅⋅+
+
∑∑∑∑
∑∑∑∑
∑∑∑
===
+1
=0
==
+
=1
=0
===10
n
i
k
ii
n
i
k
i
n
i
k
i
n
i
k
i
n
i
ii
n
i
k
i
n
i
i
n
i
i
n
i
i
n
i
k
i
n
i
i
x y x x x
x y x x x
y x xn
11
2
1
1
1
11
1
1
2
1
111
κ
κ
κ
β β β
β β β
β β β
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 14/23
Fitting a Linear Function of Se)eral *aria+les
ε β β β β κ +++++= 210 k x x x y 21
Rormal eLuations@
=
+⋅⋅⋅+
+
=
=
+⋅⋅⋅+
+
=
+⋅⋅⋅+
+
∑∑∑∑
∑∑∑∑
∑∑∑
===1
=0
===1
=0
===10
n
i
ik i
n
i
ik
n
i
iik
n
i
ik
n
i
ii
n
i
ik i
n
i
i
n
i
i
n
i
i
n
i
ik
n
i
i
x y x x x x
x y x x x x
y x xn
11
2
1
1
1
1
1
1
1
1
2
1
1
111
1
κ
κ
κ
β β β
β β β
β β β
Q6 criterion @
Minimi$e % ( )2
1
1
n
i i k
i
y x x xκ β β β β 0 1 2=
− + + + ⋅ ⋅ ⋅ + ∑
( )k β β β β ,,, 1 0=
6et : , :,1, 2, , j
D for j k β ∂ = = ⋅ ⋅ ⋅∂
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 15/23
Matri, For of Multiple Regression +y LS
+
=
1
0
nk nk nn
k
k
n x x x
x x x
x x x
y
y
y
ε
ε
ε
β
β
β
2
1
21
22221
11211
2
1
1
1
1
(Rote@ ij x ith
observation of the Tth independent variable)
or y > in short
Q6 criterion is@
min( ) ( )J+'yUJ+'yVVU
1
2
∑====
n
ii
D ε
.
6et /. =
∂∂
D , and result in@ /.-y-
0
)'(U =
3he Q6 solutions are@ ( ) y11 .2−=⇒ S
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 16/23
Measure of 3oodness-of-Fit
2
0oefficient of %etermination
( )∑
=
−
∑=−=
n
1i
2y
i
y
n
1i
2i
V
1
1 ' W of variation in the dependent variable, y, une&plained by
the regression eLuation;
W of variation in the dependent variable, y, e&plained by theregression eLuation.
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 17/23
!,aple 2 "LS Method%
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 18/23
!,aple 2 "LS Method%
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 19/23
LS !,aple
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 20/23
LS !,aple "Matri, Approach%
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 21/23
LS !,aple "+y Minita+ w4 /%
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 22/23
LS !,aple "+y Minita+ w4o /%
8/16/2019 Correlation & Regression (Students)
http://slidepdf.com/reader/full/correlation-regression-students 23/23
LS !,aple "5utput &lots%
Top Related