ECE 450 - Lecture #9 Part 2 Overviedvanalp/ECE 450/ECE 450 Lectures/ece_450... · 2016. 3. 15. ·...
Transcript of ECE 450 - Lecture #9 Part 2 Overviedvanalp/ECE 450/ECE 450 Lectures/ece_450... · 2016. 3. 15. ·...
1
ECE 450 - Lecture #9 Part 2 Overview
• Bivariate Moments
– Mean or Expected Value of Z = g(X, Y)
– Correlation and Covariance of 2 RV’s
• Functions of 2 RV’s: Z = g(X, Y); finding fZ(z)
– Method 1: First find F(z), by definition; then differentiate to
find f(z);
– Method 2: Method of Auxiliary Variables
2
Expected Value of Z = g(X, Y)
For continuous RV’s: E(Z) = E[g(X, Y)]
For discrete RV’s:
E(Z) = E[g(X, Y)]
dydx)y,x(f)y,x(g XY
dz)z(fz Z
(often hard
to find)
)yYxXPr()y,x(g k,jk
kjj
3
Property/Example: Expected Value
• Say Z = X + Y
• E(Z) = E(X + Y) =
• Note: Expectation is linear!
• Similarly, E(aX + bY + c)= ____________________
dydx)y,x(f)yx( XY
dydx)y,x(fydydx)y,x(fx XYXY
)Y(E)X(Edy)y(fydx)x(fx YX
4
Correlation of RV’s
• Definition: The correlation of RV’s X and Y is: E(______).
• Calculation: E(XY) =
• Concept: Correlation is a measure of similarity –
– (Large) positive correlation X, Y tend to be both negative or both positive, on the average
– (Large, in abs. value) negative correlation X, Y tend to be opposite in sign, on the average
• Note E(XY) E(X) E(Y), in general
• Problem with correlation as a measure of similarity: the number may be “large” just due to the fact the RV’s take large values.
• Ex: correlation between gpa’s and entry-level salaries would be larger if we measured salaries in cents rather than dollars.
dydx)y,x(fxy XY
5
Covariance of RV’s
• Definition: The covariance of RV’s X and Y is
cov(X, Y)
• Calculation:
• Property #1: cov(X, Y) = E(____) – E(___) E(___)
– Mean of the product minus the product of the means
– Similar to the expression for variance
– Covariance is a (partially) normalized measure of similarity between RV’s
)}YY()XX{(E
dydx)y,x(f)Yy()Xx()Y,Xcov( XY
6
Proof of Property #1
Cov(X, Y) =
= E(___) - “term2” – “term3” + E(__) E(__)
where terms 2 and 3 are computed on the next page:
dydx)y,x(f)Yy()Xx( XY
dydx)y,x(f)YXYxyXxy( XY
dydx)y,x(fyXdydx)y,x(fxy XYXY
dydx)y,x(fYXdydx)y,x(fYx XYXY
7
Proof of Property #1, continued
So far, cov(X, Y) = E(_____) - “term2” – “term3” + E(____) E(____)
where term2
and where term3 =
cov(X, y)
dydxyxfyXdydxyxfyX XYXY ),(),(
YXdyyfyXdyydxyxfX YXY )(),(
dydxyxfxYdydxyxfYx XYXY ),(),(
XYdxxxfYdxxdyyxfY XXY )(),(
YXXYEYXXYYXXYE )()(
8
More About Covariances
• Var(X Y) = Var(X) + Var(Y) 2 cov(X, Y)
• Definition: 2 RV’s X and Y are uncorrelated if
cov(X, Y) = 0.
• Property: X, Y independent X, Y uncorrelated
(not, in general, conversely)
9
Correlation Coefficient
• Defn: The correlation coefficient between 2 RV’s X and Y
is
r = (often denoted: r)
• Measures the degree to which X and Y are statistically
related in a linear sense.
• Property 1: -1 r 1
• Property 2: If Y = aX + b, where a 0
then r = 1 (r = 1 if a > 0)
YX
)Y,Xcov(
Note: the parameter r (or r) in the pdf for joint Gaussian RV’s
is the correlation coefficient for X and Y.
10
Correlation Intuition
re: Correlation Coefficient, r
• Say we run an experiment in which we measure the outcomes of RV’s
X and Y, and plot the resulting points in the x-y plane:
Case 1: r .9 Case 2: r -.9
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1
(highly correlated in the positive
sense; nearly fall on a line with
positive slope.
(highly correlated in the negative
sense); nearly fall on a line with
negative slope.
11
Correlation Intuition, continued
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
-1 -0.75 -0.5 -0.25 0 0.25 0.5 0.75 1
Case 3: r 0 Case 4: r 0
(little correlation; little dependence
of any kind between X and Y)
-5 -4 -3 -2 -1 0 1 2 3 4 5
-4
-3
-2
-1
0
1
2
3
4
(little correlation; little linear
dependence, but obviously there
exists a strong dependence
between X and Y)
12
Example 1
Givens: E(X) = 0, E(Y) = 2, var(X) = 4, var(Y) = 1, rXY = .4;
W = X + Y; Z = 2X + 3Y
Find
a) The mean of W and the mean of Z;
E(W) = E(X + Y) = _____ + _____ = ___ + ___ = ___
E(Z) = E(2X + 3Y) = 2 _____ + 3 ____ = 2(___) + 3(___) =
6
b) The variance of W and the variance of Z;
var(W) = var(X + Y) = var(X) + var(Y) + 2 cov(X, Y)
(need this)
13
Example 1, continued
Thus, var(W) = var(X + Y) = var(X) + var(Y) + 2 cov(X, Y)
= 4 + 1 + 2(.8) = 6.6
And, var(Z) = var(U + V), where U = 2X, V = 3Y
= var(U) + var(V) + 2 cov(U, V)
var(U) = var(2X) = 4 var(X) = 4(4) = 16;
var(V) = var(3Y) = 9 var(Y) = 9(1) = 9;
cov(U, V) = E(UV) – E(U) E(V)
= E[ (2X) (3Y) ] – 2E(X) 3E(Y)= 6 E(XY)
8.)1)(2(4.r)Y,Xcov()Y,Xcov(
r YXXYYX
XY
(need
this)
14
Example 1, continued
To find E(XY): cov(XY) = E(XY) – E(X) E(Y)
.8 = E(XY) – (0) (2) E(XY) = .8
Thus, cov(U, V) = 6 E(XY) = 6 (.8) = 4.8
Hence, var(Z) = var(U) + var(V) + 2 cov(U, V)
= 16 + 9 + 2(4.8) = 34.6
c) The correlation coefficient, rWZ, of W and Z.
First we will find cov(W, Z).
15
Example 1, continued
Cov(W, Z) = E(WZ) – E(W) E(Z)
E(WZ) = E[(X+Y)(2X + 3Y)] = E[2X2 + 5XY + 3Y2]
= 2 E(X2) + 5 E(XY) + 3 E(Y2)
var(X) = E(X2) – (E(X))2
4 = E(X2) – 0 E(X2) = 4
var(Y) = E(Y2) – (E(Y))2
1 = E(Y2) – (2)2 E(Y2) = 5
Thus, E(WZ) = 2 E(X2) + 5 E(XY) + 3 E(Y2)
= 2 (4) + 5(.8) + 3 (5) = 27
Therefore, Cov(W, Z) = 27 – (2)(6) = 27 – 12 = 15
16
Example 2 (continued from Lecture 8)
• Recall fXY(x, y) = 6 (1 – x – y), on a triangle:
• So far: fX(x) = 3(1 - x)2 0 x < 1
0 else
fY(y) = 3(1 – y)2 0 y < 1
else
• Find (a) cov(X, Y) and (b) rXY.
• (a) Solution: we need E(XY), E(X), and E(Y) since
cov(X, Y) = E(XY) – E(X) E(Y)
17
Example 2, continued
• E(X) =
• E(Y) =
• E(XY) =
=
1
0
2X 4
1dx)x1(x3dx)x(fx
1
0
2y 4
1dy)y1(y3dy)y(fy
dydx)y,x(fxy XY
dydx)yx1(xy61
0 1x
y
y = -x + 1
1
0x
x1
0y
22 dxdy)xyyxxy(6
18
Example 2, continued
• Inner Integral:
• Thus (evaluating the outer integral),
E(XY) =
cov(X, Y) = E(XY) – E(X) E(Y) = (1/20) – (1/4) (1/4)
= (1/20) – (1/16) = -1/80
1
0x
3
201dx)x1(x
x1
0y
322
6
)x1(xdy)xyyxxy(
19
Example 2, continued
b) Find rXY
To get var(X), we need E(X2); to get var(Y), we need E(Y2)
E(X2) =
Thus, var(X) = var(Y) = E(X2) – (E(X))2 =(1/10) – (1/16) =3/80
rXY = = -1/3
)Y(E10
1dx)x1(x3dx)x(fx 21
0
22X
2
YXYXXY
)Y(E)X(E)XY(E)Y,Xcov(r
(need var(X), var(Y)
for denominator)
80/3
801
)Y,Xcov(
YX
20
Functions of 2 RV’s – Method 1
• Let Z = g(X, Y), where X and Y are RV’s
• FZ(z) = Pr(Z z) = Pr(g(X, Y) z)
FZ(z) =
• Then: fZ(z) =
dydxyxf
zR
XY ),(
)(
Defines a subset, say R(z), of the
xy-plane, meeting this condition
)(zFdz
dZ
21
Example
• Let Z = X2 + Y2, and
fXY(x, y) =
• So FZ(z) = Pr(Z z) =
• Polar Coordinates:
2
22
2 2exp
2
1
yx
dydx)y,x(f
zyx
XY22
r2 r dr dq
)z(Ue1ddr2
rexp
2
1r)z(F )2/(z
2
2
2
2
0
z
0rz
2
q
q
eu du
Find fZ(z).
22
Example, continued
• Thus, fz(z) =
=
dz
dzF
dz
dZ )(
)(1 )2/( 2
zUe z
2)2/(z)2/(z
2
1e)z(U)z(e1
22
product
rule
)z(Ue2
1 )2/(z2
2
23
Method 2: Auxiliary Variables(not shown in Cooper & McGillem)
• Given 2 inputs X, Y, and 1 output, Z = g(X, Y);
• Find fZ(z).
• Method of Auxiliary Variable: create a new (auxiliary)
variable, W = h(X, Y) = h(X).
g(x, y)X
Y ZModel of actual
problem
Zg(x, y)X
Y
h(x, y) W(aux.)
Aux. Variable
Model for
same problem
24
Method 2: Auxiliary Variables
- 4-step Approach -
1. Solve the output equations “backwards” for X and Y
2. Find the Jacobian of the transformation:
3. fZW(z, w) =
(analogous to fY(y) = , for transformation of a single RV)
4. fZ(z) =
w,z
XY
)y,x(J
)y,x(f
dw)w,z(fZW (getting rid of the auxiliary
variable)
dx
dy
)x(fX
x-1
25
Method 2, Example 1:
Let Z = X + Y; find fZ(z)
1. Solve the output equations “backwards” for X and Y:
Y = Z – X = _______ (in terms of Z, W)
X = W
g(x, y)X
Y
h(x, y) W = X(aux.)
Z = X + Y
26
Method 2, Example 1, continued:
(Z = X + Y; aux: W = X)
2. Find the Jacobian of the transformation, J(x, y):
J(x, y) =
3. fZW(z, w) =
4. fZ(z) = (**)
)wz,w(f|1|
)wz,w(f
)y,x(J
)y,x(fXY
XY
w,z
XY
101
11
dy
dw
dx
dw
dy
dz
dx
dz
det"old"d
"new"ddet
dw)wz,w(fdw)w,z(f XYZW
27
Method 2, Example 1 – Going Further
• Special Case of Interest: Z = X + Y where X, Y
independent
• Repeating the (**) result from p. 25:
fZ(z) =
• If X and Y are independent, this becomes:
fZ(z) =
d)z(f)(f
yYX
______________
d)z,(fdw)wz,w(f XYXY
28
Example 1, continued
Summary: If Z = X+Y, and if X and Y are independent
RV’s, then
fZ(z) = fX(z) * fY(z)
(This generalizes to the sum of an arbitrary # of
independent RV’s)
Specific Example: If X and Y are both U(0, 1), and if X
and Y are independent, find the pdf of Z = X + Y.
29
Example 1 – Specific Example, continued
Solving by Graphical Convolution:
0 1
fX(x)
x0 1
fY(y)
y
0 2
fZ(z)
z
1(Convolution
details will be
reviewed during
class.)
Check: area = 1
30
Other Things to Recall about Convolution
• Important for Discrete RV’s:
(x) * f(x) = f(x)
(x - a) * f(x) = f(x - a)
• In class application/example:
– Discrete RV X takes values 0 and 5 with equal
probability
– Discrete RV Y takes values 1 and -1 with equal
probability
– Find the pdf of RV Z = X + Y if X and Y are
independent.
Discrete RV X takes values 0 and 5 with equal probability
Discrete RV Y takes values 1 and -1 with equal probability
Find the pdf of RV Z = X + Y
_______________________________________________
_______________________________________________
Sketch of pdf’s:
31
32
Method 2, Example 2: Let Z = XY;
use aux. variable W = X; find fZ(z)
1. Solve the output equations “backwards” for X and Y:
X = W; Y = Z/W
2. Find the Jacobian of the transformation:
J(x, y) =
3. fZW(z, w) =w
)w/z,w(f
)y,x(J
)y,x(f XY
w,z
XY
x01
xy
dy
dw
dx
dw
dy
dz
dx
dz
det"old"d
"new"ddet
4. fZ(z) =