7/27/2019 lectr11a

1/18

1

As we have seen in section 4 conditional probability density

functions are useful to update the information about an

event based on the knowledge about some other relatedevent (refer to example 4.7). In this section, we shall

analyze the situation where the related event happens to be a

random variable that is dependent on the one of interest.

From (4-11), recall that the distribution function ofXgiven

an eventB is

(11-1)

.

)(

))((|)()|(

BP

BxXPBxXPBxFX

PILLAI

11. Conditional Density Functions and

Conditional Expected Values

7/27/2019 lectr11a

2/18

2

Suppose, we let

Substituting (11-2) into (11-1), we get

where we have made use of (7-4). But using (3-28) and (7-7)

we can rewrite (11-3) as

To determine, the limiting case we can let

and in (11-4).

.)( 21 yYyB

(11-3)

(11-2)

,

)()(

),(),(

))((

)(,)()|(

12

12

21

2121

yFyF

yxFyxF

yYyP

yYyxXPyYyxF

YY

XYXY

X

.

)(

),()|(

2

1

2

1

21

y

yY

x y

yXY

X

dvvf

dudvvufyYyxF

(11-4)

),|( yYxFX yy 1

yyy 2

PILLAI

7/27/2019 lectr11a

3/18

3

This gives

and hence in the limit

(To remind about the conditional nature on the left hand

side, we shall use the subscript X| Y(instead ofX) there).

Thus

Differentiating (11-7) with respect tox using (8-7), we get

(11-5)

(11-6).)(

),()|(lim)|(

0 yf

duyufyyYyxFyYxF

Y

x

XY

Xy

X

(11-7)

(11-8)

yyf

yduyuf

dvvf

dudvvufyyYyxF

Y

x

XY

yy

yY

x yy

yXY

X

)(

),(

)(

),()|(

.

)(

),()|(

|

yf

duyufyYxF

Y

x

XY

YX

.)(

),()|(|

yf

yxfyYxf

Y

XYYX

PILLAI

7/27/2019 lectr11a

4/18

4

It is easy to see that the left side of (11-8) represents a valid

probability density function. In fact

and

where we have made use of (7-14). From (11-9) - (11-10),(11-8) indeed represents a valid p.d.f, and we shall refer to it

as the conditional p.d.f of the r.vXgiven Y=y. We may also

write

From (11-8) and (11-11), we have

(11-9)|

( , )( | ) 0

( )

XY

X Y

Y

f x yf x Y y

f y

,1)(

)(

)(

),()|(

|

yf

yf

yf

dxyxfdxyYxf

Y

Y

Y

XY

YX(11-10)

(11-11)

,)(

),()|(

| yf

yxfyxf

Y

XY

YX

(11-12)

PILLAI

).|()|( || yxfyYxf YXYX

7/27/2019 lectr11a

5/18

5

and similarly

If the r.vsXand Yare independent, thenand (11-12) - (11-13) reduces to

implying that the conditional p.d.fs coincide with their

unconditional p.d.fs. This makes sense, since ifXand Yare

independent r.vs, information about Yshouldnt be of any

help in updating our knowledge aboutX.In the case of discrete-type r.vs, (11-12) reduces to

)()(),( yfxfyxf YXXY

(11-13)

(11-14)

(11-15)

.)(

),()|(|

xf

yxfxyf

X

XYXY

),()|(),()|( || yfxyfxfyxf YXYXYX

.)(

),(|

j

ji

jiyYP

yYxXPyYxXP

PILLAI

7/27/2019 lectr11a

6/18

6

Next we shall illustrate the method of obtaining conditional

p.d.fs through an example.

Example 11.1: Given

determine and

Solution: The joint p.d.f is given to be a constant in theshaded region. This gives

Similarly

and

(11-16)

,otherwise,0

,10,),(

yxkyxfXY

.212

),(1

0

1

0

0 k

kdyykdydxkdxdyyxf

y

XY

)|(| yxf YX ).|(| xyf XY

x

y

1

1

Fig. 11.1

,10),1(),()(1

xxkdykdyyxfxf xXYX (11-17)

.10,),()(

0 yykdxkdxyxfyf

y

XYY(11-18)

PILLAI

7/27/2019 lectr11a

7/18

7

From (11-16) - (11-18), we get

and

We can use (11-12) - (11-13) to derive an important result.

From there, we also have

or

But

and using (11-23) in (11-22), we get

(11-19)

(11-20)

(11-21)

,10,1

)(

),()|(| yx

yyf

yxfyxf

Y

XYYX

.10,1

1

)(

),()|(|

yx

xxf

yxfxyf

X

XYXY

)()|()()|(),( || xfxyfyfyxfyxf XXYYYXXY

.)(

)()|()|( ||

xf

yfyxfxyf

X

YYXXY (11-22)

|

)()|(),()( dyyfyxfdyyxfxf YYXXYX (11-23)

PILLAI

7/27/2019 lectr11a

8/18

8

Equation (11-24) represents the p.d.f version of Bayes

theorem. To appreciate the full significance of (11-24), one

need to look at communication problems where

observations can be used to update our knowledge about

unknown parameters. We shall illustrate this using a simpleexample.

Example 11.2: An unknown random phase is uniformly

distributed in the interval and where

n Determine

Solution: Initially almost nothing about the r.v is known,

so that we assume its a-priori p.d.f to be uniform in the

interval

.)()|(

)()|()|(

|

|

dyyfyxf

yfyxfxyf

YYX

YYX

YX (24)

),2,0( ,nr

).,0( 2N ).|( rf

).2,0( PILLAI

7/27/2019 lectr11a

9/18

9

In the equation we can think ofn as the noise

contribution and ras the observation. It is reasonable to

assume that and n are independent. In that case

since it is given that is a constant, behaves

like n. Using (11-24), this gives the a-posteriori p.d.f of

given r to be (see Fig. 11.2 (b))

where

,nr

),()|( 2 Nrf (11-25)

,20,)(

2

1)()|(

)()|()|(

22

22

22

2/)(

2

0

2/)(

2/)(

2

0

r

r

r

er

de

e

dfrf

frfrf

.2

)(2

0

2/)(22

der

r

(11-26)

nr

PILLAI

7/27/2019 lectr11a

10/18

10

Notice that the knowledge about the observation ris

reflected in the a-posteriori p.d.f of in Fig. 11.2 (b). It is

no longer flat as the a-priori p.d.f in Fig. 11.2 (a), and it

shows higher probabilities in the neighborhood of .r

)|(| rf r

(b) a-posteriori p.d.f of

r

Fig. 11.2

Conditional Mean:

We can use the conditional p.d.fs to define the conditional

mean. More generally, applying (6-13) to conditional p.d.fs

we get

)(f

(a) a-priori p.d.f of

2

1

2

PILLAI

7/27/2019 lectr11a

11/18

11

( ) | ( ) ( | ) .XE g X B g x f x B dx

(11-27)

and using a limiting argument as in (11-2) - (11-8), we get

to be the conditional mean ofXgiven Y=y. Notice

that will be a function ofy. Also

In a similar manner, the conditional variance ofXgiven Y

=y is given by

we shall illustrate these calculations through an example.

|| )|(| dxyxfxyYXE YXYX(11-28)

)|( yYXE

.)|(|

||

dyxyfyxXYE XYXY (11-29)

.|)(

)|(|)|(

2

|

222

|

yYXE

yYXEyYXEYXVar

YX

YX

(11-30)

PILLAI

7/27/2019 lectr11a

12/18

12

Example 11.3: Let

Determine andSolution: As Fig. 11.3 shows,

in the shaded area, and zero elsewhere.

From there

and

This gives

and

.otherwise,0

,1||0,1),(

xyyxfXY (11-31)

)|( YXE ).|( XYE

,10,2),()(

xxdyyxfxfx

xXYX

1

| |( ) 1 1 | |, | | 1,Y

yf y dx y y

,1||0,||1

1

)(

),()|(|

xy

yyf

yxfyxf

Y

XYYX

.1||0,

2

1

)(

),()|(| xy

xxf

yxfxyf

X

XYXY

(11-32)

(11-33)

x

y

1

Fig. 11.3

1),( yxfXY

PILLAI

7/27/2019 lectr11a

13/18

13

Hence

It is possible to obtain an interesting generalization of the

conditional mean formulas in (11-28) - (11-29). More

generally, (11-28) gives

But

.1||,2

||1

|)|1(2

||1

2|)|1(

1

|)|1()|()|(

21

||

2

1

|||

yy

y

yx

y

dxy

xdxyxfxYXE

y

yYX

.10,022

1

2)|()|(

2

|

xy

xdy

x

ydyxyyfXYE

x

x

x

xXY

(11-34)

(11-35)

|

( )|

( ) ( ) ( ) ( ) ( , )

( ) ( , ) ( ) ( | ) ( )

( ) | ( )

X XY

XY X Y Y

E g X Y y

Y

E g X g x f x dx g x f x y dydx

g x f x y dxdy g x f x y dx f y dy

E g X Y y f y dy E E

( ) | .g X Y y

|

( ) | ( ) ( | ) .X YE g X Y y g x f x y dx

(11-36)

11-37 PILLAI

7/27/2019 lectr11a

14/18

14

Obviously, in the right side of (11-37), the inner

expectation is with respect toXand the outer expectation is

with respect to Y. Letting g(X) = Xin (11-37) we get the

interesting identity

where the inner expectation on the right side is with respect

to X and the outer one is with respect to Y. Similarly, we

have

Using (11-37) and (11-30), we also obtain

,)|()( yYXEEXE

.)|()( xXYEEYE

(11-38)

(11-39)

.)|()( yYXVarEXVar (11-40)

PILLAI

7/27/2019 lectr11a

15/18

15

Conditional mean turns out to be an important concept in

estimation and prediction theory. For example given an

observation about a r.vX, what can we say about a related

r.v Y? In other words what is the best predicted value ofYgiven that X=x ? It turns out that if best is meant in the

sense of minimizing the mean square error between Yand

its estimate , then the conditional mean ofYgivenX=x,

i.e., is the best estimate forY(see Lecture 16

for more on Mean Square Estimation).

We conclude this lecture with yet another application

of the conditional density formulation.Example 11.4 : Poisson sum of Bernoulli random variables

Let represent independent, identically

distributed Bernoulli random variables with

Y

)|( xXYE

3,2,1,, iXi

qpXPpXP ii 1)0(,)1(

7/27/2019 lectr11a

16/18

16

andNa Poisson random variable with parameter that is

independent of all . Consider the random variables

Show that YandZare independent Poisson random variables.

Solution : To determine the joint probability mass functionofYandZ, consider

.,1 YNZXY

N

ii

(11-41)

iX

PILLAI

nm

i i

N

ii

nmNPmXP

nmNPnmNmXP

nmNPnmNmYP

nmNmYPnYNmYPnZmYP

1

1

)()(

)()(

)()(

),(),(),(

(11-42)

7/27/2019 lectr11a

17/18

17

)),(~( oftindependenareandthatNote1

NsXpnmBX i

nm

ii

(11-43)

( )!

! ! ( )!

m nm nm n p q e

m n m n

!

)(

!

)(

n

qe

m

pe

nq

mp

).()( nZPmYP

PILLAI

Thus

and YandZare independent random variables.

Thus if a bird lays eggs that follow a Poisson random

variable with parameter , and if each egg survives

)(~)(~ and qPZpPY (11-44)

7/27/2019 lectr11a

18/18

18

with probabilityp, then the number of chicks that survive

also forms a Poisson random variable with parameter .p

PILLAI