Download - Recitation 2006

Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science

6.041/6.431: Probabilistic Systems Analysis (Spring 2006)

Recitation 1 February 9, 2005

1. Problem 1.2, page 52 of text.

Let A and B be two sets.

(a) Show the following two equalities

Ac = (Ac ∩ B) ∪ (Ac ∩ Bc), Bc = (A ∩ Bc) ∪ (Ac ∩ Bc)

(b) Show that(A ∩ B)c = (Ac ∩ B) ∪ (Ac ∩ Bc) ∪ (A ∩ Bc)

(c) Consider rolling a six-sided die. Let A be the set of outcomes where the roll is an odd number. Let B be the set of outcomes where the roll is less than 4. Calculate the sets on both sides of the equality in part (b), and verify that the equality holds.

2. Problem 1.5, page 53 of text.

Out of the students in a class, 60% are geniuses, 70% love chocolate, and 40% fall into both categories. Determine the probability that a randomly selected student is neither a genius nor a chocolate lover.

3. Example 1.5, page 13 of text.

Romeo and Juliet have a date at a given time, and each will arrive at the meeting place with a delay between 0 and 1 hour, with all pairs of delays being equally likely. The first to arrive will wait for 15 minutes and will leave if the other has not yet arrived. What is the probability that they will meet?

of 1



Recitation 1 Solutions February 9, 2005

1. Problem 1.2, page 52 of text. See online solutions.


3. Example 1.5, page 13 of text. See solutions in text.

Page 1 of ??




1. A coin is tossed twice. Alice claims that the event of two heads is at least as likely if we know that the first toss is a head than if we know that at least one of the tosses is a head. Is she right? Does it make a difference if the coin is fair or unfair? How can we generalize Alice’s reasoning?

2. We are given three coins: one has head on both faces, the second has tails on both faces, and the third has a head on one face and a tail on the other. We choose a coin at random, toss it, and it comes up heads. What is the probability that the opposite face is tails?

3. Fischer and Spassky play a sudden death chess match. Each game ends up with either a win by Fischer, this happens with probability p, a win for Spassky, this happens with probability q, or a draw, this happens with probability 1 − p − q. The match continues until one of the players wins a game (and the match).

(a) What is the probability that Fischer will win the last game of the match?

(b) Given that the match lasted no more than 5 games, what is the probability that Fischer won in the first game?

(c) Given that the match lasted no more than 5 games, what is the probability that Fischer won the match?

(d) Given that Fischer won the match, what is the probability that he won at or before the 5th game?

Page 1 of 1



Recitation 2: Solutions February 14, 2006



3. (a)

P(Fischer wins) = p + p(1 − p − q) + p(1 − p − q)2 + · · · p

= 1 − (1 − p − q)

p= p+q

We may also find the solution through a simpler method:

P(Fischer wins) P(Fischer wins | Someone wins) =

P(Someone wins) p= p+q

Fischer wins

Spassky wins

p

q

1-p-q Draw p

q

1-p-q

Fischer wins

Spassky wins

Draw

p q

1-p-q

Fischer wins

Spassky wins

Draw p

q

1-p-q

Fischer wins

Spassky wins

Draw

p

q

1-p-q

Fischer wins

Spassky wins

. . .

(b) P(the match lasted no more than 5 games) = (p+ q)+(p+ q)(1 −p− q)+(p+ q)(1 −p− q)2 +(p+ q)(1 −p− q)3 +(p+ q)(1 −p− q)4

= (p+q)[1−(1−p−q)5]1−(1−p−q)

= 1 − (1 − p − q)5

P(Fischer wins in the first game ∩ the match lasted no more than 5 games)= p

Therefore, P(Fischer wins | the match lasted no more than 5 games)P(Fischer wins ∩ the match lasted no more than 5 games)=

P(the match lasted no more than 5 games) p

1−(1−p−q)5=

Page 1 of 2

� � � �



(c) P(the match lasted no more than 5 games) = 1 − (1 − p − q)5

P(Fischer wins ∩ the match lasted no more than 5 games) = p + p(1 − p − q) + p(1 − p − q)2 + p(1 − p − q)3 + p(1 − p − q)4

= p[1−(1−p−q)5]1−(1−p−q)

= p[1−(1−p−q)5]p+q

Therefore, P(Fischer wins | the match lasted no more than 5 games) P(Fischer wins ∩ the match lasted no more than 5 games)=

P(the match lasted no more than 5 games) p= p+q

(d) P(Fischer wins at or before the 5th game | Fischer wins) P(Fischer wins at or before the 5th game ∩ Fischer wins)=

P(Fischer wins) = p[1−(1−p−q)5] p

p+q / p+q

= 1 − (1 − p − q)5

This part may be solved by observing that the events {Fischer wins} and {the matchlasted no more than 5 games} are independent (we know this from parts (a) and (c)):P(the match lasted no more than 5 games | Fischer wins)= P(the match lasted no more than 5 games)= 1 − (1 − p − q)5

Page 2 of 2




1. A particular class has had a history of low attendance. The annoyed professor decides that she will not lecture unless at least k of the n students enrolled in the class are present. Each student will independently show up with probability pg if the weather is good, and with probability pb if the weather is bad. Given the probability of bad weather on a given day, calculate the probability that the professor will teach her class on that day.

2. Consider two coins, a blue and a red one. We choose one of the two coins at random, each being chosen with probability 1/2. Let H1 be the event that the first toss results in heads, and H2 be the event that the second toss results in heads. The coins are biased: with the blue coin, the probability of heads in any given toss is 0.99, whereas for the red coin it is 0.01.

(a) Are the events H1 and H2 (unconditionally) independent?

(b) Given that the blue coin was selected, are the events H1 and H2 (conditionally) independent?

3. For each one of the following statements, indicate whether it is true or false, and provide a brief explanation.

(a) If P (A | B) = P (A), then P (B | Ac) = P (B).

(b) If 5 out 10 independent fair coin tosses resulted in tails, the events “first toss was tails” and “10th toss was tails” are independent.

(c) If 10 out 10 independent fair coin tosses resulted in tails, the events “first toss was tails” and “10th toss was tails” are independent.

(d) If the events A1, . . . , An form a partition of the sample space, and if B, C are some other events, then

P (B | C) = P (Ai | C)P (B | Ai). n �

i=1

Page 1 of 1




1. Problem 1.35, page 61. See online solutions.

2. Example 1.21, page 37 of text. See solutions in text.

3. (a) True If P (A | B) = P (A), then A and B are independent. And if B is independent of A, then B is also independent of Ac. This implies, by the definition of independence:

False

P (B | Ac) = P (B)

(b) Since there are only 5 tails out of ten, knowledge of one coin toss provides knowledge about the other coin tosses, which means the two events are not independent. In other words, the knowledge that the first coin toss was a tails influences the probability that the tenth coin toss is a tails.

(c) True Here, all tosses are tails, so knowledge of one coin toss provides no additional knowledge about the tenth coin toss. Therefore the two events are independent.

(d) Ai

False On the left hand side of the expression, since ’s are disjoint,

P (B ∩ C)P (B | C) =

P (C)

=

=

n �

i=1 n �

i=1

P (Ai)P (B ∩ C | Ai) P (C)

P (Ai ∩ B ∩ C) P (C)

However, the right hand side of the given expression shows, n �

i=1

n �

i=1 n �

i=1

P (Ai ∩ C) P (B ∩ Ai) P (C) P (Ai)

P (Ai ∩ B ∩ C) P (C)P (Ai)

P (Ai | C)P (B | Ai) =

=

where the last line is ONLY TRUE if the events Ai ∩ C and B ∩ Ai are independent ofeach other.Note also for the expression to be true, i = 1 and A1 has to be the entire sample space,i.e. P (A1) = 1. Therefore, the given expression only holds if Ai ∩ C and B ∩ Ai are independent and i = 1.

Page 1 of 1

Massachusetts Institute of TechnologyDepartment of Electrical Engineering & Computer Science

6.041/6.431: Probabilistic Systems Analysis(Spring 2006)

Recitation 4February 23, 2006

1. The birthday problem. Problem 1.45, page 66 in the text.

Consider n people who are attending a party. What is the probability that each person hasa distinct birthday?

Assume that each person has an equal probability of being born on each day during the year,independently of everyone else, and ignore the additional complication presented by leap years(i.e., nobody is born on February 29).

2. Recall from Lecture 4 the different cases that arise from the problem of selecting/sampling k

balls from an urn containing n numbered balls, numbered 1 through n:

• Sampling with replacement and ordering

• Sampling without replacement and ordering

• Sampling without replacement and without ordering

• Sampling with replacement and without ordering

The objective of this problem is to study the fourth case. A distinct solution may be expressedin terms of the vector of nonnegative integers (N1, N2, . . . , Nn), where Ni is the number oftimes the ball numbered i gets selected.

(a) Explain why we must have N1 + N2 + · · · + Nn = k.

(b) How many distinct solutions does the equation above have? Explain why this is answerto the number of distinct results of sampling with replacement, without ordering.

(c) Let X1 denote the number of balls selected that are numbered 1. For any ℓ ∈ {0, 1, . . . , k},find the number of distinct samplings with replacement, without ordering such thatX1 = ℓ. Use this to state an identity for binomial coefficients.

3. 4 buses carrying 148 job-seeking MIT students arrive at a job fair. The buses carry, respec-tively, 40, 33, 25, and 50 students. One of the students is randomly selected. Let X denotethe number of students that were on the bus carrying this randomly selected student. Also,one of the 4 bus drivers is also randomly selected. Let Y denote the number of students onhis bus.

(a) Do you think E[X] and E[Y ] are equal? If not, which is larger? Give your reasoninginformally.

(b) Compute E[X] and E[Y ].

Page 1 of 1

�

�




1. Problem 1.45, page 66. See online solutions.

2. (a) The Nis are the numbers of times each ball is selected, so the sum of the Nis must be the total number of draws from the urn.

(b) There is a nice visualization for this. Make a dot for each drawn ball, grouped according the ball’s identity:

· · · · · · · · · �� ··· NnN1 N2

There is a total of k dots put in n groups. Think of there being a separator mark between groups, so there are n − 1 separator marks:

· · · | · · · | | · · · �� ··· NnN1 N2

This gives a grand total of k + n − 1 dots and marks. The number of solutions is the � k + n − 1number of ways to place k dots in k + n − 1 locations: .

k

(c) If we know that X1 = ℓ, then applying the result of the previous part to the remaining

balls and remaining draws from the urn gives �

(k − ℓ) + (n − 1) − 1 as the desired number. k − ℓ

Since this is just a way of breaking down the problem of the previous part, we have

k � � k + n − ℓ − 2

� �� k + n − 1 = .

k − ℓ k

ℓ=0

3. (a) Students might say they are equal (both being the average number of students per bus) or have the correct intuition.

(b) Make sure to define the PMFs of X and Y . Then

40 33 25 50 E[X] = · 40 + · 33 + · 25 + · 50 ≈ 39.3

148 148 148 148 1 1 1 1

E[Y ] = · 40 + · 33 + · 25 + · 50 = 37 4 4 4 4

Page 1 of 1




Review/discussion: The following five types of discrete random variables arise frequently in applications and in the

remainder of the course. Their properties are tabulated on pages 116–117 of the text. Make sure you understand how these random variables arise and how to derive their means and variances.

• Discrete uniform over [a, b] (or uniform over {a, a + 1, . . . , b})

• Bernoulli with parameter p

• Binomial with parameters p and n

• Geometric with parameter p

• Poisson with parameter λ

Problems:

1. Problem 2.22, page 123 in the text. Two coins are simultaneously tossed until one of them comes up a head and the other a tail. The first coin comes up a head with probability p and the second with probability q. All tosses are assumed independent.

(a) Find the PMF, the expected value, and the variance of the number of tosses.

(b) What is the probability that the last toss of the first coin is a head?

2. Prove the following version of the Total Expectation Theorem:

E[X] =n �

i=1

P(Ai)E[X | Ai]

whenever A1, A2, . . . , An is a partition of the sample space.

3. Suppose a discrete random variable X can have only non-negative integer values. Show that

∞

E[X] =� k=0

P(X > k).

Page 1 of 1

�

� �



Recitation 5 SolutionsFebruary 28, 2006

1. Problem 2.22, page 123 in the text. See online solutions.

2. Total expectation follows easily from total probability. This could be a good time to point out that the Total Probability Theorem and Total Expectation Theorem each have versions phrased with (a) conditioning on events forming a partition; and (b) conditioning on a discrete random variable. These are equivalent because the collection of events {Y = y} over all y is a partition. You could also point out that technically, when we write

E[X] = pY (y)E[X | Y = y] y

we better only include in the summation y such that P(Y = y) > 0.

3. The result follows by rewriting the expectation summation in the following manner:

∞ ∞ �

k �

∞ ∞ � � � � � E[X] = kpX (k) = 1 pX (k) = pX (k)

k=0 k=1 �=1 �=1 k=� ∞ ∞

= P(X > � − 1) = P(X > n). �=1 n=0

The manipulations could look unmotivated, but if you sketch the k-� plane, then the interchange of summations is clear.

Page 1 of 1



Recitation 6 March 2, 2006

1. Problem 2.32, page 127 in the text. D. Bernoulli’s problem of joint lives. Consider 2m persons forming m couples who live together at a given time. Suppose that at some later time, the probability of each person being alive is p, independently of other persons. At that later time, let A be the number of persons that are alive and let S be the number of couples in which both partners are alive. For any number of total surviving persons a, find E[S|A = a].

2. Problem 2.38, page 132 in the text. Alice passes through four traffic lights on her way to work, and each light is equally likely to be green or red, independently of the others.

(a) What are the PMF, the mean, and the variance of the number of red lights that Alice encounters?

(b) Suppose that each red light delays Alice by exactly two minutes. What is the variance of Alice’s commuting time?

3. Problem 2.40, page 132 in the text. A particular professor is known for his arbitrary grading policies. Each paper receives a grade from the set {A,A−, B+, B,B−, C+}, with equal probability, independently of other papers. How many papers do you expect to hand in before you receive each possible grade at least once?

Page 1 of 1

of 1



Recitation 6 Answers March 2, 2006

1. See online solutions.



�




1. The random variable X is exponentially distributed with parameter λ.

λe−λx x ≥ 0 fX(x) =

0, otherwise

(a) Calculate E[X], var(X) and find P(X ≥ E[X]).

Hint: � ∞

P(x ≥ k) = fX(x)dx k

(b) Find P(X ≥ t + k|X > t).

2. You are allowed to take a certain test three times, and your final score will be the maximum of the test scores. Your score in test i, where i = 1, 2, 3 takes one of the values from i to 10 with equal probability 1/(11− i), independently o f the scores in other tests. What is the PMF of the final score?

3. Wanting to browse the net, Oscar uses his high-speed 300-baud modem to connect through his Internet Service Provider. The modem transmits bits in such a fashion that -1 is sent if a given bit is zero and +1 is sent if a given bit is one. The telephone line has additive zero-mean Gaussian (normal) noise with variance σ2 (so, the receiver on the other end gets a signal which is the sum of the transmitted signal and the channel noise). The value of the noise is assumed to be independent of the encoded signal value.

2

0 1

Modem Oscar’s -1

+1

Noise ~ N(0, )σ

Channel

Receiver/ Decoder Decision (0 or 1?)

Binary Signal

Encoded Signal

PC

We assume that the probability of the modem sending −1 is p and the probability of sending 1 is 1 − p.

(a) Suppose we conclude that an encoded signal of −1 was sent when the value received on the other end of the line is less than a (where −1 < a < +1), and conclude +1 was sent when the value is more than a. What is the probability of making an error?

(b) Answer part (a) assuming that p = 2/5, a = 1/2 and σ2 = 1/4.

Page 1 of ??

{

( ) ( )



Recitation 07 (Answers) March 07, 2006

1 11. (a) E[X] = λ, var(X) = 1 and P(X ≥ E[X]) =

λ2 e

(b)

P(X > t + k|X > t) = e −λ(k)

Note: the exponential random variable is memoryless.

2. We first compute the CDF FX(x) and then obtain the PMF as follows

FX(k) − FX(k − 1) if k = 3, ...10, pX(k) =

0, otherwise.

We have,

0, k < 3,

k k−1 k−2 3 ≤ k ≤ 10,FX(k) = 10 9 8 1 10 ≤ k.

3. (a)

P(error) = P(R1|S0)P(S0) + P(R0|S1)P(S1)

= P(Z − 1 > a)(p) + P(Z + 1 < a)(1 − p) ( ( )) ( ) a − (−1) a − 1

= p · 1 − Φ + (1 − p) · Φ σ σ ( ) ( ( ))

a + 1 1 − a = p − p · Φ + (1 − p) · 1 − Φ

σ σ

a + 1 1 − a = 1 − p · Φ − (1 − p) · Φ

σ σ

3/2 1/2(b) P(error) = 1 − 0.4 · Φ(1/2 ) − 0.6 · Φ(1/2 )

Page 1 of 1

�




1. Random variables X and Y have the joint PDF shown below:

y

x1.0 2.0-1.0

-1.0

1.0

2.0 f (x,y) = 0.1X,Y

-2.0

(a) Prepare neat, fully labeled sketches of fX (x), fY (y), fY |X (y|x) and fX|Y (x|y).

(b) Are X and Y independent?

(c) Find fX,Y |A(x, y), where the event A corresponds to points (x, y ) within the unit circle centered at the origin.

(d) Find E[X|Y = y] and var(X|Y = y).

2. Alexei is vacationing in Monte Carlo. The amount X (in dollars) he takes to the casino each evening is a random variable with a PDF of the form

ax if 0 ≤ x ≤ 40 fX (x) =

0 otherwise

At the end of each night, the amount Y that he has when leaving the casino is uniformly distributed between zero and twice the amount that the came with.

(a) Determine the joint PDF fX,Y (x, y )

(b) What is the probability that on a given night Alexei makes a positive profit at the casino?

(c) Find the PDF of Alexei’s profit Y −X on a particular night, and also determine its expected value.

Page 1 of 1

{



Recitation 08 AnswersMarch 09, 2006

1. (a) The marginal distributions are obtained by integrating the joint distribution along the X and Y axes and is shown in the following figure.

y y0 0

2.0 f

x,y(x

0,y

0) = 0.1

1.0

x 0

-1.0

-2.0

0.2

-1.0 1.0 2.0

0.4

0.2

0.2

f(x)

yf (y)

0.3

x

1.0 2.0-1.0

-1.0

1.0

2.0

-2.0

x

0

Figure 1: Marginal probabilities fX(x) and fY (y) obtained by integration along the y and x axes respectively

The conditional PDFs are as shown in the figure below.

(b) X and Y are NOT indepenent since fXY (x, y) 6= fX(x)fY (y). Also, from the figures we have fX|Y (x|y) 6 fX(x). =

(c)

fX,Y ((x,y) (x, y) ∈ A

fX,Y |A(x, y) = P(A)

0 otherwise

=

{ 0.1 π0.1 (x, y) ∈ A 0 otherwise

(d)

E[X|Y = y] =

0 1 2 0

−2.0 ≤ y ≤ −1.0 −1.0 ≤ y ≤ 1.0 1.0 ≤ y ≤ 2.0

Page 1 of 3



f (x|y) {1 < y <= 2} x|y

f (x|y) x|y

-1.0 2.0

1/3

{-1<y<=1}

f (x|y) x|y

-1.0 1.0

1/2 {-2<y<=2}

f (y|x) y|x

f (y|x) y|x

y 0 y

0

1.0-1.0

1/2

2.0

-2.0

1/4

1.0

-1.0

1/2

y0

x 01.0 2.0-1.0

-1.0

1.0

2.0 f (x ,y ) = 0.1

0x,y 0

-2.0

{-1<x<=1} {1<x<=2}

Figure 2: Conditional Probabilities

Page 2 of 3



The conditional variance var(X|Y = y) is given by

var(X|Y = y) =

4 12 −2.0 ≤ y ≤ −1.0 9 12 −1.0 ≤ y ≤ 1.0 4 12 1.0 ≤ y ≤ 2.0

2. (a) We have a = 1/800, so that

fXY (x, y) =

{ 1/1600 if 0 ≤ x ≤ 40 and 0 ≤ y ≤ 2x 0, otherwise.

}

(b) P(Y > X) = 1/2

(c) Let Z = Y − X. We have

fZ(z) =

1 1600 z + 1

40 , if − 40 ≤ z ≤ 0, − 1

1600 + 1 40 , if 0 ≤ z ≤ 40,

0, otherwise.

E[Z] = 0.

Page 3 of 3

�

�



Recitation 09March 21, 2006

1. Al and Bo are in a race. Denote Al’s and Bo’s elapsed times with random variables X and Y , respectively. These independent random variables are described by the following PDFs:

0.5 1 < x < 3 fX (x) =

0 elsewhere

0.5 2 < y < 4 fY (y) =

0 elsewhere

(a) Determine P (A), the probability that Al wins the race.

(b) Determine the probability that Al wins a total of exactly 7 of the next 10 races. Assume all races are independent. You may use P (A) symbolically in your answer. (As long as your answer is explicit, compact, and fully explained, it need not be simplified.)

(c) Determine, carefully sketch, and label the PDF for W , the elapsed time for the winner of the race. Fully explain each step of your work.

2. Random variables X and Y are independent and have PDFs as shown below.

fX(x) fY(y)

5 4 3 2 1

5 4 3 2 1

0.2 0.4 0.6 0.8 1.0 x 0.2 0.4 0.6 0.8 1.0 y

Let W = X + Y , and find fW (w) using a graphical argument.

3. Alice and Bob flip bias coins independently. Alice’s coin comes up heads with probability 1/4, while Bob’s coin comes up head with probability 3/4. Each stop as soon as they get a head; that is, Alice stops when she gets a head while Bob stops when he gets a head. What is the PMF of the total amount of flips until both stop? (That is, what is the PMF of the combined total amount of flips for both Alice and Bob until they stop?)

Page 1 of 1

( )

∫

∑

∑

( )



Recitation 09 AnswersMarch 21, 2006

1.

(a) P [A] = 78

(b) P [Al wins 7 out of 10 races] = 10 (7 )7(1 )3 7 8 8

1 , 1 < w0 ≤ 2 2 (c) fw(w0) = 7

4 − w2 0 , 2 < w0 ≤ 3 0, otherwise

2. ∞

fW (w) = fX (x)fY (w − x)dx −∞

for w = x + y and x, y independent. This operation is called the convolution of fX (x) and fY (y).

5w, 0 ≤ w ≤ 0.1 0.5, 0.1 ≤ w ≤ 0.9 5(0.1 + (w − 0.9)), 0.9 ≤ w ≤ 1.0

fW (w) = 5(0.1 + (1.1 − w)), 1.0 ≤ w ≤ 1.1 0.5, 1.1 ≤ w ≤ 1.9 5(2.0 − w), 1.9 ≤ w ≤ 2.0 0, otherwise

3. Let X and Y be the number of flips until Alice and Bob stop, respectively. Thus, X + Y is the total number of flips until both stop. The random variables X and Y are independent geometric random variables with parameters 1/4 and 3/4, respectively. By convolution, we have

∞

pX+Y (j) = pX (k)pY (j − k) k=−∞

j−1

= (1/4)(3/4)k−1(3/4)(1/4)j−k−1

k=1

j−11 ∑

= 3k

4j k=1

1 3j − 1 = − 1

4j 3 − 1

3 (3j−1 − 1) = ,

2 4j

Page 1 of 2



if j ≥ 2, and 0 otherwise. (Even though X + Y is not geometric, it roughly behaves like one with parameter 3/4.)

Page 2 of 2

Massachusetts Institute of Technology

Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis

(Spring 2006)

Recitation 10

March 23, 2006

1. Suppose X is uniformly distributed between a and b.

a) Find the transform of X.

b) Use the transform in (a) to find the mean and the variance of X.

2. A three sided die is described by the following probabilities:

1 1 1 P (X = 1) = , P (X = 2) = , P (X = 3) = .

2 4 4

a) Find the transform of the above random variable.

b) Use the transform to find the first three moments, E[X], E[X2], E[X3].

c) Check your answers in (b) by computing the moments directly.

3. Suppose a nonnegative discrete random variable has one of the following two expressions as its transform:

−1) (i) MX(s) = e2(es−1

(ii) MX(s) = e2(es −1)

(a) Explain why one of the two could not possibly be its transform, and indicate which one is the true transform.

(b) Find P (X = 0).

Page 1 of 1

�

�

� �

�

�

�

�

�

� ��

� � � �

= + + �

�

�

�



(Spring 2006)

Recitation 10 Solutions

March 23, 2006

1. a) To find the transform, we integrate the density function over its full domain, against an exponential. This is often expressed as finding the expected value of the function e −rx .

�

b −rx e E[e −rx] =

a b − a −ra −rb e − e

= . r(b − a)

b) To find the mean and the variance we use the moment generating properties of the transform, namely:

d E[Xn] = (−1)n E[e −rx]

�

�

dr r=0

Thus we have:

d E[X] = − E[e −rx]

�

�

dr r=0

1 �

−rb −ra � �

−ra � e − e 1 be −rb − ae = − + �

b − a r2 b − a r �

r=0

b2 − a 2 a 2 − b2 ′(L Hopital) = − −

b − a b − a

b + a = .

2

To find the Variance we need to find E[X2] and thus we need to take the second derivative of the transform and evaluate at r = 0,

d2 E[X2] = E[e −rx]�

dr2 r=0

2 �

−ra −rb ae −ra − be −rb 1 � 2 −ra b2 −rb � e − e 2 a e − e

b − a r3 b − a r2 b − a r �

r=0

b3 b3 a 3 − b3 31 − a 3 − a′(L Hopital) = + +

3 b − a b − a b − a

= 1(b2 + ab + a 2)3

and therefore we have:

V ar[X] = E[X2]− E[X]2 =1(b2 + ab + a 2)−

b + a �2

3 2

2. The transform for nonegative integer valued random variables is defined as:

∞

T px (z) = z xiP (X = xi) = E[z X ]

i=1

Page 1 of 2

�

�

�

�

�

�

�

�

�

�

�

�



(Spring 2006)

and therefore we have: 1 1 3E[z X ] = z +

1 z 2 + z .

2 4 4

b) We observe from above that if we take n derivatives of the transform and evaluate at z = 1 then we will have a linear combination of the first n moments.

d E[z X ]� = E[X]

dz z=1

d2 E[z X ]� = E[X2]−E[X]

dz2 z=1

d3 �

E[z X ]� = E[X3]− 3E[X2] + 2E[X]dz3 z=1

and therefore we find:

E[X] = dE[z X ]�

dz z=1

1 1 3 7 = + + = 2 2 4 4

and similarly,

d2 E[X2] = E[z X ]� + E[X]

dz2 z=1

1 3 7 15 = + + = 2 2 4 4

and finally,

d3 E[X3] = E[z X ]� + 3E[X2]− 2E[X]

dz3 z=1

6 45 14 37 = + + = 4 4 4 4

c) Direct computation thankfully produces the same results.

3. a) Note that by the definition of the transform,

MX(s) = e sx pX(x) x

and therefore when evaluated at s = 0, the transform should equal 1. We see that only the secondoption satisfies this requirement.b) It is observed that the transform is that of a Poisson random variable with parameter λ = 2.Hence the pdf is given as follows:

λk pX(k) = e

−λ

k! pX(0) = e

−2

Page 2 of 2



(Spring 2006)

Recitation 11

April 4, 2006

1. A number p is drawn from the interval [0, 1] according to the uniform distribution, and then a sequence of independent Bernoulli trials is performed, each with success probability p. What is the mean and the variance of the number of successes in k trials?

2. Imagine that the number of people that enter a bar in a period of 15 minutes has a Poisson distribution with rate λ. Each person who comes in buys a drink. If there are N types of drinks, and each person is equally likely to choose any type of drink, independently of what anyone else chooses, find the expected number of different types of drinks the bartender will have to make.

Page 1 of 1



Recitation 11 April 4, 2006

1. k

E[X] = 2 .

k k2

var(X) = +6 12

.

2. E[D] = N − Ne− .

λ N

Page 1 of 1



(Spring 2006)

Recitation 12

April 6, 2006

1. Widgets are packed into cartons which are packed into crates. The weight (in pounds) of a widget, X, is a continuous random variable with PDF

fX(x) = λe −λx , x ≥ 0 .

The number of widgets in any carton, K, is a random variable with PMF

µke−µ pK(k) = , k = 0, 1, 2, . . . .

k!

The number of cartons in a crate, N , is a random variable with PMF

pN (n) = p n−1(1 − p), n = 1, 2, 3, . . . .

Random variables X, K, and N are mutually independent. Determine

(a) The probability that a randomly selected crate contains exactly one widget.

(b) The expected value and variance of the number of widgets in a crate.

(c) The transform or the PDF for the total weight of the widgets in a crate.

(d) The expected value and variance of the total weight of the widgets in a crate.

2. Using a fair three-sided die (construct one, if you dare), we will decide how many times to spin a fair wheel of fortune. The wheel of fortune is calibrated infinitely finely and has numbers between 0 and 1. The die has the numbers 1,2 and 3 on its faces. Whichever number results from our throw of the die, we will spin the wheel of fortune that many times and add the results to obtain random variable Y .

(a) Determine the expected value of Y .

(b) Determine the variance of Y .

Page 1 of 1

{

{



(Spring 2006)

Recitation 12 Solutions

April 5, 2006

1. (a) First note that X should be a r.v., not a number. In particular, we are to minimize over all X that can be expressed as functions of Y . From lecture, ˆr.v.’s ˆ X = E[X|Y ].

Now, take conditional expectations, to get Y = E[Y |Y ] = E[X|Y ] + E[W |Y ]. Since there is complete symmetry between X and W , we also have E[X|Y ] = E[W |Y ], which finally yields E[X|Y ] = Y/2.

(b) In the dependent case, we cannot simply conclude that the distribution fX,W (x,w) is symmetric in its two argument (i.e., fX,W (x,w) = fX,W (w, x)), even though the marginals fX(x), fW (w) are the same. Since fX,W (x,w) is not symmetric, E[X|Y ] 6= E[W |Y ] in general. So in this case, one cannot really solve the problem with the available information, we really need the joint distribution in order to compute the conditional expectations. The solution given in the independent case still works, though, for any symmetric distribution.

2. (a) The minimum mean squared error estimator g(Y ) is known to be g(Y ) = E[X|Y ]. Let us first find fX,Y (x, y). Since Y = X + W , we can write

1 if x− 1 ≤ y ≤ x+ 1 fY |X(y|x) = 2

0 otherwise

and, therefore,

1 if x− 1 ≤ y ≤ x+ 1 and 5 ≤ x ≤ 10 fX,Y (x, y) = fY |X(y|x) · fX(x) = 10

0 otherwise

as shown in the plot below.

o ox , y x,y

f (yo

5

10

) = 1/10

5 10 xo

We now compute E[X|Y ] by first determining fX|Y (x|y). This can be done by looking at the horizontal line crossing the compound PDF. Since fX,Y (x, y) is uniformly distributed in the defined region, fX|Y (x|y) is uniformly distributed as well. Therefore,

5+(y+1) if 4 ≤ y < 6 2

g(y) = E[X|Y = y] = y if 6 ≤ y ≤ 9

10+(y−1) if 9 < y ≤ 11 2

The plot of g(y) is shown here.

Page 1 of 3



(Spring 2006)

4

5

6

7

8

9

10

11

o )

g (y

4 5 6 7 8 9 10 11 yo

(b) The linear least squares estimator has the form

cov(X,Y ) gL(Y ) = E[X] +

σ2 (Y − E[Y ])

Y

where cov(X,Y ) = E[(X − E[X])(Y − E[Y ])]. We compute E[X] = 7.5, E[Y ] = E[X] + E[W ] = 7.5, σ2 = (10 − 5)2/12 = 25/12, σ2 = (1 − (−1))2/12 = 4/12 and, using the fact

X W

that X and W are independent, σ2 = σ2 + σ2 = 29/12. Furthermore, Y X W

cov(X,Y ) = E[(X − E[X])(Y − E[Y ])] = E[(X − E[X])(X − E[X] + W − E[W ])] = E[(X − E[X])(X − E[X])] + E[(X − E[X])(W − E[W ])] = σ2 + E[(X − E[X])]E[(W − E[W ])] = σ2 = 25/12.

X X

Note that we use the fact that (X − E[X]) and (W − E[W ]) are independent and E[(X − E[X])] = 0 = E[(W − E[W ])]. Therefore,

25 gL(Y ) = 7.5 + (Y − 7.5).

29

The linear estimator gL(Y ) is compared with g(Y ) in the following figure. Note that g(Y ) is piecewise linear in this problem.

4

5

6

7

8

9

10

11

Linear

predictor

o L

o )

g (y

)

, g

(y

4 5 6 7 8 9 10 11 yo

Page 2 of 3

∫ ∫



(Spring 2006)

23. The problem asks us to find P (x2 + x2 ≤ α). The information given completely determines the 1

joint density function, so we need only to perform the integration:

2 21 +xx1 22 2 −P (x ≤ α) dx1dx2+ x 2= e1 2

≤α 2π2 2+xx1 2

∫

α∫ 2π 1 r

−

2

2 rdrdθ= e 0 0 2π

= 1 − e −α 2

2

Page 3 of 3



Recitation Solutions April 11, 2006

1. E[AB] = E[(W + X)(X + Y )] = E[WX + X2 + XY + WY ] = E[X2] = var(X) + E[X]2 = 1

E[AC] = E[(W + X)(Y + Z)] = E[WY + XY + XZ + WZ] = 0

2. Solution is in the text, pp. 264–265.


Page 1 of 1

13




1. Suppose four random variables, W , X, Y and Z, are known to be pairwise uncorrelated and to satisfy

E[W ] = E[X] = E[Y ] = E[Z] = 0

andvar(W ) = var(X) = var(Y ) = var(Z) = 1.

Let A = W +X, B = X +Y and C = Y +Z. Compute E[AB] and E[AC], or the correlation between A & B and A & C, respectively.

2. (Problem 4.25) Correlation Coefficient. Consider the correlation coefficient

cov(X, Y )ρ(X, Y ) = �

var(X)var(Y )

of two random variables X and Y that have positive variances. Show that:

(a) |ρ(X, Y )| ≤ 1. Hint : Use the Schwarz inequality:

2(E[XY ])2 ≤ E[X2]E[Y ].

(b) If Y − E[Y ] is a positive multiple of X −E[X], then ρ(X, Y ) = 1.

(c) If Y − E[Y ] is a negative multiple of X − E[X], then ρ(X, Y ) = −1.

(d) If ρ(X, Y ) = 1, then, with probability 1, Y − E[Y ] is a positive multiple of X −E[X].

(e) If ρ(X, Y ) = −1, then, with probability 1, Y −E[Y ] is a negative multiple of X −E[X].

3. (Problem 4.29) Let X and Y be two random variables with positive variances.

(a) Let XL be the linear least mean squares estimator of X based on Y . Show that

E[(X − XL)Y ] = 0.

Use this property to show that the correlation of the estimation error X − XL with Y is zero.

(b) Let X = E[X | Y ] be the least mean squares estimator of X given Y . Show that

E[(X − X)h(Y )] = 0,

for any function h.

(c) Is it true that the estimation error X − E[X | Y ] is independent of Y ?

Page 1 of 1

� �



Recitation 14 Solutions April 11, 2006

1. We know that: Cov(X1,X2)

ρ(X1,X2) = σX1 σX2

Therefore we first find the covariance:

Cov(A,B) = E[AB] − E[A]E[B] = E[WX + WY + X2 + XY ] = E[X2] = 1

and √

σA = Var(A) = 2 √

σB = Var(B) = 2

and therefore: 1

ρ(A,B) = . 2

We proceed as above to find the correlation of A,C.

Cov(A,C) = E[AC] − E[A]E[C] = E[WY + WZ + XY + XZ] = 0

and thereforeρ(A,C) = 0.



Page 1 of 1




1. Let X and Y be random variables, and let a and b be scalars; X takes nonnegative values.

(a) Use the Markov inequality on the random variable esY to show that

−sbMY (s),P (Y ≥ b) ≤ e

for every s > 0, where MY (s) is the transform of Y .

2. Joe wishes to estimate the true fraction f of smokers in a large population without asking each and every person. He plans to select n people at random and then employ the estimator F = S/n, where S denotes the number of people in a size-n sample who are smokers. Joe would like to sample the minimum number of people, but also guarantee an upper bound p on the probability that the estimator F differs from the true value f by a value greater than or equal to d i.e., for a given accuracy d and given confidence p, Joe wishes to select the minimum n such that

P(|F − f | ≥ d) ≤ p .

For p = 0.05 and a particular value of d, Joe uses the Chebyshev inequality to conclude that n

must be at least 50,000. Determine the new minimum value for n if:

(a) the value of d is reduced to half of its original value.

(b) the probability p is reduced to half of its original value, or p = 0.025.

3. Let X1,X2, . . . be a sequence of independent random variables that are uniformly distributed between 0 and 1. For every n, we let Yn be the median of the values of X1,X2, . . . ,X2n+1. [That is, we order X1, . . . ,X2n+1 in increasing order and let Yn be the (n+1)st element in this ordered sequence.] Show that the sequence Yn converges to 1/2, in probability.

Page 1 of 1

�

�

� �

� �




1. See textbook pg. 399

2. (a) N = 200, 000.

(b) N = 100, 000.

3. Let us fix some ǫ > 0. We will show that P (Yn ≥ 0.5 + ǫ) converges to 0. By symmetry, this will imply that P (Yn ≤ 0.5 − ǫ) also converges to zero, and it will follow that Yn converges to 0.5, in probability.

For the event {Yn ≥ 0.5 + ǫ} to occur, we must have at least n + 1 of the random variables X1,X2, . . . ,X2n+1 to have a value of 0.5 + ǫ or larger. Let Zi be a Bernoulli random variable which is equal to 1 if and only if Xi ≥ 0.5 + ǫ:

1 if Xi ≥ 0.5 + ǫ Zi =

0 otherwise

{Z1, Z2, ....} are i.i.d random variables and E[Zi] = P (Zi = 1) = P (Xi ≥ 0.5 + ǫ) = 0.5 − ǫ.

Hence, for the event {Yn ≥ 0.5 + ǫ} to occur, we must have at least n + 1 of the {Zi} to take value 1,

2n+1

P (Yn ≥ 0.5 + ǫ) = P ( Zi ≥ n + 1) i=1 �2n+1 Zi n + 1

= P ( i=1 ≥ )2n + 1 2n + 1�2n+1 Zi 1

= P ( i=1 ≥ 0.5 + )2n + 1 2(2n + 1)�2n+1 Zi

≤ P ( i=1 ≥ 0.5) 2n + 1

Note that P (Zi = 1) = 0.5 − ǫ. By the weak law of large numbers, the sequence (Z1 +

· · · + Z2n+1)/(2n + 1) converges to 0.5 − ǫ. To show that P Z1+···+Z2n+1 ≥ 0.5 converges 2n+1

to zero, we need to show that for any given ǫ > 0, there exists N such that for all n > N ,

P Z1+···+Z2n+1 ≥ 0.5 < ǫ. The fact that the sequence (Z1 + · · ·+ Z2n+1)/(2n + 1) converges to 2n+1 �2n+1

Zii=1 0.5− ǫ ensures the existence of such N . Since P (Yn ≥ 0.5 + ǫ) is bounded by P ( 2n+1 ≥ 0.5),

it also converges to zero.

Page 1 of 1

of 1



Recitation 16April 25, 2006

1. (Example 5.3) A computer executes two types of tasks, priority and non priority, and operates in discrete time units (slots). A priority task arises with probability p at the beginning of each slot, independently of other slots and requires one full slot to complete. A non priority task is always available and is executed at a given slot if no priority task is available. In this context, it may be important to the know the probabilistic properties of the time intervals available for non priority tasks.

With this in in mind let us call a slot busy if within this slot the computer executes a priority task, and otherwise let us call it idle. We call a string of idle (or busy) slots , flanked by busy (or idle, respectively) slots an idle period or busy period respectively. Derive the PMF, mean and variance of the following random variables.

(a) T = the time index of the first slot;

(b) B = the length (number of slots) of the first busy period;

(c) I = the length of the first idle period.

(d) Z = the number of slots after the first slot of the first busy period up to and including the first subsequent idle slot.

2. (Problem 5.4) Consider a Bernoulli process with probability of success in each trial equal to p.

(a) Relate the number of failures before the rth success (sometimes called a negativebinomial random variable) to a Pascal random variable and derive its PMF.

(b) Find the expected value and variance of the number of failures before the rth success.

(c) Obtain an expression for the probability that the ith failure occurs before the rth success.

3. (Problem 5.6) Sum of a geometric number of independent random variables. Let Y = X1 + X2 + ... + XN , where the random variables Xi are geometric with parameter p, and N is geometric with parameter q. Assume that the random variables N, X1,X2, ... are independent. Show, without using transforms, that Y is geometric with parameter pq. Hint : Interpret the various random variables in terms of a split Bernoulli process.

of ??




1. (Example 5.3) See textbook, page 276.

2. (Problem 5.4) See textbook, page 302.


of 1




1. An amateur criminal is contemplating shoplifting from a store. Police officers walk by the store according to a Poisson process of rate λ per minute. If an officer walks by while the crime is in progress, the criminal will be caught.

(a) If it takes the criminal t seconds to commit the crime, find the probability that the criminal will be caught.

(b) Repeat part (a) under the new assumption that the criminal will only be caught if two police officers happen to walk by while the crime is in progress.

2. (Problem 5.12) Beginning at time t = 0 we begin using bulbs, one at a time, to illuminate a room. Bulbs are replaced immediately upon failure. Each new bulb is selected independently by an equally likely choice between a Type-A bulb and a Type-B bulb.

The lifetime, X, of any particular bulb of a particular type is an independent random variable with the following PDF:

� For Type-A Bulbs: fX (x) =

e−x

0 x ≥ 0 elsewhere

For Type-B Bulbs: fX (x) =

� 3e−3x

0 x ≥ 0 elsewhere

(a) Find the expected time until the first failure.

(b) Find the probability that there are no bulb failures before time t.

(c) Given that there are no failures until time t, determine the conditional probability that the first bulb used is a type-A bulb.

(d) Find the variance of the time until the first bulb failure.

(e) Find the probability that the 12th bulb failure is also the 4th type-A bulb failure.

(f) Up to and including the 12th bulb failure, what is the probability, that a total of exactly 4 Type-A bulbs have failed?

(g) Determine either the PDF or the transform associated with the time until the 12th bulb failure.

(h) Determine the probability that the total period of illumination provided by the first two Type-B bulbs is longer than that provided by the first Type-A bulb.

3. (Problem 5.16) Consider a Poisson process. Given that a single arrival occurred in a given interval [0, t], show that the PDF of the arrival time is uniform over [0, t].

of 1




1. (a) The criminal will be caught if the first officer comes by in fewer than t seconds. Since the time until the first arrival is exponentially distributed, the desired probability is 1 − e−tλ .

(b) We are interested in the probability that the second arrival occurs before time t. The Erlang PDF of order 2 is fY2(y) = λ2ye−λy. The desired probability is obtained by integrating by parts and is 1 − e−λt(λt + 1).


3. (a) The arrival time of each of the three calls is uniformly distributed in the interval of 90 minutes (see Problem 16 in Chapter 5 of the text). Furthermore, the three arrival times are independent of each other. This follows intuitively from the definition of the Poisson process: given that there was an arrival at some particular time, this gives us no information on what may have happened at other times. Therefore the probability that all three occur within the first 30 minutes is: (1/3)3 = 1/27.

(b) The probability that at least one ocurs in the first 30 minutes is, by the same reasoning as above, 1 − (8/27) = 19/27.



Recitation 18 May 2, 2006

1. (Example 5.15) Competing Exponentials. Two light bulbs have independent and exponentially distributed lifetimes Ta and Tb, with parameters λa and λb, respectively. What is the distribution of Z = min{Ta, Tb}, the first time when a bulb burns out ?

2. (Example 5.16) More on Competing Exponentials. Three light bulbs have independent exponentially distributed lifetimes with a common parameter λ. What is the expected value of the time until the last bulb burns out ?

3. (Problem 5.17a) Let X1 and X2 be independent and exponentially distributed, with parameters λ1 and λ2, respectively. Find the expected value of max{X1,X2} .

4. (Problem 5.21) The number of Poisson arrivals during an exponentially distributed interval. Consider a Poisson process with parameter λ, and an independent random variable T , which is exponential with parameter ν. Find the PMF of the number of Poisson arrivals during the time interval [0, T ].

Page 1 of 1



Recitation 18May 2, 2006

1. Example 5.15 in text, page 296


3. Problem 5.17a in text, page 307

4. Problem 5.21 in text, page 310

Page 1 of 1



Recitation 19May 4, 2006

1. (Example 6.3) A machine can be either working or broken down on a given day. If it is working, it will break down in the next day with probability b, and will continue working with probability 1 − b. If it breaks down on a given day, it will be repaired and be working in the next day with probability r, and will continue to be broken down with probability 1−r. Suppose that whenever the machine remains broken for a given number of l days, despite the repair efforts, it is replaced by a new working machine. Model this machine as a Markov chain.

2. (Problem 6.3) A spider and a fly move along a straight line in unit increments. The spider always moves towards the fly by one unit. The fly moves towards the spider by one unit with probability 0.3, moves away from the spider by one unit with probability 0.3, and stays in place with probability 0.4. The initial distance between the spider and the fly is integer. When the spider and the fly land in the same position, the spider captures the fly.

(a) Construct a Markov chain that describes the relative location of the spider and fly.

(b) Identify the transient and recurrent states.

(c) Assume that the initial distance between the spider and the fly is 2 units. Provide a recursive formula to evaluate the n-step transition probabilities rij (n), i, j ∈ {0, 1, 2}. Compute r2i(3), i ∈ {0, 1, 2}. Can you infer the limiting behavior of the n-step transition probabilities ?

3. (Problem 6.4) Existence of a recurrent state. Show that in a Markov chain at least one recurrent state must be accessible from any given state, i.e., for any i, there is at least one recurrent j in the set A(i) of accessible states from i.

Page 1 of 1



Recitation 19 May 4, 2006


2. parts (a) and (b) Problem 6.3 in text, page 354 (c) the n-step transition probabilities can be generated by the recursive formula

2 ∑ rij(n) = rik(n − 1)pkj for n > 1, and all i, j

k=0

starting with rij(1) = pij where

1 0 0

[pij ] = 0.4 0.6 0 0.3 0.4 0.3

Plugging into the above formula gives :

1 0 0

[rij (2)] = 0.64 0.36 0 0.55 0.36 0.09

Similarly

1 0 0 1 0 0 1 0 0

[rij (3)] = 0.784 0.216 0 , [rij(5)] = 0.922 0.078 0 , [rij (10)] = 0.994 0.006 0 0.721 0.252 0.027 0.897 0.100 0.003 0.992 0.008 0

Eventually the spider will catch the fly, thus :

1 0 0

lim [rij (n)] = 1 0 0 n→∞

1 0 0

3. Problem 6.4 in text, page 354

Page 1 of 1



Recitation 20 Markov Chains: Steady State Behavior

May 09, 2006

1. (Problem 6.9) A professor gives tests that are hard, medium or easy. If she gives a hard test, her next test will be either medium or easy, with equal probability. However, if she gives a medium or easy test, there is a 0.5 probability that her next test will be of the same difficulty, and a 0.25 probability of each of the other two levels of difficulty. Construct an appropriate Markov chain and find the steadystate probabilities.

2. (Problem 6.10) Alvin likes to sail each Saturday to his cottage on a nearby island off the coast. Alvin is an avid fisherman, and enjoys fishing off his boat on the way to and from the island, as long as the weather is good. Unfortunately, the weather is good on the way to or from the island with probability p, independently of what the weather was on any past trip (so the weather could be nice on the way to the island, but poor on the way back). Now, if the weather is nice, Alvin will take one of his n fishing rods for the trip, but if the weather is bad, he will not bring a fishing rod with him. We want to find the probability that on a given leg of the trip to or from the island the weather will be nice, but Alvin will not fish because all his fishing rods are at his other home.

(a) Formulate an appropriate Markov chain model with n + 1 states and find the steady state probabilities.

(b) What is the steadystate probability that on a given trip, Alvin sails with nice weather but without a fishing rod?

3. (Problem 6.13) Ehrenfest model of diffusion. We have a total of n balls, some of them black, some white. At each time step, we either do nothing, which happens with probability �, where 0 < � < 1, or, we select a ball at random, so that each ball has probability (1− �)/n > 0 of being selected. In the latter case, we change the color of the selected ball (if white it becomes black, and vice versa), and the process is repeated indefinitely. What is the steadystate distribution of the number of white balls?

Page 1 of 1



Solutions for Recitation 20Markov Chains: Steady State Behavior

May 09, 2006




Page 1 of 1



Recitation 21 Markov Chains: Absorption Probabilities and Expected Time to Absorption

May 11, 2006

1. Josephina is currently a 61 student. On each day that she is a 61 student, she has a probability of 1/2 of being a course 61 student the next day. Otherwise, she has an equally likely chance of becoming a 62 student, a 63 student, a course 9 student or a course 15 student the next day. On any day she is a 63 student, she has a probability of 1/4 of switching to course 9, a probability of 3/8 of switching to 61 and a probability of 3/8 of switching to 62 the next day. On any day she is a 62 student, she has a probability of 1/2 of switching to course 15, a probability of 3/8 of switching to 61 and a probability of 1/8 of switching to 63 the next day.

In answering the questions below, assume Josephina will be a student forever. Also assume, for parts (a)(f ) that if Josephina switches to course 9 or course 15, she will stay there and will not change her course again.

(a) What is the probability that she eventually will leave course 6?

(b) What is the probability that she will eventually be in course 15?

(c) What is the expected number of days until she leaves course 6?

(d) Every time she switches into 61 from 62 or 63, she buys herself an ice cream cone at Tosci’s. She can only afford so much ice cream, so after she’s eaten 2 ice cream cones, she stops buying herself ice cream. What is the expected number of ice cream cones she buys herself before she leaves course 6?

(e) Her friend Oscar started out just like Josephina. He is now in course 15. You don’t know how long it took him to switch. What is the expected number of days it took him to switch to course 15? [Hint: He had no particular aversion to course 9.]

(f) Josephina decides that course 15 is not in her future. Accordingly, when she is a course 61 student, she stays 61 for another day with probability 1/2, and otherwise she has an equally likely chance of becoming any of the other options. When she is 62, her probability of entering 61 or 63 are in the same proportion as before. What is the expected number of days until she is in course 9?

(g) Suppose that if she is course 9 or course 15, she has probability 1/8 of returning to 61, and otherwise she remains in her current course. What is the expected number of days until she is 61 again? (Notice that we know today she is 61, so if tomorrow she is still 61, then the number of days until she is 61 again is 1).

Page 1 of 1



Solutions for Recitation 21Markov Chains: Absorption Probabilities and Expected Time to Absorption

May 11, 2006

1. (a) The Markov chain is shown below.

1/8

9

1/4

1

1/2

3/8

1/8 3/8

1/8

1/8

1/2

1

6-3 6-2

6-1

15

3/8

1/8

By inspection, the states 6-1, 6-2, and 6-3 are all transient, since they each have paths leading to either state 9 or state 15, from which there is no return. Therefore she eventually leaves course 6 with probability 1 .

(b) This is simply the absorption probability for the recurrent class consisting of the state course-15. Let us denote the probability of being absorbed by state 15 conditioned on being in state i as ai. Then

a15 = 1

a9 = 0 1 1 1 1 1

a6−1 = a6−1 + (1) + a6−2 + (0) + a6−32 8 8 8 8 1 3 1

a6−2 = (1) + a6−1 + a6−32 8 8 1 3 3

a6−3 = (0) + a6−1 + a6−24 8 8

Solving this system of equations yields

105 a6−1 = ≈ 0.571

184

We will keep the other ai’s around as well - they will be useful later:

a6−2 = 0.77717

a6−3 = 0.50543

Page 1 of 5



(c) This corresponds to an expected time until absorption for the transient state 6 − 1. Let µi

be the expected time until absorption conditioned on being in state i. Then

µ15 = 0

µ9 = 0 1 1 1 1 1

µ6−1 = 1 + µ6−1 + (0) + µ6−2 + (0) + µ6−32 8 8 8 81 3 1

µ6−2 = 1 + (0) + µ6−1 + µ6−32 8 81 3 3

µ6−3 = 1 + (0) + µ6−1 + µ6−24 8 8


162 81 µ6−1 = = ≈ 3.522

46 23

(d) The student buys one ice cream cone every time she goes from 6-2 to 6-1 or from 6-3 to 6-1, and buys no more than 2 ice cream cones. Let us denote vi(j) as the probability that she transitions from from 6 − 2 to 6 − 1 or from 6 − 3 to 6 − 1 j times before leaving course 6, conditioned on being in state i. Then we are interested in the expected value of the random variable N , which denotes the number of cones bought before leaving course 6, and takes on the values 0, 1, or 2. So

E[N ] = (0)v6−1(0) + (1)v6−1(1) + (2)(1 − v6−1(0) − v6−1(1))

We use the total probability theorem, conditioning on the next day, to yield the following set of recursive equations:

v15(0) = 1

v9(0) = 1 1 1 1 1 1

v6−1(0) = v6−1(0) + v6−2(0) + v6−3(0) + (1) + (1) 2 8 8 8 83 1 1

v6−2(0) = (0) + v6−3(0) + (1) 8 8 23 3 1

v6−3(0) = (0) + v6−2(0) + (1) 8 8 4

Solving this system of equations yields:

46 v6−1(0) = ≈ 0.754

61

We still need to find v6−1, and we do this by again conditioning on the second following day:

1 1 1 1 1 v6−1(1) = v6−1(1) + v6−2(1) + v6−3(1) + (0) + (0)

2 8 8 8 83 1 1

v6−2(1) = v6−1(0) + v6−3(1) + (0) 8 8 23 3 1

v6−3(1) = v6−1(0) + v6−2(1) + (0) 8 8 4

Page 2 of 5



Notice in the second and third equations that when she goes into state 6-1, this automatically sets v to 1, so we require that there be no more transitions from 6-2 to 6-1 or from 6-3 to 6-1 after the second day (that is, v=0 starting in state 6-1, whose probability we found before). Solving this system of equations yields:

690 v6−1(1) = ≈ 0.185

3721

Finally, we can solve for the expected number of cones:

E[N ] = (0)v6−1(0) + (1)v6−1(0) + (2)(1 − v6−1(0) − v6−1(1)) 690 225

= + 2( )3721 37211140

= ≈ 0.306 3721

(e) We want to find the expected time to absorption conditioned on the event that the student eventually ends up in state 15, which we will call A. So

Pi,j|A = P(Xn+1 = j|Xn = i, X∞ = 15)

P(X∞ = 15|Xn+1 = j)P(Xn+1 = j|Xn = i)

= P(X

∞ = 15|Xn = i) aj Pi,j

= ai

where ak is the absorption probability of eventually ending up in state 15 conditioned on being in state k, which we found in part (b). So we may modify our chain with these new conditional probabilities and calculate the expected time to absorption on the new chain. Note that state 9 now disappears. Also, note that Pj,j|A = Pj,j , but Pi,j|A 6= Pi,j for i 6= j, which means that we may not simply renormalize the transition probabilities in a uniform fashion after conditioning on this event. Let us denote the new expected time to absorption, conditioned on being in state i as µi Our system of equations now becomes

µ15 = 0 a6−1 1 a6−2 1 a6−3 1

µ6−1 = 1 + µ6−1 + 0 + µ6−2 + 0 + µ6−3 a6−1 2 a6−1 8 a6−1 8

a6−1 3 a6−3 1 µ6−2 = 1 + 0 + µ6−1 + µ6−3

a6−2 8 a6−2 8 a6−1 3 a6−2 3

µ6−3 = 1 + 0 + µ6−1 + µ6−2 a6−3 8 a6−3 8


1763 µ6−1 = ≈ 3.65

483

(f) The new Markov chain is shown below.

Page 3 of 5



1/4

1

1/2

3/8

6-3 6-2

6-1

9

3/8

1/6

1/6

1/6 3/4

1/4

This is another expected time to absorption question on the new chain. Let us define µk

to be the expected number of days it takes the student to go from state k to state 9 in this new Markov chain:

1 1 1 1 µ6−1 = 1 + µ6−1 + µ6−2 + µ6−3 + (0)

2 6 6 63 1

µ6−2 = 1 + µ6−1 + µ6−34 43 3 1

µ6−3 = 1 + µ6−1 + µ6−2 + (0) 8 8 4

Solving this system of equations yields:

86 µ6−1 = ≈ 6.615

13

(g) The corresponding Markov chain is the same as the one in part (a) except p9,6−1 = 18 , p9,9 =

7 1 7 , p15,6−1 = , p15,15 = instead of p9,9 = 1, p15,15 = 1. 8 8 8 We can consider state 6-1 as an absorbing state. Let µk be the expected number of transitions to be absorbed if we start at state k

1 7 µ9 = + (1 + µ9) ⇒ µ9 = 8

8 81 7

µ15 = + (1 + µ15) ⇒ µ15 = 8 8 83 3 1

µ6−3 = + (1 + µ6−2) + (1 + µ9)8 8 43 1 1

µ6−2 = + (1 + µ6−3) + (1 + µ15)8 8 2

344 312 ⇒ µ6−2 = , µ6−3 =

61 61

Page 4 of 5



Let R be the number of days until she is 6-1 again. We find E[R] by using the total expectation theorem, conditioned on what happens on the first transition.

E[R] = E[E[R|X2]]1 1 1 1 1

= (1) + (1 + µ9) + (1 + µ15) + (1 + µ6−2) + (1 + µ6−3)2 8 8 8 8265

= 61

Notice that this chain consists of a single recurrent aperiodic class. Another approach to solving this problem uses the steady state probabilites of this chain, which are π6−1 = 61 11 9 79 105 , π6−2 = , π6−3 = , π9 = , π15 = . The expected frequency of visits to 6-1 is 265 265 265 265 265 π6−1, so the expected number of days between visits to 6-1 is

π6

1 −1

. Since she is currently 1 265 6-1, the expected number of days until she is 6-1 again is

π6−1 = 61 .

Page 5 of 5



Recitation 22 Central Limit Theorem

May 16, 2006

1. (Example 7.8) We load on a plane 100 packages whose weights are independent random variables that are uniformly distributed between 5 and 50 pounds. What is the probability that the total weight will exceed 3000 pounds? Find an approximate answer using the Central Limit Theorem.

2. (Problem 7.6) Before starting to play the roulette in a casino, you want to look for biases that you can exploit. You therefore watch 100 rounds that result in a number between 1 and 36, and count the number of rounds for which the result is odd. If the count exceeds 55, you decide that the roulette is not fair. Assuming that the roulette is fair, find an approximation for the probability that that you will make the wrong decision.

3. (Problem 7.7) During each day, the probability that your computer’s operating system crashes at least once is 5%, independent of every other day. You are interested in the probability of at least 45 crashfree days out of the next 50 days.

(a) Find the probability of interest by using the normal approximation to the binomial.

(b) Repeat part (a), this time using the Poisson approximation to the binomial.

Page 1 of 1



Solutions for Recitation 22 Central Limit Theorem

May 16, 2006

1. See solution in text, page 390.



Page 1 of 1