Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business...

92
Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 1 This handout is available at http://www.f.waseda.jp/toyoizumi.

Transcript of Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business...

Page 1: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Stochastic Models in Business

Hiroshi Toyoizumi 1

March 21, 2013

1This handout is available at http://www.f.waseda.jp/toyoizumi.

Page 2: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Contents

1 Basics in Probability Theory 81.1 Why Probability? . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2 Probability Space . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3 Conditional Probability and Independence . . . . . . . . . . . . . 111.4 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . 131.5 Expectation, Variance and Standard Deviation . . . . . . . . . . . 141.6 How to Make a Random Variable . . . . . . . . . . . . . . . . . . 171.7 News-vendor Problem, “How many should you buy?” . . . . . . . 171.8 Covariance and Correlation . . . . . . . . . . . . . . . . . . . . . 181.9 Value at Risk . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191.10 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2 Markov chain 242.1 Series of Random Variables . . . . . . . . . . . . . . . . . . . . . 242.2 Discrete-time Markov chain . . . . . . . . . . . . . . . . . . . . 242.3 Time Evolution of Markov Chain . . . . . . . . . . . . . . . . . . 262.4 Stationary State . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.5 Matrix Representation . . . . . . . . . . . . . . . . . . . . . . . 272.6 Stock Price Dynamics Evaluation Based on Markov Chain . . . . 282.7 Google’s PageRank and Markov Chain . . . . . . . . . . . . . . . 29

3 Birth and Death process and Poisson Process 333.1 Definition of Birth and Death Process . . . . . . . . . . . . . . . 333.2 Differential Equations of Birth and Death Process . . . . . . . . . 343.3 Infinitesimal Generator . . . . . . . . . . . . . . . . . . . . . . . 343.4 System Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . 353.5 Poisson Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2

Page 3: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

CONTENTS 3

3.6 Why we use Poisson process? . . . . . . . . . . . . . . . . . . . 363.7 Z-trnasform of Poisson process . . . . . . . . . . . . . . . . . . . 373.8 Independent Increment . . . . . . . . . . . . . . . . . . . . . . . 373.9 Interearrival Time . . . . . . . . . . . . . . . . . . . . . . . . . . 373.10 Memory-less property of Poisson process . . . . . . . . . . . . . 373.11 PASTA: Poisson Arrival See Time Average . . . . . . . . . . . . 383.12 Excercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

4 Introduction of Queueing Systems 404.1 Foundation of Performance Evaluation . . . . . . . . . . . . . . . 404.2 Starbucks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414.3 Specification of queueing systems . . . . . . . . . . . . . . . . . 414.4 Little’s formula . . . . . . . . . . . . . . . . . . . . . . . . . . . 424.5 Lindley equation and Loynes variable . . . . . . . . . . . . . . . 444.6 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

5 PASTA 475.1 What will be seen by customers . . . . . . . . . . . . . . . . . . 475.2 Stochastic Intensity . . . . . . . . . . . . . . . . . . . . . . . . . 485.3 Lack of Bias Assumption . . . . . . . . . . . . . . . . . . . . . . 495.4 Reverse Stochastic Intensity . . . . . . . . . . . . . . . . . . . . 495.5 Poisson arrivals see time average . . . . . . . . . . . . . . . . . . 505.6 Excercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

6 M/M/1 queue 526.1 M/M/1 queue as a birth and death process . . . . . . . . . . . . . 526.2 Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 536.3 Waiting Time Estimation . . . . . . . . . . . . . . . . . . . . . . 53

6.3.1 Waiting Time by Little‘s Formula . . . . . . . . . . . . . 546.3.2 Waiting Time Distribution of M/M/1 Queues . . . . . . . 54

6.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 556.5 Excercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

7 Reversibility 577.1 Output from Queue . . . . . . . . . . . . . . . . . . . . . . . . . 577.2 Reversibibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

7.2.1 Definition of Reversibility . . . . . . . . . . . . . . . . . 577.2.2 Local Balance for Markov Chain . . . . . . . . . . . . . . 57

Page 4: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

4 CONTENTS

7.3 Output from M/M/1 queue . . . . . . . . . . . . . . . . . . . . . 597.4 Excercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

8 Network of Queues 608.1 Open Network of queues . . . . . . . . . . . . . . . . . . . . . . 608.2 Global Balance of Network . . . . . . . . . . . . . . . . . . . . . 608.3 Traffic Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 618.4 Product Form Solution . . . . . . . . . . . . . . . . . . . . . . . 618.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

9 Examples of Queueing System Comparison 649.1 Single Server vs Tandem Servers . . . . . . . . . . . . . . . . . . 649.2 M/M/2 queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659.3 M/M/1 VS M/M/2 . . . . . . . . . . . . . . . . . . . . . . . . 669.4 Two M/M/1 VS One M/M/2 . . . . . . . . . . . . . . . . . . . 679.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

10 Stochastic Integral 6910.1 Diffusion Process . . . . . . . . . . . . . . . . . . . . . . . . . . 6910.2 Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7110.3 Definition of Stochastic Integral . . . . . . . . . . . . . . . . . . 7210.4 Martingale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7410.5 Calculus of Stochastic Integral . . . . . . . . . . . . . . . . . . . 74

11 Examples of Stochastic Integral 7811.1 Evaluation of E[W (t)4] . . . . . . . . . . . . . . . . . . . . . . . 7811.2 Evaluation of E[eαW (t)] . . . . . . . . . . . . . . . . . . . . . . . 80

12 Differential Equations 8112.1 Ordinary Differential Equation . . . . . . . . . . . . . . . . . . . 8112.2 Geometric Brownian Motion . . . . . . . . . . . . . . . . . . . . 8212.3 Stochastic Process and Partial Differential Equation . . . . . . . . 83

13 Portfolio Dynamics 8513.1 Portfolio Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 8513.2 Rate of Return of Stock and Risk-free Asset . . . . . . . . . . . . 8713.3 Arbitrage and Portfolio . . . . . . . . . . . . . . . . . . . . . . . 87

Page 5: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

CONTENTS 5

14 Pricing via Arbitrage 8914.1 Way to Black-Scholes Model . . . . . . . . . . . . . . . . . . . . 89

Page 6: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Bibliography

T. Bjork. Arbitrage Theory in Continuous Time. Oxford Finance. Oxford Univ Pr,2nd edition, 2004.

R. Durrett. Probability: Theory and Examples. Thomson Learning, 1991.

M. El-Taha and J. S. Stidham. Sample-path analysis of queueing systems.Kluwer’s international press, 1999.

P. Glynn, B. Melamed, and W. Whitt. Estimating customer and time averages,1993. URL citeseer.nj.nec.com/glynn93estimating.html.

A. Langville and C. Meyer. Google’s PageRank and Beyond: The Science ofSearch Engine Rankings. Princeton University Press, 2006.

B. Melamed and W. Whitt. On arrival tha see time averages: a martingale ap-proach. J. of Applied Probability, 27:376 – 384, 1990.

B. Melamed and D. Yao. The asta property, 1995. URL citeseer.nj.nec.com/melamed95asta.html.

B. Oksendal. Stochastic Differential Equations: An Introduction With Applica-tions. Springer-Verlag, 2003.

R. K. P. Bremaud and R. Mazumdar. Event and time averages: a review. Adv.Appl. Prob., 24:377 – 411, 1992.

S. M. Ross. Applied Probability Models With Optimization Applications. DoverPubns, 1992.

N. N. Taleb. Fooled by Randomness: The Hidden Role of Chance in the Marketsand in Life. Random House Trade Paperbacks, 2005.

6

Page 7: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

BIBLIOGRAPHY 7

H. Toyoizumi. Applied probability and mathematical finance theory.http://www.f.waseda.jp/toyoizumi/classes/classes.html, 2008. URL http://www.f.waseda.jp/toyoizumi/classes/classes.html.

Page 8: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 1

Basics in Probability Theory

1.1 Why Probability?Example 1.1. Here’s examples where we use probability:

• Lottery.

• Weathers forecast.

• Gamble.

• Baseball,

• Life insurance.

• Finance.

Problem 1.1. Name a couple of other examples you could use probability theory.

Since our intuition sometimes leads us mistake in those random phenomena,we need to handle them using extreme care in rigorous mathematical framework,called probability theory. (See Exercise 1.1).

1.2 Probability SpaceBe patient to learn the basic terminology in probability theory. To determine theprobabilistic structure, we need a probability space, which is consisted by a sam-ple space, a probability measure and a family of (good) set of events.

8

Page 9: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

1.2. PROBABILITY SPACE 9

Definition 1.1 (Sample Space). The set of all events is called sample space, andwe write it as Ω. Each element ω ∈Ω is called an event.

Example 1.2 (Lottery). Here’s an example of Lottery.

• The sample space Ω is first prize, second prize,..., lose.

• An event ω can be first prize, second prize,..., lose, and so on.

Sometimes, it is easy to use sets of events in sample space Ω.

Example 1.3 (Sets in Lottery). The following is an example in Ω of Example 1.2.

W = win= first prize, second prize,..., sixth prize (1.1)L = lose (1.2)

Thus, we can say that “what is the probability of win?”, instead of saying“what is the probability that we have either first prize, second prize,..., or sixthprize?”.

Example 1.4 (Coin tosses). Let us consider tossing coins 10 times. Then, bywriting ”up =1” and ”down = 0”, the corresponding sample space Ω is

Ω = ω = (x1,x2, . . . ,x10) : xi = 0 or 1. (1.3)

Problem 1.2. Find the set S = the number of up is 5 in Example 1.4.

Definition 1.1 (Probability measure). The probability of A, P(A), is defined foreach set of the sample space Ω, if the followings are satisfyed:

1. 0≤ P(A)≤ 1 for all A⊂Ω.

2. P(Ω) = 1.

3. For any sequence of mutually exclusive A1,A2...

P(∞⋃

i=1

Ai) =∞

∑i=1

P(Ai). (1.4)

In addition, P is said to be the probability measure on Ω. The third conditionguarantees the practical calculation on probability.

Page 10: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

10 CHAPTER 1. BASICS IN PROBABILITY THEORY

Mathematically, all function f which satisfies Definition 1.1 can regarded asprobability. In other words, we need to be careful to select which function issuitable for probability.

Example 1.5 (Probability Measures in Lottery). Suppose we have a lottery suchas 10 first prizes, 20 second prizes · · · 60 sixth prizes out of total 1000 tickets,then we have a probability measure P defined by

P(n) = P(win n-th prize) =n

100(1.5)

P(0) = P(lose) =79100

. (1.6)

It is easy to see that P satisfies Definition 1.1. According to the definition P, wecan calculate the probability on a set of events:

P(W ) = the probability of win= P(1)+P(2)+ · · ·+P(6)

=21

100.

Of course, you can cheat your customer by saying you have 100 first prizesinstead of 10 first prizes. Then your customer might have a different P satisfyingDefinition 1.1. Thus it is pretty important to select an appropriate probabilitymeasure. Selecting the probability measure is a bridge between physical worldand mathematical world. Don’t use wrong bridge!

Problem 1.3. In Example 1.4, find the appropriate probability measure P on Ω.For example, calculate

P(S), (1.7)

where S = the number of up is 5.

Remark 1.1. There is a more rigorous way to define the probability measure. In-deed, Definition 1.1 is NOT mathematically satisfactory in some cases. If you arefamiliar with measure theory and advanced integral theory, you may proceed toread [Durrett, 1991].

Page 11: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

1.3. CONDITIONAL PROBABILITY AND INDEPENDENCE 11

1.3 Conditional Probability and IndependenceNow we introduce one of the most uselful and probably most difficult concepts ofprobability theory.

Definition 1.2 (Conditional Probability). Define the probability of B given A by

P(B | A) = P(B & A)P(A)

=P(B∩A)

P(A). (1.8)

We can use the conditional probability to calculate complex probability. It isactually the only tool we can rely on. Be sure that the conditional probabilityP(B|A) is different with the regular probability P(B).

Example 1.6 (Lottery). Let W = win and F = first prize in Example 1.5.Then we have the conditional probability that

P(F |W ) = the probability of winning 1st prize given you win the lottery

=P(F ∩W )

P(W )=

P(F)

P(W )

=10/1000

210/1000=

121

6= 101000

= P(F).

Remark 1.2. Sometimes, we may regard Definition 1.2 as a theorem and callBayse rule. But here we use this as a definition of conditional probability.

Problem 1.4 (False positives1). Answer the followings:

1. Suppose there are illegal acts in one in 10000 companies on the average.You as a accountant audit companies. The auditing contains some uncer-tainty. There is a 1% chance that a normal company is declared to havesome problem. Find the probability that the company declared to have aproblem is actually illegal.

2. Suppose you are tested by a disease that strikes 1/1000 population. Thistest has 5% false positives, that mean even if you are not affected by thisdisease, you have 5% chance to be diagnosed to be suffered by it. A medicaloperation will cure the disease, but of course there is a mis-operation. Giventhat your result is positive, what can you say about your situation?

1Modified from [Taleb, 2005, p.207].

Page 12: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

12 CHAPTER 1. BASICS IN PROBABILITY THEORY

Problem 1.5. In Example 1.4, find the conditional probability

P(A|S), (1.9)

where A = the first 4 tosses are all up and S = the number of up is 5.

Definition 1.3 (Independence). Two sets of events A and B are said to be indepen-dent if

P(A&B) = P(A∩B) = P(A)P(B) (1.10)

Theorem 1.1 (Conditional of Probability of Independent Events). Suppose A andB are independent, then the conditional probability of B given A is equal to theprobability of B.

Proof. By Definition 1.2, we have

P(B | A) = P(B∩A)P(A)

=P(B)P(A)

P(A)= P(B),

where we used A and B are independent.

Example 1.7 (Independent two dices). Of course two dices are independent. So

P(The number on the first dice is even while the one on the second is odd)= P(The number on the first dice is even)P(The number on the second dice is odd)

=12· 1

2.

Example 1.8 (More on two dice). Even though the two dices are independent,you can find dependent events. For example,

P(The first dice is bigger than second dice even while the one on the second is even) =?

How about the following?

P(The sum of two dice is even while the one on the second is odd ) =?.

See Exercise 1.4 for the detail.

Page 13: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

1.4. RANDOM VARIABLES 13

1.4 Random VariablesThe name random variable has a strange and stochastic history2. Although itsfragile history, the invention of random variable certainly contribute a lot to theprobability theory.

Definition 1.4 (Random Variable). The random variable X = X(ω) is a real-valued function on Ω, whose value is assigned to each outcome of the experiment(event).

Remark 1.3. Note that probability and random variables is NOT same! Randomvariables are function of events while the probability is a number. To avoid theconfusion, we usually use the capital letter to random variables.

Example 1.9 (Lottery). A random variable X can be designed to formulate a lot-tery.

• X = 1, when we get the first prize.

• X = 2, when we get the second prize.

Example 1.10 (Bernouilli random variable). Let X be a random variable with

X =

1 with probability p.0 with probability 1− p.

(1.11)

for some p ∈ [0,1]. The random variable X is said to be a Bernouilli randomvariable. Coin toss is a typical example of Bernouilli random variable with p =1/2.

Sometimes we use random variables to indicate the set of events. For example,instead of saying the set that we win first prize, we write as ω ∈Ω : X(ω) = 1,or simply X = 1.

Definition 1.5 (Probability distribution). The probability distribution function F(x)is defined by

F(x) = PX ≤ x. (1.12)

2J. Doob quoted in Statistical Science. (One of the great probabilists who established probabil-ity as a branch of mathematics.) While writing my book [Stochastic Processes] I had an argumentwith Feller. He asserted that everyone said “random variable” and I asserted that everyone said“chance variable.” We obviously had to use the same name in our books, so we decided the issueby a stochastic procedure. That is, we tossed for it and he won.

Page 14: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

14 CHAPTER 1. BASICS IN PROBABILITY THEORY

The probability distribution function fully-determines the probability structureof a random variable X . Sometimes, it is convenient to consider the probabilitydensity function instead of the probability distribution.

Definition 1.6 (probability density function). The probability density functionf (t) is defined by

f (x) =dF(x)

dx=

dPX ≤ xdx

. (1.13)

Sometimes we use dF(x) = dPX ≤ x= P(X ∈ (x,x+dx]) even when F(x)has no derivative.

Lemma 1.1. For a (good) set A,

PX ∈ A=∫

AdPX ≤ x=

∫A

f (x)dx. (1.14)

Problem 1.6. Let X be an uniformly-distributed random variable on [100,200].Then the distribution function is

F(x) = PX ≤ x= x−100100

, (1.15)

for x ∈ [100,200].

• Draw the graph of F(x).

• Find the probability function f (x).

1.5 Expectation, Variance and Standard DeviationLet X be a random variable. Then, we have some basic tools to evaluate randomvariable X . First we have the most important measure, the expectation or mean ofX .

Definition 1.7 (Expectation).

E[X ] =∫

−∞

xdPX ≤ x=∫

−∞

x f (x)dx. (1.16)

Page 15: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

1.5. EXPECTATION, VARIANCE AND STANDARD DEVIATION 15

Remark 1.4. For a discrete random variable, we can rewrite (1.16) as

E[X ] = ∑n

xnP[X = xn]. (1.17)

Lemma 1.2. Let (Xn)n=1,...,N be the sequence of possibly correlated random vari-ables. Then we can change the order of summation and the expectation.

E[X1 + · · ·+XN ] = E[X1]+ · · ·+E[XN ] (1.18)

Proof. See Exercise 1.6.

E[X ] gives you the expected value of X , but X is fluctuated around E[X ]. Sowe need to measure the strength of this stochastic fluctuation. The natural choicemay be X −E[X ]. Unfortunately, the expectation of X −E[X ] is always equal tozero (why?). Thus, we need the variance of X , which is indeed the second momentaround E[X ].

Definition 1.8 (Variance).

Var[X ] = E[(X−E[X ])2]. (1.19)

Lemma 1.3. We have an alternative to calculate Var[X ],

Var[X ] = E[X2]−E[X ]2. (1.20)

Proof. See Exercise 1.6.

Unfortunately, the variance Var[X ] has the dimension of X2. So, in some cases,it is inappropriate to use the variance. Thus, we need the standard deviation σ [X ]which has the order of X .

Definition 1.9 (Standard deviation).

σ [X ] = (Var[X ])1/2. (1.21)

Example 1.11 (Bernouilli random variable). Let X be a Bernouilli random vari-able with P[X = 1] = p and P[X = 0] = 1− p. Then we have

E[X ] = 1p+0(1− p) = p. (1.22)

Var[X ] = E[X2]−E[X ]2 = E[X ]−E[X ]2 = p(1− p), (1.23)

where we used the fact X2 = X for Bernouille random variables.

Page 16: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

16 CHAPTER 1. BASICS IN PROBABILITY THEORY

In many cases, we need to deal with two or more random variables. Whenthese random variables are independent, we are very lucky and we can get manyuseful result. Otherwise...

Definition 1.2. We say that two random variables X and Y are independent whenthe sets X ≤ x and Y ≤ y are independent for all x and y. In other words,when X and Y are independent,

P(X ≤ x,Y ≤ y) = P(X ≤ x)P(Y ≤ y) (1.24)

Lemma 1.4. For any pair of independent random variables X and Y , we have

• E[XY ] = E[X ]E[Y ].

• Var[X +Y ] =Var[X ]+Var[Y ].

Proof. Extending the definition of the expectation, we have a double integral,

E[XY ] =∫

xydP(X ≤ x,Y ≤ y).

Since X and Y are independent, we have P(X ≤ x,Y ≤ y) = P(X ≤ x)P(Y ≤ y).Thus,

E[XY ] =∫

xydP(X ≤ x)dP(Y ≤ y)

=∫

xdP(X ≤ x)∫

ydP(X ≤ y)

= E[X ]E[Y ].

Using the first part, it is easy to check the second part (see Exercise 1.9.)

Example 1.12 (Binomial random variable). Let X be a random variable with

X =n

∑i=1

Xi, (1.25)

where Xi are independent Bernouilli random variables with the mean p. The ran-dom variable X is said to be a Binomial random variable. The mean and varianceof X can be obtained easily by using Lemma 1.4 as

E[X ] = np, (1.26)Var[X ] = np(1− p). (1.27)

Problem 1.7. Let X be the number of up’s in 10 tosses.

1. Find E[X ] and Var[X ] using 1.12.

2. Find the probability measure P, and compute E[X ] using Definition 1.7.

Page 17: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

1.6. HOW TO MAKE A RANDOM VARIABLE 17

1.6 How to Make a Random VariableSuppose we would like to simulate a random variable X which has a distributionF(x). The following theorem will help us.

Theorem 1.2. Let U be a random variable which has a uniform distribution on[0,1], i.e

P[U ≤ u] = u. (1.28)

Then, the random variable X = F−1(U) has the distribution F(x).

Proof.

P[X ≤ x] = P[F−1(U)≤ x] = P[U ≤ F(x)] = F(x). (1.29)

1.7 News-vendor Problem, “How many should youbuy?”

Suppose you are assigned to sell newspapers. Every morning you buy in x newspa-pers at the price a. You can sell the newspaper at the price a+b to your customers.You should decide the number x of newspapers to buy in. If the number of thosewho buy newspaper is less than x, you will be left with piles of unsold newspa-pers. When there are more buyers than x, you lost the opportunity of selling morenewspapers. Thus, there seems to be an optimal x to maximize your profit.

Let X be the demand of newspapers, which is not known when you buy innewspapers. Suppose you buy x newspapers and check if it is profitable whenyou buy the additional ∆x newspapers. If the demand X is larger than x+∆x, theadditional newspapers will pay off and you get b∆x, but if X is smaller than x+∆x,you will lose a∆x. Thus, the expected additional profit is

E[profit from additional ∆x newspapers]= b∆xPX ≥ x+∆x−a∆xPX ≤ x+∆x= b∆x− (a+b)∆xPX ≤ x+∆x.

Whenever this is positive, you should increase the stock, thus the optimum stockx should satisfy the equilibrium equation;

PX ≤ x+∆x= ba+b

, (1.30)

Page 18: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

18 CHAPTER 1. BASICS IN PROBABILITY THEORY

for all ∆x > 0. Letting ∆x→ 0, we have

PX ≤ x= ba+b

, (1.31)

Using the distribution function F(x) = PX ≤ x and its inverse F−1, we have

x = F−1(

ba+b

). (1.32)

Using this x, we can maximize the profit of news-vendors.

Problem 1.8. Suppose you are a newspaper vender. You buy a newspaper at theprice of 70 and sell it at 100. The demand X has the following uniform distribu-tion,

PX ≤ x= x−100100

, (1.33)

for x ∈ [100,200]. Find the optimal stock for you.

1.8 Covariance and CorrelationWhen we have two or more random variables, it is natural to consider the relationof these random variables. But how? The answer is the following:

Definition 1.10 (Covariance). Let X and Y be two (possibly not independent)random variables. Define the covariance of X and Y by

Cov[X ,Y ] = E[(X−E[X ])(Y −E[Y ])]. (1.34)

Thus, the covariance measures the multiplication of the fluctuations aroundtheir mean. If the fluctuations are tends to be the same direction, we have largercovariance.

Example 1.13 (The covariance of a pair of indepnedent random variables). LetX1 and X2 be the independent random variables. The covariance of X1 and X2 is

Cov[X1,X2] = E[X1X2]−E[X1]E[X2] = 0,

Page 19: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

1.9. VALUE AT RISK 19

since X1 and X2 are independent. Thus, more generally, if the two random vari-ables are independent, their covariance is zero. (The converse is not always true.Give some example!)

Now, let Y = X1 +X2. How about the covariance of X1 and Y ?

Cov[X1,Y ] = E[X1Y ]−E[X1]E[Y ]= E[X1(X1 +X2)]−E[X1]E[X1 +X2]

= E[X21 ]−E[X1]

2

=Var[X1] = np(1− p)> 0.

Thus, the covariance of X1 and Y is positive as can be expected.

It is easy to see that we have

Cov[X ,Y ] = E[XY ]−E[X ]E[Y ], (1.35)

which is sometimes useful for calculation. Unfortunately, the covariance has theorder of XY , which is not convenience to compare the strength among differentpair of random variables. Don’t worry, we have the correlation function, which isnormalized by standard deviations.

Definition 1.11 (Correlation). Let X and Y be two (possibly not independent)random variables. Define the correlation of X and Y by

ρ[X ,Y ] =Cov[X ,Y ]σ [X ]σ [Y ]

. (1.36)

Lemma 1.5. For any pair of random variables, we have

−1≤ ρ[X ,Y ]≤ 1. (1.37)

Proof. See Exercise 1.11

1.9 Value at RiskSuppose we have one stock with its current value x0. The value of the stockfluctuates. Let X1 be the value of this stock tomorrow. The rate of return R can bedefined by

R =X1− x0

x0. (1.38)

The rate of return R can be positive or negative. We assume R is normally dis-tributed with its mean µ and σ .

Page 20: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

20 CHAPTER 1. BASICS IN PROBABILITY THEORY

Problem 1.9. Why did the rate of return R assume to be a normal random variable,instead of the stock price X1 itself.

We need to evaluate the uncertain risk of this future stock.

Definition 1.12 (Value at Risk). The future risk of a property can be evaluated byValue at Risk (VaR) zα , the decrease of the value of the property in the worst casewhich has the probability α , or

PX1− x0 ≥−zα= α, (1.39)

or

Pzα ≥ x0−X1= α. (1.40)

In short, our damage is limited to zα with the probability α .

Figure 1.1: VaR: adopted form http://www.nomura.co.jp/terms/english/v/var.html

By the definition of rate of return (1.38), we have

P

R≥−zα

x0

= α, (1.41)

or

P

R≤−zα

x0

= 1−α. (1.42)

Page 21: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

1.9. VALUE AT RISK 21

Since R is assumed to be normal random variable, using the fact that

Z =R−µ

σ, (1.43)

is a standard normal random variable, where µ = E[R], and σ =√

Var[R], wehave

1−α = P

R≤−zα

x0

= P

Z ≤ −zα/x0−µ

σ

. (1.44)

Since the distribution function of standard normal random variable is symmetric,we have

α = P

Z ≤ zα/x0 +µ

σ

(1.45)

Set xα as

α = PZ ≤ xα , (1.46)

or

xα = F−1 (α) , (1.47)

which can be found in any standard statistics text book. From (1.45) we have

zα/x0 +µ

σ= xα , (1.48)

or

zα = x0(F−1 (α)σ −µ). (1.49)

Now consider the case when we have n stocks on our portfolio. Each stockshave the rate of return at one day as,

(R1,R2, . . . ,Rn). (1.50)

Thus, the return rate of our portfolio R is estimated by,

R = c1R1 + c2R2 + · · ·+ cnRn, (1.51)

where ci is the number of stocks i in our portfolio.

Page 22: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

22 CHAPTER 1. BASICS IN PROBABILITY THEORY

Let q0 be the value of the portfolio today, and Q1 be the one for tomorrow.The value at risk (VaR) Zα of our portfolio is given by

PQ1−q0 ≥−zα= α. (1.52)

We need to evaluate E[R] and Var[R]. It is tempting to assume that R is anormal random variable with

µ = E[R] =n

∑i=1

E[Ri], (1.53)

σ2 =Var[R] =

n

∑i=1

Var[Ri]. (1.54)

This is true if R1, . . . ,Rn are independent. Generally, there may be some correla-tion among the stocks in portfolio. If we neglect it, it would cause underestimateof the risk.

We assume the vectore

(R1,R2, . . . ,Rn), (1.55)

is the multivariate normal random variable, and the estimated rate of return of ourportfolio R turns out to be a normal random variable again[Toyoizumi, 2008, p.7].

Problem 1.10. Estimate Var[R] when we have only two different stocks, i.e., R =R1 +R2, using ρ[R1,R2] defined in (1.36).

Using µ and σ of the overall rate of return R, we can evaluate the VaR Zα justlike (1.49).

1.10 References

There are many good books which useful to learn basic theory of probability.The book [Ross, 1992] is one of the most cost-effective book who wants to learnthe basic applied probability featuring Markov chains. It has a quite good styleof writing. Those who want more rigorous mathematical frame work can select[Durrett, 1991] for their starting point. If you want directly dive into the topic likestochatic integral, your choice is maybe [Oksendal, 2003].

Page 23: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

1.11. EXERCISES 23

1.11 ExercisesExercise 1.1. Find an example that our intuition leads to mistake in random phe-nomena.

Exercise 1.2. Define a probability space according to the following steps.

1. Take one random phenomena, and describe its sample space, events andprobability measure

2. Define a random variable of above phenomena

3. Derive the probability function and the probability density.

4. Give a couple of examples of set of events.

Exercise 1.3. Explain the meaning of (1.4) using Example 1.2

Exercise 1.4. Check P defined in Example 1.5 satisfies Definition 1.1.

Exercise 1.5. Calculate the both side of Example 1.8. Check that these events aredependent and explain why.

Exercise 1.6. Prove Lemma 1.2 and 1.3 using Definition 1.7.

Exercise 1.7. Prove Lemma 1.4.

Exercise 1.8. Let X be the Bernouilli random variable with its parameter p. Drawthe graph of E[X ], Var[X ], σ [X ] against p. How can you evaluate X?

Exercise 1.9. Prove Var[X +Y ] = Var[X ] +Var[Y ] for any pair of independentrandom variables X and Y .

Exercise 1.10 (Binomial random variable). Let X be a random variable with

X =n

∑i=1

Xi, (1.56)

where Xi are independent Bernouilli random variables with the mean p. The ran-dom variable X is said to be a Binomial random variable. Find the mean andvariance of X .

Exercise 1.11. Prove for any pair of random variables, we have

−1≤ ρ[X ,Y ]≤ 1. (1.57)

Page 24: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 2

Markov chain

As we saw in Chapter 1, the concepts like probability spaces and random variablesare quite useful. However, they are not enough. If you need to model physicalphenomena, you soon realize the concept of time in your model, but a randomvariable will not good for modeling time-evolution of stochastic systems.

Markov chain is a good answer to such demand!

2.1 Series of Random VariablesWait a minute, you may use a simple series of random variables instead of Markovchain...

Let X1,X2,X3, . . . be a sequence of independent and identically distributed ran-dom variables. This is the most simple model introducing ”time” in stochasticsystems.

Example 2.1. Let Xi be the result of i-th coin toss. Then, the resulted sequenceX1,X2,X3, . . . represents the coin toss evolution.

Example 2.2. Consider up and down of a stock price. Let Xi be the 1 if thestock price went up, 0 if it went down. Then, the resulted sequence X1,X2,X3, . . .represents the stock price dynamics.

2.2 Discrete-time Markov chainMarkov chain is one of the most basic tools to investigate dynamical features ofstochastic phenomena. Roughly speaking, Markov chain is used for any stochastic

24

Page 25: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

2.2. DISCRETE-TIME MARKOV CHAIN 25

processes for first-order approximation.

Definition 2.1 (Rough definition of Markov Process). A stochastic process X(t)is said to be a Markov chain, when the future dynamics depends probabilisticallyonly on the current state (or position), not depend on the past trajectory.

Example 2.3. The followings are examples of Markov chains.

• Stock price

• Brownian motion

• Queue length at ATM

• Stock quantity in storage

• Genes in genome

• Population

• Traffic on the internet

Definition 2.1 (discrete-time Markov chain). (Xn) is said to be a discrete-timeMarkov chain if

• state space is at most countable,

• state transition is only at discrete instance,

and the dynamics is probabilistically depend only on its current position, i.e.,

P[Xn+1 = xn+1|Xn = xn, ...,X1 = x1] = P[Xn+1 = xn+1|Xn = xn]. (2.1)

Definition 2.2 ((time-homogenous) 1-step transition probability).

pi j ≡ P[Xn+1 = j | Xn = i]. (2.2)

The probability that the next state is j, assuming the current state is in i.

Similarly, we can define the m-step transition probability:

pmi j ≡ P[Xn+m = j | Xn = i]. (2.3)

We can always calculate m-step transition probability by

pmi j = ∑

kpm−1

ik pk j. (2.4)

Problem 2.1. Consider coin tosses. Let Xn be the number of up’s up to n-th trial.Find the transition probability pi j.

Page 26: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

26 CHAPTER 2. MARKOV CHAIN

2.3 Time Evolution of Markov Chain

Let πi(n) be the probability that the state at time n is i, i.e.,

πi(n) = PXn = i. (2.5)

Theorem 2.1 (Time Evolution of Markov Chain). Time evolution of Markov chaincan be described by using its transition probabilities:

π j(n) = ∑i

πi(n−1)pi j. (2.6)

Problem 2.2. Use the definition of conditional probability to show Theorem 2.1.

2.4 Stationary State

The initial distribution: the probability distribution of the initial state. The initialstate can be decided on the consequence of tossing a dice...

Definition 2.3 (Stationary distribution). The probability distribution π j is said tobe a stationary distribution, when the future state probability distribution is alsoπ j if the initial distribution is π j.

PX0 = j= π j =⇒ PXn = j= π j (2.7)

In Markov chain analysis, to find the stationary distribution is quite important.If we find the stationary distribution, we almost finish the analysis.

Remark 2.1. Some Markov chain does not have the stationary distribution. Inorder to have the stationary distribution, we need Markov chains to be irreducible,positive recurrent. See [Ross, 1992, Chapter 4]. In case of finite state space,Markov chain have the stationary distribution when all state can be visited with apositive probability.

Page 27: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

2.5. MATRIX REPRESENTATION 27

2.5 Matrix Representation

Definition 2.4 (Transition (Probabitliy) Matrix).

P≡

p11 p12 . . . p1mp21 p22 . . . p2m...

... . . . ...pm1 pm2 . . . pmm

(2.8)

The matrix of the probability that a state i to another state j.

Definition 2.5 (State probability vector at time n).

π(n)≡ (π1(n),π2(n), ...) (2.9)

πi(n) = P[Xn = i] is the probability that the state is i at time n.

Theorem 2.2 (Time Evolution of Markov Chain).

π(n) = π(0)Pn (2.10)

Given the initial distribution, we can always find the probability distributionat any time in the future.

Problem 2.3. Suppose we have a transition matrix P such as

P =

(1/3 2/33/4 1/4 .

)(2.11)

Given the initial state X0 = 1, find the probability distribution of X1.

Theorem 2.3 (Stationary Distribution of Markov Chain). When a Markov chainhas the stationary distribution π , then

π = πP (2.12)

Problem 2.4. Suppose we have the transition matrix P as in Problem . Find itsstationary distribution.

Page 28: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

28 CHAPTER 2. MARKOV CHAIN

2.6 Stock Price Dynamics Evaluation Based on MarkovChain

Suppose the up and down of a stock price can be modeled by a Markov chain.There are three possibilities: (1) up, (2) down and (3) hold. The price fluctuationtomorrow depends on the todayfs movement. Assume the following transitionprobabilities:

P =

0 3/4 1/41/4 0 3/41/4 1/4 1/2

(2.13)

For example, if today’s movement is “up”, then the probability of “down” againtomorrow is 3/4.

Problem 2.5. 1. Find the steady state distribution.

2. Is it good idea to hold this stock in the long run?

Solutions:

1.

π = πP (2.14)

(π1,π2,π3) = (π1,π2,π3)

0 3/4 1/41/4 0 3/41/4 1/4 1/2

(2.15)

Using the nature of probability (π1+π2+π3 = 1), (2.15) can be solved and

(π1,π2,π3) = (1/5,7/25,13/25). (2.16)

2. Thus, you can avoid holding this stock in the long term.

Page 29: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

2.7. GOOGLE’S PAGERANK AND MARKOV CHAIN 29

2.7 Google’s PageRank and Markov ChainGoogle uses an innovative concept called PageRank1 to quantify the importanceof web pages. PageRank can be understood by Markov chain. Let us take a lookat a simple example based on [Langville and Meyer, 2006, Chapter 4].

Suppose we have 6 web pages on the internet2. Each web page has some linksto other pages as shown in Figure 2.1. For example the web page indexed by 1refers 2 and 3, and is referred back by 3. Using these link information, Google

Figure 2.1: Web pages and their links. Adopted from [Langville and Meyer, 2006,p.32]

decide the importance of web pages. Here’s how.Assume you are reading a web page 3. The web page contains 3 links to

other web pages. You will jump to one of the other pages by pure chance. Thatmeans your next page may be 2 with probability 1/3. You may hop the web pagesaccording to the above rule, or transition probability. Now your hop is governed

1PageRank is actually Page’s rank, not the rank of pages, as written inhttp://www.google.co.jp/intl/ja/press/funfacts.html. “The basis of Google’s search technol-ogy is called PageRank, and assigns an ”importance” value to each page on the web and gives ita rank to determine how useful it is. However, that’s not why it’s called PageRank. It’s actuallynamed after Google co-founder Larry Page.”

2Actually, the numbe of pages dealt by Google has reached 8.1 billion!

Page 30: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

30 CHAPTER 2. MARKOV CHAIN

by Markov chain, you can the state transition probability P as

P = (pi j) =

0 1/2 1/2 0 0 00 0 0 0 0 0

1/3 1/3 0 0 1/3 00 0 0 0 1/2 1/20 0 0 1/2 0 1/20 0 0 1 0 0

, (2.17)

where

pi j = PNext click is web page j|reading page i (2.18)

=

1

the number of links outgoing from the page i i has a link to j,

0 otherwise.(2.19)

Starting from web page 3, you hop around our web universe, and eventually youmay reach the steady state. The page rank of web page i is nothing but the steadystate probability that you are reading page i. The web page where you stay themost likely gets the highest PageRank.

Problem 2.6. Are there any flaws in this scheme? What will happen in the long-run?

When you happened to visit a web page with no outgoing link, you may jumpto a web page completely unrelated to the current web page. In this case, your nextpage is purely randomly decided. For example, the web page 2 has no outgoinglink. If you are in 2, then the next stop will be randomly selected.

Further, even though the current web page has some outgoing link, you may goto a web page completely unrelated. We should take into account such behavior.With probability 1/10, you will jump to a random page, regardless of the pagelink.

Thus the transition probability is modified, and when i has at least one outgo-ing link,

pi j =9

101

the number of links outgoing from the page i+

110

16. (2.20)

On the other hand, when i has no outgoing link, we have

pi j =16. (2.21)

Page 31: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

2.7. GOOGLE’S PAGERANK AND MARKOV CHAIN 31

The new transition probability matrix P is

P =160

1 28 28 1 1 1

10 10 10 10 10 1019 19 1 1 19 11 1 1 1 28 281 1 1 28 1 281 1 1 55 1 1

. (2.22)

Problem 2.7. Answer the followings:

1. Verify (2.22).

2. Compute π(1) using

π(1) = π(0)P, (2.23)

given that initially you are in the web page 3.

By Theorem 2.3, we can find the stationary probability π satisfying

π = πP. (2.24)

It turns out to be

π = (0.0372,0.0539,0.0415,0.375,0.205,0.286), (2.25)

As depicted in Figure 2.2, according to the stationary distribution, we can say thatweb page 4 has the best PageRank.

Page 32: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

32 CHAPTER 2. MARKOV CHAIN

Figure 2.2: State Probability of Google’s Markov chain

Page 33: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 3

Birth and Death process and PoissonProcess

As we saw in Chapter 2, we can analyze complicated system using Markov chains.Essentially, Markov chains can be analyzed by solving a matrix equation. How-ever, instead of solving matrix equations, we may find a fruitful analytical result,by using a simple variant of Markov chains.

3.1 Definition of Birth and Death ProcessBirth and death process is a special continuous-time Markov chain. The very basicof the standard queueing theory. The process allows two kinds of state transitions:

• X(t) = j→ j+1 :birth

• X(t) = j→ j−1 :death

Moreover, the process allows no twin, no death at the same instance. Thus, forexample,

Pa birth and a death at t= 0. (3.1)

Definition 3.1 (Birth and Death process). Define X(t) be a Birth and Death pro-cess with its transition rate;

• P[X(t +∆t) = j+1|X(t) = j] = λ j∆t +o(∆t)

• P[X(t +∆t) = j−1|X(t) = j] = µ j∆t +o(∆t).

33

Page 34: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

34 CHAPTER 3. BIRTH AND DEATH PROCESS AND POISSON PROCESS

3.2 Differential Equations of Birth and Death Pro-cess

The dynamics of birth and death process is described by a system of differentialequations. Using Markov property, for a sufficiently small ∆t, we have

Pj(t +∆t) = Pj(t)1− (λ j +µ j)∆t+Pj−1(t)λ j∆t +Pj+1(t)µ j+1∆t +o(∆t).(3.2)

Dividing ∆t in the both side and letting ∆t→ 0, we have

ddt

Pj(t) = λ jPj−1(t)− (λ j +µ j)Pj(t)+Pj+1(t)µ j+1, (3.3)

for j ≥ 1. For j = 0, we have

ddt

P0(t) =−(λ0)P0(t)+P1(t)µ1. (3.4)

Problem 3.1. Can you solve this system of differential equations? If not, whatkind of data do you need?

3.3 Infinitesimal GeneratorUnlike the case of discrete-time, we need the transition rate qi j for continuous-time Markov chains. However, It is also much convenient to use matrix form.

Let qi j be

q j j+1 = lim∆t→0

1∆t

PX(t +∆t) = j+1|X(t) = j= λ j, (3.5)

q j j−1 = lim∆t→0

1∆t

PX(t +∆t) = j−1|X(t) = j= µ j (3.6)

A birth and death process is described by its infinitesimal generator Q as

Q =

−λ0 λ0µ1 −(λ1 +µ1) λ1

µ2 −(λ2 +µ2) λ2. . . . . . . .

(3.7)

Note that in addition to above we need to define the initail condition, in orderto know its probabilistic behaviour.

Page 35: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

3.4. SYSTEM DYNAMICS 35

3.4 System DynamicsDefinition 3.2 (State probability). The state of the system at time t can be definedby the infinite-dimension vector:

P(t) = (P0(t),P1(t), · · ·), (3.8)

where Pk(t) = P[X(t) = k].

The dynamics of the state is described by the differential equation of matrix:

dP(t)dt

= P(t)Q. (3.9)

Formally, the differential equation can be solved by

P(t) = P(0)eQt , (3.10)

where eQt is matrix exponential defined by

eQt =∞

∑n=0

(Qt)n

n!. (3.11)

Remark 3.1. It is hard to solve the system equation, since it is indeed an infinite-dimension equation. (If you are brave to solve it, please let me know!)

3.5 Poisson ProcessDefinition 3.3 (Poisson Process). Poisson process is a specail birth and deathprocess, which has the following parameters:

• µk = 0 : No death

• λk = λ : Constant birth rate

Then, the corresponding system equation is,

dPk(t)dt

=−λPk(t)+λPk−1(t) for k ≥ 1 : internal states (3.12)

dP0(t)dt

=−λP0(t) : boundary state (3.13)

Page 36: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

36 CHAPTER 3. BIRTH AND DEATH PROCESS AND POISSON PROCESS

Also, the initial condition,

Pk(0) =

1 k = 00 otherwise,

(3.14)

which means no population initially.Now we can solve the equation by iteration.

P0(t) = e−λ t (3.15)

P1(t) = λ te−λ t (3.16)· · ·

Pk(t) =(λ t)k

k!e−λ t , (3.17)

which is Poisson distribution!

Problem 3.2. Show that (3.17) satisfies (3.12).

Theorem 3.1. The popution at time t of a constant rate pure birth process hasPoisson distribution.

3.6 Why we use Poisson process?• IT is Poisson process!

• It is EASY to use Poisson process!

Theorem 3.2 (The law of Poisson small number). Poisson process ⇔ Countingprocess of the number of independent rare events.

If we have many users who use one common system but not so often, then theinput to the system can be Poisson process.

Let us summarize Poisson process as an input to the system:

1. A(t) is the number of arrival during [0, t).

2. The probability that the number of arrival in [0, t) is k is given by

P[A(t) = k] = Pk(t) =(λ t)k

k!e−λ t .

Page 37: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

3.7. Z-TRNASFORM OF POISSON PROCESS 37

3. The mean number of arrival in [0, t) is given by

E[A(t)] = λ t, (3.18)

where λ is the arrival rate.

3.7 Z-trnasform of Poisson processZ-transform is a very useful tool to investigate stochastic processes. Here’s someexamples for Poisson process.

E[zA(t)] = e−λ t+λ tz (3.19)

E[A(t)] =ddz

E[zA(t)] |z=1= λ (3.20)

Var[A(t)] = Find it! (3.21)

3.8 Independent IncrementTheorem 3.3 (Independent Increment).

P[A(t) = k | A(s) = m] = Pk−m(t− s)

The arrivals after time s is independent of the past.

3.9 Interearrival TimeLet T be the interarrival time of Poisson process, then

P[T ≤ t] = 1−P0(t) = 1− e−λ t : Exponential distribution. (3.22)

3.10 Memory-less property of Poisson processTheorem 3.4 (Memory-less property). T: exponential distribution

P[T ≤ t + s | T > t] = P[T ≤ s] (3.23)

Page 38: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

38 CHAPTER 3. BIRTH AND DEATH PROCESS AND POISSON PROCESS

Proof.

P[T ≤ t + s | T > t] =P[t < T ≤ t + s]

P[T > t]= 1− e−λ s (3.24)

3.11 PASTA: Poisson Arrival See Time AveragePoisson Arrival will see the time average of the system. This is quite importantfor performance evaluation.

3.12 Excercise1. Find an example of Markov chains which do not have the stationary distri-

bution.

2. In the setting of section 2.6, answer the followings:

(a) Prove (2.16)

(b) When you are sure that your friend was in Aizu-wakamatsu initiallyon Monday, where do you have to go on the following Wednesday, tojoin her? Describe why.

(c) When you do not know when and where she starts, which place do youhave to go to join her? Describe why.

3. Show E[A(t)] = λ t, when A(t) is Poisson process, i.e.,

P[A(t) = k] =(λ t)k

k!e−λ t

.

4. When A(t) is Poisson process, calculate Var[A(t)], using z-transform.

5. Make the graph of Poisson process and exponential distribution, using Math-ematica.

6. When T is exponential distribution, answer the folloing;

Page 39: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

3.12. EXCERCISE 39

(a) What is the mean and variance of T ?

(b) What is the Laplace transform of T ? (E[e−sT ])

(c) Using the Laplace transform, verify your result of the mean and vari-ance.

Page 40: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 4

Introduction of Queueing Systems

4.1 Foundation of Performance Evaluation

Queueing system is the key mathematical concept for evaluation of systems. Thefeatures for the queueing systems:

1. Public: many customers share the system

2. Limitation of resources: There are not enough resources if all the customerstry to use them simultaneously.

3. Random: Uncertainty on the Customer’s behaviors

Many customers use the limited amount of resources at the same time withrandom environment. Thus, we need to estimate the performance to balance thequality of service and the resource.

Example 4.1. Here are some example of queueing systems.

• manufacturing system.

• Casher at a Starbucks coffee shop.

• Machines in Lab room.

• Internet.

40

Page 41: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

4.2. STARBUCKS 41

4.2 StarbucksSuppose you are a manager of starbucks coffee shop. You need to estimate thecustomer satisfaction of your shop. How can you observe the customer satisfac-tion? How can you improve the performance effectively and efficiently?

Problem 4.1. By the way, most Starbucks coffee shops have a pick-up station aswell as casher. Can you give some comment about the performance of the system?

4.3 Specification of queueing systemsSystem description requirements of queueing systems:

• Arrival process(or input)

• Service time

• the number of server

• service order

Let Cn be the n-th customer to the system. Assume the customer Cn arrives tothe system at time Tn. Let Xn = Tn+1−Tn be the n-th interarrival time. Supposethe customer Cn requires to the amount of time Sn to finish its service. We call Snthe service time of Cn.

We assume that both Xn and Sn are random variable with distribution functionsPXn ≤ x and PSn ≤ x.Definition 4.1. Let us define some terminologies:

• E[Xn] =1λ

: the mean interarrival time.

• E[Sn] =1µ

: the mean service time.

• ρ = λ

µ: the mean utilizations. The ratio of the input vs the service. We often

assume ρ < 1,

for the stability.

Let Wn be the waiting time of the n-th customer. Define the sojourn time Yn ofCn by

Yn =Wn +Sn. (4.1)

Problem 4.2. What is Wn and Yn in Starbucks coffee shop?

Page 42: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

42 CHAPTER 4. INTRODUCTION OF QUEUEING SYSTEMS

4.4 Little’s formulaOne of the ”must” for performance evaluation. No assumption is needed to provethis!

Definition 4.1. Here’s some more definitions:

• A(t) : the number of arrivals in [0, t).

• D(t) : the number of departures in [0, t).

• R(t) : the sum of the time spent by customer arrived before t.

• N(t): the number of customers in the system at time t.

The relation between the mean sojourn time and the mean queue length.

Theorem 4.1 (Little’s formula).

E[N(t)] = λE[Y ].

Proof. Seeing Figure 4.4, it is easy to find

A(t)

∑n=0

Yn =∫ t

0N(t)dt = R(t). (4.2)

Dividing both sides by A(t) and taking t→ ∞, we have

E[Y (t)] = limt→∞

1A(t)

A(t)

∑n=0

Yn = limt→∞

tA(t)

1t

∫ t

0N(t)dt =

E[N(t)]λ

, (4.3)

since λ = limt→∞ A(t)/t.

Example 4.2 (Starbucks coffee shop). Estimate the sojourn time of customers, Y ,at the service counter.

We don’t have to have the stopwatch to measure the arrival time and the re-ceived time of each customer. In stead, we can just count the number of ordersnot served, and observe the number of customer waiting in front of casher.

Page 43: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

4.4. LITTLE’S FORMULA 43

t

A(t)D(t)

Figure 4.1: Little’s formula

Page 44: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

44 CHAPTER 4. INTRODUCTION OF QUEUEING SYSTEMS

Then, we may find the average number of customer in the system,

E[N(t)] = 3. (4.4)

Also, by the count of all order served, we can estimate the arrival rate ofcustomer, say

λ = 100. (4.5)

Thus, using Little’s formula, we have the mean sojourn time of customers in Star-bucks coffee shop,

E[Y ] =E[N]

λ= 0.03. (4.6)

Example 4.3 (Excersize room). Estimate the number of students in the room.

• E[Y ] = 1: the average time a student spent in the room (hour).

• λ = 10: the average rate of incoming students (students/hour).

• E[N(t)] = λE[Y ] = 10: the average number of students in the room.

Example 4.4 (Toll gate). Estimate the time to pass the gate.

• E[N(t)] = 100: the average number of cars waiting (cars).

• λ = 10: the average rate of incoming cars (students/hour).

• E[Y ] = E[N]λ

= 10: the average time to pass the gate.

4.5 Lindley equation and Loynes variableHere’s the ”Newton” equation for the queueing system.

Theorem 4.2 (Lindley Equation). For a one-server queue with First-in-first-outservice discipline, the waiting time of customer can be obtained by the followingiteration:

Wn+1 = max(Wn +Sn−Xn,0). (4.7)

Page 45: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

4.6. EXERCISE 45

The Lindley equation governs the dynamics of the queueing system. Althoughit is hard to believe, sometimes the following alternative is much easier to handlethe waiting time.

Theorem 4.3 (Loyne’s variable). Given that W1 = 0, the waiting time Wn can bealso expressed by

Wn = maxj=0,1,...,n−1

n−1

∑i= j

(Si−Xi),0. (4.8)

Proof. Use induction. It is clear that W1 = 0. Assume the theorem holds for n−1.Then, by the Lindley equation,

Wn+1 = max(Wn +Sn−Xn,0)

= max

(sup

j=1,...,n−1

n−1

∑i= j

(Si−Xi),0+Sn−Xn,0

)

= max

(sup

j=1,...,n−1

n

∑i= j

(Si−Xi),Sn−Xn,0

)

= maxj=0,1,...,n

n

∑i= j

(Si−Xi),0.

4.6 Exercise1. Restaurant Management

Your friend owns a restaurant. He wants to estimate how long each customerspent in his restaurant during lunch time, and asking you to cooperate him.(Note that the average sojourn time is one of the most important index foroperating restaurants.) Your friend said he knows:

• The average number of customers in his restaurant is 10,

• The average rate of incoming customers is 15 per hour.

How do you answer your friend?

Page 46: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

46 CHAPTER 4. INTRODUCTION OF QUEUEING SYSTEMS

2. Modelling PC

Consider how you can model PCs as queueing system for estimating its per-formance. Describe what is corresponding to the following terminologiesin queueing theory.

• customers

• arrival

• service

• the number of customers in the system

3. Web site administration

You are responsible to operate a big WWW site. A bender of PC-serverproposes you two plans , which has the same cost. Which plan do youchoose and describe the reason of your choice.

• Use 10 moderate-speed servers.

• Use monster machine which is 10 times faster than the moderate one.

Page 47: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 5

PASTA

5.1 What will be seen by customersIn general, the system seen by customers at arrival is not always time average.

Let Ln be the number of customers in the system seen by the n-th arrival, andL(t) be the number of customer in the system at time t. Since the means can beapproximated by the sample average, we have

E[L(t)]∼ 1T

∫ T

0L(s)ds, (5.1)

and

E[Ln]∼1N

N

∑n=0

Ln, (5.2)

for large T and N.However, in general,

E[Ln] 6= E[L(t)], (5.3)

even for the stationary state of the system.

Example 5.1 (Pair arrival). Assume a pair of customers (always two!) arrives toa system. In addition, the pairs arrive according to a Poisson Process. Then, theevenly-indexed customers will see always +1 customers in the system comparedto oddly-indexed customers. Thus,

E[L2k] = E[L2k−1]+1 = E[L(t)]+1, (5.4)

where we used ”PASTA” explained below in the second equation.

47

Page 48: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

48 CHAPTER 5. PASTA

5.2 Stochastic IntensityLet us define so-called stochastic intensity for stationary counting processes. LetTn be the time of n-th event and A(t) be its counting process, i.e. the number ofevents in (0, t].

Definition 5.1 (Intensity of counting process). The intensity of the stationarycounting process A(t) is defined by

λ =E[A(t + s)−A(t)]

s. (5.5)

Definition 5.2 (The stochastic intensity of A(t)). The stochastic intensity of thecounting process A(t) given L(t) is defined by

λ (t) = lims→0

E[A(t + s)−A(t)|L(t−)]s

= lims→0

P[A(t + s)−A(t) = 1|L(t−)]s

. (5.6)

The stochastic intensity λ (t) is a random variable representing the intensityconditioned by the state of system just before the time t.

Theorem 5.1 (The relation between intensity and the stochastic intensity).

E[λ (t)] = λ . (5.7)

Proof.

E[λ (t)] = E[

lims→0

E[A(t + s)−A(t)|L(t−)]s

]= lim

s→0E[

E[A(t + s)−A(t)|L(t−)]s

]= lim

s→0

E[A(t + s)−A(t)]s

= λ

Example 5.2 (Stochastic intensity of the departure process of M/M/1 queue).Consider an M/M/1 queue with the arrival rate λ and the service rate µ . Then,the stochastic intensity for the departure process D(t) is

λ (t) = µ1L(t−)>0. (5.8)

Let ρ = λ/µ . Since we know that P[L(t−)> 0] = P[L(t)> 0] = ρ , we have

E[λ (t)] = µP[L(t−)> 0] = µρ = λ .

Page 49: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

5.3. LACK OF BIAS ASSUMPTION 49

5.3 Lack of Bias AssumptionLet us consider the difference between the number of customers L seen just beforethe event Tn.

Theorem 5.2 (Cross covariance Equation). P. Bremaud and Mazumdar [1992]

E[L(Tn−)]−E[L(t)] =Cov[

L(t−), λ (t)λ

]. (5.9)

Proof. For a small s

E [L(t−)|A(t, t + s] = 1] =E[L(t−)1A(t,t+s]=1

]P [A(t, t + s] = 1]

=E [L(t−)P[A(t, t + s] = 1|L(t−)]]

P [A(t, t + s] = 1].

Letting s→ 0 on both sides, we have

E [L(Tn−)] =E [L(t−)λ (t)]

λ, (5.10)

where λ = E[λ (1)] = E[A(1)] is the intensity of A(t). Thus, we have

E[L(Tn−)]−E[L(t)] =Cov[

L(t−), λ (t)λ

]. (5.11)

Thus, when Cov[L(0−),λ (0)] = 0, the event will see the time average.The condition Cov[L(0−),λ (0)] = 0 is known to be the lack of bias assump-

tion (LBA), and is intensively studied to find the processes with LBA in the con-text of ASTA (arrivals see time-average) El-Taha and S. Stidham [1999]; Melamedand Yao [1995]; Glynn et al. [1993]; Melamed and Whitt [1990]; P. Bremaud andMazumdar [1992].

5.4 Reverse Stochastic IntensityNow, we consider the reverse process. Let D(t) be the left-continuous countingprocess i.e. the number of events during [0, t). By reversing the sample path ofprocess L(t), the departure of original process are consider to be arrivals to thereversed process.

Page 50: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

50 CHAPTER 5. PASTA

Definition 5.3 (Reversed Stochastic Intensity). Define the reversed stochastic in-tensity λD(t) of D(t) by

λD(t) = lims→0

E[D(t)−D(t− s)|L(t+)]

s. (5.12)

Intuitively, λD(t) can be regarded as the stochastic intensity conditioned by thenumber of users left behind at the departure. Then by using (5.9) for this reversedprocess, we have

E[L(Dn+)]−E[L(t)] =Cov[

L(t+),λD(t)

λ

], (5.13)

since E[D(1)] = E[A(1)] = λ for stationary systems.

Remark 5.1. If the arrival process is simple and the sojourn time of each useris independent, then E[L(Dn+)] = E[L(Tn−)], and we have Cov[L(0−),λ (0)] =Cov[L(0+),λD(0)].

5.5 Poisson arrivals see time averageIt is known that Poisson arrivals see time average. We call this property PASTA(not an Italian dish though)!

Theorem 5.3. Poisson arrivals see time average of the system.

Proof. Since Poisson process lose the memory of the past, we have

λ (t) = lims→0

E[A(t + s)−A(t)|L(t−)]s

= lims→0

E[A(t + s)−A(t)]s

= λ ,

which is constant. Thus,

Cov[L(0−),λ (0)] =Cov[L(0−),λ ] = 0.

Hence, E[L(Tn−)] = E[L(t)]

Remark 5.2. It can be shown that not only the mean but the probability distributionseen by Poisson arrival and stationary distribution is equivalent.

Page 51: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

5.6. EXCERCISE 51

5.6 Excercise1. For an M/M/2 queue,

(a) Find the stochastic intensity of departure process.

(b) Verify that E[λ (t)] = λ .

Page 52: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 6

M/M/1 queue

The most important queueing process!

• Arrival: Poisson process

• Service: Exponetial distribution

• Server: One

• Waiting room (or buffer): Infinite

6.1 M/M/1 queue as a birth and death processLet N(t) be the number of customer in the system (including the one being served).N(t) is the birth and death process with,

• λk = λ

• µk = µ

P(Exactly one cusmter arrives in [t, t +∆t]) = λ∆t (6.1)P(Given at least one customer in the system,exactly one service completes in [t, t +∆t]) = µ∆t (6.2)

An M/M/1 queue is the birth and death process with costant birth rate (arrivalrate) and constant death rate (service rate).

52

Page 53: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

6.2. UTILIZATION 53

Theorem 6.1 (Steady state probability of M/M/1 queue). When ρ < 1,

pk = P[N(t) = k] = (1−ρ)ρk, (6.3)

where ρ = λ/µ .

Proof. The balance equation:

λ pk−1 = µ pk (6.4)

Solving this,

pk =λ

µpk−1 =

µ

)k

p0. (6.5)

Then, by the fact ∑ pk = 1, we have

pk = (1−ρ)ρk. (6.6)

It is easy to see,

E[N(t)] =ρ

(1−ρ). (6.7)

6.2 UtilizationThe quantity ρ is said to be a utilization of the system.

Since p0 is the probability of the system is empty,

ρ = 1− p0 (6.8)

is the probability that the em is under service.When ρ ≥ 1, the system is unstable. Indeed, the number of customers in the

system wil go to ∞.

6.3 Waiting Time EstimationFor the customer of the system the waiting time is more important than the numberof customers in the system. We have two different ways to obtain the imformationof waiting time.

Page 54: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

54 CHAPTER 6. M/M/1 QUEUE

6.3.1 Waiting Time by Little‘s FormulaLet Sn be the service time of n-th customer and Wn be his waiting time. Define thesojourn time Yn by

Yn =Wn +Sn. (6.9)

Theorem 6.2.

E[W ] =ρ/µ

1−ρ. (6.10)

Proof. By Little’s formula, we have

E[N] = λE[Y ]. (6.11)

Thus,

E[W ] = E[Y ]−E[S] =E[N]

λ− 1

µ

=ρ/µ

1−ρ.

6.3.2 Waiting Time Distribution of M/M/1 QueuesLittle‘s formula gives us the estimation of mean waiting time. But what is thedistribution?

Lemma 6.1 (Erlang distribution). Let Sii=1,...,m be a sequence of independentand identical random variables, which has the exponetial distribution with itsmean 1/µ . Let X = ∑

mi=1 Si, then we have

dPX ≤ tdt

=µ(µt)m−1

(m−1)!e−µt . (6.12)

Proof. Consider a Poisson process with the rate µ . Let A(t) be the number ofarrivals by time t. Since t < X ≤ t+h= A(t) = m−1,A(t+h) = m, we have

Pt < X ≤ t +h= PA(t) = m−1,A(t +h) = m= PA(t +h) = m | A(t) = m−1PA(t) = m−1

= (µh+o(h))(µt)m−1

(m−1)!e−µt .

Dividing both sides by h, and taking h→ 0, we have (6.12).

Page 55: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

6.4. EXAMPLE 55

Theorem 6.3. Let W be the waiting time of the M/M/1 queue. Then we have

P[W ≤ w] = 1−ρe−µ(1−ρ)w. (6.13)

Further, let V be the sojourn time (waiting time plus service time) of the M/M/1queue, then V is exponential random variable, i.e.,

P[V ≤ v] = 1− e−µ(1−ρ)v. (6.14)

Proof. By condition on the number of customers in the system N at the arrival,we have

P[W ≤ w] =∞

∑n=0

P[W ≤ w | N = n]P[N = n]

Let Si be the service time of each customer in the system at the arrival. By thelack of memory, the remaining service time of the customer in service is againexponentially distibuted. P[W ≤ w | N = n] = P[S1 + ...+ Sn ≤ w] for n > 0.Using Lemma 6.1, we have

P[W ≤ w] =∞

∑n=1

∫ w

0

µ(µt)n−1

(n−1)!e−µtdtP[N = n]+P[N = 0]

=∫ w

0

∑n=1

µ(µt)n−1

(n−1)!e−µt(1−ρ)ρndt +1−ρ

= 1−ρe−µ(1−ρ)w. (6.15)

Problem 6.1. Prove (6.14).

6.4 ExampleConsider a WWW server. Users accses the server according to Poisson processwith its mean 10 access/sec. Each access is processed by the sever according tothe time, which has the exponential distribution with its mean 0.01 sec.

• λ = 10.

• 1/µ = 0.01.

Page 56: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

56 CHAPTER 6. M/M/1 QUEUE

• ρ = λ/µ = 10×0.01 = 0.1.

Thus the system has the utilizaion 0.1 and the number of customer waiting isdistributed as,

P[N(t) = n] = (1−ρ)ρn = 0.9×0.1n (6.16)

E[N(t)] =ρ

(1−ρ)= 1/9. (6.17)

6.5 Excercise1. For an M/M/1 queue,

(a) Calculate the mean and variance of N(t).

(b) Show the graph of E[N(t)] as a function of ρ . What can you say whenyou see the graph?

(c) Show the graph of P[W ≤ w] with different set of (λ ,µ). What canyou say when you see the graph?

2. In Example 6.4, consider the case when the access rate is 50/sec

(a) Calculate the mean waiting time.

(b) Draw the graph of the distribution of N(t).

(c) Draw the graph of the distribution of W .

(d) If you consider 10 seconds is the muximum time to wait for the WWWaccess, how much access can you accept?

3. Show the equation (6.15).

Page 57: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 7

Reversibility

7.1 Output from QueueWhat is output from a queue?

To consider a network of queues, this is very important question, since theoutput from a queue is indeed the input for the next queue.

7.2 ReversibibilityReversibility of stochastic process is useful to consider the output for the system.If the process is reversible, we may reverse the system to get the output.

7.2.1 Definition of ReversibilityDefinition 7.1. X(t) is said to be reversible when (X(t1),X(t2), ...,X(tm)) has thesame distribution as (X(τ− t1), ...,X(τ− tm)) for all τ and for all t1, ..., tm.

The reverse of the process is stochatically same as the original process.

7.2.2 Local Balance for Markov ChainTransition rate from i to j :

qi j = limτ→0

P[X(t + τ) = j | X(t) = i]τ

(i 6= j) (7.1)

qii = 0 (7.2)

57

Page 58: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

58 CHAPTER 7. REVERSIBILITY

We may have two balance equations for Markov chain.

1. Global balance equation:

∑j

piqi j = ∑j

p jq ji. (7.3)

2. Local balance equation:piqi j = p jq ji. (7.4)

Theorem 7.1 (Local Balance and Revesibility). A stationary Markov chain isreversible, if and only if there exists a sequence pi with

∑i

pi = 1 (7.5)

piqi j = p jq ji (7.6)

Proof. 1. Assume the process is reverisble, then we have

P[X(t) = i,X(t + τ) = j] = P[X(t) = j,X(t + τ) = i].

Using Bayse theorem and taking τ → 0, we have

piqi j = p jq ji.

2. Assume there exists pi satisfying (7.5) and (7.6). Since the process isMarkov, we have

P[X(u)≡ i for u ∈ [0, t),X(u)≡ j for u ∈ [t, t + s)] = pie−qitqi je−q js,

where qi = ∑ j qi j.

Using the local balance, we have

pie−qitqi je−q js = p je−qitq jie−q js.

Thus

P[X(u)≡ i for u ∈ [0, t),X(u)≡ j for u ∈ [t, t + s)]= P[X(u)≡ i for u ∈ [−t,0),X(u)≡ j for u ∈ [−t− s,−t)].

Page 59: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

7.3. OUTPUT FROM M/M/1 QUEUE 59

7.3 Output from M/M/1 queueTheorem 7.2 (Burke’s Theorem). If ρ = λ/µ < 1, the output from an equilibiumM/M/1 queue is a Poisson process.

Proof. Let N(t) be the number of the customers in the system at time t. The pointprocess of upward changes of N(t) corresponds to the arrival process of the queue,which is Poisson. Recall the transition rate of N(t) of the M/M/1 queue satisfies

λPn−1 = µPn, (7.7)

which is the local balance eqaution. Thus, by Theorem 7.1 N(t) is reversibleand N(−t) is stochastically identical to N(t). Thus the upward changes of N(t)and N(−t) are stochastically identical. In fact, the upward chang of N(−t) cor-responds to the departures of the M/M/1 queue. Thus, the departure process isPoisson.

7.4 Excercise1. If you reverse an M/M/1 queue, what is corresponding to the waiting time

of n-th customer in the original M/M/1? (You may make the additionalassumption regarding to the server, if necessary.)

2. Consider two queues, whose service time is exponetially distributed withtheir mean 1/µ . The input to one queue is Poisson process with rate λ , andits output is the input to other queue. What is the average total sojourn timeof two queues?

3. Can you say that M/M/2 is reversible? If so, explain why.

4. Can you say that M/M/m/m is reversible? If so, explain why. (You shouldclearly define the output from the system.)

Page 60: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 8

Network of Queues

8.1 Open Network of queuesWe will consider an open Markovian-type network of queues (Jackson network).

Specifications:

• M: the number of nodes (queues) in the network.

• Source and sink : One for entire network.

• i :index of node.

• ri j: routing probability from node i to j (Bernouilli trial).

• Open network : allow the arrival and departure from outside the network.

– ris: routing probability from node i to sink (leaving the network).

– rsi: probability of arrival to i from outside (arrival to the network).

• The arrival form the outside : Poisson process (λ )

• The service time of the each node: exponential (µ)

8.2 Global Balance of Network• (N∪0)M :State space (M-dimensional).

• n = (n1, ...,nM): the vector of the number customers in each node.

60

Page 61: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

8.3. TRAFFIC EQUATION 61

• 1i = (0, ...,0,i1,0, ...,0)

• p(n): the probability of the number customers in the network is n.

The global balance equation of Network:

Theorem 8.1 (Global Balance Equation of Jackson Network). In the steady stateof a Jackson network, we have

[λ +M

∑i=1

µi]p(n) =M

∑i=1

λ rsi p(n−1i)+M

∑i=1

µiris p(n+1i)+M

∑i=1

M

∑j=1

µ jr ji p(n+1 j−1i).

(8.1)

Proof. The left-hand side correspond to leaving the state n with an arrival or de-parture. Each terms on the right correspond to entering the state n with an ar-rival, departure and routing. Thus, the both side should be balanced in the steadystate.

8.3 Traffic EquationLet Λi be the average output (throughput) of a queue i.

Theorem 8.2 (Traffic Equations).

Λi = λ rsi +M

∑j=1

r jiΛ j (8.2)

Proof. Consider the average input and output from a queue i. The output from ishould be equal to the internal and external input to i.

Remark 8.1. The average output (Λi) can be obtained by solving the system ofequations (8.2).

8.4 Product Form SolutionTheorem 8.3 (The steady state probabilities of Jackson network). The solution ofthe global balance equation (8.1):

Page 62: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

62 CHAPTER 8. NETWORK OF QUEUES

p(n) =M

∏i=1

pi(ni), (8.3)

andpi(ni) = (1−ρi)ρ

nii , (8.4)

where ρi = Λi/µi.

Proof. First we will show pi, which satisfy the ”local balance”-like equation

Λi p(n−1i) = µi p(n), (8.5)

also satisfyies (8.1). Rearranging (8.2), we have

λ rsi = Λi−M

∑j=1

r jiΛ j. (8.6)

Use this to eliminate λ rsi from (8.1), then we have

[λ +M

∑i=1

µi]p(n) =M

∑i=1

Λi p(n−1i)−M

∑i=1

M

∑j=1

r jiΛi p(n−1i)

+M

∑i=1

µiris p(n+1i)+M

∑i=1

M

∑j=1

µ jr ji p(n+1 j−1i).

Using (8.5), we have

λ =M

∑i=1

risΛi, (8.7)

which is true because the net flow into the network is equal to the net flow outof the network. Thus, a solution satisfying the local balance equation will beautomatically the solution of the global balance equation. By solving the localbalance equation, we have

p(n) =Λi

µip(n−1i)

= (Λi/µi)ni p(n1, ...,0, ...,nM).

Page 63: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

8.5. EXERCISES 63

Using this argument for different i, we have

p(n) =M

∏i=1

(Λi/µi)ni p(0).

Taking summation over n and find p(0) to be the product of

pi(0) = 1− Λi

µi. (8.8)

Remark 8.2. The steady state probability of Jackson network is the product of thesteady state probability of M/M/1 queues.

8.5 Exercises1. Consider two queues in the network.

• λ : external arrival rate, µi: the service rate of the node i.

• rs1 = 1, rs2 = 0

• r11 = 0, r12 = 1, r1s = 0

• r21 = 3/8, r22 = 2/8, r2s = 3/8

(a) Estimate the throughputs Λi for each queue.

(b) Estimate the steady state probability of the network.

(c) Draw the 3-D graph of the steady state probability for λ = 1 and µ1 = 3and µ2 = 4.

(d) Estimate the mean sojourn time of the network of (1c) by using Little’sLaw.

Page 64: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 9

Examples of Queueing SystemComparison

Problem 9.1. Which is better,

1. one fast server or two moderate servers?

2. tandem servers or parallel servers?

3. independent servers or joint servers?

4. Starbucks or Doutor Coffee?

9.1 Single Server vs Tandem Servers

Example 9.1 (Starbucks vs Doutor Coffee). Suppose we have a stream of cus-tomer arrival as Poisson process with the rate λ to a coffee shop.

At Doutor Coffee, they take ordering, payment and coffee serving at the sametime. Assume it will take an exponentially-distributed service time with its mean1/µ . On the other hand, Starbucks coffee separate the ordering and coffee service.Assume both will take independent exponentially-distributed service times withits mean 1/2µ . Thus, the mean overall service time in both coffee shop is sameand equal to 1/µ .

Problem 9.2. Which coffee shop is better?

64

Page 65: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

9.2. M/M/2 QUEUE 65

At Doutor Coffee, we have only one M/M/1 queue, and as we saw in Chapter6 the expected number of customer in line including the one serving is obtainedby

E[NDoutor] =ρ

1−ρ, (9.1)

where ρ = λ/µ as usual. At Starbucks, we have two queues N1 at the orderingarea, and N2 at the serving area. This can be modeled by a tandem queue. ByTheorem 7.2, we know that the output of M/M/1 queue ordering area is againPoisson process with the rate λ . Thus, the second queue at the serving area isagain modeled by M/M/1 queue. Since the service rate of both queue is 2µ , wehave the expected number of customers at Starbucks as

E[NStarbucks] = E[N1]+E[N2]

=λ/2µ

1−λ/2µ+

λ/2µ

1−λ/2µ

=2ρ

2−ρ. (9.2)

Consequently,

E[NDoutor] =ρ

1−ρ>

2−ρ= E[NStarbucks], (9.3)

and by Little’s formula, the expected sojourn time is obtained by

E[YDoutor]> E[YStarbucks]. (9.4)

Problem 9.3. Give an intuitive reason of this consequence.

9.2 M/M/2 queueSpecifications:

• Arrival: Poisson process

• Service: Exponential distribution

• Server: Two

Page 66: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

66 CHAPTER 9. EXAMPLES OF QUEUEING SYSTEM COMPARISON

• Waiting room (or buffer): Infinite

Birth and death coefficients:

λk = λ (9.5)

µk =

µ k = 12µ k ≥ 2

(9.6)

The balance equations for stationary state probability:

−λP0 +µP1 = 0 (9.7)−(λ +µ)P1 +λP0 +2µP2 = 0 (9.8)

−(λ +2µ)Pk +λPk−1 +2µPk+1 = 0 ( k ≥ 2), (9.9)

or

µP1 = λP0 (9.10)2µPk = λPk−1 (k ≥ 1). (9.11)

Solving the equations inductively:

pk = 2p0

)k

for k ≥ 1 (9.12)

Using Σpk = 1, we have

p0 =1− λ

1+ λ

(9.13)

The average number of customers in the system:

E[N(t)] =λ/µ

1− (λ/2µ)2 (9.14)

9.3 M/M/1 VS M/M/2

Consider two servers (A1,A2) and one server (B) which is two-times faster thanthe two servers. So the average power of M/M/2 (A1,A2) and M/M/1 (B) areequivalent.

Page 67: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

9.4. TWO M/M/1 VS ONE M/M/2 67

• λ :The mean arrival rate for both queues.

• 2/µ :The mean service time of the servers Ai.

• 1/µ :The mean service time of the servers B.

Set ρ = λ/µ .The mean queue lengths:

• E[NM/M/2] =2ρ

1−ρ2

• E[NM/M/1] =ρ

1−ρ

It is easy to see

E[NM/M/2] =2ρ

1−ρ2 >ρ

1−ρ= E[NM/M/1]. (9.15)

Thus, by using Little’s formula, the sojourn time of the system (delay) can beevaluated by:

E[YM/M/2]> E[YM/M/1]. (9.16)

Problem 9.4. Give an intuitive reason of this consequence.

9.4 Two M/M/1 VS One M/M/2

Consider two systems which have the same arrival rete:

1. Two independent M/M/1 queue.

2. One M/M/2 queue, i.e sharing two servers forming one queue.

Four servers have the same speed.

• λ :The mean arrival rate for both systems.

• 1/µ :The mean service time of the servers.

Set ρ = λ/(2µ).

E[NM/M/2] =ρ

1−ρ2/4<

ρ

1−ρ/2= E[NM/M/1]+E[NM/M/1]. (9.17)

Thus, by using Little’s formula, the sojourn time of the system (delay) can beevaluated by:

E[YM/M/2]< E[Y2(M/M/1)]. (9.18)

Page 68: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

68 CHAPTER 9. EXAMPLES OF QUEUEING SYSTEM COMPARISON

9.5 Exercises1. In the setting of Section 9.3:

(a) Draw the graph of the steady state probabilities of M/M/1 and M/M/2vs ρ .

(b) Draw the graph of the mean sojourn time of M/M/1 and M/M/2 vsρ .

(c) What can you say about when ρ → 1?

2. Web site administration (revisited) You are responsible to operate a bigWWW site. A bender of PC-server proposes you two plans , which hasthe same cost. Which plan do you choose and describe the reason of yourchoice.

• Use 2 moderate-speed servers.

• Use monster machine which is 2 times faster than the moderate one.

3. Web site administration (extended) You are responsible to operate a bigWWW site. Your are proposed two plans , which has the same cost. Whichplan do you choose and describe the reason of your choice.

• Use 2 moderate-speed servers and hook them to different ISPs.

• Use 2 moderate-speed servers and hook them to single ISP, using aload-balancer. (Assume the Load-balancer ideally distribute the loadto each server.)

Page 69: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 10

Stochastic Integral

In basic mathematical financial theory, the stock price dynamic is assumed to bea brownian motion on continuous time. We need to extend the notion of integralto stochastic integral based on brownian motions, which is quite different withnormal (Riemann) integral.

10.1 Diffusion ProcessDiffusion process is a building block of stochastic integral.

Definition 10.1 (Diffusion process). A stochastic process X(t) is said to be adiffusion process if

X(t +∆t)−X(t) = µ(t,X(t))∆t +σ(t,X(t))Z(t). (10.1)

Here, Z(t) is independent normally distributed disturbance term, µ is drift term,and σ is so-called diffusion term.

Definition 10.2 (Wiener process). A stochastic process W (t) is said to be a Wienerprocess if

1. W (0) = 0.

2. Independent increment, i.e., for r < s≤ t < u

W (u)−W (t),W (s)−W (r) (10.2)

are independent.

69

Page 70: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

70 CHAPTER 10. STOCHASTIC INTEGRAL

3. For s < t, W (t)−W (s) is normally distributed as N[0, t− s].

4. W(t) has a continuous trajectory.

Remark 10.1. Sometimes, Wiener process is called by Brownian motion.

Using the Wiener process W (t) in (10.1), we can rewrite it as

X(t +∆t)−X(t) = µ(t,X(t))∆t +σ(t,X(t))∆W (t), (10.3)

where ∆W (t) =W (t+∆t)−W (t). Dividing both sides with ∆t and letting ∆t→ 0,we have

dX(t)dt

= µ +σv(t). (10.4)

Here,

v(t) =dW (t)

dt. (10.5)

Remark 10.2. The process v(t) cannot be exist, while the Wiener process W (t)exists mathematically. Indeed W (t) cannot be differentiable almost everywhere.

Instead of dividing both sides with ∆t, just take ∆t→ 0 in (10.1), then we havedX(t) = µdt +σdW (t),X(0) = a.

(10.6)

Integrating (10.28) over [0, t], we have

X(t) = a+∫ t

0µds+

∫ t

0σdW (s). (10.7)

The term∫ t

0 µds is normal integral, so it is OK. However, we have a problem forthe term

∫ t0 σdW (s), which cannot be defined by ordinary integral.

Thus, in the following section, we need to study the integral like∫ t

0g(s)dW (s). (10.8)

Page 71: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

10.2. INFORMATION 71

10.2 InformationDefinition 10.3 (Information of X(t)). Define F X

t by the information generatedby X(s) on s ∈ [0, t]. Roughly speaking, F X

t is “what has happened to X(s) up totime t.”

When we can decide an event A happened or not by the the trajectory of X(t),we say that A is F X

t -measurable, and

A ∈F Xt . (10.9)

When a random variable Z can be completely determined by the trajectory of X(t),we write

Z ∈F Xt . (10.10)

Definition 10.4. A stochastic Y is said to be adopted to the filtration F Xt when

Y (t) ∈F Xt , (10.11)

for all t.

Example 10.1. These are examples of information.

1.

A = X(s)≤ 3.14 for all s≤ 9. (10.12)

Then, A ∈F X9 .

2.

A = X(10)> 8. (10.13)

Then, A ∈F X10, but A 6∈F X

9 .

3.

Z =∫ 5

0X(s)ds. (10.14)

Then, Z ∈F X5 .

Page 72: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

72 CHAPTER 10. STOCHASTIC INTEGRAL

4. Let W be the Wiener process, and

X(t) = sups≤t

W (s). (10.15)

Then, X is adopted to F Xt . However, when

Y (t) = sups≤t+1

W (s), (10.16)

Y is not adopted to F Xt .

10.3 Definition of Stochastic IntegralDefinition 10.5.

g ∈L 2[a,b], (10.17)

when

1.∫ b

a E[g(s)2]ds < ∞,

2. g is adopted FWt .

Our objective is to define the stochastic integral∫ b

a g(s)dW (s) on g∈L 2[a,b].Assume g is simple, i.e.,

g(s) = g(tk) for s ∈ [tk, tk+1). (10.18)

Then, the stochastic integral can be defined by

∫ b

ag(s)dW (s) =

n−1

∑k=0

g(tk)[W (tk+1−W (tk)]. (10.19)

Remark 10.3. Note that this stochastic integral is forward incremental.

Problem 10.1. Prove g ∈L 2[a,b].

When g is not simple, we will proceed as follows:

1. Approximate g with simple functions gn.

Page 73: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

10.3. DEFINITION OF STOCHASTIC INTEGRAL 73

2. Take n→ ∞ in ∫ b

agn(s)dW (s)→

∫ b

ag(s)dW (s). (10.20)

Theorem 10.1. Here are some important properties for our stochastic integral.When g ∈L 2, we have

1.

E[∫ b

ag(s)dW (s)

]= 0, (10.21)

2.

E

[(∫ b

ag(s)dW (s)

)2]=∫ b

aE[g2(s)]ds, (10.22)

3. ∫ b

ag(s)dW (s):F X

b -measurable. (10.23)

Proof. We prove the case only when g(t) is simple. Set g(t) = A1[a,b] with A ∈F X

a . Since A is independent with the future W (b)−W (a), we have

E[∫ b

ag(s)dW (s)

]= E[A(W (b)−W (a))]

= E[A]E[W (b)−W (a)] = 0.

Similarly, we have

E

[(∫ b

ag(s)dW (s)

)2]= E[A2(W (b)−W (a))2]

= E[A2]E[(W (b)−W (a))2]

= E[A2]t

=∫ b

aE[g2(s)]ds.

We can prove the general case using the approximation with simple functions.

Page 74: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

74 CHAPTER 10. STOCHASTIC INTEGRAL

10.4 MartingaleDefinition 10.6. Given a filtration Ft (the history observed up time t), a stochasticprocess Xt is said to be martingale, when

1. Xt is adopted to Ft ,

2. E[|X(t)|] for all t,

3. for all s≤ t,

E[X(t)|Ft ] = X(s). (10.24)

Theorem 10.2. Every stochastic integral is martingale. More precisely,

X(t) =∫ t

0g(s)dW (s), (10.25)

is an FWt -martingale.

Theorem 10.3. A stochastic process X(t) is martingale if and only if

dX(t) = g(t)dW (t), (10.26)

which means X(t) has no dt-term.

10.5 Calculus of Stochastic IntegralWe consider a stochastic process X(t) satisfying following stochastic integral:

X(t) = a+∫ t

0µ(s)ds+

∫ t

0σ(s)dW (s). (10.27)

Equivalently, we write dX(t) = µ(t)dt +σ(t)dW (t),X(0) = a.

(10.28)

(10.28) is called stochastic differential equation.

Remark 10.4. We can see the infinitesimal dynamics more easily in the stochasticdifferential equations.

Page 75: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

10.5. CALCULUS OF STOCHASTIC INTEGRAL 75

Remark 10.5. Wiener process W (t) is continuous but perfectly rugged. Thus, weneed unusual Ito-calculus.

Since W (t)−W (s) is N[0, t− s],

E[∆W ] = 0, (10.29)

E[(∆W )2] = ∆t, (10.30)Var[∆W ] = ∆t, (10.31)

Var[(∆W )2] = 2(∆t)2. (10.32)

For small ∆t, Var[(∆W )2] rapidly vanish, so deterministically we have

(∆W )2 ∼ ∆t. (10.33)

Thus, we can set

(dW )2 = dt, (10.34)

or ∫ t

0(dW )2 = t. (10.35)

Remark 10.6. Intuitive proof can be found in text Bjork [2004] p26.

Theorem 10.4 (Ito’s formula). Given a stochastic process X(t) with

dX(t) = µdt +σdW (t), (10.36)

set

Z(t) = f (t,X(t)). (10.37)

Then, we have the following change of variables rule:

dZ(t) = d f (t,X(t)) (10.38)

=

∂ f∂ t

+µ∂ f∂x

+12

σ2 ∂ 2 f

∂x2

dt +σ

∂ f∂x

dW (t), (10.39)

or more conveniently,

d f =∂ f∂ t

dt +∂ f∂x

dX +12

∂ 2 f∂x2 (dX)2. (10.40)

Page 76: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

76 CHAPTER 10. STOCHASTIC INTEGRAL

Roughly speaking, Ito’s formula reveals that the dynamics of a function ofstochastic process depends not only on the first order but also the second order ofthe underlying stochastic process.

Proof. Take Taylor expansion of f ,

d f =∂ f∂ t

dt +∂ f∂x

dX +12

∂ 2 f∂x2 (dX)2 +

12

∂ 2 f∂ t2 (dt)2 +

∂ 2 f∂x∂ t

dtdX . (10.41)

Note that the the term (dX)2 should be ignored in normal differential, but instochastic differential this terms survive as we see in the following.

By (10.36), formally,

(dX)2 = µ2(dt)2 +2µσdtdW +σ

2(dW )2. (10.42)

Now we can igonre those terms with (dt)2 and dtdW which converges much fasterthan dt. Also, by (10.34), (dW )2 = dt. Substituting these, we have the result.

Lemma 10.1. You can use the following conventions:

(dt)2 = 0, (10.43)dtdW = 0, (10.44)

(dW )2 = dt. (10.45)

Theorem 10.5 (n-dimensional Ito’s formula). Given an n-dimensional stochasticprocess

X(t) = (X1(t),X2(t), . . . ,Xn(t)), (10.46)

with n-dimensional diffusion equation,

dX(t) = µdt +σdW (t), (10.47)

set

Z(t) = f (t,X(t)). (10.48)

Then, we have the following change of variables rule:

d f =∂ f∂ t

dt +∑i

∂ f∂xi

dXi(t)+12 ∑

i, j

∂ 2 f∂xix j

dXi(t)dX j(t). (10.49)

Page 77: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

10.5. CALCULUS OF STOCHASTIC INTEGRAL 77

Corollary 10.1. Let X(t) and Y (t) be diffusion processes, then

d(X(t)Y (t)) = X(t)dY (t)+Y (t)dX(t)+dX(t)dY (t). (10.50)

Proof. Use Ito’s formula (Theorem 10.5) and

dWi(t)dWj(t) = δi jdt, (10.51)dWidt = 0. (10.52)

Page 78: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 11

Examples of Stochastic Integral

For pricing via arbitrage theory, we need to evaluate the expectation of Wienerprocess. We can use Ito’s formula for the evaluation.

11.1 Evaluation of E[W (t)4]

We already know that the mean and variance of W (t) by definition, i.e.,

E[W (t)] = 0, (11.1)Var[W (t)] = t. (11.2)

But how about the higher moments?

Theorem 11.1. Let W (t) be a Wiener process, then

E[W (t)4] = 3t2. (11.3)

Proof. Let f (t,x) = x4 and

Z(t) = f (t,W (t)) =W (t)4. (11.4)

In Ito’s formula (Theorem 10.4), we can set µ = 0 and σ = 1 and,

dX(t) = 0dt +1dW (t) = dW (t). (11.5)

Then,

dZ(t) =∂ f∂ t

dt +∂ f∂x

dX +12

∂ 2 f∂x2 (dX)2. (11.6)

78

Page 79: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

11.1. EVALUATION OF E[W (T )4] 79

Check∂ f∂ t

= 0, (11.7)

∂ f∂x

= 4x3, (11.8)

∂ 2 f∂x2 = 12x2, (11.9)

∂ 2 f∂ t2 = 0, (11.10)

and we have

dZ(t) = 6W (t)2dW (t)2 +4W (t)3dW (t). (11.11)

Since dW (t)2 = dt by Lemma 10.1, we can rewrite the dynamics for Z(t) =W (t)4

as d(W (t)4) = 6W (t)2dt +4W (t)3dW (t),Z(0) = 0.

(11.12)

Integrating the both side of (11.12), we have

W (t)4 = 6∫ t

0W (s)2ds+4

∫ t

0W (s)3dW (s). (11.13)

Taking the expectation, we have

E[W (t)4] = 6E[∫ t

0W (s)2ds

]+4E

[∫ t

0W (s)3dW (s)

](11.14)

= 6∫ t

0E[W (s)2]ds+4E

[∫ t

0W (s)3dW (s)

]. (11.15)

Since W (s) is N[0,√

s], we have

E[W (s)2]= s. (11.16)

Also, by Theorem 10.1,

E[∫ t

0W (s)3dW (s)

]= 0. (11.17)

Note that we used the fact W (s)3 ∈L 2. Now we can evaluate (11.14).

E[W (t)4] = 6∫ t

0sds = 3t2. (11.18)

Page 80: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

80 CHAPTER 11. EXAMPLES OF STOCHASTIC INTEGRAL

11.2 Evaluation of E[eαW (t)]

Theorem 11.2. Let W (t) be a Wiener process, then

E[eαW (t)] = eα22 t . (11.19)

Proof. Set Z(t) = eαW (t). Then, by Ito’s forumula,

dZ(t) =12

α2eαW (t)dt +αeαW (t)dW (t). (11.20)

Problem 11.1. Show the above equation by using Ito’s formula.

Rewriting this, we have the stochastic differential equation,

dZ(t) =12

α2Z(t)dt +αZ(t)dW (t), (11.21)

Z(0) = 1. (11.22)

In integral form, we have

Z(t) = 1+12

α2∫ t

0Z(s)ds+α

∫ t

0Z(s)dW (s). (11.23)

Since E[∫ t

0 Z(s)dW (s)] = 0, taking the expectation, we have

m(t) = E[Z(t)] = 1+12

α2∫ t

0m(s)ds. (11.24)

Or equivalently,

m′(t) =12

α2m(t), (11.25)

m(0) = 1. (11.26)

By solving this, we have

E[eαW (t)] = m(t) = eα2t/2. (11.27)

Page 81: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 12

Differential Equations

12.1 Ordinary Differential EquationTheorem 12.1. Consider the following ordinary differential equation,

dB(t)dt

= rB(t), (12.1)

B(0) = B0. (12.2)

The solution of this equation is given by

B(t) = B0ert . (12.3)

Proof. By reordering (12.1), we have

dB(t)B(t)

= rdt. (12.4)

Integrating the both sides, we have∫ t

0

dB(s)B(s)

=∫ t

0rds. (12.5)

Thus,

logB(t) = rt +C, (12.6)

or

B(t) =Cert . (12.7)

81

Page 82: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

82 CHAPTER 12. DIFFERENTIAL EQUATIONS

Using the initial condition (12.2), we finally have

B(t) = B0ert . (12.8)

12.2 Geometric Brownian MotionDefinition 12.1 (Geometric Brownian Motion). A stochastic process X(t) satis-fying the following dynamics is called geometric brownian motion.

dX(t) = αX(t)dt +σX(t)dW (t), (12.9)X(0) = x0, (12.10)

where α is called the drift and σ is called the volatility.

Theorem 12.2 (SDE of Geometric Brownian Motion). The solution of the equa-tion

dX(t) = αX(t)dt +σX(t)dW (t), (12.11)X(0) = x0, (12.12)

is given by

X(t) = x0e(α−σ2/2)t+σW (t). (12.13)

Proof. Here the tricks we used in Theorem 12.1 may not work here because dW (t)terms should be considered as the stochastic difference.

To simplify the proof, we assume X(t)> 0. Set

Z(t) = f (t,X(t)) = logX(t). (12.14)

Then by Ito’s formula (Theorem 10.4), we have

dZ(t) =∂ f∂ t

dt +∂ f∂x

dX(t)+12

∂ 2 f∂x2 (dX(t))2

=dX(t)X(t)

− 12(dX(t))2

X(t)2 . (12.15)

Page 83: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

12.3. STOCHASTIC PROCESS AND PARTIAL DIFFERENTIAL EQUATION83

By (12.11),

(dX(t))2 = α2X(t)2(dt)2 +2σX(t)dW (t)αX(t)dt +σ

2X(t)2(dW (t))2

= σ2X(t)2dt. (12.16)

Using this in (12.15) as well as (12.11), we have

dZ(t) = αdt +σdW (t)− 12

σ2dt. (12.17)

Luckily, the right hand side can be integrated directly here!

Z(t)−Z(0) =∫ t

0(α− 1

2)ds+σ

∫ t

0dW (s) (12.18)

= (α− 12

σ2)t +σdW (t). (12.19)

Using the initial condition (12.12), we have Z(0) = logx0. Thus,

Z(t) = (α− 12

σ2)t +σdW (t)+ logx0, (12.20)

or

X(t) = eZ(t) = x0e(α−σ2/2)t+σW (t). (12.21)

Remark 12.1. This proof has not prove the existence of the solution yet. Morerigorous treatment of the proof, see the textbook like Oksendal [2003].

12.3 Stochastic Process and Partial Differential Equa-tion

Theorem 12.3 (Feynman-Kac Represenation). Consider the following equationof a function F = F(t,x):

∂F∂ t

+µ∂F∂x

+12

σ2 ∂ 2F

∂x2 − rF = 0, (12.22)

Page 84: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

84 CHAPTER 12. DIFFERENTIAL EQUATIONS

with its boundary condition,

F(T,x) = Φ(x). (12.23)

In addition, consider the following stochastic differential equation,

dX(t) = σdt +σdW (t). (12.24)

Then, the solution of (12.22) has the following Feynman-Kac representation:

F(t,x) = e−r(T−t)E[Φ(XT )|X(t) = x]. (12.25)

Page 85: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 13

Portfolio Dynamics

13.1 Portfolio ModelTheorem 13.1. Assume there is a market including N different stocks. Considerwe have a portfolio consisted with these N stocks. Let Si(t) be the price of stock i.Let hi(t) be the number of stocks in our portfolio at time t. Then, the value of ourportfolio V (t) = S(t)h(t) has the following dynamics:

dV (t) = h(t)dS(t)− c(t)dt, (13.1)

which can be superficially understood as the change of value of our portfolio aredue to the change of the stock price and the consumption we made. Particularly,when c(t) = 0,

dV (t) = h(t)dS(t). (13.2)

Proof. Analyze the dynamics of the value of our portfolio V (t). The value of ourportfolio at just before t, V (t−∆t), can be estimated by

V (t−∆t) = h(t−∆t)S(t) =N

∑i=1

hi(t−∆t)Si(t), (13.3)

for a small ∆t.At time t, within our budget V (t−∆t), we will make change of our portfolio

from h(t−∆t) to h(t) based on the observation of the stock price S(t). When weallow to consume a part of our portfolio, we have

V (t−∆t) = h(t)S(t)+ c(t)∆t, (13.4)

85

Page 86: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

86 CHAPTER 13. PORTFOLIO DYNAMICS

where c(t) is the consumption rate at time t. Thus, we have the budget equationas,

S(t)(h(t)−h(t−∆t))+ c(t)∆t = 0. (13.5)

Remark 13.1. It is wrong to conclude

S(t)dh(t)+ c(t)dt = 0, (13.6)

by letting ∆t → 0 in (13.5). Our stochastic calculus needs to have the forwarddifference as,

S(t)(h(t +∆t)−h(t))→ S(t)dh(t). (13.7)

We will rewrite (13.5) by subtracting the term S(t−∆t)(h(t)−h(t−∆t)), as

(S(t)−S(t−∆t))(h(t)−h(t−∆t))+S(t−∆t)(h(t)−h(t−∆t))+ c(t)∆t = 0.

This has the appropriate forms of (present value) × (future difference). Thus, wecan take the limit to have the proper budget equation as,

dS(t)dh(t)+S(t)dh(t)+ c(t)dt = 0. (13.8)

On the other hand, we can evaluate the current value of our portfolio as

V (t) = S(t)h(t). (13.9)

Using Ito’s formula (see Corollary 10.1) on this, we have

dV (t) = h(t)dS(t)+Sdh(t)+dS(t)dh(t). (13.10)

By (13.8),

dV (t) = h(t)dS(t)− c(t)dt, (13.11)

which can be superficially understood as the change of value of our portfolio aredue to the change of the stock price and the consumption we made. Particularly,when c(t) = 0,

dV (t) = h(t)dS(t). (13.12)

Page 87: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

13.2. RATE OF RETURN OF STOCK AND RISK-FREE ASSET 87

13.2 Rate of Return of Stock and Risk-free AssetIf the dynamics of the value process of some asset does not have any dW (t)-terms,the value process inceases exponentially without any risk.

Definition 13.1 (Risk-free asset). The stochastic process B(t) is said to be risk-free with its rate of return r(t) if

dB(t)dt

= r(t)B(t). (13.13)

Assumption 13.1. Let S(t) be the price of a stock at time t, and the dynamics ofS(t) is governed by the stochastic differential equation as,

dS(t) = S(t)αdt +S(t)σdW (t). (13.14)

Remark 13.2. Let us compare the rate of return of the risk-free asset B(t) and thestock S(t). By Definition 13.1, formally, we have the rate of return of B(t) as

dB(t)B(t)dt

= r(t), (13.15)

which is observable at time t. On the other hand, the rate of return of S(t) isobtained by

dS(t)S(t)dt

= α +σdW (t)

dt. (13.16)

Here we have the term of dW (t) that is essentially future information.

Definition 13.1. Let B(t) be the value of a risk-free asset and S(t) be the stockprice at time t. The Black-Scholes model is composed by the (B(t),S(t)), whichhas the dynamics as

dB(t) = rB(t)dt, (13.17)dS(t) = αS(t)dt +σS(t)dW (t). (13.18)

13.3 Arbitrage and PortfolioAccording to the result in Section 13.1, we have the budget dynamics of the port-folio by

dV (t) = h(t)dS(t)− c(t)dt, (13.19)

Page 88: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

88 CHAPTER 13. PORTFOLIO DYNAMICS

Generally, we have dW (t) terms in the dynamics of underlying stock price dy-namics dS(t). So, overall, portfolio dynamics dV (t) contain the dW (t) terms. Butin some special case, we can make our portfolio so cleverly that we can avoid thedW (t) terms, which means we have risk-free portfolio. In that case, the followinghappens.

Theorem 13.2. Suppose there is a risk free asset as

dB(t) = r(t)B(t)dt. (13.20)

Consider a portfolio h(t) having the dynamics as,

dV (t) = k(t)V (t)dt, (13.21)

for some function k(t). Apparently, the value of our portfolio V (t) does not haveany risk. Assume we have no arbitrage, then the rate of return of our risk-freeportfolio is r(t), i.e.,

k(t) = r(t). (13.22)

Proof. Assume r(t) and k(t) to be constant r and k for simplicity. First, let ussuppose k > r. Then, we can sell the risk-free asset B(t) and use the money in ourportfolio V (t). The rate of return r for selling the risk-free asset can be coveredby our portfolio, which has the rate of return k. This contradicts the no-arbitrageassumption.

In the case of k < r, we can reverse the argument.

Remark 13.3. Thus, there is a unique rate of return for risk-free asset, which turnsout to be the short-term interest rate of bank account.

Page 89: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Chapter 14

Pricing via Arbitrage

14.1 Way to Black-Scholes ModelWe would like to evaluate the price of an option at time t,

Π(t) = Π(t,S(t)), (14.1)

given that the option is expired at time T and have the value

Π(T ) = Φ(S(T )). (14.2)

Typically,

Φ(S(T )) = (S(T )−K)+ = max(S(T )−K,0) (14.3)

Here’s the procedure to derive Black-Scholes equation:

1. Consider the Black-Scholes model,

dB(t) = rB(t)dt, (14.4)dS(t) = αS(t)dt +σS(t)dW (t). (14.5)

2. Show we can build a risk-free portfolio h(t) using the Black-Scholes model,which has no dW (t)-terms.

3. The rate of return of our portfolio should be the short-term interest rate rbecause of Theorem 13.2.

89

Page 90: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

90 CHAPTER 14. PRICING VIA ARBITRAGE

4. Based on the assumption above, evaluate the dynamics of the option price

Π(t), (14.6)

using Ito’s formula.

5. Solving the dynamics and use so-called Feyman-Kac representation (Theo-rem 12.3) to have

Π(t,S(t)) = e−r(T−t)E[(S(T )−K)+|S(t)]. (14.7)

Page 91: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

Bibliography

T. Bjork. Arbitrage Theory in Continuous Time. Oxford Finance. Oxford Univ Pr,2nd edition, 2004.

R. Durrett. Probability: Theory and Examples. Thomson Learning, 1991.

M. El-Taha and J. S. Stidham. Sample-path analysis of queueing systems.Kluwer’s international press, 1999.

P. Glynn, B. Melamed, and W. Whitt. Estimating customer and time averages,1993. URL citeseer.nj.nec.com/glynn93estimating.html.

A. Langville and C. Meyer. Google’s PageRank and Beyond: The Science ofSearch Engine Rankings. Princeton University Press, 2006.

B. Melamed and W. Whitt. On arrival tha see time averages: a martingale ap-proach. J. of Applied Probability, 27:376 – 384, 1990.

B. Melamed and D. Yao. The asta property, 1995. URL citeseer.nj.nec.com/melamed95asta.html.

B. Oksendal. Stochastic Differential Equations: An Introduction With Applica-tions. Springer-Verlag, 2003.

R. K. P. Bremaud and R. Mazumdar. Event and time averages: a review. Adv.Appl. Prob., 24:377 – 411, 1992.

S. M. Ross. Applied Probability Models With Optimization Applications. DoverPubns, 1992.

N. N. Taleb. Fooled by Randomness: The Hidden Role of Chance in the Marketsand in Life. Random House Trade Paperbacks, 2005.

91

Page 92: Stochastic Models in Business - Waseda University · 2013-03-21 · Stochastic Models in Business Hiroshi Toyoizumi 1 March 21, 2013 ... 4 Introduction of Queueing Systems40 ... The

92 BIBLIOGRAPHY

H. Toyoizumi. Applied probability and mathematical finance theory.http://www.f.waseda.jp/toyoizumi/classes/classes.html, 2008. URL http://www.f.waseda.jp/toyoizumi/classes/classes.html.