The University of Queensland...The University of Queensland ACEMS Workshop on Stochastic Processes...

Birth-Death Processes and Orthogonal Polynomials

Phil. Pollett

The University of Queensland

ACEMS Workshop on Stochastic Processes and Special Functions

August 2015

Phil. Pollett (The University of Queensland) Birth-Death Processes and Orthogonal Polynomials 1 / 37

Example: a metapopulation model (illustrating quasi stationarity)

0 1 2 3 4 5 6 7 80

20A metapopulation model: Xt is the number of occupied patches at time t

↑ λn = (c/N )n(N − n) (c = 8)

↓ µn = en (e = 2)

Quasi-stationary distribution

0 1 2 3 4 5 6 7 80

20A metapopulation model: Xt is the number of occupied patches at time t

↑ λn = (c/N )n(N − n) (c = 8)

↓ µn = en (e = 2)

mj = limt→∞ Pr(Xt = j|Xt > 0) (j = 1, . . . ,20)

Main message

pij(t) : = Pr(Xs+t = j |Xs = i)

∫ ∞

e−txQi(x)Qj(x) dψ(x)

Birth-death processes What are they?

Birth-death processes

A birth-death process is a continuous-time Markov chain (Xt , t ≥ 0) taking values inS ∪ {−1}, where S ⊆ {0, 1, . . . }, with

Pr(Xt+h = n + 1|Xt = n) = λnh + ◦(h)

Pr(Xt+h = n − 1|Xt = n) = µnh + ◦(h)

Pr(Xt+h = n|Xt = n) = 1− (λn + µn)h + ◦(h)

(as h→ 0). Other transitions happen with probability ◦(h).

The birth rates (λn, n ≥ 0) and the death rates (µn, n ≥ 0) are all strictly positiveexcept perhaps µ0, which could be 0. State −1 is a “extinction state”, which can bereached if µ0 > 0.

A birth-death process is a continuous-time Markov chain (Xt , t ≥ 0) taking values inS ∪ {−1}, where S ⊆ {0, 1, . . . }, with

Pr(Xt+h = n + 1|Xt = n) = λnh + ◦(h)

Pr(Xt+h = n − 1|Xt = n) = µnh + ◦(h)

Pr(Xt+h = n|Xt = n) = 1− (λn + µn)h + ◦(h)

(as h→ 0). Other transitions happen with probability ◦(h).

The birth rates (λn, n ≥ 0) and the death rates (µn, n ≥ 0) are all strictly positiveexcept perhaps µ0, which could be 0. State −1 is a “extinction state”, which can bereached if µ0 > 0.

The birth rates (λn, n ≥ 0) and the death rates (µn, n ≥ 0) are all strictly positive,except perhaps µ0, which could be 0. State −1 is a “extinction state”, which can bereached if µ0 > 0.

µ0 > 0

−1 0 1 ..... n − 1 n n + 1 .....

λn−1 λn

µn µn+1

µ0 = 0

0 1 ..... n − 1 n n + 1 .....

λn−1 λn

µn µn+1

The birth rates (λn, n ≥ 0) and the death rates (µn, n ≥ 0) are all strictly positive,except perhaps µ0, which could be 0. State −1 is a “extinction state”, which can bereached if µ0 > 0.

µ0 > 0

−1 0 1 ..... n − 1 n n + 1 .....

λn−1 λn

µn µn+1

µ0 = 0

0 1 ..... n − 1 n n + 1 .....

λn−1 λn

µn µn+1

Birth-death processes Explosive birth-death processes

Explosive birth-death processes

Suppose that λn = 22n, µn = 22n−1 (n ≥ 1), and µ0 = 0, with S = {0, 1, . . . }.

An “equilibrium distribution” exists: πn = ( 12)n+1. But . . .

When a jump occurs it is a birth with probability

pn =22n

22n + 22n−1=

Thus births are twice as likely as deaths, and so the process will have positive drift.

pn =22n

22n + 22n−1=

pn =22n

22n + 22n−1=

0 1 2 3 4 5 6 7 8 90

30Birth-death simulation (λn = 22n , µn = 22n−1) - Explosion

0 1 2 3 4 5 6 7 8 90

0 5 10 15 20 25 30 350

20Birth-death simulation (λn = 22n , µn = 22n−1) - Restart in State 0 after explosion

0 5 10 15 20 25 30 350

20Birth-death simulation (λn = 22n , µn = 22n−1) - Regular boundary

0 5 10 15 20 25 30 350

The Kolmogorov differential equations For a general Markov chain

The Kolmogorov differential equations

The conditions we have imposed ensure that the transition probabilitiespij(t) = Pr(Xs+t = j |Xs = i) (i , j ∈ S , s, t ≥ 0) do not depend on s.

For any such time-homogeneous continuous-time Markov chain with (conservative)transition rate matrix Q = (qij), the transition function P(t) = (pij(t)) satisfies thebackward equations

P ′(t) = QP(t) (BE)

but not necessarily the forward equations

P ′(t) = P(t)Q (FE)

(the derivative is taken elementwise). Note that Q = P ′(0+).

There is however a minimal solution F (t) = (fij(t)) to (BE) and this satisfies (FE).

Non-explosivity corresponds to F being the unique solution to (BE). Otherwise Fgoverns the process up to the time of the (first) explosion.

P ′(t) = QP(t) (BE)

P ′(t) = P(t)Q (FE)

P ′(t) = QP(t) (BE)

P ′(t) = P(t)Q (FE)

P ′(t) = QP(t) (BE)

P ′(t) = P(t)Q (FE)

The Kolmogorov differential equations For birth-death processes

For birth-death processes the full range of behaviour is possible.

Here the transition rate matrix restricted to S = {0, 1, . . . } has the form

−(λ0 + µ0) λ0 0 0 0 . . .

µ1 −(λ1 + µ1) λ1 0 0 . . .0 µ2 −(λ2 + µ2) λ2 0 . . .0 0 µ3 −(λ3 + µ3) λ3 . . ....

......

Returning to the example where λn = 22n, µn = 22n−1 (n ≥ 1), and µ0 = 0, we have . . .

For birth-death processes the full range of behaviour is possible.

Here the transition rate matrix restricted to S = {0, 1, . . . } has the form

−(λ0 + µ0) λ0 0 0 0 . . .

µ1 −(λ1 + µ1) λ1 0 0 . . .0 µ2 −(λ2 + µ2) λ2 0 . . .0 0 µ3 −(λ3 + µ3) λ3 . . ....

......

Returning to the example where λn = 22n, µn = 22n−1 (n ≥ 1), and µ0 = 0, we have . . .

The process governed by F (the “minimal process”)

0 1 2 3 4 5 6 7 8 90

A process where P satisfies (BE) but not (FE)

0 5 10 15 20 25 30 350

20Birth-death simulation (λn = 22n , µn = 22n−1) - Restart in State 0 after explosion

A process where P satisfies both (BE) and (FE)

0 5 10 15 20 25 30 350

Birth-death processes and orthogonal polynomials The birth-death polynomials

The birth-death polynomials

Define a sequence (Qn, n ∈ S) of polynomials by

Q0(x) = 1

−xQ0(x) = −(λ0 + µ0)Q0(x) + λ0Q1(x)

−xQn(x) = µnQn−1(x)− (λn + µn)Qn(x) + λnQn+1(x),

and a sequence of strictly positive numbers (πn, n ∈ S) by π0 = 1 and, for n ≥ 1,

πn =λ0λ1 . . . λn−1

µ1µ2 . . . µn.

A explicit expression for pij(t)

Theorem (Karlin and McGregor (1957))

Let P(t) = (pij(t)) be any transition function that satisfies both the backward and theforward equations (for example the minimal one). Then, there is a probability measure ψwith support [0,∞) such that

pij(t) = πj

∫ ∞0

e−txQi (x)Qj(x) dψ(x) (i , j ≥ 0, t ≥ 0).

1Karlin, S. and McGregor, J.L. (1957) The differential equations of birth-and-death processes, and

the Stieltjes Moment Problem. Trans. Amer. Math. Soc. 85, 489–546.

pij(t) = πj

∫ ∞0

Since pij(0) = δij , it is clear that (Qn) are orthogonal with orthogonalizing measure ψ:∫ ∞0

Qi (x)Qj(x) dψ(x) =δijπj

(i , j ≥ 0).

pij(t) = πj

∫ ∞0

Proof . We use mathematical induction: on i using (BE) with j = 0 and then on j using(FE). But, showing that there is a probability measure ψ with support [0,∞) whoseLaplace transform is p00(t), that is

p00(t) =

∫ ∞0

e−tx dψ(x)

(= π0

∫ ∞0

e−txQ0(x)Q0(x) dψ(x)

is not completely straightforward. More on this later.

A explicit expression for pij(t) - Why is it useful?

pij(t) = πj

∫ ∞0

pij(t) = πj

∫ ∞0

pij(t) = πj

∫ ∞0

pi j(t) =

∫ ∞0

e−txQi (x)πjQj(x) dψ(x) (i , j ≥ 0, t ≥ 0).

pi j(t) =

∫ ∞0

e−txQi (x)πjQj(x) dψ(x) (i , j ≥ 0, t ≥ 0).

pij(t) = πj

∫ ∞0

This formula, together with the myriad of properties of (Qn) and ψ, are used to developtheory peculiar to birth-death processes.

Birth-death processes and orthogonal polynomials Some properties of (Qn) and ψ

Some properties of (Qn) and ψ

Of particular interest and importance is the “interlacing” property of the zeros xn,i(i = 1, · · · , n) of Qn: they are strictly positive, simple, and they satisfy

0 < xn+1,i < xn,i < xn+1,i+1, (i = 1, · · · , n, n ≥ 1),

from which it follows that the limits ξi = limn→∞ xn,i (i ≥ 1) exist and satisfy0 ≤ ξi ≤ ξi+1 <∞.

Interestingly, ξ1 := inf supp(ψ) and ξ2 := inf{supp(ψ) ∩ (ξ1,∞)}, quantities that areparticularly important in the theory of quasi-stationary distributions.

Birth-death processes and orthogonal polynomials Some properties of (Qn) and ψ

Some properties of (Qn) and ψ

Of particular interest and importance is the “interlacing” property of the zeros xn,i(i = 1, · · · , n) of Qn: they are strictly positive, simple, and they satisfy

0 < xn+1,i < xn,i < xn+1,i+1, (i = 1, · · · , n, n ≥ 1),

from which it follows that the limits ξi = limn→∞ xn,i (i ≥ 1) exist and satisfy0 ≤ ξi ≤ ξi+1 <∞.

Interestingly, ξ1 := inf supp(ψ) and ξ2 := inf{supp(ψ) ∩ (ξ1,∞)}, quantities that areparticularly important in the theory of quasi-stationary distributions.

Birth-death processes and orthogonal polynomials Quasi-stationary distributions

The time to extinction

Consider the case µ0 > 0:

−1 0 1 ..... n − 1 n n + 1 .....

λn−1 λn

µn µn+1

Suppose that∑∞

n=0(λnπn)−1 =∞, which ensures that the extinction state −1 is reachedwith probability 1 (and necessarily the process is non-explosive).

Let T = inf{t ≥ 0 : Xt = −1} be the time to extinction.

Clearly Pr(T > t|X0 = i)→ 0 as t →∞, but how fast?

Claim. inf

{a ≥ 0 :

∫ ∞0

eat Pr(T > t|X0 = i) dt =∞}

= ξ1.

2Van Doorn, E.A. and Pollett, P.K. (2013) Quasi-stationary distributions for discrete-state models.

Invited paper. European J. Operat. Res. 230, 1–14.

−1 0 1 ..... n − 1 n n + 1 .....

λn−1 λn

µn µn+1

Suppose that∑∞

Claim. inf

{a ≥ 0 :

∫ ∞0

= ξ1.

−1 0 1 ..... n − 1 n n + 1 .....

λn−1 λn

µn µn+1

Suppose that∑∞

Claim. inf

{a ≥ 0 :

∫ ∞0

= ξ1.

−1 0 1 ..... n − 1 n n + 1 .....

λn−1 λn

µn µn+1

Suppose that∑∞

Claim. inf

{a ≥ 0 :

∫ ∞0

= ξ1.

−1 0 1 ..... n − 1 n n + 1 .....

λn−1 λn

µn µn+1

Suppose that∑∞

Claim. inf

{a ≥ 0 :

∫ ∞0

= ξ1.

Quasi-stationary distributions

A distribution u = (un, n ≥ 0) is called a limiting conditional distribution (or sometimesquasi-stationary distribution) if uij(t) := Pr(Xt = j |T > t,X0 = i)→ uj as t →∞.

Theorem

If ξ1 > 0 then uij(t)→ uj := µ−10 ξ1πjQj(ξ1). If ξ1 = 0 then uj(t)→ 0.

Again, how fast?

Claim. inf

{a ≥ 0 :

∫ ∞0

eat |uij(t)− uj | dt =∞}

= ξ2 − ξ1 (same for all i , j ∈ S).

3van Doorn, E.A. (1991) Quasi-stationary distributions and convergence to quasi-stationarity of

birth-death processes. Adv. Appl. Probab. 23, 683–700.

4Kijima, M., Nair, M.G., Pollett, P.K. and van Doorn, E.A. (1997) Limiting conditional distribu-

tions for birth-death processes. Adv. Appl. Probab. 29, 185–204.

Theorem

Again, how fast?

Claim. inf

{a ≥ 0 :

∫ ∞0

Theorem

Again, how fast?

Claim. inf

{a ≥ 0 :

∫ ∞0

Theorem

Again, how fast?

Claim. inf

{a ≥ 0 :

∫ ∞0

Birth-death processes and orthogonal polynomials Why does this work?

Recall . . .

pij(t) = πj

∫ ∞0

Proof . We use mathematical induction: on i using (BE) with j = 0 and then on j using(FE). But, showing that there is a probability measure ψ with support [0,∞) whoseLaplace transform is p00(t), that is

p00(t) =

∫ ∞0

e−tx dψ(x)

(= π0

∫ ∞0

e−txQ0(x)Q0(x) dψ(x)

is not completely straightforward. More on this later.

Why does this work?

pij(t) = πj

∫ ∞0

Why is there a ψ whose Laplace transform is p00(t): p00(t) =

∫ ∞0

e−tx dψ(x) ?

Answer . Weak symmetry: πiqij = πjqji (πiλi = πi+1µi+1)

Why does this work?

pij(t) = πj

∫ ∞0

e−tx dψ(x) ?

Why does this work?

pij(t) = πj

∫ ∞0

e−tx dψ(x) ?

Finite state Markov chains - some linear algebra

Let (Xt , t ≥ 0) be a continuous-time Markov chain taking values in S = {0, 1, . . . ,N}with (conservative) transition rate matrix Q. So, there is collection π = (πj , j ∈ S) ofstrictly positive numbers such that πQ = 0, that is∑

πiqij = πj

∑i∈S

qji (j ∈ S).

Suppose that Q is weakly symmetric with respect to π: πiqij = πjqji .

Let A be the symmetric matrix with entries aij =√πiqij/

√πj . It is orthogonally similar to

a diagonal matrix D = diag{d0, d1, . . . , dN}: A = MDM> . . . , et cetera, . . . leading tothe spectral solution of P ′(t) = QP(t) (BE):

pij(t) = πj

N∑k=0

edk tQ(k)i Q

(k)j , where Q(k)

i =Mik√πi.

πiqij = πj

∑i∈S

qji (j ∈ S).

pij(t) = πj

N∑k=0

edk tQ(k)i Q

(k)j , where Q(k)

i =Mik√πi.

πiqij = πj

∑i∈S

qji (j ∈ S).

a diagonal matrix D = diag{d0, d1, . . . , dN}: A = MDM> . . . , et cetera, . . .

leading tothe spectral solution of P ′(t) = QP(t) (BE):

pij(t) = πj

N∑k=0

edk tQ(k)i Q

(k)j , where Q(k)

i =Mik√πi.

πiqij = πj

∑i∈S

qji (j ∈ S).

pij(t) = πj

N∑k=0

edk tQ(k)i Q

(k)j , where Q(k)

i =Mik√πi.

General symmetric Markov chains - some functional analysis

Let π = (πj , j ∈ S) be a collection of strictly positive numbers and suppose that P isweakly symmetric with respect to π: πipij(t) = πjpji (t) (i , j ∈ S).

Define Tt : `2 → `2 by

(Ttx)j =∑i∈S

xi (πi/πj)1/2pij(t) (i ∈ S , x ∈ `2).

Then (Tt , t ≥ 0) is a semigroup which is self adjointing 〈Ttx , y〉 = 〈x ,Tty〉.

Kendall used a result of Riesz and Sz.-Nagy on the spectral representation of self-adjointsemigroups to show that there is a finite signed measure γij with support [0,∞) such that

pij(t) = (πj/πi )1/2

∫[0,∞)

e−tx dγij(x).

Furthermore, γii is a probability measure.

5Kendall, D.G (1959) Unitary dilations of one-parameter semigroups of Markov transition opera-

tors, and the corresponding integral representations for Markov processes with a countable infinity

of states. Proc. London Math. Soc. 9, 417–431.

(Ttx)j =∑i∈S

xi (πi/πj)1/2pij(t) (i ∈ S , x ∈ `2).

∫[0,∞)

e−tx dγij(x).

(Ttx)j =∑i∈S

xi (πi/πj)1/2pij(t) (i ∈ S , x ∈ `2).

∫[0,∞)

e−tx dγij(x).

(Ttx)j =∑i∈S

xi (πi/πj)1/2pij(t) (i ∈ S , x ∈ `2).

∫[0,∞)

e−tx dγij(x).

General symmetric Markov chains - speculation

In can be seen from the definition of the birth-death polynomials Q = (Qn, n ∈ S),

Q0(x) = 1

−xQ0(x) = −(λ0 + µ0)Q0(x) + λ0Q1(x)

−xQn(x) = µnQn−1(x)− (λn + µn)Qn(x) + λnQn+1(x),

and the form of transition rate matrix restricted to S = {0, 1, . . . },

−(λ0 + µ0) λ0 0 0 0 . . .

µ1 −(λ1 + µ1) λ1 0 0 . . .0 µ2 −(λ2 + µ2) λ2 0 . . .0 0 µ3 −(λ3 + µ3) λ3 . . ....

......

that Q = Q(x) as a column vector satisfies QQ = −xQ (Q(x) is an x-invariant vectorfor Q), and R = R(x), where Rj(x) = πjQj(x), as a row vector satisfies RQ = −xR(R(x) is an x-invariant measure for Q).

One might speculate that

pij(t) = πj

∫ ∞0

e−txQi (x)Qj(x) dψ(x) (i , j ≥ 0, t ≥ 0)

holds more generally under weak symmetry (πiqij = πjqji ) for a function systemQ = (Qn, n ∈ S) (necessarily orthogonal with respect to ψ) satisfying QQ = −xQ.

It might perhaps be too much to expect that

pij(t) =

∫ ∞0

e−txQi (x)Rj(x) dψ(x) (i , j ≥ 0, t ≥ 0)

holds with just πQ = 0 for function systems Q = (Qn, n ∈ S) and R = (Rn, n ∈ S)satisfying QQ = −xQ and RQ = −xR, and, of necessity,∫ ∞

Qi (x)Rj(x) dψ(x) = δij (i , j ≥ 0).

One might speculate that

pij(t) = πj

∫ ∞0

e−txQi (x)Qj(x) dψ(x) (i , j ≥ 0, t ≥ 0)

holds more generally under weak symmetry (πiqij = πjqji ) for a function systemQ = (Qn, n ∈ S) (necessarily orthogonal with respect to ψ) satisfying QQ = −xQ.

It might perhaps be too much to expect that

pij(t) =

∫ ∞0

e−txQi (x)Rj(x) dψ(x) (i , j ≥ 0, t ≥ 0)

holds with just πQ = 0 for function systems Q = (Qn, n ∈ S) and R = (Rn, n ∈ S)satisfying QQ = −xQ and RQ = −xR, and, of necessity,∫ ∞

Qi (x)Rj(x) dψ(x) = δij (i , j ≥ 0).

The University of Queensland...The University of Queensland ACEMS Workshop on Stochastic Processes...

Documents

Transcript of The University of Queensland...The University of Queensland ACEMS Workshop on Stochastic Processes...