Martingale

•First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

Martingale problems and stochastic equations forMarkov processes

• Review of basic material on stochastic processes

• Characterization of stochastic processes by their martingale properties

• Weak convergence of stochastic processes

• Stochastic equations for general Markov process in Rd

• Martingale problems for Markov processes

• Forward equations and operator semigroups

• Equivalence of martingale problems and stochastic differential equations

• Change of measure

• Filtering

• Averaging


• Control

• Exercises

• Glossary

• References


1. Review of basic material on stochastic processes

• Filtrations

• Stopping times

• Martingales

• Optional sampling theorem

• Doob’s inequalities

• Stochastic integrals

• Local martingales

• Semimartingales

• Computing quadratic variations

• Covariation

• Ito’s formula


Conventions and caveats 4

State spaces are always complete, separable metric spaces (sometimes called Polishspaces), usually denoted (E, r).

All probability spaces are complete.

All identities involving conditional expectations (or conditional probabilities) onlyhold almost surely (even when I don’t say so).

If the filtration Ft involved is obvious, I will say adapted, rather than Ft-adapted, stopping time, rather than Ft-stopping time, etc.

All processes are cadlag (right continuous with left limits at each t > 0), unlessotherwise noted.

A process is real-valued if that is the only way the formula makes sense.


References 5

Kurtz, Lecture Notes for Math 735

http://www.math.wisc.edu/˜kurtz/m735.htm

Seppalainen, Basics of Stochastic Analysis

http://www.math.wisc.edu/˜seppalai/sa-book/etusivu.html

Ethier and Kurtz, Markov Processes: Characterization and Convergence

Protter, Stochastic Integration and Differential Equations, Second Edition

http://www.math.wisc.edu/~kurtz/m735.htm

http://www.math.wisc.edu/~seppalai/sa-book/etusivu.html


Filtrations 6

(Ω,F , P ) a probability space

Available information is modeled by a sub-σ-algebra of F

Ft information available at time t

Ft is a filtration. t < s implies Ft ⊂ Fs

A stochastic process X is adapted to Ft if X(t) is Ft-measurable for each t ≥ 0.

An E-valued stochastic process X adapted to Ft is Ft-Markov if

E[f(X(t+ r))|Ft] = E[f(X(t+ r))|X(t)], t, r ≥ 0, f ∈ B(E)

An R-valued stochastic process M adapted to Ft is an Ft-martingale if

E[M(t+ r)|Ft] = M(t), t, r ≥ 0


Stopping times 7

τ is an Ft-stopping time if for each t ≥ 0, τ ≤ t ∈ Ft.

For a stopping time τ ,

Fτ = A ∈ F : τ ≤ t ∩ A ∈ Ft, t ≥ 0

Exercise 1.1 1. Show that Fτ is a σ-algebra.

2. Show that for Ft-stopping times σ, τ , σ ≤ τ implies that Fσ ⊂ Fτ . In particular,Fτ∧t ⊂ Ft.

3. Let τ be a discrete Ft-stopping time satisfying τ < ∞ = ∪∞k=1τ = tk = Ω.Show that Fτ = σA ∩ τ = tk : A ∈ Ftk , k = 1, 2, . . ..

4. Show that the minimum of two stopping times is a stopping time and that the maxi-mum of two stopping times is a stopping time.


Examples and properties 8

Define Ft+ ≡ ∩s>tFs. Ft is right continuous if Ft = Ft+ for all t ≥ 0. If Ft isright continuous, then τ is a stopping time if and only if τ < t ∈ Ft for all t > 0.

If K ⊂ E is closed, τhK = inft : X(t) or X(t−) ∈ K is a stopping time, but

inft : X(t) ∈ K may not be; however, if Ft is right continuous and complete,then for anyB ∈ B(E), τB = inft : X(t) ∈ B is an Ft-stopping time. This resultis a special case of the debut theorem, a very technical result from set theory. Notethat

ω : τB(ω) < t = ω : ∃s < t 3 X(s, ω) ∈ B = projΩ(s, ω) : X(s, ω) ∈ B, s < t

Piecewise constant approximations

ε > 0, τ ε0 = 0,

τ εi+1 = inft > τ ε

i : r(X(t), X(τ εi )) ∨ r(X(t−), X(τ ε

i )) ≥ ε

Define Xε(t) = X(τ εi ), τ ε

i ≤ t < τ εi+1. Then r(X(t), Xε(t)) ≤ ε.

If X is adapted to Ft, then the τ εi are Ft-stopping times and Xε is Ft-

adapted.


Martingales 9

An R-valued stochastic process M adapted to Ft is an Ft-martingale if

E[M(t+ r)|Ft] = M(t), t, r ≥ 0

Every martingale has finite quadratic variation:

[M ]t = lim∑

(M(t ∧ ti+1)−M(t ∧ ti))2

where 0 = t0 < t1 < · · ·, ti →∞, and the limit is in probability as max(ti+1−ti) → 0.More precisely, for ε > 0 and t0 > 0,

limPsupt≤t0

|[M ]t − lim∑

(M(t ∧ ti+1)−M(t ∧ ti))2| > ε = 0.

For standard Brownian motion W , [W ]t = t.

Exercise 1.2 Let N be a Poisson process with parameter λ. Then M(t) = N(t) − λt is amartingale. Compute [M ]t.


Square integrable martingales 11

M a martingale satisfying E[M(t)2] <∞. Then

M(t)2 − [M ]t

is a martingale. In particular, for t > s

E[(M(t)−M(s))2] = E[[M ]t − [M ]s].


Doob’s inequalities 12

Let X be a submartingale. Then for x > 0,

Psups≤t

X(s) ≥ x ≤ x−1E[X(t)+]

Pinfs≤t

X(s) ≤ −x ≤ x−1(E[X(t)+]− E[X(0)])

If X is nonnegative and α > 1, then

E[sups≤t

X(s)α] ≤(

α

α− 1

)α

E[X(t)α].

Note that by Jensen’s inequality, if M is a martingale, then |M | is a submartingale.In particular, if M is a square integrable martingale, then

E[sups≤t

|M(s)|2] ≤ 4E[M(t)2].


Stochastic integrals 13

Definition 1.3 For cadlag processes X , Y ,

X− · Y (t) ≡∫ t

0

X(s−)dY (s)

= limmax |ti+1−ti|→0

∑X(ti)(Y (ti+1 ∧ t)− Y (ti ∧ t))

whenever the limit exists in probability.

Sample paths of bounded variation: If Y is a finite variation process, the stochasticintegral exists (apply dominated convergence theorem) and∫ t

0

X(s−)dY (s) =

∫(0,t]

X(s−)αY (ds)

αY is the signed measure with

αY (0, t] = Y (t)− Y (0)


Existence for square integrable martingales 14

If M is a square integrable martingale, then

E[(M(t+ s)−M(t))2|Ft] = E[[M ]t+s − [M ]t|Ft]

For partitions ti and ri

E[(∑

X(ti)(M(ti+1 ∧ t)−M(ti ∧ t))

−∑

X(ri)(M(ri+1 ∧ t)−M(ri ∧ t)))2]

= E

[∫ t

0

(X(t(s−))−X(r(s−)))2d[M ]s

]= E

[∫(0,T ]

(X(t(s−))−X(r(s−)))2α[M ](ds)

]t(s) = ti for s ∈ [ti, ti+1) r(s) = ri for s ∈ [ri, ri+1)


Cauchy property 15

Let X be bounded by a constant. As sup(ti+1 − ti) + sup(ri+1 − ri) → 0, the rightside converges to zero by the dominated convergence theorem.

MtiX (t) ≡

∑X(ti)(M(ti+1 ∧ t)−M(ti ∧ t)) is a square integrable martingale, so

E

[supt≤T

(∑X(ti)(M(ti+1 ∧ t)−M(ti ∧ t))

−∑

X(ri)(M(ri+1 ∧ t)−M(ri ∧ t)))2]

≤ 4E

[∫(0,t]

(X(t(s−))−X(r(s−)))2α[M ](ds)

]A completeness argument gives existence of the stochastic integral and the unifor-mity implies the integral is cadlag.


Local martingales 16

Definition 1.4 M is a local martingale if there exist stopping times τn satisfying τ1 ≤τ2 ≤ · · · and τn →∞ a.s. such that M τn defined by M τn(t) = M(τn ∧ t) is a martingale.M is a local square-integrable martingale if the τn can be selected so that M τn is squareintegrable.

τn is called a localizing sequence for M .

Remark 1.5 If τn is a localizing sequence for M , and γn is another sequence of stop-ping times satisfying γ1 ≤ γ2 ≤ · · ·, γn → ∞ a.s. then the optional sampling theoremimplies that τn ∧ γn is localizing.


Local martingales with bounded jumps 17

Remark 1.6 If M is a continuous, local martingale, then τn = inft : |M(t)| ≥ nwill be a localizing sequence. More generally, if |∆M(t)| ≤ c for some constant c, thenτn = inft : |M(t)|∨|M(t−)| ≥ nwill be a localizing sequence. Note that |M τn| ≤ n+c,so M is local square integrable.


A metric 19

For cadlag stochastic processes X and Y , define

dt(X,Y ) = infε > 0 : Psups≤t

r(X(s), Y (s)) ≥ ε ≤ ε

andd(X, Y ) =

∫ ∞

0

e−tdt(X,Y )dt.

Under d, the space of cadlag (or cadlag and adapted) stochastic processes is a com-plete metric space.


Semimartingales 20

Definition 1.7 Y is an Ft-semimartingale if and only if Y = M + V , where M isa local square integrable martingale with respect to Ft and V is an Ft-adapted finitevariation process.

In particular, if X is cadlag and adapted and Y is a semimartingale, then∫X−dY

exists.


Computing quadratic variations 21

Let ∆Z(t) = Z(t) = Z(t−).

Lemma 1.8 If Y is finite variation, then

[Y ]t =∑s≤t

∆Y (s)2

Lemma 1.9 If Y is a semimartingale, X is adapted, and Z(t) =∫ t

0X(s−)dY (s), then

[Z]t =

∫ t

0

X(s−)2d[Y ]s.

Proof. Check first for piecewise constant X and then approximate general X bypiecewise constant processes.


Covariation 22

The covariation of Y1, Y2 is defined by

[Y1, Y2]t ≡ lim∑

i

(Y1(ti+1 ∧ t)− Y1(ti ∧ t)) (Y2(ti+1 ∧ t)− Y2(ti ∧ t))

Exercise 1.10 1. Show that if Y1 is cadlag and Y2 is finite variation, then

[Y1, Y2]t =∑s≤t

∆Y1(s)∆Y2(s).

2. Using the fact that martingales have finite quadratic variation, show that semimartin-gales have finite quadratic variation.

3. Using the above results, show that the covariation of two semimartingales exist.


Ito’s formula 23

If f : R → R is C2 and Y is a semimartingale, then

f(Y (t)) = f(Y (0)) +

∫ t

0

f ′(Y (s−))dY (s) +

∫ t

0

1

2f ′′(Y (s))d[Y ]cs

+∑s≤t

(f(Y (s))− f(Y (s−))− f ′(Y (s−))∆Y (s)

where [Y ]c is the continuous part of the quadratic variation given by

[Y ]ct = [Y ]t −∑s≤t

∆Y (s)2.


Ito’s formula for vector-valued semimartingales 24

If f : Rm → R is C2, Y1, . . . , Ym are semimartingales, and Y = (Y1, . . . , Ym), thendefining

[Yk, Yl]ct = [Yk, Yl]t −

∑s≤t

∆Yk(s)∆Yl(s),

f (Y (t)) = f (Y (0)) +m∑

k=1

∫ t

0

∂kf (Y (s−)) dYk(s)

+m∑

k,l=1

1

2

∫ t

0

∂k∂lf (Y (s−)) d[Yk, Yl]cs

+∑s≤t

(f (Y (s))− f (Y (s−))−m∑

k=1

∂kf (Y (s−)) ∆Yk(s)).


Examples 25

W standard Brownian motion

Z(t) = expW (t)− 1

2t =

∫ t

0

Z(s)d(W (s)− 1

2s) +

∫ t

0

1

2Z(s)ds

=

∫ t

0

Z(s)dW (s)

Exercise 1.11 Consider the stochastic differential equation

X(t) = X(0) +

∫ t

0

aX(s)dW (s) +

∫ t

0

bX(s)ds.

Find α and β so thatX(t) = X(0) expαW (t) + βt

is a solution.


2. Characterization of stochastic processes by their martingale prop-erties

• Levy’s characterization of Brownian motion

• Watanabe’s characterization of the Poisson process

• Strong Markov property for Poisson processes

• Intensity for a counting process

• Martingale problems for counting processes

• Multivariate counting processes

• Continuous time Markov chains

• Martingale problems for diffusion processes


Levy’s characterization of Brownian motion 27

Theorem 2.1 Let M be a continuous local martingale with [M ]t = t. Then M is standardBrownian motion.

Remark 2.2 Note that

E[M τn(t)2] = E[[M τn ]t] = E[τn ∧ t] ≤ t

and E[sups≤tMτn(s)2] ≤ 4E[M τn(t)2] ≤ 4t and it follows by the dominated convergence

theorem and Fatou’s lemma that M is a square integrable martingale.


Proof. Applying Ito’s formula,

eiθM(t) = 1 +

∫ t

0

iθeiθM(s)dM(s)− 1

2θ2

∫ t

0

eiθM(s)ds,

where the second term on the right is a martingale. Consequently,

E[eiθM(t+r)|Ft] = eiθM(t) − 1

2θ2

∫ t+r

t

E[eiθM(s)|Ft]ds

and

ϕt(θ, r) ≡ E[eiθ(M(t+r)−M(t)|Ft] = 1− 1

2θ2

∫ t+r

t

E[eiθM(s)−M(t)|Ft]ds

= 1− 1

2θ2

∫ r

0

ϕt(θ, u)du

so ϕt(θ, r) = e−12θ2r. It follows that for 0 = t0 < t1 < · · · < tm,

E[m∏

k=1

eiθk(M(tk)−M(tk−1)] =∏

e−12θ2k(tk−tk−1)

and hence M has independent Gaussian increments.


Continuous, finite variation martingales 29

Theorem 2.3 Suppose that M is a continuous local martingale and Tt(M) < ∞ a.s. forevery t ≥ 0. Then M is constant in time.

Proof. Replace M by M − M(0). Since M is a continuous local martingale, it islocally square integrable (Remark 1.6) and there exist stopping times τn →∞ suchthat

E[M(t ∧ τn)2] = E[[M ]t∧τn ] = 0.


Original form of Levy’s theorem 30

Lemma 2.4 Suppose that M is a continuous local martingale and that M2(t) − t is acontinuous local martingale also. Then [M ]t = t.

Proof. By Ito’s formula

M2(t) =

∫ t

0

2M(s)dM(s)− [M ]t.

Consequently,

[M ]t − t = M2(t)− t+

∫ t

0

2M(s)dM(s)

is a finite variation, continuous local martingale and is zero by Lemma 2.3.

It follows that Levy’s theorem (Theorem 2.1) holds with the condition [M ]t = treplaced by the assumption that M(t)2 − t is a local martingale.


Dominated convergence theorem 31

Theorem 2.5 Let Xn → X and Yn → Y in probability. Suppose that |Xn| ≤ Yn a.s. andE[Yn|D] → E[Y |D] in probability. Then

E[Xn|D] → E[X|D] in probability

Proof. A sequence converges in probability iff every subsequence has a furthersubsequence that converges a.s., so we may as well assume almost sure conver-gence. Let Dm,c = supn≥mE[Yn|D] ≤ c. Then

E[Yn1Dm,c |D] = E[Yn|D]1Dm,c

L1→ E[Y |D]1Dm,c = E[Y 1Dm,c |D].

Consequently, E[Yn1Dm,c ] → E[Y 1Dm,c ], so Yn1Dm,c → Y 1Dm,c in L1 by the ordinarydominated convergence theorem. It follows that Xn1Dm,c → X1Dm,c in L1 andhence

E[Xn|D]1Dm,c = E[Xn1Dm,c |D]L1→ E[X1Dm,c |D] = E[X|D]1Dm,c .

Since m and c are arbitrary, the lemma follows.


Poisson processes 32

A Poisson process N can be defined to be a counting process with stationary, in-dependent increments, and it follows that the increments must be Poisson dis-tributed.

E[eiθ(N(b)−N(a))] = E[eiθN( b−an

)]n

= (1 + PN(b− a

n) = 1(eiθ − 1) + E[eiθN( b−a

n)1N( b−a

n)>1])

n.

Exercise 2.6 Show that limn→∞ nPN( b−an

) > 1 = 0.

By the identity and the exercise:

E[eiθ(N(b)−N(a))] = limn→∞

enPN( b−an

)=1(eiθ−1),

so λb−a ≡ limn→∞ nPN( b−an

) = 1 exists. The assumption that N has stationary,independent increments then implies λb−a = λ1(b− a). Letting λ = λ1, we have

E[eiθ(N(b)−N(a))] = eλ(b−a)(eiθ−1).


Time inhomogeneous Poisson processes 33

Exercise 2.7 Extend the previous result to stochastically continuous counting processeswith independent increments.

Definition 2.8 X is stochastically continuous if for every t ≥ 0 and ε > 0,

lims→t

Pr(X(s), X(t)) > ε = 0.


Watanabe’s characterization of the Poisson process 34

Theorem 2.9 Let N be a counting process and λ > 0. If M(t) ≡ N(t) − λt is a localmartingale, then N is a Poisson process with parameter λ.

Remark 2.10 Since the discontinuities of M are bounded by 1, M is local square inte-grable. (See Remark 1.6.) Consequently, there exist τn →∞ such that

E[M(t ∧ τn)2] = E[[M ]t∧τn ] = E[N(t ∧ τn)] = E[t ∧ τn],

soM is square integrable by Fatou’s lemma. Further more, sups≤tM(t∧τn)2 sups≤tM(s)2,

E[sups≤t

M(s)2] = limE[sups≤t

M(t ∧ τn)2] ≤ 4t.

The dominated convergence theorem then gives

E[M(t+ r)|Ft] = limE[M((t+ r) ∧ τn|Ft] = limM(t ∧ τn) = M(t)


Proof.

eiθN(t) = 1 +

∫ t

0

(eiθ(N(s−)+1) − eiθN(s−))dM(s) +

∫ t

0

(eiθ(N(s−)+1) − eiθN(s−))λds

where the second term on the right is a martingale. Consequently,

E[eiθN(t+r)|Ft] = eiθN(t) + λ(eiθ − 1)

∫ t+r

t

E[eiθN(s)|Ft]ds

and

ϕt(θ, r) ≡ E[eiθ(N(t+r)−N(t)|Ft] = 1 + λ(eiθ − 1)

∫ t+r

t

E[eiθN(s)−N(t)|Ft]ds

= 1 + λ(eiθ − 1)

∫ r

0

ϕt(θ, u)du

so ϕt(θ, r) = eλr(eiθ−1). It follows that for 0 = t0 < t1 < · · · < tm,

E[m∏

k=1

eiθk(N(tk)−N(tk−1)] =∏

eλ(tk−tk−1)(eiθ−1)

and hence N has independent Poisson increments.


Strong Markov property 36

A Poisson process N is compatible with a filtration Ft, if N is Ft-adapted andN(t+ ·)−N(t) is independent of Ft for every t ≥ 0.

Lemma 2.11 Let N be a Poisson process with parameter λ > 0 that is compatible withFt, and let τ be a Ft-stopping time such that τ <∞ a.s. Define Nτ (t) = N(τ + t)−N(τ). ThenNτ is a Poisson process that is independent of Fτ and compatible with Fτ+t.

Proof. Let M(t) = N(t)− λt. By the optional sampling theorem,

E[M((τ + t+ r) ∧ T )|Fτ+t] = M((τ + t) ∧ T ),

so

E[N((τ + t+ r) ∧ T )−N((τ + t) ∧ T )|Fτ+t] = λ((τ + t+ r) ∧ T − (τ + t) ∧ T ).

By the monotone convergence theorem

E[N(τ + t+ r)−N(τ + t)|Fτ+t] = λr

which gives the lemma.


Intensity for a counting process 37

If N is a Poisson process with parameter λ and N is compatible with Ft, then

PN(t+ ∆t) > N(t)|Ft = 1− e−λ∆t ≈ λ∆t.

For a general counting process N , at least intuitively, a nonnegative, Ft-adaptedstochastic process λ(·) is an Ft-intensity for N if

PN(t+ ∆t) > N(t)|Ft ≈ E[

∫ t+∆t

t

λ(s)ds|Ft] ≈ λ(t)∆t.

Let σn be the nth jump time of N .

Definition 2.12 λ is an Ft-intensity for N if and only if for each n = 1, 2, . . ..

N(t ∧ σn)−∫ t∧σn

0

λ(s)ds

is a Ft-martingale.


Modeling with intensities 38

Let Z be a stochastic process (cadlag, E-valued for simplicity) that models “exter-nal noise.” Let Dc[0,∞) denote the space of counting paths (zero at time zero andconstant except for jumps of +1).

Condition 2.13

λ : [0,∞)×DE[0,∞)×Dc[0,∞) → [0,∞)

is measurable and satisfies λ(t, z, v) = λ(t, zt, vt), where zt(s) = z(s ∧ t) (λ is nonantic-ipating), and ∫ t

0

λ(s, z, v)ds <∞

for all z ∈ DE[0,∞) and v ∈ Dc[0,∞).

Let Y be a unit Poisson process that is Ft-compatible and assume that Z(s) isF0-measurable for every s ≥ 0. (In particular, Z is independent of Y . Consider

N(t) = Y (

∫ t

0

λ(s, Z,N)ds). (2.1)


Solution of the stochastic equation 39

Theorem 2.14 For n = 1, 2, . . ., there exists a unique solution of (2.1) up to σn, τ(t) =∫ t

0λ(s, Z,N)ds is a Fu-stopping time, and


0

λ(s, Z,N)ds

is a Fτ(t)-martingale.


Proof. Existence and uniqueness follows by solving from one jump to the next. LetY r(u) = Y (r ∧ u) and let

N r(t) = Y r(

∫ t

0

λ(s, Z,N r)ds).

Then N r(t) = N(t), if τ(t) =∫ t

0λ(s, Z,N)ds ≤ r. Consequently,

τ(t) ≤ r = ∫ t

0

λ(s, Z,N r)ds ≤ r ∈ Fr,

as is τ(t ∧ σn) ≤ r. By the optional sampling theorem

E[M(τ((t+ v)∧ σn)∧ T )|Fτ(t)] = M(τ((t+ v)∧ σn))∧ τ(t)∧ T ) = M(τ(t∧ σn)∧ T ).

We can let T → ∞ by the monotone convergence argument used in the proof ofthe strong Markov property for Poisson processes.


Martingale problems for counting processes 41

Definition 2.15 Let Z be a cadlag, E-valued stochastic process, and let λ satisfy Con-dition 2.13. A counting process N is a solution of the martingale problem for (λ, Z)if


0

λ(s, Z,N)ds

is a martingale with respect to the filtration

Ft = σ(N(s), Z(r) : s ≤ t, r ≥ 0)

Theorem 2.16 If N is a solution of the martingale problem for (λ, Z), then N has thesame distribution as the solution of the stochastic equation (2.1).


Proof. Suppose λ is an intensity for a counting process N and∫∞

0λ(s)ds = ∞ a.s.

Let γ(u) satisfy

γ(u) = inft :

∫ t

0

λ(s)ds ≥ u.

Then, since γ(u+ v) ≥ γ(u),

E[N(γ(u+v)∧σn∧T )−∫ γ(u+v)∧σn∧T

0

λ(s)ds|Fγ(u)] = N(γ(u)∧σn∧T )−∫ γ(u)∧σn∧T

0

λ(s)ds.

The monotone convergence argument lets us send T and n to infinity, and we have

E[N(γ(u+ v))− (u+ v)|Fγ(u)] = N(γ(u))− u.

so Y (u) = N(γ(u)) is a Poisson process. But γ(τ(t)) = t, so (2.1) is satisfied.

If∫∞

0λ(s)ds < ∞ with positive probability, then let Y ∗ be a unit Poisson process

that is independent of Ft for all t ≥ 0 and consider N ε(t) = N(t) + Y ∗(εt). N ε hasintensity λ(t) + ε, and Y ε, obtained as above, converges to

Y (u) =

N(γ(u)) u < τ(∞)

N(∞) + Y ∗(u− τ(∞)) u ≥ τ(∞)

(except at points of discontinuity).


Multivariate counting processes 43

Dcd[0,∞) d-dimensional counting paths

Condition 2.17 λk : [0,∞) × Dcd[0,∞) × DE[0,∞) → [0,∞), measurable, nonantici-

pating ∫ t

0

∑k

λk(s, z, v)ds <∞, v ∈ Dcd[0,∞), z ∈ DE[0,∞)

Z cadlag, E-valued and independent of independent Poisson processes Y1, . . . , Yd.

Nk(t) = Yk(

∫ t

0

λk(s, Z,N)ds) (2.2)

where N = (N1, . . . , Nd). Existence and uniqueness holds (including for d = ∞)and

Nk(t ∧ σn)−∫ t∧σn

0

λk(s, Z,N)ds

is a martingale for σn = inft :∑

k Nk(t) ≥ n, but what is the correct filtration?


Multiparameter optional stopping theorem 44

I is a directed set with partial ordering t ≤ s. If t1, t2 ∈ I, there exists t3 ∈ I suchthat t1 ≤ t3 and t2 ≤ t3.

Ft, t ∈ I, s ≤ t implies Fs ⊂ Ft.

A stochastic process X(t) indexed by I is a martingale if and only if for s ≤ t,

E[X(t)|Fs] = X(s).

An I valued random variable is a stopping time if and only if τ ≤ t ∈ Ft, t ∈ I.

Fτ = A ∈ F : A ∩ τ ≤ t ∈ Ft, t ∈ I

Lemma 2.18 (Kurtz [6]) Let X be a martingale and let τ1 and τ2 be stopping times as-suming countably many values and satisfying τ1 ≤ τ2 a.s. If there exists a sequenceTm ⊂ I such that limm→∞ Pτ2 ≤ Tm = 1, limm→∞E[|X(Tm)|1τ2≤Tmc ] = 0, andE[|X(τ2)|] <∞, then

E[X(τ2)|Fτ1 ] = X(τ1)


Proof. Define

τmi =

τi on τi ≤ TmTm on τi ≤ Tmc

Then τmi is a stopping time, since

τmi ≤ t = (τm

i ≤ t ∩ τi ≤ Tm) ∪ (τmi ≤ t ∩ τi ≤ Tmc

= (∪s∈Γ,s≤t,s≤Tmτi = s) ∪ (Tm ≤ t ∩ τi ≤ Tmc

Let Γ ⊂ I be countable and satisfy Pτi ∈ Γ = 1 and Tm ⊂ Γ. For A ∈ Fτ1 ,∫A∩τm

1 =tX(τm

2 )dP =∑

s∈Γ,s≤Tm

∫A∩τm

1 =t∩τm2 =s

X(s)dP

=∑

s∈Γ,s≤Tm

∫A∩τm

1 =t∩τm2 =s

X(Tm)dP

=

∫A∩τm

1 =tX(Tm)dP

=

∫A∩τm

1 =tX(t)dP =

∫A∩τm

1 =tX(τm

1 )dP


Multiple time change 46

I = [0,∞)d, u ∈ I, Fu = σ(Yk(sk) : sk ≤ uk, k = 1, . . . , d). Then

Mk(u) ≡ Yk(uk)− uk

is a Fu-martingale. For

Nk(t) = Yk(

∫ t

0

λk(s, Z,N)ds),

define τk(t) =∫ t

0λk(s, Z,N)ds and τ(t) = (τ1(t), . . . , τd(t)). Then τ(t) is a Fu-

stopping time.

Lemma 2.19 Let Gt = Fτ(t). If σ is a Gt-stopping time, then τ(σ) is a Fu-stoppingtime.


Approximation by discrete stopping times 47

Lemma 2.20 If τ is a Fu-stopping time, then τ (n) defined by

τ(n)k =

[τk2n] + 1

2n

is a Fu-stopping time.

Proof.

τ (n) ≤ u = ∩kτ (n)k ≤ uk = ∩k[τk2n] + 1 ≤ [uk2

n] = ∩kτk <[uk2

n]

2n

Note that τ (n)k decreases to τk.


Martingale problems for multivariate counting processes 48

Let σn = inft :∑

k Nk(t) ≥ n.

Theorem 2.21 Let Condition 2.17 hold. For n = 1, 2, . . ., there exists a unique solutionof (2.2) up to σn, τk(t) =

∫ t

0λk(s, Z,N)ds defines a Fu-stopping time, and


0

λk(s, Z,N)ds

is a Fτ(t)-martingale.

Definition 2.22 Let Z be a cadlag, E-valued stochastic process, and let λ = (λ1, . . . , λd)satisfy Condition 2.17. A multivariate counting process N is a solution of the martingaleproblem for (λ, Z) if for each k,


0

λk(s, Z,N)ds

is a martingale with respect to the filtration

Gt = σ(N(s), Z(r) : s ≤ t, r ≥ 0)


Existence and uniqueness for the martingale problem 49

Theorem 2.23 Let Z be a cadlag, E-valued stochastic process, and let λ = (λ1, . . . , λd)satisfy Condition 2.17. Then there exists a unique solution of the martingale problem for(λ, Z).


Continuous time Markov chains 50

Let X be a Markov chain with values in Zd. Let Nl(t) be the number of jumps withX(s)−X(s−) = l up to time t. Then

X(t) = X(0) +∑

l

lNl(t).

Define βl(k) = qk,k+l, qk,k+l is the usual intensity for a transition from k to k + l.Then

X(t) = X(0) +∑

l

lYl(

∫ t

0

βl(X(s))ds).


Generator for a Markov chain 51

f(X(t)) = f(X(0)) +∑

l

∫ t

0

(f(X(s−) + l)− f(X(s−))dNl(s)

= f(X(0)) +∑

l

∫ t

0

(f(X(s−) + l)− f(X(s−))dMl(s)

+

∫ t

0

∑l

βl(X(s))(f(X(s) + l)− f(X(s))ds

= f(X(0)) +

∫ t

0

Af(X(s))ds+Mf (t)

whereAf(k) =

∑l

βl(k)(f(k + l)− f(k))


Martingale problem for a Markov chain 52

Definition 2.24 X is a solution of the martingale problem for A with domain D(A) ifand only if there is a filtration Ft such that

f(X(t))− f(X(0))−∫ t

0

Af(X(s))ds (2.3)

is a (local) Ft-martingale for each f ∈ D(A).

Remark 2.25 Usually, one can take D(A) to be the collection of functions with finite sup-port.

If Af is bounded, then (2.3) will be martingale.


Chemical reactions 53

Standard notation for chemical reactions

A+Bκ C

is interpreted as “a molecule ofA combines with a molecule ofB to give a moleculeof C.

A+B C

means that the reaction can go in either direction, that is, a molecule of C candissociate into a molecule of A and a molecule of B.

A + Bκ1 C at a rate proportional to the numbers of molecules of A and B and

inversely proportional to the volume.

Cκ2 A+B at a rate proportional to the number of molecules of C.

N = Avogadro’s number times the volume

Let θ = (1, 1,−1)T . Then X(t) = (XA(t), XB(t), XC(t))T satisfies

X(t) = X(0)− θY1(κ1

∫ t

0

XA(s)XB(s)

Nds) + θY2(κ2

∫ t

0

XC(s)ds)


Law of large numbers for concentrations 54

ZNA (t) = N−1XA(t) is the concentration of A.

ZN(t) = ZN(0)−N−1Y1(κ1

∫ t

0

XA(s)XB(s)

Nds)θ +N−1Y2(κ2

∫ t

0

XC(s)ds)θ

= ZN(0)−N−1Y1(Nκ1

∫ t

0

ZNA (s)ZN

B (s)ds)θ +N−1Y2(Nκ2

∫ t

0

ZNC (s)ds)θ

Since Yi(Nu)N

≈ u,

ZN(t) = ZN(0)− κ1

∫ t

0

ZNA (s)ZN

B (s)ds)θ + κ2

∫ t

0

ZNC (s)ds)θ + εN(t)

Let

Z(t) = Z(0)− κ1

∫ t

0

ZA(s)ZB(s)ds)θ + κ2

∫ t

0

ZC(s)ds)θ.

Then there exists a constant K depending on κ1, κ2, ZN(0), and Z(0) such that

|ZN(t)− Z(t)| ≤ |ZN(0)− Z(0)|+ sups≤t

|εN(t)|+K

∫ t

0

|ZN(s)− Z(s)|ds


Gronwall’s inequality 55

Lemma 2.26 Let µ be a Borel measure on [0,∞), let ε ≥ 0, and let f be a Borel measurablefunction that is bounded on bounded intervals and satisfies

0 ≤ f(t) ≤ ε+

∫[0,t)

f(s)µ(ds).

Thenf(t) ≤ εeµ[0,t).

Proof. Iterate the inequality.


Martingale problems for diffusion processes 56

The Ito equation for a diffusion process is

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

b(X(s))ds

W m-dimensional standard Brownian motion

σ : Rd → Md×m b : Rd → Rd

Define a(x) = σ(x)σT (x) (a d× d-matrix). Then by Ito’s formula

f(X(t)) = f(X(0)) +

∫ t

0

∇f(X(s))Tσ(X(s))dW (s) +

∫ t

0

Lf(X(s))ds

whereLf(x) =

1

2

∑aij(x)∂xi

∂xjf(x) +

∑i

bi(x)∂xif(x)

Consequently,

f(X(t))− f(X(0))−∫ t

0

Lf(X(s))ds

is a local martingale.


Simplified martingale problem 57

Note that

M(t) = X(t)−X(0)−∫ t

0

b(X(s))ds =

∫ t

0

σ(X(s))dW (s)

is a local martingale, as is

Mi(t)Mj(t)−∫ t

0

aij(X(s))ds

Lemma 2.27 If M1 and M2 are continuous local martingales and

M1(t)M2(t)−∫ t

0

C(s)ds

is a local martingale, then

[M1,M2]t =

∫ t

0

C(s)ds


Proof. By Ito’s formula

M1(t)M2(t) = M1(0)M2(0) +

∫ t

0

M1(s)dM2(s) +

∫ t

0

M2(s)dM1(s) + [M1,M2]t,

so M1(t)M2(t)− [M1,M2]t is a local martingale as is∫ t

0

C(s)ds− [M1,M2]t.

Since this process is finite variation it must be identically zero a.s.


Equivalence of martingale problems 59

Theorem 2.28 Let a : Rd → Md×d and b : Rd → Rd be measurable and bounded onbounded sets. Suppose that X is a continuous Rd-valued process and that there exists afiltration Ft such that

M(t) = X(t)−X(0)−∫ t

0

b(X(s))ds and Mi(t)Mj(t)−∫ t

0

aij(X(s))ds

are local martingales. Then X is a solution of the martingale problem for L with D(L) =C2

c (Rd).


Proof. Note that [Mi,Mj]t = [Xi, Xj]t so that

f(X(t)) = f(X(0)) +

∫ t

0

∇f(X(s))TdX(s) +1

2

∑i,j

∫ t

0

aij(X(s))∂xi∂xj

f(X(s))ds

= f(X(0)) +

∫ t

0

∇f(X(s))TdM(s) +

∫ t

0

Lf(X(s))ds.

For f ∈ D(L), f and Lf are bounded, so

f(X(t))− f(X(0))−∫ t

0

Lf(X(s))ds

is an Ft-martingale.


Equivalence of martingale problem and SDE 61

Theorem 2.29 Suppose there exists ε > 0 such that ε|z|2 ≤ zTa(x)z ≤ ε−1|z|2. Thenthere exists a symmetric, nonsingular matrix σ(x) such that σ(x)2 = a(x). If X is asolution of the martingale problem for L and M(t) ≡ X(t)−

∫ t

0b(X(s))ds, then

W (t) =

∫ t

0

σ−1(X(s))dM(s)

is a standard Brownian motion and

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

b(X(s))ds. (2.4)

Proof. W is a local martingale and

[Wi,Wj]t =

∫ t

0

∑k,l

σ−1ik (X(s))σ−1

jl (X(s))d[Mk,Ml]s = δijt.

(2.4) follows from the fact that∫ t

0

σ(X(s))dW (s) =

∫ t

0

σ(X(s))σ−1(X(s))dM(s) = M(t)−M(0).


3. Weak convergence for stochastic processes

• General definition of weak convergence

• Prohorov’s theorem

• Skorohod representation theorem

• Skorohod topology

• Conditions for tightness in the Skorohod topology

• Skorohod representation theorem

• Continuous mapping theorem

• Martingale central limit theorem

• Diffusion approximations


Topological proof of convergence 63

(S, d) metric space

Fn : S → R

Fn → F in some sense (e.g., xn → x implies Fn(xn) → F (x))

Fn(xn) = 0

1. Show that xn is compact

2. Show that any limit point of xn satisfies F (x) = 0

3. Show that the equation F (x) = 0 has a unique solution x0

4. Conclude that xn → x0


Convergence in distribution 64

(S, d) complete, separable metric space

Xn S-valued random variable

Xn converges in distribution to X (PXn converges weakly to PX) if for each f ∈C(S)

limn→∞

E[f(Xn)] = E[f(X)].

Denote convergence in distribution by Xn ⇒ X .

Equivalent statements

Xn converges in distribution to X if and only if

lim infn→∞

PXn ∈ A ≥ PX ∈ A, each open A,

or equivalently

lim supn→∞

PXn ∈ B ≤ PX ∈ B, each closed B,


Tightness and Prohorov’s theorem 65

A sequence Xn is tight if for each ε > 0, there exists a compact set Kε ⊂ S suchthat

supnPXn /∈ Kε ≤ ε.

Theorem 3.1 Suppose that Xn is tight. Then there exists a subsequence n(k) alongwhich the random variables converge in distribution.


Skorohod topology on DE[0,∞) 66 (E, r) complete, separable metric

space

DE[0,∞) space of cadlag, E-valued functions

xn → x ∈ DE[0,∞) in the Skorohod (J1) topology if and only if there exist strictlyincreasing λn mapping [0,∞) onto [0,∞) such that for each T > 0,

limn→∞

supt≤T

(|λn(t)− t|+ r(xn λn(t), x(t))) = 0.

The Skorohod topology is metrizable so that DE[0,∞) is a complete, separablemetric space.

Note that 1[1+ 1n

,∞) → 1[1,∞) in DR[0,∞), but (1[1+ 1n

,∞),1[1,∞)) does not converge inDR2 [0,∞).


Conditions for tightness 67

Sn0 (T ) collection of discrete Fn

t -stopping times q(x, y) = 1 ∧ r(x, y)

Theorem 3.2 Suppose that for t ∈ T0, a dense subset of [0,∞), Xn(t) is tight. Then thefollowing are equivalent.

a) Xn is tight in DE[0,∞).

b) (Kurtz) For T > 0, there exist β > 0 and random variables γn(δ, T ) such that for0 ≤ t ≤ T , 0 ≤ u ≤ δ, and 0 ≤ v ≤ t ∧ δ

E[qβ(Xn(t+ u), Xn(t)) ∧ qβ(Xn(t), Xn(t− v))|Fnt ] ≤ E[γn(δ, T )|Fn

t ]

limδ→0

lim supn→∞

E[γn(δ, T )] = 0,

andlimδ→0

lim supn→∞

E[qβ(Xn(δ), Xn(0))] = 0. (3.1)

c) (Aldous) Condition (3.1) holds, and for each T > 0, there exists β > 0 such that

Cn(δ, T ) ≡ supτ∈Sn

0 (T )

supu≤δ

E[ supv≤δ∧τ

qβ(Xn(τ + u), Xn(τ)) ∧ qβ(Xn(τ), Xn(τ − v))]

satisfies limδ→0 lim supn→∞Cn(δ, T ) = 0.


Example 68

η1, η2, . . . iid, E[ηi] = 0, σ2 = E[η2i ] <∞

Xn(t) =1√n

[nt]∑i=1

ηi

ThenE[(Xn(t+ u)−Xn(t))2|FXn

t ] =[n(t+ u)]− [nt]

nσ2 ≤ (δ +

1

n)σ2

for u ≤ δ.


Uniqueness of limit 69

Theorem 3.3 If Xn is tight in DE[0,∞) and

(Xn(t1), . . . , Xn(tk)) ⇒ (X(t1), . . . , X(tk))

for t1, . . . , tk ∈ T0, T0 dense in [0,∞), then Xn ⇒ X .

For the example, this condition follows from the central limit theorem.


Skorohod representation theorem 70

Theorem 3.4 Suppose that Xn ⇒ X . Then there exists a probability space (Ω,F , P ) andrandom variables, Xn and X , such that Xn has the same distribution as Xn, X has thesame distribution as X , and Xn → X a.s.

Continuous mapping theorem

Corollary 3.5 Let G(X) : S → E and define CG = x ∈ S : G is continuous at x.Suppose Xn ⇒ X and that PX ∈ CG = 1. Then G(Xn) ⇒ G(X).


Some mappings on DE[0,∞) 71

πt : DE[0,∞) → E πt(x) = x(t)

Cπt = x ∈ DE[0,∞) : x(t) = x(t−)

Gt : DR[0,∞) → R Gt(x) = sups≤t x(s)

CGt = x ∈ DR[0,∞) : lims→t−

Gs(x) = Gt(x) ⊃ x ∈ DR[0,∞) : x(t) = x(t−)

G : DR[0,∞) → DR[0,∞), G(x)(t) = Gt(x), is continuous

Ht : DE[0,∞) → R Ht(x) = sups≤t r(x(s), x(s−))

CHt = x ∈ DE[0,∞) : lims→t−

Hs(x) = Ht(x) ⊃ x ∈ DE[0,∞) : x(t) = x(t−)

H : DE[0,∞) → DR[0,∞), H(x)(t) = Ht(x), is continuous


Level crossing times 72

τc : DR[0,∞) → [0,∞) τc(x) = inft : x(t) > c

τ−c : DR[0,∞) → [0,∞) τ−c (x) = inft : x(t) ≥ c or x(t−) ≥ c

Gτc = Gτ−c= x : τc(x) = τ−c (x)

Note that τ−c (x) ≤ τc(x) and that xn → x implies

τ−c (x) ≤ lim infn→∞

τ−c (xn) ≤ lim supn→∞

τc(xn) ≤ τc(x)


Localization 73

Theorem 3.6 Suppose that for each α > 0, ταn is a stopping time, and Xn(· ∧ τα

n ) isrelatively compact. If

limα→∞

supnPτα

n > α = 0,

then Xn is relatively compact.

Compactification of the state space

Theorem 3.7 Let E ⊂ E0 where E0 is compact and the topology on E is the restriction ofthe topology onE0. Suppose that for each n,Xn is a process with sample paths inDE[0,∞)and that Xn ⇒ X in DE0 [0,∞). If X has sample paths in DE[0,∞), then Xn ⇒ X inDE[0,∞).


Simplified conditions for tightness 74

Sn0 (T ) collection of discrete Fn

t -stopping times q(x, y) = 1 ∧ r(x, y)

Theorem 3.8 Assume the following:

a) For t ∈ T0, a dense subset of [0,∞), Xn(t) is tight.

b) For T > 0, there exist β > 0 and random variables γn(δ, T ) such that for 0 ≤ t ≤ T ,0 ≤ u ≤ δ,

E[qβ(Xn(t+ u), Xn(t))|Fnt ] ≤ E[γn(δ, T )|Fn

t ]

andlimδ→0

lim supn→∞

E[γn(δ, T )] = 0.

Then Xn is tight in DE[0,∞).


Martingale central limit theorem 75

Theorem 3.9 Let Mn be a martingale such that

limn→∞

E[sups≤t

|Mn(s)−Mn(s−)|] = 0

and[Mn]t → ct

in probability. Then Mn ⇒√cW .

Theorem 3.10 (Vector-valued version) If for each 1 ≤ i ≤ d

limn→∞

E[sups≤t

|M in(s)−M i

n(s−)|] = 0

and for each 1 ≤ i, j ≤ d,[M i

n,Mjn]t → cijt,

then Mn ⇒ σW , where W is d-dimensional standard Brownian motion and σ is a sym-metric d× d-matrix satisfying σ2 = c = ((cij)).


Proof of tightness 76

To avoid a truncation argument, assume that Mn is square integrable and thatE[|[Mn]t − ct|] → 0. In particular, supnE[Mn(t)2] = supnE[[Mn]t] < ∞, so Mn(t)is tight.

By monotonicity and the dominated convergence theorem, this assumption im-plies

E[supt≤T

|[Mn]t − ct|] → 0. (3.2)

E[(Mn(t+ u)−Mn(t))2|Fnt ] = E[[Mn]t+u − [Mn]t|Fn

t ]

≤ E[supr≤T

([Mn]r+δ − [Mn]r)|Fnt ],

for t ≤ T and 0 ≤ u ≤ δ, and (3.2) implies

limδ→0

limn→∞

E[supr≤T

([Mn]r+δ − [Mn]r)] = limδ→0

cδ = 0.

Consequently, Mn is tight.


Martingale properties of the limit 77

Assume that Mn ⇒ M , selecting a subsequence if necessary. Since Mn is a martin-gale

E[(Mn(t+ s)−Mn(t))m∏

i=1

fi(Mn(ti))] = 0

for 0 ≤ t1 < · · · < tm ≤ t < t + s and fi ∈ C(R). Convergence in distributionand the uniform integrability of the sequence (Mn(t+s)−Mn(t))

∏mi=1 fi(Mn(ti))

implies

E[(M(t+ s)−M(t))m∏

i=1

fi(M(ti))] = 0

which in turn implies M is a martingale.

Let τ rn = inft : |Mn(t)| ≥ r. Then M τr

n (t)2 ≤ 2([Mn]t + r2), so M τrn (t)2 is uni-

formly integrable. By the same argument as above, it follows that

M(t)2 − ct

is a local martingale, and hence, M is a Brownian motion.


Example 78

Let Y be a unit Poisson process. Define

X(t) =

∫ t

0

(−1)N(s)ds

and

Xn(t) =1

nX(n2t) = n

∫ t

0

(−1)N(n2s)ds

Since

(−1)N(t) = 1−∫ t

0

2(−1)N(s−)dN(s) = 1−∫ t

0

2(−1)N(s−)dM(s)−∫ t

0

2(−1)N(s−)ds

Consequently,

Xn(t) =1− (−1)N(n2t)

2n− 1

n

∫ n2t

0

(−1)N(s−)dM(s) = εn(t) +Mn(t)

Since [Mn] = N(n2t)n2 → t and supt |εn(t) → 0, Xn ⇒ W .


Convergence of coupled counting processes 79

Theorem 3.11 For each n, letNn1 , . . . , N

nm be counting processes satisfying [Nn

i , Nnj ]t = 0

for i 6= j (that is, there are no simultaneous jumps). Suppose that Hni are nondecreasing

processes with Hni (t)−Hn

i (t−) ≤ 1 for all i and t ≥ 0, that

Nni −Hn

i , i = 1, . . . ,m

are Gnt -martingales, and thatHn

i (t) is Gn0 -measurable for each i and t ≥ 0. If (Hn

1 , . . . , Hnm) ⇒

H = (H1, . . . , Hm) in the Skorohod topology on DRm [0,∞), then (Nn1 , . . . , N

nm) ⇒

(N1, . . . , Nm) where (N1, . . . , Nm) are counting processes with joint distribution deter-mined by

ϕf (t) = E[e−

∑i=1

∫ t0 fi(s)dNi(s)

∣∣∣H]= 1 +

m∑i=1

∫ t

0

ϕf (u−)(e−fi(u) − 1)dHi(u)

for all nonnegative, continuous, Rm-valued functions f = (f1, . . . , fm).


Proof. Using the fact that there are no simultaneous jumps among the Nni ,

e−∑m

i=1

∫ t0 fi(s)dNn

i (s) = 1 +m∑

i=1

∫ t

0

(e−fi(u) − 1)e−∫ u−0 fi(s)dNn

i (s)dNni (u) (3.3)

= 1 +m∑

i=1

∫ t

0


i (s)d(Nni (u)−Hn

i (u))

+m∑

i=1

∫ t

0


i (s)dHni (u) .

Using the martingale assumption and the measurability assumption, conditioningboth sides of (3.3) on Hn, we have

ϕnf (t) = E

[e−

∑i=1

∫ t0 fi(s)dNn

i (s)∣∣∣Hn

]1 +

m∑i=1

∫ t

0

ϕnf (u−)(e−fi(u) − 1)dHn

i (u)

and the convergence of Hn to H implies the convergence of ϕnf to ϕf . Convergence

of the finite dimensional distributions follows. Convergence in distribution of theprocesses under the Skorohod topology follows by Exercise 14.


Diffusion approximations 81

Theorem 3.12 Let a : Rd → Md×d be continuous, symmetric and nonnegative definite,and let b : Rd → Rd be continuous. Suppose that for each ν ∈ P(Rd), the martingaleproblem for (L, ν) with D(L) = C2

c (Rd) has a unique solution in CRd [0,∞).

Let Xn and Bn be a cadlag, Rd-valued processes and let An be a cadlag, Md×d-valuedprocess. Suppose

Mn ≡ Xn −Bn and M inM

jn − Aij

n , i, j = 1, . . . , d

are Fnt -local martingales and that for each T > 0, and i, j = 1, . . . , d,

limn→∞

E[supt≤T

|Xn(t)−Xn(t−)|2]+E[supt≤T

|Bn(t)−Bn(t−)|2]+E[supt≤T

|Aijn (t)−Aij

n (t−)|] = 0

and

supt≤T

|Bin(t)−

∫ t

0

bi(Xn(s))ds|+ supt≤T

|Aijn (t)−

∫ t

0

aij(Xn(s))ds| → 0

in probability.

Let X be a solution of the martingale problem for L, and suppose Xn(0) ⇒ X(0). ThenXn ⇒ X .


Example 82

Markov chain: H : E × U → E, ξ1, ξ2, . . . iid U -valued, X0 E-valued and indepen-dent of ξk. for k ≥ 0, define Xk+1 = H(Xk, ξk+1). Then Xk is a Markov chain.Taking E = R, let

Xnk+1 = Xn

k +1√nGn(Xn

k , ξk+1).

For ∫U

Gn(x, u)ν(du) =1√nbn(x) an(x) =

∫U

Gn(x, u)2ν(du),

Mnm = Xn

m −Xn0 −

1

n

m−1∑k=0

bn(Xnk ) (Mn

m)2 − 1

n

m−1∑k=0

(an(Xnk )− 1

nb2n(Xn

k ))

are martingales. Set

Xn(t) = Xn[nt]

Bn(t) =1

n

[nt]−1∑k=0

bn(Xnk ) =

∫ [nt]n

0

bn(Xn(s))ds


An(t) =1

n

[nt]−1∑k=0

(an(Xnk )− 1

nb2n(Xn

k )) =

∫ [nt]n

0

an(Xn(s))ds− 1

n

∫ [nt]n

0

b2n(Xn(s))ds


Wright-Fisher model 84

PXnk+1 =

l

n|Xn

k = x =

(n

l

)xl(1− x)n−l

E[Xnk+1 −Xn

k |Xnk = x] = 0 E[(Xn

k+1 −Xnk )2|Xn

k = x] =1

nx(1− x)

Let Xn(t) = Xn[nt].

Psupt≤T

|Xn(t)−Xn(t−)| ≥ ε ≤[nt]−1∑k=0

P|Xnk+1−Xn

k | ≥ ε ≤ [nt]

ε4n4max

xE[(ξ[nt],x−[nt]x)4]

where ξm,x is binomially distributed with parameters m and x. Then

limn→∞

E[supt≤T

|Xn(t)−Xn(t−)|2] = 0

by uniform integrability. Xn is approximated by the diffusion with generator

Lf(x) =1

2x(1− x)f ′′(x)


4. Stochastic equations for general Markov processes in Rd

• Poisson random measures

• Stochastic integrals for space-time Poisson random measures

• Stochastic integrals for centered space-time Poisson random measures

• Stochastic equations for Markov processes


Poisson distribution 86

Definition 4.1 A random variable X has a Poisson distribution with parameter λ > 0(write X ∼ Poisson(λ)) if for each k ∈ 0, 1, 2, . . .

PX = k =λk

k!e−λ.

E[X] = λ V ar(X) = λ

and characteristic function of X is

E[eiθX ] = eλ(eiθ−1).

Since the characteristic function of a random variable characterizes its distribution,a direct computation gives

Proposition 4.2 If X1, X2, . . . are independent random variables with Xi ∼ Poisson(λi)and

∑∞i=1 λi <∞, then

X =∞∑i=1

Xi ∼ Poisson

(∞∑i=1

λi

)


Poisson sums of Bernoulli random variables 87

Proposition 4.3 Let N ∼ Poisson(λ), and suppose that Y1, Y2, . . . are i.i.d. Bernoullirandom variables with parameter p ∈ [0, 1]. If N is independent of the Yi, then

∑Ni=0 Yi ∼

Poisson(λp).

For j = 1, . . . ,m, let ej be the vector in Rm that has all its entries equal to zero,except for the jth which is 1.

For θ, y ∈ Rm, let

〈θ, y〉 =m∑

j=1

θjyj.

Proposition 4.4 Let N ∼ Poisson(λ). Suppose that Y1, Y2, . . . are independent Rm-valued random variables such that for all k ≥ 0 and j ∈ 1, . . . ,m

PYk = ej = pj,

where∑m

j=1 pj = 1. Define X = (X1, ..., Xm) =∑N

k=0 Yk. If N is independent of the Yk,then X1, . . . , Xm are independent random variables and Xj ∼ Poisson(λpj).


Poisson random measures 88

Let ν be a σ-finite measure on U and (U, dU) be a complete, separable metric space.Let N (U) denote the collection of counting measures on U .

Definition 4.5 A Poisson random measure on U with mean measure ν is a randomcounting measure ξ (that is, a N (U)-valued random variable) such that

a) For A ∈ B(U), ξ(A) has a Poisson distribution with expectation ν(A)

b) ξ(A) and ξ(B) are independent if A ∩B = ∅.

For f ∈M(U), f ≥ 0, define

ψξ(f) = E[exp−∫

U

f(u)ξ(du)] = exp−∫

(1− e−f )dν

(Verify the second equality by approximating f by simple functions.)


Existence 89

Proposition 4.6 Suppose that ν is a measure on U such that ν(U) <∞. Then there existsa Poisson random measure with mean measure ν.

Proof. The case ν(U) = 0 is trivial, so assume that ν(U) ∈ (0,∞). Let N be aPoisson random variable defined on a probability space (Ω,F , P ) with E[N ] =ν(U). Let X1, X2, . . . be iid U -valued random variables such that for every A ∈B(U),

PXj ∈ A =ν(A)

ν(U),

and assume that N is independent of the Xj .

Define ξ by ξ(A) =∑N

k=0 1Xk∈A. In other words ξ =∑N

k=0 δXkwhere, for each

x ∈ U , δx is the Dirac mass at x.

Extend the existence result to σ-finite measures by partitioning U = ∪iUi, whereν(Ui) <∞.


Identities 90

Let ξ be a Poisson random measure with mean measure ν.

Lemma 4.7 Suppose f ∈M(U), f ≥ 0. Then

E[

∫f(y)ξ(dy)] =

∫f(y)ν(dy)

Lemma 4.8 Suppose ν is nonatomic and let f ∈M(N (U)× U), f ≥ 0. Then

E[

∫U

f(ξ, y)ξ(dy)] = E[

∫U

f(ξ + δy, y)ν(dy)]


Proof. Suppose 0 ≤ f ≤ 1U0 , where ν(U0) < ∞. Let U0 = ∪kUnk , where the Un

k aredisjoint and diam(Un

k ) ≤ n−1. If ξ(Unk ) is 0 or 1, then∫

Unk

f(ξ, y)ξ(dy) =

∫Un

k

f(ξ(· ∩ Un,ck ) + δy, y)ξ(dy)

Consequently, if maxk ξ(Unk ) ≤ 1,∫

U0

f(ξ, y)ξ(dy) =∑

k

∫Un

k

f(ξ(· ∩ Un,ck ) + δy, y)ξ(dy)

Since ξ(U0) <∞, for n sufficiently large, maxk ξ(Unk ) ≤ 1,

E[

∫U

f(ξ, y)ξ(dy)] = E[

∫U0

f(ξ, y)ξ(dy)]

= limn→∞

∑k

E[

∫Un

k

f(ξ(· ∩ Un,ck ) + δy, y)ξ(dy)]

= limn→∞

∑k

E[

∫Un

k

f(ξ(· ∩ Un,ck ) + δy, y)ν(dy)]

= E[

∫U

f(ξ + δy, y)ν(dy)].


Note that the last equality follows from the fact that

f(ξ(· ∩ Un,ck ) + δy, y) 6= f(ξ + δy, y)

only if ξ(Unk ) > 0, and hence, assuming 0 ≤ f ≤ 1U0 ,

|∑

k

∫Un

k

f(ξ(· ∩ Un,ck ) + δy, y)ν(dy)−

∫U0

f(ξ + δy, y)ν(dy)| ≤∑

k

ξ(Unk )ν(Un

k ),

where the expectation of the right side is∑

k ν(Unk )2 =

∫U0ν(Un(y))ν(dy) ≤

∫U0ν(U0∩

B1/n(y))ν(dy), where Un(y) = Unk if y ∈ Un

k . limn→∞ ν(U0 ∩ B1/n(y)) = 0, since ν isnonatomic.


Space-time Poisson random measures 93

Let ξ be a Poisson random measure on U × [0,∞) with mean measure ν× ` (where` denotes Lebesgue measure).

ξ(A, t) ≡ ξ(A× [0, t]) is a Poisson process with parameter ν(A).

ξ(A, t) ≡ ξ(A× [0, t])− ν(A)t is a martingale.

Definition 4.9 ξ is Ft compatible, if for each A ∈ B(U), ξ(A, ·) is Ft adapted andfor all t, s ≥ 0, ξ(A× (t, t+ s]) is independent of Ft.


Stochastic integrals for Poisson random measures 94

For i = 1, . . . ,m, let ti < ri and Ai ∈ B(U), and let ηi be Fti-measurable. Let

X(u, t) =∑

i

ηi1Ai(u)1[ti,ri)(t),

and note thatX(u, t−) =

∑i

ηi1Ai(u)1(ti,ri](t). (4.1)

DefineIξ(X, t) =

∫U×[0,t]

X(u, s−)ξ(du× ds) =∑

i

ηiξ(Ai × (ti, ri]).

Then

E [|Iξ(X, t)|] ≤ E

[∫U×[0,t]

|X(u, s−)|ξ(du× ds)

]=

∫U×[0,t]

E[|X(u, s)|]ν(du)ds

and if the right side is finite, E[Iξ(X, t)] =∫

U×[0,t]E[X(u, s)]ν(du)ds.


Estimates in L1,0 95

|Iξ(X, t)| ∧ 1 ≤∫

U×[0,t]

|X(u, s−)| ∧ 1ξ(du× ds)

and

E

[supt≤T

|Iξ(X, t)| ∧ 1

]≤∫

U×[0,T ]

E[|X(u, s)| ∧ 1]ν(du)ds

Definition 4.10 Let L1,0(U, ν) denote the space of B(U)×B[0,∞)×F-measurable map-pings (u, s, ω) → X(u, s, ω) such that

∫∞0e−s∫

UE[|X(u, s)| ∧ 1]ν(du)ds <∞.

Let S− denote the collection of B(U)×B[0,∞)×F measurable mappings (u, s, t) →∑mi=1 ηi(ω)1Ai

(u)1(ti,ri](t) defined as in (4.1).

Lemma 4.11

d1,0(X, Y ) =

∫ ∞

0

e−s

∫U

E[|X(u, s)− Y (u, s)| ∧ 1]ν(du)ds

defines a metric on L1,0(U, ν), and the definition of Iξ extends to the closure of S− inL1,0(U, ν).


The predictable σ-algebra 96

Warning: LetN be a unit Poisson process. Then∫∞

0e−sE[|N(s)−N(s−)|∧1]ds = 0,

but P∫ t

0N(s)dN(s) 6=

∫ t

0N(s−)dN(s) = 1− e−t.

Definition 4.12 Let (Ω,F , P ) be a probability space and let Ft be a filtration in F .The σ-algebra P of predictable sets is the smallest σ-algebra in B(U) × B[0,∞) × Fcontaining sets of the form A× (t0, t0 + r0]×B for A ∈ B(U), t0, r0 ≥ 0, and B ∈ Ft0 .

Remark 4.13 Note that for B ∈ Ft0 , 1A×(t0,t0+r0]×B(u, t, ω) is left continuous in t andadapted and that the mapping (u, t, ω) → X(u, t−, ω), where X(u, t−, ω) is defined in(4.1), is P-measurable.

Definition 4.14 A stochastic process X on U × [0,∞) is predictable if the mapping(u, t, ω) → X(u, t, ω) is P-measurable.

Lemma 4.15 If the mapping (u, t, ω) → X(u, t, ω) is B(U) × B[0,∞) × F-measurableand adapted and is left continuous in t, then X is predictable.

Proof.Let 0 = tn0 < tn1 < · · · and tni+1 − tni ≤ n−1. Define Xn(u, t, ω) = X(u, tni , ω)for tni < t ≤ tni+1. Then Xn is predictable and limn→∞Xn(u, t, ω) = X(u, t, ω) for all(u, t, ω).


Stochastic integrals for predictable processes 97

Lemma 4.16 LetG ∈ P ,B ∈ B(U) with ν(B) <∞ and b > 0. Then 1B×[0,b](u, t)1G(u, t, ω)is a predictable process and

Iξ(1B×[0,b]1G, t)(ω) =

∫U×[0,t]

1B×[0,b](u, s)1G(u, s, ω)ξ(du× ds, ω) a.s. (4.2)

and

E[

∫U×[0,t]

1B×[0,b](u, s)1G(u, s, ·)ξ(du× ds)] = E[

∫U×[0,t]

1B×[0,b]1G(u, s, ·)ν(du)ds]

(4.3)

Proof. Let

A = ∪mi=1Ai × (ti, ti + ri]×Gi : ti, ri ≥ 0, Ai ∈ B(U), Gi ∈ Fti.

ThenA is an algebra, (4.2) holds by definition, and (4.3) holds by direct calculation.The collection ofG that satisfy (4.2) and (4.3) is closed under increasing unions anddecreasing intersections, and the monotone class theorem (see Theorem 4.1 of theAppendix of [2]) gives the lemma.


98

Lemma 4.17 LetX be a predictable process satisfying∫∞

0e−s∫

UE[|X(u, s)|∧1]ν(du)ds <

∞. Then∫

U×[0,t]|X(u, t)|ξ(du× ds) <∞ a.s. and

Iξ(X, t)(ω) =

∫U×[0,t]

X(u, t, ω)ξ(du× ds, ω) a.s.

Proof. Approximate by simple functions.


Consequences of predictability 99

Lemma 4.18 If X is predictable and∫

U×[0,t]|X(u, s)| ∧ 1ν(du)ds <∞ a.s. for all t, then∫

U×[0,t]

|X(u, s)|ξ(du× ds) <∞ a.s. (4.4)

and ∫U×[0,t]

X(u, s)ξ(du× ds)

exists a.s.

Proof. Let τc = inft :∫

U×[0,t]|X(u, s)| ∧ 1ν(du)ds ≥ c, and consider Xc(s, u) =

1[0,τc](s)X(u, s). Then Xc satisfies the conditions of Lemma 4.17, so∫U×[0,t]

|X(u, s)| ∧ 1ξ(du× ds) <∞ a.s.

But this implies ξ(u, s) : s ≤ t, |X(u, s)| > 1 <∞, so (4.4) holds.


Martingale properties 100

Theorem 4.19 Suppose X is predictable and∫

U×[0,t]E[|X(u, s)|]ν(du)ds < ∞ for each

t > 0. Then ∫U×[0,t]

X(u, s)ξ(du× ds)−∫ t

0

∫U

X(u, s)ν(du)ds

is a Ft-martingale.


Proof. LetA ∈ Ft and defineXA(u, s) = 1AX(u, s)1(t,t+r](s). ThenXA is predictableand

E[1A

∫U×(t,t+r]

X(u, s)ξ(du× ds)] = E[

∫U×[0,t+r]

XA(u, s)ξ(du× ds)]

= E[

∫U×[0,t+r]

XA(u, s)ν(du)ds]

= E[1A

∫U×(t,t+r]

X(u, s)ν(du)ds]

and hence

E[

∫U×(t,t+r]

X(u, s)ξ(du× ds)|Ft] = E[

∫U×(t,t+r]

X(u, s)ν(du)ds|Ft].


Local martingales 102

Lemma 4.20 If ∫U×[0,t]

|X(u, s)|ν(du)ds <∞ a.s. t ≥ 0,

then ∫U×[0,t]

X(u, s)ξ(du× ds)−∫

U×[0,t]

X(u, s)ν(du)ds


Proof. If τ is a stopping time and X is predictable, then 1[0,τ ](t)X(u, t) is pre-dictable. Let

τc = t > 0 :

∫U×[0,t]

|X(u, s)|ν(du)ds ≥ c.

Then ∫U×[0,t∧τc]

X(u, s)ξ(du× ds)−∫

U×[0,t∧τc]

X(u, s)ν(du)ds

=

∫U×[0,t]

1[0,τc](s)X(u, s)ξ(du× ds)−∫

U×[0,t]

1[0,τc](s)X(u, s)ν(du)ds.


is a martingale.


Representation of counting processes 104

Let U = [0,∞) and ν = `. Let λ be a nonnegative, predictable process, and defineG = (u, t) : u ≤ λ(t). Then

N(t) =

∫[0,∞)×[0,t]

1G(u, s)ξ(du× ds) =

∫[0,∞)×[0,t]

1[0,λ(s)](u)ξ(du× ds)

is a counting process with intensity λ.

Stochastic equation for a counting process

λ : Dc[0,∞) × [0,∞) → [0,∞), λ(z, t) = λ(zt, t), t ≥ 0, λ(z, t) cadlag for eachz ∈ Dc[0,∞).

N(t) =

∫[0,∞)×[0,t]

1[0,λ(N,s−)](u)ξ(du× ds)


Semimartingale property 105

Corollary 4.21 If X is predictable and∫

U×[0,t]|X(u, s)| ∧ 1ν(du)ds < ∞ a.s. for all t,

then∫

U×[0,t]|X(u, s)|ξ(du× ds) <∞ a.s.∫

U×[0,t]

X(u, s)ξ(du× ds)

=

∫U×[0,t]

1|X(u,s|≤1X(u, s)ξ(du× ds)−∫ t

0

∫U

1|X(u,s)|≤1X(u, s)ν(du)ds︸︷︷︸local martingale

+

∫ t

0

∫U

1|X(u,s)|≤1X(u, s)ν(du)ds+

∫U×[0,t]

1|X(u,s|>11X(u, s)ξ(du× ds)︸︷︷︸finite variation

is a semimartingale.


Stochastic integrals for centered Poisson random measures 106

Let ξ(du× ds) = ξ(du× ds)− ν(du)ds

ForX(u, t−) =

∑i

ηi1Ai(u)1(ti,ri](t).

as in (4.1), define

Iξ(X, t) =

∫U×[0,t]

X(u, s−)ξ(du×ds) =

∫U×[0,t]

X(u, s)ξ(du×ds)−∫ t

0

∫U

X(u, s)ν(du)ds

and note thatE[Iξ(X, t)

2]

=

∫U×[0,t]

E[X(u, s)2]ν(du)ds

if the right side is finite.

Then Iξ(X, ·) is a square-integrable martingale.


Extension of integral 107

The integral extends to predictable integrands satisfying∫U×[0,t]

|X(u, s)|2 ∧ |X(u, s)|ν(du)ds <∞ a.s. (4.5)

so that ∫U×[0,t∧τ ]

X(u, s)ξ(du× ds) =

∫U×[0,t]

1[0,τ ](s)X(u, s)ξ(du× ds) (4.6)

is a martingale for any stopping time satisfying

E

[∫U×[0,t∧τ ]

|X(u, s)|2 ∧ |X(u, s)|ν(du)ds]<∞,

and (4.6) is a local square integrable martingale if∫U×[0,t]

|X(u, s)|2ν(du)ds <∞ a.s.


Quadratic variation 108

Note that if X is predictable and∫U×[0,t]

|X(u, s)| ∧ 1ν(du)ds <∞ a.s. t ≥ 0,

then ∫U×[0,t]

|X(u, s)|2 ∧ 1ν(du)ds <∞ a.s. t ≥ 0,

and[Iξ(X, ·)]t =

∫U×[0,t]

X2(u, s)ξ(du× ds).

Similarly, if ∫U×[0,t]

|X(u, s)|2 ∧ |X(u, s)|ν(du)ds <∞ a.s.,

[Iξ(X, ·)]t =

∫U×[0,t]

X2(u, s)ξ(du× ds).


Semimartingale properties 109

Theorem 4.22 Let Y be a cadlag, adapted process. If X satisfies (4.4), Iξ(X, ·) is a semi-martingale and ∫ t

0

Y (s−)dIξ(X, s) =

∫U×[0,t]

Y (s−)X(u, s)ξ(du× ds),

and if X satisfies (4.5), Iξ(X, ·) is a semimartingale and∫ t

0

Y (s−)dIξ(X, s) =

∫U×[0,t]

Y (s−)X(u, s)ξ(du× ds)


Levy processes 110

Theorem 4.23 Let U = R and∫

R |u|2 ∧ 1ν(du) <∞. Then

Z(t) =

∫[−1,1]×[0,t]

uξ(du× ds) +

∫[−1,1]c×[0,t]

uξ(du× ds)

is a process with stationary, independent increments with

E[eiθZ(t)] = expt∫

R(eiθu − 1− iθu1[−1,1](u))ν(du)


Proof.

eiθZ(t) = 1 +

∫ t

0

iθeiθZ(s−)dZ(s−) +∑s≤t

(eiθZ(s) − eiθZ(s−) − iθeiθZ(s−)∆Z(s))

= 1 +

∫[−1,1]×[0,t]

iθeiθZ(s−)uξ(du× ds) +

∫[−1,1]c×[0,t]

iθeiθZ(s−)uξ(du× ds)

+

∫R×[0,t]

(eiθ(Z(s−)+u) − eiθZ(s−) − iθeiθZ(s−)u)ξ(du× ds)

= 1 +

∫[−1,1]×[0,t]

iθeiθZ(s−)uξ(du× ds)

+

∫R×[0,t]

eiθZ(s−)(eiθu − 1− iθu1[−1,1](u))ξ(du× ds)

Taking expectations

ϕ(θ, t) = 1 +

∫R×[0,t]

ϕ(θ, s)(eiθu − 1− iθu1[−1,1](u))ν(du)ds

so ϕ(θ, t) = expt∫

R(eiθu − 1− iθu1[−1,1](u))ν(du)


Scaling 112

Let U = R and write ξ =∑δ(ui,si). Define ξa,b =

∑δ(aui,bsi). Then ξa,b is a Poisson

random measure with mean measure b−1νa(du)ds where νa(c, d) = ν(a−1c, a−1d). Ifν has a density γ, then γa(u) = a−1γ(a−1u).

Let

Za,b(t) =

∫[−1,1]×[0,t]

uξa,b(du× ds) +

∫[−1,1]c×[0,t]

uξa.b(du× ds)

=

∫[−a−1,a−1]×[0,b−1t]

auξ(du× ds) +

∫[−a−1,a−1]c×[0,b−1t]

auξ(du× ds)

= aZ(b−1t) +

∫ ∞

−∞au(1[−1,1](u)− 1[−a−1,a−1](u))ν(du)b

−1t

Example: γ(u) = c|u|−1−α. Then the measure for ξa,b is caα|u|−1−αb−1duds and the“drift” term on the right vanishes by symmetry. Consequently, if b = aα, thenZa,b(t) = aZ(a−αt) has the same distribution as Z.


Approximation of Levy processes 113

For 0 < ε < 1, let

Zε(t) =

∫[−1−ε)∪(ε,1]×[0,t]

uξ(du× ds) +

∫[−1,1]c×[0,t]

uξ(du× ds)

=

∫(−∞,−ε)∪(ε,∞)×[0,t]

uξ(du× ds)− t

∫[−1−ε)∪(ε,1]

uν(du)

that is, throw out all jumps of size less than or equal to ε and the correspondingcentering. Then

E[|Zε(t)− Z(t)|2] = t

∫[−ε,ε]

u2ν(du).

Consequently, since Zε − Z is a square integrable martingale, Doob’s inequalitygives

limε→0

E[sups≤t

|Zε(s)− Z(s)|2] = 0.


Summary on stochastic integrals 114

If X is predictable and∫

U×[0,t]|X(u, s)| ∧ 1ν(du)ds <∞ a.s. for all t, then∫

U×[0,t]

|X(u, s)|ξ(du× ds) <∞ a.s.

∫U×[0,t]

X(u, s)ξ(du× ds)

=

∫U×[0,t]

1|X(u,s)|≤1X(u, s)ξ(du× ds)−∫ t

0

∫U

1|X(u,s)|≤1X(u, s)ν(du)ds︸︷︷︸local martingale

+

∫ t

0

∫U

1|X(u,s)|≤1X(u, s)ν(du)ds+

∫U×[0,t]

1|X(u,s)|>1X(u, s)ξ(du× ds)︸︷︷︸finite variation

is a semimartingale.

Exercise 4.24 Give an example of a right continuous, adapted processX such that∫

U×[0,t]|X(u, s)|∧

1ν(du)ds <∞ a.s. but∫

U×[0,t]|X(u, s)|ξ(du× ds) = ∞ a.s.


115

If X is predictable and∫U×[0,t]

|X(u, s)|2 ∧ |X(u, s)|ν(du)ds <∞ a.s.,

then∫U×[0,t]

X(u, s)ξ(du× ds)

= limε→0+

∫U×[0,t]

1|X(u,s)|≥ε(s)X(u, s)ξ(du× ds)

= limε→0+

(∫U×[0,t]

1|X(u,s)|≥εX(u, s)ξ(du× ds)−∫ t

0

∫U

1|X(u,s)|≥εX(u, s)ν(du)ds

)exists and is a local martingale.


Markov processes 116

Markov chain: Xn+1 = H(Xn, ηn+1), ηn iid and independent of X0.

Rd-valued Markov process:

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

b(X(s))ds (4.7)

+

∫U1×[0,t]

α1(X(s−), u)ξ(du× ds) +

∫U2×[0,t]

α2(X(s−), u)ξ(du× ds)

where σ : Rd → Md×m, b : Rd → Rd, and for each compact K ⊂ Rd,

supx∈K

(|σ(x)|+ |b(x)|+

∫U1

|α1(x, u)|2ν(du) +

∫U2

|α2(x, u)| ∧ 1ν(du))<∞ (4.8)


Uniqueness and the Markov property 117

If X is a solution of (4.7), then

X(t) = X(r) +

∫ t

r

σ(X(s))dW (s) +

∫ t

r

b(X(s))ds

+

∫U1×(r,,t]

α1(X(s−), u)ξ(du× ds) +

∫U2×(r,t]


Uniqueness implies X(r) is independent of W (· + r) −W (r) and ξ(A × (r, ·]) andthat X(t), t ≥ r is determined by X(r), W (· + r) −W (r) and ξ(A × (r, ·]), whichgives the Markov property.


Random measures 119

Π is a random measure on U × [0,∞) if for each ω ∈ Ω, Π(·, ω) is a measure onB(U) × B[0,∞) and for each G ∈ B(U) × B[0,∞), Π(G) is a random variable withvalues in [0,∞]. Π is adapted if for each D ∈ B(U), the process Π(D × [0, t]) isadapted, and Π is predictable if for each D ∈ B(U), the process Π(D × [0, t]) ispredictable.


Dual predictable projection 120

The following result is essentially the existence of the dual predictable projectionof a random measure. (See, for example, Jacod and Shiryaev [4], Theorem II.1.8.)

Lemma 4.25 Let Ft be a complete, right-continuous filtration. Let Γ be a randommeasure on U × [0,∞) (not necessarily adapted) satisfying Γ(U ×0) = 0. Suppose thatthere exists a strictly positive predictable process H such that

E

[∫U×[0,∞)

H(u, s)Γ(du× ds)

]<∞.

Then there exists a predictable random measure Γ such that for each predictable Z satisfy-ing |Z| < K for some constant K <∞,

MZ(t) = E[

∫U×[0,t]

Z(u, s)H(u, s)Γ(du×ds)|Ft]−∫

U×[0,t]

Z(u, s)H(u, s)Γ(du×ds),

(4.9)is an Ft-martingale. In addition, there exist a kernel γ from ([0,∞)× Ω,P) to U and anondecreasing, right-continuous, predictable process A such that

Γ(du× ds, ω) = γ(s, ω, du)dA(s, ω).


Proof. See Appendix 1.


A martingale inequality 122

Discrete time version: Burkholder [1]

Continuous time version: Lenglart, Lepingle, and Pratelli [9]

See Lemma 2.4 of [8] for an easy proof due to Ichikawa [3].

If M is a local square integrable martingale, there exists a nondecreasing pre-dictable process 〈M〉 such that M(t)2 − 〈M〉t is a local martingale. In particular,[M ] − 〈M〉 is a local martingale. Recall that a left-continuous, adapted process ispredictable.

Lemma 4.26 For 0 < p ≤ 2 there exists a constant Cp such that for any local squareintegrable martingale M with Meyer process 〈M〉 and any stopping time τ

E[sups≤τ

|M(s)|p] ≤ CpE[〈M〉p/2τ ]


123

Proof. For p = 2 the result is an immediate consequence of Doob’s inequality. Let0 < p < 2. For x > 0, let σx = inft : 〈M〉t > x2. Since σx is predictable, there existsa strictly increasing sequence of stopping times σn

x → σx. Noting that 〈M〉σnx≤ x2,

we have

Psups≤τ

|M(s)| > x ≤ Pσnx < τ+ P sup

s≤τ∧σnx

|M(s)| > x

≤ Pσnx < τ+

E[〈M〉τ∧σnx]

x2

≤ Pσnx < τ+

E[x2 ∧ 〈M〉τ ]x2

,

and letting n→∞, we have

Psups≤τ

|M(s)| > x ≤ P〈M〉τ ≥ x2+E[x2 ∧ 〈M〉τ ]

x2. (4.10)

Using the identity ∫ ∞

0

E[x2 ∧X2]pxp−3dx =2

2− pE[|X|p],

the lemma follows by multiplying both sides of (4.10) by pxp−1 and integrating.


Computation of Meyer process 124

Let ξ be a Poisson random measure on U × [0,∞) with mean measure ν × `, andlet W be a standard Brownian motion. Assume that ξ and W are compatible witha filtration Ft.

IfX is predictable and∫

U×[0,t]X2(u, s)ν(du)ds <∞ a.s., thenM(t) =

∫U×[0,t]

X(u, s)ξ(du×ds) is a local square integrable martingale with

〈M〉t =

∫U×[0,t]

X2(u, s)ν(du)ds <∞.

If X is adapted and∫ t

0X(s)2ds < ∞ a.s., then M(t) =

∫ t

0X(s)dW (s) is a local

square integrable martingale with

〈M〉t =

∫ t

0

X(s)2ds.


5. Martingale problems for Markov processes

• The martingale problem for a generator

• Generator for an SDE

• Pure jump processes

• Dynkin’s identity

• Moment estimates

• Lyapunov functions


Martingale problems 127

An E-valued process X is an Ft-Markov process if X is Ft-adapted and

E[g(X(t+ s))|Ft] = E[g(X(t+ s))|X(t)], g ∈ B(E).

The generator of a Markov process determines its short time behavior

E[g(X(t+ ∆t))− g(X(t))|Ft] ≈ Ag(X(t))∆t

Definition 5.1 X is a solution of the martingale problem forA if and only if there existsa filtration Ft such that X is Ft-adapted and

g(X(t))− g(X(0))−∫ t

0

Ag(X(s))ds (5.1)

is an Ft-martingale for each g ∈ D(A).

For ν ∈ P(E), X is a solution of the martingale problem for (A, ν) if X is a solution of themartingale problem for A and X(0) has distribution ν.


Generator for the SDE 128

Let

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

b(X(s))ds

+

∫U1×[0,t]

α1(X(s−), u)ξ(du× ds) +

∫U2×[0,t]

α2(X(s−), u)ξ(du× ds) .

Then

f(X(t))− f(X(0))−∫ t

0

Af(X(s))ds

=

∫ t

0

∇f(X(s))Tσ(X(s))dW (s)

+

∫U1

(f(X(s−) + α1(X(s−), u))− f(X(s−))ξ(du× ds)

+

∫U2

(f(X(s−) + α2(X(s−), u))− f(X(s−))ξ(du× ds)

Note that, assuming (4.8), f ∈ C2c (Rd), and that X exists for all t ≥ 0, the right side

is a local square integrable martingale


Form of the generator 129

Af(x) =1

2

∑aij(x)∂i∂jf(x) + b(x) · ∇f(x)

+

∫U1

(f(x+ α1(x, u))− f(x)− α1(x, u) · ∇f(x))ν(du)

+

∫U2

(f(x+ α2(x, u))− f(x))ν(du)

Let D(A) be a collection of functions for which Af is bounded. Then a solution ofthe SDE is a solution of the martingale problem for A.


Pure jump processes 130

ConsiderAf(x) = λ(x)

∫E

(f(y)− f(x))µ(x, dy)

λ ≥ 0, µ a transition function.

The corresponding Markov process stays in a state x for an exponential length oftime with parameter λ(x) and then jumps to a new point with distribution µ(x, ·).

There exists a space U0, a probability measure ν0 ∈ P(U0), and a measurable map-ping H : E × U0 → E such that µ(x,Γ) = ν0(u : H(x, u) ∈ Γ), that is∫

U0

f(H(x, u))ν0(du) =

∫E

f(y)µ(x, dy).


Stochastic equation for a pure jump process 131

Let ξ be a Poisson random measure on U0 × [0,∞) × [0,∞) with mean measureν0 × `× `. Then there exists an E-valued process satisfying

f(X(t))

= f(X(0))

+

∫U0×[0,∞)×[0,t]

1[0,λ(X(s−))](u1)(f(H(X(s−), u0))− f(X(s−)))ξ(du0 × du1 × ds)

= f(X(0)) +

∫ t

0

Af(X(s))ds+

∫U0×[0,∞)×[0,t]

Bf(X(s−), u0, u1)ξ(du0 × du1 × ds)

up to

τ∞ = limn→∞

inft :

∫U0×[0,∞)×[0,t]

1[0,λ(X(s−))](u1)ξ(du0 × du1 × ds) ≥ n

If E ⊂ Rd, then write

X(t) = X(0)+

∫U0×[0,∞)×[0,t]

1[0,λ(X(s−))](u1)(H(X(s−), u0)−X(s−))ξ(du0×du1×ds).


Lipschitz condition 132

α2(x, u) = 1[0,λ(x)](u1)(H(x, u0)− x)

so ∫U

|α2(x, u)− α2(y, u)|ν(du)

≤∫

U

|1[0,λ(x)](u1)− 1[0,λ(y)](u1)||(H(x, u0)− x)|ν0(u0)du1

+

∫U

1[0,λ(y)](u1)||(H(x, u0)−H(y, u0)− (x− y))|ν0(u0)du1

≤∫

U0

|λ(x)− λ(y)||(H(x, u0)− x)|ν0(u0)

+

∫U0

λ(y)|(H(x, u0)−H(y, u0)− (x− y))|ν0(u0)

Exercise 5.2 Try estimating∫|α2(x, u)− α2(y, u)|2ν(du).


Dynkin’s identity 133

Lemma 5.3 Suppose X is a solution of the martingale problem for A. Then for a stoppingtime τ ,

E[f(X(t ∧ τ))] = E[f(X(0))] + E[

∫ t∧τ

0

Af(X(s))ds]

Proof. Apply the optional sampling theorem.


Moment estimates 134

Suppose

|a(x)|+ |b(x)|2 +

∫U1

|α1(x, u)|2ν(du) (5.2)

+

∫U2

|α2(x, u)|2ν(du) +

(∫U2

|α2(x, u)|ν(du))2

≤ K1 +K2|x|2

Then for f(x) = |x|2, Af(x) ≤ C1 + C2|x|2.

Af(x) =1

2


+

∫U1


+

∫U2

(f(x+ α2(x, u))− f(x))ν(du)


By (5.4) and (5.5),

E[(|X(t)| ∧ (c

2))2] = E[(|X(t ∧ τc)| ∧ (

c

2))2] ≤ (E[|X(0)|2] + C1t)e

C2t.

Consequently, the monotone convergence theorem gives

E[|X(t)|2] ≤ (E[|X(0)|2] + C1t)eC2t.


Lyapunov functions 138

Lemma 5.4 Suppose there exist ϕn ∈ D(A) such that infn,x ϕn(x) > −∞, ϕn(x) ϕ(x), x ∈ E, supn,xAϕn(x) <∞, and Aϕn(x) → ψ(x). Then

ϕ(X(t))− ϕ(X(0))−∫ t

0

ψ(X(s))ds

is a supermartingale.

Proof.

E[(ϕ(X(t+ r))−∫ t+r

t

ψ(X(s))ds)∏

hi(X(ti))]

≤ limn→∞

E[(ϕn(X(t+ r))−∫ t+r

t

Aϕn(X(s))ds)∏

hi(X(ti))]

= limn→∞

E[ϕn(X(t)))∏

hi(X(ti))]

= E[ϕ(X(t)))∏

hi(X(ti))]

The inequality follows by Fatou’s lemma, the first equality by the martingale con-dition, and the last equality by the monotone convergence theorem.


Estimates on return times 139

Suppose there exists ε > 0 such that x : ψ(x) ≥ −ε is compact. Let τ = inft :ψ(X(t)) ≥ −ε. The supermartingale property implies

E[ϕ(X(t ∧ τ)] ≤ E[ϕ(X(0))] + E[

∫ t∧τ

0

ψ(X(s))ds] ≤ E[ϕ(X(0))]− εE[t ∧ τ ]

Rearranging the equality and letting t→∞, we have

E[τ ] ≤ E[ϕ(X(0))]− infx ϕ(x)

ε.


Uniform stochastic boundedness 140

Suppose that ϕ ≥ 0, x : ϕ(x) ≤ c is compact for each c > 0, and for some ε > 0and K ∈ R, ψ(x) ≤ K − εϕ(x), x ∈ E. Then

h(t) ≡ E[ϕ(X(t))] ≤ E[ϕ(X(r))] + E[

∫ t

r

ψ(X(s))ds]

≤ E[ϕ(X(r))] + E[

∫ t

r

(K − εϕ(X(s)))ds],

so

eεth(t) = h(0) +∑

(eεti+1h(ti+1)− eεtih(ti))

= h(0) +∑

(eεti+1 − eεti)h(ti+1) +∑

eεti(h(ti+1)− h(ti))

≤ h(0) +

∫ t

0

εeεsh(η(s))ds+

∫ t

0

eεη(s)(K − εh(s))ds

≤ h(0) +

∫ t

0

εeεs(h(η(s))− h(s))ds+

∫ t

0

ε(eεs − eεη(s))h(s)ds+Kε−1(eεt − 1)

where η(s) = ti+1 and η(s) = ti, for ti ≤ s < ti+1. By judicious choice of the ti,

h(t) ≤ h(0)e−εt +Kε−1(1− e−εt).


Exponential martingales 141

Suppose f ∈ D(A) and infx f(x) > 0. Then

f(X(t)) exp−∫ t

0

Af(X(s))

f(X(s))ds = f(X(0)) +

∫ t

0

exp−∫ r

0

Af(X(s))

f(X(s))dsdf(X(r))

−∫ t

0

exp−∫ r

0

Af(X(s))

f(X(s))dsAf(X(r))dr

= f(X(0)) +

∫ t

0

exp−∫ r

0

Af(X(s))

f(X(s))dsdMf (r)

is a martingale. If ϕn, ϕ, and ψ are as above and infx,n ϕn(x) > 0, then

ϕ(X(t)) exp−∫ t

0

ψ(X(s))ds

ϕ(X(s))ds

is a supermartingale. In particular,

E[ϕ(X(t+ r)) exp−∫ t+r

t

ψ(X(s))ds

ϕ(X(s))ds|Ft] ≤ ϕ(X(t)).


Check that

lim infn→∞

ϕn(X(t+r)) exp−∫ t+r

t

Aϕn(X(s))ds

ϕn(X(s))ds ≥ ϕ(X(t+r)) exp−

∫ t+r

t

ψ(X(s))ds

ϕ(X(s))ds


6. Forward equations and operator semigroups

• The forward equation for a general Markov process

• The operator semigroup associated with a Markov process

• The generator for a semigroup and the Hille-Yosida theorem

• Equivalence of the forward equation and the martingale problem

• Markov mapping theorem


The forward equation for a general Markov process 144

If X is a solution of the martingale problem for A and νt is the distribution of X(t),then

0 = E[f(X(t))− f(X(0))−∫ t

0

Af(X(s))ds] = νtf − ν0f −∫ t

0

νsAfds

so

νtf = ν0f +

∫ t

0

νsAfds, f ∈ D(A). (6.1)

(6.1) gives the weak form of the forward equation.

Definition 6.1 A measurable mapping t ∈ [0,∞) → νt ∈ P(E) is a solution of theforward equation for A if (6.1) holds.


Fokker-Planck equation 145

Let Af = 12a(x)f ′′(x) + b(x)f ′(x), f ∈ D(A) = C2

c (R). If νt has a C2 density, then

νtAf =

∫ ∞

−∞Af(x)ν(t, x)ds =

∫ ∞

−∞f(x)

(1

2

∂2

∂x2(a(x)ν(t, x))− ∂

∂x(b(x)ν(t, x))

)dx

and the forward equation is equivalent to

∂

∂tν(t, x) =

1

2

∂2

∂x2(a(x)ν(t, x))− ∂

∂x(b(x)ν(t, x)),

known as the Fokker-Planck equation in the physics literature.


The operator semigroup associated with a Markov process 146

Let X be a time homogeneous Markov process and for f ∈ B(E), define

T (t)f(x) = E[f(X(s+ t))|X(s) = x],

that is, T (t)f ∈ B(E) is a measurable function such that

T (t)f(X(s)) = E[f(X(s+ t))|X(s)] a.s.

and temporal homogeneity simply means that T (t)f can be selected independentlyof s.

By the Markov property

T (s)T (t)f(x) = E[T (t)f(X(s))|X(0) = x]

= E[E[f(X(s+ t))|X(s)]|X(0) = x]

= E[f(X(s+ t))|X(0) = x]

= T (t+ s)f(x)

T (t) is a contraction in the sense that

‖T (t)f‖ = supx|T (t)f(x)| ≤ ‖f‖.


Determining the finite dimensional distributions 147

For t2 > t1 ≥ 0,

E[f1(X(t1))f2(X(t2))|X(0) = x]

= T (t1)(f1T (t2 − t1)f2)(x)

and in general

E[f1(X(t1)) · · · fk(X(tk))|X(0) = x]

= E[f1(X(t1)) · · · fk−1(X(tk−1))T (tk − tk−1)fk(X(tk−1))|X(0) = x]

Note that, for these identities to determine the finite dimensional distributions ofa process, T (t) must be given by a transition function

T (t)f(x) =

∫E

f(y)P (t, x, dy) (6.2)

P (t, x, ·) ∈ P(E) for all t, x and (t, x) → P (t, x,D) is measurable for all D ∈ B(E).


A continuity condition 148

If T (t) is given by a transition function, then

bp− limn→∞

fn = 0 (6.3)

impliesbp− lim

n→∞T (t)fn = 0. (6.4)


The generator for a semigroup 149

The strong generator for a contraction semigroup: f is in the domain D(A) of thestrong generator for T (t) if

Af(x) ≡ limt→0

T (t)f(x)− f(x)

t

exists uniformly in x. f is in the domain of the weak generator A if the limit existsboundedly and pointwise.

Lemma 6.2 If f ∈ D(A), then T (t)f ∈ D(A)

AT (t)f = T (t)Af

T (t)f = f +

∫ t

0

AT (s)fds = f +

∫ t

0

T (s)Afds

‖T (t+ h)f − T (t)f‖ ≤ h‖Af‖

and T (t) is strongly continuous on L = D(A). If (6.4) holds for each sequence fnsatisfying (6.3), then the conclusion holds with A replaced by A.


Dissipativity and the positive maximum principle 150

For λ > 0,

‖λf − t−1(T (t)f − f)‖ ≥ (λ+ t−1)‖f‖ − t−1‖T (t)f‖ ≥ λ‖f‖

so A is dissipative‖λf − Af‖ ≥ λ‖f‖, λ > 0.

Definition 6.3 A satisfies the positive maximum principle if f(x) = ‖f‖ impliesAf(x) ≤ 0.

Lemma 6.4 The weak generator for a Markov process satisfies the positive maximum prin-ciple.

Lemma 6.5 Let E be compact and D(A) ⊂ C(E). If A satisfies the postive maximumprinciple, then A is dissipative.


Hille-Yosida theorem 151

Theorem 6.6 A linear operatorA on a Banach spaceL is the strong generator of a stronglycontinuous, contraction semigroup on L if and only if

1. D(A) is dense in L.

2. A is dissipative.

3. R(λ− A) = L for some (and hence all) λ > 0.

Remark 6.7 If Condition 3 is replaced by the condition thatR(λ−A) be dense in L, thenthe closure of A generates a strongly continuous contraction semigroup.


Uniqueness for the forward equation 152

Lemma 6.8 If νt and µt are solutions of the forward equation for A with ν0 = µ0 andR(λ − A) is separating for each λ > 0, then

∫∞0e−λtνtdt =

∫∞0e−λtµtdt and if ν and µ

are weakly right continuous, νt = µt for all t ≥ 0.

Proof.

λ

∫ ∞

0

e−λtνtfdt = ν0f + λ

∫ ∞

0

e−λt

∫ t

0

νsAfds dt

= ν0f + λ

∫ ∞

0

∫ ∞

s

e−λtνsAfdt ds

= ν0f +

∫ ∞

0

e−λsνsAf ds

and hence ∫ ∞

0

e−λtνt(λf − Af)dt = ν0f.

Since R(λ− A) is separating, the result holds.


The semigroup and the martingale problem 153

Theorem 6.9 If T (t) is a semigroup corresponding to a Markov process X and A is itsweak generator, then X is a solution of the martingale problem for A.

Proof.

E[f(X(t+ r))− f(X(t))−∫ t+r

t

Af(X(s))ds|FXt ]

= T (r)f(X(t))− f(X(t))−∫ t+r

t

T (s− t)Af(X(t))ds

= T (r)f(X(t))− f(X(t))−∫ r

0

T (s)Af(X(t))ds

= 0


The semigroup and the forward equation 154

If ν0 ∈ P(E), then

ν0T (t)f = ν0f +

∫ t

0

ν0T (s)Afds

and if T (t) is given by a transition function, νt =∫

EP (t, x, ·)ν0(dx) satisfies

νtf = ν0f +

∫ t

0

νsAfds, f ∈ D(A).

If A is the strong generator and L is separating, then uniqueness holds for theforward equation.


Uniqueness for martingale problems 155

Theorem 6.10 If any two solutions of the martingale problem forA satisfying PX1(0)−1 =

PX2(0)−1 also satisfy PX1(t)

−1 = PX2(t)−1 for all t ≥ 0, then the f.d.d. of a solution X

are uniquely determined by PX(0)−1.

If X is a solution of the MGP for A and Ya(t) = X(a+ t), then Ya is a solution of theMGP for A.

Theorem 6.11 If the conclusion of the above theorem holds, then any solution of themartingale problem for A is a Markov process.

Theorem 6.12 If for each ν0 ∈ P(E) uniqueness holds for the forward equation for(A, ν0), then uniqueness holds for the martingale problem for (A, ν0).


Digression on the proof of the Hille-Yosida theorem 156

The conditions of the Hille-Yosida theorem 6.6 imply (I − n−1A)−1 exists and

‖(I − n−1A)−1f‖ ≤ ‖f‖.

In addition‖(I − n−1A)−1f − f‖ ≤ 1

n‖Af‖.

One proof of the Hille-Yosida theorem is to show that

Tn(t)f = (I − n−1A)−[nt]f

is a Cauchy sequence and to observe that

Tn(t)f = f +1

n

[nt]∑k=1

(I − n−1A)−kAf = f +

∫ [nt]n

0

Tn(s+ n−1)Afds


Probabilistic interpretation 157

(n − A)−1 =∫∞

0e−ntT (t)dt and (I − n−1A)−1 = n

∫∞0e−ntT (t)dt. If T (t) is given

by a transition function, then

ηn(x, dy) = n

∫ ∞

0

e−ntP (t, x, dy)dt

is a transition function. If Y nk is a Markov chain with transition function ηn, then

E[f(Y nk )] = E[(I − n−1A)−kf(Y0)] = E[f(X(

∆1 + · · ·+ ∆k

n))]

and Xn(t) = Y n[nt] can be written as

Xn(t) = X(1

n

[nt]∑k=1

∆k)


Construction of solution of martingale problem 158

Assume that E is compact, A ⊂ C(E) × C(E), (1, 0) ∈ A, and D(A) is dense inC(E).

Assume that A satsifies the positive maximum principle (and is consequently dis-sipative). Note that

D((I − n−1A)−1) = R(I − n−1A)

For each x ∈ E, ηxf = (I − n−1A)−1f(x) is a linear functional on R(I − n−1A) andsatisfies |ηxf | ≤ ‖f‖ and ηx1 = 1. The Hahn-Banach theorem implies ηx extends toa positive linear functional on C(E) (hence a probability measure).

Γx = η ∈ P(E) : ηf = (I − n−1A)−1f(x), f ∈ R(I − n−1A)

is closed and lim supy→x Γy ⊂ Γx. The measurable selection theorem implies that thereexists a transition function satisfying∫

E

f(y)ηn(x, dy) = (I − n−1A)−1f(x).


Approximating Markov chain 159

If Y nk is a Markov chain with transition function ηn, then

E[f(Y nk+1)|Fn

k ] = (I − n−1A)−1f(Y nk ),

for f ∈ D(A) and fn = f − n−1Af and

fn(Y nm)− fn(Y n

0 )− 1

n

m−1∑k=0

Af(Y nk )

is a martingale. Recall the tightness condition, and defining Xn(t) = Y n[nt], note that

E[(g(Xn(t+ u))− g(Xn(t)))2|Fnt ]

= E[g2(Xn(t+ u))− g2(Xn(t))|Fnt ]− 2g(Xn(t))E[g(Xn(t+ u))− g(Xn(t))|Fn

t ]

≤ 2‖g2 − hn‖+ 4‖g‖‖g − fn‖+[nu] + 1

n(‖Ah‖+ 2‖g‖‖Af‖)

Tightness for g(Xn) for each bounded continuous g and the compactness of Eimplies tightness for Xn.


Stationary distributions 160

Definition 6.13 A stochastic process X is stationary if the distribution of Xt ≡ X(t+ ·)does not depend on t.

Definition 6.14 µ is a stationary distribution for the martingale problem for A if thereexists a stationary solution of the martingale problem for A with marginal distribution µ.

Theorem 6.15 Suppose that D(A) and R(λ − A) are separating and that for each ν ∈P(E), there exists a solution of the martingale problem for (A, ν). If∫

E

Afdµ = 0, f ∈ D(A),

then µ is a stationary distribution for A.


Echeverria’s theorem 161

Theorem 6.16 LetE be compact, and letA ⊂ C(E)×C(E) satisfy the positive maximumprinciple. Suppose that D(A) is an algebra and dense in C(E). If µ ∈ P(E) satisfies∫

E

Afdµ = 0, f ∈ D(A),

then µ is a stationary distribution of A.

Example 6.17 E = [0, 1], Af(x) = 12f ′′(x)

D(A) = f ∈ C2[0, 1] : f ′(0) = f ′(1) = 0, f ′(13) = f ′(2

3)

Let µ(dx) = 3I[ 13, 23](x)dx.


Outline of proof 162

We constructed ηn so that∫E

fn(y)ηn(x, dy) =

∫E

(f(y)− 1

nAf(y))ηn(x, dy) = f(x)

Consequently∫E

∫E

fn(y)ηn(x, dy)µ(dx) =

∫E

f(x)µ(dx) =

∫E

(f(x)− 1

nAf(x))µ(dx) =

∫E

fn(x)µ(dx)

For F (x, y) =∑m

i=1 hi(x)(fi(y)− 1nAfi(y)) + h0(y), fi ∈ D(A), define

ΛnF =

∫ [ m∑i=1

hi(x)fi(x) + h0(x)

]µ(dx)

If Λn is given by a measure νn, then both marginals are µ, and letting ηn satisfyνn(dx, dy) = ηn(x, dy)µ(dx), for f ∈ D(A),∫

(f(y)− 1

nAf(y))ηn(x, dy) = f(x), µ− a.s.

The work is to show the Λn is a positive linear functional.


Extensions 163

Theorem 6.18 LetE be locally compact (e.g., E = Rd), and letA ⊂ C(E)×C(E) satisfythe positive maximum principle. Suppose that D(A) is an algebra and dense in C(E). Ifµ ∈ P(E) satisfies ∫

E

Afdµ = 0, f ∈ D(A),

then µ is a stationary distribution of A.

Proof. Let E = E ∪ ∞ and extend A to include (1, 0). There exists an E-valuedstationary solution X of the martingale problem for the extended A, but PX(t) ∈E = µ(E).


Complete, separable E 164

E complete, separable. A ⊂ C(E)× C(E).

Assume that gk is closed under multiplication. Let I be the collection of finitesubsets of positive integers, and for I ∈ I, let k(I) satisfy gk(I) =

∏i∈I gi. For each

k, there exists ak ≥ |gk|. Let

E = z ∈∞∏i=1

[−ai, ai] : zk(I) =∏i∈I

zi, I ∈ I.

Note that E is compact, and define G : E → E by

G(x) = (g1(x), g2(x), . . .).

Then G has a measurable inverse defined on the (measurable) set G(E).

Lemma 6.19 Let µ ∈ P(E). Then there exists a unique measure ν ∈ P(E) satisfying∫Egkdµ =

∫Ezkν(dz). In particular, if Z has distribution ν, thenG−1(Z) has distribution

µ.


Equivalence of the forward equation and the MGP 165

Suppose

νtf = ν0f +

∫ t

0

νsAfds

DefineBλf(x, θ) = Af(x, θ) + λ(

∫E

f(y,−θ)ν0(dy)− f(x, θ))

andµλ = λ

∫ ∞

0

e−λtνtdt× (1

2δ1 +

1

2δ−1).

Then ∫E

Bλfdµλ = 0, f(x, θ) = f1(x)f2(θ), f1 ∈ D(A).

Let τ1 = inft > 0 : Θ(t) 6= Θ(0), τk+1 = inft > τk : Θ(t) 6= Θ(τk).

Theorem 6.20 Let (Y,Θ) be a stationary solution of the martingale problem for Bλ withmarginal distribution µλ. Let τ1 = inft > 0 : Θ(t) 6= Θ(0), τk+1 = inft > τk : Θ(t) 6=Θ(τk). Define X(t) = Y (τ1 + t). Then conditioned on τ2 − τ1 > t0, X is a solution of themartingale problem for A and the distribution of X(t) is νt for 0 ≤ t ≤ t0.


Images of a Markov process 166

X a Markov process with generator A and initial distribution µ0

γ : E → E0, Borel measurable

Y = γ X

• When is Y Markovian?

• If Y is Markovian, what is its generator?

• What is the conditional distribution of X given Y ?


Technical conditions 167

Definition 6.21 An operator A ⊂ B(E) × B(E) is a pre-generator if A is dissipativeand there are sequences of functions µn : E → P(E) and λn : E → [0,∞) such that, foreach (f, g) ∈ A,

g(x) = limn→∞

λn(x)

∫E

(f(y)− f(x))µn(x, dy), (6.5)

for each x ∈ E. Note that we have not assumed that µn and λn are measurable functionsof x.

A is bp-separable if there exists a countable collection gk ⊂ D(A) such that A iscontained in the bounded, pointwise closure of the linear span of (gk, Agk).


Markov mapping theorem 168

Theorem 6.22 A ⊂ C(E)× C(E) a pre-generator with bp-separable graph.

D(A) closed under multiplication and separating.

γ : E → E0, Borel measurable.

α a transition function from E0 into E satisfying

α(y, γ−1(y)) = 1

Let µ0 ∈ P(E0), ν0 =∫α(y, ·)µ0(dy), and define

C = (∫

E

f(z)α(·, dz),∫

E

Af(z)α(·, dz)) : f ∈ D(A) .

If Y is a solution of the MGP for (C, µ0), then there exists a solution Z of the MGP for(A, ν0) such that Y = γ Z and Y have the same distribution on ME0 [0,∞).


Interpretation of α 169

E[f(Z(t))|FYt ] =

∫f(z)α(Y (t), dz)

(at least for almost every t).


Uniqueness 170

Corollary 6.23 If uniqueness holds for the MGP for (A, ν0), then uniqueness holds for theME0 [0,∞)-MGP for (C, µ0). If Y has sample paths in DE0 [0,∞), then uniqueness holdsfor the DE0 [0,∞)-martingale problem for (C, µ0).

Existence for (C, µ0) and uniqueness for (A, ν0) implies existence for (A, ν0) and unique-ness for (C, µ0), and hence that Y is Markov.


7. Equivalence of SDE and MGP

1. Weak solutions of stochastic equations

2. Ito equations

3. General equations for Markov processes

4. Yamada-Watanabe-Engelbert theorem


Weak solutions of stochastic equations 172

Stochastic equation: Given Γ : S1 × S2 → R and S2-valued Y , find X such that

Γ(X, Y ) = 0 a.s. (7.1)

Definition 7.1 Suppose Y has distribution µY . (X, Y ) is a weak solution (or distribu-tional solution) of (7.1) if Y has distribution µY and Γ(X, Y ) = 0 a.s.

For stochastic processes, X is compatible with Y if

E[f(Y )|FX,Yt ] = E[f(Y )|FY

t ], t ≥ 0.

Remark 7.2 If Y has independent increments, then compatibility is just the requirementthat Y (t+ ·)− Y (t) is independent of FX,Y

t for every t ≥ 0.


Equivalence of SDE and MGP 173

If

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

b(X(s))ds,

then X is a solution of the martingale problem for

Af(x) =1

2

∑ij

aij(x)∂2

∂xi∂xj

f(x) +∑

i

bi(x)∂

∂xi

f(x)

where ((aij)) = σσT .

Conversly, if X is a solution of the MGP for A, then X is a weak solution of theSDE. If σ is invertible, then we should have

W (t) =

∫ t

0

σ−1(X(s))dX(s)−∫ t

0

σ−1(X(s))b(X(s))ds.


Existence of cadlag version 174

Lemma 7.3 Let M be a martingale defined on (Ω,F , P ). Then there exsits Ω′ ⊂ Ω withP (Ω′) = 1 such that for each ω ∈ Ω′,

M+(t, ω) = lims∈Q→t+

M(s, ω) and M−(t, ω) = lims∈Q→t−

M(s, ω)

exists for all t > 0 and the first limit exists for t = 0.

Definition 7.4 Let X and Y be E-valued stochastic processes defined on (Ω,F , P ). Y isa modification of X if and only if PX(t) = Y (t) = 1 for all t ≥ 0.

Lemma 7.5 Let E be compact and let D(A) ⊂ C(E) be separating. Suppose that thereexists a countable subset fi ⊂ D(A) such that fi separates points. If X is a solutionof the martingale problem for A, then there exists a cadlag modification of X .


Proof. Let

Mi(t) = fi(X(t))−∫ t

0

Afi(X(s))ds

There exists Ω′ with P (Ω′) = 1 such that M+i (·, ω) exists for each ω ∈ Ω′. By the

compactness of E and the fact that fi separates points, there exists Y such that

M+i (t, ω) = fi(Y (t, ω))−

∫ t

0

Afi(X(s, ω))ds, ω ∈ Ω′

and Y (t, ω) = lims∈Q→t+X(s, ω) Consequently,

E[f(Y (t))− f(X(t))|FXt ] = lim

s∈Q→t+E[

∫ s

t

Af(X(r))dr|FXt ] = 0,

and hence E[f(Y (t))|FXt ] = f(X(t)) and

E[f(Y (t))g(X(t))] = E[f(X(t))g(X(t))] (7.2)

for evey g ∈ B(E) and f ∈ D(A). Since D(A) is separating, (7.2) holds for allf ∈ B(E) and

E[(g(Y (t))− g(X(t)))2] = E[g2(Y (t)) + g2(X(t))− 2g(Y (t))g(X(t))] = 0.


Proof of continuity 176

Let f ∈ C2c (Rd). A little calculus shows that

∑4k=0(−1)4−k

(4k

)Afk(x)f(x)4−k = 0 and

hence

E[(f(X(t+ h))− f(X(t)))4|Ft]

= E[4∑

k=0

(−1)4−k

(4

k

)f(X(t+ h))kf(X(t))4−k|Ft]

=4∑

k=0

(−1)4−k

(4

k

)E[

∫ h

0

Afk(X(t+ s))ds|Ft]f(X(t))4−k

=4∑

k=0

(−1)4−k

(4

k

)E[

∫ h

0

Afk(X(t+ s))f(X(t+ s))4−kds|Ft]

−4∑

k=0

(−1)4−k

(4

k

)E[

∫ h

0

Afk(X(t+ s))(f(X(t+ s))4−k − f(X(t))4−k)ds|Ft]

= −4∑

k=0

(−1)4−k

(4

k

)E[

∫ h

0

Afk(X(t+ s))(f(X(t+ s))4−k − f(X(t))4−k)ds|Ft]


Setting h = T/n and ηn(t) = [tn]h,

E[n−1∑l=0

E[(f(X((l + 1)h))− f(X(lh)))4]

= −4∑

k=0

(−1)4−k

(4

k

)E[

∫ T

0

Afk(X(s))(f(X(s))4−k − f(X(ηn(s)))4−k)ds]

Letting n→∞, the right side goes to zero, so by Fatou’s lemma, we have

E[∑s≤T

(f(X(s))− f(X(s−)))4] ≤ limn→∞

E[n−1∑l=0

E[(f(X((l + 1)h))− f(X(lh)))4] = 0,

which implies the continuity of X .


Solution of MGP is weak solution of SDE 178

Theorem 7.6 Suppose X is a solution of the martingale problem for

Af(x) =1

2

∑ij

aij(x)∂2

∂xi∂xj

f(x) +∑

i

bi(x)∂

∂xi

f(x), f ∈ C2c (Rd),

where ((aij)) = σσT and σ(x) is invertible and that sups≤t |X(s)| < ∞ for each t > 0.Then X is a weak solution of the stochastic differential equation

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

b(X(s))ds.


Proof. Note that

M(t) = X(t)−∫ t

0

b(X(s))ds

and

Mi(t)Mj(t)−∫ t

0

aij(X(s))ds

are FXt -local martingales, and hence

W (t) =

∫ t

0

σ−1(X(s))dX(s)−∫ t

0

σ−1(X(s))b(X(s))ds =

∫ t

0

σ−1(X(s))dM(s)

is also, and[Wi,Wj]t = 〈Wi,Wj〉t = δijt

Compatibility follows from the fact that W (t+ ·)−W (t) is independent of FXt .


Alternative approach 180

Let d = 1. DefineZ(t) = Z(0) +W (t) mod 1

Then (X,Z) is a solution of the MGP for

Af(x, z) =1

2a(x)

∂2

∂x2f(x, z) + σ(x)

∂2

∂x∂zf(x, z)

+1

2

∂2

∂z2f(x, z) + b(x)

∂

∂xf(x, z)

Conversely, if (X,Z) is a solution of the martingale problem for A, then the corre-sponding W can be recovered from Z.


Verification of solution 181

For example, setting U1(t) = cos 2πZ(t) and U2(t) = sin 2πZ(t),

U1(t) +

∫ t

0

2π2U1(s)ds and U2(t) +

∫ t

0

2π2U2(s)ds

are martingales and

W (t) =1

2π

(∫ t

0

U1(s)dU2(s)−∫ t

0

U2(s)dU1(s)

)is a Brownian motion. To see that X is a weak solution of the SDE, use the martin-gale properties to compute

E

[(X(t ∧ τ)−X(0)−

∫ t∧τ

0

σ(X(s))dW (s)−∫ t∧τ

0

b(X(s))ds)

)2].


Equivalence to original MGP 182

Let

αf(x) =

∫ 1

0

f(x, z)dz.

ThenαAf(x) = Aαf(x),

and any solution of the MGP for A corresponds to a solution of A and hence is aweak solution of the SDE.


Processes with jumps 183

Af(x) =1

2


+

∫U1


+

∫U2

(f(x+ α2(x, u))− f(x))ν(du)

Let D(A) = C2c (Rd) and suppose that Af is bounded for f ∈ D(A).

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

b(X(s))ds (7.3)

+

∫U1×[0,t]

α1(X(s−), u)ξ(du× ds) +

∫U2×[0,t]


Then a solution of the SDE is a solution of the martingale problem for A.


Stationary representations for PRM 184

Let Di ⊂ B(U) be a countable collection of sets with ν(Di) <∞ that is closed un-der intersections, and generates B(U). Then ξ is completely determined by ξ(Di, t).Define

Zi(t) = Zi(0)(−1)ξ(Di,t)

where Zi(0) is ±1. Note that

ξ(Di, t) = −1

2

∫ t

0

Zi(s−)dZi(s), (7.4)

and if the Zi(0) are iid with PZi(0) = 1 = PZi(0) = −1 = 12

and independentof ξ, then for each t ≥ 0, the Zi(t) are iid and independent of ξ.

Let z ∈ E1 = −1, 1∞ and define

(−1)δuz = ((−1)1D1(u)z1, (−1)1D2

(u)z2, . . .).


Joint generator 185

For f ∈ D(A) and g ∈ B(E1) satisfying supz

∫|g((−1)δuz) − g(z)|ν(du) < ∞, let

h(x, z) = f(x)g(z)

Ah(x, z)

=1

2g(z)

(∑aij(x)∂i∂jf(x) + b(x) · ∇f(x)

)+

∫U1

(f(x+ α1(x, u))g((−1)δuz)− f(x)g(z)− g(z)α1(x, u) · ∇f(x, z))ν(du)

+

∫U2

(f(x+ α2(x, u))g((−1)δuz)− f(x)g(z))ν(du)


Markov mapping 186

E = Rd × E1

E0 = Rd

γ(x, z) = x

α(x, dy × dz) = δx(dy)∏

i(12δ1(dzi) + 1

2δ−1(dzi))

Setting g =∫g(z)

∏i(

12δ1(dzi) + 1

2δ−1(dzi)),

αh = gf(x)

andαAh(x) = gAf(x) = Aαh(x).


Joint solution measure 187

Definition 7.7 Given Γ : S1 × S2 → R and ν ∈ P(S2), µ ∈ P(S1 × S2) is a jointsolution measure for (Γ, ν) if µ(S1 × ·) = ν and

Γ(x, y) = 0, a.s. µ.

Joint uniqueness in law holds if there is at most one joint solution measure.

Lemma 7.8 The collection SΓ,ν of joint solution measures is convex.

Let S1 = DE1 [0,∞) and S2 = DE2 [0,∞).

Lemma 7.9 Let Cν be the collection of µ ∈ P(S1 × S2) with the following properties:

a) µ(S1 × ·) = ν

b) If (X, Y ) has distribution µ, then X is compatible with Y .

Then Cν is convex.


Proof. X is compatible with Y if for each f ∈ B(DE2 [0,∞)),

E[f(Y )|FX,Yt ] = E[f(Y )|FY

t ], t ≥ 0,

which is equivalent to

infh∈B(DE1×E2

[0,t])E[(f(Y )− h(X,Y ))2] = inf

h∈B(DE2[0,t])

E[(f(Y )− h(Y ))2].

Note that the right side is determined by ν, so µ ∈ Cν if µ(S1 × ·) = ν and∫S1×S2

(f(y)− h(x, y)2µ(dx× dy) ≥ infh∈B(DE2

[0,t])

∫S2

(f(y)− h(y))2ν(dy),

for each h ∈ B(DE1×E2 [0, t]), f ∈ B(DE2 [0,∞)), and t ≥ 0. Each of these inequalitiesis preserved under convex combinations.


Conditions for compatibility 189

Lemma 7.10 X is compatible with Y if and only if for each t ≥ 0 and each g ∈ B(DE1 [0, t]),

E[g(X)|Y ] = E[g(X)|FYt ] a.s. (7.5)

Proof. Suppose thatX is compatible with Y . Then for f ∈ B(S2) and g ∈ B(DE1 [0, t]),

E[f(Y )g(X)] = E[E[f(Y )|FXt ∨ FY

t ]g(X)]

= E[E[f(Y )|FYt ]g(X)]

= E[E[f(Y )|FYt ]E[g(X)|FY

t ]]

= E[f(Y )E[g(X)|FYt ]],

and (7.5) follows.

Conversely, suppose (7.5) holds. For f ∈ B(S2), g ∈ B(DE1 [0, t]), and h ∈ B(DE2 [0, t])

E[E[f(Y )|FYt ]g(X)h(Y )] = E[E[f(Y )|FY

t ]E[g(X)|FYt ]h(Y )]

= E[f(Y )E[g(X)|Y ]h(Y )]

= E[f(Y )g(X)h(Y )],

and compatibility follows.


Pathwise uniqueness 190

Definition 7.11 Let X1, X2, and Y be defined on the same probability space. Let X1 andX2 be S1-valued and Y be S2-valued. (X1, X2) are jointly compatible with Y if

E[f(Y )|FX1t ∨ FX2

t ∨ FYt ] = E[f(Y )|FY

t ], t ≥ 0, f ∈ B(S2).

Pathwise uniqueness holds for compatible solutions of (Γ, ν), if for every triple of pro-cesses (X1, X2, Y ) defined on the same sample space such that µX1,Y , µX2,Y ∈ SΓ,ν ∩ Cν

and (X1, X2) is jointly compatible with Y , X1 = X2 a.s.

Definition 7.12 A solution (X, Y ) is strong if there exists a measurable F : S2 → S1

such that X = F (Y ) a.s.


Yamada-Watanabe-Engelbert theorem 191

Theorem 7.13 Suppose SΓ,ν ∩ Cν 6= ∅. The following are equivalent:

a) Pathwise uniqueness holds.

b) µ ∈ SΓ,ν ∩ Cν is unique and there is a strong, compatible solution.

Proof. Let µ, γ ∈ SΓ,ν ∩ Cν and let Y , ξ1, and ξ2 be independent, µY = ν, and ξ1 andξ2 uniform on [0, 1]. Then there exist Fµ : S2 × [0, 1] → S1 and Fγ : S2 × [0, 1] → S1

such that (Fµ(Y, ξ1), Y ) has distribution µ and (Fγ(Y, ξ2), Y ) has distribution γ.

Claim: X1 = Fµ(Y, ξ1) and X2 = Fγ(Y, ξ2) are jointly compatible.

For f ∈ B(DE1 [0, t]),

E[f(Fµ(Y, ξ1))|Y, ξ2] = E[f(Fµ(Y, ξ1))|Y ] = E[f(Fµ(Y, ξ1))|FYt ].


Consequently, for f ∈ B(S2), g1, g2 ∈ B(DE1 [0, t]), and h ∈ B(DE2 [0, t]),

E[f(Y )g1(X1)g2(X2)h(Y )]

= E[f(Y )E[g1(X1)|Y, ξ2]g2(X2)h(Y )]

= E[f(Y )E[g1(X1)|FYt ]g2(X2)h(Y )]

= E[E[f(Y )|FX2t ∨ FY

t ]E[g1(X1)|FYt ]g2(X2)h(Y )]

= E[E[f(Y )|FYt ]E[g1(X1)|Y, ξ2]g2(X2)h(Y )]

= E[E[f(Y )|FYt ]g1(X1)g2(X2)h(Y )],

giving the joint compatibility. By pathwise uniqueness, Fµ(Y, ξ1) = Fγ(Y, ξ2) a.s.The independence of ξ1 and ξ2 implies that the solution is strong.

Conversely, the strong solution must give the unique µ ∈ SΓ,ν ∩ Cν . Consequently,there exists F : S2 → S1 such that µXY = µ implies X = F (Y ) almost surely, andpathwise uniqueness follows.


8. Change of measure

• Absolute continuity and the Radon Nikodym theorem

• Applications of absolute continuity

• Bayes formula

• Local absolute continuity

• Martingales under a change of measure

• Change of measure for Brownian motion

• Change of measure for Poisson processes


Absolute continuity and the Radon-Nikodym theorem 194

Definition 8.1 Let P and Q be probability measures on (Ω,F). Then P is absolutelycontinuous with respect to Q (P << Q) if and only if Q(A) = 0 implies P (A) = 0.

Theorem 8.2 If P << Q, then there exists a random variable L ≥ 0 such that

P (A) = EQ[1AL] =

∫A

LdQ, A ∈ F .

Consequently, Z is P -integrable if and only if ZL is Q-integrable, and

EP [Z] = EQ[ZL].

Standard notation: dPdQ

= L.


Maximum likelihood estimation 195

Suppose for each α ∈ A,

Pα(Γ) =

∫Γ

LαdQ

andLα = H(α,X1, X2, . . . Xn)

for random variables X1, . . . , Xn. The maximum likelihood estimate α for the“true” parameter α0 ∈ A based on observations of the random variablesX1, . . . , Xn

is the value of α that maximizes H(α,X1, X2, . . . Xn).

For example, under certain conditions the distribution of

Xα(t) = X(0) +

∫ t

0

σ(Xα(s))dW (s) +

∫ t

0

b(Xα(s), α)ds,

will be absolutely continuous with respect to the distribution of X satisfying

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) . (8.1)


Sufficiency 196

If dPα = LαdQ whereLα(X, Y ) = Hα(X)G(X, Y ),

then X is a sufficient statistic for α. Without loss of generality, we can assumeEQ[G(X,Y )] = 1 and hence dQ = G(X, Y )dQ defines a probability measure.

Example 8.3 If (X1, . . . , Xn) are iid N(µ, σ2) under P(µ,σ) and Q = P(0,1), then

L(µ,σ) =1

σnexp

−1− σ2

2σ2

n∑i=1

X2i +

µ

σ2

n∑i=1

Xi −µ2

σ2

so (∑n

i=1X2i ,∑n

i Xi) is a sufficient statistic for (µ, σ).


Parameter estimates and sufficiency 197

Theorem 8.4 If θ(X, Y ) is an estimator of θ(α) and ϕ is convex, then

EPα [ϕ(θ(α)− θ(X, Y ))] ≥ EPα [ϕ(θ(α)− EQ[θ(X, Y )|X])]

Proof.

EPα [ϕ(θ(α)− θ(X, Y ))] = EQ[ϕ(θ(α)− θ(X, Y ))Hα(X)]

= EQ[EQ[ϕ(θ(α)− θ(X, Y ))|X]Hα(X)]

≥ EQ[ϕ(θ(α)− EQ[θ(X, Y )|X])Hα(X)]


Other applications 198

Finance: Asset pricing models depend on finding a change of measure underwhich the price process becomes a martingale.

Stochastic Control: For a controlled diffusion process

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

b(X(s), u(s))ds

where the control only enters the drift coefficient, the controlled process can beobtained from an uncontrolled process satisfying (8.1) via a change of measure.


Examples 200

For real-valued random variables with a joint density X,Y ∼ fXY (x, y), condi-tional expectations can be computed by

E[g(Y )|X = x] =

∫∞−∞ g(y)fXY (x, y)dy

fX(x)

that is, setting h(x) equal to the right side,

E[g(Y )|X] = h(X).

For general random variables, suppose X and Y are independent on (Ω,F , Q). LetL = H(X, Y ) ≥ 0, and E[H(X, Y )] = 1. Define

νY (Γ) = QY ∈ ΓdP = H(X, Y )dQ.

Bayes formula becomes

EP [g(Y )|X] =EQ[g(Y )H(X, Y )|X]

EQ[H(X, Y )|X]=

∫g(y)H(X, y)νY (dy)∫H(X, y)νY (dy)


Local absolute continuity 201

Theorem 8.6 Let (Ω,F) be a measurable space, and let P and Q be probability measureson F . Suppose Dn ⊂ Dn+1 and that for each n, P |Dn << Q|Dn . Define Ln = dP

dQ

∣∣∣Dn

.

Then Ln is a nonnegative Dn-martingale on (Ω,F , Q) and L = limn→∞ Ln satisfiesEQ[L] ≤ 1. If EQ[L] = 1, then P << Q on D =

∨nDn.

Proof. If D ∈ Dn ⊂ Dn+1, then P (D) = EQ[Ln1D] = EQ[Ln+11D] which impliesE[Ln+1|Dn] = Ln. If E[L] = 1, then Ln → L in L1, so

P (D) = EQ[L1D], D ∈ ∪nDn,

hence for all D ∈∨

nDn.

Proposition 8.7 P << Q on D if and only if Plimn→∞Ln <∞ = 1.

Proof. The dominated convergence theorem implies

PsupnLn ≤ K = lim

m→∞EQ[1supn≤m Ln≤KLm] = EQ[1supn Ln≤KL].

Letting K →∞ we see that EQ[L] = 1.


Martingales and change of measure 202

(See [10], Section III.6.)

Let Ft be a filtration and assume that P |Ft << Q|Ft and that L(t) is the corre-sponding Radon-Nikodym derivative. Then as before, L is an Ft-martingale on(Ω,F , Q).

Lemma 8.8 Z is a P -local martingale if and only if LZ is a Q-local martingale.

Proof. For a bounded stopping time τ , Z(τ) is P -integrable if and only if L(τ)Z(τ)is Q-integrable. Furthermore, if L(τ ∧ t)Z(τ ∧ t) is Q-integrable, then L(t)Z(τ ∧ t)is Q-integrable and

EQ[L(τ ∧ t)Z(τ ∧ t)] = EQ[L(t)Z(τ ∧ t)].

By Bayes formula, EP [Z(t+ h)−Z(t)|Ft] = 0 if and only if EQ[L(t+ h)(Z(t+ h)−Z(t))|Ft] = 0 which is equivalent to

EQ[L(t+ h)Z(t+ h)|Ft] = EQ[L(t+ h)Z(t)|Ft] = L(t)Z(t),

so Z is a martingale under P if and only if LZ is a martingale under Q.


Semimartingale decompositions under a change of measure 203

Theorem 8.9 If M is a Q-local martingale, then

Z(t) = M(t)−∫ t

0

1

L(s)d[L,M ]s (8.3)

is a P -local martingale. (Note that the integrand is 1L(s)

, not 1L(s−)

.)

Proof. Note that LM − [L,M ] is a Q-local martingale. We need to show that LZ isa Q-local martingale. But letting V denote the second term on the right of (8.3), wehave L(t)V (t) =

∫ t

0V (s−)dL(s) +

∫ t

0L(s)dV (s) and hence

L(t)Z(t) = L(t)M(t)− [L,M ]t −∫ t

0

V (s−)dL(s).

Both terms on the right are Q-local martingales.


Change of measure for Brownian motion 204

Let W be standard Brownian motion, and let ξ be an adapted process. Define

L(t) = exp∫ t

0

ξ(s)dW (s)− 1

2

∫ t

0

ξ2(s)ds

and note that

L(t) = 1 +

∫ t

0

ξ(s)L(s)dW (s).

Then L(t) is a local martingale.

Assume EQ[L(t)] = 1 for all t ≥ 0. Then L is a martingale. Fix a time T , and restrictattention to the probability space (Ω,FT , Q). On FT , define dP = L(T )dQ.

For t < T , let A ∈ Ft. Then

P (A) = EQ[1AL(T )] = EQ[1AEQ[L(T )|Ft]]

= EQ[1AL(t)]︸︷︷︸has no dependence on T

(crucial that L is a martingale)


New Brownian motion 205

Theorem 8.10 W (t) = W (t)−∫ t

0ξ(s)ds is a standard Brownian motion on (Ω,FT , P ).

Proof. Since W is continous and [W ]t = t a.s., it is enough to show that W isa local martingale (and hence a martingale). But since W is a Q-martingale and[L,W ]t =

∫ t

0ξ(s)L(s)ds, Theorem 8.9 gives the desired result.


Changing the drift of a diffusion 206

Suppose that

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s)

and setξ(s) = b(X(s)).

Note that X is a diffusion with generator 12σ2(x)f ′′(x). Define

L(t) = exp∫ t

0

b(X(s))dW (s)− 1

2

∫ t

0

b2(X(s))ds,

and assume that EQ[L(T )] = 1 (e.g., if b is bounded). Set dP = L(T )dQ on (Ω,FT ).


Transformed SDE 207

Define W (t) = W (t)−∫ t

0b(X(s))ds. Then

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) (8.4)

= X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

σ(X(s))b(X(s))ds

so under P , X is a diffusion with generator

Af(x) =1

2σ2(x)f ′′(x) + σ(x)b(x)f ′(x). (8.5)


Conditions that imply local absolute continuity 208

Let Af(x) = 12σ2(x)f ′′(x) + σ(x)b(x)f ′(x).

Condition 8.11 If (Xn, τn), n = 1, 2, . . . satisfy

f(Xn(t ∧ τn))− f(Xn(0))−∫ t∧τn

0

Af(Xn(s))ds,

an Fnt -martingale for each f ∈ C2

c (R) and

τn = inft :

∫ t

0

b2(Xn(s))ds ≥ n,

then limn→∞ Pτn ≤ T = 0 for each T > 0.

Theorem 8.12 Suppose Condition 8.11 holds, and letW be a Browian motion on (Ω,F , Q).If X is a solution of

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s)

on (Ω,F , Q), then for each T , there is a change of measure dP = L(T )dQ such that X on(Ω,FT , P ) is a solution of the martingale problem for A on [0, T ].


Proof. Let

L(t) = exp∫ t

0

b(X(s))dW (s)− 1

2

∫ t

0

b2(X(s))ds,

and define τn = inft :∫ t

0b2(X(s))ds > n. Then EQ[L(T ∧ τn)] = 1 and we can

define dP = L(T ∧ τn)dQ on FT∧τn . On (Ω,FT∧τn , P ),

W (t ∧ τn) = W (t ∧ τn)−∫ t∧τn

0

b(X(s))ds

is a Brownian motion stopped at τn and

X(t) = X(0) +

∫ t

0

σ(X(s))dW (s) +

∫ t

0

σ(X(s))b(X(s))ds

for t ≤ T ∧ τn. Then (X, τn), n = 1, 2, . . . satisfies Condition 8.11, and since

PL(T ∧ τn) > K = P∫ T∧τn

0

b(X(s))dW (s) +1

2

∫ T∧τn

0

b2(X(s))ds > logK

we can apply Proposition 8.2 to conclude that P << Q on FT , that is, E[L(T )] = 1.


Change of measure for Poisson processes 210

Theorem 8.13 Let N be an unit Poisson process on (Ω,F , Q) that is compatible withFt. If Λ is nonnegative, Ft-predictable, and satisfies∫ t

0

Λ(s)ds <∞ a.s., t ≥ 0,

then

L(t) = exp

∫ t

0

ln Λ(s)dN(s)−∫ t

0

(Λ(s)− 1)ds

satisfies

L(t) = 1 +

∫ t

0

(Λ(s)− 1)L(s−)d(N(s)− s) (8.6)

and is a Q-local martingale. If E[L(T )] = 1 and we define dP = L(T )dQ on FT , thenN(t)−

∫ t

0Λ(s)ds is a P -local martingale.


Proof. By Theorem 8.9,

Z(t) = N(t)− t−∫ t

0

1

L(s)(Λ(s)− 1)L(s−)dN(s)

=

∫ t

0

1

Λ(s)dN(s)− t

is a local martingale under P . Consequently,∫ t

0

Λ(s)dZ(s) = N(t)−∫ t

0

Λ(s)ds

is a local martingale under P .


Construction of counting processes by change of measure 212

Let J [0,∞) denote the collection of nonnegative integer-valued cadlag functionsthat are constant except for jumps of +1. Suppose that λ : J [0,∞)×[0,∞) → [0,∞),∫ t

0

λ(x, s)ds <∞, t ≥ 0, x ∈ J [0,∞)

and that λ(x, s) = λ(x(· ∧ s), s) (that is, λ is nonanticipating). If we take Λ(t) =λ(N, t) and let τn = inft : N(t) = n, then defining dP = L(τn)dQ on Fτn , N(· ∧ τn)

on (Ω,Fτn , P ) has the same distribution as N(· ∧ τn), where N is the solution of

N(t) = Y (

∫ t

0

λ(N , s)ds)

for Y a unit Poisson process and τn = inft : N(t) = n.


Change of measure for Poisson random measures 213

ξ a Poisson random measure on U × [0,∞) with mean measure ν(du) × dt. λ apositive, predictable process satisfying∫

U×[0,t]

(λ(u, s)− 1)2 ∧ |λ(u, s)− 1|ν(du)ds <∞ a.s., t ≥ 0.

Let Mλ(t) =∫

U×[0,t](λ(u, s)− 1)ξ(du× ds) and let L be the solution of

L(t) = 1 +

∫ t

0

L(s−)dMλ(s) = 1 +

∫U×[0,t]

(λ(u, s)− 1)L(s−)ξ(du× ds). (8.7)

Then L is a local martingale. If∫

U×[0,t]|λ(u, s)− 1|ν(du)ds <∞ a.s., t ≥ 0, then

L(t) = exp∫

U×[0,t]

log λ(u, s)ξ(du× ds)−∫

U×[0,t]

(λ(u, s)− 1)ν(du)ds,

and in general,

L(t) = exp∫

U×[0,t]

log λ(u, s)ξ(du× ds) +

∫U×[0,t]

(log λ(u, s)− λ(u, s) + 1)ν(du)ds.


Intensity for the transformed counting measure 214

If E[L(T )] = 1, then for A ∈ B(U) with ν(A) <∞,

MA(t) =

∫A×[0,t]

λ(u, s)ξ(du× ds)

is a local martingale under Q and

[MA, L]t =

∫A×[0,t]

λ(u, s)(λ(u, s)− 1)L(s−)ξ(du× ds).

Consequently,

ZA(t) =

∫A×[0,t]

λ(u, s)ξ(du× ds)−∫

A×[0,t]

1

L(s)λ(u, s)(λ(u, s)− 1)L(s−)ξ(du× ds)

= ξ(A, t)−∫

A×[0,t]

λ(u, s)ν(du)ds

is a local martingales under dP = L(T )dQ.


Change for stochastic equations 215

U = U1 ∪ U2, U1 ∩ U2 = ∅,

Af(x) =

∫U1

(f(x+ α(x, u))− f(x)− α(x, u) · ∇f(x))ν(du)

+

∫U2

(f(x+ α(x, u))− f(x))ν(du)

Let D(A) = C2c (Rd) and suppose that Af is bounded for f ∈ D(A). Then X satisfy-

ing

X(t) = X(0) +

∫U1×[0,t]

α(X(s−), u)ξ(du× ds) +

∫U2×[0,t]

α(X(s−), u)ξ(du× ds)

is a solution of the martingale problem for A.


New martingale problem 216

Let λ(u, s) = λ(u,X(s−)). Under Q

f(X(t))− f(X(0))−∫ t

0

Af(X(s))ds

=

∫U×[0,t]

(f(X(s−) + α(X(s−), u))− f(X(s−))ξ(du× ds)

is a local martingale, so under P ,

f(X(t))− f(X(0))−∫ t

0

Af(X(s))ds

−∫ t

0

∫U

(f(X(s) + α(X(s), u))− f(X(s))(λ(u,X(s))− 1)ν(du)ds

=

∫U×[0,t]

(f(X(s−) + α(X(s−), u))− f(X(s−))(ξ(du× ds)− λ(u,X(s))ν(du)ds)



New generator 217

X is a solution of the martingale problem for

Af(x) =

∫U1

(f(x+ α(x, u))− f(x)− α(x, u) · ∇f(x))λ(u, x)ν(du)

+

∫U2

(f(x+ α(x, u))− f(x))λ(x, u)ν(du)

+

∫U1

α(x, u)(λ(x, u)− 1)ν(du) · ∇f(x)


9. Filtering

• Observation of a signal in noise

• Continuous time filtering in Gaussian white noise

• Zakai equation

• Kushner-Stratonovich equation

• Point process observations

• Convergence of stochastic integrals

• Convergence of exchangeable families

• Consistancy of Monte Carlo approximation of filtering equations

• Finite dimensional approximations


Observation of a signal in noise 219

Signal: X1, X2, . . .

Observation: Yk = h(Xk) + ζk, ζk iid “noise”

Filtering problem: Compute πn(Γ) = PXn ∈ Γ|FYn .

Suppose ζk has a strictly positive density γ with respect to Lebesgue measure.

Theorem 9.1 Suppose that under Q, Yk are iid with density γ(z) and are independentof Xk. Then

Ln =n∏

k=1

γ(Yk − h(Xk))

γ(Yk)

is a martingale and under dP = LndQ, (Y1, . . . , Yn) has the same distribution as

(h(X1) + ζ1, . . . , h(Xn) + ζn).


Proof.

EQ[g(Y1, . . . , Yn)Ln]

=

∫Rd

· · ·∫

Rd

EQ[g(z1, . . . , zn)n∏

k=1

γ(zk − h(Xk))]dz1 · · · dzn

=

∫Rd

· · ·∫

Rd

EQ[g(h(X1) + z1, . . . , h(Xn) + zn)n∏

k=1

γ(zk)]dz1 · · · dzn

= EQ[g(h(X1) + ζ1, . . . , h(Xn) + ζn)]


Unnormalized conditional distributions 222

Define φ0(dz) = π0(dz) and∫E

g(z)φn(dz) =

∫E

g(z)γ(Yn − h(z))P (dz|x)φn−1(dx)

Then ∫E

g(x)πn(dx) =

∫Eg(x)φn(dx)

φn(E)


Continuous time filtering in Gaussian white noise 223

Suppose Y (n)k = h(X

(n)k ) 1

n+ 1√

nζk, X(n)

k ≈ X(k/n), and ζk are iid with mean zero

and variance σ2. Then Yn(t) =∑[nt]

k=1 Y(n)k is approximately

Y (t) =

∫ t

0

h(X(s))ds+ σW (t)

Then

EP [g(X(t))|FYt ] =

EQ[g(X(t))L(t)|FYt ]

EQ[L(t)|FYt ]

where underQ,X and Y are independent, Y is a Brownian motion with mean zeroand variance σ2t, and

L(t) = exp∫ t

0

h(X(s))

σdY (s)− 1

2

∫ t

0

h2(X(s))

σ2ds

that is,

L(t) = 1 +

∫ t

0

h(X(s))

σL(s)dY (s)


Monte Carlo solution 224

Let X1, X2, . . . be iid copies of X that are independent of Y under Q, and let

Li(t) = 1 +

∫ t

0

h(Xi(s))

σLi(s)dY (s).

Note thatφ(g, t) ≡ EQ[g(X(s))L(s)|FY

s ] = EQ[g(Xi(s))Li(s)|FYs ]

Claim:1

n

n∑i=1

g(Xi(s))Li(s) → EQ[g(X(s))L(s)|FYs ]


Zakai equation 225

Assume X is a diffusion

X(t) = X(0) +

∫ t

0

σ(X(s))dB(s) +

∫ t

0

b(X(s))ds,

where under Q, B and Y are independent. Since

g(X(t)) = g(X(0)) +

∫ t

0

g′(X(s))σ(X(s))dB(s) +

∫ t

0

Ag(X(s))ds

g(X(t))L(t) = g(X(0)) +

∫ t

0

L(s)dg(X(s)) +

∫ t

0

g(X(s))dL(s)

= g(X(0)) +

∫ t

0

L(s)g′(X(s))σ(X(s))dB(s) +

∫ t

0

L(s)Ag(X(s))ds

+

∫ t

0

g(X(s))h(X(s))L(s)dY (s)


Monte Carlo derivation of Zakai equation 226

Xi(t) = Xi(0) +

∫ t

0

σ(Xi(s))dBi(s) +

∫ t

0

b(Xi(s))ds,

where (Xi(0), Bi) are iid copies of (X(0), B).

g(Xi(t))Li(t) = g(Xi(0)) +

∫ t

0

Li(s)g′(Xi(s))σ(Xi(s))dBi(s) +

∫ t

0

Li(s)Ag(Xi(s))ds

+

∫ t

0

g(Xi(s))h(Xi(s))Li(s)dY (s)

and hence

φ(g, t) = φ(g, 0) +

∫ t

0

φ(Ag, s)ds+

∫ t

0

φ(gh, s)dY (s)


Kushner-Stratonovich equation 227

π(g, t) = EP [g(X(t))|FYt ] =

φ(g, t)

φ(1, t)

=φ(g, 0)

φ(1, 0)+

∫ t

0

1

φ(1, s)dφ(g, s)−

∫ t

0

φ(g, s)

φ(1, s)2dφ(1, s)

+

∫ t

0

φ(g, s)

φ(1, s)3d[φ(1, ·)]s −

∫ t

0

1

φ(1, s)2d[φ(g, ·), φ(1, ·)]s

= π(g, 0) +

∫ t

0

π(Ag, s)ds+

∫ t

0

(π(gh, s)− π(g, s)π(h, s))dY (s)

+

∫ t

0

σ2π(g, s)π(h, s)2ds−∫ t

0

σ2π(gh, s)π(h, s)ds

= π(g, 0) +

∫ t

0

π(Ag, s)ds+

∫ t

0

(π(gh, s)− π(g, s)π(h, s))(dY (s)− π(h, s)ds)


Counting process observations 228

Model: X is a diffusion

X(t) = X(0) +

∫ t

0

σ(X(s))dB(s) +

∫ t

0

b(X(s))ds (9.1)

and

Y (t) = V (

∫ t

0

λ(X(s))ds),

where V is unit Poisson process independent of B.

Reference measure construction: Under Q, X is the diffusion given by (9.1) and Yis an independent, unit Poisson process. The change of measure is given by (8.6):

L(t) = 1 +

∫ t

0

(λ(X(s))− 1)L(s−)d(Y (s)− s)


Zakai equation 229

g(X(t))L(t) = g(X(0)) +

∫ t

0

g(X(s))dL(s) +

∫ t

0

L(s)dg(X(s))

= g(X(0)) +

∫ t

0


∫ t

0

L(s)Ag(X(s))ds

+

∫ t

0

g(X(s))(λ(X(s))− 1)L(s−)d(Y (s)− s)

The unnormalized conditional distribution φ(g, t) = EQ[g(X(t))L(t)|FYt ] satisfies

φ(g, t) = φ(g, 0) +

∫ t

0

φ((A− C)g, s)ds+

∫ t

0

φ(Cg, s−)dY (s),

where Cg(x) = (λ(x)− 1)g(x).


Kushner-Stratonovich equation 230

For π(g, t) ≡ φ(g,t)φ(1,t)

,

π(g, t) = π(g, 0) +

∫ t

0

(π(Ag, s)− π(λg, s) + π(λ, s)π(g, s))ds

+

∫ t

0

(π(λg, s−)− π(λ, s−)π(g, s−)

π(λ, s−)

)dY (s)


Solution of the Zakai equation 231

Let T (t) be the semigroup given by

T (t)f(x) = E[f(X(t))e−∫ t0 (λ(X(s))−1)ds]

Suppose the jump times of Y satisfy 0 < τ1 < · · · < τm < t < τm+1. Then

φ(g, t) = φ(T (t− τm)g, τm)

φ(g, τk+1−) = φ(T (τk+1 − τk)g, τk)

φ(g, τk+1) = φ(λg, τk+1−) = φ(T (τk+1 − τk)λg, τk).

If φ(dx, t) = φ(x, t)dx, then

φ(·, t) = T ∗(t− τm)φ(·, τm)

φ(·, τk+1−) = T ∗(τk+1 − τk)φ(·, τk)φ(·, τk+1) = λ(·)φ(·, τk+1−) = λ(·)T ∗(τk+1 − τk)φ(·, τk).


Point process observations 232

Model: X is adapted to Ft, and

Y (t,Γ)−∫ t

0

∫Γ

λ(X(s), u)ν(du)ds

is an Ft-martingale for all Γ ∈ B(U) with ν(Γ) <∞.

supx

∫U

|λ(x, u)− 1| ∧ |λ(x, u)− 1|2ν(du) <∞.

Reference measure construction: Under Q, X is the diffusion given by (9.1) andY is an independent, Poisson random measure with mean measure ν. The changeof measure is given by (8.7):

L(t) = 1 +

∫ t

0

L(s−)dMλ(s) = 1 +

∫U×[0,t]

(λ(X(s), u)− 1)L(s−)Y (du× ds).


Zakai equation 233

g(X(t))L(t) = g(X(0)) +

∫ t

0

g(X(s))dL(s) +

∫ t

0

L(s)dg(X(s))

= g(X(0)) +

∫ t

0


∫ t

0

L(s)Ag(X(s))ds

+

∫U×[0,t]

g(X(s))(λ(X(s), u)− 1)L(s−)Y (du× ds)

Assume that ν(U) <∞. Setting λ(x) =∫

Uλ(x, u)ν(du) and Cg(x) = (λ(x)− 1)g(x),

the unnormalized conditional distribution φ(g, t) = EQ[g(X(t))L(t)|FYt ] satisfies

φ(g, t) = φ(g, 0) +

∫ t

0

φ((A− C)g, s)ds+

∫ t

0

φ((λ(·, u)− 1)g, s−)Y (du× ds).


Solution of the Zakai equation 234

Let T (t) by the semigroup given by

T (t)f(x) = E[f(X(t))e−∫ t0 (λ(X(s))−1)ds]

Suppose Y =∑

k δ(τk,uk), and 0 < τ1 < · · · < τm < t < τm+1. Then

φ(g, t) = φ(T (t− τm)g, τm)

φ(g, τk+1−) = φ(T (τk+1 − τk)g, τk)

φ(g, τk+1) = φ(λ(·, uk+1)g, τk+1−) = φ(T (τk+1 − τk)λ(·, uk+1)g, τk).

If φ(dx, t) = φ(x, t)dx, then

φ(·, t) = T ∗(t− τm)φ(·, τm)

φ(·, τk+1−) = T ∗(τk+1 − τk)φ(·, τk)φ(·, τk+1) = λ(·, uk+1)φ(·, τk+1−) = λ(·, uk+1)T

∗(τk+1 − τk)φ(·, τk).


Stock prices as noisy observations of stock value 235

Stock value:

S(t) = S(0) +

∫ t

0

σS(s)dB(s) +

∫ t

0

µS(s)ds

B standard Brownian motion

N(t) number of trades up to time t (Assume Poisson with parameter λ.)

Pi = S(τi) + ηi price of ith trade, ηi iid with density ρ(z)

Y (t,Γ) cumulative number of trades at prices in Γ.

Y (t,Γ)−∫ t

0

∫Γ

λρ(z − S(s))dzds

is a martingale, Γ ∈ B[0,∞).


Filtering and Bayesian parameter estimation 236

Assume prior on unknown parameters σ, µ

Signal: (S(t), σ, µ)

Problem: Computeπσ,µ(D, t) = P(σ, µ) ∈ D|FY,R

t


Uniformity conditions for sequences of semimartingales 237

Yn = Mn + An, a semimartingale adapted to Fnt

Tt(An), the total variation of An on [0, t]

[Mn]t, the quadratic variation of Mn on [0, t]

Uniform tightness (UT) [JMP]: For Sn0 , the collection of piecewise constant Fn

t -adapted processes,

H0t = ∪∞n=1|

∫ t

0

Z(s−)dYn(s)| : Z ∈ Sn0 , sup

s≤t|Z(s)| ≤ 1

is stochastically bounded.

Uniformly controlled variations (UCV) [KP]: Tt(An), n = 1, 2, . . . is stochasti-cally bounded, and for each α > 0, there exist stopping times τα

n such that

supnE[[Mn]t∧τα

n] <∞

andlim

α→∞sup

nPτα

n ≤ α = 0.


Examples 238

Lemma 9.2 Suppose ηi are iid, E[ηi] = 0, V ar(ηi) = σ2 <∞. Then

Yn(t) =1√n

[nt]∑i=1

ηi

defines a sequence satisfying the uniformity conditions and Yn ⇒ Y , for Y = σW .

Lemma 9.3 Suppose that λn(t) is nondecreasing and satisfies limn→∞ supt≤T |λn(t)−t| =0 for each T > 0. If Y is a Ft-semimartingale, then Yn = Y λn is a Fλn(t)-semimartingale, Yn satisfies the uniformity conditions, and Yn → Y in the Skorohodtopology.

More generally, if Yn satisfies the uniformity conditions and Yn ⇒ Y , then Yn λnsatisfies the uniformity conditions and Yn λn ⇒ Y .

λn can be stochastic provided λn(t) is a Fnt -stopping time for each t.


Basic convergence theorem 239

Theorem 9.4 [5, 7]

(Xn, Yn) Fnt -adapted in DMkm×Rm [0,∞).

Yn = Mn + An an Fnt -semimartingale

Assume that Yn satisfies either UT or UCV.

If (Xn, Yn) ⇒ (X, Y ) in DMkm×Rm [0,∞) with the Skorohod topology, then

(Xn, Yn,

∫Xn(s−)dYn(s)) ⇒ (X, Y,

∫X(s−)dY (s))

in DMkm×Rm×Rk [0,∞)

If (Xn, Yn) → (X, Y ) in probability in the Skorohod topology on DMkm×Rm [0,∞), then

(Xn, Yn,

∫Xn(s−)dYn(s)) → (X, Y,

∫X(s−)dY (s))

in probability in DMkm×Rm×Rk [0,∞)

“IN PROBABILITY” CANNOT BE REPLACED BY “A.S.”


Proof. Suppose |Xn −Xεn| ≤ ε. Then the uniformity conditions imply

limε→0

supnPsup

t≤T|∫ t

0

Xn(s−)dYn(s)−∫ t

0

Xεn(s−)dYn(s)| ≥ δ = 0

Let θ be uniform [0, 1] and independent of Xn, τn,ε0 = 0, and

τn,εk+1 = inft > τn,ε

k : |Xn(t)−Xn(τn,εk )|+ |Yn(t)− Yn(τn,ε

k )| ≥ θε.

Define Xεn(t) = Xn(τn,ε

k ), τn,εk ≤ t < τn,ε

k+1. Claim: (Xεn, Yn) ⇒ (Xε, Y ) and∫

XεndYn ⇒

∫XεdY,

specifically, (Xn(τn,εk ), Yn(τn,ε

k )) ⇒ (X(τ εk), Y (τ ε

k))


Continuity lemma 241

Lemma 9.5 Let Zn be cadlagE-valued processes, and let τnc = inft : r(Zn(t), Zn(0)) ≥

c. If Zn ⇒ Z, then for all but countably many c, (Zn(τnc ), τn

c ) ⇒)Z(τc), τc).


Convergence of exchangeable families 242

Lemma 9.6 For n = 1, 2, . . ., let ξn1 , . . . , ξ

nNn be exchangeable (allowing Nn = ∞.) Let

Ξn be the empirical measure (defined as a limit if Nn = ∞),

Ξn =1

Nn

Nn∑i=1

δξni.

Assume

• Nn →∞

• For each m = 1, 2, . . ., (ξn1 , . . . , ξ

nm) ⇒ (ξ1, . . . , ξm) in Sm.

Then

ξi is exchangeable and setting ξni = s0 ∈ S for i > Nn, Ξn, ξn

1 , ξn2 . . . ⇒ Ξ, ξ1, ξ2, . . .

in P(S)× S∞, where Ξ is the deFinetti measure for ξi.

If for each m, ξn1 , . . . , ξ

nm → ξ1, . . . , ξm in probability in Sm, then Ξn → Ξ in proba-

bility in P(S).


Proof. Ξn ⇒ Ξ if and only if E[〈f,Ξmn 〉] → E[〈f,Ξm〉] for each m ≥ 1 and each

f ∈ C(Sm). If Nn = ∞,

E[〈f,Ξmn 〉] = E[f(ξn

1 , . . . , ξnm)] → E[f(ξ1, . . . , ξm)] = E[〈f,Ξm〉]

If Nn <∞,

E[〈f,Ξmn 〉] =

(Nn

m

)N−m

n E[f(ξn1 , . . . , ξ

nm)] +O(1−

(Nn

m

)N−m

n )‖f‖

→ E[f(ξ1, . . . , ξm)] = E[〈f,Ξm〉]

f ∈ C(Sk+m)

E[〈f(ξn1 , . . . , ξ

nk , ·),Ξm

n 〉] =

(Nn − k

m

)N−m

n E[f(ξn1 , . . . , ξ

nk+m)]+O(1−

(Nn − k

m

)N−m

n )‖f‖


Convergence lemma for processes 244

Lemma 9.7 Let Xn = (Xn1 , . . . , X

nNn

) be exchangeable families of DE[0,∞)-valued ran-dom variables such that Nn ⇒∞ and Xn ⇒ X in DE[0,∞)∞. Define

Ξn = 1Nn

∑Nn

i=1 δXni∈ P(DE[0,∞))

Ξ = limm→∞1m

∑mi= δXi

Vn(t) = 1Nn

∑Nn

i=1 δXni (t) ∈ P(E)

V (t) = limm→∞1m

∑mi=1 δXi(t)

Then

a) For t1, . . . , tl /∈ t : E[Ξx : x(t) 6= x(t−)] > 0

(Ξn, Vn(t1), . . . , Vn(tl)) ⇒ (Ξ, V (t1), . . . , V (tl)).

b) If Xn ⇒ X in DE∞ [0,∞), then Vn ⇒ V in DP(E)[0,∞). If Xn → X in probabilityin DE∞ [0,∞), then Vn → V in DP(E)[0,∞) in probability.


Properties of cadlag processes 245

a) The set DΞ = t : E[Ξx : x(t) 6= x(t−)] > 0 is at most countable.

b) If for i 6= j, with probability one, Xi and Xj have no simultaneous discontinu-ities, then DΞ = ∅ and convergence of Xn to X in DE[0,∞)∞ implies convergencein DE∞ [0,∞).


Approximating the filtering equation 246

Consider

Y (t) =

∫ t

0

h(X(s))ds+ σW (t)

where X is a Markov process with generator A.

φ(g, t) = φ(g, 0) +

∫ t

0

φ(Ag, s)ds+

∫ t

0

φ(gh, s)dY (s)

φ(g, t) = limm→∞

1

m

m∑i=1

g(Xi(t))Li(t) =

∫E×R

g(x)zV (t, dx× dz)

where Xi are independent copies of X ,

Li(t) = 1 +

∫ t

0

h(Xi(s))

σLi(s)dY (s),

and

V (t) = limm→∞

1

m

m∑i=1

δ(Xi(t),Li(t))


Continuous dependence on signal 247

Suppose Xn has generator An and Xn ⇒ X , and hence, (Xn, Y ) ⇒ (X, Y ). Then

Ln(t) = exp∫ t

0

h(Xn(s))

σdY (s)− 1

2

∫ t

0

h2(Xn(s))

σ2ds

and if h is continuous, (Xn, Y, Ln) ⇒ (X, Y, L). More generally, ((Xni , L

ni ), Y ) ⇒

((Xi, Li), Y ), and Vn ⇒ V .

φn(g, t) ≥∫g(x)z ∧ cVn(t, dx× dz) ⇒

∫g(x)z ∧ cV (t, dx× dz)

EQn [|φn(g, t)−∫g(x)z ∧ cVn(t, dx× dz)|] ≤ ‖g‖EQn [

∫(z − z ∧ c)Vn(t, dx× dz)]

≤ ‖g‖(1− EQn [

∫z ∧ cVn(t, dx× dz))]

→ ‖g‖(1− EQ[

∫z ∧ cV (t, dx× dz))]

= ‖g‖EQ[L(t)− L(t) ∧ c]


Consistency of Monte Carlo approximation 248

Suppose X(t) = X(0) +∫ t

0σ(X(s))dB(s) +

∫ t

0b(X(s))ds, Euler approximation:

Bh(t) = B([t/h]h), Ah(t) = [t/h]h, Y h(t) = Y ([t/h]h).

Xh(t) = X(0) +

∫ t

0

σ(Xh(s))dBh(s) +

∫ t

0

b(Xh(s))dAh(s)

Xhi (t) = Xi(0) +

∫ t

0

σ(Xhi (s))dBh

i (s) +

∫ t

0

b(Xhi (s))dAh(s)

Lhi (t) = 1 +

∫ t

0

h(Xhi (s))

σLh

i (s)dYh(s)

Claim: As h → 0, ((Xhi , L

hi ), Y h) → ((Xi, Li), Y ) in probability, and hence, as

h→ 0 and n→∞,1

n

n∑i=1

δ(Xhi ,Lh

i ) → limm→∞

1

m

m∑i=1

δ(Xi,Li)

and1

n

n∑i=1

g(Xhi (t))Lh

i (t) → φ(g, t)


Finite dimensional approximation 249

Assume En ⊂ E is finite. Xn(t) ∈ En, t ≥ 0, and

Anf(z) =∑y∈En

λ(z, y)(f(y)− f(z))

Let Xni be independent copies of Xn, let

Lni (t) = 1 +

∫ t

0

h(Xni (s))

σLn

i (s)dY (s)

and define

φn(g, t) = limm→∞

1

m

m∑i=1

g(Xni (t))Ln

i (t).

Then

φn(g, t) = φn(g, 0) +

∫ t

0

φn(Ang, s)ds+

∫ t

0

φn(gh, s)dY (s)


φn given by finite dimensional system 250

gx = 1x, x ∈ En, φn(x, t) = φn(gx, t). Then

φn(x, t) = φn(x, 0) +

∫ t

0

(∑z 6=x

λ(z, x)φn(z, s)−∑y 6=x

λ(x, y)φn(x, s)

)ds

+

∫ t

0

h(x)φn(x, s)dY (s)

If Xn ⇒ X , thenφn(g, t) → φ(g, t)


10. Averaging

• Convergence of random measures

• Diffusion with rapidly varying coefficients

• Markov process with two time scales

• Diffusion approximation for random evolutions


Convergence of random measures 252

M(S) the space of finite measures on S with the weak topology:

µn → µ if and only if∫fdµn →

∫fdµ, f ∈ C(S)

Prohorov metric:

ρ(µ, ν) = infε > 0 : µ(B) ≤ ν(Bε) + ε, ν(B) ≤ µ(Bε) + ε, B ∈ B(S), (10.1)

where Bε = s ∈ S : infy∈B d(x, y) < ε. The following lemma is a simple conse-quence of Prohorov’s theorem.

Lemma 10.1 Let Γn be a sequence of M(S)-valued random variables. Then Γn is rela-tively compact if and only if Γn(S) is relatively compact as a family of R-valued randomvariables and for each ε > 0, there exists a compact K ⊂ S such that supn PΓn(Kc) >ε < ε.


Proof. The proof of necessity is left to the reader. To prove sufficiency, fix η > 0.Then there exist compact sets Kk ⊂ S such that

supnPΓn(Kc

k) > η2−(k+1) < η2−(k+1)

and a constant c such that

supnPΓn(S) > c < η

2.

Define K = µ ∈ M(S) : µ(Kck) ≤ η2−(k+1), k = 1, 2, . . . , µ(S) ≤ c, and observe

that PΓn ∈ K ≥ 1− η. Prohorov’s theorem implies that K is a compact subset ofM(S), and consequently, again by Prohorov’s theorem, Γn is relatively compact.


A convergence lemma 254

Let L(S) be the space of measures on [0,∞) × S such that µ([0, t] × S) < ∞ foreach t > 0, and let Lm(S) ⊂ L(S) be the subspace on which µ([0, t] × S) = t. Forµ ∈ L(S), let µt denote the restriction of µ to [0, t] × S. Let ρt denote the Prohorovmetric on M([0, t]× S), and define ρ on L(S) by

ρ(µ, ν) =

∫ ∞

0

e−t1 ∧ ρt(µt, νt)dt,

that is, µn converges in ρ if and only if µtn converges weakly for almost every

t.

Lemma 10.2 Let (xn, µn) ⊂ DE[0,∞)×L(S), and (xn, µn) → (x, µ). Let h ∈ C(E×S). Define

un(t) =

∫[0,t]×S

h(xn(s), y)µn(ds× dy), u(t) =

∫[0,t]×S

h(x(s), y)µ(ds× dy)

zn(t) = µn([0, t]× S), and z(t) = µ([0, t]× S).

a) If x is continuous on [0, t] and limn→∞ zn(t) = z(t), then limn→∞ un(t) = u(t).


b) If (xn, zn, µn) → (x, z, µ) inDE×R[0,∞)×L(S), then (xn, zn, un, µn) → (x, z, u, µ)in DE×R×R[0,∞) × L(S). In particular, limn→∞ un(t) = u(t) at all points of conti-nuity of z.

c) The continuity assumption on h can be replaced by the assumption that h is con-tinuous a.e. νt for each t, where νt ∈ M(E × S) is the measure determined byνt(A×B) = µ(s, y) : x(s) ∈ A, s ≤ t, y ∈ B.

d) In both (a) and (b), the boundedness assumption on h can be replaced by the as-sumption that there exists a nonnegative convex function ψ on [0,∞) satisfyinglimr→∞ ψ(r)/r = ∞ such that

supn

∫[0,t]×S

ψ(|h(xn(s), y)|)µn(ds× dy) <∞

for each t > 0.


Proof. Let h ∈ C(E × S). For each ε > 0 and t > 0, there exists a compact K ⊂ Swith supn µn([0, t]×Kc) ≤ ε. If x is continuous, then

limn→∞

supy∈K,s≤t

|h(xn(s), y)− h(x(s), y)| = 0,

and if zn(t) → z(t), it follows that

lim supn→∞

∣∣∣∣∫[0,t]×S

h(xn(s), y)µn(ds× dy)−∫

[0,t]×S


∣∣∣∣ ≤ 2‖h‖ε.

If (xn, zn) → (x, z) in the Skorohod topology, then there exist continuous, strictlyincreasing mappings ηn of [0,∞) onto [0,∞) such that ηn(t) → t for each t and (xnηn, zn ηn) → (x, z) uniformly on bounded intervals. Define µn so that µn([0, t] ×H) = µn([0, ηn(t)]×H) and observe that µn → µ in L(S). But the uniformity of theconvergence of xn ηn to x and zn ηn to z implies∫

[0,ηn(t)]×S

h(xn(s), y)µn(ds× dy) =

∫[0,t]×S

h(xn ηn(s), y)µn(ds× dy)

→∫

[0,t]×S


for each fixed t.


Diffusion with rapidly varying coefficients 258

Let Y be a stationary process. An event of the form Y ∈ Γ is invariant if Y (t+·) ∈Γ does not depend on t. A stationary process Y is ergodic if every invariant eventhas probability zero or one. If Y is ergodic, the ergodic theorem implies

limT→∞

1

T

∫ T

0

g(Y (s))ds =

∫gdµ,

where µ is the marginal distribution of Y .

Let Y be an ergodic stationary process, independent of W , and

Xn(t) = X(0) +

∫ t

0

σ(Xn(s), Y (ns))dW (s) +

∫ t

0

b(Xn(s), Y (ns))ds

What happens as n→∞?

Note that∫ t+δ

t

b(x, Y (ns))ds =1

n

∫ n(t+δ)

nt

b(x, Y (u))du→ δ

∫E

b(x, y)µ(dy),

suggesting that the drift term is asymptotic to∫ t

0b(Xn(s))ds, where b(x) =

∫Eb(x, y)µ(dy).


Martingale approach 259

LetAf(x, y) =

1

2a(x, y)f ′′(x) + b(x, y)f ′(x),

and

Γn([0, t]×B) =

∫ t

0

1B(Yn(s))ds

Then, assuming relative compactness of Xn and observing that Γn → m× µ

f(Xn(t))−∫ t

0

Af(Xn(s), Yn(s))ds = f(Xn(t))−∫

[0,t]×E

Af(Xn(s), y)Γn(ds× dy)

⇒ f(X(t))−∫ t

0

∫E

Af(X(s), y)µ(dy)ds

so any limit point is a solution of the martingale problem for

Af(x)−∫

E

Af(x, y)µ(dy)


Conditions for tightness 260

Theorem 10.3 For n = 1, 2, . . ., let Xn be a cadlag, E-valued process adapted to a fil-tration Fn

t , and let D ⊂ C(E) be closed under multiplications and separate points.Suppose

a) (Compact containment) For each ε > 0 and T > 0, there exists a compact Kε,T ⊂ Esuch that

infnPXn(t) ∈ Kε,T , 0 ≤ t ≤ T ≥ 1− ε.

b) For each n = 1, 2, . . ., f ∈ D, T > 0, there exists γnf,T (δ) ≥ 0, δ ≥ 0, such that

|E[f(Xn(t+ u))− f(Xn(t))|Fnt ]| ≤ E[γn

f,T (δ)|Fnt ], 0 ≤ t ≤ T, 0 ≤ u ≤ δ,

andlimδ→0

lim supn→∞

E[γnf,T (δ)] = 0.

Then Xn is relatively compact.


Proof. Note that

|E[(f(Xn(t+ u))− f(Xn(t)))2|Fnt ]|

≤ |E[f 2(Xn(t+ u))− f 2(Xn(t))|Fnt ]|

+2‖f‖|E[f(Xn(t+ u))− f(Xn(t))|Fnt ]|

≤ E[γnf2,T (δ)|Fn

t ] + 2‖f‖E[γnf,T (δ)|Fn

t ]


Verification of compact containment 262

|Xn(t)|2 = |Xn(0)|2 +

∫ t

0

2Xn(s)Tσ(Xn(s), Yn(s))dW (s)

+

∫ t

0

(d∑

i=1

aii(Xn(s), Yn(s)) + 2Xn(s) · b(Xn(s), Yn(s))

)ds

Simple condition:

d∑i=1

aii(x, y) + 2x · b(x, y) ≤ K1 +K2|x|2 +K3|y|2

τc = inft|Xn(t)| ≥ c.


Estimates for compact containment 263

Assume that supn suptE[|Yn(t)|2] <∞.

E[|Xn(t ∧ τc)|2]≤ E[|Xn(0)|2] +

+

∫ t

0

(K1 +K2E[|Xn(s ∧ τc)|2] +K3E[|Yn(s)|2]

)ds


Markov process with two time scales 265

Suppose

Xn(t) = X(0) +

∫ t

0

σ(Xn(s), Yn(s))dW (s) +

∫ t

0

b(Xn(s), Yn(s))ds

Yn(t) = Y (0) +

∫ t

0

√βnα(Xn(s), Yn(s))dW (s) +

∫ t

0

βnc(Xn(s), Yn(s))ds

For simplicity, assume all coefficents are bounded. Then

f(Xn(t))−∫ t

0

Af(Xn(s), Yn(s))ds

and

g(Yn(t))−∫ t

0

βnBg(Xn(s), Yn(s))ds

are martingales for f ∈ C2c (Rd), g ∈ C2

c (Rm),A,B appropriately defined differentialoperators.


Abstract setting 266

Suppose that for A ⊂ C(E1)× C(E1 × E2),

f(Xn(t))−∫ t

0

Af(Xn(s), Yn(s))ds+ εfn(t), (10.2)

is a Fnt -martingale and E[|εfn(t)|] → 0, t ≥ 0, and that forB ⊂ C(E2)× C(E1×E2)

g(Yn(t))−∫ t

0

βnBg(Xn(s), Yn(s))ds+ δgn(t)

is an Fnt -martingale, βn →∞, and for each t ≥ 0,

limn→∞

E[β−1n |δg

n(t)|] = 0.

Then if (Xn,Γn) is relatively compact,∫[0,t]×E2

Bg(X(s), y)Γ(ds× dy) (10.3)

is a martingale, and since (10.3) is continuous and of bounded variation∫[0,t]×E2

Bg(X(s), y)Γ(ds× dy) = 0, t ≥ 0, a.s. (10.4)


Local stationarity 267

Suppose that there exists a countable subset D ⊂ D(B) such that the bounded,pointwise closure of (g,Bg) : g ∈ D is the same as the bounded, pointwiseclosure of (g,Bg) : g ∈ D(B). (For example, ifD(B) = C∞

c (Rd) and B is a secondorder differential operator with locally bounded coefficients.) By (10.4),∫ t

0

∫E2

Bg(X(s), y)γs(dy)ds = 0

for all t a.s., and hence ∫E2

Bg(X(s), y)γs(dy) = 0 (10.5)

a.e. m a.s. Consequently, with probability one, there exists a single set Q ⊂ [0,∞)

with m(Q) = 0 such that (10.5) holds for all g ∈ D and all s ∈ [0,∞) − Q. But thechoice of D ensures that (10.5) holds for all g ∈ D(B) and all s ∈ [0,∞)−Q.


Limiting martingale problem 268

Define Bx : D(B) → C(E2) by Bxg(y) = Bg(x, y). Suppose that there is a uniquemeasure πx in P(E2) satisfying∫

E2

Bxgdπx = 0, g ∈ D(B).

(If Bx is the generator for an E2-valued Markov process, this assumption is essen-tially the assertion that there is a unique stationary distribution corresponding toBx.) Then we can take γs = πX(s), and defining C on D(A) by

Cf(x) =

∫E2

Af(x, y)πx(dy),

it follows that X is a solution of the martingale problem for C.


Averaging for stochastic equations 269

W a standard Brownian motion in R

Xn(0) independent of W

Y a stochastic process with state space U , independent of W and Xn(0)

Define Yn(t) = Y (nt).

Xn(t) = Xn(0) +

∫ t

0

σ(Xn(s), Yn(s))dW (s)

+

∫ t

0

b(Xn(s), Yn(s))ds

Define Mn(A, t) =∫ t

01A(Yn(s))dW (s) and Vn(A, t) =

∫ t

01A(Yn(s))ds, so that

Xn(t) = Xn(0) +

∫U×[0,t]

σ(Xn(s), u)Mn(du× ds)

+

∫U×[0,t]

b(Xn(s), u)Vn(du× ds)


Convergence of Driving Processes 270

Define

Mn(ϕ, t) =

∫ t

0

ϕ(Yn(s))dW (s)

Vn(ϕ, t) =

∫ t

0

ϕ(Yn(s))ds

Assume that1

t

∫ t

0

ϕ(Y (s))ds→∫

U

ϕ(u)ν(du)

in probability for each ϕ ∈ C(U).

Observe that

[Mn(ϕ1, ·),Mn(ϕ2, ·)]t =

∫ t

0

ϕ1(Yn(s))ϕ2(Yn(s))ds

→ t

∫U

ϕ1(u)ϕ2(u)ν(du)


Limiting Equation 271

The functional central limit theorem for martingales implies Mn(ϕ, t) ⇒ M(ϕ, t)where M is Gaussian with

E[M(ϕ1, t)M(ϕ2, s)] = t ∧ s∫

U

ϕ1(u)ϕ2(u)ν(du)

and Vn(ϕ, t) → t∫

Uϕ(u)ν(du)

Can we conclude that Xn ⇒ X satisfying

X(t) = X(0) +

∫ t

0

∫U

σ(X(s), u)M(du× ds)

+

∫ t

0

∫U

b(X(s), u)ν(du)ds


Construction of a partition of unity 272

Lemma 10.4 Let (S, d) be a complete, separable metric space, and let xk be a countabledense subset of S. Then for each ε > 0, there exists a sequence ψε

k ⊂ C(S) such thatsuppψε

k ⊂ Bε(xk), 0 ≤ ψεk ≤ 1, |ψε

k(x)−ψεk(y)| ≤ 4

εd(x, y), and for each compact K ⊂

S, there exists NK <∞ such that∑NK

k=1 ψεk(x) = 1, x ∈ K. In particular,

∑∞k=1 ψ

εk(x) =

1 for all x ∈ S.

Proof.Fix ε > 0. Let ψk(x) = (1 − 2εd(x,Bε/2(xk)) ∨ 0. Then 0 ≤ ψk ≤ 1, ψk(x) = 1,

x ∈ Bε/2(xk), and ψk(x) = 0, x /∈ Bε(xk). Note also that |ψk(x) − ψk(y)| ≤ 2εd(x, y).

Define ψε1 = ψ1, and for k > 1, ψε

k = maxi≤k ψi −maxi≤k−1 ψi. Clearly, 0 ≤ ψεk ≤ ψk

and∑k

i=1 ψεi = maxi≤k ψi. In particular, for compact K ⊂ S, there exists NK < ∞

such that K ⊂ ∪NKk=1Bε/2(xk) and hence

∑NK

k=1 ψεk(x) = 1 for x ∈ K. Finally,

|ψεk(x)− ψε

k(y)| ≤ 2 maxi≤k

|ψi(x)− ψi(y)| ≤4

εd(x, y)


Construction of Stochastic Integral for SMRM 273

H a Banach space

Y is an H#-semimartingale if

• For each ϕ ∈ H , Y (ϕ, ·) is an Ft-semimartingale.

• Y (∑m

k=1 akϕk, ·) =∑m

k akY (ϕk, ·) a.s.

ϕk dense in H , ψεk partition of unity with supp(ψε

k) ⊂ Bε(ϕk).

For X cadlag, H-valued, Ft-adapted, define

Xε(t) =∑

k

ψεk(X(t))ϕk.

Then Xε is cadlag and Ft-adaptd and ‖X −Xε‖H ≤ ε.

Define

Xε− · Y (t) =

∑k

∫ t

0

ψεk(X(s−))dY (ϕk, s) (=

∫U×[0,t]

Xε(t, u)Y (du× ds)).


Good Integrator Condition 274

Let S be the collection of cadlag, adapted processes of the formZ(t) =∑m

k=1 ξk(t)ϕk.Define

Z− · Y =∑

k

∫ t

0

ξk(s−)dY (ϕk, s).

Basic assumption:

Ht = sups≤t

|Z− · Y (s)| : Z ∈ S, sups≤t

‖Z(s)‖H ≤ 1

is stochastically bounded.


Definition of integral 275

Stochastic boundedness of Ht implies there exists K(t, δ) such that

Psups≤t

|Z− · Y (s)| ≥ K(t, δ) ≤ δ

for all Z ∈ S satisfying sups≤t ‖Z(s)‖H ≤ 1

Y aH#-semimartingale satisfying the good itegrator condition,X a cadlag, adapted,H-valued process, Xε as above. Then

X− · Y ≡ limε→0

Xε− · Y

exists in the sense that for all η > 0

limε→0

Psups≤t

|X · Y (s)−Xε · Y (s)| > η = 0

Since ‖Xε1(s)−Xε2(s)‖H ≤ ε1 + ε2,

Psups≤t

|Xε1− · Y (s)−Xε2

− · Y (s)| ≥ (ε1 + ε2)K(δ, t) ≤ δ

and Xε− · Y is Cauchy in probability.


Convergence for H#-semimartingales 276

H a separable Banach space of functions on U

Yn an Fnt -H#-semimartingale (for eachϕ ∈ H , Y (ϕ, ·) is an Fn

t -semimartingale)

Xn cadlag, H-valued processes

(Xn, Yn) ⇒ (X, Y ), if

(Xn, Yn(ϕ1, ·), . . . , Yn(ϕm, ·))⇒ (X,Y (ϕ1, ·), . . . , Y (ϕm, ·))

in DH×Rm [0,∞) for each choice of ϕ1, . . . , ϕm ∈ H .


Convergence for Stochastic Integrals 277

LetHn,t = sup

s≤t|Z− · Yn(s)| : Z ∈ Sn, sup

s≤t‖Z(s)‖H ≤ 1,

Definition: Yn is uniformly tight if ∪nHn,t is stochastically bounded for each t.

Theorem 10.5 Assume that Yn is uniformly tight. If (Xn, Yn) ⇒ (X, Y ), then thereis a filtration Ft, such that Y is an Ft-adapted, standard, H#-semimartingale, X isFt-adapted and

(Xn, Yn, Xn− · Yn) ⇒ (X, Y,X− · Y )

If (Xn, Yn) → (X, Y ) in probability, then (Xn, Yn, Xn− · Yn) → (X,Y,X− · Y ) in proba-bility.


Averaging theorem 278

Xn(t) = Xn(0) +

∫ t

0

σ(Xn(s), Yn(s))dW (s) +

∫ t

0

b(Xn(s), Yn(s))ds

= Xn(0) +

∫U×[0,t]

σ(Xn(s), u)Mn(du× ds) +

∫U×[0,t]

b(Xn(s), u)Vn(du× ds)

where Mn(A, t) =∫ t

01A(Yn(s))dW (s) and Vn(A, t) =

∫ t

01A(Yn(s))ds.

Then Mn(ϕ, t) ⇒M(ϕ, t), where M is Gaussian with

E[M(ϕ1, t)M(ϕ2, s)] = t ∧ s∫

U

ϕ1(u)ϕ2(u)ν(du)

and Vn(ϕ, t) → t∫

Uϕ(u)ν(du).

Let H = L2(ν). If x ∈ R → σ(x, ·) ∈ H and x ∈ R → b(x, ·) ∈ L1(ν) are boundedand continuous, then Xn ⇒ X satisfying

X(t) = X(0) +

∫ t

0

∫U

σ(X(s), u)M(du× ds) +

∫ t

0

∫U

b(X(s), u)ν(du)ds


Diffusion approximation for random evolutions 279

Consider a sequence of stochastic ordinary differential equations in Rk of the form

Xn(t) = G(Xn(t), Y (n2t)) + nH(Xn(t), Y (n2t)) (10.6)

where Y is a stochastic process representing the noise in the system.

Let Y be a continuous time Markov chain with state space E = 1, . . . ,m andintensity matrix Q.

Assume that Q is irreducible and hence that there is a unique stationary distribu-tion π.

Suppose ΣβH(x, β)πβ = 0 and define Vn and Wn by

V βn (t) =

∫ t

0

1β(Y (n2s))ds→ πβt

and

W βn (t) = n(V β

n (t)− πβt) = n

∫ t

0

(1β(Y (n2s))− πβ)ds .


Equation driven by Vn and Wn 280

Then (10.6) becomes

Xn(t) = Xn(0) +m∑

β=1

∫ t

0

G(Xn(s), β)dV βn (s) +

m∑β=1

∫ t

0

H(Xn(s), β)dW βn (s) .

Wn converges to a Brownian motion, but does not satisfy the uniformity condi-tions of the stochastic integral convergence theorem.

Let hβ satisfym∑

k=1

qjkhβ(k) = 1β(j)− πβ

(hβ exists by the uniqueness of π), and note that Yn defined by

Y βn (t) = n

∫ t

0

(1β(Y (n2s))− πβ)ds− 1

nhβ(Y (n2t)) +

1

nhβ(Y (0))

is a martingale and Wn = Yn + Zn, where

Zβn (t) =

1

nhβ(Y (n2t))− 1

nhβ(Y (0)).


Integration by parts 281

Note that∫ t

0

H(Xn(s), β)dW βn (s) =

∫ t

0

H(Xn(s), β)dY βn (s) +

∫ t

0

H(Xn(s), β)dZβn (s)

and that Yn satisfies the uniformity conditions, so the first stochastic integralconverges appropriately.∫ t

0

H(Xn(s), β)dZβn (s)

= H(Xn(t), β)Zβn (t)−H(Xn(0), β)Zβ

n (0)−∫ t

0

Zβn (s)dH(Xn(s), β)

= −∑

γ

∫ t

0

Zβn (s)H ′(Xn(s), β)G(Xn(s), γ)dV γ

n (s)

−∑

γ

∫ t

0

Zβn (s−)H ′(Xn(s), β)H(Xn(s), γ)dY γ

n (s)

−∑

γ

∫ t

0

H ′(Xn(s)β)H(Xn(s), γ)Zβn (s−)dZγ

n(s)


Convergence 282

Let Nij(t) denote the number of transitions of Y from state i to state j up to time t.Then

[Y βn , Y

γn ]t =

m∑ij=1

Nij(n2t)

n2(hβ(j)− hβ(i))(hγ(j)− hγ(i))

[Y βn , Z

γn ]t = −[Y β

n , Yγn ]t

and∫ t

0

Zβn (s−)dZγ

n(s) =m∑

ij=1

Nij(n2t)

n2hβ(i)(hγ(j)− hγ(i))−

1

nhβ(Y (0))Zγ

n(t) .

[Y βn , Y

γn ]t → Cβγt ≡

m∑ij=1

πiqij(hβ(j)− hβ(i))(hγ(j)− hγ(i))t∫ t

0

Zβn (s−)dZγ

n(s) → Dβγt ≡m∑

ij=1

πiqijhβ(i)(hγ(j)− hγ(i))t .


Limit theorem 283

The martingale central limit theorem gives Yn ⇒ Y where Y is a Brownian motionwith infinitesimal covariance C = ((Cβγ)).

Theorem 10.6 Let Y in (10.6) be a finite Markov chain with state space E = 1, . . . ,mand intensity matrix Q, and let Xn(0) be independent of Y . Let G be bounded and con-tinuous, and let H be bounded and have bounded and continuous first and second deriva-tives. Assume that Q is irreducible and that π, ((Cβγ)) and ((Dβγ)) are as above. DefineG : R → R by G(x) = ΣβG(x, β)πβ . If Xn(0) ⇒ X(0), then Xn is relatively compactand any limit point satisfies

X(t) = X(0) +

∫ t

0

G(X(s))ds+∑

β

∫ t

0

H(X(s), β)dY β(s)

+∑β,γ

∫ t

0

H ′(X(s), β)H(X(s), γ)Dγβds .


11. Control

• Examples

• Controlled martingale problems

• Standard cost structures

• Relaxed controls

• Pasting lemma

• Bellman principle and the Nisio semigroup

• Hamilton-Jacobi-Bellman equation

• Linear programming formulation

• Relationship of linear program to HJB equation

• Viscosity solutions


Controlled queueing networks 285


Deviations from a target value 286


Recalibration 287


Controlled martingale problems 288

E state space F control space

A : D(A) ⊂ C(E) → C(E × F )

Definition 11.1 (X,U) is a solution of the controlled martingale problem forA if thereis a filtration Ft such that

f(X(t))−∫ t

0

Af(X(s), U(s))ds

is an Ft-martingale for each f ∈ D(A).

Examples of generators

Af(x, u) =1

2

∑aij(x, u)

∂2

∂xi∂j

f(x) +∑

bi(x, u)∂

∂xi

f(x)

E = Rd

Af(x, u) = λ(x, u)

∫E

(f(y)− f(x))µ(x, u, dy)


Standard cost structuresFinite horizon:

Running cost: c(x, u)

Terminal cost: g(x)∫ t

0

c(X(s), U(s))ds+ g(X(t)) or∫ t

0

∫F

c(X(s), u)λs(du)ds+ g(X(t))

Discounted infinite horizon: α > 0∫ ∞

0

e−αsc(X(s), U(s))ds or∫ ∞

0

e−αs

∫F

c(X(s), u)λs(du)ds

Long run average:

limt→∞

1

t

∫ t

0

c(X(s), U(s))ds or limt→∞

1

t

∫ t

0

∫F

c(X(s), u)λs(du)ds


Relaxed controls 290

L(F ) space of measures on F × [0,∞) with µ(F × [0, t]) <∞ for each t > 0.

Lm(F ) ⊂ L(F ) subspace with µ(F × [0, t]) = t.

Lm,T (F ) space of measures on F × [0, T ] that are restrictions of measures in Lm(F ).

Definition 11.2 A process (X,Λ) in DE[0,∞) × Lm(F ) is a relaxed solution of thecontrolled martingale problem for A if there exists a filtration Ft such that (X,Λ) isFt-adapted and for each f ∈ D(A)

Mf (t) = f(X(t))−∫

F×[0,t]

Af(X(s), u)Λ(du× ds)

is an Ft-martingale.

Λ(du× ds) = λs(du)ds so

Mf (t) = f(X(t))−∫ t

0

∫F

Af(X(s), u)λs(du)ds


Pasting lemma 291

Lemma 11.3 Let (E, r), (S1, ρ1), and (S2, ρ2) be complete separable metric spaces, letP1 ∈ P(S1) and P2 ∈ P(S2), and suppose that X1 : S1 → E and X2 : S2 → E are Borelmeasurable. If P1X1 ∈ · = P2X2 ∈ · = µ ∈ P(E), then there exists P ∈ P(S1 × S2)such that for each Z1 ∈ B(S1) and Z2 ∈ B(S2),

EP [Z1Z2] =

∫E

EP1 [Z1|X1 = x]EP2 [Z2|X2 = x]µ(dx)

and X1 = X2 a.s. P .


Properties of the collection of solutions 292

Suppose that every solution has a cadlag modification, so we can identify a solu-tion with its distribution on DE[0,∞)× Lm(F ).

Let Γ ⊂ P(DE[0,∞)×Lm(F )) be the collection of relaxed solutions of the controlledmartingale problem A, and let Γν ⊂ Γ be the collection of solutions with intitialdistribution ν.

For P ∈ Γ, t > 0, µ = PX(t)−1, and P ′ ∈ Γµ, there exists Q ∈ Γ such that

Q(X(· ∧ t),Λ(· ∧ t)) ∈ A, (X(t+ ·),Λ(t+ ·)− Λ(t)) ∈ B

=

∫E

P(X(· ∧ t),Λ(· ∧ t)) ∈ A|X(t) = x

P ′(B|X(0) = x)µ(dx)


Compactness lemma 294

Lemma 11.4 Let

Γkν = P ∈ Γν : EP [

∫ ∞

0

e−αt

∫F

c(X(t), u)Λ(du× dt)] ≤ k.

Suppose D(A) is closed under multiplication and separates points. If E and F are com-pact (e.g., E = Rd ∪ ∞) and (11.1) holds for all f ∈ D(A), then Γk

ν is compact inP(DE[0,∞)× Lm(F ))


Bellman principle and the Nisio semigroup 295

Lemma 11.5 Suppose that c and g are lower semicontinuous and bounded below. Define

S(t)g(ν) = inf(X,λ)∈Γν

E

[∫ t

0

∫F

c(X(s), u)λs(du)ds+ g(X(t))

](11.2)

Under the conditions of the compactness lemma, the infimum is achieved, and settingS(t)g(x) = S(t)g(δx),

S(t)g(ν) =

∫E

S(t)g(x)ν(dx).

Proof. Check that for 0 ≤ a ≤ 1,

S(t)g(aν1 + (1− a)ν2) = aS(t)g(ν1) + (1− a)S(t)g(ν2)


Semigroup property 296

S(t+ r)g(ν) = infP∈Γν

EP[ ∫ t

0

∫F

c(X(s), u)λs(du)ds

+

∫ t+r

t

∫F

c(X(s), u)λs(du) + g(X(t+ r))]

≥ infP∈Γν

EP

[∫ t

0

∫F

c(X(s), u)λs(du)ds+ S(r)g(X(t))

]= S(t)S(r)g(ν),


S(t)S(r)g(ν) = infQ∈Γν

EQ

[∫ t

0

∫F

c(X(s), u)λs(du)ds+ S(r)g(X(t))

]≥ inf

Q∈Γν

(EQ

[∫ t

0

∫F

c(X(s), u)λs(du)ds

]+ inf

P∈ΓQX(t)−1

EP

[∫ r

0

∫F

c(X(s), u)λs(du)ds+ g(X(r))

])

≥ infQ∈Γν

(EQ

[∫ t

0

∫F

c(X(s), u)λs(du)ds+

∫ t+r

t

∫F

c(X(s), u)λs(du)ds+ g(X(t+ r))

]= S(t+ r)g(ν)


Properties of the semigroup 298

S(t) is a nonlinear contraction semigroup

S(t+ r)g(ν) = S(t)S(r)g(ν)

‖S(t)g − S(t)h‖∞ ≤ ‖g − h‖∞


Nonlinear semigroup generators 299

If E and F are compact, c is continuous, and g ∈ D(A), then

S(t)g(x)− g(x)

= inf(X,Λ)∈Γx

E

[∫ t

0

∫F

(c(X(s), u) + Ag(X(s), u))λs(du)ds

]and hence

limt→0

S(t)g(x)− g(x)

t= Cg(x) = inf

u(c(x, u) + Ag(x, u))

so S(t)g should satisfy the Hamilton-Jacobi-Bellman equation

d

dtS(t)g = CS(t)g.


Hamilton-Jacobi-Bellman equation (discounted case) 300

Define Rαh(ν) = infP∈Γν EP [∫∞

0e−αs(

∫Fc(X(s), u)λs(du) + h(X(s)))dt]

Again Rαh(ν) =∫

ERαh(x)ν(dx)

S(t)Rαh(ν)

= infP∈Γν

EP [

∫ t

0

∫R

c(X(s), u)λs(du) +Rαh(X(t))]

= infP∈Γν

EP [

∫ t

0

∫F

c(X(s), u)λs(du) +

∫ ∞

t

e−α(s−t)(

∫F

c(X(s), u)λs(du) + h(X(s))ds]

= infP∈Γν

EP [

∫ t

0

(1− e−αs)

∫R

c(X(s), u)λs(du)]

+(eαt − 1)EP [

∫ ∞

t

e−αs(

∫F

c(X(s), u)λs(du) + h(X(s))ds]

−EP [

∫ t

0

e−αsh(X(s))ds] + EP [

∫ ∞

0

e−αs(

∫F

c(X(s), u)λs(du) + h(X(s)))dt]

limt→0

t−1(S(t)Rαh(x)−Rαh(x)) = αRαh(x)− h(x)


For h ≡ 0, set Vα(x) = Rαh(x). Then we should have

αVα(x)− CVα(x) = αVα(x)− infu

(c(x, u) + AVα(x, u)) = 0


Linear programming formulation 302

If (X,λ) is a solution of the controlled martingale problem and νt ∈ P(E × F ) isdefined by

νth ≡∫hdνt = E[

∫F

h(X(t), u)λt(du)],

then

νtf = ν0f +

∫ t

0

νsAfds, f ∈ D(A). (11.3)

In the discounted case, the goal is to minimize∫ ∞

0

e−αtνtcdt = E[

∫ ∞

0

e−αt

∫F

c(X(t), u)λt(du)]

(11.3) implies

e−αtνtf = ν0f +

∫ t

0

e−αs(νsAf − ανsf)ds = ν0f +

∫ t

0

e−αsνs(Af − αf)ds

and henceαν0f = α

∫ ∞

0

e−αsνs(αf − Af)ds ≡ π(αf − Af)


Linear Program 303

Minimizeπc =

∫E×F

c(x, u)π(dx× du)

subject toπ(αf − Af) = αν0f, ∀f ∈ D(A).


Relationship of linear program to HJB equation 304

∫c(x, u)π(dx× du) =

∫(c(x, u) + Af(x, u)− αf(x))π(dx× du) + αν0f

≥∫ [

infu

(c(x, u) + Af(x, u)− αf(x))]π(dx× du) + αν0f.

Consequently, if Vα is a solution of the HJB equation∫c(x, u)π(dx× du) ≥ αν0Vα

LetΓ = (x, u) : (c(x, u) + AVα(x, u)− αVα(x)) = 0.

If there exists a solution of the controlled martingale problem for (A, ν0) with∫ t

0

∫U

1Γ(X(s), u)λs(du)ds = t, t ≥ 0,

thenE[

∫ ∞

0

e−αt

∫c(X(t), u)λt(du)dt] =

∫Vα(y)ν0(dy),

proving that∫Vαdν0 is the minimal cost.


Conditions on generators 305

A ⊂ B(E) × B(E) is a pre-generator if A is dissipative and there are sequences offunctions µn : E → P(E) and λn : E → [0,∞) such that for each (f, g) ∈ A

g(x) = limn→∞

λn(x)(

∫E

(f(y)− f(x))µn(x, dy)

for each x ∈ E.

A is bp-separable if there exists a countable subset gk ⊂ D(A)∩ C(E) such that thegraph of A is contained in the bounded, pointwise closure of (gk, Agk).

i) A : D(A) ⊂ C(E) → C(E × U), 1 ∈ D(A), and A1 = 0.

ii) There exist ψA ∈ C(E × U), ψA ≥ 1, and constants af , bf , f ∈ D(A), such that

|Af(x, u)| ≤ afψA(x, u), ∀(x, u) ∈ E × F.

iii) Defining A0 = (f, ψ−1A Af) : f ∈ D(A), A0 is bp-separable and for each

u ∈ F , the operator Au defined by Auf(x) = A0f(x, u) is a pre-generator.

iv) D(A) is closed under multiplication and separates points.


Existence of solutions of the martingale problem 306

Theorem 11.6 Suppose that A satisfies the condtions above. Let ν0 ∈ P(E) and π ∈P(E × F ) satisfy ∫

E×F

ψAdπ <∞

andπ(αf − Af) = αν0f, f ∈ D(A). (11.4)

Then there exists a solution of the controlled martingale problem for (A, ν0) such that

E[α

∫ ∞

0

e−αt

∫F

h(X(t), u)λt(du)dt] =

∫E×F

h(x, u)π(dx× du)


Existence of optimal solution 307

Theorem 11.7 Suppose that c ≥ 0 satisfies (x, u) : c(x, u) ≤ k is compact for every0 ≤ k < ∞ and that there exists ρ : [0,∞) → [0,∞) satisfying limr→∞ r−1ρ(r) = ∞such that

ρ(ψA(x, u)) ≤ c(x, u).

If there exists π ∈ P(E × F ) satisfying (11.4) and∫

E×Fcdπ < ∞, then there exists

π0 ∈ P(E×F ) satisfying (11.4) such that π0 achieves the minimum in the linear program.


Viscosity solutions 308


12. Miscellaneous

• Feynman-Kac formula

• Duality

• Stochastic equations for spin-flip models


Feynman-Kac formula 310

X a solution of the martingale problem for A

Mf (t) = f(X(t))− f(X(0))−∫ t

0

Af(X(s))ds

Then

f(X(t))e∫ t0 β(X(s))ds = f(X(0)) +

∫ t

0

e∫ s0 β(X(r))drdf(X(s))

+

∫ t

0

e∫ s0 β(X(r))drβ(X(s))f(X(s))ds

= f(X(0)) +

∫ t

0

e∫ s0 β(X(r))drdMf (s)

+

∫ t

0

e∫ s0 β(X(r))dr(β(X(s))f(X(s)) + Af(X(s)))ds


Corresponding semigroup 311

DefineT (t)f(x) = E[f(X(t))e

∫ t0 β(X(s))ds|X(0) = x]

T (t+ r)f(x) = E[f(X(t+ r))e∫ t+r0 β(X(s))ds|X(0) = x]

= E[E[f(X(t+ r))e∫ t+r0 β(X(s))ds|Ft]|X(0) = x]

= E[E[f(X(t+ r))e∫ t+r

t β(X(s))ds|Ft]e∫ t0 β(X(s))ds|X(0) = x]

= E[T (r)f(X(t))e∫ t0 β(X(s))ds|X(0) = x]

= T (t)T (r)f(x)

and

T (t)f(x) = f(x) +

∫ t

0

T (s)(βf + Af)(x)ds.


Contact process 312

E space of counting measures on a countable set S with ηx ∈ 0, 1.

λyx ≥ 0

Af(η) =∑y 6=x

λyx(1− ηx)ηy(f(η + δx)− f(η)) +∑

y

µyηy(f(η − δy)− f(η))

HΓ(η) = H(η,Γ) =∏

x∈Γ(1− ηx)

AHΓ(η) = −∑x∈Γ

∑y/∈Γ

λyx(1− ηx)ηyH(η,Γ) +∑y∈Γ

µyηyH(η,Γ− y)

= −∑x∈Γ

∑y/∈Γ

λyxηyH(η,Γ) +∑y∈Γ

µy(H(η,Γ− y)−H(η,Γ))

=∑x∈Γ

∑y/∈Γ

λyx(H(η,Γ ∪ y)−H(η,Γ)) +∑y∈Γ

µy(H(η,Γ− y)−H(η,Γ))

= BHη(Γ)


Duality 313

X and Y are independent, X has generator A, Y has generator B

d

dsE[H(X(s), Y (t− s))] = E[

d

dsT (s)S(t− s)H(X(0), Y (0))]

= E[T (s)S(t− s)AH(X(0), Y (0))]

−E[T (s)S(t− s)BH(X(0), Y (0))]

which is zero if AH(x, y) = BH(x, y). But then

E[H(X(t), Y (0)] = E[H(X(0), Y (t))]

In particular,

E[H(ηt,Γ0)] = E[H(η0,Γt)]

Pη(t,Γ0) = 0 = PΓt ⊂ y : η0y = 0

Existence of finite process implies uniqueness of infinite process.


Spin-flip models 314

A spin-flip model on Zd is a stochastic process whose state η = ηi : i ∈ Zd assignsto each lattice point i ∈ Zd the value ±1.

For each i ∈ Zd, specify a flip rate ci which determines the rate at which the associ-ated state variable ηi changes sign.

The rates ci may depend on the full configuration η.

Xi counts the cumulative number of sign changes

ηi(t) = ηi(0)(−1)Xi(t)−Xi(0).

The specification of the rates ci corresponds to the requirement that there exist afiltration Ft such that for each i ∈ Zd,

Mj(t) = Xi(t)−∫ t

0

ci(X(s))ds.

is an Ft-martingale. Assume no simultaneous jumps.


Generator for spin-flip model 315

Suppose f only depends on finitely many coordinates. Then

f(X(t)) = f(X(0)) +∑j∈Jf

∫ t

0

(f(X(s−) + ej)− f(X(s−)))dXj(s)

= f(X(0)) +∑j∈Jf

∫ t

0

(f(X(s−) + ej)− f(X(s−)))dMj(s)

+

∫ t

0

∑j∈Jf

cj(X(s))(f(X(s−) + ej)− f(X(s−)))ds


Stochastic equation for spin-flip model 316

Let ξi : i ∈ Zd be independent Poisson random measures on [0,∞)× [0,∞) withLebesgue mean measure.

Xi(t) = Xi(0) +

∫[0,∞)×[0,t]

1[0,ci(X(s−))](u)ξi(du× ds)

Let J = ki, i ∈ Zd : ki ∈ Z+. Assume

|ci(x+ ej)− ci(x)| ≤ aij |ci(x)| ≤ bi

for all x ∈ J and i, j ∈ Zd, where ej is the element of J such eji = 0 for i 6= j andejj = 1.


Conditions 318

Setting ρi(t) = E[|Xi(t)− Yi(t)|] and K = supi

∑j

αiaij

αj

supiαiρi(t) ≤ sup

iαiρi(0) +K

∫ t

0

supjαjρj(s)ds. (12.1)

Alternative, set K =∑

i supj

∑j

αiaij

αj. Then

∑i

αiρi(t) ≤∑

i

αiρi(0) +K

∫ t

0

∑j

αjρj(s)ds. (12.2)

Warning: To apply Gronwall to (12.1), need to know that sups≤t supj αjρj(s) <∞ (true if supj αjbj < ∞), and similarly for (12.2), that sups≤t

∑j αjρj(s) (true if∑

j αjbj <∞).


Example 319

Let aij = ρ(j − i), where ρ has bounded support, and bi ≡ b.

Take αi = 11+|i|d+1 so that

supi

∑j

(1 + |j|d+1)ρ(j − i)

1 + |i|d+1≤ sup

i

∑j

(1 + |j + i|d+1)ρ(j)

1 + |i|d+1

≤ supi

supk∈suppρ

(1 + |k + i|d+1)

1 + |i|d+1

∑j

ρ(j)


Exercises 320

1. Show that Fτ is a σ-algebra.

2. Show that for Ft-stopping times σ, τ , σ ≤ τ implies that Fσ ⊂ Fτ . In particular, Fτ∧t ⊂ Ft.

3. Let τ be a discrete Ft-stopping time satisfying τ < ∞ = ∪∞k=1τ = tk = Ω. Show that Fτ =σA ∩ τ = tk : A ∈ Ftk , k = 1, 2, . . ..

4. Let τ be a discrete stopping time with values ti. Show that

E[Z|Fτ ] =∑

i

E[Z|Fti ]1τ=ti.

5. Show that the minimum of two stopping times is a stopping time and that the maximum of two stoppingtimes is a stopping time.

6. Let N be a Poisson process with parameter λ. Then M(t) = N(t)− λt is a martingale. Compute [M ]t.

7. Show that if Y1 is cadlag and Y2 is finite variation, then

[Y1, Y2]t =∑s≤t

∆Y1(s)∆Y2(s).

8. Using the fact that martingales have finite quadratic variation, show that semimartingales have finitequadratic variation.

9. Using the above results, show that the covariation of two semimartingales exist.

10. Consider the stochastic differential equation

X(t) = X(0) +

∫ t

0

aX(s)dW (s) +

∫ t

0

bX(s)ds.


Find α and β so thatX(t) = X(0) expαW (t) + βt

is a solution.

11. Show that limn→∞ nPN( b−an

) > 1 = 0.

12. Let N be a stochasticly continuous counting processes with independent increments. Show that theincrements of N are Poisson distributed.

13. Verify the statements regarding the mappings on page 71.

14. Let Nn and N be counting processes. Suppose that (Nn(t1), . . . , Nn(tm)) ⇒ (N(t1), . . . , N(tm)) forall choices of t1, . . . , tm in T0, where T0 is dense in [0,∞). Show that Nn ⇒ N under the Skorohodtopology.

15. Let N be Poisson distributed with parameter λ, and let Y1, Y2, . . . be independent Bernoulli randomvariables with PYi = 1 = 1 − PYi = 0 = p. Define Z =

∑Ni=1 Yi. Compute the characteristic

function for Z.

16. Suppose the Y1, . . . , Ym are independent Poisson random variables with E[Yi] = λi. Let Y =∑m

i=1 Yi.Compute PYi = 1|Y = 1. More generally, compute PYi = k|Y = m.

17. Let ξ be a Poisson random measure on (U, dU ) with mean measure ν. For f ∈ M(U), f ≥ 0, show that

E[exp−∫

U

f(u)ξ(du)] = exp−∫

(1− e−f )dν.

18. Give an example of a right continuous process X such that∫

U×[0,t]|X(u, s)| ∧ 1ν(du)ds < ∞ a.s. but∫

U×[0,t]|X(u, s)|ξ(du× ds) = ∞ a.s.

19. Let Y , ξ1, and ξ2 be independent random variables with values in complete, separable metric spaces, E,S1, and S2. Suppose that G : E × S1 → E0 and H : E × S2 → E0 are Borel measurable functions and


that G(Y, ξ1) = H(Y, ξ2) a.s. Show that there exists a Borel measurable function F : E → E0 such thatF (Y ) = G(Y, ξ1) = H(Y, ξ2) a.s.


Appendix 1: Proof of Lemma 4.25 323

Proof. For D ∈ PU , define ν(D) = E[∫

U×[0,∞)ID(x, s)H(x, s)Γ(dx × ds)] and for C ∈ P , define ν0(C) =

ν(U × C). Since U is Polish, there exists a transition function γ0 from ([0,∞)× Ω,P) into U such that

ν(D) =

∫[0,∞)×Ω

γ0(s, ω, D(s,ω))ν0(ds× dω),

where D(s,ω) = x : (x, s, ω) ∈ D. In particular, for each G ∈ B(U), γ0(·, ·, G) is P-measurable. LetA0(t) =

∫U×[0,t]

H(x, s)Γ(dx× ds), and note that

ν0(C) = E[

∫[0,∞)×U

IC(s)H(x, s)Γ(dx× ds)] = E[

∫ ∞

0

IC(s)dA0(s)], C ∈ P,

soν(D) =

∫Ω

∫ ∞

0

γ0(s, ω, D(s,ω))dA0(s, ω)P (dω) = E[

∫ ∞

0

γ0(s, ·, D(s,·))dA0(s)],

for every D ∈ PU which implies

E[

∫U×[0,∞)

Z(x, s)H(x, s)Γ(dx× ds)] = E[

∫ ∞

0

∫U

Z(x, s)γ0(s, ·, dx)dA0(s)]

for every bounded, predictable Z.

There exists a nondecreasing, right-continuous, predictable process A, such that A0 − A is a martingale andhence

ν0(C) = E[

∫ t

0

IC(s)dA(s)], C ∈ P,


andν(D) = E[

∫ ∞

0

γ0(s, ω, D(s,ω))dA(s)].

For G ∈ B(U × [0,∞)), define

Γ(G, ω) =

∫ ∞

0

∫Gs

1

H(x, s)γ0(s, ω, dx)dA(s),

and observe that

E

[ ∫U×[0,t]

Z(x, s)H(x, s)Γ(dx× ds)

]= E[

∫ t

0

∫U

Z(x, s)γ0(s, dx)dA(s)]

= E[

∫ t

0

∫U

Z(x, s)γ0(s, dx)dA0(s)]

= E[

∫[0,t]×U

Z(x, s)H(x, s)Γ(dx× ds)].

Let t, r > 0, and let R be bounded andFt-measurable. Note that if Z is predictable, then Z(x, s) = RI(t,t+r](s)Z(x, s)is predictable. It follows that

E[(MZ(t + r)−MZ(t))R]

= E

[(E[

∫U×[0,t+r]

Z(x, s)H(x, s)Γ(dx× ds)|Ft+r]

−E[

∫U×[0,t]

Z(x, s)H(x, s)Γ(dx× ds)|Ft])R

]−E

[ ∫U×(t,t+r]

Z(x, s)H(x, s)Γ(dx× ds)R

]


= E

[( ∫U×[0,t+r]

Z(x, s)H(x, s)Γ(dx× ds)

−∫

U×[0,t]

Z(x, s)H(x, s)Γ(dx× ds))R

]−E

[ ∫U×(t,t+r]

Z(x, s)H(x, s)Γ(dx× ds)R

]= E

[( ∫U×(t,t+r]

RZ(x, s)H(x, s)Γ(dx× ds)

]−E

[ ∫U×(t,t+r]

RZ(x, s)H(x, s)Γ(dx× ds)

]= 0,

so MZ is a martingale.


GlossaryA directed set is a set A together with a binary relation ≤ having the following properties:

– (reflexivity) a ≤ a for all a in A.

– (transitivity) If a ≤ b and b ≤ c, then a ≤ c.

– (directedness) For a, b ∈ A, there exists a c ∈ A with a ≤ c and b ≤ c.


References

[1] D. L. Burkholder. Distribution function inequalities for martingales. Ann. Probability, 1:19–42, 1973.

[2] Stewart N. Ethier and Thomas G. Kurtz. Markov processes. Wiley Series in Probability and MathematicalStatistics: Probability and Mathematical Statistics. John Wiley & Sons Inc., New York, 1986. Characteri-zation and convergence.

[3] Akira Ichikawa. Filtering and control of stochastic differential equations with unbounded coefficients.Stochastic Anal. Appl., 4(2):187–212, 1986.

[4] Jean Jacod and Albert N. Shiryaev. Limit theorems for stochastic processes, volume 288 of Grundlehren derMathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences]. Springer-Verlag, Berlin,1987.

[5] A. Jakubowski, J. Memin, and G. Pages. Convergence en loi des suites d’integrales stochastiques surl’espace D1 de Skorokhod. Probab. Theory Related Fields, 81(1):111–137, 1989.

[6] Thomas G. Kurtz. The optional sampling theorem for martingales indexed by directed sets. Ann. Probab.,8(4):675–681, 1980.

[7] Thomas G. Kurtz and Philip Protter. Weak limit theorems for stochastic integrals and stochastic differen-tial equations. Ann. Probab., 19(3):1035–1070, 1991.

[8] Thomas G. Kurtz and Philip E. Protter. Weak convergence of stochastic integrals and differential equa-tions. II. Infinite-dimensional case. In Probabilistic models for nonlinear partial differential equations (Monte-catini Terme, 1995), volume 1627 of Lecture Notes in Math., pages 197–285. Springer, Berlin, 1996.

[9] E. Lenglart, D. Lepingle, and M. Pratelli. Presentation unifiee de certaines inegalites de la theorie desmartingales. In Seminar on Probability, XIV (Paris, 1978/1979) (French), volume 784 of Lecture Notes inMath., pages 26–52. Springer, Berlin, 1980. With an appendix by Lenglart.


[10] Philip E. Protter. Stochastic integration and differential equations, volume 21 of Applications of Mathematics(New York). Springer-Verlag, Berlin, second edition, 2004. Stochastic Modelling and Applied Probability.

Martingale

Documents

Transcript of Martingale