Cl ement Foucart - arxiv.org

20
ASYMPTOTIC BEHAVIOUR OF ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS Cl´ ement Foucart 1 and Martin M¨ ohle 2 Abstract. Consider the population model with infinite size associated to subcritical continuous-state branching processes (CSBP). Individuals reproduce independently ac- cording to the same subcritical offspring distribution. We study the long-term behaviour of the ancestral lineages as time goes to the past and show that the flow of ancestral lineages, properly renormalized, converges almost surely to the inverse of a drift-free subordinator whose Laplace exponent is explicit in terms of the branching mechanism. We provide an interpretation in terms of the genealogy of the population. In particu- lar, we show that the inverse subordinator is partitioning the current population into ancestral families with distinct common ancestors. When Grey’s condition is satisfied, the population comes from a discrete set of ancestors and the ancestral families are i.i.d and distributed according to the quasi-stationary distribution of the CSBP conditioned on non-extinction. When Grey’s condition is not satisfied, the population comes from a continuum of ancestors which is described as the set of increase points S of the lim- iting inverse subordinator. The Hausdorff dimension of S is given. The proof is based on a general result for stochastically monotone processes of independent interest, which relates θ-invariant measures and θ-invariant functions for a process and its Siegmund dual. 1. Introduction Continuous-state branching processes (CSBPs) are positive Markov processes satisfying the branching property. They arise as scaling limits of Galton-Watson processes and form a fundamental class of random population models. Their longterm behavior has received a great deal of attention since the seventies. We refer to the early works of Bingham [Bin76] and Grey [Gre74]. In the seminal work [BLG00], Bertoin and Le Gall showed how to encode a complete genealogy of a random branching population by considering a flow of subordinators (X s,t (x),s t, x 0). In this representation the population has an infinite size at all times and all individuals have arbitrarily old ancestors. More precisely, individuals at time s with descendants at time t are the locations of the jumps of the subordinator x 7X s,t (x) and the descendants at time t of the individuals in the population at time s are represented by the jump intervals. We shall provide more background on the flow of subordinators in the sequel. Date : December 7, 2020. 2010 Mathematics Subject Classification. 60J80, 60J70, 92D25. Key words and phrases. branching processes, continuous-state space, inverse subordinators, ancestral lineage, almost sure asymptotics, Siegmund dual, invariant function, invariant measure, stochastic flows. 1 Universit´ e Sorbonne Paris Nord and Paris 8, Laboratoire Analyse, G´ eom´ etrie & Applications, UMR 7539. Institut Galil´ ee, 99 avenue J.B. Cl´ ement, 93430 Villetaneuse, France; [email protected] 2 Eberhard Karls Universit¨ at ubingen, Auf der Morgenstelle 10, 72076 ubingen, Germany; [email protected]. 1 arXiv:2012.02857v1 [math.PR] 4 Dec 2020

Transcript of Cl ement Foucart - arxiv.org

Page 1: Cl ement Foucart - arxiv.org

ASYMPTOTIC BEHAVIOUR OF ANCESTRAL LINEAGES INSUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

Clement Foucart 1 and Martin Mohle 2

Abstract. Consider the population model with infinite size associated to subcriticalcontinuous-state branching processes (CSBP). Individuals reproduce independently ac-cording to the same subcritical offspring distribution. We study the long-term behaviourof the ancestral lineages as time goes to the past and show that the flow of ancestrallineages, properly renormalized, converges almost surely to the inverse of a drift-freesubordinator whose Laplace exponent is explicit in terms of the branching mechanism.We provide an interpretation in terms of the genealogy of the population. In particu-lar, we show that the inverse subordinator is partitioning the current population intoancestral families with distinct common ancestors. When Grey’s condition is satisfied,the population comes from a discrete set of ancestors and the ancestral families are i.i.dand distributed according to the quasi-stationary distribution of the CSBP conditionedon non-extinction. When Grey’s condition is not satisfied, the population comes froma continuum of ancestors which is described as the set of increase points S of the lim-iting inverse subordinator. The Hausdorff dimension of S is given. The proof is basedon a general result for stochastically monotone processes of independent interest, whichrelates θ-invariant measures and θ-invariant functions for a process and its Siegmunddual.

1. Introduction

Continuous-state branching processes (CSBPs) are positive Markov processes satisfyingthe branching property. They arise as scaling limits of Galton-Watson processes and forma fundamental class of random population models. Their longterm behavior has receiveda great deal of attention since the seventies. We refer to the early works of Bingham[Bin76] and Grey [Gre74]. In the seminal work [BLG00], Bertoin and Le Gall showed howto encode a complete genealogy of a random branching population by considering a flowof subordinators (Xs,t(x), s ≤ t, x ≥ 0).

In this representation the population has an infinite size at all times and all individualshave arbitrarily old ancestors. More precisely, individuals at time s with descendants attime t are the locations of the jumps of the subordinator x 7→ Xs,t(x) and the descendantsat time t of the individuals in the population at time s are represented by the jumpintervals. We shall provide more background on the flow of subordinators in the sequel.

Date: December 7, 2020.2010 Mathematics Subject Classification. 60J80, 60J70, 92D25.Key words and phrases. branching processes, continuous-state space, inverse subordinators, ancestral

lineage, almost sure asymptotics, Siegmund dual, invariant function, invariant measure, stochastic flows.1Universite Sorbonne Paris Nord and Paris 8, Laboratoire Analyse, Geometrie & Applications, UMR

7539. Institut Galilee, 99 avenue J.B. Clement, 93430 Villetaneuse, France; [email protected] Karls Universitat Tubingen, Auf der Morgenstelle 10, 72076 Tubingen, Germany;

[email protected]

arX

iv:2

012.

0285

7v1

[m

ath.

PR]

4 D

ec 2

020

Page 2: Cl ement Foucart - arxiv.org

2 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

Most works on CSBPs focus on their long-term behaviour forward in time. We referto Bertoin et al. [BFM08], Duquesne and Labbe [DL14], Labbe [Lab14] and Foucartand Ma [FM19] for studies in the framework of the flow of subordinators. However, therepresentation of the population model through (Xs,t(x), s ≤ t, x ≥ 0) allows one to followthe ancestral lineages backward in time. In this article, we are interested in the backwardgenealogy of the continuous population and how it behaves on the long-term. To thebest of our knowledge, fewer works on CSBPs have been done in this direction. We referhowever to Labbe [Lab14], Lambert [Lam03], Lambert and Popovic [LP13] and Foucart et

al. [FMM19]. The latter work initiates the study of the inverse flow (Xs,t(x), s ≤ t, x ≥ 0)defined for s ≤ t and x ∈ [0,∞], as

(1.1) Xs,t(x) := infy ≥ 0 : X−t,−s(y) > x.

This random variable represents the ancestor at time −t of the individual x in the popu-lation at time −s. From now on consider an arrow of time pointing to the past, and callXt(x) := X0,t(x), the ancestor at time t ≥ 0 (backwards) of the individual x of the pop-

ulation at time 0. The two-parameter flow (Xt(x), t ≥ 0, x ≥ 0) is therefore representing

the ancestral lineages of the individuals in the current population. We call (Xt(x), t ≥ 0)the ancestral lineage process. This is a Feller process with no positive jumps, namelyP(

supt>0(Xt(x) − Xt−(x)) > 0)

= 0. Moreover, for any x 6= y, whenever (Xt(x), t ≥ 0)

and (Xt(y), t ≥ 0) cross, they coalesce and such a coalescence represents the occurrencein the past of a common ancestor of the individuals x and y. We stress that coales-cence can be multiple in the sense that more than two lineages can coalesce at the sametime. We refer to [FMM19] for a study of the coalescent processes embedded in the flow

(Xt(x), t ≥ 0, x ≥ 0).

We focus on subcritical CSBPs. In such a setting, it is known, see [FMM19, Proposition

2.8], that for all x ∈ (0,∞), the Markov process (Xt(x), t ≥ 0) is transient. We shall find

an almost sure renormalisation of (Xt(x), t ≥ 0) as t goes to ∞ and study the limitingprocess in the variable x. This leads to an almost sure description of the long-termbehaviour of the ancestral lineages for all subcritical CSBPs, including those not satisfyingGrey’s condition, see Section 2 and Theorem 3.1.

The paper is organised as follows. Further background on CSBPs and their representa-tion in terms of flow of subordinators are provided in Section 2. Fundamental propertiesof the ancestral lineage process (Xt(x), t ≥ 0), such as its Siegmund duality relation withthe CSBP (Xt(x), t ≥ 0) and the representation of its semigroup, are also recalled. Ourmain results are stated in Section 3 and proven in Section 4. The proof is based on ageneral result, established in Theorem 4.1, for stochastically monotone Markov processesby showing how to link (infinite) θ-invariant measures of a process (Xt, t ≥ 0) with (in-

creasing) θ-invariant functions of its Siegmund dual process (Xt, t ≥ 0). We apply thisresult in the setting of CSBPs.

2. Background on CSBPs and the flow of subordinators

We first recall basic definitions and properties of CSBPs and their representation interms of flows. These processes are continuous time and continuous space analogue ofGalton-Watson Markov chains. They have been introduced by Lamperti [Lam67] and

Page 3: Cl ement Foucart - arxiv.org

ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS 3

Jirina [Jir58]. CSBPs are positive Markov processes satisfying the branching property:for any x, y ≥ 0 and fixed time t ≥ 0,

(2.2) Xt(x+ y) = X ′t(x) +X ′′t (y),

where (Xt(x+y), t ≥ 0) is a CSBP started from x+y, and (X ′t(x), t ≥ 0) and (X ′′t (y), t ≥ 0)are two independent copies of the process started respectively from x and y. We refer thereader to [Li11, Chapter 3] for an introduction to CSBPs. Denote by L the generator of(Xt(x), t ≥ 0). For any q ≥ 0, set eq(x) := e−qx for any x ≥ 0. The operator L acts onthe exponential functions as follows. For any q ≥ 0, and x ∈ [0,∞)

(2.3) Leq(x) = Ψ(q)xeq(x),

where Ψ is a Levy-Khintchine function and is called the branching mechanism. We refere.g. to Silverstein [Sil68]. The linear span of exponential functions A := Span(eq, q ∈[0,∞)) is a core for generator L.

We shall merely be interested in subcritical CSBPs for which Ψ is of the form

(2.4) Ψ(u) =σ2

2u2 + γu+

∫ ∞0

(e−ux − 1 + ux)π(dx) for all u ≥ 0,

where γ = Ψ′(0+) > 0, σ ≥ 0, and π is a Levy measure, i.e. a Borel measure such that∫∞0

(x ∧ x2)π(dx) < ∞. We assume that either π 6= 0 or σ > 0, so that Ψ is not linear.The semigroup of (Xt(x), t ≥ 0) satisfies for any λ ∈ (0,∞), t ≥ 0 and x ∈ [0,∞)

(2.5) E[e−λXt(x)] = e−xvt(λ)

with t 7→ vt(λ) for any λ ∈ (0,∞), as the solution to the integral equation

(2.6)

∫ λ

vt(λ)

du

Ψ(u)= t.

Note that t 7→ vt(λ) solves ddtvt(λ) = −Ψ(vt(λ)) with v0(λ) = λ. As a first consequence

of (2.5), the process (Xt(x), t ≥ 0) gets extinct at time t with probability e−xvt(∞) wherevt(∞) := lim

λ→∞vt(λ) ∈ (0,∞]. The latter is finite if and only if Ψ satisfies Grey’s condition

(2.7)

∫ ∞ du

Ψ(u)<∞.

Lambert [Lam07] and Li [Li00] have studied the subcritical process (Xt(x), t ≥ 0)conditioned on non-extinction and established the following weak convergence when (2.7)holds:

P(Xt(x) ∈ · |Xt(x) > 0) −→t→∞

ν∞(·),

where ν∞, the so-called quasi-stationary distribution of the CSBP, has Laplace transform

(2.8)

∫ ∞0

e−qxν∞(dx) = 1− κ∞(q) := 1− e−Ψ′(0+)∫∞q

duΨ(u) , q ≥ 0.

The branching property (2.2) can be translated in terms of independence and stationar-ity of the increments of the process (Xt(x), x ≥ 0) for any fixed time t. The latter is there-fore a subordinator and according to (2.5), its Laplace exponent is λ 7→ vt(λ). Startingfrom this observation, Bertoin and Le Gall in [BLG00] showed that a complete populationmodel can be associated to CSBPs through a flow of subordinators (Xs,t(x), s ≤ t, x ≥ 0).

Page 4: Cl ement Foucart - arxiv.org

4 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

More precisely, the collection of processes (Xs,t(x), s ≤ t, x ≥ 0) is satisfying the fol-lowing properties:

(1) For every s ≤ t, x 7→ Xs,t(x) is a cadlag subordinator with Laplace exponentλ 7→ vt−s(λ).

(2) For every t ∈ R, (Xr,s, r ≤ s ≤ t) and (Xr,s, t ≤ r ≤ s) are independent.(3) For every r ≤ s ≤ t, Xr,t = Xs,t Xr,s.

The two-parameter flow (Xt(x), x ≥ 0, t ≥ 0) := (X0,t(x), x ≥ 0, t ≥ 0) is a flow of CSBPswith branching mechanism Ψ, each starting from an initial population of arbitrarily largesize x. The three-parameter flow above provides a complete genealogy of the underlying(infinite) population: let y ∈ (0,∞), if Xs,t(y−) < Xs,t(y), then the individual y at times has descendants at time t and those are represented by the interval (Xs,t(y−), Xs,t(y)];see Figure 1.

y

Xs,t(y−)

Xs,t(y)

x 7→ Xs,t(x)

Figure 1. Symbolic representation of the genealogy forward in time

We work now on the probability space on which the flow (Xs,t(x), s ≤ t, x ≥ 0) isdefined. Recall the definition (1.1) of the inverse flow and set

(Xt(x), t ≥ 0) := (X0,t(x), t ≥ 0).

We summarize here fundamental properties of the ancestral lineage process (Xt, t ≥ 0),see [FMM19, Section 3]. We stress that here, apart if this is explicitely mentioned,the branching mechanism Ψ is general and not assumed to be subcritical. The process(Xt(x), t ≥ 0), started from x > 0, is a non-explosive cadlag Feller process with no positivejumps. Property (3) of the flow of subordinators (Xs,t(x), s ≤ t, x ≥ 0) entails that for allr ≤ s ≤ t,

Xr,t = Xs,t Xr,s a.s.

This entails that the flow of processes(Xt(x), t ≥ 0, x ∈ (0,∞)

)is coalescing, in the

sense that, when two ancestral lineages (Xt(x), t ≥ 0) and (Xt(y), t ≥ 0) cross, theymerge. Such a coalescence represents the occurrence in the past of a common ancestorof the individuals x and y. We refer the reader to [FMM19, Section 5.2] for a study of

coalescent processes embedded in the flow (Xs,t, s ≤ t) .

Page 5: Cl ement Foucart - arxiv.org

ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS 5

The processes X and X are linked through the following duality relations, see [FMM19,Lemma 3.3-(1) and Equation (3.5), Section 3]. For any t ≥ 0 and x, y ∈ (0,∞)

(2.9) Xt(x) < y = x < X−t,0(y) almost surely.

In particular, since for any t ≥ 0, X−t,0(y) has the same law as Xt(y),

(2.10) P(Xt(x) < y

)= P

(x < Xt(y)

).

The following other duality relations will be more convenient to work with. It also allowsone to identify the Markov process (Xt(x), t ≥ 0) as the Siegmund dual of (Xt(x), t ≥ 0),see the forthcoming Section 4.1.

Lemma 2.1. For any t ≥ 0 and x, y ∈ (0,∞)

(2.11) Xt(x) > y = x > X−t,0(y) almost surely,

and

(2.12) P(Xt(x) > y

)= P

(x > Xt(y)

).

Proof. Let t ≥ 0 and x, y ∈ (0,∞). We establish that Xt(x) ≤ y = x ≤ Xt(y)almost surely. Note that this is equivalent to (2.11). According to (2.9), Xt(x) < y =

x < Xt(y) almost surely, therefore we only need to focus on the events Xt(x) = yand Xt(y) = x. Recall the definition of Xt(x) in (1.1),

Xt(x) = y = X−t,0(y−) ≤ x < X−t,0(y) ∪ X−t,0(y−) = X−t,0(y) = x.Since X−t,0 is a subordinator, it has no almost sure fixed discontinuities and the eventX−t,0(y−) < X−t,0(y) for fixed y has probability 0. Thus

Xt(x) = y) = X−t,0(y) = x) a.s,

and (2.11) is established. Since X−t,0(y) has the same law as Xt(y),

P(Xt(x) = y) = P(X−t,0(y) = x) = P(Xt(y) = x),

and the identity (2.12) holds.

The next theorem characterizes the semigroup of (Xt, t ≥ 0).

Theorem 2.2 (Theorem 3.5, Proposition 3.6 in [FMM19]). For any continuous functionf defined on (0,∞) and any q > 0,

(2.13) E[f(Xt(eq))] = E[f(evt(q))],

where for any λ ∈ (0,∞), eλ is an exponential random variable with parameter λ.

The process admits an entrance boundary at 0+ if and only if (2.7) is satisfied.

When (2.7) is satisfied, we denote by (Xt(0+), t ≥ 0) the process started from 0, defined

at any time, as Xt(0+) := limx↓0

Xt(x) a.s. This corresponds to the first individual at time

t with descendants at time 0.

We complete these fundamental results by showing that X satisfies some properties ofregularity. For x, y ∈ (0,∞), set

(2.14) Ty := inft > 0 : Xt(x) > y = inft > 0 : Xt(x) = y.

Page 6: Cl ement Foucart - arxiv.org

6 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

We shall sometimes write Ty = T xy to emphasize on the initial state of the process.

Lemma 2.3 (Regularity). If −Ψ is not the Laplace exponent of a subordinator, then theprocess is regular on (0,∞), namely for any x < y,

Px(Ty <∞) > 0.

Proof. Let x, y ∈ (0,∞), for any t > 0, Px(Ty < t) ≥ P(Xt(x) > y

)= P

(x > Xt(y)

).

By assumption, −Ψ is not the Laplace exponent of a subordinator, this ensures that theCSBP is not almost surely non-decreasing and that the event Xt(y) −→

t→∞0 has positive

probability. Hence, Px(Ty <∞) ≥ limt→∞

P(x > Xt(y)

)≥ P(Xt(y) −→

t→∞0) > 0. 2

In the subcritical case, for which Ψ′(0+) = γ > 0, one has limt→∞

vt(λ) = 0 and all families

forward in time are getting extinct. As mentioned in the introduction, in this case theancestral lineages are transient.

Proposition 2.4 (Proposition 3.8 in [FMM19]). Assume Ψ′(0+) > 0. For any x ∈(0,∞), the ancestral lineage process (Xt(x), t ≥ 0) is transient, i.e Xt(x) −→

t→∞∞ a.s.

The main aim of this article is to obtain an almost sure renormalisation of the inverseflow and to interpret it in terms of the genealogy. Siegmund dual processes of discretebranching Markov processes have been studied by Pakes et al. [LPLG08] and Pakes[Pak17]. A striking difference for continuous-state space processes is that when Grey’scondition (2.7) does not hold, the latter are persistent in the sense that although sub-critical, they are not getting absorbed at 0, but are decreasing towards 0 while keepingpositive mass at all times.

3. Results

Assume Ψ′(0+) > 0. For any λ ∈ (0,∞), define the map

(3.15) κλ : θ 7→ e−Ψ′(0+)∫ λθ

duΨ(u) .

We shall see that the function κλ is the Laplace exponent of a drift-free subordinator. Wedenote its Levy measure by νλ. The latter is finite if and only if

∫∞ duΨ(u)

< ∞ (Grey’s

condition). In this case, the function κ∞ defined in (2.8), is the Laplace exponent ofcompound Poisson process with jump law ν∞, the quasi-stationary distribution of theΨ-CSBP conditioned on non-extinction.

Our main result is the following.

Theorem 3.1. Assume Ψ′(0+) > 0. Fix λ ∈ (0,∞). Then, almost surely

vt(λ)Xt(x) −→t→∞

W λ(x), for all x /∈ Jλ := x > 0 : W λ(x+) > W λ(x),

where (W λ(x), x ≥ 0) is a process with non-decreasing left-continuous sample paths andits right-continuous inverse process (W λ(y), y ≥ 0), defined for any y ≥ 0, by W λ(y) :=

infx ≥ 0 : W λ(x) > y, is a drift-free subordinator with Laplace exponent κλ.

i) If∫∞ du

Ψ(u)=∞, then for any λ ∈ (0,∞) the process (W λ(x), x ≥ 0) has continu-

ous sample paths almost surely.

Page 7: Cl ement Foucart - arxiv.org

ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS 7

ii) If∫∞ du

Ψ(u)<∞, then for any λ ∈ (0,∞) the process (W λ(x), x ≥ 0) has piecewise

constant sample paths almost surely, and

vt(∞)Xt(x) −→t→∞

W∞(x) for all x /∈ J∞ almost surely,

where (W∞(x), x ≥ 0) is the inverse of a compound Poisson process with Laplaceexponent κ∞.

The next observation ensures that the choice of the parameter λ is arbitrary. A changein λ only affects the limit by a multiplicative factor.

Lemma 3.2. For any λ′ 6= λ ∈ (0,∞) and x ∈ (0,∞)

W λ′(x) = cλ′,λWλ(x) almost surely, with cλ′,λ = eΨ′(0+)

∫ λ′λ

duΨ(u) .

Recall π the Levy measure in the Levy-Khintchine form (2.4) of Ψ. The following

corollary shows that the ancestral lineage process (Xt(x), t ≥ 0) has an exponential growthwhen the measure π satisfies an L logL condition.

Corollary 3.3. For any λ > 0, vt(λ) ∼t→∞

cλe−Ψ′(0+)t for some constant cλ > 0 if and

only if∫∞

1u log uπ(du) <∞. Moreover, under this latter condition, almost surely

e−Ψ′(0+)tXt(x) −→t→∞

W (x), for all x /∈ J,

where J := x > 0 : W (x+) > W (x) and W is the inverse of a subordinator W withLaplace exponent

κ : θ ∈ [0,∞) 7→ θe−Ψ′(0+)

∫ θ0

(1

Ψ′(0+)u− 1

Ψ(u)

)du.

Example 3.4. Let γ > 0. Consider the subcritical Neveu CSBP whose branching mech-anism is defined by Ψ(u) := γ(u + 1) log(u + 1) for all u ≥ 0. Note that Ψ′(0+) = γ > 0and

∫∞ duΨ(u)

=∞. Solving (2.6) yields

vt(λ) = (λ+ 1)e−γt − 1 ∼

t→∞log(1 + λ)e−γt.

By Corollary 3.3, almost surely

e−γtXt(x) −→t→∞

W (x) for all x ≥ 0,

where W is the inverse of a subordinator W with Laplace exponent

κ(θ) = γ log(1 + θ) =

∫ ∞0

(1− e−θx)ν(dx),

with ν(dx) := γ e−x

xdx. The limiting process W is therefore an inverse Gamma subordina-

tor.

The process (W λ(x), x ≥ 0) can be interpreted as follows. Define a random equivalencerelation A on (0,∞) via

xA∼ y if and only if W λ(x) = W λ(y).

Page 8: Cl ement Foucart - arxiv.org

8 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

This induces a random partition of the set (0,∞) into intervals of constancy of W λ. Asimple application of Lemma 3.2 ensures that this partition does not depend on λ. Bydefinition, the subintervals of the partition A are made of individuals whose ancestrallineages have the same asymptotic behaviour. The next proposition states that A corre-sponds actually to the families of current individuals having a common ancestor.

Proposition 3.5. For any x, y ∈ (0,∞),

xA∼ y if and only if Xt(x) = Xt(y) for some t ≥ 0.

Theorem 3.1 and Proposition 3.5 complete the results obtained under Grey’s condition,in Foucart et al. [FMM19, Sections 4 and 5.3], on the long-term behavior of the ancestral

lineages. When∫∞ du

Ψ(u)<∞ (Grey’s condition), the process (W∞(x), x ≥ 0) is the left-

continuous inverse of a compound Poisson process whose jump law is ν∞. Therefore ithas piecewise constant sample paths with intervals of constancy of i.i.d lengths with lawν∞. The partition A is thus constituted of i.i.d. families with lengths of law ν∞, i.e ofthe form

A =((0, x1], (x1, x2], . . .

)a.s.,

where (xi, i ≥ 1) is a random renewal process with jump law ν∞. Set x0 = 0. Theancestral lineage of a given family (xi−1, xi], for i ≥ 1, escapes towards ∞ at speed

t 7→ W∞(xi)

vt(∞).

Conditionally on xi, we have that W∞(xi) = e1 + . . . + ei, where the random variablese1, e2, ... are independent and standard exponential. The following figure provides a sym-bolic representation of the families, their lineages and the process W∞, under Grey’scondition.

Xt(x1) ∼ W∞(x1)/vt(∞)

time 0

x1

x2

Xt(x3) ∼ W∞(x3)/vt(∞)

Xt(x2) ∼ W∞(x2)/vt(∞)

x3

x1

W∞(x1)

x2

W∞(x2)

W∞(x3)

x3

Figure 2. Symbolic representation of ancestral families under Grey’s condition

Coalescences between the ancestral lineages inside each family are possibly multipleand can be described using the notion of consecutive coalescents, see [FMM19, Section 5].

Conditionally given A , for any j > i, when W∞(xj) − W∞(xj−1) = ej > W∞(xi) −W∞(xi−1) = ei, the ancestor of the family (xj−1, xj] is found asymptotically earlier inthe past than that of (xi−1, xi]. We may thus interpret the exponential random variables(ei, i ≥ 1) as measures of ages of the families in A . In the figure above, the divergence

Page 9: Cl ement Foucart - arxiv.org

ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS 9

towards ∞ of the ancestral lineage of (x2, x3] is quicker than that of (x1, x2], namelye3 > e2, and the family (x1, x2] is older than (x2, x3].

When∫∞ du

Ψ(u)=∞, the description is more involved since the process (W λ(x), x ≥ 0)

has singular continuous paths and any fixed subinterval of (0,∞) of finite length containsinfinitely many small families with positive probability. More precisely, the ancestralfamilies are separated by points xi, i ∈ I, in the support S of the associated singularStieltjes measure dW λ.

Proposition 3.6. Set Ψ′(∞) := limu→∞

Ψ(u)u∈ (0,∞]. For any x > 0, the Hausdorff

dimension of S ∩ [0, x] is

(3.16) dimH(S ∩ [0, x]) =Ψ′(0+)

Ψ′(∞)∈ [0, 1) a.s.

From (2.4), one sees that Ψ′(∞) = σ2

2· ∞ + γ +

∫∞0xπ(dx) ∈ (0,∞]. In the case

of a CSBP with unbounded variation, namely with σ > 0 or∫ 1

0xπ(dx) = ∞, one has

Ψ′(∞) = ∞ and the Hausdorff dimension of S is zero. In the bounded variation case,(3.16) can be rewritten as

dimH(S ∩ [0, x]) =γ

γ +∫∞

0xπ(dx)

a.s.

Example 3.7. Let Ψ be the branching mechanism with drift γ = 1 and Levy measureπ(dx) = x−α−1e−xdx with α ∈ (0, 2).

i) If α ∈ (1, 2), then Grey’s condition (2.7) holds, Ψ′(∞) = ∞, and the ancestralfamilies A are separated along a discrete set S and dimH(S ∩ [0, x]) = 0 a.s.

ii) If α = 1 (Neveu case), then Grey’s condition (2.7) does not hold, Ψ′(∞) =∞, andthe ancestral families A are separated along the set S and dimH(S ∩ [0, x]) = 0a.s.

iii) If α < 1, then Grey’s condition (2.7) does not hold, Ψ′(∞) <∞, and the ancestralfamilies A are separated along the set S and dimH(S ∩ [0, x]) = 1

1+Γ(1−α)a.s.

In general, inverse subordinators do not have the Markov property. The joint density ofthe finite-dimensional marginals of (W λ(x), x ≥ 0) are thus rather involved. We refer tothe works of Lageras [Lag05] and Veillette and Taqqu [VT10] for information on inversesubordinators. The following proposition is a side result on the one-dimensional lawsof the limiting process (W λ(x), x ≥ 0). Recall that νλ denotes the Levy measure ofthe subordinator (W λ(x), x ≥ 0). Set νλ the tail of the measure νλ: for any x ≥ 0,νλ(x) := νλ

((x,∞)

).

Proposition 3.8. The law of W λ(x) admits the density gλx defined on (0,∞) by

(3.17) gλx(u) :=

∫ x

0

νλ(x− z)P(W λ(u) ∈ dz).

When∫∞ du

Ψ(u)<∞, W∞(x) has density g∞x :

(3.18) g∞x (u) := e−u∞∑n=0

un

n!

∫ x

0

ν∞(x− z)ν?n∞ (dz).

Page 10: Cl ement Foucart - arxiv.org

10 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

4. Proofs

We first establish a general result of independent interest relating θ-invariant functionsof stochastically monotone processes, with θ-invariant measures of their dual processes,see Section 4.1. We then apply this result in our setting. The asymptotics of a certainθ-invariant function for the dual process X is studied. It enables to show an almost sureconvergence for the process (Xt(x), t ≥ 0) started from a fixed value x, see Lemma 4.11.We study the associated limiting process in x and show that it satisfies the propertiesstated in Theorem 3.1, see Lemma 4.13. We will see from the proof of Lemma 4.11and Lemma 4.12 that Proposition 3.5 holds true. Proposition 3.6 will be a consequenceof Theorem 3.1. The proof of Theorem 3.1 is divided into several lemmas. The firstare needed to show the almost sure convergence towards some positive random variableW λ(x). Finally, we identify the law of the process (W (x), x ≥ 0).

4.1. Invariant functions of stochastically monotone Markov processes. In thissection, we consider a “general” standard Markov process X := (Xt, t ≥ 0) with statespace [0,∞), and denote by (Xt(y), t ≥ 0) the process started from y ∈ [0,∞). Recallthat the process X is said to be stochastically monotone if for any t ≥ 0 and x ∈ [0,∞),the map y 7→ P(Xt(y) ≥ x) is non-decreasing. Siegmund [Sie76] has established that ifthe process X is stochastically monotone, non-explosive or with boundary ∞ absorbing,and that for any fixed t and z, the map y 7→ P(Xt(y) ≥ z) is right-continuous then thereexists a Markov process X, the so-called Siegmund dual process, such that for any t andx, y

(4.19) P(Xt(y) ≥ x) = P(Xt(x) ≤ y).

The latter identity can be rewritten as

(4.20) P(Xt(x) > y

)= P

(x > Xt(y)

).

Our first result shows how to find fundamental martingales for the Siegmund dual process(Xt(x), t ≥ 0) of any stochastically monotone Markov process (Xt(x), t ≥ 0). Recall Tydefined in (2.14).

Theorem 4.1 (Invariant functions of X). Let (Pt, t ≥ 0) be the semigroup of the process(Xt, t ≥ 0). Let θ ∈ R. If µθ is a positive Borel measure on (0,∞) satisfying for anyt ≥ 0, µθPt = eθtµθ, then the functions x 7→ µθ([0, x)) and x 7→ µθ((x,∞)), provided theyare well-defined, are θ-invariant functions, namely functions fθ such that for any t ≥ 0and x ∈ [0,∞),

E[fθ(Xt(x))] = eθtfθ(x),

so that

(4.21)(e−θtfθ(Xt(x)), t ≥ 0

)is a martingale.

In particular, if the process (Xt, t ≥ 0) has no positive jumps and µθ is finite on [0, x)for all x > 0, then fθ : x 7→ µθ([0, x)) is a well-defined increasing and left-continuousfunction, and for all y ≥ x ≥ 0,

(4.22) Ex[e−θTy ] =µθ([0, x))

µθ([0, y)).

Page 11: Cl ement Foucart - arxiv.org

ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS 11

Remark 4.2. A measure µθ verifying µθPt = θµθ is sometimes called an eigen-measureor a θ-invariant measure. If θ < 0 and µθ is a probability measure, then µθ is a quasi-stationary distribution whose rate of decay is θ. Indeed, let T be a killing time, typicallyT = inft > 0 : Xt = 0, then for any t > 0, µθPt = θµθ is equivalent to Pµθ(Xt ∈ ·|T >t) = µθ(·) and Pµθ(T > t) = e−θt.

Remark 4.3. Let L be the generator of the process (Xt, t ≥ 0). The Kolmogorov forwardequation entails that the condition µθPt = eθtµθ for all t ≥ 0, is equivalent to µθL = θµθwhere µθL is by definition the measure such that 〈µθL, f〉 =

∫Lf(x)µθ(dx) for any

function f ∈ C2b ((0,∞)).

Proof. Set fθ(x) = µθ([0, x)) for all x > 0. For any x ∈ (0,∞) and any t ≥ 0,

Ptfθ(x) = E[fθ(Xt(x)

)]= E

[∫1Xt(x)>yµθ(dy)

]= E

[∫1x>Xt(y)µθ(dy)

]by the duality relation (4.20)

= µθPt([0, x)

)= eθtµθ

([0, x)

)= eθtfθ(x).

The martingale property (4.21) follows readily from the Markov property. Note that themap fθ : x 7→ µθ([0, x)) is left-continuous. We now apply the bounded optional stopping

time theorem at time t ∧ Ty:

E[e−θt∧Tyfθ

(Xt∧Ty(x)

)]= fθ(x).

Since fθ is non-decreasing and Xt∧Ty(x) ≤ y a.s, one has for any t ≥ 0, fθ(Xt∧Ty(x)

)≤

fθ(y). On the event Ty <∞, the left-continuity of fθ and the absence of negative jumps

in the process (Xt, t ≥ 0) ensure that fθ(Xt∧Ty(x)

)−→t→∞

fθ(y). This yields

fθ(x) = limt→∞

E[e−θt∧Tyfθ

(Xt∧Ty(x)

)]= E

[e−θTyfθ(y)1Ty<∞

],

which provides the identity (4.22). 2

Remark 4.4. Theorem 4.1 holds in general for any stochastically monotone Markov pro-cess. In particular, the process is not required to have one-sided jumps. The state space[0,∞] could also be replaced by a more general nice ordered state space.

Remark 4.5. Let L denote the generator of the process X. An invariant function fθ for thesemigroup of X can be thought as a solution to the equation Lfθ = θfθ. However, whenthe process has jumps, L is an integro-differential operator and no general theory allowsone for identifying solutions of this equation. Lemma 4.1 reveals that for stochasticallymonotone processes finding a θ-invariant function corresponds to finding a θ-invariantmeasure for the dual process. This is reminiscent to a result of Cox and Rosler [CR84].

4.2. Application to CSBPs and their ancestral lineages. Recall the definition of(Xt(x), t ≥ 0) as the right-continuous inverse of (X−t,0(x), t ≥ 0) and the duality relation(2.10). We start by establishing a second duality relation, which will allow us to applyTheorem 4.1.

Page 12: Cl ement Foucart - arxiv.org

12 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

Lemma 4.6. For any t ≥ 0, x, y ∈ [0,∞)

(4.23) P(Xt(x) > y

)= P

(x > Xt(y)

).

Remark 4.7. We stress here on the strict inequalities. Lemma 4.23 entails that (Xt, t ≥ 0)is the Siegmund dual, in the sense of Section 4.1, of the CSBP (Xt, t ≥ 0).

Recall the action (2.3) of the generator L on exponential functions. We now look forthe θ-invariant measures µθ in our setting and their Laplace transforms explicitly in termsof Ψ. The following Lemma holds for general branching mechanism (i.e not necessarilysubcritical).

Lemma 4.8. For any θ > 0, the map cθ : q 7→ e−θ∫ q1

duΨ(u) is the Laplace transform of a

Borel measure µθ on (0,∞). Moreover, the measure µθ is θ-invariant for the semigroup(Pt, t ≥ 0) of the CSBP (Xt, t ≥ 0).

Proof. Recall that L denotes the generator of the CSBP (Xt(x), t ≥ 0). Let θ ≥ 0. It is

easily checked from the expression of cθ that (−1)nc(n)θ ≥ 0 on (0,∞). Bernstein theorem,

see e.g. [Wid41, Theorem 12.b, page 161], guarantees that there exists a certain Borelmeasure µθ (possibly infinite) on [0,∞) such that cθ(q) =

∫(0,∞)

e−qxµθ(dx) for all q > 0.

We now check that the measure µθ solves µθL = θµθ. Let q > 0, recall that λ > 0 is fixed.Observe first that cθ satisfies the equation

(4.24) −Ψ(q)c′θ(q) = θcθ(q), cθ(1) = 1.

Recall eq : x 7→ e−qx for any x, q ≥ 0. Since the linear span of exponential functions isa core for the generator L, it is enough to verify that

(4.25) 〈µθL, eq〉 :=

∫ ∞0

Leq(x)µθ(dx) = 〈θµθ, eq〉 = θ

∫ ∞0

e−qxµθ(dx).

One has on the other hand Leq(x) = Ψ(q)xeq(x) and (4.25) is equivalent to

(4.26) Ψ(q)

∫ ∞0

xe−qxµθ(dx) = θ

∫ ∞0

e−qxµθ(dx),

which holds true by using (4.24). 2

Remark 4.9. In the subcritical or critical cases, the map cθ is completely monotone on(0,∞) and not defined at 0, the measure µθ is infinite. In the supercritical case, cθ iscompletely monotone and well-defined and right-continuous at 0, and µθ is finite.

According to Theorem 4.1, the map fθ : x 7→ µθ([0, x)) is a θ-invariant function for

(Xt, t ≥ 0). The following simple calculation provides an expression of the Laplace trans-form of fθ. For any q > 0,

(4.27) ξθ(q) :=

∫ ∞0

fθ(y)e−qydy =

∫ ∞0

∫ ∞0

1u<ye−qyµθ(du)dy =

1

qcθ(q).

Inverting ξθ in order to find fθ does not seem to be feasible in a general setting, howeverwe shall see in the next lemma that ξθ has regular variation properties at 0, Tauberiantheorems will then allow us to find an equivalent at∞ of the function fθ and hence enableus to investigate more precisely the martingale (e−θtfθ(Xt(x)), t ≥ 0).

Page 13: Cl ement Foucart - arxiv.org

ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS 13

Lemma 4.10. Assume Ψ′(0+) 6= 0. The map R : q 7→ e−∫ q1

duΨ(u) is regularly varying at 0

with index −1/Ψ′(0+). In particular it takes the form R(q) = q− 1

Ψ′(0+)L1(1/q), where L1

is a slowly varying function at ∞. Moreover, for any θ > −Ψ′(0+),

(4.28) fθ(y) ∼y→∞

Ψ′(0+)1

Γ(

1 + θΨ′(0+)

)L1(y)θ =1

Γ(

1 + θΨ′(0+)

)R(1/y)θ.

Proof. For any q > 0,∫ 1

q

du

Ψ(u)=

∫ 1

q

(1

Ψ(u)− 1

Ψ′(0+)u

)du+

∫ 1

q

du

Ψ′(0+)u

=

∫ 1/q

1

(1

u2Ψ(1/u)− 1

Ψ′(0+)u

)du.

Set ε(u) = 1uΨ(1/u)

− 1Ψ′(0+)

for u > 0 and notice that ε(u) −→u→∞

0. One has

R(q) = q− 1

Ψ′(0+) exp

(∫ 1/q

1

ε(u)

udu

)

and by Karamata’s representation theorem, we see that L1(x) := exp(∫ x

1ε(u)u

du)

is slowly

varying at ∞. Recall cθ(q) = e−θ∫ q1

duΨ(u) . By (4.27), ξθ : q 7→ cθ(q)

qis regularly varying at

0 with index ρ = −1 − θΨ′(0+)

. The Tauberian theorem with monotone density, see e.g.

[Fel71, Chapter XIII.5, Theorem 4], provides (4.28). 2

From now on, we focus on the subcritical case Ψ′(0+) > 0. The next lemma providesan almost sure convergence of the process for fixed x and a representation of the limitW λ(x) in terms of the first passage time.

Lemma 4.11. Let λ > 0. For any x > 0, and y0 > x, the following almost sureconvergence holds:

(4.29) vt(λ)Xt(x) −→t→∞

W λ(x) := c(λ, y0)e−Ψ′(0+)Txy0 ,

where c(λ, y0) is a constant independent of x.

Moreover, for any λ > 0 and x < y < y0,

(4.30)Xt(x)

Xt(y)−→t→∞

W λ(x)

W λ(y)= e−Ψ′(0+)(Txy0−T

yy0

) = e−Ψ′(0+)Txy a.s.

Proof. We first establish the almost sure convergence towards some random variableW λ(x). By applying Theorem 4.1, the process (e−tf1(Xt(x)), t ≥ 0) is a positive martin-gale. Therefore, the latter converges almost surely towards some random variable Z(x).

Set β(1, λ) := λξ1(λ)Γ(1+1/Ψ′(0+))

and Rλ(1/y) := exp(∫ λ

1/ydu

Ψ(u)

)= exp

(∫ λ1

duΨ(u)

)R(1/y). Re-

call that Xt(x) −→t→∞∞ a.s, see Proposition 2.4. By Lemma 4.10,

e−tf1

(Xt(x)

)∼t→∞

e−tβ(1, λ)Rλ

(1/Xt(x)

)−→t→∞

Z(x) a.s.

Page 14: Cl ement Foucart - arxiv.org

14 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

Hence

(4.31) Rλ

(1/Xt(x)

)∼t→∞

etZ(x)

β(1, λ).

Now, using the fact that Rλ is non-decreasing and regularly varying at 0 with index− 1

Ψ′(0+), we see that R−1

λ is regularly varying at ∞ with index −Ψ′(0+). Taking R−1λ in

the asymptotic equivalence (4.31) yields

1

Xt(x)∼t→∞

R−1λ

(etZ(x)

β(1, λ)

)∼t→∞

(Z(x)

β(1, λ)

)−Ψ′(0+)

R−1λ (et).

By the definition of λ 7→ vt(λ), see (2.6), one has Rλ

(vt(λ)

)= et. This allows us to

conclude that

vt(λ)Xt(x) −→t→∞

W λ(x) :=(Z(x)/β(1, λ)

)Ψ′(0+)a.s.

We now explain the relation between W λ(x) and the first passage time. Note that since

the process (Xt(x), t ≥ 0) is transient, for any y0 > x, T xy0<∞ a.s. Let y0 > y. It suffices

to observe that Z(x) := limt→∞

e−tf1(Xt(x)) coincides with f1(y0)e−Txy0 . Denote by (Ft)t≥0

the natural filtration of X. Let (Mt, t ≥ 0) be the martingale defined by Mt := E[e−Txy0 |Ft]

for any t ≥ 0. Note that Mt −→t→∞

e−Txy0 almost surely. Conditional on T xy0

> t, we have

that T xy0= T

Xt(x)y0 + t. We get by the strong Markov property

Mt = E[e−Txy0 |Ft] = EXt(x)[e

−(TXt(x)y0

+t)] = e−tf1(Xt(x))/f1(y0) −→t→∞

e−Txy0 a.s.

Hence, Z(x) = f1(y0)e−Txy0 a.s, and setting c(λ, y0) :=

(f1(y0)/β(1, λ)

)−Ψ′0+), we have

W λ(x) = c(λ, y0)e−Txy0 a.s. The relation (4.30) is plainly checked using the fact that, since

there are no positive jumps, T xy + T yy0= T xy0

a.s. This achieves the proof of Lemma 4.11.2

The following lemma is a direct consequence of Lemma 4.11.

Lemma 4.12. For λ > 0 and x < y, W λ(x) = W λ(y) almost surely if and only if there

exists T <∞ such that Xt(x) = Xt(y) for all t ≥ T a.s.

Proof. According to Lemma 4.11, if W λ(x) = W λ(y), then for any y0 > x, y, T yy0= T xy0

.

This entails that at time T := T xy0, XT (x) = XT (y) and thus that Xt(x) = Xt(y) for all

t ≥ T a.s. The other implication is trivial. 2

The almost sure convergence in Lemma 4.11 holds for fixed x. We now further investi-gate the limiting process in the variable x and establish a stronger convergence.

Lemma 4.13. The process (W λ(x), x ≥ 0) admits a non-decreasing left-continuous mod-

ification. Moreover, setting Jλ := x > 0 : W λ(x+) > W λ(x), one has almost surely,

(4.32) ∀x /∈ Jλ, vt(λ)Xt(x) −→t→∞

W λ(x).

Page 15: Cl ement Foucart - arxiv.org

ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS 15

Proof. Recall that by Lemma 4.11, for any fixed x, vt(λ)Xt(x) converges almost surely

as t goes to ∞ towards a random variable denoted by W λ(x). Consider the almost sure

event Ω1 on which all random variables (W λ(q), q ∈ Q+) are defined. Since for any t ≥ 0,

the process (Xt(x), x ≥ 0) is non-decreasing then for any rational numbers q′ ≥ q ≥ 0,

one has W λ(q′) ≥ W λ(q). We work deterministically on Ω1 and define for all non-negativereal number x,

˜W λ(x) := lim

q↑x,q∈Q+

W λ(q).

The process (˜W λ(x), x ≥ 0) is well-defined on Ω1 and we define it as the null process on

Ω \ Ω1. By construction, (˜W λ(x), x ≥ 0) is left-continuous. It has right limits since it

is non-decreasing. We now show that for any x ≥ 0, P(W λ(x) =˜W λ(x)) = 1. By the

identity (4.30), on the event Ω1, we have that for all q ∈ Q+ such that q < x

(4.33) W λ(q) = W λ(x)e−Ψ′(0+)T qx a.s.

where we recall that T qx := inft ≥ 0 : Xt(q) > x. We now verify that T qx −→q→xq<x

0 a.s.

Since almost surely for all q ≤ q′, Xt(q) ≤ Xt(q′), then T q

′x ≤ T qx . Recall the Laplace

transform of T qx given in Theorem 4.1, since the function x 7→ µθ([0, x)) is left-continuous,

we have that E[e−θTqx ] = µθ([0,q))

µθ([0,x))−→q→xq<x

1; hence T qx −→q→xq<x

0 a.s. The identity (4.33) entails

that W λ(q) −→q↑x,q∈Q

W λ(x), hence W λ(x) =˜W λ(x) a.s. and

˜W λ is a left-continuous version

of the process W λ. The first statement of the lemma is established. It remains to see thatthe almost sure pointwise convergence holds outside the set of jumps Jλ. Almost surelyfor all x /∈ Jλ, one can choose q′ and q two rational numbers such that q′ < x < q. SinceXt(q

′) ≤ Xt(x) ≤ Xt(q) for all t, one has

˜W λ(q′) ≤ lim inf

t→∞vt(λ)Xt(x) ≤ lim sup

t→∞vt(λ)Xt(x) ≤ ˜

W λ(q).

Since˜W λ has left-continuous paths with right limits and x /∈ Jλ, both sides of the

inequalities above converge towards the same value W λ(x) when q′ ↑ x and q ↓ x. Thisallows us to claim (4.32). 2

From now on we work with the left-continuous version of W λ and give up the notation˜W λ. The next lemma sheds some light on the role of the parameter λ and provides Lemma3.2.

Lemma 4.14. For any λ > 0 and λ′ > 0,

(4.34)vt(λ)

vt(λ′)−→t→∞

cλ,λ′ := eΨ′(0+)∫ λλ′

duΨ(u) .

Moreover W λ(x) = cλ,λ′Wλ′(x) for all x ∈ (0,∞) almost surely.

Page 16: Cl ement Foucart - arxiv.org

16 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

Proof. Since Ψ′(0+) ≥ 0, vt(λ) −→t→∞

0. Moreover, limt→∞↑ Ψ(vt(u))

vt(u)= Ψ′(0+). Recall that

ddλvt(λ) = Ψ(vt(λ))

Ψ(λ). Therefore for any λ 6= λ′,

vt(λ)

vt(λ′)= exp

(∫ λ

λ′

d

dulog vt(u)du

)= exp

(∫ λ

λ′

Ψ(vt(u))

vt(u)

du

Ψ(u)

)and by monotone convergence vt(λ)

vt(λ′)−→t→∞

exp(

Ψ′(0+)∫ λλ′

duΨ(u)

). We see from Lemma 4.13

that W λ(x) = cλ,λ′Wλ′(x) for all x ∈ (0,∞) almost surely. 2

Lemma 4.15. The map κλ : q 7→ e−Ψ′(0+)∫ λq

duΨ(u) is the Laplace exponent of a drift-free

subordinator W λ. Its Levy measure, denoted by νλ, is finite if and only if∫∞ du

Ψ(u)<∞.

Proof. For any θ > 0 and any y ∈ (0,∞), since X−t,0(z) has the same law as X0,t(z) forall z and t ≥ 0, one gets by (2.5) and by applying Lemma 4.14

(4.35) E[e−θX−t,0

(y

vt(λ)

)] = e

−y vt(θ)vt(λ) −→

t→∞e−yκλ(θ).

At any time t, the process (X−t,0(y/vt(λ)), y ≥ 0) is a subordinator, the function κλ istherefore the Laplace exponent of a certain subordinator (W λ(y), y ≥ 0). We show that

there is no drift in the subordinator. Recall Ψ′(∞) := limu→∞

Ψ(u)u∈ (0,∞]. Since the

case of linear branching mechanism is discarded, i.e Ψ(q) 6≡ bq, one has by convexity,Ψ′(∞) > Ψ′(0+). Choose δ ∈ (Ψ′(0+),Ψ′(∞)). There exists λ0 such that for all u ≥ λ0,Ψ(u)u≥ δ and thus 1

Ψ(u)≤ 1

δu. Therefore

limθ→∞

∫ θ

λ0

(1

Ψ′(0+)u− 1

Ψ(u)

)du ≥

∫ ∞λ0

(1

Ψ′(0+)− 1

δ

)du

u=∞.

One deduces that

κλ(θ)

θ=

1

λexp

(−Ψ′(0+)

∫ θ

λ

(1

Ψ′(0+)u− 1

Ψ(u)

)du

)−→θ→∞

0,

which entails that there is no drift. Letting θ to ∞ in κλ(θ), we see that

limθ→∞

κλ(θ) = νλ((0,∞)) = eΨ′(0+)∫∞λ

duΨ(u) ,

which is finite if and only if∫∞ du

Ψ(u)<∞. Therefore the Levy measure νλ is finite if and

only if∫∞ du

Ψ(u)<∞. 2

Lemma 4.16. The process (W λ(x), x ≥ 0) has the same finite-dimensional law as

((W λ)−1(x), x ≥ 0)

where W λ is a cadlag subordinator with Laplace exponent κλ and

(W λ)−1(x) := infy ≥ 0 : W λ(y) ≥ x.

Moreover, if∫∞ du

Ψ(u)=∞, then the process W λ has continuous paths almost surely.

Page 17: Cl ement Foucart - arxiv.org

ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS 17

Proof. By independence and stationarity of the increments of (X−t,0(y/vt(λ)), y ≥ 0), theconvergence in law of the one-dimensional marginal (X−t,0(y/vt(λ)), t ≥ 0) as t goes to∞ towards W λ(y), established in (4.35), entails the convergence of the finite-dimensionalmarginals. Since W λ has no drift, the range of the subordinator W λ contains almostsurely no fixed point, see [Ber99, Proposition 1.9-(i)]. Hence for any x ∈ (0,∞) andy ∈ (0,∞), P(W λ(y) = x) = 0. The weak convergence of the finite-dimensional marginalsof (X−t,0(y/vt(λ), y ≥ 0) entails thus that for any 0 < y1 < . . . < yn and 0 < x1 < . . . < xn

limt→∞

P(X−t,0(y1/vt(λ)) ≥ x1, X−t,0(y2/vt(λ)) ≥ x2, . . . , X−t,0(yn/vt(λ)) ≥ xn

)= P

(W λ(y1) ≥ x1, W

λ(y2) ≥ x2, . . . ,Wλ(yn) ≥ xn

).

By applying Lemma 4.11, the definition of the flow (Xt(x), x ≥ 0) and the duality relation(2.11), one gets the following identities

P(W λ(x1) ≤ y1, W

λ(x2) ≤ y1, . . . , Wλ(xn) ≤ yn

)(4.36)

= limt→∞

P(vt(λ)Xt(x1) ≤ y1, vt(λ)Xt(x2) ≤ y2, . . . , vt(λ)Xt(xn) ≤ yn

)= lim

t→∞P(X−t,0(y1/vt(λ)) ≥ x1, X−t,0(y2/vt(λ)) ≥ x2, . . . , X−t,0(yn/vt(λ)) ≥ xn

)= P

(W λ(y1) ≥ x1, W

λ(y2) ≥ x2, . . . ,Wλ(yn) ≥ xn

)= P

((W λ)−1(x1) ≤ y1, (W λ)−1(x2) ≤ y2, . . . , (W λ)−1(xn) ≤ yn

).

The fact that when∫∞ 1

Ψ(u)du = ∞, there are no jumps, i.e Jλ = ∅ a.s, comes from

the fact the processes (W λ)−1 and W λ, where W λ is a subordinator with infinite Levy

measure, have the same law. Since there is no drift in W λ, the sample paths of W λ arepure singular continuous functions. 2

Remark 4.17. Recall the expression of W λ(x) in terms of T xy0given in Lemma 4.11. Lemma

4.16 ensures that the process W λ defined at any y ≥ 0 by

W λ(y) := infx > 0 : W λ(x) > y = infx > 0 : T xy0

< (log c(λ, y0)− log y) /Ψ′(0+)

is a cadlag subordinator with Laplace exponent κλ.

Remark 4.18. When∫∞ du

Ψ(u)=∞, for any time t > 0, the subordinator (X−t,0(x), x ≥ 0)

has an infinite Levy measure, the process (Xt(x), x ≥ 0) is therefore continuous increasing.

Since its limit (W λ(x), x ≥ 0) is continuous, Dini’s theorems ensure that the almost sureconvergence (4.32) holds true locally uniformly.

Proof of Theorem 3.1 The main theorem is obtained by combining Lemma 4.13, Lemma4.16 and Lemma 4.15. 2

Proof of Corollary 3.3. Recall the statement of Corollary 3.3. For any λ > 0, by (2.6),one has for any time t,∫ λ

vt(λ)

(1

Ψ′(0+)u− 1

Ψ(u)

)du =

1

Ψ′(0+)log

vt(λ)

)− t =

1

Ψ′(0+)log

vt(λ)e−Ψ′(0+)t

).

Page 18: Cl ement Foucart - arxiv.org

18 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

Recall that vt(λ) −→t→∞

0. Therefore, the asymptotics vt(λ) ∼t→∞

cλe−Ψ′(0+)t holds for some

constant cλ > 0 if and only if

(4.37)

∫ λ

0

(1

Ψ′(0+)u− 1

Ψ(u)

)du <∞.

One has, when this is the case, cλ = λe−Ψ′(0+)

∫ λ0

(1

Ψ′(0+)u− 1

Ψ(u)

)du

. Since Ψ(u) ∼u→0

Ψ′(0+)u,

the convergence (4.37) is equivalent to∫ λ

0

(Ψ(u)−γu

u2

)du < ∞, where we recall that γ =

Ψ′(0+) is the linear drift in Ψ, see (2.4). Simple calculations from the Levy-Khintchineform (2.4) ensure that the latter integral converges if and only if

∫∞u log uπ(du) < ∞.

We refer for instance to the calculations around Proposition 3.14 in [Li11]. By Theorem3.1, almost surely for all x /∈ Jλ,

e−Ψ′(0+)tXt(x) −→t→∞

1

cλW λ(x).

The process ( 1cλW λ(x), x ≥ 0) is the inverse of the subordinator (W λ(cλx), x ≥ 0), whose

Laplace exponent is θ 7→ cλκλ(θ). Recall κλ, one easily checks that

cλκλ(θ) = λe−Ψ′(0+)

[∫ λ0

(1

Ψ′(0+)u− 1

Ψ(u)

)du−

∫ θλ

duΨ(u)

]= θe

−Ψ′(0+)∫ θ0

(1

Ψ′(0+)u− 1

Ψ(u)

)du.

2

Proof of Proposition 3.6. Recall the statement of Proposition 3.6 where S denotesthe support of the random measure dW λ. By Lemma 4.14, W λ = cλ,λ′W

λ′ almost surely,this entails that S does not depend on the parameter λ. Moreover, we see by Lemma4.16 that S is the range of a subordinator W λ with Laplace exponent κλ. Having theLaplace exponent of W λ at hand, one can directly apply known results on the geometryof the range of a subordinator. By [Ber99, Theorem 5.1], for all x > 0, almost surely

dimH(S ∩ [0, x]) = ind(κλ) and dimH(S ∩ [0, x]) = ind(κλ)

with

ind(κλ) := lim infq→∞

log κλ(q)

log qand ind(κλ) := lim sup

q→∞

log κλ(q)

log q.

Recall κλ, one gets

ind(κλ) = lim infq→∞

Ψ′(0+)

log q

∫ q

λ

du

Ψ(u).

If Ψ′(∞) := limu→∞

Ψ(u)u

< ∞, then we see that∫ qλ

duΨ(u)

∼q→∞

1Ψ′(∞)

log q and the result is

established. If now Ψ′(∞) =∞, then for any large D > 0, for large q,∫ q

λ

du

Ψ(u)≤ Cλ +

log q

D,

for some constant Cλ. We see therefore that ind(κλ) ≤ Ψ′(0+)D

. Since D is arbitrarily large,

one can conclude that dimH(S ∩ [0, x]) = Ψ′(0+)Ψ′(∞)

= 0 a.s. 2

We now establish Proposition 3.8 and look for the density of W λ(x) for fixed x.

Page 19: Cl ement Foucart - arxiv.org

ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS 19

Proof of Proposition 3.8. Let λ > 0 and θ > 0. Recall κλ and denote by eθ, eqtwo independent exponential random variables with parameter θ and q respectively. ByLemma 4.16, we have that P(W λ(eθ) ≥ eq) = P(eθ ≥ W λ(eq)), or equivalently

E[e−θWλ(eq)] = 1− E[e−qW

λ(eθ)].

We deduce that∫ ∞0

E[e−θWλ(x)]e−qxdx =

1

q

(1− E[e−κλ(q)eθ ]

)=

1

q

(1− θ

κλ(q) + θ

)=

1

θ

κλ(q)

q

θ

κλ(q) + θ

=1

θ

κλ(q)

qE[e−κλ(q)eθ ] =

1

θ

∫ ∞0

e−quνλ(u)du

∫ ∞0

e−qzP(W λ(eθ) ∈ dz),

where we have used that

E[e−qWλ(eθ)] = E[e−κλ(q)eθ ] and

κλ(q)

q=

∫ ∞0

e−quνλ(u)du.

By the change of variable x = u+ z, we obtain∫ ∞0

E[e−θWλ(x)]e−qxdx =

1

θ

∫ ∞0

e−qx∫ ∞

0

νλ(x− z)P(W λ(eλ) ∈ dz)

and deduce that

E[e−θWλ(x)] =

1

θ

∫ x

0

νλ(x− z)P(W λ(eθ) ∈ dz)

=

∫ ∞0

e−θu∫ x

0

νλ(x− z)P(W λ(u) ∈ dz)du.

Thus, the density of W λ(x) is gλx(u) :=∫ x

0νλ(x−z)P(W λ(u) ∈ dz). In the case

∫∞ duΨ(u)

<

∞, (W∞(u), u ≥ 0) is a compound Poisson process with intensity ν∞, and since ν∞(0) = 1,the formula (3.18) can be plainly checked. 2

Acknowledgements: C.F’s research is partially supported by LABEX MME-DII (ANR11-LBX-0023-01).

References

[Ber99] Jean Bertoin, Subordinators: Examples and Applications, pp. 1–91, Springer Berlin Heidelberg,Berlin, Heidelberg, 1999. 17, 18

[BFM08] Jean Bertoin, Joaquin Fontbona, and Servet Martınez, On prolific individuals in a supercriticalcontinuous-state branching process, J. Appl. Probab. 45 (2008), no. 3, 714–726. MR 24551802

[Bin76] N. H. Bingham, Continuous branching processes and spectral positivity, Stochastic ProcessesAppl. 4 (1976), no. 3, 217–242. MR 0410961 1

[BLG00] Jean Bertoin and Jean-Francois Le Gall, The Bolthausen-Sznitman coalescent and the geneal-ogy of continuous-state branching processes, Probab. Theory Related Fields 117 (2000), no. 2,249–266. MR 1771663 1, 3

[CR84] J. Theodore Cox and Uwe Rosler, A duality relation for entrance and exit laws for Markovprocesses, Stochastic Process. Appl. 16 (1984), no. 2, 141–156. MR 724061 11

[DL14] Thomas Duquesne and Cyril Labbe, On the Eve property for CSBP, Electron. J. Probab. 19(2014), no. 6, 31. MR 3164759 2

[Fel71] William Feller, An Introduction to Probability Theory and Its Applications. Vol. II, Secondedition, John Wiley & Sons, Inc., New York-London-Sydney, 1971. MR 0270403 13

Page 20: Cl ement Foucart - arxiv.org

20 ANCESTRAL LINEAGES IN SUBCRITICAL CONTINUOUS-STATE BRANCHING POPULATIONS

[FM19] Clement Foucart and Chunhua Ma, Continuous-state branching processes, extremal processesand super-individuals, Ann. Inst. H. Poincare Probab. Statist. 55 (2019), no. 2, 1061–1086. 2

[FMM19] Clement Foucart, Chunhua Ma, and Bastien Mallein, Coalescences in continuous-state branch-ing processes, Electron. J. Probab. 24 (2019), no. 103. 2, 4, 5, 6, 8

[Gre74] D. R. Grey, Asymptotic behaviour of continuous time, continuous state-space branching pro-cesses, J. Appl. Probab. 11 (1974), 669–677. MR 0408016 1

[Jir58] Miloslav Jirina, Stochastic branching processes with continuous state space, CzechoslovakMath. J. 8 (83) (1958), 292–313. MR 0101554 3

[Lab14] Cyril Labbe, Genealogy of flows of continuous-state branching processes via flows of partitionsand the eve property, Ann. Inst. H. Poincare Probab. Statist. 50 (2014), no. 3, 732–769. 2

[Lag05] Andreas Nordvall Lageras, A renewal-process-type expression for the moments of inverse sub-ordinators, J. Appl. Probab. 42 (2005), no. 4, 1134–1144. 9

[Lam67] John Lamperti, Continuous state branching processes, Bull. Amer. Math. Soc. 73 (1967),382–386. MR 0208685 2

[Lam03] Amaury Lambert, Coalescence times for the branching process, Adv. in Appl. Probab. 35(2003), no. 4, 1071–1089. MR 2014270 2

[Lam07] Amaury Lambert, Quasi-stationary distributions and the continuous-state branching processconditioned to be never extinct, Electron. J. Probab. 12 (2007), no. 14, 420–446. MR 22999233

[Li00] Zeng-Hu Li, Asymptotic behaviour of continuous time and state branching processes, J. Austral.Math. Soc. Ser. A 68 (2000), no. 1, 68–84. MR 1727226 3

[Li11] Zenghu Li, Measure-valued branching Markov processes, Probability and its Applications (NewYork), Springer, Heidelberg, 2011. MR 2760602 3, 18

[LP13] Amaury Lambert and Lea Popovic, The coalescent point process of branching trees, Ann. Appl.Probab. 23 (2013), no. 1, 99–144. MR 3059232 2

[LPLG08] Yangrong Li, Anthony G. Pakes, Jia Li, and Anhui Gu, The limit behavior of dual Markovbranching processes, J. Appl. Probab. 45 (2008), no. 1, 176–189. MR 2409319 6

[Pak17] Anthony G. Pakes, Convergence rates and limit theorems for the dual Markov branching pro-cess, J. Probab. Stat. (2017), 13. 6

[Sie76] D. Siegmund, The equivalence of absorbing and reflecting barrier problems for stochasticallymonotone Markov processes, Ann. Probab. 4 (1976), no. 6, 914–924. MR 0431386 10

[Sil68] M. L. Silverstein, A new approach to local times, J. Math. Mech. 17 (1967/1968), 1023–1054.MR 0226734 3

[VT10] Mark Veillette and Murad S. Taqqu, Using differential equations to obtain joint moments offirst-passage times of increasing Levy processes, Statistics and Probability Letters 80 (2010),no. 7, 697 – 705. 9

[Wid41] D. V. Widder, Laplace transform, Mathematics Series, Princeton Univ. Press, 1941. 12