THE CONVERGENCE RATE AND ASYMPTOTIC DISTRIBU- TION...
Transcript of THE CONVERGENCE RATE AND ASYMPTOTIC DISTRIBU- TION...
Applied Probability Trust (8 February 2011)
THE CONVERGENCE RATE AND ASYMPTOTIC DISTRIBU-
TION OF BOOTSTRAP QUANTILE VARIANCE ESTIMATOR
FOR IMPORTANCE SAMPLING
JINGCHEN LIU AND XUAN YANG,∗ Columbia University
Abstract
Importance sampling is a widely used variance reduction technique to compute
sample quantiles such as value-at-risk. The variance of the weight sample
quantile estimator is usually a difficult quantity to compute. In this paper,
we present the exact convergence rate and asymptotic distributions of the
bootstrap variance estimators for quantiles of weighted empirical distributions.
Under regularity conditions, we show that the bootstrap variance estimator is
asymptotically normal and has relative standard deviation of order O(n−1/4).
1. Introduction
In this paper, we derive the asymptotic distributions of the bootstrap quantile
variance estimators for weighted samples. Let F be a cumulative distribution
function (c.d.f.), f be its density function, and αp = infx : F (x) ≥ p be
its p-th quantile. It is well known that the asymptotic variance of the p-th
sample quantile is inversely proportional to f(αp) (c.f. [6]). When f(αp) is
close to zero (e.g. p is close to zero or one), the sample quantile becomes very
unstable since the “effective samples” size is small. In the scenario of Monte
Carlo, one solution is using importance sampling for variance reduction by
distributing more samples around the neighborhood of the interesting quantile
αp. Such a technique has been widely employed in multiple disciplines. In
portfolio risk management, the p-th quantile of a portfolio’s total asset price
is an important risk measure. This quantile is also known as the value-at-risk.
∗ Postal address: 1255 Amsterdam Ave, Department of Statistics, New York, NY 10027, U.S.A.
1
2 Jingchen Liu and Xuan Yang
Typically, the probability p in this context is very close to zero (or one). A
partial list of literature of using importance sampling to compute the value-at-
risk includes [25, 26, 28, 42, 43, 44, 14, 23]. A recent work by [30] discussed
efficient importance sampling for risk measure computation for heavy-tailed
distributions. In the system stability assessment of engineering, the extreme
quantile evaluation is of interest. In this context, the interesting probabilities
are typically of a smaller order than those of the portfolio risk analysis.
Upon considering p being close to zero or one, the computation of αp can
be viewed as the inverse problem of rare-event simulation. The task of the
latter topic is computing the tail probabilities 1−F (b) when b tends to infinity.
Similar to the usage in the quantile estimation, importance sampling is also a
standard variance reduction technique for rare-event simulation. The first work
on this topic is given by [41], which not only presents an efficient importance
sampling estimator but also defines a second-moment-based efficiency measure.
We will later see that such a measure is also closely related to the asymptotic
variance of the weighted quantiles. Such a connection allows people to adapt
the efficient algorithms designed for rare-event simulations to the computation
of quantiles (c.f. [24, 30]). More recent works of rare-event simulations for
light-tailed distributions include [19, 39, 17] and for heavy-tailed distributions
include [2, 4, 18, 31, 7, 8, 10, 11]. There are also standard textbooks such as
[13, 3].
Another related field of this line of work is survey sampling where unequal
probability sampling and weighted samples are prevailing (c.f. [32, 36]). The
weights are typically defined as the inverse of the inclusion probabilities.
The estimation of distribution quantile is a classic topic. The almost sure
result of sample quantile is established by [6]. The asymptotic distribution of
(unweighted) sample quantile can be found in standard textbook such as [15].
Estimation of the (unweighted) sample quantile variance via bootstrap was
proposed by [37, 38, 40, 5, 22]. There are also other kernel based estimators
Bootstrap for Weighted Quantile Variances 3
(to estimate f(αp)) for such variances (c.f. [21]).
There are several pieces of works immediately related to the current one.
The first one is [29], which derived the asymptotic distribution of the bootstrap
quantile variance estimator for unweighted i.i.d. samples. Another one is given
by [27] who derived the asymptotic distribution of weighted quantile estimators;
see also [14] for a confidence interval construction. A more detailed discussion
of these results is given in Section 2.2.
The asymptotic variance of weighted sample quantile, as reported in [27],
contains the density function f(αp), whose evaluation typically consists of
computation of high dimensional convolutions and therefore is usually not
straightforward. In this paper, we propose to use bootstrap method to com-
pute/estimate the variance of such a weighted quantile. Bootstrap is a generic
method that is easy to implement and does not consist of tuning parameters in
contrast to the kernel based methods for estimating f(αp). This paper derives
the convergence rate and asymptotic distribution of the bootstrap variance
estimator for weighted quantiles. More specifically, the main contributions are
to first provide conditions under which the quantiles of weighted samples have
finite variances and develop their asymptotic approximations. Second, we de-
rive the asymptotic distribution of the bootstrap estimators for such variances.
Let n denote the sample size. Under regularity conditions (for instance, moment
conditions and continuity conditions for the density functions), we show that
the bootstrap variance estimator is asymptotically normal with a convergence
rate of order O(n−5/4). Given that the quantile variance decays at the rate of
O(n−1), the relative standard deviation of a bootstrap estimator is O(n−1/4).
The technical challenge lies in that many classic results of order statistics
are not applicable. This is mainly caused by the variations introduced by the
weights, which in the current context is the Radon-Nikodym derivative, and
the weighted sample quantile does not map directly to the ordered statistics.
In this paper, we employed Edgeworth expansion combined with the strong
4 Jingchen Liu and Xuan Yang
approximation of empirical processes ([33]) to derive the results.
This paper is organized as follows. In Section 2, we present our main results
and summarize the related results in literature. A numerical implementation
is given in Section 3 to illustrate the performance of the bootstrap estimator.
The proofs of the theorems are provided in Sections 4 and 5.
2. Main results
2.1. Problem setting
Consider a probability space (Ω,F , P ) and a random variable X admitting
cumulative distribution function (c.d.f.) F (x) = P (X ≤ x) and density func-
tion
f(x) = F ′(x)
for all x ∈ R. Let αp be its p-th quantile, that is,
αp = infx : F (x) ≥ p.
Consider a change of measure Q, under which X admits a cumulative distribu-
tion function G(x) = Q(X ≤ x) and density
g(x) = G′(x).
Let
L(x) =f(x)
g(x),
and X1,...,Xn be i.i.d. copies of X under Q. Assume that P and Q are
absolutely continuous with respect to each other, then EQL(Xi) = 1. The
corresponding weighted empirical c.d.f. is
FX(x) =
∑ni=1 L(Xi)I(Xi ≤ x)∑n
i=1 L(Xi). (1)
A natural estimator of αp is
αp(X) = infx ∈ R : FX(x) ≥ p. (2)
Bootstrap for Weighted Quantile Variances 5
Of interest in this paper is the variance of αp(X) under the sampling distribution
of Xi, that is
σ2n = V arQ(αp(X)). (3)
The notations EQ(·) and V arQ(·) are used to denote the expectation and
variance under measure Q.
Let Y1, ..., Yn be i.i.d. bootstrap samples from the empirical distribution
G(x) =1
n
n∑i=1
I(Xi ≤ x).
The bootstrap estimator for σ2n in (3) is defined as
σ2n =
n∑i=1
Q (αp(Y) = Xi) (Xi − αp(X))2, (4)
where Y = (Y1, ..., Yn) and Q is the measure induced by G, that is, under Q
Y1, ..., Yn are i.i.d. following empirical distribution G. Note that both G and Q
depend on X. To simplify the notations, we do not include the index of X in
the notations of Q and G.
Remark 1. There are multiple ways to form an estimate of F . One alternative
to (1) is
FX(x) =1
n
n∑i=1
L(Xi)I(Xi ≤ x). (5)
The analysis of FX is analogous to and simpler than that of (1). This is because
the denominator is a constant. We will briefly mention the corresponding results
for (5) without a rigorous proof.
The weighted sample c.d.f.’s in (1) and (5) are useful under different situa-
tions. If one can only compute L(x) up to an unknown normalizing constant,
then it is necessary to consider (1) for the quantile computation. On the other
hand, for some situations such as rare-event simulation, the measure P is
sometimes not absolutely continuous with respect to Q. In that case, (1) is
a consistent estimator of the conditional c.d.f. and αp(X) is the conditional
6 Jingchen Liu and Xuan Yang
quantile estimator given event A, where A = ∩B : Q(B) = 1. In order to
compute the quantile of F under this situation, (5) is appropriate.
Remark 2. The bootstrap method is a generic estimation procedure and only
requires that the samples are i.i.d.. In addition, this method does not involve
choosing appropriate tuning parameters. From the computation point of view,
one typically uses Monte Carlo to compute the bootstrap estimator in (4).
More precisely, let Y(k)1 , ..., Y
(k)n be i.i.d. samples with replacement from the set
X1, ..., Xn. Define Y(k) = (Y(k)1 , ..., Y
(k)n ) for k = 1, ...,m, where Y(k)’s are
independent. A Monte Carlo estimator of σ2n is
1
m
m∑k=1
(α(Y(k))− α(X))2.
In this paper, we do not pursue the analysis of Monte Carlo error for computing
σ2n.
Another advantage of bootstrap is that one does not have to regenerate the
samples in contrast to evaluating σ2p via Monte Carlo directly by regenerating
samples from Q. Sometimes, generating a single X from Q requires nonig-
norable computational cost, for instance, X =∑m
i=1 Zi with m large. A nice
feature of bootstrap is that the computation only consists of X (not the Zi’s)
and therefore is free of the complexity of the system.
2.2. Related results
In this section, we present two related results in the literature. First, [29]
established asymptotic distribution of the the bootstrap variance estimators for
the (unweighted) sample quantiles. In particular, it showed that if the density
function f(x) is Holder continuous with index 12 + δ0 then
n5/4(σ2n − σ2n)⇒ N(0, 2π−1/2[p(1− p)]3/2f(αp)−4)
as n→∞. This is consistent with the results in Theorem 2 by setting L(x) ≡ 1.
This paper can be viewed as a natural extension of [29], though the proof
techniques are different.
Bootstrap for Weighted Quantile Variances 7
In the context of importance sampling, as shown by [27], if EQ|L(x)|3 <∞,
the asymptotic distribution of a weighted quantile is
√n(αp(X)− αp)⇒ N
(0,V arQ(Wp)
f(αp)2
)(6)
as n → ∞, where Wp = L(X)(I(X < αp) − p). More general results in terms
of weighted empirical processes are given by [30]
Remark 3. An alternative variance estimator to the bootstrap method is the
approximation V arQ(Wp)/f2(αp). Such an approximation requires the evalua-
tion of V arQ(Wp) and f(αp). The computation of f(αp) is usually difficult for
complex systems.
We illustrate this issue by one example (chapter 4 in [13]). Consider n
i.i.d. random variables W1, ...,Wm. Let Sm =∑m
i=1Wi. We are interested
in computing the p-th quantile of Sm. Note that the density of Sm is the
convolution of m density functions. The computation overhead for evaluating
this density function with certain relative error is substantial especially when it
is of a small value. For more complicated systems, the computation of marginal
density is even harder.
Remark 4. Note that the weak convergence in (6) requires weaker conditions
than those in Theorems 1 and 2 in the following subsection. The weak conver-
gence does not require αp(X) to have a finite variance. In contrast, in order to
apply the bootstrap variance estimator, one needs to have the estimand well
defined, that is, V arQ(αp(X)) <∞.
We now provide a brief discussion on the efficient quantile computation via
importance sampling. The sample quantile admits a large variance when f(αp)
is small. One typical situation is that p is very close to zero or one. To fix
ideas, we consider the case where p tends to zero. The asymptotic variance of
the p-th quantile of n i.i.d. samples is
1− pnp
p2
f(αp)2.
8 Jingchen Liu and Xuan Yang
Then, in order to obtain an estimate of an ε error with at least 1−δ probability,
the necessary number of i.i.d. samples is proportional to p−1 p2
f2(αp)which
grows to infinity as p → 0. Typically, the inverse of the hazard function,
p/f(αp), varies slowly as p tends to zero. For instance, p/f(αp) is bounded if
X is a light-tailed random variable and grows at the most linearly in αp for
most heavy-tailed distributions (e.g. regularly varying distribution, log-normal
distribution).
The asymptotic variance of the quantiles of FX defined in (5) is
V arQ(L(X)I(X ≤ αp))np2
p2
f(αp)2.
There is a wealth of literature on the design of importance sampling algo-
rithms particularly adapted to the context in which p is close to zero. A
well accepted efficiency measure is precisely based on the relative variance
p−2V arQ(L(X)I(X ≤ αp)) as p→ 0. More precisely, the change of measure is
called strongly efficient, if p−2V arQ(L(X)I(X ≤ αp)) is bounded for arbitrarily
small p. A partial list of recent developments of importance sampling algorithms
in the rare event setting includes [1, 9, 10, 12, 19]. Therefore, the change of
measure designed to estimate p can be adapted without much additional effort
to the quantile estimation problem. For a more thorough discussion, see [30, 14].
The current paper is presented based on such efficient algorithms. Whereas,
we take a simplified approach by considering a fixed p and sending the sample
size n to infinity.
2.3. The main results
In this subsection, we provide an asymptotic approximation of σ2n and the
asymptotic distribution of σ2n. We first list a set of conditions which we will
refer to in the statements of our theorems.
A1 There exists an α > 4 such that
EQ|L(X)|α <∞.
Bootstrap for Weighted Quantile Variances 9
A2 There exists a β > 3 such that
EQ|X|β <∞.
A3 Assume thatα
3>β + 2
β − 3.
A4 There exists a δ0 > 0 such that the density functions f(x) and g(x) are
Holder continuous with index 12 +δ0 in a domain of αp, that is, there exists
a constant c such that
|f(x)− f(y)| ≤ c|x− y|1
2+δ0 , |g(x)− g(y)| ≤ c|x− y|
1
2+δ0 ,
for all x and y in a domain of αp.
A5 The measures P and Q are absolutely continuous with respect to each
other. The likelihood ratio L(x) ∈ (0,∞) is Lipschitz continuous in a
domain of αp.
A6 Assume f(αp) > 0.
Theorem 1. Let F and G be the cumulative distribution functions of a random
variable X under probability measures P and Q respectively. The distributions
F and G have density functions f(x) = F ′(x) and g(x) = G′(x). We assume
that conditions A1 - A6 hold. Let
Wp = L(X)I(X ≤ αp)− pL(X),
and αp(X) be as defined in (2). Then,
σ2n , V arQ(αp(X)) =V arQ(Wp)
nf(αp)2+ o(n−5/4), EQ(αp(X)) = αp + o(n−3/4)
as n→∞.
Theorem 2. Suppose that the conditions in Theorem 1 hold and L(X) has
density under Q. Let σ2n be defined as in (4). Then, under Q
n5/4(σ2n − σ2n)⇒ N(0, τ2p ) (7)
10 Jingchen Liu and Xuan Yang
as n→∞, where “⇒” denotes weak convergence and
τ2p = 2π−1/2L(αp)f(αp)−4(V arQ(Wp))
3/2.
Remark 5. Conditions A1-3 are imposed to insure that αp(X) has a finite
variance under Q. In the context of light-tailed systems, such moment con-
ditions are typically satisfied. However, for very heavy-tailed simulations, the
moment conditions may not be in place, e.g. regularly varying distributions
with very heavy tails (c.f. [18, 10, 11]).
The continuity assumptions on the density function f and the likelihood ratio
function L (conditions A4 and A5) are typically satisfied in practice. Condition
A6 is necessary for the quantile to have a variance of order O(n−1).
Remark 6. Once a consistent estimate of σ2n has been obtained, one can use
the weak convergence result in (6) to construct approximate confidence intervals
for αp.
If one considers the empirical distribution FX defined as in (5) and quantile
estimator
αp(X) = infx ∈ R : FX(x) ≥ p.
The corresponding result to those in Theorem 1 is that
σ2∗n = V arQ(αp(X)) =V arQ(Wp)
nf(αp)2+ o(n−5/4), (8)
where Wp = L(X)I(X ≤ αp). Under Q, the corresponding bootstrap variance
estimator has asymptotic distribution
n5/4(σ2∗n − σ2∗n)⇒ N(0, τ2p ) (9)
as n→∞, where
τ2p = 2π−1/2L(αp)f(αp)−4(V arQ(Wp))
3/2.
The proofs of (8) and (9) are similar to those of Theorems 1 and 2 and therefore
are omitted.
Bootstrap for Weighted Quantile Variances 11
p α1−p α1−p σ2n σ2
n σ2n
0.05 15.70 15.67 0.0032 0.0031 0.0027
0.04 16.16 16.13 0.0034 0.0033 0.0029
0.03 16.73 16.70 0.0037 0.0036 0.0032
0.02 17.51 17.47 0.0042 0.0041 0.0037
0.01 18.78 18.74 0.0054 0.0052 0.0047
σ2n: the quantile variance computed by crude Monte Carlo.
σ2n: the asymptotic approximation of σ2
n in Theorem 1.
σ2n: the bootstrap estimate of σ2
n.
Table 1: Comparison of variance estimators. The standard error of Monte Carlo errors are
reported in the parentheses, n = 10000.
3. A numerical example
In this section, we provide one numerical example to illustrate the perfor-
mance of the bootstrap variance estimator. In order to compare the boot-
strap estimator with the asymptotic approximation in Theorem 1, we choose
an example for which the marginal density f(x) is in a closed form and αp
can be computed numerically. This example is simply for comparison and
illustration purpose. The proposed bootstrap estimator is applicable to much
more complicated situations. Consider a partial sum
X =
10∑i=1
Zi
where Zi’s are i.i.d. exponential random variables with rate one. Then, the
density function of X is
f(x) =x9
9!e−x.
We are interested in computing X’s (1− p)-th quantile via exponential change
of measure that is
dQθdP
=
10∏i=1
eθZi−φ(θ), (10)
12 Jingchen Liu and Xuan Yang
where φ(θ) = − log(1 − θ) for θ < 1. This family of change of measure is
equivalent to the exponential family generated by the Gamma distribution
itself.
We further choose θ = θ∗ so that θ∗ = arg supθ(θαp − 10φ(θ)). Note that
such a choice of change of measure is the optimal change of measure, in terms
of minimizing EQ(L2(X);X ≥ αp), among all the change of measures under
which Zi’s are i.i.d..
We generate n i.i.d. replicates of (Z1, ..., Z10) fromQθ∗ , that is, (Z(k)1 , ..., Z
(k)10 )
for k = 1, ..., n; then, use Xk =∑10
i=1 Z(k)i , k = 1, ..., n and the associated
weights to form an empirical distribution and further α1−p(X). Let σ2n =
V arQθ∗ (α1−p(X)), σ2n be the approximation of σ2n in Theorem 1, and σ2n be
the bootstrap estimator of σ2n in Theorem 2. We use Monte Carlo to compute
both σ2n and σ2n by generating independent replicates of αp(X) under Q and
bootstrap samples under Q respectively. In particular, they are computed via
1000 independent Monte Carlo simulations with which the Monte Carlo error
is small enough to be ignored. The variance estimates are reported in Table 1.
With sample size n = 10000, the bootstrap variance estimator is very closed to
the (estimated) true variance and its asymptotic approximation. Nonetheless,
when computable, the asymptotic approximation σ2n provides a more accurate
estimate of σ2n.
4. Proof of Theorem 1
Throughout our discussion we use the following notations for asymptotic
behavior. We say that 0 ≤ g(b) = O(h(b)) if g(b) ≤ ch(b) for some constant
c ∈ (0,∞) and all b ≥ b0 > 0. Similarly, g(b) = Ω(h(b)) if g(b) ≥ ch(b) for all
b ≥ b0 > 0. We also write g(b) = Θ(h(b)) if g(b) = O(h(b)) and g(b) = Ω(h(b)).
Finally, g(b) = o(h(b)) as b→∞ if g(b)/h(b)→ 0 as b→∞.
Before the proof of Theorem 1, we first present a few useful lemmas.
Bootstrap for Weighted Quantile Variances 13
Lemma 1. X is random variable with finite second moment, then
EX =
∫x>0
P (X > x)dx−∫x<0
P (X < x)dx
EX2 =
∫x>0
2xP (X > x)dx−∫x<0
2xP (X < x)dx
Proof of Lemma 1. For each nonnegative random variable X,
EX =
∫x>0
xP (dx) =
∫x>0
∫0<u<x
duP (dx) =
∫u>0
P (X > u)du,
and
EX2 =
∫x>0
x2P (dx) =
∫x>0
∫0<u<x
2uduP (dx) =
∫u>0
2uP (X > u)du.
For a general random variable, let X = max(X, 0)−max(−X, 0) and apply the
above derivation to the positive part and negative part separately. Thereby, we
conclude the proof.
Lemma 2. Let X1, ..., Xn be i.i.d. random variables with EXi = 0 and E|Xi|α <
∞ for some α > 2. For each ε > 0, there exists a constant κ depending on ε,
EX2i and E|Xi|α such that
E
∣∣∣∣∣n∑i=1
Xi/√n
∣∣∣∣∣α−ε
≤ κ
for all n > 0.
Proof of Lemma 2. Let σ2 = EX21 . According to Theorem 2.18 in [16], we
obtain that for all δ > 0, x > 0, and n > 1
P
(n∑i=1
Xi >√nx
)≤ P ( max
1≤i≤nXi > δ
√nx) +
(3
1 + δσ−2x2
)1/δ
.
One can first choose δ small enough such that the second term decays fast
enough. For the first term, by Chebyshev inequality and the fact that α > 2,
for x > 1 and n > 1
P
(max1≤i≤n
Xi > δ√nx
)= 1− P (X1 ≤ δ
√nx)n
≤ 1− (1− κ0n−α/2x−α)n
≤ κ1x−α.
14 Jingchen Liu and Xuan Yang
Therefore, the tail probability has a bound
P
(n∑i=1
Xi >√nx
)≤ κ1x−α +
(3
1 + δσ−2x2
)1/δ
,
which is free of n. For the cases that P (Sn < −√nx), the development is
completely analogous. Then, the conclusion follows.
Lemma 3. Let h(x) be a non-negative function. There exists ζ0 > 0 such that
h(x) ≤ xζ0 for all x sufficiently large. Then, for all ζ1, ζ2, λ > 0 such that
λ(ζ1 − 1) < ζ2, we obtain∫ nλ
0h(x)Φ(−x+ o(xζ1n−ζ2))dx =
∫ nλ
0h(x)Φ(−x)dx+ o(n−ζ2),
as n→∞, where Φ is the c.d.f. of a standard Gaussian distribution.
Proof of Lemma 3. We first split the integral into∫ nλ
0h(x)Φ(−x+ o(xζ1n−ζ2))dx
=
∫ (logn)2
0h(x)Φ(−x+ o(xζ1n−ζ2))dx+
∫ nλ
(logn)2h(x)Φ(−x+ o(xζ1n−ζ2))dx.
Note that the second term∫ nλ
(logn)2h(x)Φ(−x+ o(xζ1n−ζ2))dx ≤ nλ(ζ0+1)Φ(− (log n)2 /2) = o(n−ζ2).
For the first term, note that for all 0 ≤ x ≤ (log n)2,
Φ(−x+ o(xζ1n−ζ2)) = (1 + o(xζ1+1n−ζ2))Φ(−x).
Then∫ (logn)2
0h(x)Φ(−x+ o(xζ1n−ζ2))dx = (1 + o(n−ζ2))
∫ (logn)2
0h(x)Φ(−x)dx.
Therefore, the conclusion follows immediately.
Bootstrap for Weighted Quantile Variances 15
Proof of Theorem 1. Let αp(X) be defined in (2). To simplify the notation,
we omit the index X and write αp(X) as αp. We use Lemma 1 to compute the
moments. In particular, we need to approximate the following probability,
Q(n1/2(αp − αp) > x
)= Q(FX(αp + xn−1/2) < p) (11)
= Q
(n∑i=1
L(Xi)(I(Xi ≤ αp + xn−1/2)− p
)< 0
).
For some λ ∈ ( 14(α−2) ,
18), we provide approximations for (11) of the following
three cases: 0 < x ≤ nλ, nλ ≤ x ≤ c√n, and x >
√n. The development for
Q(n1/2(αp − αp) < x)
on the region that x ≤ 0 is the same as that of the positive side.
Case 1: 0 < x ≤ nλ.
Let
Wx,n,i = L(Xi)(I(Xi ≤ αp + xn−1/2)− p
)− F (αp + xn−1/2) + p.
According to Berry-Esseen bound (c.f. [20]),
Q
(n∑i=1
L(Xi)(I(Xi ≤ αp + xn−1/2)− p
)< 0
)
= Q
1n
∑ni=1Wx,n,i√
1nV ar
QWx,n,1
< −F (αp + xn−1/2)− p√1nV ar
QWx,n,1
= Φ
− (F (αp + xn−1/2)− p)√
1nV ar
QWx,n,1
+D1(x). (12)
There exists a constant κ1 such that
|D1(x)| ≤ κ1
(V arQWx,n,1)3/2
n−1/2.
16 Jingchen Liu and Xuan Yang
Case 2: nλ ≤ x ≤ c√n.
Thanks to Lemma 2, for each ε > 0, the (α−ε)-th moment of 1√n
∑ni=1Wx,n,i
is bounded. By Chebyshev’s inequality, we obtain that
Q
(n∑i=1
L(Xi)(I(Xi ≤ αp + xn−1/2)− p) < 0
)= Q
(1√n
n∑i=1
Wx,n,i ≤√n(p− F (αp + x/
√n)))
≤ κ1
(1
√n(F (αp + xn−1/2)− p)
)α−ε≤ κ2x−α+ε.
Since λ > 14(α−2) , we choose ε small enough such that
∫ c√n
nλxQ(αp − αp > xn−1/2)dx = O(n−λ(α−2−ε)) = o(n−1/4). (13)
Case 3: x > c√n.
Note that
Q
(n∑i=1
L(Xi)(I(Xi ≤ αp + xn−1/2)− p) < 0
)
is a non-increasing function of x. Therefore, for all x > c√n, from Case 2, we
obtain that
Q
(n∑i=1
L(Xi)(I(Xi ≤ αp + xn−1/2)− p) < 0
)≤ κ2(c
√n)−α+ε = κ3n
−α/2+ε/2.
For c√n < x ≤ nα/6−ε/6, we have that
Q
(n∑i=1
L(Xi)(I(Xi ≤ αp + xn−1/2)− p) < 0
)≤ κ3n−α/2+ε/2 ≤ κ3x−3.
In addition, note that for all xβ−3 > n1+β/2,
Q(αp > αp + x/
√n)≤ Q(sup
iXi > αp + xn−1/2) = 1−Gn(αp + xn−1/2)
≤ O(1)n1+β/2x−β = O(x−3).
Therefore, Q (αp > αp + x/√n) = O(x−3) on the region c
√n < x ≤ nα/6−ε/3∪
x > nβ+2
2(β−3) . Since α3 > β+2
β−3 , one can choose ε small enough such that
Bootstrap for Weighted Quantile Variances 17
x > nα/6−ε/6 implies xβ−3 > n1+β/2. Therefore, for all x > c√n, we obtain
that
Q(αp > αp + x/
√n)≤ x−3,
and ∫ ∞c√nxQ(αp > αp + x/
√n)
= O(n−1/2). (14)
A summary of Cases 1, 2, and 3.
Summarizing the Cases 2 and 3, more specifically (13) and (14), we obtain
that ∫ ∞nλ
xQ(αp > αp + x/
√n)dx = o(n−1/4).
Using the result in (12), we obtain that∫ ∞0
xQ(αp > αp + x/
√n)dx
=
∫ nλ
0x
Φ
− (F (αp + xn−1/2)− p)√
1nV ar
QWx,n,1
+O(n−1/2)
dx+ o(n−1/4)
=
∫ nλ
0xΦ
− (F (αp + xn−1/2)− p)√
1nV ar
QWx,n,1
dx+O(n2λ−1/2) + o(n−1/4).(15)
Given that λ < 18 , we have that O(n2λ−1/2) = o(n−1/4). Thanks to condition
A4 and the fact that V arQ(Wx,n,1) = (1 +O(xn−1/2))V arQ(W0,n,1), we have
−(F (αp + xn−1/2)− p
)√1nV ar
QWx,n,1
= − xf(αp)√V arQW0,n,1
+O(x3
2+δ0n−1/4−δ0/2).
Insert this approximation to (15). Together with the results from Lemma 3, we
obtain that∫ nλ
0xQ(αp > αp + x/
√n)dx =
∫ nλ
0xΦ
(− xf(αp)√
V arQW0,n,1
)dx+ o(n−1/4).
18 Jingchen Liu and Xuan Yang
Therefore∫ ∞0
xQ (αp − αp > x) dx =1
n
∫ ∞0
xQ(αp > αp + x/
√n)dx
=1
n
∫ ∞0
xΦ
(− xf(αp)√
V arQW0,n,1
)dx+ o(n−5/4)
=V arQW0,n,1
nf2(αp)
∫ ∞0
xΦ(−x)dx+ o(n−5/4)
=V arQW0,n,1
2nf2(αp)+ o(n−5/4).
Similarly,∫ ∞0
Q (αp > αp + x) dx =1√n
∫ ∞0
Q(αp > αp + x/
√n)dx
=1√n
∫ ∞0
Φ
(− xf(αp)√
V arQW0,n,1
)dx+ o(n−3/4).
For Q(αp < αp − x) and x > 0, the approximations are completely the same
and therefore are omitted. We summarize the results of x > 0 and x ≤ 0 and
obtain that
EQ (αp − αp)2 =
∫ ∞0
xQ (αp > αp + x) dx+
∫ ∞0
xQ (αp < αp − x) dx
= n−1[V arQW0,n,1
f2(αp)+ o(n−1/4)
]EQ (αp − αp) =
∫ ∞0
Q (αp > αp + x) dx−∫ ∞0
Q (αp < αp − x) dx
= o(n−3/4).
5. Proof of Theorem 2
We first present a lemma that localizes the event. This lemma can be proven
straightforwardly by standard results of empirical processes (c.f. [33, 34, 35])
along with the strong law of large numbers and the central limit theorem.
Therefore, we omit it. Let Y1, ..., Yn be i.i.d. bootstrap samples and Y be a
generic random variable equal in distribution to Yi. Let Q be the probability
measure associated with the empirical distribution G(x) = 1n
∑ni=1 I(Xi ≤ x).
Bootstrap for Weighted Quantile Variances 19
Lemma 4. Let Cn be the set in which the following events occur
E1 EQ|L(Y )|ζ < 2EQ|L(X)|ζ for ζ = 2, 3, α; EQ|L(Y )|2 > 12E
Q|L(X)|2;
EQ|X|β ≤ 2EQ|X|β.
E2 Suppose that αp = X(r). Then, assume that |r/n−G(αp)| < n−1/2 log n
and |αp − αp| < n−1/2 log n.
E3 There exists δ ∈ (0, 1) such that for all 1 < x <√n
δ ≤∑n
i=1 I(X(i) ∈ (αp, αp + xn−1/2])
nQ(αp < X ≤ αp + n−1/2x))≤ δ−1,
and
δ ≤∑n
i=1 I(X(i) ∈ (αp − xn−1/2, αp])nQ(αp − n−1/2x < X ≤ αp)
≤ δ−1.
Then,
limn→∞
Q(Cn) = 1.
Lemma 5. Under conditions A1 and A5, let Y be a random variable following
c.d.f. G. Then, for each λ ∈ (0, 1/2)
sup|x|≤cnλ−1/2
∣∣∣V arQ[L(Y )(I(Y ≤ αp + x)− p)]− V arQ[L(X)(I(αp + x)− p)]∣∣∣ = Op(n
−1/2+λ).
Proof of Lemma 5. Note that
EQL2(Y )(I(Y ≤ αp + x)− p)2 =1
n
n∑i=1
L2(Xi)(I(Xi ≤ αp + x)− p)2
=1
n
n∑i=1
L2(Xi)(I(Xi ≤ αp + x)− p)2
+L2 (αp)Op
(1
n
n∑i=1
I(min(αp, αp) ≤ Xi − x ≤ max(αp, αp))
).
For the first term, by central limit theorem, continuity of L(x), and Taylor’s
expansion, we obtain that
sup|x|≤cn−1/2+λ
∣∣∣∣∣ 1nn∑i=1
L2(Xi)(I(Xi ≤ αp + x)− p)2 − EQ(L2(X)(I(X ≤ αp + x)− p)2
)∣∣∣∣∣ = Op(n−1/2+λ).
20 Jingchen Liu and Xuan Yang
Thanks to the weak convergence of empirical measure and αp−αp = O(n−1/2),
we have that the second term
L2 (αp)Op
(1
n
n∑i=1
I(min(αp, αp) ≤ Xi − x ≤ max(αp, αp))
)= Op(n
−1/2).
Therefore,
sup|x|≤cn−1/2+λ
∣∣∣EQL2(Y )(I (Y ≤ αp + x)− p)2 − EQ(L2(X)(I(X ≤ αp + x)− p)2)∣∣∣ = Op(n
−1/2+λ).
With a very similar argument, we have that
sup−cn−1/2≤x≤cn−1/2
∣∣∣EQL(Y )(I (Y ≤ αp + x)− p)− EQ(L(X)(I(X ≤ αp + x)− p))∣∣∣ = Op(n
−1/2+λ).
Thereby, we conclude the proof.
Proof of Theorem 2. Let X(1), ..., X(n) be the order statistics of X1, ..., Xn in
an ascending order. Since we aim at proving weak convergence, it is sufficient
to consider the case that X ∈ Cn as in Lemma 4. Throughout the proof, we
assume that X ∈ Cn.
Similar to the notations in the proof of Theorem 1, we write αp(X) as αp and
keep the notation αp(Y) to differentiate them. We use Lemma 1 to compute
the second moment of αp(Y)− αp under Q, that is,
σ2n =
∫ ∞0
xQ(αp(Y) > αp + x)dx+
∫ ∞0
xQ(αp(Y) < αp − x)dx.
We first consider the case that x > 0 and proceed to a similar derivation as
that of Theorem 1. Choose λ ∈ ( 14(α−2) ,
18).
Case 1: 0 < x ≤ nλ.
Similar to the proof of Theorem 1 by Berry-Esseen bound, for all x ∈ R
Q(n1/2(αp(Y)− αp) > x
)= Φ
−∑ni=1 L(Xi)
(I(Xi ≤ αp + xn−1/2)− p
)√nV arQWx,n
+D2,
Bootstrap for Weighted Quantile Variances 21
where
Wx,n = L(Y )(I(Y ≤ αp + xn−1/2)− p
)− 1
n
n∑i=1
L(Xi)(I(Xi ≤ αp + x/
√n)− p
),
and (thanks to E1 in Lemma 4)
|D2| ≤3EQ
∣∣∣Wx,n
∣∣∣3√n(V arQWx,n
)3/2 = O(n−1/2).
In what follows, we further consider the cases that x > nλ. We will essentially
follow the Cases 2 and 3 in the proof of Theorem 1.
Case 2: nλ ≤ x ≤ c√n.
Note thatn∑i=1
L(Xi) (I(Xi ≤ αp)− p) = O(1).
With exactly the same argument as in Case 2 of Theorem 1 and thanks to E1
in Lemma 4, we obtain that for each ε > 0
Q(αp(Y)− αp > xn−1/2
)≤ κ
(1√n
n∑i=1
L(Xi)(I(Xi ≤ αp + x/
√n)− p
))−α+ε
= κ
(1√n
n∑i=1
L(Xi)I(αp < Xi ≤ αp + x/√n) +O(1/
√n)
)−α+εFurther, thanks to E3 in Lemma 4, we have
Q(αp(Y)− αp > xn−1/2
)= O(x−α+ε).
With ε sufficiently small, we have∫ √nnλ
xQ(αp(Y)− αp > xn−1/2
)dx = O(n−λ(α−ε−2)) = o(n−1/4).
Case 3: x > c√n.
Note that
Q
(n∑i=1
L(Yi)(I(Yi ≤ αp + xn−1/2)− p
)< 0
),
22 Jingchen Liu and Xuan Yang
is a monotone non-increasing function of x. Therefore, for all x > c√n, from
Case 2, we obtain that
Q
(n∑i=1
L(Yi)(I(Yi ≤ αp + xn−1/2)− p
)< 0
)≤ κ3n−α/2+ε/2.
For x ≤ nα/6−ε/6, we obtain that
Q
(n∑i=1
L(Yi)(I(Yi ≤ αp + xn−1/2)− p
)< 0
)≤ κ3n−α/2+ε/2 ≤ κ3x−3.
Thanks to condition A3, with ε sufficiently small, we have that x > nα/6−ε/6
implies that xβ−3 > n1+β/2. Therefore, because of E1 in Lemma 4, for all
x > nα/6−ε/6 (therefore xβ−3 > n1+β/2)
Q(α(Y) > αp+xn−1/2) ≤ Q(sup
iYi > αp+xn
−1/2) = O(1)n1+β/2x−β = O(x−3)
Therefore, we have that
∫ ∞c√nxQ(αp(Y)− αp > xn−1/2
)dx = O(n−1/2).
Summary of Cases 2 and 3.
From the results of Cases 2 and 3, we obtain that for X ∈ Cn
∫ ∞nλ
xQ(αp(Y) > αp + x/√n)dx = o(n−1/4). (16)
With exactly the same proof, we can show that
∫ ∞nλ
xQ(αp(Y) < αp − x/√n)dx = o(n−1/4). (17)
Bootstrap for Weighted Quantile Variances 23
Case 1 revisit.
Cases 2 and 3 imply that the integral in the region where |x| > nλ can be
ignored. In the region 0 ≤ x ≤ nλ, on the set Cn, for λ < 1/8, we obtain that∫ nλ
0xQ(αp(Y) > αp + x/
√n)dx
=
∫ nλ
0x
Φ
−∑ni=1 L(Xi)
(I(Xi ≤ αp + xn−1/2)− p
)√nV arQWx,n
+D2
dx=
∫ nλ
0xΦ
−∑ni=1 L(Xi)
(I(Xi ≤ αp + xn−1/2)− p
)√nV arQWx,n
dx
+o(n−1/4). (18)
We now take a closer look at the integrand. Note that
n∑i=1
L(Xi)(I(Xi ≤ αp + xn−1/2)− p
)=
n∑i=1
L(Xi) (I(Xi ≤ αp)− p) +
n∑i=1
L(Xi)I(αp < Xi ≤ αp + xn−1/2).(19)
Suppose that αp = X(r). Then,
r∑i=1
L(X(i)) ≥ pn∑i=1
L(Xi), and
r−1∑i=1
L(X(i)) < p
n∑i=1
L(Xi).
Therefore,
p
n∑i=1
L(Xi) ≤n∑i=1
L(Xi)I(Xi ≤ αp) < L(αp) + p
n∑i=1
L(Xi).
We plug this back to (19) and obtain that
n∑i=1
L(Xi)(I(Xi ≤ αp + xn−1/2)− p
)= O(L(αp)) +
n∑i=1
L(Xi)I(αp < Xi ≤ αp + xn−1/2). (20)
In what follows, we study the dominating term in (18) via (20). For all
24 Jingchen Liu and Xuan Yang
x ∈ (0, nλ), thanks to (20), we obtain that
Φ
−∑ni=1 L(Xi)I(Xi ≤ αp + xn−1/2)− np√
nV arQWx,n
= Φ
−∑ni=1 L(Xi)I(αp < Xi ≤ αp + xn−1/2)√
nV arQWx,n
+O(n−1/2)
. (21)
Note that the above display is a functional of (X1, ..., Xn) and is also a stochastic
process indexed by x. In what follows we show that it is asymptotically a
Gaussian process. The distribution of (21) is not straightforward to obtain.
The strategy is to first consider a slightly different quantity and then connect it
to (21). For each (x(r), r) such that |x(r)−αp| ≤ n−1/2 log n and |r/n−G(αp)| ≤
n−1/2 log n, conditional on X(r) = x(r), X(r+1), ..., X(n) are equal in distribution
to the order statistics of (n−r) i.i.d. samples from Q(X ∈ ·|X > x(r)). Thanks
to the fact that L(x) is locally Lipschitz continuous and E3 in Lemma 4, we
obtain
Φ
−∑ni=r+1 L(X(i))I(x(r) < X(i) ≤ x(r) + xn−1/2)√
nV arQWx,n
+O(n−1/2)
= Φ
− L(x(r))√nV arQWx,n
n∑i=r+1
I(X(i) ∈ (x(r), x(r) + xn−1/2]) +O(x2n−1/2)
.
Note that the above display equals to (21) if αp = X(r) = x(r). For the time
being, we proceed by conditioning only on X(r) = x(r) and then further derive
the conditional distribution of (21) given αp = X(r) = x(r). Due to Lemma 5,
we further simplify the denominator and the above display equals to
Φ
(−
L(x(r))√nV arQW0,n
n∑i=r+1
I(X(i) ∈ (x(r), x(r) + xn−1/2]) +O(x2n−1/2+λ)
).
(22)
Let
Gx(r)(x) =
G(x(r) + x)−G(x(r))
1−G(x(r))= Q(X ≤ x(r) + x|X > x(r)).
Bootstrap for Weighted Quantile Variances 25
Thanks to the result of strong approximation ([33, 34, 35]), given X(r) = x(r),
there exists a Brownian bridge B(t) : t ∈ [0, 1], such that
n∑i=r+1
I(X(i) ∈ (x(r), x(r) + xn−1/2])
= (n− r)Gx(r)(xn−1/2) +
√n− rB(Gx(r)
(xn−1/2)) +Op(log(n− r)),
where the Op(log(n − r)) is uniform in x. Again, we can localize the event by
considering a set in which the error term in the above display is O(log(n−r))2.
We plug this strong approximation back to (22) and obtain
Φ
(−
L(x(r))√nV arQW0,n
(n− r)Gx(r)(xn−1/2) +Op(x
2n−1/2+λ(log n)2)
)(23)
−ϕ
(−
L(x(r))√nV arQW0,n
(n− r)Gx(r)(xn−1/2) +Op(x
2n−1/4(log n)2)
)
×(n− r)1/2L(x(r))√
nV arQW0,n
B(Gx(r)(xn−1/2)).
In addition, thanks to condition A4,
L(x(r))√nV arQW0,n
(n− r)Gx(r)(xn−1/2) =
f(x(r))√V arQW0,n
x+O(xδ0+3/2n−1/4−δ0/2).
(24)
Let
ξ(x) =(n− r)1/2L(x(r))√
nV arQW0,n
B(Gx(r)(xn−1/2)) (25)
which is a Gaussian process with mean zero and covariance function
Cov(ξ(x), ξ(y)) =(n− r)L2(x(r))
nV arQW0,nGx(r)
(x/√n)(1−Gx(r)
(y/√n))
= (1 +O(n−1/4+λ/2))L(x(r))f(x(r))
V arQW0,n
x√n
(26)
for 0 ≤ x ≤ y ≤ nλ. Insert (24) and (25) back to (23) and obtain that given
26 Jingchen Liu and Xuan Yang
X(r) = x(r)
∫ nλ
0Φ
−∑ni=r+1 L(X(i))I(x(r) < X(i) ≤ x(r) + xn−1/2)√
nV arQWx,n
+O(n−1/2)
dx
=
∫ nλ
02xΦ
(−
f(x(r))√V arQW0,n
x+ o(x2n−1/4)
)dx (27)
−∫ nλ
02xϕ
(−
f(x(r))√V arQW0,n
x+ o(x2n−1/4)
)ξ(x)dx+ o(n−1/4),
where ϕ(x) is the standard Gaussian density function. Due to Lemma 3 and
|x(r) − αp| ≤ n−1/2 log n, the first term on the right side of (27) is
∫ nλ
02xΦ
(−
f(x(r))√V arQW0,n
x+ o(x2n−1/4)
)dx (28)
= (1 + o(n−1/4))
∫ ∞0
2xΦ
(− f(αp)√
V arQW0,n
x
)dx+ op(n
−1/4).
The second term on the right side of (27) converges weakly to a Gaussian
distribution with mean zero and variance
∫ nλ
0
∫ nλ
0xyϕ
(−
f(x(r))√V arQW0,n
x+ o(x2n−1/4)
)
ϕ
(−
f(x(r))√V arQW0,n
y + o(x2n−1/4)
)Cov(ξ(x), ξ(y))dxdy
=1 + o(1)√
n
∫ nλ
0
∫ nλ
04xyϕ
(−
f(x(r))√V arQW0,n
x
)ϕ
(−
f(x(r))√V arQW0,n
y
)
×L(x(r))f(x(r))
V arQW0,nmin(x, y)dxdy
= (1 + o(1))n−1/2L(αp)
(V arQW0,n
)3/2f4(αp)
√π
. (29)
We insert the estimates in (28) and (29) back to (27) and obtain that conditional
Bootstrap for Weighted Quantile Variances 27
on X(r) = x(r),
n1/4∫ nλ
0Φ
−∑ni=r+1 L(X(i))I(x(r) < X(i) ≤ x(r) + xn−1/2)√
nV arQWx,n
+O(n−1/2)
dx
−∫ ∞0
2xΦ
(− f(αp)√
V arQW0,n
x
)dx
=⇒ N(0, τ2p /2) (30)
as n − r, r → ∞ subject to the constraint that |r/n − G−1(αp)| ≤ n−1/2 log n,
where τ2p is defined in the statement of the theorem. One may consider that the
left-hand-side of (30) is indexed by r and n− r. The limit is in the sense that
both r and n−r tend to infinity in the region that |r/n−G−1(αp)| ≤ n−1/2 log n.
The limiting distribution of (30) conditional on αp = X(r) = x(r).
We now consider the limiting distribution of the left-hand-side of (30) further
conditional on αp = X(r) = x(r). To simplify the notation, let
Vn = n1/4∫ nλ
02xϕ
(− f(αp)√
nV arQW0,n
x+ o(x2n−1/4)
)
L(αp)
∑ni=r+1 I(x(r) < X(i) ≤ x(r) + xn−1/2)− (n− r)Gx(r)
(xn−1/2)√nV arQW0,n
dx.
Then,∫ nλ
0Φ
−∑ni=r+1 L(X(i))I(x(r) < X(i) ≤ x(r) + xn−1/2)√
nV arQWx,n
+O(n−1/2)
dx
−∫ ∞0
2xΦ
(− f(αp)√
V arQW0,n
x
)dx
= n−1/4Vn + o(n−1/4).
The weak convergence result in (30) says that for each compact set A,
Q(Vn ∈ A|X(r) = x(r))→ P (Z ∈ A),
as n − r, r → ∞ subject to the constraint that |r/n − G−1(αp)| ≤ n−1/2 log n,
where Z is a Gaussian random variable with mean zero and variance τ2p /2. Note
28 Jingchen Liu and Xuan Yang
that αp = X(r) = x(r) is equivalent to
0 ≤ H =
r∑i=1
L(X(i))(1− p)− pn∑
i=r+1
L(X(i)) ≤ L(x(r)).
Let
Un =
n∑i=r+1
I(x(r) < X(i) ≤ x(r) + nλ−1/2)− (n− r)Gx(r)(nλ−1/2)
and
Bn =|Un| ≤ nλ/2+1/4 log n
.
Note that, given the partial sum Un, H is independent of the Xi’s in the interval
(x(r), x(r) + nλ−1/2) and therefore is independent of Vn. For each compact set
A and An = Vn ∈ A ∩Bn, we have
Q(An|αp = X(r) = x(r)
)(31)
=Q(0 ≤ H ≤ L(x(r))|X(r) = x(r), An
)Q(0 ≤ H ≤ L(x(r))|X(r) = x(r)
) Q(An|X(r) = x)
= EQ
[Q(0 ≤ H ≤ L(x(r))|X(r) = x(r), Un
)Q(0 ≤ H ≤ L(x(r))|X(r) = x(r)
) ∣∣∣∣∣X(r) = x(r), An
]Q(An|X(r) = x(r))
The second step of the above equation uses the fact that on the set Bn
Q(0 ≤ H ≤ L(x(r))|X(r) = x(r), Un
)= Q
(0 ≤ H ≤ L(x(r))|X(r) = x(r), Un, An
).
Note that Un only depends on the Xi’s in (x(r), x(r) + nλ−1/2), while H is the
weighted sum of all the samples. Therefore, on the setBn =|Un| ≤ nλ/2+1/4 log n
Q(0 ≤ H ≤ L(x(r))|X(r) = x(r), Un
)Q(0 ≤ H ≤ L(x(r))|X(r) = x(r)
) = 1 + o(1), (32)
and the o(1) is uniform in Bn. The rigorous proof of the above approximation
can be developed using the Edgeworth expansion of density functions straight-
forwardly, but is tedious. Therefore, we omit it. We plug (32) back to (31).
Note that Q(Bn|X(r) = x(r))→ 1 and we obtain that for each A
Q(Vn ∈ A|αp = X(r) = x(r)
)−Q(Vn ∈ A|X(r) = x(r))→ 0.
Bootstrap for Weighted Quantile Variances 29
Therefore, we obtain that conditional on αp = X(r), |αp − αp| ≤ n−1/2 log n,
and |r/n−G−1(αp)| ≤ n−1/2 log n, as n→∞
n1/4
[∫ nλ
0Q(αp(Y) > αp + x/
√n)dx−
∫ ∞0
2xΦ
(− f(αp)√
V arQW0,n
x
)dx
]
= n1/4∫ nλ
0Φ
−∑ni=r+1 L(X(i))I(αp < X(i) ≤ αp + xn−1/2)√
nV arQWx,n
+O(n−1/2)
dx
−∫ ∞0
2xΦ
(− f(αp)√
V arQW0,n
x
)dx
+ op(1)
= Vn + op(1) =⇒ N(0, τ2p /2).
Together with E2 in Lemma 4, this convergence indicates that asymptotically
the bootstrap variance estimator is independent of αp. Therefore, the uncon-
ditional asymptotic distribution is
n1/4
[∫ nλ
0Q(αp(Y) > αp + x/
√n)dx−
∫ ∞0
2xΦ
(− f(αp)√
V arQW0,n
x
)dx
]=⇒ N(0, τ2p /2).
(33)
With exactly the same argument, we have the asymptotic distribution of the
negative part of the integral
n1/4
[∫ nλ
0Q(αp(Y) < αp − x/
√n)dx−
∫ ∞0
2xΦ
(− f(αp)√
V arQW0,n
x
)dx
]=⇒ N(0, τ2p /2).
(34)
Using a conditional independence argument, we obtain that the negative part
and the positive part of the integral are asymptotically independent. Putting
together the results in Theorem 1, (16), (17), (33), (34), and the moment
calculations of Gaussian distributions, we conclude that
σ2n =
∫ ∞0
2x[Q (αp(Y) < αp − x) + Q (αp(Y) > αp + x)
]dx
=1
n
∫ ∞0
2x[Q(αp(Y) < αp − x/
√n)
+ Q(αp(Y) > αp + x/
√n)]dx
d=
V arQ(Wp)
nf(αp)2+ Zn−5/4 + o(n−5/4)
= σ2n + Zn−5/4 + o(n−5/4).
30 Jingchen Liu and Xuan Yang
where Z ∼ N(0, τ2p ).
References
[1] R.J. Adler, J.H. Blanchet, and J.C. Liu. Efficient monte carlo for large
excursions of gaussian random fields. Preprint, 2009.
[2] S. Asmussen, K. Binswanger, and B. Hojgaard. Rare events simulation for
heavy-tailed distributions. Bernoulli, 6:303–322, 2000.
[3] S. Asmussen and P. Glynn. Stochastic Simulation: Algorithms and
Analysis. Springer, New York, NY, USA, 2007.
[4] S. Asmussen and D. Kroese. Improved algorithms for rare event simulation
with heavy tails. Advances in Applied Probability, 38:545–558, 2006.
[5] G. J. Babu. A note on bootstrapping the variance of sample quantile.
Annals of the Institute of Statistical Mathematics, 38:439–443, 1986.
[6] R. R. Bahadur. A note on quantiles in large samples. Annals of
Mathematical Statistics, 37(3), 1966.
[7] J. Blanchet and P. Glynn. Effcient rare event simulation for the maxi-
mum of heavy-tailed random walks. Annals of Applied Probability, 2008.
[8] J. Blanchet, P. Glynn, and J.C. Liu. Effcient rare event simulation for
multiserver queues. preprint. 2007.
[9] J. H. Blanchet. Efficient importance sampling for binary contingency
tables. The Annals of Applied Probability, 19:949–982, 2009.
[10] J. H. Blanchet and J. C. Liu. State-dependent importance sampling for
regularly varying random walks. Advances in Applied Probability, 40:1104–
1128, 2008.
Bootstrap for Weighted Quantile Variances 31
[11] J. H. Blanchet and J. C. Liu. Efficient importance sampling in ruin
problems for multidimensional regularly varying random walks. Journal of
Applied Probability, 47:301–322, 2010.
[12] J.H. Blanchet, P. Glynn, and J.C. Liu. Fluid heuristics, lyapunov bounds
and efficient importance sampling for a heavy-tailed g/g/1 queue. Queueing
Syst., 57(2-3):99–113, 2007.
[13] J. Bucklew. Introduction to Rare Event Simulation. Springer, New York,
NY, USA, 2004.
[14] F. Chu and M.K. Nakayama. Confidence intervals for quantiles and value-
at-risk when applying importance sampling. In Proceedings of the 2010
Winter Simulation Conference, 2010.
[15] H. A. David and H. N. Nagaraja. Order Statistics. Wiley-Interscience,
New Jersey, 3rd edition, 2003.
[16] V. H. de la Pena, T. L. Lai, and Q. Shao. Self-Normalized Processes.
Springer-Verlag, Berlin Heidelberg, 2009.
[17] P. Dupuis and A. Sezer a H. Wang. Dynamic importance sampling for
queueing networks. Ann. Appl. Probab., 17:1306–1346, 2007.
[18] P. Dupuis, K. Leder, and H. Wang. Importance sampling for sums of
random variables with regularly varying tails. ACM Trans. Model. Comput.
Simul., 17(3):14, 2007.
[19] P. Dupuis and H. Wang. Dynamic importance sampling for uniformly
recurrent markov chains. The Annals of Applied Probability, 15:1–38, 2005.
[20] R. Durrett. Probability: theory and examples. Duxbury Press, 4th edition,
2010.
[21] M. Falk. On the estimation of the quantile density function. Statistics &
Probability Letters, 4:69–73, 1986.
32 Jingchen Liu and Xuan Yang
[22] M. Ghosh, W. C. Parr, K. Singh, and G. J. Babu. A note on bootstrapping
the sample median. Annals of Statistics, 12:1130–1135, 1985.
[23] P. Glasserman, P. Heidelberger, and Shahabuddin P. Variance reduction
techniques for estimating value-at-risk. Management Science, 46:1349–
1364, 2000.
[24] P. Glasserman, P. Heidelberger, and Shahabuddin P. Portfolio value-at-risk
with heavy-tailed risk factors. Math. Financ., 12:239–270, 2002.
[25] P Glasserman, P. Heidelgerger, and P. Shahabuddin. Importance sampling
and stratification for value-at-risk. In Computational Finance 1999 (Proc.
of the Sixth Internat. Conf. Comput. Finance), Cambridge, MA. MIT
Press.
[26] P. Glasserman and J. Li. Importance sampling for the portfolio credit risk.
Management Science, 50(11):1643–1656, 2005.
[27] P. W. Glynn. Importance sampling for monte carlo estimation of quantiles.
Mathematical Methods in Stochastic Simulation and Experimental Design:
Proc. 2nd St. Petersburg Workshop on Simulation, pages 180–185, 1996.
[28] Hult. H and J. Svensson. Efficient calculation of risk measures by
importance sampling - the heavy tailed case. Preprint, 2009.
[29] P. Hall and M. A. Martin. Exact convergence rate of bootstrap quantile
variance estimator. Probability Theory and Related Fields, 80:261–268,
1988.
[30] H. Hult and J. Svensson. Efficient calculation of risk measures by
importance sampling - the heavy tailed case. Preprint, 2009.
[31] S. Juneja and P Shahabuddin. Simulating heavy tailed processes using
delayed hazard rate twisting. ACM Trans. Model. Comput. Simul.,
12(2):94–118, 2002.
Bootstrap for Weighted Quantile Variances 33
[32] L. Kish. Survey sampling. John Wiley & Sons, New York, NY, 1st edition,
1965.
[33] J. Komlos, P. Major, and G. Tusnady. An approximation of partial sums of
independent rv’s and the sample df. i. Wahrsch verw Gebiete/Probability
Theory and Related Fields, 32:111–131, 1975.
[34] J. Komlos, P. Major, and G. Tusnady. Weak convergence and embedding.
Limit Theorems of Probability Theory. Colloquia Mathematica Societatis
Janos Bolyai, 11:149–165, 1975.
[35] J. Komlos, P. Major, and G. Tusnady. An approximation of partial sums of
independent r.v.’s and the sample df. ii. Wahrsch verw Gebiete/Probability
Theory and Related Fields, 34:33–58, 1976.
[36] S. L. Lohr. Sampling: design and analysis. Duxbury Press, Pacific Grove,
CA, 1st edition, 1999.
[37] J. S. Maritz and R. G. Jarrett. A note on estimating the variance of the
sample median. Journal of the American Statistical Association, 73:194–
196, 1978.
[38] J. W. McKean and R.M. Schrader. A comparison of methods for studen-
tizing the sample median. Communications in Statistics - Simulation and
Computation, 13:751–773, 1984.
[39] J. Sadowsky. On Monte Carlo estimation of large deviations probabilities.
Ann. Appl. Probab., 6:399–422, 1996.
[40] S. J. Sheather. A finite sample estimate of the variance of the sample
median. Statistics & Probability Letters, 4:1815–1842, 1986.
[41] D. Siegmund. Importance sampling in the monte carlo study of sequential
tests. The Annals of Statistics, 4:673–684, 1976.
34 Jingchen Liu and Xuan Yang
[42] L. Sun and L. J. Hong. A general framework of importance sampling for
value-at-risk and conditional value-at-risk. In Proceedings of 2009 Winter
Simulation Conference, pages 415–422, 2009.
[43] L. Sun and L. J. Hong. Asymptotic representations for importance-
sampling estimators of value-at-risk and conditional value-at-risk. Op-
erations Research Letters, 38:246–251, 2010.
[44] R.-H. Wang, S.-K. Lin, and C.-D. Fuh. An importance sampling method
to evaluate value-at-risk for asset with jump risk. Asia-Pacific Journal of
Financial Studies, 38(5):745–772, 2009.