Estimation and Testing of Nonproportional Weibull Hazard...
Transcript of Estimation and Testing of Nonproportional Weibull Hazard...
-
Estimation and Testing of Nonproportional WeibullHazard Models
Thomas W. Zuehlke
Department of Economics, Florida State University, Tallahassee, FL 32306, USA
August 3, 2011
Abstract
Most applications of the Weibull hazard model specify a common shape parameter. This is a proportionalhazard model that imposes a common rate of duration dependence. A wide class of nonproportional Weibullmodels may be estimated by making the shape parameter a linear function of observable regressors. Thelog-likelihood function for these models is well behaved. The conditions under which this generalization isuseful are essentially the same conditions under which interaction terms are useful in classical regression.Since the nonproportional model nests the proportional model, a formal test for nonproportionality maybe conducted by likelihood ratio test. Estimation and testing of nonproportional models is illustrated withdata sets for housing sales, out-of-court settlements, and oil field exploration. Finally, estimation of aproportional Weibull model after adding temporal interaction terms to the regressors that specify the scaleparameter is shown to be a fundamental misspecification. The standard log-likelihood function fails torecognize the stochastic nature of temporal interaction terms and the resulting estimates often fall outsidethe parameter space of the Weibull.
Key words: Weibull, Nonproportional, Duration Dependence
JEL Classification: C41
E-mail: [email protected] (Thomas W. Zuehlke).
-
Estimation and Testing of Nonproportional Weibull Hazard Models
Abstract
Most applications of the Weibull hazard model specify a common shape parameter. This is
a proportional hazard model that imposes a common rate of duration dependence. A wide class
of nonproportional Weibull models may be estimated by making the shape parameter a linear
function of observable regressors. The log-likelihood function for these models is well behaved.
The conditions under which this generalization is useful are essentially the same conditions under
which interaction terms are useful in classical regression. Since the nonproportional model nests the
proportional model, a formal test for nonproportionality may be conducted by likelihood ratio test.
Estimation and testing of nonproportional models is illustrated with data sets for housing sales,
out-of-court settlements, and oil field exploration. Finally, estimation of a proportional Weibull
model after adding temporal interaction terms to the regressors that specify the scale parameter is
shown to be a fundamental misspecification. The standard log-likelihood function fails to recognize
the stochastic nature of temporal interaction terms and the resulting estimates often fall outside
the parameter space of the Weibull.
-
I. Introduction
Weibull hazard models have been used to analyze a wide variety of problems in economics,
including duration of unemployment, duration of labor strikes, duration of litigation in legal dis-
putes, and even traffic congestion. These models have also been used extensively in other fields of
study including political science and biometrics. The most commonly used specification makes the
scale parameter a function of covariates, while imposing a common shape parameter. The shape
parameter determines the duration elasticity of the hazard function. This specification is one of the
class of proportional hazard models.1 Recent examples using this specification include Hernandez
and Dresdner (2010), Kumazawa (2010), and Giles (2007).
The assumption of a common shape parameter is adopted for convenience, and no attempt
has been made to justify the assumption. In fact, relatively simple economic models can provide
reasons to expect duration elasticities to differ across observations. Furthermore, the assumption
of a common duration elasticity is not necessary from an econometric perspective. Weibull hazard
models may be generalized to allow both the scale and shape parameters to depend on covariates.
This results in a class of nonproportional Weibull models. The conditions for identification are
similar to those necessary when using interaction terms in classical regression. In this case, the
distribution of duration must not be degenerate, and the regressors employed cannot be perfectly
collinear. This generalization may also be used in conjunction with methods that allow for unob-
served heterogeneity in the scale parameter, using either a parametric specification of the mixing
distribution, as in Emons and Sheldon (2009), or the nonparametric method of Laird (1978) and
Lindsay (1981).
Estimation of a proportional Weibull model after including temporal interaction terms in
the covariates that specify the scale parameter has been suggested as a means of testing for non-
proportionality. Unfortunately, this simple procedure is a fundamental misspecification that fails to
recognize the stochastic nature of temporal interaction terms. The resulting estimates of the shape
parameter often fall outside the parameter space of the Weibull.
Section II provides a brief discussion of proportional Weibull models. Section III specifies
a class of nonproportional Weibull models by making the shape parameter a linear function of
1With a proportional hazard model, the ratio of the hazard functions for any two observations is constant at all pointsalong the temporal profile.
1
-
regressors. The resulting log-likelihood function is well behaved, and any root to the score equations
is a global MLE. The limitations of testing for nonproportionality with asymptotic t-tests are
discussed, and a likelihood ratio test that overcomes these limitations is presented. Section IV
shows that use of temporal interaction terms in conjunction with a proportional Weibull model
is a fundamental misspecification. Section V presents a trio of applications that illustrate the
value of the nonproportional Weibull model. The first two applications provide relatively simple
specifications that illustrate interpretation of the coefficients. These applications are accompanied
by estimates of a proportional model that has been augmented with temporal interaction terms. The
resulting parameter estimates are shown to be outside the parameter space. In a final application,
estimation of a proportional hazard model (with common shape parameter) results in coefficient
estimates for the scale parameter that are largely insignificant. By most standards, the model
would be considered poor. When the nonproportional Weibull model is used, a large block of
regressors (relating to price and volatility) become significant determinants of both the scale and
shape parameters. The null hypothesis of proportionality is easily rejected using a likelihood ratio
test. The final section presents conclusions.
II. Proportional Weibull Hazard Models
Hazard models are typically specified by the choice of either the hazard function, h(t), or the
survival function, S(t). The survival function is simply the complement of the distribution function
for duration. Given some specification for the hazard function, the survival function is determined
as
S(t) = exp[−t∫
0
h(s)ds] (1)
Given a specification for the survival function, the hazard function is determined as
h(t) = −∂ ln[S(t)]∂t
=f(t)
S(t)(2)
where f(t) is the density function for duration. Perhaps the most commonly used parametric form
for hazard models is the Weibull.2 The survival function of the Weibull is
S(t) = exp(−eαtβ) (3)
2A good exposition of the basics of the Weibull hazard model is provided by Lancaster (1979).
2
-
where t > 0 and β >0. The parameter α is the scale parameter of the Weibull, and the parameter
β is the shape parameter. By equation (2), the hazard function of the Weibull is
h(t) = eαβtβ−1 (4)
The log transformation reveals the fundamental assumption of the Weibull model.
ln[h(t)] = [α + ln(β)] + (β − 1) ln(t) (5)
The log of the hazard function is linear in the log of duration. The coefficient (β-1) is the elasticity
of the hazard function with respect to duration.
A typical sample will have both complete and incomplete observations for duration. Com-
pleted duration is the total length of time spent in a state, and is observed if a spell ends during the
sampling period. An observation of completed duration provides quantitative information about
duration. Here, the density function is the relevant term in the likelihood function. Incomplete du-
ration is the time spent in a state as of the end of the sampling period. An observation of incomplete
duration only provides qualitative information about duration. All that is known is that completed
duration exceeds observed duration. In this case, the survival function is the relevant term in the
likelihood function. Let Ti denote observed duration for a random sample of n observations, and
let Ji denote a binary variable that equals one if duration is complete and equals zero otherwise.
With statistically independent observations, the log-likelihood function is:
lnL(α, β) =∑n
i=1{Ji ln[f(Ti)] + (1− Ji) ln[S(Ti)]} (6)
Since f(t) = h(t)S(t), this may be written as
lnL(α, β) =∑n
i=1{ln[S(Ti)] + Ji ln[h(Ti)]} (7)
In the case of the Weibull, the log-likelihood function is
lnL(α, β) =∑n
i=1
{(−eαT βi ) + Ji[ln(eαT
β−1i ) + ln(β)]
}(8)
Since T β−1=T β/T and eαT β=exp[ α + β ln(T )], this may be written as
lnL(α, β) =∑n
i=1{− exp[α + β ln(Ti)] + Ji[α + β ln(Ti)− ln(Ti) + ln(β)]} (9)
3
-
Most statistical software for estimation of the Weibull hazard model will incorporate regres-
sors by letting αi = Xiδ, where Xi is a row vector of length k, and where the first element of Xi is
one. This just provides an observation-specific intercept in the linear relationship between the log-
hazard function and log-duration. The null hypothesis of a common scale parameter involves testing
the restrictions δ2 = δ3 = · · · = δk=0. With a common shape parameter, the hazard functions of
any two observations are proportional at all points along the temporal profile. The assumption of a
common shape parameter is adopted for convenience. Lancaster (1990) states that ‘no econometri-
cian ... has ever given an economic-theoretical justification of why hazards should be proportional,
or even approximately so.’ More recently, Keele (2010) states that ‘... for proportional hazards
models such as the Weibull, there is no method for the detection for nonproportional hazards.’
III. A Class of Nonproportional Weibull Models
Nonproportionality in the temporal profile of the hazard function may be introduced by
making the shape parameter a function of regressors. This allows an observation-specific slope to
the linear relationship between the log-hazard function and log-duration. Specifically, let βi = Ziγ,
where Zi is a row vector of length q, and where the first element of Zi is one. The null hypothesis of a
common shape parameter involves testing the restrictions γ2 = γ3 = · · · = γq=0. This generalization
provides a natural framework for testing deviations from proportionality. With these changes, the
log-likelihood function is:
lnL(δ, γ) =∑n
i=1{− exp[Xiδ + (Ziγ) ln(Ti)] + Ji[Xiδ + (Ziγ) ln(Ti)− ln(Ti) + ln(Ziγ)]} (10)
This may be further simplified by letting Vi=[ Xi ln(Ti)·Zi ] and θ′=[ δ′ γ′ ]. Then,
lnL(δ, γ) =∑n
i=1{− exp(Viθ) + Ji[Viθ − ln(Ti) + ln(Ziγ)]} (11)
The estimates of δ and γ that maximize the log-likelihood function in equation (11) are the
solution to a set of simultaneous nonlinear implicit functions. The score equations are:
∂ lnL(δ, γ)
∂δ=
∑ni=1{Ji − exp(Viθ)}X
′
i (12)
and
∂ lnL(δ, γ)
∂γ=
∑ni=1{[Ji − exp(Viθ)][ln(Ti) · Z
′
i ] + Ji(Ziγ)−1Z
′
i} (13)
4
-
Given the added complexity introduced by an observation-specific shape parameter, it is natural to
ask whether the problem is still sufficiently well behaved to guarantee a unique root to the score
equations. This is in fact the case, as the Hessian matrix is negative definite for all values of (δ, γ).
The Hessian matrix is composed of
∂2 lnL(δ, γ)
∂δ∂δ′=
∑ni=1{− exp(Viθ)}X
′
iXi (14)
∂2 lnL(δ, γ)
∂δ∂γ′=
∑ni=1{− exp(Viθ)}X
′
i [ln(Ti) · Zi] (15)
and∂2 lnL(δ, γ)
∂γ∂γ′=
∑ni=1{− exp(Viθ)[ln(Ti) · Zi]′[ln(Ti) · Zi]− Ji(Ziγ)−2Z
′
iZi} (16)
Let V denote the nx(k+q) data matrix with rows [ Xi ln(Ti)·Zi ], B denote an nxn
diagonal matrix with diagonal elements bii=exp(Viθ), and W denote an nx(k+q) matrix with rows
[ 0 Ji(Ziγ)−1Zi ], then the Hessian may be written as -(V
′BV+W
′W). Since B is diagonal with
strictly positive diagonal elements, and V has full column rank by assumption, V′BV is positive
definite. Any matrix of the form W′W is positive semi-definite. Since the sum of a positive definite
and positive semi-definite matrix must be positive definite, (V′BV+W
′W) is positive definite and
−(V′BV+W′W) is negative definite. Thus, any root to the score equations is a unique global MLE.
Note that V has rows Vi=[ Xi ln(Ti)·Zi ] and will have full column rank if both X and Z have
full column rank. This simply excludes perfect multicollinearity in the choice of regressors. This is
true even if X and Z are chosen to include the same set of regressors, provided that ln(Ti) is not
degenerate. When X and Z are identical, the conditions are basically those that apply when using
interaction terms in classical regression. In practice, the use of interaction terms may, but need
not, introduce the possibility of near multicollinearity. This is only a problem when the regressors
used to form the interaction term lack sufficient independent sources of variation. In the current
application, near collinearity of Xi and ln(Ti)·Xi is a problem when Xi can predict ln(Ti) with a
high degree of accuracy. The precision of both δ̂ and γ̂ will suffer when this occurs, and individual
t-tests of the null hypotheses γj = 0 may give misleading conclusions regarding the presence of
nonproportionality.
An alternative test statistic is available, however. Since the nonproportional Weibull model
nests the proportional model under the null hypothesis γ2 = γ3 = · · · = γq=0, proportionality may
5
-
be tested using a likelihood ratio test. This statistic is based on differences in fit of the model
rather than the precision of the individual coefficient estimates and is not subject to problems of
near multicollinearity. If θ̃ denotes the null restricted ML estimates, and θ̂ denotes the unrestricted
ML estimates, then -2[lnL(θ̃)-lnL(θ̂)] is asymptotically χ2 with q − 1 degrees of freedom. The null
hypothesis of proportionality is rejected if this sample statistic exceeds the critical value that leaves
the desired α level in the upper tail of the χ2(q-1) distribution function.
Concerns about collinearity should not preclude estimation of a nonproportional model.
Collinearity need not occur (it is sample specific), and its presence is readily detected with the aid
of the LR test statistic for proportionality. In addition, while multicollinearity results in imprecise
coefficient estimates, it does not bias coefficients, standard errors, or test statistics. Failure to
account for nonproportionality when it is present involves imposition of an invalid restriction, the
common shape parameter, resulting in misspecification bias.
IV. Temporal Interaction Terms in Proportional Weibull Models
At first glance, the log-likelihood function in equation (11) appears to suggest that a Weibull
model allowing for nonproportionality in the hazard profiles may be estimated with standard sta-
tistical software by simply adding interaction terms for Zi and ln(t) to the set of regressors in Xi
and estimating a proportional Weibull model (with common duration elasticity).3 Unfortunately,
this procedure is a fundamental misspecification that involves a failure to recognize the stochastic
nature of the interaction terms, ln(t) ·Zi. To see why, note that the log of the survival function for
the nonproportional Weibull model is
ln[S(t)] = −eXiδtZiγ = −eXiδ+[ln(t)·Zi]γ (17)
The second equality of equation (17) shows that exactly the same survival function is obtained
under the assumptions αi = Xiδ and βi = Ziγ as under the assumptions αi = Xiδ + [ln(t) · Zi]γ
and βi=0. That is, making the shape parameter a function of the regressors, Zi, is observationally
equivalent to making the scale parameter of function of the regressors Xi plus interaction terms
for Zi and ln(t). Regardless of the interpretation that is adopted, the first equality in equation
(2), which relates the hazard function to the derivative of the survival function, shows that the
3This procedure has been suggested by both Yamaguchi (1991) and Box-Steffensmeier and Zorn (2001).
6
-
corresponding hazard function is
h(t) = eXiδ(Ziγ)t(Ziγ−1) (18)
or in log terms,
ln[h(t)] = Xiδ + (Ziγ) ln(t)− ln(t) + ln(Ziγ) (19)
This is the relevant hazard function regardless of whether the shape parameter is made a linear
function of regressors or the scale parameter is made a linear function of regressors that include
interaction terms with ln(t). With either interpretation, the final term in the log-likelihood function
is∑n
i=1 Ji ln(Ziγ). The practical impact of this term is to bound values of the observation-specific
shape parameters, β̂i = Ziγ̂, away from zero. When a proportional Weibull model is estimated after
adding temporal interaction terms, the final term in the log-likelihood function is only n1ln(γ1),
where γ1 is the common shape parameter and n1 is the number of observations of completed dura-
tion. This procedure will allow negative values of Ziγ̂ so long as γ̂1 >0. The empirical section of
this paper will show that inclusion of temporal interaction terms when estimating a proportional
Weibull hazard model will often result in estimated values of the shape parameter that are outside
the parameter space.
V. Applications
Equation (5) provides a good context within which to consider alternative specifications of
the hazard function. It states that the log of the hazard function is linear in the log of duration. The
generalization αi = Xiδ allows an observation-specific intercept in the time profile of the log-hazard
function. This specification is provided by virtually all econometric software for hazard estimation.
The slope of the time profile, β-1, is the duration elasticity. The specification βi = Ziγ allows an
observation-specific slope. Perhaps the simplest application involves specifying X and Z to include
a column of ones plus a common binary covariate. The interpretation of the coefficients is similar
to that obtained with the use of binary regressors in classical regression. The coefficients of the
column of ones in X and Z correspond to the intercept and slope of the log-hazard profile for the
zero case of the binary, while the coefficients of the binary are intercept and slope shifters for the
unit case. This specification allows a separate log-hazard profile for the two cases defined by the
binary. Of course, one could always estimate a separate model for each of these cases, but as with
7
-
classical regression, the real power of this method comes when one wants to allow intercept and
slope shifts for the binary while imposing common coefficients for other regressors in the model.
Many economic models predict differential rates of duration dependence across individuals
or groups. Lentz and Tranaes (2005) provide a theoretical model of job search characterized by
positive duration dependence. As search progresses and wealth is reduced, search intensity increases
resulting in positive duration dependence. The duration elasticity varies with the initial level of
wealth as well as the per-period costs of search. Zuehlke (1987) makes a similar argument with
respect to the hazard probability of sale for single-family housing. Ceteris paribus, the seller of
a vacant house has a higher opportunity cost than the seller of an occupied house. This has a
differential impact on the search strategies of the two groups. As search progresses and wealth
declines, the seller of a vacant house has an incentive to be more accommodating. The hazard
function should exhibit a higher duration elasticity for vacant houses than for occupied houses.
This specification can be modeled using the log-likelihood function in equation (10) where both X
and Z contain a column of ones and a binary indicating vacancy status of the dwelling. This allows
both the intercept and slope of the log-hazard profile to differ depending on whether the house
being offered for sale is occupied or vacant.
Table (1) presents moments for a Multiple Listing Service (MLS) sample of houses. Approx-
imately 52% of the houses were sold and approximately 43% were vacant (VAC). The average value
of time on the market was 129 days. The first block of 3 columns in Table (2) present estimates
of a nonproportional Weibull model using the single binary regressor VAC in both X and Z. The
estimates of α and β for occupied houses (VAC=0) are δ̂1 = -4.73 and γ̂1 = 0.87 respectively. The
corresponding duration elasticity for occupied houses is γ̂1-1 = -0.13. The hazard probability of
sale for occupied houses is decreasing in duration (negative duration dependence). The coefficients
of VAC in X and Z are the intercept and slope shifters for vacant houses. These are δ̂2 and γ̂2
respectively. Both are significantly different than zero at conventional levels. The estimates of
α and β for vacant houses are δ̂1+δ̂2 = -7.53 and γ̂1+γ̂2 = 1.36 respectively. The corresponding
duration elasticity is γ̂1+γ̂2-1 = 0.36. The hazard probability of sale for vacant houses is increasing
in duration (positive duration dependence). The specification presented here is a deliberate over-
simplification made for illustrative purposes. While not the focus of this paper, these differences
persist with a more fully specified model.
8
-
For this specification, the null hypothesis of proportionality is γ2 = 0. This hypothesis is
rejected at the 1% level with both the t-test and LR test. There is strong evidence of nonproportion-
ality. Because of the simplicity of the specification, the LR statistic is identical to that one would
obtain by estimating separate models for the vacant and occupied subsamples, pooling the samples
and estimating a common model, and then constructing a LR test for pooling of samples. Using
tests for pooling of subsamples to test for nonproportionality, suggested by Box-Steffensmeier and
Zorn (2001), works well for a single binary regressor, but becomes very cumbersome with continuous
regressors or sets of regressors.
The second block of 3 columns in Table (2) presents the estimates of a proportional Weibull
model when an interaction term for VAC with log-duration is added to X. The coefficient of the
interaction term is denoted δ3 here. As noted in section IV, this approach is a fundamental misspec-
ification that fails to recognize that interaction terms with log-duration are stochastic. When this
approach is taken, the final term in the log-likelihood function is n1ln(γ1), whereas with a properly
specified model, the final term is∑n
i=1 Ji ln(γ1 + δ3·V ACi). This misspecification has a significant
impact on the estimates. The estimates of α and β for occupied houses are δ̂1 = -7.40 and γ̂1 =
1.39 respectively, with a corresponding duration elasticity of 0.39. In contrast with a properly spec-
ified model, occupied houses now appear to exhibit positive duration dependence. The estimates
of α and β for vacant houses are δ̂1+δ̂2 = 0.29 and γ̂1+δ̂3 = -0.17, respectively. Since the shape
parameter of the Weibull must be positive, the estimated value of β for vacant houses is outside
the parameter space! The conclusion from all of this is that simply adding interaction terms with
log-duration to the list of regressors used to specify a proportional Weibull model is not an effective
way to test for non-proportionality in the hazard profile.
Another interesting application of nonproportional hazard models concerns the dynamics of
pretrial negotiations. Bebchuk (1984) presents a theoretical model of the settlement process that
predicts the likelihood of settlement to be decreasing in the monetary compensation at stake in the
trial. The stakes of a trial are generally subjective and can be measured any number of ways, but
one possible measure is provided by the ‘ad damnum’ claim requested by the plaintiff on initial
filing. This is a measure of the alleged damages. Spier (1992) generalizes the model of Bebchuk
(1984) and finds the presence of a ‘deadline effect,’ where the probability of settling increases as
the trail date approaches. This explains why settlement often occurs ‘on the court-house steps.’
Spier’s model also predicts that fee shifting, where the judge can award court costs and attorney
9
-
fees to the prevailing party, will have a differential impact on the probability of settlement. Cases
subject to fee shifting initially have a lower probability of settlement, but their hazard probabilities
increase more rapidly as the court date approaches.
Table (3) presents moments for a sample of civil cases from the U.S. Court system. Ap-
proximately 50% of the cases were settled during the sample period and approximately 8% were
subject to fee shifting. The mean value of the log of alleged damages is 3.3381, which corresponds
to approximately 28 thousand dollars. There is substantial variation however, and a value for al-
leged damages one standard deviation above the mean corresponds to about 261 thousand dollars.
The average duration of litigation was 425 days. The first block of 3 columns in Table (4) present
estimates of a nonproportional Weibull model that includes the fee-shifting binary (BRITISH) and
the log of alleged damages (DAMAGES) in both X and Z. This specification makes the scale and
shape parameters linear functions of DAMAGES, where the intercepts of these linear functions
differ depending on whether the case is subject to fee shifting, but the coefficient of DAMAGES
is common to both populations. As with the previous example, this specification has been simpli-
fied to illustrate interpretation of the coefficients, but the results are robust to more fully specified
models. See Fournier and Zuehlke (1996).
The coefficients of BRITISH and DAMAGES are statistically significant in both the scale
and shape parameters. Absent fee shifting (BRITISH=0), the scale parameter αi is estimated as
δ̂1 + δ̂2·DAMAGESi = -6.46372-0.57630·DAMAGESi.
Cases with larger alleged damages are initially less likely to settle. The corresponding shape pa-
rameter βi is estimated as
γ̂1 + γ̂2·DAMAGESi = 1.02326+0.06605·DAMAGESi.
The duration elasticity is positive and increasing in the size of alleged damages. The hazard prob-
ability of settlement increases more rapidly for cases with larger alleged damages. When evaluated
at the mean of DAMAGES, the values of α̂ and β̂ for cases that are not subject to fee shifting are
-8.3875 and 1.2437 respectively. The corresponding duration elasticity is 0.2437.
For cases subject to fee shifting (BRITISH=1), αi is estimated as
(δ̂1 + δ̂3) + δ̂2 ·DAMAGESi = (−6.46372− 0.57543)− 0.57630 ·DAMAGESi
= −7.03915− 0.57630 ·DAMAGESi
10
-
while βi is estimated as
(γ̂1 + γ̂3) + γ̂2 ·DAMAGESi = (1.02326 + 0.13559) + 0.06605 ·DAMAGESi
= 1.15885 + 0.06605 ·DAMAGESi.
At the mean of DAMAGES, the estimated values of α and β for cases subject to fee shifting are
-8.9629 and 1.3793 respectively. For cases subject to fee shifting, the initial hazard probability is
slightly lower, but the rate of duration dependence is higher.
These estimates also provide strong evidence of nonproportionality. Both γ̂2 and γ̂3 are
significantly different from zero at the 1% level. As one would expect, the LR test statistic of the
joint hypothesis γ2 = γ3 = 0 is also significant at the 1% level.
The second block of 3 columns in Table (4) presents the estimates that result when a pro-
portional Weibull model is estimated after including temporal interaction terms for both DAM-
AGES and BRITISH in X. The coefficients of these interaction terms are denoted δ4 and δ5
respectively. The final term in the log-likelihood function of a properly specified model would
be∑n
i=1 Ji ln(γ1 + δ4·DAMAGESi+δ5·BRITISHi), rather than simply n1ln(γ1). As with the
MLS example, this misspecification has a significant impact on the estimates. Absent fee shift-
ing (BRITISH=0), the scale parameter αi is estimated as
δ̂1 + δ̂2·DAMAGESi = -10.76187+1.80398·DAMAGESi.
The corresponding shape parameter βi is estimated as
γ̂1 + δ̂4·DAMAGESi = 1.67762-0.31109·DAMAGESi.
With this method, cases with larger alleged damages are initially more likely to settle, but have
lower rates of duration dependence. These conclusions are just the opposite of those found with
a properly specified model. Worse yet, for sufficiently large values of DAMAGES, the estimated
value of the shape parameter, βi, is outside the parameter space. The value of β̂i is negative if
DAMAGES exceeds 5.3927, which occurs for 21.93 percent of the BRITISH=0 subsample.
Similar results apply for cases subject to fee shifting (BRITISH=1). The scale parameter αi
is estimated as
(δ̂1 + δ̂3) + δ̂2 ·DAMAGESi = (−10.76187 + 6.00072) + 1.80398 ·DAMAGESi
= −4.76115 + 1.80398 ·DAMAGESi.
11
-
The corresponding shape parameter βi is estimated as
(γ̂1 + δ̂5) + δ̂4 ·DAMAGESi = (1.67762− 0.90866)− 0.31109 ·DAMAGESi
= 0.76896− 0.31109 ·DAMAGESi.
The value of β̂i is negative if DAMAGES exceeds 2.4718, which occurs for 85.62 percent of the
BRITISH=1 subsample.
The final application in this section is interesting in that regressors of theoretical interest are
found to be insignificant determinants of the scale parameter when a proportional Weibull model
is estimated, but are highly significant determinants of both the scale and shape parameters when
a nonproportional model is estimated. Favero et al. (1994) present a dynamic model of irreversible
investment decisions where the level (P) and volatility (VARDP) of oil prices interact to determine
the development lag for new oil field discoveries. Table (5) presents moments for a sample of North
Sea oil fields. The mean time lag until development (TIME) is 63 months. The size of the oil
field (SIZE) is recoverable reserves in millions of barrels. The mean water depth of the oil field
(WATERD) is 126 meters. The mean volume of gas reserves (GASRES) is 179 billion cubic feet.
The mean value of the real after-tax price of oil (P) is 2.02 measured in 1960 dollars. A measure of
the volatility of real oil prices (VARDP) is obtained as the squared standard errors from a recursive
regression of the temporal change in oil price on a constant.4
The first block of 3 columns in Table (6) present estimates of a proportional hazard model
using the regressors discussed above to specify the scale parameter. Since Favero et al. (1994)
conclude that ‘both our theoretical model and our empirical results suggest the importance of a
nonlinear interaction of the level of oil prices and the volatility of oil prices in determining the
development lag,’ a quadratic term for price (P2) and an interaction term for price and volatility
(P*VARDP) are also included. Note that in contrast with the two previous examples, we have
dispensed with the use of temporal interaction terms that misspecify the model. The purpose of
this example is to show that imposing a common shape parameter can diminish the significance
of the regressors used to specify the scale parameter. With the proportional Weibull specification,
only the coefficient of SIZE is significant at an α level less than 10 percent. The estimate of the
common shape parameter of 1.52266 is also significantly positive at conventional levels.
4See Favero et al. (1994) for details.
12
-
The second block of 3 columns in Table (6) present estimates of a nonproportional hazard
model where both the scale and shape parameters are specified using the same set of regressors as in
the proportional model. In this case, the coefficients of price, price squared, and the price-volatility
interaction term, are all significant at α levels less that 10 percent in both the scale and shape
parameters. The null hypothesis of proportionality is easily rejected at conventional levels. The
p-value of the LR test statistic is less than 0.00001.
Even with the nonlinear terms, interpretation of the coefficients is straightforward. The
marginal impact of price on either the scale or shape parameter is a linear function of price and
volatility. The marginal impact of price on the scale parameter is estimated as
∂α̂i∂Pi
= δ̂2 + δ̂3 · VARDPi + 2δ̂4 · Pi = −35.73529 + 2.57974 · VARDPi + 16.21826 · Pi
The marginal impact of price on the scale parameter is increasing in both price and volatility. The
marginal impact of price on the shape parameter is estimated as
∂β̂i∂Pi
= γ̂2 + γ̂3 · VARDPi + 2γ̂4 · Pi = 7.68352− 0.681759 · VARDPi − 3.36374 · Pi
The marginal impact of price on the shape parameter is decreasing in price and volatility. An
increase in either price or volatility twists the temporal profile of the hazard function, resulting in
a higher initial hazard rate, but one that diminishes at a faster rate as time passes. The interesting
aspect of this example is that by imposing a common shape parameter, the significance of the price
and volatility measures is washed out. Only when a nonproportional model is estimated is the
importance of these variables revealed.
VI. Conclusions
A wide class of nonproportional hazard models may be estimated by making the shape
parameter of the Weibull hazard a linear function of observable regressors. The log-likelihood
function for these models is well behaved. Any root to the score equations is a unique global MLE.
The regressors used to specify the shape parameter can even be the same set used to specify the scale
parameter. The only potential limitation introduced by using the same regressors to specify both
parameters is the risk of collinearity when the log of duration can be predicted with a high degree
of accuracy by the regressors involved. The conditions under which this generalization is useful are
essentially the same conditions under which interaction terms are useful in classical regression.
13
-
Interpretation of the coefficients is straightforward. For the Weibull, the log of the hazard
function is linear in the log of duration. Making the scale parameter a function of regressors, a
specification available with most econometric software, allows an observation-specific intercept in
this linear relationship. A Weibull specified in this manner is one member of the class of proportional
hazard models. Making the shape parameter a function of regressors simply allows an observation-
specific slope to the temporal profile, resulting in a well-behaved class of nonproportional hazard
functions. This generalization is a parametric specification of the hazard function with the same
benefits and limitations as other parametric specifications relative to nonparametric estimators; the
additional structure imposed by parametric models typically results in more precise estimation, but
with the risk of bias when that structure is invalid.
Finally, simply adding temporal interaction terms to the regressors that specify the scale
parameter while estimating a proportional Weibull model is a fundamental misspecification. The
standard log-likelihood function fails to recognize the stochastic nature of temporal interaction
terms. The empirical section of this paper shows that the estimates of the shape parameter that
result from this method often fall outside the parameter space of the Weibull, and that the failure to
properly constrain the sign of the shape parameter has important spill-over effects on the remaining
coefficient estimates, which often switch sign in a properly specified nonproportional model.
14
-
Table 1. Sample Moments for MLS Housing Data
Variable Mean Std. Dev.
TIME 129.2489 102.1318SOLD 0.5236 0.4994VAC 0.4292 0.4950
Table 2. Hazard Model for Housing Sales
Nonproportional Weibull Proportional Weibull
Scale Parameter (αi = Xiδ)
Coefficient Variable Coeff. Std. Err. p-value Coeff. Std. Err. p-value
δ1 CON -4.76784 0.55907 0.00001 -7.40270 0.39222 0.00001δ2 VAC -2.75976 0.98243 0.00540 7.69401 1.54744 0.00001δ3 VAC*LOGT -1.56223 0.31023 0.00001
Shape Parameter (βi = Ziγ)
Coefficient Variable Coeff. Std. Err. p-value Coeff. Std. Err. p-value
γ1 CON 0.86714 0.10700 0.00001 1.39351 0.07123 0.00001γ2 VAC 0.49377 0.18291 0.00746
LR Test: Proportionality 9.1226H0:γ2=0 χ
2(1) p-value = 0.0025
15
-
Table 3. Sample Moments for Settlement Data
Variable Mean Std. Dev.
TIME 425.3323 377.8770SETTLED 0.5045 0.5000DAMAGES 3.3381 2.2264BRITISH 0.0826 0.2752
Table 4. Hazard Model for Settlement of Litigation
Nonproportional Weibull Proportional Weibull
Scale Parameter (αi = Xiδ)
Coefficient Variable Coeff. Std. Err. p-value Coeff. Std. Err. p-value
δ1 CON -6.46372 0.11774 0.00001 -10.76187 0.07223 0.00001δ2 DAMAGES -0.57630 0.03185 0.00001 1.80398 0.02116 0.00001δ3 BRITISH -0.57543 0.22390 0.01018 6.00072 0.14414 0.00001δ4 DAMAGES*LOGT -0.31109 0.00361 0.00001δ5 BRITISH*LOGT -0.90866 0.02582 0.00001
Shape Parameter (βi = Ziγ)
Coefficient Variable Coeff. Std. Err. p-value Coeff. Std. Err. p-value
γ1 CON 1.02326 0.01741 0.00001 1.67762 0.00987 0.00001γ2 DAMAGES 0.06605 0.00469 0.00001γ3 BRITISH 0.13559 0.03260 0.00003
LR Test: Proportionality 244.9572H0:γ2 = γ3=0 χ
2(2) p-value = 0.00001
16
-
Table 5. Sample Moments for Oil Field Investment Data
Variable Mean Std. Dev.
TIME 63.0189 55.2287SIZE 328.8868 590.9644WATERD 126.5283 28.0608GASRES 178.7925 485.3050P 2.0296 0.7818VARDP 1.5438 0.7605
Table 6. Hazard Model for Oil Field Investment
Proportional Weibull Nonproportional Weibull
Scale Parameter (αi = Xiδ)
Coefficient Variable Coeff. Std. Err. p-value Coeff. Std. Err. p-value
δ1 CON -5.29150 2.80579 0.06577 21.79120 10.53594 0.04529δ2 P -0.78067 3.43764 0.82138 -35.73529 12.99638 0.00900δ3 P*VARDP -0.30119 0.27238 0.27472 2.57974 1.15459 0.03126δ4 P
2 0.39617 0.80497 0.62500 8.10913 3.04035 0.01108δ5 SIZE 0.00113 0.00049 0.02560 -0.00007 0.00256 0.97927δ6 WATERD -0.00476 0.00700 0.49946 -0.02103 0.03427 0.54301δ7 GASRES 0.00002 0.00075 0.97677 0.00001 0.00473 0.99782
Shape Parameter (βi = Ziγ)
Coefficient Variable Coeff. Std. Err. p-value Coeff. Std. Err. p-value
γ1 CON 1.52266 0.18481 0.00001 -4.29491 2.99665 0.15976γ2 P 7.68352 3.56912 0.03758γ3 P*VARDP -0.681759 0.34284 0.05381γ4 P
2 -1.68187 0.84440 0.05343γ5 SIZE 0.00052 0.00100 0.60712γ6 WATERD 0.00345 0.00864 0.69224γ7 GASRES 0.00003 0.00146 0.98619
LR Test: Proportionality 15.965H0:γ2 = · · · = γ7=0 χ2(6) p-value = 0.00001
17
-
References
Bebchuk, L. A. (1984) Litigation and settlement under imperfect information, The Rand Journal
of Economics, 15, 404–415.
Box-Steffensmeier, J. M. and Zorn, C. J. W. (2001) Duration models and proportional hazards in
political science, American Journal of Political Science, 45, 972–988.
Emons, W. and Sheldon, G. (2009) The market for used cars: New evidence of the lemons phenom-
enon, Applied Economics, 41, 2867–2885.
Favero, C. A., Pesaran, M. H. P. and Sharma, S. (1994) A duration model of irreversible oil invest-
ment: Theory and empirical evidence, Journal of Applied Econometrics, 9, S95–S112.
Fournier, G. M. and Zuehlke, T. W. (1996) The timing of out-of-court settlements, The RAND
Journal of Economics, 27, 310–321.
Giles, D. E. (2007) Survival of the hippest: Life at the top of the hot 100, Applied Economics, 39,
1877 – 1887.
Hernandez, A. and Dresdner, J. (2010) The effect of temporal closures and individual quotas on
fishing trip duration: A hazard function analysis, Applied Economics, 42, 3767 – 3776.
Keele, L. (2010) Proportionally difficult: Testing for nonproportional hazards in cox models, Polit-
ical Analysis, 18, 189–205.
Kumazawa, R. (2010) Promotion speed and its effect on attrition of navy-enlisted personnel: Ad-
dressing heterogeneity in high school credentials, Applied Economics, 42, 2563 – 2576.
Laird, N. (1978) Nonparametric maximum likelihood estimation of a mixing distribution, Journal
of the American Statistical Association, 73, 805–811.
Lancaster, T. (1979) Econometric methods for the duration of unemployment, Econometrica, 47,
939–956.
Lancaster, T. (1990) The Econometric Analysis of Transition Data, Econometric Society Mono-
graphs, Cambridge University Press.
Lentz, R. and Tranaes, T. (2005) Job search and savings: Wealth effects and duration dependence,
Journal of Labor Economics, 23, 467–489.
Lindsay, B. (1981) Properties of the maximum likelihood estimator of a mixing distribution, in
Statistical Distributions in Scientific Work: Vol. 5 (Eds.) C. Taillie, G. Patil and B. Baldessari,
Kluwer Academic Publishers, pp. 95–109.
Spier, K. E. (1992) The dynamics of pretrial negotiation, Review of Economic Studies, 59, 93–108.
18
-
Yamaguchi, K. (1991) Event History Analysis, Sage Publications.
Zuehlke, T. W. (1987) Duration dependence in the housing market, Review of Economics and
Statistics, 69, 701–709.
19
AbstractI. IntroductionII. Proportional Weibull Hazard ModelsIII. A Class of Nonproportional Weibull ModelsIV. Temporal Interaction Terms in Proportional Weibull ModelsV. ApplicationsVI. ConclusionsReferences