The cost of uniqueness in groundwater model calibration

Advances in Water Resources 29 (2006) 605–623

www.elsevier.com/locate/advwatres

The cost of uniqueness in groundwater model calibration

Catherine Moore *, John Doherty

School of Engineering, University of Queensland, St. Lucia, Queensland 4072, Australia

Received 25 February 2004; received in revised form 13 June 2005; accepted 7 July 2005Available online 29 August 2005

Abstract

Calibration of a groundwater model requires that hydraulic properties be estimated throughout a model domain. This generallyconstitutes an underdetermined inverse problem, for which a solution can only be found when some kind of regularization device isincluded in the inversion process. Inclusion of regularization in the calibration process can be implicit, for example through the useof zones of constant parameter value, or explicit, for example through solution of a constrained minimization problem in whichparameters are made to respect preferred values, or preferred relationships, to the degree necessary for a unique solution to beobtained.

The ‘‘cost of uniqueness’’ is this: no matter which regularization methodology is employed, the inevitable consequence of its use isa loss of detail in the calibrated field. This, in turn, can lead to erroneous predictions made by a model that is ostensibly ‘‘well cal-ibrated’’. Information made available as a by-product of the regularized inversion process allows the reasons for this loss of detail tobe better understood. In particular, it is easily demonstrated that the estimated value for an hydraulic property at any point within amodel domain is, in fact, a weighted average of the true hydraulic property over a much larger area. This averaging process causesloss of resolution in the estimated field. Where hydraulic conductivity is the hydraulic property being estimated, high averagingweights exist in areas that are strategically disposed with respect to measurement wells, while other areas may contribute very littleto the estimated hydraulic conductivity at any point within the model domain, this possibly making the detection of hydraulic con-ductivity anomalies in these latter areas almost impossible. A study of the post-calibration parameter field covariance matrix allowsfurther insights into the loss of system detail incurred through the calibration process to be gained. A comparison of pre- and post-calibration parameter covariance matrices shows that the latter often possess a much smaller spectral bandwidth than the former. Itis also demonstrated that, as an inevitable consequence of the fact that a calibrated model cannot replicate every detail of the truesystem, model-to-measurement residuals can show a high degree of spatial correlation, a fact which must be taken into accountwhen assessing these residuals either qualitatively, or quantitatively in the exploration of model predictive uncertainty.

These principles are demonstrated using a synthetic case in which spatial parameter definition is based on pilot points, andcalibration is implemented using both zones of piecewise constancy and constrained minimization regularization.� 2005 Elsevier Ltd. All rights reserved.

Keywords: Regularization; Model calibration; Pilot points; Resolution matrix; PEST

1. Introduction

Model calibration is the process whereby modelparameters are adjusted until a satisfactory fit between

0309-1708/$ - see front matter � 2005 Elsevier Ltd. All rights reserved.doi:10.1016/j.advwatres.2005.07.003

* Corresponding author.E-mail addresses: [email protected] (C. Moore), jdo-

[email protected] (J. Doherty).

model outputs and field measurements is achieved. Inmost cases the calibration process seeks to infer a uniqueparameter set, which can then be used by the model inmaking predictions of future system behavior. Howeverin all but the most homogenous of geological environ-ments, solution of the groundwater inverse problem isinherently nonunique, as model parameters representingthe hydraulic properties of a heterogeneous and complex‘‘real world’’ must be inferred from only a handful of

mailto:[email protected]



606 C. Moore, J. Doherty / Advances in Water Resources 29 (2006) 605–623

measurements made at discrete points, contaminatedwith measurement ‘‘noise’’.

Notwithstanding the inherent complexity of ground-water systems, a unique solution to the inverse problemcan generally be obtained by introducing additional‘‘simplifying’’ information to the calibration process.Traditionally this additional information takes the formof a user-supplied zonation pattern, in which the modeldomain is subdivided into a limited number of regions ofconstant parameter value. The spatial zonation patternmay, or may not, be inferable from geological mapping,and is normally used to represent medium to large-scalevariability in hydraulic properties. Alternatively, addi-tional information can be introduced through the inclu-sion of mathematical regularization in the parameterestimation process. If regularized inversion is imple-mented using a so-called ‘‘Tikhonov scheme’’ this extrainformation can take the form of some ‘‘preferredparameter condition’’ (for example a smoothness orhomogeneity condition), from which departures will betolerated only to the extent that they are demanded bythe calibration dataset. The end product of regularizedinversion can be considered to be a parameter field thatreflects, to the greatest extent possible, any informationpertaining to spatial hydraulic property detail that isavailable in the calibration dataset. However whereinformation is lacking, the preferred parameter condi-tion prevails, this being a necessity if a unique solutionto the inverse problem is to be attained. Alternatively‘‘subspace methods’’ such as ‘‘truncated singular valuedecomposition’’ can be employed for regularization pur-poses. Using these methods a stable solution to theparameter estimation problem is sought within an ‘‘esti-mable subspace’’ of parameter space normally associ-ated with parameter variations of low spatialfrequency. Estimation of the high spatial frequencyparameter components is often compromised by systemnoise, or even rendered numerically intractable due tosingularity of the inverse problem when attempts aremade to estimate parameters to this level of detail.Though the discussion in the present paper is limitedto Tikhonov regularization, results documented hereinare just as applicable to model calibration contexts inwhich regularized inversion is implemented using sub-space methods.

While calibration based on parameter zonation orregularization allows the estimation of a unique param-eter set, both of these methods yield a vastly simplifiedpicture of the real world. The extent to which such sim-plification is necessary to achieve a unique solution tothe inverse problems depends on the information con-tent (or lack of information content) of the calibrationdataset. Whether this simplification matters or not de-pends on the types of predictions the model will be re-quired to make. Where a model prediction is sensitiveto spatial hydraulic property detail, this prediction

may be seriously flawed. Contaminant transport is anobvious example of a process that is highly sensitive tohydraulic property detail.

The smoothing or averaging that is an inherent partof model calibration has been the subject of a numberof theoretical and practical studies. For example Backusand Gilbert [5–7], Jackson [34], Menke [43], Lines andTreitel [38], Vasco et al. [59] (to mention just a few) inthe geophysical context, and Kitanidis [35], Guadagniniand Neuman [28], Yeh and Liu [64], Clemo et al. [19],Gorokhovski [27], Vasco et al. [58] and Butler et al.[11] in the groundwater (particularly tomography) con-text, all discuss the ability (or lack thereof) of the cali-bration process to infer spatial detail of earthproperties from a set of continuous or point-based mea-surements. The literature on this subject is not confinedto earth sciences, for it is a commonly recurring themewherever imaging (for example medical imaging) isundertaken; see for example Lutkenhoner and Menen-dez [39], Menendez et al. [42] and Babiloni et al. [4].All of these studies demonstrate that the interpretationof measurements made externally to a system, or at anumber of discrete points within a system, leads to spa-tially integrated system properties, with the integrationkernel being dependent on the disposition and spatialdensity of field measurements.

Regularized inversion presents a powerful means ofrapidly fitting model outputs to field data. There is alsoa wealth of information available as a by-product of theregularized inversion process which can be used to gainimportant insights into the limitations of the calibrationprocess in inferring details of a particular groundwatersystem. These limitations exist whether or not formalregularization is employed in the calibration process.In fact, because estimation of zone-based parameterscan be easily recast as a regularized inversion problem(as is demonstrated in this paper), the same methodsof analysis can be used in conjunction with traditionalcalibration methodologies. Hence the conclusions drawnin this study on the limitations of the calibration processare as applicable to traditional zone-based model cali-bration as they are to formal regularized inversion.

The purpose of this paper is to examine the use ofregularized inversion in the context of groundwatermodel calibration. Regularized inversion has been dis-cussed in the groundwater literature prior to this paper(see for example Doherty [22] and Mantoglou [41]).The use of regularized inversion in the interpretationof densely-sampled groundwater data at the site scalehas also been documented; see for example Vasco andcoworkers [57,58,60] and Clemo et al. [19]. However lit-erature reports of its application, of the benefits to begained through its use, and of the information availableas a by-product of its deployment, in cases that can beconsidered typical of those encountered in everydaygroundwater modeling work are, to our knowledge,

C. Moore, J. Doherty / Advances in Water Resources 29 (2006) 605–623 607

rare. Given the increasing use of regularized inversion ingroundwater model calibration as a result of its readyavailability in public domain and commercial ground-water modeling packages, further discussion of thesematters is, in our opinion, urgently needed. Further-more, whether calibration is achieved through regularizedinversion, or using tradition ‘‘manual’’ regularizationdevices such as zones of piecewise constancy, it is ourimpression that there is insufficient appreciation in thegroundwater modeling community of the relationshipbetween true hydraulic properties and those estimatedthrough the calibration process. This paper attemptsto address these issues by presenting some of the theoryof regularized inversion, and some extensions of thistheory as it applies to groundwater model calibration.Through application of this theory to a synthetic casestudy, some important aspects of the calibration processthat are often unappreciated in everyday modeling workare made readily apparent.

We first present a brief overview of the theory of reg-ularized inversion and demonstrate the means by whichthis theory can be applied to the task of groundwatermodel calibration. Next we describe the ‘‘resolution ma-trix’’ as a device for exploring the relationship between acalibrated hydraulic property field on the one hand, and‘‘hydraulic reality’’ on the other. The concept of param-eter resolution then forms the basis of the discussionsand illustrations that follow. In particular:

(1) Resolution analysis is employed to compare theproperties of parameters estimated on the basisof traditional zone based calibration methods withthose estimated through regularized inversion.

(2) The concept of parameter resolution is extended tothat of the ‘‘calibrated field covariance matrix’’,this being an explicit indicator of the ability ofthe calibration process to capture (or not) thestructural fine detail existing within an aquiferwhose stochastic properties are known.

(3) The nature of ‘‘structural noise’’ as a componentof ‘‘measurement error’’, and the dependence ofthis noise on the hydraulic property heterogeneityof an aquifer is described.

(4) A methodology through which inferences of ‘‘truehydraulic property heterogeneity’’ can be drawnon the basis of a calibrated property field ispresented.

The concepts introduced above are illustrated using asynthetic model. This model is also used to demonstratethe loss of parameter detail incurred by the calibrationprocess, and the degree of predictive error that this lossof detail can potentially incur, even where a perfect fit isobtained between model outputs and field measure-ments. It is hoped that the model is simple enough forthe matters raised herein to be demonstrated in a man-

ner that is unclouded by issues unrelated to those we en-deavor to discuss, yet is complex enough to berepresentative of many real-world groundwater model-ing applications.

2. Regularized parameter estimation

2.1. Background

Classical parameter estimation theory as presented inworks such as Bard [8], Mikhail [44], Draper and Smith[24], and many others demonstrates that the number ofparameters estimated through the calibration processcannot exceed the number of observations used for theirestimation if these parameters are to be estimated un-iquely. In fact in most practical cases, the number ofparameters that can be uniquely estimated is much smal-ler than the number of observations comprising the cal-ibration dataset due to the limited information contentof most such datasets. Modelers are advised to ‘‘livewithin their means’’ when formulating an inverse prob-lem, basing that problem on a parameterization schemethat conforms to the dictates of the ‘‘principle of parsi-mony’’; see for example Hill [31], Carerra and Neuman[13], Akaike [1]. Where zones of parameter constancyare used as a method of spatial parameterization forgroundwater models, a limited number of these zonesis thus employed.

Limitations in the information content of a calibra-tion dataset with respect to the parameters that requireestimation result in parameter insensitivity and correla-tion; see for example Hill and Osterby [32]. At best thisleads to a high degree of uncertainty in estimatedparameters; at worst it leads to numerical instability inthe parameter estimation package used for numericalsolution of the inverse problem, often making the solu-tion of that problem impossible. Fortunately, statisticalinformation forthcoming from the parameter estimationprocess allows the modeler to readily identify insensitiveand correlated parameters; the parameter estimationproblem can then be reformulated to mitigate this effect,normally through reducing the number of parametersthat require estimation.

Mathematical regularization as a device for modelcalibration provides an important and useful alternativeto spatial parameterization based on a necessarily smallnumber of zones of constant parameter value. In itsbroadest terms, mathematical regularization can be con-sidered as a means of obtaining a unique solution to aninherently nonunique inverse problem, with the neces-sary level of simplification required to achieve this solu-tion being determined as part of the inversion processitself. When applied to the estimation of spatially dis-tributed parameters, it often involves the imposition ofsmoothness or equality constraints on those parameters;


see for example Tikhonov [54], Haber [29] and Englet al. [25]. While routinely used in geophysical datainterpretation (e.g. Mackie and Madden [40], Yao andRoberts [63], Nolet et al. [48], Bertete-Aguirre et al. [9]and many others including references already cited), reg-ularized inversion is rarely employed in groundwatermodel calibration, though, as stated above, it is pres-ently gaining increasing acceptance.

The principal advantage of using regularized inver-sion as a means of groundwater model calibration is thata large number of parameters can be distributedthroughout the model domain to define the spatial dis-tribution of the hydraulic property requiring estimation.To prevent the onset of numerical instability that wouldinevitably result from the estimation of so many param-eters, so-called ‘‘Tikhonov schemes’’ are often employedas a regularization device. Using such a scheme, eachparameter is accompanied by at least one ‘‘observation’’pertaining directly to that parameter (and maybe to oth-ers as well), thereby defining a ‘‘preferred condition’’ forthat parameter while at the same time ensuring thatthere are more ‘‘observations’’ than parameters presentin the inversion process. Parameter uniqueness, andhence numerical stability of the inverse problem, is guar-anteed through the assignment of appropriate weights tosuch ‘‘regularization observations’’, often through theapplication of a ‘‘weight multiplier’’ to all of them. Thisweight multiplier is calculated as part of the regularizedinversion process such that it is high enough to allowparameter uniqueness to be attained, yet not so highthat it compromises the goodness of fit between modeloutputs and field data beyond that which is requiredto prevent ‘‘over fitting’’ to measurement noise. As al-ready mentioned, other regularization methodologiesare also available, notably those based on parametersubspaces. For simplicity, these will not be discussedherein. However conclusions drawn below pertainingto loss of parameterization detail, and its effect on modelpredictive error, are just as pertinent to these schemes asthey are to Tikhonov schemes.

In the present study, regularized model calibration isundertaken in a similar manner to that described byDoherty [22], and is numerically implemented throughthe use of the PEST [23] parameter estimation package.The theory is now briefly outlined.

2.2. Basic theory

In unregularized, or ‘‘overdetermined’’ linear param-eter estimation, parameter values are calculated usingthe equation:

p ¼ ðXtQcXÞ�1XtQcc ð1Þ

where p is the vector of estimated parameters; c is thevector of measurements; X is the model Jacobian matrix(i.e. the matrix of derivatives of model outputs to which

there are observed equivalents with respect to adjustablemodel parameters, often referred to also as the ‘‘sensitiv-ity matrix’’); and Qc is the measurement weight matrix,often chosen as the inverse of the measurement covari-ance matrix which we denote by C(e), e specifying thenoise associated with the measurements c.

Eq. (1) is derived through minimization of an objec-tive function (see Eq. (4)) comprised of weightedsquared differences between model outputs and fieldmeasurements; see for example Bard [8], Mikhail [44],Draper and Smith [24] and many others. For nonlinearmodels X is a function of parameter values. Eq. (1) isthen solved by an iterative procedure in which X is re-calculated during every iteration, and in which parame-ter improvements are calculated on the basis of currentmodel-to-measurement residuals, rather than directlyfrom the measurements themselves.

If we supplement c with a set of ‘‘regularizationobservations’’ z, and if we supplement the model witha set of ‘‘regularization conditions’’ on estimated param-eters encapsulated in a matrix Z, the following equationdescribing regularized parameter estimation is obtainedthrough a simple extension of Eq. (1):

p ¼ ðXtQcXþ lZtQzZÞ�1ðXtQccþ lZtQzzÞ ð2aÞ

where Z is a matrix which describes the relationships be-tween parameters that collectively define a ‘‘preferredsystem state’’. These relationships will prevail in the ab-sence of any evidence to the contrary available throughobservations encapsulated in c; z is the vector of valuesdefining the ‘‘preferred system state’’, referred to hereinas the set of ‘‘regularization observations’’; Qz is a ‘‘reg-ularization weight matrix’’; and l is a ‘‘weight multi-plier’’ applied to all regularization observations.

For the remainder of this paper we consider onlycases for which regularization observations are all zero,and therefore for which the ZtQzz term of Eq. (2a) iszero. This involves no loss of generality, as it can easilybe achieved in most practical cases through appropriateparameter transformation and/or offsetting. With zequated to zero, Eq. (2a) can be re-written:

p ¼ ðXtQcXþ lZtQzZÞ�1XtQcc ð2bÞ

The regularization relationships encapsulated in Z maybe linear or nonlinear; in the latter case Z (which thenrepresents the derivatives of those relationships with re-spect to parameter values), must be recalculated duringevery optimization iteration. Often-used regularizationrelationships include discrete parameter equality (inwhich case Z is simply a diagonal matrix), or equalityof parameters to each other (in which case each row ofZ is comprised of a ‘‘+1’’ and a ‘‘�1’’ at elements per-taining to the respective parameters).

As well as being considered a ‘‘weight multiplier’’, itcan be shown that l is also equivalent to the reciprocal


of a Lagrange multiplier used in the solution of the con-strained minimization problem posed as follows:

Minimize Ur ¼ ðZp� zÞtQzðZp� zÞ ð3aÞ

or as z = 0 in this case,

Minimize Ur ¼ ptZtQzZp ð3bÞ

under the constraint that

Um ¼ ðXp� cÞtQcðXp� cÞ 6 Ulm ð4Þ

where Ur is referred to herein as the ‘‘regularizationobjective function’’, Um is referred to as the ‘‘measure-ment objective function’’, and Ul

m is referred to as the‘‘limiting measurement objective function’’.

As formulated through Eqs. (3) and (4), Tikhonov-based regularized inversion seeks to minimize the sumof squared outcomes of the regularization relationshipsencapsulated in Z (and hence maximize the extent towhich estimated parameters respect the preferred systemcondition encapsulated in those relationships), whileinsisting that the departure of model outputs from fieldobservations, as measured in the weighted least squaressense, is no greater than a user-defined threshold, Ul

m.This threshold should be set such that the sum ofsquared model-to-measurement residuals is commensu-rate with expected field measurement uncertainty as de-scribed by the measurement covariance matrix C(e); thus‘‘over fitting’’ or ‘‘under fitting’’ is prevented. In imple-menting the constrained minimization problem de-scribed by Eqs. (3) and (4), a new weight factor lmust be computed during each optimization iteration.In PEST, this is done using an iterative procedure basedon local linearization of the constrained optimizationproblem; see Doherty [23] for details.

2.3. The resolution matrix

With appropriate choice of l, Eq. (2) provides a un-ique solution to the constrained minimization problemdescribed by Eqs. (3) and (4). However it does not pro-vide a unique solution for the estimated parameter set pitself, for different solutions can be obtained using differ-ent regularization operators Z. Some regularizationoperators are more appropriate for use in some circum-stances than others (see, for example, Portniaguine andZhdanov [50] and Vogel and Oman [62]), their appropri-ateness being judged partly on the basis of the character-istics of the estimated property field p produced as anoutcome of their deployment. One way in which thecharacteristics of the estimated property field can be ex-plored, and in which different regularization methodolo-gies can be compared, is through the so-called‘‘resolution matrix’’, which will now be defined.

The relationship between the set of field measure-ments c and the true hydraulic property field p is de-scribed by the equation:

c ¼ Xpþ e ð5Þwhere e is a ‘‘noise vector’’, as described above, normallyassumed to be random with a mean of zero and a covari-ance matrix C(e). If Eq. (5) is substituted into Eq. (3) thefollowing equation is obtained:

p ¼ ðXtQcXþ lZtQzZÞ�1XtQcðXpþ eÞ ð6aÞ

or

p ¼ RpþGe ð6bÞ

where the ‘‘resolution matrix’’ R is given by theequation:

R ¼ ðXtQcXþ lZtQzZÞ�1XQcX ð7aÞ

and G is defined by:

G ¼ ðXtQcXþ lZtQzZÞ�1XtQc ð7bÞ

If measurement noise is small enough to be ignored Eq.(6) becomes simply:

p ¼ Rp ð8Þ

Thus the matrix R describes the relationship between thetrue property field p and its estimated counterpart p. Eq.(8) tells us that each element of p is the scalar product ofthe pertinent row of R with the true property vector p. Ifthe matrix R is the identity matrix (implying perfect res-olution of the estimated field), p and p would be equal.In practice the best that can be hoped for is that R isdiagonally dominant, and that the components of R

decay rapidly to the left and right of its diagonal. Tothe extent that off-diagonal elements of R are significant,the estimated hydraulic property at any point within themodel domain (for example the i�th component of p) is aweighted average of true hydraulic property values. Inmany circumstances, this averaging can occur over alarge part of the model domain. Thus the ‘‘image’’ ofp (i.e. p) produced as an outcome of the calibration pro-cess is ‘‘blurred’’, and the estimated field at any point isunlikely to be equal to the true field at that point. Thecloser that R approaches the identity matrix, the morerepresentative is the calibrated field of the true field.Conversely, the greater the bandwidth of R, the largeris the spatial extent of the ‘‘integration kernel’’ throughwhich the estimated field is derived by spatial averagingof the true field. Note that for over-determined inversionbased on Eq. (1), R is equal to I. This means that themodel calibration process yields perfect resolution ofsystem complexity under the assumption that the com-plexity of the real world is no greater than that of themodel which represents it.

2.4. The regularized parameter covariance matrix

Prior to model calibration much is often knownabout hydraulic properties prevailing within a study


area as a result of geological investigations and othersite characterization activities. This knowledge usuallytakes the form of most likely values for hydraulic prop-erties within different parts of the model domain basedon lithologic inference or pump test interpretation, to-gether with the degree of variability to be expected ofthese properties. Conceptually, pre-calibration parame-ter uncertainty can be summarized through the use ofa parameter covariance matrix C(p). If enough informa-tion is available within a study area to construct ahydraulic property variogram, C(p) can be calculatedfrom that.

Once C(p) is known (or estimated) simple matrix rela-tionships for propagation of covariance allow us to cal-culate a covariance matrix of the estimated field, i.e.C(p), from Eqs. (6) as

CðpÞ ¼ ðXtQcXþ lZtQzZÞ�1XtQcXCðpÞ

� XtQcXðXtQcXþ lZtQzZÞ�1

þ ðXtQcXþ lZtQzZÞ�1XtCðeÞ

� XðXtQcXþ lZtQzZÞ�1 ð9aÞ

where, for simplicity, C(e), the covariance matrix ofmeasurement noise, is assumed to be the inverse of Qc.Note that derivation of Eq. (9a) from Eq. (6) assumesthat no correlation exists between measurement noiseand hydraulic property values in different parts of themodel domain.

If measurement noise is neglected Eq. (9a) becomes:

CðpÞ ¼ ðXtQcXþ lZtQzZÞ�1XtQcXCðpÞ

� XtQcXðXtQcXþ lZtQzZÞ�1 ð9bÞ

A short example may improve understanding of the C(p)matrix. Suppose that a thousand different realisations ofa hydraulic conductivity field are generated within amodel domain based on the same C(p). Suppose thateach one of these hydraulic conductivity fields is thenused to parameterise a model, which is then run in orderto generate a set of outputs. Suppose now that each ofthese outputs is sampled at identical locations, and thatregularized inversion (using the same inversion method-ology in each case) is then undertaken based on eachsynthetic measurement set. The result would be onethousand estimated hydraulic property fields, the statis-tical properties of which would be approximated by C(p)(‘‘approximated’’ because the model is not linear, andbecause even if it were, identical values of the weight fac-tor l would not be calculated in each case). The diago-nal elements of C(p) describe variability of individualparameters between these fields, that is, parameter vari-ability at any one point in the model domain over thenotional one thousand calibration exercises. The off-diagonal elements represent the degree to which fieldestimates at different points within the model domainare statistically related.

2.5. Spectral decomposition of the true

and estimated fields

In many instances of groundwater model calibration,the matrix X does not have full column rank becauseeither the number of parameters exceeds the numberof observations, and/or because the information contentof the calibration dataset is insufficient to provide un-ique estimation of parameters pertaining to all of the de-tail represented in the model. The matrix XtQcX of Eqs.(9a) and (9b) is thus rank-deficient (Koch [36] p. 28),having a maximum rank equal to the number of obser-vations (i.e. the dimension of c), though often consider-ably less than this. The matrix C(p) calculated by thoseequations is thus rank-deficient, and is thereforesingular.

Notwithstanding its singularity, C(p) is a symmetric,positive semidefinite matrix. As such it has real eigen-values, with the number of non-zero eigenvalues beingequal to its rank. These non-zero eigenvalues, and corre-sponding eigenvectors, can easily be determined throughsingular value decomposition. As a result of such ananalysis, the stochastic character of the estimatedparameter field p can be represented by independentrandom variations in directions which are mutuallyorthogonal in parameter space. These directions arethose of the eigenvectors of C(p) that correspond to itsnon-zero eigenvalues; the variance, or stochastic vari-ability, in each of these directions is given by the corre-sponding eigenvalue.

Similarly, eigenvalue/eigenvector decomposition ofC(p), the spatial covariance matrix of the true field, re-sults in a set of independent, uncorrelated random vari-ables pertaining to orthogonal directions in parameterspace (i.e. the directions of the eigenvectors of C(p)).The variance in each such direction is equal to the cor-responding eigenvalue. A comparison of the eigen-values/eigenvectors of C(p) with those of C(p) canreveal much about the loss of detail incurred throughthe calibration process as will be demonstrated for asynthetic example later in this paper.

Note that discussions of covariance matrix decompo-sition in relation to inversion problems in the stochasticframework can be found in Braud and Obled [10] andDelay et al. [21].

2.6. Inference of true hydraulic property variability

Eqs. (9) relate the spatial variability of an estimatedhydraulic property field to that of a true hydraulic prop-erty field. Consequently they provide a mechanism fortesting plausibility of a posited true field covariance ma-trix C(p), once an estimated hydraulic property field hasbeen obtained through regularized inversion.

To test the validity of a hypothesized C(p), that C(p)can be inserted into the right side of either of Eqs. (9).


The hypothesis that C(p) represents a true descriptor ofhydraulic property variability within the model domaincan then be accepted or rejected at a certain level of con-fidence by testing the compatibility of the estimated fieldp derived using Eqs. (2), with C(p) calculated throughEqs. (9). A testing procedure is now described.

For the ith non-zero eigenvalue mi of C(p), define gi as

gi ¼ ptei=m1=2i ð10Þ

where ei is the eigenvector of unit length correspondingto mi. If C(p) as calculated by Eqs. (9) properly character-izes the spatial covariance structure of p, then each gishould have a variance of 1, and will be statisticallyindependent from all other gi. Furthermore, if C(p) ismulti-Gaussian and if model outputs are reasonably lin-ear with respect to parameters, then the gi should benormally distributed, and possess a mean of zero. Thusthe sum of all squared gi values corresponding to non-zero eigenvalues of C(p) should belong to a chi-squaredistribution with order equal to the number of elementssummed. In practice, not all of these eigenvalues can beused for testing chi-square behaviour due to amplifica-tion of numerical noise in directions of low spatial var-iability as a result of near-zero denominators in Eq. (10)corresponding to low eigenvalues. Hence the summationmust be restricted to a sufficient number of terms to ac-count for most of the variability of C(p) as calculatedthrough summing eigenvalues, without contaminatingthis sum through the inclusion of noise.

(It is worth noting in passing that rarely, if ever, isreal-world heterogeneity characterized by a multi-Gaussian stochastic structure. The analysis aboveshould, however, in many circumstances provide someinsights into the level of ‘‘uncaptured heterogeneity’’prevailing in the real hydraulic property field; in mostcircumstances such insights are likely to be qualitativerather than quantitative.)

2.7. Zones as a regularization device

As has already been discussed, in many groundwatermodeling applications, zones of constant parametervalue are used to represent geological heterogeneity. Afundamental distinction between parameters based ona limited number of zones, and the use of regularizedinversion based on a large number of parameters is thenumber of degrees of freedom that exist in the spaceof estimated parameters under these two different cali-bration methodologies. As was discussed above, whereparameters outnumber observations in regularizedinversion, the number of degrees of freedom in parame-ter space from which the calibrated field can be con-structed is potentially as high as the number ofobservations (though it may be considerably less thanthis if the information provided by different observa-tions is not distinct or is shrouded in measurement

noise). Furthermore regularized inversion allows theparameter estimation process to automatically place het-erogeneity where it is most needed. In contrast, whereregularization is achieved by restricting the calibrationprocess to the estimation of parameters pertaining to alimited number of zones of piecewise constancy, thenumber of degrees of freedom that is available in param-eter space is equal to the number of zones. This placesrestrictions on the fit that can be achieved between mod-el outcomes and field data, and potentially suppressessome of the information residing in field measurements.

Through a slight expansion of the concepts presentedso far, the relationship between estimated and truehydraulic properties when zones of piecewise constancyare employed as a pre-calibration regularization devicecan be explored using a ‘‘zonal resolution matrix’’. Sup-pose that zonal parameters are represented by the vectors whose dimensionality is low enough to ensure that aunique solution to the inverse problem is obtained usingEq. (1); that is

s ¼ ðYtQcYÞ�1YtQcc ð11Þ

which is identical to Eq. (1) except for the fact that Y isused instead of X, where Y is the Jacobian matrix as itpertains to the limited number of lumped parametersencapsulated in s, and thus has far fewer columns thanX, but has the same number of rows. If Eq. (5) is nowsubstituted for c we obtain:

s ¼ ðYtQcYÞ�1YtQcXpþ ðYtQYÞ�1

Qce ð12aÞThat is

s ¼ R0pþG0e ð12bÞwhere

R0 ¼ ðYtQcYÞ�1YtQcX ð13Þ

and

G0 ¼ ðYtQcYÞ�1YtQc ð14Þ

It is apparent that each row of the matrix R 0 representsthe ‘‘integration kernel’’ or ‘‘averaging weights’’ bywhich the elements of the lumped parameters s areobtained from the ‘‘true’’ hydraulic properties p.

2.8. Spatial correlation of structural noise

Suppose that the presence of measurement noise dic-tates that a less than perfect fit be sought between modeloutputs and field measurements through the regularizedinversion process. The target measurement objectivefunction Ul

m of Eq. (4) should thus be set at a value thatreflects the level of expected measurement noise. FromEqs. (5) and (6):

c ¼ Xpþ e ¼ Xp� Xpþ Xpþ e

¼ Xpþ XðI� RÞpþ ðI� XGÞe ð15Þ


The last two terms of Eq. (15) define model-to-measure-ment misfit of the calibrated model (i.e. residuals). Thelast of these is present even where calibration is effectedusing over-determined inversion based on Eq. (1). Themiddle term, however, appears because of the fact thatmodel calibration is undertaken using regularized inver-sion; its disappearance in the overdetermined case (inwhich the system is assumed to be completely character-ized by the limited number of parameters employed bythe model) is readily demonstrated by setting R in thisequation to I. Thus this term is a ‘‘structural noise’’ termthat describes the action of the model operator X onparameterization detail unrepresented in the calibratedmodel; that is, it represents the action of X on p � p.

Where regularization is based on pre-calibrationparameter lumping a similar equation can be derived.In that case we again start with the equation:

c ¼ Xpþ e

but through substitution of Eq. (12) we obtain:

c ¼ Ysþ ðX� YR0Þpþ ðI� YG0Þe ð16ÞOnce again, structural noise is represented by the secondterm on the right side of this equation. It expresses thedifference in system states c calculated using the ‘‘truemodel’’ X on the one hand and its lumped equivalentY on the other hand, acting on the ‘‘true parameters’’p in the first case and their lumped equivalents R 0p inthe second case.

Measurement noise e is often assumed to be uncorre-lated. However structural noise does indeed show spatialcorrelation; therefore so too will calibration residuals.To demonstrate this, we first represent the structuralnoise term by d using the equation:

d ¼ XðI� RÞp ð17aÞor

d ¼ ðX� YR0Þp ð17bÞdepending on whether regularization is undertakenthrough the regularized inversion process itself, orthrough pre-emptive parameter zonation. The covari-ance matrix of structural noise is then easily calculatedas

CðdÞ ¼ XðI� RÞCðpÞðI� RÞtXt ð18aÞor

CðdÞ ¼ ðX� ZR0ÞCðpÞðX� ZR0Þt ð18bÞIt is apparent from Eqs. (18) that C(d) is a function ofC(p) and thus depends on the level of heterogeneity pres-ent (though not necessarily capturable) in the real-worldsystem that is simulated by the model. Furthermore,C(d) will not, in general, be a diagonal matrix; thusstructural noise is likely to be spatially correlated.

This short analysis demonstrates an important,though seldom appreciated, property of model-to-mea-

surement residuals. Conventional wisdom as applied togroundwater model calibration leads us to seek, at theend of a calibration exercise, model-to-measurementresiduals that are uncorrelated in space, see Andersonand Woessner [3] and modelling guidelines such asSTOWA [53] and Murray–Darling Basin Commission[47]. However Eqs. (18) inform us that model-to-mea-surement residuals will, in fact, have a spatial correla-tion structure that depends on the level and type ofheterogeneity prevailing in the true property field, andon the regularization process employed to achieve aunique solution to the inverse problem. An example ofspatial correlation of model-to-measurement residualsin a zone-based model will be presented later in thispaper.

The fact that model-to-measurement residuals arelikely to show spatial correlation can have importantrepercussions when attempts are made to quantify theuncertainty of parameters, and/or of model predictions,using methods such as that proposed by Poeter and Hill[49] and Hill [31] (based on an assumed linear relation-ship between model parameters and model outputs) orby Vecchia and Cooley [61], Christensen and Cooley[15,16] in which no such linearity assumption is invoked.In all of these cases, the calculated confidence intervalsof model parameters and/or predictions are based onthe assumed probability structure of model-to-measure-ment noise; in most cases statistical independence of thisnoise is assumed. However where model-to-measure-ment misfit is principally an outcome of ‘‘model struc-tural error’’ (as it usually is), neglecting the spatiallycorrelated nature of this term invalidates calculation ofparameter and prediction uncertainty. This has beenrecognised by Cooley [20], Christensen and Cooley[17], and Christensen et al. [18] who have explicitly in-cluded C(d) in their analysis of parameter and modelpredictive error variance.

3. Synthetic case

3.1. Description

Fig. 1a shows the domain of a groundwater model. Itis rectangular, with dimensions 500 m by 800 m andthickness 10 m. Along the top of the model is an inflowboundary, with water entering the system at a rate of0.1 m3 per day per metre of boundary. Along the bottomof the domain is a fixed head boundary, with the headset at 0.0 m uniformly along this boundary. There isno recharge within the model domain itself. The simu-lated aquifer is confined and conditions are steady state.

An hydraulic conductivity field was synthesised usinga multi-Gaussian field generator based on a log-expo-nential variogram. The equation for this type of vario-gram is given by:

Fig. 1. (a) Model domain and grid, 104 pilot point locations (crosses), and 12 observation well locations (circles), (b) ‘‘true’’ hydraulic conductivityfield, (c) piezometric contours (solid lines) and particle path (dotted line).


cðhÞ ¼ bð1� e�ahÞ ð19Þwhere b is the sill, and the coefficient a is often consideredto be about one third of the effective range of the vario-gram. In the present case b was assigned a value of 0.2while a was assigned a value of 200 m; an average logconductivity value of 0.0 (corresponding to an hydraulicconductivity of 1.0 m/day) was employed. This conduc-tivity distribution was used as the basis for generationof ‘‘field data’’ employed in the calibration exercisesdocumented below. In the discussion that follows, thisis referred to as the ‘‘true’’ hydraulic conductivity distri-bution for the study area; it is depicted in Fig. 1b.

Groundwater flow was simulated using MODFLOW2000 [30] using a finite difference grid comprised of 4000cells, each 10 m square. Particle movement was simu-lated using the MODFLOW 2000 ADV package [2].

Fig. 1a shows the locations of 12 observation wells(as circles), evenly distributed throughout the model do-main. For many of the calibration exercises documentedin this paper, the heads at these 12 wells comprised thecalibration dataset. In other exercises the heads in agreater number of wells were used for model calibration.

Fig. 1c shows piezometric contours throughout themodel domain calculated by MODFLOW on the basisof the true hydraulic conductivity field shown inFig. 1b. Also shown is the MODFLOW-calculated trackof a particle released into the model domain 100 m southof the upper inflow boundary. Time of travel to the low-er fixed-head boundary is calculated as 3256 days; this isreferred to as the ‘‘true’’ particle travel time.

3.2. Parameter estimation

For the purpose of model calibration, pilot pointswere employed to represent spatial hydraulic conductiv-

ity variation within the groundwater model domain.Using this scheme, each element of p represents the logof hydraulic conductivity at the location of a pilot point.During the inversion process, hydraulic conductivity isestimated at each of these locations; hydraulic conduc-tivities for all of the cells of the model grid are then cal-culated through spatial interpolation (using kriging inthe present case) from this set of pilot points. The useof pilot points as a device for groundwater modelparameterization dates back to Certes and de Marsily[14]. However their use in conjunction with regularizedinversion (which allows deployment of a far greaternumber of pilot points than would otherwise be possi-ble) was first demonstrated by Doherty [22]. In the pres-ent study the estimated hydraulic conductivitydistribution was characterized using 104 pilot points;these are shown as crosses in Fig. 1a. It is readily admit-ted that the use of pilot points, like the use of zones,introduces a certain degree of ‘‘pre-emptive averaging’’into the estimation of hydraulic properties undertakenthrough the calibration process; the smaller the numberof pilot points employed, the greater the degree of thisaveraging. However this effect is ignored in the presentstudy as the use of 104 pilot points is sufficient to ensurethat the conclusions drawn herein are minimally affectedby pilot-point induced parameter averaging [46].

Though many options exist for the Z matrix of Eqs.(2), only two are employed in this study. The first option(implementing what is referred to herein as ‘‘smoothingregularization’’) is the same as that described by Doh-erty [22]. The basis of this scheme is that the differencebetween the logs of hydraulic conductivity values atpairs of pilot points is ‘‘observed’’ to be zero. Becauseeach row of the Z matrix of Eq. (2) thus constitutesa relationship in which the difference between two


parameter values is taken, each such row contains onlytwo non-zero elements, these being ‘‘+1’’ and ‘‘�1’’. Adiagonal Qz matrix was employed, the magnitude ofeach diagonal element being greater for pilot pointswhich are close together than for pilot points whichare further apart; this weight was calculated as thesquare of the inverse of the hydraulic conductivity vari-ogram value for the respective pilot point separation dis-tance. This regularization scheme thus attempts to forcepilot points which are closer together to have similarvalues (unless this is contradicted by the calibrationdataset), with specified weights taken from the vario-gram which is assumed to describe spatial variabilityof that hydraulic property.

For the second Z matrix option (implementing whatis referred to herein as ‘‘preferred value regularization’’),the log of the hydraulic conductivity at each pilot pointwas individually assigned a preferred value of zero (cor-responding to an actual hydraulic conductivity value of1.0 m/day). While this preferred value is obviously suit-able for the synthetic model used in this study given thefact that generation of the synthetic hydraulic conduc-tivity field used to represent ‘‘reality’’ was based on amean log-value of zero, it is important to note that this‘‘preferred value’’ regularization scheme is readily ex-tended to more complex calibration contexts where themean hydraulic property value differs from unity. Forexample, parameter deviations from another uniformvalue, or from a ‘‘trend value’’ - possibly determinedas part of the calibration process itself - can easily beaccommodated. In the present work, the regularizationcovariance matrix Qz of Eq. (2) was selected as the in-verse of the spatial covariance matrix describing thehydraulic conductivity distribution, i.e. C�1(p); howeverthis is not essential for implementation of ‘‘preferredvalue’’ regularization.

3.3. Smoothing of hydraulic conductivity fieldsand predictive error

In order to explore the ability of the calibration pro-cess to reproduce spatial variability of hydraulic proper-ties, the model of Fig. 1 was calibrated using ‘‘observed’’heads in different numbers of wells spread throughoutthe model domain. The ‘‘preferred value’’ regularizationscheme described above was employed in this exercise.

Fig. 2a–f shows the estimated hydraulic conductivityfields resulting from 6 different calibration exercises; thelocations of wells whose heads were used in each of thesecalibration exercises are superimposed on these esti-mated fields. In all cases model-to-measurement headmisfit at all of these locations was negligible (as theyshould be, as no noise was added to the MODFLOW-calculated heads before using these heads for model cal-ibration). The grey scale used to represent the log ofhydraulic conductivity in each of these plots is the same

as that used to represent the true field in Fig. 1. Collec-tively, these figures provide a graphic illustration of therelationship between true and estimated hydraulic prop-erty fields. At no point in the model domain is thecalibrated field equal to the true field, in spite of thenear-perfect fit between model outputs and field mea-surements. More importantly, as less data is made avail-able for calibration, there is a serious reduction in theability of the calibrated field to reproduce spatial detailcontained in the true field.

This loss of ability to infer hydraulic property detailmay or may not matter, depending on how the modelis to be used. However, if the model is used to make aprediction that is dependent on system detail beyondthat which can be captured by the calibration process,then the consequences of using the calibrated field tomake that prediction can be substantial. For each ofFig. 2a–f, the particle travel time predicted using therespective calibrated field is recorded below the figure;recall that the ‘‘true’’ travel time (i.e. the travel time cal-culated on the basis of the true hydraulic conductivitydistribution) is 3256 days. It is immediately obvious thatsignificant errors are incurred when any of the ‘‘perfectlycalibrated’’ models illustrated in Fig. 2 are used to pre-dict this travel time, with this error generally increasingas data support for the calibration process is reduced.

Fig. 3a–d shows another series of estimated hydraulicconductivity fields. For each of these cases heads from84 wells were used in the calibration process (seeFig. 2a for the locations of these wells). However forthese experiments progressively larger amounts ofGaussian noise were added to the head measurementsin order to explore the effect of measurement inaccuracyon the calibration process. With the addition of Gauss-ian noise, the user-specified value for the limiting mea-surement objective function (Ul

m of Eq. (4)) wasincreased to a level commensurate with this noise. In thisway the regularized inversion process was preventedfrom ‘‘over fitting’’ the measurement data beyond thelevel of noise that was present within it; nor was it allowedto ‘‘under fit’’ the measurement data through acceptanceof an inappropriately large level of model-to-measure-ment misfit.

It is apparent from Fig. 3 that the addition of noise tothe calibration dataset has a similar effect to the progres-sive removal of data. That is, the ability of thecalibration process to infer hydraulic property detaildecreases with increasing measurement noise. An inspec-tion of predicted travel times recorded under thesefigures reveals that this loss of detail has, once again,serious consequences for the accuracy of these modelpredictions.

In order to compare predictive error introducedthrough zone-based parameterization with that intro-duced through regularized inversion, calibration of thesynthetic model was repeated using zones. Fig. 4a shows

Fig. 2. Hydraulic conductivity fields and particle exit times calculated by the model calibrated against heads measured in (a) 84 wells, (b) 42 wells,(c) 24 wells, (d) 12 wells, (e) 6 wells, (f) 1 well.

Fig. 3. Hydraulic conductivity fields and particle exit times calculated by the model calibrated against head measurements in 84 wells with (a) nonoise; (b) noise with a standard deviation of 0.05 m; (c) noise with a standard deviation of 0.1 m; (d) noise with a standard deviation of 0.2 m. Thelocations of the 84 wells are shown in Fig. 2a.


the zonation pattern chosen for this exercise; boundariesfor three (non-contiguous) zones were drawn against thebackground of the true field in an attempt to ensure that

zone boundaries coincided, as much as possible, withareas of distinctly different conductivities in the true field(a luxury not available in normal modeling practice).

Fig. 4. Model domain showing (a) disposition of zones of parameterconstancy; (b) calibrated hydraulic conductivity field based on thiszonation; particle path and measurement well locations are alsoshown.


Estimation of the three zone-based hydraulic conductiv-ities was undertaken using Eq. (1) on the basis of a cal-ibration dataset comprised of 12 measured heads withno noise added to these heads. The estimated hydraulicconductivity field is shown in Fig. 4b; the root meansquare misfit between model outputs and field measure-ments was 0.065 m, this being the minimum that couldbe achieved using this parameterization scheme. Particletravel time to the lower boundary was calculated as6365 days, demonstrating, once again, the potentialerror associated with model predictions as an outcomeof the loss of detail incurred through the calibrationprocess.

This exercise was repeated with 5 and then 8 non con-tiguous zones. While the root mean square fit improvedslightly to 0.063 m and 0.039 m respectively, the particletravel time to the lower boundary remained similar at6466 days and 6249 days respectively.

We conclude this sub-section by pointing out that itwould be wrong to draw the conclusion that travel timespredicted by calibrated models are always larger thanthose which occur in reality. Indeed, other numericalexperiments that we have undertaken using the samemodel framework, but with different ‘‘true’’ hydraulicconductivity fields, have demonstrated that the oppositecan occur just as easily. What is constant in all caseshowever is that the calibration process fails to reproducethe detail of the true field. When an estimated field isthen used to make a prediction that is as sensitive tothe existence and disposition of hydraulic property het-erogeneity as travel time predictions are, that prediction

has the potential to be considerably in error as a directresult of that loss of detail.

3.4. The resolving power of calibration

The resolution matrix was discussed in Section 2.3 ofthis paper. There it was pointed out that each row of thismatrix contains the weights by which true hydraulicproperties at different locations within the model do-main are multiplied in order to calculate the hydraulicproperty value assigned to a particular point of that do-main. In the present context, where spatial parameteri-zation is achieved through the use of pilot points, eachrow of the resolution matrix represents the weights bywhich true hydraulic properties at pilot point locationsare multiplied in order to obtain the estimated hydraulicconductivity at the pilot point pertaining to that row.(Actually, this is not strictly correct as our ‘‘true field’’was not generated on the basis of pilot points, but wasactually generated on a cell-by-cell basis by our multi-Gaussian simulator; nevertheless, because of the largenumber of pilot points used in the calibration process,this is a close enough approximation for the purposesof this discussion. Numerical experiments have demon-strated that errors incurred through the making of thisapproximation are slight; see Moore [46].)

The spatial averaging that is inherent in model cali-bration is best demonstrated if elements of the resolu-tion matrix are plotted in a spatial setting. This isdone in Fig. 5, which depicts elements of the resolutionmatrix calculated on the basis of a calibration datasetcomprised of 12 wells (shown as circles in that figure),for a number of specific pilot points. Model calibrationwas achieved using ‘‘smoothing regularization’’. Each ofthe four plots comprising this figure depicts componentsof one row of the resolution matrix; the pilot point cor-responding to each such row is marked in the figure as across. Each element of the respective row of the resolu-tion matrix is plotted at the location of its pertinent pilotpoint; components pertaining to these elements are thencontoured. Thus each of these contour maps representsthe ‘‘weighting function’’ through which true hydraulicproperties over the model domain are averaged to ob-tain the estimated hydraulic property at the pilot pointto which the map pertains.

The maps of Fig. 5 possess important features thatare relevant not just to the present calibration exercise,but to groundwater model calibration in general. Firstly,the peak value of the weighting factor is nowhere greaterthan 0.14 for these particular pilot points, providing agraphic illustration of the fact that hydraulic conductiv-ities estimated through the calibration process are repre-sentative of no one point in space, but are rather aspatial average of true hydraulic conductivities occur-ring throughout the model domain; furthermore, someof the averaging weights involved in this process are

Fig. 5. Each of these pictures depicts the elements of one row of the resolution matrix; the pilot point to which the row pertains is shown as a cross.The zero contour is enhanced.


actually negative. Secondly (and perhaps surprisingly),the largest positive contribution to the estimatedhydraulic conductivity at any particular point withinthe model domain does not necessarily come from theimmediate vicinity of that point. Rather, in the presentcase at least, it comes from the area between the nearestpair of measurement wells (in the direction of ground-water flow). Thirdly, there are many locations withinthe model domain that do not contribute significantlyto the hydraulic conductivity estimated for any pilotpoint (predominantly places that do not lie betweenpairs of measurement wells along lines of head gradient).Thus the hydraulic conductivities in these areas are sim-ply unsampled through the calibration process. Othernumerical studies that we have undertaken indicate thatlarge, and important, hydraulic property anomalies

Fig. 6. Each of these pictures depicts the elements of one row of thezonal resolution matrix; the zone to which the row pertains ishighlighted in figures (a) and (b), and the remainder of the modeldomain which comprises the third zone is shown in figure (c). The zerocontour is enhanced. (Note that for the central figure the parameterzone is non-contiguous, being comprised of three spatially separatesubzones.)

could exist in such areas, and be virtually undetectedby the calibration process.

Fig. 6 shows similar plots to those provided in Fig. 5.However these pertain to each of the three rows of the‘‘zonal resolution matrix’’ defined in Eq. (13); recall thateach row of this matrix depicts the weights by which‘‘true parameter values’’ are multiplied in formulationof zonal parameter values. It is obvious from this figurethat estimated zonal parameter values do not representaverage hydraulic properties within each respectivezone. While, for each of the zones illustrated in Fig. 6,the bulk of the contribution to the estimated zonal prop-erty originates from within the zone, the averaging ker-nel is far from uniform within the zone. Furthermore,substantial contributions are made by parts of the modeldomain existing outside the pertinent zone.

3.5. Eigen-analysis of pre- and post-calibration

parameter covariance matrices

Section 2.5 of this paper discussed eigenvalue/eigen-vector analysis as a mechanism for comparing C(p) withC(p).

Fig. 7 shows eigenvalues, arranged in order ofdecreasing value, calculated for C(p) and C(p) where cal-culation of p was achieved through regularized inversion(‘‘preferred value’’ option) based on a measurementdataset comprised of heads (with no noise added) in12 observation wells. As has been discussed, the spatialvariation of estimated hydraulic conductivity p is char-acterised by 104 pilot points, the locations of whichare shown in Fig. 1a. These same points were used tosample the true hydraulic conductivity field p for thepurpose of calculating the eigenvectors and eigenvaluesof C(p). Because this sampled p has 104 components,C(p) has 104 eigenvalues. Thus spatial variation of pcan be characterised by independent variation of 104random variables in parameter space; these individual

0.01

0.1

1

10

0 20 40 60 80 100 120

Eigenvalue number

Eig

enva

lue

Covariance matrix of true field

Covariance matrix of estimated field

Fig. 7. Eigenvalues of true and estimated covariance matrices for thecase where 12 observations are used for model calibration.


random values pertain to linear combinations of the 104elements of p corresponding to the respective eigenvec-tors of C(p). The components of the normalized eigen-vectors corresponding to the highest three eigenvaluesand the lowest three eigenvalues of C(p) are shown inFig. 8. It is readily apparent that the directions in

Fig. 8. Components of normalised eigenvectors (plotted at locationsof corresponding parameters) of the true covariance matrix corre-sponding to (a–c) the largest three eigenvalues, and (d–f) the smallestthree eigenvalues.

parameter space of highest variability for the trueparameter field correspond to broad scale variations(i.e. variations of low spatial frequency), while directionsof smallest variability correspond to small scale varia-tions (i.e. variations of high spatial frequency).

In stark contrast to C(p), the spectrum of C(p) onlyhas 12 non-zero eigenvalues, this being a direct outcomeof the fact that only 12 observations are used for modelcalibration and that C(p) is thus singular with a rank of12. Furthermore, as shown in Fig. 7, variability in C(p)decreases more rapidly with eigenvalue number than forC(p). Components of the normalized eigenvectors corre-sponding to the highest and lowest three eigenvalues ofC(p) are shown in Fig. 9.

It is once again apparent from this example that mostof the ‘‘fine detail’’ present in the true parameter field islost in the calibrated field. This results from the fact thatvariability in the estimated field is restricted to only asmany independent directions in parameter space as thereare observations; furthermore, variability in thosedirections corresponding to non-zero eigenvalues isattenuated, with increasing attenuation occurring in

Fig. 9. Components of normalised eigenvectors (plotted at locationsof corresponding parameters) of the covariance matrix of the estimatedhydraulic conductivity field based on 12 observations, correspondingto (a–c) the largest three eigenvalues and (d–f) the smallest threeeigenvalues.

100

0.2 0.4 0.6 0.8 1

200

300

400

500

80% 90% 95%

Variogram sill

Var

iogr

am 'a

' val

ue (

m)

Fig. 10. Chi-square confidence regions for true field variogram sill anda values consistent with calibrated hydraulic conductivity field ofFig. 2d.

Fig. 11. Contoured residuals for zone-based calibration; the shadedarea depicts that part of the model domain where residuals arenegative. Observation wells are shown as filled circles.


directions that reflect finer system detail. Thus, concep-tually at least, calibration can be viewed as a kind of lowpass spatial filter, in which low spatial frequencies repre-sented in the true hydraulic conductivity field are al-lowed to pass through to the estimated field, whereashigher spatial frequencies are either attenuated, or lostcompletely.

Consideration of the calibration process from a spec-tral point of view allows a sharp philosophical divide tobe drawn between calibration based on a set of user-sup-plied zones on the one hand, and calibration based on alarge number of parameters estimated through regular-ized inversion on the other. Earlier in this paper, modelcalibration based on 12 observations was undertakenusing three zones of parameter constancy. In thiszone-based calibration the fit between model outputsand field data was not as good as that achieved throughregularized inversion. Depending on the modeling cir-cumstances, this may not necessarily be a bad thing.However of larger concern when using zone-based cali-bration is the fact that the estimated parameter field canhave only as many directions of variability in parameterspace as there are zones, three in that case. In contrast,when calibration is undertaken using regularized inver-sion, the estimated field is characterized by as many de-grees of freedom as the data allows (12 in this case). It isthus apparent that regularized inversion allows a greateramount of heterogeneity information to be extractedfrom a given calibration dataset than zone based param-eterization, as use of the latter device can preclude someof this measurement information being reflected in theestimated field by placing inappropriate restrictions onthe number of degrees of freedom through which thisinformation can express itself.

3.6. Inferring the geostatistical structure of the true

hydraulic properties

For the synthetic calibration problem of Fig. 1, wetested the compatibility of a suite of possible true fieldvariograms with the estimated field depicted inFig. 2d; recall that this field was calculated using ‘‘pre-ferred value’’ regularization on the basis of 12 observa-tion wells (with no noise added to the heads in thesewells). Fig. 10 shows the results of this analysis forhypothesized variograms with sills ranging from 0.01to 1, and variogram a values ranging from 25 m to500 m, where a is the coefficient of distance in the expo-nent of Eq. (19). Our �true� field variogram (which has asill of 0.2 and an a value of 200 m) falls well within the80% confidence interval depicted in this figure.

Of note in Fig. 10 is the fact that the variability ofvariogram sills that are admissible at the 80% confidencelevel decreases to a minimum at a variogram range ofabout 100 m, and increases for variogram ranges aboveand below this value. For variogram ranges that are very

small, spatial variability of the true hydraulic propertyfield is virtually undetectable through the calibrationprocess as Fig. 7 shows; hence the amplitude of spatialvariation becomes less and less consequential as vario-gram range is reduced. For high variogram ranges, high-er sills are also possible because the variogram range iscomparable with (or larger than) the size of the modeldomain itself. The calibration process is thus able to‘‘see’’ only that variability of the true field which occurswithin the domain of the model, with the ‘‘effectiverange’’ of the variogram thus reduced to the size ofthe model domain, the ‘‘effective sill’’ of the variogramis limited to that variability which can occur within thateffective range; thus higher ‘‘true sills’’ can be compati-ble with the calibrated model.

as

-1.0 1.0

Correlation

(a) (b) (c)

Fig. 12. Residual correlation coefficients for three head measurement points, each such head measurement location being depicted with a cross.


3.7. Spatial correlation of residuals

Fig. 11 shows contoured residuals for the zone-basedcalibration of Fig. 4. Recall that the calibration datasetin this case consisted of 12 measurement heads. No noisewas added to the data; however structural noise with anrms value of 0.065 m was incurred through the use ofzones as a parameterization device. It is hard to recon-cile this picture (which typifies the spatial distributionof residuals encountered in most cases of model calibra-tion) with the notion of zero spatial correlation of resid-uals as suggested by some authors as being indicative ofa good calibration, as discussed in a previous section. Tofurther illustrate the point that spatial correlation ofresiduals is in fact to be expected, Eq. (18b) can be usedto calculate the spatial correlation structure of model-to-measurement residuals for our synthetic case. Fig. 12a–cdepicts contoured correlation coefficients between resid-uals at each of three measurement wells (these wells aremarked by a cross in each case) and all of the other wellsused in the calibration process. As expected, a correla-tion coefficient of 1 exists at the measurement well towhich each plot pertains; however the fact that correla-tion is significantly non-zero at all other measurementwell locations (being positive at some locations and neg-ative at others) is obvious from these figures.

4. Discussion and conclusions

Rightly or wrongly, most modelling exercises (oftenpursued at great expense) seek to produce a ‘‘calibratedmodel’’. Implicit in this endeavour is the pursuit of a un-ique solution to the inverse problem of model calibra-

tion, and thus the use of some kind of regularizationdevice.

It is apparent from the examples and theory pre-sented in this paper that, whether regularization isundertaken manually through use of zones of piecewiseconstancy, or as an inherent part of the inversion meth-odology itself, an estimated property value assigned toany point within a model domain through the calibra-tion process is unlikely to be equal to the true hydraulicproperty value at that point. In fact, calibration-deter-mined hydraulic property values can be locally quiteerroneous, especially where data density is low. Further-more, no matter how regularization is achieved, detail inthe estimated parameter field cannot exist beyond a levelthat is sustainable by the calibration dataset. Fortu-nately, information made available as a by-product ofthe regularized inversion process allows an assessmentto be made of the degree to which such detail is poten-tially lost. Sadly, however, this loss of detail may seri-ously degrade a model�s ability to make accuratepredictions of those aspects of system behaviour thatare sensitive to this lost detail. In fact, such predictionsmay be seriously in error. This is the cost of obtaining aunique solution to the inverse problem!

Examination of the spatial covariance structure of thecalibrated field has allowed insights to be gained into thenature of the calibration process. In this paper an anal-ogy has been drawn between model calibration and lowpass spatial filtering. The increasing attenuation of spa-tial detail with decreasing spatial wavelength (i.e.increasing spatial frequency) is an outcome of the factthat the calibration process is actually a process of spa-tial integration as evinced by the role of the resolutionmatrix in the calibration process. As discussed by many


authors (e.g. Stearns [52]) integration is actually a formof low-pass filtering. Hence it is to be expected that anestimated parameter field will possess a much lower spa-tial frequency content than the true field from which itwas derived.

An analogy can be drawn between model calibrationand sampling theory. Sampling theory tells us that sam-ple points need to be spaced at an interval that is lessthan half the smallest wavelength contained in the sam-pled field if a valid reconstruction of that field from itssamples is to take place. As the number of observationscomprising any calibration dataset is finite, there is thusa lower bound on spatial wavelengths that can be repre-sented in an estimated field. Because observation pointsare normally separated by a greater distance than halfthe wavelength of the maximum spatial frequency repre-sented in the true hydraulic property field, shorter spa-tial wavelengths existing within this sampled field are‘‘aliased’’ to appear as larger spatial wavelengths inthe calibrated field.

The use of many parameters in conjunction with reg-ularized inversion as a methodology for groundwatermodel calibration allows the calibration process to cap-ture more information from a calibration dataset thanthat which can be captured using a limited number ofzones of piecewise constancy. Looked at in the ‘‘spacedomain’’, this is an outcome of the fact that use of amultiplicity of parameters increases the chance thatparameters exist where they need to exist, allowing thecalibration process to introduce heterogeneity into themodel domain where this is required in order to repro-duce field measurements. The more parameters thatare used (i.e. the more pilot points in our case), the morelikely this is to be true. However the sampling theoryanalogy outlined above may allow us to place limitson the number of pilot points required for a particularcalibration exercise if something is known about the de-gree of heterogeneity that is likely to exist within the truehydraulic property field, and hence the estimated fieldthrough application of Eqs. (9). Alternatively becausethe shortest spatial wavelength that can be representedin an estimated field is ultimately determined by the den-sity of sample points (for example, observation wells),sampling theory allows us to determine how many pilotpoints will be required for adequate representation ofspatial heterogeneity in fields that are estimated on thebasis of a limited calibration dataset. Basically, a pilotpoint density that is commensurate with observationpoint density should be employed, with the spatial den-sity of the former increasing where the spatial density ofthe latter increases (thereby allowing inference of agreater range of heterogeneity potentially existing inthe real property field). However where observationpoint density is very large, there may not be a need forobservation points to be matched ‘‘one for one’’ by pilotpoints, for if observation point spacing is less than half

the shortest wavelength represented in the hydraulicproperty field, then its sample density is more than ade-quate; thus there is no requirement that pilot point den-sity should be commensurate with it.

Knowing that the calibration process can incur a con-siderable loss of detail, it follows that model predictionsmust be viewed with great caution in situations wherethese predictions depend on system detail beyond thatwhich can be inferred through the calibration process.Contaminant movement is an obvious example of a pro-cess that is highly dependent on system detail. Otherexamples abound, for example the transient responseof water levels to climatic or pumping extremes, ground-water-surface water interaction, etc. Predictions of thesetypes (and others), made with parameter fields estimatedthrough calibration against an always incomplete data-set may be seriously in error. An assessment of the pos-sible magnitude of this error should be an essentialcomponent of model deployment. Means by which thiscan be achieved are presently available through methodssuch as calibration-constrained Monte Carlo analysis(see for example Gomez Hernandez [26], Ramaraoet al. [51] and LaVenue et al.�s [37], Capilla et al. [12],Hu [33]). Unfortunately the computational cost of thesemethods is often very high. An alternative, and muchless computationally expensive, method, is presented in[45].

It is anticipated that the use of regularized inversionas a methodology for groundwater model calibrationwill increase dramatically in the future, for the cali-brated parameter field produced as an outcome of itsuse is more responsive to the information content ofan often costly environmental dataset, thus ‘‘doing jus-tice’’ to the cost of acquiring that data in the first place.Furthermore, information made available to the model-ler as a by-product of regularized inversion, some ofwhich has been discussed in this paper, allows the mod-eller to assess a calibrated model not just in terms of thedetail which it has captured, but also in terms of the de-tail that it has failed to capture. In addition to this, thecomputational cost of undertaking regularized inver-sion, and of including many parameters in the formula-tion of the inverse problem to take maximum advantageof the benefits that regularized inversion has to offer,need not be great. The use of adjoint state methodsfor calculation of derivatives where parameters outnum-ber observations can reduce this burden considerably(see, for example, Townley and Wilson [56] and Clemoet al. [19]). So too can the use of ‘‘SVD-assisted’’ param-eter estimation [55], which combines the benefits ofsubspace regularization with that of Tikhonovregularization in a single, computationally efficient, hy-brid scheme that dispenses with the need to calculatederivatives of all model outputs with respect to allparameters during all iterations of a nonlinear inversionprocess.


References

[1] Akaike H. A new look at the statistical model identification. IEEETrans Automat Contr 1974;AC19(6):716–23.

[2] Anderman ER, Hill MC. MODFLOW-2000, the US GeologicalSurvey modular ground-water model—documentation of theADVective-Transport observations (ADV2) Package: US Geo-logical Survey Open-File Report 01-54, 2001, 69p.

[3] Anderson MP, Woessner WW. Applied groundwater modeling—simulation of flow and advective transport. Academic Press; 1992.

[4] Babiloni F, Carducci F, Cincotti F, Del Gratta C, Pizzella V,Romani GL, et al. Linear inverse source estimate of combinedEEG and MEG data related to voluntary movements. Hum BrainMap 2001;14(4):197–209.

[5] Backus G, Gilbert F. Numerical application of a formalism forgeophysical inverse problems. Geophys J Roy Astr S1967;13:247–76.

[6] Backus G, Gilbert F. The resolving power of gross earth data.Geophys J Roy Astr S 1969;16:169–205.

[7] Backus GE, Gilbert F. Uniqueness in the inversion of inaccurategross earth data. Philos Tr R Soc S A 1970;266(1173):123–92.

[8] Bard Y. Nonlinear parameter estimation. New York: AcademicPress; 1974.

[9] Bertete-Aguirre H, Cherkaev E, Oristaglio M. Non-smoothgravity problem with total variation penalization functional.Geophys J Int 2002;149(2):499–507.

[10] Braud I, Obled C. On the use of empirical orthogonal functionsanalysis in the simulation of random fields: stochastic hydrol.Hydraul 1991;5(2):125–34.

[11] Butler Jr JJ, McElwee CD, Bohling GC. Pumping tests innetworks of multilevel sampling wells: Motivation and method-ology. Water Resour Res 1999;35(11):3553–60.

[12] Capilla JE, Rodrigo J, Gomez-Hernandez JJ. Simulation of non-Gaussian hydraulic conductivity fields honoring piezometric dataand integrating soft and secondary information. Math Geol1999;31(7):907–27.

[13] Carrera J, Neuman SP. Estimation of aquifer parameters undertransient and steady-state conditions. 2. Uniqueness, stability, andsolution algorithms. Water Resour Res 1986;22(2):211–27.

[14] Certes C, de Marsily G. Application of the pilot points method tothe identification of aquifer transmissivities. Adv Water Resour1991;14(5):284–300.

[15] Christensen S, Cooley RL. Evaluation of prediction intervals forexpressing uncertainties in groundwater flow model predictions.Water Resour Res 1996;35(9):2627–39.

[16] Christensen S, Cooley RL. Evaluation of confidence intervals fora steady-state leaky aquifer model. Adv Water Resour 1999;22(8):807–17.

[17] Christensen S, Cooley RL. Experiences from the testing of atheory for modeling groundwater flow in heterogeneous media.Proceedings of ModelCARE�2002, Prague, Czech Republic, 17–20June 2002. IAHS publ. no. 2002; p. 22–7.

[18] Christensen S, Moore C, Doherty J. Comparison of stochastic andregression based methods for quantification of predictive uncer-tainty of model-simulated wellhead protection zones in heteroge-neous aquifers. ModelCARE�2005 conference paper, The Hague,Netherlands, 6–9 June 2005.

[19] Clemo T, Michaels P, Lehman RM. Transmissivity ResolutionObtained from the Inversion of Transient and Pseudo-SteadyDrawdown Measurements. MODFLOW and More 2003: Under-standing through Modeling—Conference proceedings 2003; Poe-ter,Zheng, Hill & Doherty. Downloadable from: www.mines.edu/igwmc.

[20] Cooley RL. A theory for Modeling Ground-Water Flow inHeterogeneous Media. Reston, VA, US Geological Survey,Professional Paper 1679, 2004.

[21] Delay F, Buoro A, Marsily Gde. Empirical orthogonal functionanalysis applied to the inverse problem in hydrogeology: Evalu-ation of uncertainty and simulation of new solutions. MathGeology 2001;33(8):927–49.

[22] Doherty J. Groundwater model calibration using pilot points andregularization. Ground Water 2003;41(2):170–7.

[23] Doherty J. Version 5 of PEST Manual. Watermark NumericalComputing, Brisbane Australia; 2003. Downloadable from:http://www.sspa.com/pest.

[24] Draper NR, Smith H. Applied regression analysis. Third ed. NewYork: John Wiley & Sons, Inc.; 1998.

[25] Engl HW, Hanke M, Neubauer A. Regularization of inverseproblems. Kluwer Academic Publishers; 1996.

[26] Gomez-Hernandez JJ, Sahuquillo A, Capilla JE. Stochasticsimulation of hydraulic conductivity fields conditional to bothhydraulic conductivity and piezometric data. 1. Theory J Hydrol1997;20:162–74.

[27] Gorokhovski VM. Problem-dependence of ground-water modelidentification: significance, extent, and treatment. Ground Water1996;34(3):461–9.

[28] Guadagnini A, Neuman SP. Nonlocal and localized analyses ofconditional mean steady state flow in bounded, randomly uniformdomains, 1, theory and computational approach. Water ResourRes 1999;35(10):2999–3018.

[29] Haber E. Numerical strategies for the solution of inverseproblems. PhD thesis, University of British Columbia, Canada,1997.

[30] Harbaugh AW, Banta ER, Hill MC, McDonald MG. The USGeological Survey Modular Ground-Water Model—User Guideto Modularization Concepts and the Ground-Water Flow pro-cess. US Geological Survey Open File Report 00-92. Reston,Virginia, 2000.

[31] Hill MC. Methods and guidelines for effective model calibration.US Geological Survey, Water-Resources Investigations Report98-4005, 1998.

[32] Hill MC, Osterby O. Determining extreme parameter correlationin Ground Water Models. Ground Water 2003;41(4):420–30.

[33] Hu LY. Gradual deformation and iterative calibration of gauss-ian-related stochastic models. Math Geol 2000;32(1):87–108.

[34] Jackson DD. Interpretation of inaccurate, insufficient and incon-sistent data. Geophys J Roy Astr S 1972;28:97–109.

[35] Kitanidis PK. The minimum structure solution to the inverseproblem. Water Resour Res 1997;33(10):2263–72.

[36] Koch KR. Parameter estimation and hypothesis testing in linearmodels. Berlin: Springer-Verlag; 1987.

[37] LaVenue AM, RamaRao BS, de Marsily G, Marietta MG. Pilotpoint methodology for automated calibration of an ensemble ofconditionally simulated hydraulic conductivity fields. 2. ApplWater Resour Res 1995;31(3):495–516.

[38] Lines LR, Treitel ST. A review of least-squares inversion and itsapplication to geophysical problems. Geophys Prospect 1984;32(2):159–86.

[39] Lutkenhoner B, Menendez RGDP. The resolution-field concept.Electroen Clin Neuro 1997;102(4):326–34.

[40] Mackie RL, Madden TR. Three-dimensional magnetotelluricinversion using conjugate gradients. Geophys J Int 1993;115:215–29.

[41] Mantoglou A. Estimation of heterogeneous aquifer parametersfrom piezometric data using ridge functions and neural networks.Stoch Env Res Risk A 2003;17(5):339–52.

[42] Menendez RGDP, Andino SLG, Lutkenhoner B. Figures of meritto compare distributed linear inverse solutions. Brain Topogr1996;9(2):117–24.

[43] Menke W. Geophysical data analysis: discrete inverse the-ory. Academic Press Inc; 1984.

http://www.mines.edu/igwmc

http://www.mines.edu/igwmc

http://www.sspa.com/pest


[44] Mikhail EM with contributions by Ackerman F. Observationsand Least Squares. New York: IEP—ADun-Donnelley publisher;1976.

[45] Moore C, Doherty J. The role of the calibration process inreducing model predictive error. Water Resour Res 2005.

[46] Moore C. The use of regularized inversion in groundwater modelcalibration and prediction uncertainty analysis. PhD Thesis.University of Queensland, Australia, 2005.

[47] Murray Darling Basin Commission. Murray–Darling BasinCommission Groundwater Flow Modeling Guideline. AquaterraConsulting Pty Ltd. Project 125. 2000. Downloadable from:http://www.mdbc.gov.au/publications/pdf/model_guide.pdf.

[48] Nolet G, Montelli R, Virieuz J. Explicit, approximate expressionsfor the resolution and a posteriori covariance of massive tomo-graphic systemes. Geophys J Int 1999;138:36–44.

[49] Poeter EP, Hill MC. Documentation of UCODE, a computercode for universal inverse modeling. US Geological Survey Water-Resources Investigations 1998; Report 98-4080, 122p.

[50] Portniaguine O, Zhdanov MS. Focusing geophysical inversionimages. Geophysics 1999;64(3):874–87.

[51] RamaRao BS, LaVenue AM, de Marsily G, Marietta MG. Pilotpoint methodology for automated calibration of an ensemble ofconditionally simulated hydraulic conductivity fields, 1, theoryand computational experiments. Water Resour Res 1995;31(3):475–93.

[52] Stearns SD. Digital signal analysis. NJ: Hayden Book Company,Inc; 1975.

[53] STOWA. Good Modeling Practice Handbook. STOWA 1999;Report 99-05 file gmpuk.pdf. Downloable from: http://water-land.net/riza/aquest.

[54] Tikhonov AN, Arsenin VY. Solution of ill-posed prob-lems. Washington DC: VH Winston and Sons; 1977.

[55] Tonkin M, Doherty J. A hybrid regularised inversion methodol-ogy for highly parameterised environmental models. WaterResour Res 2005.

[56] Townley LR, Wilson JL. Computationally efficient algorithms forparameter estimation and uncertainty propagation in numericalmodels of groundwater flow. Water Resour Res 1985;21(12):1851–60.

[57] Vasco DW, Akhil Datta-Gupta. Asymptotic solutions of solutetransport: A formalism for tracer tomography. Water Resour Res1990;35(1):1–16.

[58] Vasco DW, Datta-Gupta A, Long JCS. Resolution and uncer-tainty in hydrologic characterization. Water Resour Res1997;33(3):379–97.

[59] Vasco DW, Petersen JE, Majer EL. Resolving seismic anisotropy:sparse matrix methods for geophysical inverse problems. Geo-physics 1998;63(3):970–83.

[60] Vasco DW. Estimation of flow properties using surface deforma-tion and head data: A trajectory-based approach. WRR2004;40:W10104. doi:10.1029/2004WR003272.

[61] Vecchia AV, Cooley RL. Simultaneous confidence and predictionintervals for nonlinear regression models with application to agroundwater model. Water Resour Res 1987;23(7):1237–50.

[62] Vogel CR, Oman ME. Fast total variation based reconstructionof noisy, blurred images. IEEE T Image Process 1998;7(6):813–24.

[63] Yao ZS, Roberts RG. A practical regularization for seismictomography. Geophys J Int 1999;138:293–9.

[64] Yeh TCJ, Liu SY. Hydraulic tomography: Development of a newaquifer test method. Water Resour Res 2000;36(8):2095–105.

http://www.mdbc.gov.au/publications/pdf/model_guide.pdf

http://waterland.net/riza/aquest

http://waterland.net/riza/aquest

http://dx.doi.org/10.1029/2004WR003272

The cost of uniqueness in groundwater model calibration

Documents

Transcript of The cost of uniqueness in groundwater model calibration