Controlling model error of underdamped forecast models in ... · can signiﬁcantly improve...

Quarterly Journal of the Royal Meteorological Society Q. J. R. Meteorol. Soc. 00: 1–15 (0000)

Controlling model error of underdamped forecast models insparse observational networks using a variance limiting

Kalman filter

Lewis Mitchell a ∗ and Georg A. Gottwalda∗aSchool of Mathematics and Statistics, University of Sydney, NSW 2006, Australia

∗Correspondence to: E-mail: [email protected]; [email protected]

The problem of controlling covariance overestimation due to underdampedforecast models and sparsity of the observational network in an ensembleKalman filter setting is considered. It is shown in a variational setting thatlimiting the analysis error covariance to stay below the climatological value anddriving the mean towards the climatological mean for the unobserved variablescan significantly improve analysis skill over standard ensemble Kalman filters.These issues are explored for a Lorenz-96 system. It is shownthat for largeobservation intervals the climatological information assures that the statisticalproperties of the unobserved variables are recovered, providing superioranalysis skill. This skill improvement is increased for larger observational noise.Copyright c© 0000 Royal Meteorological Society

Key Words: Data assimilation; Ensemble Kalman filter; model error

Received . . .

Citation: . . .

1. Introduction

To account for uncertainties in both the initial conditionsand forecast model and to combat the associated forecasterror and flow-dependent predictability, ensemble methodshave become popular for producing numerical weatherforecasts (Molteni and Palmer 1993; Toth and Kalnay1993) and for performing data assimilation (Evensen1994). The idea behind ensemble based methods is thatthe nonlinear chaotic dynamics of the underlying forecastmodel and the associated sensitivity to initial conditionscause an ensemble of trajectories to explore sufficientlylarge parts of the phase space in order to deduce meaningfulstatistical properties of the dynamics. However, all currentlyoperational ensemble systems are underdispersive, meaningthat the true atmospheric state is on average outside thestatistically expected range of the forecast or analysis(e.g. Buizzaet al. (2005); Hamill and Whitaker (2011)).This detrimental underdispersiveness can be linked to twoseparate sorts of model error: adynamical model errordue to a misrepresentation of the physics of the forecastmodel and anumericalmodel error due to the choice of thenumerical method used to simulate those forecast models.It has been argued (Palmer 2001) that dynamical model

error is largely due to a misrepresentation of unresolvedsub-grid scale processes. The very active field of stochasticparametrization is aimed at reintroducing statisticalfluctuations of the unresolved degrees of freedom anda direct coupling between the resolved and unresolveddegrees of freedom into the equations of motion (seePalmer and Williams (2010) for a recent review on currenttrends). Indeed, using a low-dimensional toy modelMitchell and Gottwald (2012) showed that stochasticparametrizations can lead to superior skill performance inensemble data assimilation schemes in situations where thedynamics involves rapid large amplitude excursions such asin regimes switches or in intermittent dynamics.Numerical model error is often introduced deliberately tocontrol numerical instabilities arising in the simulationofgeophysical fluids, such as in numerical weather predictionor climate dynamics. In such simulations numericalcodes tend to produce large errors linked to numericalinstabilities at the grid resolution level, which limits thereliability of the forecast over the forecast window. In adata assimilation context this numerical “noise” can lead toan unrealistic overestimation of forecast error covariancesat small scales. The standard approach to dealing withthese numerical instabilities is to add various forms of

Copyright c© 0000 Royal Meteorological Society

Prepared usingqjrms4.cls [Version: 2011/05/18 v1.02]

2 L. Mitchell and G.A. Gottwald

artificial dissipation to the model (Durran 1999). Thishowever has several well documented drawbacks. In thecontext of ensemble data assimilation artificial viscositydiminishes the spread of the forecast ensemble, causingdetrimental underestimation of the error covariances(Houtekamer and Mitchell 1998; Burgerset al. 1998;Anderson and Anderson 1999; Houtekameret al. 2005;Charronet al. 2010). Most notably, artificial dissipationimplies unrealistic and excessive drainage of energy out ofthe system (Shutts 2005), effecting processes ranging fromfrontogenesis to large scale energy budgets. For example,Palmer (2001) reports on the unphysical dissipation ofkinetic turbulent energy caused by mountain gravity wavedrag parametrizations. Skamarock and Klemp (1992)introduce divergence damping to control unwanted gravitywave activity in hydrostatic and non-hydrostatic elasticequations to stabilize the numerical scheme, leading tonumerical energy dissipation (see also Durran (1999)).Blumen (1990) controls the collapse of near-surface frontalwidth by artificially introducing momentum diffusion in asemi-geostrophic model of frontogenesis, with the effectof producing unrealistic energy dissipation rates. In semi-Lagrangian advection schemes Cote and Staniforth (1988)and Ritchie (1988) find unrealistic energy dissipation ratesthat are caused by interpolation. Besides effecting thesmall-scale dynamics, numerical dissipation also has amajor effect on the large scales and their energy spectra(Palmer 2001; Shutts 2005). In ideal fluids, which oftenserve as a good approximation of the large scale dynamics(Salmon 1998), numerical dissipation destroys dynamicallyimportant conservation laws which may be undesirablein long time simulations such as in climate modelling(Thuburn 2008). Using statistical mechanics Dubinkinaand Frank (2007, 2010) show how the choice of theconservation laws respected by a numerical scheme effectsthe statistics of the large scale fields.There exists a plethora of methods to remedy the unphysicalenergy loss due to artificial diffusion and reinject energyback into the numerical model, for example via energybackscattering (Frederiksen and Davies 1997; Shutts 2005)or systematic stochastic model reductions (Majdaet al.1999).

Here we will look at these issues in the context ofensemble data assimilation. Rather than controllingnumerical instabilities of the dynamic core via appropriatediscretization schemes of the numerical forecast model,we will modifiy the actual ensemble data assimilationprocedure, avoiding excessive numerical dissipation.Underdamping in dynamic cores causes the ensemble toacquire unrealistically large spread. This overdispersionis exacerbated in sparse observational networks wherefor the unobserved variables unrealistically large forecastcovariances are poorly controlled by observational data(Liu et al. 2008; Whitakeret al. 2009). The questions weaddress in this article are:can one use numerical forecastingmodels which are not artificially damped, yet still controlthe resulting overestimation of forecast covariances withinthe data assimilation procedure?, and,is it possible to usethe increased spread induced by numerical instabilities ina controlled fashion to improve the skill of ensemble dataassimilation schemes?In a recent article Gottwaldet al. (2011) proposed avariance limiting ensemble Kalman filter (VLKF) whichcontrols overestimation in sparse observational networks

by including climatological information of the unobservedvariables. In a perfect model data assimilation schemeunrealistic overestimation of the error covariances occursin sparse observational networks as a finite size effect; itwas shown that for short observational intervals of up to10hours the VLKF produces superior skill when comparedto the standard ensemble transform Kalman filter (ETKF)(Bishop et al. 2001; Tippett et al. 2003; Wanget al.2004). Here we will apply the variance limiting Kalmanfilter in the case of an imperfect, underdamped forecastmodel, and for large observation intervals. Unlike in theperfect model case where for large observation intervals theforecast error covariances approach the climatic variance,when forecasting with underdamped forecast models theerror covariances will be larger than the climatic variance.This will be controlled by the VLKF, invoking a weakconstraint onto the mean and the variance of the unobservedvariables, leading to an improvement in the analysis skill.We will study this effect using the standard Lorenz-96model (Lorenz 1996), where we assume that the truth isevolving to the standard set of parameters. To investigatewhether underdamped models can be used as forecastmodels we then use forecast models with smaller valuesof the linear damping term of the Lorenz-96 system togenerate ensembles with forecast spreads exceeding thetrue climatology.

Our main result is, that VLKF produces significantlybetter skill than ETKF, increasing with the strength ofthe model error, the sparsity of the observational networkand the observational noise covariance. We will establishthat a variance limiting weak constraint may be used asan adaptive alternative to artificial viscosity damping,counteracting model error on the fly when underdampingof the numerical forecast model causes an unrealisticoverestimation of the forecast ensemble. However, we willsee that there is a trade-off as for large model error the skillof VLKF becomes comparable to the skill of an analysisinvolving no forecast but using an analysis comprised of theobservations and the climatological mean for the observedand unobserved variables, respectively. It will be shownthat this is important for large observation times whenthe analysis is not tracking anymore due to either modelerror or the chaotic nature of the underlying dynamics,and one can only improve the analysis, assuring a reliablereproduction of statistical properties of the dynamics.

In the next section we briefly describe the variancelimiting Kalman filter. In Section 3 we introduce theLorenz-96 model and describe the underdamped forecastmodel used. In Section 4 we present results showing underwhat conditions the VLKF produces better skill than theusual ensemble transform Kalman filter. We conclude witha discussion and outlook in Section 5.

2. The variance limiting Kalman filter

The variance limiting Kalman filter (VLKF) is a variationof a standard ensemble Kalman filter designed to controloverestimation of error covariances caused by finiteensemble sizes in sparse observational grids (Gottwaldet al. 2011). Whereas usually only direct observations andmodel forecasts are combined to produce an analysis in dataassimilation, it was proposed to incorporate climatologicalinformation to fill the data voids in sparse observational

Copyright c© 0000 Royal Meteorological Society Q. J. R. Meteorol. Soc.00: 1–15 (0000)

Prepared usingqjrms4.cls

Controlling model error using a variance limiting Kalman fil ter 3

grids. In the atmospheric sciences data voids are a hugeobstruction to climate reanalyses (Bengtssonet al. 2007;Whitakeret al.2004; Compoet al.2011) when assimilatingdata from pre-radiosende era, or when attempting to includethe mesospehere into numerical weather forecasts wherethe energetics is dominated by unobserved gravity waves(Polavarapuet al. 2005; Sankeyet al. 2007; Eckermannet al. 2009). Climatic information is used to limit theposterior error covariance of the unobserved variables tonot exceed their climatic covariance. It was shown thatthis limitation of the analysis error covariance controlledundesirable overestimation of error covariances whichare caused by the finite size of the forecast ensembles.Furthermore, the VLKF drives the mean of the unobservedvariables towards the climatological mean significantlyincreasing the analysis skill. We remark that one mayuse other values than the climatic covariance to controlthe analysis error covariance if one interprets the varianceconstraint as a numerical tool only to control overestimationof the error covariances. We present a brief summary ofthe algorithm here. For a detailed derivation the reader isreferred to Gottwaldet al. (2011).

Given anN -dimensional dynamical system

zt = F(zt) , (1)

which is observed at discrete timesti = i∆tobs, dataassimilation aims at producing the best estimate of thecurrent state given a typically chaotic, possibly inaccuratemodel z = f(z) and noisy observations of the true statezt (Kalnay 2002). We assume that the state space isdecomposable according toz = (x,y) with observablesx ∈ R

n, for which direct observations are available,and pseudo-observablesy ∈ R

m, for which only someintegrated or statistical climatological information isavailable. Here we will assumen + m = N , however thisis not essential.

Observationsyo ∈ Rn are expressed as a perturbed truth

according toyo(ti) = Hzt(ti) + ro ,

where the observation operatorH : RN → R

n maps fromthe whole space into observation space, andro ∈ R

n

is assumed to be independent identically distributedobservational Gaussian noise with associated errorcovariance matrixRo.

We further introduce an operatorh : RN → R

m

which maps from the whole space into the space of thepseudo-observables spanned byy. We assume that thepseudo-observables have varianceAclim ∈ R

m×m andconstant meanaclim ∈ R

m. This is the only informationavailable for the pseudo-observables, and may be estimated,for example, from historical data or long time simulations.The error covariance of those pseudo-observations isdenoted byRw ∈ R

m×m.

In an ensemble Kalman filter (EnKF) (Evensen 2006) anensemble withk memberszk

Z = [z1, z2, . . . , zk] ∈ RN×k

is propagated by the model dynamics according to

Z = f(Z) , f(Z) = [f(z1), f(z2), . . . , f(zk)] ∈ RN×k .

This forecast ensemble is split into its meanzf andensemble deviation matrixZ′

f . The ensemble deviationmatrixZ′

f can be used to approximate the ensemble forecastcovariance matrix via

Pf (t) =1

k − 1Z′(t) [Z′(t)]

T ∈ RN×N .

Note thatPf (t) is rank-deficient fork < N which is thetypical situation in numerical weather prediction whereNis of the order of109 andk of the order of100.

Given the forecast meanzf , the forecast error covariancePf and an observationyo, an analysis mean is given by

za = zf − Ko [Hzf − yo] − Kw [hzf − aclim] , (2)

where

Ko = PaHTR−1

o

Kw = PahTR−1

w ,

with analysis error covariance matrix

Pa =(

P−1

f + HTR−1o H + hT R−1

w h)

−1

.

The variance of the unresolved pseudo-observableshz iscontrolled by setting

hPahT = Aclim . (3)

This yields upon using the Sherman-Morrison-Woodburyformula (see for example Golub and Loan (1996)) theequation for the as yet unspecified error covariance matrixfor the pseudo-observablesRw

R−1w = A−1

clim−

(

hPhT)−1

,

where

P =(

P−1

f + HTR−1o H

)

−1

is the analysis error covariance of an equivalent EnKFwithout variance constraint. We remark that the matrixRw itself is never required in the VLKF algorithm, soan expensive inversion ofR−1

w is not explicitly needed.Furthermore, the matrixP can also be calculated withoutcomputing inverses of the forecast covariance matrix bymeans of the Sherman-Morrison-Woodbury formula. It isreadily seen that for sufficiently small background errorcovariancePf , the error covarianceRw is not positivesemi-definite. To ensure that the error covariance is non-negative we diagonalizeR−1

w = VDVT ; introducing Dwith Dii = Dii for Dii ≥ 0 andDii = 0 for Dii < 0, wesetR−1

w = VDVT . Hence the variance limiter acts onlyon those eigendirections ofhPhT whose correspondingsingular eigenvalues are larger than those of the prescribed(climatic) covarianceAclim. The diagonalisation ofR−1

w

poses an additional computational complexity whencompared to ETKF.

To determine an ensembleZa which is consistent withthe analysis error covariancePa and satisfies

Pa =1

k − 1Z′

a [Z′

a]T

,




we use the method of ensemble square root filters (Simon2006). In particular we use the ensemble transform Kalmanfilter (ETKF) (Bishop et al. 2001; Tippettet al. 2003;Wanget al.2004), which seeks a transformationS ∈ R

k×k

such that the analysis deviation ensembleZ′

a is givenas a deterministic perturbation of the forecast ensembleZf via Z′

a = Z′

fS. Details of the implementation can befound in (Bishopet al. 2001; Tippettet al. 2003; Wanget al. 2004). Alternatively one could choose the ensembleadjustment filter (Anderson 2001) in which the ensembledeviation matrixZ′

f is pre-multiplied with an appropriatelydetermined matrixA ∈ R

N×N . Yet another method toconstruct analysis deviations was proposed in Bergemannet al. (2009) where the continuous Kalman-Bucy filteris used to calculateZ′

a without using any computationsof matrix inverses, which may be advantageous in high-dimensional systems.A new forecast is obtained by propagatingZa with thenonlinear forecast model to the next observation time,where a new analysis cycle will be started.

In the perfect model case, whenAclim and aclim

are given by the climatology, the forecast model willacquire comparable forecast error covariancePf for largeobservational intervals∆tobs using either the ETKF orVLKF. Hence both filters will have approximately thesame input parameters prior to the analysis step, and aswas shown in Gottwaldet al. (2011) the filters performequally well. This is not true for imperfect models with anunderdamping forecast model; when the variance limitingconstraint is switched on, the background covariance ofthe pseudo-observables is set toAclim = σ2

climI for theVLKF, much smaller than that for the ETKF when usingan underdamped model. Note that this implies that theforecast error covariance is also typically much smaller forthe VLKF than ETKF over short observation intervals.

3. The Lorenz-96 model

Consider the following modification of the Lorenz-96model (Lorenz 1996)

dzj

dt= zj−1(zj+1 − zj−2) − γzj + F j = 1, ..., D (4)

with z = (z1, ..., zD)T and periodiczj = zj+D, where wehave introduced a variable linear damping parameterγ. TheLorenz-96 model is a standard test bed for data assimilation(see for example Ottet al. (2004); Fisheret al. (2005))as it is computationally manageable but still incorporatescrucial ingredients of real midlatitude atmospheric flowssuch as nonlinear energy conservation, advection, forcingand linear damping. We set the number of degrees offreedom asD = 40 andF = 8 for the forcing. Forγ = 1,the classical value of the damping in the Lorenz-96 system,this corresponds to a time scaling of 5 days per modeltime unit, ane-folding time of 2.1 days and a distancebetween adjacent grid points of750 km, roughly equallingthe Rossby radius of deformation (Lorenz 1996).

Figure 1 illustrates how the climatic meanµclim andvariance σ2

clim depend on the value of the dampingparameterγ. The climatic values are obtained fromsimulations of one long time trajectory (1000 days). Notethat in the Lorenz-96 system all variableszj share the same

statistical behaviour. Furthermore, we show the variancesσ2 when calculated from simulations running only overa finite length of time, such as analysis cycle length. Asexpected, the variance increases as the dynamics gets lessand less damped for fixed length of time, and increase withthe simulation length∆tobs for fixed damping parameterγ(asymptoting to the climatic variance).

Two characteristic time scales of the Lorenz-96 system(4) will be important in the following, thee-foldingtime, measured here by the inverse of maximal Lyapunovexponentλmax and the advection timeTadv, which is ameasure for the time taken for a front to propagate fromone sitej to the nextj + 1. In Figure 2 we show how thesecharacteristic time scales vary with the damping parameterγ. Firstly, we show the inverse of the maximal Lyapunovexponentλmax which is a measure for the predictabilitytime of the Lorenz-96 dynamics and correlates with thedecorrelation time. For the original Lorenz-96 model withγ = 1 we reproduce thee-folding time of 50 hours, i.e.2.1 days. In accordance with the observed increase invariance, the predictability time decreases with decreasingdamping. As a second characteristic time scale we showthe advective timeTadv. Since the nonlinear term in (4)can be interpreted as a finite difference discretization of anadvection termzzx, information from one grid pointj ismainly transported to its right neighbours in an asymmetricfashion. We estimateTadv by initializing an integrationof the Lorenz-96 model with only one non-zero variableand then measuring the time taken for the resulting frontto propagate across sites. The characteristic time scale ofthis information transport is important when discussing theinfluence of the sparsity of an observational network on thefiltering skill. Figure 2 shows that this time scale decreaseswith increasing model error.

4. Numerical results

We now investigate how and when the variance limitingKalman filter can be used to control the increasedcovariances of an underdamped forecast model. Inparticular, we will study the situation of a sparseobservational grid.

We will address two issues. First, we will investigateunder what conditions the VLKF as described in Section 2produces better analysis skill than the ETKF. Second, wewill investigate when the VLKF produces better skill than aworst-case analysis which does not involve any forecast butsimply constructs the analysis by taking the observationaldata and the climatic mean for the observed and theunobserved analysis variables, respectively (a “poor-man’sanalysis”). This latter unfortunate situation can occur, forexample, for moderate observation intervals∆tobs, as wewill see. Besides the degree of underdampingγ, the skillwill be seen to depend on the sparsity of the observationalgrid used as well as on the observation interval and theobservational noise level.

The main result will be that the VLKF framework canindeed provide skill improvement over the standard ETKF.However, as the level of model damping is decreasedwe find that there is a trade-off between increasingimprovement over ETKF on the one hand, and decreasing




0 0.2 0.4 0.6 0.8 10.5

1

1.5

2

2.5

γ

µclim

0.2 0.4 0.6 0.8 10

25

50

75

100

125

150

γ

σ2

∆tobs = 12 hrs

∆tobs = 24 hrs

∆tobs = 48 hrs

Climatic

Figure 1. Dependence of climatic mean (left) and variance (right) on damping parameterγ. As well as the climatic variance (black, circles, solid) weshow the average variance calculated over forecast intervals ∆tobs = 12 hours (red, squares, dotted),∆tobs = 24 hours (green, triangles, dash-dotted)and∆tobs = 48 hours (blue, crosses, dashed).

0 0.2 0.4 0.6 0.8 1

10

20

30

40

50

γ

1/λ

max

0 0.2 0.4 0.6 0.8 18

9

10

11

12

13

14

γ

Tadv

Figure 2. Characteristic time scales for the Lorenz-96 system (4) as afunction of the damping parameterγ. Left: Inverse of the maximal Lyapunovexponentλmax in hours. Right: Advection timeTadv in hours.

improvement over the poor-man’s analysis on the other. Theskill improvement can be explained by the improved abilityof the VLKF to ensure the correct statistical propertiesof the analysis, as opposed to pathwise tracking of thetruth, for small forecast model damping. We find analogousresults as the observational sparsityNobs or the magnitudeof the observational noise is increased.

We generate a truthzt from a realisation of the Lorenz-96system (4) withγ = 1. Observationsyo are then generatedfor the observablesHz with Gaussian error covarianceRo = (ησclim)2I, and are available after each observationinterval ∆tobs at everyN th

obssite. The observations are

then assimilated in an ensemble filter framework using theLorenz-96 model (4) with damping parameterγ < 1 asforecast model, mimicking the situation of underdampedforecast models. We perform twin experiments where thesame truth and observations are used for data assimilationwith both the VLKF and ETKF. Both filters are run withk = 41 ensemble members and the same forecast model(4) with γ < 1, making the forecasts underdamped withrespect to the truthzt. Both filters are spun up for 100

cycles to ensure statistical stationarity of the ensembles.We employ 5% constant global multiplicative covarianceinflation in both filters.

We use an implicit mid-point rule (see for exampleLeimkuhler and Reich (2005)) to simulate the system (4)to a timeT = 60 time units with a time stepdt = 1/240.The total integration time corresponds to 300 days, and theintegration timestepdt corresponds to half an hour. Theclimatic mean and standard deviation over all variableswith γ = 1 are estimated as temporal averages from a longtrajectory asµclim = 2.34 and σclim = 3.63 respectively,which determines our true climatology.

In Figures 3 and 4 we show sample analyses using bothfilters, ETKF and VLKF, with Nobs = 4 and dampingfactorγ = 0.5 for both∆tobs = 24 and∆tobs = 48 hours,exemplifying moderate and large observation intervalsrespectively. We show analyses for an arbitrary unobservedcomponent and an arbitrary observed component of theLorenz-96 model. Both filters track the truth for theobserved components reasonably well for both moderate




and large observation intervals, with the analysis tendingmore towards the observations for large∆tobs. However,for the pseudo-observables with moderate observationinterval ∆tobs = 24 hours, the ETKF tracks the truthbetter than the VLKF despite the inclusion of additionalclimatological information. For large observation intervals∆tobs = 48 hours, neither filter tracks the truth for thepseudo-observables well. However, the additional inputof the climatological information into the VLKF leads toan analysis with statistics which better resemble the trueclimatology, thereby improving the overall RMS errors andskill. It is also clearly seen that the main contribution to theerror stems from the pseudo-observables.

Figures 3 and 4 suggest that the notion of whatconstitutes a good performance of a filter depends on thelength∆tobs of the observation interval. When aiming forimproved performance in the case of moderate observationintervals, one is interested in tracking the truth for boththe observables and pseudo-observables. In the case oflarge observation intervals, one can only expect to trackthe truth for the observed variables and produce analyseswhich exhibit the same statistical behaviour as the truthfor the pseudo-observables. We remark that for very largeobservational sparsityNobs tracking cannot be demandedeven for moderate observation intervals; for example with∆tobs = 6 hours the analysis of an observational grid withNobs = 20 does not produce tracking analyses for anyvariablezj .

In Figure 5 we show the average maximal singularvalues sH and sh of the forecast and analysis errorcovariances projected onto the observation and pseudo-observation subspaces respectively. As expected thecovariances increase with increasing model error, as wellas with increasing sparsity. The variance constraint of theVLKF markedly controls the analysis covariance of theunobserved variables to always assume values below theclimatic variance. This causes the forecast covariancesto be limited as well, albeit at values greater than theclimatic varianceσ2

clim. For large observational sparsityNobs, the analysis covariance of the unobserved variablesis set to the climatic variance for the VLKF, whereas forthe ETKF the analysis covariances grow larger thanσ2

clim

with increasing sparsity. Interestingly, the VLKF alsolimits the covariances for the observed variables. As wewill see this may have negative effects and may lead toan underdispersive ensemble which can spoil the VLKFanalysis for the observed variables.

In Figure 6 we show how for long observation intervals∆tobs = 48 hours the true climatic mean and variance arebetter preserved by the VLKF analysis than by the ETKFanalysis. The true climatic meanµclim is reasonably wellpreserved by the VLKF analysis whereas the ETKF greatlyunderestimates it. The VLKF analysis underestimates thetrue climatic varianceσ2

clim, but not as severely as theETKF does. We will see that this leads to a marked increasein skill, especially for large model errorγ < 1.The fact that the variance of the analysis〈(za − 〈za〉)2〉decreases with decreasing damping may seemcounterintuitive at first sight. This can be justified byconsidering a simple two-dimensional system wherexis observed andy is unobserved. The Kalman analysisupdate (cf. Equation (2)) for the unobserved variables

is ya = yf − Pf,xy/(Pf,xx + Ro)(xf − xo) wherePf,xx > 0 andPf,xy measures the forecast error covariancebetween the unobserved and observed variable. Note thatdue to finite size ensemble sizes one can havePf,xy 6= 0even if dynamically there are no correlations betweenobserved and unobserved variables. We argue that thedecrease in variance of the analysis〈(za − 〈za〉)2〉 withdecreasing damping is due to the analysis being dampedstronger for smaller values ofγ in the case of the Lorenz-96system (4). For the simple system this is the case whenPf,xy/(Pf,xx + Ro)(xf − xo) > 0 on average, providedthat yf > 0 also on average. Since the climatic meanis positive and decreases with decreasing dampingγ(cf. Figure 1), zf > 0 and Hzf − yo < 0 on average.Moreover, as all variableszj are statistically equal due tothe periodicity of the Lorenz-96 system, the forecast errorcovariances between observed and unobserved variablesHPfh

T (equivalent toPf,xy in the simple model) mustalso be negative on average. Hence the variance of theanalysis time series〈(za − 〈za〉)2〉 has the same decreasingtendency with decreasing dampingγ as the climatic mean.

We now quantify the performance of the VLKF and thestandard ETKF by the site-averaged RMS error

E =

√

√

√

√〈 1

LDG

L∑

l=1

‖za(l∆tobs) − zt(l∆tobs)‖2G〉 (5)

between the truthzt and the ensemble meanza, whereL = ⌊T/∆tobs⌋ is the number of analysis cycles andDG denotes the number of variables involved. Weintroduced the norm‖a‖2

G= aT Ga to investigate the

error over all variablesE using G = I, the error Eo

distributed over the observed variables usingG = HTHand the errorEuo distributed over the unobserved variablesusing G = hTh. We also introduce a measureE toquantify how well a filter performs against the poor-man’s analysis of using the observations and the climaticmean for the observed and unobserved components of theanalysis respectively, implyingH(za − zt) ∼ N (0,Ro)and h(za − zt) ∼ N (0,Aclim). Assuming thatH(za −zt) and h(za − zt) are uncorrelated and settingAclim =

σ2clim

Im, we can calculate the RMS errorE in the poor-man’s analysis as

E2 =1

LD

L∑

i=1

[

||H(za(ti) − zt(ti))||2

+ ||h(za(ti) − zt(ti))||2]

whereL = ⌊T/∆tobs⌋ is the number of analysis cycles asin (5), giving the RMS error

E =

√

1

Nobs

η2σ2clim

+

(

1 − 1

Nobs

)

σ2clim

= σclim

√

1 +(η2 − 1)

Nobs

, (6)

with Ro = (ησclim)2I.




0 10 20 30 40 50 60 70 80 90 100

−5

0

5

10

t

z4

0 10 20 30 40 50 60 70 80 90 100

−5

0

5

10

15

z1

Figure 3. Sample ETKF (blue-green, solid line) and VLKF (red, solid line) analyses over 100 days for observedz4 (bottom panel) and unobservedz1

(top panel) component for∆tobs = 24 hours and model errorγ = 0.5. The dashed line is the truth, the crosses are observations.Parameters used wereNobs = 4 andRo = (0.25σclim)2 I.

0 10 20 30 40 50 60 70 80 90 100−10

−5

0

5

10

t

z4

0 10 20 30 40 50 60 70 80 90 100

−5

0

5

10

z1

Figure 4. As in Figure 3, but with∆tobs = 48 hours.

To quantify the proportional improvement in RMS errorsfor the VLKF over ETKF we introduce the analysis skill

S =EE

EV,

where the RMS errorsE are defined by (5). Values ofS > 1 indicate an improvement of the VLKF over ETKF.We also define the corresponding skills for proportionalimprovement of the ETKF and VLKF over the poor man’sanalysis

SE =EEE

and SV =EEV

.

We first investigate the dependence of skill upon theobservation interval∆tobs. In Figure 7 the skillsS, SE

and SV are shown as a function of the model errorγ fordifferent observation intervals∆tobs with fixed sparsityNobs = 4. The figures reveal an intricate dependence ofthe skill on both the observation interval and the modelerror. We have checked that for∆tobs > 2.1 days, thee-folding time for the Lorenz-96 model withγ = 1, theskill curvesS(γ) collapse onto one curve, independent ofthe observation interval. This indicates that the forecasterror covariance, which saturates for large observational

times to the climatic covariance, is the main factor drivingfilter performance.

We will treat the near-perfect model caseγ close to oneand the strong model error case with smallγ separately.The perfect model case withγ = 1 was already consideredin Gottwaldet al. (2011). We confirm here that the VLKFis superior to the standard ETKF, its skill increasingwith decreasing observation intervals∆tobs. For smallerobservation intervals skill improvements of more than20%can be achieved, and the analysis of the pseudo-observablestrack the truth surprisingly well (cf. Gottwaldet al.(2011)).Note that both filters perform better than the poor man’sanalysis withSE > 1 andSV > 1 for γ = 1. The variancelimiting filter here successfully controls the overestimationof the analysis error covariance caused by the combinationof finite ensemble sizes and sparse observations.

The case of model error withγ significantly smallerthan 1 – our main objective in this work – shows aninteresting dependence on both the observation interval andthe model error. For the discussion below it is importantto note that for sufficiently large model error the analysiserror covariances projected onto the pseudo-observablesexceed the climatic varianceσ2

clim; hence the background




2 4 6 8 100

5

10

15

20

25

sH

Nobs

2 4 6 8 100

20

40

60

80

100

Nobs

sh

2 4 6 8 10

0.65

0.7

0.75

0.8

Nobs

sH

2 4 6 8 100

10

20

30

40

50

60

70

Nobs

sh

Figure 5. Average maximal singular valuessH (left panels) andsh (right panels) forHPf HT andhPfhT (top) andHPaHT andhPahT (bottom),

respectively, as a function of the sparsityNobs. The solid lines depict values obtained from the ETKF and thedashed lines from the VLKF. We useγ = 1 (blue, crosses),γ = 0.9 (green, circles) andγ = 0.8 (red, triangles). We usedRo = (0.25σclim)2I and∆tobs = 48 hours.

0 0.2 0.4 0.6 0.8 11.8

1.9

2

2.1

2.2

2.3

2.4

2.5

γ

〈za〉

0 0.2 0.4 0.6 0.8 16

7

8

9

10

11

12

13

14

γ

〈(z

a−

〈za〉)

2〉

Figure 6. Mean 〈za〉 (left) and variance〈(za − 〈za〉)2〉 (right) of the analysis for ETKF (blue, crosses) and VLKF (green, circles), averaged overtime and sites. The dashed black lines shows the climatic mean and varianceµclim andσ2

climof the truth withγ = 1. We usedRo = (0.25σclim)2I,

Nobs = 4, ∆tobs = 48 hours.

error covariance matrices which are used to initialize ananalysis cycle are larger thanσ2

clim for the ETKF. On theother hand, the VLKF limits the analysis error covariance(projected onto the pseudo-observables) from above toσ2

clim, independent of the observation interval.

It is seen in Figure 7 that for moderate observationintervals∆tobs (24 hours), the VLKF performs worse thanthe ETKF with skills S < 1 decreasing with increasingmodel error (i.e. decreasing values ofγ). On the contrary,for large observation intervals with∆tobs > 36 hours,the skill S > 1 increases with increasing model error.

Furthermore, for moderate observation intervals the VLKFperforms increasingly worse than the poor man’s analysiswith increasing model error, but outperforms it for allvalues of γ for large observation intervals. The ETKF,however, always performs worse than the poor man’sanalysis for all observation intervals in the presence ofsufficiently large model error. We first explain the behaviorof the skill observed for moderate observation intervals andthen for large observation intervals.

Over moderate observation intervals the forecastensemble acquires only little spread, as already encountered




in Figure 1. This small spread in conjunction with modelerror can lead to instances where the analysis meanperforms large excursions away from the truth, therebyspoiling the analysis. This effect is stronger for the VLKFas it is initialized with a smaller ensemble spread associatedwith the constrained analysis error covariance as discussedabove. Increasing the model error by decreasingγ increasesthe probability of the ensemble mean diverging fromthe truth, thereby decreasing the skill. This explains thepoor performance of the VLKF over ETKF withS < 1for moderate∆tobs, and the poor performance of boththe ETKF and VLKF over the poor man’s analysis withSE < 1 andSV < 1 as illustrated in Figure 7.

For large observation intervals the situation is different.Large observation intervals are characterized by theforecast dynamics having sufficiently decorrelated sothat the forecast ensemble will have equilibrated with(γ-dependent) covariance, having lost memory of thebackground initialization. This has the effect that bothfilters will acquire comparable forecast error covariancePf

over the forecast interval∆tobs. The large spread which isacquired by the forecast ensemble during large observationintervals prohibits the analysed pseudo-observablestracking the truth well (cf. Figure 4). Skill improvementscan only be achieved by 1.) tracking the truth for theobserved variables as guided by the observations, and2.) assuring the correct climatological behaviour for thepseudo-observables. This latter point is where the variancelimiting method is beneficial, as illustrated in Figure 6. Themore accurate statistical properties of the VLKF analysisresults in improved skill withS > 1 for all values ofγ asshown in Figure 7.

However, Figure 7 also shows that the increased skillS for large observation intervals with increasing modelerror comes at the price of the VLKF becoming lessable to outperform the poor man’s analysis with errorE . As expected,SV asymptotes to1 from above fordecreasing values ofγ, whereasSE becomes less than1for sufficiently large model error. We found that the VLKFperforms better than the poor man’s analysis for all valuesof γ for ∆tobs > 36 hours.

We conclude that the VLKF may be beneficial forensemble based data assimilation with large underdampingmodel error and large observation intervals∆tobs. Forsmaller observation intervals∆tobs < 18 hours withsufficient model error (not shown) the standard ETKFperforms better than the VLKF. However, for small∆tobs

with a near-perfect model (γ ≈ 1) it is beneficial touse the VLKF, as shown in Gottwaldet al. (2011). Forlarge observation intervals the overall skill increases withincreasing model errorγ due to the imposed control of theincreased error covariances. However, there is a trade-offbetween increased skill improvement of the VLKF overETKF, and of the skill improvement of the VLKF over thepoor man’s analysis.

We now look at the dependence of the skill on thesparsity of the observational gridNobs in the presenceof model error. Numerical simulations show that thecritical observation interval above which we observe skillimprovement withS > 1 for all values of γ decreases

with increasing sparsityNobs. Note that this impactson the notion of what is considered to be moderate orlarge observation interval in the discussion above. Thisdependence of skill on observational sparsity can beexplained by noticing that sparser observations imply anincreased forecast error covariance and ensemble spreadfor a given observation interval, as illustrated in Figure 5.We have already seen that the main contribution to theoverall RMS error stems from the pseudo-observables.Therefore one may be tempted to conclude that skillimprovement consistently gets better for increasingNobs.This is correct when only comparing the VLKF and ETKF.However, as in the case of increasing model error forfixed sparsity discussed above, the increased sparsity forfixed model error causes the VLKF to be less effective inoutperforming the poor-man’s analysis. Figure 8 illustrateshow increased sparsity affects the overall RMS errorE .It is seen that the ETKF increasingly performs worsethan the poor-man’s analysis with increasing model errorand increasingNobs. In contrast, the RMS errors for theVLKF are below the RMS error of the poor man’s analysisE for all values ofγ, and they asymptote toE for largesparsity and large model error, in accordance with ourobservations in Figure 5 on the singular values of the errorcovariances of the VLKF. The trade-off between gain inskill of the VLKF over ETKF and less good skill withrespect to the poor-man’s analysis for increasing sparsityNobs can be explained as follows. For sparse observationsinformation from the observables cannot propagate todistant pseudo-observables within the observation interval∆tobs (cf. Figure 2). This causes the forecast and analysisfor those pseudo-observables to be essentially randomvariables with temporal meanµclim and varianceσ2

clim

for the VLKF, andµ(γ) andσ2(γ) for the ETKF. Henceincreased sparsity leads toEV asymptoting from below toEfor all values ofγ, with increasingly worse performance fordecreasing dampingγ. In contrast, for the ETKFEE > Efor sufficiently large sparsity, for all values ofγ.

It is illuminating to see how the RMS errors aredistributed over the observed variables and the unobservedvariables. In Figure 9 we depict the RMS errorsEo

of the observed variables andEuo of the unobservedvariables, as defined in (5). We restrict our discussion tolarge observation intervals∆tobs = 48 hours. Consistentwith our observation that the overall skill improvementis dominated by the superior analysis of the pseudo-observables,Euo exhibits the same behaviour and trends asthe overall RMS errorE (cf. Figures 7 and 8). The errorfor the observed variables shows different behaviour. It isseen that the standard ETKF produces better or equal RMSerrors than the VLKF for all values of the model errorγ.The inferior performance of the VLKF over ETKF for theobserved variables is an instance of underestimation ofthe error covariances – the familiar situation in ensembledata assimilation. As seen in Figure 5, where we show theaverage maximal singular valuesH of HPaH

T for theETKF and VLKF, the variance constraint of the VLKFfor the pseudo-observations also significantly decreasesthe covariance in the observed subspace. The observedpoor performance of the VLKF for the observed variablessuggests that the limited covariance in the observedsubspace is insufficient to provide enough ensemble spreadfor a reliable analysis.




0 0.2 0.4 0.6 0.8 10.85

0.9

0.95

1

1.05

1.1

1.15

1.2

γ

S

0 0.2 0.4 0.6 0.8 10.8

0.85

0.9

0.95

1

1.05

1.1

1.15

γ

SV

0 0.2 0.4 0.6 0.8 10.8

0.85

0.9

0.95

1

1.05

1.1

1.15

γ

SE

Figure 7. Skill S as a function of dampingγ for ∆tobs = 24 hours (red, squares),∆tobs = 36 hours (green, circles) and∆tobs = 48 hours (blue,crosses). We usedRo = (0.25σclim)2I, Nobs = 4 and averaged all results over 200 realizations where one realization was performed over300 days.With ∆tobs = 48 hours, the RMS error in the ETKF analysisEE ranges from 3.13 (forγ = 1) to 3.57 (forγ = 0.05), while the RMS error in theVLKF analysisEV ranges from 3.08 (forγ = 1) to 3.15 (forγ = 0.05).

2 4 6 8 102.5

3

3.5

4

Nobs

E

ETKF

2 4 6 8 102.5

3

3.5

4

Nobs

VLKF

Figure 8. Overall RMS errorE as a function of the observational grid sparsityNobs for different values ofγ. We show the perfect model caseγ = 1

(circles, blue),γ = 0.5 (crosses, green) andγ = 0.05 (triangles, red). The dashed line denotesE . The left panel shows results for the ETKF, the rightpanel for the VLKF. We usedRo = (0.25σclim)2I and∆tobs = 48 hours.

For small model error, both filters produce worse analysesthan an analysis consisting of observations only. Thisis again linked to insufficient spread of the forecastensembles. We have confirmed that for larger ensemblesizesEo <

√Ro for all γ with both the ETKF and VLKF.

Underestimation of error covariances due to finite sizeeffects in ensemble filters are usually controlled by someform of covariance inflation, such as multiplicative inflation

whereby the prior forecast error covariance is multiplied byan inflation factorδ > 1 (Anderson and Anderson 1999).In Figure 10 we show how the RMS error of the observedvariablesEo decreases below the RMS error of the observedvariables forγ = 1 and increasing inflation factorδ. Wenote that for very large values ofδ in the perfect modelsituation the ETKF and VLKF produce analyses of equalRMS error. Increasing the forecast covariance inflation




has the same effect as increasing the model error (i.e.decreasingγ) in producing larger forecast ensemble spread.

Further, it is seen in Figure 9 that for sufficiently largemodel error, the RMS errors of the observablesEo for theVLKF and ETKF converge to the sameγ-independentvalue Eo <

√Ro. This can be explained by the greater

inflation for less damped forecast models; less modeldamping increases the forecast error covariancePf , and sothe analysis tends to trust the observations more. As such,Eo → √

Ro asγ → 0 for both filters, albeit slightly reducedby the Kalman filtering. Due to the variance constraint theRMS error of the observed variablesEo asymptotes toEo atlarger model error for a given observational sparsity.The value of the model dampingγ below which theforecast variance is sufficiently large for the filter to trustthe observations depends on the sparsityNobs. The largerthe sparsityNobs, the larger the value ofγ for which theRMS errorEo of the two filters collapses onto the constantvalueEo. ForNobs = 8, Eo ≈ Eo for all values ofγ for theETKF andEo assumes this value at roughlyγ < 0.7 for theVLKF. For Nobs = 3, Eo ≈ Eo for γ < 0.7 for the ETKFand forγ < 0.25 for the VLKF. This is linked again to theincreased forecast error covariances caused by the sparsity,as illustrated in Figure 5. The top left panel in Figure 5shows that for a given observation interval and model errorγ, the forecast error covariance increases with sparsityNobs, causing the observations to be weighted more by theanalysis.

We remark that while for the observed variablesthere is generally little or no improvement in the RMSerror of the VLKF over ETKF, the VLKF still createsmore reliable ensembles, as characterised by the rankedprobability histogram (also known as Talagrand diagram)(Anderson 1996; Hamill and Colucci 1997; Talagrandet al. 1997). In such a histogram the forecast ensembleis sortedXf = [xf,1, xf,2, ..., xf,k] and bins(−∞, xf,1],(xf,1, xf,2], ... , (xf,k,∞) are generated at each forecaststep. We then increment whichever bin the truth fallsinto at each forecast step to produce a histogram ofprobabilities Pi of the truth being in bini. A reliableensemble is considered to be one where the truth and theensemble members can be viewed as drawn from the samedistribution. A flat probability density histogram thereforeis seen as indicating a reliable ensemble for which eachensemble member has equal probability of being nearest tothe truth. A convex probability density histogram indicatesa lack of spread of the ensemble, while a concave diagramindicates an ensemble which exhibits too large spread. InFigure 11 we show ranked probability histograms for anobserved variable withγ = 0.5, and all other parametersas in Figure 9. The figure shows that forNobs = 3, 4 and8 the VLKF makes the forecast ensemble more reliable bycontrolling the spread.

In Figure 12 we show how the overall skill, as well asthe climatic skillsSE andSV, vary with the observationalnoiseη, whereRo = (ησclim)2I. The VLKF outperformsboth the ETKF and the observations/climatology for largeenough noiseη ≥ 0.25. Large observational noise givespreference to the pseudo-observables, whose unrealisticallylarge error covariances as seen in Figure 5 are limited bythe VLKF. This is particularly true for large model error.

It is seen that for sufficiently large observational noiseη ≥ 0.25, the VLKF performs better than a poor man’sanalysis for all values of the model errorγ.

5. Discussion

Our numerical results suggest that including climatologicalinformation of the mean and variance of the unobservedvariables in an ensemble filter can be beneficial in the casewhen the forecast error covariances are overestimated.In particular, this occurs when sparse observations aretaken at large observation intervals and the forecastmodel is sufficiently underdamped. However, we havealso encountered situations where the additional varianceconstraint provided by the VLKF causes error covariancesto be underestimated, spoiling the analysis (albeit only forthe observables and not, as has to be stressed, for the overallRMS error). This is the case for moderate observationintervals with large model error, and also for largeobservation intervals in the case of near-perfect models,but only if one is interested in the analysis of just theobservables. This latter situation is not particularly limiting,as RMS errors for the observables are relatively smallcompared to those for pseudo-observables, and the finitesize effects can be eliminated here by basic multiplicativecovariance inflation. For sufficiently large observationalnoise levels, when the analysis puts more weight on theforecast, controlling the unrealistic overestimation causedby underdamping model error using the VLKF becomesthe more effective strategy.

We argue that model error associated with anunderdamped forecast model leads to an increased forecasterror covariance and increased associated ensemble spreadby the same mechanism by which covariance inflationis beneficial. The variance limiting filter controls thisincreased spread and keeps it within the bounds of theclimatologically reasonable bounds, thereby leading to alower RMS error and improving the overall skill.

We remark that one may choose to drive the systemtowards values other than the climatological mean andvariance. The variance limitation can be viewed as a simplestrategy to numerically stabilize the filtering process. Assuch it may be more beneficial to choose values differingfrom climatology which are given by numerical necessitiesrather than by dynamical and physical considerations. Forexample, in non-equilibrium situations where the dynamicsis locally off the attractor, such as in the presence of strongfronts, the climatological values are not meaningful. Ourapproach still provides a means to control the dynamicallynatural overestimation to numerically stabilize the filteringprocedure, however this would imply careful tuning ofthe limiters Aclim and aclim. Furthermore, it would becomputationally impossible in a full-scale model to treat allof the unobserved variables as pseudoobservables, so onewould need to choose carefully which of the unobservedvariables to apply the VLKF to.

It was noted in Gottwaldet al.(2011) that the RMS errorsare not sensitive to uncertainties in the estimation ofaclim

andAclim, which may be obtained from historical data orfrom long time simulations of numerical models. However,instead of damping all pseudoobservables towards the




0 0.2 0.4 0.6 0.8 1

0.9

0.95

1

1.05

Eo

γ0 0.2 0.4 0.6 0.8 1

0.9

0.95

1

1.05

γ0 0.2 0.4 0.6 0.8 1

0.9

0.95

1

1.05

γ

0 0.2 0.4 0.6 0.8 1

3.4

3.6

3.8

4

4.2

Euo

γ0 0.2 0.4 0.6 0.8 1

3.4

3.6

3.8

4

4.2

γ0 0.2 0.4 0.6 0.8 1

3.4

3.6

3.8

4

4.2

γ

Figure 9. RMS errors of observed variablesEo (top) and of unobserved variablesEuo (bottom) for the ETKF (blue, dashed, crosses) and VLKF(green, dashed, circles) forNobs = 3 (left), Nobs = 4 (centre) andNobs = 8 (right). Top and bottom horizontal dashed lines show

√Ro andσclim,

respectively. We usedRo = (0.25σclim)2I and∆tobs = 48 hours.

0.05 0.1 0.15 0.2 0.250.85

0.9

0.95

1

1.05

δ

Eo

Figure 10. RMS errorsEo for ETKF (blue, dashed, crosses) and VLKF (green, dashed, circles) for increasing forecast covariance inflationδ. The dashedline shows

√Ro. We usedγ = 1, ∆tobs = 48 hrs,Nobs = 3.

same spatially homogeneous climatological valueaclim,one may wish to damp towards a spatially inhomogeneousfinite-time statistics. One may, for example, use a runningmean of the past history of analysesza instead of theconstantaclim we have used here. However, for the longobservation times used here, the analysis is not tracking(cf. Figure 4). Therefore driving the pseudoobservablestowards non-climatological values ofaclim will introduceon average larger RMS errors, degrading the skill. It isexactly the situation of large observational intervals wherethe quality of an analysis cannot be measured by how wellit estimates the actual truth but by how well the statistics ofthe truth is reproduced. The situation of large observation

intervals and sparse observational grids is particularlyrelevant for climate reanalysis in the pre-radiosonde era.Inthis case long time climatology was shown to be beneficialas additional information in data assimilation. Our resultsshow that model error may provide the necessary ensemblespread needed to combat finite ensemble size effects,similarly to covariance inflation; VLKF controls this spreadby limiting unrealistically large analysis error covariances.

The problem we addressed here is closely related toforecast bias correction schemes (Takacs 1996). Dee andDa Silva (1998) introduced a filtering strategy where aforecast bias is estimated using a running mean which




1 10 20 30 400

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

Pi,o

i1 10 20 30 40

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

i1 10 20 30 40

0

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

i

Figure 11. Ranked probability histograms for the observables for ETKF(blue) and VLKF (green), withNobs = 3 (left), Nobs = 4 (centre) andNobs = 8 (right). We usedγ = 0.5, all other parameters as in Figure 9.

0 0.2 0.4 0.6 0.8 10.95

1

1.05

1.1

1.15

γ

S

0 0.2 0.4 0.6 0.8 1

0.9

0.95

1

1.05

1.1

γ

SE

0 0.2 0.4 0.6 0.8 1

0.9

0.95

1

1.05

1.1

γ

SV

Figure 12. Overall skillS (top) and climatic skillsSE (bottom left) andSV (bottom right) as a function ofγ for different levels of observational noiseη, whereRo = (ησclim)2I. Hereη = 1 (blue, crosses),η = 0.5 (green, circles),η = 0.25 (red, squares) andη = 0.15 (blue-green, triangles). Weused∆tobs = 48 hours andNobs = 4.

was then subtracted from the forecast prior to the analysis.This strategy was applied to the Goddard Earth ObservingSystem moisture analysis (Dee and Todling 2000). Weare dealing here with the additional problem of sparseobservational grids. Both approaches, however, gain theirskill improvement from driving the dynamics towards amean which was estimated from the preceding dynamicsusing a running average rather than allowing the analysisto drift away due to forecast bias caused as in our case bymodel error.

We have used here underdamping as a model fornumerically unstable dynamic cores; our forecast models

still produce stable dynamics, albeit with significantlyincreased covariances. The effectiveness of our approachat controlling actual numerical instabilities remains anopen question planned for further research. We mentionthe work by Grote and Majda (2006) who addressed asimilar problem and designed a filter which assures anobservability condition in the case of unstable dynamicsin the dynamic core where the unstable dynamics is tamedby appropriately incorporating observations. Furthermore,we have not addressed the important issue raised inthe introduction whether the proposed method is able toproduce realistic energy spectra and control the energy




dissipation rates. This again may require tuning of thelimiters and is planned for further research.

Acknowledgements

We thank Sebastian Reich for bringing the problemof underdamping model error to our attention. LMacknowledges the support of an Australian PostgraduateAward. GAG acknowledges support from the AustralianResearch Council.

References

Anderson JL. 1996. A method for producing and evaluating probabilisticforecasts from ensemble model integrations.Journal of Climate9(7):1518–1530.

Anderson JL. 2001. An ensemble adjustment Kalman filter for dataassimilation.Monthly Weather Review129(12): 2884–2903.

Anderson JL, Anderson SL. 1999. A Monte Carlo implementation of thenonlinear filtering problem to produce ensemble assimilations andforecasts.Monthly Weather Review127(12): 2741–2758.

Bengtsson L, Arkin P, Berrisford P, Bougeault P, Folland CK,GordonC, Haines K, Hodges KI, Jones P, Kallberg P, Rayner N, SimmonsA, Stammer D, Thorne PW, Uppala S, Vose RS. 2007. Theneed for a dynamical climate reanalysis.Bulletin of the AmericanMeteorological Society88(4): 495–501.

Bergemann K, Gottwald GA, Reich S. 2009. Ensemble propagation andcontinuous matrix factorization algorithms.Quarterly Journal of theRoyal Meteorological Society135(643): 1560–1572.

Bishop CH, Etherton BJ, Majumdar SJ. 2001. Adaptive sampling withthe Ensemble Transform Kalman Filter. Part I: Theoretical aspects.Monthly Weather Review129(3): 420–436.

Blumen W. 1990. A semi-geostrophic Eady-wave frontal modelincorporating momentum diffusion. Part II: Kinetic energyandenstrophy dissipation.Journal of the Atmospheric Sciences47: 2903–2908.

Buizza R, Houtekamer PL, Pellerin G, Toth Z, Zhu Y, Wei M. 2005.A comparison of the ECMWF, MSC, and NCEP global ensembleprediction systems.Monthly Weather Review133(5): 1076–1097.

Burgers G, van Leeuwen PJ, Evensen G. 1998. Analysis scheme inthe ensemble Kalman filter.Monthly Weather Review126(6): 1719–1724.

Charron M, Pellerin G, Spacek L, Houtekamer PL, Gagnon N, MitchellHL, Michelin L. 2010. Toward random sampling of model error inthe Canadian ensemble prediction system.Monthly Weather Review138(5): 1877–1901.

Compo GP, Whitaker JS, Sardeshmukh PD, Matsui N, Allan RJ, YinX, Gleason BE, Vose RS, Rutledge G, Bessemoulin P, BronnimannS, Brunet M, Crouthamel RI, Grant AN, Groisman PY, Jones PD,Kruk MC, Kruger AC, Marshall GJ, Maugeri M, Mok HY, NordliV, Ross TF, Trigo RM, Wang XL, Woodruff SD, Worley SJ. 2011.The Twentieth Century Reanalysis Project.Quarterly Journal of theRoyal Meteorological Society137(654): 1–28.

Cote J, Staniforth A. 1988. A two-time-level semi-Lagrangian semi-implicit scheme for spectral models.Monthly Weather Review116(10): 2003–2016.

Dee DP, Da Silva AM. 1998. Data assimilation in the presence offorecast bias.Quarterly Journal of the Royal Meteorological Society124: 269–295.

Dee DP, Todling R. 2000. Data assimilation in the presence offorecastbias: The GEOS moisture analysis.Monthly Weather Review128:3268–3282.

Dubinkina S, Frank J. 2007. Statistical mechanics of Arakawa’sdiscretizations.Journal of Computational Physics227(2): 1286 –1305.

Dubinkina S, Frank J. 2010. Statistical relevance of vorticityconservation in the Hamiltonian particle-mesh method.Journal ofComputational Physics229(7): 2634 – 2648.

Durran DR. 1999. Numerical Methods for Wave Equations inGeophysical Fluid Dynamics. Springer: New York.

Eckermann SD, Hoppel KW, Coy L, McCormack JP, Siskind DE,Nielsen K, Kochenash A, Stevens MH, Englert CR, Singer W, Hervig

M. 2009. High-altitude data assimilation system experiments for thenorthern summer mesosphere season 2007.Journal of Atmosphericand Solar-Terrestrial Physics71(3-4): 531–551.

Evensen G. 1994. Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast errorstatistics.Journal of Geophysical Research99(C5): 10 143–10 162.

Evensen G. 2006.Data Assimilation: The Ensemble Kalman Filter.Springer: New York.

Fisher M, Leutbecher M, Kelly GA. 2005. On the equivalence betweenKalman smoothing and weak-constraint four-dimensional variationaldata assimilation.Quarterly Journal of the Royal MeteorologicalSociety131(613): 3235–3246.

Frederiksen JS, Davies AG. 1997. Eddy viscosity and stochasticbackscatter parameterizations on the sphere for atmosphericcirculation models.Journal of the Atmospheric Sciences54(20):2475–2492.

Golub GH, Loan CFV. 1996.Matrix Computations. The Johns HopkinsUniversity Press: Baltimore, 3rd edn.

Gottwald GA, Mitchell L, Reich S. 2011. Controlling overestimation oferror covariance in ensemble Kalman filters with sparse observations:A variance limiting Kalman filter.Monthly Weather Review139(8):2650–2667.

Grote MJ, Majda AJ. 2006. Stable time filtering of strongly unstablespatially extended systems.Proceedings of the National Academy ofSciences103(20): 7548–7553.

Hamill TM, Colucci SJ. 1997. Verification of Eta/RSM short-rangeensemble forecasts.Monthly Weather Review125(6): 1312–1327.

Hamill TM, Whitaker JS. 2011. What constrains spread growthinforecasts initialized from ensemble Kalman filters.Monthly WeatherReview139: 117–131.

Houtekamer PL, Mitchell HL. 1998. Data assimilation using anensemble Kalman filter technique.Monthly Weather Review126(3):796–811.

Houtekamer PL, Mitchell HL, Pellerin G, Buehner M, Charron M,Spacek L, Hansen B. 2005. Atmospheric data assimilation with anensemble Kalman filter: Results with real observations.MonthlyWeather Review133(3): 604–620.

Kalnay E. 2002. Atmospheric Modeling, Data Assimilation andPredictability. Cambridge University Press: Cambridge.

Leimkuhler B, Reich S. 2005.Simulating Hamiltonian Dynamics.Cambridge University Press: Cambridge.

Liu J, Fertig EJ, Li H, Kalnay E, Hunt BR, Kostelich EJ, SzunyoghI, Todling R. 2008. Comparison between local ensemble transformKalman filter and PSAS in the NASA finite volume GCM - perfectmodel experiments.Nonlinear Processes in Geophysics15(4): 645–659.

Lorenz EN. 1996. Predictability - a problem partly solved. In:Predictability, Palmer T (ed), European Centre for Medium-RangeWeather Forecast, Shinfield Park, Reading, UK.

Majda AJ, Timofeyev I, Vanden Eijnden E. 1999. Models for stochasticclimate prediction.Proceedings of the National Academy of Sciences96(26): 14 687–14 691.

Mitchell L, Gottwald GA. 2012. Data assimilation in slow-fast systemsusing homogenized climate models.Journal of the AtmosphericSciences69(4): 1359–1377.

Molteni F, Palmer TN. 1993. Predictability and finite-time instabilityof the northern winter circulation.Quarterly Journal of the RoyalMeteorological Society119: 269–298.

Ott E, Hunt B, Szunyogh I, Zimin A, Kostelich E, Corrazza M, KalnayE, Yorke J. 2004. A local ensemble Kalman filter for atmosphericdata assimilation.Tellus A56(5): 415–428.

Palmer T, Williams P (eds). 2010.Stochastic physics and climatemodelling. Cambridge University Press: Cambridge.

Palmer TN. 2001. A nonlinear dynamical perspective on modelerror:A proposal for non-local stochastic-dynamic parametrization inweather and climate prediction models.Quarterly Journal of theRoyal Meteorological Society127(572): 279–304.

Polavarapu S, Shepherd TG, Rochon Y, Ren S. 2005. Some challengesof middle atmosphere data assimilation.Quarterly Journal of theRoyal Meteorological Society131(613): 3513–3527.

Ritchie H. 1988. Application of the semi-Lagrangian methodto aspectral model of the shallow-water equations.Monthly WeatherReview116: 1587–1598.




Salmon R. 1998.Lectures on Geophysical Fluid Dynamics. OxfordUniversity Press: New York.

Sankey D, Ren S, Polavarapu S, Rochon Y, Nezlin Y, Beagley S. 2007.Impact of data assimilation filtering methods on the mesosphere.Journal of Geophysical Research112(D24): 104–118.

Shutts GJ. 2005. A stochastic kinetic energy backscatter algorithm foruse in ensemble prediction systems.Quarterly Journal of the RoyalMeteorological Society131(612): 3079–3102.

Simon DJ. 2006.Optimal State Estimation. John Wiley & Sons, Inc.:New York.

Skamarock WC, Klemp JB. 1992. The stability of time-split numericalmethods for the hydrostatic and the nonhydrostatic elasticequations.Monthly Weather Review120(9): 2109–2127.

Takacs LL. 1996. A simple bias correction algorithm for use in dataassimilation.DAO Office Note 96-25.

Talagrand O, Vautard R, Strauss B. 1997. Evaluation of probabilisticprediction systems. In:Proceedings of the ECMWF Workshop onPredictability. ECMWF.

Thuburn J. 2008. Some conservation issues for the dynamicalcores ofNWP and climate models.Journal of Computational Physics227(7):3715–3730.

Tippett MK, Anderson JL, Bishop CH, Hamill TM, Whitaker JS. 2003.Ensemble square root filters.Monthly Weather Review131(7): 1485–1490.

Toth Z, Kalnay E. 1993. Ensemble forecasting at NMC: The generationof perturbations.Bulletin of the American Meteorological Society74:2317–2330.

Wang X, Bishop CH, Julier SJ. 2004. Which is better, an ensembleof positive-negative pairs or a centered spherical simplexensemble?Monthly Weather Review132(7): 1590–1505.

Whitaker JS, Compo GP, Thepaut JN. 2009. A comparison of variationaland ensemble-based data assimilation systems for reanalysis ofsparse observations.Monthly Weather Review137(6): 1991–1999.

Whitaker JS, Compo GP, Wei X, Hamill TM. 2004. Reanalysis withoutradiosondes using ensemble data assimilation.Monthly WeatherReview132(5): 1190–1200.



Controlling model error of underdamped forecast models in ... · can signiﬁcantly improve...

Documents

Transcript of Controlling model error of underdamped forecast models in ... · can signiﬁcantly improve...