Observability, Data Assimilation with the Extended Kalman Filter 1

Observability,a Problem in Data Assimilation

Chris DanforthDepartment of Applied Mathematicsand Scientific Computation, UMD

March 10, 2004

AdvisorsJoaquim Ballabrera, UMD/ESSIC

James Yorke, UMD/IPSTEugenia Kalnay, UMD/METO

D.J. Patil, UMD/IPSTBob Cahalan, NASA/GSFC

Observability, Data Assimilation with the Extended Kalman Filter 2

Sources of Numerical Forecast Error

● Displacementerror (standard chaos)• initial conditions are approximate• indistinguishable conditions of the atmosphere diverge

● Model error• improper physical parameterizations• sub-grid scale phenomena

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 3

Ensemble Forecasts and Shadowing

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 4

Ensemble Forecasts and Shadowing

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 5

Ensemble Forecasts and Shadowing

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 6

Ensemble Forecasts and Shadowing

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 7

Model Error and Nudging

● Conservation law∂q∂t

= F(q)

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 8

Model Error and Nudging

● Conservation law∂q∂t

= F(q)

● Nudge model forecast to truth through relaxation

∂q∂t

= F(q)+qobs−q

τ

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 9

Model Error and Nudging

● Conservation law∂q∂t

= F(q)

● Nudge model forecast to truth through relaxation

∂q∂t

= F(q)+qobs−q

τ● Hourly nudging terms correct state-dependent tendency error● Time-averaged nudging terms represent systematic model error

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 10

Data Assimilation

● Data Assimilation Cycle:

•Start with best guess of initial conditions,background• Integrate model to generate prediction,forecast•Make measurements of truth,observations•Combine model prediction with observations,analysis

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 11

Data Assimilation

● Data Assimilation Cycle:

•Start with best guess of initial conditions,background• Integrate model to generate prediction,forecast•Make measurements of truth,observations•Combine model prediction with observations,analysis

● Sources of difficulty:

•Model gridvs observational grid•Model variablesvs observations•Observability: Does the model respond to measurements?

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 12

Kalman Filter

● Analysis cycle: Combine forecast with observations

K[o−H f

]+ f = a

● OperatorH transforms model forecast statefinto the space of observationo

● Kalman Gain matrixK weights the observational incrementwith knowledge of confidence in measurements and forecast

● Analysis statea is our new best guess

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 13

Lorenz Model

dxdt

= σy(t)−σx(t)

dydt

= ρx(t)−x(t)z(t)−y(t)

dzdt

= x(t)y(t)−βz(t)

● Solutions represent simplified convection in the atmosphere● Chaotic for certain parameter values● Suitable for testing data assimilation techniques

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 14

Twin Experiments

● Generate reference state (truth) from model integrationof an arbitrary initial condition

● Start forecast from adifferent arbitrary initial state● Observe truth at relevant time steps, combine with forecast● Generate analysis (best estimate) of current state

● Does the forecast stay close to the truth?

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 15

Plot of y(t) , assimilatingx every two time steps

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 16

Relative Error remains small observingx,y and combinations

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 17

Plot of x(t) , assimilatingz every time step

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 18

Observability Conclusions

● The EKF fails to push forecasts to truth in the classic toy weathermodel of Lorenz, when measuring the variablez

● Nonlinear systems do not necessarily respond to assimilation ofall state variables, not all measurements are the same!

● Operational weather models need to be tested for observability

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 19

Current Work

● Researching 40-d Lorenz model to develop techniques of ensem-ble variance inflation

● Model error experiments with Marshall and Molteni global 3-levelQG model show nudging terms correct model bias

● Displacement error and model error cooperate to destroy weatherforecasts......to keep the truth contained within our ensemble ellipse, and toevaluate/ correct model error, we MUST effectively assimilate ac-curate and representative observations!

Department of Applied Mathematics and Scientific Computing/University of Maryland

Observability, Data Assimilation with the Extended Kalman Filter 20

References

[1] Robert Miller, Michael Ghil, Francois Gauthiez,Advanced DataAssimilation in Strongly Nonlinear Dynamical Systems, Journal ofthe Atmospheric Sciences, Vol. 51, No. 8, April 1994.[2] Eugenia Kalnay,Atmospheric Modeling, Data Assimilation andPredictability, Cambridge University Press, 2002[3] Edward Lorenz, ”Predictability - a problem partly solved”, inPredictability, edited by T. Palmer, European Centre for Medium-Range Weather Forecasting, Shinfield Park, Reading, UK, 1996.[4] D.J. Patil, E. Ott, B.R. Hunt, E.Kalnay, J.A. Yorke,Local lowdimensionality of atmospheric dynamics, Physical Review Letters,Vol. 86, No 26, 2001, 5878–5881.

The End — Thank you Contact: danforth@math.umd.edu