1

Hybrid variational-ensemble data assimilation

Daryl T. Kleist

Kayo Ide, Dave Parrish, John Derber, Jeff Whitaker

Weather and Chaos Group Meeting 07 March 2011

2

Variational

Data Assimilation

c''

o1T''

o'1

VarT''

Var 21

21 JJ HxyRHxyxBxx

J

: Penalty (Fit to background + Fit to observations + Constraints)x’

: Analysis increment (xa

– xb

) ; where xb

is a backgroundBVar

: Background error covarianceH

: Observations (forward) operator

R

: Observation error covariance (Instrument + representativeness)

yo

’

: Observation innovationsJc

: Constraints (physical quantities, balance/noise, etc.)

B

is typically static and estimated a-priori/offline

3

Motivation from Var

•

Current background error covariance (applied operationally) in VAR –

Isotropic recursive filters

–

Poor handle on cross-variable covariance–

Minimal flow-dependence added

•

Implicit flow-dependence through linearization in normal mode constraint (Kleist et al. 2009)

•

Flow-dependent variances (only for wind, temperature, and pressure) based on background tendencies

–

Tuned NMC-based estimate (lagged forecast pairs)

4

Kalman

Filter in Var

Setting

KFKF BKHIA

ab xx M

yHxKxx bba

1TKF

TKF

HHBRHBK

QMAMB TKFKF

Forecast Step

Analysis

•

Analysis step in variational

framework (cost function)

Extended Kalman Filter

''o

1T''o

'1KF

T''KF 2

121 HxyRHxyxBxx J

•

BKF

: Time evolving background error covariance•

AKF

: Inverse [Hessian of JKF

(x’)]

5

Motivation from KF

TbbKF 1

1 XXB

K

ab XX

•

Problem: dimensions of AKF

and BKF

are huge, making this practically impossible for large systems (GFS for example).

•

Solution: sample and update using an ensemble instead of evolving AKF

/BKF

explicitly

TaaKF 1

1 XXA

K

Ensemble Perturbations

ba XX Forecast Step:

Analysis Step:

6

Why Hybrid?

VAR (3D, 4D)

EnKF Hybrid References

Benefit from use of flow dependent ensemble covariance instead of static B

x x Hamill

and Snyder 2000; Wang et al. 2007b,2008ab, 2009b, Wang 2011; Buehner

et al. 2010ab

Robust for small ensemble x Wang et al. 2007b, 2009b; Buehner

et al. 2010b

Better localization for integrated measure, e.g. satellite radiance

x Campbell et al. 2009

Easy framework to add various constraints

x x

Framework to treat non-

Gaussianity

x x

Use of various existing capabilities in VAR

x x

7

Hybrid Variational-Ensemble

•

Incorporate ensemble perturbations directly into variational

cost function through extended control

variable–

Lorenc

(2003), Buehner

(2005), Wang et. al. (2007), etc.

111

ef

't

'o

1T't

'o

1Te

'f

1T'ff

'f 2

121

21 HxyRHxyLxBxx ,J

K

kkk

1

e'f

't xxx

f

& e

: weighting coefficients for fixed and ensemble covariance respectivelyxt

: (total increment) sum of increment from fixed/static B

(xf

) and ensemble B k

: extended control variable; :ensemble perturbationL: correlation matrix [localization on ensemble perturbations]

ekx

8

Importance of Ensemble Generation Method?

•

GEFS (already operational)–

80 cycled members–

ETR•

Virtually no computational cost•

Uses analysis error mask derived for 500 mb

streamfunction•

Tuned for medium range forecast spread and fast “error growth”–

T190L28

version of the GFS model–

Viable for hybrid paradigm?

•

EnKF–

80 cycled members–

Perturbations specifically designed to represent analysis and background errors

–

T254L64

version of the GFS–

Extra computational costs worth it for hybrid?

9

EnKF/ETR Comparison

EnKF

(green) versus ETR (red) spread/standard deviation for surface

pressure (mb) valid 2010101312

Surface Pressure spread normalized difference (ETR has much less spread,

except poleward

of 70N)

10

EnKF/ETR Comparison

EnKF

zonal wind (m/s) ensemble standard deviation valid 2010101312

ETR zonal wind (m/s) ensemble standard deviation valid 2010101312

11

EnKF/NMC B

Compare

EnKF

zonal wind (m/s) ensemble standard deviation valid 2010101312

Zonal Wind standard deviation (m/s) from “NMC-method”

12

Hybrid with (global) GSI

•

Control variable has been implemented into GSI 3DVAR*–

Full B

preconditioning•

Working on extensions to B1/2

preconditioned minimization options–

Spectral filter for horizontal part of A•

Eventually replace with (anisotropic) recursive filters–

Recursive filter used for vertical–

Dual resolution capability•

Ensemble can be from different resolution than background/analysis (vertical levels are the exception)

–

Various localization options for A•

Grid units or scale height•

Level dependent (plans to expand)–

Option to apply TLNMC (Kleist et al. 2009) to analysis increment

K

k

ekk

1

'f

' xxCx

*Acknowledgement: Dave Parrish for original implementation of extended control variable

13

Single Observation

Single 850mb Tv

observation (1K O-F, 1K error)

14

Single Observation

Single 850mb zonal wind observation (3 m/s

O-F, 1m/s error) in Hurricane Ike circulation

EnKFmember update

member 2 analysis

high resforecast

GSIHybrid Ens/Var

high resanalysis

member 1 analysis

member 2 forecast

member 1 forecast

recenteranalysis ensemble

Dual-Res Coupled Hybrid

member 3 forecast

member 3 analysis

Previous Cycle Current Update Cycle

16

Hybrid Var-EnKF

GFS experiment

•

Model–

GFS deterministic (T574L64; post July 2010 version –

current operational version)–

GFS ensemble (T254L64)•

80 ensemble members, EnKF

update, GSI for observation operators

•

Observations–

All operationally available observations (including radiances)–

Includes early (GFS) and late (GDAS/cycled) cycles as in production

•

Dual-resolution/Coupled•

High resolution control/deterministic component–

Includes TC Relocation on guess•

Ensemble is recentered

every cycle about hybrid analysis –

Discard ensemble mean analysis

•

Satellite bias corrections –

Coefficients come from GSI/VAR

•

Parameter settings•

1/3 static B, 2/3 ensemble•

Fixed localization: 800km & 1.5 scale heights

•

Test Period–

15 July 2010 –

15 October 2010 (first two weeks ignored for “spin-up”)

17

500 hPa

Anom.Corr.Northern Hemisphere Southern Hemisphere

18

AC Frequency DistributionsNorthern Hemisphere Southern Hemisphere

19

Geopotential

Height RMSENorthern Hemisphere Southern Hemisphere

Significant reduction in mean height errors

20

Stratospheric Fits

Improved fits to stratospheric observations

21

Forecast Fits to Obs

(Tropical Winds)

Forecasts from hybrid analyses fit observation much better.

22

23

24

25

HVEDAS (3D) for GDAS/GFS•

Prototype dual-resolution, two-way coupled hybrid Var/EnKF

system outperforms standard 3DVAR in GFS experiments–

2010 Hurricane Season (August 15 through October 31 2010) run off-

site

–

Emphasis on AC, RMSE, TC Tracks

•

Plan underway to implement into GDAS/GFS operationally–

Target: Spring 2012 (subject to many potential issues)•

Porting of codes/scripts back to IBM P6•

Cost analysis (will everything fit in production suite?)•

More thorough (pre-implementation) testing and evaluation–

More test periods (including NH winter)–

Other/more verification metrics•

Potential moratorium associated with move to new NCEP facility

•

Issues–

Weighting between ensemble and static B–

Localization–

How should EnKF

be used within ensemble forecasting

paradigm?

26

HVEDAS Extensions and Improvements

•

Expand hybrid to 4D–

Hybrid within ‘traditional 4DVAR’

(with adjoint)–

Pure ensemble 4DVAR (non-adjoint)–

Ensemble 4DVAR with static B supplement (non-adjoint)*

•

EnKF

improvements–

Explore alternatives such as LETKF–

Adaptive localization and inflation

•

Non-GFS applications in development–

NASA GEOS-5 (GMAO)–

NAM (Dave Parrish, others)–

Hurricanes/HWRF (Mingjing

Tong, HFIP, many collaborators)–

Storm-scale initialization (Jacob Carley, collaborators)–

RR (Xuguang

Wang, Ming Xue, Stan Benjamin, Jeff Whitaker, Steve Weygandt, others)

•

NCEP strives to have single DA system to develop, maintain, and run operationally (global, mesoscale, severe weather, hurricanes, etc.)

–

GSI (including hybrid development) is community code supported through DTC–

EnKF

used for GFS-based hybrid being expanded for use with other applications