The recent developments of WRFDA(WRF-Var)

Hans Huang, NCAR

WRFDA: WRF Data AssimilationWRF-Var: WRF Variational data assimilation

Acknowledge: NCAR/ESSL/MMM/DAG, NCAR/RAL/JNT/DATC,AFWA, USWRP, NSF-OPP, NASA, AirDat, KMA, CWB, CAA, BMB, EUMETSAT

Outline

1. WRFDA overview

2. A few new capabilities

3. Incremental formulation and outer-loop

4. Forecast error sensitivity to observations

5. Future plan and Summary

WRF-Var (WRFDA) Data Assimilation Overview

• Goal: Community WRF DA system for • regional/global, • research/operations, and • deterministic/probabilistic applications.

• Techniques: • 3D-Var• 4D-Var (regional)• Ensemble DA, • Hybrid Variational/Ensemble DA.

• Model: WRF (ARW, NMM, Global)• Support:

• NCAR/ESSL/MMM/DAG• NCAR/RAL/JNT/DATC

• Observations: Conv.+Sat.+Radar

The WRF-Var Program• NCAR staff: 15FTE

• Non-NCAR collaborators: ~10FTE.

• Community users: ~30 (more in 6000 general WRF downloads?).

WRF-Var Observations In-Situ:

- Surface (SYNOP, METAR, SHIP, BUOY).- Upper air (TEMP, PIBAL, AIREP, ACARS).

Remotely sensed retrievals:- Atmospheric Motion Vectors (geo/polar).- Ground-based GPS Total Precipitable Water.- SSM/I oceanic surface wind speed and TPW.- Scatterometer oceanic surface winds.- Wind Profiler.- Radar radial velocities and reflectivities.- Satellite temperature/humidities.- GPS refractivity (e.g. COSMIC).

Radiative Transfer:- RTTOVS (EUMETSAT).- CRTM (JCSDA).

2004082600 ~ 2004092812

Threshold = 5.0mm

TIME

3 6 9 12 15 18 21 240.0

0.2

0.4

0.6

0.8

1.0

0.00

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

Th

reat

Sco

re

Bias

KMA Pre-operational Verification:

(with/without radar)

WRF-Var Radiance Assimilation StatusLiu and Auligne

• BUFR 1b radiance ingest.

• RTM interface: RTTOV or CRTM

• NESDIS microwave surface emissivity model

• Range of monitoring diagnostics.

• Quality Control for HIRS, AMSU, AIRS, SSMI/S.

• Bias Correction (Adaptive, Variational in 2008)

• Variational observation error tuning

• Parallel: MPI

• Flexible design to easily add new satellite sensors

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

DMSP(SSMI/S)

Aqua (AMSU, AIRS)

NOAA (HIRS, AMSU)

The first WRF-Var tutorial

• July 21-22, 2008

• 9 hours lectures and 4 hours hands on

• 53+ participants, US and international

WRF-Var tutorial agenda

http://www.mmm.ucar.edu/events/tutorial_708/agenda/agenda.php

WRF-Var tutorial presentations

http://www.mmm.ucar.edu/wrf/users/tutorial/tutorial_presentation.htm

WRF-Var online tutorial and user guide

http://www.mmm.ucar.edu/wrf/users/docs/user_guide_V3/users_guide_chap6.htm

Outline

1. WRFDA overview

2. A few new capabilities

3. Incremental formulation and outer-loop

4. Forecast error sensitivity to observations

5. Future plan and summary

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

AIRS assimilation: Experiments

T8 domain

45km horizontal resolution

57 vertical levels (up to 10hPa)

Domain-specific background error covariances (ensemble-based)

48h forecasts at 00UTC and 12UTC

Experiments: Conventional Data (CONV) Conventional + AIRS (AIRS)

AIRS assimilation: Impact on forecast

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

BIASBIAS RMSERMSECONVCONV AIRSAIRS

UU VV

TT QQ

UU VV

TT QQ

24h forecasts minus conventional observations over a 30-day period

WRF 4D-Var Summary• 4D-Var included within WRF-Var.• Linear/adjoint models based on WRF-

ARW.• Status:

• Parallel code, JcDFI, limited physics.

• Delivered to AFWA in 2006 and 2007. (2008)

• Current focus: PBL/microphysics, optimization.

• Advantages of 4D-Var • Flow-dependent response to obs

• Better treatment of cloud/precip obs

• Forecast model as a constraint• Obs at obs-times

=> Xin’s 4D-Var talk

4D-Var

3D-Var

WRF-Var and NMM (Pattanayak and Rizvi)Analysis increments

Global WRF-Var (Rizvi and Duda)

Analysis increments

Use ensemble information in QC (Yongsheng Chen)

2007.08Number of rejected observations

Outline

1. WRFDA overview

2. A few new capabilities

3. Incremental formulation and outer-loop

4. Forecast error sensitivity to observations

5. Future plan and summary

J =12

x−xb( )TB−1 x−xb( ) +

12

y−H x( )( )TR−1 y−H x( )( )

J =12

x−xg + xg −xb( )TB−1 x−xg + xg −xb( ) +

12

y−H xg( ) + H xg( )−H x( )( )TR−1 y−H xg( ) + H xg( )−H x( )( )

d =y−H xg( ) H x( )−H xg( ) ≈Hδx

J =12

δx+ xg −xb( )TB−1 δx+ xg −xb( ) +

12

d−Hδx( )T R−1 d−Hδx( )

δx = x − xg

J =12δxTB−1δx+

12

d−Hδx( )T R−1 d−Hδx( )

3D-Var (4D-Var replace H by HM)

The incremental formulation (in the general form, !)xg ≠xb

The first outer-loop: xg = xb

Outer-loop:

d (and QC, etc) … nonlinear!

Inner-loop: minimization

update xg

Investigate impact on observation rejection algorithm due to multiple “outer-loops”.

• Code Development

– Activated analysis “outer-loop”;

– Added observation rejection check to outer-loop;

– Generated statistics for data utilization/rejection for each outer-loop;

– Graphic tools developed to monitor data utilization/rejection.

Minimization

i≥ntmaxor

|Jnew|< eps• |J|

Update first-guess

j ≥ max_ext_its

No

Yes

No

Yes

Outer

Inner

First-guess & Observation

Update first-guess &Observation rejection

- Led by Rizvi Syed

• Rejected Observation locations and number

b) Investigate impact on observation rejection algorithm due to multiple “outer-loops”.

Number of rejected sondes

Outer-loop 1Outer-loop 2

ptop 1000 900 800 600 400 300 250 200 150 100 50 0obs type var pbot 1200 999.9 899.9 799 599.9 399.9 299.9 249.9 199.9 149.9 99.9 2000--------- ------- ------- --------- ------ ------- ------- ------- ------- ------- ------- ------- -------- ------ ---------sound U used 19 50 40 45 79 40 38 37 42 65 96 551

rej 4 12 11 5 0 0 0 0 0 0 0 32sound V used 21 42 39 46 79 40 38 37 42 65 96 545

rej 2 20 12 4 0 0 0 0 0 0 0 38sound T used 35 96 112 380 494 181 78 67 96 164 273 1976

rej 0 0 0 1 0 0 0 0 0 0 6 7

WRF-Var data utilization statistics for outer iteration 1

Outer-loop 1Outer-loop 2

The obs. rejection can be monitored either in tabular form & graphical form.

Analysis difference between the first and second outerloop at the 10th Eta level

Outline

1. WRFDA overview

2. A few new capabilities

3. Incremental formulation and outer-loop

4. Forecast error sensitivity to observations

5. Future plan and summary

< MA,B >=< A,MTB >

F =<δxf ,δxf >=< Mδxa,δxf >=<δxa,M Tδxf >=< Kd,M Tδxf >=< d,K TM Tδxf >

′J = 0δxa = BHT HBHT + R( )-1

d = Kd

∂F

∂y=

∂F

∂d= KTMTδx f

Simply Math:

Linear assumption:

Analysis:

Forecast error:

Forecast error sensitivity to initial state:

Forecast error sensitivity to observations:

∂F

∂xa=

∂F

∂δxa= MTδx f

δx f = Mδxa

Observation(y) WRF-VAR

Data Assimilation

WRF-ARWForecast

Model

Forecast(xf)

DeriveForecastAccuracy

Background(xb)

Analysis(xa)

Adjoint of WRF-ARW

ForecastTL Model

(WRF+)

ObservationSensitivity

(F/ y)

BackgroundSensitivity(F/ xb)

AnalysisSensitivity(F/ xa)

Observation Impact<y-H(xb)> (F/ y)

Adjoint of WRF-VAR

Data Assimilation

Obs Error Sensitivity(F/ ob)

Adjoint sensitivity (Thomas Auligne)

Gradient of F

(F/ xf)

DefineForecastAccuracy

ForecastAccuracy

(F)

Bias CorrectionSensitivity(F/ k)

Adjoint of WRF-VAR DA: Introduction

• Analysis incrementsδx = xa - xb = K.[y-H(xb)] = K.d

• Adjoint of analysisF/y = KT.F/xa

• Various methods– Ensemble Transform Kalman Filter (ETKF, Bishop et al. 2001)

– Dual approach (PSAS, Baker and Daley 2000)

– Exact calculation of adjoint of DA (Zhu and Gelaro, 2007)

– Leading eigenvectors of Hessian (Fisher and Courtier 1995)

– Lanczos algorithm (Fisher 1997, Tremolet 2008)

Adjoint of WRF-VAR DA: Lanczos Algorithm

• New minimisation packageSolve variational problem with Lanczos algorithm:

• Minimize Cost Function

• Estimate Analysis Error

• EXACT adjoint of analysis gain KT

• Link with current Conjugate GradientDue to theoretical similarities b/w the two approaches,

the convergence and solutions are IDENTICAL

• Computer issues – Adjoint of analysis is calculated during minimization with no overhead

– Extra storage is required during minimization

– New orthonormalization of gradients results in faster convergence

• Products– Observation Sensitivity & Impact !

– Estimation of condition number of DA system !!

– Efficient preconditioner for multiple outer loops & ensemble DA !!!

– Estimation of Analysis Error covariance matrix !!!!

200 hPa

500 hPa

Impact (Jb) per observation

Adjoint of WRF-VAR DA: Observation Impact

Impact (Jb) per observation type

SOUND

SYNOPPILOT

SATEM

GEO AMV

AIREP

GPSRF

METARSHIP

PROFILER

BUOY

SONDE`_SFC

N15 AMSUA

N16 AMSUA

N15 AMSUB

N16 AMSUB

N17 AMSUB

METOP AMSUA

SSMIS

Outline

1. WRFDA overview

2. A few new capabilities

3. Incremental formulation and outer-loop

4. Forecast error sensitivity to observations

5. Future plan and summary

Future Plans

General Goals:

• Unified, multi-technique WRF DA system.

• Retain flexibility for research, multi-applications.

• Leverage international WRF community efforts.

WRF-Var Development (MMM Division):

• 4D-Var (additional physics, optimization).

• Sensitivities tools (adjoint, ensemble, etc.).

• EnKF within WRF-Var -> WRFDA.

• Instrument-specific radiance QC, bias correction, etc.

Data Assimilation Testbed Center (DATC):

• Technique intercomparison: 3/4D-Var, EnKF, Hybrid

• Obs. impact: AIRS, TMI, SSMI/S, METOP.

• New Regional testbeds: US, India, Arctic, Tropics.

Applications:

• Hurricanes/Typhoons

• OSEs and OSSEs

• Reanalysis (Arctic System Reanalysis)

Summary1. WRFDA overview

• Unified system: 3D-Var, 4D-Var, ETKF, Hybrid

• Observations: conventional, satellite, radar

• Community support

2. A few new capabilities• AIRS

• Optimization of 4D-Var (Xin’s talk)

• WRF-NMM interface

• global ARW interface

• Ensemble QC

3. Incremental formulation and outer-loop• Background and guess(es)

• Handling of nonlinear aspects: QC, M and H

• Resolution changes between inner- and outer-loops

4. Forecast error sensitivity to observations• Sensitivity to initial state: adjoint of forecast model

• Sensitivity to observations: need adjoint of analysis