Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems
description
Transcript of Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems
![Page 1: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/1.jpg)
Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems
Yanqiu Zhu
Contributions from: John Derber, Andrew Collard, Dick Dee, Russ
Treadon, George Gayno, Jim Jung, Emily Liu, Min-Jeong
![Page 2: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/2.jpg)
Part I: The enhanced radiance bias correction scheme
Part II: New emissivity predictor to handle large land-sea differences with the new CRTM (will be presented at the 6th WMO Symposium on Data Assimilation)
Part III: Radiance bias correction for all-sky radiance bias correction
![Page 3: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/3.jpg)
Part I: Enhanced radiance bias correction scheme
The original radiance bias correction scheme
The motives to enhance the original scheme – the goals for the enhanced scheme
Methodology for the enhanced scheme
Experiment setup and results
Conclusions
Yanqiu Zhu John Derber, Andrew Collard, Dick Dee, Russ Treadon, George Gayno, Jim Jung
![Page 4: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/4.jpg)
Why radiance bias correction – to generate unbiased analysis
• the error associated with the instrument and calibration
• the error of the radiative transfer model
• Inconsistencies among the assumptions of data usage and model as well as algorithms
Derber et al. (1991), Harris and Kelly (2001), Dee (2004), etc
4
![Page 5: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/5.jpg)
The control variables and are estimated by minimizing (Derber et al., 1991, Derber and Wu, 1998)
),(~
),(~
2
1
)()(2
1)()(
2
1),(
1
11
xhyRxhy
BxxBxxxJ
T
bT
bbxT
b
is radiative transfer model )(xh
x
Air-mass dependent in GSI
Separate scan-angle dependent
Original radiance bias correction scheme
A two-step procedure
,),()(~ angleair bxbxhh
where )(),(1
xpxb ii
N
ii
air
is pre-specified parameter
x
5
is predictor term with predictor coeff. ip i
i
![Page 6: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/6.jpg)
What we expect to achieve with the enhanced radiance bias correction scheme
Combine the scan angle and air-mass bias components inside the GSI variational framework. Eliminate the duplications and potential risk of opposite drifting
between the two components
Remove the pre-specified parameter for each predictor, so avoid the trial-and-error procedure for any new bias predictors
Apply a new pre-conditioning to the predictor coefficients taking into account of observation contribution, speeding up the convergence
6
![Page 7: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/7.jpg)
What we expect to achieve with the enhanced radiance bias correction scheme (cont.)
Radiance bias correction is based on the data passing the quality control
Reasonable radiance bias correction estimate need to be manually derived prior to the use of any new radiance data or during the re-analysis process with the original scheme
Quality control is performed on the bias corrected data
7
![Page 8: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/8.jpg)
What we expect to achieve with the enhanced radiance bias correction scheme (cont.)
Automatically detect any new/missing/recovery of radiance data and initialize new radiance data; Quickly capture any changes in the data and the system.
A new approach of specifying the background error variances for the predictor coefficients
An new initialization procedure for any new radiance dataAdd capability of bias correction for passive channels
We can use any new radiance data with initial radiance bias correction set to be zero 8
![Page 9: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/9.jpg)
• is computed outside GSI, a running average of OMF
• The predictors of air-mass are computed inside the GSI
Enhanced scheme: a one-step procedure
Original scheme: a two-step procedureangleb
kK
kkN
angleb
1
• Variational angle bias correction, is computed inside the GSI along with other predictor terms
• satang file no longer needed
• Special attention to the initialization of bias coefficients: using quality-controlled data from previous run; or mode of all data
angleb
9
![Page 10: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/10.jpg)
Change of the Preconditioning
• Hessian w.r.t. predictor coeff. of the cost function
aa ββ
ββ1T
β1
β
ββ2
2
βHHRHB
β
hJ
~ ,
Modified block-diagonal preconditioning is set to be the inverse of (Dee 2004)
Current preconditioner Modified preconditioner
βM
β
x
B
BB
0
0
β
x
M
BB
0
0
2
2
β J
For simplicity, only diagonal elements of are considered at each analysis cycle
βM
10
![Page 11: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/11.jpg)
Currently scheme
Nii
,,1
,0.102
Set to be the estimate of analysis error variance of the coeff. from previous analysis cycle
New data: is initialized to be 10000.0
Missing data: is twice as much as that of the previous analysis cycle
Enhanced scheme
),,( 22
1 NdiagB
2
i
2
i
2
iAutomatically adjust variances of coeff.
More & high quality data
Smaller variance
Background variance for predictor coeff.
11
![Page 12: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/12.jpg)
Bias correction for passive channels (passive_bc=.true.)
• Make preparation for assimilating passive channels data
• Aid quality control of other channels
),(~
),(~
2
1)()(
2
1)( 11 a
T
abT
b xhyRxhyBF
At the end of analysis, minimizing the functional
angleai
N
iiaa bxpxhxh
)()( ),(~
1
where
)()( ),(~
1ai
KN
iiaa xpxhxh
or If adp_anglebc=.t.12
![Page 13: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/13.jpg)
Experiment configuration
• High Resolution: T574L64 3DVAR and T254L64 EnKF with 80 ensemble members
• 0.25 for static B and 0.75 for ensemble• Two-way hybrid coupling• EnKF relies on the GSI to perform the radiance bias
correction
CTL: prda141b, current radiance bias correction
RBC : prda141br, enhanced radiance bias correction
Experiment period: 00Z July 5, 2012 to 00Z Aug. 16, 2012
Spin-up period: 00ZJuly 5, 2012 to 18Z July 13, 2012 13
![Page 14: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/14.jpg)
Convergence comparison
00Z July 15, 2012 14
![Page 15: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/15.jpg)
Variance vs. Observation number
A big drop of obs number prompts a jump of variance
AIRS CH22 15
![Page 16: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/16.jpg)
Global offset and total bias terms: AMSUA-NOAA18
16
![Page 17: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/17.jpg)
OMF with and without bias correction
CTL RBC
Without BC
With BC
AMSUA-N18, CH4 00Z July 10, 2012 17
![Page 18: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/18.jpg)
Mean Temperature Analysis & Difference 700hpa
18
![Page 19: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/19.jpg)
HGT 500hpa NH Anomaly Correlation
19
![Page 20: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/20.jpg)
HGT 500hpa SH Anomaly Correlation
20
![Page 21: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/21.jpg)
RMSE Wind Vector 200hpa Tropics
21
![Page 22: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/22.jpg)
24h & 48h Forecasts Fits to Obs
22
![Page 23: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/23.jpg)
Conclusions
• One-step bias correction procedure
• Improved convergence rate
• Made it easier to add new bias predictors
• Added the capability to automatically detect any new/missing/recovery of radiance data
• New radiance data can be used without providing prior bias correction estimate
• Neutral forecast skills in NH and Tropics, better in SH
• The scheme was also tested on NDAS/NAM
• Currently is used in the wind energy project (Jacob’s talk at 6th WMO Symposium on Data Assimilation)
23
![Page 24: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/24.jpg)
Part III: Radiance bias correction with all-sky radiance assimilation
Yanqiu Zhu, Emily Liu, Min-Jeong Kim, Andrew Collard, John Derber
Two other talks on all-sky radiance assimilation at NCEP: Emily Liu’s and Min-Jeong Kim’s talks
![Page 25: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/25.jpg)
Bias correction for Operational clear radiance
• Angle bias correction
• Air-mass bias correction with five predictor terms: global offset, a zenith angle term, cloud liquid water, lapse rate, and the square of the lapse rate.
All-sky radiance assimilation
• Control variable: total cw/normalized cw/normalized RH
• State variables: Hydrometeors ql, qi, (qr, qs, qg, qh)
• The cloud liquid water predictor is removed
• Observation error based on averaged CLW
• Quality control: cloud filtering is removed
• Linearized moist physics
![Page 26: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/26.jpg)
Issues of radiance bias correction with all-sky radiance assimilation
• Different error characteristics between clear-sky and cloudy radiance
• Different cloud information between radiance observation and first guess
1. O:clear vs. F:clear
2. O:clear vs. F:cloudy eliminate cloud
3. O:cloudy vs. F:clear add cloud
4. O:cloudy vs. F:cloudy
Ideally, Gaussian distributed (O:clr,F:clr) & (O:cld,F:cld), with one hump ((O:clr,F:cld), (O:cld,F:clr)) on each side
![Page 27: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/27.jpg)
Issues of radiance bias correction with all-sky radiance assimilation (cont.)
OmF before BC OmF after BC
Lots of (O:cld, F:cld), too fewer other categories
Over bias corrected (O:cld,F:cld) and (O:clr,F:clr)
![Page 28: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/28.jpg)
Issues of radiance bias correction with all-sky radiance assimilation (cont.)
OmF before BC OmF after BC
METOPA AMSUA Ch. 15
![Page 29: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/29.jpg)
Issues of radiance bias correction with all-sky radiance assimilation (cont.)
METOPA AMSUA CH. 15
OmF before BC OmF after BC
(O:cld, F:cld)
![Page 30: Enhancements of Radiance Bias Correction in NCEP’s Data Assimilation Systems](https://reader038.fdocuments.net/reader038/viewer/2022103007/5681548c550346895dc29a74/html5/thumbnails/30.jpg)
• What may have hurt the radiance bias correction
• Experiments are underway, and results will be presented later at the GSI meeting
1. Bias correction coeff. are retrieved using only (O:clr,F:clr) and (O:cld,F:cld); Fixed previous coeff. are used to calculate the bias corrected OmF for (O:clr,F:cld) and (O:cld,F:clr).
2. Variational quality control
1. Inclusion of the (O:clr,F:cld) and (O:cld,F:clr) in bias correction
2. Radiance data with large OmF but small obs error
Experiment strategies: