1 Climate Test Bed Seminar Series 24 June 2009 Bias Correction & Forecast Skill of NCEP GFS Ensemble...
-
Upload
corey-bridges -
Category
Documents
-
view
213 -
download
0
Transcript of 1 Climate Test Bed Seminar Series 24 June 2009 Bias Correction & Forecast Skill of NCEP GFS Ensemble...
1
Climate Test Bed Seminar Series24 June 2009
Bias Correction & Forecast Skill of NCEP GFS Ensemble Week 1 & Week 2
Precipitation & Soil Moisture Forecasts
Yun Fan & Huug van den Dool
Acknowledgement: Jae Schemm, John Janowiak, Doug Lecomte, Jin Huang, Pingping Xie,
Viviane Silva, Peitao Peng, Vern Kousky, Wayne Higgins
2
Outline
• Motivation• Methodology • Performance of NCEP GFS Week1 & Week2
Ensemble Precipitation Forecasts• Analysis of Week1 & Week2 Biases & Errors• Application: land model forced with bias
corrected week1 week2 P & T2m forecast• Future Work
3
History of Soil Moisture “Dynamical” Outlook CPC Leaky Bucket Hydrological Model
Forced With Week 1 & Week 2 GFS Forecasts
Single member HR MRF (started around 1997 & CONUS)
Ensemble GFS (started late 2001 & CONUS)
Bias corrected Ensemble GFS (started late 2003 & CONUS)
Bias corrected Ensemble GFS (started late 2007 & global land)
:
The prediction skill of soil moisture crucially depends on our ability to predict precipitation Early stage (both good and bad comments)
Recent years (more & more good comments)
So its time to verify & quantify:daily GFS ensemble week 1 & week 2 precip forecast skills & statistics
4
The quality of soil moisture prediction largely or almost entirely depends on the quality of precipitation prediction
5
Daily bias correction based on
last 30 (or 7) day forecast errors
Week1
Future
Week2
Today
Past
Last 30 day
1/N Σ [ Pf (week1) – Po (week1) ] = Bias1
1/N Σ [ Pf (week2) – Po (week2) ] = Bias2
Pf : GFS ensemble week1 & week2 precip forecast
Po: Observed week1 & week2 precip from CPC daily global Unified Precip
N = ( 30, 7..….)
6
North America
Seasonal cycle with
Large day to day fluctuation
On 0.5x0.5 obs grid
7
South America
On 0.5x0.5 obs grid
8
Asia-Australia
On 0.5x0.5 obs grid
9
Africa
On 0.5x0.5 obs grid
10
How good is GFS?
On 0.5x0.5 obs grid
Seasonal cycle with
Large day to day fluctuation
11
On 0.5x0.5 obs grid
How good is GFS?
Seasonal cycle with
Large day to day fluctuation
12
Comparison (based on last 30-day forecast errors) Obs grids (regrid model grids to 0.5x0.5 obs grids) Model grids (regrid obs grids to 2.5x2.5 model grids)
Question: Does grid matter for skills assessment?
13Skill does not depend much on the grid
14
Comparison
bias corrected skills (based on last 30-day forecast errors) bias corrected skills (based on last 7-day forecast errors)
Question: Does the bias estimate influence skill?
Week1
Future
Week2
Today
Past
Last 30 day
1/N Σ [ Pf (week1) – Po (week1) ] = Bias1
1/N Σ [ Pf (week2) – Po (week2) ] = Bias2
Pf : GFS ensemble week1 & week2 precip forecast
Po: Observed week1 & week2 precip from CPC daily global Unified Precip
N = ( 30, 7..….)
151) Skill depends on the definition of bias
2) 30-day bias correction better than 7-day bias correction
16
Comparison
bias corrected skills (based on last 30-day forecast errors) raw forecast skills (no bias correction applied)
Question: Does bias correction improve skill
in terms of Spatial Correlation and RMSE?
17
Bias correction is time &
location dependent
18
Bias correction helps everywhere
19
Table 1. Averaged (May 1, 2008 – June 7, 2009) spatial correlations over different monsoon regions
Week 1 Week 2
Bias Correction
No Bias Correction
Bias Correction
No Bias Correction
North America 0.49 0.48 0.24 0.26
South America 0.45 0.25 0.31 0.18
Asia Australia 0.47 0.40 0.29 0.26
Africa 0.40 0.24 0.25 0.13
Table 2. Averaged (May 1, 2008 – June 7, 2009) RMSE over different monsoon regions (unit: mm/week)
Week 1 Week 2
Bias Correction
No Bias Correction
Bias Correction
No Bias Correction
North America 19.18 22.82 21.61 23.58South
America 29.55 41.06 32.27 41.72Asia
Australia 22.65 27.62 25.24 29.15
Africa 17.06 19.47 17.66 19.33
The effectiveness of bias correction is mainly space dependent.
Bias correction can correct spatial
distribution of Pf & reduce its error.
Similarity of Pf & Po
Distance of Pf & Po
Reduced by 28%
Reduced by 23%
Increased by 80%
Increased by 67%
20
CONUS
30-day running mean
21
In terms of Spatial Anomaly Correlation, bias correction helps:1) very little over North America
2) considerably over South America & Africa
3) a little over Asia-Australia
In terms of RMSE:Bias correction helps everywhere
Questions: Why bias correction works but varies in space and time?
What biases look like?
Are biases removable & to what extent are they removable?
22
Temporal-spatial structures of last 30-day biases:
Daily Bias1 & Bias2 used to correct GFS ensemble week1 & week2 forecasts
Week1
Future
Week2
Today
Past
Last 30 day
1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1
1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2
M
mmm sEOFtPCMeantsBias
1
)()(),(2,1
23
Annual Mean Bias or Raw Forecast Error
Week-1 mean Bias
Week-2 mean Bias
24
Mean Bias of Daily R2 & Observed Precip (1979-2006)
25
summer
winter
26
winter
summer
27
Temporal-spatial structures of last 30-day biases:
Daily Bias1 & Bias2 used to correct GFS ensemble week1 & week2 forecasts
Week1
Future
Week2
Today
Past
Last 30 day
1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1
1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2
Large-scale & low-frequency (annual or semi-annual cycles) are prominent
First two EOF modes of Bias1 & Bias2 explain about 60% total variances
GFS has prominent annual cycle errors (lesson for model development?)
28
Temporal-spatial structures of real time raw forecast errors:
Daily GFS week1 & week2 forecast errors without bias correction
Week1
Future
Week2
Today
Past
Last 30 day
1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1
1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2Pf (week1) – Po (week1) = Error1
Pf (week2) – Po (week2) = Error2
No bias correction applied
M
mmm sEOFtPCMeantsError
1
)()(),(2,1
29
30
31
Temporal-spatial structures of real time raw forecast errors:
Daily GFS week1 & week2 forecast errors without bias correction
Week1
Future
Week2
Today
Past
Last 30 day
1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1
1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2
Pf (week1) – Po (week1) = Error1
Pf (week2) – Po (week2) = Error2
No bias correction applied
Raw forecast errors are dominated by the 1st, 2nd or 3rd EOFs in Bias1 & Bias2
First two EOF modes of Error1 & Error2 explain about 23~35% total variances
At least this amount of error is removable. But so far bias correction was not done by EOF analysis
32
Temporal-spatial structures of real time forecast errors:
GFS week1 & week2 forecast errors with last 30-day bias correction
Week1
Future
Week2
Today
Past
Last 30 day
1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1
1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2
Pf (week1) – Po (week1) = Error1
Pf (week2) – Po (week2) = Error2
Bias correction:
Error1 = Error1 – Bias1Error2 = Error2 – Bias2
M
mmm sEOFtPCMeantsError
1
)()(),(2,1
33
Annual Mean Forecast Error after bias correction
5 times smaller than mean bias or raw forecast error
Week-1 mean forecast error
Week-2 mean forecast error
34
35
36
Temporal-spatial structures of real time forecast errors:
GFS week1 & week2 forecast errors with last 30-day bias correction
Week1
Future
Week2
Today
Past
Last 30 day
1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1
1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2
Pf (week1) – Po (week1) = Error1
Pf (week2) – Po (week2) = Error2
Bias correction:
Error1 = Error1 – Bias1Error2 = Error2 – Bias2
Bias Corrected Forecast Errors are much more random (in time mainly, EOFs more “white”).
Leading EOF modes of Bias1, Bias2, & Error1, Error2 Show that GFS has prominent large-scale & low-frequency errors or GFS has difficulty to reproduce those observed Precip patterns & their evolution. However, to some extent they can be corrected through bias correction, especially in winter season.
37
Application
Soil Moisture “Dynamical” Outlook CPC Leaky Bucket Hydrological Model
Forced With Week-1 & Week-2 GFS Ensemble Forecasts
(Daily data from 01Nov2003 to present)
All initial conditions & verification datasets are from leaky bucket model forced with daily observed P & T2m
38
Some Thoughts:
• Once this (SST, w) was the lower boundary….• Both SST and w have (high) persistence• Old ‘standard’ in meteorology: If you cannot beat
persistence …..• For instance: dw/dt = P – E - R = F
or w(t+1)=w(t) + F
• Clearly if we do not know F with sufficient skill, the forecast loses against persistence (F=0).
39
40
41
42
30-day running mean
P1=0.9511, C1=0.9512PR1=16.27, FR1=18.02
P2=0.9015, C2=0.8957PR2=23.67, FR2=26.56
4330-day running mean
Precip
44
Even moderate forecast skill at right time still help a lot
45
30-day running mean for week-2
Hybrid persistence = week-1 forecast persists to week-2
46
• Moderate week-1 & week-2 GFS P forecast skills• Last 30-d biases dominated by low-frequency &
large-scale errors• Bias corrections are time & location dependent• Soil moisture forecast skill hardly beats its
persistence over CONUS• The inability to outperform persistence relates
to the skill of precipitation not being above a threshold (AC>0.5 is required)
Summary
47
• Is PDF bias correction better?
• GFS Week3 & Week4 Precip Assessment
• GFS hindcasts?
• How about New CFSRR?
Future Work