Validation of Global Ocean Forecast System (GOFS) 3 · 3 Horizontal grid: 1/12° equatorial...
Transcript of Validation of Global Ocean Forecast System (GOFS) 3 · 3 Horizontal grid: 1/12° equatorial...
Validation of Global Ocean Forecast System (GOFS) 3.1
O.M Smedstad1, E.J. Metzger2, P.J. Hogan, P.G. Posey, A.J. Wallcraft2, D.S. Franklin1 , L. Zamudio3 and M.W. Phelps4
1Vencore, Inc. 2Naval Research Laboratory
3Florida State University 4Jacobs Engineering
Layered Ocean Model Workshop 2-4 June 2015
Copenhagen, Denmark
2
GOFS 3.0: 1/12° 32 layer HYCOM
NCODA-3DVAR
Modular Ocean Data Assimilation System (MODAS) energy-loan ice
Operational system running on Navy DSRC IBM iDataPlex computers
GOFS 3.1: 1/12° 41 layer HYCOM (9 additional layers in the upper ocean)
NCODA-3DVAR
Improved Synthetic Ocean Profiles (ISOP)
Los Alamos Community Ice CodE (CICE)
Currently in operational testing mode (OPTEST)
GOFS 3.5: 1/25° 41 layer HYCOM (Transition scheduled for Fall 2016)
NCODA-3DVAR
ISOP
CICE
tides
Arctic Cap: Sub region of GOFS 3.0 north of 40°N
CICE
GOFS Descriptions and Status
3
Horizontal grid: 1/12° equatorial resolution 4500 x 3298 grid points, ~6.5 km spacing on average, ~3.5 km at pole
Mercator 79°S to 47°N, then Arctic dipole patch
Vertical coordinate surfaces: 41 for σ2*
KPP mixed layer model
Community Ice CodE (CICE v4) sea-ice model Coupling between ocean and ice via the Earth System Modeling Framework (ESMF)
Surface forcing: wind stress, wind speed, thermal forcing, precipitation, relaxation to climatological SSS
Monthly river runoff (986 rivers)
Initialize from January climatology (GDEM 4.2) T and S
No subsurface relaxation to climatology
GOFS 3.1 Configuration
4
Community
Ice
Code
(CICE V4)
HYbrid
Coordinate
Ocean
Model
(HYCOM)
Atmospheric Forcing
0.5° NAVGEM
HYCOM/NCODA/CICE
Model Output
Ice Drift
Ice Thickness
Ice Concentration
Ocean Currents
Ocean Temp
Ocean Salinity
Navy
Coupled
Ocean
Data
Assimilation
(NCODA)*
First Guess
24-hour forecast
Ocean currents,
sss and sst
Ice concentration,
ice temperature,
ice drift,
shortwave
through ice
* ocean observations (sst, profiles, altimeter)
and ice concentration observations
Hourly exchange+
+ hourly fields exchanged via
Earth System Modeling Framework (ESMF)
NAVy Global Atmospheric
Prediction System
5
Sequential Incremental Update Cycle Analysis-Forecast-Analysis
3Dvar - simultaneous analysis ice concentration and 5 ocean
variables: temperature, salinity, geopotential, layer pressure,
velocity (u,v)
HYCOM
CICE
Ocean QC
3D Var
Ocean Obs
SST: GAC/LAC
MCSST, GOES,
Ship, Buoy
Profile: XBT, CTD,
PALACE Float,
Fixed Buoy,
Drifting Buoy
Altimeter SSHA
SSM/I Sea Ice
Innovations
Increments
First Guess Improved Synthetic Ocean Profiles
(ISOP) used as the vertical projection
technique of the surface observations
Navy Coupled Ocean Data Assimilation
6
1) Perform first NCODA analysis centered on tau = -12
2) Run HYCOM using incremental updating ( ) over the first 6 hours
3) Run HYCOM in forecast mode out to tau = 168
FGAT – First Guess at Appropriate Time
0
00Z Nowcast
00Z
NCODA analysis windows centered at this time using receipt time and FGAT
using observations received since the previous analysis and looking back:
-96 hours for profile data
-120 hours for altimeter data
00Z 00Z 00Z
+24 +48 +72 +96
00Z
+120
Hindcast Forecast
-12 tau = +168
00Z
GOFS 3.1 Runstream
1/12 Global HYCOM/CICE
7
Snapshot of Sea Surface Temperature
8
GOFS 3.1 Temperature Observations Pacific Ocean
Downward projection into the interior at all locations with SSH and SST (synthetics)
Ocean Validation Regions Used in the Validation Test Report (VTR)
9
Validation regions are defined by the Naval Oceanographic Office
10
Ocean Validation – Temperature Profiles GOFS 3.0 vs. GOFS 3.1 Nowcast Time
Mea
n
Mea
n
RM
S R
MS
Temperature (°C) vs. depth error analysis in the upper 500 m against unassimilated profile observations at the “nowcast” time for the eight regions defined on the previous slide spanning the hindcast period August 2013 – April 2014. The gray lines in the ME plots are the tolerances set by NAVOCEANO for the temperature bias in the GOFS 3.0 OPTEST
Red curves: GOFS 3.0 Black curves: GOFS 3.1
11
Ocean Validation – Temperature Profiles GOFS 3.1 Forecast Horizons (5,10,14 days)
Mea
n
Mea
n
RM
S R
MS
Temperature (°C) vs. depth error analysis in the upper 500 m against unassimilated profile observations for the eight analysis regions for the 14-day forecasts initialized from the hindcast period August 2013 – April 2014.
Black curves: Nowcast Cyan curves: 5-day forecast Red curves: 10-day forecast Green curves: 14-day forecast
Not a lot of forecast skill degradation Out to 14 day forecast horizon.
Ocean Validation – Mixed Layer Depth GOFS 3.0 vs. GOFS 3.1Nowcast Time
12
GOFS 3.1 GOFS 3.0
Lower RMSE in all regions in GOFS 3.1
August 2013 – April 2014
Mean error (bias)
Model ML too deep
Model ML too shallow
13
Ice Concentration (%) Ice Thickness (m)
Black line is the independent ice edge analysis from the National Ice Center (NIC)
GOFS 3.1 includes 2-way nested CICE 30-day animation starting on 7 April 2015
Polar (Ice) Validation Regions
14
Arctic Antarctic
Compare independent observations against GOFS 3.1 and ACNFS hindcast output (1 June 2012 – 31 May 2013)
15
Ice Edge Error Arctic
Ice edge error (km) at nowcast time vs. time, (1 June 2012 – 31 May 2013)
28.4 km 36.4 km
22%
38.4 km 43.6 km
12%
38.9 km 44.9 km
13%
28.8 km 25.6 km
-13%
Due to an assimilation error that has been corrected
The GOFS 3.1 and ACNFS 5% ice concentration isolines are compared against the independent National Ice Center ice edge analysis
Mean Error
16
Validation period is 1 June 2012 – 31 May 2013
Region GOFS 3.1
Amery Sea 34.2
Shackleton Sea 30.6
Ross Sea 29.2
Amundsen Sea 37.0
Bellinghausen Sea 39.9
Weddell Sea 47.3
Mean Ice Edge Location Error (km)
Antarctic
Take-home message: Ice edge errors in the Southern Hemisphere have similar magnitudes as ice edge errors in the Northern Hemisphere
“IceBridge” Flights (in lieu of satellite obs)
17
• Black arrows indicate flight data comparison shown on the next slide
• GOFS 3.1 has generally lower thickness error north of Alaska (Beaufort Sea) and the Canadian Archipelago
• ACNFS generally has lower thickness error north of Greenland
18
Select 2013 IceBridge Thickness Comparisons
Flight Bias Absolute Bias RMS Difference
GOFS 3.1 ACNFS GOFS 3.1 ACNFS GOFS 3.1 ACNFS
20130321 -0.43 0.60 0.98 0.90 1.22 1.09
20130322 0.39 0.98 0.54 1.08 0.67 1.33
20130323 0.23 1.04 0.55 1.33 0.77 1.59
20130324 0.59 0.82 0.82 1.01 1.05 1.32
20130326 -0.76 0.76 0.96 1.09 1.23 1.32
20130327 -1.89 -1.11 1.91 1.45 2.14 1.93
20130422 -0.57 0.80 0.83 0.85 1.00 0.99
20130424 -1.33 -0.11 1.40 0.62 1.87 0.94
20130425 -0.28 1.46 0.63 1.47 0.79 1.55
IceBridge GOFS 3.1
ACNFS
Ice Thickness vs. IceBridge
19
Ice Drift • Compared 24-hour forecast ice drift against 129
International Arctic Buoy Program drifting buoys • Initial results showed GOFS 3.1 was 35% too fast
and ACNFS was 15% too fast • GOFS 3.1 used ocean currents averaged over 3 m
but ACNFS used currents averaged over 10 m – Options:
• Use consistent depth for averaging ocean currents • Modify the ice-ocean drag coefficient
• Ice-ocean drag coefficient doubled and a new Jan-Aug 2014 hindcast was integrated to compute new ice drift errors
20
Twenty-four hour separation distance (km) between the International Arctic Buoy Program (IABP) ice drifting buoy 169312003533373 and GOFS 3.1 (black) and ACNFS (red) over the period 15 March - 3 September 2014. The mean separation distance for GOFS is 7.0 km and 7.5 km for ACNFS.
Drifting Buoy Comparison
21
Ice Drift
Variable Observed GOFS 3.1 ACNFS
GOFS -
Observed
ACNFS -
Observed
Statistics over the period January-August 2014
Speed 8.78 9.97 9.59 1.19 (14%) 0.81 (9%)
Statistics over the period January-March 2014
Speed 7.90 9.43 9.96 1.53 (19%) 2.06 (26%)
Statistics over the period June-August 2014
Speed 10.41 11.20 9.87 0.79 (8%) -0.54 (-5%)
• ACNFS has lower overall (Jan-Aug) error • GOFS 3.1 has lower error in the winter (Jan-Mar) • ACNFS has lower error in summer (Jun-Aug) • Even though ACNFS slightly outperformed GOFS 3.1 in ice
drift, the NIC agreed that in the net, GOFS 3.1 outperformed ACNFS (edge, concentration, thickness, etc.)
Observed and forecast ice speed (cm/s) against all IABP drifters
22
• SSMIS ≈ 25 km resolution
• AMSR2 ≈ 10 km resolution
• IMS ≈ 4 km resolution
• Implemented 2 Feb 2015 in real-time GOFS 3.1 runstream
• Significant improvement in edge location error
High resolution ice assimilation
Region
GOFS 3.1
SSMIS AMSR2
and IMS
AMSR2 +
SSMIS and
IMS
GIN Sea 72 19 19
Barents/Kara Seas 47 22 22
Laptev Sea 59 24 24
Bering/Chukchi/ Beaufort 57 22 22
Canadian Archipelago
83 31 31
Total Arctic 64 25 25
Percent improvement over
SSMIS --- 62% 62%
GOFS 3.1 ice edge location error (km) using various ice assimilation data sources
Hindcast period: Jun-Aug 2014
23
GOFS 3.5 Demonstration 1/25 HYCOM/CICE/NCODA with tides running in demonstration mode at
Navy DSRC on Cray XC30
Total SSH (including the barotropic tidal signal)
24
GOFS 3.5 Demonstration Steric SSH reveals the generation locations and propagation of internal waves
25
GOFS 3.5 Demonstration
Total SSH Steric SSH
Barotropic tides Internal waves at tidal frequencies
1/25 HYCOM/CICE/NCODA with tides running in demonstration mode at Navy DSRC on Cray XC30
26
Thanks!
Questions?
Observations move the forecast from the background trajectory
(Xb) to the trajectory starting from the new analysis (Xa)
“Observation impact” is the combined
effect of all of the observations on the
difference in forecast error (ef - eg)
Observation Impact: Concept
OBSERVATIONS
ASSIMILATED
Fore
cast
Err
or
VERIFYING
ANALYSIS
Forecast Lead Time
Xg
Xf
Xb
Xa
t=-72 t=-48 t=0
●
●
●
●
●
eg
ef
Observation Impact: Equations
Forecast error cost function
Impact for each
observation
𝜕𝐽
𝜕𝑥𝑓= 𝑒𝑓 − 𝑒𝑔
𝑒𝑔 = (𝑥72 − 𝑥0)(𝑥72 − 𝑥0) 𝑒𝑓 = (𝑥48−𝑥0)(𝑥48 − 𝑥0)
𝜕𝐽
𝜕𝑥𝑎= 𝐿𝑇
𝜕𝐽
𝜕𝑥𝑓 Initial condition sensitivity
LT adjoint forecast model
𝜕𝐽
𝜕𝑦= 𝐾𝑇
𝜕𝐽
𝜕𝑥𝑎 Observation sensitivity
KT adjoint assimilation system
𝛿𝑒𝑓𝑔= 𝑦 − 𝐻𝑥𝑏 ,
𝜕𝐽
𝜕𝑦
Observation innovation
Forecast error sensitivity
in observation space
Langland and Baker (2004) Tellus 56A:189-201 28
29
Observation Impact: Interpretation
< 0.0 the observation is BENEFICIAL -
forecast errors decrease
For any observation assimilated, if ...
g
fe
g
fe > 0.0 the observation is NON-BENEFICIAL -
forecast errors increase
Non-beneficial impacts:
- not expected, all observations should decrease forecast error
- if occurs (and is persistent), look for problems in data QC,
instrument accuracy, model error, specification of assimilation
error statistics (observation error, background error)
30
Observation Data Impact Argo Float Temperature
Blue: Positive impact on the forecast Yellow/Red: Negative impact on the forecast
31
Observation Data Impact Animal Borne Temperature
Blue: Positive impact on the forecast Yellow/Red: Negative impact on the forecast
32
GOFS 3.0: 1/12° 32 layer HYCOM
NCODA-3DVAR
Modular Ocean Data Assimilation System (MODAS) energy-loan ice
Operational system running on Navy DSRC IBM iDataPlex computers
GOFS 3.1: 1/12° 41 layer HYCOM (9 additional layers in the upper ocean)
NCODA-3DVAR
Improved Synthetic Ocean Profiles (ISOP)
Los Alamos Community Ice CodE (CICE)
Currently in operational testing mode (OPTEST)
GOFS 3.5: 1/25° 41 layer HYCOM (Transition scheduled for Fall 2016)
NCODA-3DVAR
ISOP
CICE
tides
Arctic Cap: Sub region of GOFS 3.0 north of 40N
CICE
GOFS Descriptions and Status
Ocean Validation – Mixed Layer Depth Forecast
33
Nowcast time 5-day forecast 10-day forecast 14-day forecast
August 2013 – April 2014