Six years class4 intercomparison with IV-TT...
Transcript of Six years class4 intercomparison with IV-TT...
![Page 1: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/1.jpg)
mercator-ocean.eu/marine.copernicus.eu
Six years class4 intercomparison with
IV-TT dataset
OceanPredict’19 meeting - May 8, 2019
REGNIER Charly
DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and IV-TT members
![Page 2: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/2.jpg)
Introduction
This Intercomparison is based on class4 files « model equivalents of the observations in
time and space (best estimate, forecast and persistence at different lead times and clim) »
Used to:
❖ Control and Monitor the quality of the systems
❖ Described the main differences in 2D and 3D space
❖ Measure the forecast accuracy
Aim of this presentation is to show you this three aspects but also new metrics like
contengency table approach and perspectives with a new CLASS4 UV dataset
![Page 3: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/3.jpg)
Materials
• Since 2013 daily production of Class4
• Datasets used (available on crypted NOAA ftp server)
• SST dritings Buoys since 2013
• Temperature and salinity from Argo profiles since 2013
• Sea Ice concentration from AMS2 satellite since 2015
• L3 along track sla dataset from CMEMS since 2013
• UV drifters from CMEMS since April 2019
• ParticipantsGlobal systems• UK with Met Office (FOAM system ¼°)
• Canada with Environnement Canada (GIOPS system ¼°)
• US with NOAA (RTOFS system 1/12°)
• Australia with Bluelink (OMAPS system 1/10°)
• France with Mercator Ocean International (PSY3 and PSY4 systems)
Regional systems
• Arctic with NERSC system ¼° (2014 – 2018)
• Italia with INGV (MFS system)
• Brasil
• China with NMEFC
![Page 4: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/4.jpg)
Monitoring the quality
![Page 5: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/5.jpg)
Monitoring the performance of the systems
• Scores are available for all the GODAE regions with other scores (anomaly correlation, skill scores)• Need to go a step further to have a real time access to these scores
Oversampling in the ArcticBad representationof Pacific SummerWaters in november2018
![Page 6: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/6.jpg)
Future evolutions
https://moniqua.mercator-ocean.fr
Number of observations
Misfit average for Temp (obs-mod)Oper 1/12°
Ran 1/12°
Frontend is ok, just need a short time to adapt the database to the statistics
![Page 7: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/7.jpg)
Describing the quality in 2D and 3D
![Page 8: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/8.jpg)
RMSD Salinity 0-500m in 2018
Best Analysis
BLK
GIOPS
FOAMPSY4
ENSEMBLE MEAN
HYCOM-RTOFS
![Page 9: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/9.jpg)
GIOPS
RMSD Salinity 0-500m in 2018
Forecast 3D
BLK
FOAMPSY4
ENSEMBLE MEAN
HYCOM-RTOFS
![Page 10: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/10.jpg)
Temperature error spread (Forecast 3D)
Period 2017-2018Thick blue line : mean diffDark blue shading : 75 % of distributionLight blue shading : 95% of distribution
![Page 11: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/11.jpg)
Salinity error spread (Forecast 3 days)
![Page 12: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/12.jpg)
Forecast Accuracy
![Page 13: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/13.jpg)
Forecast accuracy in 2017
Evolution of the RMSD as a fonction of forecast lead timeThick line : percentile 75% distrThin Line : Percentile 95% distr
![Page 14: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/14.jpg)
Forecast accuracy using skill scores
• Skill = 1 – (RMSDforecast /RMSDreference)
• 2 types of skill scores :
• Persistence Skill Score with persistence as a reference
• Climatology skill scores with Climatology WOA as a reference
![Page 15: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/15.jpg)
PSS skill score 3D in 2018 (Ensemble mean)
Temperature (0-500m) Salinity (0-500m)
SSTSLA
![Page 16: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/16.jpg)
New metrics and dataset
![Page 17: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/17.jpg)
Derived quantities with Ice concentration
Contingency table approach
• Following the contingency table method described in Smith et al (2015)
• Theses statistics are computed on a stereopolar grid (with 0.2 threshold for ice/no ice)
• Possible to compute the following derived quantities▪ Proportion Correct Ice, PCI=a/(a+c)
▪ Proportion Correct Water, PCW=d/(b+d)
▪ Proportion Correct Total, PCT=(a+d)/n
▪ Frequency Bias, BIAS=(a+b)/(a+c)
n : number of points
![Page 18: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/18.jpg)
PCT scores Forecast 3D
ARC
ANT
Update: add sea iceconcentration in DA
![Page 19: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/19.jpg)
New dataset: Class4 UV drifters
OBS MODEL
• First set of 1 year already produced• Zonal currents are underestimated in the model• Need to work on filtered dataset without inertial oscillations to compare to daily
model outputs• Correct the data from errors (slippage and windage corrections)
![Page 20: Six years class4 intercomparison with IV-TT datasetgodae-data/OP19/5.6.2-presentation_cregnier_16... · REGNIER Charly DREVILLON, Marie, SZCZYPTA, Camille, HERNANDEZ, Fabrice and](https://reader033.fdocuments.net/reader033/viewer/2022043013/5f84b34a710cd8225e455a5d/html5/thumbnails/20.jpg)
Conclusion
▪ Class4 statistics are fundamental for operationnal verification
▪ Convergence in the scores for all the global systems
▪ Each system as not the same strenghts and weaknesses but are complementary
▪ The ensemble mean of all the global systems exhibits very good performances
▪ Forecast accuracy is improved over time, except for SST all the systems are close
▪ Still work to do on new metrics (threshold metrics, integrated values…)
▪ Work to do to put UV class4 in operationnal and to produce new statistics