1 NUOPC National Unified Operational Prediction Capability Update to COPC 27 – 28 May 2015 Dave...

1 NUOPCNational Unified Operational Prediction Capability

Update to COPC27 – 28 May 2015

Dave McCarren, NUOPC DPM


2

Agenda

National ESPC Update

NUOPC UEO Committee Update

NUOPC CMA Committee Update

Questions and Discussion


.

The National Earth System Prediction Capability (National ESPC)

3

The National Earth System Prediction Capability ESPC 4

National ESPC Update • National ESPC Strategy 2-pager created for appropriate distribution to

get support for the project; sent to Liaisons for review and taken to AMS meeting in Phoenix

• National ESPC staff drafting Strategy paper for submission to BAMS

• ESG meeting 26 Jan 2015

• ESG Principals +1 meeting hosted by Navy 5 May 2015



• ESG meeting 26 Jan 2015 Action Items:– Modify the proposed National ESPC Management Structure to show

reporting relationships and communications. Project office will provide proposal with fully coordinated recommendation, options, with pros and cons.

– Coordinate a topical ESG meeting within 90 days for various topics, N2/6E to host. Semi-closed session: just staff and Principals + 1.

– Verify that the ESMF paper contains proper mention of National ESPC. Provide the ESG principals background on ESMF management, funding, and requirements process.

– Coordinate specific sessions for National ESPC at upcoming conferences.

– Coordinate a response to the Presidential Executive Order on Coordination of National Efforts in the Arctic on the role of ESPC in Arctic prediction, including a potential National ESPC brief by the principals to OSTP. This response may include the possible development of an integrated NAVY/NOAA operational Arctic Modeling Strategy.



• ESG meeting 5 May 2015 draft Action Items:– Revise the ESPC and NUOPC charters and create a new

National ESPC Charter. Include recommendations from Decision Brief on Management Structure/Plan, and Personnel Issues. Work with climate community to help define what is operational for products to support strategic decision making, Define components that will be part of National ESPC - spiral 1

– Coordinate AF and Navy collaboration to determine the DOD way ahead and converge on a common core

National Unified Operational Prediction Capability NUOPC 7

UEO Committee Update


Unified Ensemble Operations

• NUOPC metrics from NCEP, FNMOC, and AFWA briefed at 26 Jan ESG meeting

• ½ Degree Data Exchange Upgrade on schedule for Summer 2015 implementation at NCEP, CMC, and FNMOC

• NUOPC/NAEFS mini workshop on 24-25 Feb– Face to face meeting of the Co-Chairs of the UEO Committee– Agenda focused on the development of NUOPC that includes data

exchange (of global and wave ensemble data, post process, ensemble week 3&4 forecast and future plans.

– Research following a concern by FNMOC over the timing of the CMC raw ensemble data resulted in NCEP starting raw data processing 90 minutes earlier, much closer to real time and bias corrected data.

• AF moved up operational GEPS processing to take advantage of this; GEPS suite had 5,000 unique users and about 4M product hits in the last 12 months

½ Degree Data Exchange Implementation Schedule

NCEP CMC FNMOC

Test hi-res data Available

Pre-production hi-res data Jan 2015 Sep-Oct 2014

Production hi-res data Apr 2015 Nov-Dec 2014 Oct 2014

Ready to receive hi-res production data Jan 2015

Test hi-res NAEFS products Mar 2015 Apr 2015 (SFC-3HR)

Pre-production hi-res NAEFS products Apr 2015

Production hi-res NAEFS products May 2015 Jul 2015(SFC-3HR)

Earliest practical date for each stage at each center

5-day forecast for surface wind (U)


NH T2m

NA T2m

13

Northern HemisphereWind Speed

The spread ratio for wind speed was greater than one for the entire period (under-dispersive) with NAVGEM and GFS members both being over-dispersive and GEM members also under-dispersive.

There was a spike in May when NOGAPS was replaced by NAVGEM.

GEM members are under-dispersive while NAVGEM and GFS members are over-dispersive

14

Ensemble MeanNorthern Hemi -2 Meter Temps

Ensemble mean in solid red. GFS members in dotted green, NAVGEM members in dotted blue, and GEM members in dotted purple. The solid green is the GFS control member and the solid purple is the CMC control member. We do not currently receive a NAVGEM control member.

Ensemble mean has nearly no bias. GFS members have a cold bias, and NAVGEM members have a warm bias


CMA Committee Update


NUOPC

Common Model Architecture

• Draft manuscript for ESPS paper submitted to BAMS; CMA Committee revising first draft for 1 June deadline

• CMA Committee drafting new whitepaper on Using ESPS

• PI Group building a working Physics Driver prototype with a common physics interface to GFS physics, by June 2015, to address NGGPS requirement to begin testing how it works and use in the selection of a new Dynamc Core, using a common set of physics.

• PI Group presented modernized Kalnay Rules paper at AMS


Questions & Discussion

NUOPC Verification Metrics

EMC/NCEP

May 18 2015

Acknowledgement: Dr. Yan Luo

For all three individual bias corrected ensemble forecast (NCEP/GEFS, CMC/GEFS and FNMOC/GEFS) and combined (NUOPC) ensemble (equal weights) against UKMet analysis

Period: April 1st 2011 – May 15 2015

Ratio of RMS error over spread

Northern Hemisphere 500hPa height:

30-day running mean scores of day-5 CRP scoreRMS error and ratio of RMS error / spreadAnomaly correlation

All other regions could be seen from: http://www.emc.ncep.noaa.gov/gmb/yluo/naefs/VRFY_STATS/NUOPC_bc_COMB_spr2015_ts.html

NH 500hPa anomaly correlation

5-day forecast

NH 500hPa CRP scores

Under-dispersion

Over-dispersion

NH 500hPa RMS errors

http://www.emc.ncep.noaa.gov/gmb/yluo/naefs/VRFY_STATS/NUOPC_bc_COMB_fal2014_ts.html





30-day running mean scores of day-10 CRP scoreRMS error and ratio of RMS error / spreadAnomaly correlation

All other regions could be seen from: http://www.emc.ncep.noaa.gov/gmb/yluo/naefs/VRFY_STATS/NUOPC_bc_COMB_spr2015_ts.html

10-day forecast

NH 500hPa CRP scores

NH 500hPa anomaly correlation

NH 500hPa RMS errors




NH CRP scoresNH RMS errors

NA CRP scoresNA RMS errors

5-day forecast for surface temperature

10-day forecast for surface temperature

5-day forecast for surface wind (V)

10-day forecast for surface wind (V)

Last winter statistic scores

Dec. 1st 2014 – Feb. 28th 2015


Latest 3-month winter scores: CRPS skill scoreRMS error and ratio of RMS error / spreadAnomaly correlation

All other regions/scores could be seen from: http://www.emc.ncep.noaa.gov/gmb/yluo/NUOPC/NUOPC_bc_win1415.html

NH CRPS skill scores

NH anomaly correlation

500hPa Height

NH RMS errors

http://www.emc.ncep.noaa.gov/gmb/yluo/NUOPC/NUOPC_bc_win1415.html



NH CRPS skill scores NH RMS errors

NA CRPS skill scores NA RMS errors

Surface temperature

NH CRPS skill scores NH RMS/Spread scores

For surface wind (U)

NH CRPS skill scoresNH RMS/Spread scores

For surface wind (V)

Comparing of NCEP/GEFS, NAEFS and NUOPC for all bias corrected ensemble forecasts, against UKMet analysis

Period: April 1st 2011 – May 15th 015

Day-5 NH 500hPa height

Day-10 NH 500hPa height

NH T2m

NA T2m

Aim High…Fly, Fight, Win

NUOPC Ensemble Performance

1 Jan 2015 – 1 Apr 2015

Mr. Bob Craig16 WS/WXN

Aim High…Fly, Fight, Win 34

Overview

Method

Brier Skill Scores

Spread

Ensemble Mean


Method

All models verified against 25km UKMO analysis except for total cloud cover that uses WWMCA

Data shown is mainly for the northern hemisphere domain containing Navy - NAVGEM CMC - GEM NCEP - GFS

On ensemble mean slides, ALL refers to the GEPS ensemble

For skill scores, climatology was used as reference forecast


Brier Skill Score

Brier Score (BS) for ensembles is equivalent to Root Mean Square error for deterministic models. Model probabilities for an event are compared to the

relative frequency of the event To get the skill score, the BS is compared to climatology

Brier skill scores greater than 0 indicate skill


Brier Skill ScoreTotal Cloud > 80%

Total Cloud > 80% - looking at the CONUS domain, significant skill is indicated out to about 108hrs. For global and hemisphere domains, little significant skill is indicated. Some reasons for this can be seen on the next slide.


Typical Total Cloud ForecastTotal Cloud > 80%

Black squares are non-zero probabilities and green “1” are observation hits. Over high latitudes there tends to be less hits compared to lower latitudes. Cloud coverage tends to be over- forecasted


Brier Skill ScoreWind Speed > 25kts

For wind speeds >= 25kts, global coverage and northern hemisphere have significant skill out to 240 hours. Higher winds speeds (>35, and >50) also have skill, but an insignificant number of events resulted in much larger error bars.

25Kts

25Kts


Ensemble Spread

Spread measures the difference between the ensemble mean and ensemble forecasts Typically the RMSE of the ensemble mean is compared

to the standard deviation of the members from the mean (RMSE/Stdev)

An ideal ensemble will have the same size of ensemble member deviation from the mean as Root Mean Square error of the ensemble mean

An ensemble spread greater than one means the ensemble does not contain enough spread in the members to account for the errors in the ensemble system


Northern Hemisphere500mb Heights

This chart depicts the ensemble spread ratio of the ensemble members compared to the ensemble mean over the last year for the 120 hour forecast. Green represents the spread ratio of the combined ensemble (ALL) compared to the spread for GFS (purple), Navy (red), and CMC (blue). Ideally, the spread ratio should be around one. A spread ratio greater than one indicates the ensemble is under-dispersive and the opposite is true for spread ratio less than one.

This chart depicts the ensemble spread by forecast hour. The ensemble starts off under-dispersive in early forecast hour getting better in later forecasts, green line approaching one


Northern HemisphereWind Speed

The spread ratio for wind speed was greater than one for the entire period (under-dispersive) with NAVGEM and GFS members both being over-dispersive and GEM members also under-dispersive.

There was a spike in May when NOGAPS was replaced by NAVGEM.

GEM members are under-dispersive while NAVGEM and GFS members are over-dispersive


Northern HemisphereTemperature

The spread ratio was above 1 (under_dispersive) for most of the winter but approaching 1 as Spring arrives

Over the last several months, the spread starts under-dispersive with early forecast hours but becomes neutral by the end of the period


Ensemble Mean

Ensemble mean is the average value of the ensemble members

The ensemble mean error should be less than the error of all the ensemble members for a good ensemble

One of the ensemble members from each center is the control and is configured like the deterministic model from the center though its resolution is degraded We do not currently get a control member from

NAVGEM, but that will be added when available


Ensemble MeanNorthern Hemi -2 Meter Temps

Ensemble mean in solid red. GFS members in dotted green, NAVGEM members in dotted blue, and GEM members in dotted purple. The solid green is the GFS control member and the solid purple is the CMC control member. We do not currently receive a NAVGEM control member.

Ensemble mean has nearly no bias. GFS members have a cold bias, and NAVGEM members have a warm bias


Ensemble MeanNorthern Hemi -10 Meter Winds

GFS members have the lowest error and NAVGEM members have the highest

The ensemble mean has a positive bias and the NAVGEM members seem to be pulling it that way


Ensemble MeanNorthern Hemi – 500mb Hgts

NAVGEM members have the highest error out to 168 hrs, then all members are similar

Ensemble mean bias is near 0 out to about 168 hours then starts to rise as GEM and NVGEM members pull it positive


Ensemble MeanNorthern Hemi – 250mb Winds

A three model members are close for this field

Ensemble mean has a negative bias with all model members contributing. GFS members show a discontinuity after 192 hrs in their bias


Ensemble MeanNorthern Hemi – 850mb Temps

GFS members slightly lower error in early forecast hours

Ensemble mean has a cold bias with GEM and GFS members pulling it into down


Summary

The GEPS ensemble system is performing as expected Brier skill scores for wind speed indicated the GEPS

ensemble has skill out to 240hrs Precipitation forecasts weren’t included here due to recent

problem with UM GRIB files The GEPS ensemble spread score tends to be greater

than 1 with the mean RMSE > member standard deviation, indicating the ensemble members are under-dispersed which is a common ensemble issue


Summary

Ensemble mean forecasts have lower errors than the members which is expected for correctly designed ensembles.

1 NUOPC National Unified Operational Prediction Capability Update to COPC 27 – 28 May 2015 Dave...

Documents

Transcript of 1 NUOPC National Unified Operational Prediction Capability Update to COPC 27 – 28 May 2015 Dave...