HFIP Regional Ensemble Call Audio = 800-857-6535 Passcode = 71511 Meeting ID: 492-533-786
Objective Evaluation of 2010 HFIP Stream 1.5 Candidates
description
Transcript of Objective Evaluation of 2010 HFIP Stream 1.5 Candidates
![Page 1: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/1.jpg)
Objective Evaluation of 2010 HFIP Stream 1.5 Candidates
Louisa Nance, Christopher Williams, Michelle Harrold, Kathryn Newman, Paul Kucera, and Barb Brown
National Center for Atmospheric Research (NCAR)
Research Applications Laboratory (RAL)Joint Numerical Testbed (JNT)
Tropical Cyclone Modeling Team (TCMT)
Acknowledgements:National Hurricane Center – case selection and verification metricsHFIP Verification Team – verification metricsParticipating Modeling Groups – retrospective forecasts
![Page 2: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/2.jpg)
HFIP Stream 1.5 Concept
• Stream 1: Yearly upgrades to operational numerical weather prediction capabilities
• Stream 2: Enhancements to operations that require multiple year applied research, development and transition-to-operations work
• Stream 1.5: Improved models (mainly) that the NHC, based on prior assessments, wants to access in real-time during a particular hurricane season, but which can’t be made available to the NHC by the operational modeling center in conventional “production” mode (typically due to limits in computing capability and/or programmer time)
![Page 3: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/3.jpg)
2010 Stream 1.5Retrospective Cases
Eastern Pacific:2008 – 5 storms 2009 – 6 storms
Atlantic:2008 – 8 storms2009 – 9 storms
![Page 4: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/4.jpg)
2010 Stream 1.5 Participants
Participants
Organization GFDL NCAR/MMM NRL FSU
Model GFDL* AHW COAMPS-TC ARW
ATCF ID GFD5 AHW1 COTC ARFS
Resolution 1/12 deg 1.3 km 5 km 4 km
Basins Atlantic & Eastern Pacific Atlantic Atlantic &
Eastern Pacific Atlantic
Initialization times 00, 06, 12, 18 00, 12 00, 06, 12, 18 00, 12
![Page 5: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/5.jpg)
Data Inventory
% Expected 1)Storms & time
periods2)Planned basins
& # of runs/day
![Page 6: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/6.jpg)
Methodology
Graphics SS tables
forecast
errors
NHC Vx
error distribution properties
forecast
errors
NHC Vx
forecast
errors
NHC Vx
forecast
errors
NHC Vx
…….
…….
…….
…….
…….
…….
Stream 1.5 Candidate Operational Baseline
median, mean, interquartile range, 95% CI, outliers
Track & Intensity (along- & cross-
track too)
Median ± 95%CI does not include zero
pairwise differences
matching – homogeneous sample
![Page 7: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/7.jpg)
Baseline ComparisonsOperational Baseline Stream 1.5 configuration Lead times evaluated
GFDL Stream 1.5 Every 6 h out to 120 h
Consensus (at least 2 available) - Track: GFS, UKMET, NOGAPS, GFDL, HWRFIntensity: DSHP, LGEM, GFDL, HWRF
Consensus + Stream 1.5 Every 6 h out to 120 h
GFS (track)DSHP (intensity) Stream 1.5
Official forecast times:00, 12, 24, 36, 48, 72, 96, 120 h
Homogeneous average of previous year’s top flight models – Track: GFS, UKMET, GFDLIntensity: GFDL, HWRF, DSHP, LGEM
Stream 1.5Official forecast times:00, 12, 24, 36, 48, 72, 96, 120 h
![Page 8: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/8.jpg)
Error DistributionsAbsolute Intensity ErrorGFDL vs GFD5Atlantic Basin
Difference DistributionsGFDL-GFD5Atlantic Basin
![Page 9: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/9.jpg)
Statistically Significant Differences
AHW1
ARFS COTC GFD50
4
8
12
16
20
ATL-OPATL-1.5EP-OPEP-1.5
# of
lead
tim
es
AHW1
ARFS COTC GFD50
4
8
12
16
20
ATL-OPATL-1.5EP-OPEP-1.5
# of
lead
tim
es
AHW1
ARFS COTC GFD50
4
8
12
16
20
ATL-OPATL-1.5EP-OPEP-1.5
# of
lead
tim
es
AHW1
ARFS COTC GFD50
4
8
12
16
20
ATL-OPATL-1.5EP-OPEP-1.5
# of
lead
tim
es
GFDL Baseline Consensus Baseline
Trac
kIn
tens
ity
![Page 10: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/10.jpg)
NHC’s 2010 Stream 1.5 DecisionAccepted• GFD5
– Accepted prior to TCMT evaluation• AHW1
– Statistically significant improvements at numerous time periods when added to the operational consensus with no statistically significant degradations
– Substantial improvements over the consensus at 96 and 120 h (not statistically significant)
Note: Sample provided smaller than desiredNot accepted• ARFS
– Largely neutral impact on the consensus and limited sample size. • COTC
– Not sufficiently strong or consistent enough to warrant inclusion Note: Provided a significant sample size
![Page 11: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/11.jpg)
Sample Size Impact Full COTC sample vs sample consistent w/AHW1
full reduced02468
101214161820
ATL-OPATL-1.5EP-OPEP-1.5
# of
lead
tim
es
full reduced02468
101214161820
ATL-OPATL-1.5EP-OPEP-1.5
# of
lead
tim
es
full reduced02468
101214161820
ATL-OPATL-1.5EP-OPEP-1.5
# of
lead
tim
es
full reduced02468
101214161820
ATL-OPATL-1.5EP-OPEP-1.5
# of
lead
tim
es
GFDL Baseline Consensus Baseline
Trac
kIn
tens
ity
![Page 12: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/12.jpg)
Basin SensitivityAtlantic Eastern Pacific
![Page 13: Objective Evaluation of 2010 HFIP Stream 1.5 Candidates](https://reader035.fdocuments.net/reader035/viewer/2022070421/568161c9550346895dd1ae8e/html5/thumbnails/13.jpg)
2011 Stream 1.5Retrospective Cases
Eastern Pacific:2009 – 6 storms 2010 – 7 storms
Atlantic:2008 – 4 storms2009 – 7 storms2010 – 16 storms