Assessing Forecast Uncertainty in the NDFD
description
Transcript of Assessing Forecast Uncertainty in the NDFD
![Page 1: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/1.jpg)
Assessing Forecast Assessing Forecast Uncertainty in the Uncertainty in the NDFDNDFD
Report to MDLReport to MDL
Matt Peroutka, Greg Zylstra, John Matt Peroutka, Greg Zylstra, John WagnerWagner
July 1, 2005July 1, 2005
![Page 2: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/2.jpg)
OverviewOverview
Overall ProcessOverall Process MethodsMethods
– Data SourcesData Sources– Transformation to PercentilesTransformation to Percentiles– Diagnostic DataDiagnostic Data
ResultsResults– Transformation to PercentilesTransformation to Percentiles– Modeling the Joint DistributionModeling the Joint Distribution– Diagnostic DataDiagnostic Data
PlansPlans
![Page 3: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/3.jpg)
Overall ProcessOverall Process
NUNCA
NDFD Forecast
NDFD Performance
Related Guidance
Expected Distribution of Observations
![Page 4: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/4.jpg)
What’s in a name?What’s in a name?
NDFD UNCertainty Assessment NDFD UNCertainty Assessment (NUNCA)(NUNCA)
Numerical Uncertainty Numerical Uncertainty Assessment of NDFD via Assessment of NDFD via Climatology and Ensembles Climatology and Ensembles (NUANCE) (NUANCE)
![Page 5: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/5.jpg)
Development PhaseDevelopment Phase
Observations (x)
Forecasts (f)
Diagnostic Data (d)
MergeJoint
Distribution Model p(f,x,d)
Transform to
Percentiles
![Page 6: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/6.jpg)
Implementation PhaseImplementation Phase
Forecasts (f)
Diagnostic Data (d)
Joint Distribution
Model p(f,x,d)
Extract
Inferred Conditional Distribution
p(x | f,d)
Transform from
Percentiles
Transform to
Percentiles
![Page 7: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/7.jpg)
Data SourcesData Sources
Ideally, NDFD grids and Analysis of RecordIdeally, NDFD grids and Analysis of Record Prototype with NDFD points and METAR Prototype with NDFD points and METAR
observationsobservations– October 2004 to April 2005, inclusiveOctober 2004 to April 2005, inclusive
US Historical Climatological Network (USHCN)US Historical Climatological Network (USHCN) Ensemble MOS (ENSMOS) archivesEnsemble MOS (ENSMOS) archives
– One bulletin from control runOne bulletin from control run– Five bulletins created from runs with positive Five bulletins created from runs with positive
perturbationsperturbations– Five bulletins created from runs with negative Five bulletins created from runs with negative
perturbationsperturbations
![Page 8: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/8.jpg)
Transformation to Transformation to PercentilesPercentiles Addresses lack of cases in Addresses lack of cases in
development data with extreme development data with extreme values of values of ff or or xx..
Encourages combining of data.Encourages combining of data. NDFD has a short history.NDFD has a short history. NDFD includes a variety of NDFD includes a variety of
forecasting techniques.forecasting techniques.
![Page 9: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/9.jpg)
Diagnostic DataDiagnostic Data
Standard Deviation (SD) of 11 Standard Deviation (SD) of 11 ENSMOS forecasts.ENSMOS forecasts.
““Ensemble Deviation” (ED)Ensemble Deviation” (ED)– Difference each perturbed forecast Difference each perturbed forecast
with control forecast;with control forecast;– Compute r. m. s.Compute r. m. s.
![Page 10: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/10.jpg)
Results: Results: Transformation to Transformation to PercentilesPercentiles Obtained daily MaxT observations for 168 Obtained daily MaxT observations for 168
stations from USHCN.stations from USHCN. Percentile function computed at 5-day Percentile function computed at 5-day
intervals throughout the year.intervals throughout the year. Standard probability distributions used to Standard probability distributions used to
model distribution.model distribution.– Percentile function fitted to observations.Percentile function fitted to observations.– Fit parameters expressed as cosine series over day Fit parameters expressed as cosine series over day
of the year.of the year.– Quality of fit judged subjectively.Quality of fit judged subjectively.– Additional terms added to cosine series, if needed.Additional terms added to cosine series, if needed.
![Page 11: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/11.jpg)
![Page 12: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/12.jpg)
![Page 13: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/13.jpg)
![Page 14: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/14.jpg)
Statistical Statistical DistributionsDistributionsDistributioDistribution n
Variable Variable CommentComment
Normal Normal MaxT MaxT Poor fit “in the tails” Poor fit “in the tails”
Normal Normal ln (MaxT) ln (MaxT) Improved fit “in the tails” Improved fit “in the tails”
Binormal Binormal MaxT MaxT Skewness improved fit for some Skewness improved fit for some stations. Poor fit “in the tails.”stations. Poor fit “in the tails.”
Logistic Logistic MaxT MaxT Better fit than either version of Better fit than either version of Normal Normal
Laplace Laplace MaxT MaxT Worst fitWorst fit
Gumbel Gumbel MaxT MaxT Skewness improved fit for some Skewness improved fit for some stations. stations.
GumbelGumbel -(MaxT) -(MaxT) Skewness improved fit for some Skewness improved fit for some stations. stations.
Generalized Generalized Lambda Lambda
MaxTMaxT Best overall fitBest overall fit
![Page 15: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/15.jpg)
![Page 16: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/16.jpg)
Results: Modeling Results: Modeling Joint DistributionJoint Distribution Modeling Modeling p(f,x,d)p(f,x,d) straightforward straightforward
for prototype.for prototype.– Small number of stations.Small number of stations.– All values can be retained in All values can be retained in
memory.memory.– Will be re-designed for grids.Will be re-designed for grids.
![Page 17: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/17.jpg)
![Page 18: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/18.jpg)
![Page 19: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/19.jpg)
Results: Diagnostic Results: Diagnostic DataData Stratified scatter diagrams for Stratified scatter diagrams for
day 7 day 7 – ED < 6 ED < 6 °F°F– ED ED ≥ ≥ 6 6 °F°F
![Page 20: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/20.jpg)
![Page 21: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/21.jpg)
![Page 22: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/22.jpg)
Results: SD vs. EDResults: SD vs. ED
NDFD percentile errors binned NDFD percentile errors binned (interval 0.1 (interval 0.1 °F) °F) by associated by associated ED/SD value.ED/SD value.
Plot NDFD percentile error vs. Plot NDFD percentile error vs. ED/SD.ED/SD.
![Page 23: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/23.jpg)
![Page 24: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/24.jpg)
![Page 25: Assessing Forecast Uncertainty in the NDFD](https://reader035.fdocuments.net/reader035/viewer/2022062518/56814bf1550346895db8db15/html5/thumbnails/25.jpg)
Future PlansFuture Plans
Quantitatively assess uncertainty.Quantitatively assess uncertainty.– Variations by forecast projection.Variations by forecast projection.
Expand to include MinT.Expand to include MinT. Prototype products.Prototype products.
– 50% confidence interval around NDFD50% confidence interval around NDFD– Probability Density FunctionProbability Density Function– Exceedence probabilities for key valuesExceedence probabilities for key values