Lessons Learned and

Initial Best Practices

Converting from MDL matrices to GFE Formatters

  

David ZaffNOAA/NWS Weather Forecast Office,

Albany, New YorkNovember 4, 2003

ALB IFPS Methodology Before(prior to 06/03)

• 1st draft: GFE

• 2nd draft: Matrix– All fine tuning done through matrix– All updates done through matrix – Grids would

not change for up to 12 hrs.

• 3rd draft: post editing ZFP

• 4th draft: post editing SAF

ALB IFPS Methodology Before(prior to 06/03)

• More time was often spent on the matrix than on the grids

• Little detail in grids

• Grids showed poor consistency – Between offices

• (no use of ISC)– Meteorologically across our grids

• (ex Def R- w/ a 50% PoP)– Forecast vs. Reality

• Both in short term and meteorologically (unrealistic temp changes)

ALB IFPS Methodology NOW

• GFE always open– GRIDS are the “flagship product”

• ZFP/SAF produced from the GRIDS– (Actually all text products are produced from grids)

ALB IFPS Methodology NOW

• Biggest change:– We’ve taken a step out of the forecast

process– We have a better relationship between the

grids and the text

Lessons Learned

• 1) Any Workload improvement?

• 2) Matrix vs GFE Formatters – any time savings?

• 3) Four Reasons the text is post edited

• 4) Amount of text editing varies from forecaster to forecaster

• 5) The SAF and ZFP

Lessons Learned• 1) Is there any workload improvement??

Lessons Learned• 1) Is there any workload improvement??

• YES!!!• Because you remove a step in the

forecast process – there is indeed a workload improvement

• BUT – you spend more time and effort in the grids

• Don’t forget – this is where the meteorology is!

“grid ache”

Lessons Learned

• 2) Is producing text quicker via GFE Formatters than with the Matrix?

Lessons Learned

• 2) Is producing text quicker via GFE Formatters than with the Matrix?

• Yes and No!

– GFE Formatters are a HUGE time saver– GFE Formatters do not save any time because you have to put

much more effort into the grids

• The point is that while in the grids, you spend more time thinking in terms of meteorology and less time working with text output

Lessons Learned

• 3) Four Reasons the text is post edited (in the post Matrix era):

– 1) Grid problems – 2) Large zone groupings (we use Xnow)– 3) Cosmetic wording changes– 4) Formatter problems

Lessons Learned

• Four Reasons the text is post edited:– 1) Grid Problems:

• Most text issues are grid related– Often the grids do not match the desired text

• Problem 1:– It is easy to modify the grids to “fit” the desired text. The

result is that the grids are not consistent in reality or meteorologically

– There is a learning curve here! The ability to adjust grids to match text w/out compromising grids takes time

Lessons Learned

• Four Reasons the text is post edited:– 1) Grid Problems:

• Most text issues are grid related– Often the grids do not match the desired text

• Problem 2: – Often it is too easy to simply edit the text.

» It takes time to adjust the grids and rerun the formatter

– This leaves the problem for the next forecaster

Lessons Learned

• Four Reasons the text is post edited:– 2) Large zone groupings

• We use Xnow – there are other alternatives

• Problem: – The zone groupings are determined

independent of GRIDS, and output therefore doesn’t always turn out as expected.

• Solution 1: Regroup zones• Solution 2: Nudge grids (not recommended in

most cases)

Lessons Learned

• Four Reasons the text is post edited:– 2) Large zone groupings

• We use Xnow – there are other alternatives

• Ultimate solution: One zone/county one group!

– While this will increase the size of the ZFP and therefore increase proofreading time, it will greatly improve formatter’s capability – less errors overall!

Lessons Learned

• Four Reasons the text is post edited:– 3) Cosmetic wording changes

• Forecasters need to let go of wording that do not meteorologically add to the forecast.

• Example:– GFE output - Partly cloudy this morning then becoming mostly

cloudy– Modified text - Increasing clouds

• While the above change makes for a cleaner forecast, it does not add meteorology to the forecast.

Lessons Learned

• Four Reasons the text is post edited:– 3) Cosmetic wording changes

• Cosmetic wording changes are not an efficient use of forecaster time.

• Our expertise and therefore our time should be put into the grids.

Lessons Learned

• Four Reasons the text is post edited:– 4) Formatter Problems:

• Pros:–You have total control

• Cons:–It takes time to fix problems without

overriding grids or introducing new errors

Lessons Learned

• 4) Amount of text editing varies from forecaster to forecaster

– Tough problem• The GFE has a very steep learning curve

– (and you thought differential equations were hard!!)

• Some take to the grids like a video game, and pick up smart tools and and ways to make the “grids work” such that the text comes out as expected.

Lessons Learned

• 5) The SAF and ZFP– This is ALY’s biggest problem:

• We massage the grids for the ZFP, with (hopefully) minimal text edits.

• THEN, we have do it all over again with the SAF

Lessons Learned

• 5) The SAF and ZFP• The SAF contains large sampling areas• Larger areas mean more potential wording issues• Same problem with large zone groupings

Lessons Learned

• 5) The SAF and ZFP

• Solution??

• We don’t have one yet!– An easy one would be to ABANDON THE

SAF, and simply use the ZFP– Better – add in local effects.

Initial Best Practices

• Quickly test wording by looking at one zone group and one period output.

• Control text detail without modifying grids

• Test/Non-operational GFE box

• Get everyone involved!

• Issuance Times

Lessons Learned

• One more comment:

• ALY switched from Matrix to GFE formatters in June, AND we switched from NWS directive C-11 to 10-503

• How many folks contacted us about the change???

Lessons Learned

• One more comment:

• ALY switched from Matrix to GFE formatters in June, AND we switched from NWS directive C-11 to 10-503

• How many folks contacted us about the change???– ZERO!!!!

Initial Best Practices

• Quickly test wording by looking at one zone group and one period output.– It normally takes 20s-30s to run the formatter for all zone

groupings and all forecast periods To make a change to the text you:

• edit the grids• rerun the formatter• wait ANOTHER 20-30 seconds…

– This process begs the forecaster to ignore the grids and focus on the text, introducing grid-text discrepancies!

– The formatter runs VERY quickly when running through one zone, one period!!!!!!!!

Initial Best Practices

• Quickly test wording by looking at one zone group and one period output.

– To run, you need to choose three items:• Choose issuance time • Choose period of interest • Choose zone grouping (ex NYZ052>054)

– Then you can rerun the above over and over again easily after quick grid edits.

Initial Best Practices

• Control Detail text detail without modifying grids

– GFE Formatters come with a default set of statistical sampling techniques can be overridden semi-permanently, but these cannot easily be changed on the fly.

– The MDL matrix allows the forecaster to change the wording detail

– ALY formatters preserve this useful capability via a GUI designed by Vasil Koleci and local formatter adjustments.

Initial Best Practices

• Control Detail text detail without modifying grids– With this tool, you can have a high amount of detail in the grids, and

gloss over some of this in the text, or leave ALL the detail if you prefer.

– Problem: forecasters reluctant to simplify text, and high detail in wording

can be confusing.

Initial Best Practices

• Test/Non-operational GFE box – This is ABSOLUTELY CRITICAL to testing

nearly everything related to the GFE:• Smart Tools• Smart Initialization• And most importantly – FORMATTERS

– You are SEVERELY limited and will place yourself in a dangerous spot if you work on formatters/GRIDS that are operational.

Initial Best Practices

• Test/Non-operational GFE box – The GFE provides the software needed to

quickly transfer GRIDS, Formatters, Smart tools, etc from one machine to another.

– With an RPP box, you can work on operational grids and formatters in realtime….but offline

Initial Best Practices

• Get everyone involved!– The grids are an ever changing, always

current database.

– Note – ALY is not doing this…

Initial Best Practices

• Get everyone involved!– Why write nowcasts saying it’s going to rain in

2 hrs, when the grids only have a chance of rain?

• Springfield MO: the whole staff is involved– HMTs, Interns, all levels of forecasters and

the management team

Initial Best Practices

• Get everyone involved!– One person on short term– One person on next significant Wx event– One person on extended– Leave all other grids as is to preserve

consistency and prevent “yo-yo-ing” forecast• (Use MOS data to update Max/MinT if you find this

improves point-based verification stats)

Initial Best Practices

• Issuance Times– If you have an “always up to date set of grids”,

then why have issuance times?

– You ought to be able to send out a new text product once an hour…