An algorithm developer’s tool

Post on 25-Feb-2016

46 views 2 download

Tags:

description

An algorithm developer’s tool. Valliappa.Lakshmanan@noaa.gov National Severe Storms Laboratory Norman OK, USA http://w ww.wdssii.org/. A developer’s tool. The Warning Decision Support System – Integration Information (WDSS-II) - PowerPoint PPT Presentation

Transcript of An algorithm developer’s tool

An algorithm developer’s tool

Valliappa.Lakshmanan@noaa.govNational Severe Storms LaboratoryNorman OK, USAhttp://www.wdssii.org/

April 22, 2023 lakshman@ou.edu 2

A developer’s tool

The Warning Decision Support System – Integration Information (WDSS-II) A collection of meteorological algorithms for severe

weather analysis, diagnosis and prediction Hail, tornadoes, wind, lightning

An integrated set of loosely coupled tools for: Severe weather diagnosis Image processing Statistical validation Ground-truth verification

April 22, 2023 lakshman@ou.edu 3

WDSS-II applications (algorithms and tools) are just executables. launched on the command line. In deployed systems through scripts.

Can easily change input to filtered form, or accumulate a different product (such as rainfall rate or hail size)

Applications exist for many tasks: Image processing (smoothing, dilating, eroding, etc.) Objective analysis (Cressman, Barnes, Gaussian, etc.) Scoring grids (error statistics) Statistical skill based on ground truth

WDSS-II Applications

w2accumulator –i xmllb:/data/realtime/radar/KTLX/code_index.lb \

-I MaxShear_0-3km \

-o /data/realtime/radar/KTLX/ -r -t “30 60 120”

April 22, 2023 lakshman@ou.edu 4

Creating a new algorithm

An algorithm is essentially a data filter Takes some data as input Produces new data as output

The algorithm developer should be able to specify the scientific processing in the middle Without having to worry about data ingest, data

formats, notification, etc. But provide a library of common computations on

the typical data used.

April 22, 2023 lakshman@ou.edu 5

w2algcreator

W2algcreator is a WDSS-II tool To write the format-independent code for ingesting

data into your application. The algorithm developer writes an XML file

specifying the inputs and adaptable parameters. The algorithm itself is auto-generated!

With a “fill in the blank” for the scientific computation

WDSS-II class libraries can be used for common computations.

Easy to add new input and output formats.

April 22, 2023 lakshman@ou.edu 6

Display

The WDSS-II displays are highly configurable to aid trouble-shooting.Display of intermediate outputs is easy and

convenient.

April 22, 2023 lakshman@ou.edu 7

Example intermediate product

Created with no modification of the display. Just configuration files.

Algorithm developer marks the radar associated with each detection. For easy debugging.

April 22, 2023 lakshman@ou.edu 8

The end-result

So, what kinds of algorithms have been developed in WDSS-II?

April 22, 2023 lakshman@ou.edu 9

Single-radar/Multi-sensor algorithms

Some single-radar (multi-sensor) algorithms in WDSS-II

April 22, 2023 lakshman@ou.edu 10

Multi-radar/multi-sensor algorithms

A typical multi-radar deployment of WDSS-II

April 22, 2023 lakshman@ou.edu 11

Relevance to Q2

Which of those are relevant to Q2?Some existing severe weather algorithms

may be relevantProbability of hail for identifying radar echoes

with potential hail contamination.More likely:

Developing new algorithmsBuilding algorithms as data filtersExisting lower-level tools.

April 22, 2023 lakshman@ou.edu 12

Accumulation algorithm

Six Hour Path of Rotational Shear Accumulation of shear to form rotation tracks.

Accumulation could be as: Maximum Rate Total

This tool could be used for rainfall depth from rainrate for example.

April 22, 2023 lakshman@ou.edu 13

Motion Estimation

Uses K-Means clustering and Kalman filters

Forecast dBZActual dBZ

30 min30 min

April 22, 2023 lakshman@ou.edu 14

Need for new approach

Traditional centroid tracking Accurate at small scales, but not at large scales Inaccurate when storms merge or split Possible to extract trends from the information

Flow-based tracking Cross-correlation, Lagrangian methods, etc. Are accurate at large scales, but not at small scales Not useful in decision support because trends of

storm properties can not be extracted

April 22, 2023 lakshman@ou.edu 15

K-Means clustering

K-Means clustering is a hybrid approachCluster the input data to find clusters

Like centroid-based tracking methodsBut at different scales.

Track the clusters using flow-based methods (minimization of cost-functions)

Like flow-based methodsDoes not involve cluster matching (e.g: Titan)

April 22, 2023 lakshman@ou.edu 16

Example clusters

Two different scales shown

Both scales are tracked

April 22, 2023 lakshman@ou.edu 17

Extrapolation

Smooth the motion estimates spatially using OBAN

techniques (Gaussian kernel)

temporally using a Kalman filter (assuming constant velocity)

Repeat at different scales and choose scale appropriate to extrapolation time period.

April 22, 2023 lakshman@ou.edu 18

Trends

The clusters can be used to extract trends of any gridded field. Configurable to extract minimum, maximum, count,

sum, time-delta, etc. of gridded fields within cluster Even fuzzy combination of multiple fields

Extremely useful for research! Statistical properties of storms Changing drop-size distributions with time Which clusters are convective? Trends in rain-rates … Trends in cloud-top temperatures …

April 22, 2023 lakshman@ou.edu 19

Polygon statistics

Using cluster trends is useful for deriving storm properties.What about extracting statistics around a

fixed location?Maybe around rain gages?

WDSS-II has a tool to provide polygon statistics from any gridded field(s)The polygons can change with time (e.g:

weather service watch areas)

April 22, 2023 lakshman@ou.edu 20

Quality Control Neural Network (QCNN)

Developed for MDA false alarms in non-storm echo. With QCNN, shows over 90%

reduction in the non-storm MDA false alarms and zero change to detections within storm echo.

The same QC technique would be useful in estimating precipitation as well.

Based on local statistics of reflectivity, velocity and spectrum width fields, vertical statistics and morphological image processing

Handles AP/GC, radar artifacts and some biological signals.

Neural network for optimal combination of inputs.

After QC

Before QC

April 22, 2023 lakshman@ou.edu 21

Current uses of WDSS-II in the NWS

WDSS-II is a leading edge system Provides capabilities not yet in the “official” National Weather Service

systems. The Storm Prediction Center

defines daily threat areas launch a WDSS-II domain

automatically configures the data ingest and starts the algorithms. NWS forecast offices

WDSS-II products are converted into AWIPS format and piped the AWIPS displays in several NWS forecast offices.

But the AWIPS display is too restrictive. Therefore … The 4D WDSS-II display is to be implemented as a separate app on

AWIPS but controlled from within D2D. Concept of algorithm development capabilities

Being considered for next redesign of AWIPS

April 22, 2023 lakshman@ou.edu 22

In summary

How can WDSS-II be useful in Q2 As an algorithm development toolkit

Multi-sensor inputs in real-time and for archived cases But limited to user workstations JADE will provide web-based capabilities.

Individual tools Objective Analysis tools and other low-level tools. Image processing filters Quality control of radar data Motion estimation and extrapolation (short-term QPF) Storm statistics Polygon statistics

Please visit this website: http://www.wdssii.org/