Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul...

44
Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods and Observations (CIMO) Upper Air and Remote Sensing Technologies (UA&RST)

Transcript of Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul...

Page 1: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Radar Quality Control and Quantitative Precipitation Estimation

Intercomparison ProjectStatus

Paul JoeEnvironment Canada

Commission of Instruments, Methods and Observations (CIMO)Upper Air and Remote Sensing Technologies (UA&RST)

Page 2: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Outline

• Project Concept

• The Problem

• Overview of Data Quality Techniques

• Pre-RQQI Results

• Status

Page 3: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

External Factors

Page 4: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Segmenting the DQ Process for Quantitative Precipitation Estimation

Remove Artifacts- Cleaned Up 3D volume

Estimating Surface/3D Reflectivity

Estimating Surface/3DPrecipitation

MosaicingSpace-Time Estimation

Focus on Reflectivity

Page 5: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

NowcastingClear Air Echo as Information

Page 6: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Segmenting the DQ Process

Remove Artifacts- Cleaned Up 3D volume

Estimating Surface/3D Radar Moments

Estimating Surface3D Precipitation (Classification)

MosaicingSpace-Time Estimation in 3D

Reflectivity

Radial Velocity

Dual-Polarization

Page 7: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Every radar has clutter due to environment!

Page 8: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Sea Clutter and Ducting

Page 9: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Electromagnetic Interference

Page 10: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Techniques

Page 11: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

CAPPI is a classic technique to overcome ground clutter

VVO

5o

4

3

2

1

0

Lines are elevation angles at 1o spacing, orange is every 5o.

Page 12: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Canada Australia

U.S./China VCP21 Whistler Valley Radar

3.0 CAPPI

1.5 CAPPI

There are a variety of Scan Strategies(CAPPI Profiles)

Make better or drop

Page 13: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

The elevation angles but nature of weather important for CAPPI

2.5o

1.5o

0.5o

PPI’s1.5km CAPPI

Page 14: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Doppler Zero Velocity Notch

1. Doppler Velocity Spectrum• Pulse pair (time domain)• FFT (frequency domain)

2. Reflectivity statistics

Before

After

Page 15: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Doppler Filtering

Page 16: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.
Page 17: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

SNOWRAIN

Too much echo removed! However, better than without filtering?

Page 18: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Data Processing plus Signal Processing

Texture + Fuzzy Logic + Spectral

Dixon, Kessinger, Hubbert

Data Processing plus Signal Processing

FUZZY LOGIC

Page 19: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.
Page 20: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Removal of Anomalous Propagation

NONQC QC

Liping Liu, CMA

Page 21: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

The Metric of Success

Page 22: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Iso-range “Variance” as an intercomparison Metric

Daniel Michelson, SMHI

Accumulation – a winter season log (Raingauge-Radar Difference)

No blockageRings of decreasing value

Difference increases range!

almost

Page 23: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Convection

Stratiform

Snow

Vertical Profile of Reflectivity is smoothed as the beam spreads in range

Due to Earth curvature and beam propagating above the weather.

Page 24: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Variance Metric

Similar to before except area of partial blockage contributes to lots of scatterAlgorithms that are able to infill data should reduce the variance in the scatter!

Michelson

Page 25: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Proposed Metric

Page 26: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Alternate MetricsAccumulation of Radial Velocity should produce the mean wind for the site.

Both look believable, maybe difference is due to different data set length

nonQC QC

Page 27: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Modality

• Need a variety of techniques

• Need a variety of scan strategies

• Need a variety of data sets that integrate to a uniform pattern

• Need weather with a variety of artifacts

Page 28: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Pilot Study

Purpose is to test the assumptions of the project modality-Short data sets for uniformity-Check the interpretation of the metric-Variety of scan strategies, algorithms, etc-Evaluate feasibility

Page 29: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Uniform Fields

Page 30: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Sample Cases

• Uniform with local clutter (XLA)• Uniform with partial blocking (WVY)• Urban Clutter/Niagara Escarpment (WKR)• Strong Anomalous Propagation Echo (TJ 2006)• Strong AP with Weather (TJ 2007)• Sea Clutter (Sydney AU, Kurnell)• Sea Clutter / Multi-path AP (Saudi 2002)• Convective Weather with Airplane Tracks - One

season (TJ Radar 2007)

Page 31: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

XLA

The data accumulates to uniform pattern. Widespread snow. A baseline case. IRIS formatted data. 24 elevation angles. Doppler (dBZT, dBZc, Vr, SPW) at low levels. Range res = 1km or 0.5 km. Az res = 1 or 0.5 degrees.

Page 32: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

WVY

The data accumulates to uniform pattern with an area of blockage. Widespread snow. A baseline case. IRIS formatted data. 24 elevation angles. Doppler (dBZT, dBZc, Vr, SPW) at low levels. Range res = 1km or 0.5 km. Az res = 1 or 0.5 degrees.

Page 33: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

WKR

The data accumulates to uniform pattern with an area of blockage. Widespread snow. Urban (skyscrapers) and small terrain clutter. IRIS formatted data. 24 elevation angles. Doppler (dBZT, dBZc, Vr, SPW) at low levels. Range res = 1km or 0.5 km. Az res = 1 or 0.5 degrees.

Page 34: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

BSCAN of Z accumulation with no filtering, Doppler and CAPPI

CAPPI

Doppler

No Filtering

Azimuth

Ran

ge [

km]

0

1

00

Page 35: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Probability Density Function of Reflectivity as a function of range

Raw Doppler CAPPI

Page 36: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

What length of data sets are needed?

Highly Variable More uniform, smoother, more continuous

Page 37: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

The Techniques

• Doppler Notching

• CAPPI 1.5km

• CAPPI 3.0km

• Mixed of Doppler Notching and CAPPI

• Radar Echo Classifier (REC)– Anomalous Propagation– Sea Clutter

• REC-CMA

Page 38: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

The Statistic

Page 39: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Spread of PDF (at constant range) for various cases and techniques…

Page 40: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.
Page 41: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.
Page 42: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Status

Page 43: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Status and Acknowledgements

•Kimata, Japan•Liu, China•Seed, Australia•Michelson, Sweden•Sempere-Torres, Spain•Howard, USA•Hubbert, USA•Calhieros, Brazil•Levizzani, Italy/IPWG•Gaussiat, UK/OPERA HUB•Donaldson, Canada

Data Providers

Algorithm Providers

Evaluation Team

Reviewers

Page 44: Radar Quality Control and Quantitative Precipitation Estimation Intercomparison Project Status Paul Joe Environment Canada Commission of Instruments, Methods.

Summary

• On-going• Data Providers, Processors

identified• ODIM_H5 format identified• BOM will host and convert

data for Data Processors• Initial Metric identified• Review

– Variety of techniques– Variety of scan strategies– Variety of data sets – Weather (e.g. convective, snow)

with a variety of artifacts

Alternate radial velocity metric