Clutter Environment Analysis using Adaptive Processing:using Adaptive Processing:

The CLEAN-AP Filter(Informational Briefing)

Sebastian Torres and David WardeCIMMS/University of OklahomaCIMMS/University of Oklahoma

and NSSL/NOAA

NEXRAD Technical Advisory Committee MeetingNorman, OK

19 November, 2009

What is the CLEAN-AP filter?

CLEAN-AP is a novel real-time, automatic integrated approach forautomatic, integrated approach for ground clutter detection and filteringthat produces data with the best possiblethat produces data with the best possible quality while meeting NEXRAD technical requirementsrequirements

NEXRAD TAC Meeting 19 November 2009 Norman, OK2

Outline

• Motivation• Current approach• Current approach – Maps/CMD + GMAP

P d lt ti• Proposed alternative– The CLEAN-AP filter

• Summary and recommendations

• Stay tuned: Performance analyses and comparisons

NEXRAD TAC Meeting 19 November 2009 Norman, OK3

analyses and comparisonswith current approach

WSR-88D Strategic DirectionsSnow et al. (2003)

• “Produce the best quality data possible from the WSR-88D throughout thefrom the WSR 88D throughout the remainder of its service life.”

– “ these applications require that quality…these applications require that quality control/assurance be applied automatically.”

– “Signal processing could beSignal processing could be improved to almost completely mitigate ground clutter…”

NEXRAD TAC Meeting 19 November 2009 Norman, OK4

WSR-88D Clutter Mitigationat the signal processing level (ORDA)

Ground Clutter Detection

Control

Ground Clutter Filtering

Unfiltered data

Filtered dataFilteringdata data

NEXRAD TAC Meeting 19 November 2009 Norman, OK5

Ground Clutter DetectionCurrent Approach

• ORDA Build 10– Static ground clutter maps (BYPASS map)

& 12Static ground clutter maps (BYPASS map)

– Operator-defined Clutter Censor Zones (CCZ)

ORDA Build 11• ORDA Build 11– Lower tilts (split cuts)

Cl tt Miti ti D i i (CMD)

CMD is good!

• Clutter Mitigation Decision (CMD)

– Upper tiltsStatic g o nd cl tte maps (BYPASS map)• Static ground clutter maps (BYPASS map)

– Operator-defined Clutter Censor Zones (CCZ)

NEXRAD TAC Meeting 19 November 2009 Norman, OK6

f f

Ground Clutter DetectionClutter Mitigation Decision (CMD)

• Uses temporal and spatial features in a fuzzy-logic system to automatically detect ground clutter contamination in real timeclutter contamination in real time

– Detections are “filled-in” using spatial filter – Requires filtered and unfiltered dataRequires filtered and unfiltered data– Functionality split between RVP-8 and RCP-8

RCP-8RVP-8 RCP 8RVP 8CMD Flags

I/Q

Clutter Det. Flags

GMAP Moment Comp. CMD

Data S tiM t

I/Q Zf

Z

Z

Doppler Spectrum

S

Sf

NEXRAD TAC Meeting 19 November 2009 Norman, OK7

SortingMoment Comp.

ZuSu

U G i d l f l d i h id h

Ground Clutter FilteringGaussian Model Adaptive Processing (GMAP)

• Uses Gaussian model for clutter to determine notch width– Suppression is limited by maximum notch width

• Needs Blackman window to achieve required suppression• Needs Blackman window to achieve required suppression– Does not produce data with best possible quality

• Uses iterative process to reduce reflectivity bias– Computationally intensive

• Needs at least 16 samples to achieve required suppressionsuppression

– Imposes limit on faster updates

• Not conducive to more spectral processing– Phase is lost from filtered signal– Affected by circular convolution biases

• Algorithm is under Vaisala control

NEXRAD TAC Meeting 19 November 2009 Norman, OK8

• Algorithm is under Vaisala control

Source: Siggia and Passarelli (2004)

The CLEAN-AP Filter (I)CLEAN AP i i• CLEAN-AP is automatic

– No need for user intervention– Real-time detectionReal time detection– No need for clutter maps

• CLEAN-AP produces data with the best possible quality

– Adaptive data windowing finds the best compromise between clutter suppression and data qualitybetween clutter suppression and data quality

• CLEAN-AP meets NEXRAD requirements– Improved suppression (not limited by max. notch width)– Requirements (Z) met with as little as 8 samples

• CLEAN-AP is integrated

NEXRAD TAC Meeting 19 November 2009 Norman, OK9

– One algorithm for ground clutter detection and filtering– Gate-by-gate operation

The CLEAN-AP Filter (II)CLEAN AP “ h ” f f h• CLEAN-AP “sets the stage” for further spectral processingPhase information is not lost– Phase information is not lost

– Immune to biases from circular convolution

• CLEAN-AP is operational on the NWRT PARp– Running in real-time since Sep 2008– Performance informally evaluated by meteorologists

and forecasters (PARISE experiments)and forecasters (PARISE experiments)

• CLEAN-AP consideration as an alternative clutter mitigation solution gmakes sense now

– Re-implementation of an automatic ground clutter detection scheme will be needed after ORDA B12

NEXRAD TAC Meeting 19 November 2009 Norman, OK10

detection scheme will be needed after ORDA B12– In principle, compatible with dual pol., SZ-2, and SPRT

How does CLEAN-AP work? (I)The “lag-1 autocorrelation spectral density” (ASD)

MeasuredTrue

ASDPSD ASD

vv vL k

v

Ground Clutter

v

Leakage

ASDPSD ASD

vvWeather

v

NEXRAD TAC Meeting 19 November 2009 Norman, OK11

v vLeakage

How does CLEAN-AP work? (II)Integrated detection and filtering

• Estimate CNR• Select appropriate data

|ASD|

DC

PSD

tect

ion window

• Compute ASDId tif t ith

Det • Identify components with

clutter contamination– Phase of ASD is near zero

v

ing

due to leakage effect

• Remove clutterzero phase

Arg(ASD)

Filt

eri e o e c utte

• Interpolate weather

ASD i d t ti t

vphase

NEXRAD TAC Meeting 19 November 2009 Norman, OK12

• ASD is used to estimate spectral moments directly

CLEAN-AP vs. Current Approach

RVP-8

MomentI/Q ZDoppler FCLEAN-AP Moment

Comp.

/Q ZDoppler Spectra

F

RCP-8RVP-8CMDCMD Flags

MomentI/Q

Clutter Det. Flags

ZDopple SfGMAP Moment

Comp. CMD Data

SortingMoment

I/Q Zf

Zu

Z

Doppler Spectrum

Su

Sf

NEXRAD TAC Meeting 19 November 2009 Norman, OK13

gComp.

Summary and Recommendation• CLEAN-AP is a real-time, automatic,

integrated approach for ground clutter d i d fil i h ddetection and filtering that produces data with the best possible quality while

h l imeeting NEXRAD technical requirements– Improved performance compared to current approach

• We recommend considering the CLEAN-AP filter as a ground clutter mitigation solution for the NEXRAD network

– TAC endorsement is needed

NEXRAD TAC Meeting 19 November 2009 Norman, OK14