Robin HoganLast Minute Productions Inc.

How to distinguish rain How to distinguish rain from hail using radar:from hail using radar:

A A cunning,cunning, variationalvariational methodmethod

OutlineOutline• Increasingly in active remote sensing (radar and lidar), many

instruments are being deployed together, and individual instruments may measure many variables– We want to retrieve an “optimum” estimate of the state of the

atmosphere that is consistent with all the measurements– But most algorithms use at most only two instruments/variables

and don’t take proper account of instrumental errors

• The “variational” approach (a.k.a. optimal estimation theory) is standard in data assimilation and passive sounding, but has only recently been applied to radar retrieval problems– It is mathematically rigorous and takes full account of errors– Straightforward to add extra constraints and extra instruments

• In this talk, it will be applied to polarization radar measurements of rain rate and hail intensity– Met Office recently commissioned new polarization radar– A variational retrieval is a very useful step towards assimilation of

polarization data

• Radiance at a particular wavelength has contributions from large range of heights

• A variational method is used to retrieve the temperature profile

PassivePassivesensingsensing Active sensingActive sensing

No attenuation With attenuation

• Isolated weighting functions (or Jacobians) so don’t need to bother with variational methods?

• With attenuation (e.g. spaceborne lidar) weighting functions are broader: variational method required

ChilboltoChilbolton 3GHz n 3GHz

radarradar: Z: Z

• We need to retrieve rain rate for accurate flood forecasts• Conventional radar estimates rain-rate R from radar reflectivity

factor Z using Z=aRb

– Around a factor of 2 error in retrievals due to variations in raindrop size and number concentration

– Attenuation through heavy rain must be corrected for, but gate-by-gate methods are intrinsically unstable

– Hail contamination can lead to large overestimates in rain rate

ChilboltoChilbolton 3GHz n 3GHz

radarradar: : ZZdrdr

• Differential reflectivity Zdr is a measure of drop shape, and hence drop size: Zdr = 10 log10 (ZH /ZV)– In principle allows rain rate to be retrieved to 25%– Can assist in correction for attenuation

• But– Too noisy to use at each range-gate– Needs to be accurately calibrated– Degraded by hail ZV

ZH

1 mm

3 mm

4.5 mm

ChilboltoChilbolton 3GHz n 3GHz

radarradar: : dpdp

phase shift

• Differential phase shift dp is a propagation effect caused by the difference in speed of the H and V waves through oblate drops– Can use to estimate attenuation– Calibration not required– Low sensitivity to hail

• But– Need high rain rate– Low resolution information:

need to take derivative but far too noisyto use at each gate: derivative can be negative!

• How can we make the best use of the Zdr and dp information?

Simple Simple ZZdrdr methodmethod

• Use Zdr at each gate to infer a in Z=aR1.5

– Measurement noise feeds through to retrieval

– Noise much worse in operational radars

Observations

Lookup

table

Noisy or Negative

Zdr Retrieval

Noisy or no retrievalRainrate

Variational methodVariational method• Start with a first guess of coefficient a in Z=aR1.5

• Z/R implies a drop size: use this in a forward model to predict the observations of Zdr and dp

– Include all the relevant physics, such as attenuation etc.

• Compare observations with forward-model values, and refine a by minimizing a cost function:

2

2

2

2

,,

12

2

,,

apdpdr a

api

fwdidpidp

n

i Z

fwdidridr aaZZ

J

Observational errors are explicitly included, and the

solution is weighted accordingly

For a sensible solution at low rainrate, add an a

priori constraint on coefficient a

+ Smoothness constraints

Finding the solutionFinding the solutionNew ray of dataFirst guess of x

Forward modelPredict measurements y and Jacobian H from state vector x using forward model H(x)

Compare measurements to forward modelHas the solution converged?2 convergence test Gauss-Newton iteration step

Predict new state vector: xi+1= xi+A-1{HTR-1[y-H(xi)]

+B-1(b-xi)}where the Hessian is

A=HTR-1H+B-1

Calculate error in retrievalThe solution error covariance matrix is S=A-1

No

Yes

Proceed to next ray

– In this problem, the observation vector y and state vector x are:

na

a

ln

ln 1

x

mdp

dp

mdr

dr

Z

Z

1

1

y

First guess of First guess of aa

First guess: a =200 everywhere

• Use difference between the observations and forward model to predict new state vector (i.e. values of a), and iterate

Rainrate

Final iterationFinal iteration• Zdr and dp are well fitted by forward model at

final iteration of minimization of cost function

Rainrate

• Retrieved coefficient a is forced to vary smoothly– Prevents random noise in measurements feeding through into

retrieval (which occurs in the simple Zdr method)

Enforcing smoothnessEnforcing smoothness• In range: cubic-spline basis functions

– Rather than state vector x containing “a” at every range gate, it is the amplitude of smaller number of basis functions

– Cubic splines solution is continuous in itself, its first and second derivatives

– Fewer elements in x more efficient!

Representing a 50-point function by 10 control points

• In azimuth: Two-pass Kalman smoother– First pass: use one ray as a constraint on the retrieval at the

next (a bit like an a priori)– Second pass: repeat in the reverse direction, constraining

each ray both by the retrieval at the previous ray, and by the first-pass retrieval from the ray on the other side

• Observations

• Retrieval– Note:

validation required!

Forward-model values at final iteration are essentially least-squares fits to the observations, but without instrument noise

Full scan from Full scan from ChilboltonChilbolton

Nominal Zdr error of ±0.2 dB Additional random error of ±1 dB

Response to observational Response to observational errorserrors

What if we What if we use only use only ZZdrdr

or or dp dp ? ? Very similar retrievals: in moderate rain rates, much more useful information obtained from Zdr than dp

Zdr

only

dp

only

Zdr

and

dp

Retrieved a Retrieval error

Where observations provide no information, retrieval tends to a priori value (and its error)

dp only useful where there is appreciable gradient with range

• Observations

• Retrieval

Difficult case: differential attenuation of 1 dB and differential phase shift of 80º

Heavy Heavy rain andrain and

hailhail

How is hail How is hail retrieved?retrieved?

• Hail is nearly spherical– High Z but much lower Zdr than

would get for rain– Forward model cannot match both

Zdr and dp

• First pass of the algorithm– Increase error on Zdr so that rain

information comes from dp

– Hail is where Zdrfwd-Zdr

> 1.5 dB and Z > 35 dBZ

• Second pass of algorithm– Use original Zdr error

– At each hail gate, retrieve the fraction of the measured Z that is due to hail, as well as a.

– Now the retrieval can match both Zdr and dp

Distribution of Distribution of hailhail

– Retrieved rain rate much lower in hail regions: high Z no longer attributed to rain

– Can avoid false-alarm flood warnings

– Use Twomey method for smoothness of hail retrieval

Retrieved a Retrieval error Retrieved hail fraction

SummarySummary• New scheme achieves a seamless transition

between the following separate algorithms:

– Drizzle. Zdr and dp are both zero: use a-priori a coefficient

– Light rain. Useful information in Zdr only: retrieve a smoothly varying a field (Illingworth and Thompson 2005)

– Heavy rain. Use dp as well (e.g. Testud et al. 2000), but weight the Zdr and dp information according to their errors

– Weak attenuation. Use dp to estimate attenuation (Holt 1988)

– Strong attenuation. Use differential attenuation, measured by negative Zdr at far end of ray (Smyth and Illingworth 1998)

– Hail occurrence. Identify by inconsistency between Zdr and dp measurements (Smyth et al. 1999)

– Rain coexisting with hail. Estimate rain-rate in hail regions from dp alone (Sachidananda and Zrnic 1987)

• Could be applied to new Met Office polarization radars– Testing required: higher frequency higher attenuation!

Hogan (2007, J. Appl. Meteorol. Climatology)

Conclusions and ongoing Conclusions and ongoing workwork

• Variational methods have been described for retrieving cloud, rain and hail, from combined active and passive sensors– Appropriate choice of state vector and smoothness constraints

ensures the retrievals are accurate and efficient– Could provide the basis for cloud/rain data assimilation

• Ongoing work: cloud– Test radiance part of cloud retrieval using geostationary-satellite

radiances from Meteosat/SEVIRI above ground-based radar & lidar– Retrieve properties of liquid-water layers, drizzle and aerosol– Incorporate microwave radiances for deep precipitating clouds– Apply to A-train data and validate using in-situ underflights– Use to evaluate forecast/climate models– Quantify radiative errors in representation of different sorts of cloud

• Ongoing work: rain– Validate the retrieved drop-size information, e.g. using a

distrometer– Apply to operational C-band (5.6 GHz) radars: more attenuation!– Apply to other radar problems, e.g. the radar refractivity method