Page 1© Crown copyright 2004 The use of an intensity-scale technique for assessing operational...

© Crown copyright 2004

The use of an intensity-scale technique for assessing operational mesoscale precipitation

forecasts

Marion Mittermaier and Clive Wilson


Outline

1. An intensity-scale technique (Casati et al. 2004)2. Model output and data description3. Value added by higher resolution for a severe

flooding event (Boscastle, August 2004)4. A modified sign-test statistic for highlighting

persistent/prevalent errors at the monthly time scale.

5. Radar vs gauge as “truth”6. Concluding remarks

1. An intensity-scale technique

….. best illustrated with an example ….

(from Casati, 2004)

Radar Model forecast

from Casati (2004)

Radar > 1 mm Forecast > 1 mm Binary error image

X > u X < u

Y > u Hits a

False Alarms

b a+b

Y < u Misses c

Correct Rejections

dc+d

a+c b+d a+b+c+d=n


MSE skill score

,

, , ,

1u u random uu

u best u random u random

MSE MSE MSESS LMSE MSE MSE

1

0

-1

-2

-3

-4

luSS ,

threshold (mm/h)

spat

ial s

cale

(km

)

[from Casati (2004)]

Axes multiples of 2


2. Model output and data description

Mesoscale version of the Unified Model (MES) runs 4 times a day at ~12 km over the UK (for Unified Model description see Davies et al., QJRMS, 2005)

Newly implemented 4-km model now runs twice a day over the UK (see Bornemann et al, this conference)

Radar-rainfall accumulations available on a 5 km x 5 km national grid

~2700 rain gauges have been used to produce a daily gridded rainfall product also on a 5 km x 5 km grid


3. Boscastle: the benefit of higher resolution?

How does one assess added benefit?

Output from the MES and 4 km model isn’t directly comparable Basis of comparison should ideally be the same.

Solution:

Average the 4 km model output to the 12 km grid and compare against the same 12-km averaged radar rainfall product.

…. consider 6-hr rainfall between 12-18Z from the 00Z run …

On 16.08.2004 over 180 mm were recorded by one gauge in a 5-hr period during a highly localised flooding event.

12-18Z

12-18Z

12-18Z

4 km 00Z 6 hr rainfall

MES 00Z 6 hr rainfall

4 km 00Z avg 6 hr rainfall E

rror

sca

le (k

m)

2x

16x

1 mm 64 mm

2x

16x

Max radar = 44 mm

68 mm 7 mm46 mm

Rainfall threshold (mm)


Distribution-free test as normality of errors can’t be assumed.

B = number of +ve skill scores for a given scale and intensity during a given time interval, e.g. 1 month.

Hypotheses:

H0 : SS >= 0 (implicit positive and skillful)

H1 : SS < 0 (less skill than a random forecast)

H0 is rejected if b <= bn,where B ~ bi(n, 0.5) for small samples (n < 40), = 0.025

The value of (n – B) / n is shaded in intensity-phase space for each scale and intensity where H0 is rejected.

4. A modified sign-test statistic


Added benefit: comparison of prevalent errors at the monthly time scale

(sub-)“grid” scale errors are more prevalent at trace rainfall totals for the 4 km model

prevalent errors at twice and four times the MES grid length for thresholds > 16 mm are less for the 4 km model (captures large totals better)

May 2005 MES vs radar May 2005 4 km avg vs radar

X X X X X X X X X X X X X X

X X XX X

X X

X48 km

32 mm


5. Radar vs gauge as “truth”

Slight shifts in the distribution of prevalent errors at the monthly time scale

Overall pattern very similar Radar-rainfall fields preferred as they are truly spatial with a

greater observation frequency

August 2004 MES vs radar August 2004 MES vs gauge

X X X X X X X X X X

X X XX X X

X X XX X X


6. Concluding remarks

1. The 4 km model contains much more detail (even when averaged to 12 km)

2. Detail does not necessarily equal accuracy! Raw model output needs to be averaged

3. Scale-intensity analyses show that the need for averaging is (almost) independent of grid length (there is always grid noise, regardless)

4. The difference between error analyses produced using radar (true spatial) and gauge (point-interpolated) fields is minimal. Recommend that radar fields are used also because of the high observation frequency.

Questions?

12-18Z

12-18Z

12-18Z

Radar 12Z 6 hr rainfall

MES 12Z 6 hr rainfall

4 km 12Z avg 6 hr rainfall E

rror

sca

le (k

m)

Rainfall threshold (mm)

19 June 2005Flash flooding caused by thunderstorms overNorth Yorkshire

Haar Wavelet filter

deviation from mean value

mean value

+

+

mean value on all the domain Casati et al., 2004, Met Apps


• Wavelets are locally defined real functions characterised by a location and a spatial scale.• Any real function can be expressed as a linear combination of wavelets, i.e. as a sum of components with different spatial scales. • Wavelet transforms deal with discontinuities better than Fourier transforms do

Haar mother wavelet 1

-1

0 1 2 4 n n+1

An intensity-scale technique using wavelets

wavelet decomposition of the binary errorScale

L

lluu EE

1,

from Casati (2004)

Page 1© Crown copyright 2004 The use of an intensity-scale technique for assessing operational...

Documents

Transcript of Page 1© Crown copyright 2004 The use of an intensity-scale technique for assessing operational...