1

On the use of radar data to verify mesoscale model precipitation forecasts

Martin Goeber and Sean MiltonModel Diagnostics and Validation group

Numerical Weather Prediction Division

Met Office, Bracknell, U.K.contributions from Clive Wilson, Dawn Harrison, Dave Futyan

and Glen Harris

2

Outline of talk

Importance of precipitation verification Verifying observations and ‘wet’ model Statistical methods and interpretation Examples from operational mesoscale model

forecasts from the wet autumn 2000

3

12 km12

km

5kmgauge

Model

5km

Nimrod

Rain gauge representative of an area of about 20 km2

on mesoscale timescales(Kitchen and Blackall 1992)

# observations per model grid box 0.1:1 5:1

4

Characteristics of the Nimrod data Ground clutter removal fixed Z-R conversion attenuation correction removal of corrupt images removal of anaprop accounting for variations in the vertical

reflectivity profile gauge adjustment

5

‘Wet’ Forecast system characteristics

3D-Var latent heat nudging, 3D-cloud from MOPS cloud microphysics with ice and explicit

calculation of transfer between phases prognostic cloud liquid water and ice penetrative convection scheme based on an

ensemble of buoyant entraining plumes with a treatment of downdraughts

6

Categorical statisticsCount concurrent event/no-event, e.g.

precipitation > 2 mm / 3 hours

Observations

Yes No

Yes Hits

aFalse Alarms

b

Fore

cast

s

No Misses

cCorrect rejections

d

7

Categorical measures (1)

( ¦ )

( ¦ )

p f o

p f o

aH

a c

bF

b d

Hit rate

False alarm rate

8

Which measures for categorical statistics ?

complete description of 2*2 contingency table description of different aspects of relationship

between forecasts and observations, e.g. independence from marginal distributions

confidence intervals interpretation relationship to value

9

Categorical measures (2)( )

( )_

p f

p o

a bfrequency bias

a c

2

( ¦ ) ( ¦ ) 1

( ) ( ) 1

( , )

( )( )O

p f o p f o

HKS accuracy event accuracy non event

Cov O F ad bcHKS H F

s a c b d

HKS

Frequency bias

Hansen-Kuipers score

1

-1

( ) / ( ) /(1 )

( ¦ ) ( ¦ )

( ¦ ) ( ¦ )

1 1

odds p x p x p p

p f o p f o

p f o p f o

H F ad

H F bc

Odds ratio

10

Categorical measures (3)

Confidence intervals

2 22 4( )( )( )

4 ( )( )

n a b c d HKSs HKS

n a b c d

/ 2 / 2

1 1 1 1(log )

log (log ) log log (log )

sa b c d

z s z s

11

Autumn 2000 Orography Accumulation

12

Accumulation autumn (SON) 200012km resolution, 18-24 h forecasts Model Nimrod-obs Bias

13

Rain/no-rain (>0.4 mm/6hrs)12km resolution, 12-18 h forecasts Bias HKS Odds

ratio

14

Heavy precipitation (>4mm/6hrs)12km resolution, 12-18 h forecasts Bias HKS

Odds ratio

15

Heavy precipitation (>4mm/6hrs)36km resolution, 12-18 h forecasts Bias HKS

Odds ratio

16

Estimates of confidence intervals

Minimum(a,b,c,d) Error(HKS) Error(log(OR))

17

Frequency bias

Hansen-Kuipers score

Odds ratio

Summary (6h)

18

Frequency bias

Hansen-Kuipers score

Odds ratio

Summary (3h)

19

Regionally integrated statistics

a) observed area, b) hourly accumulationc) wet area, d) maximum

A) Obs b) 2-3hrc) 8-9hr d) 14-15e) 20-21 f) 26-27

20

Regionally integrated statistics

21

Regionally integrated statistics

Probability for rain in one hour

Nimrod (obs) 6-12hrs. forecast 18-24 hrs. forecast

22

Regionally integrated statistics

Brier skill score for p(rain in one hour)

6-12 hrs. forecasts 18-24 hrs. forecasts

23

Summary

Nimrod (radar) data are relatively great for verifying mesoscale quantitative precipitation forecasts, because of their spatial and temporal resolution and near real time availability.

Last autumn’s extreme precipitation in England and Wales was relatively well forecasted.

Time series of regionally integrated statistics and categorical data analysis provide scientifically based, yet customer friendly, measures to verify quantitative precipitation forecasts.

24

Future developments

Application and development of tests on significance of difference between two samples (e.g. convective vs. frontal precipitation, orographic enhancement, spinup, dependency on resolution, new model formulations)

Extension of investigations on catchment scale Comparison of spatio-temporal spectral

characteristics of model and observations Lagrangian (event based) statistics