Robin HoganEwan O’ConnorDamian Wilson

Malcolm Brooks

Evaluation statistics of cloud fraction and

water content

Overview• Cloudnet level 3 data• A solution to the problem of evaluating high cloud?• Summary of errors in model cloud fraction and water

content climatologies over Europe– ECMWF model– KNMI Regional atmospheric climate model (RACMO)– Met Office mesoscale and global– SMHI Rossby Centre atmospheric model (RCA)– Meteo France ARPEGE model– DWD Lokal Modell

• Forecast skill

Cloudnet level 3 data• Level 3 files summarise the comparison of a

observations and model over a certain period:– Long-term mean of a quantity versus height– Separation into “freq. of occurrence” and “amount when

present”– PDFs in height ranges 0-3 km, 3-7 km, 7-12 km and 12-18 km– Skill scores versus height for different thresholds

• Separate level-3 files/quicklooks are produced for– Each variable: cloud fraction, LWC, IWC, high cloud fraction– Each site: 4 European, 4 ARM (so far)– Each model: 7 so far, plus persistence/climatology forecasts– Each month and each year– Different forecast lead times (Met Office meso and DWD only)– In principle: different model resolutions / parameterisations

• Over 5000 files so far!

Observations

Met Office

Mesoscale Model

ECMWF

Global Model

Meteo-France

ARPEGE Model

KNMI

RACMO Model

Swedish RCA model

Cloud fraction

What can we do about high cloud?

• All models see more cirrus than observed– We use the known radar sensitivity to remove clouds from

model that we would not expect to detect (affecting heights > 7 km)

– Does not usually remove enough cloud to bring into agreement

• Are all models wrong?– Or does radar miss more IWC than it thinks due to small

particles?

ARM Nauru 8 Nov 2003

Night-time

Radar35 GHz MMCR

LidarMerged ceilometer and micropulse lidar

October 2003: Normal processing

No periods when rain rate > 8 mm/h

Large difference between observations and ECMWF model, whether model is modified for radar sensitivity or not

…only periods of high lidar sensitivity

Consider only night-time and periods when lidar is unobscured by liquid cloud, rain or melting ice

Liquid clouds removed from comparison

Cloud fraction OK but peak 2 km too high

One month later

Model grossly overestimates high cloud fraction

To evaluate high clouds in models: need a high sensitivity lidar and appropriate sampling of data (both model and observations)

ECMWF cloud

fraction• Cabauw 2002:

– Amount when present is good

– Mean cloud fraction and frequency of occurrence too high in the boundary layer

– Need to treat snow as cloud in the model

• Chilbolton 2004 (and all mid-latitude sites 2003-2005):– Boundary layer

cloud fraction much more accurate

– Still need to treat snow as cloud

ECMWF water content• Mean LWC and IWC accurate to

observational uncertainties• Freq. of occurrence too high;

amount when present too low• Inconsistent with cloud frac.?• PDF shows occurrence of low

values is too high

Chilbolton 2004: LWC

Chilbolton 2004: IWC

RACMO• Cloud fraction errors similar to

ECMWF before 2003• Water content errors (mean,

frequency of occurrency) much as ECMWF

• Lower IWC in high cirrus

Met Office mesoscalecloud fraction

• Mean amount when present too low through most of atmosphere

• Largely due to inability of model to simulate 100% cloud fraction, as shown by the PDFs

• Error in high cloud needs to be checked using high sensitivity lidar

Cabauw 2004

Met Office global cloud

fraction• Observations show greater

frequency of cloud with increased gridbox size; opposite in model

• PDF error unchanged

Cabauw 2004

Met Office mesoscale water content

• Liquid occurrence very good

• Boundary layer perhaps too low

• Mean LWC underestimated above 3 km

• Similar to previous result found for occurrence of supercooled layers

Chilbolton 2004: LWC Chilbolton 2004: IWC

• Mean IWC very good

• Frequency of ice cloud occurrence too high above 3 km

• PDFs much better than ECMWF!

Met Office global water content• Mean LWC

similar but frequency of occurrence much lower

• IWC generally higher

Chilbolton 2004: LWC Chilbolton 2004: IWC

SMHI Rossby Centre model• Amount when present

reasonable but frequency of occurrence and overall mean much too high

• Similar picture for LWC/IWC: mean overestimated due to cloud too often

Palaiseau 2004

Meteo France

cloud fraction

• Before Apr ‘03– Amount when

present far too low

– High values rarely predicted

Cabauw 2002

• After Apr ‘03– Amount when

present very good (better than Met Office & ECMWF)

– Mean cloud fraction much better

– Amazingly, worse agreement with synoptic obs of cloud cover!

Cabauw 2004

Meteo Fr. water content

• Boundary-layer LWC too low• Frequency of supercooled liquid

much too high– Need to change the T-dependent

ice/liquid ratio

• PDF of LWC and IWC too narrow• Mean IWC too low in mid-levels

Chilbolton 2004: IWC

Chilbolton 2004: LWC

DWD cloud

fraction• Cloud fraction

generally very good – But frequency

of occurrence always overestimated by 20-30%

• PDFs particularly well simulated

Chilbolton 2004

DWD water

content

• Frequency of liquid cloud occurrence too high

• LWC in supercooled clouds too high

• Frequency of ice cloud occurrence OK

• Mean IWC and mean amount when present (in-cloud IWC) are both underestimated below 7 km

Chilbolton 2004

Equitable threat score• Measure of skill of forecasting

cloud fraction>0.05• Persistence and climatology

shown for comparison• Lower skill in summer

convective events

Skill versus lead time• Unsurprisingly UK model most accurate in UK,

German model most accurate in Germany!

• Typically 500-mb geopotential height used in operational forecast verification

• Cloud fraction a more challenging test: more rapid loss of skill with time