AN ICTPLECTURER

ParameterizationParameterizationCloud Scheme ValidationCloud Scheme Validation

tompkins@ictp.it

ADRIAN

Cloud Validation: The issuesCloud Validation: The issues

• AIM: To perfectly simulate one aspect of nature: CLOUDS• APPROACH: Validate the model generated clouds against

observations, and to use the information concerning apparent errors to improve the model physics, and subsequently the cloud simulation

Cloud observations

Cloud simulation

error parameterisation improvements

sounds easy?

Cloud Validation: The Cloud Validation: The problemsproblems• How much of the ‘error’ derives from observations?

Cloud observations error = 1

Cloud simulation error = 2

error parameterisation

improvements

Cloud Validation: The Cloud Validation: The problemsproblems• Which Physics is responsible for the error?

Cloud observations

Cloud simulation

error parameterisation

improvements

radiation

convectioncloud

physicsdynamics

turbulence

The path to improved cloud The path to improved cloud parameterisation…parameterisation…

cloud validation

parameterisation improvement

Comparison to Satellite Products

Case studies

Composite studies

NWP validation

?

Model climate - Model climate - Broadband radiative fluxesBroadband radiative fluxes

JJA 87

TSR

CY18R6-ERBE

Stratocumulus regions bad - also North Africa (old cycle!)

Can compare Top of Atmosphere (TOA) radiative fluxes with satellite observations: TOA Shortwave radiation (TSR)

Model climate - Cloud Model climate - Cloud radiative “forcing”radiative “forcing”• Problem: Can we associate these “errors” with clouds?• Another approach is to examine “cloud radiative forcing”

Note CRF sometimes defined as Fclr-F, also differences in model calculation

JJA 87

SWCRF

CY18R6-ERBE

Cloud Problems: strato-cu YES, North Africa NO!

Model climate - “Cloud Model climate - “Cloud fraction” or “Total cloud fraction” or “Total cloud cover”cover”

JJA 87

TCC

CY18R6-ISCCP

references: ISCCP Rossow and Schiffer, Bull Am Met Soc. 91, ERBE Ramanathan et al. Science 89

Can also compare other variables to derived products: CC

Climatology: The Problems Climatology: The Problems

If more complicated cloud parameters are desired (e.g. vertical structure) then retrieval can be ambiguous

Channel 1 Channel 2 …..

Liquid cloud 1Liquid cloud 2Liquid cloud 3

Ice cloud 1Ice cloud 2Ice cloud 3

Vapour 1Vapour 2Vapour 3H

eigh

t

Ass

umpt

ions

ab

out v

ertic

al

stru

ctur

es Another approach is tosimulate irradiances using model fieldsR

adia

tive

tran

sfer

m

odel

Simulating Simulating Satellite Satellite ChannelsChannels

Examples: Morcrette MWR 1991Chevallier et al, J Clim. 2001

More certainty in the diagnosis of the existence of a problem. Doesn’t necessarily help identify the origin of the problem

70°S 70°S

60°S60°S

50°S 50°S

40°S40°S

30°S 30°S

20°S20°S

10°S 10°S

0°0°

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

60°W 40°W 20°W 0° 20°E 40°E 60°E

METEOSAT 7 First Infrared Band Monday 11 April 2005 0600UTC

70°S 70°S

60°S60°S

50°S 50°S

40°S40°S

30°S 30°S

20°S20°S

10°S 10°S

0°0°

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

60°W 40°W 20°W 0° 20°E 40°E 60°E

RTTOV generated METEOSAT 7 First Inf rared Band (10 bit)Sunday 10 April 2005 12UTC ECMWF Forecast t+18 VT:Monday 11 April 2005 06UTC

A more complicated analysis is A more complicated analysis is possible:possible:

DIURNAL CYCLEOVER TROPICAL

LAND

VARIABILITY

Observations:late afternoon peak in

convectionModel:

morning peak(Common problem)

Has this Improved? 29r1, 06 Has this Improved? 29r1, 06 UTCUTC

70°S 70°S

60°S60°S

50°S 50°S

40°S40°S

30°S 30°S

20°S20°S

10°S 10°S

0°0°

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

60°W 40°W 20°W 0° 20°E 40°E 60°E

METEOSAT 7 First Infrared Band Monday 11 April 2005 0600UTC

70°S 70°S

60°S60°S

50°S 50°S

40°S40°S

30°S 30°S

20°S20°S

10°S 10°S

0°0°

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

60°W 40°W 20°W 0° 20°E 40°E 60°E

RTTOV generated METEOSAT 7 First Inf rared Band (10 bit)Sunday 10 April 2005 12UTC ECMWF Forecast t+18 VT:Monday 11 April 2005 06UTC

29r1, 12 UTC29r1, 12 UTC

70°S 70°S

60°S60°S

50°S 50°S

40°S40°S

30°S 30°S

20°S20°S

10°S 10°S

0°0°

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

60°W 40°W 20°W 0° 20°E 40°E 60°E

METEOSAT 7 First Infrared Band Monday 11 April 2005 1200UTC

70°S 70°S

60°S60°S

50°S 50°S

40°S40°S

30°S 30°S

20°S20°S

10°S 10°S

0°0°

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

60°W 40°W 20°W 0° 20°E 40°E 60°E

RTTOV generated METEOSAT 7 First Inf rared Band (10 bit)Sunday 10 April 2005 12UTC ECMWF Forecast t+24 VT:Monday 11 April 2005 12UTC

29r1, 18UTC29r1, 18UTC

70°S 70°S

60°S60°S

50°S 50°S

40°S40°S

30°S 30°S

20°S20°S

10°S 10°S

0°0°

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

60°W 40°W 20°W 0° 20°E 40°E 60°E

METEOSAT 7 First Infrared Band Monday 11 April 2005 1800UTC

70°S 70°S

60°S60°S

50°S 50°S

40°S40°S

30°S 30°S

20°S20°S

10°S 10°S

0°0°

10°N 10°N

20°N20°N

30°N 30°N

40°N40°N

50°N 50°N

60°N60°N

70°N 70°N

60°W 40°W 20°W 0° 20°E 40°E 60°E

RTTOV generated METEOSAT 7 First Inf rared Band (10 bit)Sunday 10 April 2005 12UTC ECMWF Forecast t+30 VT:Monday 11 April 2005 18UTC

NWP forecast evaluationNWP forecast evaluation• Differences in longer simulations may not be the direct result of

the cloud scheme– Interaction with radiation, dynamics etc.– E.g: poor stratocumulus regions

• Using short-term NWP or analysis restricts this and allows one to concentrate on the cloud scheme

Introduction of Tiedtke Scheme

Time

cloud cover bias

Example over EuropeExample over Europe

30°N

35°N

40°N

45°N

50°N

55°N

60°N

65°N

70°N

50°W

50°W

40°W

40°W

30°W

30°W

20°W

20°W

10°W

10°W

0°

0°

10°E

10°E

20°E

20°E

30°E

30°E 40°E

40°E

50°E

50°E

60°E

60°E

70°E70°E

N= 10761 BIAS= -0.55 STDEV= 2.61 MAE= 1.91FC PERIOD: 20050401 - 20050412 STEP: 48 VALID AT: 12 UTC

BIAS Total Cloud Cover [octa] ECM

-8 - -3 -3 - -1 -1 - 1 1 - 3 3 - 8-1:1 1:3 3:8-3:-1-8:-3

1. What are your conclusions concerning the cloud scheme?

Example over EuropeExample over Europe

30°N

35°N

40°N

45°N

50°N

55°N

60°N

65°N

70°N

50°W

50°W

40°W

40°W

30°W

30°W

20°W

20°W

10°W

10°W

0°

0°

10°E

10°E

20°E

20°E

30°E

30°E 40°E

40°E

50°E

50°E

60°E

60°E

70°E70°E

N= 10761 BIAS= -0.55 STDEV= 2.61 MAE= 1.91FC PERIOD: 20050401 - 20050412 STEP: 48 VALID AT: 12 UTC

BIAS Total Cloud Cover [octa] ECM

-8 - -3 -3 - -1 -1 - 1 1 - 3 3 - 8-1:1 1:3 3:8-3:-1-8:-3

1.Who wants to be a Meteorologist?

Which of the following is a drawback of SYNOP observations?

(a) They are only available over land (b) They are only available during daytime

(c) They do not provide information concerning vertical structure

(d) They are made by people

Case StudiesCase Studies

• These were examples of general statistics: globally or for specific regions

• Can look concentrate on a particular location in more details, for which more data is collected:CASE STUDY

• Examples: – GATE, CEPEX, TOGA-COARE, ARM...

Evaluation of vertical cloud Evaluation of vertical cloud structurestructure

Mace et al., 1998, GRL

Examined the frequency of occurrence of ice cloud

Reasonable match to data found

ARM Site - America Great Plains

Evaluation of vertical cloud Evaluation of vertical cloud structurestructure

Hogan et al., 2000, JAM

Analysis using the Chilbolton radar and

Lidar

Reasonable Match

Hogan et al. More details Hogan et al. More details possiblepossible

Hogan Hogan et al.et al.

Found that comparison improved when snow was

taken into account

Issues Raised:Issues Raised:

• WHAT ARE WE COMPARING?– Is the model statistic really equivalent to what the

instrument measures?– e.g: Radar sees snow, but the model may not include this is

the definition of cloud fraction. Small ice amounts may be invisible to the instrument but included in the model statistic

• HOW STRINGENT IS OUR TEST? – Perhaps the variable is easy to reproduce– e.g: Mid-latitude frontal clouds are strongly dynamically

forced, cloud fraction is often zero or one. Perhaps cloud fraction statistics are easy to reproduce in short term forecasts

Can also use to validate Can also use to validate “components” of cloud “components” of cloud schemescheme

EXAMPLE: Cloud Overlap AssumptionsHogan and Illingworth, 00,

QJRMS

Issues Raised: HOW REPRESENTATIVE IS OUR CASE STUDY LOCATION?e.g: Wind shear and dynamics very different between Southern England and the tropics!!!

CompositesComposites

• We want to look at a certain kind of model system: – Stratocumulus regions– Extra tropical cyclones

• An individual case may not be conclusive: Is it typical?

• On the other hand general statistics may swamp this kind of system

• Can use compositing technique

Composites - a cloud surveyComposites - a cloud survey

Tselioudis et al., 2000, JCL

From Satellite attempt to derive

cloud top pressure and cloud optical

thickness for each pixel - Data is then

divided into regimes according

to sea level pressure anomaly

Use ISCCP simulator

1. High Clouds too thin

2. Low clouds too thick

Data Model Modal-Data

Optical depth

Clo

ud to

p pr

essu

re

1.

2.

900

500

620

750

350

250

120

-ve

SLP

900

500

620

750

350

250

120

+ve

SLP

Composites – Extra-tropical Composites – Extra-tropical cyclonescyclones

Overlay about 1000 cyclones, defined about a location of maximum optical thickness

Plot predominant cloud types by looking at anomalies from 5-day average

Klein and Jakob, 1999, MWR

High tops=Red, Mid tops=Yellow, Low tops=Blue

• High Clouds too thin

• Low clouds too thick

A strategy for cloud A strategy for cloud parametrization evaluationparametrization evaluation

Jakob, Thesis

Where are the difficulties?

Recap: The problemsRecap: The problems

• All Observations– Are we comparing ‘like with like’? What assumptions are contained

in retrievals/variational approaches?

• Long term climatologies:– Which physics is responsible for the errors?– Dynamical regimes can diverge

• NWP, Reanalysis, Column Models– Doesn’t allow the interaction between physics to be represented

• Case studies– Are they representative? Do changes translate into global skill?

• Composites As case studies.

• And one more problem specific to NWP…

NWP cloud scheme developmentNWP cloud scheme development

Timescale of validation exercise– Many of the above validation exercises are complex and

involved– Often the results are available O(years) after the project

starts for a single version of the model– NWP operational models are updated 2 to 4 times a year

roughly, so often the validation results are no longer relevant, once they become available.

RequirementRequirement: A quick and easy test bench

Example: LWP ERA-40 and recent Example: LWP ERA-40 and recent cyclescycles

23r4: June 2001 26r1: April 2003

mod

elS

SM

ID

iff

Example: LWP ERA-40 and recent Example: LWP ERA-40 and recent cyclescycles

23r4: June 2001 26r3: Oct 2003

mod

elS

SM

ID

iff

Example: LWP ERA-40 and recent Example: LWP ERA-40 and recent cyclescycles

23r4: June 2001 28r1: Mar 2004

mod

elS

SM

ID

iff

Example: LWP ERA-40 and recent Example: LWP ERA-40 and recent cyclescycles

23r4: June 2001 28r3: Sept 2004

mod

elS

SM

ID

iff

Example: LWP ERA-40 and recent Example: LWP ERA-40 and recent cyclescycles

23r4: June 2001 29r1: Apr 2005

Do ERA-40 cloud studies still have relevance for the operational model?

mod

elS

SM

ID

iff

So what is used at ECMWF?So what is used at ECMWF?

• T799-L91– Standard

“Scores” (rms, anom corr of U, T, Z)

– “operational” validation of clouds against SYNOP observations

– Simulated radiances against Meteosat 7

• T159-L91 – “climate” runs– 3 ensemble members of 13 months– Automatically produces comparisons to:

• ERBE, NOAA-x, CERES TOA fluxes• Quikscat & SSM/I, 10m winds• ISCCP & MODIS cloud cover• SSM/I, TRMM liquid water path• (soon MLS ice water content)• GPCP, TRMM, SSM/I, Xie Arkin, Precip• Dasilva climatology of surface fluxes• ERA-40 analysis winds

All datasets treated as “truth”

OLR OLR

-240

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

TOA lw ellu September 2000 nmonth=12 nens=3 Global Mean: -249 50S-50N Mean: -261

[W/m2]

-300

-270

-240

-210

-180

-150

-124.0

-240

-240

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

TOA lw CERES September 2000 nmonth=12 Global Mean: -239 50S-50N Mean: -250

[W/m2]

-300

-270

-240

-210

-180

-150

-128.0

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

Difference ellu - CERES 50N-S Mean err -11.3 50N-S rms 15.8[W/m2]

10

20

20.59

-71.57

-70

-60

-50

-40

-30

-20

-10

Model T95 L91

CERES

Difference

too high

too low

Conclusions?

-150

-300

-150

-300

150

150150

350

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

TOA sw ellu September 2000 nmonth=12 nens=3 Global Mean: 244 50S-50N Mean: 277

[W/m2]

56.85

100

150

200

250

300

350

361.2

150

250

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

TOA sw CERES September 2000 nmonth=12 Global Mean: 244 50S-50N Mean: 280

[W/m2]

50

100

150

200

250

300

350

370.4

15 60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

Difference ellu - CERES 50N-S Mean err -2.86 50N-S rms 16.2[W/m2]

15

45

75

-130-115-100-85-70-55-40-25-15

SW SW

Model T95 L91

CERES

Difference

albedo high

albedo low

Conclusions?

350

100

350

100

65

65

65

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

Total Cloud Cover ellu September 2000 nmonth=12 nens=3 Global Mean: 60.5 50N-S Mean: 58.2

[percent]

5

20

35

50

65

80

93.74

65

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

Total Cloud Cover ISCCP D2 September 2000 nmonth=12 50N-S Mean: 62.2

[percent]

13.50

20

35

50

65

80

95

95.54

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

Difference ellu - ISCCP 50N-S Mean err -4.04 50N-S rms 11.3[percent]

-60

-50

-40

-30

-20

-10

10

20

30

37.88

TCC TCC

Model T95 L91

ISCCP

Difference

TCC high

TCC low

Conclusions?

80

5

80

5

316 hPa 215 hPa

Color: MLSWhite bars: EC IWC sampled with MLS tracks

First Ice Validation: microwave limb sounders

75

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

Liquid Water Path ellu September 2000 nmonth=12 nens=3 Global Mean: 67.6

[g/m**2]

25

50

75

100

125

150

171.4

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

Liquid Water Path SSMI Wentz V5 September 2000 nmonth=12 Global Mean: 82.2

[g/m**2]

25

50

75

100

125

150

175

200

225

234.1

-50

60°S60°S

30°S 30°S

0°0°

30°N 30°N

60°N60°N

135°W

135°W 90°W

90°W 45°W

45°W 0°

0° 45°E

45°E 90°E

90°E 135°E

135°E

Difference ellu - SSMI Wentz V5 Global Mean err -14.7 RMS 26.5[g/m**2]

-200

-175

-150

-125

-100

-75

-50

-25

25

50

75

100

117.8

TCLW TCLW

LWPLWP

Model T95 L91

SSMI

Difference

high

low

Conclusions?

80

5

80

5

Map of MLS ice errorMap of MLS ice error

20°N

25°N

30°N

35°N

40°N

45°N

50°N

55°N

60°N

65°N

50°W50°W

45°W

45°W

40°W

40°W

35°W

35°W

30°W

30°W

25°W

25°W

20°W

20°W

15°W

15°W

10°W

10°W

5°W

5°W

0°

0°

5°E

5°E

10°E

10°E

15°E

15°E

20°E

20°E

25°E

25°E

30°E

30°E35°E

35°E

40°E

40°E

45°E

45°E

50°E

50°E

55°E

55°E

60°E

60°E

65°E

65°E

METEOSAT 7 First Infrared Band Monday 11 April 2005 1200UTC

20°N

25°N

30°N

35°N

40°N

45°N

50°N

55°N

60°N

65°N

50°W50°W

45°W

45°W

40°W

40°W

35°W

35°W

30°W

30°W

25°W

25°W

20°W

20°W

15°W

15°W

10°W

10°W

5°W

5°W

0°

0°

5°E

5°E

10°E

10°E

15°E

15°E

20°E

20°E

25°E

25°E

30°E

30°E35°E

35°E

40°E

40°E

45°E

45°E

50°E

50°E

55°E

55°E

60°E

60°E

65°E

65°E

RTTOV generated METEOSAT 7 First Inf rared Band (10 bit)Sunday 10 April 2005 12UTC ECMWF Forecast t+24 VT:Monday 11 April 2005 12UTC

Daily Report 11Daily Report 11thth April 2005 April 2005Lets see what you think? Lets see what you think?

“Going more into details of the cyclone, it can be seen that the model was able to reproduce the very peculiar spiral structure in the clouds bands. However large differences can be noticed further east, in the warm sector of the frontal system attached to the

cyclone, were the model largely underpredicts the typical high-cloud shield. Look for example in the two maps below where a clear deficiency of clod cover is evident in the model generated satellite images north of the Black Sea. In this case this was systematic

over different forecasts.” – Quote from ECMWF daily report 11th April 2005

Same Case, water vapour Same Case, water vapour channelschannels

METEOSAT 7 Water Vapour Band Monday 11 April 2005 2000UTC RTTOV generated METEOSAT 7 Water Vapour Band (10 bit)Sunday 10 April 2005 12UTC ECMWF Forecast t+30 VT:Monday 11 April 2005 18UTC

Blue: moistRed: Dry

30 hr forecast too dry in front regionIs not a FC-drift, does this mean the cloud scheme is at fault?

Future: Long term ground-based Future: Long term ground-based validation, CLOUDNETvalidation, CLOUDNET

• Network of stations processed for multi-year period using identical algorithms, first Europe, now also ARM sites

• Some European provide operational forecasts so that direct comparisons are made quasi-realtime

• Direct involvement of Met services to provide up-to-date information on model cycles

Cloudnet ExampleCloudnet Example

• In addition to standard quicklooks, longer-term statistics are available

• This example is for ECMWF cloud cover during June 2005

• Includes preprocessing to account for radar attenuation and snow

• See www.cloud-net.org for more details and examples!

Future: Improved remote sensing Future: Improved remote sensing capabilities, CLOUDSATcapabilities, CLOUDSAT

• CloudSat is an experimental satellite that will use radar to study clouds and precipitation from space. CloudSat will fly in orbital formation as part of the A-Train constellation of satellites (Aqua, CloudSat, CALIPSO, PARASOL, and Aura)

• Launched 28th April

CloudsatCloudsattransect

Zonal mean cloud cover

A chasm still exists!!!

In summary: Many methods In summary: Many methods for examining cloudsfor examining clouds

parameterisation improvement

cloud validation

but all too often...