Stochastic representations of model uncertainties in the IFS

Martin Leutbecher, Pirkka Ollinaho, Sarah-Jane Lock, Simon Lang,Peter Bechtold, Anton Beljaars, Alessio Bozzo, Richard Forbes,

Thomas Haiden, Robin Hogan and Irina Sandu

ECMWF/WWRP workshop April 2016

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 1

Model uncertainty representations in the IFS

• Operational in medium/extended-range and seasonal ensembles (ENS, SEAS)• SPPT (Stochastically Perturbed Parametrisation Tendencies) with 3-scales• SKEB (Stochastic Kinetic Energy Backscatter)

• Operational in ensemble of data assimilations (EDA)• SPPT with 1-scale (fast & small-scale)

• Research• modifications of SPPT (cf. talks by Weisheimer and Christensen)• Development of a new scheme “SPP”: Stochastically Perturbed Parameterisations• ENS with SPP versus ENS with SPPT• EDA with SPP• EDA with 3-scale SPPT

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 2

Model uncertainty representations in the IFS

• Operational in medium/extended-range and seasonal ensembles (ENS, SEAS)• SPPT (Stochastically Perturbed Parametrisation Tendencies) with 3-scales• SKEB (Stochastic Kinetic Energy Backscatter)

• Operational in ensemble of data assimilations (EDA)• SPPT with 1-scale (fast & small-scale)

• Research• modifications of SPPT (cf. talks by Weisheimer and Christensen)• Development of a new scheme “SPP”: Stochastically Perturbed Parameterisations• ENS with SPP versus ENS with SPPT• EDA with SPP• EDA with 3-scale SPPT

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 2

Stochastic Kinetic Energy Backscatter (SKEB)

• Rationale: A fraction of the dissipated energy is backscattered upscale and acts asstreamfunction forcing for the resolved-scale flow (Shutts and Palmer 2004,Shutts 2005, Berner et al. 2009)

• Streamfunction forcing = [bD]1/2 F (λ, φ, σ, t),where b,D,F denote the backscatter ratio, the (smoothed) total dissipation rateand the 3-dim evolving pattern, respectively

• Total dissipation rate: sum of• “Numerical” dissipation: Loss of KE by numerical diffusion + interpolation in

semi-Lagrangian advection; estimated from biharmonic diffusion• an estimate of the deep convective KE production

• Resolution upgrade (32→ 19 km) in March 2016:Spectral viscosity approach for cubic octahedral grid is inconsist with biharmonicdiffusion assumed by SKEB and used previously with the linear grid ( ⇒contribution from numerical dissipation deactivated, i.e. SKEB → SCB ⇒ SKEBis less active )

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 3

Stochastic Kinetic Energy Backscatter (SKEB)

• Rationale: A fraction of the dissipated energy is backscattered upscale and acts asstreamfunction forcing for the resolved-scale flow (Shutts and Palmer 2004,Shutts 2005, Berner et al. 2009)

• Streamfunction forcing = [bD]1/2 F (λ, φ, σ, t),where b,D,F denote the backscatter ratio, the (smoothed) total dissipation rateand the 3-dim evolving pattern, respectively

• Total dissipation rate: sum of• “Numerical” dissipation: Loss of KE by numerical diffusion + interpolation in

semi-Lagrangian advection; estimated from biharmonic diffusion• an estimate of the deep convective KE production

• Resolution upgrade (32→ 19 km) in March 2016:Spectral viscosity approach for cubic octahedral grid is inconsist with biharmonicdiffusion assumed by SKEB and used previously with the linear grid ( ⇒contribution from numerical dissipation deactivated, i.e. SKEB → SCB ⇒ SKEBis less active )

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 3

Stochastically Perturbed Parameterization Tendencies(SPPT)

• Total physics tendencies P perturbed by ∆P = µrP, with r a random pattern andµ a tapering profile (0 in BL and stratosphere, 1 in free troposphere)

• Improved version of the original SPPT scheme (stochastic physics, Buizza, Miller& Palmer, 1999)

• Random pattern r(lat, lon, t) uses AR-1 processes in spectral space and is“continuous” in space and time

• Multi-scale pattern with three components:τ =6h, 3 d, 30 d and L =500 km, 1000 km, 2000 km with standard deviations ofσ =0.52, 0.18, 0.06, respectively

• Gaussian distribution (limited to range [−1, 1] )• Same pattern r for T , q, u, v

• see Tech Memo 598, Palmer et al. (2009) and Shutts et al (2011), ECMWFNewsletter 129

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 4

Stochastically Perturbed Parameterization Tendencies(SPPT)

• Total physics tendencies P perturbed by ∆P = µrP, with r a random pattern andµ a tapering profile (0 in BL and stratosphere, 1 in free troposphere)

• Improved version of the original SPPT scheme (stochastic physics, Buizza, Miller& Palmer, 1999)

• Random pattern r(lat, lon, t) uses AR-1 processes in spectral space and is“continuous” in space and time

• Multi-scale pattern with three components:τ =6h, 3 d, 30 d and L =500 km, 1000 km, 2000 km with standard deviations ofσ =0.52, 0.18, 0.06, respectively

• Gaussian distribution (limited to range [−1, 1] )• Same pattern r for T , q, u, v

• see Tech Memo 598, Palmer et al. (2009) and Shutts et al (2011), ECMWFNewsletter 129

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 4

SPPT patterncomposed of 3 random fields

stdev=0.52 0.18 0.06

5

Multi-scale SPPT

500 km6 h

1000 km3 d

2000 km30 d

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 5

What happens without representation of modeluncertainties?

Ensemble standard deviation

0

2

4

6

8

10

12

14

RM

S

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

spread_em

u200hPa, Northern Extra-tropics

IP only

SPPT

SPPT+SKEB

0

2

4

6

8

RM

S

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

spread_em

u200hPa, Tropics

IP only

SPPT

SPPT+SKEB

TL399/255, resolution change at D15, 20 members

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 6

What happens without representation of modeluncertainties?Probabilistic skill (I)

0

2

4

6

8

CR

PS

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

ContinuousRankedProbabilityScore

u200hPa, Northern Extra-tropics

IP only

SPPT

SPPT+SKEB

0

1

2

3

4

5

CR

PS

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

ContinuousRankedProbabilityScore

u200hPa, Tropics

IP only

SPPT

SPPT+SKEB

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 7

What happens without representation of modeluncertainties?Probabilistic skill (II)

2

3

4

DS

2M

OM

EN

TS

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

ContinuousIgnoranceScoreGaussianNN

u200hPa, Northern Extra-tropics

IP only

SPPT

SPPT+SKEB 3

4

DS

2M

OM

EN

TS

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

ContinuousIgnoranceScoreGaussianNN

u200hPa, Tropics

IP only

SPPT

SPPT+SKEB

Proper two-moment score of Dawid and Sebastiani (1999) ≡ log-score of Gaussian distribution withthe two moments given by ensemble mean and ensemble variance

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 8

Planned and potential upgrades of SPPT

• fix global integral of perturbed tendency to the value of the unperturbed tendencyto address lack of conservation (Antje Weisheimer, Simon Lang and Jost vonHardenberg)

• independent patterns for different processes / groups of processes (iSPPT:Hannah Christensen and Sarah-Jane Lock)

• re-assessment of supersaturation limiter (Sarah-Jane Lock)

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 9

Towards process-level specification of uncertaintiesAim: Improve physical consistency of model uncertainty representation

Subgrid-scaleorographic drag

Turbulent diffusion

Deepconvection

Cloud

Shallowconvection

Long-waveflux

Short-waveflux

Latentheatflux

Non-orographicwave drag

Sensibleheat flux

Short-waveradiation

Cloud

O3 Chemistry

Wind Waves

CH4 OxidationLong-waveradiation

Ocean model

Surface

IFS documentation

• Flux perturbations at TOAand sfc that are consistentwith tendency perturbation inatmospheric column

• Conservation of water

• No ad hoc tapering in BL andstratosphere

• Include multi-variate aspectsof uncertainties

• Embed stochasticity within IFS physics

• Local stochastic perturbations toparameters and variables with specifiedspatial and temporal correlations

• Target uncertainties that matter

• Stochastic parameterisation convergesto deterministic IFS physics in limit ofvanishing variance

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 10

Towards process-level specification of uncertaintiesAim: Improve physical consistency of model uncertainty representation

Subgrid-scaleorographic drag

Turbulent diffusion

Deepconvection

Cloud

Shallowconvection

Long-waveflux

Short-waveflux

Latentheatflux

Non-orographicwave drag

Sensibleheat flux

Short-waveradiation

Cloud

O3 Chemistry

Wind Waves

CH4 OxidationLong-waveradiation

Ocean model

Surface

IFS documentation

• Flux perturbations at TOAand sfc that are consistentwith tendency perturbation inatmospheric column

• Conservation of water

• No ad hoc tapering in BL andstratosphere

• Include multi-variate aspectsof uncertainties

• Embed stochasticity within IFS physics

• Local stochastic perturbations toparameters and variables with specifiedspatial and temporal correlations

• Target uncertainties that matter

• Stochastic parameterisation convergesto deterministic IFS physics in limit ofvanishing variance

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 10

Stochastically Perturbed Parametrisations (SPP)The distributions sampled by SPP

Stochastic parameterisation samplesparameter ξj by a log-normaldistribution

ξj = ξ̂j exp (Ψj)

with unperturbed parameter ξ̂j

Ψj ∼ N (µj , σ2j )

µ = 0⇒ median of ξj is equal to ξ̂j0 1 2 3 4 5

ξj/ξ̂j

0.0

0.2

0.4

0.6

0.8

1.0

1.2

pd

f

cloud vert. decorrel. height (McICA)

fractional stdev. hor. distr. water content

effective radius of cloud water /ice

Development started with parameter perturbationsthat target the cloud-radiation interaction

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 11

The distributions sampled by SPPturbulent diffusion and subgrid oro. cloud and large-scale precipitation

0 1 2 3 4 5

ξj/ξ̂j

0.0

0.5

1.0

1.5

2.0

2.5

pd

f

transfer coeff. momentum (oc)

transfer coeff. momentum (lnd)

turbulent orogr. form drag

stdev subgrid orography

vert. mixing length scale, stable BL

0 1 2 3 4 5

ξj/ξ̂j

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

pd

f

RH threshold stratiform condensation

diff. coeff. for evaporation of turb. mixing

crit. cloud water content

threshold for snow autoconversion

radiation convection

0 1 2 3 4 5

ξj/ξ̂j

0.0

0.2

0.4

0.6

0.8

1.0

1.2

pd

f

cloud vert. decorrel. height (McICA)

fractional stdev. hor. distr. water content

effective radius of cloud water /ice

aerosol vert. distrib. scale height

aerosol optical thickness

0 1 2 3 4 5

ξj/ξ̂j

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

pd

f

entrainment rate

shallow entr. rate

detrainment rate

conversion cloud-rain

adjustment time scale

conv. momentum transport

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 12

Correlation scales matterEnsemble standard deviation and CRPS

0

2

4

6

RM

S

0 3 6 9 12 15fc-step (d)

2013120400-2014112100 (23)

spread_em

u200hPa, Tropics

IP_only

PPL

PPInf

PPM

0

1

2

3

4

CR

PS

0 3 6 9 12 15fc-step (d)

2013120400-2014112100 (23)

ContinuousRankedProbabilityScore

u200hPa, Tropics

IP_only

PPL

PPInf

PPM

Exp L (km) τ (h)

PPM 500 6PPL 2000 72

PPInf ∞ ∞

horizontal correlation length scale Lcorrelation time τfor pattern ΨOllinaho et al. (2016, submitted to QJ)

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 13

Amplitude matters tooEnsemble standard deviation and CRPS

0

2

4

6

RM

S

0 3 6 9 12 15fc-step (d)

2013120400-2014112100 (23)

spread_em

u200hPa, Tropics

IP_only

PPL

PPL0.5

0

1

2

3

4

CR

PS

0 3 6 9 12 15fc-step (d)

2013120400-2014112100 (23)

ContinuousRankedProbabilityScore

u200hPa, Tropics

IP_only

PPL

PPL0.5

PPL0.5 has all standard deviations σj halved compared to PPL.The correlation scales are 2000 km and 72 h in both experiments.

Ollinaho et al. (2016, submitted to QJ)

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 14

Intercomparison of SPP and SPPTEnsemble stdev of 0–3 h temperature tendencies (K[3 h]−1)

SPPT SPP

no initialpertns.TL255,

20 member

← level 64∼ 500 hPa

← level 9110m above

surface

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 15

Comparing ensembles of temperature tendenciesSPP versus SPPT (no initial perturbations)

0.0 0.9 1.8 2.7

20

40

60

80

STD

EV

TR

0.0 0.9 1.8

20

40

60

80

NH

0-3h SPPT 0-3h SPP

0.0 0.5 1.0

20

40

60

80

CO

RR

STD

EV

0.0 0.5 1.0

20

40

60

80

• Stdev of 0–3 h tendency

• SPP induces larger (smaller)tendency perturbations in(above) BL than SPPT

• regions where tendencies aremost uncertain become moresimilar with increasing leadtime (bottom: corr stdev)

• 6 boreal winter cases; Unit(top panels): K/(3 h)

Ollinaho et al. (2016,submitted to QJ)

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 16

Comparing ensembles of temperature tendenciesSPP versus SPPT (no initial perturbations)

0.0 0.9 1.8 2.7

20

40

60

80

STD

EV

TR

0.0 0.9 1.8

20

40

60

80

NH

21-24h SPPT 21-24h SPP

0.0 0.5 1.0

20

40

60

80

CO

RR

STD

EV

0.0 0.5 1.0

20

40

60

80

• Stdev of 21–24 h tendency

• SPP induces larger (smaller)tendency perturbations in(above) BL than SPPT

• regions where tendencies aremost uncertain become moresimilar with increasing leadtime (bottom: corr stdev)

• 6 boreal winter cases; Unit(top panels): K/(3 h)

Ollinaho et al. (2016,submitted to QJ)

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 16

Change of ensemble stdev: 200 hPa zonal windSPP versus SPPT relative to initial perturbation only

-0.1

0.1

0.3

0.5

0.7

0.9

RM

S

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

spread_em [sign p=0.0500]

u200hPa, Northern Extra-tropics

IP only

SPPT

SPP

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

RM

S

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

spread_em [sign p=0.0500]

u200hPa, Tropics

IP only

SPPT

SPP

TCo255/159, resolution change at D15,20 member

Exp L (km) τ (h)

SPP 2000 72SPPT 3-scale 3-scale

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 17

Change of CRPS: 200 hPa zonal windSPP versus SPPT relative to initial perturbation only

-0.2

-0.1

0

0.1

CR

PS

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

fairContinuousRankedProbabilityScore [sign p=0.0500]

u200hPa, Northern Extra-tropics

IP only

SPPT

SPP

-0.4

-0.3

-0.2

-0.1

0

0.1

CR

PS

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

fairContinuousRankedProbabilityScore [sign p=0.0500]

u200hPa, Tropics

IP only

SPPT

SPP

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 18

Change of CRPS: 200 hPa zonal windSPP+SPPT1 and . . . relative to initial perturbation only

-0.2

-0.1

0

0.1

CR

PS

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

fairContinuousRankedProbabilityScore [sign p=0.0500]

u200hPa, Northern Extra-tropics

IP only

SPPT

SPP

SPPT1+SPP

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

CR

PS

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

fairContinuousRankedProbabilityScore [sign p=0.0500]

u200hPa, Tropics

IP only

SPPT

SPP

SPPT1+SPP

Exp L (km) τ (h)

SPP 2000 72SPPT1 500 6

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 19

Impact on model climate

W20

0

W70

0

W92

515

10

5

0

5

10

15

RM

SE C

HA

NG

E (

%)

TR WINDS

TP(1

)

TP(2

)

PRECIPITATION

TCC

CLOUD COVER

TCW

V(1)

TCW

V(2)

TCWV

TTR

TSR

RADIATION

SPPSPPT

Relative change of RMSerror of annual meanfields with respect tounperturbed forecasts.

ERA-Interim (tropicalwinds) and satellite obs.are used as reference.

Model configuration:uncoupled TL255, 4 startdates, initial monthomitted.

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 20

EDA first guess (3 h) stdev: Meridional wind (m s−1)

90°S60°S30°S0°N30°N60°N90°N

20

40

60

80

100

120

-0.5 -0.42 -0.33 -0.25 -0.17 -0.08 -0.01 0.01 0.08 0.17 0.25 0.33 0.42 0.5

SPPT − noTenPert

90°S60°S30°S0°N30°N60°N90°N

20

40

60

80

100

120

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.5

noTenPert

SPPT:oper.1-scale

SPPT3 andSPP

propagaterandom

fields acrosscycles

90°S60°S30°S0°N30°N60°N90°N

20

40

60

80

100

120

-0.5 -0.42 -0.33 -0.25 -0.17 -0.08 -0.01 0.01 0.08 0.17 0.25 0.33 0.42 0.5

SPPT3 − noTenPert

90°S60°S30°S0°N30°N60°N90°N

20

40

60

80

100

120

-0.5 -0.42 -0.33 -0.25 -0.17 -0.08 -0.01 0.01 0.08 0.17 0.25 0.33 0.42 0.5

SPP − noTenPert

mean over16 12-hourassimilationcycles inJune 2015

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 21

Summary

• The operational schemes SPPT (+SKEB) contribute significantly to theprobabilistic skill in medium range and extended range

• A new stochastic scheme has been developed for representing model uncertaintiesat the process level in IFS: The SPP scheme provides a framework to buildstochastic parameterisations that are guided by existing deterministicparameterisations

• A first attempt to represent model uncertainties in the main physical processes in aphysically consistent way.

• Further extensions of SPP are envisaged and ideas are welcome

• Proximity to processes implies a scheme that is less parsimonious than SPPT.

• Further development could benefit from validating the ensemble for variables thatare close to the processes.

• Initial operational implementation could consider a combination of SPPT and SPP

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 22

Current ECMWF ideas on future plansFor working group discussions

• Move towards consistent approach to represent model uncertainties inassimilations and forecasts at all lead times

• Process-oriented diagnostics of ensembles(e.g. radiative fluxes, precipitation, skin temperature)

• Ideas for extending scope of SPP:• thermodynamic coupling between surface and atmosphere• vertical mixing above boundary layer• atmospheric composition: trace gas sources/sinks

• How to evolve from current SKEB?Stochastic Convective Backscatter (SCB, Shutts 2015)and/or stochastic dynamical core

Vacancy VN16-13 at ECMWF for Scientist to work on specification of global surfacecharacteristics for land, ocean, biosphere and cryosphere and model uncertainty, seehttp://www.ecmwf.int/en/about/jobs/jobs-ecmwf

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 23

Current ECMWF ideas on future plansFor working group discussions

• Move towards consistent approach to represent model uncertainties inassimilations and forecasts at all lead times

• Process-oriented diagnostics of ensembles(e.g. radiative fluxes, precipitation, skin temperature)

• Ideas for extending scope of SPP:• thermodynamic coupling between surface and atmosphere• vertical mixing above boundary layer• atmospheric composition: trace gas sources/sinks

• How to evolve from current SKEB?Stochastic Convective Backscatter (SCB, Shutts 2015)and/or stochastic dynamical core

Vacancy VN16-13 at ECMWF for Scientist to work on specification of global surfacecharacteristics for land, ocean, biosphere and cryosphere and model uncertainty, seehttp://www.ecmwf.int/en/about/jobs/jobs-ecmwf

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 23

extra slides . . .

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 24

What happens without representation of modeluncertainties?

Ensemble standard deviation

0

2

4

6

8

10

12

RM

S

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

spread_em

u200hPa, Northern Extra-tropics

IP only

SPPT

SPPT+SKEB

0

2

4

6

8

RM

S

0 5 10 15 20 25 30fc-step (d)

2013120100-2014112600 (46)

spread_em

u200hPa, Tropics

IP only

SPPT

SPPT+SKEB

TCo255/159, resolution change at D15, 20 members

M. Leutbecher et al. Stochastic representations of model uncertainties in the IFS ECMWF/WWRP workshop April 2016 25