Theme D: Model Processes, Errors and Inadequacies

Mat Collins, College of Engineering, Mathematics and Physical Sciences, University of Exeter and Met Office Hadley Centre

© Crown copyright Met Office

A Much Used Quote

• “All models are wrong, some are useful” - George Box

• Easy to test when a model is wrong, much harder to say when it is useful

climate models

Uncertainties and Errors in Climate Models

• Uncertainties in parameters, sampled using Perturbed Physics Ensembles (PPEs)

• “Structural” uncertainties, at least partially sampled by Multi-Model Ensembles (MMEs)

• Coding errors!• Errors common to all models (examples follow)• Missing processes

Why do Errors Matter

• Predictions and projections

• Detection & attribution

• Understanding climate and climate change

• Risk assessment for natural events (cat modelling)

• …

Types of Errors & Inadequacies

Incapabilities

Iconic Error 1: Blocking

© Crown copyright Met OfficeTim Hinton, Gill Martin

Figure 8.5

Iconic Error 2: Double ITCZ

Source: IPCC Fourth Assessment Report

Figure 8.5

observations

models - MME

• Boé J, Hall A, Qu X (2009), September Sea-Ice Cover in the Arctic Ocean Projected to Vanish by 2100, Nature Geosci, 2: 341-343

• Hall A, Qu X (2006) Using the current seasonal cycle to constrain snow albedo feedback in future climate change. Geophys. Res. Lett., 33, L03502

Errors known to be important for e.g. climate projections

Poorly Observed but Important Variables

Figure 10.15

Source: IPCC Fourth Assessment Report

Which model is right?

Known Missing Processes• Land use change (in HadCM3Q)• Soot/black carbon (in HadCM3Q)• …• Dynamic ice sheets• Methane hydrate release• Stratosphere• …

Unknown Missing Processes

Dealing with Model Errors and Inadequacies

Dealing with Model Errors and Inadequacies

• Do nothing• Do nothing but at least discuss the implications of

doing nothing• Embark on an ambitious programme of metrics

and intercomparisons• Use a discrepancy term• …• (Continue to improve models, of course)

Metrics, metrics, metricsMME Error Characteristics

Reichler and Kim, 2008

Relative model errors

HadGEM Traffic Lights

© Crown copyright Met Office

Introduce a Discrepancy e.g. Rougier (2007)

y = {yh,yf} climate variables (vector)

f = Climate model e.g. HadCM3

x* = best point in HadCM3 parameter space – for observable and non-observable fields

d = discrepancy – irreducible/”structural” model error (vector)

How to determine d?

d)f(xy *

Estimating Discrepancy in UKCP09 (David Sexton Talk)

• Use the multi-model ensemble from IPCC AR4 (CMIP3) and CFMIP (models from different centres)

• For each multi-model ensemble member, find point in HadCM3 parameter space that is closest to that member

• There is a distance between climates of this multi-model ensemble member and this point in parameter space i.e. effect of processes not explored by perturbed physics ensemble

• Pool these distances over all multi-model ensemble members

• Uses model data from the past and the future

© Crown copyright Met Office

Scottish Snow

• Clear sky SW TOA flux over HadCM3 Scotland grid point

• Most extreme discrepancy found

© Crown copyright Met Office

Theme D: Questions/Topics for Discussion

• Develop language/classification for errors and inadequacies

• Explore ways of specifying discrepancy• Explore alternative strategies for dealing with

errors and inadequacies• Any more?

Blocking

The frequency of blocking events in the perturbed physics HadCM3 ensemble (PPE_A1B, red lines) for winter (DJF, top) and summer (JJA, bottom) together with that estimated from ERA40 (thick black lines). The blocking index is calculated following Pelly and Hoskins (2003) and uses a variable latitude to track the location of the model storm track (in contrast to other indices which used a fixed latitude).

Murphy et al. 2009 UKCP09

Types of Errors and Inadequacies

• Incapabilities – e.g. due to resolution• Known errors/biases in simulating mean climate and

variability, leading to metrics, iconic errors• Errors which are known to be important for

projection, D&A, …• Potential errors in poorly observed variables• Known missing processes• Unknown missing processes

Systematic Errors in All Models

Collins et al. 2010