Why forecasts fail: ensemble prediction, early warnings and the challenges of emergency management...

Why forecasts fail: ensemble prediction, early warnings and the

challenges of emergency management

Sébastien Nobert

Centre National de la Recherche Scientifique (CNRS),

INSHS, SET UMR-5603

WWOSC 2014, Montréal

18 August 2014

www.cnrs.frwww.cnrs.fr

Outline of Talk Today Outline of Talk Today 1. Communicating risk and uncertainty;

2. Research and Understanding EPS;

1. Breakdowns and the moments of i) Reception; ii) Attention; iii) Acceptance; iv) Prioritisation and v) Mobilisation;

1. Conclusions.


1.Communicating risk and uncertainty• Risk communication: often promoted as a tool for fixing the apparent

deficit in public understanding, or for achieving other ends, like encouraging behaviour change, deflecting blame, assuaging public outrage, or securing trust.

• Questions of what should be communicated?

• BUT…also how this information should be transmitted, why

and by whom?

• Several models for communicating risks other than deficit model whereby it is assumed that risk communication is simply a one-way process of transmitting factual, scientific knowledge to the ignorant.


Demeritt, D. and Nobert, S. (2014) Models of best practice in flood risk communication and management, Environmental Hazards, http://dx.doi.org/10.1080/17477891.2014.924897

Understanding forecast (mis)communication and response

1-Work in psychology and decision science emphasises the cognitive challenges to effective risk communication and the difficulties of conveying complicated forecast products to non-experts in ways they can understand and thus act upon appropriately.

2- Institutionalist approaches to political science suggest that even if forecasts are received and understood, appropriate responses to them may be inhibited by external constraints, such as resource limitations or political authorisation and mandate, or internal organisational factors.

How these approaches inform us about the break-downs in responding to pre-warnings?


2. 2. Background of research and Understanding EPS

• 72 interviews with operational forecasters and civil protection officials in 17 European countries involved in flood forecasting, warning and emergency response;

• Participant observation at forecasting workshops &

conferences and during visits to forecasting centres;

• All interview quotations are anonymised;

“what we call our data are really our constructions of other people’s constructions of what they are up to” (Geertz 1973: 9)


EFAS as an example of risk EFAS as an example of risk communication?communication?

European Centre for

Medium Range Weather

ForecastsEFASEFAS

The public

Civil Protection Agencies

Uses ECMWF ensemble

Produces 3-15 day real time ensemble flood forecasts

• EFAS: both user and producer of ensemble forecasts;

• Nicely illustrates the challenges of communicating and using ensembles for operational decision making;

• Potential lessons for you about how such forecasts/science are received and understood.

Provided to cooperating flood forecasting agencies across Europe

National level agencies issue official flood warnings to:


Modelling break-downs in the response to early-warnings


Meyer et al. (2011) outlined five steps at which breakdowns in the transmission, understanding, and response to a warning may occur:

Reception: a signal may be screened out by operational processing routines such that it fails to reach a recipient with sufficient authority to act; Attention: new information may not attract the notice of recipients distracted by other information sources or agenda overload due to other competing concerns;

Acceptance: recipients may reject the issue framing of a warning or dismiss its specific predictions about the probability or consequences of an event as insufficiently credible;

Prioritisation: even if a prediction is received and believed by its recipients, they may not be convinced that the issue at hand is sufficiently pressing given other demands on resources;

Mobilisation: resource constraints or other exogenous factors, such as political opposition, may prevent decision-makers mobilising action in response to warnings they find credible and pressing.

i-ii) Moment of Reception and Attention “I realized that we almost forgot to show

our use of the probabilistic tool designed by [the person from EFAS] attending our meeting but the most embarrassing thing was that nobody else in the room could remember the password to access it either [laughs].” (Flood forecaster, France)

“I’m receiving the national forecasts, I can look at ECMWF and make my own opinion of what is coming, I rarely pay attention to other probabilisitic models, simply because I am used to looking at ECMWF output in relation to my local models” (Flood forecaster, Czech Republic).


iii) Acceptance and the challenges of iii) Acceptance and the challenges of crediting EPScrediting EPS

“So, each forecaster has in his brain what we could call a personal climatology and this is what is valorised and that is always working. It is this personal climatology, with its conceptual models and experiences that is confronted to the probabilities of ensemble prediction…this is what we have to deal with when taking a decision (Weather forecaster, France).


iv) Prioritization and the challenges of error and blame

“Tuesday 5th of August: prediction for the weekend were 90mm/m² with very high uncertainty. The weather experts told us to wait till Friday for warning. Friday 6th of August: prediction for weekend were 30-50mm/m²so we gave a warning for a moderate local flood danger. However, Saturday 7th of August we had a 100 year event on a local river after heavy rainfall reached 80-120mm/m“. (CPA, Austria).

“There is also a fear to circulate probabilistic information for the risk of false alarms and the lost of trust in the General Inspectorate of Emergency Situations as well as in the governmental risk management. It is the story we have here in Romania about the little boy crying wolf, if people are given wrong information it will be difficult to convince them to evacuate.” (CPA, Romania


v) Mobilization and the challenge of political authorisation


“We can’t afford a system of pre-warnings. By law, we need to send the information of evacuation between 120 to 60 minutes in advance and this means that local Fire Rescue Service are actioning their evacuation plans” (CPA, Czech Republic).

“this stuff is increasingly applied throughout the scientific world but we are aware of the fact that when it comes to formal legislation, especially when we talk about restrictions and constraints through administrative interventions, we face an uphill battle.” (CPA, Germany)

“It does not operate on uncertainty”

View from ECMWF computer room, looking at IBM super computers, Reading, UK

Despite the work done in forecasting science to improve modelling capacity and measurement of uncertainty, CPAs and local authorities take their decisions according to a binary scheme in which uncertainty has little if no role in their final decision.

"I saw some of these news feeds that I've been watching upstairs of people sitting on the beach at Asbury Park…Get the hell off the beach at Asbury Park and get out. You're done. It's 4:30, you've maximized your tan. Get off the beach. Get in your cars and get out of those areas.“ (Chris Cristie, Governor of New-Jersey, 26th of August, before Hurrican Irene).


6. Conclusions1-Civil protection authorities do see spread or probabilities as interesting information but the novelty of EPs in flood forecasting and management makes operational actors often judge them too complicated to be communicated to civil protection judge them too complicated to be communicated to civil protection authoritiesauthorities.

2-Although the content of probabilistic tools are tailored to the needs of forecasters, experience of weather events comes to displace the relevance of computer modelling information such as EPs. This, in turn, influences the information used by forecasters in their operations and what is provided to CPAs to build their mobilisation and action protocols.

3-Receiving information with a high degree of uncertainty might only restrain CPAs of acting instead of activating mitigation strategies acting instead of activating mitigation strategies and thus increase their chance of missing an event.

4-Problem of binary decision makingProblem of binary decision making.


Why forecasts fail: ensemble prediction, early warnings and the challenges of emergency management...

Documents

Transcript of Why forecasts fail: ensemble prediction, early warnings and the challenges of emergency management...