The precautionary principle fragility and black swans from policy actions

EXTREME RISK INITIATIVE —NYU SCHOOL OF ENGINEERING WORKING PAPER SERIES

The Precautionary Principle: Fragility and BlackSwans from Policy Actions

Nassim Nicholas Taleb⇤, Yaneer Bar-Yam†, Raphael Douady‡, Joseph Norman†, Rupert Read§

⇤School of Engineering, New York University †New England Complex Systems Institute‡ Institute of Mathematics and Theoretical Physics, C.N.R.S., Paris

§School of Philosophy, University of East Anglia

F

Abstract—The precautionary principle (PP) states that if an actionor policy has a suspected risk of causing severe harm to the publicdomain (affecting general health or the environment globally), the actionshould not be taken in the absence of scientific near-certainty about itssafety. Under these conditions, the burden of proof about absence ofharm falls on those proposing an action, not those opposing it. PP isintended to deal with uncertainty and risk in cases where the absenceof evidence and the incompleteness of scientific knowledge carriesprofound implications and in the presence of risks of "black swans",unforeseen and unforeseable events of extreme consequence. Here weformalize PP, placing it within the statistical and probabilistic structureof “ruin” problems, in which a system is at risk of total failure, andin place of risk we use a formal"fragility" based approach. In theseproblems, what appear to be small and reasonable risks accumulateinevitably to certain irreversible harm. Traditional cost-benefit analyses,which seek to quantitatively weigh outcomes to determine the bestpolicy option, do not apply, as outcomes may have infinite costs. Evenhigh-benefit, high-probability outcomes do not outweigh the existenceof low probability, infinite cost options—i.e. ruin. Uncertainties resultin sensitivity analyses that are not mathematically well behaved. ThePP is increasingly relevant due to man-made dependencies thatpropagate impacts of policies across the globe. In contrast, absenthumanity the biosphere engages in natural experiments due to randomvariations with only local impacts. Our analysis makes clear that thePP is essential for a limited set of contexts and can be used to justifya limited set of actions. We discuss the implications for nuclear energyand GMOs. GMOs represent a public risk of global harm, while harmfrom nuclear energy is comparatively limited and better characterized.PP should be used to prescribe severe limits on GMOs.

August 17, 2014

1 INTRODUCTION

THe aim of the precautionary principle (PP) is toprevent decision makers from putting society as a

whole—or a significant segment of it—at risk from theunexpected side effects of a certain type of decision. ThePP states that if an action or policy has a suspected riskof causing severe harm to the public domain (such asgeneral health or the environment), and in the absenceof scientific near-certainty about the safety of the action,

Corresponding author: N N Taleb, email [email protected]

the burden of proof about absence of harm falls on thoseproposing the action. It is meant to deal with effects ofabsence of evidence and the incompleteness of scientificknowledge in some risky domains.1

We believe that the PP should be evoked only inextreme situations: when the potential harm is systemic(rather than localized) and the consequences can involvetotal irreversible ruin, such as the extinction of humanbeings or all life on the planet.

The aim of this paper is to place the concept ofprecaution within a formal statistical and risk-analysisstructure, grounding it in probability theory and theproperties of complex systems. Our aim is to allowdecision makers to discern which circumstances requirethe use of the PP and in which cases evoking the PP isinappropriate.

2 DECISION MAKING AND TYPES OF RISK

Taking risks is necessary for individuals as well as for de-cision makers affecting the functioning and advancementof society. Decision and policy makers tend to assume allrisks are created equal. This is not the case. Taking intoaccount the structure of randomness in a given systemcan have a dramatic effect on which kinds of actions are,or are not, justified. Two kinds of potential harm must beconsidered when determining an appropriate approachto the role of risk in decision-making: 1) localized non-spreading impacts and 2) propagating impacts resultingin irreversible and widespread damage.

Traditional decision-making strategies focus on thecase where harm is localized and risk is easy to calculatefrom past data. Under these circumstances, cost-benefitanalyses and mitigation techniques are appropriate. Thepotential harm from miscalculation is bounded.

1. The Rio Declaration on Environment and Development presentsit as follows: "In order to protect the environment, the precautionaryapproach shall be widely applied by States according to their capabil-ities. Where there are threats of serious or irreversible damage, lackof full scientific certainty shall not be used as a reason for postponingcost-effective measures to prevent environmental degradation."

1

EXTREME RISK INITIATIVE —NYU SCHOOL OF ENGINEERING WORKING PAPER SERIES 2

On the other hand, the possibility of irreversible andwidespread damage raises different questions about thenature of decision making and what risks can be reason-ably taken. This is the domain of the PP.

Criticisms are often levied against those who arguefor caution portraying them as unreasonable and pos-sibly even paranoid. Those who raise such criticismsare implicitly or explicitly advocating for a cost benefitanalysis, and necessarily so. Critics of the PP have alsoexpressed concern that it will be applied in an overreach-ing manner, eliminating the ability to take reasonablerisks that are needed for individual or societal gains.While indiscriminate use of the PP might constrainappropriate risk-taking, at the same time one can alsomake the error of suspending the PP in cases when it isvital.

Hence, a non-naive view of the precautionary princi-ple is one in which it is only invoked when necessary,and only to prevent a certain variety of very precisely de-fined risks based on distinctive probabilistic structures.But, also, in such a view, the PP should never be omittedwhen needed.

The remainder of this section will outline the differ-ence between the naive and non-naive approaches.

2.1 What we mean by a non-naive PPRisk aversion and risk-seeking are both well-studiedhuman behaviors. However, it is essential to distinguishthe PP so that it is neither used naively to justify any actof caution, nor dismissed by those who wish to courtrisks for themselves or others.

The PP is intended to make decisions that ensuresurvival when statistical evidence is limited—becauseit has not had time to show up —by focusing on theadverse effects of "absence of evidence."

Table 1 encapsulates the central idea of the paper andshows the differences between decisions with a risk ofharm (warranting regular risk management techniques)and decisions with a risk of total ruin (warranting thePP).

Standard Risk Management Precautionary Approachlocalized harm systemic ruinnuanced cost-benefit avoid at all costsstatistical fragility basedstatistical probabilistic non-statisticalvariations ruinconvergent probabibilities divergent probabilitiesrecoverable irreversibleindependent factors interconnected factorsevidence based precautionarythin tails fat tailsbottom-up, tinkering top-down engineeredevolved human-made

Table 1: Two different types of risk and their respectivecharacteristics compared

2.2 Harm vs. Ruin: When the PP is necessaryThe purpose of the PP is to avoid a certain class of what,in probability and insurance, is called “ruin" problems

[1]. A ruin problem is one where outcomes of riskshave a non-zero probability of resulting in unrecoverablelosses. An often-cited illustrative case is that of a gamblerwho loses his entire fortune and so cannot return tothe game. In biology, an example would be a speciesthat has gone extinct. For nature, "ruin" is ecocide: anirreversible termination of life at some scale, which couldbe planetwide. The large majority of variations thatoccur within a system, even drastic ones, fundamentallydiffer from ruin problems: a system that achieves ruincannot recover. As long as the instance is bounded, e.g.a gambler can work to gain additional resources, theremay be some hope of reversing the misfortune. This isnot the case when it is global.

Our concern is with public policy. While an individualmay be advised to not "bet the farm," whether or not hedoes so is generally a matter of individual preferences.Policy makers have a responsibility to avoid catastrophicharm for society as a whole; the focus is on the aggregate,not at the level of single individuals, and on global-systemic, not idiosyncratic, harm. This is the domain ofcollective "ruin" problems.

3 WHY RUIN IS SERIOUS BUSINESS

The risk of ruin is not sustainable. By the ruin theorems,if you incur a tiny probability of ruin as a "one-off" risk,survive it, then do it again (another "one-off" deal), youwill eventually go bust with probability 1. Confusionarises because it may seem that the "one-off" risk isreasonable, but that also means that an additional oneis reasonable. This can be quantified by recognizing thatthe probability of ruin approaches 1 as the number ofexposures to individually small risks, say one in tenthousand, increases For this reason a strategy of risktaking is not sustainable and we must consider anygenuine risk of total ruin as if it were inevitable.

The good news is that some classes of risk can bedeemed to be practically of probability zero: the earthsurvived trillions of natural variations daily over 3 bil-lion years, otherwise we would not be here. By recog-nizing that normal risks are not in the category of ruinproblems, we recognize also that it is not necessary oreven normal to take risks that involve a possibility ofruin.

3.1 Harm vs. Ruin: When the PP is necessaryIt is important to contrast and not conflate the PP andrisk management. Risk management involves variousstrategies to make decisions based upon accounting forthe effects of positive and negative outcomes and theirprobabilities, as well as seeking means to mitigate harmand offset losses. Risk management strategies are impor-tant for decision-making when ruin is not at stake. How-ever, the only risk management strategy of importancein the case of the PP is ensuring that actions which canresult in ruin are not taken, or equivalently, modifying


potential choices of action so that ruin is not one of thepossible outcomes.

More generally, we can identify three layers associatedwith strategies for dealing with uncertainty and risk.The first layer is the PP which addresses cases thatinvolve potential global harm, whether probabilities areuncertain or known and whether they are large or small.The second is risk management which addresses the caseof known probabilities of well-defined, bounded gainsand losses. The third is risk aversion or risk-seekingbehavior, which reflects quite generally the role of per-sonal preferences for individual risks when uncertaintyis present.

3.2 Ruin is foreverA way to formalize the ruin problem in terms of the de-structive consequences of actions identifies harm as notabout the amount of destruction, but rather a measureof the integrated level of destruction over the time itpersists. When the impact of harm extends to all futuretimes, i.e. forever, then the harm is infinite. When theharm is infinite, the product of any non-zero probabilityand the harm is also infinite, and it cannot be balancedagainst any potential gains, which are necessarily finite.This strategy for evaluation of harm as involving theduration of destruction can be used for localized harmsfor better assessment in risk management. Our focushere is on the case where destruction is complete fora system or an irreplaceable aspect of a system.

For example, for humanity global devastation cannotbe measured on a scale in which harm is proportionalto level of devastation. The harm due to complete de-struction is not the same as 10 times the destructionof 1/10 of the system. As the percentage of destructionapproaches 100%, the assessment of harm diverges toinfinity (instead of converging to a particular number)due to the value placed on a future that ceases to exist.

Because the “cost” of ruin is effectively infinite, cost-benefit analysis (in which the potential harm and po-tential gain are multiplied by their probabilities andweighed against each other) is no longer a usefulparadigm. Even if probabilities are expected to be zerobut have a non-zero uncertainty, then a sensitivity anal-ysis that considers the impact of that uncertainty resultsin infinities as well. The potential harm is so substantialthat everything else in the equation ceases to matter. Inthis case, we must do everything we can to avoid thecatastrophe.

4 SCIENTIFIC METHODS AND THE PPHow well can we know either the potential conse-quences of policies or their probabilities? What doesscience say about uncertainty? To be helpful in policydecisions, science has to encompass not just expectationsof potential benefit and harm but also their probabilityand uncertainty.

Just as the imperative of analysis of decision-makingchanges when there is infinite harm for a small, non-zerorisk, so is there a fundamental change in the ability toapply scientific methods to the evaluation of that harm.This influences the way we evaluate both the possibilityof and the risk associated with ruin.

The idea of precaution is the avoidance of adverse con-sequences. This is qualitatively different from the idea ofevidentiary action (from statistics). In the case of the PP,evidence may come too late. The non-naive PP bridgesthe gap between precaution and evidentiary action usingthe ability to evaluate the difference between local andglobal risks.

4.1 Precautionary vs. Evidentiary Action

Statistical-evidentiary approaches to risk analysis andmitigation count the frequency of past events (robuststatistics), or calibrate parameters of statistical distribu-tions to generate probabilities of future events (para-metric approach), or both. Experimental evidentiarymethods follow the model of medical trials, computingprobabilities of harm from side effects of drugs or in-terventions by observing the reactions in a variety ofanimal and human models. Generally they assume thatthe risk itself (i.e. nature of harm and their probabil-ity) is adequately determined by available information.However, the level of risk may be hard to gauge asits probability may be uncertain, and, in the case ofpotential infinite harm, an uncertainty that allows fora non-zero probability results in infinities so that theproblem is ill-defined mathematically.

While evidentiary approaches are often considered toreflect adherence to the scientific method in its purestform, it is apparent that these approaches do not applyto ruin problems. In an evidentiary approach to risk(relying on evidence-based methods), the existence ofa risk or harm occurs when we experience that risk orharm. In the case of ruin, by the time evidence comes itwill by definition be too late to avoid it. Nothing in thepast may predict one fatal event as illustrated in Fig. 2.Thus standard evidence-based approaches cannot work.

More generally, evidentiary action is a frameworkbased upon the quite reasonable expectation that welearn from experience. The idea of evidentiary action isembodied in the kind of learning from experience that isfound in how people often react to disasters—after thefact. When a disaster occurs people prepare for the nextone, but do not anticipate it in advance. For the case ofruin problems, such behavior guarantees extinction.

4.2 Invalid Empirical Arguments Against Ruin

In the case of arguments about ruin problems, claimsthat experience thus far has not provided evidence forruin, and thus it should not be considered, are not valid.


4.3 Unknowability, Uncertainty and UnpredictabilityIt has been shown that the complexity of real worldsystems limits the ability of empirical observations todetermine the outcomes of actions upon them [2]. Thismeans that a certain class of systemic risks will remaininherently unknown. In some classes of complex sys-tems, controlled experiments cannot evaluate all of thepossible systemic consequences under real-world con-ditions. In these circumstances, efforts to provide assur-ance of the "lack of harm" are insufficiently reliable. Thisruns counter to both the use of empirical approaches(including controlled experiments) to evaluate risks, andto the expectation that uncertainty can be eliminated byany means.

4.4 Distinguishing Global and Local RisksSince there are mathematical limitations to predictabilityof outcomes in a complex system, the central issueto determine is whether the threat of harm is local(hence globally benign) or carries global consequences.Scientific analysis can robustly determine whether a riskis systemic, i.e. by evaluating the connectivity of thesystem to propagation of harm, without determining thespecifics of such a risk. If the consequences are systemic,the associated uncertainty of risks must be treated differ-ently than if it is not. In such cases, precautionary actionis not based on direct empirical evidence but on analyti-cal approaches based upon the theoretical understandingof the nature of harm. It relies on probability theorywithout computing probabilities. The essential questionis whether or not global harm is possible or not. Theoryenables generalizing from experience in order to apply itto new circumstances. In the case of the PP, the existenceof a robust way to generalize is essential.

The relevance of the precautionary principle today isgreater than in the past, owing to the global connectivityof civilization that makes the spreading of effects toplaces previously insulated.

5 FAT TAILS AND FRAGILITY5.1 Thin and Fat TailsTo figure out whether a given decision involves the riskof ruin and thus warrants the use of the PP, we mustfirst understand the relevant underlying probabilisticstructures.

There are two classes of probability distributions ofevents: one in which events are accompanied by wellbehaved, mild variations (e.g. Gaussian or thin tails), andthe other where small probabilities are associated withlarge variations that have no characteristic scale (e.g.power law or fat tails). Allegorically these are illustratedby Mediocristan and Extremistan (Figs. 1 and 2), theformer being typical of human weight distributions, andthe latter of human wealth distributions. Given a seriesof events (a sequence of measurements of weight orwealth), in the case of thin tails the sum is proportional

to the average, and in the case of fat tails a sum overthem may be entirely dominated by a single one. Thus,while no human being can be heavier than, say, tenaverage adults (since weight is thin-tailed), a singleindividual can be richer than the poorest two billionhumans (since wealth is fat tailed).

In thin tailed domains (Fig 1) harm comes from thecollective effect of many, many events; no event alonecan be consequential enough to affect the aggregate. Itis practically impossible for a single day to account for99% of all heart attacks in a given year (the probability issmall enough to be practically zero), for an illustration).Statistical distributions that belong to the thin-taileddomain include: Gaussian, Binomial, Bernoulli, Poisson,Gamma, Beta and Exponential.

In fat tailed domains of risk (Fig. 2) harm comes fromthe largest single event. Examples of relevant statisticaldistributions include: Pareto, Levy-Stable distributionswith infinite variance, Cauchy, and power law distribu-tions, especially with larger exponents.

Figure 1: Thin Tails from Tinkering, Bottom-Up, Evolu-tion. In nature no individual variation represents a largeshare of the sum of the variations. Natural boundariesprevent cascading effects from propagating globally.Mass extinctions arise from the rare cases where largeimpacts (meteorite hits and vulcanism) propagate acrossthe globe through the atmosphere and oceans.


Figure 2: Fat Tails from a Top-Down, Engineered De-sign In human made variations the tightly connectedglobal system implies a single deviation will eventuallydominate the sum of their effects. Examples includepandemics, invasive species, financial crises and mono-culture.

5.2 Why interdependence brings fat tailsWhen variations lead to independent impacts locally, theaggregate effect of those variations is small according tothe central limit theorem, guaranteeing thin-tailed distri-butions. When there is interdependence, the central limittheorem does not apply, and aggregate variations maybecome much more severe due to mutual reinforcement.Interdependence arises because of the coupling of behav-ior in different places. Under these conditions, cascadespropagate through the system in a way that can causelarge impacts. Whether components are independent ordependent clearly matters to systemic disasters such aspandemics and financial or other crises. Interdependenceincreases the probability of ruin, ultimately to the pointof certainty.

Consider the global financial crash of 2008. As finan-cial firms became increasingly interdependent duringthe latter part of the 20th century, small fluctuationsduring periods of calm masked the vulnerability of thesystem to cascading failures. Instead of a local shock inan independent area of the system, we experienced aglobal shock with cascading effects. The crisis of 2008,in addition, illustrates the failure of evidentiary riskmanagement. Since data from the time series beginningin the 1980s exhibited stability, causing the period to bedubbed "the great moderation," it deceived those relyingon historical statistical evidence.

6 WHAT IS THE RISK OF HARM TO THEEARTH?At the systemic largest scale on Earth, nature has thintails, though tails may be fat at smaller length scales orsufficiently long time scales; occasional mass extinctionsoccur at very long time scales. This is characteristic of abottom-up, local tinkering design process, where thingschange primarily locally and only mildly and iterativelyon a global scale.

In recent years, it has been shown that natural systemsoften have fat tail (power law) behaviors associated withthe propagation of shocks [3]. This, however, applies toselected systems that do not have barriers that limit thosepropagations. The earth has an intrinsic heterogeneityof oceans/continents, deserts, mountains, lakes, riversand climate differences that limit the propagation ofvariations from one area to another. There are alsosmaller natural boundaries associated with organismsizes and those of local groups of organisms. Amongthe largest propagation events we commonly observe areforest fires, but even these are bounded in their impactscompared to a global scale. The various forms of barriers

limit the propagation of cascades that enable large scaleevents.

At longer time scales of millions of years, mass extinc-tions can achieve a global scale. Connectivity of oceansand the atmosphere enables propagation of impacts, i.e.gas, ash and dust propagating through the atmospheredue to meteor impacts and volcanism, is considered ascenario for these extinction events [4]. The variabilityassociated with mass extinctions can especially be seenin the fossil record of marine animal species; those ofplants and land insects are comparatively robust. It isnot known to what extent these events are driven extrin-sically, by meteor impacts, geological events includingvolcanos, or cascading events of coupled species extinc-tions, or combinations of them. The variability associatedwith mass extinctions, however, indicates that there arefat tail events that can affect the global biosphere. Themajor extinction events during the past 500 million yearsoccur at intervals of millions of years [5]. While mass ex-tinctions occur, the extent of that vulnerability is drivenby both sensitivity to external events and connectivityamong ecosystems.

The greatest impact of human beings on this naturalsystem connectivity is through dramatic increases inglobal transportation. The impact of invasive species andrapid global transmission of diseases demonstrates therole of human activity in connecting previously muchmore isolated natural systems. The role of transporta-tion and communication in connecting civilization itselfis apparent in economic interdependence manifest incascading financial crises that were not possible even ahundred years ago. The danger we are facing today isthat we as a civilization are globally connected, and thefat tail of the distribution of shocks extends globally, toour peril.

Had nature not imposed sufficiently thin-tailed varia-tions in the aggregate or macro level, we would not behere today. A single one of the trillions, perhaps the tril-lions of trillions, of variations over evolutionary historywould have terminated life on the planet. Figures 1 and 2show the difference between the two separate statisticalproperties. While tails can be fat for subsystems, natureremains predominantly thin-tailed at the level of theplanet [6]. As connectivity increases the risk of extinctionincreases dramatically and nonlinearly [7].

6.1 Risk and Global InterventionismCurrently, global dependencies are manifest in the ex-pressed concerns about policy maker actions that nom-inally appear to be local in their scope. In just recentmonths, headlines have been about Russia’s involvementin Ukraine, the spread of Ebola in east Africa, expansionof ISIS control into Iraq, ongoing posturing in North Ko-rea and Israeli-Palestinian conflict, among others. Theseevents reflect upon local policy maker decisions thatare justifiably viewed as having global repercussions.The connection between local actions and global risks


compels widespread concern and global responses toalter or mitigate local actions. In this context, we pointout that the broader significance and risk aszsociatedwith policy actions that impact on global ecological andhuman survival is the essential point of the PP. Payingattention to the headline events without paying attentionto these even larger risks is like being concerned aboutthe wine being served on the Titanic.

7 FRAGILITY

The PP applies only to the largest scale impacts dueto the inherent fragility of systems that maintain theirstructure. As the scale of impacts increases the harmincreases non-linearly up to the point of destruction. Wedefine fragility in the technical appendix as "is harmedby uncertainty", with the mathematical result that whatis harmed by uncertainty has a certain type on nonlinearresponse to random events.

7.1 Fragility as Nonlinear Response

Everything that has survived is necessarily non-linear toharm. If I fall from a height of 10 meters I am injuredmore than 10 times than if I fell from a height of 1 meter,or more than 1000 times than if I fell from a heightof 1 centimeter, hence I am fragile. In general, everyadditional meter, up to the point of my destruction, hurtsme more than the previous one.

Similarly, if I am hit with a big stone I will be harmeda lot more than if I were pelted serially with pebbles ofthe same total weight.

Everything that is fragile and still in existence (thatis, unbroken), will be harmed more by a certain stressorof intensity X than by k times a stressor of intensityX/k, up to the point of breaking. If I were not fragile(susceptible to harm more than linearly), I would bedestroyed by accumulated effects of small events, andthus would not survive. This non-linear response iscentral for everything on planet earth.

This explains the necessity of considering scale wheninvoking the PP. Polluting in a small way does notwarrant the PP because it is essentially less harmful thanpolluting in large quantities, since harm is non-linear.

Figure 3: Nonlinear response compared to linear re-sponse. The PP should be evoked to prevent impactsthat result in complete destruction due to the nonlin-ear response of natural systems, it is not needed forsmaller impacts where risk management methods canbe applied.

7.2 Why is fragility a general rule?The statistical structure of stressors is such that smallvariations are much, much more frequent than largeones. Fragility is intimately connected to the ability towithstand small impacts and recover from them. Thisability is what makes a system retain its structure. Everysystem has a threshold of impact beyond which it willbe destroyed, i.e. its structure is not sustained.

Consider a coffee cup sitting on a table: there aremillions of recorded earthquakes every year; if the coffeecup were linearly sensitive to earthquakes and accumu-lated their effects as small deteriorations of its form, itwould not persist even for a short time as it would havebeen broken down due to the accumulated impact ofsmall vibrations. The coffee cup, however, is non-linearto harm, so that the small or remote earthquakes onlymake it wobble, whereas one large one would break itforever.

This nonlinearity is necessarily present in everythingfragile.

Thus, when impacts extend to the size of the sys-tem, harm is severely exacerbated by non-linear effects.Small impacts, below a threshold of recovery, do notaccumulate for systems that retain their structure. Largerimpacts cause irreversible damage. We should be careful,however, of actions that may seem small and local butthen lead to systemic consequences.

7.3 Fragility, Dose response and the 1/n ruleAnother area where we see non-linear responses to harmis the dose-response relationship. As the dose of somechemical or stressor increases, the response to it growsnon-linearly. Many low-dose exposures do not causegreat harm, but a single large-dose can cause irreversibledamage to the system, like overdosing on painkillers.

In decision theory, the 1/n heuristic is a simple rulein which an agent invests equally across n funds (orsources of risk) rather than weighting their investmentsaccording to some optimization criterion such as mean-variance or Modern Portfolio Theory (MPT), which dic-tates some amount of concentration in order to increasethe potential payoff. The 1/n heuristic mitigates the riskof suffering ruin due to an error in the model; thereis no single asset whose failure can bring down theship. While the potential upside of the large payoff isdampened, ruin due to an error in prediction is avoided.This heuristic works best when the sources of variationsare uncorrelated and, in the presence of correlation ordependence between the various sources of risk, the totalexposure needs to be reduced.


Hence, because of non-linearities, it is preferable todiversify our effect on the planet, e.g. distinct types ofpollutants, across the broadest number of uncorrelatedsources of harm, rather than concentrate them. In thisway, we avoid the risk of an unforeseen, disproportion-ately harmful response to a pollutant deemed "safe" byvirtue of responses observed only in relatively smalldoses.

8 THE LIMITATION OF TOP-DOWN ENGINEER-ING IN COMPLEX ENVIRONMENTS

In considering the limitations of risk-taking, a key ques-tion is whether or not we can analyze the potentialoutcomes of interventions and, knowing them, identifythe associated risks. Can’t we just "figure it out?” Withsuch knowledge we can gain assurance that extremeproblems such as global destruction will not arise.

Since the same issue arises for any engineering effort,we can ask what is the state-of-the-art of engineering?Does it enable us to know the risks we will encounter?Perhaps it can just determine the actions we should,or should not, take. There is justifiably widespread re-spect for engineering because it has provided us withinnovations ranging from infrastructure to electronicsthat have become essential to modern life. What is notas well known by the scientific community and thepublic, is that engineering approaches fail in the face ofcomplex challenges and this failure has been extensivelydocumented by the engineering community itself [8].The underlying reason for the failure is that complexenvironments present a wide range of conditions. Whichconditions will actually be encountered is uncertain.Engineering approaches involve planning that requiresknowledge of the conditions that will be encountered.Planning fails due to the inability to anticipate the manyconditions that will arise.

This problem arises particularly for “real-time” sys-tems that are dealing with large amounts of informationand have critical functions in which lives are at risk. Aclassic example is the air traffic control system. An effortto modernize that system by traditional engineeringmethods cost $3-6 billion and was abandoned withoutchanging any part of the system because of the inabilityto evaluate the risks associated with its implementation.

Significantly, the failure of traditional engineering toaddress complex challenges has led to the adoption ofinnovation strategies that mirror evolutionary processes,creating platforms and rules that can serve as a basisfor safely introducing small incremental changes thatare extensively tested in their real world context [8].This strategy underlies the approach used by highly-successful, modern, engineered-evolved, complex sys-tems ranging from the Internet, to Wikipedia, to iPhoneApp communities.

9 WHY SHOULD GMOS BE BANNED BUT NOTNUCLEAR ENERGY?As examples that are relevant to the discussion of thedifferent types of strategies, we consider the differencesbetween concerns about nuclear energy and GM crops.

9.1 Nuclear energyMany are justifiably concerned about nuclear energy.It is known that the potential harm due to radiationrelease, core meltdowns and waste can be large. At thesame time, the nature of these risks has been extensivelystudied, and the risks from local uses of nuclear energyhave a scale that is much smaller than global. Thus,even though some uncertainties remain, it is possibleto formulate a cost benefit analysis of risks for localdecision-making. The large potential harm at a local scalemeans that decisions about whether, how and how muchto use nuclear energy, and what safety measures to use,should be made carefully so that decision makers and thepublic can rely upon them. Risk management is a veryserious matter when potential harm can be large andshould not be done casually or superficially. Those whoperform the analysis must not only do it carefully, theymust have the trust of others that they are doing it care-fully. Nevertheless, the known statistical structure of therisks and the absence of global systemic consequencesmakes the cost benefit analysis meaningful. Decisionscan be made in the cost-benefit context—evoking the PPis not appropriate for small amounts of nuclear energy,as the local nature of the risks is not indicative of thecircumstances to which the PP applies.

In large quantities, we should worry about an unseenrisk from nuclear energy and invoke the PP. In smallquantities, it may be OK—how small we should deter-mine by direct analysis, making sure threats never ceaseto be local.

In addition to the risks from nuclear energy useitself, we must keep in mind the longer term risksassociated with the storage of nuclear waste, which arecompounded by the extended length of time they remainhazardous. The problems of such longer term “lifecycle”effects is present in many different industries. It arisesnot just for nuclear energy but also for fossil fuels andother sources of pollution, though the sheer duration oftoxicity effects for nuclear waste, enduring for hundredsof thousands of years in some cases, makes this problemparticularly intense for nuclear power.

Weighing different options is an important part of pol-icy decision-making. The general idea is that we shouldlimit pollution to small sources, and allow for significanterrors in the estimates of harm, even if proponents andexperts deem them safe.

9.2 GMOsGenetically Modified Organisms (GMOs) and their riskare currently the subject of debate [9]. Here we ar-gue that they fall squarely under the PP because their


risk is systemic. There are two aspects of systemicrisk, the widespread impact on the ecosystem and thewidespread impact on health.

Ecologically, in addition to intentional cultivation,GMOs have the propensity to spread uncontrollably, andthus their risks cannot be localized. The cross-breeding ofwild-type plants with genetically modified ones preventstheir disentangling, leading to irreversible system-wideeffects with unknown downsides. The ecological impli-cations of releasing modified organisms into the wild arenot tested empirically before release.

Health wise, the modification of crops impacts every-one. Corn, one of the primary GMO crops, is not onlyeaten fresh or as cereals, but is also a major component ofprocessed foods in the form of high-fructose corn syrup,corn oil, corn starch and corn meal. In 2014 in the USalmost 90% of corn and 94% of soybeans are GMO [11].Foods derived from GMOs are not tested in humansbefore they are marketed.

The widespread impacts of GMOs on ecologies andhuman health imply they are in the domain of thePP. This should itself compel policy makers to takeextreme caution. However, there is a difficulty for manyin understanding the abstract nature of the engagementin risks and imagining the many possible ways that harmcan be caused. Thus, we summarize further the natureof the risks that are involved.

9.3 GMOs in detailThe systemic global impacts of GMOs arise from acombination of (1) engineered genetic modifications, (2)monoculture—the use of single crops over large areas.Global monoculture itself is of concern for potentialglobal harm, but the evolutionary context of traditionalcrops provides important assurances. Invasive speciesare frequently a problem but one might at least arguethat the long term evolutionary testing of harmful im-pacts of organisms on local ecological systems mitigatesif not eliminates the largest potential risks. Monoculturein combination with genetic engineering dramaticallyincreases the risks being taken. Instead of a long historyof evolutionary selection, these modifications rely notjust on naive engineering strategies that do not appropri-ately consider risk in complex environments, but also ex-plicitly reductionist approaches that ignore unintendedconsequences and employ very limited empirical testing.

Ironically, at a time when engineering is adoptingevolutionary approaches due to the failure of top-downstrategies, biologists and agronomists are adopting top-down engineering strategies and taking global systemicrisks in introducing organisms into the wild.

One argument in favor of GMOs is that they are nomore "unnatural" than the selective farming our ances-tors have been doing for generations. In fact, the ideasdeveloped in this paper show that this is not the case.Selective breeding over human history is a process inwhich change still happens in a bottom-up way, and can

be expected to result in a thin-tailed distribution. If thereis a mistake, some harmful variation, it will not spreadthroughout the whole system but end up dying out dueto local experience over time. Human experience overgenerations has chosen the biological organisms that arerelatively safe for consumption. There are many that arenot, including parts of and varieties of the crops we docultivate [12]. Introducing rapid changes in organisms isinconsistent with this process. There is a limited rate atwhich variations can be introduced and selection will beeffective [13].

There is no comparison between tinkering with theselective breeding of genetic components of organismsthat have previously undergone extensive histories ofselection and the top-down engineering of taking a genefrom a fish and putting it into a tomato. Saying that sucha product is natural misses the process of natural selec-tion by which things become “natural." While there areclaims that all organisms include transgenic materials,those genetic transfers that are currently present weresubject to selection over long times and survived. Thesuccess rate is tiny. Unlike GMOs, in nature there is noimmediate replication of mutated organisms to become alarge fraction of the organisms of a species. Indeed, anyone genetic variation is unlikely to become part of thelong term genetic pool of the population. Instead, justlike any other genetic variation or mutation, transgenictransfers are subject to competition and selection overmany generations before becoming a significant part ofthe population. A new genetic transfer engineered todayis not the same as one that has survived this process ofselection.

An example of the effect of transfer of biologicallyevolved systems to a different context is that of zoonoticdiseases. Even though pathogens consume their hosts,they evolve to be less harmful than they would oth-erwise be. Pathogens that cause highly lethal diseasesare selected against because their hosts die before theyare able to transmit to others. This is the underlyingreason for the greater dangers associated with zoonoticdiseases—caused by pathogens that shift from the hostthat they evolved in to human beings, including HIV,Avian and Swine flu that transferred from monkeys(through chimpanzees), birds and hogs, respectively.

More generally, engineered modifications to ecologicalsystems (through GMOs) are categorically and statisti-cally different from bottom up ones. Bottom-up modifi-cations do not remove the crops from their long termevolutionary context, enabling the push and pull ofthe ecosystem to locally extinguish harmful mutations.Top-down modifications that bypass this evolutionarypathway unintentionally manipulate large sets of inter-dependent factors at the same time, with dramatic risksof unintended consequences. They thus result in fat-tailed distributions and place a huge risk on the foodsystem as a whole.

For the impact of GMOs on health, the evaluation ofwhether the genetic engineering of a particular chemical


(protein) into a plant is OK by the FDA is based uponconsidering limited existing knowledge of risks associ-ated with that protein. The number of ways such an eval-uation can be in error is large. The genetic modificationsare biologically significant as the purpose is to stronglyimpact the chemical functions of the plant, modifying itsresistance to other chemicals such as herbicides or pesti-cides, or affecting its own lethality to other organisms—i.e. its antibiotic qualities. The limited existing knowl-edge generally does not include long term testing ofthe exposure of people to the added chemical, even inisolation. The evaluation is independent of the ways theprotein affects the biochemistry of the plant, includinginteractions among the various metabolic pathways andregulatory systems—and the impact of the resultingchanges in biochemistry on health of consumers. Theevaluation is independent of its farm-ecosystem com-bination (i.e. pesticide resistant crops are subject to in-creased use of pesticides, which are subsequently presentin the plant in larger concentrations and cannot bewashed away). Rather than recognizing the limitationsof current understanding, poorly grounded perspectivesabout the potential damage with unjustified assumptionsare being made. Limited empirical validation of bothessential aspects of the conceptual framework as wellas specific conclusions are being used because testing isrecognized to be difficult.

We should exert the precautionary principle here – ournon-naive version – because we do not want to discovererrors after considerable and irreversible environmentaland health damage.

9.4 Red herring: How about the risk of famine with-out GMOs?An argument used by those who advocate for GMOs isthat they will reduce the hunger in the world. Invokingthe risk of famine as an alternative to GMOs is a deceitfulstrategy, no different from urging people to play Russianroulette in order to get out of poverty.

The evocation of famine also prevents clear thinkingabout not just GMOs but also about global hunger. Theidea that GMO crops will help avert famine ignoresevidence that the problem of global hunger is due topoor economic and agricultural policies. Those who careabout the supply of food should advocate for an imme-diate impact on the problem by reducing the amountof corn used for ethanol in the US, which burns foodfor fuel consuming over 40% of the US crop that couldprovide enough food to feed 2/3 of a billion people [14].

One of the most extensively debated cases for GMOsis a variety of rice—"golden rice"—to which has beenadded a precursor of vitamin A as a potential meansto alleviate this nutritional deficiency, which is a keymedical condition affecting impoverished populations.Since there are alternatives, including traditional vitaminfortification, one approach is to apply a cost benefitanalysis comparing these approaches. Counter to this

approach stands both the largely unknown risks asso-ciated with the introduction of GMOs, and the needand opportunities for more systemic interventions toalleviate not just malnutrition but poverty and hungerworldwide. While great attention should be placed onimmediate needs, neglecting the larger scale risks is un-reasonable [10]. Here science should adopt an unyieldingrigor for both health benefit and risk assessment, in-cluding careful application of the PP. Absent such rigor,advocacy by the scientific community not only fails tobe scientific, but also becomes subject to challenge forshort term interests, not much different from corporateendorsers. Thus, cutting corners on tests, including testswithout adequate consent or approvals performed onChinese children [15], undermines scientific claims tohumanitarian ideals. Given the promotion of "goldenrice" by the agribusiness that also promote biofuels, theirinterest in humanitarian impacts versus profits gainedthrough wider acceptance of GMO technology can belegitimately questioned [16].

9.5 GMOs in summaryIn contrast to nuclear energy (which, as discussed insection 9.1 above, may or may not fall under the PP,depending on how and where (how widely) it is imple-mented), Genetically Modified Organisms, GMOs, fallsquarely under the PP because of their systemic risk.The understanding of the risks is very limited and thescope of the impacts are global both due to engineeringapproach replacing an evolutionary approach, and dueto the use of monoculture.

Labeling the GMO approach “scientific" betrays a verypoor—indeed warped—understanding of probabilisticpayoffs and risk management. A lack of observations ofexplicit harm does not show absence of hidden risks.Current models of complex systems only contain thesubset of reality that is accessible to the scientist. Natureis much richer than any model of it. To expose anentire system to something whose potential harm isnot understood because extant models do not predict anegative outcome is not justifiable; the relevant variablesmay not have been adequately identified.

Given the limited oversight that is taking place onGMO introductions in the US, and the global impactof those introductions, we are precisely in the regimeof the ruin problem. A rational consumer should say:We do not wish to pay—or have our descendants pay—for errors made by executives of Monsanto, who arefinancially incentivized to focus on quarterly profitsrather than long term global impacts. We should exertthe precautionary principle—our non-naive version—simply because we otherwise will discover errors withlarge impacts only after considerable damage.

9.6 Vaccination, Antibiotics, and Other ExposuresOur position is that while one may argue that vaccina-tion is risky, or risky under some circumstances, it does


not fall under PP owing to the lack of systemic risk.The same applies to such interventions as antibiotics,provided the scale remains limited to the local.

10 PRECAUTION AS POLICY AND NAIVE IN-TERVENTION

When there is a risk of ruin, obstructionism and policyinaction are important strategies, impeding the rapidheadlong experimentation with global ruin by thosewith short-term, self-centered incentives and perspec-tives. Two approaches for policy action are well justified.In the first, actions that avoid the inherent sensitivity ofthe system to propagation of harm can be used to free thesystem to enable local decision-making and explorationwith only local harm. This involves introducing bound-aries, barriers and separations that inhibit propagation ofshocks, preventing ruin for overly connected systems. Inthe second, where such boundaries don’t exist or cannotbe introduced due to other effects, there is a need foractions that are adequately evaluated as to their globalharm. Scientific analysis of such actions, meticulouslyvalidated, is needed to prevent small risks from causingruin.

What is not justified, and dangerous, are actions thatare intended to prevent harm by additional intervention.The reason is that indirect effects are likely to createprecisely the risks that one is intending to avoid.

When existing risks are perceived as having the po-tential for ruin, it may be assumed that any preventivemeasure is justified. There are at least two problemswith such a perspective. First, localized harm is oftenmistaken for ruin, and the PP is wrongly invoked whererisk management techniques should be employed. Whena risk is not systemic, overreaction will typically causemore harm than benefits, like undergoing dangeroussurgery to remove a benign growth. Second, even if thethreat of ruin is real, taking specific (positive) action inorder to ward off the perceived threat may introducenew systemic risks. It is often wiser to reduce or removeactivity that is generating or supporting the threat andallow natural variations to play out in localized ways.

Preventive action should be limited to correcting sit-uations by removing threats via negativa in order tobring them back in line with a statistical structure thatavoids ruin. It is often better to remove structure orallow natural variation to take place rather than to addsomething additional to the system.

When one takes the opposite approach, taking specificaction designed to diminish some perceived threat, oneis almost guaranteed to induce unforeseen consequences.Even when there appears to be a direct link from aspecific action to a specific preventive outcome, theweb of causality extends in complex ways with con-sequences that are far from the intended goal. Theseunintended consequences may generate new vulnerabil-ities or strengthen the harm one is hoping to diminish.Thus, when possible, limiting fragilizing dependencies is

better than imposing additional structure that increasesthe fragility of the system as a whole.

11 FALLACIOUS ARGUMENTS AGAINST PPIn this section we respond to a variety of arguments thathave been made against the PP.

11.1 Crossing the road (the paralysis fallacy)Many have countered the invocation of the PP with“nothing is ever totally safe.” “I take risks crossing theroad every day, so according to you I should stay homein a state of paralysis.” The answer is that we don’t crossthe street blindfolded, we use sensory information tomitigate risks and reduce exposure to extreme shocks.

Even more importantly in the context of the PP, theprobability distribution of death from road accidents atthe population level is thin-tailed; I do not incur the riskof generalized human extinction by crossing the street—a human life is bounded in duration and its unavoidabletermination is an inherent part of the bio-social system[17]. The error of my crossing the street at the wrongtime and meeting an untimely demise in general doesnot cause others to do the same; the error does notspread. If anything, one might expect the opposite effect,that others in the system benefit from my mistake byadapting their behavior to avoid exposing themselves tosimilar risks. Equating risks a person takes with his orher own life with risking the existence of civilization isan inappropriate ego trip. In fact, the very idea of thePP is to avoid such a frivolous focus.

The paralysis argument is often used to present the PPas incompatible with progress. This is untrue: tinkering,bottom-up progress where mistakes are bounded is howprogress has taken place in history. The non-naive PPsimply asserts that the risks we take as we innovate mustnot extend to the entire system; local failure serves asinformation for improvement. Global failure does not.

11.2 The Psychology of Risk and Thick Tailed Distri-butionsOne concern about the utility of the PP is that its evoca-tion may become commonplace because of risk aversion.Is it true that people overreact to small probabilitiesand the PP would feed into human biases? While wehave carefully identified the scope of the domain ofapplicability of the PP, it is also helpful to review theevidence of risk aversion, which we find not to be basedupon sound studies.

Certain empirical studies appear to support the exis-tence of a bias toward risk aversion, claiming evidencethat people choose to avoid risks that are beneficial,inconsistent with cost-benefit analyses. The relevant ex-periments ask people questions about single probabilityevents, showing that people overreact to small probabil-ities. However, those researchers failed to include theconsequences of the associated events which humans


underestimate. Thus, this empirical strategy as a wayof identifying effectiveness of response to risk is funda-mentally flawed [18].

The proper consideration of risk involves both prob-ability and consequence, which should be multipliedtogether. Consequences in many domains have thicktails, i.e. much larger consequences can arise than areconsidered in traditional statistical approaches. Overre-acting to small probabilities is not irrational when theeffect is large, as the product of probability and harmis larger than expected from the traditional treatment ofprobability distributions.

11.3 The Loch Ness fallacyMany have countered that we have no evidence thatthe Loch Ness monster doesn’t exist, and, to take theargument of evidence of absence being different fromabsence of evidence, we should act as if the Loch Nessmonster existed. The argument is a corruption of theabsence of evidence problem and certainly not part ofthe PP.

The relevant question is whether the existence of theLoch Ness monster has implications for decisions aboutactions that are being taken. We are not considering adecision to swim in the Loch Ness. If the Loch Nessmonster did exist, there would still be no reason toinvoke the PP, as the harm he might cause is limitedin scope to Loch Ness itself, and does not present therisk of ruin.

11.4 The fallacy of misusing the naturalistic fallacySome people invoke “the naturalistic fallacy,” a philo-sophical concept that is limited to the moral domain.According to this critique, we should not claim thatnatural things are necessarily good; human innovationcan be equally valid. We do not claim to use nature toderive a notion of how things "ought" to be organized.Rather, as scientists, we respect nature for the extent ofits experimentation. The high level of statistical signifi-cance given by a very large sample cannot be ignored.Nature may not have arrived at the best solution to aproblem we consider important, but there is reason tobelieve that it is smarter than our technology based onlyon statistical significance.

The question about what kinds of systems work (asdemonstrated by nature) is different than the questionabout what working systems ought to do. We can takea lesson from nature—and time—about what kinds oforganizations are robust against, or even benefit from,shocks, and in that sense systems should be structured inways that allow them to function. Conversely, we cannotderive the structure of a functioning system from whatwe believe the outcomes ought to be.

To take one example, Cass Sunstein—who has writtenan article critical of the PP [19]—claims that there is a"false belief that nature is benign." However, his concep-tual discussion fails to distinguish between thin and fat

tails, local harm and global ruin. The method of analysismisses both the statistical significance of nature and thefact that it is not necessary to believe in the perfection ofnature, or in its "benign" attributes, but rather in its trackrecord, its sheer statistical power as a risk evaluator andas a risk manager in avoiding ruin.

11.5 The "Butterfly in China" fallacyThe statement “if I move my finger to scratch my nose,by the butterfly-in-China effect, owing to non-linearities,I may terminate life on earth," is known to be flawed. Theexplanation is not widely understood. The fundamentalreason arises because of the existence of a wide rangein levels of predictability and the presence of a largenumber of fine scale degrees of freedom for every largescale one [20]. Thus, the traditional deterministic chaos,for which the butterfly effect was named, applies specif-ically to low dimensional systems with a few variablesin a particular regime. High dimensional systems, likethe earth, have large numbers of fine scale variables forevery large scale one. Thus, it is apparent that not allbutterfly wing flaps can cause hurricanes. It is not clearthat any one of them can, and, if small perturbationscan influence large scale events, it happens only underspecific conditions where amplification occurs.

Empirically, our thesis rebuts the butterfly fallacy withthe argument that, in the aggregate, nature has experi-enced trillions of small variations and yet it survives.Therefore, we know that the effects of scratching one’snose fall into the thin tailed domain and thus do notwarrant the precautionary principle.

As described previously, barriers in natural systemslead to subsystems having a high-degree of indepen-dence. Understanding how modern systems with a high-degree of connectivity have cascading effects is essentialfor understanding when it is and isn’t appropriate to usethe PP.

11.6 The potato fallacyMany species were abruptly introduced into the OldWorld starting in the 16th Century that did not causeenvironmental disasters (perhaps aside from diseasesaffecting Native Americans). Some use this observationin defense of GMOs. However, the argument is fallaciousat two levels:

First, by the fragility argument, potatoes, tomatoesand similar "New World" goods were developed locallythrough progressive, bottom-up tinkering in a complexsystem in the context of its interactions with its envi-ronment. Had they had an impact on the environment,it would have caused adverse consequences that wouldhave prevented their continual spread.

Second, a counterexample is not evidence in the riskdomain, particularly when the evidence is that takinga similar action previously did not lead to ruin. Lack ofruin due to several or even many trials does not indicatesafety from ruin in the next one. This is also the Russianroulette fallacy, detailed below.


11.7 The Russian roulette fallacy (the counterexam-ples in the risk domain)The potato example, assuming potatoes had not beengenerated top-down by some engineers, would still notbe sufficient. Nobody says "look, the other day therewas no war, so we don’t need an army," as we knowbetter in real-life domains. Nobody argues that a giantRussian roulette with many barrels is "safe" and a greatmoney making opportunity because it didn’t blow upsomeone’s brains last time.

There are many reasons a previous action may nothave led to ruin while still having the potential to doso. If you attempt to cross the street with a blindfoldand earmuffs on, you may make it across, but this is notevidence that such an action carries no risk.

More generally, one needs a large sample for claimsof absence of risk in the presence of a small probability ofruin, while a single “n = 1" example would be sufficientto counter the claims of safety—this is the Black Swanargument [?]. Simply put, systemic modifications requirea very long history in order for the evidence of lack ofharm to carry any weight.

11.8 The Carpenter fallacyRisk managers skeptical of the understanding of riskof biological processes, such as GMOs, by the expertsare sometimes asked "are you a biologist?". But nobodyasks a probabilist dealing with roulette sequences if he isa carpenter. To understand the gambler’s ruin problemby roulette betting, we know to ask a probabilist, nota carpenter. No amount of expertise in carpentry canreplace rigor in understanding the properties of longsequences of small probability bets. Likewise, no amountof expertise in the details of biological processes can bea substitute for probabilistic rigor.

Moreover, the track record of the experts in under-standing biological and medical risks has been extremelypoor, and we need policies to be robust to their miscal-culations . The “expert problem” is manifest in a verypoor record historically of risks taken with innovationsin biological products, ranging from biofuels to transfatto nicotine, etc. Consider the recent major drug recallssuch as Thalidomide, Fen-Phen, Tylenol and Vioxx—all of these show chronic blindness on the part of thespecialist to large scale risks of Black Swan events. Yetmost of these risks were local and not systemic (withthe exception of biofuel impacts on global hunger andsocial unrest): with the systemic the recall happens toolate, which is why we need this strong version of the PP.

11.9 The technological salvation fallacyIatrogenics is harm done by a healer despite positiveintentions, see Appendix A for a list of innovationsin care that have extensive documentation of adverseconsequences. Each of these underwent best practicestesting that did not reveal the iatrogenic consequences

prior to widespread application. The controlled teststhat are used to evaluate innovations for potential harmcannot replicate the large number of conditions in whichinterventions are applied in the real world. Adverseconsequences are exposed only by extensive experiencewith the combinatorial number of real world condi-tions. Natural, i.e. evolutionary, selection implements asa strategy the use of selection of lack of harm undersuch conditions in a way that bounds the consequencesbecause the number of replicates is increased only grad-ually during the process in which success is determined.In contrast, traditional engineering of technological solu-tions does not. Thus, the more technological a solution toa current problem—the more it departs from solutionsthat have undergone evolutionary selection—the moreexposed one becomes to iatrogenics owing to combinato-rial branching of conditions with adverse consequences.

Our concern here isn’t mild iatrogenics, but the sys-temic case.

11.10 The pathologization fallacyToday many mathematical or conceptual models thatare claimed to be rigorous are based upon unvalidatedand incorrect assumptions. Such models are rationalin the sense that they are logically derived from theirassumptions, except that it is the modeler who is usingan incomplete representation of the reality. Often themodelers are not familiar with the dynamics of com-plex systems or use Gaussian statistical methods thatdo not take into account fat-tails and make inferencesthat would not be acceptable under different classes ofprobability distributions. Many biases, such as the onesused by Cass Sunstein (mentioned above), about theoverestimation of the probabilities of rare events in factcorrespond to the testers using a bad probability modelthat is thin-tailed. See Ref. [6] for a deeper discussion.

It has became popular to claim irrationality for GMOand other skepticism on the part of the general public—not realizing that there is in fact an "expert problem"and such skepticism is healthy and even necessary forsurvival. For instance, in The Rational Animal, the authorspathologize people for not accepting GMOs although"the World Health Organization has never found evi-dence of ill effects," a standard confusion of evidenceof absence and absence of evidence. Such pathologizingis similar to behavioral researchers labeling hyperbolicdiscounting as "irrational" when in fact it is largely theresearcher who has a very narrow model and richermodels make the "irrationality" go away.

These researchers fail to understand that humans mayhave precautionary principles against systemic risks, andcan be skeptical of the untested consequences of policiesfor deeply rational reasons.

12 CONCLUSIONSThis formalization of the two different types of un-certainty about risk (local and systemic) makes clear


when the precautionary principle is, and when it isn’t,appropriate. The examples of GMOs and nuclear energyhelp to elucidate the application of these ideas. We hopethis will help decision makers to avoid ruin in the future.

ACKNOWLEDGMENTS

Gloria Origgi, Maya Bialik, David Boxenhorn, JessicaWoolley, ...

CONFLICTS OF INTEREST

One of the authors (Taleb) reports having received mon-etary compensation for lecturing on risk managementand Black Swan risks by the Institute of Nuclear PowerOperations, INPO, the main association in the UnitedStates, in 2011, in the wake of the Fukushima accident.

REFERENCES

[1] Asmussen, S., & Albrecher, H., 2010, Ruin probabilities (Vol. 14).World Scientific.

[2] Bar-Yam, Y., 2013, The Limits of Phenomenology: From Behaviorismto Drug Testing and Engineering Design, arXiv 1308.3094

[3] Bak, P., 2009, How nature works. Copernicus.[4] Schulte, P., Alegret, L., Arenillas, I., Arz, J. A., Barton, P. J., Bown,

P. R., ... & Willumsen, P. S., 2010. The Chicxulub asteroid impactand mass extinction at the Cretaceous-Paleogene boundary. Sci-ence, 327(5970), 1214-1218.

[5] Alroy, J., 2008. Dynamics of origination and extinction in themarine fossil record. Proceedings of the National Academy ofSciences, 105(Supplement 1), 11536-11542.

[6] Taleb, N.N., 2014, Silent Risk: Lectures on Fat Tails, (Anti)Fragility,and Asymmetric Exposures, SSRN

[7] Rauch, E.M. and Y. Bar-Yam, 2006, Long-range interactions andevolutionary stability in a predator-prey system, Physical Review E73, 020903

[8] Bar-Yam, Y., 2003, When Systems Engineering Fails — TowardComplex Systems Engineering in International Conference onSystems, Man & Cybernetics Vol. 2, IEEE Press, Piscataway, NJ,2003, pp. 2021- 2028.

[9] Thompson, P.B. (Ed.), 2007. Food biotechnology in ethical per-spective (Vol. 10). Springer.

[10] Read, R., Hutchinson, P., 2014. What is Wrong With GM Food?,Philosophers’ Magag.

[11] Recent Trends in GE Adoption, Adoption of Genetically Engi-neered Crops in the U.S., USDA Economics Research Service,

[12] See e.g. List of poisonous plants, Wikipedia[13] Nowak, M., Schuster, P.,1989. Error thresholds of replication in

finite populations mutation frequencies and the onset of Muller’sratchet. Journal of Theoretical Biology, 137, 375-395.

[14] Albino, D.K., Bertrand, K.Z., Bar-Yam, Y., 2012, Food for fuel: Theprice of ethanol. arXiv:1210.6080.

[15] Qiu, J., 2012, China sacks officials over Golden Rice controversy.Nature News, 10.

[16] Harmon, A., 2013, Golden Rice: Lifesaver?[17] Taleb, N.N., 2007, Black swans and the domains of statistics. The

American Statistician, 61, 198-200.[18] Taleb, N.N. and Tetlock, P.E., 2014, On the Difference be-

tween Binary Prediction and True Exposure with Implicationsfor Forecasting Tournaments and Decision Making Researchhttp://dx.doi.org/10.2139/ssrn.2284964

[19] Sunstein, C.R., Beyond the Precautionary Principle (January 2003).U Chicago Law & Economics, Olin Working Paper No. 149; U ofChicago, Public Law Working Paper No. 38.

[20] Bar-Yam, Y., Complex Systems: The Science of Prediction,[21] Pitman, E. (1980). Subexponential distribution functions. J. Aus-

tral. Math. Soc. Ser. A, 29(3):337âAS347.[22] Teugels, J. L. (1975). The class of subexponential distributions. The

Annals of Probability, 3(6):1000âAS1011.

[23] Chistyakov, V. (1964). A theorem on sums of independent positiverandom variables and its applications to branching random pro-cesses. Theory of Probability & Its Applications, 9(4):640âAS648.

[24] Embrechts, P., Goldie, C. M., and Veraverbeke, N. (1979). Subexpo-nentiality and infinite divisibility. Probability Theory and RelatedFields, 49(3):335âAS347.


Medical Intervention Intended Effects Unintended Effects

Rofecoxib (Vioxx, Ceoxx, Ceeoxx) relieve osteoarthritis, dysmenorrhoea myocardial infarctions [?]Thalidomide (Immunoprin, Talidex, Tal-izer, Thalomid)

sedative severe birth defects [?]

Fen-phen (Pondimin) weight loss valvular heart disease, pulmonary hy-pertension [?]

Diethylstilbestrol (Distilbene,Stilbestrol, Stilbetin)

reduce miscarriage cancerous tumors in daughters exposedin utero [?]

Cerivastatin (Baycol, Lipobay) lower cholesterol, reduce cardiovasculardisease

Rhabdomyolysis leading to renal failure[?]

lobotomy improve mental disorder loss of personality, intellect [?]Troglitazone (Rezulin, Resulin,Romozin, Noscal)

antidiabetic, antiinflammatory drug-induced hepatitis [?]

Terfenadine (Seldane, Triludan, Tel-dane)

antihistamine cardiac arrhythmia [?]

Phenylpropanolamine (Accutrim) appetite suppressant, stimulant, decon-gestant

increased stroke [?]

hospitalization patient treatment and monitoring nosocomial infection; medication errors[?]

antibiotics clear bacterial infections treatment-resistant bacteria [?]antidepressants relieve depression increased suicide risk [?]Encainide (Enkaid), flecainide (Tambo-cor)

reduced arrhythmia increased mortality [?]

Acetaminophen (Tylenol) pain relief liver damage [?]coronary angioplasty increased blood flow increased risk of death/myocardial in-

farction [?]cosmetic surgery improved aesthetics infection, death, deformity, other mal-

function [?]obsessive hygiene keeping safe from ‘germs’ autoimmune disorders [?]ear-tubes otitis media with effusion tympanosclerosis [?]

Table 2: Examples of iatrogenics in the medical field. The upper portion of the table shows medications andtreatments whose use has been significantly reduced or completely discontinued due to their undesired effects(which were discovered only after significant damage had been done). The lower portion of the table lists exampleswhere unintended side effects are significant but treatment continues to be applied due to expected benefits.

APPENDIX AA SAMPLE OF IATROGENICS, ERRORS DISCOVERED TOO LATE


APPENDIX BDEFINITION OF FAT TAILS AND DISTINCTIONBETWEEN MEDIOCRISTAN AND EXTREMISTAN

Probability distributions range between extreme thin-tailed (Bernoulli) and extreme fat tailed [6]. Among thecategories of distributions that are often distinguisheddue to the convergence properties of moments are: 1)Having a support that is compact but not degenerate, 2)Subgaussian, 3) Gaussian, 4) Subexponential, 5) Powerlaw with exponent greater than 3, 6) Power law withexponent less than or equal to 3 and greater than 2, 7)Power law with exponent less than or equal to 2. Inparticular, power law distributions have a finite meanonly if the exponent is greater than 1, and have a finitevariance only if the exponent exceeds 2.

Our interest is in distinguishing between cases wheretail events dominate impacts, as a formal definition ofthe boundary between the categories of distributionsto be considered as Mediocristan and Extremistan. Thenatural boundary between these occurs at the subexpo-nential class which has the following property:

Let X = (Xi

)

1in

be a sequence of independent andidentically distributed random variables with support inthe positive real numbers (R+), with cumulative distri-bution function F . The subexponential class of distribu-tions is defined by [22],[21].

lim

x!+1

1� F ⇤2(x)

1� F (x)= 2

where F ⇤2= F 0 ⇤ F is the cumulative distribution

of X1

+ X2

, the sum of two independent copies of X .This implies that the probability that the sum X

1

+ X2

exceeds a value x is twice the probability that eitherone separately exceeds x. Thus, every time the sumexceeds x, for large enough values of x, the value ofthe sum is due to either one or the other exceeding x—the maximum over the two variables—and the other ofthem contributes negligibly.

More generally, it can be shown that the sum of nvariables is dominated by the maximum of the valuesover those variables in the same way. Formally, thefollowing two properties are equivalent to the subex-ponential condition [23],[24]. For a given n � 2, letSn

= ⌃

n

i=1

xi

and Mn

= max

1in

xi

a) limx!1

P (Sn>x)

P (X>x)

= n,

b) limx!1

P (Sn>x)

P (Mn>x)

= 1,

Thus the sum Sn

has the same magnitude as thelargest sample M

n

, which is another way of saying thattails play the most important role.

Intuitively, tail events in subexponential distributionsshould decline more slowly than an exponential distri-bution for which large tail events should be irrelevant.

K

Prob Density

Ξ!K, s" # $s"" % #"&

K !x "'" f Λ!s_#$s"" !x" ) xΞ!K, s"" % #

"&

K !x "'" f Λ!s_" !x" ) x

Figure 4: A definition of fragility as left tail-vega sensi-tivity, in other words how an increase in uncertainty(which includes errors) affects adverse outcomes. Thefigure shows the effect of the perturbation of the lowersemi-deviation s� on the tail integral ⇠ of (x – ⌦) belowK, ⌦ being a centering constant. Our detection of fragilitydoes not require the specification of f the probabilitydistribution.

Indeed, one can show that subexponential distributionshave no exponential moments:

Z 1

0

e✏x dF (x) = +1

for all values of " greater than zero. However,the con-verse isn’t true, since distributions can have no ex-ponential moments, yet not satisfy the subexponentialcondition.

We note that if we choose to indicate deviations asnegative values of the variable x, the same result holdsby symmetry for extreme negative values, replacingx ! +1 with x ! �1. For two-tailed variables, wecan separately consider positive and negative domains.

APPENDIX CMATHEMATICAL DERIVATIONS OF FRAGILITY

The following offers a formal definition of fragility as"vega", negative expected response from uncertainty. Italso shows why this is necessarily linked to acceleratedresponse, how "size matters". The derivations explain,among other things"

• How spreading risks are dangerous compared tolimited one; the derivations show the notion of riskspreading as a non-concave response.

• Why error is a problem in the presence of nonlin-earity.

• Why polluting "a little" is qualitatively differentfrom pollution "a lot".

• Eventually, why fat tails arise from acceleratingresponse.

Intrinsic and Inherited Fragility: Our definition offragility is two-fold. First, of concern is the intrinsic


fragility, the shape of the probability distribution of avariable and its sensitivity to s-, a parameter controllingthe left side of its own distribution. But we do not oftendirectly observe the statistical distribution of objects,and, if we did, it would be difficult to measure theirtail-vega sensitivity. Nor do we need to specify suchdistribution: we can gauge the response of a given objectto the volatility of an external stressor that affects it. Forinstance, an option is usually analyzed with respect tothe scale of the distribution of the “underlying” security,not its own; the fragility of a coffee cup is determinedas a response to a given source of randomness or stress;that of a house with respect of, among other sources, thedistribution of earthquakes. This fragility coming fromthe effect of the underlying is called inherited fragility.The transfer function, which we present next, allowsus to assess the effect, increase or decrease in fragility,coming from changes in the underlying source of stress.

Transfer Function: A nonlinear exposure to a certainsource of randomness maps into tail-vega sensitivity(hence fragility). We prove that

Inherited Fragility , Concavity in exposure on the leftside of the distribution

and build H, a transfer function giving an exact map-ping of tail vega sensitivity to the second derivativeof a function. The transfer function will allow us toprobe parts of the distribution and generate a fragility-detection heuristic covering both physical fragility andmodel error.

Taking z as a stress level and ⇧(z) the harm function,it suffices to see that, with n > 1,

⇧(nz) < n⇧(z) for all 0 < nz < Z⇤

where Z⇤ is the level (not necessarily specified) atwhich the item is broken. Such inequality leads to ⇧(z)having a negative second derivative at the initial valuez.

So if a coffee cup is less harmed by n times astressor of intensity Z than once a stressor of nZ, thenharm (as a negative function) needs to be concave tostressors up to the point of breaking; such stricture isimposed by the structure of survival probabilities andthe distribution of harmful events, and has nothing todo with subjective utility or some other figments. Justas with a large stone hurting more than the equivalentweight in pebbles, if, for a human, jumping one mil-limeter caused an exact linear fraction of the damageof, say, jumping to the ground from thirty feet, then theperson would be already dead from cumulative harm.Actually a simple computation shows that he wouldhave expired within hours from touching objects orpacing in his living room, given the multitude of suchstressors and their total effect. The fragility that comesfrom linearity is immediately visible, so we rule it outbecause the object would be already broken and theperson already dead. The relative frequency of ordinary

events compared to extreme events is the determinant.In the financial markets, there are at least ten thousandtimes more events of 0.1% deviations than events of10%. There are close to 8,000 micro-earthquakes dailyon planet earth, that is, those below 2 on the Richterscale —about 3 million a year. These are totally harmless,and, with 3 million per year, you would need them tobe so. But shocks of intensity 6 and higher on the scalemake the newspapers. Accordingly, we are necessarilyimmune to the cumulative effect of small deviations, orshocks of very small magnitude, which implies that theseaffect us disproportionally less (that is, nonlinearly less)than larger ones.

Model error is not necessarily mean preserving. s-,the lower absolute semi-deviation does not just expresschanges in overall dispersion in the distribution, suchas for instance the “scaling” case, but also changes inthe mean, i.e. when the upper semi-deviation from ⌦ toinfinity is invariant, or even decline in a compensatorymanner to make the overall mean absolute deviationunchanged. This would be the case when we shift thedistribution instead of rescaling it. Thus the same vega-sensitivity can also express sensitivity to a stressor (doseincrease) in medicine or other fields in its effect on eithertail. Thus s�(l) will allow us to express the sensitivityto the "disorder cluster" in Antifragile: i) uncertainty,ii) variability, iii) imperfect, incomplete knowledge, iv)chance, v) chaos, vi) volatility, vii) disorder, viii) entropy,ix) time, x) the unknown, xi) randomness, xii) turmoil,xiii) stressor, xiv) error, xv) dispersion of outcomes.

C.1 Tail Sensitivity to UncertaintyWe construct a measure of "vega", that is, the sensitivityto uncertainty, in the left tails of the distribution thatdepends on the variations of s the semi-deviation belowa certain level W , chosen in the L1 norm in orderto ensure its existence under "fat tailed" distributionswith finite first semi-moment. In fact s would exist asa measure even in the case of undefined moments to theright side of W .

Let X be a random variable, the distribution of whichis one among a one-parameter family of pdf f

�

,� 2 I ⇢R. We consider a fixed reference value ⌦ and, from thisreference, the left-semi-absolute deviation:

s�(�) =

Z⌦

�1(⌦� x)f

�

(x)dx (1)

We assume that � ! s–(�) is continuous, strictly in-creasing and spans the whole range R

+

= [0, +1),so that we may use the left-semi-absolute deviations– as a parameter by considering the inverse function�(s) : R

+

! I , defined by s� (�(s)) = s for s 2 R+

.This condition is for instance satisfied if, for any given

x < ⌦, the probability is a continuous and increasingfunction of �. Indeed, denoting

F�

(x) = Pf�(X < x) =

Zx

�1f�

(t) dt, (2)


an integration by parts yields:

s�(�) =

Z⌦

�1F�

(x) dx

This is the case when � is a scaling parameter, i.e., X ⇠⌦+ �(X

1

� ⌦) indeed one has in this case

F�

(x) = F1

✓⌦+

x� ⌦

�

◆,

@F�

@�(x) =

⌦� x

�2

f�

(x) and s�(�) = � s�(1).

It is also the case when � is a shifting parameter, i.e.X ⇠ X

0

� � , indeed, in this case F�

(x) = F0

(x+ �) and@s

�

@�

(x) = F�

(⌦).For K < ⌦ and s 2 R+, let:

⇠(K, s�) =

ZK

�1(⌦� x)f

�(s

�)

(x)dx (3)

In particular, ⇠(⌦, s–) = s–. We assume, in a firststep, that the function ⇠(K,s–) is differentiable on(�1, ⌦] ⇥ R

+

. The K-left-tail-vega sensitivity of X at stresslevel K < ⌦ and deviation level s� > 0 for the pdf f

�

is:

V (X, f�

,K, s�) =@⇠

@s�(K, s�) =

Z⌦

�1(⌦� x)

@f�

)

@�dx

!✓ds�

d�

◆�1

(4)

As in the many practical instances where thresholdeffects are involved, it may occur that ⇠ does not dependsmoothly on s–. We therefore also define a finite differenceversion of the vega-sensitivity as follows:

V (X, f�

,K, s�) =1

2�s

�⇠(K, s� +�s)� ⇠(K, s� ��s)

�

=

ZK

�1(⌦� x)

f�

(s� +�s)(x)� f�

(s� ��s)(x)

2� sdx (5)

Hence omitting the input �s implicitly assumes that�s ! 0.

Note that ⇠(K, s�) = �E(X|X < K) Pf�(X < K). It

can be decomposed into two parts:

⇠�K, s�(�)

�= (⌦�K)F

�

(K) + P�

(K) (6)

P�

(K) =

ZK

�1(K � x)f

�

(x) dx (7)

Where the first part (⌦�K)F�

(K) is proportional to theprobability of the variable being below the stress levelK and the second part P

�

(K) is the expectation of theamount by which X is below K (counting 0 when it isnot). Making a parallel with financial options, while s–(�)is a “put at-the-money”, ⇠(K,s–) is the sum of a put struckat K and a digital put also struck at K with amount ⌦ –K; it can equivalently be seen as a put struck at ⌦ witha down-and-in European barrier at K.

Figure 5: The different curves of F�

(K) and F�

0(K)

showing the difference in sensitivity to changes at dif-ferent levels of K.

Letting � = �(s–) and integrating by part yields

⇠�K, s�(�)

�= (⌦�K)F

�

(K) +

ZK

�1F�

(x)dx =

Z⌦

�1FK

�

(x) dx (8)

Where FK

�

(x) = F�

(min(x,K)) = min (F�

(x), F�

(K)),so that

V (X, f�

,K, s�) =@⇠

@s(K, s�)

=

R⌦

�1@F

K�

@�

(x) dxR⌦

�1@F�@�

(x) dx(9)

For finite differences

V (X, f�

,K, s�,�s) =1

2� s

Z⌦

�1�FK

�,�s

(x)dx (10)

Where �+

s

and ��s

are such that s(�+

s

�) = s� + �s,s(��

s

�) = s� ��s and �FK

�,�s

(x) = FK

�s+(x)� FK

�s�(x).

C.2 Mathematical Expression of FragilityIn essence, fragility is the sensitivity of a given risk mea-sure to an error in the estimation of the (possibly one-sided) deviation parameter of a distribution, especiallydue to the fact that the risk measure involves parts of thedistribution – tails – that are away from the portion usedfor estimation. The risk measure then assumes certainextrapolation rules that have first order consequences.These consequences are even more amplified when therisk measure applies to a variable that is derived fromthat used for estimation, when the relation between thetwo variables is strongly nonlinear, as is often the case.


C.2.1 Definition of Fragility: The Intrinsic CaseThe local fragility of a random variable X

�

depending onparameter �, at stress level K and semi-deviation levels–(�) with pdf f

�

is its K-left-tailed semi-vega sensitivityV (X, f

�

,K, s�).

The finite-difference fragility of X�

at stress level K andsemi-deviation level s�(�)±�s with pdf f

�

is its K-left-tailedfinite-difference semi-vega sensitivity V (X, f

�

,K, s�,�s).In this definition, the fragility relies in the unsaid

assumptions made when extrapolating the distributionof X

�

from areas used to estimate the semi-absolutedeviation s–(�), around ⌦, to areas around K on whichthe risk measure ⇠ depends.

C.2.2 Definition of Fragility: The Inherited CaseNext we consider the particular case where a randomvariable Y = '(X) depends on another source of risk X,itself subject to a parameter �. Let us keep the abovenotations for X, while we denote by g

�

the pdf ofY ,⌦

Y

= '(⌦) and u�(�) the left-semi-deviation of Y.

Given a “strike” levelL = '(K), let us define, as in the case of X :

⇣�L, u�

(�)�=

ZK

�1(⌦

Y

� y)g�

(y) dy (11)

The inherited fragility of Y with respect to X at stresslevel L = '(K) and left-semi-deviation level s�(�) of Xis the partial derivative:

VX

�Y, g

�

, L, s�(�)�=

@⇣

@s

�L, u�

(�)�=

ZK

�1(⌦

Y

� Y )

@g�

@�(y)dy

!✓ds�

d�

◆�1

(12)

Note that the stress level and the pdf are definedfor the variable Y, but the parameter which is usedfor differentiation is the left-semi-absolute deviation ofX, s–(�). Indeed, in this process, one first measures thedistribution of X and its left-semi-absolute deviation,then the function ' is applied, using some mathematicalmodel of Y with respect to X and the risk measure ⇣ isestimated. If an error is made when measuring s–(�), itsimpact on the risk measure of Y is amplified by the ratiogiven by the “inherited fragility”.

Once again, one may use finite differences and definethe finite-difference inherited fragility of Y with respect toX, by replacing, in the above equation, differentiation byfinite differences between values �+ and �–, where s–(�+)= s– + �s and s–(�–) = s– – �s.

C.3 Effect of Nonlinearity on Intrinsic FragilityLet us study the case of a random variable Y = '(X); thepdf g

�

of which also depends on parameter �, relatedto a variable X by the nonlinear function '. We arenow interested in comparing their intrinsic fragilities.We shall say, for instance, that Y is more fragilefragile

at the stress level L and left-semi-deviation level u�(�)

than the random variable X, at stress level K and left-semi-deviation level s�(�) if the L-left-tailed semi-vegasensitivity of Y

�

is higher than the K-left-tailed semi-vega sensitivity of X

�

:

V (Y, g�

, L, µ�) > V (X, f

�

,K, s�) (13)

One may use finite differences to compare thefragility of two random variables:V (Y, g

�

, L,�µ) >V (X, f

�

,K,�s). In this case, finite variations must becomparable in size, namely �u/u– = �s/s–.

Let us assume, to start, that ' is differentiable, strictlyincreasing and scaled so that ⌦

Y

= '(⌦) = ⌦. We alsoassume that, for any given x < ⌦, @F�

@�

(x) > 0.In this case, as observed above, � ! s–(�) is also

increasing.Let us denote G

y

(y) = Pg�(Y < y) . We have:

G�

(�(x)) = Pg� (Y < �(y)) = P

f�(X < x) = F�

(x).(14)

Hence, if ⇣(L, u–) denotes the equivalent of ⇠(K), s�

with variable (Y, g�

) instead of (X, f�

), we have:

⇣�L, u�

(�)�=

Z⌦

�1FK

�

(x)d�

dx(x)dx (15)

Because ' is increasing and min('(x),'(K)) ='(min(x,K)). In particular

µ�(�) = ⇣

�⌦, µ�

(�)�=

Z⌦

�1FK

�

(x)d�

dx(x) dx (16)

The L-left-tail-vega sensitivity of Y is therefore:

V�Y, g

�

, L, u�(�)�=

R⌦

�1@F

K�

@�

(x)d�dx

(x) dxR⌦

�1@F�@�

(x)d�dx

(x) dx(17)

For finite variations:

V (Y, g�

, L, u�(�),�u) =

1

2�u

Z⌦

�1�FK

�,�u

(x)d�

dx(x)dx

(18)Where �+

u

� and ��u

� are such that u(�+

u

�) = u�+�u,

u(�+

u

�) = u� ��u and FK

�,�u

(x) = FK

�

+u(x)� FK

�

�u(x).

Next, Theorem 1 proves how a concave transformation'(x) of a random variable x produces fragility.

Fragility Transfer TheoremTheorem 1: Let, with the above notations, ' : R ! R

be a twice differentiable function such that '(⌦) = ⌦

and for any x < ⌦, d'

dx

(x) > 0. The random variableY = '(X) is more fragile at level L = '(K) and pdf g

�

than X at level K and pdf f�

if, and only if, one has:Z

⌦

�1HK

�

(x)d2'

dx2

(x)dx < 0


Figure 6: The Transfer function H for different portionsof the distribution: its sign flips in the region slightlybelow ⌦

Where

HK

�

(x) =@PK

�

@�(x)

�@PK

�

@�(⌦)�@P

�

@�(x)

�@P

�

@�(⌦) (19)

and where

P�

(x) =

Zx

�1F�

(t)dt (20)

is the price of the “put option” on X�

with “strike” xand

PK

�

(x) =

Zx

�1FK

�

(t)dt

is that of a "put option" with "strike" x and "Europeandown-and-in barrier" at K.H can be seen as a transfer function, expressed as the

difference between two ratios. For a given level x ofthe random variable on the left hand side of ⌦, thesecond one is the ratio of the vega of a put struck atx normalized by that of a put "at the money" (i.e. struckat ⌦), while the first one is the same ratio, but where putsstruck at x and ⌦ are "European down-and-in options"with triggering barrier at the level K.

The proof is detailed in [?] and [?].Fragility Exacerbation TheoremTheorem 2: With the above notations, there exists a

threshold ⇥

�

< ⌦ such that, if K ⇥

�

then HK

�

(x) > 0

for x 2 (1,�

] with K < l

ambda < ⌦.As a con-sequence, if the change of variable ' is concave on(�1,

�

] and linear on [�

,⌦], then Y is more fragileat L = '(K)than X at K.

One can prove that, for a monomodal distribution,⇥

�

< �

< ⌦ (see discussion below), so whatever thestress level K below the threshold ⇥

�

, it suffices that thechange of variable ' be concave on the interval (�1,⇥

�

]

and linear on [⇥

l

ambda,⌦] for Y to become more fragileat L than X at K. In practice, as long as the change ofvariable is concave around the stress level K and haslimited convexity/concavity away from K, the fragilityof Y is greater than that of X .

Figure 6 shows the shape of HK

�

(x) in the case ofa Gaussian distribution where � is a simple scalingparameter (� is the standard deviation �) and ⌦ = 0.We represented K = –2� while in this Gaussian case, ⇥

�

= –1.585�.

Figure 7: The distribution of G�

and the various deriva-tives of the unconditional shortfalls

DISCUSSION

Monomodal caseWe say that the family of distributions (f

�

) is left-monomodal if there exists K

�

< ⌦ such that @f�

@�

> 0 on (–1,

�

] and @f�

@�

6 0 on [µ�

,⌦]. In this case @P�@�

is a convexfunction on the left half-line (–1, µ

�

], then concaveafter the inflexion point µ

�

. For K µ�

, the function@P

K�

@�

coincides with @P�@�

on (–1, K], then is a linearextension, following the tangent to the graph of @P�

@�

inK (see graph below). The value of @P

K�

@�

(⌦) correspondsto the intersection point of this tangent with the verticalaxis. It increases with K, from 0 when K ! –1 to avalue above @P�

@�

(⌦) when K = µ�

. The threshold ⇥

�

corresponds to the unique value of K such that @P

K�

@�

(⌦) =

@P�@�

(⌦) . When K < ⇥

�

then G�

(x) =@P�@�

(x).

@P�@�

(⌦)

and GK

�

(x) =

@P

K�

@�

(x).

@P

K�

@�

(⌦) are functions such thatG

�

(⌦) = GK

�

(⌦) = 1 and which are proportional for x K, the latter being linear on [K, ⌦]. On the other hand,if K < ⇥

�

then @P

K�

@�

(⌦) <@P�@�

(⌦) and G�

(K) < GK

�

(K),which implies that G

�

(x) < GK

�

(x) for x K. Anelementary convexity analysis shows that, in this case,the equation G

�

(x) = GK

�

(x) has a unique solution �

with µl

ambda < �

< ⌦. The “transfer” function HK

�

(x)is positive for x <

�

, in particular when x µ�

andnegative for

�

< x < ⌦.Scaling ParameterWe assume here that � is a scaling parameter, i.e. X

�

=

⌦+ �(X1

� ⌦). In this case, as we saw above, we have

f�

(x) =1

�f1

✓⌦+

x� ⌦

�

◆, F

�

(x) = F1

✓⌦+

x� ⌦

�

◆

P�

(x) = �P1

✓⌦+

x� ⌦

�

◆and s�(�) = �s�(1).

Hence


⇠(K, s�(�)) = (⌦�K)F1

✓⌦+

K � ⌦

�

◆

+ �P1

✓⌦+

K � ⌦

�

◆(21)

@⇠

@s�(K, s�) =

1

s�(1)

@⇠

@�(K,�)

=

1

s�(�)

⇣P�

(K) + (⌦�K)F�

(K) + (⌦�K)

2

f�

(K)

⌘

(22)

When we apply a nonlinear transformation ', theaction of the parameter � is no longer a scaling: whensmall negative values of X are multiplied by a scalar �,so are large negative values of X. The scaling � applies tosmall negative values of the transformed variable Y witha coefficient d'

dx

(0), but large negative values are subjectto a different coefficient d'

dx

(K), which can potentially bevery different.

C.4 Fragility DriftFragility is defined at as the sensitivity – i.e. the firstpartial derivative – of the tail estimate ⇠ with respect tothe left semi-deviation s–. Let us now define the fragilitydrift:

V 0K

(X, f�

,K, s�) =@2⇠

@K@s�(K, s�) (23)

In practice, fragility always occurs as the result offragility, indeed, by definition, we know that ⇠(⌦, s–) = s–,hence V(X, f

�

, ⌦, s–) = 1. The fragility drift measures thespeed at which fragility departs from its original value1 when K departs from the center ⌦.

Second-order FragilityThe second-order fragility is the second order derivative

of the tail estimate ⇠ with respect to the semi-absolutedeviation s–:

V 0s

�(X, f�

,K, s�) =@2⇠

(@s�)2

(K, s�)

As we shall see later, the second-order fragility drives thebias in the estimation of stress tests when the value ofs– is subject to uncertainty, through Jensen’s inequality.

C.5 Definitions of Robustness and AntifragilityAntifragility is not the simple opposite of fragility, as wesaw in Table 1. Measuring antifragility, on the one hand,consists of the flipside of fragility on the right-hand side,but on the other hand requires a control on the robustnessof the probability distribution on the left-hand side.From that aspect, unlike fragility, antifragility cannot besummarized in one single figure but necessitates at leasttwo of them.

When a random variable depends on another sourceof randomness: Y

�

= '(X�

), we shall study the an-tifragility of Y

�

with respect to that of X�

and to theproperties of the function '.

DEFINITION OF ROBUSTNESSLet (X

�

) be a one-parameter family of random vari-ables with pdf f

�

. Robustness is an upper control on thefragility of X, which resides on the left hand side of thedistribution.

We say that f�

is b-robust beyond stress level K < ⌦ ifV(X

�

, f�

, K’, s(�)) b for any K’ K. In other words, therobustness of f

�

on the half-line (–1, K] is

R(�1,K]

(X�

, f�

,K, s�(�)) = max

K

06K

V (X�

, f�

,K 0, s�(�)),

(24)so that b-robustness simply means

R(�1,K]

(X�

, f�

,K, s�(�)) 6 b

We also define b-robustness over a given interval [K1, K2]by the same inequality being valid for any K’ 2 [K1, K2].In this case we use

R[K1,K2]

(X�

, f�

,K, s�(�)) =

max

K16K

06K2

V (X�

, f�

,K 0, s�(�)). (25)

Note that the lower R, the tighter the control and themore robust the distribution f

�

.Once again, the definition of b-robustness can be trans-

posed, using finite differences V(X�

, f�

, K’, s–(�), �s).In practical situations, setting a material upper bound

b to the fragility is particularly important: one need tobe able to come with actual estimates of the impactof the error on the estimate of the left-semi-deviation.However, when dealing with certain class of models,such as Gaussian, exponential of stable distributions,we may be lead to consider asymptotic definitions ofrobustness, related to certain classes.

For instance, for a given decay exponent a > 0,assuming that f

�

(x) = O(eax) when x ! –1, the a-exponential asymptotic robustness of X

�

below the levelK is:

Rexp

(X�

, f�

,K, s�(�), a)

= max

K

06K

⇣ea(⌦�K

0)V (X

�

, f�

,K 0, s�(�))⌘

(26)

If one of the two quantities ea(⌦�K

0)f

�

(K 0) or

ea(⌦�K

0)V (X

�

, f�

,K 0, s�(�)) is not bounded from abovewhen K ! –1, then Rexp = +1 and X

�

is considered asnot a-exponentially robust.

Similarly, for a given power ↵ > 0, and assuming thatf�

(x) = O(x–↵) when x ! –1, the ↵-power asymptoticrobustness of X

�

below the level K is:

Rpow

(X�

, f�

,K, s�(�), a) =

max

K

06K

⇣(⌦�K 0

)

↵�2

V (X�

, f�

,K 0, s�(�))⌘


If one of the two quantities

(⌦�K 0)

↵f�

(K 0)

(⌦�K 0)

↵�2V (X�

, f�

,K 0, s�(�))

is not bounded from above when K 0 ! �1, thenR

pow

= +1 and X�

is considered as not ↵-power robust.Note the exponent ↵ – 2 used with the fragility, forhomogeneity reasons, e.g. in the case of stable distribu-tions, when a random variable Y

�

= '(X�

) depends onanother source of risk X

�

.Definition 1: Left-Robustness (monomodal distribu-

tion). A payoff y = '(x) is said (a, b)-robust belowL = '(K) for a source of randomness X with pdff�

assumed monomodal if, letting g�

be the pdf ofY = '(X), one has,for any K 0 K and L = '(K):

VX

�Y, g

�

, L0, s�(�)�6 aV

�X, f

�

, K 0, s�(�)�+ b (27)

The quantity b is of order deemed of “negligibleutility” (subjectively), that is, does not exceed sometolerance level in relation with the context, while a isa scaling parameter between variables X and Y.

Note that robustness is in effect impervious to changesof probability distributions. Also note that this measurerobustness ignores first order variations since owing totheir higher frequency, these are detected (and remedied)very early on.

The precautionary principle fragility and black swans from policy actions

Documents

Transcript of The precautionary principle fragility and black swans from policy actions