Interval Analysis

Scott Ferson, scott@ramas.com11 September 2007, Stony Brook University, MAR 550, Challenger 165

Outline of Interval Analysis• What they are and why they’re used• Forward calculations

– Elementary formulas– Outward-directed rounding– Example: contaminant plume travel time– Repeated parameters– Tricks and subinterval reconstitution– Propagating uncertainty through black boxes – Dependencies among intervals

• Specifying intervals• Updating

– Aggregation– Constraint propagation

• Backcalculation• Exercises

An interval x = [x1, x2]

• Used when upper and lower bounds are known reliably• Sometimes absolutely sure, sometimes just a judgment

range

x1 x2

Why intervals are a good idea

• Intervals are easy to understand

• Calculations with them are easy

• They work with even the crappiest data

• Often sufficient for a decision

Rigorousness

• The computations are guaranteed to enclose the true results (so long as the inputs do)

• “Automatically verified calculations”

• You can still be wrong, but the method won’t be the reason if you are

General interval arithmetic

x + y = [x1 + y1, x2 + y2]x y = [x1 y2, x2 y1]x y = [min(x1y1, x1y2, x2y1, x2y2), max(x1y1, x1y2, x2y1, x2y2)]x y = x [1/y2, 1/y1], if y doesn’t include zero

Examples: x = [3, 5]y = [1, 2]x + y = [2, 7]x y = [1, 6]x y = [5, 10]

Other simple arithmetic operations

min(x, y) = [min(x1, y1), min(x2, y2)]

max(x, y) = [max(x1, y1), max(x2, y2)]

env(x, y) = [min(x1, y1), max(x2, y2)]

intersect(x, y) = [max(x1, y1), min(x2, y2)], if x and y overlap

Examples: x = [0.3, 0.5], y = [0.1, 0.2]

min(x,y) = [0.1, 0.2]

env(x, y) = [0.1, 0.5]

intersect(x, y) doesn’t exist

Other functions

• If the function is monotone (e.g., exp, log, sqrt) just apply it to the two endpoints

• Trickier if it isn’t monotone (e.g., abs, sin, cos)

• Note that generally non-monotone functions may be monotone over a particular interval

Examples: x = [0.3, 0.5]exp(x) = [1.34, 1.65]sin(x) = [0.295, 0.480]abs(x) = [0.3, 0.5]

Outward-directed rounding

• To maintain the rigor of interval calculations, any necessary truncation of digits should be done with outward-directed rounding

Example:

[2,4] / 3 = [ 0.66666666666, 1.33333333333]

would be written as [0.66, 1.34]

even though 2/3 ordinarily rounds to 0.67,

and 4/3 ordinarily rounds to 1.33

_ _

Example: contaminant plume

• Hydrocarbon in groundwater near some wells• Constant, one-dimensional, uniform Darcian flow• Homogeneous properties (e.g., no pipes, conduits,

barriers or differential permeability among layers)• Linear retardation• No dispersion (just Darcian advection)

• How long before the contaminant reaches the wells?

Darcy’s Law

• Statistical description of fluid flow in porous media • Expresses conservation of momentum• Flow rate is proportional to

– cross-sectional area

– hydraulic conductivity and

– gradient in the hydraulic head

• Analogous to– Fick’s Law in diffusion

– Ohm’s Law (I=V/R)

• Derivable from Navier-Stokes by homogenization

Travel time (Lobascio)

iK

LKocfocBDnT

ParameterL source-receptor distancei hydraulic gradientK hydraulic conductivityn effective soil porosityBD soil bulk densityfoc fraction organic carbonKoc organic partition coefficient

Min800.00033000.215000.00015

Max1200.000830000.3517500.00520

Mean1000.0005510000.2516500.0025510

Stdv11.550.00014437500.051000.0014153

Unitsmm/mm/yr-kg/m3

-m3/kg

Results

• The most optimistic outcome would be that the plume takes ___ years to reach the well

• But, in the worst case, it could take __ years to reach the well

Answers: 233800 years, 31.6 years

[31.6, 233800] years

• Is ‘6’ the last decimal digit of the lower bound?

• Did you check that the units balance?

• Did you include units in the answer?

How to understand this result

• Highly reliable result, given the assumptions– Can’t get any worse

• Represents parametric uncertainty– Neglects (possibly big) model uncertainty

• Expresses only best and worst cases– How likely is 32 years? 50 years? 100 years?

Lobascio’s original formulation

Kd= foc Koc = [ 0.0005, 0.1] m3 kg-1

R = 1 + BD Kd / n = [ 3.143, 876] V = K i / (n R) = [ 0.000293, 3.82] m yr-1

T = L/V = [ 20.95, 408800] yr

Quickest plume reaches the well = 20.95 yrLongest plume reaches the well = 408,800 yr

What explains the difference? (hint: n is repeated above)

Repeated parameters

a = [1,2]b = [2,3]c = [2, 5]

z = a × (b + c) zz = a × b + a × c

b + c = [0, 8] a × b = [2, 6]z = [0, 16] a × c = [4, 10] zz = [2, 16]

inflated uncertainty

What to do about repeated parameters

• Always rigorous, but maybe not best possible when uncertain parameters are repeated

• Inconsequential if all are non-negative and all operations are increasing (+, ×, but not – or ÷)

• Use cancellation to reduce repetitions, e.g., caia/m + cwiw/m + cdid/m = (caia + cwiw + cdid)/m

• Cancellation not always possible, e.g., (a + b) / (a + c) = ??

If you can’t cancel

• Use tricks with algebrae.g., a² + a = (a +½)² – ¼

• Employ subinterval reconstitutionA brute-force and computationally intensive strategy

Workable if there aren’t too many repeated parameters

• Live with the suboptimalityDecisions may not require perfect precision

TricksBasic identities

u + 0 = u

u – 0 = u

0 – u = –u

u 0 = 0

u 1 = u

u / 1 = u

u0 = 1

u1 = u

u & 1 = u

u | 1 = 1

u & 0 = 0

u | 0 = u

u & u = u

u | u = u

u & not(u) = 0

u | not(u) = 1

(u&a) | (u&b) = u&(a | b)

(u | a)&(u | b) = u | (a&b)

etc.

One repetitionu + u = 2uu – u = 0u u = u2

u / u = 1(1+u) / u = 1/u + 1(1+u)/(1–u) = (1/tan(acos(u)/2))2

au + bu = u(a + b)au – bu = u(a – b)a/u + b/u = (a + b) / ua/u – b/u = (a – b) / uu/a + u/b = u(b + a)/(ab)u/a – u/b = u(b – a)/(ab)aub + cub = (a + c) ub

aub cud = a c u(b + d)

au bu = exp(u (ln(a) + ln(b)))u2 + u = (u + ½)2 – ¼u2 – u = –¼ sin(2 asin(sqrt(u)))2

u2 + au = (u + a/(2))2 – a2/4etc.

u, v, etc. represent the uncertain numbers a, b, etc. represent arbitrary expressions

Two repetitionsu + v – uv = 1 – (1 – u) (1 – v)(u + v) / (1 – uv) = tan(atan(u) + atan(v))(u – v) / (1 + uv) = tan(atan(u) – atan(v))(1 + uv) / (u – v) = 1 / tan(atan(u) – atan(v))(1 – uv) / (u + v) = 1 / tan(atan(u) + atan(v))(uv – 1) / (u + v) = –1 / tan(atan(u) + atan(v))u sqrt(1 – v2) + v sqrt(1 – u2) = sin(asin(u) + asin(v))u sqrt(1 – v2) – v sqrt(1 – u2) = sin(asin(u) – asin(v))u v + sqrt(1 – u2) sqrt(1 – v2) = cos(acos(u) – acos(v))u v – sqrt((1 – u2) (1 – v2)) = cos(acos(u) + acos(v))u v – sqrt(1 – u2 – v2 + u2 v2) = cos(acos(u) + acos(v))sin(u) sqrt(1 – sin(v)2) + sin(v) sqrt(1 – sin(u)2) = sin(u + v)cos(u) cos(v) – sin(u) sin(v) = cos(u + v)sin(u) cos(v) – cos(u) sin(v) = sin(u – v)sqrt((1 + u) / (1 – u)) = 1 / tan(acos(u)/2)etc.

Subinterval reconstitution

• Partition each repeated interval into subintervals• Compute the function for every subinterval• The union of all the results contains the true range

i j k

kji zyxwvufzyxwvuf ),...,,,,...,,(...),...,,,...,,(

i j k

kji wwvvuu ...;;;

where u, v, …, w are repeated intervals and x, y,…, z are other interval and scalar inputs, and

Example: (a + b)a, a = [0.1, 1], b = [0,1]

Partition the repeated uncertain a:ai = [(i 1)w/m + a, iw/m + a] where i = 1,2,…, m, and m is the number of subintervals, w is thewidth of a, and a is its lower bound

m U(ai+b)ai

1 [ 0.1, 2]2 [ 0.282, 2]3 [ 0.398, 2]4 [ 0.473, 2]5 [ 0.525, 2]10 [ 0.624, 2]100 [ 0.686, 2]1,000 [ 0.692, 2]10,000 [ 0.692, 2]a

0 0.5 1

m U(ai+b)ai

1 [ 0.1, 2]2 [ 0.282, 2]3 [ 0.398, 2]4 [ 0.473, 2]5 [ 0.525, 2]10 [ 0.624, 2]100 [ 0.686, 2]1,000 [ 0.692, 2]10,000 [ 0.692, 2]

Cauchy-deviate method

• Propagates intervals through black-box model– Don’t need to know, but have to be able to query it

• “Sample” from around interval – Points not necessarily inside the interval!

• Scale results to get an asymptotically correct estimate of the interval uncertainty of the output

(Trejo and Kreinovich 2001)

Cauchy-deviate method

• Depends on the number of samples, not inputs– Works just as well for 2000 variables as 20– Similar in performance to Monte Carlo

• Need about 200 samples to obtain 20% relative accuracy of half-width of output range– With fewer samples, we’d get lower accuracy, but

we can compensate for this by scaling by N, which works under the linearity assumption

Limitations of the method

• Asymptotically correct, but not rigorous

• Intervals narrow relative to the nonlinearity– Function almost linear OR uncertainties small– Could combine with subinterval reconstitution

• Most efficient when dimensionality is high

• Only handles interval uncertainty

Dependence among intervals

Dependence in interval analysis

• Interval analysis automatically accounts for all possible dependencies

• Information about dependencies isn’t usually used to tighten results, but it could be

Dependencies

• Any restriction on the possible pairings of inputs (any subset of the unit square) May also require each value of u to match with at least v, and vice versa

• The null restriction is the full unit square“Nondependence” or “noninteractivity” rather than “independence”

• D denotes the set of all possible dependencies Set of all subsets (power set) of the unit square D = 2[0,1][0,1]

Three special cases

0 u 1 0 u 1 0 u 1

1

v

0

1

v

0

1

v

0

Perfect(comonotonic)

Nondependent(the Fréchet case)

Opposite(countermonotonic)

Correlation

• A model of dependence that’s parameterized by a (scalar) value called the “correlation coefficient”

: [1, +1] D = 2[0,1][0,1]

• A correlation model is called “complete” if

(1) = , (0) = , (+1) =

r = 1 r = 0 r = +1

Corner-shaving dependence

D(r) = { (u,v) : max(0, ur, u1+r) v min(1, u+1r, u+2+r)} u [0,1], v [0,1] 

f (A, B) = { c : c = f (u (a2 – a1) + a1, v (b2 – b1) + b1), (u,v) D }A+B = [env(w(A, r)+b1, a1+w(B,r)), env(a2+w(B,1+r),w(A,1+r)+b2)]

a1 if p < 0w([a1,a2], p) = a2 if 1 < p p(a2a1)+a1 otherwise

Other complete correlation familiesr = 1 r = 0 r = +1

Elliptic dependence

• Not complete (because r = 0 isn’t nondependence)

A+B = [d, d] + (a1 + a2 + b1 + b2) / 2

d = (d12 + d2

2 + r d1 d2), d1 = (a2 a1)/2, d2 = (b2 b1)/2

r = 1 r = 0 r = +1

A+B = [d, d] + (a1 + a2 + b1 + b2) / 2

d = (d12 + d2

2 + r d1 d2), d1 = (a2 a1)/2, d2 = (b2 b1)/2

[ 5, 14] Perfect

[ 8, 11] Opposite

[ 7.1, 11.9] Corner-shaving (r = 0.7)

[ 7.27, 11.73] Elliptic (r = 0.7)

[ 5, 14] Upper, left

[ 5, 11] Lower, left

[ 8, 14] Upper, right

[ 5, 14] Lower, right

[ 6.5, 12.5] Diamond

[ 5, 14] Nondependent

A + B

A = [2,5]B = [3,9]

Implications

• Intervals aren’t more widely used is because they’re thought to be (and often are) too wide to be helpful

• They can be much tighter if we used all available information, such as about intervariable dependence

• (Other information, like about moments, could also be used to tighten interval calculations)

Computing

• Sequence of binary operations– Need to deduce dependencies of intermediate results

with each other and the original inputs– Different calculation order can give different results

(which should be intersected)

• Do all at once in one multivariate calculation – Can be much more difficult computationally– Can produce much better tightening

Specifying input intervals

Interval uncertainty

• Statisticians often ignore this uncertainty

• “Interval uncertainty doesn’t exist in real life”(Tony O’Hagan et

al.)

Hammer salesmen saying screws don’t exist?

When do intervals arise?

• Periodic observationsWhen did the fish in my aquarium die during the night?

• Plus-or-minus measurement uncertaintiesCoarse measurements, measurements from digital readouts

• Non-detects and data censoringChemical detection limits, studies prematurely terminated

• Privacy requirementsEpidemiological or medical information, census data

• Theoretical constraintsConcentrations, solubilities, probabilities, survival rates

• Bounding studies Presumed or hypothetical limits in what-if calculations

Ways to characterize intervals

• Theoretical constraints

• Modeled from other intervals

• Expert assertions

• Discounting (widening) intervals (Shlyakhter)

• Confidence procedures (Grosof)– But 95% confidence isn’t the same as surety– Use in interval calculations requires an assumption

Problems with confidence intervals

• Cannot be combined in arithmetic or logical operations without an assumption

• Don’t measure epistemic belief anyway

Example (Walley): For instance, a 90% confidence interval could have zero chance of containing the value. For example, suppose X ~ normal(, 1), where 0 < . If the sample mean happens by chance to be 1.3, the 90% confidence interval on the mean is the empty set (Walley, P. 1991, Statistical Reasoning with Imprecise Probabilities, Chapman & Hall, page 379).

Why we have to be careful

• Interval analysis yields contingent results• Results are contingent on assumptions that model inputs are within

their respective intervals• But all analysis results are contingent on similar assumptions that the

models they came from are true

• Naïve elicitation has big problems• Intervals are usually unrealistically narrow• People make incoherent statements

• Can’t mix together different kinds• Not clear how to translate data into intervals

Determining endpoints

• The largest observed may not be the largest possible (and it usually isn’t)

• Sampling theory >> theory of extremes

• Rigor of analysis is contingent on inputs

• If you’re nervous, just widen the bounds

Point sample data

Range (envelope)

Extreme value model

Tolerance interval

Confidence interval

Central value and width

P-box

Level cutcut

Model

Output rangesimulation, etc.Plus-minus interval

Shlyakhter widening

Envelope

Interval function

Distribution

Support

Percentile range

Trejo-KreinovichCauchy deviates

Tolerance solutionbackcalculation

Certain and tenable ranges

Intersection

Credibility interval

Prediction interval

Percentile range

Support

Eliciting dependence

• As hard as getting intervals (maybe a bit worse)

• Theoretical or “physics-based” arguments

• Inference from empirical data – Risk of loss of rigor at this step (just as there is

when we try to infer intervals from data)

Updating

Aggregation (updating)

• How do you combine different sources?

• If you trust them all, take the intersection– [max(x1, y1, z1, …), min(x2, y2, z2, …)]– What if there is no intersection (right<left)?

• If you’re not sure which is right, use the envelope– [min(x1, y1, z1, …), max(x2, y2, z2, …)]– But are you sure this is wide enough?

Example

• Suppose we have two rigorous interval estimates of the same quantity: [1,7] & [4,10]

• Their intersection [4,7] is also a rigorous interval for the quantity

AB

0 5 10

Constraint analysis (updating)

• Using knowledge of how variables are related to tighten their estimates

• Removes internal inconsistency and explicates unrecognized knowledge

• Also called ‘constraint updating’ or ‘editing’

• Also called ‘natural extension’

Example

• Suppose we know

W = [23, 33] m

H = [112, 150] m

A = [2000, 3200] m2

• Does knowing W H = A let us to say any more?

Answer

• Yes! We can infer that

W = [23, 28.57]

H = [112, 139.13]

A = [2576, 3200]

• The formulas are just W = intersect(W, A/H), etc.

To get the largest possible W, for instance, let A be as large as possible and H as small as possible, and solve for W =A/H.

Updating with p-boxes

2000 3000 40000

1

20 30 400

1

120 140 1600

1 A W H

2000 3000 40000

1

20 30 400

1

120 140 1600

1

Answers

intersect(W, A/H) intersect(H, A/W) intersect(A, WH)

A W H

Bayesian strategy

20003200

])3200,2000[(

112150

])150,112[(

2333

])33,23[(),,Pr(

AIHIWI

AHW

)(),,|( HWAAHWHWAL

),,Pr()()|,,( AHWHWAHWAAHWf

Prior

Likelihood

Posterior

Bayes’ rule

• Concentrates mass onto the manifold of feasible combinations of W, H, and A

• Answers have the same supports as intervals

• Computationally complex• Needs specification of priors• Yields distributions that are not justified

(coming from the choice of priors)• Expresses less uncertainty than is present

Backcalculation

Backcalculation

• Needed for cleanup and remediation planning

• Untangles an equation in uncertain numbers when we know all but one of the variables

• For instance, backcalculation finds B such that A+B = C, from estimates for A and C

Can’t just invert the equation

Dose = Concentration × Intake

Concentration = Dose / Intake

When concentration is put back into the forward equation, the resulting dose is wider than planned

prescribed knownunknown

Exampledose = [0, 2] milligram per kilogramintake = [1, 2.5] litermass = [60, 96] kilogram

conc = dose * mass / intake [ 0, 192] milligram liter-1

dose = conc * intake / mass [ 0, 8] milligram kilogram-1

Doses four times larger than tolerable levels we planned

Untangling backcalculations

• Solving for B given A + B = C

B = backcalc(A, C) = [C1 A1, C2 A2]

• Solving for B given A B = C

B = factor(A, C) = [C1 / A1, C2 / A2]

• Sometimes called “tolerance solutions”

1

2

3

4

5

6

1 1.5 2A

Bsh

ell

Kernal versus shell

Given A [1,2] C [2,6] C A BThere are two different ways to solve for B

kern

el

Shell (united solution)B C A [0,5]

Kernel (tolerance solution)B backcalc(A,C) [1,4]

When you need for

A + B C

A – B C

A B C

A / B C

A ^ B C

2A C

A² C

And you have estimates for

A, BA, CB ,CA, BA, CB ,CA, BA, CB ,CA, BA, CB ,CA, BA, CB ,C

ACAC

Use this formulato find the unknownC = A + BB = backcalc(A,C)A = backcalc (B,C)C = A – BB = –backcalc(A,C)A = backcalc(–B,C)C = A * BB = factor(A,C)A = factor(B,C)C = A / BB = 1/factor(A,C)A = factor(1/B,C)C = A ^ BB = factor(log A, log C)A = exp(factor(B, log C))C = 2 * AA = C / 2C = A ^ 2A = sqrt(C)

Interval algebra

• Commutativity a+b=b+a, a×b=b×a• Associativity (a+b)+c=a+(b+c), (a×b)×c=a×(b×c)• Neutral elements a+0=0+a=a, a×1=1×a=a

• Subdistributivity a×(b+c) a×b+a×c• Subcancellation a a+bb, a a×b/b• No inverse elements a+( a) 0, a×(1/a) 1

Conclusions

• Interval analysis is a worst case analysis (that also includes the best case)

• Repeated uncertain parameters can cause unnecessary inflation of uncertainy

• Results will always be rigorous, but might not be best possible

• Moving a uncertain parameter to the other side of an equal sign often requires backcalculation

Exercises

1. Do the inputs in the travel time example seem dependent?2. What does subinterval reconstitution with m=100 on the

original Lobascio formulation give for the travel time?3. What contaminant concentrations C in water will lead to

doses D no larger than 6 mg per kg per day if it comes from both drinking and eating fish as D = (Iwater C) / W + (Ifish B C) / W, where

Iwater = [1.5, 2.5] liters per day // water intake

Ifish = [0, 8] g per day // dietary ingestion of fishB = [0.9, 2.1] liters per mg // bioaccumulation factorW = [60, 90] kg // receptor biomass

How do you check the solution?4. Is there a Bayesian analog of backcalculation?

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////Answer to #1

L source-receptor distancei hydraulic gradientK hydraulic conductivityn effective soil porosityBD soil bulk densityfoc fraction organic carbonKoc organic partition coefficient

Porosity and conductivity would likely be positively relatedPorosity and density would likely be inversely relatedThus, conductivity and density should be inversely related tooFraction organic carbon and the partition coefficient would likely be positively related

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////Answer to #2

L = [80,120] m // source-receptor distancei = [0.0003,0.0008] m per m // hydraulic gradientK = [300,3000] m per yr // hydraulic conductivityn = [0.2,0.35] // effective soil porosityBD = [1500,1750] kg per m{3} // soil bulk densityfoc = [0.0001,0.005] // fraction organic carbonKoc = [5 ,20] m{3} per kg // organic partition coefficient Kd = foc * KocR = 1 + BD * Kd / n V = K * i / (n * R)T = L/V T [ 20.952380952, 408800] yr

func TT() return L/(K * i / ($1 * (1 + BD * foc * Koc / $1)))

b = left(n)e = TT(b)w = width(n)/100for k=1 to 100 do begin s = [ (k-1), k] * w + b e = env(e, TT(s)) ende [ 31.480562448, 235352] yr

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////Answer to #3

Iwater = [1.5, 2.5] liters per day // water intakeIfish = [0, 8] g per day // dietary ingestion of fish tissueB = [0.9, 2.1] liters per g // bioaccumulation factorW = [60, 90] kg // receptor biomass

D = [0, 6] mg per kg per day // tolerable dose

// forward equation// D = (Iwater * C) / W + (Ifish * B * C) / W

// condense repeated parameters// D = C * (Iwater + Ifish * B) / W

// solve for C by factoring the quotient (Iwater+Ifish*B)/W out of DC = factor((Iwater + Ifish * B) / W, D) C [ 0, 18.652849741] mg liters-1

// put C back into original forward equation (and ensure correct units)d = (Iwater * C) / W + (Ifish * C * B) / W + 0 mg per kg per day

d [ 0, 6] mg kg-1 day-1

(break)

Interval probability

Outline for Interval Probability

• Logical operations under independence• Dependence among events

– Perfect and opposite dependence– Frank and Pearson correlation– Fréchet inequalities– Signed dependence

• Example: rupture of a pump system• Exercises• Conclusions• References

Interval arithmetic for probabilities

x + y = [x1, x2] + [y1, y2] = [x1 + y1, x2 + y2]

x y = [x1, x2] [y1, y2] = [x1 y2, x2 y1]

x y = [x1, x2] [y1, y2] = [x1 y1, x2 y2]

x y = [x1, x2] [y1, y2] = [x1 y2, x2 y1]

Examples: x = [0.3, 0.5]y = [0.1, 0.2]x + y = [0.4, 0.7]x y = [0.1, 0.4]

Rules are simpler because intervals confined to [0,1]

Probabilistic logic

• Conjunctions (and)

• Disjunctions (or)

• Negations (not)

• Exclusive disjunctions (xor)

• Modus ponens (if-then)

• etc.

Conjunction (and)

P(A |&| B) = P(A) P(B)

Example: P(A) = [0.3, 0.5]

P(B) = [0.1, 0.2]

A and B are independent

P(A |&| B) = [0.03, 0.1]

Disjunction (or)

P(A || B) = = 1 (1 P(A))(1 P(B))

Example: P(A) = [0.3, 0.5]

P(B) = [0.1, 0.2]

A and B are independent

P(A || B) = [0.37, 0.6]

Disjunction (or)

P(A || B) = P(A) + P(B) P(A) P(B)

Example: P(A) = [0.3, 0.5]

P(B) = [0.1, 0.2]

A and B are independent

P(A || B) = [0.3, 0.67]

repeatedparameters!

Negation (not)

P(not A) = 1 P(A)

Example: P(A) = [0.3, 0.5]

P(not A) = [0.5, 0.7]

1 2 3 4 5

Dependence

Independent

Probabilities are depicted hereas the areas in Venn diagrams

Perfect Opposite

Perfect dependence

P(A /&/ B) = min(P(A), P(B)) P(A // B) = max(P(A), P(B))

Examples: P(A) = [0.3, 0.5]P(B) = [0.1, 0.2]A and B are perfectly dependentP(A /&/ B) = [0.1, 0.2]P(A // B) = [0.3, 0.5]

Opposite dependence

P(A \&\ B) = max(P(A) + P(B) 1, 0)

P(A \\ B) = min(1, P(A) + P(B))

Examples: P(A) = [0.3, 0.5]

P(B) = [0.1, 0.2]

A and B are oppositely dependent

P(A \&\ B) = 0

P(A \\ B) = [0.4, 0.7]certain

Correlated events (Frank)

where r is the correlation, s = tan((1r)/4), a = P(A), b = P(B)

otherwise

1 if

0 if

1 if

)1/()1)(1(1log

)0,1max(

),min(

),,(and)&P(

s

Frank

r

r

r

sss

ba

ab

ba

rbaBA

ba

)&P( BAr

otherwise.

1 if

0 if

1 if

)1/()1)(1(1log-1

)1,min(

)1)(1(1

),max(

),,(or) P(

11s

Frank

r

r

r

sss

ba

ba

ba

rbaBA

ba

) ∨P( BA r

0

0.1

0.2

0.3

0.4

0.5

0.6

1 0.75 0.5 0.25 0 0.25 0.5 0.75 1

Frank correlation coefficient, r

Pro

babi

lity

orFrank(0.29, 0.22, r)

andFrank(0.29, 0.22, r)

Examples of Frank correlations

Other correlation models

• Pearson correlation (on indicator functions)

–

– doesn’t allow all possible correlation coefficients– permits negative and larger-than-one probabilities

• But any complete copula would work fine– Frank, Clayton, Mardia, etc.– Nelsen (1999) is the essential reference on copulas

P(A & B) = [max(0,P(A)+P(B)–1), min(P(A), P(B))]

P(A or B) = [max(P(A), P(B)), min(1, P(A)+ P(B))]

• Makes no assumption about the dependence

• Rigorous (guaranteed to enclose true value)

• Best possible (cannot be any tighter)

Fréchet case

The proofs are elementaryP(A  B) = P(A) + P(B) P(A & B) implies P(A) + P(B) P(A  B) = P(A & B).

P(A  B) 1, since probabilities are no bigger than 1, so P(A) + P(B) 1 P(A & B).

0 P(A & B), since probabilities are positive, so max(0, P(A) + P(B) 1) P(A & B).

This gives the lower bound on the conjunction.

To get the upper bound, recall that P(A & B) = P(A|B) P(B) = P(B|A) P(A).

P(A|B) 1 and P(B|A) 1, so P(A & B) P(A) and P(A & B) P(B).

Therefore, P(A & B) min(P(A), P(B)), which is the upper bound.

The best-possible nature of these bounds follows from observing that

they are realized by some dependency between the events A and B.

Comparable bounds on the disjunction are similarly derived.

Fréchet examples

Examples: P(A) = [0.3, 0.5]P(B) = [0.1, 0.2]P(A & B) = [0, 0.2]P(A B) = [0.3, 0.7]

P(C) = 0.29P(D) = 0.22P(C & D) = [0, 0.22]P(C D) = [0.29, 0.51]

}certain

}uncertain

Sign of the dependence

P(A &+ B) = [ab, min(a,b)]

P(A + B) = [1(1a)(1b), max(a, b)]

P(A & B) = [max(a+b1, 0), ab]

P(A B) = [1(1a)(1b), min(1, a+b)]

where a = P(A), b = P(B)

RelayK2

Pressureswitch S

Timerrelay R

Pump

Motor

RelayK1

Fromreservoir

Pressure tank T Outletvalve

SwitchS1

S tartupSw itc h S1 c lo sedR e lay K1 c lo sedR e lay K2 c lo sedT im e r re la y c lo sedP re ssu re sw it ch c lo sed

Start pum pR e lay K2 c lo sedT im e r re la y s ta r tsP re ssu re sw it ch c lo sed

ReadyS w itch S1 o p enR e lay K1 c lo sedR ela y K2 o p enT im e r re la y c lo sedPre ssure sw itch o p en

System offS w itch S1 o p enR ela y K1 o p enR ela y K2 o p enT im e r re la y c lo sedP re ssu re sw it ch c lo sed

Shut downR ela y K1 o p enR ela y K2 o p enT im er re la y o p enP re ssu re sw it ch c lo sed

P um pingS w itch S1 o p enR e lay K1 c lo sedR e lay K2 c lo sedT im e r re la y c lo sedP re ssu re sw it ch c lo sed

S top pum pR ela y K2 o p enT im er re lay re se tsPre ssure sw itch o p en

Example: pump system

What’s the chance thetank ruptures under pumping?

E1 = “tank rupturing under pressurization”

Fault tree

Vesely et al. (1981)

E1

T

K2

E2

E3

S

E4

S1

E5

K1

or and

or

or

or

R

Boolean expression

E1 = T (K2 (S & (S1 (K1 R))))

ComponentPressure tank T Relay K2 Pressure switch S Relay K1 Timer relay R On-switch S1 

Probability5 106

3 105

1 104

3 105

1 104

3 105

Different dependency models

Vesely et al. (all variables precise, independent)E1 = T || (K2 || (S |&| (S1 || (K1 || R))))

Mixed dependencies

E1 = T || (K2 (S &r (S1 || (K1 // R))))Frank to Fréchet

E1 = T || (K2 (S & (S1 || (K1 // R))))All Fréchet

E1 = T (K2 (S & (S1 (K1 R))))

Interval probabilities

ComponentPressure tank T Relay K2 Pressure switch S Relay K1 Timer relay R On-switch S1 

Probability interval[4.5 106, 5.5 106][2.5 105, 3.5 105][0.5 104, 1.5 104][2.5 105, 3.5 105][0.5 104, 1.5 104][2.5 105, 3.5 105]

105 104 103

Probability of tank rupturing under pumping

Mixed dependencies

Frank to Fréchet

All Fréchet

Mixed dependence with intervals

Vesely et al. (all independent)

Results

3.5105

[3.499105, 3.504105]

[3.50105, 1.35104]

[3105, 1.4104]

[2.9105, 4.1105]

Strategies to handle repetitions

• Interval analysis is always rigorous, but maybe not best possible when parameters are repeated

• Use cancellation to reduce repetitions, e.g.,(A & B) (A & C) (A & D) = A & (B C D)

• When cancellation is not possible, mathematical programming may be necessary to get best possible result

Subtle dependencies

• May also require mathematical programming to obtain the best possible result

• But rigorous bounds are always easy to get with the artful use of the Fréchet case

Advances in practices

“It practice, independence among events and event probabilities is often assumed for the sake of convenience because it makes otherwise intractable problems easier to solve.” (Vick 2002, page 412)

“…Independent and conditional relationships [among events] could not be established without the subjective elements of one’s state of knowledge, making it otherwise all but impossible to apply the [probability] calculus to most situations of practical importance.” (ibid.)

Conclusions

• Easy to compute rigorous bounds

• Mathematical programming may be needed to get answers that are also best possible

• Rigor of analysis is contingent on inputs

• If you’re nervous, just widen the bounds

Exercises

5. Calculate the probability of tank rupture under pumping that assumes the interval inputs and makes no assumption about the dependencies among the events.

6. Develop an fault tree for establishment of snake populations on a Hawaiian island (or a star exploding).

7. Compute the probability of the conjunction of two events having probabilities 0.29 and 0.22, assuming a Pearson correlation of 1.0. Compare the result to the Fréchet range for such probabilities. What’s going on?

8. Derive an algorithm to compute the probability that n of k events occur, given intervals for the probability of each event, assuming they’re independent. Derive an analogous algorithm for the Fréchet case.

Vesely et al. (all independent)3.5105

[3.499105, 3.504105]

[3.50105, 1.35104]

[3105, 1.4104]

[2.9105, 4.1105]

[ 2.5105, 1.905 104]

Snakes on a plane (or boat)Introduction

Stowaway on plane OR Stowaway on ship OR Arrives on natural raft OR Brought as pet OR Brought for zoo

AND Escape and ReproductionPregnant snake escapes OR Male and female escape AND Successful reproduction OR Female escapes AND Parthenogenic reproduction

AND Survives predationDomestic and feral dogs ANDFeral cats ANDWild pigs

AND Prey is accessibleGround nesting bird eggsOR Rodents

AND Survives eradication programEludes detection by biologistsOR Reproduction too quickOR Dispersal too quick

Overall conclusions

Why bounding?

• Often sufficient to specify a decision • Possible even when estimates are impossible• Usually easy to compute and simple to combine• Rigorous, rather than an approximation

(after N.C. Rowe 1988)

Reasons to use interval analysis

• Requires very little data• Applicable to all kinds of uncertainty• Can be comprehensive• Fast and easy to compute answers• Conservative when correlations unknown • Can be made “best possible”• Backcalculations easy• Updating relatively easy

Reasons not to use it

• Same thing as worst case analysis

• Doesn't say how likely extreme event is

• Repeated parameters are cumbersome

• Not optimal when there’s a lot of data

• Can't use distribution information

• Doesn't use correlation information

Interval (worst case ) analysisHow?

– bound inputs, a = [a1, a2], where a1 a2

– addition: [a1, a2] + [b1, b2] = [a1+b1, a2+b2]– subtraction: [a1, a2] – [b1, b2] = [a1–b2, a2–b1]– multiplication, division, etc. are a little more complex

Why?– natural for scientists and easy to explain to others– works no matter where uncertainty comes from

Why not?– paradoxical: can’t give exact value but can give exact bounds– ranges could grow quickly, yielding very wide results– doesn’t give probabilities of extreme outcomes (tail risks)

Interval probabilityHow?

– bound event probabilities, p = [p1, p2], where 0 p1 p2 1

– evaluate event trees as composition of ANDs, ORs, etc.

– standard probabilistic rules if events are independent

– Fréchet rules if their dependence is unknown

– other dependency relations can also be represented

Why?

– can capture incertitude about event probabilities

Why not?

– paradoxical: can’t give exact value but can give exact bounds

– ranges can grow quickly, especially without independence

ReferencesDwyer, P. 1951. Linear Computations. John Wiley & Sons, New York.Ferson, S. 2002. RAMAS Risk Calc 4.0: Risk Assessment with Uncertain Numbers. Lewis Publishers, Boca Raton.Grosof, B.N. 1986. An inequality paradigm for probabilistic knowledge: the logic of conditional probability intervals.

Uncertainty in Artificial Intelligence. L.N. Kanal and J.F. Lemmer (eds.), Elsevier Science Publishers, Amsterdam.Hailperin, T. 1986. Boole’s Logic and Probability. North-Holland, Amsterdam.Kyburg, H.E., Jr. 1998. “Interval Valued Probabilities,” http://ippserv.rug.ac.be/documentation/interval_prob/interval_prob.html,

The Imprecise Probabilities Project, edited by G. de Cooman and P. Walley, http://ippserv.rug.ac.be/home/ipp.html.Lobascio, M.C. 1993. Uncertainty analysis tools for environmental modeling: application of Crystal Ball® to predict

groundwater plume traveling times. ENVIRONews 1: 6-10.Loui, R.P. 1986. Interval based decisions for reasoning systems. Uncertainty in Artificial Intelligence. L.N. Kanal and

J.F. Lemmer (eds.), Elsevier Science Publishers, Amsterdam.Moore, R.E. 1966. Interval Analysis. Prentice-Hall, Englewood Cliffs, New Jersey.Moore, R. 1979. Methods and Applications of Interval Analysis. SIAM, Philadelphia.Rowe, N.C. 1988. Absolute bounds on the mean and standard deviation of transformed data for constant-sign-derivative

transformations. SIAM Journal of Scientific Statistical Computing 9: 1098–1113. Shlyakhter A. 1994. Improved framework for uncertainty analysis: accounting for unsuspected errors. Risk Analysis

14(4):441-447.Tessem, B. 1992. Interval probability propagation. International Journal of Approximate Reasoning 7: 95-120.Trejo, R. and V. Kreinovich. 2001. Error estimations for indirect measurements: randomized vs. deterministic

algorithms for ‘black-box’ programs. Handbook on Randomized Computing, S. Rajasekaran, P. Pardalos, J. Reif, and J. Rolim (eds.), Kluwer, 673–729. http://www.cs.utep.edu/vladik/2000/tr00-17.pdf

Vesely, W.E., F.F. Goldberg, N.H. Roberts, D.F. Haasl. 1981. Fault Tree Handbook. Nuclear Regulatory Commission, Washington, DC.

Vick, S.G. 2002. Degrees of Belief: Subjective Probability and Engineering Judgment. ASCE Press, Reston, Virginia.

End

For next time

• Do exercise #1 and either #2 or #3

• Try to do one of the exercises #5, #6 or #7