Monte Carlo AnalysisDavid M. Hassenzahl

Purpose of lectureIntroduce Monte Carlo Analysis as a tool for managing uncertaintyDemonstrate how it can be used in the policy settingDiscuss its uses and shortcomings, and how they are relevant to policy making processes

What is Monte Carlo Analysis?It is a tool for combining distributions, and thereby propagating more than just summary statisticsIt uses random number generation, rather than analytic calculationsIt is increasingly popular due to high speed personal computers

Background/HistoryMonte Carlo from the gambling town of the same name (no surprise)First applied in 1947 to model diffusion of neutrons through fissile materials Limited use because time consumingMuch more common since late 80sToo easy now?Nameis EPA gambling with peoples lives (anecdotal, but reasonable).

Why Perform Monte Carlo Analysis?Combining distributions With more than two distributions, solving analytically is very difficultSimple calculations lose information Mean mean = mean95% %ile 95%ile 95%ile!Gets worse with 3 or more distributions

Monte Carlo AnalysisTakes an equation example: Risk = probability consequenceInstead of simple numbers, draws randomly from defined distributionsMultiplies the two, stores the answerRepeats this over and over and overThen the set of results is displayed as a new, combined distribution

Simple (hypothetical) exampleSkin cream additive is an irritantMany samples of cream provide information on concentration:mean 0.02 mg chemicalstandard dev. 0.005 mg chemicalTwo tests show probability of irritation given applicationlow freq of effect per mg exposure = 5/100/mghigh freq of effect per mg exposure = 10/100/mg

Analytical resultsRisk = exposure potencyMean risk = 0.02 mg 0.075 / mg = 0.0015or 15 out of 10,000 applications will result in irritation

Analytical resultsConservative estimateUse upper 95th %ile

Risk = 0.03 mg 0.0975 / mg = 0.0029

Monte Carlo: Visual exampleExposure = normal(mean 0.02 mg, s.d. = 0.005 mg)potency = uniform (range 0.05 / mg to 0.10 / mg)

Random draw onep(irritate) = 0.0165 mg 0.063/mg = 0.0010

Random draw twop(irritate) = 0.0175 mg 0.089 /mg = 0.0016

Summary: {0.0010, 0.0016}

Random draw threep(irritate) = 0.152 mg 0.057 /mg = 0.0087

Summary: {0.0010, 0.0016, 0.00087}

Random draw fourp(irritate) = 0.0238 mg 0.085 /mg = 0.0020

Summary: {0.0010, 0.0016, 0.00087, 0.0020}

After ten random drawsSummary{0.0010, 0.0016, 0.00087, 0.0020, 0.0011, 0.0018, 0.0024, 0.0016, 0.0015, 0.00062}

mean 0.0014

standard deviation (0.00055)

Using softwareCould write this program using a random number generatorBut, several software packages out there.I use Crystal Balluser friendlycustomizabler.n.g. good up to about 10,000 iterations

100 iterations (about two seconds)Monte Carlo resultsMean0.0016Standard Deviation0.00048Conservative estimate0.0026Compare to analytical resultsMean 0.0015standard deviationn/aConservative estimate0.0029

Summary chart - 100 trials

Summary - 10,000 trialsMonte Carlo resultsMean0.0015Standard Deviation0.000472Conservative estimate0.0024Compare to analytical resultsMean 0.0015standard deviationn/aConservative estimate0.0029

Summary chart - 10,000 trialsAbout 1.5 minutes run time

Policy applicationsWhen there are many distributional inputsConcern about excessive conservatismmultiplying 95th percentilesmultiple exposuresBecause we canBayesian calculations

Issues: Sensitivity AnalysisSensitivity analysis looks at which input distributions have the greatest effect on the eventual distributionHelps to understand which parameters can both be influenced by policy and reduce risksHelps understand when better data can be most valuable (information isnt freenor even cheap)

Issues: CorrelationTwo distributions are correlated when a change in one causes a change in anotherExample: People who eat lots of peas may eat less broccoli (or may eat more)Usually doesnt have much effect unless significant correlation (||>0.75)

Generating DistributionsInvalid distributions create invalid results, which leads to inappropriate policiesTwo optionsempiricaltheoretical

Empirical DistributionsMost appropriate when developed for the issue at hand. Example: local fish consumptionsurvey individuals or otherwise estimatedata from individuals elsewhere may be very misleadingA number of very large data sets have been developed and published

Empirical DistributionsChallenge: when theres very little dataExample of two data pointsuniform distribution?triangular distribution?not a hypothetical issueis an ongoing debate in the literatureKey is to state clearly your assumptionsBetter yetdo it both ways!

Which Distribution?

Random number generationShouldnt be an issue@Risk and Crystal Ball are both good to at least 10,000 iterations10,000 iterations is typically enough, even with many input distributions

Theoretical DistributionsAppropriate when theres some mechanistic or probabilistic basisExample: small sample (say 50 test animals) establishes a binomial distributionLognormal distributions show up often in nature

Some CaveatsBeware believing that youve really understood uncertaintyBeware: misapplication ignorance at bestfraudulent at worstporcine hoof blister

Example (after Finkel)Alar versus aflatoxinExposure has two elementsPeanut butter consumptionaflatoxin residueJuice consumptionAlar/UDMH residuePotency has one elementaflatoxin potencyUDMH potencyRisk = (consumption residue potency)/body weight

Inputs for Alar & aflatoxin

Variable

Units

Mean

5th %ile

95th %ile

Percentile location of the mean.

Peanut butter consumption

g/day

11.38

2.00

31.86

66

Apple juice consumption

g/day

136.84

16.02

430.02

69

aflatoxin residue

(g/g

2.82

1.00

6.50

61

UDMH residue

(g/g

13.75

0.5

42.00

67

aflatoxin potency

kg-day/mg

17.5

4.02

28.23

61

UDMH potency

kg-day/mg

0.49

0.00

0.85

43

Alar and aflatoxin point estimatesaflatoxin estimates:Mean

= 0.028Conservative = 0.29Alar (UDMH) estimates:Mean = 0.046Conservative = 0.77

Alar and aflatoxin Monte Carlo10,000 runsGenerate distributions (dont allow 0)Dont expect correlation

Aflatoxin and Alar Monte Carlo results (point values)

Aflatoxin

Analytical

Monte Carlo

Mean

0.028

0.028

Conservative

0.29

0.095

Alar

Analytical

Monte Carlo

Mean

0.046

0.046

Conservative

0.77

0.18

Aflatoxin and Alar Monte Carlo results (distributions)

Aflatoxin and Alar Monte Carlo results (distributions)

Aflatoxin and Alar Monte Carlo results (distributions)

Aflatoxin and Alar Monte Carlo results (distributions)

Aflatoxin and Alar Monte Carlo results (distributions)

Aflatoxin and Alar Monte Carlo results (distributions)

References and Further ReadingBurmaster, D.E and Anderson, P.D. (1994). Principles of good practice for the use of Monte Carlo techniques in human health and ecological risk assessments. Risk Analysis 14(4):447-81Finkel, A (1995). Towards less misleading comparisons of uncertain risks: the example of aflatoxin and Alar. Environmental Health Perspectives 103(4):376-85.Kammen, D.M and Hassenzahl D.M. (1999). Should We Risk It? Exploring Environmental, Health and Technological Problem Solving. Princeton University Press, Princeton, NJ.Thompson, K. M., D. E. Burmaster, et al. (1992). "Monte Carlo techniques for uncertainty analysis in public health risk assessments." Risk Analysis 12(1): 53-63.Vose, David (1997) Monte Carlo Risk Analysis Modeling in Molak, Ed., Fundamentals of Risk Analysis and Risk Management.