Monte Carlo AnalysisDavid M. Hassenzahl
Purpose of lectureIntroduce Monte Carlo Analysis as a tool for managing uncertaintyDemonstrate how it can be used in the policy settingDiscuss its uses and shortcomings, and how they are relevant to policy making processes
What is Monte Carlo Analysis?It is a tool for combining distributions, and thereby propagating more than just summary statisticsIt uses random number generation, rather than analytic calculationsIt is increasingly popular due to high speed personal computers
Background/HistoryMonte Carlo from the gambling town of the same name (no surprise)First applied in 1947 to model diffusion of neutrons through fissile materials Limited use because time consumingMuch more common since late 80sToo easy now?Nameis EPA gambling with peoples lives (anecdotal, but reasonable).
Why Perform Monte Carlo Analysis?Combining distributions With more than two distributions, solving analytically is very difficultSimple calculations lose information Mean mean = mean95% %ile 95%ile 95%ile!Gets worse with 3 or more distributions
Monte Carlo AnalysisTakes an equation example: Risk = probability consequenceInstead of simple numbers, draws randomly from defined distributionsMultiplies the two, stores the answerRepeats this over and over and overThen the set of results is displayed as a new, combined distribution
Simple (hypothetical) exampleSkin cream additive is an irritantMany samples of cream provide information on concentration:mean 0.02 mg chemicalstandard dev. 0.005 mg chemicalTwo tests show probability of irritation given applicationlow freq of effect per mg exposure = 5/100/mghigh freq of effect per mg exposure = 10/100/mg
Analytical resultsRisk = exposure potencyMean risk = 0.02 mg 0.075 / mg = 0.0015or 15 out of 10,000 applications will result in irritation
Analytical resultsConservative estimateUse upper 95th %ile
Risk = 0.03 mg 0.0975 / mg = 0.0029
Monte Carlo: Visual exampleExposure = normal(mean 0.02 mg, s.d. = 0.005 mg)potency = uniform (range 0.05 / mg to 0.10 / mg)
Random draw onep(irritate) = 0.0165 mg 0.063/mg = 0.0010
Random draw twop(irritate) = 0.0175 mg 0.089 /mg = 0.0016
Summary: {0.0010, 0.0016}
Random draw threep(irritate) = 0.152 mg 0.057 /mg = 0.0087
Summary: {0.0010, 0.0016, 0.00087}
Random draw fourp(irritate) = 0.0238 mg 0.085 /mg = 0.0020
Summary: {0.0010, 0.0016, 0.00087, 0.0020}
After ten random drawsSummary{0.0010, 0.0016, 0.00087, 0.0020, 0.0011, 0.0018, 0.0024, 0.0016, 0.0015, 0.00062}
mean 0.0014
standard deviation (0.00055)
Using softwareCould write this program using a random number generatorBut, several software packages out there.I use Crystal Balluser friendlycustomizabler.n.g. good up to about 10,000 iterations
100 iterations (about two seconds)Monte Carlo resultsMean0.0016Standard Deviation0.00048Conservative estimate0.0026Compare to analytical resultsMean 0.0015standard deviationn/aConservative estimate0.0029
Summary chart - 100 trials
Summary - 10,000 trialsMonte Carlo resultsMean0.0015Standard Deviation0.000472Conservative estimate0.0024Compare to analytical resultsMean 0.0015standard deviationn/aConservative estimate0.0029
Summary chart - 10,000 trialsAbout 1.5 minutes run time
Policy applicationsWhen there are many distributional inputsConcern about excessive conservatismmultiplying 95th percentilesmultiple exposuresBecause we canBayesian calculations
Issues: Sensitivity AnalysisSensitivity analysis looks at which input distributions have the greatest effect on the eventual distributionHelps to understand which parameters can both be influenced by policy and reduce risksHelps understand when better data can be most valuable (information isnt freenor even cheap)
Issues: CorrelationTwo distributions are correlated when a change in one causes a change in anotherExample: People who eat lots of peas may eat less broccoli (or may eat more)Usually doesnt have much effect unless significant correlation (||>0.75)
Generating DistributionsInvalid distributions create invalid results, which leads to inappropriate policiesTwo optionsempiricaltheoretical
Empirical DistributionsMost appropriate when developed for the issue at hand. Example: local fish consumptionsurvey individuals or otherwise estimatedata from individuals elsewhere may be very misleadingA number of very large data sets have been developed and published
Empirical DistributionsChallenge: when theres very little dataExample of two data pointsuniform distribution?triangular distribution?not a hypothetical issueis an ongoing debate in the literatureKey is to state clearly your assumptionsBetter yetdo it both ways!
Which Distribution?
Random number generationShouldnt be an issue@Risk and Crystal Ball are both good to at least 10,000 iterations10,000 iterations is typically enough, even with many input distributions
Theoretical DistributionsAppropriate when theres some mechanistic or probabilistic basisExample: small sample (say 50 test animals) establishes a binomial distributionLognormal distributions show up often in nature
Some CaveatsBeware believing that youve really understood uncertaintyBeware: misapplication ignorance at bestfraudulent at worstporcine hoof blister
Example (after Finkel)Alar versus aflatoxinExposure has two elementsPeanut butter consumptionaflatoxin residueJuice consumptionAlar/UDMH residuePotency has one elementaflatoxin potencyUDMH potencyRisk = (consumption residue potency)/body weight
Inputs for Alar & aflatoxin
Variable
Units
Mean
5th %ile
95th %ile
Percentile location of the mean.
Peanut butter consumption
g/day
11.38
2.00
31.86
66
Apple juice consumption
g/day
136.84
16.02
430.02
69
aflatoxin residue
(g/g
2.82
1.00
6.50
61
UDMH residue
(g/g
13.75
0.5
42.00
67
aflatoxin potency
kg-day/mg
17.5
4.02
28.23
61
UDMH potency
kg-day/mg
0.49
0.00
0.85
43
Alar and aflatoxin point estimatesaflatoxin estimates:Mean
= 0.028Conservative = 0.29Alar (UDMH) estimates:Mean = 0.046Conservative = 0.77
Alar and aflatoxin Monte Carlo10,000 runsGenerate distributions (dont allow 0)Dont expect correlation
Aflatoxin and Alar Monte Carlo results (point values)
Aflatoxin
Analytical
Monte Carlo
Mean
0.028
0.028
Conservative
0.29
0.095
Alar
Analytical
Monte Carlo
Mean
0.046
0.046
Conservative
0.77
0.18
Aflatoxin and Alar Monte Carlo results (distributions)
Aflatoxin and Alar Monte Carlo results (distributions)
Aflatoxin and Alar Monte Carlo results (distributions)
Aflatoxin and Alar Monte Carlo results (distributions)
Aflatoxin and Alar Monte Carlo results (distributions)
Aflatoxin and Alar Monte Carlo results (distributions)
References and Further ReadingBurmaster, D.E and Anderson, P.D. (1994). Principles of good practice for the use of Monte Carlo techniques in human health and ecological risk assessments. Risk Analysis 14(4):447-81Finkel, A (1995). Towards less misleading comparisons of uncertain risks: the example of aflatoxin and Alar. Environmental Health Perspectives 103(4):376-85.Kammen, D.M and Hassenzahl D.M. (1999). Should We Risk It? Exploring Environmental, Health and Technological Problem Solving. Princeton University Press, Princeton, NJ.Thompson, K. M., D. E. Burmaster, et al. (1992). "Monte Carlo techniques for uncertainty analysis in public health risk assessments." Risk Analysis 12(1): 53-63.Vose, David (1997) Monte Carlo Risk Analysis Modeling in Molak, Ed., Fundamentals of Risk Analysis and Risk Management.
Top Related