Random Number Generators. Based upon specific mathematical algorithms Which are repeatable and...

Random Number Random Number GeneratorsGenerators

Random Number GeneratorsRandom Number Generators

Based upon specific mathematical Based upon specific mathematical algorithmsalgorithms

Which are repeatable and sequentialWhich are repeatable and sequential

RandomRandom

Truly RandomTruly Random– Exhibiting true randomnessExhibiting true randomness

PseudorandomPseudorandom– Appearance of randomness but having a Appearance of randomness but having a

specific repeatable patternspecific repeatable pattern Quasi-randomQuasi-random

– Having a set of non-random numbers in Having a set of non-random numbers in a randomized ordera randomized order

ProblemsProblems

Difficult to isolateDifficult to isolate– Often need to replace current generatorOften need to replace current generator– RequireRequire

Knowledge of current generatorKnowledge of current generatorSometimes in-depth understanding of Sometimes in-depth understanding of

random number generators themselvesrandom number generators themselves Large scale tests cause most Large scale tests cause most

problemsproblems– Needing sometimes millions or billions Needing sometimes millions or billions

of random numbersof random numbers

Desirable PropertiesDesirable Properties

When performing Monte Carlo When performing Monte Carlo SimulationsSimulations– Attributes of each particle should be Attributes of each particle should be

independent of those attributes of any independent of those attributes of any other particleother particle

– Fill the entire attribute space in a Fill the entire attribute space in a manner which is consistent with the manner which is consistent with the physicsphysics

Random Number CycleRandom Number Cycle

BasisBasis– sequence of pseudorandom integerssequence of pseudorandom integers

Some exceptionsSome exceptions

Integers (“Fixed”)Integers (“Fixed”)– Manipulated arithmetically to yield Manipulated arithmetically to yield

floating point (“real”)floating point (“real”) Can be presented in either Integer or Can be presented in either Integer or

Real numbersReal numbers

CycleCycle

What Does This Show Us?What Does This Show Us?

Properties of pseudorandom Properties of pseudorandom sequences of integers sequences of integers – The sequence has a finite number of The sequence has a finite number of

integers integers – The sequence gets traversed in a The sequence gets traversed in a

particular orderparticular order– The sequence repeats if the period of The sequence repeats if the period of

the generator is exceeded the generator is exceeded

LCGLCG

Most commonly used RNGMost commonly used RNG– Linear Congruential GeneratorLinear Congruential Generator

Requires initial “seed” denoted as X0Requires initial “seed” denoted as X0Appears random because of Modulo functionAppears random because of Modulo functionNext “random” number depends heavily on Next “random” number depends heavily on

previous Xprevious X– Typical of linear, congruential generatorsTypical of linear, congruential generators– Restricts periodRestricts period

Equations - LCGEquations - LCG

Using LCGUsing LCG

Choosing Correct Input is KeyChoosing Correct Input is Key LCG (a,c,m,X0)LCG (a,c,m,X0)

– LCG (5, 1, 16, 1)LCG (5, 1, 16, 1) Yields – Yields –

1,6,15,12,13,2,11,8,9,14,7,4,5,10,3,0,1,6,15,12,13,2,11,8,9,14,7,4,5,10,3,0, 1,6,15,12,13,2,11,8,9,14,…1,6,15,12,13,2,11,8,9,14,…

When the next result depends upon only the When the next result depends upon only the previous integer, the longest period possible is P=Mprevious integer, the longest period possible is P=M

Odd/Even patternOdd/Even pattern lack of randomness results from using a power of two lack of randomness results from using a power of two

for Mfor M

Cycle LCG(5,1,16,1)Cycle LCG(5,1,16,1)

TableTable

Example #2Example #2

LCG(5,0,16,1)LCG(5,0,16,1)– Yields - 1,5,9,13,1,5,9,13,…Yields - 1,5,9,13,1,5,9,13,…– M is a power of 2 (here: 2^4)M is a power of 2 (here: 2^4)

C=0C=0Maximum period is going to be 2^(m-2)Maximum period is going to be 2^(m-2)

– Correlation (each differ by 4)Correlation (each differ by 4)

Cycle (5,0,16,1)Cycle (5,0,16,1)

SeedSeed

Using the date and timeUsing the date and time– Enter the date and time into an Enter the date and time into an

equations and return an integer then equations and return an integer then make sure it is oddmake sure it is odd

– Standard seed for these equationsStandard seed for these equations

Overflow & Negative NumbersOverflow & Negative Numbers

Using large values of a and large Using large values of a and large values of M are neededvalues of M are needed– Often 31 bits longOften 31 bits long

On 32 bit machinesOn 32 bit machines

– A*M results in 62 bit numberA*M results in 62 bit number– OverflowOverflow

Can result in 32Can result in 32ndnd bit being a negative bit being a negative

N-Tuple GeneralizationN-Tuple Generalization

Choose R1 and R2Choose R1 and R2– Choose Rn and R(n+1)Choose Rn and R(n+1)– Then plot this point of interest in a Then plot this point of interest in a

surrounding area.surrounding area.Plot these points in successionPlot these points in successionThe area will be uniformly covered by the The area will be uniformly covered by the

LCG in a “random” orderLCG in a “random” order– Covering of only part of the unit or certain areas Covering of only part of the unit or certain areas

of the unit would prove to be not useful for Monte of the unit would prove to be not useful for Monte Carlo MethodsCarlo Methods

Embarrassingly Parallel'Embarrassingly Parallel'

Little or no interprocessor Little or no interprocessor communicationcommunication

Easy to codeEasy to code

N StreamsN Streams

N StreamsN Streams– N independent random numbersN independent random numbers– N independent processesN independent processes

Need to find N seeds far away from Need to find N seeds far away from each other on the cycleeach other on the cycle

Find SeedsFind Seeds

Find SeedsFind Seeds– LCG rule successively applied: LCG rule successively applied:

Lagged Fibonacci Generators Lagged Fibonacci Generators

Increasingly popularIncreasingly popular– Lags are k and lLags are k and l– M is power of 2M is power of 2

With proper choice of k and LWith proper choice of k and L Period of Generator can bePeriod of Generator can be

– [(2^L)-1] * [2^(m-1)][(2^L)-1] * [2^(m-1)]

LFGLFG

Computationally simpleComputationally simple– Integer addInteger add– Logical ANDLogical AND– Decrement of 2 array pointersDecrement of 2 array pointers

– Must keep L words current in memoryMust keep L words current in memory– LCG needs only oneLCG needs only one

LFG (cont)LFG (cont) LFG are an attempt to improve LCGLFG are an attempt to improve LCG Similar to Combined LCGSimilar to Combined LCG

– Take 2 previous numbers in the sequence to Take 2 previous numbers in the sequence to produce a new numberproduce a new number

– Where p and q are the “lags”Where p and q are the “lags”– Some arithmetic computation is performedSome arithmetic computation is performed– Then mod that answer for the next numberThen mod that answer for the next number

Monte Carlo MethodsMonte Carlo Methods

Overview Overview

IntroductionIntroduction HistoryHistory ExamplesExamples ApplicationsApplications Real Life practices Real Life practices

IntroductionIntroduction

Define Monte Carlo MethodDefine Monte Carlo Method– The Monte Carlo method is a numerical The Monte Carlo method is a numerical

method for solving mathematical method for solving mathematical problems using stochastic sampling. problems using stochastic sampling.

– It performs simulation of any process It performs simulation of any process whose development is influenced by whose development is influenced by random factors, but also if the given random factors, but also if the given problem involves no chance, the method problem involves no chance, the method enables artificial construction of a enables artificial construction of a probabilistic model. probabilistic model.

Introduction cont…Introduction cont…

Similarly, Monte Carlo methods randomly Similarly, Monte Carlo methods randomly select values to create scenarios of a select values to create scenarios of a problem. These values are taken from problem. These values are taken from within a fixed range and selected to fit a within a fixed range and selected to fit a probability distribution [e.g. bell curve, probability distribution [e.g. bell curve, linear distribution, etc.]. This is like rolling a linear distribution, etc.]. This is like rolling a dice. The outcome is always within the dice. The outcome is always within the range of 1 to 6 and it follows a linear range of 1 to 6 and it follows a linear distribution - there is an equal opportunity distribution - there is an equal opportunity for any number to be the outcome.for any number to be the outcome.


MC method is often referred to as the MC method is often referred to as the “method of last resort”, as it is apt to “method of last resort”, as it is apt to consume large computing resources;consume large computing resources;

Characteristics:Characteristics:– consuming vast computing resources consuming vast computing resources – have historically had to be executed upon have historically had to be executed upon

the fastest computers available at the time the fastest computers available at the time – and employ the most advanced algorithmsand employ the most advanced algorithms– implemented with substantial implemented with substantial

programming acumen. programming acumen.


Major components of Monte Carlo Major components of Monte Carlo methods:methods:– Probability distribution functionsProbability distribution functions – Random number generatorRandom number generator – ScoringScoring– Error estimationError estimation – Variance reduction techniquesVariance reduction techniques – Parallelization and vectorizationParallelization and vectorization

HistoryHistory

Where does Monte Carlo method come Where does Monte Carlo method come from? When? Who?from? When? Who?– The name "Monte Carlo" comes from the city of The name "Monte Carlo" comes from the city of

Monte Carlo in the principality of Monaco, Monte Carlo in the principality of Monaco, famous for its gambling housefamous for its gambling house

– Birth date of the Monte Carlo method is 1949, Birth date of the Monte Carlo method is 1949, when an articale entitled "The Monte Carlo when an articale entitled "The Monte Carlo Method"( by N. Metropolis and S. Ulam ) Method"( by N. Metropolis and S. Ulam ) appeared. appeared.

– The American mathematicians J. Neyman and The American mathematicians J. Neyman and S. Ulam are considered its originators. S. Ulam are considered its originators.

History cont…History cont…

History cont…History cont… The theoretical foundation of the method The theoretical foundation of the method

had been known long before first articles had been known long before first articles were published. were published.

Well before 1949 certain problems in Well before 1949 certain problems in statistics were sometimes solved by means statistics were sometimes solved by means of random sampling of random sampling

However, simulation of random variables However, simulation of random variables by hand is a laborious process by hand is a laborious process

Use of the Monte Carlo method as a Use of the Monte Carlo method as a universal numerical technique became universal numerical technique became practical only with the advent of computers practical only with the advent of computers and high-quality pseudorandom number and high-quality pseudorandom number generators generators

History cont…History cont… Buffon's needle problemBuffon's needle problem

– In 1768 Buffon, a French mathematician, In 1768 Buffon, a French mathematician, experimentally determined a value of experimentally determined a value of ππ by by casting a needle on a ruled gridcasting a needle on a ruled grid

Lord Rayleigh even delved into this field Lord Rayleigh even delved into this field near the turn of the century.near the turn of the century.

Fredericks and Levy in 1928 showed how Fredericks and Levy in 1928 showed how the method could be used to solve the method could be used to solve boundary value problemsboundary value problems

Enrico Fermi in the 1930's used Monte Enrico Fermi in the 1930's used Monte Carlo in the calculation of neutron Carlo in the calculation of neutron diffusion (involving nuclear reactors )diffusion (involving nuclear reactors )

History cont…History cont…

In the 1940's, a formal foundation for In the 1940's, a formal foundation for the Monte Carlo method was the Monte Carlo method was developed by von Neumann (PDE)developed by von Neumann (PDE)

Stanislaw Ulam realized the Stanislaw Ulam realized the importance of the digital computer in importance of the digital computer in the implementation of the approach the implementation of the approach from collaboration results of the work from collaboration results of the work on the Manhattan project during on the Manhattan project during World War IIWorld War II

ExamplesExamples

Simple Example to Understand: Simple Example to Understand: computing the area of a plane figure computing the area of a plane figure S. S. – completely arbitary figure with a completely arbitary figure with a

curvilinear boundary, given graphically curvilinear boundary, given graphically or analytically, connected or consisting or analytically, connected or consisting of several pieces of several pieces

– assume that it is contained completely assume that it is contained completely within the unit square. within the unit square.

Examples cont…Examples cont…

Figure S in the unit square, being covered with sampling points randomly


Applying Randomness to the Applying Randomness to the example:example:– Choose at random N points in the Choose at random N points in the

square and designate the number of square and designate the number of points lying inside S by N'. It is points lying inside S by N'. It is geometrically obvious that the area of S geometrically obvious that the area of S is approximately equal to the ratio N'/N. is approximately equal to the ratio N'/N. The greater the N, the greater the The greater the N, the greater the accuracy of this estimate. accuracy of this estimate.


Buffon's Needle:Buffon's Needle:– A simple Monte Carlo method for the A simple Monte Carlo method for the

estimation of the value of estimation of the value of ππ, 3.1415926, 3.1415926– Assumptions:Assumptions:

Suppose you have a tabletop with a number Suppose you have a tabletop with a number of parallel lines drawn on it, which are of parallel lines drawn on it, which are equally spaced (say the spacing is 1 inch, for equally spaced (say the spacing is 1 inch, for example). example).

Suppose you also have a pin or needle, Suppose you also have a pin or needle, which is also an inch long. which is also an inch long.


Dropping needles on the tablet:Dropping needles on the tablet:– The needle crosses or touches one of the lines The needle crosses or touches one of the lines – The needle crosses no linesThe needle crosses no lines

Keep dropping this needle over and over Keep dropping this needle over and over on the tableon the table

Record the statistics.Record the statistics.– Keep track of both the total number of times Keep track of both the total number of times

that the needle is randomly dropped on the that the needle is randomly dropped on the table N, and the number of times that it table N, and the number of times that it crosses a line N’. crosses a line N’.


Findings:Findings:– 2N/N’=2N/N’=ππ– Because, Because, the probability on any given the probability on any given

drop of the needle that it should cross a drop of the needle that it should cross a line is given by 2/pi line is given by 2/pi

– After many tries, N/N’ will approach the After many tries, N/N’ will approach the probability number.probability number.

Applications Applications

Monte Carlo methods can help in Monte Carlo methods can help in design and prediction of behavior of design and prediction of behavior of systems in nuclear applications and systems in nuclear applications and radiation physicsradiation physics

The use of MC in the area of nuclear The use of MC in the area of nuclear power has undergone an important power has undergone an important evolution. Notable are the extensions evolution. Notable are the extensions to compute burnup in reactor cores, to compute burnup in reactor cores, and full core neutronic simulations. and full core neutronic simulations.

Applications cont…Applications cont…

help researchers understand the help researchers understand the probability of the occurrence of an probability of the occurrence of an adverse effect associated with adverse effect associated with exposures to chemicals. Monte Carlo exposures to chemicals. Monte Carlo sampling simulates the distribution of sampling simulates the distribution of total exposures, by simulating random total exposures, by simulating random samples of factors associated with each samples of factors associated with each exposure route and accumulating them exposure route and accumulating them to arrive at an individual total exposure. to arrive at an individual total exposure.


The use of MC methods to model physical The use of MC methods to model physical problems allows us to examine more problems allows us to examine more complex systems than we otherwise can. complex systems than we otherwise can. Solving equations which describe the Solving equations which describe the interactions between two atoms is fairly interactions between two atoms is fairly simple; solving the same equations for simple; solving the same equations for hundreds or thousands of atoms is hundreds or thousands of atoms is impossible. With MC methods, a large system impossible. With MC methods, a large system can be sampled in a number of random can be sampled in a number of random configurations, and that data can be used to configurations, and that data can be used to describe the system as a whole. describe the system as a whole.


Random numbers generated by the Random numbers generated by the computer are used to simulate computer are used to simulate naturally random processesnaturally random processes

many previously intractable many previously intractable thermodynamic and quantum thermodynamic and quantum mechanics problems have been mechanics problems have been solved using Monte Carlo techniquessolved using Monte Carlo techniques

Real Life Practice Real Life Practice

Quantum Monte CarloQuantum Monte Carlo– The microscopic world is described by The microscopic world is described by

quantum mechanics. We need to use quantum mechanics. We need to use simulation techniques to “solve” many-simulation techniques to “solve” many-body quantum problems.body quantum problems.

– Both the wavefunction and expectation Both the wavefunction and expectation values are determined by the values are determined by the simulations.simulations.

– QMC gives most accurate method for QMC gives most accurate method for general quantum many-body systems.general quantum many-body systems.

Real Life Practice cont…Real Life Practice cont…

• WeatherWeather Equipment ProductivityEquipment Productivity Soil ConditionsSoil Conditions

Projects are often associated with a high Projects are often associated with a high degree of uncertainty resulting from degree of uncertainty resulting from the unpredictable nature of eventsthe unpredictable nature of events

Real Life Practice cont…Real Life Practice cont…

Risk Analysis and Risk Management Risk Analysis and Risk Management – Monte Carlo Simulation is a valuable modeling Monte Carlo Simulation is a valuable modeling

tool that generates multiple scenarios tool that generates multiple scenarios depending upon the data and the assumptions depending upon the data and the assumptions fed into the model. fed into the model.

– Simulation calculates multiple scenarios by Simulation calculates multiple scenarios by repeatedly inserting different sampling values repeatedly inserting different sampling values from probability distribution for the uncertain from probability distribution for the uncertain variables into the computerized spread-sheet.variables into the computerized spread-sheet.

– probability or percentage chance that a probability or percentage chance that a particular forecast value will fall within a certain particular forecast value will fall within a certain specified range. specified range.

Why has the Monte Carlo method Why has the Monte Carlo method become so popular? become so popular?

Analytic methods tend to be Analytic methods tend to be prohibitive prohibitive (but some very difficult (but some very difficult problems have finally been solved problems have finally been solved using MC)using MC)

Monte Carlo is somewhat Monte Carlo is somewhat intuitive intuitive (and several good books have now (and several good books have now been written on the subject)been written on the subject)

Computers continue to get Computers continue to get faster faster and and cheapercheaper

ReferenceReference

http://random.mat.sbg.ac.at/links/mohttp://random.mat.sbg.ac.at/links/monte.htmlnte.html

http://www.ccs.uky.edu/csep/http://www.ccs.uky.edu/csep/csep.htmlcsep.html

Random Number Generators. Based upon specific mathematical algorithms Which are repeatable and...

Documents

Transcript of Random Number Generators. Based upon specific mathematical algorithms Which are repeatable and...