MonteCarlo Oxford

Numerical Methods IIProf. Mike Giles

[email protected]

Oxford University Mathematical Institute

MC Lecture 2 p. 1

Random Number GenerationMonte Carlo simulation starts with random numbergeneration, which often is split into 3 stages:

generation of independent uniform (0, 1) randomvariablesconversion into independent Normal N(0, 1) randomvariablesconversion into correlated Normal N(0, 1) randomvariables

This lecture will cover:what you need to know as a userbackground information to make you better informed

MC Lecture 2 p. 2

Uniform Random Variables

Generating good uniform random variables istechnically complexNever write your own generator; always use a wellvalidated generator from a reputable source

MatlabNAGIntel MKLAMD ACMLnot MS Excel, C rand function or Numerical Recipes

What you need to know is what to look for in a goodgeneratorUseful background knowledge is how they work

MC Lecture 2 p. 3

Uniform Random VariablesPseudo-random number generators use a deterministic(i.e. repeatable) algorithm to generate a sequence of(apparently) random numbers on (0, 1) interval.

What defines a good generator?a long period how long it takes before the sequencerepeats itself232 is not enough need at least 240

various statistical tests to measure randomnesswell validated software will have gone through thesechecks

MC Lecture 2 p. 4

Uniform Random VariablesPractical considerations:

computational cost RNG cost can be as large as restof Monte Carlo simulationtrivially-parallel Monte Carlo simulation on a computecluster requires the ability to skip-ahead to an arbitrarystarting point in the sequencefirst computer gets first 106 numberssecond computer gets second 106 numbers, etc

MC Lecture 2 p. 5


Multiplicative congruential algorithms based on

ni = (a ni1) mod m

choice of integers a and m is crucial(0,1) random number given by ni/mtypical period is 257, a bit smaller than mcan skip-ahead 2k places at low cost by repeatedlysquaring a, mod m

MC Lecture 2 p. 6


Mersenne twister is very popular in finance:

developed in 1997 so still quite newhuge period of 2199371; I think this is the main reasonits popular for Monte Carlo applicationsIve heard conflicting comments on its statisticalproperties

MC Lecture 2 p. 7

Uniform Random VariablesFor more details see

Intel MKL informationwww.intel.com/cd/software/products/asmo-na/eng/266864.htm

NAG library informationwww.nag.co.uk/numeric/CL/nagdoc cl08/pdf/G05/g05 conts.pdf

Matlab informationwww.mathworks.com/moler/random.pdf

Wikipedia informationen.wikipedia.org/wiki/Random number generationen.wikipedia.org/wiki/List of random number generatorsen.wikipedia.org/wiki/Mersenne Twister

MC Lecture 2 p. 8

Normal Random Variables

To generate N(0, 1) Normal random variables, we start witha sequence of uniform random variables on (0, 1).

There are then 4 main ways of converting them into N(0, 1)Normal variables:

Box-Muller methodMarsaglias polar methodMarsaglias ziggurat methodinverse CDF transformation

MC Lecture 2 p. 9

Normal Random VariablesThe Box-Muller method takes y1, y2, two independentuniformly distributed random variables on (0, 1) and defines

x1 =2 log(y1) cos(2piy2)

x2 =2 log(y1) sin(2piy2)

It can be proved that x1 and x2 are N(0, 1) randomvariables, and independent.

A log, cos and sin operation per 2 Normals makes this aslightly expensive method.

MC Lecture 2 p. 10


Marsaglias polar method is as follows:Take y1, y2 from uniform distribution on (1, 1)Accept if r2 = y2

1+ y2

2< 1, otherwise get new y1, y2

Define x1 =2 log(r2)/r2 y1

x2 =2 log(r2)/r2 y2

Again it can be proved that x1 and x2 are independentN(0, 1) random variables.Despite approximately 20% rejections, it is slightly moreefficient because of not needing cos, sin operations.However, rejection of some of the uniforms spoils theskip-ahead capability.

MC Lecture 2 p. 11


Marsaglias ziggurat method is the fastest, but also thehardest to explain. The details arent really important, so Illjust give an outline.The unit area under the standard Normal distribution inbroken up into a number of rectangles (accounting for over98% of the area) and other bits.

4 3 2 1 0 1 2 3 40

0.1

0.2

0.3

0.4

x

N

(

x

)

MC Lecture 2 p. 12


Given a uniformly distributed (0, 1) random input:a lookup table is used to determine if it corresponds toone of the rectanglesif so, then the corresponding output is uniformlydistributed within the rectangle and so can be computedvery easilyif not, the calculation is much more complex, but thisonly happens 2% of the timeMatlab uses this approach in randnwww.mathworks.com/moler/random.pdf

MC Lecture 2 p. 13


The transformation method takes y, uniformly distributed on(0, 1), and defines

x = 1(y),

where (x) is the Normal cumulative distribution functiondefined earlier.

1(y) is approximated in software in a very similar way tothe implementation of cos, sin, log, so this is just as accurateas the other methods.

It is also a more flexible approach because well need1(y) later for stratified sampling and quasi-Monte Carlomethods.

MC Lecture 2 p. 14


4 2 0 2 40

0.2

0.4

0.6

0.8

1

x

(

x

)

0 0.5 14

3

2

1

0

1

2

3

4

x

1

(

x

)

MC Lecture 2 p. 15


Some useful weblinks:home.online.no/pjacklam/notes/invnorm/code for 1 function in many different languageslib.stat.cmu.edu/apstat/241/single and double precision code in FORTRAN(coming soon in next version of NAG libraries)en.wikipedia.org/wiki/Normal distributionWikipedia definition of matches minemathworld.wolfram.com/NormalDistribution.htmlmathworld.wolfram.com/DistributionFunction.htmlGood Mathworld items, but their definition of isslightly different; they call the cumulative distributionfunction D(x).

MC Lecture 2 p. 16


The Normal CDF (x) is related to the error function erf(x)through

(x) = 12+ 1

2erf(x/

2) = 1(y) =

2 erf1(2y1)

This is the function I use in Matlab code:

% x = ncfinv(y)%% inverse Normal CDF

function x = ncfinv(y)

x = sqrt(2)*erfinv(2*y-1);

MC Lecture 2 p. 17

Correlated Normal Random VariablesThe final step is to generate a vector of Normally distributedvariables with a prescribed covariance matrix.Suppose x is a vector of independent N(0, 1) variables, anddefine a new vector y = Lx.Each element of y is Normally distributed, E[y] = LE[x] = 0,and

E[y yT ] = E[LxxT LT ] = LE[xxT ]LT = LLT .

since E[xxT ] = I becauseelements of x are independent = E[xi xj ] = 0 for i 6= jelements of x have unit variance = E[x2i ] = 1

MC Lecture 2 p. 18

Correlated Normal Random Variables

To get E[y yT ] = , we need to find L such that

LLT =

L is not uniquely defined. Simplest choice is to use aCholesky factorization in which L is lower-triangular, with apositive diagonal.

MC Lecture 2 p. 19

Correlated Normal Random Variables

Pseudo-code for Cholesky factorization LLT = :

for i from 1 to Nfor j from 1 to ifor k from 1 to j1

ij := ij LikLjkend

if j=iLii :=

ii

elseLij := ij/Ljj

endifend

end

MC Lecture 2 p. 20

Correlated Normal Random VariablesAlternatively, if has eigenvalues i 0, and orthonormaleigenvectors ui, so that

ui = i ui, = U = U then

= U UT = LLT

whereL = U 1/2.

This is the PCA decomposition; it is no better than theCholesky decomposition for standard Monte Carlosimulation, but is often better for stratified sampling andquasi-Monte Carlo methods.

MC Lecture 2 p. 21

Final advice

always use mathematical libraries as much aspossibleusually they will give you uncorrelated Normals, andyou have to convert these into correlated Normalslater with stratified sampling and quasi-Monte Carlomethods, we will use the inverse cumulative Normaldistribution to convert (quasi-)uniforms into(quasi-)Normals

MC Lecture 2 p. 22

Random Number GenerationUniform Random VariablesUniform Random VariablesUniform Random VariablesUniform Random VariablesUniform Random VariablesUniform Random VariablesNormal Random VariablesNormal Random VariablesNormal Random VariablesNormal Random VariablesNormal Random VariablesNormal Random VariablesNormal Random VariablesNormal Random VariablesNormal Random VariablesCorrelated Normal Random VariablesCorrelated Normal Random VariablesCorrelated Normal Random VariablesCorrelated Normal Random VariablesFinal advice

MonteCarlo Oxford

Documents

Transcript of MonteCarlo Oxford