Course Overview Course Outline Bibliography Introduction

M4A42 APPLIED STOCHASTICPROCESSES

G.A. Pavliotis

Department of Mathematics Imperial College London, UK

LECTURE 1 12/10/2009

Course Overview Course Outline Bibliography Introduction

Lectures: Mondays 09:00-11:00, Huxley 139, Tuesdays09:00-10:00, Huxley 144.

Office Hours: Mondays 14:00-15:00, Tuesdays14:00-15:00 or by appointment.

Course webpage:http://www.ma.imperial.ac.uk/~pavl/M4A42.htm

Text: Lecture notes, available from the course webpage.Also, recommended reading from various textbooks.

Course Overview Course Outline Bibliography Introduction

This is an introductory course on stochastic processes andtheir applications, aimed towards students in appliedmathematics.

The emphasis of the course will be on the presentation ofanalytical tools that are useful in the study of stochasticmodels that appear in various problems in appliedmathematics, physics, chemistry and biology.

We will also discuss about numerical methods forsimulating stochastic processes, solving stochasticdifferential equations etc.

Time permitting, some applications of stochasticprocesses such as the modeling of molecular motors orchemical reactions will be discussed.

Course Overview Course Outline Bibliography Introduction

PrerequisitesElementary probability theory.Ordinary and partial differential equations.Linear algebra.Some familiarity with analysis (measure theory, linearfunctional analysis) is desirable but not necessary.

Course ObjectivesBy the end of the course you are expected to be familiarwith the basic concepts of the theory of stochasticprocesses in continuous time and to be able to use variousanalytical and computational techniques to study stochasticmodels that appear in applications.

Course assessmentCoursework: 3 sets of assessed coursework 10% of thefinal mark.Final exam (May/June 2010).

Course Overview Course Outline Bibliography Introduction

Probability theory and random variables (3 lectures) .Basic definitions, probability spaces, probability measuresetc. Random variables, conditional expectation,characteristic functions, limits theorems.

Stochastic processes (8 lectures) . Basic definitions.Brownian motion. Stationary processes. Other examplesof stationary processes. The Karhunen-Loeve expansion.

Markov processes (4 lectures) . Introduction andexamples. Basic definitions. The Chapman-Kolmogorovequation. The generator of a Markov process and itsadjoint. Ergodic and stationary Markov processes.

Diffusion processes (4 lectures) . Basic definitions andexamples. The backward and forward (Fokker-Planck)Kolmogorov equations. Connection between diffusionprocesses and stochastic differential equations.

Course Overview Course Outline Bibliography Introduction

The Fokker-Planck equation (7 lectures) . Basicproperties of the FP equation. Examples of diffusionprocesses and of the FP equation. TheOrnstein-Uhlenbeck process. Gradient flows andeigenfunction expansions.

Stochastic Differential Equations (4 lectures) . Basicproperties of SDEs. Itô’s formula. Numerical solution ofSDEs.

Course Overview Course Outline Bibliography Introduction

Lecture notes will be provided for all the material that wewill cover in this course. The notes are available from thecourse webpage.

There are many excellent textbooks/review articles onapplied stochastic processes, at a level and style similar tothat of this course.Standard textbooks that cover the material on probabilitytheory, Markov chains and stochastic processes are:

Grimmett and Stirzaker: Probability and RandomProcesses.Karlin and Taylor: A First Course in Stochastic Processes.Lawler: Introduction to Stochastic Processes.Resnick: Adventures in Stochastic Processes.

Course Overview Course Outline Bibliography Introduction

Books on stochastic processes with a view towardsapplications, mostly to physics, are:

Horsthemke and Lefever: Noise induced transitions.Risken: The Fokker-Planck equation.Gardiner: Handbook of stochastic methods.van Kampen: Stochastic processes in physics andchemistry.Mazo: Brownian motion: fluctuations, dynamics andapplications.Chorin and Hald: Stochastic tools for mathematics andscience.Gillespie; Markov Processes.

Course Overview Course Outline Bibliography Introduction

The rigorous mathematical theory of probability andstochastic processes is presented in

Koralov and Sinai: Theory of probability and randomprocesses.Karatzas and Shreeve: Brownian motion and stochasticcalculus.Revuz and Yor: Continuous martingales and Brownianmotion.Stroock: Probability theory, an analytic view.

Books on stochastic differential equations and theirnumerical solution are

Oksendal: Stochastic differential equations.Kloeden and Platen, Numerical Solution of StochasticDifferential Equations.

An excellent book on the theory and the applications ofstochastic processes is

Bhatthacharya and Waymire: Stochastic processes andapplications.

Course Overview Course Outline Bibliography Introduction

A stochastic process is used to model systems thatevolve in time and whose laws of evolution are probabilisticin nature.

The state of the system evolves in time and can bedescribed through a state variable x(t).The evolution of the state of the system depends on theoutcome of an experiment. We can write x = x(t, ω), whereω denotes the outcome of the experiment.

Examples:The random walk in one dimension.Brownian motion.The exchange rate between the British pound and the USdollar.Photon emission.The spread of the SARS epidemic.

Course Overview Course Outline Bibliography Introduction

The One-Dimensional Random Walk

We let time be discrete, i.e. t = 0, 1, . . . . Consider the followingstochastic process Sn:

S0 = 0;

at each time step it moves to ±1 with equal probability 12 .

In other words, at each time step we flip a fair coin. If theoutcome is heads, we move one unit to the right. If the outcomeis tails, we move one unit to the left.Alternatively, we can think of the random walk as a sum ofindependent random variables:

Sn =

n∑

j=1

Xj ,

where Xj ∈ {−1, 1} with P(Xj = ±1) = 12 .

Course Overview Course Outline Bibliography Introduction

We can simulate the random walk on a computer:

We need a (pseudo)random number generator togenerate n independent random variables which areuniformly distributed in the interval [0,1].

If the value of the random variable is ≥ 12 then the particle

moves to the left, otherwise it moves to the right.

We then take the sum of all these random moves.

The sequence {Sn}Nn=1 indexed by the discrete time

T = {1, 2, . . . N} is the path of the random walk. We use alinear interpolation (i.e. connect the points {n, Sn} bystraight lines) to generate a continuous path .

Course Overview Course Outline Bibliography Introduction

0 5 10 15 20 25 30 35 40 45 50

−6

−4

−2

0

2

4

6

8

50−step random walk

Figure: Three paths of the random walk of length N = 50.

Course Overview Course Outline Bibliography Introduction

0 100 200 300 400 500 600 700 800 900 1000

−50

−40

−30

−20

−10

0

10

20

1000−step random walk

Figure: Three paths of the random walk of length N = 1000.

Course Overview Course Outline Bibliography Introduction

Every path of the random walk is different: it depends onthe outcome of a sequence of independent randomexperiments.We can compute statistics by generating a large number ofpaths and computing averages. For example,E(Sn) = 0, E(S2

n) = n.The paths of the random walk (without the linearinterpolation) are not continuous: the random walk has ajump of size 1 at each time step.This is an example of a discrete time, discrete spacestochastic processes.The random walk is a time-homogeneous (theprobabilistic law of evolution is independent of time)Markov (the future depends only on the present and noton the past) process.If we take a large number of steps, the random walk startslooking like a continuous time process with continuouspaths.

Course Overview Course Outline Bibliography Introduction

0 0.5 1−2

0

2First 50 steps

0 0.5 1−2

0

2First 1000 steps

0 0.5 1−2

0

2First 100 steps

0 0.5 1−2

0

2First 5000 steps

0 0.5 1−2

0

2First 200 steps

0 0.5 1−2

0

2First 25000 steps

Figure: Space-time rescaled random walks.

Course Overview Course Outline Bibliography Introduction

Consider the sequence of continuous time stochasticprocesses

Z nt :=

1√n

Snt .

In the limit as n → ∞, the sequence {Z nt } converges (in

some appropriate sense) to a Brownian motion withdiffusion coefficient D = ∆x2

2∆t = 12 .

Course Overview Course Outline Bibliography Introduction

0 0.2 0.4 0.6 0.8 1−1.5

−1

−0.5

0

0.5

1

1.5

2

t

U(t)

mean of 1000 paths5 individual paths

Figure: Sample Brownian paths.

Course Overview Course Outline Bibliography Introduction

Brownian motion W (t) is a continuous time stochasticprocesses with continuous paths that starts at 0(W (0) = 0) and has independent, normally. distributedGaussian increments.

We can simulate the Brownian motion on a computer usinga random number generator that generates normallydistributed, independent random variables.

Course Overview Course Outline Bibliography Introduction

We can write an equation for the evolution of the paths of aBrownian motion Xt with diffusion coefficient D starting atx:

dXt =√

2DdWt , X0 = x .

This is an example of a stochastic differential equation .

The probability of finding Xt at y at time t , given that it wasat x at time t = 0, the transition probability densityρ(y , t) satisfies the PDE

∂ρ

∂t= D

∂2ρ

∂y2 , ρ(y , 0) = δ(y − x).

This is an example of the Fokker-Planck equation .

The connection between Brownian motion and thediffusion equation was made by Einstein in 1905.

Course Overview Course Outline Bibliography Introduction

Why introduce randomness in the description of physicalsystems?

To describe outcomes of a repeated set of experiments.Think of tossing a coin repeatedly or of throwing a dice.To describe a deterministic system for which we haveincomplete information: we have imprecise knowledge ofinitial and boundary conditions or of model parameters.

ODEs with random initial conditions are equivalent tostochastic processes that can be described usingstochastic differential equations.

To describe systems for which we are not confident aboutthe validity of our mathematical model.

Course Overview Course Outline Bibliography Introduction

To describe a dynamical system exhibiting verycomplicated behavior (chaotic dynamical systems).Determinism versus predictability.

To describe a high dimensional deterministic system usinga simpler, low dimensional stochastic system. Think of thephysical model for Brownian motion (a heavy particlecolliding with many small particles).

To describe a system that is inherently random. Think ofquantum mechanics.