MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.
-
Upload
ethan-jordan -
Category
Documents
-
view
221 -
download
0
description
Transcript of MA3264 Mathematical Modelling Lecture 5 Discrete Probabilistic Modelling.
MA3264 Mathematical ModellingLecture 5
Discrete Probabilistic Modelling
Markov Chains
vertices represent states and are labeled 1,..,K
Can be illustrated using labeled directed graphs
edges represent transitions between states and are labeled by numbers in the interval [0,1]
Page 217, Figure 6.1
1 2
p1
q1
qp
Transition Matrix T=
qpqp
11
NOTE Our TM = transpose book’s TM
Transition Probabilities
The label on a directed edge from vertex j to vertex i is the probability that if the system is in state j at time n then it will be in state i at time n+1 FOR EVERY n (in other words these so called transition probabilities (TP) do not change over time), for convenience we place these TP into a transition matrix T
Imagine a dynamic* system that can be in one of K states during each time interval [n,n+1)
1 2
p1
q1
qp
qpqp
11
* marked by usually continuous and productive activity or change
Transition ProbabilitiesFor the situation displayed in the graph below, if a system is in state 1 at time n = 1, then the probability that the system will change to be is state 2 at time n = 2 equals 1-p and the probability that the system will stay the same to be in state 1 at time n = 2 equals p, similar considerations apply if the system is in state 2 at time n = 1
1 2
p1
q1
qp
qpqp
11
The sum of the elements in each column of T equals 1
State VectorsIf at time n, the probability of a system being in state 1 equals a(n) and the probability of being in state 2 equals b(n) then this may be represented by a state vector
)()(
)(nbna
nv
Clearly the entries in this matrix are in [0,1] and their sum equals 1
Dynamics of State Vectors
The dynamics of this vector are derived from probability
)()()()(
)1(2221
1211
nbTnaTnbTnaT
nv
)()()(
2221
1211 nTvnbna
TTTT
Law of Large NumbersIf we have two (or more) large populations of individuals, each of whom can be in state 1 or state 2 at each time n, then a(n), b(n) can be interpreted as the frequency (or fractions) of individuals that are in state 1, 2 at time n
Likewise, if a state vector has entries that represent frequencies, then these frequencies can be interpreted as probabilities of an individual who is chosen randomly to be in state 1, 2 at time n
Likewise, the entries of a transition matrix can be either interpreted as probabilities or as frequencies
This dual interpretation aspect can be initially confusing but becomes much more obvious through applications
Rental Car ApplicationExample 1. Rental Car Company (pages 217-218)
1 24.
3.
7.6.
7.4.3.6.
T
If we let nn qp , the fraction of cars in Orlando, Tampa
at time n, then
nnn qpp 3.6.1
nnn qpq 7.4.1
n
n
n
n
qp
Tqp
1
1
11
OrlandoTampa
Long Term Behavior
Can we find the long-term behavior ?
7.4.3.6.
T
0
0
2
22
1
1
qp
Tqp
Tqp
Tqp n
n
n
n
n
n
n
This matrix notation gives n-step transitions
61.52.39.48.2T
583.556.417.444.3T
5749.5668.4251.4332.4T
5717.5710.4283.4290.6T
Long Term Behavior
If 1,lim
ba
bbaa
T k
kthen
bbaa
bbaa
T
baba
ba
T7.4.3.6.
ba
ba
T
5714.4286.
ba
MATLAB Experiment>> T = [.6 .3;.4 .7]
T =
0.6000 0.3000 0.4000 0.7000
>> for k = 1:20Tk = T^k;a1(k) = Tk(1,1);b1(k) = Tk(2,1);a2(k) = Tk(1,2);b2(k) = Tk(2,2);End
>> plot([a1' a2' b1' b2'])
Radioactive DecayModel decay of an atom of Polonium 209 to Lead 205after its half life (=102 years), its state vector evolves as
1 2
21
0
121
Transition Matrix T=
10
2121
1. Po209, 2. Pb205
1615161
8781
4341
2121
01
10
10
Radioactive DecayState vector dynamics of # Po209 atoms remaining(after 102 years)
1 2
41
1
1000
21
41
21
2141
T
1. two Po209 2. one Po209 3. zero Po209
3214
1
21
1
21
Traffic Light
With fixed transition times of one minute
1 2 1
010001100
T
1. red 2. green 3. yellow
31 1
1
001
100
010
001
100
010
001
Convergence CriteriaTheorem (Perron-Frobenius) If T is an n x n transition matrix then the following three statements are true
]111[v
ST kk
lim(ii)
(i) 1 is an eigenvalue of T since vT = v where
exists (converges) if and only if all other
(iii) If there exists an integer p > 0 such that every entry of the matrix is positive then the if condition in (ii) is satisfied and every entry of w is positive.
PT
eigenvalues have modulus < 1,then the right eigenvector w (Tw = w) for eigenvalue 1 (normalized so the sum of the entries of w equals 1), is the steady state of T and each column of S equals w
Numerical Examples
7.4.3.6.
Matrix Eigenvalues Steady State Vector
15.05.
4.2.2.2.6.05.4.2.75.
1,3.
5714.4286.
1,5.
10
1,55.,2.
2500.1944.5556.
Suggested Reading&Problems in Textbook
6.1 Probabilistic Modelling with Discrete Systems, pages 217-222
http://aix1.uottawa.ca/~jkhoury/markov.htm
Recommended Websites
http://en.wikipedia.org/wiki/Markov_chain
http://www.eng.buffalo.edu/~kofke/applets/MarkovApplet1.html
Tutorial 5 Due Week 29 Sept – 3 Oct
Page 222. Problem 1.
Page 222. Problems 2
Page 222-223. Project Problem 1