Markov Chain Monte Carlo Method - 大阪大学kikuchi/kougi/MCMC_Kobe.pdfWhy temperature statistical...

Why temperature statistical mechanics Markov Chain Monte Carlo New methods

Markov Chain Monte Carlo Method

Macoto Kikuchi

Cybermedia Center, Osaka University

6th July 2017


Thermal Simulations

1 Why temperature2 Statistical mechanics in a nutshell3 Temperature in computers4 Introduction to Markov Chain Monte Carlo

method5 Remarks6 New methodology


Why temperature is important

Thermal motion (fluctuation) is important for

Liquid state

Ordeing phanomena (crystal growth)Soft matters

Macromolecules, polymer, gel

Biomolecules (in vivo, in vitro)DNA, Proteins, Membrane


Thermal fluctuation of Kinesin


cont.Phase transitions

liquid-solid, liquid-gas, paramagnet-ferromagnetSuperconductivity, Superfluiditymelting of metalic materials

Electric conductionResistance by lattice vibration


Ferromagnetic transition

0

0.2

0.4

0.6

0.8

1

0.6 0.8 1 1.2 1.4

m

T/Tc

0

2

4

6

8

10

0.6 0.8 1 1.2 1.4

chi

T/Tc

magnetization and magnetic susceptibility


computer simulations

To treat thermal effect in molecular-level simulations

Molecular dynamics (MD)

Markov Chain Monte Carlo method (MCMC orMetropolis method)

Both method is for simulating thermal equilibrium(and nonequilibrium, with special care)


Statistical mechanics in a nutshell

Consider a many-particle system

N ≃ 1023 in real systems

As many particles as we can treat in computersimulations

We would like to know properties of matters in thethermal equilibrium state


thermal equilibrium

Macroscopic state of matter reached after along time under a certain external condition

Stable as long as the external condition is keptunchangedDistinguished by only a few thermodynamic(macroscopic) quantities:

temperature, pressure, volume, total energy etc.


Two levels of ”state” of matter

Microscopic (molecular level) statesparticle configurations distinguishablemicroscopically

Macroscopic statesdistinguishable only by macroscopic(thermodynamic) quantities such as total energy

Thermal equilibrium is macroscopically still. Butfrom the microscopic point of view, the matterchanges its microscopic state rapidly.


Basic formula

Boltzman’s formula for entropy

S(E ) = kB logW (E )

W (E ): number of microscopic states havingenergy E

kB : Bolzman’s constant

Thermodynamic definition of temperature

1

T=

∂S(E )

∂E


Simple model system

Collection of N elements each of which can takeone of two states

two levels of energy

Energy of i -th element

ei = 0, ϵ

Energy of the total system

E =∑i

ei


Microscopic states

One assignment of enegy for all the elementsdefines one microscopic state

2N distinguishable microscopic states in total

Macroscopic states

Distinguishable by total energy E = nϵ (Weassume N ≫ n)

Number of the corresponding microscopicstates is

W (E ) = W (nϵ) =

(N

n

)


Consider a closed systemNo interaction with external environmentTotal energy E is kept unchanged(conservation law of energy)

Principle of equal weight

The microscopic states having the same totalenergy realize in the same probability

All the microscopic states of the same energyare equally probable to appear

While the total energy is kept constant, thesystem constantly itenerates from a microstateto another to another....

This is the basic assumption for thermal equilibriumstate


(math.) Stirling’s formula

log n! ≃ n log n − n

Boltzman’s entropy

S(nϵ) = kB log

(N

n

)≃ N logN − n log n − (N − n) log(N − n)


Temperature

1

T=

∂S

∂E

=kBϵ

∂

∂nlogW

=kBϵlog

N − n

n

≃ kBϵlog

N

n


Consider a part of the system (called ”subsystem”)consisting of m elements (N ≫ m, and estimate theprobability that the subsystem has the energy lϵ.

Number of microscopic states of the totalsystem in which the subsystem has the energylϵ:

ω(lϵ) =

(m

l

)(N −m

n − l

)Probability that such microscopic states realize:

P(lϵ) =ω(lϵ)

W (nϵ)


From the Stirling’s formula (via a tediouscalculation)

log

(N−mn−l

)(Nn

)=(N −m) log(N −m) + (N − n) log(N − n)

+ n log n − (n − l) log(n − l)

− (N −m − n + l) log(N −m − n + l)− N logN

≃l logn

N= − ϵ

kBT


Finally, we have an important result

P(lϵ) =

(m

l

)exp

(− lϵ

kBT

)Probability is

(Number of microscopic states of the subsystemhaving the energy lϵ)×(exp(−energy/kBT ))


General result for the thermal equilibrium state ofany system contacting with a very large system(heat bath) of the temperature T

Boltzman distributionAppearance probability of a microscopic statehaving the energy ε is

P(ε) ∼ exp

(− ε

kBT

)


Thermal averages

e.g. Average energy for the thermal equilibriumstate of the temperature T can be calculated as

⟨E ⟩ = 1

Z

∑i

εiexp

(− εikBT

)with

Z =∑i

exp

(− εikBT

)i is the index for microscopic states.


Markov Chain Monte Carlo

Sample microscopic states of a fixed temperatureusing the computer simulations of a Markov chainspecially designed to realize thermal equilibriumstate

The same method can also be used for BaysianinferenceSimilar to the Boltzman machine in the field ofAIAlso used in the simulated annealing foroptimizationBasis of Metropolis light transport in the fieldof CGetc.


Construct a Markov process of that the averagequantities (e.g. energy) in the steady state coincidewith the thermal averages at the temperature T

goal

limN→∞

1

N

N∑i=1

Ai = ⟨A⟩

l.h.s.: avarage in the steady state of theMarkov process

r.h.s.: thermal average


Markov process is defined by a set of the transitionprobability wij from jth microscopic state to ithstates

requirement1

0 ≤ wij ≤ 1

2 ∑i

wij = 1


Consider the probability distribution of microscopicstate i at t-th step Pi(t), then∑

i

Pi(t) = 1

One step of evolution of the state according to thetransition probability is

Pi(t + 1) =∑j

wijPj(t)


In the vector and matrix notation

P⃗(t + 1) = WP⃗(t)

W: Markov matrixThe large step limit

P⃗(∞) = limn→∞

W nP⃗(t0)

Simce the largest eigenvalue of the Markov matrix is1, P⃗∞ is a steady state that satisfies

WP⃗(∞) = P⃗(∞)


requirement for W

ErgodicitySystem at an arbitrary state can reach all thestates in finite steps

State space should be singly connected, otherwisethe steady state is not uniquely determined.

Since the number of states is finite, the steady stateis reached in a finite steps.


We require that the steady state coincides with thethermal equilibrium state

requirement

Pi(∞) ∝ exp

(− εikBT

)The following is the sufficient condition

Detailed balance

wij exp

(− εjkBT

)= wji exp

(− εikBT

)


or

Detailed balance 2

wij

wji= exp

(−∆εijkBT

)where

∆εij ≡ εi − εj

The most widely used transition probability is

Metropolis transition probability

wij = min

[1, exp

(−∆εijkBT

)]


ProblemThe state space is usually astronomically huge

In case of the two-state system with 1000elements (very small considering today’scomputing power), the number of themicroscopic states is 21000 ≃ 10300. Thus theMarkov matrix is 10300 × 10300.

Solution

Instead of having the distribution vector P⃗ , we carrya single microscopic state and follow its trajectory inthe state spece by simulating the stochastic process.


ProblemWe cannot obtain the absolute probability Pi ,becarse we do not know the number of microstatesω(ε). Instead, we sample microstates in the relativeprobability that is proportional to Pi

SolutionCompute only the thermal averages. Forget aboutcomputing Pi itself.


Procedure1 Prepare any initial state i2 Make a candidate state j for the next step3 Generate a random number R in [0, 1] and

compare to the transition probability wji

4 If R ≤ wji , change the state to j . Otherwise,keep the state i .

5 Repeat many times

After sufficiently long steps, the system reaches thethermal equilibrium state. After that, the statesobtained by the simulation are samples from thethermal equilibrium.


A simple example

Ising model

A model for ferromagnet, binary alloy, neuralnetwork etc.

Defined on a lattice with N lattice pointsTwo-state elements S(called ”spin”) arelocated on the lattice points.

each element can take one of the two statesS = ±1 (called ”up” and ”down”)Total number of the microscopic states is 2N


Ising model (cont’d)

If the two spins located in a neibouring latticepoints have the same value, they have theenergy −J , otherwise have the energy J

The energy of the two spins are defined as

εij = −JSiSj

(i , j indicates the lattice points)The total energy of the system is

ε = −J∑ij

SiSj

(The sum is taken over all the neighboring latticepoints)


Procedure (Metropolis method)

Preparation: Assign +1 or −1 to all the spinsFlip:

Choose a spin to flipCalculate the energy difference ∆ε due to a flip.Note that the energy difference can be calculatedlocallyIf ∆ε < 0 then flip the spin. Otherwise make a

random number R . If R < exp(− ∆ε

kBT

)then flip

the spin

Repeat many times


Remarks

Initial relaxationMicroscopic states obtained in the earlier stages ofthe simulation is not the sample from theequilibrium state, because the effect of the initial(arbitrary) state remains. Therefore, samples fromearlier steps should be discarded (thermalizationprocess


Sampling interval

Microscopic states obtained from nearby steps areclose to each other. So they are not the statisticallyindependent samples. Samples should be collectedwith a sufficient interval.


Statistical analysis

Since we have only a small fraction of microstatesamong all the possible states, thermal averages aresuffered from statistical error. So, the standarderror analyses as those used in experimental scienceshould be employed. In that sense, MCMC is a kindof computer ”experiment”. Many sophisticatedstatistical analysis methods have been proposed.


Random number

Random numbers are pseudorandom

Random numbers generated (RNG) by anyalgorithm are not truely random. Therefore, qualityof the random number generator is important.

ex. Mersenne-Twister (MT) is one of thecandidate of the good RNG

Physical RNG using noises in electric circuit isa good candidate, because it’s truely random.But such RNGs are not always available. Andalso they lack the repeatability.


CautionThere are very ”bad” RNGs, sometimes even instandard libraries. Thus selecting a good RNGis really important in MCMC.

There is no RNG that is good for any purpose.So, the RNGs should be tested for eachapplication. For example, MT is good for mostMCMC simulations, but is not appropriate forusing in encryption.


New methods and rare event sampling


Histogram reweighting

If we get the histogram of the energy H(ε,T ) byMCMC at temperature T

H(ε,T ) ∝ ω(ε) exp

(− ε

kBT

)where ω(ε) is the number of microstates with ε.Then the energy distribution for a differenttemperature T ′ can be estimated as

P(ε,T ′) ∝ H(ε,T ) exp

{(1

kBT− 1

kBT ′

)ε

}


-1800 -1600 -1400 -1200 -1000E

0

1000

2000

3000P(E;β)

図でのシミュレーションで作ったヒストグラム

-1800 -1600 -1400 -1200 -1000E

0

2∗10-4

4∗10-4

6∗10-4

8∗10-4

P(E;β’)

図からへシフトしたヒストグラム

-1800 -1600 -1400 -1200 -1000E

0

1∗1019

2∗1019

P(E;β’)


Original histogram


-1800 -1600 -1400 -1200 -1000E

0

1000

2000

3000

P(E;β)


-1800 -1600 -1400 -1200 -1000E

0

2∗10-4

4∗10-4

6∗10-4

8∗10-4

P(E;β’)


-1800 -1600 -1400 -1200 -1000E

0

1∗1019

2∗1019

P(E;β’)


-1800 -1600 -1400 -1200 -1000E

0

1000

2000

3000

P(E;β)


-1800 -1600 -1400 -1200 -1000E

0

2∗10-4

4∗10-4

6∗10-4

8∗10-4

P(E;β’)


-1800 -1600 -1400 -1200 -1000E

0

1∗1019

2∗1019

P(E;β’)


Reweighted histogram

Reweighted histograms become jaggy at theshifted side. But it is a basis of the extendedensemble methods


Extended ensemble methodsAccelarate the relaxation especially in lowtemperature or crossing the energy barrier

Calculate thermal quantities for wide range oftemperatures by a single simulation

Count the number of microstates and computeentropy

Rare event sampling


Exchange method

Simulate many identical systems with differenttemperature simultaneously, and exchange theirtemperature from time to time according to thetransition rate that satisfies the following condition:

W (1, 2 → 2, 1)

W (2, 1 → 1, 2)= exp

{−(

1

kBT1− 1

kBT2

)(ε2 − ε1)

}Then simultaneous equilibrium at all thetemperatures is reaches.


Multicanonical methodEquilibrium distribution is made inverselyproportional to ω(ε)

P(ε) ∝ 1

ω(ε)

Very broad histogram is obtainedTransition rate is determinde through”Learning process” (machine learning)

Most frequently used method is Wang-Landaumethod

Thermal equilibrium can be obtained by thehistogram reweighting method


Conceptual difference between the conventionalMCMC and the multicanonical MC


Multicanonical method to unphysical direction

In order to bipass some physical constraint, we canuse multicanonical method that relaxes theconstraint

example: Multi-self-overlap ensemble for latticepolymer


Relaxing the self-avoidance condition, the polymercan readily transit among these three configurations.


Rare event sampling using multicanonical method

We can generate very rare configuration usingmulticanonical MC and estimate its appearanceprobability

This method can be applied even tonon-physical systems by defining appropriateenergy function


example

Count the number of magic squares

Number of 30× 30 magic square wasestimated to be 6.56(29)× 102056


Report: Make a MCMC program for Ising model.

Markov Chain Monte Carlo Method - 大阪大学kikuchi/kougi/MCMC_Kobe.pdfWhy temperature statistical...

Documents

Transcript of Markov Chain Monte Carlo Method - 大阪大学kikuchi/kougi/MCMC_Kobe.pdfWhy temperature statistical...