IERG5300 Tutorial 1 Discrete-time Markov Chain Peter Chen Peng Adapted from Qiwen Wang’s Tutorial Materials.