Parameter Estimation in Stochastic Chemical Kinetic Models

by

Rishi Srivastava

A dissertation submitted in partial fulfillment of

the requirement for the degree of

Doctor of Philosophy

(Chemical Engineering)

at the

University of Wisconsin-Madison

2012

Date of final oral examination: 9/20/12

The dissertation is approved by the following members of the Final Oral Committee:

James B. Rawlings, Professor, Chemical and Biological Engineering

John Yin, Professor, Chemical and Biological Engineering

Michael D. Graham, Professor, Chemical and Biological Engineering

Jennifer L. Reed, Assistant Professor, Chemical and Biological Engineering

David F. Anderson, Assistant Professor, Mathematics

Copyright by Rishi Srivastava 2012

All Rights Reserved

iTo my mom Manju

DISCARD THIS PAGE

ii

TABLE OF CONTENTS

Page

LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi

ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 The stochastic chemical kinetic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.1 The master equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.2 Exact simulation of stochastic chemical kinetic models: Stochastic simulation al-

gorithm (SSA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.3 Application of SSA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4 Approximate simulation of stochastic chemical kinetic models . . . . . . . . . . . . 8

2.4.1 leap and Langevin approximation . . . . . . . . . . . . . . . . . . . . . . . . 82.4.2 Reaction equilibrium approximation . . . . . . . . . . . . . . . . . . . . . . . 92.4.3 Quasi-steady state approximation . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Parameter estimation in stochastic chemical kinetic models . . . . . . . . . . . . . . . 11

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.2 The negative log likelihood minimization problem . . . . . . . . . . . . . . . . . . . 133.3 The algorithm for the negative log likelihood minimization problem . . . . . . . . . 14

3.3.1 The quadratic model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.3.2 The notion of adequacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3.3 Obtaining number of SSA simulations N and smoothing/noise parameter R 153.3.4 The UOBYQA-Fit algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.4 Estimation of confidence regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.4.1 The confidence region estimation algorithm . . . . . . . . . . . . . . . . . . . 183.4.2 The verification of confidence region algorithm . . . . . . . . . . . . . . . . . 20

3.5 Application: RNA dynamics in Escherichia coli . . . . . . . . . . . . . . . . . . . . . 20

iii

Page

3.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4 New methods to obtain sensitivities of stochastic chemical kinetic models . . . . . . 25

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254.2 The estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.3.1 Sensitivity of expected value of population of a species in a reaction network 274.3.2 Sensitivity of the negative log likelihood function . . . . . . . . . . . . . . . . 294.3.3 Sensitivity of a rare state probability . . . . . . . . . . . . . . . . . . . . . . . . 354.3.4 Sensitivity of a fast fluctuating species . . . . . . . . . . . . . . . . . . . . . . . 40

4.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5 Model reduction using the Stochastic Quasi-Steady-State Assumption . . . . . . . . . 44

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.2.1 Pap operon regulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465.2.2 Biochemical Oscillator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 525.2.3 Fast fluctuation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

6 Conclusions and Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

APPENDICES

Appendix A: Proof of exact likelihood of the experimental data . . . . . . . . . . . . . . 81Appendix B: Supplementary information for Chapter 4 . . . . . . . . . . . . . . . . . . . 82Appendix C: Supporting Information for Chapter 5 . . . . . . . . . . . . . . . . . . . . . 83

DISCARD THIS PAGE

iv

LIST OF TABLES

Table Page

3.1 System and optimization parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.2 Parameter estimates and confidence regions . . . . . . . . . . . . . . . . . . . . . . . . 23

4.1 Parameter values for example 4.3.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.2 Parameter value for example 4.3.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.3 Reaction stoichiometry and reaction rates for example 4.3.3. . . . . . . . . . . . . . . . 37

4.4 Parameter values for example 4.3.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.1 Reaction stoichiometry and reaction rates for pap operon regulation. . . . . . . . . . . 46

5.2 Parameters for the biochemical oscillator . . . . . . . . . . . . . . . . . . . . . . . . . . 53

5.3 Comparison of the full, sQSPA reduced and dQC reduced models . . . . . . . . . . . . 58

5.4 Initial population and reaction rate constants for the fast fluctuation example. . . . . 60

B.1 Experimental data for example 4.3.2, Sensitivity of negative log likelihood function 82

DISCARD THIS PAGE

vLIST OF FIGURES

Figure Page

2.1 Multiple SSA simulations and generation of histogram from them . . . . . . . . . . . . 6

2.2 Evolving Probability density function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.1 10 replicates of mRNA vs time experimental data . . . . . . . . . . . . . . . . . . . . . . 21

3.2 An ellipse and its bounding box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.3 Verification of confidence region using bootstrapping . . . . . . . . . . . . . . . . . . . 23

4.1 A typical simulation of the network involving reaction (4.1) and (4.2). . . . . . . . . . . 27

4.2 Comparison of CRN and CFD estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

4.3 Experimental data for example 4.3.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.4 Convergence of sensitivity estimate from CRN estimator . . . . . . . . . . . . . . . . . 33

4.5 Convergence of sensitivity estimate from CFD estimator . . . . . . . . . . . . . . . . . 34

4.6 Comparison of convergences of CRN and CFD estimators . . . . . . . . . . . . . . . . . 35

4.7 Schematic diagram of the Pap regulatory network . . . . . . . . . . . . . . . . . . . . . 36

4.8 Reduced system in the slow time scale regime . . . . . . . . . . . . . . . . . . . . . . . . 38

4.9 Estimated sensitivity from the CRN, CFD and SRN estimators . . . . . . . . . . . . . . 40

4.10 A typical SSA simulation of the network of reactions (4.33)(4.35). . . . . . . . . . . . . 42

4.11 Comparison of standard deviations of CRN and CFD estimators . . . . . . . . . . . . . 43

5.1 Schematic diagram of the Pap regulatory network . . . . . . . . . . . . . . . . . . . . . 47

5.2 Reduced system in the slow time scale regime . . . . . . . . . . . . . . . . . . . . . . . . 47

vi

Figure Page

5.3 Comparison of full model and sQSPA slow time scale reduced model at t = 10s . . . . 49

5.4 A comparison of the full model and the sQSPA reduced model of pap operon regulation 51

5.5 Stochastic simulation of the biochemical oscillator. . . . . . . . . . . . . . . . . . . . . 56

5.6 Sensitivity of the full and sQSPA reduced models to parameters . . . . . . . . . . . . . 59

5.7 Comparison of the full model simulation and the sQSPA- simulation of the fastfluctuation example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

5.8 The step size and the frequency of the fluctuation of SSA simulation versus time . . . 63

5.9 The noise in full SSA and hybrid SSA- for the rapidly increasing species . . . . . . . . 65

5.10 Comparison of probability density of C . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

vii

ABSTRACT

Recent years have seen increasing popularity of stochastic chemical kinetic models due to their ability

to explain and model several critical biological phenomena. Several developments in high resolution

fluorescence microscopy have enabled researchers to obtain protein and mRNA data on the single cell

level. The availability of these data along with the knowledge that the system is governed by a

stochastic chemical kinetic model leads to the problem of parameter estimation. This thesis develops a

new method of parameter estimation for stochastic chemical kinetic models. There are three

components of the new method. First, we propose a new expression for likelihood of the experimental

data. Second, we use sample path optimization along with UOBYQA-Fit, a variant of of Powells

unconstrained optimization by quadratic approximation, for optimization. Third, we use a variant of

Efrons percentile bootstrapping method to estimate the confidence regions for the parameter

estimates. We apply the parameter estimation method in an RNA dynamics model of E. coli. We test

the parameter estimates obtained and the confidence regions in this model. The testing of the

parameter estimation method demonstrates the efficiency, reliability and accuracy of the new method.

1Chapter 1

Introduction

The traditional approach to model biological systems deterministically had seen difficulties in

explaining increasing evidence of stochastic phenomena [61, 101, 107]. A deterministic model can only

simulate one pathway of evolution for a particular network. However there is usually noise in biological

networks due to the presence of several components at small copy numbers. This noise can lead to

phenomena usually inexplicable by a deterministic model. A stochastic model which mimics the

typical noise and average behavior of these biological systems is imperative for successful explanation

of several biological phenomena.

Elowitz et al. [26] in their experimental work on Escherichia coli (E. coli) have shown that the intrinsic

stochastic nature of gene expression process and cell to cell differences contribute significantly in

explaining overall variation in gene expression. This work reinforces the idea that identical cells under

similar conditions1 can exhibit different expression of the same gene. Arkin et al. [5], in their work on

phage2 , infected E. coli cells have shown that a stochastic model can predict the number of lytic

(destroying cell) and lysogenic (staying dormant) type of infections. Their work suggests that stochastic

fluctuations in concentration of two regulatory proteins acting at low concentrations can produce

pathways that might lead to phenotype bifurcation ultimately. Works on HIV [107] have shown that

viruses adopt stochastic networks, whereby a probabilistic pathway for active infection and latent

infection may be chosen to optimize the chances of viral-escape from cellular defenses.

Intrinsic noise with its ability to drive processes of significant biological interests and stochastic

chemical kinetic models with their ability to explain intrinsic noise have seen significant attention

recently [5, 7, 26, 57, 72, 77, 85]. Intrinsic noise depicts itself when similar cells give rise to markedly

1Infection by genetically identical viruses under identical environmental conditions2A virus that infects a bacteria

2different behaviors [29, 111]. Stochastic chemical kinetic model, which is known as continuous time

discrete jump Markov chain in Mathematics and Probability theory, models such similar cells as

realizations of a Markov chain. One useful tool for understanding these models is the chemical master

equation, which describes the evolution of the probability density of the system. The solution of the

master equation is computationally tractable only for simple systems. Rather, approximation

techniques such as finite state projections [74] or the stochastic simulation algorithm (SSA) [33, 34] are

employed to reconstruct a systems probability distribution and statistics (usually the mean and

variance). Applying these techniques to solve models of biological processes leads to significant

improvements in our understanding of intrinsic noise and its effect on cellular behavior.

With the advent of fluorescence microscopy [66], which can give high-throughput data for gene

expression, data at the protein level have become available. Usually proteins in cells are present in the

range of 103106, however mRNA are present in the level of 11000 molecules. Recently mRNAdata [40, 41, 68] have also been quantified using fluorescence microscopy. The availability of mRNA

level data, along with stochastic chemical kinetic models for the experimental systems makes

parameter identification of intracellular stochastic model possible for the first time.

While the motivation of the parameter estimation method described in this thesis comes from biology,

the method is general enough and is applicable to any scenario where continuous time discrete jump

Markov chain is the underlying model.

3Overview of Thesis

Chapter 2 The stochastic chemical kinetic model : In the following chapter, we introduce the

stochastic chemical kinetic modeling framework. We define the governing equation, the method to

simulate, and approximate methods to simulate a stochastic chemical kinetic model.

Chapter 3 Parameter estimation in stochastic chemical kinetic models: This chapter is the

main contribution of this thesis. We develop a novel method of parameter estimation in stochastic

chemical kinetic models.

Chapter 4 New methods to obtain sensitivities of stochastic chemical kinetic models :

Sensitivities are a powerful tool both in parameter estimation and in model analysis . This chapter

compares various methods of sensitivity estimation in stochastic chemical kinetic models.

Chapter 5 Model reduction using the Stochastic Quasi-Steady-State Assumption:

Reducing a stochastic chemical kinetic model can lead to ease in parameter estimation both due to the

reduced model having fewer and better constrained parameters and due to the speed up in the

simulation time of the reduced model. This chapter presents applications of stochastic

Quasi-Steady-State Assumption and a new model reduction method for certain types of models.

Chapter 6 Conclusions and Future Work: We end with a summary of the contributions of this

thesis and give recommendations for further work.

4Chapter 2

The stochastic chemical kinetic model

2.1 The master equation

The dynamic state of a well stirred mixture of N 1 species {S1,S2, . . . ,SN } under the influence of M 1reaction channels {R1,R2, . . . ,RM } can be specified as

X(t)= [X1(t ), X2(t ), . . . , XN (t )]

in which

Xi (t )=The population of species i at timet , i {1,2, . . . , N }

A probability density P (x, t ) for the random variable X can be defined as the probability of system being

in state x at time t , where we note that even though time t is a continuous variable the random variable

X takes on finitely many or countably infinite values. The underlying governing equation for P (x, t ) is a

linear first order differential equation known as master equation which can be written as :

dP (x, t )

d t=

Mj=1

k j a j (x j , t )P (x j )k j a j (x)P (x, t ) (2.1)

In equation 2.1, j is stoichiometric vector of j th reaction of the reaction network. If we had a method

whereby we could solve explicitly the time evolution of P (x, t ) for all x in the state space of the

biological network, we would be done. However, solving this system of ODEs in equation 2.1 either

analytically or numerically is challenging even for moderately large state-space systems. There are

several methods to solve the master equation approximately [9, 17, 27, 52, 60, 74]. However, the quality

of the approximation obtained by these methods depends both upon the size of the state space and a

good intuition about the concentration of the probability density.

There is a method [32] available to generate realization of this system, instead of solving for the time

evolution of the probabilities of different states explicitly. This method is known as the Gillespie direct

5method or stochastic simulation algorithm (SSA) and usually this is the preferred way to simulate

complex biological networks.

2.2 Exact simulation of stochastic chemical kinetic models: Stochasticsimulation algorithm (SSA)

The stochastic simulation algorithm (SSA) or Gillespies direct method is the most widely method used

for the simulation of stochastic chemical kinetic models. SSA is a method to obtain exact samples from

the master equation. The algorithm is as follows:

Input: Initial population vector x0 Rn , stoichiometric matrix Rmn in which m is the number ofreactions, vector of rate constants k, End time tend

Output: Vector of time points, T , corresponding to jump times, and matrix X , consisting of species

population at the jump times in T

1. Initialize t = 0, x = x0, T (1)= 0, X (:,1)= x0, l = 1

2. While t tend

(a) For each i = 1 to m, calculate a(i )= k(i )x(i ). Calculate rtot =mi=1 a(i )(b) Generate two uniform random numbers u1,u2 and set = logu1rtot(c) Find reaction , such that,

1i=1 a(i ) u2

i=1 a(i )

(d) For each i = 1 to n , update x(i ) using x(i ) x(i )+mi=1(, i )(e) t t +

(f) l l +1

(g) Store the results: T (l )= t , X (:, l )= x

(h) Go to the step 2 to check the while condition

3. Return T and X

62.3 Application of SSA

Consider the following reversible reaction

Ak1 B (2.2)

Bk2 A (2.3)

with nA0 = 100,nB0 = 0 and k1 = 2,k2 = 1. Figure 2.1(a) shows 3 different runs of SSA algorithmdescribed in section 2.2. Figure 2.1(b) shows the sample probability density of the system using 5000

SSA simulations at time t = 2, and illustrates usefulness of SSA in reconstructing probability densityfunction described by the master equation.

0

20

40

60

80

100

0 0.5 1 1.5 2 2.5 3

ni

time (sec)

nA

nB

(a)

0

50

100

150

200

250

300

350

400

450

20 25 30 35 40 45 50 55

Co

un

t

nA

(b)

Figure 2.1: (a) Multiple SSA simulations of system (2.2)(2.3) (b) Histogram at t=2sec, for 5000

SSA simulations

In-fact as the number of SSA simulations increase the SSA reconstructed probability density function,

starts getting close to the true probability density function given by the master equation.

For the same set of reactions (2.2) and (2.3), writing master equations gives us 101 coupled ODES, in

101 states. Figure 2.2 shows the probability density, obtained by solving the master equation 2.1, as a

function of number of A molecules and time.

7P (nA)

0

25

50

75

100

nA0 1 2 3 4Time (sec)

0

0.05

0.1

0.15

Figure 2.2: Evolving Probability density function

At time t = 0, all the probability resides in the state nA = 100. As time progresses the probability starts todiffuse to other states. Finally, at time t = 4, the probability distribution is fairly symmetric, withmaxima around nA = 50.

82.4 Approximate simulation of stochastic chemical kinetic models

The original stochastic simulation algorithm (SSA) proposed by Gillespie [32], tracks all the

microscopic reaction events individually and generates realization of the master equation exactly.

Tracking individual reactions events microscopically in SSA is both its strength and weakness. The

strength is that one gets exact realizations of the underlying micro-physical phenomena of the

individual reactions and the weakness is that it becomes computationally intensive for complex

biological networks. There have been several attempts to decrease the computational burden of the

SSA. One of the improvements of Gillespies first reaction method was proposed by Gibson and

Bruck [30]. Their method exploits the memorylessness property of exponential random variables and

some efficient data structures like indexed priority queue and graphs to perform the SSA.

Next we discuss three classes of model reduction normally found in the stochastic modeling literature.

We briefly discuss each and describe their relevance to the biological systems of interest here.

2.4.1 leap and Langevin approximation

To simulate multiple reaction events in a single step, Gillespie [35] has proposed a -leap method. The

-leap method essentially relies on a large population of all species and a specific selection method of

. The time interval is chosen in such a way that within this interval number of times each reaction

channel fires is large yet none of the reaction propensities change appreciably. Under these conditions

the number of times each reaction channel fires is obtained by a Poisson random variable. However

their selection method [35] can lead to negative population of species. Many

researchers [1, 3, 11, 12, 37, 58, 59, 63, 86] have proposed theoretical analysis and improvement of the

leap method that improves the efficiency and negative population problem of the original leap

method [35].

An extension of the -leap method, where under suitable conditions one can approximate the number

of times each reaction fires with a normally distributed random variable instead of a Poisson random

variable, gives rise to the well known Langevin approach. However it should be noted that both the

-leap and the Langevin approach as described by Gillespie [35] rely on a large population of every

species, which is rarely the case in the biological systems of interest here.

92.4.2 Reaction equilibrium approximation

A usual challenge of simulation of Gillespies SSA occurs due to the presence of several fast and slow

reactions together in the same reaction network. This problem is commonly known as the reaction

equilibrium problem. A fast reaction fires many times within a fast time scale whereas a slow reaction

fires rarely in this fast time scale. A full SSA on such systems spends the majority of its run time in the

simulation of these fast reactions which are usually transitory in nature and are not of any particular

interest for the evolution of the system in the slow time scale.

There have been efforts that separate the fast and slow subsystems to reduce the simulation burden of

full SSA [10, 2022, 50, 51]. Mastny [70] describes the issue in the approach of slow scale SSA adopted

by Cao et al. [10] and how one can circumvent this problem. Several papers have been directed

towards addressing the fast and slow reactions type of networks [43, 9294]. However, most of the

systems of interest here, do not have clear cut partitioning among the reactions and many of the

reactions in the network involve species at low copy numbers. As shown by Griffith et al. [44], it is

incorrect to put fast reactions involving species at low copy number in a fast reaction subset.

2.4.3 Quasi-steady state approximation

In deterministic kinetics, a quasi-steady state approximation (QSSA) is usually applied for a highly

reactive species, whereby one assumes that after an initial transience (fast time scale) the rate of

change of concentration of a species always remains at zero [15]. This usually leads to a reduced

mechanism in the slow time scale in which one can remove a highly reactive species from the model.

The advantage of using the QSSA in the deterministic case is that one does not have to estimate very

large rate constants from the data. The same advantage remains in the case of stochastic kinetic

models simplified by the QSSA. However, this simplification, at times can give significant speedup

along with the ease of parameter estimation.

There have been a few attempts to utilize the concept of QSSA in the stochastic settings. Rao and Arkin

[84] propose a QSSA approximation for stochastic settings drawing analogy from a deterministic QSSA

reduction. They assume that conditional probability density of population of fast species conditioned

on slow species is Markovian in nature and set the derivative of this conditional probability density to

zero. Their assumption that conditional probability of population of fast species conditioned on slow

10

species is Markovian seems to be rather ad-hoc and it is unclear if this approximation is indeed true in

general. Mastny et al. [71] have recently treated QSSA in stochastic settings by application of singular

perturbation analysis. They have shown that in the limit of a certain parameter approaching to zero,

their reduced model indeed converges to the full model. Appearance of the Quasi-Steady-State(QSS) is

also not common in the systems of interest here. Even when the system does have a QSS, identification

of the QSS species is not straight forward.

2.5 Summary

In this chapter we introduced the master equation, the probability evolution equation for stochastic

chemical kinetic models. We described the SSA, a method to generate exact samples from the master

equation. Finally, we described approximate methods to simulate stochastic chemical kinetic models

and the their relevance to the systems of interest here.

11

Chapter 3

Parameter estimation in stochastic chemical kinetic models

3.1 Introduction

Unlike ordinary differential equation based models for which fairly robust and efficient parameter

estimation techniques exist, there are not many such techniques for stochastic chemical kinetic

models. Boys et al. [8] propose generating many samples of the full master equation consistent with the

given measurement. They then use Markov chain Monte Carlo to obtain the posterior distribution of

the parameter. The first step is computationally intractable for the models of interest here. Golightly et

al. [42] use the Fokker-Planck approximation of the master equation. This diffusion approximation is

not generally applicable in stochastic chemical kinetics. Tian et al. [102] express the likelihood p(y |) asa product of transition densities p(y |)=ni=1 p(yi+1|yi ,). Each p(yi+1|yi ,) is evaluated using 5000SSA simulations. A genetic algorithm is used to maximize p(y |). This procedure is computationallyinefficient because 5000 SSA simulations are used for each transition. Reinker et al. [89] calculate the

likelihood analytically using an artificial maximum number of reactions that can occur within a given

time interval. They use a quasi-newton method to maximize the likelihood. The assumption about the

maximum number of reactions is unrealistic. Both Sisson et al. [99] and Toni et al. [103] use

approximate Bayesian computation approach but their approach requires the use of summary statistics

and a distance metric. It is difficult to extend their approach to the stochastic chemical kinetic setting.

Poovathingal et al. [81] propose to evaluate the likelihood using the solution to the master equation.

Their proposed function is not the likelihood, but some other merit function. They estimate the

solution of the master equation by SSA simulations. This is computationally intensive and requires a

binning strategy. They use directed evolution to optimize. Henderson et al. [53] replace stochastic

chemical kinetic model with a statistical model, and use the statistical model for obtaining parameter

estimates. The replacement with the statistical model makes their method limited to the cases where

12

such a replacement is accurate and possible. Wang et al. [110] propose a stochastic gradient descent

method that requires generating many samples of the master equation consistent with the

measurement. They then use reversible jump Markov chain Monte Carlo to obtain the posterior

distribution of the parameter. The first step is computationally intractable for the models of interest

here.

We present here a novel method of parameter estimation in stochastic chemical kinetic models. As

noted by Wets [19, 108], the method developed here employs tight integration of statistical sampling

and mathematical optimization. The method is based on constructing a new expression for the

likelihood of the data. The likelihood expression is in the form of an expectation of a smooth function

of the data, model parameters, and a smoothing parameter. We then use sample path

method [28, 46, 47, 55, 56, 78, 79, 9698] to approximate the likelihood, which is in the form of an

expected value, to a sample average. The basic idea behind the sample path method is that in

calculating the objective function, i.e. the likelihood, we use common random numbers. Since the

gradient and Hessian of the likelihood are hard to estimate both due to the requirement of computation

and the inaccuracy in their estimation, we pursue a derivative free optimization method, UOBYQA [83].

For some other derivative free methods, see references [13, 14, 69]. We adapt UOBYQA to UOBYQA-Fit

that suits the optimization problem of interest here. Unlike Deng and Ferris [16], who assume that the

objective function at the points of the fitting set are components of a multivariate normal to obtain the

number of sample paths for different iterations, we keep the number of sample paths fixed.

Point estimates of the parameter alone are not useful when one is dealing with a stochastic model.

Confidence region around the point estimate tells us the precision of the point estimate and how much

information is contained in the experimental data. We will discuss in a subsequent section that neither

finite sample distribution nor asymptotic distribution of parameter estimates are obtainable. Bootstrap

estimation of confidence regions becomes handy when one does not have much information about the

distribution of the point estimates. We use a variant of Efrons percentile method [2325] to estimate

the parameter confidence region. Along with estimation of confidence region, we test the quality of this

confidence region. The successful testing of the confidence region is a reliable indicator that the

UOBYQA-Fit along with the variant of the percentile method solves the parameter estimation problem

reliably and accurately.

13

This chapter is arranged as follows. In section 3.2, we provide the expression of the likelihood of the

data and describe the optimization problem. Section 3.3 describes the UOBYQA-Fit algorithm and

provides the pseudo-code for it. Section 3.4 describes the bootstrapping confidence estimation

algorithm and provides the pseudo code for both the confidence region estimation algorithm and the

verification of the confidence region algorithm. An application of the technique described in this

chapter is given in section 3.5. Finally, section 3.6 discusses the conclusions of this chapter and

summarizes the contributions.

3.2 The negative log likelihood minimization problem

To obtain the parameter estimates, we pursue likelihood maximization. The likelihood of data are

poorly conditioned numbers, and to improve the conditioning of these numbers we pursue an

equivalent problem to the likelihood maximization, the minimization of the negative log likelihood. We

show in Appendix A that the exact likelihood of the experimental data, composed of m replicates

y = {y1, y2, . . . , ym}, is given by

L(y |)= limR0

1

(2pi)mnd /2|R|m/2m

j=1E[e

1/2(y jx)R1(y jx)] (3.1)

in which x is the random vector of the population of species coming from the SSA simulation of the

model, m is the number of replicates, nd is the number of sample points in each replicate, R is an

nd nd positive definite matrix representing both measurement noise and smoothing, and is thevector of parameters that we are trying to estimate. For any positive definite value of R, the right hand

side of equation (3.1) after the limit is an approximation to the exact likelihood. For such a value of

R 6= 0, using sample mean as the estimator for the expectation in equation (3.1), we obtain an estimatorof L(y |)

L(y |, N )= 1(2pi)mnd /2|R|m/2N m

mj=1

Ni=1

e(1/2)(y jxi ())R1(y jxi ()) (3.2)

As N and R 0, the estimated likelihood, L(y |, N ), approaches the exact likelihood L(y |). Theestimate of negative log likelihood which we use as objective function for minimization is:

(, N )= log L(y |, N )=m

j=1log

[1

N (2pi)nd /2|R|1/2N

i=1e(1/2)(y jxi ())

R1(y jxi ())]

(3.3)

14

The parameter estimates are then given by the minimization of the negative log likelihood

= argmin

(, N ) (3.4)

3.3 The algorithm for the negative log likelihood minimization problem

We rewrite equation (3.4) to denote the dependence on randomness which itself depends on all the

random numbers used to generate N SSA sample paths.

(N ,)= argmin

(, N ,) (3.5)

To use sample path optimization, we freeze the N streams of random numbers and use these streams

repeatedly to calculate the value of (, N ,) for different value of . The freezing of the random

numbers means that the problem (3.5) becomes a deterministic optimization problem. Furthermore,

we define to be the logarithms of the rate constants to keep the deterministic optimization problem

unconstrained. We develop UOBYQA-Fit, which is a variant of Powells UOBYQA [83], for solving the

sample path optimization problem (3.5). The lack of availability of good derivative estimates for the

objective function (, N ,) makes use of derivative free methods like UOBYQA attractive here. Like

UOBYQA, UOBYQA-Fit is a model based approach which constructs a series of quadratic models

approximating the objective function. Like UOBYQA, UOBYQA-Fit uses a trust region

framework [73, 76]. However, unlike UOBYQA, which does exact interpolation to obtain the quadratic

model, UOBYQA-Fit fits the quadratic model to the sample points. As described in section 3.3.2, the use

of fitting instead of exact interpolation enables UOBYQA-Fit to have a fairly mild adequacy criteria for

the fitting points.

3.3.1 The quadratic model

At every iteration of the algorithm, UOBYQA-Fit fits a quadratic model

Q()= c+ g T (k )+1

2(k )T G(k ) (3.6)

to an adequate set of sample points Sk = {z1, z2, . . . , zL}. The value of the objective function is obtainedusing (3.3) along with N SSA simulations. We describe the notion of adequacy and how we obtain the

number N in sections 3.3.2 and 3.3.3, respectively. The fitting procedure solves the following

15

minimization problem.

{ck , gk ,Gk }= argminc,g ,G

Li=1

(Q(zi ) (zi , N ,))2 (3.7)

The point k is the center of the trust region, ck is a scalar, gk is a vector in Rn , and Gk is a symmetric

Rnn matrix. The quadratic model is expected to approximate the function around k . The number

of points in the fitting set Sk is chosen to be double the number of points that can determine a unique

quadratic, i.e.

L = (n+1)(n+2) (3.8)

The fitted quadratic that is obtained after solving minimization problem (3.7) is

Qk ()= ck + g Tk (k )+1

2(k )T Gk (k ) (3.9)

3.3.2 The notion of adequacy

Unlike UOBYQA, which requires that no nonzero quadratic function should vanish at all the

interpolation points, we impose an adequacy criteria that is relatively mild. The points in the fitting set

must neither be too close to each other, nor too far away from the current iterate k . We ensure that any

two points in the fitting set are no closer than kL , where closeness is measured by the Euclidean

distance. We also ensure that

zi k 2k , zi Sk .

The adequacy criteria ensures that the fitted quadratic model Qk is a good local approximation to the

function in the neighborhood of k .

3.3.3 Obtaining number of SSA simulations N and smoothing/noiseparameter R

As evident from the equation (3.3), the estimation of negative log likelihood requires the values of N

and R. In the absence of any external information about measurement noise, we use

R =Indnd (3.10)

in which value of depends upon N by

= cpN

. (3.11)

16

The value of c is a problem dependent parameter. To obtain N , we implement the UOBYQA-Fit

algorithm described in the next section, with N =Ni {1,2,3 . . .} values to obtain several optimalparameter estimates i . We choose the smallest N =Ni such that

i i1

in which is a small number. With c and the obtained N values, we use equation (3.11) to get value of

. , which determines R by equation (3.10), and N , the number of SSA simulations, are the two

required parameter for the estimation of of equation (3.3).

3.3.4 The UOBYQA-Fit algorithm

In this section we present the core algorithm, which inherits several basic features of the UOBYQA

algorithm. For complete details of UOBYQA, please see [83].

Starting the algorithm requires a starting point 0, and an initial trust region radius 0. At the iteration

k, we fit the quadratic model (3.6) as described in section 3.3.1. As in a classic trust region method, a

new promising point k+1 is obtained by solving the sub-problem

minsRn

Qk (k + s), subject to s k (3.12)

The new promising point s is accepted if the degree of agreement

k =(k ) (k + s)

Qk (k )Qk (k + s)(3.13)

is large enough. Otherwise the next iterate k+1 = k . If k is large enough, which indicates good matchbetween the Quadratic model Qk and the function , the point k + s is set as the iterate k+1 and it isput inside the interpolation set Sk+1.

UOBYQA-Fit algorithm

Input

Objective function which depends on the model, experimental data, N , R

Trust region parameters

0 a 0 1, 0 0 1 1 2,0,end

17

Starting point 0

Output

Parameter estimate and objective function value, , at the parameter estimate

1. Generate the initial fitting set S1. Using equation (3.3), evaluate for each point in the fitting set S1.

The first iterate is the point 1 S1 that minimizes over the points in the set S1

2. For iteration k = 1,2, . . .

(a) Construct a quadratic model of the form (3.9) which fits points in Sk by solving the optimization

problem (3.7)

(b) Solve the trust region problem (3.12). Evaluate at the new point k + s, and compute theagreement ratio, k , defined in equation (3.13)

(c) If k 1, increase the trust radius k by using

k k (1+2)/2

otherwise, decrease the trust radius by using

k k (0+1)/2

(d) If k 0, accept the point k + s as the next iterate

k k + s

otherwise, set the next iterate as the current iterate

k k

(e) If k a , improve the quality of fitting points in Sk as described in section 3.3.2

(f) Check whether any of the termination criteria is satisfied, otherwise repeat the loop. The

termination criteria include hitting the limit on the number of function evaluations and

k end

3. Evaluate and return the final solution point and value of the objective function at .

18

3.4 Estimation of confidence regions

Bootstrapping is the method of choice to estimate confidence regions when we do not have any

information about the distribution of the sample statistics [6, 18, 23, 49]. The parameter estimates

obtained for a given experimental data y , consisting of m replicates, depend upon the experimental

data y . We do not have any specific information about the distribution of y , implying that the

distribution of sample statistics, i.e. parameter estimate , is also unavailable to us.

To generate a large population of replicates from the experimental data y , we make cy copies of y and

generate a bootstrapped population of replicates Y . From the bootstrapped population of replicates,

we pick m replicates by doing random sampling with replacement. This random sampling with

replacement gives us one bootstrapped experimental data set. We repeat this random sampling with

replacement NB times to obtain NB sets of bootstrapped experimental data, each consisting of m

replicates. Next we present the algorithm to estimate the parameter confidence region, which is an

ellipsoid, given the experimental data y and confidence level .

3.4.1 The confidence region estimation algorithm

Input

Experimental data y

Number of copies of experimental data cy

Number of bootstrapped experimental data sets NB

The level of confidence for parameter estimates

Output

Center of the confidence region, c

G and b that characterize the confidence region

(c )G(c ) b

Bounding box half edge lengths l

19

1. Make cy copies of experimental data y to generate bootstrapped population of replicates Y

2. Generate NB bootstrapped experimental data sets from Y using random sampling with replacement

3. For each bootstrapped experimental data run UOBYQA-Fit of section 3.3.4 to obtain a bootstrapped

parameter estimate

4. Make a grid in the parameter space centered around the median of the bootstrapped parameter

estimates obtained from step 3

5. Evaluate objective function at each point of the grid of step 4

6. Fit a quadratic

(c )G(c )= d

through the grid points and the value of the objective functions obtained in step 5 at the grid points. c

is the median of the bootstrapped parameter estimates of step 3

7. For each bootstrapped parameter estimate obtained from step 3, evaluate f ()= (c )G(c )

8. Sort f () values obtained from step 7 in ascending order. Assign b to 100 percentile point of this

sorted f () values

9. The vector of bounding box half lengths is given by

l =

(bdiag(G1))

10. Return c ,G ,b, l

Next we check the accuracy of the obtained confidence region. We generate several sets of

experimental data, and obtain parameter estimates for each experimental data set. We obtain the

confidence region, using just the first experimental data set. We verify the accuracy of this confidence

region by checking how many of the parameter estimates fall inside this confidence region. Ideally, the

fraction of points inside the confidence region should be equal to the confidence level . We are now

ready to present, the verification procedure for the confidence region algorithm.

20

3.4.2 The verification of confidence region algorithm

Input

Ne sets of experimental data. Each experimental data set consists of m replicates

C , a vector of confidence levels

Output

Vector I with elements as Ii such that for each confidence level i C , Ii is the number of points insidei confidence hyper-ellipsoid

1. For each experimental data, obtain parameter estimate using UOBYQA-Fit algorithm

2. For i = 1,2, . . . to number of confidence levels

(a) Call the confidence region estimation algorithm of section 3.4.1 with confidence level i and first

experimental data set

(b) For each parameter estimate obtained in step 1 check

(i c )G(i c ) bi (3.14)

(c) Assign Ii to the number of parameter estimates from step 3.4.2that satisfy (3.14)

3. Return vector I

3.5 Application: RNA dynamics in Escherichia coli

Golding et al. [41] developed a florescence microscopy method to quantify molecular level of mRNAs in

individual E. coli cells. The method is based on amplification of a fluorescence protein having the

capability to bind to a reporter RNA. To obtain the number of mRNA molecules, the fluorescence flux

produced in the cells is compared with the fluorescence produced by a single mRNA molecule. The

mRNA signal was shown to rise until 80 minutes and then plateau. They fit the experimental data with

a mass action kinetic model given by reactions (3.15) (3.17).

DNASk1 DNAA (3.15)

DNAAk2 DNAS (3.16)

DNAAk3 DNAA + RNA (3.17)

21

Since our aim here is to test the algorithm developed, we test it with simulated experimental data

assuming model (3.15)(3.17) as the true system. The amount and quality of experimental data is

important to obtain good parameter estimates and tight confidence regions. In Figure 3.1 we show 10

replicates of the experiment. The tremendous replicate to replicate variability is an indicator of lack of

enough experimental data. We use m = 100 replicates as our experimental data set. For each replicatewe use sampling time of 0.5 minutes, which is the same as the successive image time of the

experimental protocol.

0

2

4

6

8

10

0 10 20 30 40 50 60 70 80

mR

NA

t

Figure 3.1: 10 replicates of mRNA vs time experimental data

We apply the UOBYQA-Fit algorithm described in section 3.3.4 to obtain the point estimates of the

parameters. The parameters used in the implementation of the algorithm are listed in Table 3.1.

Unlike Poovathingal and Gunawan [81], we provide a confidence region using the method described in

section 3.4. In 2-D the confidence region for the parameters is an ellipse and the bounding box is the

rectangle aligned with the coordinate axes and tangent to the ellipse. Figure 3.2 shows an ellipse with

its rectangular bounding box.

22

Parameter m N 0 0 end

Value 100 100

1.560.780.40

2.000000.397940.45593

1 0.004

Table 3.1: System and optimization parameters

Figure 3.2: An ellipse and its bounding box

In 3-D the confidence region for the parameters is an ellipsoid and the bounding box is the cuboid

aligned with the coordinate axes and tangent to the ellipsoid. The parameter estimates obtained and

the corresponding confidence region, both the bounding box and the extreme points of the ellipsoid,

are listed in Table 3.2.

The ratio of volumes of the 95% confidence region ellipsoid and the bounding box corresponding to

this ellipsoid is 0.28. This ratio indicates the significant stretch in the confidence region when we put

the bounding box around the 95% confidence region ellipsoid.

23

Parameter estimates 95%Bounding box 6 Extreme points of the ellipsoid1.401.160.60

1.340.971.060.560.521.02

1.06 1.63 0.58 2.11 1.80 0.880.67 1.46 1.01 1.12 0.46 1.670.24 0.81 1.36 0.32 0.90 0.15

Table 3.2: Parameter estimates and confidence regions. The true parameter values are

[1.560.780.40]

Next we verify whether the large confidence region depicted in Table 3.2 is due to the lack of

information in the experimental data or it is an artifact of our confidence region estimation algorithm.

We generate 500 sets of experimental data set where each experimental data set consists of 100

replicates. We use the verification of the confidence region algorithm of section 3.4.2 to obtain the

number of parameter estimates inside different confidence level ellipsoids.

0

50

100

150

200

250

300

350

400

450

500

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Nu

mb

ero

fpo

ints

in

con

fid

ence

ellip

soid

ActualExpected

Figure 3.3: Verification of confidence region using bootstrapping

24

Figure 3.3 shows, the number of these points that are inside level confidence region as a function of

. For small values of the number of points inside level confidence ellipsoid are close to the

expected number of points inside the level confidence ellipse. This figure illustrates the quality of the

confidence region generated by the confidence region estimation algorithm. Therefore the confidence

region algorithm generates reliable confidence regions, and the large confidence region depicted in the

table 3.2 is due to the lack of information in the experimental data.

3.6 Conclusions

In this chapter we presented a new method of parameter estimation in stochastic chemical kinetic

models. The method is based upon a negative log likelihood minimization approach in which the

likelihood expression had several nice properties. The likelihood expression was in the form of the

expectation of a function of data, parameters, and a smoothing parameter. The estimation of this

likelihood is possible even with just 1 SSA simulation. We describe a procedure to obtain the number of

SSA simulations, N , which can give us reliable parameter estimates. We give a heuristic expression for

the connection between the smoothing parameter, R, and the number of SSA simulations, N .

Equipped with the likelihood expression, N , and R, we used the sample path method to estimate the

negative log likelihood. To minimize this negative log likelihood, we developed a derivative free

optimization method UOBYQA-Fit, a variant of Powells UOBYQA algorithm. To estimate the

confidence region, we developed a variant of Efrons percentile method.

We tested the obtained parameter estimates and the confidence regions by generating several

experimental data sets. The tests indicate that both the optimization and the confidence estimation

algorithms are producing reliable parameter estimates and confidence regions, respectively.

25

Chapter 4

New methods to obtain sensitivities of stochastic chemicalkinetic models

4.1 Introduction

The stochastic chemical kinetic models that describe the biological systems of interest here depend on

parameters whose values are often unknown and can change due to changes in the environment.

Sensitivities quantify the dependence of the systems output to changes in the model parameters.

Sensitivity analysis is useful in determining parameters to which the system output is most responsive,

in assessing robustness of the system to extreme circumstances or unusual environmental conditions,

and in identifying rate limiting pathways as a candidate for drug delivery. However, one of the most

important applications of sensitivities is in parameter estimation. Sensitivities provide a way to

approximate the Hessian of the objective function through the Gauss-Newton approximation[88, p.

535].

Unbiased methods of sensitivity estimation include the likelihood ratio gradient method [39, 75] and

the infinitesimal perturbation method based on the Girsanov transform [80, 106]. The canonical

convergence rate, which is a measure of how fast the estimator error converges to a standard normal

distribution, of both the likelihood ratio gradient and the infinitesimal perturbation analysis estimators

are O(N1/2) [38], in which N is number of estimator simulations. The unbiasedness of the likelihood

ratio gradient method comes at the cost of high variance of the estimator if there are several reaction

events in the estimation of the output of interest. Girsanov transform based methods, on the other

hand, have high variance of the estimator when there are a small number of reaction events in the

estimation of the output of interest. Komorowski et al. [64] use a linear noise approximation of

stochastic chemical kinetic models for sensitivity analysis. However, use of the linear noise

26

approximation limits their analysis to only stochastic differential equation models. Gunawan et al. [45]

compare the sensitivity of the mean with the sensitivity of the entire distribution. They explain why the

sensitivity of the mean can be inadequate in determining the sensitivity of stochastic chemical kinetic

models.

Despite being easier to implement and intuitive to understand, finite difference based methods

produce biased sensitivity estimates. However, implemented with consideration of the trade-off

between the statistical error of the estimator and its bias, finite difference based methods can have a

canonical convergence rate close to the best possible convergence rate of O(N1/2) [38]. In fact,

LEcuyer and Perron [67] show that for many practical cases of interest, infinitesimal perturbation

analysis and finite difference with common random numbers have the same canonical convergence

rate.

Several different estimators using finite difference have been proposed [2, 38, 87]. Anderson [2]

proposes a new estimator, coupled finite difference (CFD), using a single Markov chain for the nominal

and perturbed processes. The CFD estimator incorporates a tight coupling between the nominal and

perturbed processes, thereby producing a significant reduction in estimator the variance [2].

In this chapter, we show the superiority of CFD over CRN in the estimation of sensitivities. We do not

discuss the independent random number [87] estimator , also known as Crude Monte Carlo [2]

estimator, because either estimator, CRN or CFD, usually has several order of magnitudes smaller

variance than this estimator. We calculate sensitivity estimates of four different quantities of interest.

In example one, the quantity of interest is the expected value of a species. Example two looks at the

likelihood of experimental data. Example three looks at the probability of a rare state. Example four

looks at the expected value of a fast fluctuating species.

This chapter is arranged as follows. Section 4.2 defines the estimators that are used in the subsequent

examples. Section 4.3 shows the results we obtain from the four examples. Finally section 4.4 discusses

the conclusions of this chapter and summarizes the contributions.

4.2 The estimators

Common random number (CRN) [38, 87]: A single simulation of the CRN estimator gives two coupled

SSA simulations: the first coupled SSA simulation uses the rate parameter k and randomness the

second one uses the perturbed rate parameter k+ and the same randomness .

27

Coupled Finite difference (CFD) [2]: A single simulation of the CFD estimator simulates a Markov

chain with an enlarged state space. The state space of this Markov chain contains the states

corresponding to both rate parameter k and k+. The Markov chain is constructed in such a way thatthere is a tight coupling between the states corresponding to both rate parameter k and k+. SeeAnderson [2] for the complete description.

As shown in the examples in the next section, the finite difference approximation of the sensitivity from

a single simulation of either of the two estimators can be obtained. Sample averaging N such finite

difference approximations gives us an estimator of the sensitivity of interest.

4.3 Examples

4.3.1 Sensitivity of expected value of population of a species in a reactionnetwork

Consider the following simple reaction network consisting of two reactions

Ak1 B (4.1)

Bk2 C. (4.2)

Figure 4.1 shows a typical SSA simulation of the network involving reactions (4.1) and (4.2).

0

20

40

60

80

100

0 0.5 1 1.5 2

Spec

ies

Po

pu

lati

on

t

A

B C

Figure 4.1: A typical simulation of the network involving reaction (4.1) and (4.2).

28

We wish to estimate the sensitivity of the expected value of B with respect to the rate constant k1,

s(t ;k1)= dEB(t ;k1)dk1

. (4.3)

The forward finite difference approximation to equation (4.3) is

s(t ;k1) EB(t ;k1+)EB(t ;k1)

. (4.4)

The forward finite difference has an error of O(). That is

s(t ;k1)= EB(t ;k1+)EB(t ;k1)

+O().

Throughout the chapter we use the forward finite difference to approximate the sensitivity. We denote

an estimator of (4.4) using either CRN or CFD as sest in which est {CRN,CFD}. Let B esti (t ;k1) andB esti (t ;k1+) denote the population of B obtained through the i th simulation of estimator est. Then theestimator sest for s(t ;k1) of (4.3) is defined as:

sest = 1N

Ni=1

B esti (t ;k1+)B esti (t ;k1)

(4.5)

The sample standard deviations ([sest]) of the estimator of equation (4.5) is given by

[sest]=(

1

N (N 1)2N

i=1

[{B esti (t ;k1+)B esti (t ;k1)}B est

]2)1/2(4.6)

in which

B est = 1N

Ni=1

B esti (t ;k1+)B esti (t ;k1)

Because the model is linear, the exact expected value of B and the exact sensitivity of the expected

value of B can be calculated and are given by

EB(t ;k1)= nB0 ek2t +nA0k1

k2k1(ek1t ek2t ) (4.7)

sex(t ;k1)= dEBdk1

= nA0(k1k2)2

[k2e

k1t +k1(k1k2)tek1t k2ek2t]

(4.8)

Figure 4.2 compares the performance of the CRN and CFD estimators, and Table 4.1 lists the

parameters used to generate Figure 4.2. Figure 4.2(a) shows a comparison of the sensitivity estimates

obtained from the CRN and CFD estimators. We define root mean squared error of the estimator as

eest =[

1

nd

ndi=1

(sest(ti ,k1) sex(ti ,k1))2]1/2

(4.9)

29

Parameter nA0 nB0 nC0 k1 k2 N

Value 100 0 0 2. 1. 0.1 100

Table 4.1: Parameter values for example 4.3.1.

in which nd = 41 is the total number of time points at which we calculate the sensitivity sest, 0 ti 2.0,and ti+1 ti = 0.05. Root mean squared errors calculated from the data of Figure 4.2(a) give ecrnecfd = 4. Avalue greater than one for this ratio demonstrates that on average across all the time points considered,

the CFD estimator tracks the exact sensitivity better than the CRN estimator. Figure 4.2(b) quantifies

the efficiency of the two estimators by comparing their standard deviations. We can see that starting

from t = 0.3, the CFD estimator has half the standard deviation of the CRN estimator. Lower standarddeviation of the CFD estimator compared to the CRN estimator verifies its higher efficiency.

-10

-5

0

5

10

15

20

0 0.5 1 1.5 2

Est

imat

edse

nsi

tivi

ty

t

(a) scrnscfdsex

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

0 0.5 1 1.5 2

Est

imat

or

std

.dev

.

t

(b) [scrn][scfd]

Figure 4.2: Comparison of CRN and CFD estimators: (a) Estimated and analytical sensitivities.

(b) Sample standard deviation of the two estimators.

4.3.2 Sensitivity of the negative log likelihood function

Consider reactions 4.1 and 4.2 again. Experimental data y from a single experiment, shown in

Figure 4.3 and reported in Appendix B, are given as a time series of B,

30

0

10

20

30

40

50

60

0 0.5 1 1.5 2

B

tFigure 4.3: Experimental data for example 4.3.2.

i.e., the experimental data are

y= (Bt1 ,Bt2 , . . . ,Btn )T

We assume that the rate constant k2 is known. As shown in Chapter 3.2 and Appendix A, an estimate of

the likelihood of experimental data y under certain reasonable assumptions is given by

L(k1, N )= 1N (2pi)nd /2|R|1/2

Ni=1

e(1/2)(yxi (k1))R1(yxi (k1)) (4.10)

in which R is a known positive definite matrix, nd is number of elements in the experimental data

vector y, and xi (k1)= (Bt1 ,Bt2 , . . . ,Btn )Ti is the time series of the population of B , obtained by the i th

SSA simulation using rate constant value k1 for reaction 4.1. Note that as the number of samples N

goes to infinity, the likelihood estimate L(k1, N ) from (4.10) approaches the true likelihood of the

experimental data. The estimate of the negative log likelihood is defined as

(k1, N )= logL(k1, N )= log[

1

N (2pi)nd /2|R|1/2N

i=1e(1/2)(yxi (k1))

R1(yxi (k1))]

(4.11)

To find the parameters that describe the experimental data, we need to minimize the estimate of the

negative log likelihood function given in equation (4.11). Sensitivities can be used to obtain gradients

31

Parameter nA0 nB0 nC0 k1 k2 N nd R

Value 100 0 0 1. 1. 0.1 4000 21 Indnd

Table 4.2: Parameter value for example 4.3.2.

required in any gradient based optimization algorithm. Here we are interested in the sensitivity

s(k1, N )= d(k1, N )dk1

(4.12)

of the estimated negative log likelihood function and the convergence of this sensitivity with the

number of samples, N . The forward finite difference approximation of (4.12) is given by

s(k1, N ) (k1+, N )(k1, N )

(4.13)

The forward finite difference has an error of O(), i.e.,

s(k1, N )= (k1+, N )(k1, N )

+O()

We write estimator est {CRN,CFD} of sensitivity s(k1, N ) of equation (4.12) as

sest(k1, N )= est(k1+, N )est(k1, N )

(4.14)

in which est(k1, N ) is the estimate of negative log likelihood obtained from equation (4.11) using

estimator est {CRN,CFD} .Figure 4.4 shows the steps in obtaining the sensitivity of the negative log likelihood function using the

CRN estimator. Table 4.2 contains the parameters used in this example. Figure 4.4(a) shows the

variation of the quadratic form (yxi (k1))R1(yxi (k1)) as a function of individual SSA simulationnumber i . Figure 4.4(a) displays the wide variation in the value of the quadratic form for different

individual SSA simulations. Figure 4.4(b) is a plot of e(1/2)(yxi (k1))R1(yxi (k1)) as a function of the

individual SSA simulation number i . The wide variation in (yxi (k1))R1(yxi (k1)) of Figure 4.4(a)leads to even wider variation in the exponential, as depicted in Figure 4.4(b). Figure 4.4(c) depicts

convergence of L(k1, N ) with N . L(k1, N ) in Figure 4.4(c) is given by equation (4.10). Large variation in

e(1/2)(yxi (k1))R1(yxi (k1)) as depicted in Figure 4.4(b) explains the sharp jumps in L(k1, N ) which occur

whenever the last SSA simulation i = n dominates all the previous 1 i n1 simulations. Figure

32

4.4(d) shows the convergence of the nominal and perturbed negative log likelihoods: crn(k1, N ) and

crn(k1+, N ). The sharp jumps in crn(k1, N ) occur at the same n values as the jumps in L(k1, N ) inFigure 4.4(c). Finally Figure 4.4(e) shows the convergence of the sensitivity of the negative log

likelihood using the CRN estimator, scrn(k1, N ), as a function of N .

Next we change focus from the CRN estimator to the CFD estimator. In Figure 4.5(a)-(e) we plot the

analogous results to Figure 4.4(a)-(e). Finally Figure 4.6 compares convergence of CRN estimator,

scrn(k1, N ), and CFD estimator, scfd(k1, N ). At the end of the maximum number of simulations (4000)

for CRN and CFD estimators, the ratio of the two offsets, the offset of CRN estimator from the estimated

true value and the offset of CFD estimator from the estimated true values, is 5.5. An estimated true

value is obtained by performing 50,000 simulations of the CFD estimator. The lower offset makes the

CFD estimator a natural choice for the estimator of sensitivity of negative log likelihood.

33

0

200

400

600

800

1000

1200

0 500 1000 1500 2000 2500 3000 3500 4000

i

(yxi (k1)

)R1 (yxi (k1))(a)

10250

10200

10150

10100

1050

100

0 500 1000 1500 2000 2500 3000 3500 4000

i

exp((1/2)(yxi (k1))R1(yxi (k1)))

(b)

10160

10140

10120

10100

1080

1060

1040

0 500 1000 1500 2000 2500 3000 3500 4000

L(k

1,N

)

N

(c)

50

100

150

200

250

300

350

0 500 1000 1500 2000 2500 3000 3500 4000

N

(d) crn(k1+, N )crn(k1, N )

-700

-600

-500

-400

-300

-200

-100

0

100

0 500 1000 1500 2000 2500 3000 3500 4000

N

(e) scrn(k1, N )

Figure 4.4: Convergence of sensitivity estimate from CRN estimator: (a) Plot of quadratic form

as a function of simulation number. (b) Plot of exponential of quadratic form as a function of

simulation number. (c) Likelihood as a function of total number of simulations. (d) Negative

likelihoods as a function of total number of simulations (e) Estimated sensitivity from CRN es-

timator as a function of total number of simulations

34

100

200

300

400

500

600

700

800

900

1000

1100

0 500 1000 1500 2000 2500 3000 3500 4000

i

(yxi (k1)

)R1 (yxi (k1))(a)

10250

10200

10150

10100

1050

100

0 500 1000 1500 2000 2500 3000 3500 4000

i

exp((1/2)(yxi (k1))R1(yxi (k1)))

(b)

10110

10100

1090

1080

1070

1060

1050

1040

0 500 1000 1500 2000 2500 3000 3500 4000

L(k

1,N

)

N

(c)

60

80

100

120

140

160

180

200

220

240

0 500 1000 1500 2000 2500 3000 3500 4000

N

(d) cfd(k1+, N )cfd(k1, N )

-200

-180

-160

-140

-120

-100

-80

-60

0 500 1000 1500 2000 2500 3000 3500 4000

N

(e) scfd(k1, N )

Figure 4.5: Convergence of sensitivity estimate from CFD estimator: (a) Plot of quadratic form

as a functions of simulation number. (b) Plot of exponential of quadratic form as a function

of simulation number. (c) Likelihood as a function of total number of simulations. (d) Nega-

tive likelihoods as a function of total number of simulations (e) Estimated sensitivity from CFD

estimator as a function of total number of simulations

35

-700

-600

-500

-400

-300

-200

-100

0

100

0 500 1000 1500 2000 2500 3000 3500 4000

N

scrn(k1, N )scfd(k1, N )

Exact

Figure 4.6: Convergence of estimated sensitivities from CRN and CFD estimators. Compared to

CRN estimator CFD estimator shows quicker convergence to the estimated true value.

4.3.3 Sensitivity of a rare state probability

Consider the pap operon regulation [74, 100] which has a biologically important rare state. In Srivastava

et al. [100], we obtained a significantly better estimate of the rare state probability using stochastic

quasi-steady-steady perturbation analysis (sQSPA), compared to both the full model and importance

sampling based methods [65, 90]. Figure 4.7 shows the schematic of the pap operon regulation.

36

Regulatory Protein

g1

g2 g3

g4

r5

r3r1

r7r6

r4

r8

r2

Figure 4.7: Schematic diagram of the Pap regulatory network. There are four possible states of

the pap operon depending on the LRP-DNA binding.

State g1 is the rare state. The master equation for the system is

dP1d t

=(r1+ r3)P1+ r2P2+ r4P3 (4.15)dP2d t

=(r2+ r5)P2+ r1P1+ r6P4 (4.16)dP3d t

=(r4+ r7)P3+ r3P1+ r8P4 (4.17)dP4d t

=(r6+ r8)P4+ r5P2+ r7P3 (4.18)

in which Pi (t ;r2): i = 1,2,3,4 is the probability of state gi and r j : j = 1,2, . . . ,8 are the rates of transitiondefined in Table 4.3. Define

Si (t ;r2)= Pir2

i = 1,2,3,4 (4.19)

37

The governing equations for sensitivities Si : i=1,2,3,4 are obtained by differentiating (4.15)(4.18) with

respect to r2.

dS1d t

=(r1+ r3)S1+P2+ r2S2+ r4S3 (4.20)dS2d t

=P2 (r2+ r5)S2+ r1S1+ r6S4 (4.21)dS3d t

=(r4+ r7)S3+ r3S1+ r8S4 (4.22)dS4d t

=(r6+ r8)S4+ r5S2+ r7S3 (4.23)

In Srivastava et al. [100], we showed that the sQSPA model reduction leads to the reaction network

Number Reaction stoichiometry Reaction rate (ri )

1 g1 g2 100.2 g2 g1 0.6253 g1 g3 100.4 g3 g1 1.0335 g2 g4 0.996 g4 g2 1.0337 g3 g4 0.998 g4 g3 0.625

Table 4.3: Reaction stoichiometry and reaction rates for example 4.3.3.

shown in Figure 4.8.

38

r1

r2

r6

r5 r8r7

g4

g3g2

Figure 4.8: Reduced system in the slow time scale regime

In the sQSPA model reduction we write probabilities Pi : i = 1,2,3,4 in a power series expression givenby

Pi =Wi 0+sqWi 1+2sqWi 2+O(3sq)

The master equation for the sQSPA reduced model is

W10 = 0 (4.24)dW20

d t=[r1+ r5]W20+ r2W30+ r6W40 (4.25)

dW30d t

=[r2+ r7]W30+ r1W20+ r8W40 (4.26)dW40

d t=[r6+ r8]W40+ r5W20+ r7W30 (4.27)

in which Wi j are the j th-order probabilities of state gi : i = 1,2,3,4, r1 = r2r3/(r1+ r3), andr2 = r1r4/(r1+ r3). One crucial equation that comes out of the sQSPA model reduction is theapproximation of probability (Psq1 ) of the rare state in terms of the probabilities of the other states

Psq1 = sq(r2W20+ r4W30) (4.28)

in which sq = 1/(r1+ r3). We are interested in the sensitivity of probability of the rare state g1, withrespect to r2.

s(t ;r2)= S1 = P1(t ;r2)r2

(4.29)

39

To estimate s(t ;r2) of equation (4.29), we use three different estimators: CRN, CFD, and sQSPA with

common random numbers (SRN). The CRN estimator is given as

scrn(t ;r2)= 1N

Ni=1

[1(pacrni (t ,r2+)= g1)1(pacrni (t ,r2)= g1)

](4.30)

in which pacrni (t ;r2) is the state of the pap operon at time t with rate parameter r2 obtained through

the i th CRN simulation, and N is the number of CRN simulations. A point to note is that the i th CRN

simulation gives us one SSA simulation with rate parameter r2, i.e., pacrni (t ;r2), and one SSA simulation

with rate parameter r2+, i.e., pacrni (t ;r2+). The indicator random variable 1(A) evaluates to 1whenever event A happens and 0 otherwise. In an analogous fashion we have the CFD estimator for

equation (4.29) as

scfd(t ;r2)=1

N

Ni=1

[1(pacfdi (t ,r2+)= g1)1(pacfdi (t ,r2)= g1)

](4.31)

The SRN estimator estimates the sensitivity of Psq1 of equation (4.28) with respect to r2. The SRN

estimator is given by

ssrn =sq

[(r2+)W20(t ;r2+)+ r4W30(t ;r2+) r2W20(t ;r2) r4W30(t ;r2)

](4.32)

in which W20(t ;r2) and W20(t ;r2+) are obtained by simulating the reduced system shown in Figure 4.8and governed by master equation (4.24)(4.27) using common random numbers and SSA simulations.

Figure 4.9 shows comparison of the CRN, CFD and SRN estimators. The ratio of root mean squared

errors of CRN and CFD estimators is ecrnecfd = 4.75. As the number of reaction events is small in the papoperon, we also apply the likelihood gradient method[75]. We perform 500 simulations for all the four

estimators. The likelihood gradient performs better than both CRN and CFD but it performs worse

than SRN. The SRN estimator tracks the true sensitivity closely except for a small initial time. This

example reveals the distinct advantage of analytical insight and model reduction, e.g. the sQSPA

analysis, over the several proposed estimators that do not use the reduced model.

40

-0.04

-0.02

0

0.02

0.04

0.06

0 0.1 0.2 0.3 0.4 0.5

Est

imat

edse

nsi

tivi

ty

t

(a) scrnscfdssrn

slrs

0

0.005

0.01

0.015

0.02

0 0.1 0.2 0.3 0.4 0.5

Est

imat

edse

nsi

tivi

ty

t

(b)

scfdssrn

slrs

Figure 4.9: Estimated sensitivity from the CRN, CFD and SRN estimators. The SRN estimator

tracks true sensitivity closely except for a small initial time. The CFD estimator performs bet-

ter than the CRN estimator. Likelihood gradient performs better than both the CRN and the

CFD but it performs worse than SRN. (a) Showing the CRN estimator (b) Not showing the CRN

estimator

4.3.4 Sensitivity of a fast fluctuating species

In a stochastic simulation of the infection cycle of vesicular stomatitis virus (VSV), there is a fast

fluctuation in a protein at low copy number along with a rapid increase in the population of the viral

genome. Such a system is expensive to simulate because the frequency of the fluctuation increases as

the simulation progresses leading to small time steps in the SSA simulation [54]. To illustrate the

phenomenon, consider the following simple 3-species, 3-reaction system

A + G k1 C + G (4.33)C + G k2 2G + A (4.34)

2Gk3 G (4.35)

with k2 k1 k3

This reaction system describes the interaction of three species in a simplified VSV replication process -

two forms of viral polymerase, A and C, and viral genome G. The two forms of the polymerase arise

because VSV has two different complexes that serve as viral transcriptase and replicase [91]. The viral

41

Parameter nA0 nC0 nG0 k1 k2 k3 N

Value 3 0 1 2105 3105 1 0.1 100

Table 4.4: Parameter values for example 4.3.4.

transcriptase form A is a complex of constituent VSV proteins L and P. The replicase form C is a

complex of L, N and P proteins. The species A is involved in the transcription reaction (4.33) to produce

messenger RNA. The transcription reaction leads to the conversion of transcriptase A into replicase C.

We further assume that produced mRNA from reaction (4.33) is short lived and hence we do not

include it in the model. Species C and G are involved in replication reaction (4.34) to produce an

additional viral genome G. The replication (4.34) reaction leads to the conversion of replicase C into

transcriptase A. Finally there is a second-order degradation reaction (4.35) of viral genome. The model

(4.33)(4.35) is insufficient to predict the full viral infection cycle, but it is instructive in understanding

the simulation challenges of the full infection cycle model used by Hensel et al. [54]. The reaction rate

constants k1,k2,k3 denote macroscopic reaction rate constants with units m3/(mols). We expressmicroscopic reaction rates in terms of macroscopic rate constants (k1,k2,k3) and the system size

r1 = 1

k1ag r2 = 1

k2cg r3 = 1

k3g (g 1)

in which the system size appears because the reactions are second order. For the purposes of this

example we take= 105m3. A stochastic simulation with the parameter values given in Table 4.4 isshown in the Figure 4.10. Species G increases continuously and this increase forces species C to

fluctuate with increasing frequency as shown in the Figure 4.10.

42

0

1

2

3

0 1 2 3 4 5 6 7

C

t (sec)

(a)

100

101

102

103

104

105

106

0 1 2 3 4 5 6 7

G

t (sec)

(b)

Figure 4.10: A typical SSA simulation of the network of (4.33)(4.35). (a) Species C vs. time (b)

Species G vs. time

We want to investigate the sensitivity:

s(t ;k3)= dEXdk3

(4.36)

In which X {C ,G}. An estimator of the sensitivities of interest (4.36) is:

sest = 1N

Ni=1

X esti (t ;k3+)X esti (t ;k3)

(4.37)

in which est {CRN,CFD}. Figure 4.11 shows comparison of standard deviations of the two estimators,CRN and CFD, obtained using equation (4.6). The parameters used to generate Figure 4.11 are shown

in Table 4.4. Figure 4.11(a) shows that for the abundant species G, CFD and CRN estimators have

similar standard deviations, which demonstrates that both CRN and CFD are capable of obtaining good

sensitivity estimates for the abundant species G. Figure 4.11(b) shows that for fast fluctuating species C,

the CFD estimator has less than one third the standard deviation of the CRN estimator, which

demonstrates that the CFD estimator captures the sensitivity of the fast fluctuating species C better

than the CRN estimator. This example again demonstrates the superiority of the CFD estimator over

the CRN estimator.

43

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

0 1 2 3 4 5 6 7

G

t

(a) G [scr n]G [sc f d ]

0

0.5

1

1.5

2

2.5

3

0 1 2 3 4 5 6 7

C

t

(b)

C[scr n]

C[sc f d ]

Figure 4.11: Comparison of standard deviations of CRN and CFD estimators: (a) Species G (b)

Species C

4.4 Conclusions

In this chapter we compared the performance of several finite difference sensitivity estimators. In all

the examples and sensitivities of interest, we found that the newly developed CFD estimator performs

significantly better than the CRN estimator. It has already been shown that the CFD estimator performs

better than the CRP estimator[2]. The comparisons made in this chapter, along with Andersons

previous results, lead to the conclusion that CFD is currently the best available estimator in the class of

finite difference estimators of stochastic chemical kinetic models.

44

Chapter 5

Model reduction using the Stochastic Quasi-Steady-StateAssumption

Note: Most of this chapter appears in Srivastava et al. [100]

5.1 Introduction

Stochastic chemical kinetic models have an ability to explain intrinsic noise and its effect on cellular

behavior. For example, Arkin and McAdams applied stochastic chemical kinetic model in the

bacteriophage infection and showed that intrinsic noise could bifurcate identically infected cells to

either dormant (lysogenic) or reproductive (lytic) states [5]. Using a stochastic chemical kinetic model,

Vilar et al. demonstrated that biochemical oscillators could still reliably function even in the presence

of such noise [105]. Weinberger et al. showed (both computationally and experimentally) that noise

could generate transient bursts of activity in HIV-1 Tat transactivation, which contributes to latency in

HIV-1 [107]. Hensel et al. performed computational studies to illustrate the dynamics of an

intracellular infection process, demonstrating coupling between a highly reactive species and a rapidly

increasing species [54].

While these works are clearly successes of the modeling community in understanding the intricacies of

intrinsic noise, the reality is that stochastic models of biologically relevant systems remain difficult to

construct and even more difficult to understand. Many of these difficulties stem primarily from the

significant cost of solving these models, particularly when using SSA simulation; see Gillespie [36] for a

recent review of progress made on this front. Moreover, for many systems biology models, the available

experimental measurements are not sufficient to confidently estimate the parameters of the

model [48]. While these models can still make well-constrained predictions, the unnecessarily large

number of parameters in these models increases the cost of calculating parameter sensitivities, i.e.,

45

how the model predictions change with perturbations to the parameters. Often these large number of

parameters lead to model stiffness, a phenomenon in which significant computational expense is

incurred in simulating some subset of the reactions in the model.

Stiffness can arise when some set of reversible reactions occurs much more frequently than the

remaining reactions, and the affected species typically remain at reasonable (nonzero) numbers. Such

stiffness is analogous to the deterministic concept of reaction equilibrium, and has been the subject of

several recent studies [10, 20, 21, 43, 51, 92, 95]. A separate but equally important source of stiffness

results from highly reactive species. This phenomenon results when reaction intermediates, known as

quasi-steady-state (QSS) species, react so rapidly that their average number throughout the simulation

is nearly zero or their average number is much smaller than the other species (the reactants and

products) [71, 84]. Rao and Arkin hypothesized that the stochastic quasi-steady-state reduction should

lead to the same reduced model as the deterministic quasi-steady-state reduction. Mastny et al. [71],

however, presented counterexamples in which the reduced models from deterministic and stochastic

quasi-steady-state reduction are different. Using singular perturbation analysis, Mastny et al. [71]

demonstrated that the QSS species can be removed from the master equation to yield a reduced master

equation for the remaining species. They termed this method stochastic QSS singular perturbation

analysis (sQSPA). Inspection of the reduced master equation yields reduced reaction expressions,

stoichiometries, and fewer parameters in comparison to the original master equation. Notably, the

reduced reaction expressions do not always correspond to those obtained by using the traditional

deterministic quasi-steady-state analysis. Another source of stiffness can occur when there are two

distinct type of species present in the system: a highly reactive species and a rapidly increasing species.

This stiffness causes small time steps in SSA and leads to significant slow down of SSA simulations. For

this stiffness, we use a variant of sQSPA, stochastic QSS singular perturbation analysis with

expansion (sQSPA-).

In this chapter, we demonstrate the utility of sQSPA and sQSPA- for simulating and understanding

stochastic reaction models. We choose two previously published models from the literature that appear

to have highly-reactive, or QSSA species: the pap operon regulation [74] and a biochemical

oscillator [105]. We also consider a simple system, a fast fluctuation, which contains a highly reactive

species coupled to a rapidly increasing species. This coupling of highly reactive species with rapidly

increasing species causes a large computational load for SSA simulations [54]. By reducing these three

46

Number Reaction stoichiometry Reaction rate (ri )

1 g1 g2 100.2 g2 g1 0.6253 g1 g3 100.4 g3 g1 1.0335 g2 g4 0.996 g4 g2 1.0337 g3 g4 0.998 g4 g3 0.625

Table 5.1: Reaction stoichiometry and reaction rates for pap operon regulation.

models using sQSPA and sQSPA-, we show that we can lower the model complexity without altering

the inherent noise characteristics of the full model. The reduced model for the pap operon regulation is

also useful in estimating a rare state probability from simulation. This rare state probability is not

accurately determined by either direct SSA simulation or Kuwahara and Muras [65] recent general

purpose method for estimating rare state probabilities in stochastic kinetic models. The reduced model

for the biochemical oscillator leads to simplification in the parameter estimation problem as well as

significant reduction in the simulation time. The reduced model for the fast fluctuation problem also

significantly reduces the simulation time.

5.2 Results

5.2.1 Pap operon regulation

Here we consider the gene state switch model of the Pyelonephritis-associated pili (Pap) regulatory

network considered by Munsky et al. [74]. This model describes the states (g1 to g4) of the pap operon

as a function of time. The schematic of four possible states and the mode of transition among them is

shown in Figure 5.1.

47

Regulatory Protein

g1

g2 g3

g4

r5

r3r1

r7r6

r4

r8

r2

Figure 5.1: Schematic diagram of the Pap regulatory network. There are four possible states of

the pap operon depending on the LRP-DNA binding.

The reaction stoichiometries and time invariant rates of transition (r1 to r8) are given in Table 5.1.

r1

r2

r6

r5 r8r7

g4

g3g2

Figure 5.2: Reduced system in the slow time scale regime

48

The master equation for the system is

dP1d t

=(r1+ r3)P1+ r2P2+ r4P3 (5.1)dP2d t

=(r2+ r5)P2+ r1P1+ r6P4 (5.2)dP3d t

=(r4+ r7)P3+ r3P1+ r8P4 (5.3)dP4d t

=(r6+ r8)P4+ r5P2+ r7P3 (5.4)

in which Pi : i = 1,2,3,4 is the probability of state gi and r j : j = 1,2, . . . ,8 is the rates of transitiondefined in Table 5.1. For the rates specified in Table 5.1, r1 and r3 are large compared to all other rates.

We apply the previously described sQSPA reduction technique [71]. In the Appendix C, we show that

equations (5.1)(5.4) can be reduced appropriately for the two time regimes: the fast time scale and the

slow time scale. The fast time scale regime occurs initially for a short period of time. During the fast

time scale regime, probabilities of all the states directly affected by r1 and r3 change rapidly. The slow

time scale follows this fast time scale regime. From Figure 5.1, we can see that the only state that is not

directly affected by r1 and r3 is state g4.

49

0

0.05

0.1

0.15