The Sensitivity Analysis of Mathematical Models Described...
Transcript of The Sensitivity Analysis of Mathematical Models Described...
The Sensitivity Analysis of Mathematical ModelsDescribed by Differential Equations
Hossein Zivaripiran
Department of Computer Science
University of Toronto
Sensitivity Analysis – p.1/15
Outline
� Modeling with Differential Equations (IVPs, DDEs)
� Modeling and Sensitivity Analysis
� Numerical Sensitivity Analysis of IVPs
� DDEs and Sensitivity Analysis
Sensitivity Analysis – p.2/15
Modeling with DE - Formulation
� An Initial Value Problem (IVP) for Ordinary Differential Equations (ODEs)
y′(t) = f(t, y(t))
y(t0) = y0
Sensitivity Analysis – p.3/15
Modeling with DE - Formulation
� An Initial Value Problem (IVP) for Ordinary Differential Equations (ODEs)
y′(t) = f(t, y(t))
y(t0) = y0
� Retarded Delay Differential Equations (RDDEs)
y′(t) = f(t, y(t), y(t− σ1), · · · , y(t− σν)) for t0 ≤ t ≤ tF
y(t) = φ(t), for t ≤ t0
σi = σi(t, y(t)) ≥ 0 delay (constant / time dependent / state dependent)
φ(t) history function (constant / time dependent)
Sensitivity Analysis – p.3/15
Modeling with DE - Formulation
� An Initial Value Problem (IVP) for Ordinary Differential Equations (ODEs)
y′(t) = f(t, y(t))
y(t0) = y0
� Retarded Delay Differential Equations (RDDEs)
y′(t) = f(t, y(t), y(t− σ1), · · · , y(t− σν)) for t0 ≤ t ≤ tF
y(t) = φ(t), for t ≤ t0
σi = σi(t, y(t)) ≥ 0 delay (constant / time dependent / state dependent)
φ(t) history function (constant / time dependent)
� Neutral Delay Differential Equations (NDDEs)
y′(t) = f(t, y(t), y(t− σ1), · · · , y(t− σν),
y′(t− σν+1), · · · , y′(t− σν+ω)) for t0 ≤ t ≤ tF
y(t) = φ(t), y′(t) = φ′(t), for t ≤ t0,
Sensitivity Analysis – p.3/15
Modeling with DE - Some Areas of Application
Area Example
Ecology predator-prey
Epidemiology spread of infections
Immunology immune response models
HIV infection
Physiology human respiration system
Neural Networks
Cell Kinetics
Chemical Kinetics The Oregonator
Physics Ring Cavity Lasers , two-body problem of electrodynamics
Sensitivity Analysis – p.4/15
Modeling with DE - Examples
� The Van der Pol Oscillator,
d2x
dt2− µ(1− x2)
dx
dt+ x = 0
using y1 = x and y2 = dxdt
,
y′
1(t) = y2(t)
y′
2(t) = µ(1− y21)y2 − y1
Sensitivity Analysis – p.5/15
Modeling with DE - Examples
� The Van der Pol Oscillator with µ = 1, y1(0) = 2, y2(0) = 0
0 5 10 15 20−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
t
y1
0 5 10 15 20−3
−2
−1
0
1
2
3
t
y2
Sensitivity Analysis – p.5/15
Modeling with DE - Examples
� An example of the famous Mackey-Glass DDEs proposed as a model for theproduction of white blood cells [Mackey and Glass 1977].
y′(t) =2y(t− 2)
1 + y(t− 2)9.65− y(t),
for t in [0, 100]. The history function is
φ(t) = 0.5 for t ≤ 0.
Sensitivity Analysis – p.5/15
Modeling with DE - Examples
� The Mackey-Glass model
0 20 40 60 80 1000.2
0.4
0.6
0.8
1
1.2
1.4
1.6
t
y
Sensitivity Analysis – p.5/15
Modeling with DE - Examples
� A neutral delay logistic Gause-type predator-prey system [Kuang 1991]
y′
1(t) = y1(t)(1− y1(t− τ)− ρy′
1(t− τ))−y2(t)y1(t)
2
y1(t)2 + 1
y′
2(t) = y2(t)
(
y1(t)2
y1(t)2 + 1− α
)
where α = 1/10 , ρ = 29/10 and τ = 21/50, for t in [0, 30]. The history functionsare
φ1(t) =33
100−
1
10t
φ2(t) =111
50+
1
10t
for t ≤ 0.
Sensitivity Analysis – p.5/15
Modeling with DE - Examples
� The predator-prey model
0 5 10 15 20 25 300.31
0.32
0.33
0.34
0.35
0.36
0.37
t
y1
0 5 10 15 20 25 302.21
2.215
2.22
2.225
2.23
2.235
2.24
2.245
2.25
2.255
t
y2
Sensitivity Analysis – p.5/15
Modeling with DE - Numerical Simulation
� Classical Theory of Step by Step Integration for IVPs.
� Runge-Kutta (RK).
� Linear Multistep (LM).
Sensitivity Analysis – p.6/15
Modeling with DE - Numerical Simulation
� Classical Theory of Step by Step Integration for IVPs.
� Runge-Kutta (RK).
� Linear Multistep (LM).
� Continuous Solution using Polynomial Approximation.
� Continuous Runge-Kutta (CRK).
� Linear Multistep methods have natural approximating polynomials.
Sensitivity Analysis – p.6/15
Modeling with DE - Numerical Simulation
� Classical Theory of Step by Step Integration for IVPs.
� Runge-Kutta (RK).
� Linear Multistep (LM).
� Continuous Solution using Polynomial Approximation.
� Continuous Runge-Kutta (CRK).
� Linear Multistep methods have natural approximating polynomials.
� DDEs : Combining an “interpolation” method (for evaluating delayed solutionvalues) with an ODE integration method (for solving the resulting “ODE”).
Sensitivity Analysis – p.6/15
Modeling with DE - Special Difficulties with DDEs
� Derivative DiscontinuitiesIn general
φ′(t0) 6= f(t0, φ(t0), φ(t0 − σ1), · · · , φ(t0 − σν))
Due to the existence of delays, discontinuities propagate along the integrationinterval.Solution is smoothed for RDDEs but in general not for NDDEs.The RK and LM methods fail in presence of discontinuities.Treatment : Tracking Discontinuities and forcing them to be mesh points.
� Vanishing Delays
limt→t⋆
σi(t, y(t)) = 0
Causes the solver to choose a sequence of very small steps near t⋆. (h ≤ σi)Treatment : Using Special Interpolants to pass t⋆
Sensitivity Analysis – p.7/15
Modeling and Sensitivity Analysis - Definitions
� Parameterized Models
� A parameterized IVP
y′(t;p) = f(t, y(t;p);p)
y(t0) = y0(p)
For example p = [µ] in the Van der Pol oscillator
y′
1(t) = y2(t)
y′
2(t) = µ(1− y21)y2 − y1
� A simple parameterized DDE
y′(t;p) = f(t, y(t;p), y(t− σ(t;p));p) for t0(p) ≤ t
y(t;p) = φ(t;p), for t ≤ t0(p)
Sensitivity Analysis – p.8/15
Modeling and Sensitivity Analysis - Definitions
� Forward Sensitivity Analysis
� The (first order) solution sensitivity with respect to the model parameter pi isdefined as the vector
si(t;p) = {∂
∂pi
}y(t;p), (i = 1, . . . ,L)
� The second order solution sensitivity with respect to the model parameterspi and pj is defined as the vector
rij(t;p) = {∂
∂pj
}si(t;p) = {∂2
∂pj∂pi
}y(t;p), (i, j = 1, . . . ,L)
Sensitivity Analysis – p.8/15
Modeling and Sensitivity Analysis - Definitions
� Adjoint Sensitivity Analysis
� We wish to evaluate the gradient ∂G∂pi
of
G(p) =
∫ tf
t0
g(t, y,p)dt
� or, alternatively, the gradient ∂g∂pi
of
g(t, y,p)
at time tf .
� More efficient if dim(g) < dim(y) and we need ∂∂p
for many parameters.
Sensitivity Analysis – p.8/15
Modeling and Sensitivity Analysis - Importance
Sensitivity information can be used to:
� Estimate which parameters are most influential in affecting the behavior of thesimulation. Such information is crucial for
� Experimental Design
� Data Assimilation
� Reduction of complex nonlinear models
� Study of Dynamical Systems : Periodic orbits, the Lyapunov exponents, chaosindicators, and bifurcation analysis are fundamental objects for the completestudy of a dynamical system, and they require computation of the sensitivitieswith respect to the initial conditions of the problem.
� Evaluate optimization gradients and Jacobians in the setting of
� Dynamic Optimization
� Parameter Estimation
Sensitivity Analysis – p.9/15
Numerical Sensitivity Analysis of IVPs - Forward
� Finite Difference Approach
{ ∂
∂pi
}y(t;p) ≈ y(t;p + ei∆pi) − y(t;p)
∆pi
Due to the rounding errors, the approximation is only O(√
Tol) with the best choice for∆pi.
Sensitivity Analysis – p.10/15
Numerical Sensitivity Analysis of IVPs - Forward
� Finite Difference Approach
{ ∂
∂pi
}y(t;p) ≈ y(t;p + ei∆pi) − y(t;p)
∆pi
Due to the rounding errors, the approximation is only O(√
Tol) with the best choice for∆pi.
� Internal Differentiation
y′(t;p) = f(t, y(t;p);p), y(t0) = y0(p)
⇓
Differentiation + Chain Rule + Clairaut’s Theorem
⇓
s′
i =∂f
∂ysi +
∂f
∂pi
, si(t0) =∂y0(p)
∂pi
, (i = 1, . . .L)
Sensitivity Analysis – p.10/15
Numerical Sensitivity Analysis of IVPs - Forward
� Finite Difference Approach
{ ∂
∂pi
}y(t;p) ≈ y(t;p + ei∆pi) − y(t;p)
∆pi
Due to the rounding errors, the approximation is only O(√
Tol) with the best choice for∆pi.
� Internal Differentiation
y′(t;p) = f(t, y(t;p);p), y(t0) = y0(p)
⇓
Differentiation + Chain Rule + Clairaut’s Theorem
⇓
s′
i =∂f
∂ysi +
∂f
∂pi
, si(t0) =∂y0(p)
∂pi
, (i = 1, . . .L)
� Taylor Series method using extended rules of AD [Barrio 2006].⇒ second(or higher)-order sensitivities.
Sensitivity Analysis – p.10/15
Numerical Sensitivity Analysis of IVPs - An Example
� The Van der Pol oscillator
y′
1(t) = y2(t)
y′
2(t) = µ(1− y21)y2 − y1
⇓
s′1(1)(t)
s′1(2)(t)
=
0 1
−2µy1y2 − 1 µ(1− y21)
s1(1)
s1(2)
+
0
(1− y21)y2
⇓
s′1(1)(t) = s1(2)
s′1(2)(t) = (−2µy1y2 − 1)s1(1)
+ µ(1− y21)s1(2)
+ (1− y21)y2
Sensitivity Analysis – p.11/15
Numerical Sensitivity Analysis of IVPs - An Example
� The Van der Pol oscillator (∆p = 0.2). The decrease of y1 at t = 20 can be explained as second
order sensitivities being dominant ( ∂y1∂µ
(t = 20) ' 0 , ∂2y1∂µ2 (t = 20) < 0).
0 5 10 15 20−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
t
y1
0 5 10 15 20−8
−6
−4
−2
0
2
4
6
t
∂y1
∂µ
0 5 10 15 20−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
t
y 1(p
+∆
p)
0 5 10 15 20−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
t
y 1(p−
∆p)
Sensitivity Analysis – p.11/15
DDEs and Sensitivity Analysis - Governing Equations
y′(t;p) = f(t, y(t;p), y(α(t, y;p);p), y′(α(t, y;p);p);p)
y(t;p) = φ(t;p), for t ≤ t0(p)
⇓
Differentiation + Chain Rule + Clairaut’s Theorem
⇓
s′i(t) =∂f
∂ysi(t) +
∂f
∂y(αk)
(
y′(αk)
(
∂α
∂ysi(t) +
∂α
∂p
)
+ si(α)
)
+∂f
∂y′(α)
(
y′′(α)
(
∂α
∂ysi(t) +
∂α
∂p
)
+ s′i(α)
)
+∂f
∂p
Sensitivity Analysis – p.12/15
DDEs and Sensitivity Analysis - Proper Handling of Jumps
� Hybrid ODE systems [Tolsma & Barton]
� Continuous transition at t = λ,
y(λ+) = y(λ−)
⇓
∂y
∂pl
(λ+) =∂y
∂pl
(λ−) +(
y′(λ−)− y′(λ+)) ∂λ
∂pl
� Triggered by,
g(t, y, y′;p) = 0
⇓
∂g
∂y′
(
∂
∂t
(
∂y
∂pl
)
+ y′′∂λ
∂pl
)
+∂g
∂y
(
∂y
∂pl
+ y′∂λ
∂pl
)
+∂g
∂pl
+∂g
∂t
∂λ
∂pl
= 0
Sensitivity Analysis – p.13/15
DDEs and Sensitivity Analysis - Proper Handling of Jumps
� DDEs
� Discontinuity points of y′(t;p),
Λ(p) ≡ {λ1(p), λ2(p), . . .}.
� Identified by,
α(λr+1(p), y;p) = λr(p) r = 1, 2, . . .
� Can be viewed as λr+1(p) being a solution of,
g(t, y;p) = α(t, y;p)− λr(p) = 0.
� Using the result for hybrid ODEs,
∂λr+1(p)
∂pl
= −
∂α∂y
∂y∂pl
+ ∂α∂pl− ∂λr(p)
∂pl
∂α∂y
y′ + ∂α∂t
for r ≥ 1
∂λ1(p)
∂pl
=∂t0(p)
∂pl
Sensitivity Analysis – p.13/15
DDEs and Sensitivity Analysis - Proper Handling of Jumps
� Integration (first order sensitivities)
1. Initialize (λ1 = t0(p)).
2. r ← 1.
3. Integrate the equations up to a C1-discontinuity point (λr+1).
4. Update the state variables (sensitivities) using
∂y
∂pl
(λ+r+1) =
∂y
∂pl
(λ−
r+1) +(
y′(λ−
r+1)− y′(λ+r+1)
) ∂λr+1(p)
∂pl
5. r ← r + 1 and restart (3).
Sensitivity Analysis – p.13/15
DDEs and Sensitivity Analysis - Examples
� Mackey-Glass model
y′(t) =2y(t− 2)
1 + y(t− 2)9.65− y(t),
for t in [0, 100]. The history function is
φ(t) = 0.5 for t ≤ 0.
Parameters are
p = [τ, n, A]
where
y′(t) =2y(t− τ)
1 + y(t− τ)n− y(t),
and
φ(t) = A for t ≤ 0.
Sensitivity Analysis – p.14/15
DDEs and Sensitivity Analysis - Examples
� Mackey-Glass model
0 20 40 60 80 1000.2
0.4
0.6
0.8
1
1.2
1.4
1.6
t
y
0 20 40 60 80 100−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
t
∂y
∂τ
0 20 40 60 80 100−0.1
−0.08
−0.06
−0.04
−0.02
0
0.02
0.04
t
∂y
∂n
0 20 40 60 80 1000
0.5
1
1.5
2
2.5
t
∂y
∂A
Sensitivity Analysis – p.14/15
DDEs and Sensitivity Analysis - Examples
� The predator-prey model
y′
1(t) = y1(t)(1 − y1(t − τ) − ρy′
1(t − τ)) −y2(t)y1(t)2
y1(t)2 + 1
y′
2(t) = y2(t)
(
y1(t)2
y1(t)2 + 1− α
)
where α = 1/10 , ρ = 29/10 and τ = 21/50, for t in [0, 30]. The history functions are
φ1(t) =33
100−
1
10t
φ2(t) =111
50+
1
10t
for t ≤ 0.
Parameters are
p = [τ, ρ, α, a, b, c, d]
where
φ1(t) = a + b t
φ2(t) = c + d t
Sensitivity Analysis – p.14/15
DDEs and Sensitivity Analysis - Examples
� The predator-prey model (structure-related parameters, y1)
0 5 10 15 20 25 300.31
0.32
0.33
0.34
0.35
0.36
0.37
t
y1
0 5 10 15 20 25 30−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
t
∂y1
∂τ
0 5 10 15 20 25 30−0.08
−0.06
−0.04
−0.02
0
0.02
0.04
0.06
0.08
t
∂y1
∂ρ
0 5 10 15 20 25 300
0.5
1
1.5
2
2.5
t
∂y1
∂α
Sensitivity Analysis – p.14/15
DDEs and Sensitivity Analysis - Examples
� The predator-prey model (structure-related parameters, y2)
0 5 10 15 20 25 302.21
2.215
2.22
2.225
2.23
2.235
2.24
2.245
2.25
2.255
t
y2
0 5 10 15 20 25 30−0.2
−0.1
0
0.1
0.2
0.3
0.4
0.5
t
∂y2
∂τ
0 5 10 15 20 25 30−0.005
0
0.005
0.01
0.015
0.02
0.025
0.03
t
∂y2
∂ρ
0 5 10 15 20 25 30−18
−16
−14
−12
−10
−8
−6
−4
−2
0
t
∂y2
∂α
Sensitivity Analysis – p.14/15
DDEs and Sensitivity Analysis - Examples
� The predator-prey model (history-related parameters, y1)
0 5 10 15 20 25 30−0.2
0
0.2
0.4
0.6
0.8
1
1.2
t
∂y1
∂a
0 5 10 15 20 25 30−0.5
−0.4
−0.3
−0.2
−0.1
0
0.1
0.2
t
∂y1
∂b
0 5 10 15 20 25 30−0.14
−0.12
−0.1
−0.08
−0.06
−0.04
−0.02
0
0.02
t
∂y1
∂c
0 5 10 15 20 25 30−0.01
0
0.01
0.02
0.03
0.04
0.05
0.06
t
∂y1
∂d
Sensitivity Analysis – p.14/15
DDEs and Sensitivity Analysis - Examples
� The predator-prey model (history-related parameters, y2)
0 5 10 15 20 25 30−0.5
0
0.5
1
1.5
2
t
∂y2
∂a
0 5 10 15 20 25 30−0.8
−0.7
−0.6
−0.5
−0.4
−0.3
−0.2
−0.1
0
t
∂y2
∂b
0 5 10 15 20 25 30−0.2
0
0.2
0.4
0.6
0.8
1
1.2
t
∂y2
∂c
0 5 10 15 20 25 30−0.45
−0.4
−0.35
−0.3
−0.25
−0.2
−0.15
−0.1
−0.05
0
0.05
t
∂y2
∂d
Sensitivity Analysis – p.14/15
The End
Thank you for your attention!
Questions?
Sensitivity Analysis – p.15/15