Model Predictive Control1. Theory

Niket S. KaisareDepartment of Chemical Engineering

Indian Institute of Technology – Madras

Summer School on Process Identification and Control NIT-Tiruchirappalli, June 25th 2008

Hierarchy of Process Control

Regulatory Control

Advanced Process Control

Real-TimeOptimizer

PlanningScheduling

StrategicDecisions

seconds

min ~ hours

hours ~ day

week ~ month

quarterly ~ annual

Time-scale

$$$

Example: Shell Heavy Oil Fractionator

� Hierarchical control strategy� PID loops for level &

inventory control� Quality control loops� Minimise energy usage

� Several operating constraints

What is Model Predictive Control?

Exemplary MPC Algorithm

Plant

input output

disturbances

observerstate / information

objectivemax productivity∑

model( )duxfx ,,=�

constraints

maxmin xxx ≤≤

controller

Dynamic Matrix Control:Industrial Origin of MPC

� Initiated at Shell Oil and other refineries (1970s)

� Various commercial software� DMCplus – Aspen Tech� RMPCT –Honeywell� Dozen+ other players (e.g., 3DMPC-ABB)

� >4500 worldwide installations

� Predominant in oil and petrochemical industries

� The range of applications is expanding

A Brief History of MPCAlgorithm Model Objective Pred. Horizon Constraints

LQG (1960) L SS min ISE IO infinity -

IDCOM (1976) L conv min ISE O p IO

DMC (1979) L conv min ISE IOM p IO

QDMC (1983) L conv min ISE IOM p IO

GPC (1987) L ARMA min ISE IO p -

IDCOM-M L conv min ISE O p IO(1988) min ISE I

SMOC (1988) L SS min ISE IO p IO

Rawlings and L SS min ISE IO infinity IO Muske (1993)

Multivariable Process ControlConventional Structure

Global SSOptimization

Plant-Wide Optimization

Local SSOptimization

Unit 1 Local Optimization Unit 2 Local Optimization

DynamicSupervisory/ConstraintControl

High/Low Select Logic

PID Lead/Lag PID

SUM SUM

ModelPredictive

Control (MPC)

MPC Structure

Basic DynamicControl

FCPC

TCLC

FCPC

TCLC

Unit 1 Distributed ControlSystem (PID)

Unit 2 Distributed ControlSystem (PID)

Design Effort

Almost all models used in MPC are typically empirical models “identified” through plant tests rather than first-principles models.

MPC Definition

Model Predictive Control (MPC) refers toa class of algorithms that utilize

an explicit process model to computea manipulated variable profile that will

optimize an open-loop performance objectiveover a future time interval.

The performance objective typicallypenalizes predicted future errors and

manipulated variable movementsubject to constraints.

MPC Definition

Model Predictive Control (MPC) refers toa class of algorithms that utilize

an explicit process model to computea manipulated variable profile that will

optimize an open-loop performance objectiveover a future time interval.

The performance objective typicallypenalizes predicted future errors and

manipulated variable movementsubject to constraints.

Receding Horizon Control Concept

k k+1 k+m-1 k+p-1

Past Future

setpoint

input

output

Analogy to Chess

hgfedcba

1

2

3

4

5

6

7

8

Me

Opponent (Plant)

Model Predictive Control

Opponent’s Move

New Piece Position

Major Components of MPC

1. Dynamic state (stored in memory)

2. Multi-step prediction equation

3. Objectives and Constraints

4. Optimization solver (Linear / Quadratic Program)

5. State update scheme

Predicted future outputs = Effect of current “state” (memory) +Future input adjustments +Feedforward measurement +Feedback correction

General Setup

Current State

Prediction Modelfor Future Outputs

Future Input Moves(To Be Determined)

To Optimization

Feedback / FeedforwardMeasurements

Prediction Measurement

Correction

Dynamic Model State:Compact representationOf the past input record

?

General Setup

Previous State(in Memory)

Current State

Prediction Modelfor Future Outputs

New Input Move(Just Implemented)

Future Input Moves(To Be Determined)

To Optimization

Feedback / FeedforwardMeasurements

State Update

Prediction Measurement

Correction ?

MPC Options

� Model Types� Finite Impulse Response Model or Step Response Model� State-Space Model� Linear or Nonlinear

� Measurement Correction� To the prediction (based on open-loop state calculation)� To the state (through state estimation)

� Objective Function� Linear or Quadratic� Constrained or Unconstrained

Step Response Model

Assumptions:• No immediate effect: S0 = 0• Stable process settles after time n: for 1j nS S j n= ≥ +

1. Dynamic Model

Step Response Model:Superposition Principle

0( ) ( )

k

iv k v i

=

= ∆∑

1. Dynamic Model

Step Response Model:Superposition Principle

1 2( ) ( 1) ( 2) ( ) ( 1) (0)( )

n n ny k v k S v k S v k n S v k n S v Sv k n Sn

= ∆ − + ∆ − + + ∆ − + ∆ − − + +

−

" "�����������������

1

1( ) ( )

n

i k ni

y k v k i S v S−

=

= ∆ − +∑

1. Dynamic Model

Extension to Multivariate Systems

1. Dynamic Model

State Definition

Possible choices of state:Past n input moves, orFuture n output response assuming v is held constant at v(k)

Effect of past inputs is kept in memory as the system state

1b. Definition of “State”

State Definition

( )0 1 1

( ) ( ) ( )n

T T T Tx k y k y k y k−

⎡ ⎤= ⎣ ⎦"

assuming ∆v(j)=0 for( ) ( )iy k y k i= +

Define:

2. Effect of past inputs on n-step future predictions is the current state:

1.

1b. Definition of “State”

1. Dynamic State for Step Response Model

� Step response model

� Definition of the state� Stores n future output response in the memory

assuming the input is kept constant at u(k-1)

( ) ( )1

( )n

i ni

y k S v k i S v k n=

= ∆ − + −∑

( )

( )( )

( )⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

− ky

kyky

kY

n 1

1

0

~#

( ) ( )( ) ( ) 01

with==+∆=∆

+="kuku

ikykyi

Multi-Step Prediction

Main interest for control: predicting future output behavior.

Æ Future output = function of (past input, hypothesized future input)

Æ Dynamic state: memory about the effect of past input

Æ Future output = function of (dynamic state, future input)

2. Multi-step Prediction

Multi-Step Prediction

Effect of future input moves (to be determined)

x(k)

2. Multi-step Prediction

2. Multi-Step Prediction( )( )

( )=

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

+

++

kpky

kkykky

#21 ( )

( )

( )⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

ky

kyky

p

#2

1

( ) ( ) ( )100

1

0

11

12

1

−+∆

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

+++∆

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

+∆

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

+−−

mku

S

ku

S

Sku

S

SS

mppp

#"

##

( ) ( ) "##

++∆

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

+∆

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

−

1

0

1

12

1

kd

S

Skd

S

SS

dp

d

dp

d

d ( )( )

( )⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

+

++

kpke

kkekke

#21

Effect of past inputs

Effect of current and future inputs

Effect of measured disturbances Unmeasured disturbances

+

+

+

2. Multi-step Prediction

2. Multi-Step PredictionAssume measured disturbance remains constant: d(k) = d(k+1) = d(k+2) = …

∆d(k+1) = ∆d(k+2) = … = 0

Previous prediction error is added as a constant bias terme(k+1) = e(k+2) = … e(k+p) = ymeasured(k) – y(k)

( )( )

( )( )

( )( )

( )

( ) ( ) ( )( )kykykd

S

SS

mku

kuku

SSS

SSS

kY

kpky

kkykky

dp

d

d

u

mppp

02

1

11

12

1

~

1

11

1

1000

~21

−

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+∆

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

+

∆

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−+∆

+∆∆

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

+

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

+

++

+−−

=

##

�� �� �

#

���� ���� �"

#%##""

#

US

M

3. Objective Function and Constraints

Constraint violation may not be acceptable

Shaded region is the setpoint tracking error

ySP

yMax

( ) ( )

( )[ ] ( )[ ]

( )[ ] ( )[ ] ⎪⎪⎭

⎪⎪⎬

⎫

⎪⎪⎩

⎪⎪⎨

⎧

+∆+∆

++−+−

∑

∑−

=

=

−+∆∆ 1

0

1

1,,min

m

l

p

iSPSP

mkukulkuRTlku

kikyyQTkikyy

"

3. Objective function and constraints

Output Trajectories

quadratic penalty

past future

Reference trajectory

quadratic penalty

past future

Setpoint

quadratic penalty

past future

Zonequadratic penalty

past future

Funnel

3. Objective function

Constraints

� Constraints include� Input magnitude constraints:

� Input rate constraints:

� Output magnitude constraints:

maxmin )( uikuu ≤+≤

max)( uiku ∆≤+∆

maxmin )( yikyy ≤+≤

3. Constraints

Constraint Formulations

Hard constraint

past future

Soft constraint

past future

quadratic penalty

past future

Setpoint approximation of soft constraint

quadratic penalty

3. Constraints

Optimisation in Receding Horizon Control

k k+1 k+m-1 k+p-1

Past Future

setpoint

input

output Which input move to choose?

3&4. Objective function and Optimisation

4. Optimization: Quadratic Program

3&4. Objective Function for QP

3&4. Constraint Formulation for QP

3&4. Constraint Formulation in QP

3&4. Constraint Formulation in QP

3&4. Constraint Formulation in QP

Receding Horizon Control Concept

k k+1 k+m-1 k+p-1

Past Future

setpoint

input

output

5. State Update Scheme

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

−

−

)()(

)()(

1

2

1

0

kyky

kyky

n

n

#

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

++

++

−

−

)1()1(

)1()1(

1

2

1

0

kyky

kyky

n

n

#

N N )()()1( 1 kvSkxMkxresponsestepshift

∆+=+

drop

S1 S2

Sn-1

Sn

Recall from previous lecture: State Update Equation

MPC Definition

Model Predictive Control (MPC) refers toa class of algorithms that utilize

an explicit process model to computea manipulated variable profile that will

optimize an open-loop performance objectiveover a future time interval.

The performance objective typicallypenalizes predicted future errors and

manipulated variable movementsubject to constraints.

Model Predictive Control2. Applications

Niket S. KaisareDepartment of Chemical Engineering

Indian Institute of Technology – Madras

Summer School on Process Identification and Control NIT-Tiruchirappalli, June 25th 2008

Example 1: Blending System Control

� Control rA and rB� Control blend flow q� Constraints on MV

Example 1: Blending System Control

� Control rA and rB� Control blend flow q� Constraints on MV

Classical Solution MPC Solution

Minimize deviation from the set-point for p steps into the future.

( ) ( )( )∑

= ⎥⎥⎦

⎤

⎢⎢⎣

⎡

−+

−+−

−++

p

i sp

spBB

spAA

uuu qq

rrrr

mkkk 1

2

22

...,,,1

1

minγ

maxmin uuu i ≤≤

Example 2: Heavy Oil Fractionator

� Objective� Keep temperature y7 above

a minimum value� Keep y1 and y2 at setpoint� Minimize u3 to maximize

heat recovery

� Several constraints

� Non-square system

Example 3: Ethylene Plant

Furnaces

PrimaryFractionator

QuenchTower

Charge GasCompressor

Chilling

DemethanizerDeethanizer

EthyleneFractionator

DebutanizerPropyleneFractionator

DepropanizerFuel OilFuel Oil

HydrogenHydrogen

MethaneMethane

EthyleneEthylene

EthaneEthane

PropylenePropylene

PropanePropane

B - BB - B

GasolineGasolineLight H-CLight H-CNaphthaNaphtha

FeedstockFeedstock

Typical Scale of Modeling and Control123456789

1011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Popularity of MPCSystematic and Integrated Solution

Popularity of MPCSystematic and Integrated Solution

Advantages over traditional APC� Integrated solution

� automatic constraint handling� Feedforward / feedback� No need for decoupling or delay compensation

� Efficient Utilization of degrees of freedom� Can handle nonsquare systems (e.g., more MVs and CVs)� Assignable priorities, ideal settling values for MVs

� Consistent, systematic methodology� Realized benefits

� Higher on-line times� Cheaper implementation� Easier maintenance

Popularity of MPCEmerging Popularity of On-line Optimization

� Process optimization and control are often conflicting objectives� Optimization pushes the process to the boundary of constraints.� Quality of control determines how close one can push the

process to the boundary.

� Implications for process control� High performance control is needed to realize on-line

optimization.� Constraint handling is a must.� The appropriate tradeoff between optimization and control is

time-varying and is best handled within a single framework

Popularity of MPCEmerging Popularity of On-line Optimization

Popularity of MPCEmerging Popularity of On-line Optimization

Bi-level Optimization in MPC

Steady-State Optimization(LP)

Dynamic Optimization(QP)

Optimal setpoints

Regulate low-level loops Measurements(feedback update)

S.S. prediction model

Economic optimization

Minimization oftracking error

Example: Model Predictive Control of Mammalian Cell Cultures

Niket S. Kaisare, Jay H. LeeA. A. Namjoshi, D. Ramkrishna

Presented at AIChE Meeting, November 2000

Overview of the Talk

� Introduction� Hybridoma cell reactor

� Cybernetic Modeling� Case Study

� Simpler microbial cell system� Full hybridoma cell system

� Conclusions

Introduction and Motivation

� Hybridoma Cells� Hybrid mammalian cells� Produce monoclonal

antibodies

� Applications of antibodies� Protein

detection/purification� Immunotherapy� Tumor imaging

B-lymphocytes

Myeloma

Hybridoma

• Tumor cells• Rapid in-vitro growth

• Produce antibodies

Hybridoma Cell Reactor

Air (O2)

• Glucose (s1f)• Glutamine (s2f)• Amino acids

• Unconsumed substrates• Biomass w/ antibodies• Waste metabolites

(Lactate, Alanine etc.)

Steady State Multiplicity

0

2

4

6

0 100 200 300Time (hrs)

Cel

l con

cn (x

106

cel

ls/m

l)

Three types of steady states observed for same input conditions

Differ in the metabolic states of the cells

•

•

Start up Experiments, Hu et al. (1998)

Hybridoma Reactor Control

� Previous Work� Detremblay et al. (1993): Estimation and PI control� Lenas et al. (1997): Adaptive fuzzy control� Dhir et al. (2000)

� Fuzzy update of model parameters� Heuristic random optimizer (HRO)

� Europa et al.(2000): Heuristic scheme

� Our focus: Model based optimal control

Bioreactor Modeling

� Monod type model

� Phenomenological modifications possible � Does not capture cellular metabolic regulations

� Detailed description of extremely large number of metabolic reactions� Unsuitable for on-line optimization and control

sKsr PS +

=→maxµ

Cybernetic Modeling (1)

� Characteristics� Captures internal cellular regulatory mechanisms� Relatively fewer metabolic reactions

� Accurate description of average cell in a batch, fed-batch or continuous culture� Diauxie growth phenomenon� Mammalian cultures� PHB storage intermediate synthesis

Cybernetic Modeling (2)

� Postulates� Kuberne’tes (Greek): “Steersman”

� A Metabolic Network has physiological objectives� Limited pool of resources� A cell directs the synthesis and activity of enzymes

in a metabolic competition� Mathematically, through cybernetic regulation variables� u modifies enzyme synthesis

� v modifies enzyme activity

� Cells are optimal strategists

^

Analysis of a Simpler Model

� Single cybernetic competition

� Bacterial cell growth� Diauxie phenomenon

(Kompala et al., 1986)� To understand cybernetic

competitions

� Model Equations

( ) ( ) 2,1=−−= icYvrSSDdtdS

iiiiifi

( ) 2,1* =+−−= irereurdtde

ii eigiiei β

( )cDrdtdc

g −=

⎥⎦

⎤⎢⎣

⎡+

= maxmax

i

i

ii

ii e

eSK

Sr µ ),max( 21

11 rr

rvsc =

S1 S2

C

The Steady State Multiplicity

� Two stable steady states for certain range of substrate feeds

� Objectives are� Maintain the reactor

at the steady state� Drive the reactor to

other steady state

Bifurcation diagram for the system(Namjoshi et al., unpublished)

Step Variations in s2f

� At steady state, we apply a series of step changes in s2f

� Measurement noise

� The system drifts to the other steady state in open loop

0 50 100 150 2000

0.02

0.04

0.06

0.08

Time (hrs)

Bio

mas

s c(

g/lit

re)

Reactor Drifts in Open Loop

0 50 100 150 200

0.16

0.18

0.2

Time (hrs)

s2f (

g/lit

re)

Model Predictive Control

� Dynamic Model� Cybernetic model

� Linear Approximation� NLP – difficult to solve� Linearize in the horizon

� Optimizer� Measurement Correction

� Extended K.F.� Glucose and biomass

feedback PREDICTION HORIZON

k k+m-1PAST FUTURE

TARGET

PROJECTEDOUTPUT

INPUTMOVES

ymax

( ) ( ) ⎥⎦⎤

⎢⎣⎡ ∆Λ+−Λ ++∆

2

2

2

2|11min ku

kkky

uUYR

Manipulated and Controlled Variables(case A: Two M.V.s and two C.V.s)

� C.V.: biomass, s2

� M.V.: D, s1f

� Controller is able to control the reactor outputs at the given set points

� However, the cells are in physiologically different state

0 20 40 60 80 100 120 140 160 180 2000

0.005

0.01

0.015

0.02

0.025

0.03

0.035

0.04

Time (hrs)

Bio

mas

s c(

g/lit

re)

:

0 20 40 60 80 100 120 140 160 180 200

0.7

0.75

0.8

0.85

0.9

0.95

Time (hrs)

Dilu

tion

Rat

e D

(1/h

our)

0 20 40 60 80 100 120 140 160 180 2000.04

0.05

0.06

0.07

0.08

0.09

time (hrs)

s1f (

Man

ipul

ated

)

Cell Mass Concentration

Case B: One M.V. and two C.V.s

� Changes implemented� Decreased weight on exit s2

concentration� M.V.: Dilution rate D

� Controlled at the desired steady state

� Behavior is better than case 1

0 50 100 150 2000.01

0.02

0.03

Time (hrs)

Bio

mas

s c(

g/lit

re)

Case 2(b)

0 50 100 150 2000.75

0.8

0.85

0.9

Time (hrs)

Dilu

tion

Rat

e D

(1/h

our)

Varying the Steady State

� At t=50 hours, step change in reference trajectory to another steady state

� White measurement noise

0 50 100 150

0.02

0.04

0.06

Time (hrs)

Bio

mas

s c(

g/lit

re)

Driving the reactor to other S.S.

MeasurementsEstimate

0 50 100 150

0.7

0.75

0.8

0.85

0.9

Time (hrs)D

ilutio

n R

ate

D(1

/hou

r)

Reference Trajectory

The controller is able to reject disturbances and track the reference trajectory

CELL

Model Equations

Hybridoma CellsMetabolic Network

C’

M2 M4

S2

S1

M1M3L

( ) ( )crrvrvvrssDdtds mmsdsdsc

f 2122111111 +++−−=

Namjoshi et al.,Session #271b

Set of 16 ODEs

4 sets of cybernetic regulation variables

Result from 4 substitutable cybernetic competitions

•

•

•

Step Response at the LOW State

� We give a 10% step down change in D in our simulated reactor

� Highly nonlinear and stiff nature of the system

� Biomass increases with decrease in D

1000 1500 2000 25000

0.1

0.210% step down in D at steady state

s1 (g

/litre

)

1000 1500 2000 25000.4

0.6

0.8

C (g

/litre

)

1000 1500 2000 25000

0.5

1

time (hrs)

C'n

etic

Var

v7s

d

LOW

Control Actions: LOW to HIGH State

� Starting the reactor at the low biomass state

� Step change in the reference trajectory

� Controller drives reactor to the high biomass steady state

0 100 200 300 400 500 6000

0.5

1

1.5

Time (hrs )

Bio

mas

s c(

g/lit

re)

Driving the reactor from to state

0 100 200 300 400 500 6000.01

0.02

0.03

0.04

Time (hrs)

Dilu

tion

Rat

e D

(1/h

our)

LOW HIGH

Step Response at the HIGH State

� We give a 10% step up change in D in our simulated reactor

� Biomass first increases with increase in D

� When metabolic change is triggered, there is a sudden decrease

� S1 behavior is relatively monotonous

1000 1500 2000 25000

0.005

0.01

0.01510% step up in D at steady state

s1 (g

/litre

)

1000 1500 2000 25000.75

1

1.25

C (g

/litre

)

1000 1500 2000 25000

0.5

1

time (hrs)

C'n

etic

Var

. v1s

d

HIGH

Control Actions: HIGH State

� Drive the reactor from high to intermediate biomass steady state

� Controlled Variable: c

� Prediction horizon is not long enough

0 100 200 300 400 500 600

0.8

0.9

1

Controlled Variable: biomass

biom

ass

C(g

/litre

)

0 100 200 300 400 500 6000

2

4

6 x 10-3

gluc

ose

s1(g

/litre

)

0 100 200 300 400 500 6000.01

0.02

0.03

0.04

time (hrs)

M.V

. - D

(1/h

our)

Control Actions: HIGH State

� Same as previous case

� Controlled var.: c and s1

� Controller takes the reactor to the intermediate state

� We use monotonicity of s1 advantageously

0 100 200 300 400 500 6000.8

1

1.2

Controlled variables: biomass and glucose

biom

ass

C(g

/litre

)

0 100 200 300 400 500 6000

0.005

0.01

0.015

gluc

ose

s1(g

/litre

)

0 100 200 300 400 500 6000.03

0.035

0.04

0.045

time (hrs)

M.V

. - D

(1/h

our)

Conclusions

� MPC algorithms were used to drive biological reactors from one state to another “multiple” state

� Successive linear approximation based controller was applied effectively to the cybernetic models displaying multiplicity features in biological systems

� Necessary to look at the entire state matrix rather than just the biomass or nutrient conc.

� Non-differentiability of the cybernetic v variable was overcome by using relevant approximations in different metabolic states