Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2019-11-06 · and...
Transcript of Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/Lecture... · 2019-11-06 · and...
Optimal Control
Prof. Daniela Iacoviello
Lecture 4
06/11/2019 Pagina 2
Systems Identification and Optimal Control
Optimal control:
Period: September - December
Credits: 6
Professor: Daniela Iacoviello
System Identification:
Period: February- May
Credits: 6
Professor: Stefano Battilotti
Pagina 206/11/2019Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 3
Systems Identification and Optimal Control
Optimal control:
Period: September - December
Credits: 6
Professor: Daniela Iacoviello
System Identification:
Period: February- May
Credits: 6
Professor: Stefano Battilotti
Pagina 306/11/2019Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 4
Prof. Daniela Iacoviello
Department of computer, control and management
engineering Antonio Ruberti
Office: A219 Via Ariosto 25
http://www.dis.uniroma1.it/~iacoviel
Prof.Daniela Iacoviello- Optimal Control Pagina 406/11/2019
06/11/2019 Pagina 5Prof.Daniela Iacoviello- Optimal Control Pagina 506/11/2019
Mailing list:
Send an e-mail to: [email protected]
Subject: Optimal Control 2019
06/11/2019 Pagina 6
Grading
Project+ oral exam
Example of project (1-3 Students):
-Read a paper on an optimal control problem
-Study: background, motivations, model, optimal control, solution, results
-Simulations
-Conclusions
-References
You must give me, before the date of the exam (about a week):
-A .doc document
-A power point presentation
-Matlab simulation files
Oral exam: Discussion of the project AND on the topics of the lectures
Pagina 606/11/2019Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 7
Some projects proposed
Application Of Optimal Control To Malaria: Strategies And Simulations
Performance Compare Between Lqr And Pid Control Of Dc Motor
Optimal Low-thrust Leo (Low-earth Orbit) To Geo (Geosynchronous-earth Orbit)
Circular Orbit Transfer
Controllo Ottimo Di Una Turbina Eolica A Velocità Variabile Attraverso Il Metodo
dell'inseguimento Ottimo A Regime Permanente
Optimalcontrol In Dielectrophoresis
On The Design Of P.I.D. Controllers Using Optimal Linear Regulator Theory
Rocket Railroad Car
Optimal Control Of Quadrotor Altitude Using Linear Quadratic Regulator
Optimal Control Of An Inverted Pendulum
Glucose Optimal Control System In Diabetes Treatment ………
Pagina 706/11/2019Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 8
Optimal Control Of Shell And Tube Heat Exchanger
Optimal Control Analysis Of A Mathematical Model For Unemployment
Time optimal control of an automatic Cableway
Glucose Optimal Control System In Diabetes Treatment
Optimal Control Of Shell And Tube Heat Exchanger
Optimal Control Analysis Of A Mathematical Model For Unemployment
Time Optimal Control Of An Automatic Cableway
Optimal Control Project On Arduino Managed Module For Automatic Ventilation Of
Vehicle Interiors
Optimal Control For A Suspension Of A Quarter Car Model
………
Pagina 806/11/2019
Some projects proposed
Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 9Pagina 906/11/2019
Example
Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 10Pagina 1006/11/2019
Example
Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 11
THESE SLIDES ARE NOT SUFFICIENT
FOR THE EXAM:YOU MUST STUDY ON THE BOOKS
Part of the slides has been taken from the References indicated below
Pagina 1106/11/2019Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 12
THESE SLIDES ARE NOT SUFFICIENT
FOR THE EXAM:YOU MUST STUDY ON THE BOOKS
Part of the slides has been taken from the References indicated below
Pagina 1206/11/2019Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 13
References
B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989
E.R.Pinch, Optimal control and the calculus of variations, Oxford science publications,
1993
C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993
A.Calogero, Notes on optimal control theory, 2017
L. Evans, An introduction to mathematical optimal control theory, 1983
H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972
D. E. Kirk, "Optimal Control Theory: An Introduction, New York, NY: Dover, 2004
D. Liberzon, "Calculus of Variations and Optimal Control Theory: A Concise
Introduction", Princeton University Press, 2011
How, Jonathan, Principles of optimal control, Spring 2008. (MIT OpenCourseWare:
Massachusetts Institute of Technology). License: Creative Commons BY-NC-SA.
……………………………………..
Pagina 1306/11/2019Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 14
Course outline
• Introduction to optimal control
• Calculus of variations
• Calculus of variations and optimal control
• The maximum principle
• The Hamilton Jacobi Bellman equation
• The linear quadratic regulator
• The minimum time problem
• The linear quadratic gaussian problem
Pagina 1406/11/2019Prof.Daniela Iacoviello- Optimal Control
06/11/2019 Pagina 15Pagina 1506/11/2019Prof.Daniela Iacoviello- Optimal Control
Calculus of
variations
Calculus of
variations
and
Optimal
control
The Pontryagin
principle
The LQ Problem
The Minimum time problem
The LQG problem
06/11/2019 Pagina 16Pagina 1606/11/2019Prof.Daniela Iacoviello- Optimal Control
Calculus of
variations
Calculus of
variations
and
Optimal
control
The Pontryagin
principle
The LQ Problem
The Minimum time problem
The LQG problem
06/11/2019 Pagina 17Pagina 1706/11/2019Prof.Daniela Iacoviello- Optimal Control
Calculus of
variations
Calculus of
variations
and
Optimal
control
The Pontryagin
principle
The LQ Problem
The Minimum time problem
The LQG problem
R.F.Hartl, S.P.Sethi. R.G.Vickson, A Survey of the
Maximum Principles for Optimal Control Problems with
State Constraints, SIAM Review, Vol.37, No.2,
pp.181-218, 1995
Prof.Daniela Iacoviello- Optimal Control Pagina 1806/11/2019
Pontryagin
Lev Semenovich Pontryagin (3 September 1908 – 3
May 1988) was a Soviet Russian mathematician. He was
born in Moscow and
lost his eyesight in a stove explosion when he was 14.
Despite his blindness he was able to
become a mathematician due
to the help of his mother who read
mathematical books and papers to him.
He made major discoveries in a number of fields of
mathematics, including the geometric parts of topology.
Prof.Daniela Iacoviello- Optimal Control Pagina 1906/11/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 2006/11/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 2106/11/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 2206/11/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 2306/11/2019
Principle of optimality
Prof. D.Iacoviello - Optimal Control 2406/11/2019
Principle of optimality
Minimizing over [t, tf] is equivalent to minimizing over [t, t1]
and [t1, tf]
t t1 tf
x(t)
xx1(t1)
x2(t1)
x3(t1)
x1(tf)
x2(tf)
x3(tf)
Prof. D.Iacoviello - Optimal Control 2506/11/2019
Principle of optimality:
OR:
( )
( )
( )
++
= )((),(),(min),(),(min),(
1
1
1
1 ,,
*f
t
tttu
t
tttu
txGduxLduxLttxJ
f
f
( )
( ) ( )
+= 11
*
,
* ),(),(),(min),(
1
1
ttxJduxLttxJ
t
tttu
Prof. D.Iacoviello - Optimal Control 2606/11/2019
•
Let us define the
Function.
If is an optimal solution for the control problem
Prof. D.Iacoviello - Optimal Control 2706/11/2019
It means that if a solution is optimal then it is optimal in any subinterval
Proof:
By contradiction: assume that the relation
is not true.
Prof. D.Iacoviello - Optimal Control 2806/11/2019
There exists a control and the state defined in
such that
I can define a new solution of the optimal control problem:
Prof. D.Iacoviello - Optimal Control 2906/11/2019
Contradiction!!!
Prof. D.Iacoviello - Optimal Control 3006/11/2019
The optimality principle:
Hamilton- Jacobi equation
The Pontryagin principle
Problem 1 : Consider the dynamical system:
with:
and initial instant and state fixed
( )tuxfx ,,=
( )RURCt
f
x
ffRUtuRtx npn
0,,)(,)(
ii xtx =)(
Pagina 3106/11/2019Prof.Daniela Iacoviello- Optimal Control
For the final values assume: where is a function
of dimension of C1 class.
Assume the constraint with
Define the performance index :
with
( ) 0),( = ff ttx
1+ nf
( ) kduxh
f
i
t
t
= ),(),(
( ) niRURCt
h
tx
hh n ,...,2,1,,
)(, 0 =
( ) ( )=f
i
t
t
f duxLtuxJ ),(),(,,( )RURC
t
L
x
LL n
0,,
Determine
• the value
• the control
• and the state
that satisfy
✓ the dynamical system,
✓ the constraint on the control,
✓ the initial and final conditions
✓ and minimize the cost index .
( ) ,if tt
)(0 RCuo
)(1 RCxo
Pagina 3206/11/2019Prof.Daniela Iacoviello- Optimal Control
Hamiltonian function
( ) ( ) ( ) )),(),((,)(,,,, 00 ttutxhuxftuxLuxH TT ++=
Pagina 3306/11/2019Prof.Daniela Iacoviello- Optimal Control
Theorem 1 (necessary condition):
Consider an admissible solution such that
IF it is a local minimum
there exist a constant , ,
not simultaneously null such that :
( )*** ,, ftux
00 *1* , fi ttC
( ) f
ff ttxrank =
*
),(
Pagina 3406/11/2019Prof.Daniela Iacoviello- Optimal Control
,
**
T
x
H
−=
( ) ( )U
ttutxHttxH
,)(,),(),()(,,),( **
0
****
0
*
Moreover there exists a vectorsuch that:fR
T
ft
T
f tH
txT
f
*
*
*
**
)()(
−=
=
R*
The discontinuities of may occur only in the instants
in which u has a discontinuity and in these instants the
Hamiltonian is continuous.
*
Pagina 3506/11/2019Prof.Daniela Iacoviello- Optimal Control
Proof: (some ideas) the necessary condition to be proven is
( ) ( )U
ttutxHttxH
,)(,),(),()(,,),( **
0
****
0
*
Pagina 3606/11/2019Prof.Daniela Iacoviello- Optimal Control
Definition of C
They do not depend on the control u
Pagina 3706/11/2019Prof.Daniela Iacoviello- Optimal Control
Definition of C
Pagina 3806/11/2019Prof.Daniela Iacoviello- Optimal Control
Let us compare this espression with the Hamilton- Jacobi equation
Let us choose
Pagina 3906/11/2019Prof.Daniela Iacoviello- Optimal Control
Let us define
Remark
If the set U coincides with Rp the minimum
condition reduces to :
0=
u
H
Pagina 4006/11/2019Prof.Daniela Iacoviello- Optimal Control
The Pontryagin principle - convex case
Problem 2: Consider the dynamical linear system:
with A and B function of C1 class; assume fixed the initial
and final instants and the initial state, and
Assume
where U is a convex set.
utBxtAx )()( +=
nfff Rtxorfixedxtx = )()(
fip tttRUtu ,)(
Pagina 4106/11/2019Prof.Daniela Iacoviello- Optimal Control
Define the performance index :
with
L convex function with respect to x(t), u(t) in
per ogni
G is a scalar function of C2 class and convex with
respect to x(tf).
( ) ( ) ( ))(),(),(, f
t
t
txGduxLuxJ
f
i
+=
( ) nittURCt
L
x
LL fi
n
i
,...,2,1,,,, 0 =
URn
fi ttt ,
Pagina 4206/11/2019Prof.Daniela Iacoviello- Optimal Control
Determine:
the control
and the state
that satisfy
the dynamical system,
the constraint on the control,
the initial and final conditions
and minimize the cost index .
fio ttCu ,0
fio ttCx ,1
Pagina 4306/11/2019Prof.Daniela Iacoviello- Optimal Control
Theorem 2 (necessary and sufficient condition):
Consider an admissible solution such that
It is a minimum normal (i.e.λ0 =1)
if and only if there exists an n-dimensional vector
such that :
( )oo ux ,
fio ttC ,1
( ) f
o
ff ttxrank =
),(
Pagina 4406/11/2019Prof.Daniela Iacoviello- Optimal Control
oTo
x
tuxH
−=
),,,(
( ) ( ) UttutxHttxH ooooo ,)(),(),()(,),(
nf Rtx )(
oT
f
fo
tdx
dGt
)()( =Moreover, if
Some special cases
06/11/2019 Pagina 45
The Pontryagin principle
Problem 3: Consider the dynamical system:
with:
Assume fixed the initial control instant and the initial and final
values :
Define the performance index :
with
( )uxfx ,=
( ) niURCx
ffRUtuRtx n
i
pn ,...,2,1,,)(,)( 0 =
Pagina 4606/11/2019Prof.Daniela Iacoviello- Optimal Control
ff
ii xtxxtx == )()(
( ) ( ) ))(()(),(,, f
t
t
f txGduxLtuxJ
f
i
+=
( ) 20 ,,...,2,1,, CGniURCx
LL n
i
=
The Pontryagin principle
Problem 3: Consider the dynamical system:
with:
Assume fixed the initial control instant and the initial and final
values :
Define the performance index :
with
( )uxfx ,=
( ) niURCx
ffRUtuRtx n
i
pn ,...,2,1,,)(,)( 0 =
Pagina 4706/11/2019Prof.Daniela Iacoviello- Optimal Control
ff
ii xtxxtx == )()(
( ) ( ) ))(()(),(,, f
t
t
f txGduxLtuxJ
f
i
+=
( ) 20 ,,...,2,1,, CGniURCx
LL n
i
=
Determine:
▪ the value ,
• the control
• the state
that satisfy:
✓ the dynamical system,
✓ the constraint on the control,
✓ the initial and final conditions
✓ and minimize the cost index
( ) ,if tt
)(0 RCuo
)(1 RCxo
Pagina 4806/11/2019Prof.Daniela Iacoviello- Optimal Control
Hamiltonian function
( ) ( ) ( )uxftuxLuxH T ,)(,,,, 00 +=
Pagina 4906/11/2019Prof.Daniela Iacoviello- Optimal Control
The Pontryagin principle
Theorem 3 (necessary condition):
Assume the admissible solution is a minimum
there exist a constant
and a n-dimensional vector
not simultaneously null such that :
( )*** ,, ftux
00
*1* , fi ttC
T
x
H*
*
−=
( ) ( )U
ttutxHttxH
,)(,),(),()(,,),( **0
****0
*
Pagina 5006/11/2019Prof.Daniela Iacoviello- Optimal Control
0*=H
The Pontryagin principle
Remark- If is fixed, the condition
is substituted by
Pagina 5106/11/2019Prof.Daniela Iacoviello- Optimal Control
0*=H
ft
fi tttkH ,,*
=
The Pontryagin principle
Problem 4: Consider the dynamical system:
with:
( )uxfx ,=
( ) niURCx
ffRUtuRtx n
i
pn ,...,2,1,,)(,)( 0 =
Pagina 5206/11/2019Prof.Daniela Iacoviello- Optimal Control
Assume fixed the initial control instant and the initial state
while for final values assume:
where is a function of dimension of C1 class.
Define the performance index :
with
ii xtx =)(
( ) 0)( = ftx
nf
( ) 20 ,,...,2,1,, CGniURCx
LL n
i
=
( ) ( ) ))(()(),(,, f
t
t
f txGduxLtuxJ
f
i
+=
Determine:
▪ the value
▪ the control
▪ and the state
that satisfy
✓ the dynamical system,
✓ the constraint on the control,
✓ the initial and final conditions
✓ and minimize the cost index .
( ) ,if tt
)(0 RCuo
)(1 RCxo
Pagina 5306/11/2019Prof.Daniela Iacoviello- Optimal Control
Theorem 4 (necessary condition):
Consider an admissible solution such that
If it is a minimum
there exist a constant and an n-dimensional vector
not simultaneously null such that :
Moreover there exists a vector such that:
( )*** ,, ftux
00
*1* , fi ttC
fftdx
drank =
*
)(
Pagina 5406/11/2019Prof.Daniela Iacoviello- Optimal Control
T
x
H*
*
−=
( ) ( )U
ttutxHttxH
,)(,),(),()(,,),( **0
****0
*
T
ff
tdx
dt
*
)()(
=
fR
0*=H
The Pontryagin principle
Remark- If is fixed condition
Is substituted by
Pagina 5506/11/2019Prof.Daniela Iacoviello- Optimal Control
0*=H
ft
fi tttkH ,,*
=
The Pontryagin principle
Problem 5: Consider the dynamical system:
with:
( )tuxfx ,,=
( ) niURCx
ffRUtuRtx n
i
pn ,...,2,1,,)(,)( 0 =
Pagina 5606/11/2019Prof.Daniela Iacoviello- Optimal Control
Assume fixed the initial control instant and the initial state
while for final values assume: where is a
function of dimension of C1 class.
Define the performance index :
with
ii xtx =)(
( ) 0),( = ff ttx 1+ nf
( ) ( )=f
i
t
t
f duxLtuxJ ),(),(,,
( ) niRURCt
L
x
LL n
i
,...,2,1,,, 0 =
Determine:
▪ the value
▪ the control
▪ and the state
that satisfy
the dynamical system,
the constraint on the control,
the initial and final conditions
and minimize the cost index .
( ) ,if tt
)(0 RCuo
)(1 RCxo
Pagina 5706/11/2019Prof.Daniela Iacoviello- Optimal Control
Theorem 5: Consider an admissible solution such that
IF it is a minimum there exist a constant and an
n-dimensional vector not simultaneously null
such that :
( )*** ,, ftux
00
*1* , fi ttC
( ) fff ttx
rank =
*
),(
Pagina 5806/11/2019Prof.Daniela Iacoviello- Optimal Control
,
**
T
x
H
−= Rkkd
HH
ft
t
=
+ ,
*
**
( ) ( ) UttutxHttxH ,)(,),(),()(,,),( **0
****0
*
Moreover there exists a vector such that: fR
T
ft
T
f
ft
Htx
tf
*
*
*
**
)()(
−=
=
Remark: If in the cost index a Bolza term is present:
Pagina 5906/11/2019Prof.Daniela Iacoviello- Optimal Control
T
ft
T
f
ft
Htx
tf
*
*
*
**
)()(
−=
=
( ) ( )=f
i
t
t
f duxLtuxJ ),(),(,,
In the necessary conditions the following modifications are required:
Example 3 (from L.C.Evans)
Control of production and consumption
Consider a factory whose output can be controlled.
Let’s set:
x(t) the amount of output produced at time t , 0 ≤ t.
Assume we consume some fraction of the output at each time
and likewise reinvest the remaining fraction u(t) .
It is our control, subject to the constraint
0 ≤ u(t) ≤ 1
Pagina 6006/11/2019Prof.Daniela Iacoviello- Optimal Control
The corresponding dynamics are:
The positive constant k represents the growth rate of our
reinvestment. We will chose K=1.
Assume as cost index the function:
The aim is to maximize the total consumption of the
output
0)0(
0),()()(
xx
ktxtkutx
=
=
( ) −=
ft
dttxtuuJ
0
)()(1))((
Pagina 6106/11/2019Prof.Daniela Iacoviello- Optimal Control
choose k=1
We apply the Pontryagin Principle; the Hamiltonian in the
normal case is:
The necessary conditions are:
( ) ( ) xuxuuxH +−= 1,,
( )
( ) ( ) 1)()()()(max)(),(),(
)()()(
0)(1)()(1)(
10−+=
=
=−−−=
ttxtutxttutxH
txtutx
tttut
u
f
Pagina 6206/11/2019Prof.Daniela Iacoviello- Optimal Control
1,0),()()()()()()()()()()()( −+−+ txtttxttxtxtuttxtutx
1,0,1)()(1)()( −− ttttu since x(t)>0
Pagina 6306/11/2019Prof.Daniela Iacoviello- Optimal Control
=
1)(0
1)(1)(
tif
tiftu
1,0,1)()(1)()( −− ttttu
1 0( )
0
s
s
if t tu t
if t t T
=
where ts is the switching time
From the equation of the costate, since ,
by continuity we deduce for t<tf, t close to tf, that
thus for such values of t.
Therefore and consequently:
More precisely so long as
and this holds for:
0)( =ft
0)( =tu
1)( t
1)( −=t ttt f −=)(
1)( t
ff ttt −1
ttt f −=)(
Pagina 6406/11/2019Prof.Daniela Iacoviello- Optimal Control
For times with t near tf -1 we have
Therefore the costate equation yields:
1− ftt 1)( =tu
( ) )(1)(1)( ttt −=−−−=
Since we have for all
and over this time interval there are no switchings
1)1( =−ft 1− ftt
1)(1
=−− tt fet
Pagina 6506/11/2019Prof.Daniela Iacoviello- Optimal Control
Therefore:
For the switching time
Homework: find the switching time
=
fs
s
tttif
ttiftu
0
01)(*
1−= fs tt
st ft
Optimal solution: we should reinvest all
the output
(and therefore consume nothing)
up to time ts and afterwards we should
consume everything
(and therefore reinvest nothing)
Bang-bang control
Pagina 6606/11/2019Prof.Daniela Iacoviello- Optimal Control