Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/... · -A power point...

Optimal Control

Prof. Daniela Iacoviello

Lecture 1

26/09/2019 Pagina 2

Systems Identification and Optimal Control

Optimal control:

Period: September - December

Credits: 6

Professor: Daniela Iacoviello

System Identification:

Period: February- May

Credits: 6

Professor: Stefano Battilotti

Pagina 226/09/2019Prof.Daniela Iacoviello- Optimal Control

26/09/2019 Pagina 3

Systems Identification and Optimal Control

Optimal control:

Period: September - December

Credits: 6

Professor: Daniela Iacoviello

System Identification:

Period: February- May

Credits: 6

Professor: Stefano Battilotti


26/09/2019 Pagina 4

Prof. Daniela Iacoviello

Department of computer, control and management

engineering Antonio Ruberti

Office: A219 Via Ariosto 25

http://www.dis.uniroma1.it/~iacoviel

Prof.Daniela Iacoviello- Optimal Control Pagina 426/09/2019

26/09/2019 Pagina 5Prof.Daniela Iacoviello- Optimal Control Pagina 526/09/2019

Mailing list:

Send an e-mail to: [email protected]

Subject: Optimal Control 2019

mailto:[email protected]

26/09/2019 Pagina 6

Grading

Project+ oral exam

Example of project (1-3 Students):

-Read a paper on an optimal control problem

-Study: background, motivations, model, optimal control, solution, results

-Simulations

-Conclusions

-References

You must give me, before the date of the exam (about a week):

-A .doc document

-A power point presentation

-Matlab simulation files

Oral exam: Discussion of the project AND on the topics of the lectures


26/09/2019 Pagina 7

Some projects proposed

Application Of Optimal Control To Malaria: Strategies And Simulations

Performance Compare Between Lqr And Pid Control Of Dc Motor

Optimal Low-thrust Leo (Low-earth Orbit) To Geo (Geosynchronous-earth Orbit)

Circular Orbit Transfer

Controllo Ottimo Di Una Turbina Eolica A Velocità Variabile Attraverso Il Metodo

dell'inseguimento Ottimo A Regime Permanente

Optimalcontrol In Dielectrophoresis

On The Design Of P.I.D. Controllers Using Optimal Linear Regulator Theory

Rocket Railroad Car

Optimal Control Of Quadrotor Altitude Using Linear Quadratic Regulator

Optimal Control Of An Inverted Pendulum

Glucose Optimal Control System In Diabetes Treatment ………


26/09/2019 Pagina 8

Optimal Control Of Shell And Tube Heat Exchanger

Optimal Control Analysis Of A Mathematical Model For Unemployment

Time optimal control of an automatic Cableway

Glucose Optimal Control System In Diabetes Treatment

Optimal Control Of Shell And Tube Heat Exchanger

Optimal Control Analysis Of A Mathematical Model For Unemployment

Time Optimal Control Of An Automatic Cableway

Optimal Control Project On Arduino Managed Module For Automatic Ventilation Of

Vehicle Interiors

Optimal Control For A Suspension Of A Quarter Car Model

………

Pagina 826/09/2019

Some projects proposed

Prof.Daniela Iacoviello- Optimal Control

26/09/2019 Pagina 9

THESE SLIDES ARE NOT SUFFICIENT

FOR THE EXAM:YOU MUST STUDY ON THE BOOKS

Part of the slides has been taken from the References indicated below


26/09/2019 Pagina 10

THESE SLIDES ARE NOT SUFFICIENT

FOR THE EXAM:YOU MUST STUDY ON THE BOOKS

Part of the slides has been taken from the References indicated below


26/09/2019 Pagina 11

References

B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989

E.R.Pinch, Optimal control and the calculus of variations, Oxford science publications,

1993

C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993

A.Calogero, Notes on optimal control theory, 2017

L. Evans, An introduction to mathematical optimal control theory, 1983

H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972

D. E. Kirk, "Optimal Control Theory: An Introduction, New York, NY: Dover, 2004

D. Liberzon, "Calculus of Variations and Optimal Control Theory: A Concise

Introduction", Princeton University Press, 2011

How, Jonathan, Principles of optimal control, Spring 2008. (MIT OpenCourseWare:

Massachusetts Institute of Technology). License: Creative Commons BY-NC-SA.

……………………………………..


26/09/2019 Pagina 12

References

B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989

E.R.Pinch, Optimal control and the calculus of variations, Oxford science publications,

1993

C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993

A.Calogero, Notes on optimal control theory, 2017

L. Evans, An introduction to mathematical optimal control theory, 1983

H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972

D. E. Kirk, "Optimal Control Theory: An Introduction, New York, NY: Dover, 2004

D. Liberzon, "Calculus of Variations and Optimal Control Theory: A Concise

Introduction", Princeton University Press, 2011

How, Jonathan, Principles of optimal control, Spring 2008. (MIT OpenCourseWare:

Massachusetts Institute of Technology). License: Creative Commons BY-NC-SA.

……………………………………..


26/09/2019 Pagina 13

Course outline

• Introduction to optimal control

• Calculus of variations

• Calculus of variations and optimal control

• The maximum principle

• The Hamilton Jacobi Bellman equation

• The linear quadratic regulator

• The minimum time problem

• The linear quadratic gaussian problem


26/09/2019 Pagina 14

Course outline

• Introduction to optimal control

• Calculus of variations

• Calculus of variations and optimal control

• The maximum principle

• The Hamilton Jacobi Bellman equation

• The linear quadratic regulator

• The minimum time problem

• The linear quadratic gaussian problem


26/09/2019 Pagina 15

Background

First course on linear systems

(free evolution, transition matrix, gramian matrix,…)


26/09/2019 Pagina 16

Notations

nRtx )(

pRtu )(

RRRRf pn → ::

kC

State variable

Control variable

Function (function with k-th derivative continuous a.e.)


26/09/2019 Pagina 17

Introduction

Optimal control is one particular branch of modern control that sets

out to provide analytical designs of a special appealing type.

The system, that is the end result of an optimal design, is supposed to

be the best possible system of a particular type

A cost index is introduced


26/09/2019 Pagina 18

Introduction

Linear optimal control is a special sort of optimal control:

✓ the plant that is controlled is assumed linear

✓ the controller is constrained to be linear

Linear controllers are achieved by working with quadratic cost

indices


26/09/2019 Pagina 19Prof.Daniela Iacoviello- Optimal Control

Introduction

Advantages of linear optimal control

✓ Linear optimal control may be applied to nonlinear systems

✓ Nearly all linear optimal control problems have computational

solutions

✓ The computational procedures required for linear optimal design

may often be carried over to nonlinear optimal problems

Pagina 1926/09/2019


History

Pagina 2026/09/2019


VARIATIONAL METHODS

❑ Early Greeks – Max. area/perimeter

❑ Hero of Alexandria – Equal angles of incidence /reflection

❑ Fermat - least time principle (Early 17th Century)

❑ Newton and Leibniz – Calculus (Mid 17th Century)

❑ Johann Bernoulli - brachistochrone problem (1696)

❑ Euler - calculus of variations (1744)

❑ Joseph-Louis Lagrange – Euler-Lagrange Equations (??)

❑ William Hamilton – Hamilton's Principle (1835)

❑ Raleigh-Ritz method – VA for linear eigenvalue problems (late 19th Century)

❑ Quantum Mechanics - Computational methods – (early 20th Century)

❑ Morse & Feshbach – technology of variational methods (1953)

❑ Solid State Physics, Chemistry, Engineering – (mid-late 20th Century)

❑ Personal computers – new computational power (1980’s)

❑ Technology of variational methods essentially lost (1980-2000)

❑ D. Anderson – VA for perturbations of solitons (1979)

❑ Malomed, Kaup – VA for solitary wave solutions (1994 – present)


History

In 1696 Bernoulli posed the

Brachistochrone problem to his contemporaries:

“it seems to be the first problem which explicitely

dealt with optimally controlling the path or the behaviour of a dynamical

system»

Suppose a particle of mass M moves from A to B along smooth wire

joining the two fixed points (not on the same vertical line),under the

influence of gravity.

What shape the wire should be in order that the bead, when released

from rest at A should slide to B in the minimum time?

E.R.Pinch, Optimal control and calculus of variations, Oxford Science Publications, 1993

Pagina 2226/09/2019

Example 1 (Evans 1983)

Reproductive strategies in social insects

Let us consider the model describing how social insects (for example

bees) interact:

represents the number of workers at time t

represents the number of queens

represents the fraction of colony effort

devoted to increasing work force

lenght of the season

)(tw

)(tq

)(tu

T

Motivations


0)0(

)()()()()(

ww

twtutsbtwtw

=

+−= Evolution of the

worker population

Death rate

Known rate at which each worker

contributes to the bee economy

( )

0)0(

)()()(1)()(

qq

twtstuctqtq

=

−+−= Evolution of the

Population

of queens

Constraint for the control:1)(0 tu


The bees goal is to find the control that maximizes the number of

queens at time T:

The solution is a bang- bang control

( ) )()( TqtuJ =


Example 2 (Evans 1983…and everywhere!) A moon lander

Aim: bring a spacecraft to a soft landing on the lunar surface, using

the least amount of fuel

represents the height at time t

represents the velocity =

represents the mass of spacecraft

represents thrust at time t

We assume

)(th

)(tv

)(tm

)(th

)(tu

1)(0 tu


Motivations

Consider the Newton’s law:

We want to minimize the amount of fuel

that is maximize the amount remaining once we have

landed

where is the first time in which

ugmthm +−=)(

0)(0)()()(

)()(

)(

)()(

−=

=

+−=

tmthtkutm

tvth

tm

tugtv

)())(( muJ =

0)(0)( == vh



Analysis of linear control systems

Essential components of a control system

✓ The plant

✓ One or more sensors

✓ The controller

Controller Plant

Sensor


Analysis of linear control systems

Feedback:

the actual operation of the control system is compared to the desired

operation and the input to the plant is adjusted on the basis of this

comparison.

Feedback control systems are able to operate satisfactorily despite

adverse conditions, such as disturbances and variations in plant

properties


Definitions

Consider a function

and

denotes the Eucledian norm

A point is a local minimum of over

If

RRf n →::

nRD

•

Dx * f nRD

)()(

0

*

*

xfxf

xxsatisfyingDxallforthatsuch

−


Definitions

Consider a function

and


A point is a strict local minimum of over

If

RRf n →::

nRD

•

Dx *f nRD

*)()(

0

*

*

xxxfxf

xxsatisfyingDxallforthatsuch

−


Definitions

Consider a function

and


A point is a global minimum of over

If

RRf n →::

nRD

•

Dx * f nRD

)()( * xfxf

Dxallfor


Definitions

The notions of a local/strict/global maximum are defined similarly

If a point is either a maximum or a minimum

is called an extremum

Unconstrained optimization - first order necessary conditions

All points sufficiently near in are in

Assume and its local minimum.

Let be an arbitrary vector.

Being in the unconstrained case:

Let’s consider:

0 is a minimum of g

x

nR

DnR

1Cf *x

)(:)( * += xfg

0

*

toenoughcloseR

Dx

+

*x

Prof.Daniela Iacoviello- Optimal Control 26/09/2019 Pagina 34

First order Taylor expansion of g around

Proof: assume

For these values of

If we restrict to have the opposite sign of

Contraddiction

0=

0)(

lim),()0(')0()(0

=++=→

ooggg

0)0(' =g



0)0(' g

)0(')(

0

gofor

thatsoenoughsmall

)0(')0(')0()( gggg +−

)0('g

)0(')0(')0()( gggg +− 0)0()( − gg

0)0(' =g

δ is arbitrary

First order necessary condition for optimality

( )fofgradienttheis

fffwherexfgT

xx n

1:)()(' * =+=

0)()0(' * == xfg

0)( * = xf

Prof.Daniela Iacoviello- Optimal Control 26/09/2019 Pagina 36



A point satisfying this condition is a stationary point*x


Assume and its local minimum. Let be

an arbitrary vector.

Second order Taylor expansion of g around

Since

nR2Cf *x

0=

0)(

lim),()0(''2

1)0(')0()(

2

2

0

22 =+++=→

oogggg

0)0(' =g

0)0('' g

Unconstrained optimization – second order conditions


Proof: suppose

For these values of

Contraddiction

0)0('' g

22 )0(''2

1)(

0

gofor

thatsoenoughsmall

0)0()( − gg



0)0('' g

By differentiating both sides with respect to

Second order necessary condition for optimality

=

+=

n

i

ix xfgi

1

* )()('

)()()0(''

)()(''

*2

1,

*

1,

*

xfxfg

xfg

T

j

n

ji

ixx

j

n

ji

ixx

ji

ji

==

+=

=

=

=

nnn

n

xxxx

xxxx

ff

ff

f

1

111

2

Hessian matrix

0)( *2 xf

26/09/2019 Pagina 40



Remark:

The second order condition distinguishes minima from maxima:

At a local maximum the Hessian must be negative semidefinite

At a local minimum the Hessian must be positive semidefinite

26/09/2019 Pagina 41



Unconstrained optimization- second order conditions

Let and

is a strict local minimum of f

0)( * = xf2Cf 0)( *2 xf

*x


Global minimum – Existence result

Weierstrass Theorem

Let f be a continuous function and D a compact set

there exist a global minimum of f over D



nRd A vector is a feasible direction at if

for small enough

*x Ddx +*

0

Feasible directions

x*

Not feasiblefeasible


Feasible directions

If not all direction d are feasible, then the condition is no

longer necessary for optimality

If is a local minimum for every feasible

direction d

If is a local minimum is true if

which together with

implies for every

feasible direction d satisfying

0)( * = xf

*x 0)( * dxf

*x 0)0('' g 0)0(' =g

dxfg = )()0(' * dxfdg T = )()0('' *2

0)( *2 dxfd T

0)( * = dxf

Procedure for finding a global minimum


0)( * = xf

0)( * = xf

Convex set


Convexity


0)( * = xf

Constrained optimization

Let

Equality constraints

Inequality constraints

Regularity condition:

1, CfRD n

1,:,0)( ChRRhxh p →=

1,:,0)( CgRRgxg q →

( )

a

a

x

a

qg

qpx

ghrank

dimensionwith

ofconstraintactivethearegwhere

,

a

*

+=


It is a technical

assumption

needed

to rule out

degenerate situations


Lagrangian function

If the stationary point is called normal

and we can assume

( ) )()()(,,, 00 xgxhxfxL TT ++=

*x00

10 =


From now on and therefore the Lagrangian is

If there are only equality constraints the are called

Lagrange multipliers

The inequality multipliers are called Kuhn – Tucker multipliers

i

10 =

( ) )()()(,, xgxhxfxL TT ++=

26/09/2019 Pagina 51



First order necessary conditions for constrained optimality:

Let and

The necessary conditions for to be a constrained local minimum are

If the functions f and g are convex and the functions h are linear these

conditions are necessary and sufficient

i

ixg

x

L

i

ii

T

T

x

=

=

0

,0)(

0

*

*

Dx * 1,, Cghf

*x



First order necessary conditions for constrained optimality:

Let and

The necessary conditions for to be a constrained local minimum are

Dx *

*x


Constrained optimization- a simple problem

Proof (sketch):



Let us consider the function:

Let us now characterize the tangent space

Note that 0 is a minimum of

Consider the first order Taylor expansion:

Consider the admissible set

Minimize the objective function


Example from Bruni et al. 2005

Necessary conditions:

Lagrangian function:

✓ The existence theorem of Weierstrass can’t be applied

✓ g is not convex

only necessary conditions can be applied


Since:

The point P is the only possible candidate.

It can be shown that IT IS the optimal solution

Second order sufficient conditions for constrained optimality:

Let and and assume the conditions

is a strict constrained local minimum if

ixgx

Liii

T

T

x

==

0,0)(0 *

*

Dx * 2,, Cghf

*x

pidx

xdhthatsuch

x

L

x

i

x

T ,...,1,0)(

0**

2

2

==



Definition:



2

2

0

j

i

Tj

i

hL

xx

h

x

Bordered Hessian

A point in which and is called

a non-degenerate critical point of the constrained problem.

*x 0L =

2

2

det 0

0

j

i

Tj

i

hL

xx

h

x

Theorem:



A necessary and sufficient condition for the non-degenerate critical point

to minimize the cost subjects to the constraints

is that for all non-zero tangent vector

E.R.Pinch, Optimal control and calculus of variations, Oxford Science Publications, 1993

*x

f *( ) 0, 1,2,...,jh x j m= =

2

20T L

x

Let us consider the problem of minimizing function

with one constraint

The previous theorem implies that we should look for and

such that:

If the normal to the constraint curve

at has its

normal parallel to

the level surface of that passes through has the

same normal direction as the constraint curve at

In R2 this means that the two curves touch at


A geometrical interpretation1 2( , )f x x

1 2( , )h x x c=

1 2( , )x x x=

( ) 0f h at x + = f h at x = −

0f 0 0h

1 2( , )h x x c= 1 2( , )x x x=

( )f x

1 2( , )f x x1 2( , )x x x=

1 2( , )x x x=

1 2( , )x x x=

Minimize the function

subject to

Introduce a Lagrange multiplier and consider:

There are three unknowns and we need another equation:

and


A geometrical interpretation - Example

2 21 2 1 2( , ) 1f x x x x= − −

21 2 2 1( , ) 1 0h x x x x= − + =

( )2 2 21 2 2 1( ) 1 ( 1 ) 0f h x x x x + = − − + − + =

1 1 22 0 2 0x x x − + = − + =

22 11 0x x− + =

1 20, 1, 2x x = = = 1 21 , 1/ 2, 1

2x x = = =


A geometrical interpretation - Example

1 20, 1, 2x x = = =

1 21 , 1/ 2, 1

2x x = = =

The minimum is at

You can find it evaluating

in the two points

Note that the level curves of f

are centered in the origin O

and f decreases

as you move away from the

origin.

1 20, 1x x= =

2 21 2 1 2( , ) 1f x x x x= − −

RVJ →:

VA


•

Functional

Vector space V,

We need to equipe the function space V with a norm :

it is a real valued function on V

- positive definite

- homogeneous

- satisfies the triangle inequality

DISTANCE or METRIC

00 yify

VyRallforyy = ,

zyzy ++

zyzyd −=),(

Function spaces

RVJ →:

VA

•


)(max0

xyybxa

=On the space a common norm is

where is the Eucledian norm

Function spaces

)],,([0 nRbaC

•

)],,([1 nRbaC

)('max)(max1

xyxyybxabxa

+=

On the space a common norm is

and so on.....

The k-norm can be used on klRbaC nl ),],,([

pb

a

p

pdxxyy

/1

)(

=

0-norm

1-norm

Function spaces


Extrema of J with respect to the 0- norm are called strong extrema

Extrema of J with respect to the 1- norm are called weak extrema

If a curve is a strong minimum then it is a weak one.The converse is not true

Function spaces

Functional

Vector space V,

is a local minimum of J over A if there exists an

RVJ →:

VA

Az *

)()(

0

*

*

zJzJ

zzsatisfyingAzallforthatsuch

−


• RVz + ,,

RVJz

→:)()()()( oJzJzJ z ++=+

0* =z

J


Function spaces

• RVz + ,,

RVJz

→:)()()()( oJzJzJ z ++=+


Function spaces

(see the proof of the necessary condition for optimality of a function)

• RVz + ,,

RVJz

→:)()()()( oJzJzJ z ++=+


Function spaces

(see the proof of the necessary condition for optimality of a function)

Question: how can we compute the first variation of a functional?

• RVJz

→:2

and

)()()()()( 222 oJJzJzJ zz +++=+

0)(*

2 z

J

Az *VA

Function spaces


The Weierstrass Theorem is still valid

If J is a convex functional and is a convex set a local minimum is

automatically a global one and the first order condition are necessary

and sufficient condition for a minimum

VA

Function spaces


•VA

Function spaces


Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/... · -A power point...

Documents

Transcript of Presentazione di PowerPoint - uniroma1.itiacoviel/Materiale/... · -A power point...