Kalman filtering: past and future

Bologna, September 4, 2002

The geometric approach

and Kalman regulator

Giovanni MARRO

DEIS, University of Bologna, Italy

The purpose of this talk is to show that not only modern system and control theorywas initiated in the early sixties by a remarkable set of scientific contributions byR.E. Kalman, but has also been driven during its development by the analysis andincreasing insight into several of his very basic and general results.

Thus the settlement of system and control theory in the last forty years followed apath that was strongly traced and influenced by R.E. Kalman.

I will try to point out the main landmarks of this path. Of course, this will bedone from a very personal, subjective standpoint: since my research field has beenalmost solely the geometric approach, I will try to explain the impact of Kalman’sresults on the geometric approach setting and, conversely, how Kalman’s results(mainly the Kalman regulator) may be interpreted from the geometric approachstandpoint.

Relating Kalman regulator and filter to the geometric approach is a relatively recenttrend in the scientific literature (H2 optimal control) which, in my opinion, is alsovery suitable for educational aims.

I will try to present my feelings in a conversational way, with a limited use ofmathematical manipulations and formulas. Since I am rather sensitive to practicalapplications, the available algorithms in Matlab, standard or home made, will bequoted throughout the talk.

1

Early references (controllability, observability, filtering and the LQR problem)

KalmanContribution to the theory of optimal control, Boletin de la Societed MathematicaMexicana, 1960.

KalmanA new appraoch to linear filtering and prediction problems, Transactions of theASME, Journal of Basic Engineering, march 1960.

KalmanOn the general theory of control systems, Proceedings of the first IFAC Congress,vol. 1, Butterworth, London, 1961.

KalmanMathematical description of linear dynamical systems, SIAM Journal Control, vol. 1,no. 2, 1963.

Kalman, Ho and NarendraControllability of linear dynamical systems, Contribution to Differential Equations,vol. 1, no. 2, 1962.

The “seminal papers” (25 in the last forty years) are shown in red.

2

Early references (geometric approach - before Wonham and Morse)

Basile, Laschi and MarroInvarianza controllata e non interazione nello spazio degli stati, L’Elettrotecnica,vol.56., n.1, 1969.

Basile and MarroControlled and conditioned invariant subspaces in linear system theory, Journal ofOptimization Theory and Applications, vol. 3, n. 5, 1969.

Basile and MarroOn the observability of linear time-invariant systems with unknown inputs, Journalof Optimization Theory and Applications, vol. 3, n. 6, 1969.

Basile and MarroL’ invarianza rispetto ai disturbi studiata nello spazio degli stati, Rendiconti dellaLXX Riunione Annuale AEI, paper 1-4-01, 1969.

Laschi and MarroAlcune considerazioni sull’ osservabilità dei sistemi dinamici con ingressi inaccessi-bili, Rendiconti della LXX Riunione Annuale AEI, paper 1-1-06, 1969.

3

Early references (geometric approach - Wonham and Morse’s first paper)

Wonham and MorseDecoupling and pole assignment in linear multivariable systems: a geometric ap-proach, SIAM Journal on Control, vol. 8, n. 1, 1970.

Books (geometric approach)

WonhamLinear Multivariable Control – A Geometric Approach, Springer Verlag, 1974-1985.

Basile and MarroControlled and Conditioned Invariants in Linear System Theory, Prentice Hall, 1992.

Trentelman, Stoorvogel and HautusControl Theory for Linear Systems, Springer Verlag, 2001.

4

The topics herein considered

Kalman-derived framework

• controllability◦ observability• Kalman regulator (LQR, H2)• Kalman dual filter◦ Kalman filter• previewed signal optimal decoupling◦ delayed filter (smoother)• H2-optimal model following◦ H2-optimal model observer

Geometric framework

• controlled invariants◦ conditioned invariants• disturbance decoupling problem• measured signal decoupling◦ unknown-input state observer• previewed signal decoupling◦ del. unknown-input state observer• exact model following◦ exact model observer

• primal problem◦ dual problem

5

1 - Controllability and Observability

Controllability and observability are basic properties of the dynamic systems. Theyare usually referred to state solvability of control and/or observation (filtering)problems.

They were introduced by Kalman around 1960, and involved some other basicconcepts, also concomitantly studied by Kalman, like• duality;• relation between continuous and discrete-time systems (sampling theorem).Controllability and observability carried some important further achievements, liketheorems on pole assignment and controlled and conditioned invariants, the basictools of the geometric approach. In fact, they are the first step towards a geometricpicture of the dynamic systems evolution in time, where the concepts of subspaceand invariant subspace give a significant insight, and algebra of subspaces providesa solid algorithmic substance.

Let us briefly recall what a trajectory in the state space means.

6

An example of a dynamic system: an electric motor

viK

amplifier

va

crω, ϑ

motor

vc

⎧⎪⎪⎨⎪⎪⎩

va(t) = Ra ia(t) + Lad iadt

(t) + k1 ω(t)

cm(t) = B ω(t) + Jdωdt

(t) + cr(t)

ω(t) = dϑdt

(t)

with cm(t)= k2 ia(t).

h

u

y

Σ

ẋ(t) = A x(t) + B u(t) + H h(t)

y(t) = C x(t) + D u(t) + G h(t)

where x := [ia ω ϑ]T, u := vi, h := cr,y := ϑ and

A =

⎡⎣ −Ra/La −k1/La 0k2/J −B/J 0

0 1 0

⎤⎦ ,

B =

⎡⎣ K/La0

0

⎤⎦ , H =

⎡⎣ 0−1/J

0

⎤⎦ ,

C =[

0 0 1]

, D = 0 , G = 0

7

State-space models: continuous-time systems

X

x(t)

ẋ(t)

L(t)

ẋ(t) = A x(t) + B u(t)

y(t) = C x(t)

(x∈X =Rn, u∈Rp, y ∈Rq)

ẋ(t) = A x(t) + B u(t)

y(t) = C x(t) + D u(t)

with feedthroughterm D u(t)

C=kerC

Fig. 1.1. A state-space trajectory.

A dynamic system without the feedthrough term is said to be purely dynamic. Thelinear variety L(t)=A x(t)+B u(t) represents the locus of all the state velocitiesdue to the control action u(t).

8

State space models: discrete-time systems

without feedthrough term

x(k+1) = Ad x(k) + Bd u(k)

y(k) = Cd x(k)

with feedthrough term

x(k+1) = Ad x(k) + Bd u(k)

y(k) = Cd x(k) + Dd u(t)

From continuous to discrete-time

The control action is applied stepwise andthe output of the system is accordinglysampled with the same sampling time T .Referring to an equivalent discrete-timesystem provides a significant insight intothe behavior of continuous-time systems.

0 t, kT

9

Controllability and observability

Refer to the continuous-time dynamicsystem

ẋ(t) = A x(t) + B u(t)

y(t) = C x(t) [+D u(t)]

R

X

Let B := imB. The reachability sub-space of (A, B), i.e., the set of all thestates that can be reached from the ori-gin in any finite time by means of controlactions, is the minimal A-invariant con-taining B, or R=minJ↓(A,B). If R=X ,the pair (A, B) is said to be completelycontrollable.

Let C :=kerC. The unobservability sub-space of (A, C), i.e., the set of allthe initial states that cannot be recog-nized from the output function, is themaximal A-invariant contained in C, orQ=maxJ↑(A, C). If Q= {0}, (A, C) issaid to be completely observable.

The above definitions also apply in the discrete-time case.

10

Algorihms and dualities

Consider the sequence

Z1 = BZi = B + AZi−1 i = 2,3, ...

minJ↑(A,B) is obtained when the se-quence stops, i.e., when (Zρ+1 =Zρ).This value of ρ is called the controlla-bility index.

Consider the sequence

Z1 = CZi = C ∩ A−1 Zi−1 i = 2,3, ...

maxJ↓(A, C) is obtained when the se-quence stops, i.e., when (Zρ+1 =Zρ).This value of ρ is called the observabilityindex.

The following dualities hold:

maxJ↓(A, C) =(minJ↑

(AT, C⊥))⊥ minJ↑(A,B) = (maxJ↓ (AT,B⊥))⊥

where the symbol ⊥ denotes the orthogonal complement.

11

Some system properties related to controllability and observability

The sampling theorem. (Kalman, 1960) Let (A, B) be controllable. The cor-responding zero-order hold pair (Ad, Bd) is controllable if the spectrum of A doesnot contain eigenvalues whose imaginary part is a multiple of π/T , where T is thesampling time.

The deadbeat control

0 kρ

u In the discrete-time case the minimum-time control from the origin to a givenfinal state is performed through a dead-beat type of control action and the min-imum time is equal at most to the con-trollability index ρ.

In the continuous-time case this problem does not admit any solution (the minimumtime should be zero and the control should be a distribution).

12

Some consequences of controllability and observability: pole assignment

State feedback

+

+

v u y

xF

Σ

ẋ(t) = (A + BF )x(t) + B v(t)

y(t) = C x(t)

Output injection

u y

Σ

G

ẋ(t) = (A + GC)x(t) + B u(t)

y(t) = C x(t)

The pole assigment theorem. The eigenvalues of A+BF are arbitrarily assignableby a suitable choice of F if and only if the system is completely controllable andthose of A+GC are arbitrarily assignable by a suitable choice of G if and only ifthe system is completely observable.

13

Other types of systems (signal processors)

udelay

y uFIR

y

Fig. 1.2. The delay and the FIR system.

- Continuous-time:

y(t) = u(t − t0)

y(t) =

∫ tf0

W(τ)u(t − τ) dτ

where W(τ), τ ∈ [0, tf ], is a q × p realmatrix of time functions, referred to asthe gain of the FIR system, while [0, tf ]is called the window of the FIR system.

- Discrete-time:

y(k) = u(k − k0)

y(k) =

kf∑l=0

W(l)u(k − l)

where W(k), k ∈ [0, kf ], is a q × p realmatrix of time functions, referred to asthe gain of the FIR system, while [0, kf ]is called the window of the FIR system.

14

Duality

System Σ : (A, B, C, D);

FIR system Σ : W(τ);

Σ1

Σ2

Σ3

u y

⎡⎢⎣

ẋ1(t)ẋ2(t)ẋ3(t)y(t)

⎤⎥⎦ =

⎡⎢⎢⎣

A1 0 0 B10 A2 0 B2

B3,1C1 B3,2C2 A3 00 0 C3 0

⎤⎥⎥⎦⎡⎢⎣

x1(t)x2(t)x3(t)u(t)

⎤⎥⎦

Dual system ΣT : (AT, CT, BT, DT).

Dual FIR system ΣT : WT(τ).

+

+ΣT3

ΣT1

ΣT2

ū ȳ

⎡⎢⎣

˙̄x1(t)˙̄x2(t)˙̄x3(t)ȳ(t)

⎤⎥⎦ =

⎡⎢⎢⎣

AT1 0 CT1 B

T3,1 0

0 AT2 CT2 B

T3,2 0

0 0 AT3 CT3

BT1 BT2 0 0

⎤⎥⎥⎦⎡⎢⎣

x̄1(t)x̄2(t)x̄3(t)ū(t)

⎤⎥⎦

The overall dual system is obtained by reversing the order of serially connectedsystems and interchanging branching points with summing junctions and vice versa.

15

A possible use of FIR systems

Controlling a stable system Σ to agiven final state

+

-

Σc

∫Σx0δ(t) e

x0

τ t

u x

e

Observing an unknown initial stateof a stable system Σ

+

-

Σ

∫Σox0δ(t) e

x0

τ t

y x̃

e

The above problems are dual to each other: Σc and Σo can be profitably realizedwith FIRs. The window of the FIR system on the left is easily computed by solvinga finite-time Kalman regulator problem for the reverse-time system of Σ.

16

Computational support with Matlab

A subspace Y is represented by an orthonormal basis matrix Y (such that Y =imY ).The operation on subspaces related to controllability and observability are

Z = sums(X,Y) Sum of subspaces.Z = ints(X,Y) Intersection of subspaces.Y = ortco(X) Orthogonal complementation of a subspace.Y = invt(A,X) Inverse transform of a subspace.Q = ker(A) Kernel of a matrix.Q = mininv(A,X) Min A-invariant containing imX.Q = maxinv(A,X) Max A-invariant contained in imX.[P,Q] = stabi(A,X) Matrices for the internal and external stability of the

A-invariant imX.P = place(A,B,p) Pole assignment by state feedback (Matlab).

The geometric approach software in Matlab was initiated with a diskette enclosedwith the 1992 Basile and Marro book. It is now freely downloadable from

http://www.deis.unibo.it/Staff/FullProf/GiovanniMarro/geometric.htm

17

2 - Controlled and Conditioned Invariants

Controlled and conditioned invariants are the tools that extend the concept ofinvariance in linear systems. They make the definition of some further systemproperties, besides controllability and observability, and the solution of some basicproblems, very straightforward.

Roughly speaking, the geometric approach studies conditions under which theKalman-derived problems (regulator and filter) admit zero-cost solutions. But thisinterpretation was not so clear at the beginning. It became clear when the minimumH2 norm interpretation of the Kalman regulator and filter was introduced.

The geometric approach has been settled through more than 30 years of contri-butions by several researchers. It gives insight into many interesting properties ofsystems, but has not yet been collected into a simple, easy to read, and completetreatise.

Controlled and conditioned invariants are often referred to as (A, B)-invariants and(C, A)-invariants in the literature, since Wonham gave them these new names in1974 (five years after their introduction).

18

Definitions and algorithms

Given a linear map A : X →X and asubspace B⊆X , a subspace V ⊆X isan (A,B)-controlled invariant if

AV ⊆ V + BThe set of all the (A,B)-controlledinvariants contained in a given sub-space C is closed with respectto the sum, hence has a maxi-mum, which will be referred to asV∗ =maxV↓(A,B, C). It is computedwith the sequence

V1 = CVi = C ∩ A−1 (Vi−1 + B) i = 2,3, . . .maxV↓(A,B, C) is obtained when thesequence stops (Vi+1 =Vi).

Given a linear map A : X →X and asubspace C ⊆X , a subspace S ⊆X isan (A, C)-conditioned invariant if

A (S ∩ C) ⊆ SThe set of all the (A, C)-conditionedinvariants containing a given sub-space B is closed with respect tothe intersection, hence has a mini-mum, which will be referred to asS∗ =minS↑(A, C,B). It is computedwith the sequence

S1 = BSi = B + A (Si−1 ∩ C) i = 2,3, . . .

minS↑(A, C,B) is obtained when thesequence stops (Si+1 =Si).

19

Invariance by state feedback and output injection

Let B be a basis matrix of B: a matrixF exists such that

(A+BF )V ⊆ Vif and only if V is an (A,B)-controlledinvariant. Hence a controlled invari-ant can be made a simple invariant bystate feedback.

V is said to be internally stabilizable ifit can also be internally stabilized as an(A+BF )-invariant by the matrix F .

Let C be a matrix such that C=kerC:a matrix G exists such that

(A+GC)S ⊆ Sif and only if S is an (A, C)-conditionedinvariant. Hence a conditioned invari-ant can be made a simple invariant byoutput injection.

S is said to be externally stabilizable ifit can also be externally stabilized asan (A+GC)-invariant by the matrix G.

Internal stabilizability of controlled invariants and external stabilizability of condi-tioned invariants can easily be checked and stabilizing feedback matrices F and Gcan easily be obtained by using algorithms based on suitable changes of bases inthe state space.

20

The meaning of controlled and conditioned invariants

V

x(0) X

ẋ(t) = A x(t) + B u(t)y(t) = C x(t)

x(k+1) = Ad x(k) + Bd u(k)y(k) = Cd x(k)

Let V∗ = maxV↓(A,B, C),V∗d = maxV↓(Ad,Bd, Cd),S∗d = minS↑(Ad, Cd,Bd).

The meaning of V∗ or V∗d : it is themaximal subspace of the state spacewhere it is possible to follow statetrajectories invisible at the output.In fact, a state trajectory can bemaintained on a subspace V ⊆X ifand only if it is an (A,B) or an(Ad,Bd)-controlled invariant.The meaning of S∗d: it is the maximalsubspace of the state space reach-able from the origin in at most ρsteps with trajectories having all thestates but the last one belonging tokerCd, hence invisible at the output.The integer ρ is the number of iter-ations required for the algorithm ofS∗d to converge.

21

Properties of dynamic systems: invariant zeros

Consider the following figure

V∗

RV∗unstable

zero

stablezero

Recall: V∗ = maxV↓(A,B, C)S∗ = minS↑(A, C,B)

Let RV∗ denote the maximum reach-able subspace on V∗ (which is pole-assignable with state feedback). It hasbeen shown (Morse, 1973) that

RV∗ = V∗ ∩ S∗

Let F be such that (A+BF )V∗ ⊆ V∗⇒ (A+BF )RV∗ ⊆ RV∗. The in-variant zeros of system (A, B, C)or (Ad, Bd, Cd) are the internalunassignable eigenvalues of the(A+BF )-invariant V∗.

A dynamic system is said to be minimum phase if all its invariant zeros are stable.

22

Properties of dynamic systems: left and right invertibility, relative degree

A system is said to be left-invertible or, simply, invertible if, starting from the zerostate, for any admissible output function y(t), t∈ [0, t1] t1 >0 or y(k), k∈ [0, k1],k1 ≥n there exists a unique corresponding input function u(t), t∈ [0, t1) or u(k),k∈ [0, k1 −1]. The left invertibility condition is

V∗ ∩ S∗ = {0}

A system is said to be right-invertible or functionally controllable if there existsan integer ρ≥1 such that, given any output function y(t), t∈ [0, t1], t1 >0 withρ-th derivative piecewise continuous and such that y(0)=0, . . . , y(ρ)(0)=0, ory(k), k∈ [0, k1], k1 ≥ ρ such that y(k)=0, k ∈ [0, ρ−1], there exists at least onecorresponding input function u(t), t∈ [0, t1) or u(k), k∈ [0, k1 −1]. The minimumvalue of ρ satisfying the above statement is called the relative degree of the system.The right invertibility condition is

V∗ + S∗ = Xand the relative degree is the least integer such that V∗ +Sρ = X in the conditionedinvariant algorithm.

23

Extension to systems with feedthrough

Extension to non purely dynamic sys-tems of the above definitions and prop-erties can be obtained through a simplecontrivance (state extension).

vΣe

uΣ

y

integratorsor delays

uΣ

yΣe

z

integratorsor delays

Refer to the second figure: system Σeis modeled by ż(t)= y(t) and the overallsystem by

˙̂x(t) = Â x̂(t) + B̂ v(t)y(t) = Ĉ x̂(t)

with

x̂ :=

[xu

]Â :=

[A 0C 0

]

B̂ :=

[BD

]Ĉ :=

[0 Iq

]

The addition of integrators at inputs or outputs does not affect the system rightand left invertibility, while the relative degree of (Â, B̂, Ĉ) must be simply reducedby 1 to be referred to (A, B, C, D). The controlled invariants obtained by the abovestate extension are called output nulling subspaces (Anderson, 1976).

24

u y

x

Σ

F

An output nulling subspace is simply acontrolled invariant V such that thereexists a state feedback F satisfying

(A+BF )V ⊆ VV ⊆ ker (C +DF )

so that the output is identically zero forany initial state on V.

The seven properties of dynamic systems

• Stability (internal and external)• Controllability• Observability• Left invertibility• Right invertibility• Relative degree• Minimality of phase

25

Computational support with Matlab

Q = mainco(A,B,X) Maximal (A, imB)-controlled invariant contained in im X.

Q = miinco(A,C,X) Minimal (A, imC)-conditioned invariant containing imX.

[P,Q] = stabv(A,B,X) Matrices for the internal and external stabilizability of the(A,im B)-controlled invariant imX.

[Q,F] = vstar(A,B,C,[D]) V∗, maximal output nulling controlled invariant of(A,B,C,[D]) and stabilizing state feedback matrix F .

[Q,G] = sstar(A,B,C,[D]) S∗, minimal conditioned invariant dual of V∗ andstabilizing output injection matrix G.

R = rvstar(A,B,C,[D]) Reachable set on V∗.z = gazero(A,B,C,[D] Invariant zeros of (A,B,C,[D]).

F = effesta(A,B,X) Stabilizing state feedback for the (A,B)-controlled invariant X.

26

3 - The disturbance decoupling problem

The “exact” disturbance decoupling problem by state feedback is the basic problemof the geometric approach, and it was studied by Basile and Marro as one of theearliest applications of the new concepts in 1969.

It was first approached without the stability requirement. Disturbance decouplingwith the stability requirement was satisfactorily settled around by Basile, Marro andSchumacher in 1982, by using self-bounded controlled invariants.

When the disturbance is measurable we have a milder solvability condition and wecan apply a feedforward unit. This is the “exact” counterpart of the Kalman dualfilter, and admits a dual, the unknown-input “exact” observer of the state or of alinear function of the state, which is the counterpart of the Kalman filter.

27

The structural condition

u

de

x

Σ

F

Let us consider the system

ẋ(t) = A x(t) + B u(t) + H h(t)e(t) = E x(t)

where u denotes the manipulable in-put, d the disturbance input. LetB := imB, H := imH, E :=kerE.The disturbance decoupling problemis: determine, if possible, a state feed-back matrix F such that disturbance hhas no influence on output e.

The system with state feedback isdescribed by

ẋ(t) = (A + B F ) x(t) + H h(t)e(t) = E x(t)

It behaves as requested if and only ifits reachable set by d, i.e., the min-imum (A+BF )-invariant containingH, is contained in E.Let V∗(B,E) :=maxV(A,B, E). Sinceany (A+BF )-invariant is an (A,B)-controlled invariant, the inaccessibledisturbance decoupling problem hasa solution if and only if

H ⊆ V∗(B,E)

28

The conditions with stability

This is a necessary and sufficientstructural condition and does not en-sure internal stability. If stability is re-quested, we have the disturbance de-coupling problem with stability. Sta-bility is easily handled by using self-bounded controlled invariants. As-sume that (A, B) is stabilizable (i.e.,that R=minJ (A, B) is externally sta-ble) and let

Vm := V∗(B,E) ∩ S∗(E ,B+H)This is the minimum self-bounded(A,B)-controlled invariant contain-ing H. It is the reachable set onV∗(B,E) corresponding to a control ac-tion through both inputs u and h.The following theorem yields the solu-tion.

If Vm is not internally stabilizable noother (A,B)-controlled invariant in-ternally stabilizable and containingH exists (Basile-Marro-Schumacher,1982-83).

Hence we have obtained the follow-ing result: the disturbance decou-pling problem with stability admits asolution if and only if

H ⊆ V∗(B,E)Vm is internally stabilizable

If the above conditions are satis-fied, a solution is provided by astate feedback matrix such that(A+BF )Vm ⊆ Vm and σ(A+BF ) isstable.

29

If the state is not accessible, disturbance decoupling may be achieved through adynamic unit similar to a state observer. This is called the disturbance decouplingproblem with dynamic measurement feedback and was also solved at the beginningof the eighties (Willems and Commault, 1982).

Extension to systems with feedthough terms

The disturbance decoupling problem for systems with feedthrough terms, like

ẋ(t) = A x(t) + B u(t) + H h(t)e(t) = E x(t) + D u(t) + G h(t)

can easily be handled by state extension.

30

4 - The Kalman Regulator (LQR, H2)

The study of the Kalman linear-quadratic regulator (LQR) is the central topic ofmost courses and treatises on advanced control systems. See, for instance, thebooks by Kwakernaak and Sivan (1973), Anderson and Moore (1989), Syrmos andLewis (1995).

More recently, in the nineties, a certain attention was given to the H2 optimalcontrol, which is substantially a rehash of the LQR with some standard and wellsettled problems of the geometric approach (for instance, disturbance decouplingwith output feedback). Feedthrough is not present in general, so that the standardRiccati-based solutions are not implementable and the existence of optimal solutionis not ensured. Books on this subject are by Stoorvogel (1992), Saberi, Sannutiand Chen (1995).

The computational method they use to solve the H2-optimal problem are linearmatrix inequalities (LMI), supported by a “special coordinate basis” that pointsout the geometric features of the systems dealt with.

An alternative route, which will be briefly presented in the following, is to treat thesingular and cheap problems, where feedthrough is not present, by directly referringto the Hamiltonian system, which can be considered as a generic dynamic system,with all the previously described features.

31

The standard LQR problem

Consider the continuous-time system

ẋ(t) = A x(t) + B u(t) , x(0)=x0

with the performance index

J =

∫ ∞0

(x(t)TQ x(t) + u(t)TR u(t)+

2 x(t)TN u(t))

dt

with

[Q N

NT R

]≥0, R >0.

Consider the discrete-time system

x(k + 1) = Ad x(k) + Bd u(k) , x(0)=x0

with the performance index

Jd =∞∑

k=0

(x(k)TQd x(k) + u(k)

TRd u(k)+

2 x(k)TNd u(k))

with

[Qd NdNTd Rd

]≥0, Rd >0.

The LQR problem: Find a control u(t), t∈ [0,∞) or u(k), k∈ [0,∞) such that Jor Jd is minimal.

If R >0 or Rd >0 the control problem is said to be regular , if R≥0 or Rd ≥0 it issaid to be singular , if R =0 or Rd =0 it is said to be cheap.

32

The solution through the Hamiltonian system

The continuous-time Hamiltonian system:[ẋ(t)ṗ(t)

]=

[A 0

−2Q −AT] [

x(t)p(t)

]+

[B

2N

]u(t)

0 =[

2NT BT] [ x(t)

p(t)

]+[

2R]

u(t)

The discrete time Hamiltonian system:[x(k+1)p(k+1)

]=

[A 0

−2A−TQ −A−T] [

x(k)p(k)

]+

[B

−2A−TN]

u(k)

0 =[ −2BTA−TQ + 2NT BTA−T ] [ x(k)

p(k)

]+[2R − 2BTA−TN ] u(k)

NOTE: In both cases the Hamiltonian system is a quadruple (Â, B̂, Ĉ, D̂) whoseoutput must be maintained at zero for the given initial state x0. Hence the LQRproblem is easily interpreted as a standard disturbance localization problem of thegeometric approach. The results of the geometric approach also give a significantinsight into its solvability in the singular and cheap cases.

33

The solution is achieved as follows:

1 - Regular case. Derive, from the algebraic condition,

u(t) = (2R)−1(2NTx(t) + BTp(t)

)or

u(k) =(2R − 2BTA−TN)−1 ((2BTA−TQ + 2NT)x(k) − BTA−Tp(k))

and substitute in the differential equations of state and costate. An overall au-tonomous differential system is obtained whose eigenvalues are stable-unstable bypairs. Let W (2n×n) be a basis matrix of the stable invariant subspace, so that[

x(t)p(t)

] [W1W2

]α(t) or

[x(k)p(k)

] [W1,dW2,d

]α(k)

hence p(t)=S x(t) or p(k)=Sd x(k), with S :=W2W−11 or Sd :=W2,dW

−11,d , and, by

substitution in the previous equations, u(t)=K x(t) or u(k)=Kd x(k) with K or Kdsuitably defined. Through a Lyapunov-Riccati equation, the optimal cost is alsoderived as xT0 S x0 or x

T0 Sd x0.

NOTE: There are simple contrivances to deal with A−1 and A−T in the discrete-timecase when A is singular and/or the system is not completely controllable (see, forinstance, Marro, Prattichizzo, Zattoni, 2002).

34

u

x0 δ(t) y

x

Σ

K

Hence the regular LQR problem canbe solved by state feedback, as shownin the figure.

u

hy

x

Σ

K

Reformulation in H2 terms

By using a standard matrix decompo-sition:

MT M =

[Q N

NT R

]and taking

[C D

]= M , the prob-

lem on hand can be restated as a min-imum H2 norm problem from input hto output y for the system

ẋ(t) = A x(t) + B u(t) + H h(t)y(t) = C x(t) + D u(t)

A similar procedure can be applied inthe discrete-time case.

‖G‖2 =(tr

(∫ ∞0

g(t) gT(t) dt

))1/2and ‖Gd‖2 =

(tr

( ∞∑k=0

gd(k) gTd (k) dt

))1/2

are the expressions of the H2 norms in terms of the impulse responses g(t) or gd(k).

35

1 - Singular and cheap cases. These can be handled by applying the geometricapproach to the Hamiltonian system (Â, B̂, Ĉ, D̂). The LQR Problem admits asolution if and only if there exists an internally stable output nulling (Â, B̂)-controlledinvariant of the overall Hamiltonian system whose projection on the state space ofthe original system contains the initial state x(0). This projection is defined as

P (V̂) ={

x :

[xp

]∈ V̂

}It can be proven that the internal unassignable eigenvalues of V̂∗, the maximaloutput nulling controlled invariant of (Â, B̂, Ĉ, D̂) are stable-unstable by pairs. Hencea solution of the LQR Problem is obtained as follows:

1. compute V̂∗;2. compute a matrix K̂ such that (Â+B̂K̂)V̂∗ ⊆ V̂∗ and the assignable eigenvalues

(those internal to RV̂∗) are stable;3. compute V̂s, the maximum internally stabilizable (Â+B̂K̂)-invariant contained

in V̂∗;4. if x(0)∈P (V̂s) the problem admits a solution K, that is easily computable as a

function of V̂s and K̂; if not, the problem has no solution.36

The above procedure also provides a state feedback matrix K corresponding tothe minimum H2 norm from h to y. This immediately follows from the expressionof the H2 norm in terms of the impulse response. In fact, the impulse responsecorresponds to the set of initial states defined by the column vectors of matrix H.Thus, the minimum H2 norm disturbance decoupling problem from h to y has asolution if and only if

H ⊆ P (V̂s) with H = imH

On the other hand, it can be proven that in the discrete-time case this problem isalways solvable, since P (V̂s) has dimension n.

0 ρ

dead-beat

regulartrajectory

A typical control sequence in the discrete-timecase is shown in the figure: as the samplingtime approaches zero, the dead-beat segmentapproaches a distribution, which is not obtain-able with state feedback. For this reason solv-ability of the H2 optimal decoupling problem ismore restricted in the continuous-time case.

37

The usefulness of the geometric tools in the LQR problem

Hence the geometric approach tools can profitably be applied to the Hamiltaniansystem for solving the LQR problem, particularly in the singular and cheap cases.

For instance, the dead-beat subarc in the previous figure is also avoided in thediscrete-time case if V̂s is restricted, like in the continuous-time case, by discardingthe zero eigenvalues of V̂∗, whose number equals the dimension of Ŝ∗.Some features of the Hamiltonian system can be expressed in geometric terms,like: if the original system is left-invertible, the Hamiltonian system is both left andright-invertible. A significant information on the features of the LQR problem inthe singular and cheap cases is provided by analyzing the number of invariant zerosof the Hamiltonian system, stable-unstable by pairs: the stable zeros are the modesof the solution.

In conclusion, the basic system properties that can be expressed in geometric terms,like left and right invertibility, relative degree and minimality of phase yield usefulinformation also in the LQR problem.

38

Dealing with non left-invertible systems

The computational support with Matlab for the infinite-time LQR problem consistsof two very efficient routines (Laub, 1974), available in the standard Control SystemToolbox:

[S,L,K] = care(A.B,Q,R,N) Solution of the continuous-time LQR problem.[S,L,K] = dare(A.B,Q,R,N) Solution of the discrete-time LQR problem.

The routine care only works in the regular case, while dare also works in the cheapand singular cases (hence providing the dead-beat subarc). Both routines do notwork if the system is not left-invertible. This is due to the non-uniqueness of thesolution: in fact, the eigenvalues on RV∗ are arbitrarily assignable, since they do notaffect the performance index.

+

+

uv y

x

ΣF

G

K1

h In the figure it is shown how the geomet-ric approach can integrate these standardLQR routines to provide handling of thedegrees of freedom in computing the statefeedback matrix K when the system is notleft-invertible.

39

Equivalence between the LQR-H2 problem and disturbance decoupling

XC1

C

Consider the LQR-H2 problem

ẋ(t) = A x(t) + B u(t) + H h(t)

y(t) = C x(t) + D u(t)

with the cost

J =

∫ ∞0

yT(t) y(t) dt

The following basic result (Stoorvogel, 1992) states a very direct correspondencebetween the LQR-H2 optimal control problem and the geometric approach: thereexist matrices C1, D1 such that, if the output equation of the given system isreplaced by

y(t) = C1 x(t) + D1 u(t)

the optimal control problem becomes zero-cost, i.e., it is transformed into a stan-dard disturbance decoupling problem solvable with geometric tools.This result is rather intuitive: the optimal state feedback provides motion ona controlled invariant that can be made output nulling by changing the outputmatrices (see the figure, referring to the cheap case).

40

Further problems which could be treated in this context

Kalman-derived framework

• controllability◦ observability• Kalman regulator (LQR, H2)• Kalman dual filter◦ Kalman filter• previewed signal optimal decoupling◦ delayed filter (smoother)• H2-optimal model following◦ H2-optimal model observer

Geometric framework

• controlled invariants◦ conditioned invariants• disturbance decoupling problem• measured signal decoupling◦ unknown-input state observer• previewed signal decoupling◦ del. unknown-input state observer• exact model following◦ exact model observer

• primal problem◦ dual problem

41

References

MorseStructural invariants of linear multivariable systems, SIAM J. Control, vol 1, no 3,1973.

AndersonOutput nulling invariants and controllability subspaces, Proceedings of the 6th IFACCongress, paper 43.6, 1975.

Basile and MarroSelf-bounded controlled invariant subspaces: a straightforward approach to con-strained controllability, J. of Optimization Theory and Applic., vol 38, no 1, 1982.

SchumacherOn a conjecture of Basile and Marro J. of Optimization Theory and Applications,vol 41, no 2, 1983.

Willems and CommaultDisturbance decoupling with measurement feedback with stability or pole place-ment, SIAM J. of Control and Optimization, vol 19, no 4, 1981.

42

Kwakernaak and SivanLinear optimal control systems, John Wiley & Sons, 1973.

Anderson and MooreOptimal Control: linear quadratic methods Prentice Hall International, 1989.

Lewis and SyrmosOptimal control, John Wiley & Sons, 1995.

StoorvogelThe singular H2 control problem, Automatica, vol. 28, no 3, 1992.The H∞ control problem: a state space approach, Prentice Hall International, 1992.Saberi, Sannuti and ChenH2 optimal control, Prentice Hall International, 1995.

Marro, Prattichizzo and ZattoniA geometric insight into the discrete-time cheap and singular LQR problems, IEEETrans. Automatic Control, vol 47, no 1, 2002.

Arnold and LaubGeneralized eigenproblem algorithms and software for algebraic Riccati equation,Proceedings IEEE, vol 72, 1984.

43

/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /Unknown

/Description >>> setdistillerparams> setpagedevice