International Symposium on Systems Optimization and Analysis: Rocquencourt, December 11–13,...
339
Lecture Notes in Control and Information Sciences Edited by A V Balakrishnan and M.Thoma 14 International Symposium on Systems Optimization and Analysis Rocquencourt, December 11-13, 1978 IRIA LABORIA Institut de Recherche d'lnformatique et d'Automatique Rocquencourt - France Edited by A. Bensoussan and J. L. Lions Springer-Verlag Berlin Heidelberg New York 1979
Transcript of International Symposium on Systems Optimization and Analysis: Rocquencourt, December 11–13,...
Lecture Notes in Control and Information Sciences Edited by A V Balakrishnan and M.Thoma
14
International Symposium on Systems Optimization and Analysis Rocquencourt, December 11-13, 1978 IRIA LABORIA Institut de Recherche d'lnformatique et d'Automatique Rocquencourt - France
Edited by A. Bensoussan and J. L. Lions
Springer-Verlag Berlin Heidelberg New York 1979
Series Editors h~ V. Balakrishnan. M. Thoma
Advisory Board L. D. Davisson • A. G. J. MacFarlane • H. Kwakernaak • Ya. Z. Tsypkin
Editors Prof. A. Bensoussan Prof. J. L. Lions
IRIA LABORIA Domaine de Voluceau - Rocquencourt F-78150 Le ChesnaytFrance
With 16 Figures
ISBN 3-540-09447-4 Springer-Verlag Berlin Heidelberg NewYork ISBN 0-387-0944?-4 Springer-Verlag NewYork Heidelberg Berlin
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically those of translation, re- printing, re-use of illustrations, broadcasting, reproduction by photocopying machine or similar means, and storage in data banks. Under § 54 of the German Copyright Law where copies are made for other than private use, a fee is payable to the publisher, the amount of the fee to be determined by agreement with the publisher. © Springer-Verlag Berlin Heidelberg 1979 Printed in Germany Printing and binding: Beltz Offsetdruck, Hemsbach/Bergstr. 2060/3020-543210
This symposium is organized by the Institut de Recherche d'Informatique et d'Automatique under the sponsorship of:
• Association Fran~.aise pour la Cybern~tique Economique et Technique (AFCET)
• International Federation of Automatic Control (IFAC) Technical Committee of Theory
Ce colloque est organis6 par l'Institut de Recherche d'Informatlque et d'Automatique (IRIA) sous le patronage de:
• Association FranCalse pour la Cybern~tique Economique et Technique (AFCET)
• International Federation of Automatic (IFAC) Technical Committee of Theory
Organicers - Organisateurs
A. BENSOUSSAN
J. L. LIONS
Organization committee - Comit6 d'organisation
A. BENSOUSSAN
P. FAURRE
A. FOSSARD
H. KWAKERNAAK
J. LESOURNE
J. L. LIONS
(IRIA/LABORIA)
(AFCET - IRIA/LABORIA)
(AFCET) (IFAC)
(CNAM)
(IRIA/LABORIA)
Scientific Secretaries - Secr6taires Scientifiques
P. NEPOMIASTCHY
Y. LEMARECHAL
(IRIA/LABORIA)
(IRIAJLABORIA)
Symposium Secretariat - Secr6tariat du Colloque
Th. BRICHETEAU (IPdM
Foreword
This international symposium on analysis and optimization was the
third of a series. Organized by IRIA with the co-sponsorship of AFCET
and IFAC, it has gathered more than 200 participants from 18 different
countries.
Five sessions were essentially dedicated to the following topics:
Economic models; identification, estimation, filtering; adaptative
control; numerical methods in optimization; distributed systems.
The conference was followed by a special two-day meeting on industrial
applications, co-organized by AFCET and IRIA. For this reason the
papers related with methodology were concentrated during the first
three days and will be found in this book.
The organizers wish to express their gratitude to IRIA for the support
given to the conference, in particular to the Department of External
Relations. Thanks should also be addressed to AFCET and IFAC for their
sponsorship.
This symposium is now regularly organized every two years in december.
The large variety of fields covered by "Analysis and Optimization"
allows the organizers to select and emphasize different topics at each
symposium.
The Springer-Verlag series "Lecture Notes in Control and Information
Sciences" edited by Professor Balakrishnan and Professor Thoma has
already published the proceedings of the second IRIA Symposium 1976.
We are quite happy to see the proceedings of the third one published
in the same series.
May, 1979 Alain Bensoussan Jacques-Louis Lions
Table of Contents - Table des matieres
ECONOMICAL MODELS MODELES ~CONOMIQUES ............................................
An international agreement as a complementarity problem M.A. Keyzer (Netherlands) ......................................
Solving nonlinear economic planning models using GRG algorithms L.S. La~don, A. Meerau~ (USA) ..................................
Specification and estimation of econometric models with generalized expectations K.D. Wall (USA) ................................................
Implementation of the model in codes for control of large econometric models A. Drud (Denmark) ...............................................
MODULECO, aide ~ la construction et ~ l'utilisation de modules " macroeconomlques P. Nepomiastchy, B. Oudet, F. Rechenmann (France) ..............
17
34
49
61
IDENTIFICATION, ESTIMATION, FILTERING IDENTIFICATION, ESTIMATION, FILTRAGE ........................... 72
A calculus of multiparameter martingales and its applications E. Wong (USA) .................................................. 73
Orthogonal transformation (square root). Implementations of the generalized Chandrasekhar and generalized Levinson algorithms f. KaiZath, A. Vieira, M. Morf (USA) ........................... 81
Shortest data description and consistency of order estimates in arma-processes J. Rissanen (USA) .............................................. 92
Spectral theory of linear control and estimation problems E.A. Jonckheere, L.M. Silverman (USA) .......................... 9g
Un algorithme de lissage M. CZerget, F. Germain (Prance) ................................ IiO
Reduced order modeling of closed-loop nash games H.K. KhakiS, B.F. Gardner Jr., J.B. Cruz Jr., P.V. Kokotovie (USA) ............................................ 119
VII
Quantum estimation theory S.K. Mitter, S.K. Young (USA) ................................. 127
ADAPTIVE CONTROL CONTROLE ADAPTATIF ............................................
Piece,wise deterministic signals K.J. Astr~m (Sweden) ..........................................
Adaptive control of Markov chains V. Borkar, P. Varaiya (USA) ...................................
Resource management in an automated warehouse Y.C. Ho, R. Suri (USA) ........................................
Dualit~ asymptotique entre les syst~mes de commande adaptative avec mod&le et les r~gulateurs ~ variance minimale auto-ajustables Y. Landau (France) ............................................
137
138
145
153
168
NUMERICAL METHODS IN OPTIMIZATION MRTHODES NUMRRIQUES EN OPTIMISATION ........................... 178
On the Bertsekas' method for minimization of composite functions B.T. PoZjak (USSR) ............................................ 179
On e-subgradient methods of non-differentiable optimization E.A. Nurminski (Austria) ..................................... 187
Non-differentiable optimization and large scale linear programming J.F. Shapiro (USA) ............................................ 196
Algorithms for non-linear multicommodity network flow problems D.P. Bertseka8 (USA) .......................................... 210
A 2-stage algorithm for minimax optimization J. Hald, K. Madsen (Denmark) .................................. 225
DISTRIBUTED SYSTEMS SYSTEMES DISTRIBUES ........................................... 240
Certain control problems in distributed systems A.G. Butkovskiy (USSR) ........................................ 241
VIII
Partitioning: the multi-model framework for estimation and control D.G. Lainiotis (USA) ...........................................
Water waves and problems of infinite time control D.L. Russel, R.M. Reid (USA) ...................................
Boundary stabilizability for diffusion processes R. Triggiani (USA) .............................................
Spline based approximation methods for control and identification of hereditary systems H.T. Banks, J.A. Burns, E.M. Cliff (USA) .......................
252
291
304
314
Stabilization of boundary control systems J. Zabczyk (Poland) ............................................ 321
ECONOMICAL MODELS
MODELES I~CONOMIQUES
AN INTERNATIONAL AGREEMENT AS A COMPLemENTARITY PROBLEM
M.A. Keyzer
Centre for World Food Studies
Free University, Amsterdam, the Netherlands
Abstract
A general equilibrium model is presented which describes
the operation of a buffer stock agreement on the world
market. The model is reformulated as a complementarity
problem and its solution through nonsmooth optimization
is described.
1. A COMPETITIVE MODEL
We consider an international economy as a system of commodity im-
porting - exporting nations who react to international prices. In such
an economy a competitive equilibrium can be defined as a system of
prices such that imports do not exceed exports for any commodity. We
take into consideration the full set of commodities the countries trade
in, thus following a general equilibrium approach as opposed to a part-
ial one. For each nation imports - exports are determined by a national
model. The possible structure of such a model will not be discussed in
this paper (cf. [6]).All that matters here is that the relation between
imports - exports and international prices is assumed to have the fol-
lowing characteristics, which closely follow the requirements set out
in Arrow and Hahn Ch. i [I].
(a) A national model is considered as a set of net demand functions of
world market prices. The functions are continuous for positive world
market prices.
For country h,h = I,...,L one has:
h h (pW,kh) z = z , defined for pW ~ 0 (i.i)
W p n-dimensional vector of world market prices h
z n-dimensional vector of net demand by nation h
(net demand = demand - supply)
k h deficit on the balance of trade of nation h.
(b) National net demand is assumed to satisfy for all nonnegative world
market prices the following balance of trade equation:
pWzh = kh(pw ) 1 (1.2)
is the nation's deficit on the balance of trade; kh(pw) is con- k h
tinuous and homogeneous of degree one in pW. The deficits on the
balances of trade are distributed over countries in such a way that:
Z k h = 0 at all pW ~ 0 (1.3) h
(c) Let d h be the nations vector of demand and yh the vector of supply
then by definition:
h d h h z = - y (1.4)
We assume that:
dh> 0 and yh> 0
There exists a finite number c such that for i=l,..,n: y~ < o C.
i Price vectors are row vectors. Quantity vectors are column vectors.
wh . p y + k h > 0 at all pW > 0
(d) Some of world net demands become infinite as any price drops to-
wards zero.
w = 0 for some i (i 5) lim Z 7. z h = + ~ , where Poi w+w i
P Po i h
(e) National net demand is homogeneous of degree zero in world market
price and balance of trade deficits:
zh(pW,kh) = zh(IpW,lkh ) ,16R V ~ > 0,p w > 0 (1.6)
Therefore world market prices can without loss of generality be
constrained to :
{ w w > 0, i = 1 .... n} (1 7) pW I ~ p = i, pi = . S = i
Given characteristics (a)-(e) and by following standard arguments (see
e.g. Arrow and Hahn Ch. 1 [I]) a competitive equilibrium can be shown
to exist on the world market, that is a price vector such that world
net demand is nonpositive:
pW" = {pW I z < 0, z = 7~ zh(pW), pW6s}
h
This implies that at prices p w~ a feasible alldcation exists; the equi-
librium can be nonunique.
The model can be said to depict a competitive equilibrium because all
countries take world market prices as parameters for their decisions.
They may well base their behaviour on some anticipated impact on world
market prices, but no international agreements are set up by which par-
ticipants agree on target levels for certain variables and on instru-
ments to reach these targets.
The model will be extended to cover such an agreement but first a dis-
tinction must be made between internal and external agreements.
In an external agreement a group of countries agrees to influence the
state of the rest of the world, while in an internal agreement targets
are formulated which do not directly affect other countries although
there may be an indirect impact. A cartel typically is an external
agreement while a customs union or a bilateral trade agreement are in-
ternal agreements. Internal agreements can be modelled without changing
the basic structure of the competitive model. The countries with the
agreement can be seen as a group which operates as a unit on the world
market, facing world market prices and balance of trade restrictions
just as a country does.
In external agreements the countries making the agreement explicitly
try to influence the value of the parameters they face on the world
market i.c. the world market prices. We present an example of such an
agreement, other examples have been given in [7].
2. AN INTERNATIONAL BUFFER STOCK AGREEMENT WITH A GIVEN PRICE TARGET
2.1 Principles
A group of countries agrees to devote part of the supplies
(i.e. makes commitments) to having stock adjustment prevail over price
adjustment on the world market, as long as stock levels permit to do so.
An international agency is created which announces that it will buy and
sell unlimited amounts of commodities as long as its stocks permit, at
a given target price. The model now must be set up so that equilibrium
prices exist which are such that:
- world net demand, including stock adjustments is nonpositive;
- exogenously specified constraints on stocks are satisfied;
- equilibrium prices only deviate from target prices for commodities
which have effective constraints on stocks and deviate upwards in
case of shortage and downwards in case of surplus;
- the commitment by the nations to the agency is met.
2.2 The Model
2.2.1 List of symbols
Except when stated otherwise all symbols refer to n- th
dimensional real vectors. The i element of such a vector refers to the • th 1
b h
d h
d -w P pW
s
t h t
u
-min u -max u
-o u -h Y
commodity.
share of country h in income transfer to agency (scalar)
demand by country h
total demand
unscaled target level of world market price (parameter)
realized level of world market price
excess supply on world market
income transfer from country h to the agency
total income transfer to the agency
actual level of buffer stock
minimum level of buffer stock
maximum level of buffer stock
initial level of buffer stock
supply by country h
(parameter)
(parameter)
(parameter)
(parameter)
Y h
Z
Z
P
total supply (parameter)
net demand by country h
total net demand
share of wealth committed to agency (scalar,parameter)
scaling factor on ~w (scalar)
upward deviation from scaled target price
downward deviation from scaled target price
Supplies are taken as parameters just for the sake of exposition.
Exogenous variables are not distinguished from parameters and indicated
with a bar.
Endogenous variables which are not explicitly generated by an equation
in the model, are indicated with greek letters (i.e. p,~,9). We call
them adjustment variables.
Price vectors are row, quantitity vectors are column vectors.
(a)
2.2.2 The equations
Net demand functions country h
h z = zh(pW,th ) (net demand country h) (2.1)
which satisfies for all pW ~ 0, t h ~ pWgh
pW.zh = -t h (2.2)
and has the characteristics (a)-(e) listed in para i.
(b) Distribution of transfer to agency over countries
t h = b h . t
bh = b h ( 9 1 , . . . . 9L,pW)
which satisfies:
- ~ bh= I h
_ bh (~1 . . . . ~L,pW) = b h ( ~ l , . . , ~ L , x p w ) , 1 6 R V X > O,p w => 0
(c) Aggregation tO world level
z=Zz h h
= ~ ~h
(d) Policy equations of the agency
u = -(z + s)+ ~c
~W(u° + ~- u) £ o
-min -max U < U < U
(2.3)
(2.4)
(2.5)
(2.6)
(actual stock) (2.7)
(commitment) (2.8)
(bounds on actual stock) (2.9)
(e)
(f)
t = u~min _ w~max w-o + p(Sw(~° + ~9)) _ p u
(transfer) (2.10)
Market equilibrium conditions
s i = max (0, -( u i + z i - ))
pW.s = 0
w P = p~W+ ~ - ~
(u - ~min) = 0
~(u - ~max) = 0
(free of disposal excess supply) (2.11)
(price definition) (2.12)
(complementar i ty relations on prices) (2.13)
p(~w(~o + ~9) _ ~Wu) = 0
pW, p,~,~ > 0
~(~w + ~i + ~i ) = 1 i
(complementarity rela- tion on commitments) (2.14)
(normalization) (2.15)
Assumptions on parameters
0 < u min < ~max < ~ + ~o (2.16)
~w~min < ~w(~o + ~) < ~w~max (2.17)
2.2.3 Remark8
(a) In case of an equality in (2.17) all stock levels are fixed by
(2.17), (2.9) and the problem reduces to a standard competitive
equilibrium.
(b) The initial stock (~o) can be thought of as a carryover from the
previous period. In this dynamic interpretation all the stocks are
physically held by the agency and ~.~w~ is a new commitment which
is used to buy up new stocks for the agency. The parameter ~ may
therefore become zero once sufficient stocks have been built up.
However, when ~ is zero,buffer stocks cannot be used to keep prices
away from long run competitive equilibrium levels, as stock limits
will be reached within a few periods.
(c) Equation (2.10) tells us that the countries commit themselves to
make available for stock holding purposes a certain amount of
wealth measured at target price (~w(~o + ~)). The agency is al-
lowed to deviate from this commitment only to finance losses and
profits due to price deviations. Note that the commitment is a
scalar. The model therefore does not imply that a stock needs to
be carried over in kind although this is one way to interpret it.
(d) Equation (2.8) tells us that whatever prices are realized, the
final value of stocks, measured at unscaled target prices, will
not be less than the commitment and eq. (2.14) adds that it will
be equal to it when measured at scaled target prices.
(e) Equations (2.12), (2.13), and (2.14) show that price realization
may deviate from price target but only according to prespecified
rules.
(f) The left hand side inequality in (2.17) implies together with
(2.8) and (2.13) that the equilibrium value of p must be strictly
positive.
2.3 Alternative Interpretations of the Model
The straightforward interpretation of the model as describing
a buffer stock agreement between countries was already given above in
an United Nations-type of context. We call this an altruistic inter-
pretation. A slightly different interpretation, however, suggest that
a cartel is formed which operates the buffer stock, assuming that other
nonparticipating countries remain price takers. The distribution of
transfers over countries then determines the type of cartel. If disjoint
sets of countries support different commodities one has commodity spec-
ific cartels. This, is a monopolistic interpretation.
A third interpretation is that prices have an inherent rigidity; the
price target is just last period's price. The rigidity is then part of
the market itself; one could call this a structural interpretation.
These three interpretations: altruistic, monopolistic and structural
can be given to several models of external agreements (cf. [7]).
2.4 A Simple Extension of the Model
In the version described above all targets have been taken
as given parameters. Considering them as prespecified functions of the
adjustment variables, does however not alter the basic structure of the
model. These functions have to be continuous and homogeneous of degree
zero in adjustment variables. It is for example not necessary for each
commodity to possess a predetermined price target ~w. The target may be
flexible or even absent. The latter case is represented by adding the
equation:
~l = p /0 for prespecified i (2.18)
The own price policy for that commodity is now fully flexible and the
corresponding buffer stocks operates in order to reach targets on other
commodities.
3. ECONOMIC EQUILIBRIUM AS A COMPLEMENTARITY PROBLEM
3.1 Complementarity problems and target planning
Following Cottle [2] we define as a complementarity problem
a model of the form:
q = q (~)
~.q = 0 > 0 (3.1)
q < 0
where
q and ~ are vectors in R m.
q is a column vector, ~ a row vector of adjustment variables.
Examples of complementarity problems can be found in [2] or in
Takayama and Judge [13]. Bimatrix games and Kuhn-Tucker optimality con-
ditions fall within this class. As will presently be shown, both the
competitive model and the model with buffer stocks can be seen as mem-
bers of this class. Typically, in an economic context ~ will indicate
some valuation, while q will measure quantities and the equation
~.q = 0 will be some formulation of the fact that value of expenditures
must equal the value of receipts (the so-called strong Walras' Law).
It is of some interest to formulate a model as a complementarity prob-
lem because this permits the planner to model both target and reali-
zation levels of variables and to explicitly formulate conditions under
which realizations may deviate from targets. The proof that the model
has a solution then becomes the proof that the plan is feasible within
the model and the computation of this solution then yields the values
of the policy instruments consistent with the plan (= agreement).
Let
x be the vector of realized values
the vector of target values
the upward deviation of realizations from target level.
Then we write:
x = x + ~ (3.2)
Let q = q(x(~)) describe the impact of x on certain variables.
As long as these variables (~) are strictly negative, the planner wants
to see his targets realized, but when qi = 0 for some i, an upward de-
viation is tolerated:
~i qi = 0 , ~i > 0 , qi < 0
In other words the target must be reached as long as the constraint as-
10
sociated to it is unbinding. The complementarity approach thus provides
a sort of language to model policies. Applications have been found not
only at the international but also at the national level (e.g. when
national government imposes bounds on imports and exports, cf. [6]).
3.2 Competitive equilibrium as a complementarity problem
Complementarity problems yield a framework for modelling
economic policies because the basic model, the competitive one can be
seen as such a problem, so that advantage can be taken of the rather
wide experience gained both in proving existence of a solution and in
computation of such a solution for the competitive model. The equiva-
lence can be seen by defining (cf. §i) :
qi = zi (world net demand) (3.3)
= p~ (world price) (3.4)
The competitive model can then be written as eq. (3.1) with the ad-
ditional restrictions2that
q(~) is continuous for ~ ~ 0
q(~) = q(l.~), IER, Vh > 0, ~ ~ 0
~.q = 0
Eq. (3.5) and (3.6)
where
= {~]~ => 0
v@ Z o
permit to introduce the restriction:
n
~i =i} i=l
(3.5)
(3.6)
(3.7)
(3.8)
3.3 Equilibrium with Buffer Stock8 as a Complementarity Problem
The free disposal equations (eq. 2.11) are disregarded below
due to characteristic (d) in 51. The model with buffer stocks reduces
to a complementarity problem if we define:
-w ~y u) cf. eq. (2.8) ql = p (~o + _ ]
-min
J q2 = u - u cf. eq. (2.9)
-max q3 = u - u cf. eq. (2.9)
(3.9)
~b I = p ] ~2 = ~ cf. eq. (2.12) ] ¢3 = ~
(3.~o)
2 Disregarding characteristics (c), and (d) in 51.
11
ql
I q2 > , ~ = (~i, ~2, ~3 ) (3.11) q = q3
The model with buffer stocks will now fit within the formulation (3.1),
(3.5)-(3.7) with the additional requirement:
where
= {~I~ > O, ~(~i ~w. + ~2i + ~3i ) = i, l
(3.12)
~i ~w + ~2 - ~3 ~ O}
3.4 Existence of Equilibrium
The proof of existence for the model with buffer stocks will
not be given as it is rather lengthy and has already been given else-
where [7]. The appendix to this paper describes its main principles.
First the complementarity problem with its additional restrictions is
transformed into a fixed point problem by defining an appropriate map-
ping. Second, the existence of the fixed point for that mapping is es-
tablished on the basis of Kakutani's Fixed Point Theorem. Third it has
to be shown that the fixed point is indeed a solution of the comple-
mentarity problem, i.e. an equilibrium. (In the proof the free dispo-
sal equations (2.12) are again taken into consideration (cf. appendix,
footnote 5)).
4. COMPUTATION OF EQUILIBRIUM BY NONSMOOTH OPTIMIZATION
4.1 The Extended Complementarity Problem
AS can be seen from equation (3.11) in the model with buffer
stock, q,~ are vectors in R 2n+l. Before describing an algorithm we now
proceed to the reduction of the dimensions of the problem by transform-
ing the complementarity problem (3.1) into an extended complementarity
problem. The latter has the structure
q = q(8)
< 0 (4.1) q =
>0
$.q = 0
If q,~ are again vectors in R 2n+l, 8 is a vector in R m with (hopefully)
m < 2n+l, 8 is now the vector of adjustment variables.
Define 8 6 Rn+l:
12
@n+l = P (cf. (2.12)) ]
5i = pW J (4.2)
From the fact that upper and lower bounds on stocks can impossibly be
effective at the same time (eq. (2.16)) follows that we can substitute~
¢1 = 6n+l --W
~2i = max ((@i - 8n+l Pi )' 0) i = l,...,n
-w ¢3i = max (-(8 i - en+l Pi )' 0) i = l,...,n
¢ = (~I' ¢2" ¢3 )
In the extended complementarity formulation the models satisfies the
additional restrictions @ > 0
q(9) ,~(@) are continuous functions V@ ~ 0
q(e) = q(~.e), ~£R, Vk > 0, e ~ 0
l.~(O) = ~(l.O), IER, V1 > 0, 8 ~ 0
~ . q = o vo ~o
Due to these restrictions we can formulate a very simple restriction
on O:
o £ 8
where
n+l 0 = {61@ ~ O, ~ 6 = i} (4.5)
i=l 1
(4.3)
( 4 . 4 )
The reformulation thus has permitted us both to reduce dimensionality
from R2n+ito R n+l and to simplify the structure of the constraint set
(3.12) vs (4.5). Observe that the gradient of ¢(6) is not unique at
all points.
4.2 Nonsmooth Optimization
The functions q(e) and ¢(e) are nonlinear, so that solving
(4.1-4.5) amounts to coping with a nonlinear system. To solve this
problem an optimization technique was selected which operates by
finding iteratively a sequence of adjustment variables (8) which aims
at reducing the value of the largest element of the vector q until it
is below an acceptable level~.
Alternative techniques are available. A simplex altorithm of the type
described by Scarf [12] could be used. It has sure convergence to an
13
equilibrium, a property which the computation through optimization does
not exhibit, but it usually involves a high computational cost. Dixon
[4] and Ginsburgh-Waelbroeck [5] have developed an efficient alter-
native, without sure convergence.
The idea when using an optimization approach is to choose a nonnegative
goal function f(8), which measures how far 8 is from an equilibrium and
which is zero if and only if 0 is an equilibrium. Then the problem is
reduced to:
min f(@)
860
where 0 is defined as in (4.5).
(4.6)
Since we can deduce from characteristic (d), §i that
lim f(8) = + = where 0oi = 0 for some i, i = l,...,n 0+0
o
and since it follows from remark 2.2.3f that the equilibrium value of
8n+ 1 is strictly positive, the constraint 0 ~ 0 can be eliminated and
replaced by the addition to f(0) of the barrier term C/0n+ 1 (c is a
small positive number).
n+l To eliminate the constraints Z 8. = i, we consider the restriction of
i=l l f to the set {8 I Z8 i = i} this restricted function has a gradient whose
components gi sum up to 0 and are given by:
~f(@) 1 n+l ~f(8) gi ~@i ~ j~l ~0j i = 1 ..... n+l (4.7)
It remains to specify the goal function. The values of the elements of
q should be mutually comparable, we scale them with constant ~. and l
define (other choices are possible):
f(8) = max (qi(0)/ei) + c (4.8) i 8n+l
We now make two differentiability assumptions on q(8) :
. For any 0 C @ such that 0 > 0 there exists a sequence 8 k 60,
8k+8 such that the gradients Vqi(8) exist and are bounded.
The function q(0) is semismooth (cf. [i0].
Once the functions q(8) are assumed to be continuous, the additional
differentiability assumptions are not much more restrictive and from
an application point of view only violated by strange functions such
as:
1 q(8) = 02 sin ~ , for 86 R
14
The goal function f(8) now has a gradient only when @ is such that there
is exactly one i such that qi(8)/£ i is maximum and the corresponding
gradient Vqi(8) exists. Otherwise we speak of a generaZized gradient.
The computation of f and its generalized gradient is described by:
(I) Check that all 8 i are positive. Compute the values of qi(@) and
their generalized gradients.
(2) Determine some index i such that qi/Ei is maximum; this gives f
and a (generalized) gradient g.
(3) Substract from g the restriction term corresponding to (4.7).
To minimize f(8) we apply some method for nonsmooth optimization des-
cribed in [10] ~. Every such method is based on the usual principle of
descent methods [9], in which a direction d of incrementation of @ is
computed from the current iterate and a line-search is performed in this
direction, hopefully yielding a stepsize t such that f(@+td) > f(8).
The common characteristic of these methods is that the direction is
computed through the solution of a quadratic programming problem in-
volving the gradients accumulated during the previous iterations.
Although no sure convergence to an equilibrium can be garanteed, at
least under some classical hypotheses convergence has been shown in [8],
where results of numerical experiments in calculating a competitive
equilibrium with different national models and different goal functions
have been reported. As the outcomes of experiments performed since
then with the model with buffer stocks are entirely in line with the
previous results, they will not be repeated here. ~ Cases where the al-
gorithm failed to converge have not been encountered as yet.
An empirical application of the model is at present under way in the
Food and Agriculture Programme of the International Institute for Applied
Systems Analysis (IIASA) where national models, are being designed. The
Centre for World Food Studies in the Netherlands participates in this
effort. The national models all satisfy the requirements listed in §i
but have different structures and are independently developed by coun-
try experts. The international model performs the linkage between na-
tional models under several types of international agreements.
3 The nonsmooth optimization software which we use was developed and made available to us by C. Lemar~chal from IRIA.
4 Just to give an idea: from an arbitrary start, a model with 5 commodities converges to an equilibrium within I °/oo of the size of the supplies in about 20 iterations.
15
APPENDIX: Conditions for the existence of equilibrium.
This appendix develops a set of conditions inherent in several economic
equilibrium problems and establishes the existence of an equilibrium
under these conditions. The existence proof follows standard arguments
(see e.g. Debreu [3]).
Consider the following four conditions.
(i) There is a vector-valued (uppersemi) continuous mapping ~ + q(~)
which satisfies equality ~.q = 0 (or more generally the inequ-
ality ~.q ~ 0).
(ii) The domain ~ is a compact convex set in R n
(iii) There is a convex subset of ~, #(q) 5 with the property that
~.q(~) ~ 0 for all ~ 6 #(q,~)) implies that q~ 0
(iv) The range Q is a compact convex set in R n 6
Given conditions (i) through (iv) there exists at least one ~ refer-
red to as an equilibrium such that q* 6 q(~) and q~ ~ 0.
The existence of such a ~ can be verified as follows:
Define the mapping q + p(q) as:
P(q) = {~Im ax V-q, ~ 6 ~(q), q £ Q}
Themapping q(~) is (uppersemi) continuous and p(q) is uppersemicon-
tinuous by the continuity of the maximization operator (see e.g.
Debreu [3] or Nikaldo [ii]. .
Consider the cartesian product ~ x Q with elements (~,q) and let F be
the symetric mapping of R 2n which accomplishes the interchange F(~,q) =
(q,~). Since F is linear and nonsingular the mapping (~,q)÷F(q(~),p(q))
is also uppersemicontinuous and maps the compact convex set ~ x Q into
a subset of itself.
Using the Kakutani fixed point theorem there exists at least one pair
(~,q~) £ F(q(~),p(q~)). Thus, since ~ £ p(q~) and q~ 6 q(~) we have
by construction of p(q) that for all ~ £ ~(q(~)), ~q~ ~ ~q~ ~ 0
Also by construction of the set ~(q) (see condition (iii)) the condi-
tion ~q(~) ~ 0 for all ~ 6 #(q) implies that q~ ~ 0. The pair (~,q*)
is therefore the desired equilibrium solution.
In the competitive model (q) = . In the model with buffer stocks:
~(q) = {~I~ 6 ~,(~i~ w + ~2 - ~3 ) .s(q) = 0}
where s is the excess supply defined in eq. (2.11).
If Q is not a compact convex set one adds a restriction which makes it compact con- vex and which subsequently can be shown to be unbinding in equilibrium (ef. Debreu[3]).
16
BIBLIOGRAPHY
[i] ARROW, K.J. and F.H. Hahn, General Competitive Analysis, Holden Day, Edinburgh, 1971.
[2] COTTLE, R.M., Complementarity and Variational Problems, Technical Report SOL 74-6, Stanford 1974.
[3] DEBREU, G., Theory of Value: An Axiomatic Analysis of Economic Equilibrium, Cowles Foundation Monograph hr. 17, Yale, 1959.
[4] DIXON, P., The Theory of Joint Optimization, North Holland, Amsterdam, 1975.
[5] GINSBURGH, V. and J. Waelbroeck, A General Equilibrium Model of World Trade, Part I: Full Format Computation of Economic Equilibria, Cowles Foundation discussion paper nr. 412, Yale, 1975.
[6] KEYZER, M.A., Analysis of a National Model with Domestic Price Policies and Quota on International Trade, IIASA, RM 77-19, Laxenburg, Austria, 1977.
[7] KEYZER, M.A., International Agreements in Models of Barter Ex- change, IIASA, RM 77-51, Laxenburg, Austria, 1977.
[8] KEYZER, M.A., C. Lemar~chal and R. Mifflin, Computing Economic Equilibria Through Nonsmooth Optimization, IIASA, RM 78-13, Laxenburg, Austria, 1978.
[9] LUENBERGER, D.G., Introduction to Linear and Nonlinear Program- ming, Addison - Wesley, 1973.
[i0] MIFFLIN, R., An Algorithm for Constrained Optimization with Semi- smooth Functions, Mathematics of Operations Research, ~, 1977.
[ii] NIKAIDO, H., Convex Structures and Economic Theory, Academic Press, New York, 1968.
[12] SCARF, H., The Computation of Economic Equilibria, Cowles Foun- dation Monograph nr. 24, Yale, 1973.
[13] TAKAYAMA, T. and G. Judge, Spatial and Temporal Price and Allo- cation Mode~s,North Holland, Amsterdam, 1971.
SOLVING NONLINEAR ECONOMIC PLANNING MODELS USING
Leon Lasdon University of Texas at Austin
and
Alexander Meeraus Development Research Center
World Bank, Washington, D. C.
GRG ALGORIT~
Intmoduoi~E.on
With the concept of optimization being such a central theme in economic
theory, planning economists have always been attracted to mathematical prograrmming
as it provides a natural framework of analysis for theoretical and applied work.
Although, nonlinear optimizing economic planning models have been specified and
studied for more than 20 years [4, 17] few were ever solved on a routine basis.
This stands in sharp contrast to linear programming models which, due to highly
reliable (commercial) software are the workhorses of many planning agencies. Even
today, most large-scale nonlinear planning models are solved with specifically
designed algorithms, tailored around some high performance LP system [9, 14, 16].
The enormous technical, as well as commercial difficulties associated
with NLP-software are well understood and need no emphasis. However, there is an
additional aspect peculiar to planning models, which is not always appreciated.
In a planning environment, the role of the model is often extended beyond its
traditional use of obtaining numerical solutions to well defined problems. Models
may be used to define the problem itself and help focus on issues. Also, percep-
tion and abstraction of complex economic phenomena and therefore models, change
continuously as the researcher or planner learns more about uncertain real-world
problems and their structures. No definite answers are expected and models are
employed as guides in planning and decision-making, or serve as moderators between
groups of people with conflicting knowledge and/or interests. Usually a system of
many loosely connected models of different types need to be developed and few, if
any, are used on a routine basis.
18
The present paper reports on preliminary experiments with two general
purpose General Reduced Gradient (GRG) codes in such an environment. The flrst
system, GRG2, is designed for small to medium-slzed problems. It has been deve-
loped and extensively tested by Lasdon and Associates [ii]. The second system,
MINOS/GRG, is an extension of Murtagh and Saunder's [13] original code to nonlinear
constraints. It is designed for large sparse, "mostly linear" problems. Three
classes of models were used to evaluate the two systems in a production environment.
Over i000 different models were solved successfully during a period of 2 months.
General characteristics of the test problems are presented in section i, algorithmic
features are discussed in section 2, and a general evaluation of computational
results is given in section 3.
19
i. Problem description
The two GRG codes were applied to three families of models: Chenery,
PROLOG and YULGOK. All models were originally solved by special purpose algorithms
designed around particular problem structures. Detailed specification of these
models can be found in the stated references, and thus, only the nonlinear features
are discussed in this section. Computer readable representations can be made
available upon request.
i. 1 Chenez~,
The Chenery class of models are based on the Chenery-Uzawa model [4], which
is quite possibly the first country-wlde, nonlinear programming model. It has been
extended by Chenery and Raduchel [5] to include CES (constant elasticity of
substitution) production functions for capital and labor and a more satisfactory
set of demand relations. Recent extensions by Chenery include endogenlzed capital
and labor coefficients as well as price ratios.
The main focus is on methodological questions relating to the role of
substitution in general equilibrium planning models. The model is highly nonlinear
with some rather complex formulations. Some examples are the definition of labor
and capital coefficients:
_ 8--oi/(l+Oi~ I/pi
Ki = ~il i + (i - ~i) ~(1-~i)/~i> i
where K i is a capital coefficient, ~ is the factor price ratio (PK/PL). The
constants Ci ~ ~i and Pi are parameters of the underlying CES production
function.
Simpler forms relate to demand equations such as,
o pi)Oi Qi = q i (~
where Qi is the final demand for commodity i, Pi is the price and , X a price o
deflator. The parameters~ Qi and @i are, respectively the initial conditions
and the price elasticities of final demand.
Typical problem sizes are:
20
Size of Typical Cheuery Problems
Type ..........
Free Fixed Other To tal
equations
variables
linear i ii 4 16
nonlinear 26 17 2 45
total 27 28 6 61
linear 0 0 6 6
nonlinear 0 2 33 35
total 0 2 39 41
1.2 PPoZo~
The PROLOG models [14,15] belong to a family of competitive equi-
librium models that can be solved by mathematical programming techniques. They
were designed to aid practical country economic analysis and are accordingly
flexible in structure and rich in detail. Realistic models in this family are very
large (I000 equations) with approximately 1/3 of the equations containing nonlinear
forms and are solved recursively over time. Special purpose software had been
developed to linearize these models automatically around initial solution values
and to generate piecewlse linearlzatlons in order to apply standard LP systems to
obtain solutions.
A two sector version, containing only the most salient features, has
been used extensively to study methodological questions. Typical nonlinear forms
arise from demand functions such as:
qij Qi = Ai Hj Pi Y '
where Q, P and Y are quantities, prices and income of the ith good, respectively,
nij the cross price elasticities, 81 the income elasticity, and A i is an
appropriate scaling value. Other nonlinearities result from differing income
definitions as follows:
Y = (Z Pi Qi )2 / Z p2 • i Qi '
i z
21
using variable budget shares and "real" prices. Here, the numcraire good is the
bundle of goods which generated GDP in the base year, and the price of that bundle
is used to deflate other prices. Typical sizes of these minI-PROLOGS were as
follows:
Size of Typical Mini-PROLOG Problems
......... Type ........... Free Fixed Other Total
equations
variables
linear 2 0 5 7
nonlinear 3 3 0 6
total 5 3 5 13
linear 0 0 3 3
nonlinear 2 0 3 5
total 2 0 6 8
1.3 Yulgok
The third family of models, YULGOK [i0] belongs to a "real" planning
exercise and is currently in active use by in the planning agency of an Asian country.
Essentially, this model is a standard 53-sector input-output, multi-period linear
programming model, which maximizes aggregate consumption. The nonlinearities are
introduced by the additional requirements of smooth exponential growth of output,
investment and national income. These flow requirements are needed to solve the
"terminal condition problem" which is characteristlcof finite time horizon models.
Unless special assumptions are made about the formation of terminal capital
stock, the model would not invest for post-termlnal consumption. Knowing the
functional form of the trajectories of most variables we can "integrate" the model
and solve Just for the terminal period subject to nonlinear integrallty constraints.
A typical example is the capacity constraint:
where ~,i
IT,i
T -Z
t=o [(l+si)T-t / k i] Io, i ( IT,i/lo,i )t/T ~ (I-8i)T Xo, i ,
- gross output from sector i at period T,
= fixed investments in sector i at period T, are endogenous.
22
The time index o represents the initial conditions and T the terminal
period. The depreciation rate of capital stock in sector i is denoted by ~i '
and the capital-output ratio for sector i by k i .
Since the model is linear for a given set of growth rates it was origi-
nally solved by an iterative technique this generates a sequence of LP models
which "converge" after 20 to 30 iterations. Sizes of typical YULGOK models by
level of aggregation are given in the following table:
Size of YULGOK Problems
.......... Type ......... Total
Free Fixed Other Total for n = 53
equations
variables
linear 1 1 2n + 3 2n + 5 iii
nonlinear 0 n n + 1 2n + I 107
total i n + i 3n + 4 4n + 6 218
linear 2n 0 2n + 2 4n + 2 214
nonlinear 0 0 2n + 2 2n + 2 108
total 2n 0 4n + 4 6n + 4 322
n is the number of sectors.
2. GRG Algorithms and Software.
2.1 GRG Algorithms
There are many possible GRG algorithms. The reader is assumed to be
familiar with their underlying concepts, see [i , ii]. This section briefly
describes the version currently implemented in GRG2 and MINOS/GRG. A more complete
description is found in [12] .
Consider the nonlinear program
minimize f(x) (2.1)
23
subject to
g(x) = 0 (2,2)
< x < u (2.3)
where x is a vector of n variables and the objective f and vector of m
constraint functions g are assumed dlfferentlable. At the start of some GRG
iteration let x be a vector satisfying (2), and J(x) be the Jacobian
matrix of g evaluated at x . This is assumed to be of full rank, a condition
which can always be satisfied by including a full set of logical or slack
variables in the vector x . J(x) and x are partitioned as
x = (x l, x 2, x 3) (2.4)
J (~) = (B I, B 2, B 3) (2.5)
where BI is nonslngular, Xl is the vector of basic variables, x2 the
superbasic variables, and x3 the nonbasic. The variables x2 are strictly
within their bounds, while the components of x3 are all at bounds.
Since BI is nonsingular, the equalities (2), may be solved (in some
neighborhood of x) for x I as a function of x 2 and x 3 . Then f may be
viewed as a function of x 2 and x 3 alone. This function is called the reduced
objective F(x2,x3). Its gradient at ~ is computed as follows:
a) Solve B1 T H = 8f/~x I (2.6)
b) ~F/~x i = ~f/Sx i - H Bi ' i = 2, 3 (2.7)
where all partial derivatives are evaluated at x . The derivatives BF/Sx 3
are only used to determine if some component in x 3 should join the superbaslc
set. Then (letting x 2 represent the possibly augmented set of superbasic
variables), ~F/~x 2 is used to form a search direction d. Both conjugate gradient
and variable metric methods have been used to determine d.
24
problem
A one dimensional search is now initiated, whose goal is to solve the
minimize F(x 2 + e d, x 3) (2.8)
This minimization is done only approximately~ and is accomplished by
choosing a sequence of positive values {el, s2,...} for e. For each e 1
dl,X 3) F(x 2 + e i must be evaluated, so the basic variables must be determined.
These satisfy the system of equations
g (Xl' ~2 + el ~' E3 ) = 0 (2.9)
This system is solved by a pseudo-Newton iteration:
k+l k k x I = x I + 6 , k = 0,i, ... (2.10)
k k th approximation to the solution of where x I is the
(2.9) and the Newton correction 6k satisfies
k ~2 + ei d , (2.11) Bl 6k = _ g (x I , x3 )
Note that the same B1 is used for each e i
The initial estimate x~ in (2.10) is computed either by linear or
quadratic extrapolation. The linear extrapolation uses the tangent vector v,
defined by
B1 v = B 2 d (2.12)
This is the vector of directional derivatives of the basic variables along the
direction ~ evaluated at x . At each el, x~ is computed as
(x~) i = (Xl)i_ 1 + e i v (2.13)
where (Xl) i is the vector of values of x I which satisfy (2.9). The quadratic
extrapolation procedure is described in [12].
25
In the case of nonlinear constraints, the one dimensional search can
terminate in three different ways. First, Newton's method may not converge. If
this occurs on the first step, el is reduced and we try again. Otherwise, the
search is terminated. Second, if the Newton method converges, some basic variables
may be in violation of their bounds. Then the codes discussed here determine a new
value such that at least one such variable is at its bound and all others are
within their bounds. If, at this new point, the objective is less than at all
previous points, the one dimensional search is terminated. A new set of basic
variables is determined and solution of a new reduced problem begins. Finally, the
search may continue until an objective value is found which is larger than the
previous value. Then a quadratic is fitted to the three =i values bracketing the
minimum. F is evaluated at the minimum of this quadratic, and the search
terminates with the "lowest F values found. The reduced problem remains the same.
2.2 The GRG$ Software S~stem
2.2.? ~nput~ Ease of Use~ a~d Output Features
GRG2 is designed to solve small to medium-slze NLP's. Unlike MINOS/GRG,
it makes no attempt to exploit sparslty or any partial linearity that is present
in the problem. This permits problem input to be quite simple. Only one sub-
routine need be provided by the user, which computes the values of the problem
functions f and g. Derivatives may be computed using a system finite differen-
cing option, or the user may code them in a separate routine. All other problem
data, e.g., problem size, bounds, initial values for Xl, tolerances, print levels,
choice of alternative methods, etc., is specified in an input file. All
quantities in this file, except problem size have default values. Most users will
specify only problem size, upper and lower bounds, and a starting point, leaving
the system to specify all internal parameters and options.
GRG2 is composed of approximately 4000 Fortran statements. Both GRG2
and MINOS/GRG have dynamic storage allocation. This means that each code contains
only a few arrays that must be dimensioned by the user, depending on problem
size. All working and data arrays are stored in these. This permits solution of
small problems using only the storage needed, while placing no limit on the size of
the problem that can be accommodated except for the amount of core available.
Another important ease of use feature is the presence of a revise capability. This
26
permits specification of a sequence of problems to be solved in a single input
file. The problems must have the same size and the same f and g functions,
but may differ in any other respect, e.g., bound values. ~ence, constraints may
be relaxed or tightened, or the objective may be changed. Such variations are
often important during In-depth study of the properties of a model.
2.2.2. Algorithmic Feature8
Both GRG2 and MINOS/GRG have a choice of methods for computing the search
direction d in (2.8). If the number of superbasic variables is less than a
user supplied value (default value n), d is computed using a variable metric
algorithm [6]. Otherwise, one of several Conjugate Gradient methods is used. The
variable metric method updates an approximation to 82F/Sx~ - rather than its
inverse. Following Murtsgh and Saunders [13], this approximate Hessian is main-
tained in factorized form as R T R, where R is an upper triangular matrix.
The matrlx R is updated using elementary orthogonal matrices.
In GRG2, equations (2.6), (2.11), and (2.12) (all involving the basic
matrix Bl' as the coefficient matrix) are solved by computing B1 -I explicitly.
In fact, only a nonsingular submatrix of B1 corresponding to the active constraints
is inverted. The inverse is recomputed at the start of each one dimensional search.
2.3 Tile MINOS/GRG Software S~stem
form
MINOS/GRG is designed to solve large sparse nonlinear programs of the
minimize x k (2.14)
subject to
A x + F(y) = h (2.15)
£ ~ (x,y) ~ u (2.16)
where A is an m x n matrix, x is an nl-veetor of variables which appear
linearly, F is an m-vector of functions (any of which may be nonlinear), and y
is an n2-vecto r of variables each of which appears nonlinearly in at least one
2?
equation of (2.15). These are called nonlinear variables. The matrix A contains
a full identity submatrlx corresponding to the logical variables, and x k is the
logical variable associated with the objective row. The program is designed
specifically for problems where m and/or (n I + n2) is large (greater than, say,
100) and where A is sparse. In such problems, it will often be the case that
relatively few constraints are nonlinear (i.e. F has many components which are
identically zero or are linear) and/or n 2 is much less than n I. Such problems
are called "mostly linear". Large NLP's which are direct descendants of LP
models will usually be of this type.
2.3.1. Input; Ease of Use~ and OutT~ut Features
MINOS/GRG input formats are compatible with industry standard MPS formats
for linear programming. Nonconstant elements in the Jacoblan matrix are specified
by a special character in the columns section. This reserves a position in the
matrix file for the element, whose value is computed as it is needed. The existence
of a nonlinear function (component of F(y)) in a particular row is indicated by
a special RHS set, which gives the row name and the index of the function.
Starting values for the nonlinear variables y are specified by a special BOUNDS
set, giving the name of each variable and its initial value. The vector F(y) is
computed in a user-provided subroutine FCOMP, which allows each component of F
to be accessed individually. As in GRG2, partial derivatives of F may be
computed by finite differencing or by user-coded formulas in FCOMP.
Solution output of MINOS/GRG is in MPS format. The code has the dynamic
storage allocation features described in section (2.2.1). It has about 8500 Fortran
statements.
2.3.2. Alqo~%hrr~ Eeatumes
Much of the data storage in MINOS/GRG is used for the Jacobian matrix
(i.e., A and the Jacoblan of F in (2.15)). This is stored in packed format
(only nonzeros stored) in column order. Equations (2.6), (2.11), and (2.12) are
solved by factorizing B 1 as
B 1 = LU
28
A version of Hellerman and Rarick's"bump and spike" algorithm p4 [8] is used to
insure that L and U inherit the sparsity of B 1 .
For large problems, reinversion of B I after each line search (as in
GRG2) is too t~me consuming, but if B 1 contains several nonlinear columns all of
these may change. MINOS/GRG simply regards each nonlinear column which has changed
as having its previous value replaced by the new one, and applies Saunders implemen-
tation of the Bartels-Golub LU update [17] to each such column. This update is
ideal since it is very stable, yet the rate of growth of the L and U files is
quite small. In problems solved thus far up to 400 updates are made before the basis
is refactorized.
The procedures for computing search directions and for performing the line-
search in MINOS/GRG are very similar to those in GRG2. The methods for adding
variables to the superbasic set differ. In GRG2, all nonbasic columns are priced
(i.e., their reduced gradient components are computed) prior to each llnesearch,
and a nonbasic is made superbaslc if its reduced gradient is of the proper sign and
is large enough (in absolute value) relative to the reduced gradient of the super-
basics. In order to deal with large problems, it is desirable not to price all
variables at each iteration. MINOS/GRG achieves this by minimizing over the
existing set of superbasics until a convergence criterion is met. Then the non-
basics are priced and the one with the most promising reduced gradient component
enters the superbaslc set.
3. Application of GHG Software
The previously described codes were used to solve over i000 different
versions of the Chenery, PROLOG and YULGOK type models. The Chenery and mini-
PROLOG models were solved on a production basis using GRG2, whereas MINOS/GRG was
tested on PROLOG and YULGOK. The users took the lazy man's attitude, using only
default parameters and carinK little about good starting values. The PROLOG
models were always started from systemsupplled initial points and one set of
initial values was sufficient to stay within reasonable computing costs for the
Chenery models. However, heavy use was made of the REVISE capability of GRG2,
which resulted in a very low solution cost of the revised models.
The overall performance of GRG2 was excellent, it solved all problems
successfully. Initial difficulties were encountered with domain violations, i.e.,
the system tried to evaluate some functional forms (logs, powers) with arguments
29
outside their domain of definition.
The performance of MINOS/GRG, which is still being developed, was mixed.
Only a few problems could be solved without specifying close initial values. With
some coaching, however, all selected PROLOG and YULGOK models were solved correctly.
The absence of revise and restart facilit'-s, however, made work with large models
difficult.
All jobs were run on a CDC Cyber-73 under NOS/BE and the FORTRAN extended
(FTN) compiler was used. The total computer cost of debugging and running was
under $1,500.
3.1 Computational Results
In addition to the general problem statistics from Section i, the numbers
of non-zero Jacobian elements are given below ( n is again the number of sectors).
Non-Zero Jacobian Elements
(excluding slacks)
Problem
Chenery
Pro log
Yulgok
n=5
n = 53
Constant
85
20
4n 2 + 18n + 7
197
12189
Variable
233
25
Total
318
45
S i z e
61 x 41
13x8
2n+2
12
108
4n 2 x 20n + 9
209
12297
(4n+6) x (6n+4)
26 x 34
218 x 322
30
Unfortunately, the codes were not instrumented with routines to gather
and store solution statistics automatically and we had to select a small number of
"representative" runs for further analysis. To report on test results of complex
optimization systems will always raise questions. Detailed results and evaluations
derived from carefully prepared tests on standard problems may be of little use in
predicting the performance of a system in a particular user's environment. On the
other hand, reports from the user's point of view usually tell more about the
strength and weaknesses of the user than the system under study.
In all cases derivatives were evaluated using finite differences. With
continuously changing and relatively small models it was more cost-effectlve to pay
the higher price for numerical differentiation than to spend time in coding and
debugging of separate routines. No numerical problems were encountered with the
numerical approximation routlnes~ except in combination with domain problems.
All models have a very small feasible space with no obvious feasible
solution values. Also models are often used to trace out Isoquants of some key
variables or an efficiency frontiers to indicate tradeoffs in possible policy
decisions, which restricts the feasible space even further. Usually all nonlinear
variables were in the optimal basis and the number of superbasics did not exceed
three.
In 95% of the runs termination was triggered by satisfaction of the Kuhn-
Tucker conditions within 10 -4 . The few runs terminating on the total fractional
change of objective criteria were scrutinized carefully for errors in formulation
or data.
The number of iterations to reach optimal solution differ widely with
the type and number of binding nonlinear equations. Typical results for GRG2
iterations required in multiples of the number of equations are:
Chenery PROLOG YULGOK 61 x 41 13 x 8 26 x 34
Cold start
Typical 1 2 failed
max 5 12 -
After revise or user initiali- zation
Typical < 1 1 5
max 2 3
31
3.2 Remaining Problems
A number of problems remain to be solved to reach the reliability and
convenience we are accustomed t o from LP-systems.
One main difference between linear and nonlinear systems of algebraic
equations is the domain of definition. Linear equations are always defined for all
points of the Eucledian space E in which they are contalned~ while nonlinear
equations are sometimes defined only on (possibly uncomlected) subsets of E. We
encountered this problem in all three classes of models. The introduction of
additional bounds on some intermediate variables was not sufficient since bounds
are never strictly enforced throughout the solution process. Tests were required
in the function evaluation routines to reset variables when domain violations
occurred, which in turn lead to discontinuities and premature termination because
of variables being trapped at their domain-bound.
Additions to both codes are under implementation which will permit
specification of "hard" boundaries that cannot be crossed at any stage of the
algorithm. An automatic approach to deal with domain problems has been suggested
by Bisschop and Meeraus [2].
Presently, efficient restart facilities need to be adapted to handle
nonlinear constraints and revise facilities in MINOS/GRG have yet to be developed.
Techniques developed for LP systems depend heavily on a model representation in
matrix form and cannot be directly extended to general nonlinear systems. This
raises the question of what are efficient representations of nonlinear models in terms
of data structures required for algorithms, data management and execution control.
Finally, new modelling languages [3, 7] and systems are being developed
which will permit the user to be completely oblivious of internal data and control
structures. Models will be written in a language that stays as close as possible
to existing algebraic conventions without concern for the underlying solution
process which will provide numerical answers.
-o0o-
32
References
i.
2.
3.
4.
5.
6.
7.
8.
9.
10.
II.
12.
13.
Abadle, J., and J. Carpentler 1969, "Generalization of the Wolf Reduced Gradient Method to the Case of Nonlinear Constraints", in R, Fletcher (ed.), Optimization, Academic Press
Bisschop, J., and A. Meeraus, 1978, "Domain Analysis and Exact po~L=- derivative generation for large nonlinear systems", Technical Note No.7 - 671-58, DRC, World Bank (mimeo)
Bisschop, J., and A. Meeraus, 1977 "General Algebraic Modeling System, DRC, World Bank, (mlmeo)
Chenery, H.B., and H. Uzawa, 1958, "Non-Linear Programming in Economic Development", in K.J. Arrow, L. Hurwitz, and H. Uzawa (eds.), Studies in Linear and Non-Linear Programming, Harvard University Press
Chenery, H.B., and W.J. Raduchel, 1971, "Substitution in Planning Models", in Hollls B. Chenery (ed.), Studies in Development PlanninK, Stanford University Press
Fletcher, R., (1970) "A new Approach to Variable Metric Algorithms", Computer Journal, 13
Fourer, R., and M. Harrison, "A Modern Approach to Computer Systems for Linear Progra=ming", M.I.T. (mimeo)
Hellerman, E., and D. Rarlek, (1972) "The Partitioned Preassigned Pivot Procedure" in D. Rose and R. Willoughby, eds., Sparse Matrices and their Applications, Plenum Press, New York
Hoffman, K., and D. Jorgenson, 1977 "Economic and Technological Models for Evaluation of Energy Policy", The Bell Journal of Economics, Vol.8, No.2
Inman, R., K.¥. Hyung, and R. Norton, (forthcoming) "A Multi-Sectoral Model with Endogenous Terminal Conditions", Journal of Development Economics
Lasdon, L.S., A.D. Warren, A. Jaln and M. Ratner, 1978 "Design and Testing of a General Reduced Gradient Code for Nonlinear Programming", ACM Trans. Math. Software 4, i, pp.34-50
Lasdon, L. and A. Warren, 1977 "General Reduced Gradient Software for Linearly and Nonllnearly Constrained Problems, Working Paper 77-85, University of Texas
Murtagh, B. and M. Saunders, 1978 "Large-scale linearly constrained optim/zatlon", Mathematical Progra~in~, 14
14. Norton, R. (et.al.), 1977 "A PROLOG Model for Korea", DRC, World Bank (mimeo)
33
15.
16.
17.
18.
Norton R., and P. Scandlzzo, 1978 "The Computation of Economic Equilibrium: Some Special Cases", DRC, World Bank
PIES - Project Independence Evaluation System Documentation, (1976) VoI. I-XV, NTIS, U.S. Department of Commerce, (mJmeo)
Samuelson, P.A., and R. Solow, 1956 "A Complete Capital Model Involving Heterogenous Capital Goods", quarterly Journal of Economics, Nov. 1956, 70, pp.537-562
Saunders, M. (1976) "A Test, Stable Implementation of the Simplex Method Using Bartels-Golub Updating" in: D.R. Bunch and D.J. Rose, eds. Sparse Matrix Computations, Academic Press, New York.
SPECIFICATION AND ESTIMATION OFE(~IC MODELS
WITH GENERIZ~DE~ATICNS
Kent D. Wall
University of Virginia
Charlottesville, Virginia 22901, U. S. A.
Abstract
Construction of econometric models containing unobserved vari-
ables has presented the econometrician with difficult proble~ns
because contemporary methodology of these variables de, rends
that they not be ignored, accepted practice has sought to find
proxies for them in terms of only observed variables. It is
argued that this substitution procedure is unnecessary. By
introducing the generalized expectations model representation,
explicit treatment of unobserved variables is perndttedo This
new representation is seen to contain the various econometric
proxies as special eases, l~/rthermore, the generalized expec-
tations representation yields a type of nonlinear state-space
model which may be estimated using the techniques already exis-
rant in the control theory literature.
INTRODUCTION
The phenomena of expectation formation lies at the heart of much conten~oorary
theory in the social sciences where an attempt is made to explain the actions of
individuals by decision-making under uncertainty. This is especially true in
economics where optimal decisions over time require information about the future
behavior of certain variables. For example, consider investment and consumption:
The neo-classical theory of investment expresses investment as a function of the
desired level of capital which, in turn, is dependent on expected future behavior
of price anddemand (see Jorgenson[1963], Jorgenson and Siebert[1968], and
Bischoff[1971]). Similarly, const~ption is usually expressed as a function of
permanent income (see Friedman[1957]) or the income stream over the "life cycle"
(see Ando & Modigliani[1963]). In either case the concept of the present value
of an individual's expected future incc~e stream is used.
35
The importance of expectation formation in economics, particularly m~cro-
econcmics, goes beyond the modeling problem; it directly i~ginges upon the theory
of econcnlic policy. Indeed, it threatens to disamntel the entire conceptual
framework for optimal policy formulation. More specifically, by hypothesizing
that all expectations are formed "rationally", economic theorists have shown in
certain special cases that the presence of these "rational expectations" tend to
totally frustrate all attempts to control the economy (see Sargent & Wallace[1975],
Kydland & Prescott[1977]). Moreover, the presence of "rational expectations" casts
doubt upon all traditional methods of econometric policy analysis by suggesting
that the model structure, estimated over past data, will change in the future due
to the application of new policies (see Lucas[1976]).
Given the status of expectations in economics the econometrician has felt com-
pelled to incorporate them in his/her statistical models. He/she immediately, however,
is confronted with a very difficult problem since expectations do not appear
anywhere in his list of observed data--they are, in fact, unobserved variables:
When it is realized that standard econometric modeling methodology only deals
with observed data, the problem can only be solved by hypothesizing some repre-
sentation of the expectational variable in terms of only observed variables.
This approach has led to the use of extrapolative representations (Goodwin[1947]),
adaptive expectation representations (Nerlove[1958]), and finally rational expec-
tations representations. In each case a very specific set of asstmptions (a
priori information) as to the expectation formation process is imposed to obtain a
model involving only observed variables.
This practice, now well established in econometric methodology, yields a
model which fits nicely into standard econometric practice, but at a high price
in terms of flexibility. Once the assumed expectation process is i~0osed on the
structure specification, all alternative specifications are automatically excluded.
Moreover, it prohibits the consideration of "partially rational" e~pectations
or "partially adaptive" e~q0ectations, i.e. the same model specification cannot
incorporate cc~binations of expectation formation processes.
Explicit incorporation of unobserved variables in econometric models is
necessary to obtain a general and flexible representation of economic phenomena.
Such an approach is, however, relatively foreign to the econometrician and requires
the use of certain concepts found in the control theory literature. More specifi-
cally, a combination of optimal state-space estimation theory, the innovations
representation of the optimal filter, sad the traditional econometric model yield
a representation ideally suited to empirical modeling of economic processes.
It is the purpose of this paper to elucidate this claim.
38
THE LINEAR E C O ~ I C MCDEL
The explicit inaorporation of unobserved variables in econometric models is
best presented in terms of the linear model. This facilitates the exposition with-
out limiting the utility of the result, since, as will be discussed below, the pre-
sence of nonlinear el6ments does not invalidate the result. Therefore, consider
the linear econometric model with expectational variables: ^
A(L)y t + B y t + C(L)z t = e t (1)
where Z t i s an Zxl v e c t o r c o n t a i n i n g a l l t h e endogenous (dependent ) v a r i a b l e s , _z t
i s an my/ v e c t o r c o n t a i n i n g a l l o f t h e exogenous ( i ndependen t ) v a r i a b l e s i n c l u d i n g ^
policy instrtm*~nts, and Yt denotes the expectation of Yt conditioned on a prescribed
set of information available at time t-l. The additive term e t is an £xl vector of
sequentially independent, identically distributed normal random variables with zero
moan. The dynamics of the model are embodied in the two polynomial matrix operators
A(L) and C(L) where
A L) -- % + AIL + 2 + ... ÷ ALP
C(L) = C O + CIL + C2L2 + . . . + Cr Lr
and L i s t h e backward s h i f t o p e r a t o r ; i . e . , Lkxev = Xt-k" Each c o e f f i c i e n t m a t r i x i n
A(L) and C(L) i s r e a l and d imens ioned £ x / and £xm, r e s p e c t i v e l y . The l e a d i n g m a t r i x
A 0 i s u s u a l l y c a l l e d t h e s t r u c t u r e m a t r i x i n s t a t i c models and i s always assuned
i n v e r t i b l e .
Al though t h i s model has been t r e a t e d e x t e n s i v e l y i n t h e l i t e r a t u r e , i t i s i n -
s t r u c t i v e t o r e v i e w how (1) i s u s u a l l y t r a n s f o r m e d in r e sponse t o t h e p r e s e n c e o f
Y t m t h e unobserved ( v e c t o r ) v a r i a b l e . Th i s w i l l i l l u m i n a t e t h e problems a s s o c i a t e d
w i t h contemporary e c o n o m e t r i c approaches t o e x p e c t a t i o n a l v a r i a b l e s , and p r o v i d e
the motivation for development of the "explicit" approach of the next section.
By restricting the conditioning information employed in forming Yt to just past
observed values of Yt' the econometrician can replace the unobserved variables with
observed variables. This is the traditional approach and results in a specification
which fits nicely into standard econometric methodology. Thus if the conditioning
information set is defined by
it_l = {yt-1 zt-1}
where yT denotes the set of all observed outputs Yt for -~<t<T (and likewise for Z T) ^
then Yt maY be replaced by S S .
= Z D.y . = X_?jLJy t (2) xt j=l ~t-3 j
if an extrapolative expectation forn~tion is hypothesized, or replaced by
37
£t = j__z0 DJyt-j-1 = j=OZ DJLJy t_l (3)
if an adaptive expectation formation is hypothesized. The infinite
geometric lag distribution (3) can either be truncated or replaced by
a rational lag operator (Kuh ~ Schmalensee [19731)to obtain a finite degree poly-
nomial operator. Truncation yields (2) where s is rather large. The rational appro-
ximation yields,
Yt = N-I(L)M(L)Y--t (4)
where N(L) is always chosen invertible.
Substitution of either (2) or (4) into (I) produces what may be called the
traditional distributed lag econometric model
A(L )z t + C(L t :
where _A(L) and C(L ) are now modified from A(L) and C(L) according to whether (2) or
(4) is employed. The disturbance term E t is just e t in case (2) is employed, or
_¢t--~(n)et if (4) is used.
The case of "rational expectations" is considerably different in its impact on
the specification of the traditional model, and represents a quantt~n jtm~ in com-
plexity. Here the information set is expanded to incl~de all information publicly
available at time t-l~ including the prevailing economic theory and all its mani-
festations, i.e. the econometric model itself'. Thus when
it_l = {yt-1, zt-l, and (1)}
^
Zt can only be replaced by the predictions of the model. This amounts to the
following manipulations: Solve (1) for Yt in terms of lagged Yt and the various
zt, i.e. obtain the reduced form representation
-I P BoY t^ _ = -A 0 {i__ZlAiYt_l + + C(L)z t} + Ao-le t . y t
take the expectat ion of y t condit ioned on I t _ l , not ing that E { Y t _ l l I t _ l } = ~ t _ l Next,
for i>l and
E{Y t l I t _ 1 } = E{E{Yt I I t_ l } I I t _ 1 } = Yt (6)
(see Shiller [1975] for a proof of (6) using martingale theory). If St denotes the
conditional expectation of z t given It_l, then the result:
^
Yt = HI(L)Yt + II2Zt + 113(L)~t
^
can be solved for Yt" This gives,
38
t (I-H2)-l(~l(L)Y_t + ~_3(L)z_t) + (I-H2)-IH~t (7)
Insertion of (7) into (1) produces the structural form model specification under
rational expectation formation. After collecting terms in Zt, and z t this becomes a
form similar to that obtained via extrapolative or adaptive expectations:
+ --C(L)--zt + ~ t = et (8) A(L)~ t
The exact definitions of A(L), C(L), and ~ follow easily from the substitution and
are not given. The only dissimilarity between this case and the other expectation ^
for~r~tion processes is the presence of zt, the rational expectation of z t conditioned
on It_ I. Thus one unobserved variable has been traded for another: The econcmetrician
is now faced with a model consistent with the rational expectation hypothesis, but at ^
the cost of additional modeling to construct z t. Note (8) cannot be estimated until
a representation has been found for z t.
In each case above the dynamic structure of the model is valid only if the
assigned expectation fora~tion process does indeed hold in reality. If the expectation
formation process is not entirely of one type then the model will be misspecified.
~hlrthermore, the rational expectations hypothesis does not eliminate the need to de- ^
velop models of unobserved variables; e.g., the presence of the z t term in (8). The
rational expectations hypothesis, however, does net provide any infon~ation as to
how the exogenous variable expectations are formed. Finally, none of the econometric
approaches provide a framework for investigating how the economy converges to the
hypothesized expectation formation process; i.e., there is no way to examine the
learning process.
GENERALIZED EXPECTATIONS
The problems associated with the contemporary economic approach stem from an
attempt to use asstm~0tions in place of a priori information so that enough structure
can be imposed to permit elimination of the unobserved variables in favor of the
observed variables. It is argued here that this need not be the ease if expectations
are treated explicitly, since there is then no need (a priori) to e}gplain the expec-
tation forrration process. The investigation of how expectations are formed can then
be left to the modeling stage and verified with en~0irical data.
Explicit incorporation of unobserved variables is, however, critically dependent
on whether or not there exists a suitable representation for these variables which
is suited' to the econometric problem. In addition, it is vital that a representa-
tion be amenable to enpirical testing and estimation. Luckily such a representation
can be found in the state-space model and the innovations representation of the
Kalman filter. Since this combination has the power, under certain circumstances, to
39
describe extrapolative, adaptive and rational expectations, it is called the
generalized expectation model. The rest of this section is devoted to its
development.
Expectation Properties. In general, expectation formation on the part of in-
telligent individuals should possess the ability to learn from the past. In addi-
tion, expectation fonration should contain "rmmT~ry" or a tendency to refrain from
rapid fluctuations in predictions--at least so long as the predictions remain fairly
accurate. The first property indicates that outside or other infonT~tion should enter
into expectation. This might include recent forecasting errors as well as recent, or
past, values of all observed variables which the individual feels relevant to making
the prediction. The second property suggests a dependence on previous predictions.
The expected value of a variable should not be too dissimilar from its previous ex-
pected value, especially if this previous prediction was fairly accurate and no
drastic changes in the environment are foreseen. An example of this latter case
would be predictions of inflation rates during a period (one to three years) of
relative economic stability. If forecasts of, say, 4~ inflation have proved rela-
tively accurate over the past, there will be little incentive to change from this
forecast over the next few months.
Generalized Expectation Models. The properties can be encompassed by a simple
difference equation model for each expectational variable in the model. Hence let
xit denote the conditional expectation of an element in Yt" Then in general
P ,
xit = filXit_l + ... + finiXit_ni + Y f k=_l~k --Xt-k
P , + ~ _g lkZt_k k=l
r
+ Z g'2kZt_k k=l
where ni>_O and fini#O.
(9)
The vectors f--k' glk and K2 k represent the dependence of the
expectation xit on other expectations, observed endogenous variables and observed
exogenous variables, respectively. To avoid duplicate effects the rows of ~ corre-
sponding to lagged xit are assumed zero.
Rewriting (9) in vector-matrix notation reveals a form very similar to that
studied in the control theory literature:
--xt = ~--t-I + GlUt + G~ t + G~ t ( I0)
40
where
F
FII F21 .-. FII
F12 F22 " ' " F21 • o • •
F£1 Fl2 . . - FZZ
n x n
l l
fil fi2 "'" fin.
1 0 ... 0
0 1 ... 0 • : ".
0 0 . . ' 1 0
n i x n i
F.. = 13
fijl fij2 fijn
? o o
o 8 o
n. xn. i 3
w' t [Y' t-l' t-2 .... Yt-p ]
X' t = [~'t_l,~'t_ 2 ..... ~'t_v 1.
GI, G2, and G 3 are real nx/p, nxn~, and n~m matrices respectively.
with (I) gives the complete model for ~eneralized expectations"
x t = :P/:C_I + G_ut
Combining (i0)
(lla)
where F is as before, G = [G 1
xt-- ~ t + ~ t + ~t
O 2 G3], u ' t = [w' t v ' t z't],
(11b)
41
and
H = -A0-1B0[E 1 E 2 E 3 ...... E£] (12)
D =-A0-1[A 1 A 2 ... % C 1 ... C r C O ] (13)
E. is an Axn. matrix with all elements zero except for the (i,l) element which is 1 1
unity. This matrix is required to "pick off" the correct element in x t to be used
as the expectation for Yit" Thus the generalized expectations model resembles
nothing more than a standard state-space model~ The states are, as is often the
case in control theory, unobserved and correspond to the optimal forecasts of the
endogenous variables. The output equations correspond to nothing more than the
original econometric model with the e~0ectations appearing explicitly.
RELATIONSHIPS TO EXPECTATION HYPOTHESES
The heuristic development of the generalized expectations econometric model,
presented above, while plausible, remains a rather hollow concept without further
substantiation. Therefore, consider the specification of an econometric model
under the three CC~T~nly encounter expectation formation hypotheses.
Extrapolative Expectations. If expectations are believed formed from only the ^
finite past history of the endogenous variables then Yt is formed from a finite length
distributed lag of past observed Yt" This stipulates an expectation process as
in (2). The "state" vector in (lla) is therefore given by x_t=Zt=D(L)Yt with
D(L)=DIL+D~L2+"z .+Ds Ls" In this case (lla) reduces to
_x t = O . x t _ 1 + [G 1 0 0 ] _v t
z t
w h e r e G 1 = [D 1 D 2 . . . D S 0 . . . 0] f o r s < p . I f s = p t h e n a l l o f w t e n t e r s i n t o
t h e f o r m a t i o n o f x t . I f s > p t h e n w t c a n b e s u i t a b l e l e n g t h e n e d t o i n c l u d e more
l a g g e d v a l u e s o f Y t w i t h o u t a l t e r i n g t h e b a s i c f o r m o f ( 1 1 ) . I n t h i s s i t u a t i o n n o t e
that there really is no concept of memory or "state" since F is the zero matrix: The
extrapolative e~oectations model is a pathological dynamic system because (lla) may
be directly substituted into (llb) and thereby removed frcm further consideration.
Adaptive Expectations. Hypothesizing (4) in place of (2) immediately yields (Ii).
To see this, assume M(L) and N(L) are defined by M(L)=MIL and N(L)=I+NIL with
NI=-(I-M/). This is the most typical case encountered in the economies literature.
By defining the state in (lla) to be identical to Zt , the generalized e~pectations
model is inmediately apparent:
42
-~t=~-t-I +[GI o 0] v t
with F~-I-M 1 and GI=[M ~ 0 ... 0]. F in this instance is always assumed non-singular
(~ is usually diagonal with the nonzero diagonal element less than unity) and already
in block cc~panion form~ Even though adaptive expectations usually appear in the
form above (i.e., containing only first order autoregressive terms in yt) the
generalized expectations formulation readily admits higher order adaptive schemes.
In addition, the generalized expectations formulation allows for current and lagged
z t to enter the process.
Rational Expectations. The hypothesis of rational expectations yields an ^
expression for Yt given by (7). This easily fits the generalized formulation by
defining a "state" vector which is composed of two distinct subvectors, i.e. ~t and
~t" Thus with
E 't I
_x t = F x t _ 1 + [G 1 G 2 0] v_t
where
F = ; FI2 0 ;22
10 :]r ] G = "" "" Hlp
All AI2 .... A 1
F22 is some nonsingular matrix, expressed in block companion form which determines
the evolution of ~t with respect to itself. The r~trixes Aij and A2j relate the ^
expectation of z t to endogenous and exogenous variables over the past.
43
Together with (I), the rational expectations hypothesis provides specific
infonm%tion concerning the specification of the first block row in F and G; it
does nothing to help specify the second block row in these matrices. While this
appears to put the generalized framework at some loss in structuring F22 and the
All, it is no worse off than the econometric treatment in this respect. Same
advantage is achieved, however, by "internalizing" the modeling of ~t since now it J
is done together with the other elements. In this sense, the generalized expectations
model carries the internal consistency argument (see Shiller [1975]) one step further
by simultaneously incorporating the exogenous variable expectation process along
with the endogenous expectation process. This internalization of the exogenous
variable expectation fo~Tnation is, of course, only necessary if the Aij~0. This
has been ass~ned a priori in the econ~netric literature. The attractiveness of (n)
is that it leads to a well posed estimation problem where Aij=O may be tested.
In s~n, it has been shown how (ii) incorporates several popular expectation
hypotheses as special cases, while at the same time retaining the ability to describe
other more general situations. It therefore provides a natural framework for the
estimation of econometric models where e~0ectations play a major role, and therefore
can be used to test various hypotheses. Furthermore, once such a model has been
constructed it yields a valid vehicle for policy analysis since the expectations pro-
cess has been explicitly incorporated. Such models cxmpletely answer the recent
criticisms leveled at economic control methodology by Lucas, Kydland & Prescott, and
others.
ESTIMATION
Estimation of (II) presents a formidible task. Not only are there the usual
difficulties associated with estimation of the econometric model (lib), there are
additional difficulties in estimating the state space model (lla) since little is
usually known of its order (dimension n) or whether or not certain elements of G
are to be fixed at zero.
In general a nonlinear estimation problem arises since elements of F, G, H and
D must be estimated along with the state. Indeed, if all the unknown elements of
F, G, H and D are collected in a vector ~ and adjoined to _x t (Ii) becomes
it = f(Ot_l, ut) (12a)
Zt = _b(0_t , ut) + At (12b)
where e_' t = [x' t ~' ]. There are nZ unknov~ el~T~nts, in general in F and
n(Zp+Zr+l) unkno~rn coefficients in G. H w i l l only have at most 12 unknowns, while
44
D may have as many as pZ2+f/ur+Zm. Without any prior information, ~t will be a
vector with at most N = nZ • n(Ip+Zrel) = Z 2 + pZ 2 + from + £m elements~
The estimation of _e t would appear practically impossible to the casual ob-
server. There are, however, several factors which take the estimation problem
posed by (12) and bring it into the realm of tractability. First, although £
might be large for an econometric model, the nm%ber of expectational variables
is ofien much less than i. If there are g<<£ expectational (or, generally
speaking, unobserved) variables then ~t can be replaced with a vector of much
s~ller din~nsion. This reduces n at the same time since fewer con~oanion blocks
are required in F. Second, economic theory can provide much prior information to
help reduce the unknown coefficients in (llb). Often there are many zeroes in A(L)
and C(L) to such an extent that we obtain over-identification of the reduced form
equations, (llb), thus permitting many of the coefficients to be set to zero
a priori. Finally, eooncmic theory can also aid in the specification of unknowns
for G in (lla). For example, lagged investment variables~would not be expected
to influence expectations of future prices, whereas general measures of the rate
of change of excess demand might have definite bearing. If monetary influences
are felt to he strong then perhaps GI=0 while G 2 and G 3 would be nonzero except
for the coefficients concerned with the money supply instr~nents.
Given that it is reasonable to expect a nmnageable n~nber of unkncmn para-
meters, the question arises as to how the estimates are to be obtained. It would
appear to the econometrician that estimation of 0 t subject to (12) is a completely
new problem for which there is no known solution. On the other hand, the formu-
lation of the econometric problem in terms of (Ii) or (12) would be readily
recognized by the control theorist as a nonlinear filtering problem. Solutions
to the nonlinear filtering problem abound in the controls literature, and
suitable algorithms for the estimation problem generated by (12) are too numerous
to mention here: (The books by McGarty[1974], Jaswinski[1970], and Gelb,et.al.
[1974], plus the article by Mehra[1974] should serve as a worthwhile starting
place for the interested reader. ) In short, there is nothing novel or unfathomed
about the nonlinear state estimation problem, and, hence, the generalized ex-
pectation model presents no new modeling problem'.
It should be mentioned in passing, however, that whereas the estimation
problem associated with (12) has been investigated at length, the related
question of the identifiability of 0 t given (12) remains partially unanswered.
Parts of the identifiability question have been answered (for exanple by Tse &
Weinert[ 1975], Cooley & Wall[ 1977] and Tse[ 1978] ), no general result is available
at this time. This is an area for active research in econometrics and control.
45
AN EX~g~E
The generalized expectations representation will be developed for a simple
mecroeconcmic model due to J. B. Taylor (see Taylor[ 1976] ). Although originally
used in a study of optin~l economic control in the presence of rational expectations,
the model provides a useful vehicle for illustrating the representation introduced
in the present work.
There are only two equations in the model; one for aggregate den~nd~
Yt' and another for the inflation rate, Pt" There is one exogenous variable, real
money balances, m t. (In the original model presented by J. B. Taylor, neminal
money balances appeared together with the aggregate price level). Both Yt and m t
are expressed in terms of the natural logarithm of the actual variables. Neglecting
constant terms and time trends, this model becomes
+ B~o t + elt Yt = ~ l Y t - 1 + 82Yt-2 + ¢3mt + 84mr-1
^
Pt = Pt-I + ~lYt + e2t
where Pt and Yt denote the expectations of Pt and Yt' respectively. In the ^ ^
original study of this model Yt and Pt were assured to be rational expectations.
If we ass~ne elt and e2t are not serially correlated, then the model in terms
of (I)be~: Yt' = [Yt' ~t ], _e't = [elt'e2t]' Y't^ = [~t,~t ], z' t = m t and
01 [o o A(L) = " B 0 = ;
I- -~I
The maximL~ lag on Yt is p=2 and the maxirmxn lag on z t is r=l. % is just the
2x2 identity nmtrix so this model is already in reduced fo~n, i.e. there is no
need to left multiply (i) by A0 -I
The generalized expectations model will have a "state" vector subdivided into ^ ^
two "sub-state" vectors, one for the evolution of Yt and the other for Pt" If we
let the memory effects for YtA ̂requlre" n I lagged values of Yt' and those for Pt
require n 2 lagged values of Pt' then F will be dimensioned nxn with n=nl+n 2.
There will be two conpanion matrices of dimension nlxn I and n2xn2, respectively,
down the diagonal of F. Furthermore,
4B
u ' t = [Yt-l' l°t-Z' Yt-2' !5t-2' mt-l ' mr]
H =
[o :]E:0 000 :] ~i 0...01 I O. .
D = 0 1 0 0 0
The "state" equations have the following general form
where n
fil
1
Fii = 0
r i + [Gll
f~2" " "finl-i
0 ...0
1 ,,,0 • o •
- o •
0 °i
i
% I% l !
fin I
0
0
0
F . . =
13 I fijl fij2 ..... fijnjl
i 0 ........ 0
0 .......
and G 1 is any real n×4 n~trix, G 2 is any real n×l matrix, G 3 is any n×l real
matrix.
If a rational expectations hypothesis is to be tested then F and G would he ^
augmented by n 3 rows representing the memory evolution of z t. H would then have
n 3 column added to its righthand end. The first n col~a~s of F would, of co~se,
be zeroed out to agree with the rational expectations restrictions on how Yt and
Pt are conlouted.
47
CONCLUSIONS
While economic theory relies heavily on unobserved, expectational variables,
conten~oorary econometric practice still seeks to remove them from consideration
by various proxy schemes. Most notable among these are the extrapolative,
adaptive, and rational expectations hypotheses. Although such a methodology is
successful in reducing the econometric problem to one with only observed variables,
it in~s severe restrictions on the model structure. Explicit treatment of ex-
pectations ~ppears in order introduced here. This representation contains all
three popular expectations hypotheses as special cases, and so provides a uni-
fying frs]rework. In addition, it transforms the econometric problem into a very
familiar estimation problem in control theory--one for which nL~nerous estimation
algorithms exist. While there are still identifiability questions yet to be
resolved, in general, there is a solid foundation existant in the controls litera-
ture from which a more cor~plete result is yet to ceme. Practical applications of
estimating 0 t subject to (12) are nL~nerous in the applied control theory literature,
but nonexistant in econometrics. It is hoped that this paper will serve to sti-
mulate an active interest in econc~etrics for the generalized expcetations approach,
and state space estimation theory in general.
REFERENCES
Ando, A. and Modigliani, F.(1963), "The 'Life Cycle' Hypothesis of Saving: Aggregate Implications and Tests", Amer. Econ. Review, Vol. 53, no. I, pp.55-84.
Bischoff, C. W., (1971), "The Effect of Alternate Lag Distributions", in G. Fron~ (ed.), Tax Incentives and Capital Spending, Washington, D. C. : Brookings Institution. (Chapter 2)
Cooley, T. F. and Wall, K. D. (1978), "On the Identification of Time-Varvin~ Para- meters", m/meograDh Univ. of California, Santa Barbara.
Friedman, M., (1958), A Theory. of the Consunption Function, Princeton, N. J. : PrinceZon University Press., (Chpts. 1-3, 6, 9).
Geld, A. (ed.), Applied Optimal Estimation, Cambridge, Mass. : MIT Press.
Goodwin, R. M., (1947), '~rn~mlical Coupline with Especial Reference to Markets Having Production Lags", Econometrica, Vol. 15, pp.181-204.
Jazwinski, A. H., (1970), Stochastic Processes and Filtering .%heory, New York: Academic Press. (Chapters 8,9).
Jorgenson, D. W. (1963), "Capital Theory and Investment Behavior", Amer. Econ. Rev., Vol. 53 (May), pp. 247-259.
Jorgenson, D. W., and Siebert, C. D., (1968), "Optimal Capital Acc~m~lation and corporate Investment Behavior", Jour. Political Econ., Vol. 76, (Nov~r/ December).
48
Kydland, F. E. and Prescott, E. C., (1977), "Rules Rather than Discretion: The Inconsistency of Optimal Plans", Jour. Political Economy, Vol. 85, (March) DD. 473-491.
Kuh. E. and Schmalensee, R. (1973), An Introduction to Applied Macroeconomics, Ansterdmn: North-Holland, (Chapter 2).
Lucas, R. E., (1976), "Econometric Policy Evaluation: A Critique", in The Phillips Curve and Labor Markets (K. Brunner and A. H. Meltzer, eds.), Amsterdam: North-Holland.
McCrarty, T. P., (1974), Stochastic Systems and State Estimation,New York: John Wiley & Sons. (Chapter 6).
Mehra, R. K. (1974), "Identification in Control and Econometrics: Similarities and Difference", Annals Econ. Soc. Measurement, Vol.3, no. i, pp. 21-48.
Nerlove, M., "Adaptive Expectations and Cobweb Phenomena". Ouarterlv Jour. Econ.. Vol. 73. (May)~o. 227-240.
Sargent. T. J. and Wallace. N.. (1975). "'Rational' I~mectations. the Optimal Monetary Instr~r~nt. and the Optimal Money Supply Rule". Jour. Political Econ., Vol. 83, (April)pp.241-254.
Shiller, R., (1975), "Rational Expectations and the Dynamic Structure of Macroeconomic Models: A Critical Review", NBER Working Paper No. 93, Computer Research Center, Cambridge, Mass. (June).
Taylor, J. B., (1976), "Estimation and Control of a Macroeconomic Model with Rational Expectations", unpublished paper, Col~nbia University, (August).
Tse, E., (1978), "A Quantitative Measure of Identifiability", IEEE Trans. Systems, Man & Cybernetics. Vol. SMC-8. No. 1 (Januarv~ Do. 1-8.
Tse. E. and Weinert. H. (1975). "Structure Determination and Parameter Identification for Multivariable Stochastic Linear Systems."IEEE Trans. Auto. Control, Vol. AC-20, no. 5, (October), pp.603-612.
IMPLEMENTATION OF THE MODEL IN CODES FOR
CONTROL OF LARGE ECONOMETRIC MODELS
Arne Drud
IMSOR
The Institute of Mathematical Statistics
and Operational research
The Technical University of Denmark
DK-2800 Lyngby, Denmark
i. Introduction.
It has become more and more clear that the description of an optimiza-
tion algorithm must include a description of the data structure that is
applied by the algorithm to store and retrieve the input data and the
intermediate results. E.g., in a large scale nonlinear optimization al-
gorithm we usually describe the representation of the Jacobian of the
constraints as a sparse matrix and we specify how the sparse matrix
computations are performed.
The nonlinear expressions in the model equations are also part of the
input data but usually we don't pay much attention to this part of the
input - we simply enter a subroutine that can compute the expressions
for a given input vector, and sometimes we indicate which variables
enter linearly in each expression. This paper will concentrate on the
question of how to represent model expressions in the computer. Section
2 describes some alternative representations, both the usual subroutine
representation and some numerical or internal representations. The use
of the model expressions in simulation and optimization of large scale
econometric models is discussed in section 3, and based on these con-
siderations suggestions for the possible implementation are given in
section 4. Many computations in optimization algorithms will be based
on partial derivatives so the computation of partial derivatives and
its dependence on the representation of the expression will be discus-
sed in section 5, and some computational experiments will be reported
on in section 6. Finally, section 7 contains a short summary and a
conclusion.
2. Alternative Representations of the Model Expressions.
The mathematical form of the econometric model equations we will con-
sider is either
50
Xit = fi(xt ' Ut' Yt' Xt-l' Ut-l' Yt-l''''' P)' i=l,''',n (i)
or
gi(~t ' ut' Yt' Xt-l' Ut-l' Yt-i '''''p)' i=l,...n (2)
where x t is the vector of endogenous variables in period t, u t are the
control variables, Yt are the exogenous variables, and p is a vector
of fixed parameters. We will in this section discuss alternative ways
of representing the generally nonlinear functions fi or gi in the com-
puter.
The standard method in connection with nonlinear optimization codes is
to write a subroutine that for given values of x t, ut' Yt' Xt-l' Ut-l'
Yt_l,---, p computes fi or gi" Usually this task is not as straight for-
ward as it seems because a real model not will be written in vector
notation as indicated in (i) or (2) but rather with variable names
like GNP, INV, CON etc. However, the transformation from one format to
the other can usually be performed with a simple text editing procedure.
Many computer systems for estimating and simulating econometric models
apply an internal numerical representation of the equations e.g.
through an oriented graph representation of the expressions. The fol-
lowing example shows how TSP (= Time Series Processer, see [5]) will
represent the expression
in(a) • b + c • d + e ~ 2 - a/c
through a graph as shown in fig. 1 or internally coded as
operator
operand 1
operand 2
result
in •
a b c
- R1 d
R1 R2 R3
+ % + / -
R2 e R4 a R6
R3 e R5 c R7
R4 R5 R6 R7 R8
where the different operators are represented by integers, and all va-
riables as well as parameters and "registers", are represented by in-
dices or addresses within one large data area.
Note that the operations can be performed in many different sequences,
51
e.g. also as indicated by the graph in fig. 2 or by the internal repre-
sentation
Figure i: A graph representing the expression in(a)*b + c*d + e**2 - a/c
Figure 2: An alternativ representation of the expression from figure i.
i
operator I
operand 1 I c e
operand 2 I d e
result . R1 R2
I + / - in • + [
I R1 a R3 a R6 R5
R2 c R4 - b R7
R3 R4 R5 R6 R7 R8
The IAS-system (= Inter - Active Simulation System, see [7]) uses a
reverse polish notation, i.e. the expression above may be represented
by the string
a, in, b, ~, c, d, ~, +, e, e, ~, +, a, c, /, -
52
Again the variables and the operators will be represented by integers
and the sign bit can be used to distinguish between an operator and an
address. The reverse polish notation is more compact than the TSP ap-
proach above because the references to intermediate results are impli-
cit through the stack notation.
In the TROLL system (= Time-Shared Reactive On-Line ~aboratory) the
expressions are also represented internally, but before they are used
in computations they are translated into machine code. This gives a much
faster execution, but the code becomes very machine dependent and we will
restrict ourselves to methods that are easily portable. The use of FOR-
TRAN subroutines as input can be seen as a way of using the efficiency
of machine code and at the same time keep the advantages of a portable
code.
3. Computations with the Model Expressions.
We will consider optimization codes based on the general scheme:
I. Compute a direction d for the control variables u.
2. For different steplengths 0 do:
2.1 Set the control variables to u 0 + 0 • d and perform a simulation
with the model.
2.2 Compute the objective function.
3. Choose a good 8-value from step 2, set u = u + 0 • d, and go to i.
If we forget step i for a moment the model expressions are only used
in the simulation step, step 2.1. The simulation is generally composed
of an initialization step where the equations and the variables are
reordered into a recursive sequence of nondecomposable simultaneous
blocks based on the incidence matrix of the model, and an iteration
step where the nondecomposable simultaneous blocks are solved one by
one. The initialization is only computed once for a specific model and
it can be considered as a step 0 in the scheme above.
The simultaneous blocks can be solved one by one either by the Gauss
-Seidel method or by Newtons or a pseudo-Newton method. In Gauss-Sei-
dels method the equations (i) within the simultaneous block are com-
puted iteratively in some prefixed order and the new values of the
left hand side is introduced immediately into the next right hand side.
The number of iterations, i.e. the number depend on the size of the
block and will often range between i0 and 20. Blocks of one equation
will converge in one iteration.
5S
In Newtons method and in related methods all the residuals in the block
are computed via (2) and the vector of residuals is multiplied by some
matrix to yield the changes in the variables in the block. If an equa-
tion is linear in the variables in the block and if the matrix with
which we multiply is properly chosen the equation will be satiesfied in
all subsequent iterations if round-off errors are neglected. Therefore,
the residuals in the linear equations should not be computed again, and
the zeros in the residual vector should be used to simplify the matrix
-vector multiplication. The number of iterations will again depend on
the size of the block, but it will usually range between 3 and 7.
In the overall optimization the total number of T-period simulations
will typically be a small multiple (1-3) of m • T where m is the number
of control variables per period and T is the length of the planning ho-
rizon.
We notice the following characteristics of the iterative procedure:
I. We would like to compute the model expressions in some specified or-
der (Gauss-Seidel) or we would like to compute subsets of the expres-
sions (Newton), but the order or the subsets is not known apriori.
2. Most of the active variables in an expression don't change between i-
terations. Only the endogenous variables from the same block change.
3. The information needed to find the order of the expressions or the
subsets mentioned in i. or the variability of the variables in 2. is
only known when the total model is specified and the initialization
step with the block decomposition has been computed.
4. When one or more equations are changed, deleted, or added the block
decomposition can change and all the information mentionded in 3. can
change.
5. Some optimization algorithms can handle bounds on the endogenous va-
riables by working with a set of "pseudo-endogenous" variables or ba-
sic variables that can change between iterations, see Drud [i]. After
a change in the set of pseudo-endogenous variables the block structure
and the variability of the variables within each equation will change.
Furthermore, the set of pseudo-endogenous variables and the block
structure will often be different from time period to time period.
4. Comments to the Implementation.
TO find the best representation of the model equations we must consider
and balance different kinds of costs: Preprocessing costs to change the
54
model equations from some user-oriented input format to the final repre-
sentation, and the cost of computing the values of the expressions re-
peatedly from this final representation.
Let us first consider a graph representation like the one shown in sec-
tion 2. All nodes can be divided into three groups: leaf nodes or atomic
nodes that represent numerical input data to the expression, the root
node that represents the final result, and the intermediate nodes that
represent intermediate results.
We consider the expression from section 2 and assume that we are itera-
ting on a block where a and b are endogenous variables of the block and
c, d, and e are parameters, exogenous variables, lagged endogenous va-
riables, or endogenous from earlier blocks. In this case the subexpres-
sion c~d + e*~2 should be computed before the iterative loop is entered.
If the second representation in section 2 is applied we just use the
first three operators outside the iterative loop and only the last five
inside the loop.
The example leads us to the concept of the variability of the intermedi-
ate nodes. An intermediate node is called fixed if its value only depends
on the values of the parameters.
It is called predetermined if its value depends on the parameters, exo-
genous variables, lagged endogenous variables, and/or unlagged endoge-
nousvariables from earlier blocks but not on endogenous variables from
the block to which the equation belongs. And a node is called active if
its value depend on endogenous variables from the current block. A node
is called a limiting node if the result of the operation in which it
takes place has a different variability than the node itself. The ope-
rators + and • will only work on two operands in the computer while ma-
thematically they can be applied to a whole set of operands in any order.
Therefore, great care must be taken that the + or • operations are or-
dered such that the fixed and predetermined nodes include as many ope-
rations as possible.
Based on these definitions an efficient simulation procedure could be:
i. Input the model and create a graph representation of the expressions.
2. Find the block structure of the model.
3. Compute the variability of all intermediate nodes and arrange the
representation of the expression such that the computation of the
fixed nodes are first in the representation, then the computation
55
4~
4.1
4 .i.i
4 .i .i.I
of the predetermined nodes and finally the computation of the
result node. Compute the values of all limiting fixed nodes by
applying the first part of the representation of the expressions.
Simulate: For each time period t=l,..-, T do:
For each block in the recursive order do:
Compute the limiting predetermined nodes of the expressions in
this block by applying the second part of the representation of
the expression. The fixed nodes are input.
Iterate until convergence: In each iteration compute the value
of the result node by applying the last part of the representa-
tion of the expression. The fixed nodes and the predetermined
nodes are input.
Different aspects of this general procedure will be discussed in the
following subsections.
4.1 The Time-S~ace_Tradez2~f
There is a trade-off between space to store the values of the intermedi-
ate nodes and the time that is saved by not computing the values repea-
tedly. Limiting fixed nodes are good in an absolute sence because the
time is reduced and at the same time the first part of the code used to
compute the fixed nodes can be deleted after the node values are compu-
ted, which saves more space than the space actually used to store the
node values. The predetermined nodes saves computer time but they re-
quire additional space because no code can be deleted after the node
values are computed.
The nodes that only depend on parameters and exogenous variables could
be defined as an additional set of interesting nodes called fixed pre-
determined nodes, and we could define a corresponding set of limiting
nodes. Of course, these nodes would only be of interest in optimizations
where we would make many simulations. We would have to store the value
of each of the limiting nodes for each time period in the planning ho-
rizon in order to save any time, and in cases with only a few simulations
in the optimization this can be too expensive in space even though we
can delete the first part of the code that was used to compute the values
of the nodes.
4.2 Numerical or FORTRAN re~sen~!2~.
We can keep the representation of the expressions as numerical data e.g.
56
using the graph representation mentioned ealier, or as a set of subrou-
tines in FORTRAN or another high level language. In the first case all
the operations mentioned on the previous pages can be done in one com-
puter program. If we want the same efficiency in the FORTRAN case we
must first run step 1 to 3 as an initialization and based on the results
of the decomposition we can write some FORTRAN subroutines. The subrou-
tines can be written by the computer code. We could write two subroutines
that with a given block number as input would compute either the values
of the limiting predetermined nodes or the values of the expressions of
the block.
The advantage of the FORTRAN approach is that we become a faster machine
code, the disadvantage is that we must break the job into different steps
where one is a FORTRAN compile step. Therefore, the numerical data re-
presentation can be recommended for interactive programs while the FOR-
TRAN approach can be recommended for large scale models and for optimi-
zation problems where the total number of calls of each expression, and
therefore the possible saving, is substantial.
4.3 Flexibilit[ against Changing Models.
When a model is changed, e.g. because an equation is added, deleted,
or changed we must start again just after step 1 and recompute the block
decomposition and the new variability of the intermediate nodes. The
set of limiting nodes for each expression will in general be changed.
Therefore, the setup cost for the changed model is almost as large as
for the original model. However, if we define predetermined nodes as
nodes that depend on parameters, exogenous variables, and lagged endo-
genous variables only, the limiting nodes can be computed for each ex-
pression independently. Actually we can build an equation bank with the
decomposition of each expression, and a model is simply extracted as a
subset of the expressions in the bank. The setup cost of a changed mo-
del will be much smaller while the computation of the expressions with-
in the innermost interative loop will be a little more expensive because
nodes that depend on unlagged endogenous variables from earlier blocks
are computed repeatedly.
The construction depend on a fixed set of endogenous and exogenous vari-
ables. If a variable that earlier was exogenous is made endogenous the
variability must be recomputed for all expressions in which the variable
is active. If the model is changes often we could save different rele-
57
vant representations of the same expression in the equation bank and
extract the one that corresponded to the current definition of exoge-
nous and endogenous variables. Or we could define predetermined nodes
as nodes that only depend on lagged variables and parameters.
The discussion in this subsection is relevant both for internal repre-
sentation and for FORTRAN representation of the expressions.
4.4 Optimization with Pseudo-Endoqenous Variables.
A large class of optimization procedures can be build by inserting the
general scheme for changing the control variables mentioned in section
3 between the three initialization steps and the simulation step. This
class includes the algorithms described by Fair [4] and Mantell & Las-
don [5]. The definition of variability and limiting nodes is as above
when the control variables are considered as exogenous variables.
However, in the reduced gradient algorithms that change the set of ba-
sic or pseudo-endogenous variables when some pseudo-endogenous variables
reach a lower or upper bound, the block-decomposition must be moved in-
side the iterative loops where it must be called each time the set of
basic variables changes. Or we must work on the total model without a
decomposition.
If we still believe in decomposition the most important consequence is
that the block structure need not be the same in all time periods such
that the variability of the intermediate nodes in general will depend
on the time period. The best practical solution will probably be to de-
fine the predetermined nodes as nodes depending on parameters, exogenous
variables and lagged endogenous variables only as was done above. We can
then avoid to restructure the expressions or keep different copies for
different time periods which, any way, only is possible with an inter-
nal representation and not with a FORTRAN representation.
The subsets of the expressions we want to compute in the simulation ite-
rations can also change. This causes no problem when the expressions are
coded internally as data, but if expressions are coded as a FORTRAN
subroutine the subroutine must be coded such that we can compute any
subset of the expressions fast.
58
5. Partial Derivatives.
Partial derivatives Of the model expressions are needed in two steps of
an optimization. In the computation of the search in a reduced gradient
algorithm there is a backward step, where the derivatives of all expres-
sions with respect to all endogenous and control variables, both lagged
and unlagged, must be computed, see [i] or [6]. In the simulation step
we need the derivatives of the expressions in each block with respect
to the variables in the same block if Newtons method is applied, and no
derivatives if Gauss-Seidel is applied.
The partial derivatives can relatively easily be computed ~rom an inter-
nal graph representation of the expressions. There are three possible
approaches:
i. We can use finite differences. In the backward step we will have to
start the computations from the fixed nodes, and in the Newton step
we can start from the predetermined nodes.
2. If we know the derivatives of two nodes with respect to a set of in-
teresting variables we can also find the derivative of a node that
is created from the first two nodes by some differentiable operator.
And since the initial nodes are easy with derivatives either zero or
one we can compute the exact derivatives for each node with respect
to all interesting variables, and we can end with the derivatives of
the total expression with respect to any subset of the input vari-
ables. Again, we can start either from the fixed nodes or from the
predetermined nodes. The disadvantage of the method is that the de-
rivatives will be zero for most combinations of nodes and variables
and great care should be taken to avoid unnecessary computations.
3. We can create a new internal code for the derivatives bases on the
ideas in 2. If we base the computations on some of the intermediate
results in the computation of the expression itself this method can
be extremely fast, but it will use more space than method 2.
In the FORTRAN subroutine case we can only use finite differences where
we still can have the advantages df fixed and predetermined limiting
nodes, or a special FORTRAN subroutine for the derivatives also based on
the intermediate nodes. A more restricted implementation of the deriva-
tive subroutine is described in Drud [2] and [3]
59
6. A simple Computational Experiment.
A simple experiment was performed where the expression from section 2
was computed 10000 times both from a FORTRAN statement and from an in-
ternal representation. In the internal representation the operators
were assigned numbers, and a computed GO TO statement transferred con-
trol to a piece of code where the proper operation was performed on the
proper operands. The experiment was performed on an IBM 370/165 computer
using both a FORTRAN G compiler and a FORTRAN H compiler with OPT=2. The
first experiment gave a factor of 4.0 to the advantage of FORTRAN over
internal code and the second gave a factor of only 2.2.
Although this very simple experiment doesn't tell the whole truth the
surprisingly low factor indicates that the number of calls of an expres-
sion must be rather large before FORTRAN will be cheaper on an overall
basis.
In computations where interactive computation is important or in situa-
tions with a rapidly changing model the internal representation must be
preferred. Even in optimizations where the number of evaluations of the
expressions can be very large the internal representation can be ad-
vantageous, especially if the model is changed often. The small diffe-
rence between FORTRAN and internal code suggests that more research
should be put into the subject of the best representation of an expres-
sion.
7. Conclusions.
The purpose of the paper hasbeen to discuss how the nonlinear expres-
sions in a nonlinear optimization problem and especially in a control
problem based on a nonlinear econometric model should be represented in
the computer. Two basicly different approaches, an internal graph re-
presentation and a FORTRAN subroutine representation have been discussed,
different implementational considerations were described, and the strong
and weak sides of both approaches, especially the computer time and the
flexibility when the problem is changed slightly, were discussed.
Some preliminary computational results indicate, that although the ma-
shine code we get from a FORTRAN subroutine can compute an expression
faster than the internal graph representation, the later has so many ad-
vantages with respect to flexibility that it cannot be ruled out imme-
diately, even based on a total computer time criterion.
80
Many computational experiments with different representations and with
different models and more theoretical research remains to be done before
we can make more accurate recommendations on which method to use under
different circumstances.
8. References.
[i] Drud, A.: "An Optimization Code for Nonlinear Econometric Models Based on Sparse Matrix Techniques and Reduced Gradients", in Annals of Economic and Social Measurements, vol. 6, 1978, pp 563-580.
[2] Drud, A.: "Input Analysing Programs for Econometric Models - User's Guide", OR. PROG. LIB. MANUAL vol. 9, IMSOR 1978.
[3] Drud, A.: "Input Analysing Programs for Econometric Models - Technical Report", IMSOR 1978.
[4] Fair, R.C.: "Methods for Computing Optimal Control Solutions. On the Solution of Optimal Control Problems as Maximization Pro- blems", in Annals of Economic and Social Measurements, vol. 3, 1974 pp 135-153.
[5] Hall, B.H.: "TSP Time Series Processor, Version 3.3, Users Manuas", Massachusetts, May 1977.
[6] Mantell, J.B. & Lasdon, L.S.: " A GRG Algorithm for Econometric Control Problems", in Annals of Economic and Social Measure- ments, vol. 6, 1978, pp 581-597.
[7] Plasser, K.: "Brief description of the IAS-System", Institutarbeit no. 100, Institute for Advanced Studies, Vienna, 1978.
[8] "TROLL: An Introduction and Demonstration", National Bureau of Economic Research, 1974.
MODULECO~ AIDE A LA CONSTRUCTION ETA L'UTILISATION
DE MODELES MACROECONOMIQUES
p° N~pomiastchy (|) , B. Oudet (2) , F. Rechenmann (3)
I. ORIGINE DU PROJET
Le souci de cerner de plus pros la r~alit~ ~conomique a conduit les ~conomis-
tes, aides par les prog=~s de l'informatique, ~ augmenter r~guli~rement la taille
(nombre d'~quations) de leurs modules. Par exemple, le module DMS [I], l'un des plus
r~cents construits par I'INSEE~ comporte |262 ~quations.
Pour des modules de cette taillep il est mat~riellement impossible de g~rer la
tr~s grande quantit~ de donn~es impliqu~es et de d~crire, sans se tromper, les ~qua-
tions du module sans l'aide d'un outil informatique sp~cialis~. On a donc vu apparal-
tre ces derni~res armies sur le march~ international une demi-douzaine de syst~mes
informatiques ayant un double objectif
a) faciliter la construction du module, par exemple en permettant la descrip-
tion des ~quations dans un langage tr~s proche de celui des ~conomistes.
b) faciliter l'utilisation du module par des commandes permettant de g~n~rer
ais~ment des varlantes et la sortie des r~sultats sous une forme lisible.
On est oblig~ de constater qu'R l'exception du syst~me XING~ dont l'usage est
limit~ aux petits modules, tousles syst~mes actuellement utilis~s en France ont ~t~
construits ~ l'~tranger. Ces syst~mes sont lou~s ~ leur constructeur avec impossibili-
t~ pratique, et parfois m~me interdiction, de les modifier. De plus, aucun 8yst~me
existant n'a ~t~ con~u pour fonctionner sur du materiel informatique ~ran~ais. Etre
priv~ de tout syst~me de construction et d'utilisation de modules constltuerait un
retour en arri~re considerable et les administrations ne peuvent pas ne pas en tenir
compte dans leurs choix d'~quipement informatique.
Plus g~n~ralement, il appara~t qu'il n'existe pas actuellement sur le mareh~
international de syst~me r~pondant aux cinq crit~re9 ci-dessous, qui nous paraissent
essentiels :
- transportabilit~ d'un ordinateur ~ un autre et, en particulier, compatibilit~
avec les materiels informatiques franGais;
( ! ) IRIA-LABORIA (2) IMAG (Grenoble) (3) IMAG et IRIA-SESORI
82
- possibilit~ d'adaptation aux besoins sp~ciflques de l'utilisateur, en parti-
culler ~ ses normes internes de stockage des donn~es ~conomiques|
- facilit~ drutilisation pour un non-informatieien, en particulier pour la g~-
n~ration de variantes;
- structure souple du syst~me permettant le remplacement ais~ d'un algorithme
ou module le eomposant par un autre plus r~eent, afin de pouvoir suivra le
progr~s scientifique;
- integration, qui permet le passage ais~ d'une t~che ~ l'autre (de l'estima-
tion ~ la simulation par exemple).
Pour r~pondre aux besoins clairement exprim~s par de nombreux ~conomistes, il
nous a paru opportun de d~velopper un syst~me informatique nouveau avec pour objectif
la satisfaction maximale des critares ci-dessus. Etant bas~ sur les techniques moder-
nes de programmation MODUlaire et ~tant destin~ aux ECOnomistes, ce syst~me a ~t~
baptis~ MODULECO.
II. APPORT DU SYSTEME MODULEC0
Les ~conomistes franGais disposent actuellement de trois logiciels specialists
dans la construction et l'utilisation de modgles macro~conomiques : SIMSYS, l'ensem-
ble XING-APACHE et TROLL. Ces trois logiciels sont ~tudi~s en details dans les annexes
lj II et III du rapport [2]. Nous avons rassembl& dans le tableau ci-apr~s leurs prin-
cipales caract~ristiques, que nous allons commenter rapidement.
On voit dans ce tableau que la portabilit~ de ces logiciels est mgdiocre, au-
eun d'entre eux ne fonctionnant d'ailleurs sur la sErie IRIS-80 de CII-HB. Grace
son utilisation conversationnelle et ~ son integration totale, TROLL permet d'estimer,
de simuler et de calculer des variantes pour un module d'une dizaine d'~quations en
une demi-journ~e environ, et ceci apr~s un apprentissaga tr~s court. Par contre,
TROLL est limit~ aux petits modules et sa portabilit~ est nulle. A l'opposE, SIMSYS
est construit pour traiter des gros modules, mais son apprentissage est long et son
utilisation est pEnible, m~me pour les habitues. L'ensemble XING-APACHE, limit~ aux
petits modules, est d'utilisation facile, mais , n'Etant pas int~gr~, il oblige l'uti-
lisateur ~ un travail plus lourd.
On constate enfin que si la panoplie d'algorithmes de ces trois Iogiciels est
tr~s riche pour l'EconomEtrie, il n'en va pas de mSme pour la simulation (uniquement
Gauss-Seidel pour SIMSYS). De plus, les programmes d'optimisation sont quasi-inexis-
rants (un seul algorithme, d'ailleurs dEmodE, pour TROLL et rien ailleurs) et rien
n'est pr~vu pour la gEnEration automatique des multiplicateurs du module.
Cependant, mis ~ part ces deux derniers points et la possibilitfi d'acc~der par
un interface ~ d'autres bases de donnEes, on retrouve rEparties parmi ces trois iogi-
ciels l'ensemble des qualitEs que devrait possEder un tel syst~me : facilitE d'utili-
sation, r~duction maximale des travaux de programmation, traitement des gros modules,
possibilit~ d'inclusion de progran~nes utilisateurs et, enfin et surtout, la portabili-
tE.
Nous pensons que contrairement ~ l'apparence, ces qualit~s ne sont pas trop
exclusives at le premier objectif de Modul~co est de ehercher R oombiner les qualit~s
actuellement dispers~e8 parmi le8 logiaiels existant8. Son originalit~ sera ensuite
d'inclure des programmes d'analyse de modules et d'optimisation et d'utiliser des
m~thodes math~matiques plus r~centes pour la simulation.
Modul~co sera un logiciel d'utilisation simple. Ceci r~sultera des possibilit~s
propres ~ un syst~me interactif, de l'utilisation de variables alphanum~riques facili-
tant le dialogue lors d'un caleul de variantes et, enfin, d'un gros effort de docu-
mentation automatique du logiciel (manuel d'utilisation on-line) et du module (biblio-
th~que on-line des mn~moniques, etc...).
matio~ Portcd~iZltd e z ~ t i o n
in~gr~tion
~pr~ntlssag~
doc'~nent, atgon
dimension
p ~ t e c ¢ i o ~
8oupZeaBe doew~encaCion
d'autrea bcmqu~
tatgZo
d ~ t ¢ o ~ crPcurs
VOup~e~8¢
a~ar~e~anea
amaZyse des ~sulta~;s
51MST$ XIkG-APACHE TROLL
FORTRAN FORTRAN
I ~ , CDC, UNIVAC
t r a [ t e m e n t par l o t s (batch)
extr~memeu~ d~sint~gr~
long
souvent obscure ; ang l a l s seulemeut
l i~i~fie aUX s~ries chronolo- glques g une dimension IO00 obse rva t ions
quelconque
code s~cur l t~ pour chaque s~r le
format fixe oui
impossible
modules grande f a i l l e ; d i s t i n c t i o n blocs
pargois seulement g l e compi la t ion
sp~c i£ ica t i~n du r e t a r d maxi- mum
branchements condit lomael~
Gauss-Seldel
passage pat l a banque de donnEes
tec~v~qges moindres carr~s ordinalre, variables instrt=aentales, R[Idreth-Lu, ~Iron, molndres car r~s eu trois passes
~OUpZ~SS~ n~eessit~ de eodage et u t { l l s a t i o n d~ f [ i c~ l e
iascrtlon d~ m s s l b l e p ~ , utiZisnteur8
PRILLIPS~ IBm. Honeywell
cra l tement par l o t s (batch)
s~parat£on APACHE et gING
une journ~e
e~ f rau~ais
I ~ a i ~ e a u x s ~ r g e s ch rono lo - glques k une dimension
60O observat ions maximum
meusuelle, t r i m e s t r i e l l e , ~p~riodique
pas de p r n t e c t l o n
f o ~ c l l b r e
non
~os=ible
modules t a i l l e moyenne ; d l s ~ i n e t i o n blocs
plus de 150 diagnost ics
sp~c l f l c a t i ons p r6a lab le s importantes
pas branch, condl t ; peu d'op~ra~gurs
Newton, Gau=s-Se£del
passage ob l iga~ol re par APACHE
l i s l e SI~SYS + 0rd-Wicken~. Ridge e~ r~g anus con t ra ln tes
format l i b r e ; u t [ l i s a t l o n simple
n~cess l te une ~d [ t f on da l ien~ de l 'ensemble APACHE
AEG (langage =aehlne)
I ~ { , seulement sous VH
conversa t lonne l
i u t ~ g r a t l o , t o t a l e
112 ~ une j o u r ~ e
en a n g l a l s , mals lecture f a c i l e ; doc i n t e r a c t i v e dans syst~me.
limit~e a~ s~r{e~ chronolo" glqueS A une dlmensfon
pas de l l m l c a t i o n
quelcouque
l l m i t e s d'acc~s poss{b les g e e r t a i n s u t i l i s a L e u r s
forma~ l i b r a
oui
z~poss lb le
modules t a i l l e moyenne ; pas de blocs
d6 tec t l on g l l e n t r ~ e des ~q~ations
sp~e{{£eat lons l i =£ t6es au type des va r i ab l e s
paa branch, c o n d l t . ; docu~, aut~a,
Newton s Gauss-$cfdel
analyse ~au~d[ate
l [ s t e de XING + Fl~ pour modules no= lln~aires
f o~ .a t l l b r e ; u c i l l s a t i o n s imple
E;nposs[bie
CARACTER I ST I - quEs
G~RALES
BA~UE DE
DONNEES
SPECIFICATIONS
DES
EqUAT ICeS
$ ItA.JU~T ION
ECONL'D~TR I E
Hodulfieo cherchera ~ all~ger au maximum les ~ravaux de programmation de l'fico-
nomiste, ce qui sera obtenu par la conception d'un syst~me totalement intfigrfi et per-
mettra de diviser par 2 ou 3 le travail de programmation par rapport g l'utilisation
de SIMSYS (l'usage de TROLL montre que c'est possible).
Modul~co permettra de traiter le8 modules de grande taille (plus de 1000 ~qua-
tions). En effet, d'une part un effort important a dgj~ &t~ fait pous specifier un
langage de mod~lisation ~ la lois plus pratique et plus complet, d'autre part, une
recherche repr~sentant six hommes/ann~e de travail a permis au LABORIA de maitre au
point des algorithmes de simulation et d'optimisation qui, parce qu'ils tiennent
compte de la structure particuli~re des modgles ~eonomiques et incluent des techni-
65
ques informatiques de pointe (d~rivation formelle entre autres), permettent une r~-
duction tr~s consid&rable des temps de calculo
Modul~co offrira ~ l'utilisateur la possibilit@ d'ajouter ses propres progran~es
et algorithmes. Cette possibilitY, ~videmment offerte aussi aux membres du Club vou-
lant moderniser le logiciel, sera rendue aisle par la conception modulaire de Modul~eo,
muni d'un langage de eommande permettant d'ajouter ou de retraneher des modules. Les
interfaces entre Modul~eo et les programmes utilisateurs seront possibles~ car Modu-
l~co sera ~erit darts un langage portable : CPL/! pour les modules traitant des carac-
t~res et un sous-ensemble portable de FORTRAN pour le ealeul num~rique.
Modul~co se distinguera des autres logiciels par ses programmes d'analyse de
modules. C'est une des originalit~s fran~aises des travaux de mod~lisation que de
donner une grande place ~ cette analyse. Modul~co incorporera progressivement les
algorithmes mis au point dans ee domaine (en particulier le calcul des valeurs pro-
preset vecteurs propres n~cessaires ~ l'~tude de la stabilit~ du module et le caleul
automatis~ des multiplicateurs) et comprendra un d~rivateur formel permettant de cal-
euler rapidement la version lin~aris~e du module.
Enfin, un gros effort sera fair pour l'utiZisation du module avec des facilit~s
informatiques pour la g~n~ration des variantes et la sortie des r~sultats, une docu-
mentation conversationnelle du module, un langage simplifi~ pour l'utilisation par
des non-sp~cialistes et l'introduction de plusieurs algorithmes d'optimisation.
66
CARACTERISTIQUES
GENERALES
BASES DE DONNEES
ESTIMATION
ECRITURE DU
MODE~E
SIMULATION
ANALYSE DU
MODELE
UTILISATION DU
MODELE
conception modulaire du syst~me (donc syst~me ~volutif). supportable pour los syst~mes standards de temps partag~ du constructeur. possibilit~ de traitement batch pour certaines t~ches. enti~rement int~gre. apprentissage rapide de l'ordre de la demi-journ~e. documentation conversationnelle + manuels utilisateurs et progranuneurs
s~ries ~ une ou deux dimensions (une par d~faut). taille sans limitation et fr~quence quelconque. code de s~curit~ en lecture et en ~criture, possibilit~ de documentation de la s~rie. facilit~ de mise ~ jour d'un sous-ensemble de la base. interface possible avec d'autres bases de donn~es.
addition progressive de nouveaux modules d'estimation. possibilit~ laiss~e g l'utilisateur d'ajouter ses propres algorithmes. facilit~ de mise en oeuvre.
possibilit~ de branehements conditionnels. utilisation de variables indie~es at de type alphanum~rique. construction par blocs. memo ~criture pour la simulation et l'estimation. analyse syntaxique ~ l'entr~e des ~quations.
possibilit~ de simulation de gros modules. m~thodes de Gauss-Seidel et de Newton. prise en compte de la structure pour acc~lfirer les calculs. d~rivation analytique des ~quations par un pr~processeur. calcul direct et automatique des statistiques sur les r~sul- tats.
g~n~ration automati~ue des matrices d'incidence. algorithmes de calcul des valeurs et vecteurs propres. g~n~ration automatique de la lin~arisation formelle. aide informatique pour le calcul des multiplicateurs.
facilit~s informatiques pour g~n~rer los variantes. algorithmes d'optimisation. documentation conversationnelle du module (table des mn~mo- niques...) sortie graphique des r~sultats. langage tr~s simple pour l'utilisation par des non-sp~cialis- t e s •
87
III. LE CLUB MODULECO
La r~alisation du projet Modul~co n~cessite des ressources importantes et des
comp~tences pl~ridisciplinaires (informaticiens, statisticiens, ~conomistes, ~cono-
m~tres et math~maticiens appliques), Aucune institution fran~aise ne souhaitant ou
ne pouvant d~gager ~ elle seule de telles ressources, le syst~me Modul~co sera r~ali-
s~ grace ~ la collaboration de plusieurs organismes. Les organismes suivants :
CEPREMAP, Direction de la Pr~vision, ENS T~l~communieations, Eeole Polytechnique
(Lab. d'Econom~trie), INSEE, IRIA-LABORIA, OCDE (Service de Statistiques), S~nat
(Division de l'Informatique), Universit~ de Grenoble (IMAG) et Universit~ Paris IX
ont manifest~ leur int~r~t pour cette collaboration et prendront leur d~cision tr~s
prochainement. Naturellement, d'autres organismes pourront s'ajouter ult~rieurement
cette liste.
Cette collaboration sera institutionnalis~e par la creation du "Club Modul~co",
association g but non lucratif dont l'objet est de coordonner les travaux des membres
afln de construire un ensemble informatique coherent, de diffuser les diff~rentes
versions du syst~me et les brochures d'utilisation et, enfin, d'assurer la maintenan-
ce.
Sans attendre la d~cision des autres organismes, I'IRIA a d~j~ engag~ des res-
sources importantes en personnel scientifique (l'~quivalent de 5,5 chercheurs niveau
th~se d'Etat), soit en son sein (LABORIA), soit en d~taehant des chercheurs ~ I'IMAG.
II serait souhaitable que le projet dispose de 2 chercheurs suppl~mentaires, de 2
progranmeurs d'exploitation et d'une secr~taire.
68
IV. APPORT SCIENTIFIQUE DE L'IRIA
L'id~e du projet Modul~co a ~tE lanc~e d~but 1977 par le LABORIA et depuis
cette date les organismes cites au paragraphe precedent ont partieip~ ~ des r~unions
r~guli~res ~ I'IRIA, animEes par P. N~pomiastchy, dont le r~sultat a ~t~ la publica-
tion le |-9-1978 d'un rapport d~taillE sur l'intEr~t et la faisabilit~ du projet [2].
L'animation du Club Modul~co continuera vraissemblablement ~ ~tre assur~e par
I'IRIA qui fournira par ailleurs la plus grande partie du personnel scientifique.
Mais l'apport de I'IRIA dans le projet Modul~co est surtout scientifique, avee
deux directions principales de recherche.
La premiere direction eoncerne la mise au point de m~thodes math~matiques sp~-
eialement adapt~es ~ la r~solution et ~ l'optimisation de modules macro~conomiques,
Les r~sultats obtenus dans cette direction sont exposes dang [3].
Une analyse syst~matique de la structure des modules macrQ~conomiques (cf. [3]),
bas~e sur des modules r~els, a permis de montrer que le jacobien du modgle est tou-
jours une matrice tr~s creuse que l'on peut ~ettre (par renumErotation des variables)
sous une forme quasi-triangulaire. Pa= exemple, le jacobien du modgle STAR de I'INSEE
[4], qui compte 91 ~quations, eat une matriee de 8281 ~l~ments dont 235 seul~ment ne
sont pas identiquement nuls et l'on peut renum~roter les variables d~ STAR de t~lle
sorte que la donn~e de 3 variables seulement suffise ~ rendre triangulaire le syst~me
des Equations du module.
Nous avons mis au point (cf. [5]) un algorithme (bas~ sur la recherche d'un
cycle dans un graphe) de renumErotation des variables du module de mani~re g rendre
son jaeobien le plus triangulaire possible.
L'int~r~t de cet algorithme r~side dang le fait que nQUS avons montr~ par ail-
leurs (el. [3] et [6]) que l'on peut adopter la m~thode d~ Newton pour la simulation
et la m~thode de l'Etat-adjoint pour l'optimisation ~ la structure quasi-triangulaire
du jacobien du module et obtenir alnsi un gain de temps tr~s important par rapport
l'utilisation de m~thodes elassiques, done ne tenant pas compte de la structure du
module.
Parall~lement ~ ees mises au point d'algorithmes, nous avons programm~ un pr~-
processeur eharg~ des t~ches suivantes :
a) g partir du code FORTRAN des ~quations du module, g~nErer la matrice d'inei-
dence du modgle. Cette matrice bool~enne, dont l'~l~ment e.. vaut | si et ij
seulement si la variable j intervient dans l'Equation i, est l'entrEe du
progra~ne de renumErotation.
69
b) ~ partir du code FORTRAN des ~quations du module et de la nouvelle num~rota-
tion des variables fournie par l'algorithme de renum~rotation, g~n~rer le
code FORTRAN du module triangula@is~.
c) ~ partir du code FORTRAN des ~quations, g~n~rer le code FORTRAN permettant
le calcul des ~l~ments non identiquement nuls du jaeobien, grace ~ un d~ri-
vateur formel inspir~ du logiciel FORMAC.
Ce pr~proeesseur, ~crit dans un langage portable (PL/I), nous a permis d'auto-
matiser int~gralement l'utilisation des algorithmes mentionn~s ci-dessus, ce qui
eon~titue une condition indispensable pour appliquer ces algorithmes ~ des modules
d~passant la centaine d'~quations.
La deuxi~me activit~ principale de I'IRIA dans le cadre du projet Modul~eo est
la programmation du logiciel charg~ de g~rer la bibliothgque des modules du syst~me
Modul~co. Ce logiciel sera eonstruit en collaboration ~troite avec les chercheurs de
I'IRIA ayant ~ r~soudre des probl~mes comparables dans le cadre du Club MODULEF.
70
V. CONCLUSION
L'objectif que nous nous fixons darts le eadre de ce projet Modul~eo est triple,
savolr :
a) la mise sur le mareh~ d'un syst~me informatique de construction et d'utili-
sation de modules macro~eonomiques qui soit op~rationnel, efficace, trans-
portable, d'utilisation als~e y compris par les uon-sp~cialistes et adapt~
aux besoins sp~cifiques et aux moyens informatiques des utilisateurs fran-
qais.
b) la valorisation de la recherche fran~aise en informatique par la mise sur
le march~ d'un produit construit g llaide des techniques de pointe de la
programmation modulaire.
c) la valorisation de la recherche fran~aise en ~conom~trie et en math~matiques
appliqu~es en permettant aux chercheurs de ces disciplines de tester leurs
m~thodes sur des modules r~els avec des donn~es r~elles sans avoir les pro-
bl~mes fastidieux de reeopie des ~quations et des donn~es et en donnant de
plus ~ ces chercheurs une garantie de l'utilisation concrete de leurs m~tho-
des chaque fois que celles-ei auront pu, grace ~ Modul~co, apporter la preu-
ve de leur comp~tltivit~ (remplaeement als~ d'un module du syst~me par un
autre).
71
REFERENCES
[ I ]
[2]
D. FOUQUET; J.M. CHARPIN; H. GUILLAUME; P.A. MUET; D, VALLET : "DMS, ModUle de
pr~vision ~ moyen terme", Economie et Statistiques, n ° 79, 1976.
P. NEPOMIASTCHY; B. OUDET : "Rapport final de la phase exploratoire du projet
Modul~co" (17 pages, annexe technique de 200 pages environ), publ. IRIA, Sept.
1978,
[3]
[4]
[s]
D, GABAY; P. NEPOMIASTCHY; M. RACHDI; A. RAVELLI : "Etude, r~solution et optimi=
sation de modules macro~conomiques", Rapport LABORIA n ° 312, juin 1978.
J. MISTRAL : "STAR, ModUle de pr~vision g court-moyen terme", S~minaire sur les
modules macro~conomiques, Gif-sur-Yvette, 22-26 Nov. 1976.
P. NEPOMIASTCHY; A. RAVELLI; F. RECHENMANN : "An Automatic Method to get an
Econometric Model in a Quasi-triangular Form", pr~sent~ au NBER Conference on
Control and Economics, Austin (USA), mai 1978, et Rapport LABORIA n ° 313, juin
1978.
[6] P4 NEPOMIASTCHY; A. RAVELLI : "Adopted Methods for Solving and Optimizing Quasi-
triangular Econometrics Models", pr~sent~ au NBER Conference on Control and
Economics, New Haven (USA), mai 1977, et Annals of Economics and Social Measu-
rement, n ° 6, Vol. 5, Ao~t 1978.
IDENTIFICATION, ESTIMATION, FILTERING
IDENTIFICATION, ESTIMATION, FILTRAGE
A CALCULUS OF MULTIPARAMETER MARTINGALES AND ITS APPLICATIONS*
E. Wong
Department of E lect r ica l Engineering and Computer Sciences and the Electronics Research Laboratory
Univers i ty of Cal i forn ia at Berkeley Berkeley, CA 94720 U.S.A.
I . INTRODUCTION
Let W t be a standard one-parameter Wiener process, and l e t X t be a process
defined by
( I . I ) X t = X 0 + + (tb ds 0 esdWs JO s
where the first integral is an Ito stochastic integral. For a continuouly twice
differentiable function f the celebrated Ito lemma [3] yields the representation
(1.2) f (Xt ) : f ( X o ) + I~f'(Xs)[OsdWs+bsdS] I r t f " (Xs)O:ds +2Jo which shows that the representation ( l . l ) of a process as the sum of a stochastic
integral plus an ordinary Lebesgue integral is unchanged under C 2 transformations.
The I to d i f f e ren t i a t i on formula (1.2) can be generalized and put into an
i n t r i ns i c form [4] as fol lows: Let X t be of the form
(1.3) X t : X O+M t +B t
where M t is a sample continuous local martingale and B t is a sample continuous pro-
cess with sample function which have bounded var ia t ions . Then, for any twice con-
t inuously d i f f e ren t iab le function f
(1.4) f(X t) = f(Xo) + '(Xs)dX s +
where <X,X> can be defined by
(1.5) <X,X) t = X t - 2
or al ternat ively as the squared variat ion of X on [O,t ] , and hence is known as the
quadratic variation process of X. Comparing ( I .4) with ( I .2) shows that even when X
has a representation of the form ( l . l ) , the d i f ferent ia t ion formula ( I .4) is both
simpler and free of any dependence on the representation ( l . l ) .
Of the various applications of the d i f ferent ia t ion formula, two make particu-
la r l y effective use of the in t r ins ic form. One of these is the characterization of
positive local martingales. Let {M t , t~O} be a sample continuous positive local
martingale with M O = I . Applying (1.4) to the function f(M t) = In M t quickly yields
Research sponsored by U.S. Army Research Office Grant DAAG29-76-G-OI89.
74
(1.6) In M t : [tM-IdM - ~ItM-2d(M,M) Jo s s ~JO s s
The f i r s t integral is a local martingale which we can denote by m t . A basic proper-
ty of stochastic integrals then y ie lds
t (1.7) (m'm)t : r JoM~2d(M,M)s
I t follows that a sample continuous pos i t ive local martingale is characterized by
the exponential form
( I .8 ) M t = exp{m t - ~ m , m ) t }
a special case being the exponential formula for l i ke l ihood ra t ios .
Let Z t be a sample continuous local martingale with respect to (~{c~} ,C-~) and
l e t C~ ' be a probab i l i t y measure absolutely continuous with respect to c~ ~ such that
the l ike l ihood ra t io is given by
EoC d- -l t) : e×p{m t t
Then Z t - (m,Z) t is a sample continuous local martingale with respect to C[), and
th is is essent ia l l y the Girsanov's theorem, generalized and expressed in i n t r i n s i c
form [5] .
The object of th is paper is to present a general izat ion of these ideas to the
case of two-parameter processes.
2. MARTINGALES IN THE PLANE
Let R+ denote the pos i t ive quadrant of the plane and l e t {W z, z~R } be a two-
parameter Gaussian process with zero mean and a covariance property
EW(s,t)W(s,,t,) = m i n ( s , s ' ) m i n ( t , t ' )
We shal l ca l l W a standard two-parameter Wiener process, and i t has the in terpreta-
t ion of being the integral of a two-parameter white Gaussian noise, i . e . ,
= n(T,a)dTdo (2.1) W(s't) 0 0
where n is a white Gaussian noise. As in one dimension, a major potent ia l appl ica-
t ion of Wiener processes with a two-dimensional parameter is to problems involv ing
signals and systems corrupted by white Gaussian noise.
Since 1974, a theory of stochastic in tegrals for two-parameter Wiener processes
has been developed [1,2,6-10] . As in the one dimensional case, the theory of sto-
chastic integrat ion is again c losely associated with a theory of martingales, but
the underlying theory is considerably more complex in the two dimensional case. In
?5
what follows, we shall use the definitions and notation of [7]. 2 For two points z = (s, t ) and z' = ( s ' , t ' ) in R+
(a) z ~- z' wi l l denote the condition s >_ s' and t >__ t ' , and z ~> z' the condi-
tion s > s' and t > t ' ,
(b) z ~ z ' wi l l denote the condition s <_ s' and t >__ t ' and z ~ z ' the condi-
tion s < s' and t > t ' , 2 2 (c) the function h(z,z') wi l l denote the indicator function on R+xR+ of the
condition z ~ z ' ,
(d) zxz ' wi l l denote the point ( s , t ' ) and zVz' the point {max(s,s'),
max(t,t' )) .
(e) i f z ~( z ' , (z ,z ' ] w i l l denote the rectangle ( s , s ' ] x ( t , t ' ] .
( f) 0 wi l l denote the origin and R the rectangle { O < ~ z } . Z (~,~,~) i)
Let be a probability space and le t {LJz' zER+} be a family o f q-sub-
f ields satisfying the following conditions:
(F I ) z K z' impl ies ~C~ z c ~C~z,,
(F 2) c'JC 0 contains a l l the nul l sets of
(F 3) for every z, ,C~ z : ,n ~C~z, l z ~ z
(FA) for every z, C~: and C'~ are conditionally independent given ~C~ z, where
z s, z 2 '~ For a stochastic process { X , zER,}, X(z,z'] w i l l denote X, ~, +X t - X , , - X ,
A process X is said to be ~ -~C~z-adapted i f f o r each z, X z~is "" ' ~ '~ ~~z-~measurable." In the ' t " ~
de f i n i t i ons tha t fo l low the process X is assumed to be ~7-adapted and fo r each Z, i
X z is in tegrab le .
De f i n i t i ons
(M l ) is a mart ingale i f z' > z impl ies E(Xz,I,C~z ) = X z a .s . X z (M 2) X z is an adapted l -mar t inga le (2-mart ingale) i f {Xs,t,LC~s,t} is a mar-
t i nga le in s f o r each f i xed t ( in t f o r each f i xed s) .
(M 3 ) is a weak martingale i f z' ~ z implies E{X(z,z']IC~ z} = 0 X z (M4) ~ o X ~ is a strong martingale i f X vanishes at the axes and E{X(z,z'] I
C '~V~} = 0 whenever z' ~} z.
A strong martingale is also a martingale which in turn is both an adapted l -
martingale and an adapted 2-martingale, either of which is also a weak martingale.
A Wiener process is a strong martingale.
Stochastic integrals with respect to a Wiener process W were defined in [1,6,7]
and d i f f e r e n t types o f i n teg ra ls correspond to d i f f e r e n t types o f mar t ingales. Sto-
chast ic in tegra ls of the f i r s t type
(2.2) [ RzO~dW~
are strong martingales, and those of the second type
76
(2.3) IRzXRz~,~ 'dW~dW~ '
are martingales but not strong martingales.
Lebesgue measure) of the f i r s t type
(2.4)
and the second type
(2.5)
Mixed integrals (w.r.t. W and the
IRz×Rz~,~'d~dW~
IRzXRz~, ,dW d~'
are respect ively adapted 1 and 2 martingales, but not martingales.
Now, assume that {C~_} is generated by a Wiener process W, i .e. ,
~C~ z = o({W~,~Rz}). Then Z,~z-martingales are representable as stochastic in tegra ls
in terms of W. A more general representation resu l t w i l l be stated below. Hence-
fo r th , we shall assume that % is generated by a Wiener process W.
Def in i t ion. X z is said to be a square-integrable semimartingale i f X z = M z +
Mlz+M2z+Bz where M is a square-integrable martingale, Mlz (M2z) is a sample-con-
tinuous square-integrable process which is an adapted l -mart ingale (2-martingale) £
and mean-square d i f f e ren t iab le in the 2-d i rect ion ( l - d i r e c t i o n ) , and B z = J |Rzb~d~
where b is an ,C~z-predictable process with I 2Eb2d~ < ~. ~R
I t fol lows from the resul ts of [6,7] that every square-integrable semimartin-
gale has a unique representation of the form
Rz Rz z z z z
+ IR b d~ Z
where 8 and b are ,C~z-predictable and square-integrable (dC-~dz measure) processes,
4, ~ and 6 are ~__~zvz,-predictable and square-integrable (d~dzdz' measure) processes.
Now, suppose that a process X is of the form ( I o i ) where the integrands sa t i s fy
the same p red i c t ab i l i t y conditions as before, but instead of being dC~dz or dCPdzdz '
square-integrable are now merely almost surely dz or dzdz' square-integrable. We
shall ca l l such a process a l oca l l y square-integrable semimartingale or a local
semimartingale for short.
3. A CALCULUS FOR LOCAL SEMI.IARTINGALES ON THE PLANE
The calculus that we shall describe is based on the fol lowing Fundamental
operations:
~oX stochastic integral
[X,X] quadratic var ia t ion
77
(3.1)
Then the stochastic integral
(3.2)
is defined by
(3.3) (@oX) z
(X,X> i ith directional variation
X*Y composition
Let X be a local semimartingale and let @ be a predictable process such that
Prob(sup l@zl<~) = l Z
(@oX)z : IR @ dX~ Z
R @ 0 dW + R ×R @~v~'~'~'dW~dW~' Z Z Z
+ I B_ _,dW.d~' + I @ b d~ R xR @~v~' ~,~ ~ R Z Z Z
+ IRzXRz@~V~,~,~,d{dW~,
Hence, @oX defines a mapping of local semimartingales into local semimartingales.
Let X be a local semimartingale and define [X,X] z as the quadratic variation of
X on the rectangle R z, i .e. ,
[X,X] z = lim a.s. Z X2(~) n-~oo v
for a sequence of nested partitions {A~ n)} of R z. I t can be shown that
(3.4) [X'X]z = I O~d~ + I h(~,~'),~,~,d~d~' R Rz×R z
so that: z
(a) [X,X] = [M,M],
(b) [X,X] is intrinsic to X and does not depend on its representation (2.6),
(c) [X,X] is of the form (2.6) once again, albeit a rather special case of i t .
For each direction i (i =1,2), X is a one-parameter local semimartingale so
that its quadratic variation in that direction (X,X) i is well defined and is again
an intrinsic property of X. Furthermore, (X,X) i can also be expressed in the form
(2.6) so that i t is a local semimartingale once again.
To express <X,X) i as the sum of stochastic integrals of various types, we need
a representation for X. The integrals in (2.6) over Rz×R z can be written as iterated
integrals, and X can alternatively be reexpressed as
(3 S-l) x z : IRx.I ÷ IRzXl where
(3.6-I) Xwl(z,~') = 0~, + I h(~,~')[~,~,dW~+~,~,d~] R z
: b~, + F h(~,~')B .,dW Xpl(Z,~') J ~,~ R z
(3.7-I)
or
78
(3s-2) x= IR + IRzX 2 where
+ I h(~,~')[O~,~,dW~ , + ,d~'] (3.6-2) XW2(~,z) = 8 6~,~ JR z
(3.7-2) Xu2(~,z) = b + I h(~,~')~,~,d~ R z
From (3.6-]) we can compute <X,X> l and find
(X'X>Iz = I X~l(Z'~')d~' R
Z
and using (3 .7- I ) , we can reexpress this as
(3.8) <X,X>Iz = [X,X]z + 21 XWI(~V~',~')[~ ~,dW d~' +~ ~,d~d~'] ~RzXR z , ,~
Similarly, (X,X> 2 is found to be given by
(3.9) <X,X)2z = [X,X] z + 21RzxRzXW2(~,~v~')[O~,~,d~dW ~, +6~,~,d~d~']
I t follows that <X,X). are local semimartingales once again. 1
We observe that (X,X) i and IX,X] are quadratic forms so that <X,Y) i and [X,Y] are easi ly defined as
[X,Y] = ¼([X+Y,X+Y]-[X-Y,X-Y])
<X,Y) i = ¼(<X+Y,X+Y>i-<X-Y,X-Y)i) Equations (3.5-i) can be viewed as partial differentiation formulas:
@i X = Xwi~i W+X i@iz. The operation cgmposition Y*X is suggested by the relationship
~2BI(Y*X) = B2YBI X or
~2~I(Y*X) = Yw2XwI~2W~I W + Y~2Xwl~2Z~l W + Yw2XI~2W@I z + Yu2Xul@2Z@l z
A precise interpretation of this formula is given by
(3.10) (Y-X) z = [ YW2(~,~V~')XwI(~V~',~')dW dW ~, ~RzXR z
+ IRzxRzY~2(~,~V~')XwI(~V~',~')d~dW~,
+ f YW2(~'~V~')X~I(~V~"~')dW~d~' RzXR z
+ I Y~2(~'~V~')XpI(~V~"~')d~d~' RzXR z
which shows that Y*X has the form (2.6) once again.
79
The d i f f e r e n t i a t i o n formula fo r local semimartingales in the plane can now be
stated e n t i r e l y in terms of the four basic operations: ~oX, IX,Y] , (X,Y) i and Y*X. th Let F: R ÷ R be four times cont inuously d i f f e r e n t i a b l e , and l e t F i denote i t s i
der iva t ive . Then,
(3.11) F(X) -- F(X O) + FI(X)oX + F2(X)o(X*X)
+ 1F2(X)o((X,X) 1 +(X,X) 2 - [X,X])
+ 12-F3(X)o(X*<X,X> l +<X,X>2*X + 2[X,X*X])
+ ~-F4(X) o<X,X) 2*(X,X> 1 which is both free of the representat ion (2.6) and far simpler than the d i f f e r e n t i a -
t ion formulas obtained in [9 ] . The der iva t ion of (3.11) is given in greater de ta i l
in [ I 0 ] .
4. A CHAP~ACTERIZATION OF POSITIVE FtARTINGALES
A problem related to the character izat ion of l i ke l i hood ra t ios [8] is the f o l -
lowing. Let X be a local weak semimartingale such that X 0 = O. What condi t ions
must X sa t i s f y in order for e X to be a local martingale? A simple app l ica t ion of
(3.11) gives us the answer.
Let X be written as
(4.1) X = m+m I +m2+b
where m is a local martingale, m i a proper local i-martingale and b a process of
bounded variat ion. Let M i = m+m i . Since e x is an i-martingale for i = 1,2,
characterization of one-parameter continuous positive local martingales yields
X = M 1 -½(X,X>l = M2-½(X,X) 2 (4.2)
Hence,
(4.3) x.x= M2 MI- 2* x.x>1 <X.X>2 MI + X.X>2 <X.X>I ½<X,X> *X
Lett ing F(X) = e x in (3.2) and making use of (4 .3) , we get
(4.4) e X = I + eXo(x+(x,x) I+(X,X)2-[X,X]+2[X,X*X]) + eXoM2*MI
Since M2*M 1 is a local mart ingale, e X is a local martingale i f and only i f
(4.5) X + 12--[(X,X} 1 +(X,X> 2 - IX,X] +2 IX,X 'X ] } : m
is a local mart ingale. This is essen t ia l l y the l i ke l i hood ra t i o formula of [8] in
i n t r i n s i c form. Furthermore, i f we denote e X by L, then L sa t i s f i es the martingale
integral equation
(4.6) L : 1 + Lom + LoM2*M 1
80
5. A RESULT OF THE GIRSANOV TYPE
Let L = e x be a pos i t ive martingale such that EL = I . Then, we can define a
new probab i l i t y measure C~ ' by dCp, E[ I Tz] : Lz
Suppose that Z is a (C~),{~C~z}) local martingale. What can we say about under c~,? Z
B r i e f l y , the answer consists of the fol lowing parts:
(a) Z-[Z,X+X*X] is a local weak martingale under Cp,,
(b) Z - <Z,X> i is a local i -mart ingale under C~ ' ,
(c) Z-(<Z,X)I+(Z,X)2-[Z,X+X*X]) is a local martingale under C] ) ' .
Detai ls of these resul ts w i l l be reported in a forthcoming paper.
REFERENCES
[ I ] Ca i ro l i , R. and Walsh, J.B. (1975). Stochastic in tegrals in the plane. Acta Math. 134, 111-183.
[2] Ca i ro l i , R. and Walsh, J.B. (1977). Martingale representation and holomorphic processes, Ann. Prob. 5, 511-521.
[3] I to , K. (1951). On a formula concerning stochastic d i f f e r e n t i a l s . Nagoya Math. J. 3, 55-65.
[4] Kunita, H. and Watanabe, S. (1967). On square integrable martingales, Nagoya Math. J. 30, 209-245.
[5] Van Schuppen, J.H. and Wong, E. (1974). Transformation of local martingales under a change of law, Ann. Prob. 2, 879-888.
[6] Wong, E. and Zakai, M. (1974). Martingales and stochastic integrals for pro- cesses with a multidimensional parameter, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 29, 109-122.
[7] Wong, E. and Zakai, M. (1976). Weak martingales and stochastic integrals in the plane, Ann. Prob. 5, 770-778.
[8] Wong, E. and Zakai, M. (1977). Likelihood rat ios and transformations of proba- b i l i t y associated with two-parameter Wiener processes, Z. Wahrscheinlichkeits- theorie und Verw. Gebiete 40, 283-308.
[9] Wong, E. and Zakai, M. (1978). D i f fe ren t ia t ion formulas for stochastic in te- grals in the plane, Stochastic Processes and the i r Applications 6, 339-349.
[ I0 ] Wong, E. and Zakai, M. (1978). An i n t r i n s i c calculus for weak martingales in the plane, Memo M78/20, Electronics Research Laboratory, Univers i ty of Ca l i fo rn ia , Berkeley.
ORTHOGONAL TRANSFORMATION (SQUARE-ROOT)
IMPLEMENTATIONS OF THE GENERALIZED CHANDRASEKHAR
AND GENERALIZED LEVINSON ALGORITHMS*
T. Kailath, A. Vieira and M. Morf Information Systems Laboratory
Department of Electrical Engineering Stanford University, Stanford, California 94305
ABSTRACT
In recent work, we have shown how least-squares estimation problems for arb i -
t rary nonstationary processes can be speeded up by using the notion of an index of
nonstat ionar i ty and a corresponding generalized Levinson algorithm. I t was also
shown that when the nonstationary processes have constant-parameter state-space mod-
els, the generalized Levinson algorithms reduce to the generalized Chandrasekhar equ-
ations. In th is paper we shall show that the e x p l i c i t equations of the above algor-
ithms can be replaced by certain i m p l i c i t l y defined J-orthogonal transformation pro-
cedures, where J is a signature matrix (zero everywhere except for ± l ' s on the dia-
gonal). In the state-space case, these methods y ie ld the previously-derived fast
square-root algorithms of Morf and Kai lath.
This work was supported by the Air Force Office of Scientific Research, Air Force Systems Command, under Contract AF44-620-74-C-0068, and in part by the Joint Services Electronics Program under Contract N00014-75-C-0601 and by the National Science Foundation under Contract ENG 78-I0003.
82
I . INTRODUCTION
In Refs. [1] and [2] , we showed in two somewhat d i f f e ren t ways how the constancy
of model parameters in the state-space l i near least-squares estimation problem could
be explo i ted to reduce the computational burden over that required by the general
Riccat i -equat ion so lu t ion. In Ref. [ I ] , th is was done by replacing the Riccat i equa-
t ion by Chandrasekhar-type equations. In Ref. [2] (see also [3]) fast square-root
algorithms were introduced where the recursive updating proceeds not via e x p l i c i t
equations but via i m p l i c i t l y defined orthogonal transformations. In both cases the
reduction in computation was seen to depend upon the rank of the matrices
a Pi ~ Pi+l " P1
where Pi is the variance matr ix of the error in the predicted estimate of the state
at time i . Despite th is fact however the deta i led der ivat ions of the algorithms in
I l l and [2] were d i f f e ren t and, except for a special case (Po = O) treated in [3] ,
no d i rec t connections were evident.
In th is paper we shal l show that there is in fact a very close re la t ionsh ip be-
tween the Chandrasekhar equations and the fas t square-root a lgor i thms-- in fact the
l a t t e r w i l l na tu ra l l y reveal themselves when we rewr i te the Chandrasekhar equations
in a cer ta in normalized form. The normal izat ion involves a cer ta in J-orthogonal
matr ix , where J = I n 0 - I m, for some f i n i t e n and m. I t turns out that such
matrices also ar ise in network theory, as "chain-scat ter ing matrices" (see, e . g . , [ 4 ] -
[5 ] ) . The impl icat ions of th is fac t fo r the problem of f ind ing ARMA (autoregressive-
moving average) models are explored in [6] .
Inspired by the above resu l ts , we shal l also show how to obtain an orthogonal
transformation implementation of the Levinson algori thm for solv ing Toep l i tz equations
(e .g . , as in the so-cal led LPC approach to speech analysis [7 ] ) . In fac t , we shal l
derive such versions of the generalized Levinson a lgor i thm, for equations wi th coef f i c -
ien t matrices that are "close to Toepl i tz " [8] .
THE DISCRETE-TIME CHANDRASEKHAR EQUATIONS
Consider the d iscrete- t ime state space system
xi+ 1 = F x + G u i
I I .
with
The
{xi ,Yi ,U i }
(la)
Yi = H x i + v i (Ib)
E vi = R 0
0 0 RO
{F,G,H,Q,R,11 O} are assumed to be known constant ( t ime- invar ian t ) matrices and
are vectors of dimension n, p, m respect ive ly .
83
A recursive solution to the linear least squares estimation ( l . l . s . e . ) problem
for such a model is given by the Kalman f i l t e r [9], [lO]:
= F x i l i _ l + Ki(R~)-I~ i (2) Xi+l l i ^
where Xi+ll i is the l . l . s . e , of x i given {Yo . . . . . Yi } and e i = Yi - H x i l i _ l , R~I = E ~ i~ . K i is the Kalman gain, which can be computed as
Ki = FPiH~ ' Pi A E xix ~ , ~i ~ xi - ~ i l i - l ' (3)
where Pi is found through the Riccati-type difference equation
Pi+l = FPiF~ + GQG~ - Ki(R~)-IKi (4)
Note also that
R~I = HPiH" + R (5)
The above equations also apply to models with time-variant parameters
{Fi,Gi,HiRi}, but in the constant case, we have shown that K i can be computed with-
out going through the Riccati equation (4). In Ref. [11, this was done by introducing
certain so-called Chandrasekhar equations, of which several different forms were given.
Here we use the form that arises naturally in the simplest approach to the Chandrase-
khar equations--the one via scattering theory (cf. [ I l l -J13] ) :
where
i ioi HLil i+l Li+l = Ki FLi~ @i
o l I i (Rr)-IL1H.
The i n i t i a l conditions are calculated as follows.
a(n O) = FRoF" + GQG" - Ko(R~)-IK ~ - ~0
Now we can factor a (nonuniquely) as
A(H O) = LML" ,
where [ is nx~ and M is ~x~ and nonsingular.
-(R )-IHti 1 Define
(6a)
(6b)
rank a(x O) (7a)
factorization have been considered by Bunch and Parlett [14].] Then the i n i t i a l con-
ditions for (6) can be specified as
LO : ~ , Rot = _ M-I , KO = ~0 H. • ROE = R + HRoH" (8)
C Remark: The nonsingularity of R i is an assumption, equivalent to the fact that the
process {yi } is non-degenerate in the sense that no element is a linear combination
r is a consequence of this assumption, as w i l l of any others. The nonsingularity of R i
(7b)
[Methods of achieving such a
84
be proved in the Appendix. Note however that though nonsingular R[ is not necess- 1
ar i l y positive definite (unlike R~)--but we shall show in Appendix I that i ts sig-
nature is constant.
I I I . THE FAST SQUARE-ROOT ALGORITHM
The form in which we have written the Chandrasekhar equations is a suggestive
one. I t shows that we can regard the Chandrasekhar equations as a transformation of
a certain pre-array of quantities at time i to a certain post-array at time i + I .
While we have an expl ic i t formula (6b) for the transformation matrix @i' i t is con-
ceivable that ~i has enough structure that i t need only be impl ic i t l y described.
This w i l l be made clear in the following analysis.
The f i r s t step is to notice the relation, easily veri f ied by using the basic
definit ion (6b), that
where
8i~i@i : ~i+l (g)
_R r 1
For later use i t wi l l be useful to rewrite (9) as
~i~i11C)i = ~TII
(I0)
( l l )
This is a weighted orthogonality relation, which we can try to simplify by absorbing
(square roots of) the normalizing factors into e i . For this, we define, as usual,
a square-root of the posit ive-definite matrix R i as any matrix (R~) I/2 such that
R~I = (R~)I/2(R~)T/2 (12)
For the nonsingular but not necessarily posit ive-definite matrix Ri I we introduce
an extended square-root as any matrix (R~) I/2 such that
where
many on S
0,I . . . . }.
r : (R~)I/2s(R~)T/2 (13) R i
S is the siqnature matrix of {R~}-- i .e., S is a diagonal matrix with as
+l 's ( - l ' s ) as R~ has positive (negative) eigenvalues. We put no subscript I because as wi l l be shown in Appendix I , S is the same for al l {R~, i =
Therefore i f in particular
~+(~_) = the number of positive (negative) (14a) r
eigenvalues of R 0
then
B5
S = - I c ~ -
With these d e f i n i t i o n s , we can fac to r ~. as
= ~ I / 2 j ~ T / 2 say i i '
and t hen r e w r i t e ( l ] ) in t he n o r m a l i z e d form
~iJg~ : J
where ~T/2~ ~-T/2 &
~i = ~i i~i+l
--(R~)I/Z
(14b)
-(Rrlll
(15)
(16) 4
This identity can now be applied to rewrite the Chandrasekhar equations in normalized
m
(R~) I/2 HI i
Ki F[i ~. (]7) 1
m
(R~+l)I/2 0
Ki+l Li+ l
0 (R~+l)I/2 w
(18)
A i (Rr) 1/2
(Ig)
form as
where
and
Ki = Ki(R~ )-T/2 ' [ i : Li(R~ )-T/2
r I / 2 ~ . r -T/2 A i = (Ri) LiH (R i )
We have pa r t i t i oned the arrays in (17), because i t can be seen that we
two block rows in i t to compute Ki and (R~) I / 2 (and hence the f i r s t
Moreover though we have given an e x p l i c i t formula (16) fo r
an__ny_J-orthogonal matr ix % such tha t P
I i x - (R~) I /2
% has the form n k K i F[ i V Z n
fo r i t is easy to v e r i f y tha t {X,Y,Z} w i l l be such tha t
need only
K i ) . 8 i ' we could use
, ( 2 0 a )
T We wr i t e (AT/2) - I as A -T/2
B6
E ~
XX" = Ri+ 1 , YX" = Ki ' ZZ" = Li+ 1 (20b)
Moreover we should note that there is no need to actually know the J-orthogonal trans-
formation matrix ~ - - i t is enough to achieve the form on the right hand side of (20a).
In particular, this may be done by a reasonably straightforward extension of the usual
Householder method [15].
Therefore we have now deduced a general fast square-root algorithm. I t may be
of interest to note the special case, studied in [2] and [3], when ~0 = O. In this r case, we see from (7) that A(~ o) > O, so that R 0 > O and J = Ip+ m, which means
that ~ is now an orthogonal transformation. Another important special case is when
F is stable and ~0 = ~' where ~ is the unique nonnegative definite solution of the
Lyapunov equation
FRoF" - ~0 + GQG" = 0 (21)
In this case, which corresponds to having stationary x(-) and y(.) processes, we
r 0 and J = Ip (~) -Ip. see from (7) that A(H O) < O, so that R 0 <
We shall now show how similar array algorithms can be achieved when no state-
space models are available.
IV. J-ORTHOGONAL TRANSFORMATION IMPLEMENTATIONS OF THE LEVINSON
AND GENERALIZED LEVINSON ALGORITHMS
Consider a stationary process { . . . . y_l,Yo,y I . . . . }, with a known covariance
function
E yky~_i = R i , i = O,l . . . . (22a)
R i = R# , i = O,l . . . . (22b)
Various estimation problems can be easily resolved once we know the innovations, ^
EN,t = Yt - Y t l [ t - l , t -N}
= Yt + An,lYt-l + "'" + AN,NYt-N ' say
= ANY N , say (23) where
~ = [AN, N .-. AN, 1
Using the defining property that
and
we see tha t
where R N
1] , Y~ = [y~_n...y~] (24)
m N , t ] - Y i ' t - N < i < t - I .
: _ ~ (25) E ~N,tY { E EN,tE~, t ~ R N
A_N can be obtained as the so lu t i on o f the l i n e a r mat r ix equat ion
~m N : [ 0 . . . 0 R~] (26)
is the Toeplitz matrix
m N = E YNY~ = [Ri~]N (27) Ji,J=O
87
G is independent of t and We note that i t is by v i r tue of s ta t i ona r i t y that R N
that the covariance matrix ~N is 'constant along diagonals' or Toepli.tz. This spe-
cial structure should be explo i tab le to solve the equations (26) with fewer than the
O(N 3) computations general ly necessary to solve N l inear equations in N unknowns.
In fac t , Levinson [16] and then, for vector processes, Whitt le and Wiggins and Robinson
(.see the references in [ I0 ] ) described a scheme for f inding ~ with O(N 2) computa-
tions (a computation being taken as one mu l t ip l i ca t ion of two real numbers). This
algorithm involves the simultaneous solut ion of (26) and of the equation
~ , : [R~ 0 . . . O] (28 )
where
~N = [ I BN, 1 . . . BN, N] . (29)
Then the so-cal led LWR al orithm can be wr i t ten as
where
and
m
R~+ 1 0
:
r 0 RN+ 1
. . . . . .
I -(R N
~N = r - I
-(R N) A N
(30a)
(30b)
N+I a N = [0 AN]JR 0 R_l . . . R_(N+I)]" = ~ AN,N+I_iR i (30c)
We have gone to the transposed quant i t ies {A~,B~} so that the analogy between the
LWR algorithm (30) and the Chandrasekhar algorithm (6) stands out more c lear ly . [ I t
should not be surpr is ing that the l a t t e r equations are rea l l y jus t a specia l izat ion
of (30) to the case where the stat ionary process has a known state-space model, as
was f i r s t shown in [17] and then in [18]. ]
Now i t is easy to check that ~N in (30b) obeys the same re lat ions as 9 N
Section I I I , v i z . ,
~N~N~N = ~N+I (31a) and
in
where
88
Then, as in Section I I I , we can def ine
AN : (RN)-T/2AN ' BN =
AIN = AN(R~)-T/2 , A2N
in terms of which (30) becomes
) I / 2 (RN+ 1 0
with
where
A~+I B~+l
r I / 2 0 (RN+ l )
(R~)-TI2B~ A~(R~) -T/2
F(R~) I/Z AIN -
A~ 0
- (R~)f /£ A2N
~NJ~ N = J , J = Ip (~) -S
S = the signature matr ix of ~N
[The proof in Appendix I can be eas i l y adapted to show that a l l the
same s ignature . ]
Now we can immediately v e r i f y the fo l low ing resu l t s :
Let T be any J-orthogonal t ransformat ion such tha t
(R~) I12 AIN
0 BN I has the form ~ o
:~;N (R~)I/2
X OI
0 WI
(31c)
Then
i N
{~i )
(32a)
(32b)
(33a)
(33b)
(33c)
have the
(34a)
WW" r XX" = RN+ 1 , = RN+I (34b)
yX-I = AN+I ' ZW-I = BN+I
I t can also be v e r i f i e d tha t when the s ta t ionary process has a state-space model, then
the above normalized LWR algor i thm can be reduced to the fas t square-root a lgor i thm
(20).
89
Nonstationary Processes and the Generalized Levinson Algorithm
The problem of finding the innovations of a nonstationary process can also be
formulated as in (23)-(26), except that now the matrix ~N of (26) wi l l no longer be
Toeplitz. In general then we would have to return to standard models of solving linear
equations, requiring O(N 3) computations as opposed to the O(N 2) sufficient for
Toeplitz matrices with the Levinson algorithm. However in some intuit ive way one
might feel that some matrices are clearly less non-Toeplitz than others and such mat-
rices should be invertible with a complexity between D(N 2) and O(N3). In fact,
this is possible, as has been explained in [8] and [19]. The key step is to consider
a so-called displacement rank of a matrix, defined as (R is NxN)
= rank aiR] , ~[R] = [R i+ l , j + l - Ri,j]N-2i,j=O
Note that when R is Toep l i t z , a = O, whi le i f R is completely a r b i t r a r y , then,
= N - I . However as shown in [8] , there are several in te res t ing intermediate cases;
here we mention only that i f R is the covariance matr ix of a process wi th a constant-
parameter model as in ( I ) , then ~ ~ n, the number of states, and in fact ~ =m- I ,
where m = rank A(H O) as defined in (7a). Just as the key step in the fast Chandra-
sekhar algorithms of Section I I was the fac to r i za t i on of the d i f ference matr ix A(RO),
here we introduce the fac to r i za t i on
a[~N] = D N S O~
where D N is an Np×~p matr ix. Then i t is a rather s t r i k i ng fac t that the equationa
(26) can be solved by an algori thm that has exact ly the same form as the Levinson
algorithm (30)-- the only changes are that the dimensions of A~ and B~ now are and r (N + l )pxp and (N + l )px(~ + l )p respect ive ly , whi le those of R N R N are
pxp (as before) and (~ + l )p×(~ + l )p respect ive ly . F i na l l y , the ' res idua l ' para-
meter A N is now computed as (cf . (30c))
N+I A N = ~ AN,N+I- iRi,o + A_~DN2
Note that when ~N is Toep l i t z , ~ = 0 = D N and everything reduces to the previously
discussed case. Moreover i t should be c lear that wi th proper re in te rp re ta t i on , a l l
the steps in the J-orthogonal form of the Levinson algori thm also go over to the gen-
era l ized form. [We may also draw a t ten t ion to the fact that when we fu r ther assume
a state-space s t ruc ture , the generalized Levinson algori thm can, a f te r some calcu la-
t ion be shown to reduce na tu ra l l y to the Chandrasekhar equations of Section l l - - see
[8] for the de ta i l s . ]
APPENDIX I
We shal l prove here that the rank and signature of {R~, i = 0, I . . . . } , as de-
f ined by (6)-(10) is constant. Consequently, the signature matr ix S defined in
(14b) is constant.
To prove that R~ is always of f u l l rank note that by assumption R~ > O, and 1 1
90
consequently, from (6c), 0 # det(I - R~)-IHLi(R~)-IL~H') = d e t ( I - (R~)-IL~H'(R~)-IHLi).
Thus, because R~ is nonsingular by construction, i t follows from (6d) that det R;
O, i = 1,2, . . . . Now recall that the signature of a symmetric matrix is defined as
the difference between the number of positive and negative eigenvalues. Denote i t by
~(.). We f i r s t note two simple results.
Lemma I . The number n+ in-) of positive (negative) eigenvalues of a symmetric
matrix R is equal to the dimension of a maximal subspace on which R > O
(R < 0). (By a maximal subspace we mean that no other l inear subspace of
higher dimension has the same property.)
Proof.
This is f a i r l y obvious. For a formal proof see [20, p. 252]. [ ]
Lemma 2. Let P > 0 and le t R be any symmetric matrix. Then
( i ) o(R + p) ~ ~(R)
( i i ) o(R - P) ~ ( R )
( i i i ) o(R - l ) = ~(R) i f det R # 0 .
Proof.
Let ~I be a maximal subspace such that RI) R' > O. Take x E~. Then x'(R + P)x
I > O. From Lemma l i t follows then that o(R + P) > o(R). > 0 and so (R + P) )11 The proof of ( i i ) is analogous and ( i i i ) is t r i v i a l . [ ]
Theorem. The {R#, i = O,l . . . . } have constant signature.
Proof.
From (6), we see that
R r = R~- L~H'(R~)-IHLi (AI) i+l i
Now apply the matr ix- inversion formula
(A + BCD) -I = A - I - A-IB(c -I + DA-IB)-IDA -I
to equation (AI) to get
Rri+l)-I : (R~) -I + (R~)-IL~H ~(R ~i+I)-IHLi(R~) - I (A2) (
NOW (AI) and Lemma 2 y ie ld ~(R~+ I) S ~(R#), while (A2) and Lemma 2 imply that ~(R~+I) = ~((R~+I)-I) ~o((R~)- l) = ~(R~). Therefore ~(R~+I) = a(R~). []
REFERENCES
[1]
[z]
[3]
[4]
T. Kailath, M. Morf and G. Sidhu, "Some new algorithms for recursive estimation in constant discrete-time linear systems," IEEE Trans. Automat.Contr., Vol. AC-19 Aug. 1974, pp.315-323.
M. Morf, T. Kailath, "Square-root algorithms for least-squares estimation," IEEE Trans. on Auto.Control, Vol. AC-20, no. 4, Aug. 1975, pp.487-497.
L.M. Silverman, "Discrete Riccati equations: alternative algorithms, asymptotic properties and system theory interpretations," in Advances in Control and Dynamic Systems: Theory and Applications, Vol. 12, L. Leondes, editor, Academic Press, 1975.
V. Belevitch, Classical Network Synthesis, San Francisco: Holden-Day, 1966.
91
[5]
[6]
[7]
[8]
[9]
[10]
[ ] l ]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
A.V. Efimov and V.P. Potapov, "J-expanding matrix functions and their role in the analytical theory of e lectr ical c i rcu i ts , " Russian Math. Surveys, vol. 28, no. I , pp.69-140, 1973.
P. Dewilde, A. Vieira and T. Kailath, "On a generalized Szeg~-Levinson rea l i - zation algorithm for optimal l inear predictors based on a network synthesis approach," IEEE Trans. on Circuits and Systems, Sept. 1978.
J. Makhoul, "Linear prediction: a tu tor ia l review," Proc. IEEE, vol. 63, pp. 561-580, Aprl. 1975.
B. Friedlander, T. Kailath, M. Morf and L. Ljung, "Extended Levinson and Chan- drasekhar equations for general discrete-time l inear estimation problems," IEEE Trans. Auto. Control, val. AC-23, pp. 653-659, Aug. 1978.
R.E. Kalman, "A new approach to l inear f i l t e r i n g and prediction )roblems," Trans. ASME, (J. Basic Eng.), Vol. 82D, pp.34-45, March 1960.
T. Kailath, Lectures on Linear Least Squares Estimation, Wien: Springer-Verlag, 1978.
T. Kailath and L. Ljung, "A scattering theory framework for fast least-squares algorithms," in Mult ivariable Analysis - IV, P.R. Krishnaiah, editor, Amsterdam: North Holland Publishing Co., 1977. (Original symposium in Dayton, Ohio, June 1975).
B. Friedlander, T. Kailath and L. Ljung, "Scattering theory and least squares estimation - I I : Discrete-time Problems," J. Franklin Inst . , Vol. 301, nos.l-2, Jan.-Feb. 1976, pp.71-82.
G. Verghese, B. Friedlander and T. Kailath, "Scattering theory and least squares estimation, Pt. I I I - The Estimates," IEEE Trans. Auto. Control, Vol. AC-24, 1979~ to appear.
I.R. Bunch and B.N. Par let t , "Direct method for solving symmetric indef in i te systems of l inear equations, SIAM J. Numer.Anal., Vol. 8, pp.639-655, 1971.
G. Stewart, Introduction to Matrix Computations, New York: Academic Press, 1973.
N. Levinson, "The Wiener RMS (root-mean-square) error cr i ter ion in f i l t e r de- sign and prediction," J. Math. Phys., Vol. 25, pp.261-278, Jan. 1947.
T. Kailath, M. Morf and G. Sidhu, "Some new algorithms for recursive estimation in constant discrete-time l inear systems," Proc. 7th Princeton Symposium Inform- ation and System Sciences, pp. 344-352, Apr i l , 1973.
A. Lindquist, "A new algorithm for optimal f i l t e r i n g of discrete-time stationary processes," SIAM J. Control, Vol. 12, 1974, pp.736-746.
T. Kailath, S. Kung and M. Morf, "Displacement rank of matrices and l inear operators," J. Math. Anal. and Applns., to appear.
W. Greub, Linear Algebra, New York: Springer-Verlag, 3rd edit ion, 1973.
SHORTEST DATA DESCRIPTION AND CONSISTENCY OF ORDER ESTIMATES IN ARMA-PROCESSES
J. Rissanen IBM Research Laboratory
San Jose, California 95193
i. Introduction
In [i] we introduced a criterion for estimation of parameters based on the
principle: Find the parameter values in a selected model capable of reproducing
the observed sequence so as to minimize the number of bits it takes to describe
the observed sequence. For this to make sense all the observed data points as well
as the real-valued parameters must be suitably truncated to keep the description
length finite.
Asymptotically, the resulting criterion is as follows:
Nlogo + k • log N ,
where the first term is the log-likelihood (with opposite sign) and k the number
of the parameters in the model. In [2] we proved that this criterion leads to
consistent order estimates in autoregressive processes. In this paper we study
the extension of the same result for autoregressive moving average (ARMA) processes.
2. Length Criterion
An observed sequence x={xi} ~ can be generated by a "model" of autoregressive
moving average (ARMA) type:
x t + al(P)iXt_ I + ... + ap(p)xt_ p = e t + bl(q)et_ 1 + ... + bq(q)et_ q
X t e t 0 for t ~ 0 , (2.1)
where O=(p,q,~), ~=(~,al(P),..',ap(p),bl(q),'..,bq(q)) are parameters to be
estimated; o is the variance parameter for the zero-mean normal distribution modeled
for e . t
When the observed sequence in fact has been generated by a gaussian
ARMA-process, it generally takes fewer bits to write down the observed sequence
93
within an agreed accuracy if the sequence e={e t} is described together with the
parameters~ and (2.1) is used to generate the observed sequence. This is because
the sequence e tends to be less correlated than x, and hence its variance tends to
be smaller than that of x.
In order to get finite length descriptions of the considered data we agree to
truncate the numbers e t
of ~ to a level ±6i/2.
manner:
to a maxlmum error level ±e/2, and the i 'th component ~i
Moreover, the parameters ~i are written in a floating point
! 1¢i[ = 10k+ 61' o -< ¢i < lO.
Then the sequence e can be written down with about
N
N % iE2 L(e/O) = ~ log 2~ + 7 e i /O
E =
units, the unit depending on the logarithm base.
(2.2)
The integers p and q require only about log pq units, which we ignore. The
~i' however, require a non-negllgible length; namely, about logI~il/6 i parameters
units each~ so that the total parameter description length is about
p+q 1 ;\2 L(@) = ~ i=0
Finally, the length of x relative to this model is about
N i 2 i L(x,@) = ~log 2~ + y ei/~ 0 + y log , (2.4)
T where ~i denotes the normal ized number ~i"
When the length (2.4) is minimized with respect to the parameters ~i' the
÷i optimizing values ~i must he truncated to their level ±~i' and the truncated
numbers are to be used to generate x. We can see that the larger ~i is chosen the
cheaper it is to describe the truncated component ~i' but the further away ~i may
be from the true value ~i with an increase in the length (2.2) as a result. Hence,
94
there is a balance, which can be determined by assuming that the truncation errors
~i-Ei ere uniformly distributed within their range [~i-6i/2, ~i÷~i/2]. When this
optimization is done we get the final length criterion [1],[2]:
+ ~ log 2 U(x,O) = (N-p-q)log Op,q i(p) + i=1 Bai(p)2 /
q
8bi(q)2/ (p+q+l) log (N+2) (2.5)
where
N = 1 ~ 2
~p,q N+2 i~= ei ' (2.6)
and where the parameters are taken modulo a power of i0 so that each is within
[o,io).
Asymptotically as N-~=we see that the criterion is given by:
U(x,O) = Nlog ~p,q + (p+q+l)log N , (2.7)
which form was obtained by Schwarz [3], using quite different Bayesian arguments.
3. Consistency in ARMA-Processes
{xi} ~ come from a stationary ARMA-process, also Let the observations denoted
by x={x.}: i
x t + alxt_ I + ''' + a x = w + + "'" + #, b m # 0 (3.1) n t-n t blWt-i bmWt-m' an '
where {w t} is an independent zero-mean stationary gaussian process. The roots of
both of the characteristic polynomials defined by the ai's and the bi's are taken
to be inside the unit circles and they have no common factors.
We intend to outline a proof of that the minimum length criterion leads to
consistent estimates of the two structure parameters p and q in the sense that the
probability of these estimates to equal n and m, respectively, goes to 1 as N-~=.
95
Let o be the minimized criterion with the true structure numbers n and m, nsm
and let ~ be the same for the numbers p and q. As log o is asymptotically p,q n,m
equivalent with the maximum likelihood criterion, which leads to consistent
parameter estimates, we see that o ~Ew 2 almost surely (a.s.) when N-x=. Next~ it n,m t
is easily shown that Op,q~EW2t if and only if the impulse response _$p,q corresponding
to the minimized system with p and q converges to the true impulse response #.
This follows from the fact that the prediction error Ew~ is achieved by a unique
optimumpredictor defined by #. For such convergence of the impulse response it
is clearly necessary that p~n and q~m. Moreover, if either p<n or q<m then all
limit points of the uniformly bounded sequence o must be above Ewe, which implies P,q
that the probability of the event that the minimizing numbers ~ and ~ satisfy ~n
and ~m goes to one as N -~°.
We next consider the case when p=n+l and q=m.
difference
From (2.7) we obtain the
^
d = Nlog On+ipm + log N (3.2)
n , m
The first term is nonpositive a.s. and the second is positive. We need an estimate
of the asymptotic distribution of the first term. Following a suggestion by M.
Deistler we outline below how such an estimate is obtained without having an
explicit formula for the minimized term u such as in the case with AR-processes. P,q
Let ~i(p,q) and 5j(p,q) denote the minimizing parameters with the structure
numbers p and q; i.e., asymptotically m.l. estimates. Then by the consistency and
asymptotic efficiency of these estimates
~i(n+l,m) - a i ~ 0 i = I, ..°, n
~n+l(n+l,m) ÷ 0
~i(n+l,m) - b i ÷ 0 for i = i, "", m
almost surely, and the distributions are asymptotically normal with zero mean and
2 -i + Joint covariance of type ~N (n l,m).
~i(n,m) - a i ÷ 0
bi(n,m) - b i ÷ 0
96
Similarly, also
for i = i, "'', n
for i = i, "--, m
almost surely with an asymptotically normal distribution with covariance matrix of
2-1 type ~N (n,m). Both SN(n,m ) and SN(n+l,m ) converge a.s. to the matrices S(n,m)
and S(n+l,m), respectively, because the elements of these matrices are analytic
functions of the covariance estimates
N
r i = g = xtxt-i ,
which clearly converge. With the notations
~i = $i (n+l'm) - $i (n'm)' i = i, ..., n
~n+l = an+l (n+l'm)
~n+l+i = bi(n+l'm) - bi(n'm)' i = i, ..', m ,
we have the expansion:
n+m+l
n,m n+l,o + E si- i - + + i,j=l J J
where si~ J denote the elements of S(n+l,m). Further, by logarithmic expansion
(3.3)
^ n+m+l
• og" On+itm^ = _ ~-i ~ s..~.~ + O(~k4) + O(i ) O n,m i,j= 1 13 z 3 npm
Because the ~i's are asymptotically gaussian with zero means and joint 8
2 -i n+l,m covariance matrix ~S(n+l,m), Nlog ^ has asymptotically a X2-distribution with
On,m at most n+m+l degrees of freedom. But then by (3.2) the probability that A is
positive tends to 1 as N-~.
In the same manner we can also handle the case pen and q=m+l. The case pen+l,
q=m+l remains. This time there is a problem due to the fact that the limit
97
S(n+l,m+l) has rank less than full, which, in turn, implies that the estimates
and bi(n+l,m+l) do not converge at all with a breakdown of the ~i (n+l,m+l) arguments
following (3.3).
This obstacle can be overcome if we instead consider the impulse responses
¢(p,q) = ~ ¢i(P,q) z-i = i=O
P
E ~i(P,q )z-i i=0 q
~, bi(p,q) z -i i=0
^ ^
where aO(p,q)=bo(p,q)=l. Because, plainly, almost surely
~(n+l,m+l) ÷ ~(n,m) as ~ ,
We may replace (3.3) by the expansion:
= ~+I,~+i + ~ + °(u4) + o(~) 8n,m i,J=0 OiJBiBJ
w h e r e
B i = #i(n, m) - #i(n+l,m+l); B = ~ BI i
lira 32~n+l,m+l Pij = ~ SPi SPj a.s. ,
(3.4)
1 ~. e 2 (3.5) °n+l,m+l = N ~= t"
In the last equation e t is to be calculated with the coefficients ~i(n+l,m+l) and
bi(n+l,~r~l), and the partial derivative in Pij is to be evaluated at ~(n+l,m+l).
The limit clearly exists because ~(n+l,uM-l) converges.
To illustrate these let n=m=0. Then with ~l(l,l)=a, bl(l,l)=b, and #i-~i(l,1)
we have t
et = ~= ~t-ixi; ~0 = l, #i = (-b)i-l(a-b)
F u r t h e r ,
98
22°1'1 = ~ t-j t-i ~18¢j N x x ,
which converges to 2Ex~=2r 0 if i=j and to O, otherwise. Because, #O(0,0)=i and
~i(0,O)=0, i>O, (3.5) gives
°0,0 = °i,i + 2ro ~i + 0(~4) + O( ) -=
One last difficulty still remains with (3.4); namely, to show that the sum has
a X2-distribution with only finitely many degrees of freedom. The ~i's are
asymptotically gaussian with zero mean and joint covariance matrix whose maximum
element is O(~); the other elements, say Yij p tend to zero exponentially uniformly
as l i-jl -~°. These follow from the assumption that both characteristic polynomials
in (3.1) are stability polynomials. Moreover, because both #(n+l,m+l) and ~(n,m)
are generated by systems of orders *a~l and m , respectively, only at most 2m+l of
the random variables ~i can be independent. The continuation of the proof is as
above following (3.3).
We have outlined arguments with which one can show that $ attains a minimum P,q
at p=n, q=m within the range p~n+l and q~m+l. Because m and n are the least numbers
with these properties, we can determine them asymptotically with probability i by
minimizing the length criterion and systematically increasing p and q by one
starting from p=q=O.
References
i. J. Rissanen, "Modeling by Shortest Data Description," Automatica, Vol. 14 1978.
2. J. Rissanen, "Consistent Order Estimates of Autoregressive Processes by Shortest
Description of Data," Analysis and Optimization of Stochastic Systems, Univ.
Oxford, 1978.
3. G. Schwarz, "Estimating the Dimension of a Model," The Annals of Statistics,
1978, Vol. 6, No. 2.
4. E. J. Hannah, Multiple Time Series, John Wiley & Sons, Inc., 1970.
99
SPECTRAL THEORY OF LINEAR CONTROL AND ESTIMATION PROBLEMS
E. A. Jonckheere and L. M. Silverman Department of Electrical Engineering University of Southern California
Los Angeles, California 90007
I. INTI~ODUCTION
Consider the finite-dimensional discrete-time linear system
x(k + I) = A x(k) + B u(k), x(i) = ~ (i)
where x(k) # R n and u(k) ~ Rr; A and B are time-invariant matrices of compatible size.
The pair (A, B) is assumed to be controllable and A is asymptotically stable (by
feedback invariance [4], this restriction does not introduce any loss of generality
here). Together with (i), define the quadratic cost
J[~, U(i, t)] = ~ [x'(k) Q x(k) + 2 x'(k) S u(k)
+ u'(k) R u(k)], (2)
where U(i, t) = [u'(i) u'(i + i) ... u'(t-l)]' and ~ = x(i). The overall weighting
matrix w = [~, ~] is symmetric, but not necessarily positive semi-definite.
The problem of minimizing the performance index (2), subject to the dynamical
constraint (i) is a standard one in control [1]-[4]. Moreover, this problem has a
wide range of interpretation and application [i], [4], [6], [8]-[12], which makes it
one of the most important problems of modern system theory.
Although this problem has been extensively studied [1]-[7], many of the important
features related to the underlying mathematical structure of the problem have not
previously been identified. This fact is most clearly witnessed by the difficulties
that have shown up in the literature in attempting to equate time-domain and frequency-
domain conditions for the existence of a lower bound to the quadratic cost. Along
that line, several published results have turned out to be false [i], [5]-[7].
The approach presented here -- namely, the spectral theoretic approach -- rectifies
this situation. It formalizes in the most natural way tl~e boundedness and related
questions, makes clear the connection between time-domain and frequency-domain
conditions for boundedness of the optimal cost, and elucidates many features that
have remained unclear.
The paper is organized as follows: Section II presents the basic definitions
and results; the boundedness problem is stated precisely, and a perturbed Toeplitz
operator whosespectrum plays a central role is introduced. Section III is essentially
concerned with the structure of the perturbed Toeplitz operator. A Toeplitz operator
whose spectrum is almost the same as that of the perturbed Toeplitz operator is
constructed. The frequency-domain characterization of the spectrum, which involves
100
a special factorization, is given. It is finally shown that the spectrum of the
perturbed Toeplitz operator is the union of a continuous spectrum and a finite set
of isolated eigenvalues of finite multiplicities• Section IV is devoted to some
sontrol and estimation theoretic interpretations of the spectrum of the perturbed
Toeplitz operator.
For the sake of conciseness, the proofs are usually omitted; they can be found
for the most part in previous publications of the authors [13]-[19].
II. BASIC DEFINITIONS AND RESULTS
Due to the linear-quadratic nature of the problem, it is clear that the cost
J[~, •(i, t)] can be written as a quadratic form in the initial state ~ and in the
control sequence U (i, t) :
J[~, U(i, t)] = U'(i, t) R(i, t) U(i, t) + 2 ~' S(i, t) (i, t]
+ ~' ~(i, t) ; (3a)
it is easily seen that t-i-2
~÷ y '
k=O
R(i, t) =
t-i-3 t-i-2-- B'(A')kQAkB ... +B'(A') S B'(~')
• • o
B'QAt-i-2B + S'At-i-3B ..° R + B'QB B'S
S'At-i-2B ... S'B R
The computation of S(i, t) and Q (i, t) is left to the Reader.
We are first concerned with conditions under which the quadratic cost
J[~, U(i, t)] can be bounded from below. By (3) and controllability, this is
equivalent to R(i, t) > 0 for all t > i. This is, however, not a useful character-
ization of boundedness, since it requires checking the positive semi-definiteness
of infinitely many matrices. To explain how to go around this difficulty, we
proceed formally for a moment. Observe that, for j < i ~ t, R(i, t] is a bottom
right-hand corner submatrix of ~(j, t). This suggests considering the limiting
behavior of R(i, t) as i + -~ to check boundedness. The limit of R(i, t) is a
symmetric semi-infinite matrix• In more precise mathematical terms, this matrix is
a representation of a bounded self-adjoint Hilbert space operator, and it is rather
clear that its positivity is the condition for the existence of a lower bound to the
cost.
Z 2 • ~ t) be the classical Hilbert To define this Hilbert space operator, let Rr~ - ,
space of sequences defined over { .... t-2, t-l} and taking value in R r. Define
Y = Y' ~ R nxn as the unique solution of the Lyapunov equation Y-A'YA = Q. Let
= A'YB + S and R = B'YB + R. Also, define the infinite controllability matrix
101
C = (... A2B AB B), C: i~r (-~, t) + Rn; obviously,~ C ois linear and bounded•
Further define the operator H = -C'YC, H: ERr( -~, t) ~ £Rr (-~, t). It is easily
seen that it is bounded, self-adjoint, compact, of finite-dimensional range, and
that it has an Hankel-like structure. Define the bounded self-adjoint Toeplitz
operator
°
T =
g
°
° •
B'S B'A'S
~'~ {'B
2 2 t). T: ZRr(-~, t) + ~Rr( -~,
We finally define the bounded self-adjoint Hilbert space operator which is, in our
approach, the central mathematical object of concern in the discrete-time linear-
quadratic control problem;
R(-~, t) = T + H , (4a)
~2 " oo £2 . ~ ( - ~ , t ) : R r ( - , t ) + R r ( - , t ) . (4b)
This operator is thus the sum of a Toeplitz operator and a compact perturbation•
We new come to the basic theorem which precisely states the equivalence between
the positivity of R( -~, t) and the existence of a lower bound to the cost; it also
relates the operator R( -~, t) to some more classical concepts of linear-quadratic
control -- like the Riccati equation, the linear matrix inequality, etc. This
theorem makes use of the quadratic form defined by R (-~, t) :
J[0, U( -~, t)] = U'( -~, t) R( -~, t) U( -~, t) . (5)
Notice that there is an abuse of notation in equating the right-hand side of (5) to
J[0, U( -°°, t)]; for more details, see [14, Section 2].
Theorem i: Consider the problem defined by (1)-(2). The following statement are
equival ant:
(a) For all t > i, there exists a symmetric matrix N(t-i) such that
J[~, U(i, t)] > ~'N(t-i)~, for all ~ and all U(i, t).
(b) For all t > i, R(i, t) > 0.
(c) R(-~, t) > 0•
(d) The Riccati e~uation
%(k-l) = A'I[(k)A + Q - IS + A'7~(k)B] [R + B'z(k)B]+[S ' + B'~(k)A], (6)
with ~(t) = 0, R + B'~(k)B >__ 0 and Ker [R + B'~(k)B] C Ker [S + A'w(k)B] admits a
global solution•
102
(e) The infimization problem
J2 = , (n] i n f {J[O U( -~ , t ) ] : U( -~ , t ]
has a solution; moreover J (n) = -n'w B with
£~r (-~, t) and CU( -~, t) = ~} (7)
= lim {Y-[C(T + gl)-Ic']-I}.
e~0
(f) The linear matrix inequality (Z = ~' • R nxn)
A'~A - ~ + Q s + A'~B A(~) = (8)
S' + B'~A R + B'~B
admits a solution ~r < 0; moreover, there exists a solution gT < 0 of this inequality
such that any other solution ~ satisfies ~_ < ~. If any of the above conditions is
satisfied, and if in addition the matrix A - B(R + B'~ B)+(S ' + B'~ A) is nonsingular,
then ~ is an anti-stabilizing solution of the corresponding algebraic Riccati
equation (ARE).
The proof of this Theorem is mainly given in [14] and is completed in the Appendix.
Remark i.- It should be stressed that the condition that A - B(R + B'~_B)+(S ' + B'~_A)
be nonsingular is require d to guarantee that ~ is an anti-stabilizing solution of
the algebraic Riccati equation. This is shown by the following example:
C 01 I> <::> A = , B = , Q = , S = 0 , R= 0.
Obviously, R(-~, t)> O. The variational problem (7)yields ~_ = Q- I ~>< 0.
It follows that the matrix A - B(R + B'~_B)+(S ' + B'~_A) is singular. It is easily
verified that z is the minimal solution of the linear matrix inequality, but that
it is not a solution of the algebraic Riccati equation. Moreover, it is also easily
verified that the algebraic Riccati equation does not admit any negative semi-definite
solution. These facts strongly contrast with the continuous-time results [i].
III. THE SPECTRUM OF R( -~, t)
There are two different approaches for determining the spectrum of R( -~, t].
The first one, used in [14], is based on the decomposition (4) of the operator. In
this approach, the spectrum of the Toeplitz part T is first determined using the
results of [21]; then the compactness of H allows the application of perturbation
theory [20, Chapter IX], [22, Chapter i0] to determine how the addition of H
perturbs the spectrum of [.
The second approach, which we shall use in this section, was introduced in [16].
It is based on a factorization of R (-~, t). The overall weighting matrix W can be
factored the following way:
W = (E F) , (ii)
103
where C, E ~R mxn, D, F ~ R mxr, and m~ rank W.
is clear that one can take (C D) = (E F)
systems whose transfer matrices are
J(z) = D + C(z I m A) -I B ,
K(Z) = F + E(Z I -- A) -I B .
For the regulator problem (W ~ 0), it
[4]. This facterization defines two
( 1 2 a )
(12b)
It is readily verified that the operator R( -~, t) can be factored the following way:
J _-
~(-~, t) = 3'K , (13a)
• o °
• • •
D 0 0
CB
CAB
D 0
CB D
K
° °
• o ,
• • °
F 0 0
.
EB
°
EAB
F 0
EB F
, (13b)
The motivation for introducing this factorization is that, if we commute the
order of the factors, the spectrum is almost unchanged, and the resulting operator
is Toeplitz. As far as spectral computations are concerned, the Toeplitz structure
is highly desirable [21] compared with that of T + H.
In more precise terms, a general result of Banach spaces asserts that
spec (J'K) - {0] = spec (KJ') - {0} ; (14)
[14, Lemma 7]. On the other hand, it is easily seen that we have
. °
• o
• o
EZC' + FD' (EZA' + FB')C' (EZA' + FB')A'C'
E(AZC' + BD') EZC' + FD' KJ' = ".
°
EA(AZC' + BD') E(AZC' + BD')
(EZA' + FB')C'
EZC' + FD'
(15)
where Z = Z' ~ R nxn is the unique (positive definite) solution of the Lyapunov
equation Z - AZA' = BB'. Also observe that
F(e jo) = K(e -j@) j, (e je)
oo
= ~ e-JkO(EZA' + FB')(A') k-1 C' + (EZC' + FD')
k=l
oo
+ ~ e jk@ EA k-I (AZC' + BD').
k=l (16)
104
Thus the block of ~]' are the Fourier coefficients of F(e jS) . Hence KJ, is the
Toeplitz o~erator associated with F ~ LRmxm(T), the Lebesque set of essentially
bounded functions defined on the unit circle ~" and taking values in R m~m.
Before applying the frequency-domain factorization techniques which give the
spectrum of the Toeplitz operator KJ', it is necessary to go around the fact that
the spectra of KJ' and J'K might differ by {0}. The substitution R(-~, t) ÷ R (-~, t)
+ 61 merely shifts the spectrum of R(-~, t) by a translation of magnitude 6. Hence
we choose 6 such that zero is not in the resulting spectrum. Obviously, any 6 >
-inf spec [~(-~, t)] is appropriate. In [14, Section IV] and [16, Section If],
procedures for determining an appropriate 6 are given. Observe that the substitution
~(-~, t) + ~(-~, t) + 6~ is equivalent to the substitution of data (A, B, Q, s, R) -~
(A, B, Q, S, R + 6I). Let the subscript 6 denote the quantity resulting from this
substitution. Then (14) becomes
spec (J~ K 6) = spec (K 6 J~) - {0} . (17)
Thus we shall compute the spectrum of the Toeplitz operator KBJ~; should zero appear
in that spectrum, it should be eliminated in order to get the spectrum of J~K 6_ , from
which the spectrum of R( -~, t) is readily determined.
The frequency-domain condition for positivity of R( -~, t), together with a
frequency-domain characterization of the spectrum of ~(-~, t), is given by the
following theorem:
Theorem 2: Consider the problem defined by (1)-(2), with A asymptotically
stable. Let 6 > - inf spec [R( -~, t)]. The following statements are equivalent:
(a) R(-~, t) > 0.
(b) R(-~, t) - I ~ is invertible for all I e (-~, 0).
(c) K6J' 6 - ~ I is invertible for all I e (-~, 6) - {0}.
(d) For all ~ # (-~, 6) - {0} , F 6 - II has an anti-analytic factorization:
- = (e- J0) ~1,6 (ej0) F 6(e -j@) I I ~,6
-i • • ~-i
~,6 1,6 ' ~Rm~xm6~)' ~I,6 1,6
the closed subspace of LRm6~ 6 (~) consisting of all functions with
vanishing negative Fourier coefficients.
Moreover, we have
spec [R( -~, t)] = {I - 6: I # 0 and F 6 - II has no anti-analytic
factorization}. (18)
Proof. See [14, Theorem 9] or [16, Theorem 2].
The so-called anti-analytic factorization is central in the spectral theoretic
approach to linear-quadratic control. It is investigated in detail in [15].
Statement (d) of Theorem 2 is believed to be the true frequency-domain condition for
105
the existence of a lower bound to the cost. Notice that a previous presumed
frequency-domain characterization of boundedness [1, Theorem 4] has turned out to
be false [5]; see also [6, Theorem 2] and [7].
Theorem 2 readily yields the following:
Theorem 3: Consider the problem defined by (1)-(2), with A asymptotically stable. . . . . m~ Let ~ > inf spec [R( -~, t)]. Let V 6 = i~__l {l'(e-J@)]:l @ [0, 2~ } . Then
ess spec [R(-% t)] = {I - 8: I ~ 0, ~ v~ .
Proof: See [16, Theorem 3].
The essential spectrum [22, Chapter IV, 5.6] of ~(-~, t) is thus readily
determined by a root-locus analysis. However, there are, in general, other elements
in the spectrum of R(-~, t). These elements can only be isolated eigenvalues of
finite multiplicities. In [16], an algorithm for computing these eigenvalues is
given. Briefly, the eigenvalues are given by the zeros of a polynomial matrix.
Another result of [16] is the following:
The6rem 4: Consider the problem defined by (1)-(2), with A asymptotically stable.
Then there is at most a finite set of isolated eigenvalues of finite multiplicities
in the spectrum of R( -~, t).
Using Theorem 3 and [16], the whole spectrum~f R(-~, t) can be determined by
a finite procedure. This resolves a famous control problem -- the determination of
finite procedure to check whether or not the cost is bounded from below; this
problem had not been adequately solved before; see [i, Theorem 4], [5], [6, Theorem 2],
and [7].
IV, CONTROL AND ESTIMATION THEORETIC INTERPRETATIONS
In this section, we summarize some results which show the interest of the spectrum
of ~(-~, t) beyond the boundedness problem.
Theorem 5 [14]: Consider the problem (1)-(2), with (A, B) controllable and A
asymptotically stable. If ess spec JR(-% t)] is a finite set of eigenvalues of
infinite multiplicities then no Riccati equation, nor any other algorithm equivalent
to the Riccati equation, is required to compute ~ .
Theorem 6 [17] : Consider the problem (i)-(2), with (A, B) controllable and A
asymptotically stable. Assume that any of the statements of Theorem 1 is verified.
Then ~ > 0 if and only if zero is not an eigenvalue of R( -~, t).
Theorem 7: Consider the problem (1)-(2), with (A, B) controllabke, A asymptotically
stable, W = (C D)'(C D) ~ 0, and J(z) = D + C(zI - A)-IB invertible and minimum
phase [4]. Then R(-~, t) has an essential spectrum only.
Proof: It relies on the fact that ~(-~, t) is the inverse of the Toeplitz
106
71j,)-I [(j,)-I where ~j,)-i is the Toeplitz operator associated with [J'(eJ@)] -I.
For an example, see [16, Section VII.
We now give a stochastic interpretation of R(-=, t). Over the probability space
(~, ~, m), define the stochastic process {Yk:
~+i : A'~ + c'~ ,
Yk = B' ~ + D' u k ,
where {Uk: ~ + Rr:
~÷Rm: k = i, i+l, ...} by
(19a)
(19b)
k = i, i+l, ...} is a Gaussian, zero-mean, uncorrelated process;
we further assume that ~ x. x' = 0 and ~ x. u' = 0. -l l 1 l
l ! Let Y(i, ~) = ( .... Yi+l" Yi) Let Q = C'C, S = C'D, and R = D'D. It is then
easily verified that ~(-~, t) = E V(i, ~) ~' [i, ~) . Hence _~(-~, t) may be considered
as the covariance matrix of a Markov process.
The interest of the spectral decomposition of R (-~, t) is that it allows a
representation of {Yk: k = i, i+l, ...} in terms of an independent increment process.
The spectral decomposition of R( -~, t) is
dP 1 ,
where {PA: A ~ (-~, ~o)} is a monotone increasing, right continuous, one parameter
family of projection operators. It can be shown that {Yk: k=i, i+l, ...} admits
a representation
Yk = ~_Z " (~) dz~ "
where {zl: ~ ~ R: A • (-~, +~)} is a zero-mean, Gaussian, independent increment
process; moreover, ~(zl)2 has the same behavior as PI -- E(zl)2 jumps if and only
if PI jumps, etc. The kernels {ak: R ~ Rm: k = i, i+l .... } are derived from
{Pl: I ~ (-~, +~) We shall not go through this here; it is postponed to a further
paper.
The above decomposition of a Markov process is not the only stochastic interpre-
tation of the spectral theory. For example, filtering interpretations of the
invariance of the spectrum of R( -~, t) = J'K under commutation of the factors are
provided in [19].
V . CONCLUSIONS
We have presented in this paper a summary of a new approach to linear-quadratic
control and estimation problems -- namely, the spectral theoretic approach. The
main result is a clarification of the connection between time-domain and frequency-
domain conditions for boundedness of the optimal cost. It is believed that the
spectral theoretic approach will prove useful in clarifying and solving other such
problems.
107
Acknowledgement: This research was supported by the National Science Foundation
under Grant No. ENG-76-14-379 and by the Joint Services Electronics Program through
AFOSR/AFSC under Contract No. F44620-71-C-0067.
REFERENCES
[i] J. C. Willems, "Least squares stationary optimal control and the algebraic
Riccati equation," IEEE Trans. Automat. Contr., vol. AC-16, pp. 621-634, 1971.
[2] J. M. Rodriguez-Canabal, "The geometry of the Riccati equation," Stochastics,
Vol. i, pp. 129-149, 1973.
[3] R. S. Bucy, "New results in asymptotic control theory," SIAM J. Control,
Vol. 4, pp. 397-402, 1966.
[4] L. M. Silverman, "Discrete Riccati equations: alternative algorithms,
asymptotic properties, and system theory interpretations," in Control and
Dynamic Systems, C. T. Leondes (ed.), Vol. 12, New York: Academic Press, 1976.
[5] J. C. Willems, "On the existence of a nonpositive solution to the Riccati
equation," IEEE Trans. Automat. Contr., Vol. AC-19, pp. 592-593, 1974.
[6] B. D. O. Anderson, "Algebraic properties of minimal degree spectral factors,"
Automatica, Vol. 9, pp. 491-500, 1973.
[7] , "Corrections to: algebraic properties of minimal degree
spectral factors," Automatica, Vol. ii, pp. 321-322, 1975.
[8] J. C. Willems, "Mechanisms for the stability and instability in feedback
systems," Proo. IEEE, Vol. 64, pp. 24-35, 1976.
[9] P. Faurre, "Realisations markoviennes de processes stationnaires," IRIA Report,
1972.
[i0] M. R. Gevers and T. Kailath, "Constant, predictable, and degenerated directions
of the discrete Riccati equation," Automatica, Vol. 9, pp. 699-711, 1973.
[ii] , "An innovation approach to least squares
estimation -- Part VI: discrete-time innovation representation and recursive
estimation," IEEE Trans. Automat. Contr., vol. AC-18, pp. 588-600, 1973.
[12] G. Picci, "Stochastic realization of Gaussian processes," Proc. IEEE, Vol. 64,
pp. 112-122, 1976.
[13] E. A. Jonckheere and L. M. Silverman, "The general discrete-time linear-quadratic
control problem," Proc. IEEE Conf. Decision and Control, New Orleans,
Louisiana, pp. 1239-1244, 1977.
[14] , "Spectral theory of the linear-quadratic
optimal control problem: discrete-time single-input case," to appear in IEEE
Trans. Circuits and Systems, Special issue on mathematical foundation of
system theory, Vol. CAS-25, 1978.
[15] , "Spectral theory of the linear-quadratic
optimal control problem: analytic factorization of rational matrix-valued
functions," submitted to SIAM J Control and Optimization.
108
[16] , "Spectral theory of the linear-quadratic
optimal control problem~ a new algorithm for spectral computations," submitted
to IEEE Trans. Automat. Contr.
[17] E.A. Jonckheere, "Spectral theory of the linear-quadratic optimal control
problem," Ph.D. dissertation, University of Southern california, Los Angeles,
1978.
[18] , "On the observability of the deformable modes in a class of
nonrigid satellites," Proc. S~[mp. Dynamics and Control of Nonrigid Spacecraft,
Frascati, Italy, May 24-26, 1976, ESA SP 117, pp. 251-262.
[19] , "Robustness of observers for estimating the state of a
deformable satellite," Conf. on Attitude and Orbit Contr. Systems, Noordwijk,
the Netherlands, October 3-6, 1977, Preprints Book, pp. 191-202.
[20] F. Riesz and B. Sz.&Nagy, L__ee~ons d' Analyse Fon ctionnelle. Paris: Gauthier-
Villars, 1968.
[21] R. G. Douglas, "Sanach algebra techniques in the theory of Toeplitz operators,"
Regional Conf. Series, Vol. 15, Amer. Math. Soc., Providence, Rhode Island, 1972.
[22] T. Kato, Perturbation Theor~ f0~ Linear Operators. New York: Springer-Verlag,
1966.
109
APPENDIX
The last claim of Theorem 1 and some implications of special interest are
proved here. The remainder of the proof can be found in [14] , [15] .
The implication (f) + (a) is easily proved. Let 4 < 0 be a solution of - - t-i
A(~) > O. It is easily verified that J[~, U(i, t)] = ~'4~ + ~ [x'(k)j'(k)]A(~].
[x' (k)u' (k)] ' - x' (t)~x(t) , and (a) follows trivially, k=l
To prove (e) + (f), we show that the matrix ~ defined in Statement (e) is an
appropriate solution of the linear matrix inequality. Obviously, by definition of
4 , we have w < O. To prove that ~ is a solution of the linear matrix inequality,
observe that (e) can be rewritten inf {J[O, U(-~, t)] + x'(t)~_x(t): U(-~, t)~
£2r(-~, t)} = O, where x(t) = CU( -~, t), that is, the terminal state resulting from
the control sequence U(-~, t). It follows that J[O, U(-~, t)] + x'(t)~_x(t) > O,
Z 2 " ~ t). This further implies -~/'4_;z + [~' u'(t-l)]w[~' u'(t-l)] for all U( -~, t)• Rr( - ,
+ [A~ + Bu(t-l)]'~_[A~ + Bu(t-l)] > O, for all ~ and all u(t-l). This last
inequality can be rewritten [~' u' (t-l)]A(~_)[~' u' (t-l)]'> O, for all ~ and all
u(t-1). Hence A(~_) > O. It remains to prove that any o~e~ solution 4 of A(~) >_ 0
is such that ~ < ~. The condition A(~) > 0 implies inf {~. [x' (k)u' (k)]A(4)
[x'(k]u' (k)]': U( -~, t)~ £ (-~, t)} = O; or, equivalently, inf{J[O, U( -~, t)] +
x'(t)~x(t): u( -~, t) ~iRr( -~, t)} = O. In other words, J[O, U( -~, t)] >-D'~n,
for all U( -~, t) ~ £Rr( -~, t) subject to the constraint CU( -~, t) = ~. This,
interpreted as a condition to be verified for all n and compared with the definition
of 4_, yields ~_ _< 7.
To prove the additional claim, we start from (e). By definition of 4_, we have
inf {J[O, U( -~, t)] + ~]'~ n: U( -~, t) • £2_r(-~, t) and C (-~, t) = ~} = O. This can
2_- be rewritten inf { [x'(k)j'(k)]A(~_) [x'(k)u'(k)]': U( -~, t) ~ ~ r(_~ , 2 t) and
Cu( -~°, t) = ~} = 0~.~since A(IT_) > O, it follows that, to reach the infinium, each
of the sum should be cancelled. 1 This is done by taking u(k) = -(R + B'~ B) + term t-I
(S' + B'4 A)x(k), and this yields ~ x' (k)K(IT)x(k) = O, where
x(k) = [A - B (R + B' ~ B)+(S ' + B' 4 A)] k-t n, (A.I)
from which Condition (lO) follows. It should be stressed that (A.I] requires that
A - B(R + B'Z B)+(S ' + B'~_A) be nonsingular. Since A(7~ ) > O, we have K(~_) >__ O.
It follows that x' (k)K(~_)x(k) = O, for all k < t. This, together with (A.I) and
K(4_) > O, yields K(4 ) = O.
UN ALGORITHME DE LISSAGE
M. CLERGET - F. GERMAIN
IRIA-Laboria
Domaine de Voluceau-Rocquencourt
BP 105
78150 Le Chesnay
FRANCE
1) Probl~me et notations.
On s'int~resse iei au lissage d'une s~rie temporelle m-veetorielle gaussienne
eentr~e, y(.), d~finie sur un intervalle'~de ~ qu'on prendra, pour des eo~modit~s
d'~eriture, sym~trique par rapport ~ l'origine et qu'on notera [-T,+T]. On supposera
que cette s~rie est g representation markovlenne au sens o~ il existe une s~rie n-
vectorielle x(.) (les dimensions met n n'~tant pas ~ priori ~gales), markovienne,
c'est-~-dire r~gie par une equation r~currente du type :
x(t+l) = F(t)x(t) + v(t)
telle qua
y(t) = H(t)x(t) + w(t) ,
Iv] ~tant un bruit blanc gaussien centre.
(1)
(2)
La s~rie temporelle markovienne x(.) est appel~e representation markovienne de
y(.) (une telle reprfisentation est minimale si sa dimension nest minimale).
Assoei~s ~ la s~rie temporelle y(.), d~finissons :
- la covariance de y(.) (*)
^(t,r) = E[y(t)y'(r)] (3)
- les espaces de Hilbert :
i) l'espace prgsent nots ~t (*)
~t = {Y(i)(t) ; l~i~m}
(*) Dans la suite M' d~signe la transpos~e et Mi la pseudo-inverse d'une matrice M, {al,...} l'espace engendrfi par al... , A/B le sous-espace projection de A sur Bet
y(i) la i ~me composante d'un vecteur y.
111
ii) l'espace global notfi
= {y(i)(T) ; ISi_<m ; V ~}
iii) l'espaee pass~ not~ ~t
~t = (y(i)(~) ; l_<i_< m ; z_<t }
÷ iv) l'espaee futur not~ ~t
+ ~t = {Y(i)(z) ; ]_<i_<m ; ~>t}
+/ - v) l'espace projection du futur sur le pass~ @~t ~t
vi) l'espace projection du p a s s ~ s u r l e futur ~ t / ~ t
~t/~t = {E[y(i)(r)/~t] ; ,<i<m, T-<t}
ou plus g~n~ralement les espaces :
+ / + vi i ) ' t+k
+ / ~t+k ~t = {E[y(i) cr)/~t] ; 1<i_<m ; T>t+k}
+
~t_k/~/t = {E[y(i)(*)/~t] ; l<i-<m ; T<t-k}.
x(.) ~tant une representation markovienne de y(.), on cherche ~ calculer la
llss~e :
~(t) = E[x(t)l~?
Avant d'en venir ~ ce calcul, rappelons quelques r~sultats sur l'ensemble des repre-
sentations markoviennes d'une s~rie temporelle y(.)~ r~sultats dent le d~tail se trou-
par exemple dans [l].
2) Rappels sur l'ensemble des representations markoviennes - filtre.
Partant d'une representation markovienne minimale (*), commen~ons par exprimer
la covarianee A(.,.) de y(.). En posant :
E[x(t)x'(t)] = P(t) (4)
E (v' (r)w' (r)) = ~t,r (5)
L\w(t)/ S' (t) R(t)
(*) Nous ne consid~rerons que des representations minimales.
112
o~ ~t~r est le symbole de Kronecker et en d~finissant G(.) et AO(.) par les rela-
tions (*) :
oct ) = FCt)eCt)~ 'Ct ) + s c t ) (6)
Ao(t) = H(t)P(t)H'(t) + R(t), (7)
A(.,.) s'fierit de la mani~re suivante :
A(t,r) = H(t)¢(t,r+l)G(r)It_ r + G'(t)O'(t,r+l)H'(r)lr_ t + Ao(t)6t, r (8)
o~ ¢(.,.) est la matriee de transition associfie ~ F(.) et o~ :
0 si ~ 0
l = T
I si ~>0.
Ainsi la eovariance d'une s~rie temporelle ~ representation markovienne satisfait
n6eessairement une relation du type (8) par un quadruplet {H(.), F(.), G(.), AO(.)}
donn6 et l'on sait r~ciproquement que l'existence d'un tel quadruplet suffit g d~-
montrer qu'une s~rie est ~ representation markovienne.
La proposition suivante r~unit ensuite, sur l'ensemble des representations
markoviennes, un certain nombre de r~sultats dont nous aurons besoin :
PROPOSITION ] (ensemble p(.), ensemble p(.)).
i) L'ensemble des representations markoviennes d'une s~rie temporelle y(.)
dont la covariance A(.,.) est d~crite par un quadruplet {H(.), F(.), G(.), AO(.)} est
isomorphe ~ l'ensemble p (.) de fonetions matricielles sym~triques et semi-d~finies
positives telles qu*en posant :
Q(t) = P(t+l) - F(c)P(t)Fr(t)
S(t) = G(t) - F(t)P(t)H'(t)
R(t) = Ao(t) - H(t)e(t)H'(t)
on ait sur [-T,+T[ :
(9)
(IO)
(If)
(*) G(t) et Ao(t) correspondent ~ E[x(t+1)y'(t)] et E[y(t)y'(t)].
t
113
(*) Vt. (**) base x.2
(12)
(chaque fonctlon P(.) est la variance d'une representation).
ii) P C.) admet pour la relation d'ordre usuelle entre matrices sym~triques
un maximum P*(.) et un minimum P,(.).
iii) Les inverses P-l(-t+]) engendrent l'ensemble p (.) associ~ ~ la eovariance :
~(t,r) = A(-t,-r) (13)
d~crite par le quadruplet {G', F', H', A~} (*). l
Enfin, au hombre des representations markoviennes d'une s~rie y(.), on distin-
guera celle qui r~soud le prohlgme de filtrage et qu'on peut caract~rlser comme suit :
PROPOSITION 2 (filtre).
Soit y(.) une s~rie temporelle ~ representation markovienne dent la eovariance
est d~crite par le quadruplet (H(.), F(.), G(.), AO(.)). Chacune des propri~t~s ~qui-
valentes suivantes caraet~rise une seule et m~me representation x,(.) de y(.) (**) •
i) ~ t-I est engendrg par les eemposantes de x,(t).
ii) x,(t) appartient ~ ~t_|,
ili) La variance de x,(.) est P,(.), ~l~ment minimum de p (.).
iv) Le bruit blanc y(.)-Hx,(.) est l'innovation v(.) de y(.).
v) x,(.) s'obtient au moyen des ~quations r~eurrentes :
x,(t+|) = F(t)x,(t) + K(t)[y(t)-H(t)x,(t)]
(]4)
x.(t O) = o
On note M(,) la fonction matrieielle d~finie ~ partir de M(.) par M(t) - M(-t)
L' unieit~ dent il est question dans ce th~or~me est modulo tout ehangement de TX.l.
114
o~ K(.), gain de filtrage, s'obtient ~ partir du syst~me suivant :
K(t) = [G-FP.H']t[Ao-HP.H']~ (15)
P.(t+1) = [FP.F'] t + [G-FP.H']t[Ao-HP.H']~[G'-HP.F']t
(16)
P,(t O) = O.
vi) Toute repr6sentation markovienne x(.) de y(.) admet x,(.) comme meilleure
estimEe au sens du filtrage :
x,(t) = E[x(t)/~ t-] 3"
En consequence, la solution unique du probl~me de filtrage est donnEe par les
Equations r6currentes et on dira que x,(.) est le filtre de la s~rie temporelle y(.).|
3) Antifiltre.
A toute sErie y(.) associons sa retourn~e 7(.) d~finie par :
~(t) = y(-t)
et puis fait l'objet de la proposition suivante :
PROPOSITION 3
Solt y(.) une sErie temporelle ~ representation markovienne et x(.) une de ses
representations dEfinies par :
x(t+I) = F(t)x(t) + v(t)
y(t) = H(t)x(t) + w(t).
(17)
Sa retournEe y(.) est ~ representation markovienne et l'une de ses representations
est :
x(t+I) - F'(-t)x(t) + y(t)
(18)
y(t) = G'(-t)x(t) + ~(t)
c~ l'on a pos~ :
x(t) = P-l(-t+l)x(-t+l) (19)
115
Pet G ~tant d~fini par (4) et (6). •
~ t~ t t~ t t t t~ .
La s~rie temporelle 7(.) est bien ~ representation markovienne puisque sa co-
variance est de la forme (8) si la s~rie initiale y(.) est elle-m~me ~ representation
markovienne. D~montrons que [ Y] est un bruit blanc.
o 1 m(t) L H(-t) -G'(-t)P-l(-t+l) I
avec pour r > 0 :
x(-t) ] x(-t+l) /
w(-t) J
xX(-t-r) (-t-r+ I w(-t-r)
)] [x '(-t) x'(-t+l) w'(-t)]
P(- t - r )# ' ( - t , - t-r) P(-t-r+l)~'(-t , -t-r+l)
S ' ( - t - r )~ ' ( - t , -t-r+])
P(-t-r)# '(- t+l , - t-r) i ] P(-t-r+l)~'(-t+l, -t-r+])
S '(- t-r)~ '(- t+; , -t-r+l)
On peut alors v~rifler que :
E [y ' ( t )~ ' ( t ) ] : 0 r > O. • IL~(t+r)J
y(.) ~tant ~ representation markovienne, on peut lui associer son filtre
X,(.). Acette representation correspond, d'apr~s la proposition I, une representa-
tion x*(.) de y(.). Comme la variance de X,(.) est minimale dans ~ (.), celle de
x*(.) est maximale dans p(.) ; done :
E[x*(t)x*'(t)] = P*(t)
o~ P*(.) d~signe l'~l~ment maximal de ~(.). On appellera antifiltre de y(.) cette
representation x*(.), qui est done d~duite du filtre X,(.) de la retourn~e 7(.) par
la relation :
x*(t) = p*(t)×.(-t+l),
+ Par ailleurs, comme les composantes de x.(t) engendrent ~t/~ t-]'
(20)
celles de x*(t)
116
On a done d~montr~ que :
Or les composantes de x,(t) et x*(t) engendrent respectlvement ~ ;/~- t-I et
t o Donc:
$(t) = E[x(t)/x*(t), x.(t)]
= E[x(t)/x.(t), x*(t)-x.(t)]
= E[x(t)/x.(t)] + E[x(t)/x*(t)-x.(t)]
ear x,(t) et [x*(t)-x,(t)] sont orthogonaux. Le th~or~me de projection donne alors :
Ainsi :
PROPOSITION 4
$(t) = x.(t) + [P(t)-P.(t)][P*(t)-P.(t)]$[x*(t)-x.(t)].
La liss~e ~(.) de x(.) est une combinaison lin~aire (g coefflc{ents matriciels)
du filtre et de l'antifiltre. Plus pr~cisament :
X(t) = [I-(P-P.)t(P*-P.)~]x.(t) + [P-P.]t[P*-P.]~tx*(t). , (21)
Du point de vue algorithmique, il est n~cessaire pour des raisons de stabilit~
num~rique et de choix des conditions initiales de caleuler l'antifiltre x*(.) ~ par-
tit du filtre X,(.).
ALGORITHME l
^
Le calcul de la liss~e ~(.) de x(.) peut s'effectuer en r~solvant s~par~ment
deux probl~mes de filtrage :
i) calcul du filtre x,(.) de y(.).
ii) calcul du filtre X.(.) de y(.).
et en combinant ces deux series ~ l'aide des relations (20) et (21). I
Comme c'est le cas pour le probl~me du filtrage, notons que eet algorithme
de lissage n'est v~ritablement applicable au cas stationnaire que sous l'hypoth~se
1 1 7
÷ engendrent ~ t _ ] ~ t"
~) Algorithme de lissase.
L'id~e sous-jacente consiste ~ exprimer la partie de x(.) qui appartient ~
cormne ¢ombinaison lin~aire du filtre x,(.) et de l'anti-filtre x*(.).
Le point le plus d~licat ~ montrer est le suivant :
dont la d~monstration est due ~ Ruckebusch [2].
i) D~finissons les espaces ~t et ~t + par :
~t-I -- ~t~/~t - I ~ ~t-I
- - ÷ ÷ - - _
et montrons que ~ t-l et ~ t sont orthogonaux g ~ t et ~ t-l' En effet ~ t-I '
inclus dans ~-I ' est orthogonal ~ la pattie du futur ~ + qui ne se projette pas t
sur le passg ~ t - l ' so i t , plus rigoureusement :
- ÷ ÷
~lt_l " ( ~ t ° V t / ~ t - I )
- ÷ - + ~t-l fitant par construction orthogonal ~ ~t/~t_l , l'est aussi ~ ~ t' On d~montre
+ de la m~me mani~re l 'or thogonal i t f i de ~t et ~ t - l '
^ ~+ ii) Montrons que ~(t) est orthogonal ~ ~'t-I et
t"
A
En offer, par construction, ~(t)-x,(t) est orthogonal ~ ~t-I donc aussi
~t-l" Comme :
il est clair que x,(t) est orthogonal ~ ~ t-]' Par consequent :
~[~<(t)/l~ t_1] = o.
÷ Un m~me type de d~monstration vaut pour ~ t'
118
de coercivit~ (F, asymptociquement stable e t [P*-P,] d~flnle poslcive). Dans ce cab
ALGORITHME 2 (cas stationnaire coercif)
^
La liss~e ~ de x(.) s'obtient par la relation :
~(t) = [P*-P][P*-P,]-Ix.(t) + [P-P.][P*-P,]-Ip*x,(-t+I)
o0 :
x,(t+;) = Fx.(t) + [G-FP,H'][Ao-HP,H']-I(y(t)-Hx,(t))
x,(-T) = 0
X,(tl+l) = F'x,(t ) + [H'-F'p*-IG][Ao-G'p*-IG]-I(y(t)-HX,(t))
×,(-T) = O. •
5) Conclusion.
L'algorithme pr~sent~ constitue la liss~e ~ par combinaison lin~aire du filtre
(estimation sur ~-) et l'antlfiltre (estimation sur ~+). Remarquons que ces deux
estlm~es ne sont pas ind~pendantes eomme le supposent ~ tort terrains travaux.
REFERENCES
[]] P. FAURRE, M. CLERGET, F. GERMAIN. "Op~rateurs ratlonnels positifs ; applica- tion aux s~ries temporelles et ~ l'hyperstabilit~". Dunod. (]978).
[2] G. RUCKEBUSCH. "Representations markoviennes de processus gaussiens station- naires". Th~se 3 eme cycle. PARIS VI. (1975).
REDUCED ORDER MODELING OF CLOSED-LOOP NASH GAMES
H. K. Khalil, B. F. Gardner, Jr., J. B. Cruz, Jr., and P. V. Kokotovic Decision and Control laboratory
Coordinated Science Laboratory and Department of Electrical E~gineerlng
University of Illinois Urbana, Illinois 61801, USA
ABSTRACT
We summarize recent results in two complementary research directions in the
study of well-posedness of singularly perturbed closed-loop Nash games. The natural
order reduction is ill-posed but a hierarchical reduction is well-posed in the sense
that a singularly perturbed game is asymptotic to the hierarchically reduced game.
I. INTRODUCTION
In a general multi-input system there may be many decision makers or
players each trying to minimize his own performance index. The system is described
by a vector differential equation and the performance indices are functions of
control input vectors and state vectors over some period of time. A particular
strategy, or rationale for choosing controls, is the Nash strategy which is appro-
priate when cooperation among the players cannot be guaranteed. It has the advan-
tage that if one player deviates unilaterally from the Nash strategy his performance
index will not improve. An early paper on Nash strategy is [i] and necessary condi-
tions for open- and closed-loop Nash strategies have been presented in [2] and [3]
respectively.
We consider Nash strategies when the system dynamics are singularly
perturbed. Player i wishes to choose his control u i to minimize his performance
index tf
Ji " ~ L(Xl'X2"Ul ....... uN't)dt (I)
o subject to
xi = f(xl'X2'Ul ...... UN't)' Xl(to) =xlO (2)
~2 = g(~l'X2'Ul ...... u~,t), x2(to) =50 (3)
where ~ > 0 is a small scalar and Xl, x2, and u i are nl- , n2-, and mi-dimensional
vectors, respectively.
This work was supported in part by the National Science Foundation under Grant ENG-74-20091, in part by the Department of Energy, Electric Energy Systems Division, under Contract EX-76-C-01-2088, and in part by the Joint Services Electronics Program (U. S. Army, U. S. Navy, U. S. Air Force) under Contract DAAG- 29-78-C-0016.
120
This problem is interpreted as a perturbation of the so-called natural
reduced problem, for ~ ~ 0,
~I = f(xl'x2'ul ..... UN't) (4)
0 - g(xl,x2,u I ..... UN,t ). (5)
Assuming that (5) has a unique root
x 2 = ~(Xl,U 1 ..... UN, t) (6)
we substitute (6) into (i) and (4). Then the natural reduced problem is: Player i
chooses his control u. to minimize l
tf
Jis = ~ L(Xl'#(Xl'Ul ..... UN't)'Ul .... 'uN't)dt
o
tf
= ~ E(Xl,U I ..... UN, t)dt (7) t o
subject to
~l = f(xl'~ (Xl'Ul ..... uN't)'Ul ..... ~N "t)
= ¥(Xl,U 1 ..... uN, t). (g)
The reduced problem (7),(8) is considerably simpler than the original problem (i)-
(3) because of the elimination of the fast variables and the reduction of the system
order. One of the tasks of singular perturbation analysis is to establish whether
the Nash solution of the full game is well-posed in the sense that it tends to the
Nash solution of a reduced game as ~- 0. If so, then we use the Nash solution of
the reduced game to approximate the Nash solution of the full game.
This paper summarizes results obtained in two complementary research direc-
tions in the study of well-posedness of singularly perturbed Nash strategies [4,5].
We begin by showing an example of a closed-loop Nash game whereby the performance
criteria of the singular perturbation as defined above are not the limits of the
corresponding criteria for the natural reduced problem [4]. We then establish that
this ill-posedness is caused by the difference between open-loop and closed-loop
solutions [5]. What would appear to he the ~atural reduced problem lacks informa-
tion of the fast dynamics and because of this difference, it is not the limit of the
full order game [5]. To avoid difficulty with ill-posedness we propose that the
reduced problem should account for information about the fast low order game. We
sunm~rize a hierarchical reduction procedure which has been proved to lead to a
well-posed singularly perturbed modified slow game [4]. This reduced order slow
game differs from the natural one in that it preserves information about the low
order fast game [4].
II. ILL-POSEDNESS OF NATURAL REDUCED CLOSED-LOOP NASH GAMES WITH RESPECT TO SINGULAR PERTURBATION
121
Consider a singularly perturbed linear time-invariant system
Xl = AllXl + A12x2 + BllUl + B12u2 ; Xl(to) -Xl0 (9)
~2 = A=lxZ + A22x2 + B21Ul + ~22u2 ; x2(t o) =x20 (lO)
and performance criteria
" ½~-t + dr; i , j~ l ,2 , i+j . (11) J i o 2 [Q12 %3J
The control vectors Ul,U 2 are chosen by Players 1 and 2 ' respect ively in accordance
with the Nash solut ion concept, and the control strategies are restr ic ted to be
linear feedback functions of the state. Denote
r".-,-, I'-I x = , A : , B i = , and Qi ~ , "
2 LA21/~ A22/~] LB2i/~j Lqi2 qi
The usual definiteness assumptions are made on Qi and Rij , i,j=l,2.
The optimal closed-loop Nash strategy for Player i for (Ii) subject to (9),
(I0) is well known [3] and given by
-ll u i = -RiiBiKix (12)
where K. is a stabilizing solution of the coupled Kiccati equations given by
_(Qi+KIA+A,Ki)+K.B.R?IB~K.+K.B R-IB, K -I ' B~R~h R~- -I , 0
for i,j =1,2; i#j. (13)
Notice that since A and B i are functions of the small parameter ~, K i is also a
function of ~.
Following the standard approach to obtaining a reduced order game we set
= 0 in (i0), solve for x2, assuming ~2 is nonsingular, and substitute in (9) and
(ii) to obtain
Xs = A0Xs + B01Uls + B02U2s ' Xs(t0) = Xl0 (14)
and
If t ̂ ^ ^ ^ Ji~ : -2 t {~s Q'11 ~s +2x~Qi2 (~2tui+~2juj s ) +U[sRiiuis +ujsRijujs O +2U~sQi3Ujs}dt i,j = 1,2, i#J. (15)
Solving for the reduced order closed-loop Nash strategies, we have
^ - 1 I I ^ ! ^
Uis = -Rii[ B0iKisXs+B2iQi2Xs+Qi3Ujs ]
= - M . X i S S
where Kis is a stabilizing solution of ^ ^
= +M. R..M. -M. R..M. +[KIsBoj+QI2B2j]M.sj 0 -(Qiz+A~Kis+KisA0 ) ' ̂ ' ̂ zs zl £s 3s i] Is ! ! ! ^F
+M~s[B0jKis+B2jQi2 l, for i,J =1,2, i#j.
(16)
(17)
(18)
122
Using the gain matrix Mis from (17), we implement the control
u i = -MisXl (19)
and apply it to the system in (9),(10). The resulting value of the suboptimal per-
formance criteria in (ii) can he expressed as
i ' (to)VisubX (to) (20) Ji = ~x sub
where Visuh satisfies the Lyapunov equation
Visuh{A-Bi[ Mis:0] -Bj[Mjs:0] } + [A-Bi[Mis. 0] -Bj [Mjs.0] } 'V. . . . . Isu b
il+MisRiiMis+MjsRijMj s j_ 0. (21)
+ , ]
L Qi2 |
The matrix Visub depends on ~ since A and B i contain ~. Hence the reduced cost is
dependent on ~.
If the optimal Nash controls given by (12) are applied to (9),(10), the
values of the optimal performance criteria are given by
I x' (22) Ji = ~ (to)Kix (to)
where K i satisfies (13). We wish to examine the nature of the optimal criteria Ji
as ~ ~ 0. In particular we wish to verify if Ji approaches Jisub as ~ approaches
zero. We will say that the reduced order game is well-posed if Ji approaches Jisuh
as ~ 0. Otherwise, we say that it is ill-posed. It has been shown in [4] that the
natural reduced closed-loop Nash game is Ill-posed.
Consider the second order system
= + I + u2 (23) 42 I/~ -2 2 /~ 2/
2 (0)j
with performance criteria
24}dt Jl = ~JO ~x [I ~x + u I + (24)
1 ,r2 + (25)
For this example, the resulting Kis and Mis from (16)-(18) are
KI s = K2 s = 2~ = .6804 (26)
and Mls = ~s = .4082. (27)
123
Calculation of the resulting values of Jl for several values of ~ are given in
Table i. Because of symmetry, Jl ~J2 = J°
Table I
.5 .2 .I .01 .005 .001 0
J 1.3012 .73245 .5425 .3724 .3630 .3630 .3536
Jsub 1.84127 .86558 .59083 .36420 .35217 .34259 .3402
It is seen that the limit of Ji as ~ - 0 is different from the corresponding limit of
Jisub. This discrepancy between the J's in the neighborhood of ~ ~ 0 indicates that
the natural reduced Nash game obtained by the standard method is ill-posed [4]. It
is important to understand why the closed-loop Nash solution fails to possess the
desired well-posedness property. The answer is to be sought in the difference
between open-loop and closed-loop solutions of a Nash game.
III. INFORMATION STRUCTURES AND WELL-POSEDNESS OF NASH GAMES
Necessary conditions for Nash solutions have been derived in [2] for open-
loop solutions, and in [3] for closed-loop solutions. In open-loop solutions, the
optimal strategy for a trajectory through a specified initial state is given as a
function of time, while in closed-loop solutions the optimal strategy is given as a
function of time and the current value of the state vector. For the full game (I)-
(3), necessary conditions for a closed-loop solution are
Xl = f(xl'x2'ul'''''UN't)' xl(to) " Xl0 (28)
~x2 = g(xl'x2'ul'''''uN't)' x 2(t o ) = x20 (29)
N ~ (t,xl,x 2) ~i " -vXlH1 - j~l ( ~x I )'Vuj~j, Pi(tf ) = o
j#i
N ~T. (t,xl,x2)) ~ql = -VX2HI" - J=~l (j#i "1 ~x2 'VujHj ' qi(tf) = 0 (31)
u i = Ti(t,Xl,X2) minimizes Hi(Xl,X2,t,~l,...,Ti.l,Ui,Ti+l,...,TN, Pi,qi) (32)
where
Hi(Xl,X2,t,u I ..... uN,Pi,q i) = Li(Xl,X2,U I ..... uN, t) +Plf(xl,x2,u I ..... uN, t)
+q~g(x l,x 2,u I ..... ~,t). (33)
For open-loop solutions the necessary conditions are the same except that the partial
derivatives in the right hand side of (30), (31) disappear since the optimal strategy
is not a function of x I or x 2. Hence (30) and (31) are replaced by
Pi = -VxlHi' Pi(tf) = 0 (34)
~qi = "Vx2Hi' qi(tf) - O. (35)
724
The partial derivative terms in the right hand side of (30) and (31) give rise to
generally different open-loop and closed-loop solutions.
Let us investigate what happens when the full game is approximated by the
reduced game. The first step is to set ~ = 0. In the set of necessary conditions
(28)-(32), setting ~ =0 replaces (29),(31) by
0 = g(xl,x2,ul ..... UN,t ) (36)
N 5V. = - Z -__i ' (37) 0 -Vx2Hi j =l (~x 2 ) VujHj "
The next step is to use the root of (36) to eliminate x 2 from (28), (30), and (32).
Eliminating x 2 automatically assumes that the optimal strategies TI,...,~N are
functions of t and x I only. Thus
~x 2 = 0, i : i .... ,N. (38)
Subsequently, ql,...~q N are eliminated using
Vx2H i = 0, i=l,...,N.
Suppose now that we first solve (28)-(32), and then let ~ ~ 0.
true that
lim V H. ~0, i =I,...,N, for all t6(to,tf) (40) ~-0 x 2 l
where the end points to,t f have been excluded from the equality in (40), then the
order reduction is generally not well posed. The reduced solution does not, in
general, satisfy the boundary conditions x2(to)-x20 , qi(tf) = 0. Hence boundary
layer correction should be added to compensate for the discrepancy in the boundary
conditions.
Investigation of (28)-(32) shows that under appropriate assumptions (llke
having L i 5e a quadratic function in x2, and f,g to be linear functions of x2) one
may he able to show that
N~_.t' lim~ H i+ ~ (?x2) VujHj) = 0, i=l ..... n, Vt((to,tf). (41) ~--0 X2 j=l
j#i The fact that (41) does not necessarily imply (40) is the reason why one should not
expect closed-loop solutions of Nash games to be well-posed, in general [5].
In open-loop Nash solutions this difficulty does not arise because (35)
does not involve the partial derivatives of u i with respect to x 2. Furthermore,
this difficulty will not arise even if the controls are functions of Xl, because the
partial derivatives of u i with respect to x I are not the reason for the trouble.
Therefore, one can think of a new class of solutions in which u i =ui(t,Xl). Such a
solution will be called partially closed-loop. It is shown in [5] that the open-
loop and partially closed-loop solutions of the linear quadratic Nash game defined
in Section II are well-posed.
(39)
If it is not
125
After showing the reason of the difficulty in closed-loop solutions, the
next logical question would be: Is there an alternative way to define a reduced
game whose closed-loop solution could be the limit of the closed-loop solution of the
full game? The answer is obvious in view of (41). Suppose that one could know in
advance lim(~u~/~x~) and substitute them in (38). Then using (38) to eliminate ql
will define the reduced game in accordance with (41). It may seem that knowing
~ (~ui/~x2) in advance is impossible. However, in the linear quadratic problem of
Section II it is possible to extract from the full game another game called the fast
game which is played with respect to the fast dynamics of the system. This fast
game can be solved independently and provides the required limits lim(~ui/bx2). This p-O
modification of the reduced slew game is explained in the following section.
IV. HIERARCHICAL REDUCTION SCHEME WHICH TRANSFERS FAST GAME INFORMATION TO A MODIFIED SLOW GAME
To derive the fast subsystem of (9),(i0), we assume that the slow variables
are constant during fast transients. Denoting the fast variables by the subscript f
we have the fast subsystem and performance indices
~f = A22 x f + B 2 lUl f + B22u2 f (42)
i Jif: { x %3xf+u iiuif+u ljujf3dt i.j-l,2; i+j (43)
0
where xf=x2-X2s. The closed-loop Nash controls for (43) subject to (42) are
Uif -- -RiilB~iKifxf , i = 1,2 (44)
where Kif is a stabilizing solution of
, -i , -I , Q K K +K B R B K +K B R B K 0=-i3-1f~2-A~2 if if21ii2iif ifZj jj 2j jf
-I , -I -I , +KjfB2jRjjB2jKif- 5 B R .R .R..B K i,j-l,2 i#J. (45) f 2j J3 ij 33 2j jf '
Next we make use of the fast control and substitute the following for n i in
our original system (9), (i0) and performance indices (II). Let
ui -- R[ B iKi 2 + (46) be our modified control. This gives a new system and performance indices given by
^ ^ ^
~i • AllXl + A12x2 + BllUl + B12u2 ; Xl(t0) =Xl0 (47)
~x2 = ~IXl + ~2x2 + B21Ul + B22u2 ; x2(t0)=x20 (48)
and
Qi2 , ^ , - I_ ^ I ~ x 2x K B u 2x K B R .R u
Ji ~t0(x',[Qi2 Qi - 2 if 2i i- 2 Jf 2j jJ ij J
^r ^ ^! ^
+ujRijuj}dt i,J ffi 1,2, i#j. (49) + uiRiiu i
To get our '~nodified slow" subsystem we formally set p = 0 in (48) and solve
126
for x 2. This gives
^-i -- -- B22%] . -- X-2 ~ "~2[~iXl + B21Ul + (50)
Substitution of (50) into (47) and (49) gives us the '~nodified slow" subsystem and
performance indices
Xsm = A0Xsm + B01Ulsm + B02U2sm Xsm(tO) =Xl0 (51)
and
= [ [x' QilX +2x' Q.12u. +2XsmQi2Ujsm+2U~smQi3Ujsm Jism to sm sm sm Ism
+u! R_.u. +u'. R..u. ]dt ; i,j--l,2, i#j. (52) Ism ii ism 3sm 13 3sm
It is shown in [4] that the reduction process described above leads to a well-posed
reduced game. Note that the modified slow subsystem and performance indices are of
the same form as in the slow problem considered in Section II. However, the system
matrices and performance coefficients preserve information about the fast low order
game.
Ill
[2]
[3]
[4]
[5]
REFERENCES
J. F. Nash, "Noncooperative Games," Annals of Math., Vol. 54, No. 2, pp. 286- 295, 1961.
J. H. Case, "Toward a Theory of Many Player Differential Games," SIAM J. Contro.______!l, Vol. 7, No. 2, pp. 179-197, 1969.
A. W. Starr and Y. C. Ho, "Nonzero-Sum Differential Games," J. of Optimization Theo~and Applications, Vol. 3, No. 3, pp. 184-206, 1969.
B. F. Gardner, Jr. and J. B. Cruz, Jr., '~ell-Posedness of Singularly Perturbed Nash Games," Journal of the Franklin Institute, Vol. 306, 1978, to appear.
H. K. Khalil and P. V. Kokotovic, "Information Structures and Well-Posedness of Singularly Perturbed Nash Games," IEEE Trans. on Automatic Control, Vol. AC-24, 1979, to appear.
QUANTUM ESTIMATION THEORY (I)
by
Sanjoy K. Mitter (2) and Stephen K. Young (3)
(2) Department of Electrical Engineering and Computer Science and Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
(3) BDM Corporation, McLean, Virginia 22101
(i) This research has been supported by NSF Grant # ENG76-02860 and NSF Grant # ENG77-28444.
128
i. INTRODUCTION.
In classical communication theory, the message to be transmitted is modulated and the resulting signal propagates through a given channel to produce a received waveform. The function of the receiver is to recover the signal from the received waveform, perhaps in an optimum manner, optimum being defined by some performance criterion. The input to the receiver may have some additive noises added to the received waveform. It is assumed that the receiver can be constructed independ- ently of the model of the received waveform and the additive noise. Moreover, it is assumed that the optimum receiver can be physically realized.
In communication at optical frequencies neither of these two assumptions are valid. No matter what measurement we make of the received field, the outcome is random whose statistics depend on the measurement being made. This is a reflection of the laws of quantum physics. Furthermore, there is no guarantee that the meas- urement characterizing the receiver can be actually implemented.
In this paper, we present a theory of quantum estimation problems. Full de- tails will he published elsewhere [i]. For related work, see [2] and [3] and [4].
It will be assumed that the reader is familiar with the notions of convex analysis in infinite dimensional spaces a% for example, presented in [8].
In the classical formulation of detection theory (Bayesian hypothesis testing) it is desired to decide which of n possible hypotheses HI,...,H is true, based on
n observation of a random variable whose probability distribution depends on the several hypotheses. The decision entails certain costs that depend on which hypo- thesis is selected and which hypothesis corresponds to the true state of the system. A decision procedure or strategy prescribes which hypothesis is to be chosen for each possible outcome of the observed data; in general, it may be necessary to use a randomized strategy which specifies the probabilities with which each hypothesis should be chosen as a function of the ohserved data. The detection problem is to determine an optimal decision strategy.
In the quantum formulation of the detection problem, each hypothesis H. cor- 3
responds to a possible 0j of the quantum system under consideration. Unlike the
classical situation, however, it is not possible to measure all relevant variables associated with the state of the system and to specify meaningful probability dis- tributions for the resulting values. For the quantum detection problem it is nec- essary to specify not only the procedure for processing the experimental data, but also what data to measure in the first place. Hence the quantum detection problem involves determining the entire measurement process, or, in mathematical terms, determining the probability operator measure corresponding to the measurement
process.
2. OBSERVABLES, STATES AND MEASUREMENT IN QUANTUM SYSTEMS.
Let H be a complex Hilbert space. The real linear space of compact self- adjoint operators K (H) with the operator norm is a Banach space whose dual is iso-
$ metrically isomorphic to the real Banach space Ts(H) of self-adjoint trace-class
operators with the trace norm, i.e.,
Ks(H)* = Ts(H) under the duality
<A,B> = tr(AB) _< IAltrIBI A g TS(H), B 8 Ks(H)
Here, IBl=sup{lB#]: ¢ a H, I~1 < I} = sup{tr(AB) : A g T (H) ]A[ < I} -- s ' tr --
129
and IAItr is the trace norm ~ I%~ < +~ where A g Ts (H) and {%i } are the eigenvalues
i of A repeated according to multiplicity. The dual of Ts(H) with the trace norm is
isometrically isomorphic to the space of all linear bounded self-adjoint operators, i.e., Ys(H)* = Ls(H) under the duality
<A,B> = tr(AB) A e T (H), B 6 i (H) . S S
Moreover the orderings are compatible in the following sense. If K (H)+, Ts(H) +, S
and Ls(H) + denote the closed convex cones of nonnegative definite operators in
Ks(H), Ts(H) , and is(H) respectively, then
[Ks(H)+]* = Ts(H)+ and [Ts(H)+]* = Ls(H) +
where the associated dual spaces are to be understood in the sense defined above.
In the classical formulation of Quantum Mechanics one is given a complex Hil- bert space H and a measurement is identified with an element A E Ls(H). Ls(H) is
termed the algebra of observables on H. The a priori statistical information about the quantum system is incorporated in the "state" p of the system, where P g T s(H)÷
and is of unit trace. In Quantum Communication problems a more general concept of a measurement (observable) is needed. As we have mentioned before this is con- veniently described in terms of an operator-valued measure. For a discussion on the need for going to generalized measurements seeDavies [5] and Holevo [2].
In quantum mechanical measurement theory, it is nearly always the case that physical quantities have values in a locally compact Hausdorff space S, e.g. a subset of R n, and we shall make this assumption. Let H be a complex Hilbert space. A (self-adjoint) operator-valued regular Borel measure on S is a map
m: B + L (H) such that <m(')~]~> is a regular Borel measure on S for every ~,~ g H. S
In particular, since for a vector-valued measure countable additivity is equivalent to weak countable additivity m(')@ is a (norm-) countably additive H-valued measure for every ~ E H; hence whenever {E } is a countable collection of disjoint subsets
n
in B then
m( U En) = ~ mCEn), n=l n=l
where the sum is convergent in the strong operator topology. We denote by M(B,[ (H)) the real linear space of all operator-valued regular Borel measures on
s S. We define scalar semivariation of m g M(B, Ls(H)) to be the norm
sup l<m(.)~r~>l(s)
where l<m(')@l@>I denotes the to ta l variat ion measure of the real-valued Borel measure E ~ <m(E)@l@>. It can be shown that scalar semivariation is always finite.
A positive operator-valued regular Borel measure is a measure m g M(B,Ls(H)) which satisfies
re(E) > 0 VE C B,
130
where by m(E) ~ 0 we mean m(E) belongs to the positive cone Ls(H) + of all non-
negative-definite operators. A probability operator measure (POM) is a positive operator-valued measure m ~ M(B,Ls(H) ) which satisfies
re(s) = I .
If m is a POM then every <m(-)Ol~> is a probability measure on S and ~(S) = i. In particular, a resolution of the identity is an m c M(B,Ls(H)) which satisfies
m(S) = I and m(E)m(F) = 0 whenever E N F = ~; it is then true that m(') is pro- jection-valued and satisfies
m(ECIF) = m(E)m(F), E,F g B. +
3. INTEGRATION WITH RESPECT TO OPERATOR-VALUED MEASURES.
In treating quantum estimation problems it is necessary to have a theory of integration with respect to operator-valued measures. We outline this theory now. First, we consider integration of real-valued functions. Basically we identify the regular Borel operator-valued measures m e M(B,Ls(H)) with the bounded linear
operators L: Co(S) ÷ Ls(H), to get a generalization of the Riesz Representation
Theorem.
Theorem 3.1 Let S be a locally compact Huasdorff space with Borel sets B. Let H be a
Hilbert space. There is an isometric isomorphism m~>L between the operator- valued regular Borei measures m g M(B,Ls(H)) and the bounded linear maps L g ~(Co(S) ,
L (H)). The correspondence m<-~L is given by S
L(g) = fg(s)m(ds), g ~ Co(S) S
where the integral is well-defined for g(-) a M(S) (bounded and totally measurable maps g: S ÷ R) and is convergent for the supremum norm on M(S). If mg-~L, then m(S) = ILl and <e(g)~I~> =~g(s) <m(.)~l~> (ds) for every ~,~ e H. Moreover L is positive (maps Co(S) + into Ls(H)+) iff m is a positive measure; L is positive and
L(1) = I iff m is a POM; and L is an algebra homomorphism with L(1) = I iff m is a resolution of the identity, in which case L is actually an isometric algebra homomorphism of Co(S) onto a norm-closed subalgebra of Ls(H).~
Remark Since every real-llnear map from a real-linear subspace of a complex space
into another real-linear subspace of a complex space corresponds to a unique "Hermitian" complex-linear map on the complex linear spaces, we could just as easily identify the (self-adjoint) operator-valued regular measures M(B,Ls(H))
with the complex-linear maps L: C (S,C) + L(H) which satisfy o
L(g) = L(~)*, g a c (s,c). O
3.1 integration of ~ (H)-valued functions. s
We now consider L(H) as a subspace of the "operations" L(T(H),T(H)), that is, bounded linear maps from T(H) into r(H). Every B ~ L(H) defines a bounded linear
function LB: 7(H) + I(H) by
L B(A) = AB, A E T(H)
131
with IBI = ILBI. In particular, A + trAB defines a continuous (complex-) linear
functional on A 8 %(H), and in fact every linear functional in T(H)* is of this form for some B e /(H). We note that if A and B are selfadjoint then tr(AB) is real linear (although it is not necessarily true that AB is selfadjoint unless AB = BA). Thus, it is possible to identify the space T (H)* of real-linear con-
s tinuous functionals on • (H) with /s(H), again under the pairing <A,B> = trAB,
S
A g Ts(H) , B g Ls (H). For our purposes we shall be especially interested in this
latter duality between the spaces T (H) and L (H), which we shall use to formulate 8 s
a dual problem for the quantum estimation situation. However, we will also need to consider L (H) as a subspace of L(T(H),T(H)) so that we may integrate T (H)-valued
S S
functions on S with respect to L (H)-valued operator measures to get an element of S
~(~).
f:
Suppose m E M(B,Ls(H)) is an operator-valued regular Borel measure, and
S ÷ T (H) is a simple function with finite range of the form s
f(s) n
= ~ 1E (s)pj j=z J
where P. e T (H) and E. are disjoint sets in B, that is f £ B~ j s j
unambiguously (by finite additivity of m) define the integral
n f f (s)m(ds) = ~. m(mj)pj.
S j=l
% (H). Then we may S
The question, of course, is to what class of functions can we properly extend the definition of the integral? Now if m has finite total variation |m (s), then the map f ÷ ff(s)m(ds) is continuous-for the supremum norm f ~ = sup f(s) It r on
B ® T (H), so that by continuity the integral map extends to a continuous linear S
map from the closure M(S,Ts(H)) of B Q Ts(H) with the I'I norm into T(H). In
particular, the integral ff(s)m(ds) is well-defined (as the limit of the integrals
S
of uniformly convergent simple functions) for every bounded and continuous function f: S ÷ T (H). Unfortunately, it is not the case that an arbitrary POM m has finite
s total variation. Since we wish to consider general quantum measurement processes as represented by POM's (in particular, resolutions of the identity), we can only assume that m has finite scalar semivariation m(S) < +0% Hence we must put stronger restrictions on the class of functions which we integrate. The answer is summarized in
Theorem 3.2 Let S be a locally compact Hausdorff space with Borel sets~. Let H be a
Hilbert space. There i~ an isometric isomorphism LI<~ m<->L 2 between the bounded
linear maps LI; Co(S) ~ ~ %(H) + T(H), (1) the operator-valued regular Borel
measures m E M(B,L(T(H),T(H))), and the bounded linear maps L2: C (S) + L(T(H), O
T(H)). The correspondence L I ~ m ~ L 2 is given by the relations
(i) For notation and facts regarding tensor products we follow Treves [ 7 ].
132
L1(f) = ~f(s)m(ds) f £ C (S) ~ T(H) S ' o
L2(g)p = Ll(g(')p) = plg(s)m(ds), g e Co(S), p E T(H)
and under this correspondence L I = m(s) = L 2 . Moreover the integral~f(s)m(ds) S
is well-defined for every f M(S) ~ T(H) and the map~/f(s)m(ds) is bounded and S
linear from M(S) ~ T(H) into T(H). ~
Corollary 3.3 If m e M(8, L s (H)) then the integral ~f(s)m(ds) is well-defined for every
S f E M(S) ~ ~(H).
In proving Theorem 3.2 we need the fact
Proposition 3.3
M(S) O~ T(H) is a subspace of M(S,T(H)). ~
Remark The above says that we may identify the tensor product space M(S) ~ Ts(H) with
a s u b s p a c e o f t h e t o t a l l y m e a s u r a b l e f u n c t i o n s f : S * T (H) i n a w e l l - d e f i n e d w a y . s ^
The reason why this is important is that the functions f ~ M(S) ~ T (H) are those s
for which we may legitimately define an integral /f(s)m(ds) fur arbitrary operator- S
valued measures m ~ ~,~s(H)), since f~f f(s)m(ds) is a continuous linear map from S
M(S) ~ I(H) into ~(H). In particular, it is obvious that Co(S) ~ ~ T(H) may be
i d e n t i f i e d w i t h a s u b s p a e e o f c o n t i n u o u s f u n c t i o n s f : S I - . . ' ~ '~ (H) i n a w e l l - d e f i n e d s
way, just as it is obvious how to define the integral f f(s)m(ds) for finite linear S
c o m b i n a t i o n s n f(s) = j~= =I gj(s)0j a Co(S) × Ts(H). What is not obvious is that
the completion of C (S) ~ Ts(H) in the tensor product norm ~ may be identified o
with a subspaee of continuous functions f: S + T (H). s
4. A FUBINI THEOREM FOR THE BAYES POSTERIOR EXPECTED COST.
In the quantum estimation problem, a decision strategy corresponds to a probability operator measure m ~ M(B,Ls(H)) with posterior expected cost
R =f tr[p(s)fC(t,s)m(dt)]~(dt) m S S
where for each ~ p(s) specifies a state of the quantum system, C(t,s) is a cost function, and ~ is a prior probability measure on S. We would like to show that the order of integration can be interchanged to yield
R = tr~ f(s)m(ds) m S
133
where
f(s) ~C(t,s)O(t)~(dt) S
is a map f: S ~ T (H) that belongs to the space M(S) ~ E(H) of functions inte- s
grable against operator-valued measures.
Let (S,B,~) be a finite nonnegative measure space, X a Banach space. A fun- tion f: S ~ X is measurable iff there is a sequence {f } of simple measurable
n functions converging pointwise to f, i.e. f (s) ~ f(s) for every s e S. A useful
n criterion for measurability is the following: f is measurable iff it is separably-
valued and for every subset V of X, f-l(V) e B. In particular, every f ~ C (S,X) o
is measurable, when S is a locally compact Hausdorff space with Borel sets B. A function f: S + X is integrabie iff it is measurable and /If(s) l.~(ds) < +~,
S in which case the integral /f(s)N(ds) is well-defined as Bochner's integral; we
denote by LI(S,B,~;X) the space of all integrable functions f: S ~ X, a normed
space under the L 1 norm Ifll =~ If(s)l~(ds). The uniform norm I'I~ on functions S
f: S + X is defined by IfI~ = suplf(s)l; M(S,X) denotes the Banach space of all seS
uniform limits of simple X-valued functions, with norm I'I~, i.e. M(S,X) is the
closure of the simple X-valued functions with the uniform norm. We abbreviate M(S,R) to M(S).
Proposition 4.1 Let S be a locally compact Hausdorff space with Borel sets B, ~ a probability
measure on S, and H a Hilbert space. Suppose p: S ~ TS(H) belongs to M(S,Ts(H)),
and C: S × S + R is a real-valued map satisfying
t ÷ C(t,.) s LI(S,B,~;M(S)).
Then for every s ~ S, f(s) is well-defined as an element of • (H) by the Bochner s
integral
f<s) =/C(t,s)~(t)~(dt); S
moreover f E M(S)~ Ts(H) and for every operator-valued measure m g M(B,Ls(H)),
we have
f f(s)m(ds) = /p(t)[fC(t,s)m(ds)]~(dt) S S S ^
Moreover if t ~ C(t,-) in fact belongs to LI(S,B,~;Co(S)) then f £ Co(S) O n ~s(H).~
5. THE QUANTUM ESTIMATION PROBLEM AND ITS DUAL.
We are now prepared to formulate the quantum detection problem in a duality framework and calculate the associated dual problem. Let S be a locally compact Hausdorff space with Borel sets B. Let H be a Hilbert space associated with the physical variables of the system under consideration. For each parameter value s E S let 0(s) be a state or density operator for the quantum system, i.e. every p(s) is a nonnegative-definite selfadjoint trace-class operator on H with trace i;
134
we assume p £ M(S,Ts(H)). We assume that there is a cost function C: S x S + R,
where C(s,t) specifies the relative cost of an estimate t when the true parameter value is s. If the operator-valued measure m E M(B,L (H)) corresponds to a given
S
measurement and decision strategy, then the posterior expected cost is
R = trfp(t) [~C(t,s)m(ds)]~, m
S s w h e r e B i s a p r i o r p r o b a b i l i t y m e a s u r e o n ( S , B ) . By Proposition 4.1
this is well-defined whenever the map t + C(t,') belongs to LI(S,B,~;M(S)), in
which case we may interchange the order of integration to get
(5.1) R m = trff(s)m(ds) S
where f ~ M(S) ~ -- Ts(H) is defined by
f(s) = I~(t)C(t,s~(ds). S
The quantum estimation problem is to minimize (5.1) over all operator-valued measures m ~ M(B, Ls(H)) which are POM's i.e. the constraints are that m(E) _> 0
for every E ~ B and m(S) = I.
We formulate the estimation problem in a duality framework. We take pertur- bations on the equality constraint m(S) = I. Define the convex function F: M(B, Ls(H)) ÷ R by
F(m) = ~>o(m) + trf f(s)m(ds), m ~ M(B,Ls(H)) , -- S
where 6>o denotes the indicator function for the positive operator-valued measures,
i.e. 6>~(m) is 0 if m(B) C Ls(H) + and += otherwise. Define the convex function
G: i ( l l ) + R by S
G(x) = 6#o}(X) x e i (H) S
i.e. G(x) is 0 if x = 0 and G(x) =+~if x # 0. Then the quantum detection prob- lem may be written
e = inf{F(m) + G(l-em): m e M(B,L (H))} o S
where L: M(B, Is(H)) + Ls (H) is the continuous linear operator
e(m) = m(S).
We consider a family of perturbed problems defined by
P(X) = inf{F(m) + g(x-Lm): m g M(B,Ls(H))} , x e i (H). S
Thus we are taking perturbations in the equality constraint, i.e. the problem P(x) requires that every feasible m be nonnegative and satisfy m(S~ = x; of course, P = P(1). Since F and G are convex, P(-) is convex L (H) ÷ R. O S
135
In order to construct the dual problem corresponding to the family of per- turbed problems P(x), we must calculate the conjugate functions of F and G. We shall work in the norm topology of the constraint space Ls(H) , so that the dual
problem is posed in i (H) ~. Clearly G* E 0. The adjoint of the operator L is s
given by
L*: is(H)* ÷ M(B, Ls(H))*: y + (m ÷ y'm(S)).
To calculate F*(L*y), we have the following lemma.
Lemma 5.1 Suppose y s is(H)* and f s M(S) ~ ~s(H) satisfy
(5.2) y.m(S) ! trf f(s)m(ds) S
for every positive operator-valued measure m £ M(B, is(H)+). Then Ysg ~ 0 and
Yac ~ f(s) for every s ~ S, where y = Yac + Ysg is the unique decomposition of y
into Yae e Ts(H) and Ysg £ Ks(H) "~
Proposition 5.2 The perturbation function P(.) is continuous at I, and hence ~P(1) # ~. In
particular, P = D and the dual problem D has optimal solutions. Moreover every O O O
solution^ ^ y ~ Ls(H)~ of the dual problem Do has 0 s ingu la r p a r t , i . e . Ysg = 0 and
Y = Yac belongs to the canonical image of Ts(H) in Ts(H)**. ~
In order to show that the problem P has solutions, we could define a family O
of dual perturbed problems D(v) for v e Co(S) O Ts(H) and show that D(') is con-
tinuous. Or we could take the alternative method of showing that the set of feasible POM's m is weak* compact and the cost function is weak*-isc when M(B, Ls(H)) ~ L(C (S) Ls(H)) is identified as the normed dual of the space
A O
Co(S) ~ Ts(H) under the pa i r ing
<f,m> = trff(s)m(ds).
Note that both methods require that f belong to the predual C (S) @w Ts(H)); it o
suffices to assume that t + C(t,-) belongs to LI(S,B,~;Co(S)).
Proposition 5.3 % The set of POM's is compact for the weak ~ E w(M(B,Ls(H)) , Co(S) Ts(H))
topology. If t ~ C(t,.) e LI(S,B,P;Co(S)) then Po has optimal solutions m.~
The following theorem summarizes the results we have obtained so far, as well as providing a necessary and sufficient characterization of the optimal solution.
MAIN THEOREM.
Let H be a Hilbert space, S a locally compact Hausdorff space with Borel sets B. Let 0 e M(S,Ts(H)), C: S × S ÷ R a map satisfying t + C(t,')ELI(S,B,~;Co(S)),
and ~ a probability measure on (S,B). Then for every m g M(B,Ls(H)),
136
trf p(t)[f C(t,s)m(ds)]~(dt) = trf f(s)m(ds) S S S
where f ~ Co(S) ~ %s(H) is defined by
f(s) = fP(t)C(t,s)~(ds). S
Define the optimization problems
Po = inf{trf f(s)m(ds) : m E M(B, Ls(H)), m(S) = l,m(E) _> 0 for every E E B} S
Do = sup{try: y g Ts(H) , y _< f(s) for every s C S}.
Then P = Do, and both P and D have optimal solutions. Moreover the following O o O
statements are equivalent for mgM(B,Ls(H)), assuming m(S) = I and m(E) ~ 0 for
every E g B:
1) m solves P o
2) ff(s)m(ds) ~ f(t) for every t c S S
3)~m(ds)f(s) < f(t) for every t g S. S
Under any of the above conditions it follows that y = f f(s)m(ds) =~ m(ds)f(s) S S
is selfadjoint and is the unique solution of D , with O
P = D = tr(y). o o
REFERENCES
i. S.K. Mitter and S.K. Young: Quantum Detection and Estimation Theory, to appear.
2. A.S. Holevo: Statistical Decision Theory for Quantum Systems, J. Multivariate Analysis, Vol. 3, pp. 337-394, 1973.
3. A.S. Holevo: The Theory of Statistical Decisions on an Operator Algebra, Dokl. Akad. Nauk SSSR, Tom 218, no. i, pp. 1276-1281, 1974.
4. H. Yuen, R. Kennedy and M. Lax: Optimum Testing of Multiple Hypotheses in Quantum Detection Theory, IEEE Trans. on Information Theory, IT-21, 1975, p. 25.
5. E.B. Davies: Quantum Theory of Open Systems, Academic Press, New York, 1976.
6. A.S. Holevo: Loc. cit.
7. F. Treves: Topological Vector Spaces, Distributions and Kernels, Academic Press, New York.
8. I. Ekeland and R. Teman: Analyse Convexe et Probl~mes Variationnels, Dunod-Gauthier-Villars, 19/4.
ADAFIIVE CONTROL
CONTROLE ADAPTIF
PIECE-WISE DETERMINISTIC SIGNALS
K. J. Astr~m
Department of Automatic Control Lund I n s t i t u t e of Technology
S-220 07 Lund 7, Sweden
I . INTRODUCTION
Mathematical models of disturbances are important in control theory. Suitable
models should al low some degree of r egu la r i t y , but to be r e a l i s t i c the model should
not al low the disturbances to be predicted exact ly . This means, fo r example, that
ana ly t i c funct ions are not su i tab le because an ana ly t i c signal can be predicted
prec ise ly provided that i t is known over an a rb i t r a r y small i n te rva l . Stochastic
processes have been used wi th great success as models fo r disturbances and have
inspired the development of stochast ic control theory. In many cases i t i s , however,
unnatural to model disturbances as stochast ic processes. Changes of set points in
process control systems is a typ ica l example. The set point is normally kept constant
over long periods with occasional changes. There is no pa r t i cu la r pattern to the
changes. They may appear regu la r l y or i r r e g u l a r l y . The amplitudes of the changes may
vary subs tan t ia l l y . In th is paper the notion of piece-wise determin is t ic signals is
introduced to capture the essent ial feature of signals which behave l i ke set point
changes. The signals can also be used to describe typ ica l load changes fo r i ndus t r i a l
processes, which are characterized by large upsets which occur i r r e g u l a r l y . The
signal class is defined in Section 2. Some of the propert ies of the signals are also
discussed in that sect ion. Predict ion of piece-wise determin is t ic s ignals are d is-
cussed in Section 3. I t is shown that the signals can be predicted in almost the
same way as stochast ic s ignals. Piece-wise determin is t ic signals are useful to
analyse determin is t ic adaptive control systems. In Section 4 an example is given
which i l l u s t r a t e s how the signals can be used in th i s context.
2. PIECE-WISE DETERMINISTIC SIGNALS
Innovations presentations play a central ro le in the theory of stochast ic
processes. In such representations a signal is represented as the output of a dyna-
mical system whose input is a white noise process. S im i la r l y in determin is t ic control
theory i t has been common to use signals which are solut ions to determin is t ic d i f f e -
rence or d i f f e r e n t i a l equations. The piece-wise determin is t ic signals share
propert ies both with stochast ic processes and wi th determin is t ic s ignals. They are
described as solut ions to ordinary d i f ference equations over cer ta in time in te rva ls ,
but they change in an unpredictable manner at cer ta in iso la ted points. The changes
are such that there are d i scon t i nu i t i es only in the highest d i f ference that appears
139
in the difference equations.
A formal definit ion is now given. The properties are stated for the discrete
time case only. I t is thus assumed that time is the set of integers. Furthermore le t
Ti(~ ) be a set of discrete integers
Ti(C ) = { . . . . t_ l , t o • t I . . . . }
such that
m in ( t i + 1 - t i ) = ~ > 1
and l e t Tr(~ ) be the complement of Ti (~ ) with respect to a l l in tegers. The points in
Ti(~ ) are obviously i so la ted . Introduce
DEFINITION l
Let Q(q-l) be a polynomial of degree n < & in the backward shi f t operator. A
signal y is called a piece-wX~e d~te:unin/~%~c signal of degree n and index ~ i f
Q(q-l) y( t ) = 0 i f t E Tr(~ ) ( I )
and
Q(q-l) y( t ) ~ 0 i f t E Ti(~ ) (2)
D
The polynomial Q(q-l) is called the gene~utor of the signal. The set Tr(~ ) is
called the set of regular points and Ti(~ ) is called the set of /:u~cgu~aJt points. The irregular points are at least ~ units apart, where ~ is the index of the signal.
The index is thus a measure of how irregular the signal is. The smaller ~ is, the
more irregular is the signal. I t follows from (1) that a piece-wise deterministic
signal can be predicted exactly in an interval that does not contain any irregular
points. This is shown in detail in the following.
The name piece-wise deterministic signal is chosen because of i ts s imi lar i ty to
the word completely de te rm in i s t i c s tochast ic process. In the ear l y l i t e r a t u r e on
stochast ic processes a process was ca l led completely de te rm in i s t i c or s ingu lar i f
I ) holds fo r a l l t . See e.g. Wold (1954).
In analogy with the terminology f o r random processes the signal v defined by
~ ( t ) = Q(q- l ) y ( t ) , (3)
Is called the i n n o v a t i o n .
The generator of a piece-wise deterministic signal is essentially unique. To
see this, assume that a signal y has two generators Ql and Q2" Let v I and v2 denote
the associated innovations. Equation (3) gives
Ql Vl = Q1 y =~ - -v2 "
" 2
140
Since Vl and ~2 are nonzero only at the i r regu lar points, i t fol lows that Q1 = k'Q2"
Simple examples of piece-wise determinis t ic signals are given below.
EXAMPLE I
A piece-wise constant signal has the generator
Q = I .q-l. The set of i r regu lar points are a l l the points where the signal changes leve l .
EXAMPLE 2
A piece-wise l inear signal has the generator
Q = l - 2 q -l+q-2 The set of i r regu lar points are a l l points such that the change of slope is immedi-
a te ly to the l e f t of the points. See Fig. I .
i x l x I 0 5 10 15 20 25
Figure I. A piece-wise l inear signal and i ts set of irregular points.
3. PREDICTION OF PIECE-WISE DETERMINISTIC SIGNALS
I t fol lows from ( I ) that a piece-wise determinist ic signal can be predicted
exact ly one step ahead except at the i r regu lar points. The general k-step predictor
for a piece-wise determinis t ic signal is given by
THEOREM I
Consider a piece-wise determin is t ic signal y with generator Q. Let F and G be
polynomials of degrees k-I and deg Q-I which are the unique solutions of
1 = F(q - I ) Q(q-l) + q-k G(q- l ) . (4)
Assume that
> k + deg Q. (5)
141
Then the k-step predictor of y is given by
~ ( t l t - k ) = G(q " I ) y ( t - k ) (6)
and the predic t ion er ror is
e ( t ) = y ( t ) - y ( t l t - k ) = F(q - I ) v ( t ) (7)
where v is the innovation of the signal y.
Proof:
Under the condition (5) i t follows from (2) and (4) that
[ l -q -kG(q- l ) ] y(t) = F(q - l ) Q(q-l) y(t) = O.
This implies that the predictor is given by (6). The prediction error is
e(t) = y(t) - y ( t l t - k ) = [l -q-kG(q-l)] y(t) = F(q - l ) Q(q-l) y( t ) ,
where the second equality follows from (4). The formula (7) now follows from (3).
G
Notice that i t follows from Theorem I that the prediction error is different
from zero at the irregular points and at their k-l r ight successors. Notice also the
similarit ies between this result and the corresponding result for predicting ARMA
processes. See e.g. Astr~m (1970).
Simple examples of predictors are given in the following example.
EXAMPLE 3
A piece-wise constant signal has the generator Q = l - q - l . Simple calculations
give F(q -1) = 1 + q-1 + . . . + q-k+l
G(q- l ) = q-k.
The pred ic tor is thus ~ ( t l t - k ) = y ( t - k ) , o
EXAMPLE 4
A piece-wise linear signal has the generator Q = l - 2q -l
lations give
F(q -1) = 1 + 2q -1 + . . . + kq -k+l
G(q - l ) = k + 1 - kq -I
The predictor is thus
~( t I t -k) = Y(t-kl + k [ y ( t - k ) - y ( t - k - l ) ] .
+ q-2 Simple calcu-
The problem of predicting a piece-wise deterministic signal can be formulated
as the problem of finding a causal operator R such that
142
Ry(t) = O.
I t follows from Definition l that R = Q makes Ry = 0 except at the irregular points.
Since R v ( t ) Ry(t) =
and v ( t i ) • 0 i t fo l lows that the signal y can not be predicted exact ly by a rat ional
predic tor of order less than 4. I f the order of the operator is larger than ~ i t may
be possible to predic t the s ignal . This is i l l u s t r a t e d by the fo l lowing example.
EXAMPLE 5
Consider a square wave with period 2p. The signal can be predicted using the
predictor
~(t+lJt) = y( t ) .
This predictor gives the correct prediction except at those points where the square
wave changes level. The square wave can thus be regarded as a piece-wise determin-
is t ic signal with generator Q = l - q - l . The square wave can, however, also be pre-
dicted exactly by the predictor
~ ( t+ l J t ) = y ( t ) - y ( t - p+ l ) + y ( t -p )
which requires that p + l past values of the signal are stored. This means that
( l _ q - l ) ( l +q-P) y ( t ) = 0 v t .
The square wave can thus also be regarded as a purely determin is t ic s ignal , a
4. APPLICATIONS TO ANALYSIS OF ADAPTIVE SYSTEMS
I t w i l l now be shown how the piece-wise determin is t ic signals can be used in
the analysis of adaptive systems. Consider the closed loop system shown in Fig. 2~
where the se l f - tun ing regulator STURE described in Astr~m and Wittenmark (1973) is
used in a simple command servo loop.
STURE
~ n
J ~'l P r o c e s s
- I
Y
Figure 2. Block diagram of a se l f - tun ing regulator in a single-degree-of-freedom conf igurat ion.
I t is assumed that only the control error e is measured and that Yr and y are not
available separately. This case is called a single-degree-of-freedom system by
143
Horowitz (1963). I t is not the best way to handle command inputs. The behaviour of
the system shown in Fig. 2 is well understood i f the reference value Yr is zero and
i f the process disturbance n is a stochastic process. See Astr~m et al. (1977). The
case when the reference value Yr is a piece-wise deterministic signal wi l l now be
discussed.
Let the process be described by
y(t) - B(q-1) u(t-k), A(q -I )
where A and B are polynomials
A(q-I alq-] -n ) = 1 + + . . . + anq ,
B(q - l ) = b 0 + blq' l + . . .+ bmq-m, b 0 ¢ 0
in the backward shi f t operator q-l. The closed system obtained when the self-tuning
regulator is connected to the process can be described by the equations
e( t+ l ) = e( t ) + P(t+ l ) ~ ( t - k+ l ) ~( t+ l )
c ( t+ l ) = - e ( t+ l ) + ~ T ( t - k + l ) [ e ( t - k + l ) - e ( t ) ]
P - l ( t + l ) = ~P- l ( t ) + ~ ( t - k+ l ) ~T( t -k+ l ) (9)
A(q - l ) e( t ) + B(q - l ) u ( t -k ) = A(q - I ) Y r ( t )
A(q - I ) e( t ) - B(q - I ) u ( t ) = O.
where
~(t) = [ e ( t ) . . . e ( t - r + l ) u( t - l ) . . . u ( t - s ) ] T
A(q-l) = _ e l ( t ) _ e2(t ) q-I _ . . . - Or(t ) q-r+l
m1(t ) + m2(t ) q-I + . . . + mr(t ) q-r+l
B(q-l) = BO + er+l (t) q-l + . . .+er+s( t ) q-S
BO + Bl(t ) q-l + . . . + Bs(t ) q-S.
The nonlinear equations (9) are fa i r l y complex. The following result on the
properties of stationary solutions gives insight into the behaviour of the closed
loop system.
THEOREM 2
Consider the system (9). Let the command signal be piece-wise deterministic
with generator Q. Let
deg A ~ deg A + deg Q - deg Icd [A,Q] - l
deg B ~ k + deg B + deg Q - deg Icd [ A , Q ] - l
where lcd [A,Q] is the largest common divisor of A and Q and
144
> deg A + deg B + 2k.
Assume that there is a so lu t ion to (9) such that e ( t ) is constant. Then the value
e 0 is such that
( i ) and
( i i )
Q div ides A and B div ides G
A _ AG B BFG "
The theorem is proven by analysing the consequences of 8 ( t ) being constant. A
f u l l proof is given in Astr~m and Gustavsson (1978).
For constant e the se l f - t un ing regulator reduces to a l i nea r time inva r ian t
system with the t rans fer funct ion A/B. The condi t ion ( i i ) of Theorem 2 implies that
A div ides A and that B div ides 8. This means that the l i m i t i n g regulator is such
that a l l poles and zeros of the process are cancelled by poles and zeros of the
regulator . This is not a desirable s i tua t ion which indicates that the conf igurat ion
shown in Fig. 2 w i l l not work we l l .
The condi t ion that Q div ides A implies that the dynamics which generates the
command signal must be included in the process dynamics. For example to have a
s ta t ionary so lu t ion when the command signal is piece-wise constant the process must
contain an in tegra to r . This can of course be achieved by a su i tab le precompensator.
To design a precompensator i t i s , however, necessary to know the generator of the
command signal a p r i o r i .
ACKNOWLEDGEMENTS
This work has been p a r t i a l l y supported by the Swedish Board of Technical
Development under contract 76-3804. I have had several useful discussions on th i s
topic wi th Dr I . Gustavsson and Dr J. Sternby.
REFERENCES
Astr~m, K.J. (1970): In t roduct ion to Stochastic Control Theory. Academic Press, New York and London.
Astr6m, K.J. and B. Wittenmark (1973): On Sel f Tuning Regulators. Automatica 9, 185-189.
Astr6m, K.J. , U. Borisson, L. Ljung, and B. Wittenmark (1977): Theory and Appl ica- t ions of Self-Tuning Regulators. Automatica 13, 457-476.
Astr~m, K.J. and I . Gustavsson (1978): Analysis oT-a Sel f - tun ing Regulator in a Servoloop. Report, Dept of Automatic Control, Lund I n s t i t u t e of Technology, Lund, Sweden.
Horowitz, I.M. (1963): Synthesis of Feedback System. Academic Press, New York and London.
Wold, H. (1954): A study in the Analysis of Stat ionary Time Series. Almqvist & Wiksel l , Stockholm.
ADAPTIVE CONTROL OF MARKOV CHAINS
V. Borkar and P. Varaiya
Department of Electrical Engineering and Computer Sciences and the Electronics Research Laboratory
University of California, Berkeley, CA 94720 U.S.A.
ABSTRACT
Consider a controlled Markov chain whose transit ion probabil i t ies depend upon
an unknown parameter ~ taking values in a f i n i t e set A. To each ~ is associated a
prespecified stationary control law @(a). The adaptive control law selects at each
time t the control action indicated by @(a t) where a t is the maximum likel ihood
estimate of s. I t is shown that s t converges to a parameter s* such that the tran-
sit ion probabil i t ies corresponding to a* and @(~*) are the same as those correspond-
ing to a 0 and ¢(s*) where a 0 is the true parameter.
INTRODUCTION
We consider a controlled Markov chain x t , t=O, l . . . . taking values in a f i n i t e
set I = {1,2 . . . . . I } . The transit ion probabil i ty at time t depends upon the control
action u t taken at t and upon a parameter a,
Prob{xt+ 1 = j l x t = i } = p ( i , j ;u t ,s ) .
At each time t , x t is observed and based upon its value u t is selected from a 0 prespecified set U. The parameter ~ has the constant value a which is not known in
advance; i t is known, however, that s O belongs to a fixed f i n i t e set A.
The selection of u t is to be made so as to guarantee that the resulting state
process performs sat isfactor i ly. A precise specification of the performance index
is not at the moment of interest; however, since the transit ion probabil i t ies depend
on a so wi l l the performance. We assume given a stationary control law @(a,-) for
each a in A, with the understanding that i f the parameter value of m is ~ say, then
the control action chosen according to the rule u t = @(G,x t) gives satisfactory per-
formance. In a later section we examine the situation when @(m,-) is the law which
minimizes the long run expected cost per unit time for a specific cost function.
We consider the following adaptive control law. At each time t the maximum
likelihood estimate a t of the unknown parameter is made,
t - l Prob{x 0 . . . . . xtlxO,u 0 . . . . . Ut_l ,a t} = g P(Xs,Xs+l;Us,at)
s=O t-1
~ g P(Xs,Xs+l;Us,a) , fo r a l l a ~ A. (1) s=O
In case the l ikelihood function is maximized at more than one value of s, then a
unique value is assumed to be picked according to some fixed pr ior i ty ordering.
Having determined s t according to (1), the control action is selected to be
u t = @(at,xt). The resulting 'closed loop' system is diagrammed below.
146
U t
*1 ' x l ; Ul 'eO)
u t
Xt÷l UNIT I x t
DELAY I
a t I MAX. LIKELY I
@(~,x t
Fig. I . The closed loop adaptive control system
Our objective is to analyze the asymptotic behavior of s t and u t . The d i f f i - culty in the analysis stems from two sources. F i rs t , since the estimate s t wi l l change over time the t ransi t ion probabi l i t ies Prob{Xt+llX t} wi l l not be stationary. The second d i f f i cu l t y is more subtle and has to do with i den t i f i ab i l i t y . Consider the following condition introduced by Mandl.
I d e n t i f i a b i l i t y Condition. For each s ~ ~ ' , there exists i E I so that
[ p ( i , l ; u , ~ ) . . . . . p ( i , I ; u , ~ ) ] ~ [ p ( i , l ; u , ~ ' ) . . . . . p ( i , I ; u , ~ ' ) ] , for a l l u E U. (2)
In a remarkable paper [ l ] , Mandl has shown that i f this condition holds, then s t
converges to s 0 almost surely for any control law u t = St(x0 . . . . . x t ) . This resul t
suggests that the i d e n t i f i a b i l i t y condition might be too res t r i c t i ve in some prac-
t ical si tuat ions. To see this consider the fami l ia r Markovian system
xt+ l = ax t+bu t + v t , t = O,l . . . . . (3)
where x t ~s a real-valued variable and the v t are i . i . d , random variables. The
unknown parameter is s = (a,b). Then, for the l inear control law u t = -gx t and two
parameter values s = (a,b) and ~' = ( a ' , b ' ) such that alb = a ' Ib ' = g,
P(Xt,Xt+l;Ut=-gxt,s) = P(Xt,Xt+l ;Ut=-gxt,~ ') , for a l l x t , xt+ l
and so (2) cannot hold. Indeed simulation examples [2,3] indicate that i f in (3)
the adaptive control law u t = -g tx t is used, where gt = - a t l b t and s t = (at ,b t) is
the parameter estimate, then s t may not converge, and i f i t does, i t need not con-
verge to the true value.
In this paper we study the asymptotic behavior of s t and u t when (2) may not
hold. Our main result is that with probabi l i ty one s t converges to a random varia-
ble s* such that
p l i , j ; $ ( ~ * , i ) , ~ * ] = pCi, j ;$(~*, i ) ,~O~, for al ; i , j . (4)
Therefore, asymptotically, the t ransi t ion probabi l i t ies of the closedloop system are
147
the same whether the parameter is m* or 0 so tha t the two parameters are i n d i s t i n -
guishable. I t would seem there fore tha t as f a r as parameter i d e n t i f i c a t i o n is con-
cerned (4) is the best r e s u l t t ha t can be expected under the proposed adapt ive con-
t ro l Iaw.
ASSUMPTIONS
I . There is ~ > 0 such tha t fo r every i , j e i t he r p ( i , j ; u , ~ ) > ~ fo r a l l u, ~, or
p ( i , j ; u , m ) = 0 fo r a l l u, m.
2. For every i , j there is a sequence i I . . . . . i r such tha t f o r a l l u, m,
P ( i s_ l , i s ;U ,m) > O, s = 1 . . . . . r+ l where i 0 = i , i r + 1 = j . The f i r s t assumption guarantees tha t the p r o b a b i l i t y measures Prob{x 0 . . . . . x t l m ,
Xo,Uo,U I , . . . . U t_ l } , m E A are mutua l l y abso lu te l y cont inuous. Since the es t imat ion
procedure w i l l , in f i n i t e t ime, e l im ina te from fu tu re cons idera t ion those parameter
values which do not y i e l d a measure w i th respect to which the measure induced by s 0
is abso lu te ly cont inuous, there fo re t h i s assumption is not r e s t r i c t i v e . The second
assumption guarantees tha t the Markov chain generated by the t r a n s i t i o n p r o b a b i l i t i e s
p ( i , j ; @ ( m , i ) , i ) has a s ing le ergodic c lass . Some such cond i t i on is c l e a r l y needed
fo r i d e n t i f i c a t i o n .
PARAMETER ESTIMATION
Sample po in ts are denoted by m. When e x p l i c i t dependence on w is to be empha-
sized we w i l l w r i t e x t ( w ) , s t (m) , u t (~) = ~ ( ~ t ( ~ ) , x t ( ~ ) ) , e t c . Let
~ t (~ ,~) = P(Xt,Xt+ l ; u t , a ) [ p ( x t , x t + l ; u t , ~ O ) ] - l , t - I
L t (~ ,~ ) = II ~ (~,~) s= 0 s
so tha t , from ( I ) ,
L t I~t (m),m ) ~ L t (~ ,~ ) , ~ E A. (5)
Let A*(m) be the set o f l i m i t po in ts o f {~ t (m) } . Note tha t s ince A is f i n i t e ,
~t(m) E A*(m) a f t e r some f i n i t e t ime. The next r e s u l t is s i m i l a r to t ha t o f Baram-
Sandell [ 4 ] .
Lemma I . There is a set N w i th zero measure such tha t f o r m ~ N
~t(mt(m),m) : ~t lmt+l(m),m) : I , t ~T(m) (6)
P ( X t , X t + l ; U t , ~ * ) = P (X t ,X t+ l ;U t ,~O) , t ~ T ( ~ ) (7)
fo r every m* E A*(m), f o r some f i n i t e T(m).
Proof. For each m, Lt(m) is a pos i t i ve mar t inga le w i th ELt(m) = I . By the
semi-mart ingale convergence theorem [5 ,§29.3 ] there is a random va r i ab le L(m) > 0
and a set N w i th zero measure so tha t f o r m ~ N
l im L t (~ ,~) = L(~,~) . (8)
148
Fix m # N and m* E A(m). Let t k, k = 1,2 . . . . be a sequence such that
From (5) we know that
Ltk(~tk) ~ L tk (~ tk+ l ) ,
hence (9) implies that
Ltk(~tk) = L tk (~ tk+ l ) ,
Ltk+l(mtk+l) L Ltk+l (mtk) ,
Ltk+l(mtk) = Ltk+ l (mtk+l ) ,
i . e . both s t and s t +I maximize the l i ke l ihood funct ion at t k, t k+ l . Since in case k of a t i e we ~ave assumed that a unique value is selected according to some f ixed
p r i o r i t y ru le , therefore ~tk = ~tk+l and so ~* = ~*. []
Corol lary I . There is a random var iable m*(m) such that for m~ N, mt(m) =
~*(~), Ut(~) = ~[~*(~),Xt(~)], for t ~ T(~).
Proof. Since mt(m) E A*(m), t ~ T(m) i t is enough to show that A*(m) contains
only one element. I f i t does not, then there must be a sequence t k and two distinct
elements m*, B* such that mtk(m) = m*, mtk+l(m ) = B*. But this contradicts Lemma 2. []
From (7) and Corol lary 1 we already have for m ~ N
P(Xt,Xt+ I;@( , , x t ) , , ) = P~Xt,Xt+ I ;~( , , x t ) , 0] , t ~ T. (I0)
Hence to prove the main resu l t (4) i t must be shown that the process x t goes through
Let N = u N .
~tk(m) = ~* for a l l k. From (5)
Ltk (~tk(m) ,m] : Ltk(~*,~) ~ Ltk(~O,~) = I ,
and so i t fol lows from (8) that
l im Lt(~*,m) = L(~*,m) > O.
Since ~t(~*,m) = L t+ l (~ * ,~ ) [ L t (~ * ,~ ) ] - I i t fol lows that l im ~t(~*,m) = I . Since ~t(~*,~) takes on only a f i n i t e number of values, therefore a f te r some T(~*,m) < ~,
~t(~*,m) = I , t ~ T(~*,m)
which c lear ly implies (7). Also since ~t(~) E A*(m) a f ter some f i n i t e time Tl(m), therefore
Lt(mt(m),m] : I , ~t(mt+l (m) ,m] : l ; t LT(m)
where T(~) = max{TI (~) ,T(~* ,~) I~*EA*(~) } . The lemma is proved. []
mtk = B* Lemma 2. Fix m~ N, and l e t t k be a sequence such that mt = m*' +I for a l l k for some m*, B* in A*(m). Then m* = F*. k
Proof. Suppose without loss of genera l i ty that t k > T(m). Then from (6) ,
1 = ~ t k (a t k+ l ) [ ~ t k (~ t k ) ] - I
= Ltk+l(~tk+l)[Ltk(atk+l)]-ILtk(~tk)[Ltk+l(~tk)]-l. (9)
149
all possible transitions. We wi l l need our assumptions to prove this.
Lemma 3. For each i the set l t - l
M i ~ {~l l im ~ s ! o l [×s(m)=i) =o}
has zero measure. Here I ( - ) is the ind ica to r funct ion of ( - ) .
Proof. Let __~t be the a - f i e l d generated by X o , . . . , x t . By the S t a b i l i t y Theo-
rem of [5,§29.1] 1 t - I
l im ~ s ! O [ I ( x s : i ) - E { I ( x s : i ) [ L C ~ s _ I } ] : 0 almost sure ly , and so
1 t - I N i = { ~ l l i m = - Z E{ l (xs= i ) IC~s_ l } =0} ( I I )
l: S= 0
differs from M i by a set of zero measure. Now
E{l(xs=i)IC~'s_l } = Z P(k,i;Us_l,~0)l(Xs_l =k) kEI
> ~ Z )l (IZ) - kES(i ( X s ' l : k ) '
where the equa l i t y fo l lows from the Markovian proper ty , and the i nequa l i t y from
Assumption 1 where S( i ) = { k l p ( k , i ; { ( ~ , k ) , ~ O) >0} . From ( I I ) , (12) we see tha t
1 t - I l im~s~ol(Xs(~)=k~ = O, mE N i , k ~ S ( i ) .
We can repeat the argument, t h i s time wi th k in place of i , and obtain a set M k
d i f f e r i n g from M i by a zero measure set such tha t
i t - I l im#s !O l (xs(m):m ) : O, m E M k, m E S(k).
Proceeding in th is way we see by Assumption 2 that there is a set M which d i f f e r s
from M i by a zero measure set such that
1 t - I lim~sZ__oIIXs(m):j ) : 0, m E M, j E I. (13)
But since x s E I,
Z I IXs(m)=J) --- l jEI
so that (13) holds only i f M has zero measure. The lemma is proved. [ ]
Lemma 4. For each i , j such tha t p ( i , j ; u , m ) > 0 the set
1 t - I Mij = { ~ l l i m ~ s ! O 1 (x s (~ )= i ,xs+ l ( ~ )= j ) = O}
has zero measure.
Proof. By the S t a b i l i t y Theorem again
I t - I l i m ~ s ! o [ l ( x s = i , X s + l = J ) - E { l ( x s : i , X s + l = j ) l ~ C ~ s } ] = 0 almost sure ly . (14)
150
E{l(xs=i,Xs+l=j)IC~ s} = p(i,j;Us,~0)l(xs=i) ~ c l(xs=i),
again by the Markovian property and Assumption 2, and so from (14),
• l t - l l t - l llmTs!ol(xs=i,Xs+l=j) _> c lim-t s=OZ l(xs=i) , almost surely.
By Lemma 3 the term on the right vanishes only on a zero measure set, hence Mij has zero measure also. []
Theorem I. There is a set N of zero measure, a random variable ~*, and a f in i te random time T such that for m ~ N, t ~T(~),
at(w) = ~*(m), ut(m) = @I~*(~),xt(m) ] , (15)
pl i , j ;¢(~*(m),i),~*(~)] =pCi,j;@(s*(m),i),~0), all i , j . (16)
Proof. Since (15) is the same as Corollary l i t only remains to prove (16). According to (lO) for almost all
P[Xt,Xt+ l ;@(, ,x t ) ,~ , ) = PiXt,Xt+ l,@( * ,x t ) , 0], t ~ T. (17)
By Lemma 4, i f ( i , j ) is such that p( i , j ;u,s) > 0 then the joint event x t = i , xt+ l = j occurs in f in i te ly often, and in particular for t ~ T; hence (16) follows from (17). []
Corollary 2. Let A* = {~*(~) I~N}. Then for every s* EA*
p l i , j ; @ ( s * , i ) , s * ) = p l i , j ; ~ ( s * , i ) , ~ O ) , a l l i , j . (18)
Thus in the closed loop con f i gu ra t i on of Fig. 1 the parameter values A* are
i n d i s t i n g u i s h a b l e from s 0. I f the i d e n t i f i a b i l i t y cond i t i on (2) holds then (18)
impl ies tha t A* = {sO}. I t is tempting to conjecture tha t instead o f (18) we have
the much more s a t i s f a c t o r y cond i t i on v i z . ,
p l i , j ; @ ( ~ * , i ) , ~ * ] = p ( i , j ;@(sO, i ) ,~O) , a l l i , j ,
so tha t , asympto t i ca l l y , the closed loop system behavior would be the same as when 0 s is known. Unfo r tuna te ly th i s con jecture is fa lse as the f o l l o w i n g example shows.
Example. Consider the two s ta te system I = {1,2} wi th the unknown parameter
E {0 .01 ,0 .02 ,0 .03} w i th the t rue value s 0 = 0.02. The feedback law is u = @(0.01)
= @(0.03) : 2 and @(0.02) = I . The t r a n s i t i o n p r o b a b i l i t i e s are given by the d ia -
gram below. The i n i t i a l s ta te is x 0 = I . Suppose u 0 = I .
0.5+2~-~u
0 . 5 - 2 ~ + ~ u ~
l
Fig. 2. Transition diagram for example.
151
Then at t = l we have the following possib i l i t ies:
( i ) x I = l , p(l , l ;uo,O.Ol) = 0.49, p(l,l;Uo,O.02) = 0.48, p(l, l;uo,O.03) = 0.47,
so that the estimate is ~l = O.Ol; or
( i i ) x I = 2, p(l,2;Uo,O.Ol) : 0.51, p(l,2;Uo,O.02) = 0.52, p(l,2;Uo,O.03) = 0.53,
so that the estimate is ~l = 0.03.
In either case u I = 2. But since p( i , j ;2 ,~) does not depend on ~ i t follows that
the estimate w i l l stay unchanged. Thus we have ~t z O.Ol i f x I = l or ~t ~ 0.03 i f
x I = 2 and so s O cannot be a l im i t point of {~t }.
PERFORMANCE OF THE ADAPTIVE CONTROLLER
From Corollary 2 and the Example we see that the choice of the stationary con-
trol laws @(~,.) interacts with the parameter estimates. To ensure satisfactory
performance we must make sure that the law @(~*,-) is adequate whenever ~* satisf ies
(18). In this section we investigate this interaction further in the case where
@(~,-) is chosen to be an optimal control law for a specific cost function.
Suppose we are given a cost function k( i ,u ) , i E I , u E U, so that over the
long run the expected cost per uni t time is l t - l
lim~Es!ok(Xs,Us ). (19)
Suppose the parameter value is ~. From our ergodicity assumption i t follows that
there is a stationary feedback control law u t = @(~,x t) which minimizes the cost
(19). Moreover l t - l
lim~-s~oklXs,@(~,Xs) ~ ~ . = J(~) almost surely, (20)
where J(~) = Z k [ i ,@(~, i ) )~ i (~ ) ,
i
and the steady state probabi l i t ies {~i(m)} give the unique solut ion of
~j = ~ ~iP( i , j ;@(~, i ) ,~) , j E I , ~. ~j = I . 3
From Theorem l and (20) we get the next result .
Theorem 2. Let x t and u t = @(~t,xt), t = O,l . . . . be the state and control
sequences generated by the adaptive control ler. Then
l t - l lim E Z k(Xs,U-) = J(~*) almost surely.
s=O
I t follows that an a pr ior i guaranteed measure of performance of the proposed
adaptive control ler is J* = max J(~*) where ~* ranges over a l l values satisfying
(18).
CONCLUSIONS
The adaptive control ler f i r s t studied by Mandl, and resembling the self-tuning
controller, is investigated when Mandl's i d e n t i f i a b i l i t y condition fa i l s . The
152
parameter estimate does converge but, as shown by an example, i t may not converge
to the true parameter. The l imi t ing estimates can, however, be evaluated a pr ior i
and from this one can also obtain a guaranteed measure of performance. Cr i t ical use
was made of the assumption that the unknown parameter was restricted to a f i n i t e
set. In a subsequent paper we hope to extend the analysis to compact sets.
ACKNOWLEDGMENTS
The authors are grateful to Han-Shing Liu and Jean Walrand for discussions, and
for research support to the National Science Foundation under Grant ENG 76-16816 and
the Joint Services Electronics Program Contract F44620-76-C-0100.
REFERENCES
[ I ] P. Mandl, Estimation and control in Markov chains, Adv. Appl. Prob. 6, 40-60, 1974.
[2] K. Astr6m and B. Wittenmark, On self-tuning regulators, Automatic 9,185-199, 1973.
[3] L. Ljung and B. Wittenmark, Asymptotic properties of self-tuning regulators, TFRT-3071, Dept. of Auto. Contr., Lund Inst i tute of Technology, 1974
[4] Y. Baram and N. Sandell, Jr . , Consistent situation of f i n i t e parameter sets with application to l inear system ident i f icat ion, IEEE Trans. Auto. Contr.~ vol. AC-23, no. 3, 451-454, June 1978.
[5] M. Lo~ve, Probability Theory, Princeton: Van Nostrand, 1960.
RESOURCE MANAGEMENT IN AN AUTOMATED WAREHOUSE
R. Suri and Y.C. Ho
Division of Applied Sciences, Harvard University Cambridge, Mass. 02138
Abstract
We study the application of decentralization to the problem of Resource Management (RM) in a Large FIAT Warehouse, where a very large number of Activities share limited resources. Our emphasis is on the feasibility aspect of the problem, that is, of keeping the warehouse operational in the face of changing characteristics of the Activities. The size of the problem, and the ill-behaved resource-usage functions, make standard techniques unsuitable. However, by replacing the feasibility problem by a suitable "Artificial" optimization problem, we can use Lagrange Multipliers to provide a simple solution through decentralization of decisions. A Theorem is presented giving simple conditions for the existence of optimal multipliers for the Artificial Problem. Algorithms to solve the RM problem are also given, having proveable convergence properties, and quadratic convergence rates. (Our theorems are proved without the usual strict convexity conditions.) Based on our results we have designed a computerized RM system for the FIAT warehouse.
1.0 INTRODUCTION
In an era where, due to rapid advances in technology, we are seeing greater and greater interconnection between systems, the study of large-scale systems is assuming a new importance. Along with this has come the realization that in most applications practicality calls for decentralized control of such systems. In this work we study the application of decentralization to one aspect of such systems, namely, the problem of Resource Management in Large Systems.
In a large operational system, where a very large number of activities share a number of limited resources, this Resource Management problem has three main objectives. The first (the "Initial Allocation" or "Design" problem) is to find an assignment of resources to every activity, such that all the system constraints are satisfied, and all activities are operating, enabling the system as a whole to operate. The second (the "New-Assignment" problem) is to find a rationale for allocating resources to new activities. It is presumed that new activities are initiated frequently enough that we do not wish to re-solve the entire problem for the combined set of old and new activities. The third objective ("Periodic Review" problem) is to find an efficient way of re-allocating resources in order to reflect the changing needs of the individual activities, as well as the changes in total resource usages.
Conventionally, the resource-allocation problem has been studied for the case where, in addition to the constraints, there exists an objective to be maximized. Our emphasis, as is reflected by the title of this work, is on the feasibility aspect of the problem, that is, of taking a large system and keeping it operational (maintaining it in the f~asible region). We shall see that this in itself is both an important problem, and h\q theoretically interesting consequences.
154
2.0 MOTIVATION
Our interest in this problem arises from a project involving the authors, along with a team from CSDL (The Charles Stark Draper Laboratory, Cambridge, Mass.), to improve the operation of the FIAT Central Spare Parts Warehouse, in Volvera (Turin, Italy). This Warehouse essentially supplies spare parts to the whole world. It covers an area exceeding that of 15 football fields, has an inventory of over 20,000 tons, contains more than 60,000 different Part-Numbers (each of which may occupy several containers), and services about 10,000 orders every day [4]. The Warehouse is divided into several different areas, used for stocking Parts with different characteristics. For instance, medium-sized items with not too high demand are stocked in a 144 x 96 metre area, where loading and retrieval of containers is done $olely by computer-controlled cranes. On the other hand, very small, fast-moving items are stored in an area where they are hand-picked by men with hand-pushed carts.
The servicing of daily orders, and the replenishment of stocks, makes use of various resources in each area, which may be particular to a given area (such as shelf space) or may be shared by several areas (such as a conveyor that passes through different areas). Naturally, these resources have limits on their capacity. In January 1977, the Storage Allocation and Resource Management (SARM) problem faced by the warehouse could be summarized as:
I. There are several different storage areas, each with several container-types, leading to 16 different storage-types.
2. Each storage-type uses several resources, some of which are shared with other storage-types. There are 24 constrained resources (such as Storage Capacity, Crane Capacity, Conveyor Capacity, Manual Picking Capacity).
3. There were 60,000 Part-Numbers assigned to the various storage-types on the basis of criteria that were long since outdated -- demand patterns and the Warehouse operations had changed considerably.
The net effect of these factors was bottlenecks in several resources, yet much spare capacity in others. This meant that while in some storage-types the daily demand (or storage requirements) could not be met, in other storage-types equipment was lying idle. Keeping in mind these problems, as well as the future operating requirements of the warehouse, the aims of our project were set down as: (I) "Get rid of the bottlenecks" i.e. improve the current allocation as quick as possible. (2) Develop a method for reviewing the situation (say) every 3 months, and making necessary reallocations (Periodic Review). (3) Develop a rationale for allocating storage to New Part-Numbers, e.g. for a new car model.
3.0 FORMAL STATEMENT OF PROBLEM
We now develop a formal model of the SARM problem, and indicate the factors that make a good solution difficult to find. Although we will state our model in terms of the Warehouse above, the reader will see that our model generalizes to other large systems [23].
3.1 Notation
Underlined lower-case letters represent column vectors. Subscripts on a symbol usually denote a component of the corresponding vector or matrix. Superscripts will be used to differentiate between symbols of the same~type, for example xl x 2, x k. Vector inequalities are to be interpreted componentwise, that is a ! h means
155
a i ~ b i for all i. The zero vector will be denoted simply by 0. ~' denotes the transpose of the vector ~. E n is the n-dimensional Euclidean vector space.
3.2 Problem Formulation
Let there be I Items (Part-Numbers) to be allocated in S Storage-types, such that R Resource-usage constraints are satisfied.
Item allocation: The total quantity of item i is Qi and its other characteristics (demand, weight, volume, @to.) are represented by a data vector d i. For each item a S-dimensional decision x~ needs to be taken, where x~ is the quantity of item i allocated to storage s. We will refer to xi as an allocation of item i.
Resource usage: A given allocation for an item, along with the item's data characteristics will result in the use of various resources (e.g. storage space, crane-time, etc.). The resource Usage function u_i(~i,x_ i) 6 E R is a vector function such that u~(di,x i) is the usage of the r th resource by an item with data ~i, when its allocation is x I. (The calculation of Ki(.,.) obviously depends on the "operating rules" of the Warehouse which may, in general, vary for different items, hence the superscript i on K above.)
Total allocation and total usages: The allocation of all items will be represented by the vector ~ "~ r ~x1),~x2~, ,(xI) ' ]'. The total resource usage by an -- -- L ~__ s ,~__ J ,.-,
allocation of all items is I
i i=I We will refer to ~ or K as "usage vectors".
Constraints on usages: ~ 8 E R is the vector of constraints on the resource usages, that is c = value of constraint on usage of resource r.
r
Statement of general problem: Let g 6 E S have each component equal to unity, i.e. i ~ [1,1,...,I]'. Then the SARM problem can be stated as the General Problem
(GP) Find ~ = [ ( I),, .... (xE)' ]'
such that ~,x i = Qi (I equations)
and x2 ~ 0 (S x I equations)
and ~(~) ! ~ (R equations)
Note that the decision ~ consists of S x I components.
3.3 Comment On Feasibility Versus Optimality
The problem (GP) only involves looking for a feasible solution; no notion of optimality has been stated. One reason for this is that the problem is so complex (see next section) that even a feasible solution is hard to find. A more satisfactory reason derives from the warehouse management's objectives, which are: to keep the warehouse operational, irrespective of the relative uses of the resources, provided these usage levels are within the limits laid down by management. The major warehouse-equipment has already been installed, the capacities are non-transferable, and the day-to-day operating cost of the warehouse is relatively indifferent to what equipment is being used. Hence no criterion for
I56
minimization can be stated, and all feasible solutions are equally palatable.
3.4 Factors Contributing To Complexity Of Problem (GP)
Immense size: In the warehouse we have I=60,000, S=16, and R=24. This leads to a decision vector of approximately one million components!
Part-Data and usa~ functions: The diversity of Part-Data (frequency of demand, quantity demanded, weight, volume, etc.) and the dependence of ui(.,.) on the physical operation of the warehouse, leads to usage functions which can be discontinuous and/or nonlinear and/or noncenvex.
c ~ n e w Dart-Numbers: In addition to the 60,000 items in the warehouse, there are 30-50 New items arriving every day. These are not replenishment stocks, but items never before stocked. Hence 30-50 new allocations ~i have to be made every day, and clearly we would like to make "reasonable" decisions (valid in the long run) without re-solving the whole problem (GP) for the combined set of old and new items.
Linear or Integer Programming techniques would thus suffer from major disadvantages: first, the decision vector of one million components would lead to an astronomical program; and second, these methods would not lead to any strategy for allocating the new parts, short of re-solving the problem. However, an appropriate reformulation of the problem (GP) leads us to better solution tools.
4.0 THE ARTIFICIAL PROBLEM AND DECENTRALIZATION
In order to put (GP) in conventional optimization terms we "Artificial" Problem
I (AP) max J(X) ~Le'x i -
i=I
• i , i (AP-2) subject to _xZ>_ 0 and Q -~_> 0 each i
(AP-3) and c-~(~) ! 0
formulate the
In other words, maximize the total quantity allocated, subject to the resource usage constraint, the non-negativity constraint, and the fact that at most we can allocate the quantity we have of each item. Let
j, ~ ~Qi.
i=I If a feasible solution exists to (GP), then the maximum value of (AP) will be J*. (Notice the analogy with the Artificial variable technique of Linear Programming. This point is amplified in [21].)
Let ~ e E R be a vector of Lagrange Multipliers. We write the Lagrangean associated with (AP) as
L(X,A) : J(~) - ~'[~(X)-g]
For each i, let X i be the set of ~ i which satisfy (AP-2), and let X be the set of such that ~ie x i for each i. Then there is the following "Saddle Point Theorem" (see for example Lasdon [11]):
157
If there exist (X*,~*) with ~*SX and ~*l 0 such that (AP-4) L(~,~*) ~ L(~*,~*) ~ n(~*,~) for all xex and I ! O, then ~* solves the problem (AP). []
The power of the above result lies in the fact that it does nQ~ depend on the form of the functions J(X) and ~(X), nor on the form of the set X. An alternative view of (AP-4) is to say that
(AP-5) x* = arg max L(x,~*) aex
(AP-6) ~* = arg min L(~*,~) k >_0
A key point to note is that for given ~ the decentralized since
I max L(_x,~k) = 2k'__c +[ m~ i{_e,~i_il, ui(di xi)}
x_eX i: I X__ ex - -
problem (AP-5) ~a~ be
Thus, for given ~, the decision for each item i can be taken independently others, by solving the (much simpler) Individual Problem
max
x_iex i
of the
We see above that a given ~, through (IP), leads to an allocation of all items, say ~(~), and corresponding total resource usages ~(~(~)). We can therefore think of~as a function of ~, say ~(~). The problem then, is to find the ~* in (AP-6), for then from (IP), (AP-5), and (AP-4) we know that ~(~*) and ~(~*) are optimal.
Arrow and Hurwicz [I] observed that (AP-5) and scheme of the form
(AP-7) k+1 arg max L(x,~ k)
xex
(AP-8) ~k+l arg min L(xk+l,~.)_
(AP-6) suggest an iterative
with an intuitively appealing economic interpretation. A "central co-ordinator" chooses a set of "prices" ~, after which the items i find their optimal decisions xi for this ~. The central co-ordinator then looks at the total resource usages and adjusts the prices to increase the cost of over-used resources, and decrease the cost of under-used resources (but never making any cost negative); in other words he adjusts prices according to excess demand. This use of decentralization in Resource Allocation problems is well known [1,3,6,11,19], and arises out of the additive nature of the objective function and the resource usage functions.
We have reduced via this means an optimization problem involving S x I (=one million) variables to an optimization problem with R (=24) variables plus a set of I (=60,000) decoupled and relatively simple problems. However, we must overcome three additional difficulties:
I. The decomposition and iteration method described above falls in the general category of "dual" methods [6]. A major shortcoming of these methods is the existence of "duality gaps" [8,11] -- although an optimal value of the Artificial Problem exists, no pair (X*,A*) exists which satisfies (AP-4).
158
2. Even if no duality gap exists, convergence of iterations is guaranteed only when strict conditions hold on the Payoff Function and Resource usage Functions [1,26] -- conditions which certainly do not hold in our problem.
3. Convergence can be very slow even given the above conditions.
We therefore look for an improved scheme. We choose to pursue this solution technique in the face of the above difficulties because the decentralized aDoroach does offer several advantages. Firstly, it makes possible the solution of a large intractable problem, by reducing it to a number of smaller problems. Secondly, suppose we are able to find an efficient iteration technique, and use it to generate a solution ~*, with corresponding allocation X(~*). When demand characteristics have changed slightly over some months, we still expect ~* to be a good starting point for iterations to find a new solution. Hence the Periodic Review problem can be solved very efficiently each time. Thirdly, given a set of multipliers ~*, the New Parts problem can be reduced to solving (IP) for each new part -- a relatively easy problem. Hence the allocation of new parts is (through ~*) made independent of the rest of the parts in the warehouse. And finally, the economic interpretation of the scheme makes it appealing to Managers, who readily understand it. Hence they prefer it to other schemes which give them no insight as to the rationale behind a particular allocation.
5.0 ON THE EXISTENCE OF OPTIMAL MULTIPLIERS
The question of existence of an optimal ~k for a given problem has, in general, only been answered in the literature under certain convexity conditions [8,11,12]. In this section we give a far more general result.
Assume that X i is a discrete set. I
Let J* _~ ~ QI, the maximum value of (AP)
i=I
and ~(~) ~ arg max L(x_,A) xex
i i i i i i ACE R with A k ~ max max lUk(d ,Xl)_Uk(d ,~2) 1 -- i^.i i_.i
i x1~x ,~2~A
Remark: The k th component of A represents the largest change in the usage of the k -~ resource, that can be caused by a single item. []
Theorem ! (see Suri [21]): If there exists an X e x with J(X)=J* and ~(X)~ ~-a~, where ~(R-1)/2, then there exists a A* !0 and an ~(~*) such that J(~(~*))=J* and ~(~(A*))~, that is, ~(~*) solves (AP). []
For a large problem with (say) several thousand activities using each resource, we would expect gA to be very small in comparison with ~. In that case we can give the following
Internretation of Theorem !: If, for a slightly tighter set of limits, the original problem is still feasible, then there will exist a ~*~0 such that the (decentralized) solution ~(~*) will also be feasible for the original problem. []
Remarks: The importance of our theorem is threefold -- (I) We have given conditions under which there will be no duality aga~ in the Artificial Problem. (2) Our conditions require no convexity and/or continuity and/or linearity
159
assumptions: the conditions are extremely mild, and likely to be true for most large systems (since ~Ais very small in comparison with ~, as explained above). (3) If each X i has N components, our theorem Justifies replacing the IxN Integer Programming problem for ~ by the R-dimensional Nonlinear Programming problem of finding a suitable ~. For instance, in the case of the FIAT warehouse, IxN has approximately one million components, whereas R has only 24. []
The~* above is not unique -- there is a range of possible values [21]. The next Theorem shows that in this range of values there also lies a & which is strictly positive. (This result will be useful later for our iteration algorithm.)
Theorem II (Suri [21]): Under the conditions of Theorem I, there also exists a ~*>0 such that ~(i*) (AP). []
solves
6.0 THE SALA TECHNIQUE
6.1 Motivation
Our approach, called SALA (for Storage Allocation Algorithm) will be as follows: We observe that the Individual Problem (IP) can be made still easier. Then we look for an iteration scheme to find ~*. We first make some simplifying assumptions. The resulting model will be analyzed; we shall propose an algorithm and study its properties. This provides us with insight as to how to extend our algorithm to the more realistic ease.
Preview Of Iteration SQhem~: Our objective is to find a ~* such that ~(~*) is optimal --for (AP). We choose a starting value of ~, say ~o, and then follow the scheme
(SA-I) k+~= arg min L*(X,A k) x_ex*
(SA ) k÷1
until we find an optimal ~. Compare with (AP-7,AP-8): firstly, our scheme replaces "max L" for xeX by "min L*" for ~ex*, where L* and X* will be such that they fUrther simplify the solution to (SA-I) as compared with (AP-7). Secondly, we have a different method of updating ~, using the Seleetion Algorithm, which will lead to a quadratic convergence rate of the above scheme.
"Min-Cost" AllocatiQn For E~ch Item: Consider the Individual Problem (IP). It is easy to see that if there exists a ~ sueh that ~(A) achieves the maximum value of (AP), then (IP) can be replaced by the following Minimum Cost Allocation Problem
(MCA) min {xi I a,~i=Qi}
This says that for a given set of "costs" ~, the i th item must find that allocation xi (of all its Quantity Qi) which minimizes its total resource usage cost. Thus, knowing the existence of an optimal ~ (see below), and summing (MCA) over all i, we have replaced (AP-7) by !SA-I), provided we define L*(X,~) ~ ~'~(~), and X* as the set of ~ such that each ~ l in X satisfies the equality constraint in (MCA). The scheme (SA-I,SA-2) then has the following interpretation: we hold the objective function J(~) at its maximum value (J*) and try to bring the total resource usage ~(X) into the feasible region ( ~ ~). The equality constraint in (MCA), and the fact that xi is now an S-dimenslonal vector, make (MCA) a relatively simple problem.
Existence 0__[ Optimal ~: Theorem I included the condition that the strategy set for
160
each ~i was discrete. Let us simply restrict each X i to a discrete set. (In theory, could let this discrete set be as finely divided as required, so this is not a major restriction.) This restriction actually has two advantages: (i) we can apply Theorem I, and (ii) it makes the solution of (MCA) simpler. Now, the conditions in Theorem I, stated informally, are "if we reduced the limits vector by the maximum resource usage of about 12 parts, the problem would still be feasible". Since we are dealing with 60,000 parts, this condition is highly likely to hold. Thus we are justified in proceeding as if an optimal ~ exists.
6.2 Discussion Of Main Assumptions
Let W be a bounded subset of the I space, which will be delimited later. We begin by listing three assumptions, all of which will b_eerelaxed later, but which are needed for our analysis of the idealized case.
(AI) Assumntion: For all ~ e W, the function ~(~) is continuous and Frechet differentiable. []
Note that we do not, however, make any convexity (or concavity) assumptions as in [I] or [26]. In view of (At) we will define the Jacobian of ~(~) at any ~" e w by the matrix A(~"), that is
Defini~iQn Aij(~") ~k]l~=~, , []
Lemma I ISin~ularitv of A): For any~" g W we have A(X") ~"= 0 []
Proof: If all costs are increased in the same proportion, then from (MCA) no allocation will change, that is ~(~"÷h~") = ~(~"), and since this is true for arbitrary h, the directional derivative of ~(~") in the direction ~" must be z e r o . [ ]
Corolla~y: The R-dimensional function 2(.) of the R-dimensional variable ~ is (at most) an R-I dimensional surface (in the ~ space). This can also be seen from the fact that the scale of ~ is arbitrary. []
Definition: The feasible region in K space is F ~ {K { 0 < u < ~} []
Definition (Pseudo-Feasibility): K(~) is Pseudo-Feasible (PF) w.r.t. F at ~=~", if the tangent hyperplane to 2(.) at ~" passes through the region F (see Fig.l). []
(A2) Assumotion: ~(~) is PF w.r.t. F, for all ~ e W. []
The PF concept could be introduced since by (AI) the tangent hyperplane to K(!) exists for all & e w. The motivation for (A2) is that a first-order approximation to ~(~) should have a non-empty intersection with the feasible region.
(A3) Assumption: The matrix A satisfies Ajj<O and Aij~O (i~J). []
Remark: This essentially restricts the system configuration. In any case Ajj must be non-positive (explained below). Our assumption strengthens this to strictly negative. The two conditions on A are part of the conditions that make (-A) a Metzler-matrix. An example of a matrix satisfying Aij>_O would be in a system with storage constraints only; or alternatively a system in which we could identify one critical resource in each storage-type. The reason for this is as follows: From (MCA), an increase in Aj (with all other k i constant) cannot cause parts not using resource j to decide to use it, and in fact it may cause some parts using resource
161
J to move to another area. Thus we have Ajj~O and Ai~0 (igJ).
Summary of First Set of Assumptions: (AI) ~(~) continuous, differentiable. (A2) Pseudo-Feasibility. (A3) Ajj<O and Ai~0.
Fi~. I." Illustration of P~udo-Feaslbilltv
[ ]
U 3
u(X) surface
F (Feasible
Region) / /
Tangent plane at ... ® intersects F
115 ,I ' I Solution set
i/ ]L U2
Ul
6.3 The Selection Algorithm
The purpose of this algorithm is to derive a value for 6_~ [see (SA-2)], by using the information from the current iteration, i.e. K(~) and A(~). (We assume, for the moment, that A(~) is known.) Let us say we have a ~=A" such that K(~") e F. We are looking for a change Au" such that ~(~")+~u" ~ ~, or to a first order approximation [using (A1)], we want AA such that ~(~")+A(~")~_jk~ ~. (For notational simplicity we shall drop the dependence of A on ~".) Defining Au=~-~(~") we want
(SA-3) A ~_k < A~ .
We need a method to solve this inequality, and in fact to choose from all the possible solutions for AA. [The existence of a solution is guaranteed by (A2).] Clearly We should not, in general, try to solve the equality A AA = Au. (This would attempt to move to the point where all the constraints are active, which may not even be physically realizable, and/or may not lie on the tangent hyperplane: either of these will cause numerical problems.) Our method is to select only certain components of ~" to be changed, and to look at the equation AsAA s =Au S, where A~ s is a vector containing only the selected components of A_k, similarly with ~u s, and A s is a submatrix of A containing only the rows and columns corresponding
162
to the selected components. We solve this for ~__~S:(AS)-lhus. Then, in the full vector A k, we set all selected components to their corresponding values in AA s, and all unselected ones are kept zero. We check to see whether A~ is indeed !Au. If not, we modify the selection to include other components, and repeat the procedure.
The Selection Algorithm is described in detail in [23]. Its idea is to find a "good" solution to (SA-3), while at each stage exerting the minimum effort that appears necessary. For example, in a typical operation of the Algorithm with R=24, we may find we have to invert a 4x4, then 6x6 and then I0xi0 matrix, at which point a suitable ~__~is found. This takes (43+63+I03)k =1280k operations (say), whereas inverting a 24x24 matrix takes 243k =13,824k operations.
The power of the Selection Algorithm however, stems not so much from its computational savings as from its properties which are described by the next result.
Theorem III (see Suri [23]): Let assumptions (A1)-(A3) hold for ~:~", and suppose that ~(~") ~ F. Then if we have
(I) The Selection Algorithm [23] terminates before all R components of k" selected,
(2) The AA,, so found satisfies
~" > 0
are
AA,, : arg min { IIA_~I I ~(~")+A(~")AA!~ } ~_k >_0
R
whe[~i,l I can the EY~ Go m~x ~ . be either Euclidean Norm (i=I )0.5, or the 1
[]
Norm (
Remarks: There are two properties of interest in (2) above. The first, that the Algorithm gives ~" I 0. This implies that if we had ~">0, we can be sure that k"+A_~">O, an important condition for the next application of the Selection Algorithm, and for several of our other results [23]. The second appealing property is that among all suitable AA ~ 0, it finds the one of minimum norm. This will be important for our convergence proof in Theorem IV.
7.0 ITERATION SCHEMES FOR THE IDEALIZED PROBLEM
7.1 Restricted System Configuration
Retaining for the present the assumption that K(~) is a continuous function of whose Jacobian A(~) can be calculated, and also that the system satisfies (A3),
we can hypothesize the following iteration scheme to solve (AP):
Algorithm (SALA-I I~eration ~ : INIT : Given some ~ini~>0
Set ~o:~init Set k=O
MCA
TEST
JACOBIAN :
SELECTION:
UPDATE :
Use kk to do a Min-Cost Allocation for each item, and calculate KCk k)
If ~(~k) e F then STOP
Calculate A(~ k)
Use Selection Algorithm to ealculateA__~ k
Set ik+ 1=~k+AAk
163
Set k=k+1 go to MCA
The convergence of this scheme is studied below.
(DI) D~finitions:
II~I ~ max luil max ~-]II~__Aij " j i ..... ; IIAII ~ i ........ ;
e i c i
d(K,F) g mln ~,,eF I1~-~"11
(A4) Assumption: K(A) is Pseudo-Feasible w.r.t, pF for all ~ e W, where p<1. (In this case we say u(.) is Strictly Pseudo-Feasible, or SPF. This is a stronger restatement of (A2).) []
(AS) AssumPtion: For all ~,~" e w,
II A(~)-A(i")II X D'II ~-~" II (D is some constant) []
R_~: (A4) assures us that the tangent hyperplane enters the interior of F, while (A5) effectively bounds the second derivatives of K(.). In convergence proofs of Newton's method [12,13] it is customary to see (A5), while our (A4) is analogous to the assumption there that the gradient of the function is non-zero at the root. []
D@finitions:
AA >_0
~_& >_0
where p is the value in (A4).
(D4) SEL(X) ~ any AA minimizing (D2) above. (The notation reminds us that such a value is generated by the Selection algorithm.)
[]
Remark: The definitions of T(.) and T"(.) are motivated by Theorem III. We use the i~ norm on A__~. []
Lemma 2 (see [23]): Let W" ~ Wn{A I ~(k)~g}. Then for ~ e W", T,'(~) and IIA(~)II are non-zero and bounded above. []
sup (D5) Definitions: a ~ ~ew" IIA(~)II ;
~ sup T"(~) ..... []
kew" 1-p
Remark: It is in view of Lemma 2 that we can define the above quantities, and both will be non-zero. []
Second Set of Assumptions: (Compare with first set.) (A1) K(~ continuous, differentiable. (A3) Ajj<0 and Ai~>O. (A4) Strict Pseudo-Feasibillty. (A5) Bounded second derivatives.
Theorem IV: Convergence of SALA-I Scheme (see [23])
164
Let the second set of Assumptions hold for some W, and let the constants D,C be as already defined. For some ~o e W, (~°>0) let the following conditions hold:
(i) II~1-k°II <a, where ~I: ~°+SEL (~°)
(ii) e<1, where e ~ aCD/2
Now let W' ~ {A I llkl-k°I] <b} where b~a/(1-e). Then if W'~ W, the SALA-I scheme generates a sequence ~} which converges to some ~* e w, such that K(k*) e F. Furthermore, the average order of convergence of the Algorithm is at least two. []
Existence of g ~ : The reader may ask that, since Ak is always 20, how can we be sure that for arbitrary choice of ~ init in the SALA-I algorithm, there exists a solution ~* !~init? In see.6.1 we Justified the existence of some optimal ~. Under the same conditions, there also exists some ~*>0 (strictly) which is also a solution (Theorem II). Now the scale of~* is arbitrary~ and only the relative magnitudes of the components matter. Thus for any k__inlt, there exists a b>O such that b~*Z~ init and bk* is a solution.
Conditions in the Theorem: These are similar to those used for Newton's method [12,13,16]. In fact our result is similar to Robinson's [16], but our method is quite different. His proof assumes A is nonsingular, which is not so in our case (Lemma I), and he uses several properties of convex processes [17,18]. Our proof depends mainly on the minimum norm properties and the SPF assumption. Intuitively, we have replaced the condition that the range of A be the whole space, by the (weaker) condition that the range of A include some point in the interior of the set F-~(~).
7.2 General System Configuration [Relaxation Of (A3)]
The restriction on the system configuration in (A3) ensured the minimum norm properties for the selection algorithm. From the insight given us by the use of these properties we propose a general programming problem, the Minimum Norm Problem:
(MNP)
a.
Find Akminimizing II~_~I
subject to A+AA !~ min
b. and K(~+A(k)A_k~
Here ~min is a given strictly positive vector. The 1 norm problem rain llA__~l is
equivalent to min y subject to y!AA i and y~-AA i for all i. Thus MNP problem can be solved using Linear Programming, and we can generalize our iteration scheme:
Al~orithm (SALA-2 Iteration Scheme): INIT : Given some ~min>o, and some ~init~min
Set &o:~init
Set k=O
MCA
TEST
Use ~k to do a Min-Cost Allocation for each item, and calculate u(kk)
If~(~ k) e F then STOP
JACOBIAN : Calculate A(~ k)
MNP : Calculate AA k as in (MNP) above
165
UPDATE : Set Ak+1:~k+A~ k Set k=k+1 go to MCA
Third Set 9_~ Assumptions: We relax the second set to get (AI) u(~) continuous, differentiable. (A4) Strict Pseudo-Feasibility. (A5) Bounded second derivatives.
Lemma ~ (see [23]): At the k th iteration of algorithm SALA-2, A__K ~ which solves MNP. []
Definitions: We modify (D2-D4) to get
(D6) T(~) ) Let these be defined as ) in (D2-D4), except that
(D7) T"(~) ) the minimizations be ) carried out over all AA
(DS) SEL(~) ) such that
[]
there exists some
Theorem V: Convergence of SALA-2 Scheme (see [23]): The statements in Theorem IV remain valid for the third set definitions (D6-D8) and the SALA-2 iteration scheme. []
of assumptions, with
8.0 EXTENSIONS AND APPLICATIONS
Due to limitations of space, only a summary of our work is given below. Refer to [25] for further information, or [23] for complete details.
Extension T_go Realistic Case: We are able to relax several assumptions, in order to extend our results to more realistic cases. In particular, we are able to remove the continuity assumption (At), and considerably relax the SPF assumption (A4). The gradient matrix A is replaced by a suitable numerical construct. A convergence proof for a modified version of the SALA-2 algorithm is then given.
D~s~gn of a Practical Resource Management System: This illustrates how, in an operational system, we implement the solution to the "Initial Allocation", "New Assignment", and "Periodic Review" problems. The features of the program package designed for FIAT/Volvera are also described.
Examole of a Design-Evaluation Problem: This illustrates the use of SALA at FIAT/Volvera with a problem involving selection of parts to be allocated to a proposed new area, and at the same time reallocating parts between existing areas to meet future requirements. (FIAT management is making extensive use of this design evaluation feature, and consider it a valuable decision-making aid.)
9.0 REVIEW OF RESULTS AND COMPARISON WITH OTHER WORK
We have used Lagrange Multipliers to solve the Resource Management problem in a large system. Everett [3] pointed out that, even under general conditions some statements could be made regarding the properties of an allocation ~(~), for any ~Z0. He did not, however, deal with the existence of optimal multipliers. Our existence theorem greatly extends the applicability of the Lagrange Multiplier technique. We also emphasize that in practice, our conditions are likely to hold
166
for large systems. This Justifies the use of our decentralized methods. We have also given procedures to find an optimal ~, and are able to incorporate realistic conditions in our model.
The final test of the validity of our assumptions is use of the algorithm in actual cases. We have used the Algorithm successfully on numerous problems, both for evaluating design of new Warehouse facilities, and for improving the operation of existing facilities [24].
The advantage of decentralized techniques is that they make possible the efficient solution of very large problems. However, the applicability of these techniques has been restricted to problems which satisfy strict conditions. The main contribution of our work is to extend the applicability of decentralized solution methods to problems where the resource usage functions are not well-behaved. The type of assumptions and conditions required for our results reflect properties of the system as a whole, rather than the properties of the individual items in the system. We feel that this is an important viewpoint for dealing with large systems (see [22]).
ACKNOWLEDGEMENTS
This paper reports the results of a research program sponsored by FIAT Ricambi. We wish to thank all the personnel from FIAT whose cooperation and interest made this study possible. We also wish to acknowledge the assistance of the following CSDL personnel: K.Gliek, S.Brosio, J.S.Rhcdes, I.Johnson, and R.Asare. Significant portions of the theoretical work reported here were made possible by support extended the Division of Applied Sciences, Harvard University, by the U.S. Office of Naval Research under the Joint Services Electronics Program by Contract N00014-75-C-0648, and by the National Science Foundation under Grant ENG76-11824.
References
[I] Arrow, K.J. and Hurwicz, L., "Decentralization and Computation in Resource Allocation", in Essays in Economics and Econometrics, R.W. Pfouts (Ed.), Univ. of North Carolina Press (1960).
[2] Aho, A.V., Hopcroft, J.E., and Ullman, J.D., The Design and Analysis of Computer Algorithms, Addison-Wesley (1974).
[3] Everett, H., "Generalized Lagrange Multiplier Method for solving problems of Optimum Allocation of Resources", Ooerations Research 11 (1963) pp.399-417.
[4] FIAT, Volvera: The car spare parts warehouse, FIAT Information and Advertising, Edition No.4398, Turin, Italy.
[5] FIAT/CSDL, Des_~ of New Operating Procedures for the FIA~ Automobile Spare Part8 Warehouse at Volvera. Turin, Italy: FIAT Ricambi (April 1977).
[6] Geoffrion, A.M., "Elements of Large-Scale Mathematical Programming", Management SQie~¢e 16 (July 1970) pp.652-691. Also in [7].
[7] Geoffrion, A.M., (Ed.) Perspectives o__n_n Optimization, Addison-Wesley (1972).
C8] Geoffrion, A.M., "Duality in Nonlinear Programming: A Simplified Applications-oriented Development", SIAM ReviewJ_i (Jan.1971) pp.I-37. Also in [7].
[9] Katkovnik, V.Ya., "Method of Averaging Operators in Iteration Algorithms for Stochastic Optimization", Cybernetics (USA) 8 (July-Aug. 1972) pp.670-679.
167
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
[25]
Kushner, H.J., "Convergence of Recursive Adaptive and Identification Procedures via Weak Convergence Theory", IEEE Trans, Aut, Cont~o1 22 (Dec.1977) pp.921-930.
Lasdon, L.S., Op.timization Theory fQ~ Large Systems, New York: Macmillan (1970).
Luenberger, D.G., Optimization by Vector Space Methods, New York: John Wiley (1969).
Luenberger, D.G., Introduction to Linear ~n~ Nonlinear Programming, Addison-Wesley (1973).
Ljung, L., "Analysis of Recursive Stochastic Algorithms", IEEE Trans. Aut. Control 22 (Aug.1977) pp.551-575.
M~moud, S. and Riordan, J.S., "Optimal Allocation of Resources in Distributed Information Networks", ACM Trans. Database Systems ! (March 1976) pp.66-78.
Robinson, S.M., "Extension of Newton's Method to Nonlinear Functions with values in a Cone", Numer, Math. 19 (1972) pp.341-347.
Robinson, S.M., "Normed Convex Processes", Trans, Amer, Math, Soc. 174 (Dec.1972) pp.127-140.
Rockafellar, R.T., Monotone Processes of Convex and Concave TVDe, Princeton: Amer. Math. See. Memoirs No. 77 (1967).
Shapiro, J.F., A Survey of La~ran~ean Techniques for Discrete Optimization, Tech. Rep. 133, Operations Research Center, M.I.T., Cambridge, MA. (May 1977).
Suri, R., SALA Reference M~nua~ and U~r'~ Guide, Report FR72400-03, C.S.Draper Lab., Cambridge, MA. (De0.1977).
Suri, R., "Existence of Optimal Multipliers for Dual Solutions to Certain Allocation Problems", submitted for publication. (Also in [23].)
Suri, R., "New Directions in Large-Scale Systems", in "A New Look at Large-Scale Systems and Decentralized Control: Recent Graduates Speak Out", Prec. 17th IEEE Conf. Decision and Control, San Diego, Calif. (Jan. 1979).
Suri, R., Resource Management in Large Systems, Ph.D. Thesis (also available as Teeh. Rep.), Division of Applied Sciences, Harvard University (1978).
Suri, R., He, Y.C., Rhodes, J.S., Johnson, I. and Motta, P.G., "Application of a New Resource Management Algorithm to a FIAT Warehouse", submitted for publication.
Suri, R. and He, Y.C., "An Algorithm for Resource Management in Large Systems", Prec. 17th IEEE Conf. Decision and Control, San Diego, Calif. (Jan. 1979).
[26] Zangwill, W.I., Nonlinear P r a ~ A Unified ADDroach, Prentice-Hall (1969).
Co~loque I n ~ r n a t i o n a l ~ u ~ l ' A ~ l y s e ~ t l ' O p t ~ m i s a ~ n d~s Sys£~m~
IRIA - 11-15 D~c~mbre 1978
DUALITE ASYMPTOTIOUE ENTRE LES SYSTEMES DE COMMANDE ADAPTATIVE AVEC MODELE ET LES
REGULATEURS A VARIANCE MINIMALE AUTO-AJUSTABLES ~)
Yoan D. LANDAU Maitre de Recherche au CNRS
Laboratoire d'Automatique de Grenoble B.P. 46 - 38402 ST MARTIN D'HERES
RESUME : On d~montre que les r~gulateurs g variance minimale auto-ajustables (RMVA)
et les syst~mes de eommande adaptative avec module de r~f~rence (SCAMR) ont un caract~-
re "dual" qui est une extension de la relation de "dualitY" existant entre la commande
variance minimale et la commande modale dans le eas des syst~mes lin~aires ~ param~-
tres connus. On montre aussi que les SCAMR de type "explieite" sont ~quivalents aux
SCAMR de type "implicite" qui utilisent un pr~dicteur adaptatif interm~diaire si le
pr~dicteur adaptatif plus la eommande ont un comportement identique au module de r~f~-
fence explicite.
ABSTRACT : It is shown that the Self Tuning Minimum Variance Regulator and the Adapti-
ve Model Following Control feature a duality character which extends the duality exis-
ting between the minimum variance control and the modal control in the linear case wit~
known parameters. It is also shown that Adaptive Model Following Control with an expli-
cit reference model is equivalent with an Adaptive Model Following Control using an in-
termediate adaptive predictor if the adaptive predictor plus the control behave like
the explicit reference model.
I - INTRODUCTION
Des travaux r~cents [I], [2], [3] ont permis d'~claircir les liaisons qui existent
entre les syst~mes de commande adaptative avec module de r~f~renee explicite (appel~s
aussi syst~mes de eommande adaptative direete [2]) o~ on adapte directement les param~-
tres du r~gulateur (fig. |) et les syst~mes de eomande adaptative avee module de r~f~-
rence implicite (appel~s aussi syst~mes de co~m~ande adaptative indireete 2 ) o~ un
pr~dicteur adaptatif d~riv~ des S.A.M.R. est utilis~ et dont les paramatres servent
pour le ealcul du r~gulateur (fig. 2). Ces deux types de schemas peuvent ~tre ~quiva-
lents si l'erreur de prediction est globalement asymptotiquement stable et si la stra-
tegic de commande est choisie afin que la sortie du pr~dieteur ait un comportement idec
tique ~ eelle d'un module de r~f~rence explicite (le pr~dieteur adaptatif et la comman-
de forment un module de r~f~rence implicite).
• ) Ce t r a v a i l a ~t~ e f~ec tu~ clans l e cadre de l ' A . T . P , n ~ 3180 du CNRS
169
/ Adaptive I Uk I
Control Ier I L Pl ant
/
Explicit " " ' Reference
Model
x k
Adaptation Mechanism
FIGURE ;
Syst~me de Conmaande Adaptative ~ ModUle de R~f~rence "explicite"
i Adaptive I Uk - Plant
Control I er I
I i I t I Implicit Ref, rence Model
] I +
i T - q
Ada pti ve Predictor
. . . . J
Yk
Adaptation Mechanism
FIGURE 2
Syst~me de Co~ande Adaptative a ModUle de R~f~rence "implicite"
170
Dans les r~f~rences [I], [4] diverses similitudes entre les systhmes de cornmande
adaptative avec mod~le destin6s g op~rer dans un environnement d~terministe et les r~-
gulateurs g variance minimale auto-ajustables destines g op~rer dans un environnement
stochastique ont gt~ analys~es (rappelons que les r~gulateurs g variance minimale au-
to-ajustable ont une structure similaire ~ celle de la fig. 2).
L'objectif principal de ce travail est de montrer qu'~tant donn~e une elasse de
r~gulateurs ~ variance minimale auto-ajustables (RMVA), il existe une classe de syst~-
mes de commande adaptative avec mod~le (SCAMR) implicite et une classe de syst~mes de
commande adaptative avec mod~le (SCAMR) explicite qui utilisent le m~me algorithme d'a-
daptation que le RMVA, ont le m~me point de convergence que le RMVA, des conditions de
stabilit~ globale dfiterministe identiqueS aux conditions de convergence avec probabili-
t~ I du RMVA et dent les lois de cormnande tendent asymptotiquement vers celles du R.M.
V.A. Par ailleurs, on d6montre que les deux classes de systgmes de commande adaptative
avec mod~le (explicite et implieite) sent ~quivalents.
Dans ce cas, il s'agit bien d'une propri~t~ de dualit~ car bien que les algorith-
mes d'adaptation et les lois de commande soient identiques, les objectifs des RMVA et
des SCAMR sent diff~rents. La pr6sentation de ce travail est faite de la fa~on suivan-
re. Dans le § 2, on ~tablit la dualit6 entre le rfigulateur g variance minimale et le
r~gulateur de type modal. Le § 3 est consacr~ g la presentation de la classe de RMVA
consid~r~e. Dans le § 4, on d~veloppe un SCAMR implicite qui utilise le m~me algorith-
me d'adaptation que le RMVA. Dans le § 5, on d~finit le concept de dualit~ asymptoti-
que entre RMVA et SCAMR et on 6tablit la dualit~ asymptotique des deux sch6mas. Le § 6
est consaer~ au d~veloppement d'un SCAMR explicite ~quivalent au SCAMR implicite de la
section 4 et qui est asymptotiquement dual par rapport au RMVA et dans le § 7, on dis-
cute les r~sultats obtenus et on indique d'autres problgmes ouverts concernant la dua-
lit6 entre RMVA et SCAMR.
I I - DUALITE ENTRE LA REGULATION A VARIANCE MINIMALE ET LA REGULATION MODALE
Soit le proeessus et l'environnement stochastique d~crit par :
n m = B(q -I) + C(q -I) Vk = i~ I a i + b i Yk A ( q - l ) Uk-I A(q - I ) Yk-i i~ l u k - i - 1 +
n
+ be Uk-l - i~l ci Vk-i + Vk
o~ Yk est la sortie (mesur~e) du processus , u k est la commandeet v k
ce de variables al~atoires ind~pendantes normales (O,l) et :
A(q -l) = I al q-l -n - .... - anq
B(q-l) be . . -m = + b] q-l .. +bm q
C(q-l) = i _ e l q I -n - . . . . - C n q
3
(2.1)
est une s~quen-
(2.2)
(2.3)
(2.4)
171
Les polynomes B(z -I) et C(z -I) sont supposes ayant des z~ros dans Izl < I.
Rappelons d'abord l'objectif de la r~gulation dans le cas d~terministe (v k E 0).
Etant donn~ Yo # O, on d~sire soit Yk E O (les p~les du syst~me boucl~ sont places
, A°( -]. tous ~ z = O) soit q )Yk = O ou A°(q -]) d~finit les pBles souhait~s du syst~me
boucl~.
Dens le cas stochastique (v k # O), (rEgulation ~ variance minimale), on souhaite :
E{y k} = O et E{y k} = min. Le r~gulateur ~ variance minimale peut se ealculer directe-
ment (voir rEf. [7]) et le r~sultat est :
I [p~ Sk 1 ] (2.5) uk_ l = -~--_ 0
o~ : T
PMV = [al-Cl' ... an-Cn, b I ... bn] (2.6)
T Sk-I = [Yk-] "'" Yk-n' Uk-2 "'" Uk-n-]] (2.7)
Mais, le m~me rEsultat (2.5) s'obtient en appliquant le th~or~me de s~paration
savoir :
I) Calcul du pr~dicteur optimal Yk/k-! ~ partir de (2.]) : n m n
Yk/k-I = i=]E a i Y - k i + i=IE b i Uk_i_ 1 + b ° Uk_ 1 - i=lE c i v k (2.8)
2) Commande du pr~dicteur (2.8) comma s'il s'agissalt d'un processus d~terminis-
te afin d'atteindre l'objeetif Yk/k-] E O. (~ noter que dans le cas d'une commande
variance minimale, Yk = Vk)" Dans le cas d~terministe, si A°(q -]) = C(q -l) (ou C(q -l)
est le polynome qui d~finit la perturbation stochastique), alors la loi de commande
qui assure C(q-l)yk = 0 est donn~ par l'~quation (2.5) d'o~ : modale
THEOREME 2.1 : (duallt~ co~mande ~ variance minimale-eommande modale). La comman-
de a variance minimale d'un processus dans un environnement stochastique d~crit par
l'~q. (2.]) est identique ~ la co~nande modale du m@me processus dans un environnement
stochastique sl et seulemen£ si le comportement d~sirE en boucle ferm~e est d~fini
par :
C(q-l)y k = O (2.9)
I l l - LE REGULATEUR A VARIANCE MINIMALE AUTO-AJUSTABLE (RMVA)
On rappelle bri~vement les ~quations d~finissant ce type de r~gulateur adaptatif.
Pour plus de d~tails, voir [lO]. On consid~re dans le but de simplifier les ealculs
que b °dans (2.1) est connu et que tousles autres param~tres sont inconnus mais cons-
tants (un d~veloppement sim~lalre peut ~tre fa~t aussi pour la cas o~ b est inconnu). o
L'~q. (2.1) du processus et de son environnement peut se r~crire sous la forme :
172
oO :
T Yk = Po qbk-1 + bo Uk-I + C ( q - I ) v k ( 3 . 1 )
T Po = [ a l . . . a n , b 1 . . . bm ] ( 3 . 2 )
T = [Yk 1 "" Yk-n" U k - 2 Ck- I - • . . . ,,X_m_I ] ( 3 . 3 )
o~ :
o~. :
L'algorithme de commande adaptative s'obtient eoneeptuellement en deux ~tapes :
Etape 1 : (pr~dicteur adaptatif)
Yk/k-I = pT(k-l)¢k-I + bo Uk-| (3.4)
p T ( k ) = [ ~ l ( k ) - e 1, . . . a n ( k ) - On, B l ( k ) . . . Bm(k)] ( 3 . 5 )
Le veeteur des param~tres ajustables ~Mv(k) est adapt6 ~ l'alde de l'algorithme :
Fk-I Ok-I o (3.6) PMV (k) = PMV (k-l) + l + CkTl Yk-I ~k-1 ek
~c = Yk - Yk/k-I (3.7)
- - T
Fkl = Fk ! 1 + I Sk - I '$k-1 0 < I < 2 ( 3 . 8 )
Etape 2 : Commande du pr~dicteur adaptatif afin que l'objectif d~terministe :
Yk/k-I E O, soit atteint.
De l'~q. (3.4), on obtient alors :
= _ _Jl [ ~ T ( k _ l ) Ck 1 ] ( 3 . 9 ) Uk-I b o
Mais l'utilisation de la loi de eommande (3.9) conduit g :
~k = Yk (3.10)
Uk_ ! = Eq. ( 3 . 9 )
et l'algorithme d'adaptation (3.6)(qui donne le vecteur ~Mv(k) intervenant dans la
loi de co~nande (3.9)) devient :
~Mv(k) = ~Mv(k-[) + Fk-1 q~k-1
Yk (3. t~) T t + ~0k_ I Fk_ I ~k-1
L'analyse de la convergence avec probabilit~ ] de ~Mv(k) donn~ par (3.11) vers
PMV donn~ par (2.5) a ~t~ faite dans [5] en utilisant la m~thode 0.D.E. de Ljung [6]
et nous rappelons ci-dessous ee r6sultat :
Th~or~me 3.1 : (Ljung) [5]. Le R.M.V.A. d~fini par les ~qs. (3.1), (3.9) et =
173
(3.11) converge avee probabilit~ I vers le R.M.V., c'est-~-dire :
P r o b . l i m PMV (k) = PMV = 1 ( 3 . 1 2 ) k-~o
si la fonction de transfert ~chantillonn~e :
H(z-l) I X C(z-l) 2 (3.13)
est strictement r~elle positive en supposant que les trajectoires de ~(k) ont une pro-
babillt~ nulle de quitter le domalne :
D {A(z -11 . B(z -I) -A(z -1) B(z -I) = O =->Izl < i } (3.14) o~ : s
A ( z - 1 ) = 1 - (~I - e l ) z - I " ' " - (~n - e n ) z - n ( 3 . 1 5 )
-m B(z -I) ffi b ° + B l z -I . . . . + 6 m z (3.16)
IV - COMMA~E ADAPTATIVE AVEC MODELE DE REFERENCE IMPLICITE {S .C.A .M.R. IMPLICITE)
Ce type de schema bien que d~velopp~ pour un environnement de type d~terministe
est construit en utilisant une extension du principe de s~paration ~ savoir [3] :
I) Construction d'un pr~dlcteur adaptatif dont la sortie converge asymptotique-
ment vers la sortie du processus.
2) Commande du pr~dicteur conform~ment aux objectifs de poursuite ou de r~gula-
tion.
Nous proposons par la suite un S.C.A.M.R. implicite qui permet de r~aliser une
commande modale adaptative. Ce type de schema assure que lim Yk = 0 et son comporte-
ment tend vers l'objectif de la commande modale d~fini pa~T'~q. (2.9).
Le processus est d~crit par :
T Yk = Po ~k-I + bo Uk-I ; Yo # 0 (4.1)
(le vecteur Po ~tant ineonnu).
Etape ] : Le pr~dicteur adaptatif. Le pr~dicteur adaptatif (de type "s~rie-paral-
l~le") est d~crit par :
Yk/k-l = Y~ = pT(k-I) ~k-I + bo Uk-I + eT ek-I (4.2)
oO :
c T Yk/k = Yk = ~T(k) ~k-I + bo Uk-I + ek-i (4.3)
O ^O ek = Yk - Yk (4.4)
£k = Yk - Yk (4.5)
T = [ e k _ I ek_ n] (4.6) ek_ I • .. T
e = [-e I, -e 2 -.. - c n] (4.7)
174
(~ et Yk sont appel~s aussi sorties "a priori" et respeetivement "a posteriori" du
prEdicteur adaptatif [9]).
Pour determiner la loi d'adaptation de ~(k) assurant lim e k = O, pour tout o k ~
8(0) - Poet ~o' on utilise le thEor~me suivant :
o I' THEOREME 4.1 : (Landau) [8i, [9] : Soit e k et e k erreur g~n~ralis~e "a priori"
et respectivement "a posteriori" d'un syst~me adaptatif ~ module de reference (SAMR)
ec soit e k donn~ par l'Equation :
e k = H(q-l)[(po - ~(k)]~k_ 1 (4.8)
Alors, l'Eq. (4.8) et le SAMR correspondant sont globalement asymptotiquement
stable si :
F k - I qSk-I o I) ~(k) --~(k-l) + 1 + ~bTk-I Fk ~k-I e k = ~(k-]) + Fk_ I ~k-I ek (4.9)
o~ : - - ~T
Fk I = Fkl-I + ~ ~k-I k-I O~< ~ < 2 (4.;0)
2) - - -(H(z-;) --g) est une fonction de transfert strictement r~elle positive (SRP).
Des Eqs . (4.1), (4.2), (4.3), on obtient :
C(q-]) ¢k = [Po - P(k)]T ~k-] (4.11)
et, donc, en appliquant le ThEor~me 4.1, l'algorithme d'adaptation sera donne par l'~q.
(4.9) et la condition 2 devient :
S .R.P. C(z-l) 2
Etape 2 : La loi de commande. La convergence de Yk
tout Uk, nous choisissons u k tel que :
AO
Yk/k-1 = Yk - 0
De l'Eq. (4.2), on dEduit alors :
] [pT(k - cT ek_l] Uk-I = - ~ o 1) qbk_ 1 -
(4.;2)
vers Yk Etant assur~e pour
(4.13)
(4.~4)
mais comme alors : ~k/k-| - O, nous obtenons :
o ek = Yk (4.15)
Ek = Yk - [~(k) - ~(k-l)] T ~k-l (4.16)
L'algorithme d'adaptation (4.9) et la loi de commande (4.14) s'fierivent alors :
F k - | ~bk_ 1 ~ ( k ) = ~ ( k - l ) + T Yk ( 4 . 1 7 )
I + d)k_ 1 Fk_ 1 qbk_ I
175
n n
I - Z c i Yk_i ] + E ci[~(k-i) - ~(k-i-1)] T uk- I = - ~ o [ p r ( k - l ) ~k- I i= l i= l ~bk-i-I
(4.18) T
Mais, conmle lim. ~ O, on conclut de (4.9) que si ] = ~k-I Fk-I ~k-I I < M, alors : k-~=
lira [~(k) ~(k-l) ]T - ~k-1 -- o (4.19)
et done : n
I I [~,(k-l) ] (4.20) l i m uk - I ffi - h - - [ p T ( k - I ) ~k- I - ~ c i Yk-I ] = - b'-- t.,v ~k- I
k + ~ o i=l o
v - ~UALITE ~VMPTOr~uE S C a R , RMVA
Nous introduisons la d~finltion suivante de la duallt~ asymptotique entre les
syst~mes de co~nande ~ module de r~f~rence (SCAMR) et les r~gulateurs ~ variance mini-
male auto-ajustable (RMVA) :
D~finition 5.1 : (Dualit~ asymptotique SCAMR - RMVA). Un S.C.A.M.R. (explicite
eu implicite) d~velopp~ pour un environnement d~terministe et asymptotiquement dual
par rapport gun RMVA d~velopp~ pour un environnement stochastique si et seulement
si :
I) les vecteurs des param~tres ajustables sont ajust~s par des algorithmes d'a-
daptation identiques (structure, vecteur des observations (4) et erreur g~n~ralis~e
(e:) identiques),
2) les conditions de positivit~ pour la stabilit~ globale asymptotique du SCAMR
et pour la convergence avec probabilit~ I du RMVA sont identiques,
3) les lois de commande pour k + ~ sont asymptotiquement identiques.
En eomparant maintenant le RMVA pr~sent~ dans la section 3 et le SCAMR implicite
pr~sent~ dans la section 4, on eonstate qu'ils v~riflent la d~finition 5.1. En effet,
les ~qs. (3.11) et (4.17) sont identlques, les ~qs. (3.13) et (4.12) sont identiques
et les ~qs. (3.9) et (4.20) sont identiques.
VI - SCAMR EXPLICITE E~UIVALENT AU SCAMR IMPLICITE DU § 4
Le processus est d~crit comme darts le § 4 par :
T Yk = Po ~k-I + bo Uk-I ; Yo ~ 0 (6.1)
On d~finit un module de r~f~rence (sp~cifiant l'objectif de la r~gulation) :
o x k = 0 (6.2)
x k = [~(k) - ~(k-1)] T ~k-I (6.3)
On eonsid~re une loi de commande adaptative de la forme :
)76
oO :
I [~T(k_l ) - c T Uk-] = - b-- ~k-! ek-l] (6.4)
o
o o
£k = Yk - Xk (6.6)
T = [ek_ 1 .ek_ n] ( 6 7) ek_ l ..
(pour les d~tails concernant la synth~se de ce type de schema, voir [9]).
D~finitlon 6.] : (~quivalence ScAMR explicite - implicite). Un SCAMR explieite
et un SCAMR implicite sont ~quivalents si et seulement si :
l) les ~quations de l'erreur g~n~ralis~e (ek) sont identiques,
2) les algorithmes d'adaptation param~triques sont identlques,
3) les conditions de positivita pour la stabilit~ asymptotique globale sont les
m~mes.
Dans le cas present, des ~qs. (6.1), (6.3) et (6.4), on obtient :
C(q-])ek = [Po - fi(k)]T ~k-I (6.8)
qui est identique ~ l'~q. (4.11). En appliquant le Th~orSrae 4.1 et en tenant compte
dans ce cas) e~ = Yk' on obtient un algorithme d'adaptation identique ~ l'~q. que
(4.17) et une condition de positivit~ identique ~ (4.]2). Ii r~sulte que les deux
schemas seront ~quivalents au sens de la d~finition 6.],
En fait, l'existenee de SCAMR explieites et implicites ~quivalents au sens de la
d~finition 6.1 peut ~tre d~montr~e dans un contexte plus g~n~ral (poursuite et r~gu-
lation).
En fair, pour ehaque SCAMR explicite, on peut construire un SCAMR implicite ~qui-
valent au sens de la d~finition 6.] et vice-versa.
V I I - CONCLUSIONS
Les principaux r~sultats de ee travail peuvent se r~sumer sous la forme du th~o-
r~me sulvant :
Th~or~me 7.1 : Le $.C.A.M.R. implicite d~crit par les ~qs. (4.1), (4.2), (4.3),
(4.]2), (4.17) et (4.18) et le S.C.A.M.R. expliclte d~crit par les ~qs. (6,1), (6.2),
(6.3), (6.4), (4.12), (4.]7) sont ~quivalents au sens de la d~flnition 6.1 et ils
sont asymptotiquement duaux au sens de la d~finition 5.1 par rapport au R.M.V.A. d~-
crit par les ~qs. (3.1), (3,4), (3.9) et (3.11).
Le travail pr~sentO n'~puise pas tousles cas de dualit~ possibles ~tant donnO
que nous avons examin~ que le RMVA "classique" utilisant un pr~dicteur de type s~rie-
parall~le (~quation d'erreur) avec l'algorithme d'adaptation des moindres carr~s.
9
177
Mais pour ehaque schema de RMVA, on dolt pouvoir construire le SCAMR (implicite ou
explicite) asymptotiquement dual.
D'autre part, la d~marche inverse doit ~tre possible, ~ savoir, construire les
RMVA duaux correspondant aux divers SCAMR d~crits dans la litt~rature (qui souvent
font intervenir un eorreeteur ~ param~tres constants ou variables agissant sur l'er-
reur e).
Ce travail permet d'affirmer que la dualit~ existante entre la commande stochas-
tique lin~aire et la commande d~terministe lin~aire s'~tend aux cas adaptatifs et que
des sch~nms de commande adaptative d~velopp~s ~ partir des idles tr~s diff~rentes con-
duisent en fait ~ des algorithmes d'adaptation identiques asymptotiquement. Les simi-
litudes entre les RMVA et SCAMR mentionn~es d~j~ dans [I], [4] se trouvent renfore~es
par les r~sultats coneernant la duallt~ des deux approches.
L'aatear t i e n t ~ remeraier l e Prof. ASTROM ~ l e Prof. LJUNG pour l es d l s c ~ s i o n s t r ~ u ~ i l ~ q u ' i l a cues avec e~x au coups de l a pr~pa~ation de ee t rava i l .
REFERENCES
[]]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
L. LJUNG, I.D. LANDAU "Mod~ Ref~enee Adaptive Systems and Se l f -Tu~ng Regu- lators - Some Connection" 71ame Congr~s IFAC (Vol. 3, pp. 1973-1980), Helsinki Juin ]978
K.S. NARENDRA, L.S. VALAVANI "Dire~and Indirect Adaptive Control" 7i~me Con- gr~s IFAC (Vol. 3, pp. 1981-1988), Helsinki, Juin ]978
H.M. SILVEIRA " C o n , ~ b L u ~ n S ~ / a synth~se des Syst~mes AdaptatZfs avee ModUle sa~ Acc~s ~ Variable6 d'Etat" Th~se d'Etat ~s Sciences Physiques, I.N.P.G., Grenoble, Mars 1978
BO EGARD "A u~i f ied Approach to Model Reference Adaptive S y s t ~ and Sel f -Tu- n g R e g ~ r' Repport TFRT - 7134, Lund Institute of Technology, Dept. of Automatic Control, Janvier 1978
L. LJUNG "On po@a~,~ve R ~ Transfer Fun~ons and the Conv~ence of some Re- cuJtSiue Seh~gs" I.E.E.E. Trans. on Aut. Contr., Vol. AC-22, n ° 4, pp. 539- 551, 1977
L. LJUNG "Ana/yS/s of Reeu~s£ve Stochastic A~ori~", Ibld, pp. 554-575
K.J. ASTROM ~'I~A~%od~o_~ to S t o e ~ e Corutrol Theory" Academic Press, New York, 1970, (Mathematics in Science and Engineering Series).
I .D . LANDAU "An Addend~ t~ Unbiased Reeursive I d e ~ i f i ~ o n ~ i n g Mod~ Re- ference Adaptive Technique" I.E.E.E. Trans. on Aut. Contr., Vol. AC-23, n ° 1, pp. 97-99, 1978
I.D. LANDAU "Adaptive Co~t~l, the Model Reference Approach" Dekker, New York, 1978 (Control and Systems Theory Series)
K.J. ASTROM, V. BORISSON, L.LJUNG, B. WITTENMARK "Th~ry and App//cat/0ns of SeZf T u n g Regu~A" Automatica, Vol. 13, pp. 457-476, 1977
IO
NUMERICAL METHODS IN OPTIMIZATION
MI~THODES NUMI~RIQUES EN OPTIMISATION
ON TEE BERTSEKAS' METHOD FOR ~NIMIZATION OF CO~POSITE FUNCTIO}~
B.T. Poljak
Institute of Control Sciences
Moscow, USSR
I. Introduction
Most conventional methods of minimizing nondifferentiable func-
tions (for instance, the subgradient method) are applicable to func-
tions of "general form". Nevertheless, a techn4que involving identi-
fication of basic classes of the functions to be minimized is a pro-
mising approach. A very widespread kind are composite functions of the
form
where i L- is a convex nondifferentiable functional and ~ is a
smooth operator. This is the form to which problems of the best
approximation in different norms, the Steiner's problem and its
extensions, and a number of optimal control problems are reduced.
Functions of the form (I) are especially frequently encountered in
using robust methods for parameter estimation in statistics.
In his recent paper I Bertsekas has proposed a method for mini-
mizing some functions of the form (I) vahereby the initial problem is
replaced by a sequence of auxiliary problems of unconstrained mini-
mization of smooth functions obtained through a special iterative
smoothing procedure. Below that method is exSended to a wider class
of functions, reinterpreted (as a proximal point method for solution
of the dual problem) so that results on the method convergence are
obtained; new applications are found for the method (such as the
Steiner's problem); and relations with other methods are discussed.
2. Problem formulation and description of the method
Let ,~i and l~z be Hilbert spaces; ~.'/~-~ ; i 2" is a funct-
ional on H, • It is required to minimize ~X)of the form (I) on H.
In designing smoothed approximating functions we shall use the proxi-
mal mapping as introduced by Moreau 2 (see also3). For a functional
M e .Z/Ldenot e o n ,,j. Z
" e; ,
The follow~_.ng Moreau's theorem i s t rue3 : i f ~ i s a convex proper
180
lower semicontinuous functional on H# , then the functional
and the mapping ~r(ZJ are determined , ,for all Z 6~4; ~F CX) is a
convex differentiable functional on ~/~ whose gradient satisfies the
Lipschitz condition with modulus I and
ytvfzj , g.,~/zj:4illz/l~ 7g/~)-~;(z), 7N, ..iz) -- ~r (zi , (3) where F ~ is the conjugate functional
F * ( z ) = "~b I"-(~,z)- F( , f f . ('+) The method f o r ~ z i n g ( I ) i s as f o l l o w s . An i t e r a t i v e sequence
of vectors X"6/~; ~{"61/~ is generated @%er the rule
where .f~ ~ ~ is a certain numerical sequence. Consequently, each
iteration of the method (5) requires minimization of the smoothed
function ~ (~) (which is smooth for a convex ~-/~2 and a smooth
~x) in compliance with the Moreau's theorem). This m~n4m~ marion
can be approximate. From among various possible criteria we shall
men~ion only one
S '< c.' vN~ ( ~ ) 7 ~. For specifio cases this criteria can be reduced to a constructive one.
Finally, let us consider a method simpler than (5) where ~--~-Q :
3. Validation of the method
The method (5) can be obtained in different ways.
The first approach (the most similar to I) is based on the method
of penalty estimates (multiplier method) for constrained ext~emum
(see reviewS). The initial problem is equivalent to
• ~ ' . ( 8 )
Compose an augmented ~ g r a x ~ i a 3 a f o r i t
= +,.~U 7/~(.)- ~ l / (9) l (x, '<',N c) ,~",<.) + I.~ ~i., . I-~) ~ -~ "~
and use t he m u l t i p l i e r method
~' "~ ' ~ ( 1 0 ) . .qx+: = .~,~ - r Cx" - ~- "
M i n i m i z a t i o n o f /. over l~ l eads to t h e f u n c ¢ i o n ~/ , i n t r o d u c e d
181
by (2):
= ~,,= ('pfg+c,<f<). t~ ~,=-" IIJf'<L" There fo re the method takes the fo rm
---5 +.c;' ( p ) 7 - ~ . ' J , and us ing t he r e l a t i o n (3 ) we o b t a i n the method ( 5 ) - Hence (5 ) i s a
mul~iplier method for ~he constrained problem (8).
In a similar way, assuming ~ Q the multiplier method beco-
mes the penalty function method
• ., ,~ ,., , _ . / / , ~ . ~ ) ( "i'1 ) • - Z a ~ . j
which c o ~ c i d e s ~ t h t h e method ( 7 ) .
Therefore to validate the methods (5) and (7) the well known
results 4'5 on convergence of the both methods for problems with
equality constraints can be used. The latter are, however, known
chiefly for the smooth case and, since the function F is, as a
i~lle, nondifferentiable, their application is restricted.
These difficulties are avoided in another approach whereby the
method (5) is interpreted as the proximal point method for solution
of the dual problem and Rockagellar's results on convergence of
proximal algorithms 6 are used. Let ~x) be a~ine mapping, i.e.
WN--Itx-~; ~ d / : ~ ' - - , . ~ be ~ l i n e a r o p e r a t o r . T h e n ~he i , ~ . t i a l problem is of the form
and its (ordinary) dual is
where ~ ~ is the operator adjoint to ~ . The proximal method for
She latter problem is to solve a sequence of problems
~ '- :.J'IF~*: ' + :~ :, :,, ,-,, . ,,~, = ddD.~ ..h~ ~ 1 ~-.:'~,a<j .//U--:/~/I./. (14-)
Ghoosing~.~ =~.'Zand using (3) we find that the dual of (1@) is the
problem (5). Hence (5) is the proximal method for solution of ~he
dual problem (f13)° Applying theorems from 6 and general propositions
on duality 3 to this problem we can obtain the following results on
182
convergence.
Theorem I. Let F be a convex lower semicontinous functional;
W('x)= /2)~- "~, .61 "/7 ''~. ./I~ is a linear bounded operator; there exist ¢ ° ~ "
~ 6 ,~, X~ ~ H such that ~ =0, ~ ~ / ~ , ~ . - ~ ~, /~z/~. Then
solutions X ~ , ~ of the primal problem (12) and the dual problem
(~3) exist. If O<C~C "~ for all ~" , then the sequence #~ genera-
ted by (5) or (6) converges weakly to a solution / and the sequence
y.~ is an approximate minimizing in the following sense. There exist
~.Y~weakly such that ,~-'~/~9~[/,2~-~#~x~ If //~ ~,~ are finite dimens- ional, a set (Z : /-(Z).-~J is nonempty and bounded for some o(
and F is continuous then the sequence ~ i s minimizing: %~ l~Lk~-,
' Theorem ~. Su~ose /~; 'Z ~ e f i n i t o dimensio,~l , 5 is ~oly- hedral convex, ~ ( X / = / ~ - # ~ is linear, there exist a solution &~" and O(q.~,C~'% Them the method (5) is finite: .~ (~(~: ) -~ /~for a l l /~ sufficiently large.
Now let us proceed to description of the method for some speci-
fic problems.
#. ~ approximation
The discrete problem of L I approximation is of the form
x e (15)
and can be reduced to (I) with "'"~/'/=K, .,Z/, =/~ .,, I-x~.I_-~.~I.&:W, .,~ -- (~,... ~,, ) E.t ~'~, ~ ;,~ I___ __ _ . , = # ~ - -6 ,
i s a matr ix wi th rows ~L' ; and~-.(~;,... ~J,~."Introduce the funct - ion
~}</~. ~ ; l~l.~ c i~ ~ , /~/ . , c
Then the method (5) can be expressed as
, , . - _ - ,s , , i i " 2 • , -..
., ,, . i . ,~ + (16) S ' - - c,~ ~ , ff , , Y- 4 c,~ J,,.'7 Thus at each step of the method (16) it is accessory to minimize a
piecewise quadratic ftuaotion. This can be done, for instance, by
using the conjugate gradient method as modified in Ref. 7, which is
exact for functions of this kind. From Theorem 2 it follows that the method (16) converges in a finite number of iterations. The same
fact can be proved in a different way: it is easily seen tha~ (16) is
equivalent to the multiplier me~hod applied to a linear programmln~
183
problem corresponding to (15). The latter method ks as shown in
Ref. 8 finite.
The method (7) takes the form
-- -,.') c~ --0 (17) x '~ , ~d , ' ~ ,< - Z ~ , l ~ a . " , x ) 6/ ,,~ ~ ~ ~ • /
and involves simply sequential approximation of the function/,~/ by
differentiable functions
c.+. / T c , J " ~'~, i f l -~ "..: ,< ~ . f T . / -- ~ l t l - ~ c , < " , i~. l>c,< .
This method is certainly inferior of (q6) on two counts: it is not
finite and it requires that ~ ~ O, which significantly complica-
tes solution of the auxiliary problems.
For a continuous problem of ~4 approximation
~..:~. i/(o.<"~.~ .xJ - C,~.s/ t.l ~ , .x ,. ,~. Q # i"
one has ~/= / ~ Z/=~/ I%A)and the method (q) reduces to J "ii
0
where the f u n c t i o n ~ (.~] ks as above.
5. Lp approximation
Let us consider a discrete problem of /p approximation
,,,<<.,, x ) - I ' =
with [<p<',..,o . In this case the function to be minimized is diffe-
rentiable but for ~ its gradient does not satisfy the Lipschitz
condition and its direct minimization encounters numerous difficul-
ties. Therefore using the above method seems practicable. The method
then takes the form
.wt":c;<< ~' . ' ( (~<'x'</- g " c" ~ l < ) , (~o)
where ~/c~I--~. ~'IC'I¢~I:-~I')is a smooth function (its derivative satisfies the Lipschitz condition) which can be tabulated or appro-
ximated analytically. Using the results of Ref. one can prove that,
if ~/~is nonsingular, the method converges and the rate of conver-
gence is linear for ~25~ >O and superlinear for ~ ~. The same
rate of convergence is maintained if the exact minimization in
184
finding X ~ is replaced by an approximate one until the condition
~<.,<lly .-y/i, r ~ i . , . ¢ < ) l i ~ "~ '< " < o < ~ ' < ~ , (2~) is satisfied.
~. L ~ approximation
The problem of Tchebyshev approximation
~ , ~ ~,</,, '<~; ~'/- g../ , "~ ~ ~ ~'l,1. "
is reduoed t o t h e f o r m (q) with F ~ ' , J : # ~ t I ' . / ~ . ~ Introduce the function
(22)
~ :'- (~ , ... ~ , , )~ ,~ .~
This function and its gradient are easily computed for any ~ d-R~
Then the method (5) is of the form
x <-- ~ ,~"~; - K (~<-#'~=) ~" ( 2 3 )
From Theorem 2 it follows that the method (23) is finite.
Hence, the problem of T~hebyshev approximation is reduced to
sequential minimization of a differentiable piecewise quadratic func-
tions (as mentioned above, this can be done by the method of conju-
gate gradients in a finite number of iterations).
7. The Steiner's problem
This problem (alias the Neber's, Fermat's etc. problem, see
Ref. 9) is to minimize the sum of distances from t~e desired point to
the specified ones
~ ' ~ Z l i ' t " "~Z i "~ ~,<.~B ~=j , j j (2~)
H= R Representing it in the form (I) wi~h ~':" //I -~ /-~J~.--,~¢ ~"//~ J :'
~. : (~<, . . . . ~ < , ) e £ ~°', ~,..-~R~. ,. ~ . . ~ < - ~ - ~ < , .. . ,x--~.,~ ) and applying the method (5) we have
l ' ~ ' # j
/ l l ~ ~ u ~ i , . '~ ,, , I ) 6 o "~'~ ,.,~.r., "~ Ii z l/ ~ # ~ # ¢ c
~{, (:~)= ( . c / / z / / . t / ~e~ , #z / / ,*e ,~ a =~j+, . . . . .v , , / ,'~,, ~. e,,i- (2~)
( 2 6 )
185
For minimization over X one can use the followim6~ iterative pro-
cess. Equation ~ = 0 can be written as X=~'~ (~i-6~,.~)/~U,~.
: ~¢ ~.):~[ X, <.,UI,,<-:~ +~=~<.= II < } a,,d succesive approximation method for this equation reduces to
X ( ~ , O . - . : "-' -~ , ~'~--- ~.(:~"J., x~°J_- x " < . (27)
These iterations continue unbil the condition I1~ -'+0- X~'~t~'~'~,~--~=,Q'~ is satisfied, following which we let ~ -- X (~*~j and ~ i is
recalculated by formula (26). Tt would be interesting to compare
that process with the one studied in Ref. 9
The l a t t e r , which i s somewhat s impler than (25) - (26) has a s i g n i - f i c a n t disadvantage: if ~x is close to ~I~ (but ~, is not a solu-
tion) the rate of convergence is low. There is no such disadvantage
in the method (25) - (26). The method can be easily formulated for
different extensions of the problem (2#) as well, for instance, for
1"~,6~. ~ Ft///,~-£,.//,) where F,.,~) i s a convex monotone func-
t i o n or for continuous analogs of the problem.
8. Other problems
The method can also be applied to other problems, in particular,
to optimal control ones
, - - /d=.+ ~ ) ~ , . . (=9)
with functionals to be minimized in the form
f<.,,m~<'oJ ~ < "~",';<<'~d (3o) where - ~ is a convex nonsmooth function.
The method is also extendible to the case where the initial
minimization problem has constraints of different kinds.
Currently the method is numerically verified for many problems.
The results will be reported separately.
186
References
1. Bertsekas D.P. Approximation procedures based on the method of
multipliers. J. Opt~niz. Th. Appl. 23 (1977), @87-510
2. Moreau J.J. Proximite et dualiSe dans un espace hilbertien. Bull.
Soc. Math. France, 93 (1965), 273-299
3. Rockafellar R.T. Convex analysis. Princeton University Press T
Princeton, N.J., 1970
4. Ber~sekas D.P. Multiplier methods: a survey, Automatica 12 (1976),
133-145
5. Poljak B.T., Tre~ja~ov N.V. The method of penalty bounds for con-
strained e~remum problems. USSR Comp. Math. Math. Phys. 13 (1973)
~2-58
6. Rockafellar R.T. Monotone operators and the proximal point
algorithm. SLAM J. Contr. and Optimiz. 14 (1976), 877-898
7. Poljak B.T. Conjugate gradient method in extremum problems. USSR
Comp. Math. Math. Phys. 9 (1969), 807-821
8. Poljak B.T., Tretjakov N.V. On an iterative method of linear
programming and its economic interpretation. Ekon. l~at. Met.
8 (1972) 740-751 (in Russian)
9. Kuhn H.W. A note on Fermat's problem. Math. Progr. 4 (1973)
98-107.
ON £-SUBGRADIENT METHODS
OF NON-DIFFERENTIABLE OPTIMIZATION
E.A. Nurminski
International Institute for Applied Systems Analysis
Schloss Laxenburg, Austria.
1979
INTRODUCTION
The theory of nondifferentiable optimization studies extremum
problems of complex structure involving interactions of many subproblems,
stochastic factors, multi-stage decisions and other difficulties.
These features of the real problems make the objective functions
and constraints ill-defined in the sense that their differentiability
properties are not as good as would be necessary for the application
of classical mathematical programming methods.
The second and even more important reason for the development of
methods of nondifferentiable optimization is that this approach gives
some ideas for solving very difficult problems with a large number of
variables and constraints.
This paper deals with the finite-dimensional unconditional extremum
problem
min f(x) (I)
x E E n
where the objective function has no continuous derivatives with respect
to the variable x = (x I, .... Xn). Various methods were discussed and
suggested in relevant literature to solve problem (I) with many types
of non-differentiable objective functions. Bibliography published in
[I] gives a fairly good notion of these works. It should be emphasized,
that the non-differentiability of objective function in problem (I) is,
as a rule, due to complexity of the function's structure. A representa-
tive example is minimax problems where the objective function f(x) is a
result of maximization of some function g(x,y) with respect to vari-
ables y:
f(x) = max g(x,y) (2) y6 y
In this case even a simple computation of the value of f in some
fixed point may be quite a time-consuming task which requires, strictly
188
speaking, an infinite number of operations. With this in mind, it seems
to be interesting from the standpoint of theory and practice to investi-
gate the feasibility of the solution of problem (I) with an approximate
computation of the function f(x) and of its subgradients (if the latter
are determined for a given type of nondifferentiability).
During the last years the main progress in nondifferentiable opti-
mization was made due to the development of different schemes, which
used some generalization of gradients. Starting with heuristic work [2]
the proof of convergence of subgradient method was given primarily in
[3] and generalized to functional spaces in [4]. Subgradient methods
were successfully applied to many practical problems, especially at the
Institute of Cybernetics of the Ukrainian Academy of Sciences, Kiev.
N.Z. Shor and his colleagues developed the subgradient methods with space
dialation [5]. Ju.M. Ermoliev proposed and investigated subgradient
schemes in the extreme problems involving stochastic factors. The re-
sults of research by himand his colleagues were summarized in monograph
[6]. Many efficient methods of solving nondifferentiable optimization
problems were developed by V.F. Demyanov and his collaborators [7]. In
the 1970s analogous work appeared in Western scientific literature.
There were proposed methods which look like numerical algorithms that are
successful in the smooth case. The review of the state-of-art in the
West can be found in [8]. A promising class of descent methods was in-
vestigated by C. Lemarechal [9]. R. Mifflin discussed the very general
class of semi-smooth functions and developed some methods for their con-
strained minimization [10,11]. Also A.M° Gupal and V.I. Norkin [12]
proposed stochastic methods for minimization of quite general functions
which can be even discontinuous.
189
PROPERTIES OF e-SUBGRADIENT MAPPINGS
There are some difficulties in gradient-like processes of non-
differentiable optimization which are associated with the nature of these
problems. Let us consider for instance the most elaborately investigated
convex case. The subgradient g of a convex function f(x) may be con-
sidered as a vector which satisfies an infinite system of inequalities:
f(y) - f(x) > g(y-x) , for any y • E , (3)
where E is a Euclidean space. We denote the set of vectors g satisfying
(3) as ~f(x). Because (3) is as nonconstructive as the definition of a
standard derivative of a smooth function, we need some kind of differen-
tial calculus to compute subgradients. Naturally we need an additonal
hypothesis about function f(x) in that case. Quite often function f(x)
has a special structure:
f(x) = sup f(x,u) , (4) u • U
where the functions f(x,-) have known differential properties in relation
to the variables x • E. These functions may be convex and differentiable
in the usual sense. The supremum (4) for a given x, takes place on the
set U(x):
U(x) = {u~U : f(x) = f(x,u)}
any vector
gx • ~x f(x'u) ' u6U(x)
belongs to the set ~f(x), and
~f(x) = c-~{~xf(X,U) , ueU(x)}
Unfortunately, the finding of any u • U(x) may be a rather complicated
problem and a time-consuming operation. Strictly speaking, it may take
an infinite amount of time.
R.T. Rockafellar [13] proposed a notion of E-subgradient. Formally
the e-subdifferential or the set of e-subgradients ~ f(x) is the set of
vectors g which satisfy an inequality
f(y) - f(x) > g(y-x) - e , (5)
for given e > 0 and any y E E. Obviously,
f(x) D ~f(x)
and so we may hope that to find some g • ~ f(x) will be easier in compar-
ison to the problem of computing some gE ~ f(x). In fact, for the
'190
function of type (4), it is easy to see that any vector !
g e cO{fx(X,U) , u~Ue(x) }
U~(X) = [u : f(x,u) < min f(x,u) + e} u
191
NUMERICAL METHODS OF e-SUBGRADIENT OPTIMIZATION
In this chapter, we will at first study the convergence of the re-
current procedures of the kind
s+1 s s X = x - psg , s = 0,1 (6)
for finding the unconditional minimum of the convex function f. In the
above relation Ps > 0 are step multipliers,g s 6 ~esf(X s) is the es -
subgradient of the objective function f at the point x s , {e s} is a
sequence of positive numbers. Requirements placed upon this sequence
will be stipulated in the following.
In studies of the procedure (6) it is important to get the con-
vergent to the minimum sequence {x s} under the assumption of the slowest
decrease at e s. The earliest results in this field were that if
Z e/~- < m (7) s
The great theoretical advantage of U (x) is that it has some con- E
tinuity properties and it gives the corresponding continuity properties
to ~ef(x). In the following we will discuss the continuity of the point-
to-set mapping ~ f(x).
The study of the continuity properties of £-subdifferentials
started with the establishment of the properties of e-subdifferentials
which are the same as the properties of subdifferentials of the convex
R + function. In [1] upper-semicontinuity of the mapping ~ef(x) : x
E + 2 E, where R + is a non-negative semiaxis and 2 E is a family of all
subsets E, was proved, as well as the convexity and boundness of the set
of e-subgradients. It is important to say that this result was obtained
in the assumption that e > 0. If we assume that ~ is strictly positive
then it is possible to get more ingenious results. The continuity of
e subdifferential mapping when e > 0 was proved directly in the author's
work [I]. After that the author became familiar with the article [1],
where the reference to the unpublished theorem by A.M. Geoffrion was
given, from which this continuity immediately follows.
The establishment of continuity properties of e-subdifferential
mappings is of important principal significance but for practical
purposes it is necessary to get a more exact estimation of the difference
between two e-subgradient sets correspondent to the different points of
the space E.
The following theorem is valid [17] stating the Lipschitz con-
tinuity of ~£f(x) in Hausdorf metric A(A,B):
192
Theorem I. For 0 < C < e' and any x,y in the compact set K, there
exists a constant, Bk, such that
B k A(~sf(X) ,~cf(Y)) < ~-IIx-Yll
In fact, this theorem follows from the theorem of B.N. Pschenichy
(Lemma ~,I in [18]) but R.M. Chaney gave a remarkable short direct proof
which replaces in [I 7] the author's lengthy one.
The Lipschitz continuity of ~ f(x) may be efficiently used for con-
struction of numerical procedures of nondifferentiable optimization.
Then under the reasonable assumptions about the function f every cluster
point of the sequence {x s} is a solution of the problem (1). The re-
quirement (7) resulted in rather rapid decrease of £s and in turn in-
volved a great computational effort in (2). After developments in
proving techniques this requirement was essentially weakened and even
for a more general class of functions than convex it was proved [19,20]
that £s + 0 is enough for convergency.
It is also important to study the method (6) for fixed nonzero
= ~ > 0. S
In this case the following theorem is valid [19].
Theorem 2. Let the objective function f(x) be convex
ps + + 0 , Z ps = ~ , £s = e > 0
Then, if the sequence ~x s} is bounded, there exists if only one con-
vergent subsequence {x k}such that
s k lim x =
and
f(x) < min f(x) + £
x6E n
The Lipschitz continuity of e-subdifferentials gives an opportunity
to build up another class of e-subgradient algorithms in which the
directions of movement in every iteration are not directions of anti-
subgradients or anti-e-subgradients, but are weighted sums of e-
subgradients computed on the previous iterations. Such weighted sums
may have more smooth properties (see Figure I) and can bring some com-
putational advantages. So in this part we will investigate iterative
processes of the kind:
193
s+1 s s x = x - ps z
s+1 s zS_gS z = z - ~ ( ) , s = 0,1 ..... s
(8)
where
s f(x s ) g • D e
s
and es, Ps' and Ys are numerical sequences with properties that will
be specified later on.
_ gS
z S
0
Figure I
In fact, the process (8) is also a variant of c-subgradient
algorithm that follows from the Lipschitz continuity of ~ f(x) and
fundamental properties of the weighted sums {z s} of the particular
£K-Subgradients, 0 < K < S.
It was shown in [17] that
Lemma. If {x s} is bounded and
Ps - - ÷ 0 , ~ P s ~ s
then
lim inf ]Iz s - gll s ÷ ~ g • ~f (x s)
is valid.
= 0
Then the following statement on the convergence of the method (8)
can be proved.
Theorem 3. If sequence of {x s} generated by (8) is bounded, and
(i) e s ~ + 0 , Ys ~ + 0 , ~ Ps = = '
194
Qs (ii) es~s 0,
then every cluster point of the sequence {x s} is a solution of problem(1)
For the case when £s = 0 the convergence of the method (8) is
given by another theorem:
Theorem ~. Let all but condition (ii) of Theorem 3 be satisfied
and
S
Ps -- -~ 0 when s + ~s
Then every cluster point of the sequence {x s} generated by (8) is a
solution of the problem (I) .
From a practical point of view, it is useful to get the results
on convergence of algorithm (8) when e s = e = constant.
Theorem 5. Let conditions of Theorem ~ be satisfied, but
e = E > 0 ; s
then there exists a subsequence {x sk} such that
s k lim X = X , k+~
f(x) < rain f(x) + g xEE
195
References
[i] Balinski, M.L. and P. Wolfe, eds., Nondifferentiable Optimization, Mathematical Programming, Study 3, North Holland-Publishing Co., Amsterdam, 1975.
[2] Rockafellar, R.T. Convex Analysis, Princeton University Press, Princeton, N.J., 1970.
[3]
[4]
Bertsecas, D.P. and S.K. Mitter, A Descent Numerical Method for Optimization Problems with Nondifferentiable Cost Functionals, SIAM Journal Control, Voi.11, 4 (1973).
Lemarechal, C., Nondifferentlable Optimization; Subgradient and E-subgradient Methods, Lecture Notes: Numerical Methods in Optimization and Operation Research, Springer Verlag, August 1975, 191-199.
[5] Rockafellar, R.T., The Multiplier Method of Hesten and Powell Applied to Convex Programming, JOTA, vol. 12, 6 (1974).
m
[6] Nurminski, E.A., The Quasigradient Method for Solving of Nonlinear Programming Problems, Cybernetics, Vol. 9, I, (Jan-Feb 1973), 145-150, Plenum Publishing Corporation, N.Y., London.
[7] Zangwill, W.I., Convergence Conditions for Nonlinear Programming Algorithms, Management Science, Vol. 16, I (1969), 1-13.
[8] Wolf, P., Convergence Theory in Nonlinear Programming, North- Holland Publishing Co., 1970, 1-36.
[9] Meyer, G.G.L., A Systematic Approach to %he Synthesis of Algorithms, Numerical Mathematics, Vol. 24, 4 (1975), 277-290.
[10] Rheinboldt, W.C., A Unified Convergence Theory for a Class of Iterative Processes, SIAM Journal Numerical Analysis, Vol. 5, I (1968).
[11] Nurminski, E.A. and A.A. Zhelikhovski, Investigation of One Regulating Step, Cybernetics, Voi. 10, 6 (Nov-Dec.1974), 1027-1031, Plenum Publishing Corporation, N.Y., London.
[12] Nurminski, E.A., Convergence Conditions for Nonlinear Programming Algorithms, Kybernetika, 6 (1972), 79-81 (in Russian).
[13] Nurminski, E.A. and A.A. Zhelikhovski, e-Quasigradient Method for Solving Nonsmooth External Problems, Cybernetics, Vol. 13, I (1977), 109-114, Plenum Publishing Corporation, N.Y., London.
[14] Nurminski, E.A. and P.I. Verchenko, Convergence of Algorithms for Finding Saddle Points, Cybernetics, Vol. 13, ~, 430-434, Plenum Publishing Corporation, N.Y., London.
NONDIFFERENTIABLE OPTIMIZATION AND LARGE SCALE LINEAR PROGRAMMINg
Jeremy F. Shapiro Massachusetts Institute of Technology Cambridge, Massachusetts 02139 / USA
i. Introduction
In recent years, there has been a rapid development in the theory of nondifferen-
tiable optimization. In practice, this theory is required most often in the analysis
and solution of large scale linear programming problems. Our goal in this paper is
to study the relationship of the theory to these practical applications. Specifi-
cally, the main purposes of the paper are twofold:
(i)
(2)
To investigate the conceptual and practical implications of
nondifferentiable optimization methods to methods for solving
large shale linear programming problems, and vice versa;
To discuss in practical and theoretical terms the extent to
which the nondifferentiability of functions derived from and
related to large scale linear programming problems is desir-
able or unavoidable.
Before proceeding further, we must define what we mean by a large scale linear
programming problem. This is any linear programming problem with special structure
and sufficient size that decomposition methods exploiting the special structure are
more efficient than solving the problem by direct methods. The special structure can
arise naturally when a linear programming model is synthesized from a number of indi-
vidual components; for example, a coal supply model consisting of a transportation
submodel and several electric utility process submodels, one for each region, that
use the transported coal as inputs along with oil, natural gas and nuclear power to
meet given electricity demands (ICF (1977)).
Specially structured large scale linear programming problems can also arise as
approximations to smaller mathematical programming problems that are not linear such
as the convex nonlinear programming problem (Dantzig and Wolfe (1961)), the integer
programming problem (Fisher and Shapiro (1974)),(Beli and Shapiro (1977)) and the
traveling salesman problem (Held and Karp (1970)). Sometimes these linear programm-
ing approximations are so large that decomposition methods are necessary to capture
their full structure in the analysis of the problem being approximated. Finally,
linear programming decomposition methods can be used to combine mathematical pro-
gramming models with other types of models such as econometric forecasting models
(Shapiro (1977)). In this paper we will discuss inter-relationships of nondifferen-
tiable optimization techniques and decomposition methods for all of these models.
197
The plan of this paper is the following, In section 2, we discuss briefly
large scale linear programming methods and their relationship to nondifferentiable
optimization techniques. Section 3 is concerned with the central role played by
nondifferentiable optimization in the analysis of discrete optimization problems.
Nondifferentiable optimization arising in economic analysis is discussed in section
4. The final section, section 5, contains conclusions and areas of future research.
2. Linear Programming Decomposition Methods
Many of the ideas relating nondifferentiable optimization to linear programming
decomposition methods can be explained by examining the classic "block-diagonal"
structured linear programming problem ii RR
v = sin c x .... + c x (la)
s.t. Qlxl .... + Q RxR Z q (ib)
Alx I
(lc)
S.to
min R
E (crxr'k) %r,k (2a) r=l keK
r
R E E (Qrxr'k) %r,k ~ q (2b)
r=l keK r
E %r,k = i for r = i ..... R (2c) keK
r
%r,k ~ 0 for all r, k, (2d)
where the x r'k for k g K r are an arbitrary collection satisfying Arx r'k = br,x r'k ~ 0.
• ARx R = b R
i .... xR (id) x ~o, , ~o.
The principle behind the decomposition methods is to separate (i) into R + 1 smaller
linear programming problems, one coordination or Master problem concerned with the
joint constraints (ib) and R subproblems, each one using the constraint set
Ax r = br~ x r > 0. There are three separate decomposition methods that correspond to
the basic variants of the simplex method: generalized linear programming, otherwise
known as Dantzig-Wolfe decomposition, which is a generalization of the primal sim-
plex method; Benders' decomposition which is a generalization of the dual simplex
method; and a generalized version of the primal-dual simplex method. Shapiro (1978)
discusses these methods in greater detail than we will be able to do in this paper.
The Master probl~m in generalized linear programming is
198
New columns for the Master problem are generated using the vector u M of optimal
shadow prices on the constraints (2b). For this reason, generalized linear pro-
gramming is a price directive decomposition method. Specifically, the method M
requires the solution at u = u of the R subproblems
Lr(u) = min (c r - uQr)x r
s.t. Arx r = b r (3)
xr>o
An optimal solution x r'M to (3) is used to generate a new column in (2) if Lr(u) is
sufficiently small. The functions Lr(u) are plecewise linear and concave.
It is well known that generalized linear progra~mning solves, in effect, a
mathematical programming problem that is dual to problem (i). This dual problem is
d R max L(u)
s.t. u ~ O, R (4)
where L(u) = uq + I Lr(u). r=l
Since the functions Lr(u) are not everywhere differentiable, the dual problem (4)
is a nondifferentiable optimization problem. See Magnanti, Shapiro and Wagner (1976)
for more discussion about generalized linear programming and Lagrangean duality.
As a practical matter, generalized linar programming has proven to be a rela-
tively poor method for solving dual problems such as (4). Orchard-Hays (1968) and
Marsten et al (1975) report on its erratic performance. Our own experience with
generalized linear programming applied to integer programming dual problems to be
discussed in the next section is that
- the dual vectors that are generated are highly dependent
on the columns in the Master problem, and therefore,
poor choices of dual vectors will tend to persist
- large and seemingly irrelevant changes in the dual vec-
tors can be produced by the Master before a stabilized
terminal phase is reached,
- a phase one procedure to generate an initial feasible
solution in the Master can be very time consuming,
- a reoptimization of the Master can require a significant
number of iterations although the new optimal solution
is not significantly different.
Means for overcoming the deficiencies of the method become much clearer when we
recognize that it is trying to solve the nondifferentiable dual optimization problem
(4). For example, a hybrid approach that has not been widely attempted is to begin
with an ascent method for (4), such as a subgradient optimization, and then switch
to generalized linear programming after the ascent method has generated a sufficient
number of "good" columns for the Master problem. Since it is difficult to know
when the switch should be made, it may be necessary or desirable to change back and
199
forth between the methods several times. Note also that, if it has a feasible solu-
tion, the Master problem (2) provides an upper bound on the maximal objective function
value d in (4) being sought by an ascent method. Thus, generalized linear progra~mmlng
can be used to provide termination criteria for an ascent method,
Benders' decomposition method for problem (i) is resource directive. The method
is derived from a reformulation of problem (i) as the nondifferentlable optimization
problem
v = mln vl(q I) + .... + vR(q R)
s.t. ql + .... + qR ~ q, (5)
where
vr(q r) = min c rxr
s.t. Qrxr = qr
Arx r = b r
r x > 0.
The functions v r are piecewise linear and convex. Each can be approximated from
below at any point qr,k by the linear function v r'k + yr,kqr where v r'k = vr(q r'k)
_ yr,kqr,k and yr,k is any subgradient of v r at qr,k. These approximations are used
to construct theM-aster problem for Benders' decomposition
min v I + .... + v R (6a)
s.t. r > vr,k + r,kqr for all k E K -- r
r = 1 ..... R (6b) 1 qR
q + . . . . + Z q , (6e )
where K i s an a r b i t r a r y i n d e x s e t o f l i n e a r a p p r o x i m a t i o n s to v r . L e t t i n g
q l , M . . r . , q R ' M d e n o t e an o p t i m a l s o l u t i o n to ( 6 ) , t h e me thod p r o c e e d s by c o m p u t i n g
v r ( q r 'M) f o r a l l r t o s e e i f new l i n e a r a p p r o x i m a t i o n s t o v r a r e n e e d e d f o r a c c u r a c y
in ( 6 ) .
Benders' decomposition method can suffer from the same deficiencies as general-
ized linear programming. This is not surprising since the two methods can be shown
to be intimately related through linear programming duality theory (Lasdon (1970)).
We have implemented Benders' method to decompose a large scale linear programming
coal supply model (Shapiro and White (1978)). For this application, the Master
problem describes the extraction of coal over T time periods by supply region and
sulfur type, and there are T subproblems, one for each time period, that describe
the distribution of coal to demand regions in order to meet fixed coal demands and
environmental constraints. The overall objective function is the minimization of
supply and distribution costs over the T periods to meet fixed coal demands. Our
experience thus far is that Benders' method tends to produce erratic values for the
resource vectors ql,M ...,qR,M in much the same way generalized linear programming
produces erratic dual vectors. We are implementing a hybrid approach combining sub-
gradient optimization and Benders' method to try to overcome this difficulty. Again,
the idea is, initially, to treat (5) as a nondifferentiable ascent problem using
200
subgradient optimization, Benders' method would be invoked after a sufficient
number of inequalities (6b) were generated.
The contrast between generalized linear programming and Benders' method is
that with the former, the Master produces an erratic sequence of dual variables
on the shared resources whereas with the latter the Master produces an erratic
sequence of primal variables partitioning the shared resources. This difficulty
can be partially overcome if additional structure can be placed on the Master prob-
lem; for example~ a priori lower bounds on the qr in the Benders' subproblems to
ensure feasibility. In the case of generalized linear programming, however, the
dual variables are induced by the actual (primal) problem being solved, and there
is little insight available for placing constraints on them. The BOXSTEP method
proposed by Marsten et al (1975) provides a solution to this difficulty by restrict-
ing the dual variables to lie within boxes or bounded regions. A systematic search
of the boxes ensures global optimality. Marsten (1975) reports on experiments
contrasting BOXSTEP and subgradient optimization.
The generalized primal-dual method has not yet received much attention and has
not been extensively tested. Nevertheless, it provides considerable insight into
the relationship between nondifferentiable optimization and large scale linear pro-
gramming. In the context of the block diagonal problem (i), the primal-dual can be
applied with problem (i) as the primal, or its dual as the primal. We will discuss
it briefly for the former case when the generalized primal-dual method can be inter-
preted as an ascent algorithm for the dual problem (4)~ At an arbitrary point u ~ 0,
the method systematically generates extreme points of the subdifferential ~L(u)
until a direction of ascent is found or u is proven to be optimal. If an ascent
direction is found, the method moves to the nearest point in that direction where L
is not differentiable, and repeats the procedure. We have tested the primal-dual
method for large scale linear programming problems arising in integer programming
(Fisher, Northup and Shapiro (1975)). The method worked well and we intend to
implement it again for further experimentation and comparison with the other decom-
position methods.
3. Lagrangean Kelaxation of Discrete Optimization Problems
Nondifferentiable optimization plays a central role in the use of large scale
linear programming problems to approximate discrete optimization problems. For
expositional convenience, we will focus our attention on the zero-one integer pro-
gramming problem, but the constructs and results are valid for a much wider class
of discrete optimization problems. We will discuss other problems at the end of
this section.
We consider the zero-one integer programming problem
v = rain cx
s.t. Ax = b (7)
x. = 0 or i for all j, J
201
where A is m x n with columns aj and A and b have integer coefficients. Large scale
linear programming approximations are systematically constructed for problem (7) by
combining methods of elementary number theory and group theory with mathematical pro-
gramming duality. Our discussion here will be brief and the reader is referred to
Bell and Shapiro (1977) and Shapiro (1978) for more details.
Our method begins with an aggregation of the equations Ax = b. Let ~ denote
a homomorphism mapping Z TM, the group of integer m-vectors under ordinary addition,
onto G~ a finite abellan group. Applying ~ to both sides of Ax = b, we obtain
n =
(Ax) jbl#(aj)xj = #(h) (8)
Since the set of zero-one vectors satisfying Ax = b is contained in the set of zero-
one vectors satisfying (8), we can append (8) to (7) without affecting it. The
result is
v = min cx (9a)
s.t. Ax = b (9b)
n
~(a,)x. = ¢(b) (9c) j=l J J
x. = 0 or 1 for all j. (9d) 3
For future reference, we define
X = {x I x satisfies (9c) and (9d)}. (i0)
An integer programming dual problem is constructed by dualizing on the original
constraints (9b). Specifically, for all u e R m, we define the Lagrangean
Z(u) = ub + min (c - uA)x (ii) xEX
and the zero-one integer programming dual problem
d = max Z(u)
s.t. u g R m • (12)
The Lagrangean calculation is a group optimization problem that can be solved by a
discrete algorithm (see Glover (1969) or Shapiro (1978)). The algorithm is quite
efficient when the order of the group G is less than i0,000. The function Z is
piecewise linear and concave and therefore, the dual problem (12) is a nondifferen-
tlable optimization problem. Moreover, it can easily be shown that Z(u) < v for all
u g Rm~ and therefore d ~ v. However, we may have for some dual problems that d < v,
i.e., there is a duality gap, due to the nonconvex structure of the zero-one integer
programming primal problem (7). This point is discussed further below.
A primary purpose of the dual construction is the establishment of the follow-
ing sufficient optimality conditions for the zero-one integer programming problem
~7). These conditions may not he met in which case the dual problem can be strength-
202
ened by changing the homomorphism ~. Since Z(u) provides a lower bound on v, a second
primary purpose of the dual construction is to provide lower bounds for %ranch and
bound methods (see Fisher, Northup and Shapiro (1975) or Shapiro (1978)).
GLOBAL OPTIMALITY CONDITIONS: The pair (x, ~) with x g X are said to satisfy the glo-
bal optimality conditions for the zero-one integer programming problem (7) if
(i) Z(u) = ub + (e - ~A)
(ii) A~ = b.
Theorem i: If (x, u) satisfy the global optimality conditions for the zero-one
integer programming problem (7), then x is optimal in (7), u is optimal in the zero-
one integer programming dual problem (12), and moreover, d = v.
Proof: See Shapiro (1978).11
The implication of theorem 1 is that, in seeking to establish the global optimality
conditions, we solve the nondifferentiable dual problem to find an optimal dual solu-
tion ], and then try to find an x £ X that satisfies the optimality conditions (i)
and (ii).
The dual problem (12) can be solved by any one of the three linear programming
decomposition methods discussed in the previous section. To see why this is so, let
{xt}T=it be an enumeration of the set X. The finiteness of X permits us to write X
problem (12) as the linear programming problem
d = max t
< ub + (c - uA) x
The linear programming dual to this problem is T
d = min E (cx t) A t t=l
T s.t. Z (Ax t) % = h
t= 1 t
T E
t=l %t = i
% >0 t --
for t = 1 ..... T. <13)
(14)
for all t.
For example, generalized linear programming can be used to generate columns for prob-
lem (14).
The central role played by nondifferentiable optimization in the solution of the
zero-one integer programming problem (7) using dual methods is exposed by relating
the above constructs to properties of the Lagrangean function Z. We define the set
T(~) = {t I Z(~) = ~b + (c- ~A) x t} ;
the vector yt = b - Ax t for any t E T(u) is a subgradient of Z at u. Moreover, the
203
set
~Z(]) = {Y I Y = Z %t Yt, Z %t = l, %t > O} tgT(u) teT(u) --
is the subdifferential of Z at u; i.e., ~Z(u) is the set of all subgradients of Z at
~. The function Z is differentiable at ~ if SZ(u) consists of a singleton. Since
(12) is a concave maximization problem, it can be shown that ~ is optimal in (12) if
and only if 0 g ~Z(u).
Theorem 2: Suppose the zero-one integer programming Lagrangean function Z is differen-
tiable at any optimal solution u to the dual problem (12). Let ~ = b - Ax for x e X
denote its gradient at that point. Then the pair (x, ~) satisfy the global optimality
conditions implying x is optimal in the zero-one integer programming problem (7).
Proof: If ~ is optimal in problem (12) and if Z is differentiable at that point, then
VZ(~) = y = 0 since (12) is the unconstrained maximization of a concave function.
Thus, Ax = b as well as Z(u) = ub + (c - uA)x establishing the global optimality con-
ditions, ll
Theorem 2 tells us that we have been fortunate in our construction of the dual
problem (12) if Z is differentiable at the optimal dual solution found by one of the
large scale simplex methods. Conversely, the dual problem will probably fail
to find an optimal solution to the primal problem if Z is not differentiable at such
a dual solution u; that is, if ~Z(u) is larger than a singleton. In this sense, non-
differentiability is an unavoidable difficulty that can arise in the use of large
scale linear programming to solve the zero-one integer programming problem (7). The
difficulty can be overcome by the construction of a stronger dual problem that we may
find to be differentiable at an optimal dual solution. The following theorem indicates
the operational point of departure for the construction.
Theorem 3: (Bell and Shapiro (1977)) Suppose the zero-one integer programming dual
problem in the large scale linear programming form (14) is solved by a simplex method.
If only one I t is positive in the computed optimal basic feasible solution, say l I = i,
then x I is optimal in the zero-one integer programming problem (7). If more than one
I t is positive in the computed optimal basic feasible solution, say, l I > 0, 12 > 0,
.... l K > 0, ~t = 0 for t > K, then Ax k # b for k = 1 ..... K.
The stronger dual is constructed by applying a number theoretic reduction proce-
dure to the indicated optimal basis for problem (14). The result is a new homomorphism
#' from Z TM to a new group G' with the properties
(i) X' ~ X,
where n X' = {x I Z ~'(aj)xj = ~'(b)} ,
and j =I n Z ~'(aj)xj k # ~'(b) for k = i ..... K. (ii)
j-i
204
The set X' is used in the construction of the new Lagrangean Z'(u) analogous to (ii)
and dual problem analogous to (12). Since, by construction, the active subgradients k K
~Z(u) for k = I,...,K satisfying ~ yk ~ = 0 are not contained in the subdifferen Y E k=l
tial ~Z'(u), we may proceed on the assumption that 0 ~ ~Z'(~) and attempt to ascend
in the new dual problem from u. Clearly, the construction of increasingly strong
dual problems must ultimately lead us to one for which the first case in theorem 3
obtains. A sufficient condition for this to occur is that the Lagrangean is differen-
tiable at the corresponding optimal dual solution.
Lagrangean duals have been proposed and used on a variety of other discrete
optimization problems (see Shapiro (1977)). A notable application to the traveling
salesman problem is due to Held and Karp (1970). They exploited an imbedded spanning
tree structure in the construction of a dual to the traveling salesman problem. The
same group theoretic procedures discussed above could be used to strengthen the
traveling salesman dual, but it has not been tested experimentally. Geoffrion (1974)
discusses the use of Lagrangean dual techniques to exploit special structures arising
in integer programming. For all of these discrete optimization problems and their
duals, the analysis embodied by the global optimality conditions and theorems i, 2
and 3 remains valid. Thus, nondifferentiable optimization is an unavoidable aspect
of discrete optimization.
4. Economic Analyses
Large scale linear programming models have found increasing use in economic
analyses of many types, particularly in combination with econometric forecasting
models. Examples of these models can be found in energy planning (Cherniavsky (1974),
Griffin (1977)),industrial planning (Goreux and Manne (1973)), international exchange
(Ginsburgh and Waelbroeck (1974)) and others. Our purpose here is not to survey these
applications, but to address briefly some of the consequences of using linear pro-
gramming models to study economic phenomena. On the one hand, the data for linear
programming models are easily derived point estimates of costs, efficiencies, scarce
resources, and so on, and large models incorporating vast quantities of data can be
optimized. On the other hand, parametric analyses of linear programming models can
produce non-smooth (i.e., nondifferentiable) curves that may cast doubt on the vali-
dity of the model.
Consider, for example, figure i which shows the demand curve for coal in the
U.S. in 1985 derived from the Brookhaven Energy System Optimization Model (BESOM;
see Cherniavsky (1974)). This model is a linear programming problem describing in
a highly aggregate manner how fixed energy end-use demands can be met at minimal
cost by converting primary supplies using electric and non-electric technologies.
The variables in BESOM are the levels of primary supplies and the energy flows through
the conversion devices. The particular model analyzed in figure i was used by ERDA
(Energy Research and Development Administration) to study the effects of a nuclear
power moratorium on the U.S. energy sector. It consists of approximately 150 con-
205
Derived Demand Curve for Coal U.S. Energy Sector - 1985
(BESOM)
price
$ I i0 6~ BTU
3,0
2.0
1.0
3.22
2.86
O
1.55
1.16
Y
" I i0
f
.380
I 20
1015 BTU
Supply price level
.-- quantity 30
BTU = British ~ermal Unit
Figure i
206
straints and 500 variables.
The step function demand curve shown in figure i was derived by treating coal
supply as a parameter in BESOM. The level portions of the function correspond to the
shadow prices associated with the optimal linear program~ning bases encountered during
the parametric analysis. If we ignore the very small level portion at $1.16/106 BTU,
the demand curve jumps significantly at a quantity of about 19 x i015 BTU from
$.38/106 BTU to $1.55/106 BTU. Since most estimates (e.g., Zimmerman (1977)) of coal
supply at this quantity put the price at about $1.00/106 BTU, the supply of coal in
BESOM is not, in fact, variable but it is a quantity effectively fixed at 19 x 1015
BTU.
The derived demand curve shown in figure 1 is an extreme example of a poten-
tially general undesirable property of large scale, nondifferentiable linear programm-
ing models for economic analysis. The model and the policy studies on which it has
been based would improve considerably by the introduction of meaningful, smooth non-
linear functions. For example, nonlinear supply curves for the other primary supplies
such as petroleum and natural gas would smooth out the derived demand curve of figure
i and introduce stability into the parametric analysis; that is, small changes in price
would cause small changes in quantity demand. Shapiro, White and Wood (1976) experi-
mented successfully with this idea and coincidentally, used generalized linear pro-
gram~ning to approximate the nonlinear supply curves.
As a final point in this regard, we mention the pseudo-data approach applied
by Griffin (1977) to a linear programming model of electric power generation to meet
given demand at minimum cost. Griffin formally derives highly nonlinear nondifferen-
tiable functions from the results of parametric linear programming analyses. The
resulting functions can then be used in other mathematical programming models to
study, for example, capacity expansion of electric utilities.
5. Conclusions and Areas of Future Research
We have tried to demonstrate in this paper the intimate relationship that exists
between nondifferentiable optimization and large scale linear programming. An impor-
tant area of future research in this regard is the experimental integration of ascent
methods of nondifferentiable optimization, such as subgradient optimization, and de-
composition methods for large scale linear programming. Hybrid algorithms using all
of the methods discussed could prove to be highly successful.
We have seen that nondifferentiable optimization is unavoidable in the analysis
of discrete optimization problems by large scale linear programming. These large
scale problems are derived from the application of mathematical progra~ning duality
theory to exploit special structures of the discrete optimization problems. There
is further research to be done on the strengthening of dual problems when there is
a duality gap. Related future research can be done on the use of dual problems and
nondifferentiable ascent methods for solving them in the context of branch and bound.
The branch and bound approach to discrete optimization effectively produces a family
207
of closely related nondifferentiable optimization problems. Properties of such a
family need to be better understood and related to more efficient algorithms for
relating ascent steps and optimal solutions among problems in the family.
We have also tried to demonstrate the possible undesirability of unsmooth or
nondlfferentlable derived supply and demand curves resulting from linear programm-
ing models of economic phenomena. This deficiency of the models can possibly be
overcome by the use of nonlinear, highly differentiable econometric functions to
summarize unsmooth linear programming parametric functions; the pseudo-data approach
suggested hy Griffin (1977). There are two research areas related to this approach.
One is to try to understand the implied choice between decomposition methods for
large scale linear programming, which are exact but nondifferentiable, and the
pseudo-data approach which is inexact but differentiable. The other research area
is the determination of second order information for nondifferentiable functions
analogous to Hessian matrices. The use of meaningful second order information could
also lead to more stable decomposition methods for large scale linear programming.
6. Acknowledgement
The research reported on here was supported in part by the U.S. Army Research
Office Contract DAAG29-76-C-0064 and in part by the National Science Foundation
Grant MCS77-24654.
208
References
i. Bell, D. E., and J. F. Shapiro (1977), '~ convergent duality theory for integer programming", Operations Research, 25, pp. 419-434.
2. Cherniavsky, E. A. (1974), "Brookhaven Energy System Optimization Models", Report BNL 19569, Brookhaven National Laboratories, December, 1974.
3. Dantzig, G. B., and A. Wolfe (1961), "The decomposition algorithm for linear programming," Econometrica, 29.
4. Fisher, M. L., and J. F. Shapiro (1974), "Constructive duality in integer programming", SlAM Journal on Applied Mathematics, 27, pp. 31-52.
5. Fisher, M. L., W. D. Northup, and J. F. Shapiro (1975), "Using duality to solve discrete optimization problems: theory and computational experience," in Math. Prog. Study 3: Nondifferentiable Optimization, pp. 56-94, M. L. Balinski and P. Wolfe (eds.), North-Holland.
6. Geoffrion, A. M. (1974), "Lagrangean relaxations for integer programming," in Mat h. Prog. Study 2: Approaches to Integer Programming, pp. 82-114, M. L. Balinski (ed.), North-Holland.
7. Ginsburgh, V. A., and J. Waelbroelk (1974), "Linear programming planning models and general equilibrium theory," Discussion paper No. 7421, Center for Operations Research and Econometrics, Louvain, Belgium.
8. Glover, F. (1969), "Integer programming over a finite additive group," SlAM Journal on Control, 7, pp. 213-231.
9. Goreux, L., and A. S. Manne (1973), (Editors), Multi-Level Planning: Case Studies in Mexico, North-Holland.
i0. Griffin, J. M. (1977), "Long-run production modeling with pseudo data: elec- tric power generation," Bell Journal of Economics, 8, pp. 112-127.
ii. Held, M., and R. M. Karp (1970), "The traveling salesman problem and minimum spanning trees," Operations Research, 18; pp. 1138-1162.
12. ICF (1977), Coal and Electric Utilities Model Documentation, ICF Inc., Wash- ington, D.C.
13. Lasdon, L. (1970), Optimization Theory for Large Systems, MeMiilan.
14. Magnanti, T. L., J. F. Shapiro and M. H. Wagner (1976), "Generalized linear programming solves the dual," Management Science, 22, pp. 1195-1203.
15. Marsten, R. E. (1975), "The use of the boxstep method in discrete optimization," Math. Prog. Study 3: Nondifferentiable Optimization, pp. 127-144, M.L. Balinski (ed.), North-Holland.
16. Marsten, R. E., W. W. Hogan and J. W. Blankenship (1975), "The boxstep method for large scale optimization," Operations Research, 23, pp. 389-405.
209
17.
18.
19.
20.
21.
22.
23.
Orchard-Hays, W. (1968), Advanced Linear Programming Computing Techniques, McGraw-Hill.
Poljak, B. T. (1967), "A general method for solving extremum problems," Soviet Mathematics DokladJ, 8, pp. 593-597.
Shapiro, J. F. (1977), "A survey of Lagrangean techniques for discrete optimization," Technical Report No. 133, Operations Research Center, Massa- chusetts Institute of Technology.
Shapiro, J. F. (1978), Mathematical Program~ning: Structures and Algorithms, (in press), John Wiley, Inc.
Shapiro, J. F., and D. E. White (1978), "Integration of nonlinear coal supply models and the Brookhaven energy system optimization model (BESOM)," Working Paper No. OR 071-78, Operations Research Center, Massachusetts Institute of Technology.
Shapiro, J. F., D. E. White and D. O. Wood (1977), "Sensitivity analysis of the Brookhaven energy system optimization model," Working Paper No. OR 060-77, Operations Research Center, Massachusetts Institute of Technology,
Zimmerman, M. B. (1977), "Modeling depletion in a mineral industry; the case of coal," Bell Journal of Economics, 8, pp. 41-65.
ALGORITHMS FOR NONLINEAR MULTICOMMODITY
NETWORK FLOW PROBLEMS
Dimitri P. Bertsekas Coordinated Science Laboratory
University of I11inois Urbana, Illinois 61801, U.S.A.
ABSTRACT
This paper presents a class of algorithms for optimization of convex multi-
commodity flow problems. The algorithms are based on the ideas of Gallager's
methods for distributed optimization of delay in data communication networks [i],
[2], and gradient projection ideas from nonlinear programming [3],[4].
ACKNOWLEDGMENT
This work was done in part at the Massachusetts Institute of Technology,
Cambridge, Massachusetts and supported by ARPA under Grant N00014-75-C-I183, and
in part at the University of Illinois, Urbana, Illinois and supported by NSF Grant
ENG 74-19332.
i. INTRODUCTION
Consider a network consisting of N nodes denoted by 1,2,...,N and L directed
links. We denote by (i,~) the link from node i to node ~, and assume that the net-
work is connected in the sense that for any two nodes m,n there is a directed path
from m to n. The set of links is also denoted by L.
We consider the following multicommodity flow problem in the variables fi%(j),
j=I,...,N, (i,~)EL:
N minimize E D..[ ~ f.~(j)]
(i,~)EL i~ j=l mE (MFP)
subject to ~60(i)fi~(j)E " -m61(i)~ fmi(J ) =ri(J) , gi=l ..... N,i#j
fi~(j ) ~ 0, V(i,~)EL, i=l ..... N, j=l ..... N
fj~(j) = 0, Y(j,%)EL, j=l ..... N,
where fi%(j ) is the flow in link (i,~) destined for node j, 0(i) and l(i) are the
sets of nodes % for which (i,~)EL and (%,i)EL respectively, and, for i#j, ri(J) is a
known traffic input at node i destined for j. Each link (i~) has associated with
it a number Ci~ , referred to as the capacity of the link, which is assumed positive
or +=. The standing assumptions throughout the paper are:
a) ri(J) ~ 0, Vi,j=I,2,...,N, i#j.
b) The functions Di~ are defined on [0,Ci% ) and are convex functions, twice con-
tinuously differentiahle with positive first and second derivative everywhere
211
on [0,Ci~). (The derivatives of Di~ at zero are defined by taking limit from
the right.)
The problem above arises in a variety of contexts, most prominently in delay
optimization problems in data cormnunication networks, and equilibrium studies of
transportation networks. In a conventional setting the problem will be solved eom-
putationally at a central location (a computer laboratory or a central node), and
the algorithms and analysis of this paper are applicable for such intended use.
However, our emphasis is in distributed algorithms for routing of flow in communica-
tion networks, where the problem is solved in real time, with each node participa-
ting in the computation by adjusting the variables under its control on the basis of
local information exchanged with its immediate neighbors. Furthermore since the
(average) traffic inputs ri(J) change with time, the algorithm is continuously in
progress, and forms an integral part of the supervisory control system of the network.
The starting point of this paper is the gradient projection method due to
Goldstein [3], and Levitin-Polyak [4]. We briefly review a version of the method in
Section 2. We subsequently show that the method is well suited for multicommodity
flow optimization provided the problem is formulated in terms of the coordinate
system of routing variables used by Gallager [I]. This leads to a fairly broad
class of algorithms including Gallager's first method [I]. Some of these algorithms
employ second derivatives and Newton-like iterations. We show how approximations to
these derivatives can be computed in a distributed manner. A convergence result is
given under a simplifying assumption on the traffic inputs.
For notational convenience we restrict ourselves to algorithms for the single
commodity problem. But these have obvious multicommodity counterparts whereby a
multicommodity iteration consists of N single commodity iterations. However it is
as yet unclear whether the single commodity iterations should be carried out simul-
taneously for all commodities (as in [i] and [2] ), sequentially (one commodity at a
time), or in (strategically chosen) blocks of commodities. This matter is currently
under investigation.
Regarding notation, we denote by R and R n the real line and n-dimensional space.
n (xi)2] ~. The usual norm in R n is denoted by ['I , i.e., for x = (x I,... ,Xn) , Ixl =[i~i.=
All vectors are considered to be column vectors. Primes denote transposition or
derivative. Vector inequalities are considered to be eomponentwise.
2. THE GOLDSTEIN-LEVITIN-POLYAK GRADIENT PROJECTION METHOD
Consider the constrained optimization problem
minimize f (x) (1)
subject to Ax =b, x 20
where f:Rn-~R is a twice continuously differentiable function, A is an nMn matrix
and b is an m-dimensional vector.
212
One version of the gradient projection method operates as follows. An initial
feasible vector x ° is given and subsequent vectors are generated by the iteration
~+i = ~k (2)
where ~k solves the problem i
minimize vf(xk)'(x-xk) +~ (X-Xk) Mk(X-Xk) (3)
subject to Ax = b, x ~ 0
and M k is a symmetric matrix which is positive definite on the nullspace N(A) of A,
i.e. x'~x > 0, Vx#0, x6N(A). (4)
The auxiliary problem (3) may be viewed as minimization of a quadratic approxi-
mation of f over the constraint set. When Mk=V2f(xk ) [with V2f(xk ) assumed posi-
tive definite on N(A)] we obtain a constrained version of Newton's method. When
problem (i) has a unique minimum ~ with V2f~): positive definite on N(A), then
iteration (2) can be shown to converge to x at a superlinear rate provided the
starting point is sufficiently close to ~ ([4], Th. 7.1). In many problems, however
solution of (3) with ~=V2f(xk ) is impractical, and often ~ is taken to be some
approximation to V2f(xk ) (for example a diagonal approximation). Note that if ~ is
invertible one may write problem (3) as
minimize ½[x-x k +~f (Xk)] '~[x-x k +ilvf (Xk)]
subject to Ax=b, x > 0.
Thus Xk is the projection of Xk-i~f(xk) on the constraint set with respect to the
norm corresponding to ~.
We now show that choosing ~ "sufficiently large" leads to a convergent algo-
rithm. Since ~k solves problem (3) we have for all feasible x
[vf(xQ +~(~ ~k)] ' C~-x) ~ 0,
and setting x=x k we obtain
Vf(xk)'CXk-Xk) = -~k-Xk)'Mk(-Xk-Xk). (5)
We also have
f(Xk+l) = f(-~k) = f(xk) +Vf(xk)' (~k-Xk)
i +~ [Vf[x k + t (~k-Xk)] -Vf (xk)] ' ~k-Xk)dt. (6) 0
If we assume that there exist scalars %>0, A>0 such that for all k
k]x! 2 < x'~x < Alx[ 2, Vx6N(A), (7)
and a scalar L such that
Ivf(y)-Vf(z)[ ~ L]y-z[, Vy,zE[x[Ax=b, x>__0} (8)
then, using (5)-(8), we have
213
f(xk)-f (Xk+l) >__ -vf (Xk)' (~k-Xk) - ~II Vf[x k + t (-xk-xk)] -~Tf %)I I%- I dt 0
I _ 2 > (Xk'Xk)'Mk(-~[k-Xk)"~0 tL[xk-xkl dt
L 2 xl%- l z -~ l~k-xkl L 2
= ~-~')l~'k-Xkl
It follows that if L [< ~ (9)
the algorithm decreases the value of the objective function at each iteration, and a
straightforward argument using (7) shows that every limit point ~ of [Xk} is a sta-
tionary point in the sense that ?f(~)'d > 0 for all feasible directions d at ~.
3. THE SINGLE COMMODITY PROBLEM
Consider the special case of (MFP) where all flow has as destination a single
node, say N. That is we have ri(J) = 0 for all i and j#N. By suppressing the com-
modity index, we can then write (MFP) as
minimize E (fi~) (i,%)Di%
subject to E f. - E ~60(i) z~ mEi(i)fmi
fi~ ~ 0, (i,~)EL,
Let t. he the total incoming traffic at node i i
t i = r i +mE~(i)fmi '
and for ti#O let ~i~
= ri, i=l,...,N-i
i=l,...,N-l.
i=l,...,N-l,
be the fraction of t. that travels on link (i,~) i
fi~ ~ i ~ ~ t . " i = l , . . . , N - 1 ( i , I ) E L .
1
(SFP)
Then it is possible to reformulate (SFP) in terms of the variables ~i~ as follows [i].
For each node i~N we fix an order of the outgoing links (i,~), ~60(i). We
identify with each collection [~i~ I (i,~)6L, i=l ..... N-l} a column vector ~ = (~i,~,
.... ~_i )', where ~i is the column vector with coordinates ~i~' ~E0(i). Let
= [~{~i~ ~ 0, ~(i)~i~=l, (i,~>CL, i=l ..... N-l)
and let ~ be the subset of ~ consisting of all ~ for which there exists a directed
path (i,%),...,(m,N) from every node i=l,...,N-I to the destination N along which
~i~ >0'''''~mN>0" Clearly ~ and ~ are convex sets, and the closure of ~ is ~. It
is shown in [i] that for every ~6~ and r = (rl,r2,...,rN_l)' with r i~0, i=l, .... N-I
there exist unique vectors t(~,r) = (tl(~,r),...,tN_l(~,r))' and f(~,r) with coordi-
nates fi~(~,r), (i,~)EL, i#N satisfying
t (%0,r) >_ 0, f(cp,r) >__ 0
214
ti(~,r) = r i+ ~ f .(~,r), i=l,2,...,N-I m61(i) ml m#N
f. (~,r) - ~ fmi (~,r) = ri, i=l ..... N-I ~60 (i) l~ m61 (i)
fiI(~,r ) = ti(~,r)~i~ , i=l,...,(i,~)6L.
Furthermore the functions g(~,r), f(~,r) are twice continuously differentiable in the
relative interior of their domain of definition ~x[rlr~0}. The derivatives at the
relative boundary can also be defined by taking limit through the relative interior.
Furthermore for every r~0 and every f which is feasible for (SFP) there exists a
~6~ such that f= f(~,r).
It follows from the above discussion that (SFP) can be written in terms of the
variables ~i~ as
minimize D (~,r) = Z D.~[ fi~ (~,r) ] (i0) (i,~)6L l~
subject to ~E~,
where we write D(~,r) == if fi~(~,r ) ~ Ci~ for some (i,~)EL. It is easy to see that
an optimal solution exists for both (SFP) and the problem above, provided the optimal
value is finite. It is possible to show that if {~k}c~ is a sequence converging to
an element ~ of ~ which does not belong to ~, then lim D(~k,r) ==. Thus for any k~
scalar D O the set {~ID(~,r) ~ D o } is compact, and if a descent algorithm is used
to solve the problem above the possible discrepancy between ~ and ~ is inconsequen-
tial. Now problem (i0) is of the type considered in the previous section, and
gradient projection is well suited for its solution because of the decomposable
nature of the constraint set. We are thus led to the iteration
k+l -k ~i = ~i i=l .... ,N-I (II)
where ~i solves the problem
~D(~k~r) ' k k , k k minimize ~Pi (q~i-~i) +~(%°i-q°i) Mi(q°i-~Pi)
(12) subject to ~0i>__O , r~pi~=l
. ~D(~k~r) . . . . . . . . . . . . . ~D (q0k, r) wnere - is Kne vecEor with coordinates Ene parKzai Gerlva~ives
0(i) evaluated at (~p ,r). This corresponds to the gradzent projection method (2)
with ~ in (3) being a block diagonal matrix with M k i' i=l,...,N-I along the diagonal.
Actually the subproblem (12) will be modified later in this section. The algorithm
(11)-(12) is introduced here in order to motivate subsequent developments.
Gradient Computation and Optimality Conditions
In (ii) we must ensure that q0k+16~ (assuming q0k6~) for otherwise the algorithm
breaks down. This can be achieved by insisting that both q0 k and q0 k+l are loopfree,
a device that not only ensures that q0k6~ for all k but also allows efficient compu- 5D
ration of the derivatives r----- needed in (12). We briefly discuss this. A detailed o%°iZ
analysis may he found in [I].
215
For a given ~6~ we say that node k is downstream from node i if there is a
directed path from i to k, and for every link (~,m) on the path we have ~m > 0. We
say that node i is upstream from node k if k is downstream from i. We say that ~ is
loopfree if there is no pair of nodes i~k such that i is both upstream and downstream
from k.
For any ~6~ and r~0 for which D(~,r) <~ the partial derivatives ~i% can be
computed using the following equations [I]
~D +~D] (i,%)EL, i=l,...,N-I (13) ~i~ = ti[Di~(fi~) ~r% '
~D + ~D] 5r i =;~i~[D~%(fi~ ) ~ , i=l .... ,N-I (14)
~D (15) = 0 ~r N
where DI~(f=A) denotes the first derivative of Di~. The equations above uniquely
determ'ne ~._--~--- and ~---and the'r computation is particularly simple if ~ is loopfree. °~i~ °ri ~D DD . . . -
In this case each node i computes ~--~ and ~--~vla (13),(14) after recezvlng the • - r L
BD i " value of ~ from all ts immedlate ~ownstream neighbors (see [i],[5]). The computa-
tion is carried out recursively and no matrix inversion is involved.
A necessary condition for optimality is given by (see [I])
~D ~D = min if ~i~ > 0
~iL mE0(i) ~im
BD ~D min if ~i~ = 0,
~i% mE0(i) ~im
where all derivatives are evaluated at the optimum. The necessary condition can be
written for ti#0
D' ~D = . , ~ D . i~ ÷ ~ min if > 0 mE 0 (i) [ Dim + ~-~--] ~i~ m
D' + 5 D ~D] i~ ~-~ ~ mE0(i)min [Dim + Or "m if ~i~=0.
Combining these relations with (14) we have that if ti#0
~ = min [D~m+~]. i mEO (i) m
In fact if the condition above holds for all i (whether ti=0 or t i> 0) then it is
sufficient to guarantee optimality (see [I], Theorem 3).
A Class of Algorithms
In order to maintain loopfreedom the subproblem (12) via which ~i is determined
must be modified, so that soma variables ~i~ are not allowed to increase from zero
thereby forming a loop. Another difficulty is that problem (i0) has in general
stationary points which are not optimal (an example is given in [i], pp. 76-77), and
the algorithm (11),(12) will effectively terminate at such points. These and other
216
considerations based on convergence analysis lead us to the following modified
algorithm: k+l _k ~i = ~i (15)
where ~i is any solution of the problem
subject to ~i ~0' ~ ~i%=I' ~i~ =0' V%EB(i;~0k), (16)
and we assume that D(~k,r) <=. The vector 8i(~k,r) has components given by
, ~D (~k~r) (17) 6i~(~pk'r) = Di~[fi~ (~k'r)] + 8r~ "
For each i for which ti(~k,r) > 0, the matrix ~i is some symmetric matrix which is
positive definite on the subspace [vil E v = 0], and
~i = 0 if ti(~k,r) = 0.
The set of indices B(i;~ k) is specified in the following definition:
Definition: For any ~6~ and i=l,...,N-i the set B(i;~), referred to as the set of
blocked nodes for ~ at i, is the set of all L60(i) such that ~1%=0' and either
~r i ~r~ , or there exists a link (m,n) referred to as an improper link
such that m=~ or m is downstream of ~ and we have ~mn > 0, 8r r
Soma of the properties of the algorithm are given in the ~ellowing n . . proposltlon,
the proof of which is given in [5].
Proposition i: a) If ~k is loopfree then ~k+l is loopfree.
b) If k is loopfree and solves problem (26) then k is optimal.
c) If ~k is optimal then k+l is also optimal. -k k
d) If ~i #~i for some i for which t.(~k,r) >0, then there exists a positive scalar ^ k ~k such that D[~k+~-~ ),r] <D(~k,r), ~6(0,~] .
4. SPECIFIC ALGORITHMS
The selection of the matrices ~. is crucial for the success of the algorithm for
it affects both convergence and rate of coavergence. We first show that a specific
choice of ~" yields the first method of Gallager [I]. Let (i,%l),(i,L2) ..... (i,~m)
be the outgoing links from i for which ~j~B(i;~k), j=l,...,m, and, by reordering
indices if necessary, assume without loss of generality that
6i~(q0k,r) ~ 6i~ (q0k,r), V~E0(i), ~B(i;~k). (Ig)
Let m
be the mX (m-l) matrix
Zk--
- 1 -
2 1 7
the columns of which form a basis for the subspace {v6RmlVl + ... +v = 0}. Let t. ' m i
=
k where ~ is a positive number. By making the change of variables ~i =~i'zk v problem
(16) can be written after a straightforward calculation as t. l
minimize g[V + ~'~ IVl 2
k k k m-I subject to Vl<_CPi~l, .,vm_ I ~ • " oPiUm_ I, -~i~ m j~ivj
I 6il (%°k,r)'6i% (~k,r)
l . m gi = ~Si(~k'r) =
6.~ (~k,r)-6~ (~k,r) h l~ m I ~m
where
Because of (18) when t i>0, the solution of this problem is easily shown to be
(6i~j-Si~m)], % = m i n [ ~ , ~i j=l . . . . . m-i (19)
and in terms of the variables ~i we have
k+l k - ~i~ = ~i~j-vj, j=l,...,m-I (20)
~k+l k m-I
i~m = %~m+J ~lv-j" Equations (19) and (20) are identical with equations (ii) and (12) of Gallager's
paper [I] (our a corresponds to his ~).
A drawback of algorithm (19),(20) is that a proper range of the parameter a is
hard to determine. In order for the algorithm to converge for a broad range of
inputs r, one must take a quite small but this leads to poor speed of convergence. -i
Furthermore scaling at each node i by t i as in (19) does not seem adequate to yield 1 _
a good direction for change of ~ ~. It seems that a better choice of M~. is based on k i
a diagonal approximation_ 5 "~ -~°f the Hessian of D(~ ,r) with respect to ~, i.e., a diagonal
matrix with t{ I D%~;~ ! , (i,~)6L along the diagonal. Unfortunately the second
derivatives are hard to compute in a simple manner. However it is possible to
compute easily upper and lower bounds to them which seem sufficiently accurate for
practical purposes.
Calculation of Upper and Lower Bounds to Second Derivatives
52D We compute ~ evaluated at a loopfree ~E~, for all links (i,~)EL for which
[ ~ 1 ~
~ 2_______LD ~ 5 [t i ' +~)} [~Di~] 2 ~i~ (Di~ "
Since %~B(i;~) and ~ is loopfree, the node ~ is not upstream of i. It follows that
218
5D~ ,, D" is the second derivative of Di~. Using again the --= 0 and ~--~i~ =Di~ti, where i% ~t.
l
8~i~ ~t i 8D~ fact that ~ is not upstream of i we have ~--~= 0, ~--~= 0 and it follows that
~2D ~ ~D ~ [t i , +~)] = ti 82D ~r~ 5~i~ = ~r~ (Di~ [~r~]2 "
~2D t2 ,, + ~2D ) [ ~i~] 2 i(Di~ [Dry] 2 "
5~i~r~ Thus we finally obtain
(21)
~2 D A little thought shows that the second derivative - -
formula [~r% ]2
~2 D = ~ q (L)qik (m)Di~ , V~,m=l,. ,N-I
~r~br m (i,k)EL ik ""
where qik(~) is the portion of a unit of flow originating at ~ which goes through 52D
link (i,k). However calculation of - - using this formula is complicated, and in [ 8r~] 2
fact there seems to be no easy way to compute this second derivative. However upper
and lower bounds to it can be easily computed as we now show. By using (14) we
obtain
82D - ~ [ ~m(D~m+5~m)}- [~r~2 ~r~
Since ~ is loopfree we have that if ~m > 0 then m is not upstream of ~ and therefore ~t~ ~D~m ,, ~r~ i and ~--~=D~m~%m. A similar reasonsing shows that
= ~2D
~r%Srm ~ { ~n(D~n n
Combining the above relations we obtain
52D E~ DY + EE~ ~ ~2D (22) [~r~] 2 m ~n ~m mn ~m ~n ~r#r n
~2D > ~ ~2D Since ~-~-~-r _u, by setting ~ to zero for m#n we obtain the lower bound
m n 2 -" ~2D
m~P~m (D%m + [ ~r m] 2)"
Now D(~,r) can be easily shown to be convex in r for fixed ~ and hence all minors of
the Hessian matrix with respect to r are positive semidefinite. Hence
~2D /~2D ~2 D
~rm~r n ~ [~rm ]2 [Drn ]2 "
Using this fact in (22) we obtain the upper bound
m~P2 ,, ~52D 2 ~mD~m + ~m~-~-~2 ) •
m [~rm ]
It is now easy to see that we have for all k
is given by the more general
219
R_~ < --52D < ~L
~r%] 2 -- [
where R_L and R~ are generated by
R% = ~%m(DLm +Rm)
R =gN=0.
-- 5D The computation is carried out by passing R~ and R~ upstream together with ~ and
this is well suited for a distributed algorithm. Upper and lower bounds ~.., [.. 2 -~ ~
for ~---~ ~ , %~B(i;~0) are obtained simultaneously by means of the equation [cf.
[5~ ] c~iL (21)]
= t 2 "D" +RI).
An Al$orithm Based on Second Derivatives
The following algorithm seems to be a reasonable choice. If t i = 0 we take ~i
I in (16) to be zero and if ti~0 we take Ff~ to be the diagonal matrix with ~iL' L60(i) along the diagonal where ~i~ is the upper bound computed above and ~ is a
positive scalar chosen experimentally. Convergence can be proved for the resulting
algorithm provided ~ is sufficiently small. A variation of the method results if we
~i~+~_i~ use in place of the upper bound ~i~ the average of the upper and lower bounds
This however requires additional computation and con~unication between nodes.
The projection problem (16) can be written for ti#0 as
subject to ~iL ~ 0, L~PiL = I, ~i~ = 0 ~Eg(i;~ k)
and can be solved using a Lagrange multiplier technique. By introducing the expres-
sion -~i~ = t2i(Di~'' +KL) and carrying out the straightforward calculation we can write
the solution as
= ) (23) D" ti( i~ +R%)
where I is a Lagrange multiplier determined from the condition
E max{0,~L ~(6iL'l) } = I. (24) ~B (i;~) t i (D~ +~)
The equation above is piecewise linear in the single variable ~ and is nearly
trivial computationally.
It can be seen that (23) is such that all routing variables ~i~ such that 6iL <k
will be increased or stay fixed at unity, while all routing variables ~i~ such that
220
8i~ >~ will be decreased or stay fixed at zero. In particular the routing variahle
with smallest 6i% will either be increased or stay fixed at unity, similarly as in
Gallager's iteration (20).
A Modification of Gallaser's Iteration
We now consider a modification of equations (19),(20) using scaling based on
Second derivatives. These equations have in effect been obtained by carrying o u t the
gradient projection iteration in the reduced space of the yariables v. The Hessian
matrix of the objective function with respect to v is ~ 52D(~k~ r) ~. If we discard
2 k [~i ] D(~ ,r) , ~2D(q>k,r) the nondiagonal terms of then along the diagonal of ~ 2 ~ we
[ ~i] 2 [ 5~i] 52D (~k,r) 52D(~k, r)
obtain the scalars [~i~.]2 [~i~ ]2 ' j=l,.,.,m-l. Thus we arrive at the scaled
form of (19),(20) ] m
k+l k - j=l, ,m-I ~i~. = ~i~.- vj, ...
3 3 ~ k m-l_
= ~i½ + j~lVj • k ff(gi~$-6i~m )
}, j=l ..... m-l. % +% +%) j j m m
It is possible to show that if at every node and every iteration there are at
most two outgoing links that carry flow, then the modified version of Gallager's
iteration yields identical results as iteration (23),(24). If however there is a
node with more than two outgoing links carrying flow the two iterations will yield
different results.
The following convergence result, obtained under the simplifying assumption
r.>0 for all i, is proved in [5]. The result is applicable to both algorithms l
described in this section. k+l -k ~i Proposition 2: Consider the iteration ~i =~i where is a solution of subprohlem
(16). Let D O be a real number and assume that:
a) ~o is loopfree and satisfies D(~°,r)<D °
~. in problem (26) satisfies for all i and k b)
ti( k r) Ivit 2 ti( k r) Ivil 2 for all v i in the subspace [vil E v.~ = 0] where ~, X, A are some positive
scalars. ~B(i;~k ) i~ '
c) r i>0 for all i=l,...,N-l.
Then there exists a scalar &> 0 (depending on Do, ~, and A) such that for all oE(0,~]
we have lim- D(~k,r) = sin D(~,r). Furthermore every limit point of [~k] minimizes
D(~,r) ever ~6~.
221
5. STEPSIZE SELECTION BY LINE SEARC}~
The Goldstein-Levitin-Polyak method can be combined with a variety of line
search rules [6]-[8] for the purpose of enhancing its convergence properties and ell-
minating the burden of determining a satisfactory constant stepsize. The use of
such line search procedures into the algorithms of the previous two sections makes
them however unsuitable for distributed operation, so that llne search is of interest
primarily in the context of centralized operation.
It is important to note that straightforward incorporation of line search into
the class of algorithms of Section 3 can lead to inefficient and possibly unreliable
operation. Consider as a first candidate the iteration
where ~k solves problem (16) and ~k is determined by minimizing D(-,r) starting from
along the .ire=tlo~ (~5, i.e.
where ~6Sk S k ; [ ~ 1 ~ + ~ - ~ k) _~ 0}. (27)
Since (#-~0 k) is ordinarily a descent direction at ~0 k [Proposition Id)], iteration
(25)-(27) appears quite natural. A perhaps better alternative is to perform a llne
search in the space of flows f rather than in the space of routing variables q0.
~hus if ~ solves problem ~16), and are the flows corresponding to ~ and
respectively, we set for all (i,~)6L
.k+l "lmEO~(i)fkm~k) i f mEO~(t)~im~k)'O ~i~ -- \ (28)
~i~ otherwise
where ~k is such that
DT[~(~k) ] = min[DT[~)]I~) > 0} (29)
and we use the notation
= ~)6LDi~ (fi~) , (30) DT (f) (i,
fki~) ; fk +~ ~ k iZ (fi~- fi% )' V~ >_0, (i, ~)6L. (31)
An argument nearly identical to the one in the proof of Lemma i in Appendix C of [I]
shows that ~DT[ fk~)] I • ~ z D'.(1A).~.~- j ) ~ (~. ~
-k k -k k E ti( ~ ,r)6i~(~o ,r)(qoi~-%oi~ ). (32)
(i,~)EL
222
Thus if # for some i for which ti(~ ,r) # 0 we have from the necessary condition
for optimality in problem (161 that ~DT[Z ~) ]=k <0, and hence (f-k_~) is a direc- ~q ~=o
tion of descent of DT(f ) at ~. This provides a Justification for the use of line I.
search (29). Since DT[f~)] is convex as a function of ~ and its values can be
calculated more easily than the values of D[~+~(~_~k),r ] I _ , . it seems that the llne
search indicated in (29) is more attractive from the practical point of view than
the one of (26).
Both algorithms (25)-(27) and (28)-(31) would be satisfactory if there was no
possibility of improper links appearing during the course of the computation. What
can happen however is that if the parameter ~ is chosen much larger than appropriate,
the stepsize ~k in (25) or (28) will tend to be less than unity for a large number
of iterations. If at iteration k we have ~k < 1 and there is an improper link in the
network then this link will very likely continue to be improper in the next iteration.
As a result it is possible that the algorithm will take a very large number of itera-
tions before setting to zero the flQw through an improper link, thus greatly slowing
down convergence.
A remedy to this situation is to search in the space of the routing variables
for an acceptable stepsize along a piecewise linear arc connecting ~0 k and ~ - at
least whenever there is an improper link in the network. The following algorithm is
an adaptation of a procedure first suggested by the author [8] in connection with
the Goldstein-Levitin-Polyak method and we refer to that paper for further relevant
a loopfree ~k with D(~k,r)<m, and ~6(0,i] denote by -- ~i~ ) a soln- discussion. Given
tion of the problem
1 minimize 6i(~k,r)'(~i-~ ) +~
subject to ~i~0, ~i~ = I, ~i~=0, ~6B(i;~k), (33)
with ~i positive definite on the subspace {v~1 ~ v=n=0} if ti(~k,r)>0 , and
k 0 otherwise. Let 86(0,11, ~E(0,~) be fixed scalars, (suggested values are M i
~E[O.I, 0.O1], ~E[0.1, 0.5]). The algorithm consists of the iteration
k+l = ~(~%) (34) where m k is the first nonnegative integer m satisfying
~D~k,r) ' [~k_~ (Bm) ] (351 D~k,r)-D[~(sm),r] ~ ~ 5~
The algorithm (33)-(35) searches for an acceptable stepsize along the piecewise
linear arc {~)i~ ~0} rather than along the line segment connecting ~k and ~ or
fk and ~, and, as we show shortly, avoids the problem with persisting improper
links described earlier.
We now consider three specific algorithms with line search. The first is
algorithm (34) and the other two are combinations of (34) with algorithms (25) and
(28) respectively. The last two algorithms are motivated by the fact that the line
223
search (26) or (29) may be computationally more efficient than the line search (34),
(35) when there are no improper links.
Line Search Al~orithm I: This algorithm consists of the iteration described by (33)-
(35).
Line Search Algorithm 2: In this algorithm if ~k has at least one improper llnk
corresponding to it then k+l is determined by the iteration described by (33)-(35).
Otherwise ~k+l is determined by the iteration described by (25)-(27).
Line Search Al$orithm 3: In this algorithm if ~ k has at least one improper llnk
corresponding to it then ~k+l is determined by the iteration described by (33)-(35).
Otherwise ~k+l is determined by the iteration described by (28)-(31).
We have the following convergence result proved in [5] :
Proposition 3: Let ~o be loopfree and satisfy D(~°,r)< ~, and let [~k] be a
sequence generated via any one of the line Search Algorithms i, 2, or 3. Assume
that:
a) ~i in problem (16) and problem (33) satisfies for all i and k
ti(~k,r)Xlvil 2 ~ vi~v i ~ ti(~k,r)^Ivll 2
for all v i in the subspaee {vii ~ v.~ =0}, where %,A are some positive scalars. ~B(i;~k ) l
b) r.>0 for all i=l,...,N-l. I
Then we have lim D(~, r)1" =min D(~,r) and every limit point of [~k} minimizes D(~,r)
over ~6~.
Computational results for the algorithms of this and the preceding section may
be found in [9].
REFERENCES
[I] Gallager, R., '~ Minimum Delay Routing Algorithm Using Distributed Computation," IEEE Trans. on Communication, Vol. COM-25, 1974, pp. 73-85.
[2] Gallager, R., "Scale Factors for Distributed Routing Algorithms," Paper ESL-P- 770, Electronic Systems Lab., Massachusetts Institute of Technology, Cambridge, Mass., August 1977.
[3] Goldsteln, A. A., "Convex Programming in Hilhert Space," Bull. Amer. Math. Soe., Vol. 70, 1964, pp. 709-710.
[4] Levitin, E. S. and B. T. Polyak, "Constrained Minimization Problems," USSK Comput. Math. Math. Phys., Vol. 6, 1966, pp. 1-50.
[5] Bertsekas, D. P., '~igorithms for Optimal Routing of Flow in Networks," Coordinated Science Lab., University of Illinois, Urbana, Iii., June 1978.
[6] Daniel, J. W., The Approximate Minimization of Funetionals, Prentice-Hail, Englewood Cliffs, N.J., 1971.
[7] Polak, E., Computational Methods in Optimization: A Unified Approach, Academic Press, N.Y., 1971.
[8] Bertsekas, D. P., "On the Goldstein-Levitin-Polyak Gradient Projection Method," IEEE Transactions on Automatic Control, Vol. AC-21, 1976, pp. 174-184.
[9] Bertsekas, D. P., E. Gafni, andK. Vastola, "Validation of Algorithms for Optimal Routing of Flow in Networks," Proc. of IEEE Conf. on Decision and Control, San Diego, calif., January 1979.
224
Note: Many thanks are due to Bob Gallager who introduced the author to the subject,
and provided stimulating comments and insights. Valuable suggestions by Ell Gafni
are also greatly appreciated.
A 2-STAGE ALGORITHM FOR MINIMAX OPTIMIZATION
J~rgen Hald and Kaj Madsen Technical University of Denmark
DK-2800 Lyngby, Denmark
Summary: The problem of minimizing the maximum of a f i n i t e set of smooth functions
can be solved by a method that uses only f i r s t order derivative information, and
normally this me~hod w i l l have a quadratic f inal rate of convergence. However, i f
some regularity condition is not f u l f i l l e d at the solution then second order infor-
mation is required in order to obtain a fast f inal convergence. We present a method
which combines the two types of algorithms. I f an i r regular i ty is detected a switch
is made from the f i r s t order method to a method which is based on approximations of
the second order information using only f i r s t derivatives. We prove that the com-
bined method has sure convergence properties and i l lus t ra te by some numerical examp-
les.
i. Introduction.
In this paper we consider algorithms for minimax optimization, i.e.
algorithms for minimizing the maximum of a finite set of smooth func-
tions. Several authors have considered this problem. Some of the re-
levant algorithms are those of Osborne and Watson [12], Bandler and
Charalambous [i], and Charalambous and Conn [4]. An excellent theore-
tical treatment of the problem can be found in the book of Dem'yanov and
Malozemov, [6].
The objective function which is minimized,
F(x) m max f (x) (i) -- Isj <_m J -- '
is in general a non-differentiable function, and normally F is not
differentiable at a local minimum. This might look like a difficulty.
However, if the functions fj , j=l,...,m , are smooth, then F has
directional derivatives at any point with respect to any direction,[6].
If these derivatives have discontinuities at a local minimum x • of F
then the minimum is better determined numerically than in the smooth
case. The situation is illustrated in figur i. A condition that ensu-
res that no smooth valey passes through the solution ~* is that any
subset of the set (~(x~) I fi (£~) = F(£~)} has maximal rank. This
is the so-called Haar-condition. Further details on this may be found
in [i0].
226
\
F non-differentiable at x ~
\ f
x •
F differentiable at x *
Fig. i.
It turns out that in the non-differentiable situation a local mi-
nimum is characterized only by some first derivatives, whereas in case
of a differentiable objective function some second derivative informa-
tion is required (positive definiteness of the Hessian). This suggest
that it is possible to construct a fast algorithm for finding a minimum
of the non-smooth F which is based only on first derivatives. If,
however, F is smooth at x* for certain directions then it seems
that some second order information is required in order to obtain fast
final convergence.
In [9] an algorithm based on first derivatives was described. This
algorithm has quadratic final convergence under the Haar-condition, [ii].
If, however, F is smooth at the solution along certain directions then
the final rate of convergence for this algorithm may be very slow, [i0].
The 2-stage algorithm that will be described here is based on the
algorithm of [9]. But if a smooth valey through the solution is detect-
ed then a switch is made'to an algorithm that uses second derivative in-
formation, and which will have a superlinear final convergence (provided
the switch is made close enough to the solution). This stage 2 iteration
is based on the following (see for instance [8]): If the solution xe
is determined by the functions fl,...,fs , i.e.
F(x ~) = fi(x ~) > fj(x~) , l<i<s , j>s , then there exist non-negative
numbers Ii,...,I s such that the following system of equations is satis-
fied at x • :
s
i=l
s
X li -I = o (2) i=l
fs(~*)-fi(x~) = 0 , i=l, .... S-i
227
Solving (2) for the minimax problem is equivalent to solving the Kuhn-
Tucker equations in non-linear programming when the number of active
constraints is less than the number of unknowns, [8].
The stage 2 iteration solves these equations by an approximate New-
ton iteration, but if it turns out that this iteration is unsuccessful
(for instance if the active set fl,...,fs has been wrongly chosen),
then a switch is made back to stage 1 : the iteration of [9]. The al-
gorithm may switch several times between stage 1 and stage 2.
The idea of solving the equations (2) for finding a solution of the
minimax problem has also been used by Hettich in 1976, [8], and by Cha-
ralambous in 1978, [2], but they don't suggest any algorithm for iden-
tifying the set of functions being active at the solution. Furthermore
they require second derivatives.
In the following section we give a detailed description of the al-
gorithm. In section 3 it is shown that the global convergence proper-
ties of the algorithm in [9] are maintained, and in section 4 some nu-
merical examples are given.
It should be noticed that we have also implemented a version that
instead of (i) minimizes the objective function F(x) = maxlfi(x) j
All theoretical results in the paper may be extended to this algorithm
without trouble.
2. Description of the 2-stage algorithm.
The algorithm consists in four parts: stage i, stage 2, and two sets
of criteria for switching.
2A. The staq~ 1 iteration: This is the iteration described in [9].
At the k'th stage of the algorithm we have an approximation ~k
of the solution and we wish to use the gradient information at ~k
to find a better approximation ~k+l " Therefore we find the incre-
ment as a solution hk of the linear minimax problem
Minimize (as a function of h )
~(Xk,h ) n max {fi(xk ) + fi(x_k)Th} (3) l<i<m
subject to the constraints
JlhII~ ~ A k ,
where A k > 0 is a bound which is adjusted during the iteration
according to the strategy described below. This linear problem can
228
be set up as a standard linear programming problem in (n+l) vari-
ables and (m+2n) constraints.
The point (Xk+h~q) is accepted as the next iterand provided that
the decrease in F exceeds a small multiple of the decrease predict-
ed.by the linear ~pproximation. It would be simpler just to ensure
that the value of F decreases, but this is not enough to guarantee
convergence. Therefore ~k+l = ~k + hk provided that
F(~k) - F(~k+hk) ~ p(F(Xk) - F(X_k,hk)) (4)
and if this inequality does not hold we let ~k+l = ~k " In our ex-
amples we have used p = 0.01 .
The cho£ce of Ak+ 1 is also based on (4), with other values of
p . If the decrease in F is rather poor compared to the decrease
in the linear approximation we wish the bound to be smaller. In fact,
if (4) is not true with p = 0.25 we let Ak+ 1 = IIhkII/4 . This en-
sures that the bound will decrease when ~k+l = ~k " In order to
speed up the rate of convergence we must also have the possibility
of increasing the bound. The rule for this is also based on (4): If
(4) is true with p = 0.75 then we let Ak+ 1 = 211h~II - In all
other cases we let Ak+ 1 = IIhkll
It is our experience that the choice of constants in this algorithm
is not critical. Small changes in the constants do not significantly
alter the rate of convergence in the examples we have tried.
2B. The staqe 2 iteration: We suppose that the active set of functions,
fl,f2,...,fs, say, has been determined in 2C. We would like to use a
quasi-Newton iteration for solving (2).
For this purpose we need an approximation ~k of the Jacobian
~(~,~) of (2) at the current approximation (X_k,~k) of the solution.
It turns out, that ~ can be subdivided into four submatrices, one of
which is the Hessian of the Lagrangian function L(x,~) = [ X i fi(~)
and the remaining three only containing gradient information. The
Hessian matrix ~(x,l) m L" (x,l) must be approximated. The initial =xx
approximation is calculated by using finite differences of the form
af'_± f! (x+te~) - f! (x) ---~x~ ~-z ~ -i- ax-" " -- t
3
(5)
e. being the j'th unit vector. After the first stage 2 iteration --3 the Hessian matrix B is updated using a rank 1 updating formula
229
n ! Lx (Xk+l" ~--k+l ) --x (Xk+l "l--k+l ) T
, = ,Is~l )T where A~k = ~k+l - ~k ' A~ = ~k+l - ~k ' ~k (li'~2 ....
T and ~ = (fsk-fl,fsk-f2, .... fsk-fS~l) (See [7] for further details).
The point (~k+l,~k+l) found by this quasi-Newton method is normally
accepted as the next iterand, even if the minimax function increases.
Only if the proposed step is too large (see (14) below) a different
point is chosen.
2C. Conditions for switchinq_t2_§taHe_2: It is not disasterous to
start a quasi-Newton iteration with a wrong set of active functions
since in that case a switch back to stage 1 will be made after some
iterations (see section 3). But in order to avoid unnecessary iter-
ations we use a rather restrictive set of criteria which must all be
satisfied before the quasi-Newton iteration is started. A switch is
made if (7), (10) and (13) or if (7) and (ii) are satisfied at some
point ~k
Since we do not want a switch to stage 2 in case of a regular solu-
tion, it is required that the solution hk to the linear problem (3)
has maximal length, i.e.
alhkll = ^k (7)
This condition will hold near a singular solution (cf. [7]) whereas it
will not be satisfied near a regular solution because of the quadratic
convergence in this case (cf. [ii]).
The active set A k at the point ~k is defined as
m { j I F(ak)-fj(£k)~llF(£k) I } (8)
where e I is a small positive number specified by the user. If we
suppose that the stage 1 algorithm converges to some point ~ and
define the set A • as
A • m { j I F(£~)=fj (£~) } (9)
then A k = A • for k>K where K is some integer, provided that ¢ 1
is sufficiently small. This gives a motivation for using
230
Ak_ 2 : Ak_ I :
Sk_> 2
(1o)
where s k is the number of elements in A k , as an indicator that
a stable active set A k has been identified.
If only one function is active at the solution then the following
inequality holds near the solution
(li) llf~(x_ k) I] <_ ~2 where fi(xk) : F(x k) ,
~2 being a small positive number specified by the user.
If (7) and (10) are satisfied then the tentative starting point of
the quasi-Newton iteration is (Xk,~k) where ~k is found as a least
squares solution of the linear system
f~ (x k) = 0 (12) [ hi --i -- ic~ k
subject to the constraint IX i = 1 .
If some of the columns in this system are linearly dependent then
the active set is reduced. For this we use a singular value decompo-
sition of (12). If li<0 for some i6A k then i is removed from
the active set and (12) is solved with the new set. The resulting ac-
tive set is denoted by A for simplicity.
Now a switch to stage 2 is made if
(13) l l ~ i ( ~ k ' ~ k ) II " II [ X i ~ l ( X - - k ) I I ~ 0 . 5 min[If~(~k)II i6A i£A
This test confirms that A could be the active set corresponding to
a solution in the neighbourhood of ~k
2D. Causes for switchinq back to staqe_!. The rules of this section
are tested for in each quasi-Newton iteration. It will be shown in
section 3 that these rules guarantee that the convergence properties
of the method used in stage 1 are not wasted by the stage 2 iteration.
A switch to stage 1 is made if one of the conditions (14), (16), (17)
or (18) below is satisfied.
In order to prevent overflows in the computation we do not allow
too large steps. If the incr@ment ~ proposed by the quasi-Newton
iteration exceeds a user specified positive number Ama x ,
231
i lh~i[ > Ama x
then we let
~k+l = {k + ~'Amax/llhkll
(14)
(15)
When (14) is unless ilh~ii > 20Ama x , in which case ~k+l = ~k true a return to stage 1 is made (notice that this test is not nes-
cessary for the convergence theorems).
When (14) is not true then ~k+l is the point found by the quasi-
Newton step.
It is not required that the minimax objective function decreases
monotonically during stage 2. However, we require that the residuals
r(x) of the non-linear equations (2) to be solved, are strictly de-
creasing. In fact, a switch is made if
ll~(~k+l) LI > 6 IL~(~k)II , (16)
where 0<~<i . We have used ~ = 0.999
If the active set has been chosen incorrectly then it may happen
that some function from outside the active set becomes dominating, i.e.
Vi£A: F(Xk+l ) > fi(~k+l ) (17)
where A is the active set used in the current quasi-Newton iteration.
In this case a stage 1 iteration is started at ~k+l
Finally, a switch is made if any of the multipliers I i , 16A ,
becomes negative, that is
~(k+l)<0. i6A (18) i
In this case a stage 1 iteration is started at ~k+l "
The local bound Ak+ 1 used in the first stage 1 iteration after a
return from stage 2 is always
Ak+ 1 = max{llXk+l-{kll, llh~ll} (19)
where hg is the increment used in the latest stage 1 iteration.
232
3. Convergence properties.
The smoothness assumption that is nescessary for convergence of the
algorithm is the following,
fj(x+h) = fj(x) + f~(x)Th + o(llhll) 9=i ..... m (20)
f~ is continuous --3
We prove that the points generated by the algorithm of section 2
will converge to the set of stationary points. A stationary point is a
point for which the generalized derivative, [5],
~*F(x) . conv{f~(~) I fi(~)= F(x)~ (21)
where cony is the conveM hull, contains 0 . It follows that a sta-
tionary point is a point where the equalities (2) are satisfied. It is
shown, for instance in [ii], that any local minimum of F is a statio-
nary point, and on the other hand that if the Haar-condition is satis-
fied at a stationary point x* , then x* is a strict local minimum.
It is also shown in [ii] that x* is stationary if and only if h = 0
is a solution of the linearized problem in stage i, (3).
We need the following lemma which is proved in [ii]:
Lemma i. If x is not a stationary point then there exist £>0 and
6>0 such that if l]~k-Xl ] ~ 6 then
F(~k) - F(~k,hk) ~ e min{6,11~kII} (22)
where hk is a solution of (3), and if A k ~ ~ then llhkll = A k .
The first convergence theorem is concerned with the situation where
all but a finite number of iterations are performed in either stage 1 or
stage 2:
Theorem 1. Let ~k ' k=l,2, .... be the sequence generated by the al-
gorithm. If {Xk } stays in a finite region, and if only a finite
number of switches between stage 1 and stage 2 are made, then
d(Xk, s) + 0 (23)
where S is the set of stationary points and d(x,S) ~ infz6s[IZ-xl I .
Proof If the last switch is made to stage 1 then the iteration is a
stage 1 iteration from this point and the theorem follows from Theorem
233
2 of [9]. Now suppose that (23) is not true. Then the last switch
has started a stage 2 iteration which will continue for the remaining
part of the iteration. Since d(Xk,S) ~ 0 ,{Xk } has an infinite num-
ber of elements that are bounded away from S , and since {X_k} is
bounded it must have a cluster point z with ~ ~ S . Thus there
exists a subsequence {Z_k} ~ {X_k} such that ~k ÷ ~ for k ~ ~ We
can suppose that all ~k are calculated in the last stage 2 iteration
and that the set of active functions here are fl' f2''''' fs
We obtain a contradiction because none of (16), (17) or (18) is
satisfied for any ~k :
Because of (16) we obtain that r(z~) + O when k + ~ and
~k ÷ ~ " This implies that fl(z) = fi(z) , i = 2,..., s , and be-
cause of (17) fj ~) = F(~) for l~j~s , which means that
fl' "''' fs belong to the active set at ~ .
(16) and (18) imply that
s
l!k) , (k) > 0 Z I (k) i=l i fi(zk ) + 0 , I i _ , i = 1 (24)
and since the der£vatives f~ --l
s . (k) f! (z) + 0 , Ai i -- --
i=l
are continuous we obtain that
l(k) I! k) : 1 i Z 0 , Z 1 (25)
But, since z is not stationary we have that
s
0 ¢ { ~ ~i fl (z) L ~i > 0 ~ ~, = i} (26) -- i = l -- -- - ' l
Since the set on the right hand side is closed this is a contradiction
to (25), and hence the assumption d(Xk,S) ~ 0 must be wrong. The
theorem is proved.
Because of theorem 1 convergence would be guaranteed if an upper
bound on the number of switches was imposed. Since it seems to be
normal that only a few switches are made (in the examples we have tried
at most 4 switches to stage 2 have been observed) an upper bound of
this kind would seem natural. However in theorem 2 we show that even
in case of an infinite number of switches the convergence properties
are satisfactory.
Theorem 2. If the sequence generated by the algorithm is convergent
then the limit is a stationary point.
234
Proof. Suppose that the limit point ~ is not a stationary point.
Then, because of theorem i, an infinite number of the points ~k '
k = 1,2, ..., must be generated by a stage 1 iteration.
Let the indices at which a stage 1 iteration is used be
£1,£2, .... Since ~k + x we obtain x. ~ x , J = £k ' k ~ ~ ,
and consequently h. ~ 0 and A. ÷ 0 for J = £k and k ÷ ~ There- --3 -- 3
fore, because of the smoothness assumption (20) and inequality (22) of
lemma 1 we obtain (as equation (29) in [9]) ,
F(xj)~ - F(£J+hv)4~ + 1 for J = £k' k + ~ (27)
F(~j) - F(~j,hj)
Thus the condition for choosing Aj+ 1 (in connection with inequality
(4)) implies that Aj+ 1 Z ]lhjll for all large values of J = Zk
Further, lemma 1 implies that when A. is small we have that 3
ll~j]J = Aj Thus AS+ I~ ~ A 5~ for all large values of j = Z k . Be-
cause of (19) the first A used after a quasi-Newton iteration is not
less than the last A used before this stage 2 iteration. Therefore
the sequence of bounds, Aj , is bounded from below by a positive num-
ber, and hence it cannot converge to 0 . This is a contradiction,
and therefore the assumption that x is not a stationary point must
be wrong. This proves Theorem 2.
The following two theorems are concerned with the conditions under
which a switch to stage 2 can be guaranteed and the conditions under
which the correct active set will be identified. The theorems will be
proved in [7] - here they are stated without proof.
In relation to [8] and [9] we define
e~ -= sup{ e I I A (x* , el) =A* } (28)
where A(x,e) -- { j I F(x)-fj(x)-<c'IF(x) I }
Theorem 3. If the sequence {Xk } generated by the stage 1 algorithm
converges to a stationary point x* violating the Haar-condition,
if El<C ~ and if the vectors {I,~T(~*)} T , i6A* , are lineary
independent, then a switch to stage 2 will take place.
The next theorem is concerned with the situation when an infinite number
of switches between the two stages take place.
235
Theorem 4. If the sequence [~j} , j = 11,£2,..., of starting points
o f t h e s t a g e 2 a l g o r i t h m c o n v e r g e s t o a s t a t i o n a r y p o i n t x* v i o -
l a t i n g the Haar-condition, if ~l<~, and if the vectors
{ 1 , _f[T(x*)}T-- , i £ A * a r e l i n e a r l y i n d e p e n d e n t , t h e n f r o m some num-
b e r g the stage 2 algorithm will be used on the set A* v
4. Numerical Results.
In the following we illustrate the performance of the algorithm
through four mathematical testexamples and one set of examples origi-
nating from network design. The results are compared to results pub-
lished [i]-[4], and to the performance of the algorithm in [9]. In all
cases we use Ama x = 1 , el = 0.01 , ~2 = 10-5 ' and t = 10 -5 In
examples 1-4 we have used A 0 = 0.5 and in 5 and 6 A 0 = 0.1 . The
computer used is an IBM 370/165, the calculations being performed in
double precision• i.e. 14 hexadecimal digits.
Example i. This is the set of functions
2 fl (~) = 10~(x2-xl)
f2 ({) = 1 - x 1
We minimize the function F(~) = maxlfi(x) I which has the same bana-
na-shaped valey as the Rosenbrock function, [13]. Starting point
(-1.2•1) T solution (i,i) T where F(x) = 0
Example 2. This is the set of functions
Xl+X2Yj - e yj , j = 1 .... ,21 fj(~) = 2 3
l+x3Yj+x4Yj+x5Y j
where yj = -i(0.i)i , and F is minimized. The example was used
in [9]. Starting point (0.5,0,0,0,0) T , solution
(.999878, .253588, -.746608 245202, -.037490) T • . , where
F(~) = 0.000122
Example 3. We consider the nonlinear programming problem : minimize
f(~) = (Xl_10)2+5(x2_12)2+x~+3(x4_ll)2 6 2 4 +10x5+7x6+x7-4x6x 7
- 10x6-8x 7
subject to
236
2 4 2 g2(x) = -2Xl-3X2-X3-4x4-5x5+127 ~ 0
g3(x) = -7Xl-3X2-10x~-x4+x5+282 ~ 0
2 2 g4(x) = -23Xl-X2-6x6+8x7+196 ~ 0
2 2 2 g5(x) = - 4 X l - X 2 + 3 X l X 2 - 2 x 3 - 5 x 6 + l l x 7 ~ 0
which is used in [4]. Following [4] the solution is found by minimi-
zing the minimax objective F where fl = f and fi = f - 10gi '
i=2,3,4,5 In 3.I the starting point is (3,3,0,5,1,3,0) T as in
[4] and in 3.~ it is (1,2,0,4,0,i,i) T as in [2]. The solution is T
(2.33050, 1.95137, -.47754, 4.36573, -.62449, 1.03813, 1.59423) ,
where F(~) = 680.06 .
Example 4. This is the Rosen-Suzuki problem, which is formulated as a
minimax problem in [4]. In example 4.I we use the same starting
point as in [4] and in 4.~ the starting point is the same as in [2].
Example 5. As in [10] we use minimax optimization for minimizing the
maximum reflection coefficient of a i0:i three-section transformer
with 100% bandwidth. We use the same two starting points as in [I0].
Example 6. The same problem as in example 5 except that the transformer
has four sections. We use the same five starting points as in [10].
The results are shown in table 1. In the 5'th and 6'th column it is
shown how many function (including gradient) evaluations was required
to obtain 5 and 14 decimals accuracy respectively. The 4'th column gives
the number of times a switch to stage 2 is made. The next column gives
the number of function evaluations the algorithm of [9] requires to ob-
tain 5 decimals accuracy. Notice that this is identical to the present
algorithm if no switch to stage 2 is allowed. The remaining 5 columns
give the number of function evaluations reported in the papers [1]-[4]
and [i0]. In none of these cases the accuracy is better than 5 decimals.
The first two examples are regular, i.e. the Haar-condition is sa-
tisfied at the solution, and no switch to stage 2 is made. Notice that
the algorithm passes through the valey of example 1 without detecting a
non-Haar solution. In all the examples a fast final convergence was ob-
served which means that the true active set has been identified in the
examples 3-6 all of which represent non-Haar solutions.
This
algorithm
n
m
Stage
e = 1
~.10-5
ex.
2
Z
1
2
2
0
21
2
5
21
0
i0
3.I
7
5
1
22
3.~
7
5
2
30
4.I
4
4
1
15
4.~
4
4
1
18
5.I
6
ii
2
48
5.~
6
ii
1
21
6.I
8
ii
2
58
6.~
8
Ii
3
59
6.III
8
Ii
2
54
6.1V
8
ii
1
26
6.V
8
ii
2
75
6.VI
8
ii
4
82
1
_-14
e =
~-lu
Other algorithms
[9]
[i]
[2]
[3]
21
21
12
i0
26
414
35
332
19
22
21
37
51
707
25
253
64
2300
64
2466
59
61
31
1097
80
1680
i01
998
155
95
61
52
[43
[i0]
21
i0
150
83
84
37
44
48
67
48
162
29
262
217
31
165
320
252
Table i.
Number of function evaluations.
Notice that in [2] second derivatives
are required.
238
These results and other results we have obtained indicate that
the algorithm is stable: In all examples with Haar-condition no
switch to stage 2 is made and we have quadratic final convergence. In
all other examples the active set is identified, and the final Quasi-
Newton converges rapidly to the solution. We think it will be an easy
matter to prove that the final rate of convergence in these cases is
super-linear.
5. References.
i. J.W. Bandler and C. Charalambous, "New algorithms for network
optimization", IEEE Trans. Microwave Theory Tech., vol. MTT-21,
pp. 815-818, Dec. 1973.
2. C. Charalambous and O. Moharram, "A new approach to minimax op-
timization". Department of Systems Design, University of Water-
loo, Ontario, Canada, pp. 1-4, 1978.
3. C. Charalambous and A.R. Conn, "Optimization of microwave net-
works", IEEE Trans. Microwave Theory Tech., vol. MTT-23, pp.
834-838, Oct. 1975.
4. C. Charalambous and A.R. Conn, "An efficient method to solve the
minimax problem directly", SIAM J. NUM. ANAL., Vol. 15, No. i,
1978, pp. 162-187.
5. F.H. Clarke, "Generalized gradients and applications", Trans-
actions of the American Mathematical Society 205 (1975), pp. 247-
262.
6. V.F. Dem'yanov and V.N. Malozemov,"Introduction to minimax"(Wiley,
New York, 1974). [Translated from: Vvedenie v minimaks
(Izdatel'stvo "Nauka", Moscow, 1972).]
7. J. Hald and K. Madsen, "A 2-stage minimax algorithm that uses
Newton's method". In preparation.
8. R. Hettich, "A Newton-method for nonlinear Chebyshev approximati-
on".In Approximation Theory, Lect. Notes in Math. 556 (1976), R.
Schaback, K. Scherer, eds., Springer, Berlin-Heidelberg-New York,
pp. 222-236.
9. K. Madsen, "An algorithm for minimax solution of overdetermined
systems of non-linear equations", Journal of the Institute of
Mathematics and its Applications 16 (1975), pp. 321-328.
239
i0. K. Madsen and H. Schj~r-Jacobsen, "Singularities in minimax op-
timization of networks", IEEE Trans. Circuits and Systems, Vol.
CAS-23, NO. 7, 1976, pp. 456-460.
ii. K. Madsen and H. Schj~r-Jacobsen, "Linearly constrained minimax
optimization", Math. Progr. 14, 1978, pp. 208-223.
12. M.R. Osborne and G.A. Watson, "An algorithm for minimax optimi-
zation in the non-linear case", Comput. J., Vol. 12, 1969, pp.
63-68.
13. H.H. Rosenbrock, "An automatic method for finding the greatest
or least value of a function". Comput. J., Vol. 3, 1960, pp.
175-184.
DISTRIBUTED SYSTEMS
SYSTEMES DISTRIBUI~S
CERTAIN CONTROL PROBLEMS IN DISTRIBUTED SYSTEMS
A.G. Butkovskiy
Institute of Control Sciences, Profsojuzmaya, 81 Moscow, USSR
ABSTRACT
The paper consists of several Sections devoted to different facets of the
distributed system control.
Section q covers control of liquifier feed distribution along the describing
line of thin sheet rolling mill rolls in order to optimally control the shape
of the roll and the gap disturbed by thermal expansion.
Section 2 deals with active suppression of harmful and generation of useful
pressure variations and gas flows in long gas pipelines via active control
of the extermal gas source.
Section 3 poses a number of control problems for mobile energy sources act-
ing on distributed plants where transfer processes take place.
Section ~ employs numerical theoretic methods to solve the problem of
determining the set of points there a distributed oscillatory system is
controllable.
Section 5 introduces within the framework of the finite control theory the
notion of fundamental finite control in analogy with fundamental solution of
the equations.
Finally, Section 6 describes the application of certain aspects of the flow-
chart and block theory to distributed systems.
INTRODUCTION
The paper will survey some various control problems of applied and theore-
tical significance.
I. In rolling a quality thin sheets the thermal state of stand roll barrel
is of great importance (1,5). During the process this state is affected by
the hot rolled sheet temperature and thermal dissipation due to deformation
work. All this contributes to thermal expansion of the roll and thus chan-
~es the gap shape. Tim latter also changes due to roll wear. The shape
242
deterioration results in lower quality of the strip, especially due to non-
flat and wavy edge formation. Presently the thermal state of rolls is cont-
rolled through roll cooling by water or another agent or roll barrel heating
by burners. The coolers (heaters) are usually deployed linearly along the
roll barrel describing line.
Let the roll temperature field is described by a function T ~ ~ j i n a
cylindrical frame of reference, natural for a cylindrical roll, OtZ~ is
the roll length, O~ ~ ~is the roll radius,O~O~2JT/~ ~Ois time. Inside
the roll this function is governed by the well-known thermal conductivity
equation. There are four kinds of boundary conditions:(1) heat exchange bet-
ween the hot metal and roll; (2) heat absorption by deformation work; (3)
heat exchange between the roll free surface and the environment; (~) heat
exchange with the coolant. The latter constraint depends on the flow of the
cooling (heating) agent //~Jat each time instant ~ with O~ ~ ~Z .
The temperature field~(~ t ~,~J causes a thermal stress field described by
the strain tensor~/Z~10,# ). The thermoelastic description is given here in
quasistatic approximation since the rates of temperature changes are very low
compared with rates of thermal motion inertial forces. The thermal strain
field ~ causes in its turn a field of thermal displacements which is desc-
ribed by a displacement vector Z6r/~,~,~ . For the fields ~ and~partial
differential equations can be written in a straightforward way or the fields
can be computed through associated thermopoten~ials which satisfy a differen-
tial equation. The boundary conditions for these fields are defined as free
from normal strains on the cylinder surface.
We are specifically interested only in the radial componeat Z ~ f Z ~ ~
with ~=~ of the thermal displacement vector on the roll surface.
A simplest way of control is to compensate the roll wear by thermal displace-
ment. Statistical methods of roll wear prediction determine the wear as a
function at each time ~ and in each point (~; ~ ) of the surface.
Now it is desirable to s~ady controllability of this system. To solve the
controllability problem. It is to find a control Z/(F, ~J (constrained as
O~Z/(~J ~L/~ax ) at which the wear is completely compensated by ther-
mal displacement, /(~ ~j =~(~R,~9~at all specified Z , 4~ and ~ .
In the case of uncontrollability the system should minimize a certain deflec-
tion norm II R, 6, 5)- If'z, )II.
One important constraint is, in a number of cases, a constraint on thermal
strain in the roll and on the surface temperature. Figure 1 shows the control
system flowchart.
A more general control function is that at each time the optimal gap shape
is maintained.
243
2. Suppression of pressure and gas flow fluctuations in long pipelines e.g.
in powerful compressor stations is a major technological problem. Passive
suppressors kindred to passive electrical filters may, be ineffective because
actual pipeline parameters can vary widely, additional power losses may occur,
the manufacture is expensive and complicated and the operation is too invol-
ved (5).
A much more effective ~thod is to complete the line with a controlled pre-
ssure or flow impulse generator. To distinguish it from passive filters this
generator is referred to as active. In a more general case this generator is
used to create a desired vibration field for certain purposes. In the case
of impulse suppression this suppressor should be controlled so as to offset
the harmful gas impulses at a specified part of the line, which is also done
to control impulses in electrical long lines, waveguides and resonators,
This function can be formulated in stringent mathematical terms. To explain
the idea let us take up one elementary example. In a pipeline of length
the gas impulses are described by the equations
a~ 8(aC-Xo) =6o • ( ~ ) - ~ = m e " - d x - - - U ( Z J , - o o ~ ~ ~ o o
where ~E~ zL) is pressure, G {~/i) is flow, /'#o is acoustical mass, O~.~o~ is the points where the active suppressing action is applied, Co is acousti-
cal capacity, ~//~Jis the volumetric flow of the suppressor. The distur-
bances are coming from the basic compressor at ~= O . The boundary condition
at ~=O is periodic over ~ (period~)
G (o, t ) : [ A Wot- at l-o L z j_
It is desired to find a law ZaZ(t '3 for the suppressor such that ate((',,,t'o~.~)
there are no impulses:
It may be shown that the desired programmed control is of the form
_ ~ . A ~ cq~ ~ o # ~ I + B,~ ~ ~o/~ ~ (4) C.oo / f b/ (-l ) = coa ~ ~ o
K = t where A 5 and ~ are Fourier coefficients of the function (2) and C is the
sound velocity. Analysis of this formula shows that for practical purposes
only the first several harmonics of the disturbance that carry the bulk of
energy should be suppressed. There is an optimal point J~e where the suppres-
sor can be connected. Fortunately this optimum is not very sensitive to ~ .
Quite naturally a feedback controller rather than a programmed one is a prac-
tical proposition. A number of rather simple controllers can be proposed in-
cluding those that act in response to disturbances. Experience shows that
even simple active controllers reduce the harmful gas impulses several times.
What is important is that the active suppressor is effective in a wide range
of flow parameters.
244
3. A wide and almost unexplored field ~here the distributed system control
theory can be applied are important technological processes and plants ~here
the sources of disturbance and control such as of heat, mass, force, etc.
are mobile (2,5). Mathematical description of processes in such plants is
normally given as various subsWance and impulse energy transfer equations
such as classical and otherwise equations of heat conductivity, diffusion,
wave equations and equations of movement for various continuous media.
In the simplest case a mobile time varying source is described by the funct-
ion t is co ora ed in basic d fe e .
tial transfer equation which describes the plant. The source mobility is des-
cribed by one of the arguments of the function ~U of the form ~-~(ZJwhere
~/~) is the point where the mobile source acts on the plant at time ~. The
fUnction ~[~)is the control action. The function may also incorporate other
control actions ~('LJwhich, change, say, the source intensity. For ~natance,
ideally point-like heating by a mobile source is described by the function
e, : j t ; s ) c ) If the interaction is rather fuzzy, then it can be dome-shaped, e.g. of the
Gaussian type :
. . . . . . . . . s" 3e-
In this (sialgle-dimensional) example Z//7)is the source center position,
~(jc,%)is intensity and ~(zj~)is variable dispersion (focusing) of the
source. The functions Z/C@), ~(~rj~), ~2 (~fp{)can act as control actions.
These control's are quite naturally additionally constrained, e.g.
Z#'~ -~ /~/(il) ~ ~/,wax ; 0 .< Z~g ('.~, ~) -~ Z~ax ; 0 -~ Z~2 ~/~r, ~j. Therefore the problems to be solved are optimal control, controllability, stability, syn-
thesis etc. Mathematically and technologically contro~ of mobile sources is
much more involved than conventional problems without mobile sources. Synthe-
sis of such structures or design of a closed-loop control system may take a
variety of forms. This problem is greatly influenced by the presence of
mobile (e.g. scanning) sensors of data on the state of the plant. In this
case the law of sensor motion is also to be selected (controlled). Consequent-
ly, for synthesis an algorithm of sensor motion is $o be found as well as She
source motion algorithm. Quite naturally these algoriShms should be sele-
cted together so as to satisfy conditions for optimality of the entire system
operation. In conclusion of this Section 3 let us describe just one simplest
problem illustrating the difficulties of a new mathematical nat-ume. It is
required to find a control Z//fJ so that the system
transform from the initial state Q(.r~O.)=~o('Jc. J f o r the final state ~r,T)=O
within S-<X-<~. The boundary conditions are ~ ~2= ~(~]=~.This principal
control problem is reduced to finding the solvability of the following infi-
nite system of "instantaneous" equalities 7-
where all ~,/< = q, 2, ... , are proportional to the Fourier coefficients
of ~ [~rj . Conditionally this problem, in analogy with the well-known
245
problem of moments (5), oanbe termed a nonlinear problem of moments.
4. Let us take up the following controllability and finite control problem
for an oscillatory system • /#
q(o, q(4 --o,.
Find a generalized function (control)~l(~zJ,O.<~-<~ so tha~
O (.~, T)= O , # ( x , T)= O , O~ x .< i (*) The p o i n t ~ f o r wh ich t h e r e i s a s o l u t i o n o£ t h i s p r o b l e m w i l l be r e f e r e d
~o as t h e c o n t r o l l a b i l i t y p o i n t . I f t h e r e i s none , t h i s i s t h e u n c o n t r o l l a -
b i l i t y p o i n t ( 7 ) .
The following Theorem obviously holds.
Theorem. Ra~iomal poi~s are uncontrollability points. This Theorem is of
simple physical significance a force concentrated in the node of some mode
cannot "extinguish" this mode. A question arises, however, ~hether there are
more uncontrollability points other than rational.
To answer this question the following Theorems can be proved.
Theorem 2. If a is one of the tramscendental Louisville numbers then d7 is
an umoontrollability point.
Theorem 3. Algebraical points are controllability points. However, although
there are transcendental u~controllabiliby points in addition to rational
points, metrical theory of Diophantime approximations may be used to prove
the following.
Theorem @. Algebraical points are controllability points.
The irratiomal number a is referred to as an algebraical one of power~ if
it acts as the root of a certain algebraical equation of order~with integer
coefficients.
The characteristic feature of these numbers is that they have a "poor" Diopham-
fine approximation and the "worst" of these has an algebraical number of pow-
er 2, or quadratic irrationalities.
Unlike algebraical numbers the Louisville numbers have a good Diophantine
approximation and, as proved first by Lousville, these numbers are transcen-
dental. These were the first examples of well constructed transcendental
(irrational and nonalgebraical) numbers. Louisville numbers are, for ~n~bance.
~o -hi ,,~ n2" 2 " , c-O Consequently, in qualitative terms it is true that controllability points are
Shose and only those point which have a poor Diophamtime approximation.
246
These results were obtained through transformation by the Wiener-Paley-
-Schwartz theorem of the finite control problem into an associated interpola-
tion problem for integer analytical functions and by using the results of
the Diophantine analysis and of the meSric theory for
Diophantine approximation of numbers.
For simplicity and clarity of presentation only a simplast example is given
here but these results san be extended to more general cases such as systems
ef other forms, other forms of the control signal (including distributed),
higher spatial regularity, etc.
Because the observability problem is conjugate with the controllability
problem, similar results are obtained for the observability problems.
5. Of essential and practical importance is the difficult distributed system
control problem which is an integral part of the finite control problem (FC).
In the latter it is required to find a control which in a finite time inter-
val takes the system from one specified state to another (~,5). Recently
several interesting problems in this field have been solved and simpler met-
hods which lead faster to a finite control, even of the closed-loop type.
It is interesting to dwell on a new, promising notion of fundamental finlte
control (FFC) which is analogous to fundamental solution of a differential
equation (8).
The basic idea of FFC is that first to determine an FC an FFC is sought,
or an FC such that corresponds to disturbances in the form of a ~ - function
(or its derivatives) and then an FC under arbitrary disturbances is obtained
by simple composition of FFC's with a disturbance of an arbitrary form. The
advantage of this approach is that for specific systems it may be much easi-
er to find an FFC. An FFC is often promoted by direct and simple physical
properties and considerations related to ~he nature of the problem. This is
the situation, for instance, for wave problems in nondissipative media.
In determination of an FFC it is generally convenient to write an FC problem
in a standard form whereby all disturbances in the system (initial and final
conditions, nonhomogeneous boundary conditions, internal disturbances) are
"translated" into the basic differential equation of the process determined
over the spatial region ~ and time ~ (if it exists) as a certain func-
tion ~#x~%J~ ~E~2 ~o, included into this equation. Then the initial
and final conditions become zero and boundary conditions, homogeneous. In a
similar way control action can also be represented as a certain function
2~r#~p~ ~E ~, ~oincluded only into the main equation (For simplicity
only the case is considered where the control3~f#r~is indeed a function of
two variables, ~ and ~).
The case where, say, ~/x~)=C~#~r)U/~where the control is essentially
L/C~is considered analogously. Then the FFC is referred to as an FC
247
g~ ~ #~ ~ which "settles" ~he system subjected to The disturbances
~/~c~ ~=#/~-~)g~##-~r returns it to the zero (norwally equilibrium)
state.
wim~,~Dr~o~, it is easy to fi~ ~c ~ ~)for ~ ~itr~ distur-
bance ~c~ ~ via the composition formula
Everything should no~, however, be "squeezed" into the right-hand part of
the basic equation. Partial £undamental finite controls, PFFC, of an appro-
priate form, for instance, associated with the specified nonzero initial
conditions. Strict relations can be established between these PFFC so that
one form of the PFFC can be calculated into another.
Le~ us give a simple example. Let a wave process described by the equation
= have initial conditions ~/~c, O J= ~e ("~,), ~(x,O)~ooundary oonditions Q f - ~ Z ) : t / K ~ ) ~ O K % ~)=0 . The F F C Z / ~ ) w i l l be associated wi th
~he i~itial conditions : ~ ('~2 0-' }: ~-Y); ~ #~r; O2=~, and is instantaneously
determined from simple physical considerations (the characteristics method,
the D'Alembert method)
< , ' ~ s p - ~ [ ~ ~ - ~ J - ~ ¢ " ~ + # ' - ~ ] , o.<~,_~_~.,% o < , , ~ ca) Therefore an FG associated with an arbitrary initial deflection is determined
by a composition formula analogous To (J) d ~Z
~' ," ~.J : l= q o <" s.~ u,- ( 4 ~jd~ = ~<~o <" ~..~[<r <e-,v) - J <~, + s.-.z ~ ] o, ~ : __7_
~ere Qo ,'~..>= ~ at z JIG ~].
6. The classical theory of distributed system flowcharts has played a major
part in automatic control and was very useful to it (6). Therefore it would
be very attractive to extend this theory to distributed systems.
A signal will be a function (vectorial function)~('~i~, ~ , ~ where
is a certain spatial region and ~ is a time (scalar) space. The signal
/~) will be termed a signal of dimensionality ~ if the region ~ is of
dimensionality Z] ~ = O, I, ... ,. A block is a certain system (of any mature)
for which the concepts of the input signal (or input) 2Z/~p~, 5~t6~land the
output signal (or output)~('3c~), ~rE~Z, have been defined. The relation
between the input and the output is univalently specified by a certain opera-
~or ~ ~) = E -~ ~ ( ~ , ~j) {~)
or operator equation:
where ~ is a cer ta in operator and ~ - 4 i t s inverse operator.
I f the input signal Zz// I~/J is of dimensional i ty ~ and the output s ignal ~ / - ~ ) i s of dimensional i ty ~ then the dimensional i ty of t h i s block
248
is Cq~ ~) with ~ referred to as input dimensionality and ~ as output
dimensionality. A (00) - dimensional block is evidently a lumped block. A
characteristic case is where the block (or its operator) is associated with
a certain mathematical physics problem and described by generally) partial
different ial equations, integro-differential equations with initial and
boundary conditions and disturbances. Within the framework of the linear
theory we will assume the existence of a Green fumetion ~ ~J~ ~; #j ~j
(or a resolvent for the integral equation) such that the relation between
~he input ~(~p ~)and the output ~(~ ~of the block is described by a line-
ar integral operator
q (z, ;s j =./S G <.'x, ;,, ÷, r ) z'-'-(s., d, o'r -",,"z,-,-(r, rJ) (3) In the stationary case
@ s, -rJu.,-<S, )dydr 0 ,~
The Laplace transform with respect to :
7 o will be the transfer function of the block. Normally~/~s a meromorphic
function of the variable ~ . The poles of this function are eigen values of
this system. The residues of this function with respec~ to the poles deter-
mine the system eigenfunctions.
The Laplace-transformed equality (@) will be recorded in the form
.8 The operation ~ will be referred to as composition. The composition iS asso-
ciative, distributive but not commutative (generally speaking). For l~mped
systems it is commutative and degenerates into a normal product of functions.
The blocks for which transfer function are known will be referred to as
elementary.
Let us introduce a signal summation block. Its input is signals ~L(~) ,
i :/~...~ #7 and the output is a si~igle signal n
L=l
The summation has evidently a sense when all input signals are of the same
diemensionality and region of definition.
Blocks can be connected in parallel. It is quite natural that this also has
a sense when all blocks are of the same dimensionality.
The ~raasfer function ~/~x2Lp) of /7 blocks connected in parallel is
,p) c8) The operation of parallelL=c'onnection is commutative.
Blocks can connected in cascade. This operation is useful only of the output
249
dimensionality of the preceding block is equal to the kuput dimensionality
of the subsequent block.
The transfer function WIJIf?L p) of /7 cascaded blocks i s /
/A/d:c,/,nJ: l,V~ (~, l,n)@... ®rill (x, -/,pJ=l~I /.v',. 6x, L p) (9) Since She composition is non-commutative so is the operation of cascaded
connect ion.
In (9) the order of multipliers is iuverse to the order of blocks. The trams-
fer function pV(~r~)of a closed-loop system from ~he input 6~-~)to the
ou~pu~ (.~(.X'~,/g)satisfies the integral Fredholm second kind equation
(~.~,p) + ~'[ 6z, ~,p) @ l..Vdz, /,p) =W(..~, I,P) (qo) where
and ~d~/(~r~Dp)is in the feedforward loop and/..t/.zC.r, DF2%ia the feedback loop.
In ma~y cases of practical interes~ this equation is solved very elementary;
for instance it is reduced to a system of linear algebraical equations (dege-
nerate kernels).
Let us give a simplest example of computing the transfer function ~$/~J~;~2~) of a closed-loop system controlling a distributed plant of any physical nature
through a lumped controller in a channel relating the disturbance ~ c 2 ~
and the variable ~o be controlled ~f2~ j (Fig. 2). The flowchart (Fig. 3) of
this system contains a transfer function ~4/~/~)of a free plant in the
feedforward channel and the transfer function ~/~('~'J>J=~/~)cf(a:-~J~('F-~ ) of the controller in the feedback channel. EquatioDa (10) and
(~q) lead to an equation for
(.:c, Lp)+l, VrpJ lv: /x,&,p)12/I/~,/:pg = l.V(~c~ -/,p) ( I~) This equation is essentially reducible to an algebraic one and easily solved.
In this the characteristic fact of the integral equation kernel degeneration
is felt.
+~/(/,J~4 (~, d, p) l¢4 (~ L p ? ( i ~ ) m d z , / , P J = t ' a 4 " z , L P ) / -
The advanced theory of flowcharts is effectively applied not only to control
systems but to problems of physics and mechanics as well.
REFERENCES
I. ~TEOBOEI4~ A.r., C~0HMM 8.H., qS~IDOTEI4H A,B, Hpo6ne~a ynpaBaeH~ TsnaoBo~
npo~Hn~poBHoM ripe HpoHaTHe HaE sa~aqa y~paBaeHMS 0~SETOM 0 pacnpe~eaeH- EM~M napaMempaM~. ABT0~aT~Ea ~ Te~eMexa~Na, ~ 4 , 1978.
2. ByTHoBCE~ A.F., ~ap~cE~ D.B., Hyc~H~EOB ~.M. YnpaB~eH~e pacnpe~e-
aOHHNIgM CHoTeMaM~ ny~eM nepeMe~e~za HC~OqM~a.AB~ouaTzEa Z Te~eMexa~H~a, 5, 1975~.
250
3. Butkovski~ A.G. Distributed Gontrol Systems. Elsevier Publishing Co., New-York, 1969.
4. @e~Gayg A.A., ~y~zoBOKZ~ A.F. MeTO~ Teopzz a~o~aTz~ec~o~o ynpa~eHza. "Hay~a", ~oc~Ba, 197I, 7~4 c~p.
5. ByTEoBoNZ~ A.P. Memoir ynpaBxeaz~ czcTeMaMz c pacnpe~eneHHRMH napaMeTpaMz. "Hay~a", Mock,a, 1975.
6. ~y~o~c~z~ A.P. C~py~Typna~ reopz~ ~ czc~eM c pacnpe~eneHH~Mz napa~eT- paMz. t'ABToMaTZEa Z Te~eMexaH~Na", N.oS, I975.
7. ByTEoBcNz~ A.~. Hpzxo~eHze He~0T0p~x pe~ynB~a~oB Te0pE~ qzce~ N npodne~e ~ZHZTHOP0 ynpa~e~z~ z ynpa~eMOC~Z ~ pacnpe~ene~H~x czo~e~ax. ~o~a~ AH CCCP, ~.227, I~ 2, 1576.
8. By~omc~zR A.F. @yn~a~eHTan~noe ~znz~Hoe ynpa~ncHze. ~o~na~m AH CCCP, .220, ~ I, 1975.
251
I
]
~ 9 .Z
,~-9, -~
~-(Z, p) F--
I F - ' - ~ I I]¢¢cf- c) i II ~ I we,.> ,- It i-
L L I
t " I
- - I
lOcfi~')[
F,' 9 3
PARTITIONING: THE MULTI-MODEL FRAMEWORK FOR ESTIMATION AND CONTROL~
I: ESTIMATION
D. G. LAINIOTIS Systems Research Center College of Engineering
State University of New York at Buffalo Amherst, New York 14260
U.S.A.
ABSTRACT
Partitioning, the multi-model framework for estimation, and control introduced
by Lainiotis [1-26], constitutes the unifying and powerful framework for optimal
estimation and control, for linear as well as nonlinear problems. Partitioning is
the natural setting for estimation problems, since it decomposes the original esti-
mation problem into a set of estimation problems of considerably reduced complexity,
the optimal or suboptimal estimators of which are far easier to derive and, most
importantly, they are far easier to implement. Using the partitioning approach,
estimation problems are treated from a global viewpoint that readily yields and
unifies previous, seemingly unrelated, results, and mos~importantly, yields funda-
mentally new classes of optimal and suboptimal estimation formulas in a naturally
decoupled, parallel-reallzation form. The partitioning estimation formulas are of
considerable theoretical significance. They provide insight into the nature of
estimation problems, and the structure of the resulting estimators. Most importantly,
the partitioning estimation formulas yield realizations of optimal and suboptimal
estimators, both filters and smoothers, that have significantly reduced complexity,
that are computationally attractive, and numerically robust, and whose practical
implementation may be done in a pipeline or parallel-processing mode. Indeed, the
flexible structure of the partitioning estimation algorithms affords a wide variety
of serial-parallel processing combinations that can meet the computational and
storage constraints of a large class of practical applications, especially real-
time ones.
I. INTRODUCTION
The design of optimal estimators constitutes an optimization problem, and as
such it necessitates the availability of the following elements: a) a mathematical
model, which is a mathematical abstraction of the underlying "physical" situation;
b) a cost functional, namely a mathematical statement of the estimation objectives;
and c) the optimization constraints, imposed by physical, economic, or realization
considerations. Of the above elements, the model constitutes the cardinal part
since it is the vital llnk between the "physical" problem in which the resulting
253
estimator will be used, and the mathematical realm in which the estimator must be
designed. The efficacy, and applicability of the estimator depends strongly on the
realism with which the associated mathematical model represents the underlying
physical situation.
Unfortunately, the more realistic the model, the greater its complexity, and
consequently the greater the difficulty of the associated estimation optimization
problem, and the greater is the difficulty in realizing the resulting estimator.
The difficulties are compounded further by the fact that as the realism of the
model increases, so does the lack of knowledge of the model. In most physical
situations complete knowledge of complex models is neither usually available, nor
readily forthcoming, and one is confronted with the design of an optimal estimator
in the face of incomplete model knowledge. To fully account for this model uncer-
tainty, necessitates increasing the model complexity even more, e.g. augmenting
the state-vector with the unknown parameter vector, thus substantially increasin~ the
effective dimensionality of the problem.
The classical approach to the optimal estimation problem has been [i] to assume
the model, whether complex or reduced, to be known, and to proceed with the deri-
vation of the optimal estimator, bearing the consequences of unrealizable estimators
if the model is complex, or the consequences of suboptimal estimators if the model
complexity is reduced unrealistically.
In contrast, the partitioning approach to estimation, Lainiotls [1-25], does
not confront directly the estimation problem with its complex model, nor does it
approximate it by a simpler model of reduced complexity. Instead it decomposes the
original estimation problem exactly into a set of estimation problems of consider-
ably reduced complexity, the optimal or suboptimal estimators of which are far easier
to derive, and most importantly, they are far easier to implement. Specifically, the
partitioning approach consists of the selection of a "parameter" vector e of the model,
each possible value of which specifies a particular realization of the model. As
such, the admissible values of s correspond to a collection of possible models for
the estimation problem at hand. Moreover, following the Bayesian viewpoint, the
choice of a particular value of u, particular model realization, is considered random
with a-prlori probability density that is consistent with the statistics of the
original complex model. Furthermore, through conditioning on this pivotal parameter
vector s the original complex, large-scale, or ~'ill-conditioned" estimation problem is
decomposed into a set of estimation suhproblems, each one corresponding to an ad-
missible parameter-conditional model. This gives rise to the multi-model name for
the partitioning approach. The above estimation subproblems are less complex, e.g.
linear instead of nonlinear, smaller-scale, namely of smaller state-vector dimension-
allty, and better conditioned than the original estimation problem. Most importantly,
these estimation subproblems are completely decoupled from each other, i.e. the
parameter conditional optimal estimate of each is obtained completely independently
254
from the other conditional estimates. Furthermore, the desired overall-optimal
estimate is given as the aggregation of these conditional estimates, the aggregation
given in a simple, weighted-sum, decoupled, parallel-realization form.
There exists large classes of practical estimation problems, both linear as
well as nonlinear, where a natural set of partitioning parameters ~ exists, such
that use of these parameters in partitioning leads to optimal estimators of much
less complexity and computational requirements, and whose practical implementation
is thus far simpler to realize. Several of these classes of estimation problems,
and the associated partitioning estimation formulas will be presented in the follow-
ing sections. It will be shown there, that using the partitioning approach, estim-
ation problems are treated from a global viewpoint that readily yields and unifies
all previous estimation results, and most importantly, yields fundamentally new
classes of optimal and suboptimal estimation formulas in a naturally decoupled,
parallel-implementation structure. The partitioning estimation formulas are of
considerable theoretical significance. They provide deep insight into the nature
of estimation problems, and the structure of the resulting estimators. Most im-
portantly from a practical standpoint, the partitioning formulas yield realizations
of optimal and suboptimal estimators, both filters and smoothers, that have signif-
icantly reduced complexity, that are computationally attractive, and numerically
robust, and whose practical implementation may be accomplished in a pipeline or
parallel-processing mode. Indeed, the remarkably flexible structure of the part-
itioning algorithms affords a wide variety of serial-parallel processing combinations
that can meet the computational and storage constraints of large classes of practical
estimation applications, especially real-time ones.
II. PARTITIONING: GENERAL RESULTS
The most general estimation problem considered in this paper is described by
the following model and statement of objective:
dx(t) = f[x(t)] + g[x(t)]w(t) dt
z(t) = h[x(t)] + v(t)
(i)
(2)
where x(t), and z(t) are the n and m-dimensional state and observation random pro-
cesses, respectively; {w(t)}, and {v(t)} are independent plant and measurement noise
random processes, respectively; they are zero-mean, W hite-gaussian with covariances
l(t), and R(t), respectively. The initial state vector x(t o) has a priori probabil-
ity p(X(to)), and is independent of {w(t)}, and {v(t)}, for t~t O.
Given the measurements l(t,to)={z(o) ; a c (to,t)} , the optimal in the mean-
square-error-sense estimate i(t/t,to) of x(t) is desired.
As previously indicated, the partitioning approach consists of deciding on a
255
"parameter" vector ~, each possible value of which specifies a particular realiz-
ation of the above model. As such, the admissible values of m correspond to a col-
lection of possible models for the estimation problem at hand, and this gives rise
to the multi-model name for the partitioning approach.
Given the choice of "parameter" ~, the desired estimate is given, Lainlotis
[i-17], by the "partition" theorem:
Theorem I:
The optimal mse estimate, R(t/t,to) , and the corresponding error-covariance
matrix P(t,t o) are given by
R(t/t,t o) = f~(t/t,to;~)p(a/t,to)da (3)
P(t,to) = f{e(t,to/~)+[~(t/t,to)-~(t/t,to;~)] [~(t/t,to)-~(t/t,to;~]T}p(~/t,to)d~ (4)
where ~(t/t,to;~), and P(t,tol~) are the model-conditional estimates, and the corres-
ponding covariance; namely they are defined as:
~(t/t,to;a ) = E[x(t) Ik(t,to);a ]
P(t,tola) = E{[x(t)-~(tlt,to;a)][x(t)-~(t/t,to;~)]T/%(t,to;a} The a-posteriori probability p(a/t,to) is given by
e(t,to/a)
P(a/t,to) = IL(t,to/a)p(a)dm P(s)
J
where L(t,t /m) is a likelihood ratio given as o
o o
where h(*) is the causal, mse estimate of h(.), namely
h(o/O,to;~ ) = E{h[x(a)]/X(a,to);a}.
(5)
(6)
(7)
2
II h(~/o,to;u) llR_l(~)d°} (8)
(9)
Remarks:
(i) We note that using partitioning the optimal mse estimate ~(t/t,t o) and its
error-covariance P(t,to) are given, Eqs. (3-4), in terms of the model-condltional
estimates ~(t/t,to;e), and the corresponding model-conditional covariance matrices,
in a weighted-sum, decoupled, parallel-realization form, which is natural for parallel-
processing. Moreover, the weights which are given by the general Bayes-rule of
Eq. (7), are themselves computed in terms of the model-condltlonal estimates
256
h(t/t,to;=).
Thus partitioning, through conditioning on a pivotal set of parameters ~,
decomposes or disaggregates a complex or large scale estimation problem into a set
of simpler estimation subproblems which are completely deeoupled from each other i.e.
each conditional estimate ~(t/t,to;~), is realized independently from the other
conditional estimates. More importantly, the desired optimal estimate ~(t/t,t o) is
given as the optimal aggregation of the conditional (or anchored at =) estimates, the
aggregation given in the simple, weight-sum, decoupled, parallel realization form
of Eqs. (3-4).
(ii) Further, it is noted that the partitioning approach has also been applied
to estimation problems with models other than those of Eqs. (1-2). For example,
Asher and Lainiotis [20] have applied it to the estimation of doubly stochastic
Poisson processes, and Sawaragi, eto al. [37] have applied it to state-estimation
for continuous-time systems with interrupted observations.
(ill) The above "partition" theorem was first obtained by Lainiotis in 1969
[I], and extensively studied since then by Lainiotis and his students [1-25].
III. OPTIMAL LINEAR ESTIMATION: PARTITIONING FORMULAS
The simplest class of estimation problems where a natural set of partitioning
parameters ~ exists is the linear, gaussian class given by the following model:
dx(t) = F(t)x(t) + G(t)w(t) (i0) dt
z(t) = H(t)x(t) + v(t) (ii)
where x(t), z(t), {w(t)}, and {v(t)}, are as described in section II. The initial
state vector x(t o) is now gaussian with a-priori mean and covariance, x(t o), and
P(to), respectively.
In the above linear estimation problem one possible "partitioning" to use,
Lainiotis [1-17], consists of decomposing the initial state-vector into the sum of
two independent random vectors Xn(t o) , and Xr(to ) and considering the remainder r.v.
Xr(to) as the unknown model parameter vector x with respect to which we adapt. For
this choice of "partitioning" vector =, namely ~ = Xr(to), and since X(to) = xn(to)+
Xr(t o) , the parameter = or Xr(t o) is gaussian with a-priori mean and covariance
specified by the following equations:
i(to ) = ~n(to) + ~r(to) , p(to) = Pn(to) + Pr(to) (12)
For the chosen partitioning, the optimal linear estimates, both filtering and
smoothing, are given in the corresponding partitioning formulas by the following
257
Corollary of Theorem I, Lainiotis [1-17].
Corollary I:
The optimal filtered estimate, and the corresponding error-eovariance matrix
are given by:
R(tlt,t o) ~ In(tlt,t ° ) + ~n(t,to)Rr(to/t) (13)
P(t,to) = Pn(t,to) + #n(t,to)Pr(to/t)#~(t,to) (14)
where the smoothed estimate Ir(to/t ) of the partial initial-state Xr(t o) and its
eovariance matrix Pr(to/t) are given by:
Jr(to/t) = Pr(to/t)[Mn(t,to) + P;l(to)Ir(to) ] (15)
Pr(to/t) = [I + Pr(to)On(t,to)]-iPr(to ) (16)
where Mn(t,to) , and On(t,t o) are given by:
Mn(t,to) =/i ~(O,to)HT(~)R-l(~)[z(o)-H(O)~n(~/O,to)]da (17)
o
I~ ~(a" t°)HT(°) R-I(°) H(o) #n(° ' t°) d° (18) On(t,to) =
o
The nominal filtered estimate ~n(t/t,to) and the corresponding nominal-error-
covariance matrix Pn(t,to) are the optimal linear estimate and its covariance for
actual initial conditions equal to the nominal ones. Namely, in(t/t,to) is given by a linear filter, e.g. a Kalman filter, with initial conditions in(to/to,to)=~n(to) , and Pn(to,to)=Pn(to). The transition matrix of the above nominal filter is ~n(t,to).
Remarks:
(i) Proof: Proofs of the above class of linear partitioning formulas were
first given by Lainiotis [5-7], as early as 1971 for the zero-nominal-initial-con-
dition case, and in 1975 [11-12,15-17] for the general version presented above.
These proofs consisted of straightforward use of the "partition" theorem with
p(~) = N{~r(to);Pr(to)}. Specifically, we note that in view of the model's llnearlty
and gaussianess, the model-conditlonal estimates are now given by:
i(tlt,to;a) = In(tit,t o) + ~n(t,to)= (19)
and the a-posteriori density of ~, using Eqs. (7-9), may be readilyshowntobegivenby:
p(a/t,to) = N{Ir(to/t); Pr(to/t)} (20)
258
Substituting Eqs. (19-20) in the "partition" theorem Eqs. (3-4) readily yields the
desired Eqs. (13-18).
(li) Stochastic Interpretation: As indicated previously the above partitioning
formula results by viewing part of the initial state, namely Xr(t o) as an unknown
random parameter and proceeding in an adaptive framework. So, the optimal estimate
has been decomposed into the "non-adaptive" part i (t/t,to), namely the filtered n
estimate of that part of x(t) due to Xn(t o) and the plant noise w(t), and the
"adaptive" part #n(~)Ir(to/t) reflecting the adaptation or learning of Xr(to) , and
given in terms of the smoothed estimate i (t /t) of r o Xr(to)"
(iii) Structure: We note that the partitioning filter formula decomposes
the original estimation problem into a better conditioned problem, namely the nominal
estimation problem with Pn(to) < P(to )' and a parameter estimation problem for the
part Xr(t o) of the initial state. Thus, the partitioning formula partitions the
filtered estimate into a part due to the excitation w(t) only, and a second part
that reflects the "actual" initial conditions.
It is of considerable theoretical and computational interest, and of much
further use in the sequel, to introduce at this point the forward-time and reverse-
time differential versions of the above partitioning formulas. They are obtained
readily by simply differentiating with respect to t, and s(reverse-time) the integ-
ral version of the formulas given in Corollary I.
Forward Partitioning Formulas: The optimal filtered and smoothed estimates and
their covariances are still given by Eqs. (13-16) but now, the remaining quantities
are given by the following forward-time differential formulas:
aMn(t,t o) ~t = ~(t'to)HT(t)R-l(t)Zn(t'to ) ; Mn(to'to) = 0 (21)
aOn(t,t o) ~t ~(t,to)HT(t)R-l(t)H(t)~n(t,to) ; On(to,t ° ) = 0 (22)
and the nominal filtered estimate In(t/t,to) , and the corresponding nominal covari-
ance matrix Pn(t,to) are given by the Kalman-Bucy-filter like equations:
~in(t/t,t o) ~t [F(t)-Kn(t,to)H(t)]~n(t/t,to)+Kn(t,to)Z(t) ; ~n(to/to,to)=~n(t o) (23)
where the nominal filter gain Kn(t,t o) z Pn(t,to)HT(t)R-l(t), and
~Pn(t,to ) ~t F(t)Pn(t'to)+Pn(t'to)FT(t)+G(t)Q(t)GT(t)-Pn(t'to)HT(t)R(t)H(t)Pn(t'to )(24)
with initial condition Pn(to,to)=Pn(to); the nominal-filter transition matrix is
given by:
a~n(t,t o) ~t [F(t)-Kn(t'to)H(t)]~n(t'to) ' ~n(to'to ) = I (25)
259
Backward Partitioning Formulas: The optimal filtered and smoothed estimates
and their covariances are again given by Eqs. (13-161 but now the remaining quantities
are obtained from the following reverse-time differential formulas:
~Mn(t,s) ~s [Fn(S)-Pn(S'S)On(t'sl]TMn(t's)-On(t'S)Xn(S's)+HT(s)R-l(s)Zn (s's)
~On(t,s)
~s
Mn(t,t) = 0
0 (t,s)F (s)+FT(s)O (t,s)+HT(s)R-I(s)H(s)-O (t,s)P (s,s)O (t,s) n n n n n n n
(26)
On(t,t) = 0 (27)
~n(t/t,s) ~s ~n(t'S)Pn(S's1[Mn(t's)+p~l(s'S)Xn(S'S)] (281
~Pn(t'S)~s ~n(t'S)Pn(S'S)~(t's) (29)
~n(t,s) ~s ~n(t's)[Fn(S)-Pn(S'S1On (t's)] ~n (t't) = I (30)
where
Fn(S ) ~ F(s)-Pn(to)HT(s)R-l(s)H(s ) (31)
Pn(S,S) E F(s)P (t)+P (t)FT(s)+G(s)Q(s)GT(s)-P (t)HT(s)R-I(s)H(s)P (t) (32) no no no no
(s,s) ~ F(S)~n(to)+P (t)HT(s)K-l(s)~ (s,s) (33) n no n
~n(S,S) ~ z(s)-H(S)~n(to) (34)
Remarks:
(iv) Family of Realizations: The linear partitioning formulas (LPF) constitute
a family of realizations of the optimal filter, one for each set of nominal initial
conditions or equivalently one for each initial-state-vector partitioning. In part-
icular, the Kalman-Buey (K-B) filter [38] is the member of this family for nominal
initial conditions equal to actual initial conditions, namely for ~r(tol = 0, and
Pr(to) = O. As such the K-B filter is a special ease of the LPF.
(v) Unifying Framework for Linear Estimation: Lainiotis in a series of papers
[7,9,11-12,15-17] has established that the LPF is the unifying framework for linear
estimation, both filtering and smoothing. Specifically, the fundamental and all-
encompassing nature of the LPF was demonstrated by showing that the LPF contains
important generalizations of previous well-known forward and backward estimation
formulas, in fact whole families of such general formulas of which previously ob-
260
tained filtering and smoothing formulas are special cases.
(vi) Theoretical Considerations: The LPF is also the natural framework in
which to study such important concepts as stochastic observability and controlabili~
Fisher information matrix, unbiasedness, etc., Lainiotis [7,9,11-12,15-19]. For
example, we note that the LPF yields interesting generalizations of stochastic ob-
servability and controlability. Specifically, as given in Eq. (18), 0n(t,t o)
constitutes an observability matrix. Moreover, we note from Eq. (16) that
Pr(to/t) ~ Onl(t,to ) (35)
therefore, see Lainiotis [7,9], o~l(t,to ) constitutes also a Fisher information and
with respect to only a part of the initial-state-vector. As such, On(-) constitutes
a generalized observability or Fisher information matrix in the sense of being a
partial observability or partial Fisher information matrix. Similarly, Pn(t,to)
constitutes a generalized controlability with respect to only part of the initial
state [7-9]. This can be readily seen from Eq. (29) or from the more familiar ver-
sion of the controlability matrix obtained from Eq. (29) for Pn(to)=0. The latter
is given by:
Pn(t,to) = Ii ~n(t,°)G(~)Q(o)GT(~)~(t,o)do
o
From the above considerations, we conclude that the actual error-covariance
matrix P(t,t o) is given in Eq. (24) explicitly in terms of the general controlability
matrix Pn(t,to), and the general observability matrix On(t,to).
(vii) Robustness: As indicated above the K-B filter is a special case of the
LPF. More importantly, the partitioning filter is more rohust than the Kalman-Bucy
filter. For example, unlike the Kalman-Bucy filter, the partitioning filter is
robust with respect to the nature of the actual initial conditions in the sense of
being applicable regardless of the nature of the initial conditions, e.g. finite
or infinite. For example, if P(t o) is infinite or very large, direct integration
of the Riccati equation associated with the Kalman-Bucy filter is not feasible,
instead the information form of the K-B filter must be used, which is given in terms
of P-l(t,to). In contrast, the partitioning filter can be readily used for this
as well as any other initial conditions. Moreover, extensive numerical simulation,
Lainiotis and Govindaraj [18-39], has shown that the partitioning filter, unlike
the K-B filter, is numerically robust with respect to "ill-conditioned" initial-
state covariance matrices, or with respect to "stiff" dynamic matrices F(t), namely
for matrices with eigenvalues differing by orders of magnitude.
(viii) Change of Initial Conditions without Data Reprocessing: The partitioni~
filter is the natural framework for the efficient change of initial conditions,
261
without reprocessin$ the data, namely without the need to reeompute Eqs. (21-25).
As can be seen from Eqs. (13-16), the optimal estimate for the actual initial con-
ditions is given explicitly in terms of: a) The remainder initial conditions
Rr(to) and Pr(to), these being the only quantities dependent on the actual initial
conditions; and b) the nominal estimates with the arbitrarily chosen nominal initial
conditions, which are independent of the actual initial conditions.
The above property of the partitioning filter is of cardinal importance in numerous
applications of linear estimation. For example it is of interest in connection with
sensitivity-analysis of linear estimators, when estimates must be obtained for poss-
ible initial conditions. Furthermore, the freedom in choosing the initial conditions
arbitrarily can have significant conputational advantages, Lainiotis [11-12,15-17,26].
Namely, a preliminary evaluation of Eqs. (21-25) may be particularly easy to
obtain for certain specially chosen nominal initial conditions. In these eases,
using the partitioning filter the optimal estimates for the actual initial conditions
may be obtained without reprocessing the data, by using the preliminary evaluation.
For example such important cases arise in connection with estimation problems with
time-invariant or at-least slowly time-varying models. Specifically, the possible
computational advantages of the Chandrasekhar realizations of the K-B filter depend
on the low-rank property of the actual initial conditions. So, for high-rank actual
initial conditions, the Chandrasekhar realization will have no computational advant-
ages. However, in this case the partitioning filter may be used with low-rank nominal
initial conditions, e.g. P (t) = 0, Lainiotis [8-12, 15-19], thus taking full advant- no
age of the possible computational advantages of the Chandrasekhar algorithm even
for high-rank initial conditions, and without reprocessing the data obtain the optim-
al estimates. Another advantageous choice of the nominal initial condition that
considerably reduces the computational burden is the choice of the steady-state
Riccati equation solution as the nominal covariance Pn(to). For this choice of
P (t), the nominal filter~ namely Eqs. (23-25), reduces to a Wiener filter from no the start of data-processing, Lainiotis [16-19,26]. The above two cases, because
of their importance, will be considered in detail in a following section.
(ix) Historical Remarks: The partitioning formulas of Corollary I~ were
first obtained by Lainiotis [15-19] in 1975. They constitute the natural generali-
zation of his previous partitioning formulas using zero nominal initial conditions~
which were first obtained in 1971 [5]. Moreover, the LPF given in this section are
a special case of the nonlinear estimation partitioning formulas obtained by
Lainiotis as early as 1969 [i] and studied extensively by him in the period 1969-
1978 [1-26].
(x) Fundamental Relationships: It is noted that the LPF are given in terms
of the fundamental quantities #n(.), On(') , and Mn(. ). The relationships between
the nominal quantities #n(-) , On(.), and Mn (') pertaining to the nominal initial
conditions, and the corresponding quantities pertaining to the actual initial con-
262
ditions are of considerable theoretical interest, and of much further use. The
latter, as previously defined by Lainiotis [7,9,15-19], are given, in complete
analogy to the nominal ones, by the following:
d~k(t, t o ) at [F(t)-P(t'to )HT(t)R-l(t)H(t)]~k(t'to ) ~k(to'to ) = I (36)
where P(t,t o) is the actual error-covariance matrix obtained from the Riccati Eq. (24)
with the initial condition P(to) ;
ItT O k ( t , t o) = 0~k(~,to)HT(o)R-l(o)H(O)~k(~,to)dC~ (37)
t o
and
~(t,to) = Ii ~k(O,to)HT(~)R-l(~)~(~,to)dO (38)
o
where ~(o,t o) - z(o)-H(o)i(o/O,to) is the innovation pertaining to the initial con-
ditions ~(to) , and P(to).
The desired interrelationships, as obtained by Lainiotis [9,11,15], are given
by the following Corollary:
Corollary II:
~k(t,to) = ~n(t,to)[I + Pr(to)On(t,to)] -I
0k(t,t o) = On(t,to)[I + Pr(to)On(t,to)] -1
~(t,to) = [I + On(t,to)Pr(to)]-l[Mn(t,to)-On(t,to)Ir(to )]
(39)
(40)
(41)
The above interrelationships are useful in deriving all previous filtering and
smoothing formulas, as well as in deriving classes of new filtering and smoothing
formulas. Most importantly, they are essential for the derivation of efficient
change of initial condition formulas, and for the derivation of efficient partitioning
realizations of filters and smoothers. These important uses of the above fundamental
relations are presented in following sections.
IV. OPTIMAL LINEAR SMOOTHING: PARTITIONING FORMULAS
In this section the all-encompassing nature of the LPF is demonstrated by
showing that the LPF yields classes of general forward and backward smoothing formula%
of which all previously obtained formulas are special cases.
263
Forward Smoothing Formulas: The forward-time differential form of the parti-
tioning smoothing formula is readily obtained by simply differentiating with respect
to t the integral version of the formulas, the latter given by Eqs. (15-16). The
partitioning forward-time smoothing formulas are given [9,15-16] in the following
corollary of Corollary I:
Corollary III:
The partial-initial-state smoothed estimate, and the corresponding error-covarl-
ance matrix are given by:
~r(t°/t)~t Pr(to/t)~r(t'tn o)Hr(t)R-l(t)[Zn(t'to)-H(t)~n(t'to)~r(to/t)] (42)
with initial condition ~r( to / to ) = Rr( to) ;
~Pr(t°/t) -P (t /t)*T(t,t )HT(t)R-I(t)H(t)* (t,t)P (t /t) (43) ~t r o n o n o r o
with initial condition Pr(to/to) = Pr(to); and where the remaining quantities are
as given in Eqs. (21-25).
We note that the smoothing formula of Corollary III constitutes a class of forward
smoothing formulas, one for each set of nominal initial conditions.
From the above class, a related class may be readily obtained, as in Lainiotis
[9,15-16], by use of Eq. (34) and by noting that
~(t,to) = ~n(t,to)-H(t)~n(t,to)~r(to/t).
The new class is given by the following variation of Corollary III:
Corollary IV:
~Xr(to/t)
~t
~Pr(to/t)
~t
Pr(to)~(t,to)HT(t)R-l(t)~(t,to)
-Pr(to)~(t,to)HT(t)R-l(t)H(t)#k(t,to)
with initial conditions Rr(to), and P (t) respectively. r o '
(44)
(45)
We note that the Meditch forward smoothing formula [40] is the member of the above
family for the special case of zero nominal initial conditions.
Moreover, by striaghtforward integration of Eqs. (44-45) one may readily obtain,
Lainlotis [9,15-16], the following family of integral smoothing formulas:
264
Corollary V:
ir(to/t ) = Rr(to) + Pr(to)~(t,to)
er(to/t) = Fr(to)-Fr(to)Ok(t,to)Pr(to)
where Mk(-), and Ok(-) are given by Eqs. (37-38).
(46)
(47)
The above family of integral smoothing formulas includes as a special case the
innovation formula of Lainiotis [9,15-16]. The latter is an explicit version of the
previously obtained innovation smoothing formula of Kailath-Frost [41]. Lainiotis
innovation formula is the member of the above class corresponding to zero nominal
initial conditions [9,15-16].
It is given [9,15-16] by
~(to/t ) = ~(to)+P(to)~(t,to) (48)
P(to/t) = P(to)-P(to)Ok(t,to)P(to) (49)
We note also that the smoothed estimate ~n(to/t) Of Xn(t o) based on the data and with
nominal initial conditions, and the corresponding covariance Pn(to/t) are given by
equations similar to Eqs. (48-49), namely
in(to/t) = In(to! + Pn(to)Mn(t,to) (50)
Pn(to/t) - Pn(to) - Pn(to)On(t,to)Pn(to) (51)
In conclusion, we note that in a similar direct manner all other previously
obtained forward smoothing formulas may be shown to be special case of the forward
partitioning formulas.
Backward Smoothin$ Formulas: Using the reverse-time version of the LPF as
given in Eqs. (26-34), we may readily obtain general classes of backward smoothing
formulas. For example such an interesting general class is the class of two-filter
smoothing algorithms. This class pertains to smoothing the part Xr(S) of the state-
vectorx(s), given the data ~(t,to) , and the initial conditions {~(to),P(to)}, where
< s < t , a n d x ( s ) = X r ( S ) + X n ( S ) . t O _ _
The desired smoothing formulas are given by the following version of Eqs. (15-16):
Rr(S/t,to) = Pr(S/t,to)[Mn(t,s)+p:l(s,to)Rr(S/S,to)] (52)
265
and
Pr(S/t,to) = [l+Pr(S,to)On(t,s)]-~r(S,to) (53)
where ~r(S/S,to) 5 R(s/S,to)-Rn(S), and Pr(S,to) = P(S,to)-Pn(S), and R(s/S,to) , and
P(s,t o) are given by the forward version of the LPF, namely Eqs. (13-18), and Eqs.
(21-25), with actual initial conditions at t o [R(to),P(to)}, while Mn(t,s) and On(t,s)
are given by the backward version of LPF, namely Eqs. (26-34). The latter utilize
only the data l(t,s), and the nominal initial conditions at s, namely {Rn(S),Pn(S)}.
Thus, this class of smoothing algorithms are given in terms of the forward filter
for ~(s/S,to) , namely Eqs. (23-25), and the backward filter for calculating Mn(t,s),
namely Eqs. (26-27).
The above smoothing formulas constitute a class of two-filter smoothing formulas~
one for each possible partitioning of the state-vector x(s). In particular, the
Mayne-Fraser [43] formula is a member of this class corresponding to the choice of
zero nominal initial conditions, Lainiotis [7,9,15-16].
As an indication of the exceptional flexibility afforded by the LPF, we note
that the above class of two-filter smoothing formulas may be given [9,15] in terms
of: a) a forward and a backward filter as indicated above; or b) in terms of the
forward filter given above, and a second forward filter for calculating Mn(t,s) , the
latter described by Eqs. (21-25); or c) in terms of a backward filter for i(s/S,to),
and a forward filter for M (t,s). n
In a similar straightforward fashion, several other general classes of backward
smoothing formulas are readily obtained [7,9,15].
General Backward Markov Models: Backward Markov models are of theoretical
interest, as well as of computational importance in several applications. These
backward Markov models are equivalent to forward process models in the sense that
when solved in the backward direction they yield the same state-covariance as the
corresponding forward model. The availability of equivalent backward-forward model
pairs provides considerable flexibility since either backward or forward filters
(smoothers) may be used to match the constraints of practical applications.
As noted above, the backward partitioning smoothing formulas are the natural
setting for such backward or forward realizations of filters and smoothers. More-
over, it has been shown, Lainiotis [7,9,11,15-16,44], that the backward version of
the LPF yields readily the most general class of backward Markov models.
V. EFFICIENT CHANGE OF INITIAL CONDITIONS
It is of some interest to obtain formulas that provide the filtered and smoothed
estimates corresponding to initial conditions {i(to),P(to)}, utilizing only the
filtered and smoothed estimates with initial conditions {In(to),Pn(to)} , and without
266
additional processing of the data. Such efficient, and explicit change of initial-
conditions formulas for filtering and smoothing are readily obtained using the LPF,
namely through trivial algebraic manipulations of corollaries I-II, and of Eqs. (48,50).
The efficient change of initial-conditions formulas are:
Filterin8
~(t/t,to) = R (t/t,t)+~. (t,t){3 (t)+P (t)P-l(t )[In(to/t)-In(to)]} n o K o r o r o n o (54)
Smoothin$
i(to/t ) = Rn(to/t)+[I-P(to)Ok(t,to)]-{ir(to)+Pr(to)Pnl(to)[Rn(to/t)-in(to)]} (55)
where all the quantities appearing in Eqs. (54-55) have been defined previously.
The above formulas provide the filtered and smoothed estimates ~(t/t,t o) and
i(to/t), for initial conditions {i(to),P(to)} , in terms of the filtering and smoothed
estimates R n ( t / t , t o ) , and ~ n ( t o / t ) , f o r i n i t i a l c o n d i t i o n s { ~ n ( t o ) , P n ( t o ) } , wi thout reprocessing the data. Indeed, the only data processing involved is the one performed
in the calculation of the nominal filtered and smoothed estimates.
VI° GENERAL CHANDRASEKHAR FORMULAS: FAST ESTIMATION ALGORITHMS
The LPF for filtering and smoothing may be realized in terms of a set of two
generalized x-y or Chandrasekhar algorithms, even for time-varying models. This
realization is of some theoretical imterest, and in the case of time-invariant models
yields considerable computational advantage. The general Chandrasekhar realization
of the LPF is obtained trivially from the backward and forward versions of it by
simple algebraic manipulations. It is given in the following corollary of Corollary
I:
Corollary VI:
The nominal estimate In(t/t,to) is now given by:
ain(t/t,t o) at [F(t)-Kn(t'to)H(t)]In(t/t'to)+Kn(t'to )z(t) (56)
with initial condition In(to/to,to)=i(to); where the gain Kn(t,t o) is given, Lainiotis
[15,18-19], by the following general Chandr~sekhar algorithm:
aKn(t,s) ~s -Yn(t,s) • S • yr(t's)Hr(t)R-l(t)n , Kn(t,t)=PnHT(t)R-l(t) (57)
~Y (t,s) n [F(t)-Kn(t,s)H(t)]Y (t,s) Yn(S,s)=L(s) (58) at n '
The smoothing quantitiy Mn(t,t ~) is given correspondingly by
267
~Mn(t't°)~t A~(t'to)[z(t)-H(t)Xn(t/t'to)] , Mn(to,t o) = 0 (59)
where the gain An(t,t o) is obtained via the following dual, general Chandrasekhar
algorithm
~An(t,s) ~s An (t't)=R-l(t)H(t)
~Bn(t,s) ~t Bn(S'S) = 0
-An(t,s)[Fn(S)-L(s) " S " Bn(t,s)] ,
LT(s) . A~(t,s)R(t)An(t,s )
(60)
(61)
The partial observability matrix On(t,to) is now given by the following quad-
rature formula:
~On(t,t O) Ar(t,t- )R(t)A (t,t) On(to,to) = 0 (62) ~t n o n o
where Fn(S), and Pn(S,S) are given by Eqs. (31)-(32), and L(s), S are obtained from
the LDU decomposition of Pn(S,S), Namely
(s,s) = L(s) - S • LT(s) n
The partial-initial-state smoothed estimate at any time ti,
given by:
Ir(to/t i) = [l+Pr(to)On(ti,to)]-iPr(to)Mn(ti,to )
and the corresponding optimal filtered estimate is computed from
(63)
Ir(to/ti), is now
(64)
i(ti/ti,t o) = In(ti/ti,to)+ir(ti,to) (65)
where Ir(ti,to) is computed from
d~r(t,t o) dt [F(t)-Kn(t'to)H(t)]~r(t'to) , ~r(to,to)=~r(to/t i) (66)
Moreover, if the total-initial-state smoothed estimate is desired, it is com-
puted via the formula:
~(to/ti) = ~(to)+P(to)Prl(to)~r(to/t i) (67)
Finally, if in addition to the above estimates, the corresponding covariances
are wanted, they may be computed as follows:
Pr(to/ti) = [l+Pr(to)On(ti,to)]-iPr(to ) (68)
268
P(ti,to) = Pn(ti,to)+Pr(ti,to) (69)
where Pn(ti,to), and Pr(ti,to) are obtained from
8P (t,s) n =-Yn(t,s) - S • yT(t,s) Pn(t t)=P Bs n ' ' n
~Pr(t,to ) ~t = [F(t)-Kn(t'to)H(t)]Pr(t'to)+Pr(t'to) [F(t)-Kn(t,to)H(t)]
(70)
(71)
Pr(to,to)=Pr(to/ti). The total-initial state error-covariance matrix is given by:
P(to/t.) = P(t )-P(t )O (t.,t)P (t /t.)p-l(t )P(t ) (72) 1 0 0 n 1 o r o 1 r 0 o
Remarks :
(i) Proof: The above version of the LPF results from simple manipulations
of the forward and backward versions of the LPF, Lainiotis [15,18-19]. Specifically,
Eqs. (56) and (64) are Eqs. (23), and (15), respectively, for the choice of initial
condition Rn(to)=R(to). The remaining equations result by first defining the variables
Kn' Yn' An' and B as follows: n
Kn(t,s) ~ e (t,s)HT(t)R-l(t) (73) n
Yn(t,s) ~ ~n(t,s)L(s) (74)
An(t,s) ~ R-l(t)H(t)~n(t,s) (75)
Bn(t,s) E eT(s)On(t,s) (76)
Then, Eqs. (59), (62), and (70) result from Eqs, (21-22), and (29), respectively
through use of the above definitions, and the LDU decomposition of Eq. (63). Post-
multiplying Eq. (29) by HT(t)R-I(t), postmultiplying Eq. (25) by L(s), premultiplying
Eq. (22) by LT(s), and finally premultiplying Eq. (30)by R-l(t)H(t), and using the
definitions yields Eqs. (57), (58), (61), and (60), respectively. The Eqs. (65), (68),
and (69) are a rewrite of Eqs. (13), (16), and (14), respectively. The diff. Eqs. (66),
and (71) are the differential version of the second term in the rhs of Eqs. (13),
and (14), respectively. Finally, (67), and (72) are readily obtained using Eqs. (40),
and (41) in Eqs. (48-49), respectively.
(ii) Time-lnvariant Models: For time-invariant models the above version of the
LPF simplifies considerably. Specifically, it reduces, Lainiotis [15,18-19], to the
following:
The nominal estimate is now given by
d~ (t) n [F_Kn(t)H]~n(t)+Kn(t)z(t ) ~ (0)=~(0) (77) dt ' n
269
dK (t) ~t Yn(t) • 9 • Y~(t)HTR -I , Kn(0)=PnHTR-I (78)
dY (t) n [F_Kn(t)H]Yn(t ) Y (0)=L (79) dt ' n
The smoothing quantity Mn(t) is given by:
dMn(t) dt A~(t)[z(t)-H~n(t)] M (0)=0 (80)
' n
dA (t) ndt An(t) [F n - L • S • Bn(t)] , An(0) = R-IH (81)
dBn(t) L T • A~(t)RAn(t) B (0)=0 (82) dt ' n
The partial observability and the partial controlability matrices are now given
by:
d0n(t) dt A~(t)RAn(t) , On(0)=0 (83)
dP (t) n Yn(t) - S • yT(t) P (0)=0 (84) dt n ' n
Finally, Eq. (66), and (71) reduce to the following simpler ones:
dl (t) r dt [F-Kn(t)H]~r(t) ' ~r(0'0)=~r(O/T)
dPr(t) dt [F-Kn(t)H]Pr(h)+Pr(t)[F-Kn(t)H] , Pr(0,0)=Pr(0/T)
(85)
(86)
The quantities Fn, Pn' and L are now time-invariant but otherwise unchanged,
while the remaining equations are as given in Corollary VI.
We note that in the above simpler version of the Chandrasekhar realization of
the LPF we have adopted a simpler notation for convenience, namely In(t)=in(t/t,to) ,
Mn(t,to) ~ Mn(t) , etc., and also have simply assumed, without any loss of generality,
that t =0. o
(iii) Theoretical Considerations: We note from Corollary VI, that the LPF,
in their general Chandrasekhar realization are given in terms of the two, dual, two-
point boundary-value problems specified by Eqs. (57-58), and (60-61), respectively.
However, for time-invariant models, the two-point boundary-value problems reduce to
the initial-value problems given by Eqs. (78-79), and (81-82), respectively.
We note further that the above LPF version constitutes a class of Chandrasekhar
realizations, one for each possible choice of initial condition P . In the followin~ n
section, it will be shown that choosing the steady-state covariance as P results n
in a eomputationally attractive, and theoretically interesting realization of optimal
270
linear estimators.
~iv) Computational Considerations: The computational requirements, and
hence the possible advantages, of the Chandrasekhar realization of the LPF depend
on the relative size of m and n, and most importantly on the rank of P • If P has n n
rank £, then the matrices K n, Yn' An' and Bn are of order nxm, nx£, mxn, and Zxn,
respectively, while O and P are nxn matrices. Hence the computations in Eqs. n n
(56-58), (59-62), and (66) require (m2+Z)n2 operations per iteration. Moreover, 3
Eqs. (64), and (67), require n operations but are computed only once at the end of
the desired interval (0,T). So as long as the filtering and smoothing estimates are
wanted for a small number of points, the Chandrasekhar realization of the LPF can have
computational advantages, especially for small m/n, and small ~.
We note that the Kalman-Bucy filter also can be realized in terms of the
Chandrasekhar algorithm corresponding to the actual initial conditions [15,18-19,45-46].
However, the possible computational advantages of the Chandrasekhar ~ersion of the
K-B filter depend on the low-rank property of the actual initial conditions. So,
for high-rank actual initial conditions the K-B-Chandrasekhar filter will have no
computational advantages whatsoever. In contrast, the Chandrasekhar version of the
LPF may be used in this case choosing low-rank nominal initial conditions, e.g.
Pn = steady-state covariance, Lainiotis [15,18-19,26], thfls taking frill advantage
of the Chandrasekhar algorithm even for High-rank initial conditions, and without
rep~ocessing the data obtain the optimal filtered and smoothed estimates.
Extensive numerical simulation results Lainiotis [15,18-19], Lainiotis et al
[27-32], indicate that the Chandrasehkar version of the LPF results in fast numerical
algorithms that are faster for the same accuracy than the K-B filter even for actual
initial conditions pertaining to low-rank Chandrasekhar realizations.
VII. EFFICIENT PARTITIONING REALIZATIONS OF LINEAR ESTIMATORS
As indicated previously, for estimation problems with time-invariant models there
is a particularly efficient realization of the optimal linear filter and smoother.
This realization is based on the LPF with nominal initial conditions: ~n(0)=~(0),
and Pn = steady-state error-eovariance. For this choice of Pn' we have
F = FP +e FT+GQ~-P HTRIHp = 0 (87) n S S S S
and the corresponding realization is given by the following:
Efficient Realization:
d i ( t )
n [ F _ K n H ] % ( t ) + K n Z ( t ) i ( 0 ) = 2 ( 0 ) (88 ) dt ' n
= PsHTR-I; where K n and
271
dM(t)
dt
HA (t) n dt
AnT(t) [z(t)-Hin(t) ]
_ _ - An(t ) [F-KnN]
M (0) = 0 n
A (0) = R-IH n
(89)
(90)
and
dOn(t) dt A~(t)RAn(t) ' On(0) = 0 (91)
d~n(t) dt [F-KnH]~n(t) " #n (0) = I (92)
which are used to compute In(ti), Mn(ti), On(ti), and ~n(ti) for use in the following
equations:
ir(0/ti) = [l+Pr(0)On(ti)]-iPr(O)M(ti ) (93)
i(ti) = Rn(ti) + ~n(ti)Ir(0/t i) (94)
and
i(01t i) = i(0) + P(0)p]l(0)Ir(0/ti! (95)
Remarks:
(i) Structure of the Efficient Realization: The structure of the efficient
realization is exceptionally simple. Namely it consists of the steady-state o~
Wiener filter given in Eq. (88), and the linear filter of Eq. (89~. We note, more-
over, that the computation of the gains of the above filters do not require the use
of the two dual Chandrasekhar algorithms of Eqs. (78-79), and (81-82). Indeed, the
two Chandrasekhar algorithms degenerate to: a) the computatfon of the constant gain dKn
matrix K P HTR -I = or to the trivial diff. equation-~ - = 0 and b) the dual n s
Chandarasekhar algorithm of Eqs. (81-82) is replaced by the linear diff. Eq. (90).
(ii) Nature of the Efficient Realization: To appreciate how interesting is the
above version of the LPF, it must be noted that the Kalman-Bucy filter in any of its
realizations, namely the standard one obtained by Kalman-Bucy or the Chandr~sekhar
version of it obtained by Kailath-Lindquist [54-55], results in a filter that is time-
varying even for time-invariant models, at least in the transient-stage. Yet the
above efficient LPF realization of the optimal filter is given in terms of the steady-
state W~ener filter from the very-first instant. This transformation of a time-
varying filter into an effectively time-invariant one is due to partitioning with the
above choice of nominal initial conditions.
272
(iii) Computational Considerations: The computaflional and storage requirements
of the efficient realization are drastically less than those of other realizations
of the LPF. It does, however, require the additional computation associated with
obtaining the steady-state solution of the Riccati equation. This computation is
done only once, and it may be accomplished via the fast, doubling algorithms,
Lainiotis [15,18-19].
VIII. OPTIMAL LINEAR ESTIMATION: FAST, PARALLEL-PROCESSING, PARTITIONING,
NUMERICAL ALGORITHMS
Partitioning, and the LPF of previous sections serve as the basis of computation-
ally effective, fast, and numerically robust algorithm for the numerical implementation
of optimal linear estimators. As will be demonstrated in subsequent sections, these
algorithms also are most effective in the implementation of wide classes of nonlinear
estimators. These partitioning numerical algorithms, have a completely decoupled
or partitioned structure that results through partitioning of the total data interval
into nonoverlapping subintervals, solving the estimation equations in each subinterval
with arbitrarily chosen nominal initial conditions for each subinterval, and connect-
ing the piece~ise solutions via the LPF of Eqs. (13-14). Thus, the desired estim ~
ation solution over the whole data interval Nas been decomposed into and is exactly
given in terms of a set of elemental solutions that are completely decoupled from
each other via anchoring the initial point of each data subinterval at the arbitrarily
chosen nominal initial conditions. As such these elemental estimation ~61utions are
computable in either a serial or parallel-processing mode. Further, the overall
solution is given exactly in terms of simple recursive operations on the elemental
solutions.
The numerical partitioning estimation algorithms result simply by choosing the
"partitioning" vector ~ to be ~ = {Xr(i);i=O,l,2 .... }, where Xr(i)~x(ti)-xn(i), and
considering e to be an unknown model parameter, with respect to which we adapt.
The resulting general lamhda algorithm for the estimation problem of section III is
given as follow~:
Lambda Algorithm:
Given the chosen partitioning of the data interval (to,tf) into the nonover ~
lapping s~bintervals (ti,ti+l) , i=0,1, . . . f-l, the most general lambda algorithm
is given by:
~(i+l,0)=%(i+l,i)+~n(i+l,i)[I+Pr(i,0)On(i+l,i)]-l-[Pr(i,0)Mn(i+l,i)+ir(i,0)] (96)
P(i+l,0)=Pn(i+l,i)+~n(i+l,i)[I+Pr(i,0)On(i+l,i)]-iPr(i,0)~(i+l,i) (97)
where i(i+l,O) E i(ti+i/ti+l,to) , and P(i+l,0) E P(ti+l,to) are the optimal estimate
273
and its eovariance matrix at ti+ 1 with actual initial conditions ~(to),and P(to) ,
at t=t o, respectively; ~n(i+l,i) E ~n(ti+l,ti) , and Pn(i+l,i) E Pn(ti+l,t i) are the
nominal estimate and the corresponding nominal covariance at t=ti+l, obtained using
Eqs. (23-24), or their Chandrasekhar versions given by Corollary VI, with nominal
initial conditions at ti,~n(1) and P (i). The remainder initial conditions at t. n i
are defined as:
Rr(i,O) 5 R(i,O) - ~n(i) (98)
Pr(i,O) ~ P(i,O) - Pn(i) (99)
Moreover, 0 (i+l,i) E On(ti+l,ti) , Mn( i+l,i ) E Mn(ti+l,ti) , and #n(i+l,i) n
~n(ti+l,ti) are given by Eqs. (18), (17), and (251, respectively, for the subinterval
(ti,ti+l) , and with nominal ~nitial conditions ~n(i), and Pn(i) at t i. We note that
On, Mn, and ~n may alternately be obtained as indicated in Corollary Vl via
Chandrasekhar algorithms.
Remarks:
(i) Parall~l-Processin$ Nature: Note that the lambda algorithm gives exactly
the optimal filtered estimate and its eovarianee at discrete times ti, i=0, 1 ,...
It does so in terms of the set of elemental nominal solutions (Rn(i+l,i),Pn(i+l,i)},
i = 0, i, 2, . .., of each subinterval, the overall optimal solution being given
by the simple recursive operations of Bqs. 496-971. Most importantly, these elemental
solutions are completely decoupled from each other, as a result of the particular
partitioning chosen. As such, they are computable either via serial or parallel-
processing, the latter drastically reducing the processing time, and storage require-
ments. Indeed, the lambda algorithm affords a wide variety of parallel-serial pro ~
cessing =ombinations that can match a wide class of applications-especlally real
time ones-- and a wide range of 5ufferlng constraints.
(ii) Design Considerations: In general, the choice of the subinterval length,
and the choice of nominal conditions - namely the choice of a partitioning scheme -
is completely arbitrary. This freedom of choice affords considerable computational
advantages for wide classes of estimation problems. Namely, just as in section VII
a judicious choice~of subinterval nominal initial conditions, and lengths, can reduce
the computational and storage requirements substantially.
(iii) Time-lnvariant Models: For this class, the lambda algorithm decreases
the computational and storage requirements if: a) the lengths of the subinterval are
the same; add b) if the nominal initial conditions for all subintervals are the same.
Under these conditions, the subinterval, nominal quantities ~n(i+l,i),On(i+l,i), and
Pn(i+l,i) are not only decoupled from those of any other subinterval hut also they
are the same for all subintervals, since they are obzained by the same differential
274
equations (because of time invariance), and with the same subinterval lengths (by
design choice). As such, Pn(i+l,i), ~n(i+l,i), and On(i+l,i) are only functions
of the model, and of the com~non subinterval length A, where 4~ti+l-ti, namely Pn(4),
#n(~), and On(A). Thus, the lambda algorithm for time-invariant models takes the
following simple form:
~(i+l,O)=~n(i+l,i)+¢n(A)[l+Pr(i,O)On(A)]-l[Pr(i,O)Mn(i+l,i)+~t(i,O)] (i00)
P(i+l,O) = Pn(A)+~n(4)[l+Pr(i,O)On(A)]-iP (i,o)~T(A) (i01) r n
where the elemental nominal estimates in(i+l,i) and smoothing quantities M (i+l,i) ' n
may be obtained hy repeated use, one for each subinterval, of the efficient algorithm
of the previous section. Moreover, ~ (4), 0 (4), and P (A) are also obtained by n n n
using the same efficient realization.
It is seen that the above lambda algorithm has considerably reduced computation-
al and storage requirements. Specifically: a) Pn(4), ~n(4), and 0 (A) need only n
be calculated once, and stored for subsequent use; b) the gains Kn(t,i) , and An(t,i)
used to compute ~ (i+l,i), and M (i+l,i), respectively, need only be computed once n n
for a time-interval of length 4 , and then stored for subsequent use. The above
computational and storage requirements contrast sharply with those of the Kalman-
Buey filter for whichever of i~s realizations. Specifically, the K-B filter requires
the evaluation of its gain ~(t,t o) on-line or off-line and storage of ~(t,to),
for the whole data interval, namely for tC(to,tf). Thus, the computational and
storage requirements of the lambda numerical algorithm will be a small fraction of
those of the K-B filter, especially for non-low rank actual initial conditions.
(iv) Slowly Time-Varying Models: For the class of time-varying estimation
problems with models which are slowly time-varying in comparison to the total data
interval it is possible to obtain reduced computational and storage requirements
for the lambda algorithm similar to the ones for time-invariant models. Specifically
one may partition the total data interval into subintervals in each of which the
model is approximately constant, although of course, the model will differ from
subinterval to subinterval. Then one proceeds to apply the simplified lambda
algorithm of Eqs. (i00-i01), in each subinterval, having previously subpartitioned
each interval into data intervals of length A. for the ith subinterval. It may be I
noted that the subinterval partitioning lengths A. need not be the same. l
It is further noted that since in each subinterval the model is approximately
time-invariant, it i~ possible to reduce further the computational and storage
requirements of the lambda algorithm by using for the computation of Pn(Ai), ~n(4i),
On(4i) , as well as of the gains Kn(t,i), An(t,i), for use in each subinterval ,
the efficient realization of the previous section based on the choice of
the subinterval nominal initial condition P = lhe steady-state-error-covariance ni
275
pertaining to the time-invariant model of each subinterval.
(v) Periodic Models: There exists a class of time-varying estimation problems
for which there is a similar decrease in the storage and computational requirements
of the lambda algorithm, as there is for time-invariant models. Specifically, if
the time-variation is periodic, then by choosing the subinterval length to be an
integer multiple of the period and by using the identical nominal initial conditions
for all subintervals, the lambda algorithm is again given by Eqs. (i00-i01) with
similar storage and computational advantages as discussed in remarks (iii)-(iv).
There are however two important differences between the periodic and the time-invar-
iant model cases. Specifically, a) in the periodic case A must be an integer multiple
of the period, while in the time-invariant case A may be chosen arbitrarily; b)
moreover, in the periodic case, unlike the time-invariant one, the efficient LPF
realization of Section Vll which utilizes a Wiener filter can not used.
Doubling Partitioning Algorithm; Delta Algorithm: As indicated above, in many
applications *ith time-invariant models, the steady-state or Wiener filter may be
used effectively to obtain the nominal estimates, thus drastically reducing the com-
putational and storage requirements without loss of the estimation efficiency,
namely without resorting to suboptimal state-estimation. However, this requires use
of the efficient LPF realization of Section VII. This in turn requires that the
steady-state solution of the associated Riccati equation be obtained first. To
reduce the integration time, and hence the computation time, and the computations
required to obtain the steady-state solution, the following doubling algorithm may
be used. This "doubling" algorithm is based on "doubling" the length of the part-
itioning interval at each iteration of Eq. (i01), and straightforward use of the
lambda algorithm. The resulting doubling partitioning algorithm, so-called delta
algorithm for short, is particularly fast, numerically robust, and essentially in-
tegration-free, Lainiotis and Gn~rindaraj [11-12], Lainiotis [15-18,33]. The delta
algorithm, obtained by Lainiotis and Govindaraj in 1975 [11-12] and [15-18,33], is
given by:
Delta Algorithm:
Given the arbitrarily chosen initial partitioning interval A, the delta algorithm
is given by the following recursive equations:
p (2n+iA) = Po(2nA)+~o(2nA)[l+P(2nA)Oo(2nA)]-ip(2nA)~(2nA) (102)
where now Po(-), ~o(.), and 0o(-) are given by the following iterative equations:
Po(2n+iA) = Po(2nA)+¢o(2nA)[l+Po(2nA)Oo(2nA)]-iPo(2nA)~(2nA ) (103)
~o(2n+iA) = ~o(2nA)[l+Po(2nA)Oo(2nA)]-l¢o(2nA) (104)
276
0o(2n+l& ) = Oo(2nA)+~(2ng)[l+Po(2nA)Oo(2nA)]-iPo(2nA)#o(2nA )
for n = 0, i, 2, ....
(lO5)
Remarks:
(i) Note the possible computational advantages of the above "doubling" algorithm.
Specifically, it only requires: a) integration of the first A interval to obtain
~o(A), Oo(A) with which to start Eqs. (102-105) and, subsequently, b) repeated use
of the simple recursive Eqs. (102-105) until II p(2n+IA)-P(2nA) II ! ~, where e is
prespeeified to give the steady-state solution to the accuracy desired. These
recursive equations require matrix inversions to obtain the Riccati equation solution
at the end of the time-interval which is twice as long as the interval in the previous
iteration, i.e., doubling. Note, however, that in (102-105) the same matrix inversion
is required. Thus in all two (nxn) matrix inversions are required, one in (102) and
the other in (103-105).
(ii) To ascertain the possible computational advantages of the lambda algorithm%
an extensive numerical &imulation study was undertaken. It was of particular interest
to assess the relative speed of the lambda algorithm vis-a-vis the X-Y algorithm and
direct Runge-Kutta (RK) integration of the Riccati equation, and also to investigate
the sensitivity of the lambda algorithm to ill-conditioned initial conditions and
"stiff" system matrices F. Furthermore, the stability of the lambda algorithm with
respect to numerical errors was of interest.
Several numerical examples were simulated for Riccati equations of order 2x2
to 20x20. It was found that the lambda algorithms have considerable numerical and
computational advantages - specifically, speed, numerical robustness, and stability.
The detailed results of this numerical study have been documented extensively else-
where [16-19,27-31].
IX. MULTI-PARTITIONING LINEAR ESTIMATION FO~S AND FAST ALGORITHMS
For the linear estimation problem defined in section III, there exists still
another possible "partitioning" to use which results in linear estimation formulas
that are theoretically interesting, and which yield numerical estimation algorithms
with considerable computa£ional advantages.
The proposed new partitioning,proposed by Lainiotis in 1975, and studied by
Lainiotis et al [22-25,47], consists of decomposing the initial state-vector into the
sum of L statistically independent gaussian, random vectors:
L
X(to) = Xn(to) + ~ xi(to) (106) i=2
277
and considering ~ m {x2(to) , x3(t o) . . . XL(to)} as the unknown model parameter-
vector with respect to which we adapt. For this choice of "partitioning" vector
a, we have the following a-pridri mean and eovariance relations:
L L i(to) = In(to) + ~ li ' e(to) = Fn(to) + ~ P" (107)
i=2 i=2 1
For the chosen partitioning scheme, the optimal filtered estimate is given in
the corresponding partitioning formulas by the following Corollary of Theorem I,
Lainiotis and Andrisani [22-25]:
Corollary Vll:
The optimal filtering estimate and the corresponding error-oovarianoe matrix
are given by:
L i(t/t,t o) = ~n(t/t,t o) + ~ #i(t,to)li(to/t) (108)
i=2 L
P(t,t o) = Pn(t,to) + ~ ~_i(t,to)Pi(to/t)~l(t,to) (109) i=2
where the nominal estimate in(t/t,to) , and its co~ariance are defined as:
in(t/t,t o) = E[x(t)/l(t,t o) ; xi(t o) = 0 , i = 2,3 ..... L] ( i i 0 )
Pn(t,to) = E{[x(t)-In(t/t,to)][x(t)-In(t/t,to)]T/1(t,to); xi(to) , i=2,3 ..... L}(lll)
The conditional partial-state smoothed estimates and the corresponding error-
covariance matrices are given for i > 2 by:
Ri(to/t) = Pi(to/t)[Mi(t,to)+P~iRi] (112)
Pi(to/t) = [l+PiOi(t,to)]-iPi (113)
Furthermore, the auxiliary equations for i > 3 are evaluated in the following re-
cursive form:
Mi(t,t o) = [l-Oi.l(t,to)Pi_l(to/t)][Mi_l(t,to)-Oi_l(t,to)Ri_l] (114)
Oi(t,to) = Oi_l(t,to)[I-Pi_l(to/t)Oi_l(t,to)] (115)
~i(t,to) = ~i_l(t,to)[l-Pi_l(to/t)Oi_l(t,to)] (116)
For i=2, the auxiliary equations are given in the following explicit integral form:
278
ft M2(t,t o) = ~Z(O,to)HT(o)R-l(~)[z(o)-H(O)~n(O/O,to)]d~~ (117) t o
°2(it°) ° Ii ( l l8) o
F i n a l l y , t h e nomif ia l e s t i m a t e i n ( t / t , t o ) , i t s c o v a r i a n c e P n ( t , t o ) , and t h e nom-
i n a l filter transition matrix, now denoted ~2(t,to) , are given either by the forward
Eqs. (23-25), o~ by their backward versionq as given in Eqs. (28-30).
Remarks:
(i) Proof: The above partitioning was proposed by Lainiotis in 1975 [23], and
extensively studied by Lainiotis and his co-workers [22-25,47]. Detailed proofs of
the above m~iti-partitioning formulas, as well as of more general ones, can be found
in their publications [22-25,47].
(ii) Computational Advantages: The computational advantage of these formulas
can be briefly indicated by considering the following situation. Assume that the
initial state-vector has a-priori mean and covariance as follows:
i 0 . . .0 T T ~ , P(t ) = v 2 X(to) = [ml m2 . . . ]~ i o
...... V L
where the v. are symmetric matrices with dimensions consistent with the vectors m.. l 1
This initial condition may be partitioned as follows:
(119)
Rn(~ o) = 32= . . . iL = 0 ' 2 ' . (120)
Pn(t°) = [Vl0
0
, P2 =
0
0 0.. . 0]
v2 _I ' PL = 0 0
0 .... 0_
0 0 . . . 0
0
0
0 .... v L
(121)
As a result of the above partitioning, and using Corollary VII, the effects of
each block of the initial state can be separated from those of other blocks. Most
importantly, this partitioning yields a considerable computational benefit in the
matrix inversions involved in the computation af Pi(to/t), Eq. (113). For example, t~
compute P2(to/t) would ordinarily require the inversion of an nxn matrix, but when
279
P n
v 2 ,
and P2 are defined as above, the matrix to be inverted is only of the size of the
This results because which means substantial computational savings.
P2(to/t) = [l+PzO2(t,to)]-ip2 =
0 . . . 0 . . . . . . . . . . . 0 . . . . 0
0 [I+v202,22(t, to) ]-iv 2 0 .... 0
0 . . . 0
0 . . . 0 . . . . . . . . . . . 0 . . . . 0
(122)
where O2,22(t,t o) is the (2,2) block of the O2(t,to) matrix corresponding in size
and location to v 2. Moreover, when the m i and v i are scalars, then the matrix in-
version is totally eliminated, being replaced with division by scalars!
(iii) Fast Algorithms: We note that the multi-partitioning formulas are given
in terms of M2(t,to), O2(t,to), ~2(t,to), and the nominal estimate ~n(t/t,to), and
its nominal covariance Pn(t,to) , just as the partitioning formulas 6f Section III.
As such, the fast estimation algorithms of Section VI, as well as the efficient
realization of Section VII, and the numerical algorithms of Section VIII can be used
also for the computation af M2(.), 02(-) , 42(-), and of ~n(t/t,to), and Pn(t,to).
These'algorithms in conjunction with the specific multi-partitioning indicated in
the above remark (ii), yield computationally very attractive, numerically robust, and
fast numerical estimation algorithms.
Due to lack of space these fast numerical algorithms are not given here. They
can be found elsewhere [22-25,47].
X. OPTIMAL NONLINEAR ESTIMATION: PARTITIONING FORMULAS
A large class of nonlinear estimation problems for which a natural set of
partitio~ing parameters ~ exists is the class of adaptive estimation problems,
Lainiotis [1-6,16-17]. This estimation class is described by the following model:
dx(t) = F(t,f)x(t) + G(t,8)w(t) (123) dt
z(t) = H(t,0)x(t) + v(t) (124)
where x(t), and z(t) are the n and m-dimensional state and measurement processes,
respectively; {w(t)}, and {v(t)} are the plant and measurement noise random processes,
respectively, which when conditioned on e, are independent, zero-mean, white gaussian
random processes with covari&nces I, and R(t), respectively. The initial state-vector
x(t o) is independent of {w(t)} and {v(t)}, for t>t , when conditioned on 0, and has -- O
a 0-conditional a-priori density p(X(to)/0 ) which is gaussian with mean ~(to/0) , and
280
covariance P(tol8 ) .
The above model is specified up to a set of unknown parameters denoted by 8.
The parameter vector e, which if known completely specifies the model, is time-
invariant. Moreover, 8 is chosen at random by nature and/or an adversary from a
population with a-priori pdf P(8/to) -- P(8).
Given the measurements ~(t,to) - {z(o),oe(to,t)}, the optimal mse filtered
estimate i(t/t,to) of x(t) is desired under the above uncertain£y of the true value
of the defining model parameter vector 8.
Clearly the above estimation problem, a joint state and parameter estimation
problem, is a nonlinear one. Namely, augmenting the state-vector x(t) with 8, i.e.
Xa(t) - [xT(t):sT] T, yields the equivalent nonlinear model equations:
dXa(t) dt f[Xa(t)] + g[Xa(t)]w(t) (125)
z(t) = h[xa(t)] + v(t) (126)
where f[Xa(t)] - [xT(t)FT(t,8) i 0] T , g[Xa(t)] =- [GT(t,8) i 0] T ,
and h[x (t)] - H(t,e)x(t). a
Thus, it is seen that the above adaptive estimation problem constitutes a class
of nonlinear estimation problems. Moreover, if the usual, non-partitioning approach
is used for its solution [48-50], it yields an optimal nonlinear filter formula
which is useless both from a realization, as well as from an interpretation stand-
point. Specifically, in these previous approaches [48-50], the optimal nonlinear
filter is specified by a denumerable infinity of coupled nonlinear differential
equations, one for each moment of the non-gaussian a-posteriori pdf P[x(t)/%(t,t O)]
all of which contain the observations! The exact implementation of the nonlinear
filter in this form is impossible, and most importantly from a pmactical standpoint,
this form is not suitable even for deriving effective suboptimal estimators. How-
ever, by utilizing the partitioning approach, the adapative nonlinear estimator is
obtained in a closed form which is suitable for fast parallel-processing realization,
and whose approximate realizations are effective readily forthcoming, and easily
assessed, Lainiotis [3-5,8,16-17].
In this problem, one possible partitioning vector ~ to use is defined as
-= [xT(to ) ! 8T] T, where x(t o) = Xn(to)+Xr(to), and Xn(to), Xr(t o) are statistically
independent when condition&d on 8. For this choice of partitioning vector a, x (t) n o
~nd x (t) are gaussian when conditioned on 8, with a-priori means and covarlances r o
satisfying the following relations:
~(to/8) = ~n o(t/8) + ~r o(t/e) , P(to/8) = Pn(to/8) + Pr(to/8)
For the chosen partitioning scheme, the corresponding nonlinear estimation
281
partitioning formulas are given by the following Corollary of Theorem I:
Corollary VIII:
The optimal filtered estimate, and the corresponding error-covariance matrix are
given by:
~(t/t,to) = I~(t/t,to;~)p(8/t,to)de
P(t,to) = I{P(t,to/e)+[~(t/t,to)-i(t/t,to;e)][i(t/t,to)-i(t/t where i(t/t,to;e), and P(t,to/8) are the model-nonditional estimate, and the corres-
ponding covarianee. They are now given by the following explicit formulas:
(128)
,to;e)]T}p(ol%%)de(129)
R(t/t,to;8) = Xn(t/t,to;8 ) + ~n(t,to;O)Rr(to/t,e)
Pn(t,to/O) = Pn(t,to/O) + ~n(t,to;8)Pr'(to/t,O)~(t,to;O)
(130)
(131)
where the conditional smoothed estimate Rr(to/t, O) of the partial-initial state
Xr(to) , and its covariance Pr(to/t'0) are given by:
Ir(tolt,o) = Pr(tolt,e) [Mn(t,to/0) + PrlCtole)*r(to/8)] (132)
Pr(to/t,8) = [I + Pr(to)On(t,to/O)]-iPr(to/8) (133)
where, moreover, Mn(t,to/e), and On(t,to/e) are given by the set of forward diff.
Eqs. (21-25), or by the set of backward diff. Eqs. (26-30), or by a m~xed set of
forward and backward diff. eqflations constituting a set of general Chandrasekhar
algorithms, as in sections VI, and VII.
The a-posteriori pdf p(8/t,t o) is still given by an equation similar to Eq. (7),
namely by
L(t,to/O)
where the likelihood ratio is now given by the following simpler expression
L ( t , t o / e ) = exp{ ~ T ( ~ / g , t o ; O ) H T ( o , 8 ) R - l ( g ) z ( a ) d a - t
o
lit - - Jl H ( a , e ) R ( ~ / a , t o ; 8 ) tJ. 2 . d ~ } (135)
2 t ~-i(~) o
Remarks :
(i) Pro6f: The abo~re class of partitioning formulas were first given by
282
Lainiotis in 1969 [i], and studied extensively since then [1-20]. The version given
above in terms of linear partitioning filters was first presented in 1976 [16-17].
Detailed proofs of the above formulas may be found in [1,3,5,8].
(ii) Structure of Nonlinear Partitioning Filters: A feature of cardinal
importance of the above partitioning realization is that it is given in terms of
linear model-c6nditional ot model-matched filters. Namely, the optimal non-
linear filter constitutes an exact decomposition or partitioning of the nonlinear
filter for i(t/t,t o) into a set of much simpler linear elemental filters, namely,
the partitioning filters, which are, moreover, completely deeoupled from each other.
Namely, each elemental model-conditinnal filter is realized completely independently
from any other. Moreover, we note that the overall-optimal mse nonlinear adaptive
estimate i(t/t,to) is given as a weighted-sum of the model-conditional estimates
i(t/t,to;e). This constitutes a natural parallel realization form. In this sense
then, the optimal nonlinear mse estimate ~(t/t,to) has been expanded into a set of
elemental basis functions, namely, the linear mse model-conditional estimates
~(t/t,to;e).
Correspondingly, the partitiongd realization of the nonlinear adaptive estimator
constitutes a partitioning of the original nonlinear mse estimation into a set of
elemental linear mse estimation subproblems, which are, moreover, completely decoupled
from each other. In this sense, the nonlinear orthogonal projection necessary to
yield the nonlinear mse estimate ~(t/t,to) has been decomposed or partitioned into
the set of linear orthogonal projections yielding the model-conditional estimates
i(t/t,to;8).
(iii) Comparison of the Partitidnin$ Approach to Previous Approaches: In most
past investigations [48-50], of the optimal continuous nonlinear estimation problem
in general, or, in particular, of the optimal continuous adaptive estimation problem,
the results were given in terms of sde's for the mse estimate la(t/t,t o) of the (n+r)-
dimensional augumented state-veator x (t), where r is the dimensionality of 8. How- a ever, given the nonlinear natume of the models that result in non-gaussian and, most
importantly, nonreproducing densities, the solution of the problem via stochastic
differential equations requires the solution of a system of coupled nonlinear
stochastic differential equations all of which contain the observations. Moreover~
the system consists of a denumerable infinity of equations, one for each order moment
of the non~g~ussian a-posteriori pdf. As such, the solution via stochastic different-
ial equations is neither, in general, easily forthcoming, if at all, nor very illumi-
nating. Further, approximatelsolutions of these equations are ad hoc, and the effects
of the approximations made cannot be easily assessed. An example of such approxim-
ation is the elimination or truncation of the sde's corresponding to the higher order
moments, accomplished by setting these moments to zero for ~ii time under consider-
ation. These highly arbitrary approximations may yield absurd results such as negative
pdf's~ etc., as well as unacceptable estimator performance, e.g., divergence [51].
283
Thus the cardinal difference between the sde approach and the partitioning
approach is that while in the former one must solve a system consisting of a de-
numerable infinity of coupled stochastic nonlinear sde's, all of which contain the
observations, the latter is decomposed into simple completely decoupled building
blocks of identical structure - namely, the model-conditional linear filters, each
of which requires the solution~of the two ordinary differential equations (23) and
(24). The Riceati equation (24) is nonlinear but does not contain the observations.
Equation ~23) contains the observations but is linear. Moreover, there is one-way
coupling from the nonlinear equation, permitting off-line evaluation of the nonlinear
equations if desired. This result, namely, the solution of a system of coupled non-
linear stochastic differential equations in terms of a related system of de-
coupled ordinary differential equations, is of independent interest in the theory of
stochastic differential equations.
As seen from the above discussion, the partitioning realization of the optimal
adaptive estimator constitutes an exact decomposition or partitioning of a larger-scale
complex nonlinear system into a set of sm~ller size linear deeoupled systems.
Spedifically, the adaptive estimator which ~s given in the sde approach in terms of
the system of coupled nonlinear sde's for the a-posteriori moments of the (n+r)-di-
mensional augmented state-vector Xa(t) is given in the partitioning approach in terms
of the pairwise decoupled ordinary de, each pair corresponding to a model-conditional
Kalman-filter of dimensionality n. Moreover, the adaptive estimator is given exachly
in terms of these linear filters in a weighted-sum parallel realization form.
One also observes from Corollary I that partitioning has decomposed the nonlinear
adaptive estimator into a linear nonadaptive part consisting of the bank of linear
Kalman-Bucy filters, each filter matched to an admissible value of e (admissible in
the sense of p(e) being nonzero for this value of 8) and a nonlinear part, consisting
of the a-posteriori pdf's p(6/t,to), that incorporates the adaptive, learning, ot
system identifying nature of the adaptive estimator.
(iv) Joint Detection-Estimation Nature of the Partitioning Approach: Adaptive
estimation constitutes a joint estimation and multihypotheses detection problem,
Lainiotis [2-6,52]. It eonstitmtes estimation with respect to x(t) and detection or
hypothesis testing with respect to the decision as to which model (or equivalently
which value of 8) generated the observed data %(t,to). This can be readily seen by
noting that L(t,to/e) is the likelihood ratio for the following set of detection
problems (one for each value of ~):
H I : z(t) = H(t,e)x(t/8)+v(t) (136)
H 0 : z(t) = v(t) (137)
Each of the above detection problems tests the hypothesis that the data were generated
284
by the model indexed by the parameter-value 8 against the null hypothesis that the
data were generated by noise only. These detection problems may be given a more
explicit system identification or adaptive estimation form by rewriting (136) and
(137) as
z(t) = Ha(t,Oa)X(t/8 ) + v(t) (13B)
where Ha(t,0 a) - BH(t,0), x(t/0) is generated by ~123)and 8a is the augmented unknown
parameter vector defined as O -- [oT!B]T; the indicator variable B takes the value a 0 or i, depending on whether hypothesis H ° or HI, respectively, is true.
Moreover, it has been shown, Lainiotis [2-6,52"] that the quantity /L(t,to/O)p(O)dO
constitutes the likelihood ratio L(t,to) for the related compound detection problem
H 1 : z(t) = y(t) + v(t)
H : z(t) + v(t) o
where y(t) may be generated by any admissible model.
(139)
(140)
The compound detection problem
may be rewritten as an adaptive estimation or system identification problem:
z(t) = h Ix(t/0); t" O a] + v(t) (141) a
where
ha[X(t/8) ; t; 8a] ~ 8fH(t,~)x(t/~)6(~-Sa)d~ (142)
and x(t/~) is generated by (i) with model parameter value ~ and e E [eTis] T. Again, a
the indicator variable takes the value 0 or i according to hypothesis H ° or HI, re-
spectively. Thus one sees once more the intimate and integral connection between
estimation, detection, and system identification.
This global and unifying viewpoint of estimation, identifieatio~ and detection
as different manifestations of the same problem has been proposed and extensively
studied by Lainiotis [1-6,52]. Indeed, it has been shown that estimation and ident-
ification can be formulated as detection problems, and, in turn detection may he
formulated as estimation or identification. This unifying viewpoint brings to bear
on each problem the powerful methods and results obtained in all three.
(v) Fault-Detection and Correction Nature of NPF: In conclusion, the author
notes that an important feature of the nonlinear partitioning formula (NPF), from a
computational standpoint, is its natural decoupled parallel structure which lends it
admirably to parallel processing. With the advent of inexpensive microprocessors,
this parallel-processing capability of the partitioned realization is of substantial
285
practical importance. Moreover, the completely deeoupled parallel nature of the
partitioned realization renders it robust with respect to failures of any of the
parallel processors. This can be demonstrated by noting that the partitioned real-
ization has a natural "fault-detection" and "correction" mechanism built into its
weighted-sum parallel form. Specifically, if a model-conditional estimate i(t/t,to;@)
diverges, the corresponding conditional innovation ~(t,to;8) will be much larger,
comparatively speaking, than the other conditional innovations. This, in turn, will
make the correspondingp(e/t,to) tend to 0, which will essentially cutoff the diverging
conditional estimate from the weighted-sum.
(vi) Special Cases: The NPF given in this section yields interesting estimation
formulas for wide classes of special cases. One such important class is the class
of estimation problems with linear models, and gaussian noises, but with nongaussian
initial state, Lainiotis [5,8,16,53-54]. Moreover, the linear partitioning formulas
of Corollary I are an obvious special case of the NPF for completely known model,
namely for p(8) = ~(0-8o). In fact, it is interesting to note that the nonlinear
partitioning formulas were obtained first, Lainiotis [1,3,5,8], and then as a mere
special case of the NPF, namely as a trivial example, the LPF were obtained [5,8].
(vii) Computational Considerations: At this point the realization of the
adaptive estimator and the associated computational requirements should be discussed.
These depend integrally on the dimensionality of the state-vector and the Anknown
parameter-vector 9, and most importantly, on whether 8 is discrete or continuous.
There exist numerous important applications where 8 is naturally discrete. Such
situations arise in the case of joint detection and estimation when estimation is to
be performed under measurement uncertainty, or, more generally, in situations where
estimation is desired under uncertainty as to whether there exists one of several
possible failures, such as in power system estimation or in the case of failure of
sensors.
In many other important applications, 8 is a continuous random variable, resulting
in a continuous a-posteriori pdf p(@/t,to). This necessitates a nondenumerable
infinity of lineam filters for the exact realization of the optimal adaptive
estimator. Thus there is an apparent penalty to be paid in using the partitioning
approach, in that one has to solve a system consisting of a nondenumerable infinity
of pairwise decoupled ordinary differential equations (the Kalman-linear-filter equat-
ions) instead of a system of denumerable infinity of coupled nonlinear differential
equations as in the sde approach. However, this penalty is only apparent since it
is much easier to solve the former than the latter. Most importantly, as pointed
out earlier, approximations are much more effectively made in the partitioned reali-
zation than in the sde realization. This was demonstrated conclusively by Lainiotis
[3-6,16-17] who has shown that several approaches can be taken to alleviate the com-
putational difficulties, resulting in approximate adaptive estimat6rs of varying
efficacy and complexity.
286
We finally note, that in the practical implementation of the NPF, we can utilize
to great advantage, the fast, and efficient linear partitioning algorithms of the
previous sections in realizing the model-conditional filters.
XI. CONCLUSIONS
Partitioning, the multi-model framework for estimation, and control introduced
by Lainioflis [1-26], has been shown to constitute the unifying and powerful frame-
work for optimal estimation, for linear as well as nonlinear problems. Partitioning
is the natural setting for estimation problems, since it decomposes the original
estimation problem into a set of estimation of considerably reduced complexity, the
optimal or suboptimal estimators of which are far easier to derive and, most import-
antly, they are far easier to implement.
The partitioning approach consists of the selection of a "parameter" vector
of the model, each possible value of which specifies a particular realization of
the model. As such, the admissible ~alues of ~ correspond to a collection of
possible models for the estimation problem at hand. Moreover, following the Bayesian
viewpoint, the choice of a particular value of e, particular model realization, is
considered random with a-priori probahility density that is consistent with the
statistics of the original complex model. Furthermore, through conditioning on this
pivotal parameter vector ~ the original complex, large scale, or "ill-condition"
estimation problem is decomposed into a set of estimation subproblems, each one corres-
ponding to an admissible parameter-conditional model. This gives ti~e to the multi-
model name for the partitioning approach. The above estimation subproblems are less
complex, e.g. linear instead of nonlinear, smaller-scale, namely of smaller state-
vector dimensionality, and better conditioned than the original estimation problem.
Most importantly, these estimation subproblems are completely decoupled from each
other, i.e. the parameter conditional optimal estimate of each is obtained completely
independently from the other eondition~l estimates. Furthermore, the desired overall-
optimal estimate is given as the aggregation of these conditional estimates, the
aggregation given in a simple, weighted-sum, deeoupled, parallel-realization form.
There exists large classes of practical estimation problems, hoth linear as
well as nonlinear, where a natural set of partitioning parameters ~ exists, such
that use of these parameters in partitioning leads to optimal estimators of much
less complexity and computational requirements, and whose practical implementation
is thus far simpler to realize. Several of these classes of estimation problems,
and the associated partitioning estimation formulas were presented in the previous
sections. It was shown there, that using the partitioning approach, estimation
problems are treated from a global viewpoint that readily yields and unfies all
previous estimation results, and most importantly, yields fundamentally new classes
of optimal and suboptimal estimation formulas in a naturally decoupled, parallel-
implementa£ion structure. The partitioning estimation formulas are of considerable
287
theoretical significance. They provide deep insight into the nature of estimation
problems, and the structure of the resulting es£imators. Most importantly from a
practical standpoint, the partitioning formulas yield realizations of optimal and
suboptimal estimators, both filters and smoothers, that have significantly reduced
complexity, that are computationally attractive, and numerically robust, and whose
practical implementation may be accomplished in a pipeline or parallel-processing
mode. Indeed, the remarkably flexible structure of the partitioning algorithms
affords a wide variety of serial-parallel processing combinations that can meet the
computational and storage constraints of large classes of practical estimation
applications, especially real-time ones.
We note that completely analogous results have also been obtained for discrete
linear and nonlinear estimation problems, [1,3,9,10,12-13,17,21,23,25,31-32], how-
ever due to space limitations these results are not included in this paper. They a
can Be found in the above references. Moreover, the partitioning approach has been
applied equally successfully to estimation problems with models other than those
of Eqs. (1-2), and (i0-ii). For example, Asher and Lainiotis [20] ha~e applied it
to the estimation of doubly stochastic Poisson processes, Sawaragi etal [37] have
applied it to state-estimation for continuous-time systems with randomly interrupted
observations, and Haddad [55]~ and Chang [56] have applied it to state-estimation in
switching enviroments.
REFERENCES
[i] D. G. Lainiotis, "Optimal Adaptive Estimation: Structure and Parameter Adapt- ation", in Proc. IEEE Symposium Adaptive Processes, Nov. 1969; and Electronics Res. Center, University of Texas, Austin, TX, Tech. Rep. no. 74, Sept. 1969.
[2] D. G. Laini~tis, "Sequential Structure and Parameter Adaptive Pattern Recognitio$ Part I: Supervised Learning", IEEE Trans. Inform. Theory, Vol. IT-16,Sept. 1970.
[3] D. G. Lainiotis, "0ptimalAdaptive Estimation: Structure and Parameter Adapt- ation", IEEE Trans. Automat. Contr., Vol. AC-16, pp 160-170, Apr. 1971.
[4] D. G. Lainiotis, "Joint Detection, Estimation, and System Identification", Inform. Control J., Vol. 19, no. 8, pp. 75-92, Aug. 1971.
[5] D. G. Lainiotis, "Optimal Nonlinear Estimation", Int. J. Control, Vol. 14, no. 6, pp. 1137-1148, 1971.
[6] D. G. Lainiotis, "Adaptive Pattern Recognition: A State Variable Approach", in Advances in Pattern Recognition, M. Watanabe, Ed. New York: Academic Press, May 1972.
[7] D. G. Lainiotis~ "Optimal Linear Smoothing: Continuous Data Case", Int. J. Control, Vol. 17, no. 5, pp 921-930, 1973.
[8] D. G. Lainiotis, "Partitioned Estimation Algorithms, I: Nonlinear Estimation", J. Inform. Sciences, Vol. 7, no. 3, pp. 203-255, 1974.
[9] D. G. Lainiotis, "Partitioned Estimation Algorithms II: Linear Estimation",
288
[25]
[26]
[27]
J. Inform. Sciences, Vol. 7, no. 3, pp. 317-340, July 1974.
[i0] D. G. Laini~tis, "Partitioned Linear Estimation Algorithms: Discrete Case", IEEE Trans. Automat. Contr., Vol. AC-20, pp. 255-257, June, 1975.
[ii] D. G. LainiOtis, and K° S. Govindaraj, "A Unifying Approach to Linear Estimation Via the Partitioned Algorithms, I: Continuous Models", Proe. 1975 IEEE Decision and Control Conf., Dec. 1975.
[12} D. G. Lainiotis, and K. S. Govindaraj, "A Unifying Approach to Linear EStimation Via the Partitioned Algorithms, II: Discrete Models", in Proe. 1975 IEEE D~cision and Control Conf., Dec. 1975.
[13] D. G. EainiOtis, "Discrete Riecati Equation Solutions: Partitioned AiKorithms", IEEE Trans. Automat. Contr., Vol. AC-20, pp 555-556, Aug. 1975.
[14] D. G. Lainiotis, and J. G. Deshpande, "Parameter Estimation Using Splines", J. Inform. Sciences, Vol. 7, no. 3, pp 101-125, 1974.
[15] D. G. Lainiotis, "A Unifying Framework for Linear Estimation: Generalized Partitioned Algorithms", J. Inform. Sciences, Vol. i0, no. 3, pp. 243-278, Apr. 1976.
[16] D. G. Lainiotis, "Partitioning: A Unifying Framework for Adaptive Systems, I: Estimation" Proc. of IEEE, Vol. 64, no. 8~ pp. 1126-1143, 1976.
[17] D. G. Lainiotis, "Partitioning: A Unifying Framework for Adaptive Systems, II: Control" Proe. of IEEE, Vol. 64, no. 8, pp 1182-1198, Aug. 1976.
[18] D. G. Laini~tis, "Partitioned Riceati Solutions and Integration-Free Doubling Algorithms" IEEE Trans. on Automat. Contr., Vol. AC-21, no. 5, pp 677-688, Oct. 1976.
[19] D. G. Lainiotis, "Generalized Chandrasekhar Algorithms: Time Varying Models", IEEE Trans. on Automat. Contr., Vol. AC-21, no. 5, pp 728-732, Oct. 1976.
[20] R. B. Asher and D. G. Lainiotis, "Adaptive Estimation of Doubly Stochastic Poisson Processes with Applications to Adaptive Optics", J. Inform. Sciences, Vol. 12, Oct. 1977.
[21] K. S. Govindaraj, and D. G. Lainiotis, "A Unifying Framework for Discrete Linear Estimation: Generalized Partitioned Algorithms", Intern. J. of Contr., 1978 (to appear).
[22] D. G. Laini~ais, and D. Andrisani, "Multi-Partitioned Solutions for State and Parameter Estimation: EContinuous Systems", Proc. of the 1978 Joint Automat. Control Conf., ISA, Pittsburgh, PA., Oct. 1978.
[23] D. G. LainiOtis, "Multi-Partitioning Linear Estimation Formulas and Fast Algorithms", Automatica, submitted for publication~ 1978.
[24] D. G. Lainiotis, and D. Andrisani, "Multi-Partltloning Linear Estimation Algorithms: Continuous Systems"~ IEEE Trans. on Automat. Contr., Submitted for publication, 1978.
D. G. Lainiotis, and D% Andrisani~ "Multi-Partitioning Linear Estimation Algorithms: Discrete Systems", Automatica, Submitted for publication, 1978.
D. G. Lainiotis, "Partitioning Filters", J. of Inform. Scienees~ Jan. 1979.
D. G. Laiuiotis,and K. S. Govindaraj, "Partitioned Riccati Equation Solution algorithms: Computer Simulation", Proe. 1975 Pittsburgh Conf. Modeling and
289
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[40]
[4i]
[42]
[43]
[44]
[45]
Simulation, Apr. 1975.
D. G. Lainiotis, "Fast Riccati Equation Solutions: Partitioned Algorithms", Proc. 1975 Milwaukee Symp. Automatic Computation and Control, Apr. 1975.
D. G. Lainiotis, "Fast Riccati Equation Solutions: Partitioned Algorithm", J. Computers and Electrical Engineering, Nov. 1975.
D. G. Lainiotis, and K. S. GovindaraJ, and D. Andrisani~ "Nonsymmetric Riceati Equations: Partitioned Algorithms", J. of Computers and Electrical Engineering, Vol. 5, pp 109-122, Oct. 1978.
K. S. Govindaraj, and D. G. Lainiotis, "Partitioned Algorithms for Estimation and Control", Tech. Rep. 1978-2, System Res. Center, State University of New York, Amherst, NY, Nov. 1978
D° G. Lainiotis, and K. S. Govindaraj, "Discrete Riccati Equation Solutions: Generalized Parti£ioned Algorithms", J. Inform. Sciences, Vol. 15, no. 3, pp. 169-185, Nov. 1978.
D. G. Lainiatis, "Partitioned Riccati Algorithms", Proc. 1975 IEEE Decision and Control Conf., Dec. 1975.
D. G. Lainiotis, "Partitioned Filters", Proc. of the Chapman Conf. on the Applications of the Kalman Filter to Hydrology, Hydraulics, and Water Resources, American Geophysical Union, May 1978
B. J. Eurlich, and D. Andrisani,and D. G. Laini~tis, "New Identification Algorithms and their Relationships to Maximum-Likelihood Methods: The Partition- ed Approach", Proc. of the 1978 Joint Automat. Contr. Conf., ISA, Pittsburgh, PA, Oct. 1978.
D. G. LainiOtis, "Estimation: A Brief~Survey '', J. Inform. Sciences, Vol. 7, no. 3, pp. 197-202, 1974.
Y. Sawaragi, and T. Katayama, and S. Fujishige, "State EStimation for Continuous- Time Systems with Interrupted Observations", IEEE Trans. on Automat. Contr., AC-19, no. 4, Aug. 1974.
R. E. Kalman, and R. S. Bucy, "New Results in Linear Filtering and Prediction Theory", Trans. ASME J. Basic Engineering, Series D, Vol. 83, pp 95-107, Dec. 1961.
J. S. Meditch "Optimal Fixed-Point Con£inuous Linear Smoothing" in Proc. 1967 Joint Automat. Contr. Conf., pp 249-257.
T. Kailath and P. Frost, "An Innovations Approach to Least-Squares Estimation- Part II: Linear Smoothing in additive white Noise", IEEE Trans. Automat. Contr., Vo. AC-13, pp 655-660, 1968.
L. Ljung, and T. Kailath, "Efficient Change of Initial Conditions, Dual Chandrasekhar Equations, and Some Applications", IEEE Trans. on Automat. Contr. Vol. AC-22, no. 3, pp 443-446, June 1977.
D. Q. Mayne, "A Solution of the Smoothing Problem for Linear Dynamic Systems", Automatica, Vol. 4, pp 73-92, 1966.
D. G. Lainiotis, "General Backwards Markov Models", IEEE Trans. on Automatic Control, Vol. AC-21, no. 4, pp 595-598, Aug. 1976.
T. Kailath, "Some New Algorihhms for Recursive Estimation in Constant Systems", IEEE Trans. Inform. Theory, Vol. IT-19, pp 750-760, Nov. 1973.
290
[46]
[47]
[48]
[49]
[5O]
[51]
[52]
[53]
[54]
[55]
[56]
A. Lindquist, "Optimal Filtering of Continuous-Time S~ationary Processes by Means of the Backward Innovation Process", SIAM J. Control, Vol. 12, no.4, Nov. 1974.
L. Segal,aand D. G. Lainiotis, "Partitioning Estimation Algorithms and their Applications to Economic Forecasting" Tech. Rep. 1978-5, Systems Res. Center, Staee University of New York a~ Buffalo, Amherst, NY% Dec. 1978.
R. L. Stratonovieh, "On the Theory of Optimal Nonlinear Filtration of Random Functions", Theory Prob. App., Vol. 4, pp. 223-225, 1959.
W. M. Wonham, "Some Applications of Stochastic D&fferential Equations to Optimal Nonlinear Filtering", SIAM J. Control, Vol. 2, pp 347-369, 1965.
H. J. Kushner, "Dynamical Equations for Optimal Nonlinear Filtering", J. Differential Equations, Vol. 3, no. 2, pp. 179-190, Apr. 1967.
A. H. Jazwinski, Stochastic Processes and Filtering Theory. New York: Academic Press, 1970.
D. G. Lainiotis, "On a General Relationship Between Estimation, Detection, and the Bhattacharyya Coefficient", !EEE Trans. Infor~ Theory, Vol. IT-15, July 1969,
D. G. Lainiotis, and S. K. Park, and R. Krishnaiah, "Optimal State-Vector Estimation for Non-Gaussian Initial State-Vector", IEEE Trans. Automat. Contr., Vol. AC-16, pp 197-198, Apr. 1971.
S. K. Park, and D. G. Lainiotis, "Monte-Carlo Study of the Optimal Nonlinear Estimator: Linear Systems with Non-Gaussian Initial State", Int. J. Control, Vol. 16, no. 6, pp 1029-1040, 1972.
R. A. Padilla, and A. H. Haddad, "On the Estimation of Uncertain Signals Using an Estimation-Detection Scheme, "IEEE Trans. on Automat. Contr., AC-21, no. 4, pp 509-512, Aug. 1976.
C. B. Chang, and M. Athans, "State Estimation for Discrete Systems with Nwitching Parameters", IEEE Trans. on Aerospace and Electronic Systems, Vol. AES-14, no. 3, pp 418-424, May 1978.
WATER WAVES AND PROBLEMS OF
INFINITE TIME CONTRO~ ~
David Lo Russell and Russell M. Reid
University of Wisconsin, Madison, WI 53706, U.S.A.
i. Background and Problem Formulation
Our objective in this brief report is to study the control of small amplitude waves
on the surface of an incompressible fluid and to show that the resulting control problem,
which requ i res a s e m i - i n f i n i t e t ime in t e rva l for i t s so lu t ion , in t roduces phenomena
which mot ivate the deve lopment of a gene ra l approach to con t ro l l ab i l i t y s i gn i f i can t ly
different from tha t commonly used for sys tems wi th a f in i t e cont ro l t ime . The r eade r
i s referred to [1] , [ 2 ] , [ 3 ] for background on wa te r w a v e s a n d t o [4] f o r a n abs t r ac t
formulat ion of cont ro l problems in g e n e r a l .
For the p resen t a r t ic le we r e s t r i c t a t t en t ion to two d imens iona l i n c o m p r e s s i b l e
and i r ro ta t iona l motion in a region ~ = ~ ( x ' t ) a ,
~= [(x,z)]0<x<~, z<z0]. The _ _ ~ _ _ ~ . . . .
boundary z = z 0 is the equilibrium -'~ ~ = ~0 -~" _ i/
surface level of the body of liquid while
x= 0, x= ~ are vertical containing walls, o ~{
For the controlled system the wall x = ~ , ~2
may be replaced by a movable wall. (See x ~ ~ u
Fig. I). ~ l Fig. 1 I!
We consider the uncontrolled system first. Let z = ~ (x, t) denote the surface
of the body of fluid at time t. Since we are assuming the motion to be incompress-
ible and irrotafional, a velocity potential %0(x, z, t) may be defined in ~ whose
gradient with respect to x, z, V%0(x, z, t), gives the fluid velocity at (x, z, t).
With ~- 8Z/sx Z + 8Z/sz z we have
A~= 0 in ~2, llm %0(x, z,t)= 0, (I.i) Z ~ --CO
a~(0 z,t)= a--~(~,z,t)= 0 -~< z<z 0. (l°Z) 8x " 8x "
The dynarrdcs are provlded by the influence of gravity which, v~th appropr/ate physical
units, g ive s the equa t ion
~ (x, z 0 t) = -C(x,t). (1.3) 8t
Here, and in the associated kinematic equation
8--9- (x, Zo, t) = ~)B---~-t (x, t) ( 1 . 4 ) 8 z
the simplifying assumptions of the linear surface wave theory are apparent: for small
displacements and velocities ~ , 8 ~/St the relationships between the ,,free surface,.
~Supported in part by the Off ice of Nava l Research under Cont rac t NR 041-404.
292
z = ~ ( X , t ) a n d t h e m o t i o n i n 9 , d e s c A b e d b y %~(x, z , t ) , a r e e n f o r c e d a t
t h e n o m i n a l s u r f a c e l e v e l z = z 0 i n s t e a d of a t t h e a c t u a l s u r f a c e l e v e l z = ~ ( x , t ) .
[ S e e [1 ] , [ Z ] f o r more d i s c u s s i o n of t h i s . )
Our system may be presented in first order form (with ~ = a</St ) as
where A is a linear operator def.tned as follows. Given ~ (x), 0 < x < = , let
T (x, z ) be the hamonic function in f~ solving the mixed Dirichlet-Neumann problem
aT (0, z) = aT T(x' z0)= ~(x)' "~x -~x (~,z) = 0, -~o < z < z0~ (1.6)
( T corresponds with llm T (x, z) = 0. (T corresponds to -8%0/8t above). Then Z~-C0
8~ (x, z0), 0 < x < ~r (LT) (AC)(x) = a--~
The system (1.5) is then just are-expressionof (i. 1) - (1.4). For general regions
the operator A is somewhat complicated ( see [ 3 ] ) but for the special region
shown in Flg. I we have a rather simple description of if.
Theorem i. The operator A is an unbounded self adlolnt o~erator o_nn L2[ 0, ~ ]
with domain ~T
~(A) : [~ e Hl(0,~)lJn-- ~(x)dx = 0]. (1.8) v
Mgre speciflcally~ A is the positive square root of the Sturm-Liouvllle 0Pergtor
(~C)(x) = -~"(x) (1.9) with domain
S0 ~ C' C' ~(L)= [Ce ~z(0,~) l C(x)dx= o, ( o ) = (~)= 0].
Thus ~ (cf. (1.5)) generates a strongly continuous group o__f bounded operators in
th___e space
H v - [(C,q)e H~(o,~) eLZ(o,~)l£ x)dx: q(x)dx= 0]. (lol0)
Sketch of Proof. Given ~ c ~(A), we first form A(~ by constructing T(x, z),
as described above, and then letting A be given by (I. 7 ). Then
(AZC)(x) = (A ~ (o, z0) ) (x). (Ln)
To compute the rlght hand side of (I. I0 ) we require a harmonic function @ (x, z )
in ~ with aT (x, a@ 8 0 e(x, z0)= %~ %)' Tx (0, z) = ~-x(~,z)= 0, llm @(x,z)= 0.
For our very special geometric configuration it may be seen that we have @ (x, z ) =
8~[ (x, z), (x z) c n Then ~z ' "
ae (x,z 0) = --az~ (x,z 0) = -~"(x), , ~ ~,Azo,,x, = ~ az z
the last equation following from AT = 0 in e together with the first equation in
(1.6). The posltlvlty follows from
: = ) ~ (x, =o)~ :f£11vT(x, ~)II z dxd~.
293
' @~ ( o , 8~ ( 0 , The cond i t i ons ( ' ( 0 ) - - ~ ( = ) = 0 fo l low from - ~ x z ) = -~-x = ) t o g e t h e r
with the fact that ~e H5/Z(f~) when ~¢ ~(L). Theformof ~)(A) Isan
interpolation result and the last statement comes from general results in [5] , [6] ,e.g.
We see that A has the orthonormal basis of elgenfunctions ~0k(X ) =
eosk x , k = i, Z, 3 .... for the space ( with induced inner product ) Tr
z(0,~)- {~ LZ(0,~)If0 ~(x)dx = o] I2 0
and elgenvalues k k = k, k = i, 2, 3 ..... The system (1.5) has the form of a
"linear oscillatoW in L 2(0, ~T):
"~ + A ~ : 0 ,
with associated (mathematical) energy form
v(¢ ,~ ) : ~fo~(x))Zdx + ~-ffllv~'(x,z)ll zdxdz =~(n,n)l +}(g,Ag) , (l. lz) fl
a form equivalent to the squared norm in H v (cf. (I. i0 ) ) . The ,'physical" energy
f o r m i s (c f . (1 .1) , (1. Z ) )
1 ('= 1 1 E(g,~)=~0(¢(x))Zdx+~ffllv~(x,z)[IZdxdz=l(g,~)+~(n,A-l~]) (i. 13) n
the last identity being derivable with ease from the relationships (I. 3 ), (1.4)
between %0 and ~ . Thls form Is equivalent to the norm in
H E ~ (~,n) ~ T.Zo(o,~) • ~oVZ(o,~) (1.14) ~V (o,~)If ¢(~)~-- o] relativeto (H(~I/Z(O, ) dualto HI0/B(0,~) = {[~:e Z
L% ( 0, ~ ) ) and if A is defined ( via the theory of dlst~Ibutions ) as an unbounded
operatoron HoI/Z(0,=) withdomaln HI0/z{0,1T), then ~ (cf. (1.5))
generates a group of bounded operators In II E . It may be seen that both of the forms
V,E are conserved for solutions of (1.5) in H V while the form E is conserved
for solutions of (1.5) in H E .
We proceed now to consider controlled systems
for ( b , d ) e H V (o r H E ) and s c a l a r con t ro l u . Wi th the e tgen func t i on r e p r e s e n -
t a t ions
~(x, t) = ~k(t)~0k(X), ~(x, t) = ~ ~k(t) ~k(X), k=l k=l
co
b(x) = ~ b k %0k(X ) ,
k=l
we have the equivalent systems
The transformation
d(x) = ~ a k %(x), k=l
.ik~\Yk] = ~F~ .IAg/\y/
(L~6)
(L 17 )
294
carries the systems
b k id k
t h e form most c o n v e n i e n t mathematically.
forms V and E are
(1. 16 ) i n to
(,)u<,, - 7\Yk) +
b k id k k = l, 2, 3 .... ,
, ~k 4g q~ '
In terms of the coordinates
(1.18)
~k' ~]k the
co co ~ co
v ( ¢ , ~) = v ( ¢ , q ) = ~ F Ckt z, z c ¢ , n ) - - - 1 0-.191 k = l 1
and in terms of the Xk' Yk coordinates we have ca
1 1 V(~,~)= ~ ~ k(ixklZ+IyklZ), E(~,q) =-~I ([XklZ+lykIZ). (l. Z0)
k=l k=l The following result can be obtained more or less by inspection from the form
{ i. 18) and the proof is omitted. /x /k
Proposition Z. Le___t ( ~ (t), ~] (t)) be th_ee soluti__..__on _of ( I. 15 ) con-es})onding to the
initial state (0, 0) and a scalar control function u £ L I(0, ~). If (b,d} • H v
the_,..~n (~(t),~(t)) • H V (cf. (I. i0)) fo.._ral_.!l t- > 0 and, lettlng the group
generated by G b__ee designated by 8{t ) :
lim 8(t) -I {t) exists in H V . t - ~ kn (t)/
If {b, d) is an element of H E then (((t),Tl(t)} • H E fo___r t >- 0 and
llm S{t) "I ({~{t)~ existsln H E . t - o t ,64
We remark that we have already indicated that 8(t ) may be regarded as a
group of bounded linear operators on either H V or H E .
For arbitrary initial states ( ~ 0' q 0 ) (I. 15 ) has the solution
(t)/-
corresponding to the control u • L 1 [ 0, ~o ) referred to in Proposition Z. Since
8{t), g(t)-l= S(-t) are, In fact, unlformlyboundedlnelther H v or H E , we
have
Proposition 3. The contro] u e L I{0, ~) steers {~(t),7]{t)) from ~0' q0 )
to { 0, 0 ) during ( 0, ~ ) ~ust in ca se
,,m ¢o) = - , (1o Z l ) t - ~ \n (t)/ q o
which, in terms of the Xk' Yk coordinates, is !ust ~ ~ 1
7kf e-ikesu(s)ds : -Xok. 6k/ elk2Su(s)ds = -Yok" (I. ZZ) -0 ~o
295
From here on we assume the basic approximate controllability condition
IbkI2+ Idkl z i 0 (hence Yk,6k i 0), k= 1,2, 3 ..... (1.23)
It may be of interest to exhibit a mode of control for which our assumptions are satls-
fled. Referring to Fig. I, we suppose that the right hand end of our "tank,' can be
moved with a single degree of freedom, so that the equation of that boundary becomes,
for an appropriate function B ( z ), z 0
x = ~ + B(z)o~(t), / B(z)dz = 0 (1.24) _=O
{the last to give conservation of volume; this condition is easlly dispensed wlth).
The lineadzed boundary conditions, approximately valid for small 0~ {t ), ~ (t)
are (cf. (i.z))
~ (~,z,t) = B(z)~(t)- B(z)u(t) (i°25) 8x
If we take the time derivative of co (t) as our control. It may be verified that this
situation corresponds to the case where {cf. {i. 16 )) d k = 0 and
JT z0 ek( z-z0 .... b k = (-l)k~ ) B(z)dz, k= I, 2, 3,
For example, with B(z)= (I+ (z-z0))eZ-Z0 , (i. 24) is satisfied and we have
~2 k 1 1 b k =4T (-l) = ~ k= 1,2,3, ;l)z . . . . .
acase for which (b, d) = (b, 0) E H E , but ~ l-I v .
Z. C o n t r o l l a b i l i t y Resu l t s
The "nu l l cont ro l tabi l t ty problem" for the s y s t e m (1 .15) on a f i n l t e l n t e r v a l
[0, T] c o n s t s t s in t ry ing to f ind, f o r a g t v e n ~0 = ( ~ 0 ' ~ ] 0 ) e HV ( o r HE, as
the c a s e may be ), a c o n t r o l u e LZ[0, T] such tha t the so lu t lon ~ ( t ) of (1 .15)
with t he g iven i n i t i a l s t a t e and con t ro l l i e s in H V ( o r H E) for 0 < t < T and
s a t i s f i e s ~ (T ) = 0 . It i s in t h i s f ramework, for example , t ha t cont ro l of t he wave
equa t ton i s d i s c u s s e d in [ 7 ] , [ 8 ] and [ 9 ] . B u t t t t u r n s out t h a t i t t s u s e l e s s to
pose t h i s f in i te t ime cont ro l problem for ( 1 . 1 5 ) . The e q u i v a l e n t moment problem in
LZ[0, T] i s ( i n te rms of the x , y c o o r d i n a t e s )
T " is Xo'k~k ~ Tel ~Ck, Ik~Su(s)ds Yo, k c i, k I,Z, 3, f0 u (s,ds - - : D O a
(cf. (1.22)). (Z. 1)
But it is known {see [i0] , Chapters I, If) that this moment problem is not generally
solvable for any T > 0. ~4Ve are led therefore to study control on the half infinite
interval [ 0, ~ ). The most immediate re-formulatlon of the null controllability
problem is the following: to find a control u e L z [ 0, ~ ) such that the ~olutlon
~{t) descrlbed earlier has the property that ~(t) ¢ H V (or HE) for 0<t< ~
and
llm II ~ ( t )II HV HE ) t-~ (or = 0. (Z.Z)
296
In t h e s e q u e l we w i l l be a b l e t o o b t a i n a n a f f i r m a t i v e r e s o l u t i o n of t h i s c o n t r o l l a b i l i t y
p r o b l e m but , in t h e p r o c e s s , we w i l l s e e t h a t i t i s no t r e a l l y t h e p r o b l e m w h i c h we
w i s h t o s o l v e and a r e f o r m u l a t i o n w i l l b e a t t e m p t e d - fo r w h i c h , h o w e v e r , o u r t h e o r y
i s q u i t e i n c o m p l e t e .
In v i e w of P r o p o s i t i o n 3, o n e m a t t e r w h i c h must b e a t t e n d e d to i s t h e e x i s t e n c e
o f t he l i m i t 8 ( t ) - l ~ ( t ) i n H V ( o r H E) . The e x i s t e n c e of t h i s l i m i t c a n n o t ,
in g e n e r a l , be e s t a b l i s h e d fo r u e LZ[ 0, ~ ) , bu t c a n be e s t a b l i s h e d , a s we h a v e
s t a t e d , fo r u e L 1 [ 0, ~ ) . Th i s , c o m b i n e d wi th ou r f o r m u l a t i o n o f t h e nu l l c o n t r o l -
l a b i l i t y p rob lem, p rompts t h e s e l e c t i o n o f c o n t r o l s u e L 1 [ 0, o0) N L z [ 0, ~ ) a s i n
t h e n e x t t h e o r e m .
T h e o r e m 4. Le t ~0 = { C0' n0 ) h a v e t h e form
q0 = \fl~k/' a, fl scalar, (Z. 3)
for some fixed k, where ~°k = Jz coskx is the eigenfunction of A
corresponding to the eigenvalue k. Then there is a control function
u e L I[0, ~) N L Z[0, ~) such that the solution ~ (t) of (1.15) with the initial
state (Z.3) and indicated control lies in H v (or H E ) and has the property
(2. z) in H v (or HE), provided (b,d) e HV(or HE).
Remark. A comparable result for initial states
= , K f/nite, nO k= 1 k ~k
clearly follows immediately - a form of approximate null controllability sometimes
called spectral controllability.
The proof of Theorem 4 will be developed in two lemmas.
Lemma 5. The function of the complex variable z :
G(~) : r((z+l) z) F (zZ) (eZ+iF (z+l)14 (2.4)
is analytic f_oor Re ( z ) > -I and uniformly bounded for Re ( z ) -> 0 . We have the
asvmptctic formulae (for any 6 > 0 )
1 G(z) ~ Izl -~' lar°zl < Z -a (2.5)
4wZeZ '
i G(i~) ~ z=i~z-, l~ l -~ , -z+5 <[a~(~)l_~0,-=_<la~(~)[< -Z-a, (2.6)
4~Xe x (.t e ) !
The function G(z) has simple zeroes at the points z = _+ik 2, k = i, Z, 3 .....
and the values G (i~) and derivatives G j (i~), j = I, Z, 3 .... are all
asymptotic, as c~ -- m through real values, to the corresponding values or deriv-
atives of 1 (I - e 2=iwz) .
4~ 2 e 2
297
Proof. The ana ly t lc l ty for Re ( z ) > -1 fol lows immediately from elementary
properties of the r function, as does the ex i s t ence of the indica ted zeroes . (There
are a l so multiple ze roes at z = O, -1, -Z . . . . ) . The other proper t ies descr ibed
above will be obtained, in two s teps , from Stifling' s formula
r ( z ) : e-ZzZ-½(z~) -~ + (el. i n ] ),
which holds uniformly as [z[ -- ~ in any sector ]arg(z)[ _< ~ - e, @ > 0.
Using this in (2.4) we have
G (z) = (2~) Zl e-ZZ-I l )2zZ-i --Tz (l+ F (l+ ~( i) ).
But
1"2Z2-1 = ( z ) ( e - 2 Z [ i + ~ - ) ] 3 , e-ZZ-l(l+gJ ~ - 1 e-1 1 z 2
The principal va lue of the logari thm of the term in b racke t s is e a s i l y seen to be
l + a s l l-° and it f o l l o w s thot 1 1
G(z) = (1+ e(~)) I z l - = 4~2e 2 ' ,
provided tha t l a r g ( z 2 ) ] , ] a r g ( z + l ) z ] remain _< l r - 6 ) , 6 ) > 0 ,
larg(z)l < Z.. e which g ives (2o5) Z 2 ' The second step c o n s i s t s in using the change of var iable z = i co
with the wel l known formula (c f . i l l ] ) 1T
r ( z ) = F ( 1 - z ) s i n ~ z "
The resul t i s
G(ico) = r(co2 ' ( sin ~ c02 ) ~oZr (~2 .2 ico) (ei°~ +1 r (ico))4 \sln w (co2 _ 21co _ I) "
Using Stirling' s formula again, with some e lementary operat ions , we have r(co 2) e - ° ( l + e(1/lco l))
cozy{ co2 - Zico) ( e Ico +I
Since log (coZ ( ( ico)ico-½ )4
lo9 ( 0~ 2 - Zico )-2ico
2 sin~ co
r(ico)) 4 4=zco2 ({ico)Ico-½)4(co 2 _ 21co) -21co
) = -21ro~ - lri + 4ico log co ,
sinn( 2 _2i~ -I)
we have
{2.7)
i.e., for
toge ther
4 ~ 2 e Z _ Z j ' ( _ ~ - - 5 . ( 2 . 8 )
e-2i~co2 Since e -4~rco -- O, lco[-- ~ , " ~ + 5 < arg co ~ O, we have the asympto t ic Tf result ( Z . 6 ) for this argument r ange . The resul t for - ~ ~ [arg ¢0[ < - ~ - - 5 is
298
obtained similarly. That the result applies to derivatives as well as function values
follows from the f~ct that all derivatives of the factor represented in ( Z. 8 ) by -4~ -Zi~ -I (I- e e ) (I+ @(i/l,~I) ) tend to zero as 00- ~ throughposltive
real value s. The boundedness of G(z) for Re(z) >- 0 follows from (2.5)
and (Z.6).
Theorem 4 follows immediately, in view of the equivalence of the moment
problem ( I. ZZ ) and the control problem, when we prove
Lemma 6. For k i i, 2, 3 ..... let l i k 2 G ( z ) - i k ~ G ( z )
1 GI 1 Gk(Z)= z(z-ik )G'{ik ½) ' G-k(Z) = z(z+ik ~) (-ik ~) "
Then G k (z), G_k (z) are Laplace transforms o__f functions gk (t), g-k (t) in
LI(0, ~) N L2{0, co) such that, fo___r k= I, Z, 3 ..... ~ = I, Z, 3 ....
u e - i k ~ t [ g _ , ( t ] d t = u e i k ~ t L g , ( t ) d t = , . (Z. 9 )
Proof. That G k ( z ), G_k ( z ) are Laplace transforms of functions gk (t),
g_k(t) inthe space L2[0, ~) follows fromthe fact that G_+k(X+ ly) is
boundedin LZ(- ~, ~) (as a function of y), uniformly for x -> 0 (see, e°g.
[ 1Z ] ) . Thus
Gk(Z) ~0 =e-zt gk (t) dr' G-k (z) ~0 e-zf = = g_k ( t ) d t
and the f o r m u l a e ( Z. 9 ) t h e n f o l l o w from 1 1 6 k Gf(-lk ½) = G_~(ik ½) = 6~,- k, ~-- i, Z, 3, G~(ik ~) = G_~(-ik ~) = ~ . . . . .
Fromthe asymptotic relationship between G(i0~) G' (ion) to 1 (I- e z~l~°2)
4~Z~ z
and its derivative, one sees very easily that G+k (it0) G' (ico) lie in LZ(-~,~o), ' _+k
sothat g+k(t), tg+k(t ) lle in L2[O,o~). That g+k £ LI[0'~) then
follows from
1 g_+k(t) = ~ h+k(t),_ h+k(t)_ =ql+t z g+k(t)_ e nZ[o, ~o),
and the proof is complete.
The eigenvectors of the operator G have the normalized form
':'k :'-k \ - i k : % / '
and, taken as initial conditions, are steered to zero during [0, ~) by the controls
u(t) = -gk(t)/A,k, u(t) = -gk(t)/6k
respectively. It follows that the null controllability problem for an arbitrary initial
state having the formal expansion
299
~0 _- ~ xo, k yk + Yo, k ~{ k ~0 k=l
we have the corresponding formal cont ro l Xo, k Yo, k
u(t)=- gk(t)-- g_k(t). (Z. iO) ~k 6k
To show that this is a bona fide control in our sense it would be necessary to establish
the convergence of {2.10) in L I[0, co) N L 2[0, ==). Such convergence would be
dependent on the relative rates of decay of xO, k, Yo, k' Yk' 6k as k -- ~ as
well as estimates on the gk ( t ), g-k (t) a very involved project and, in a
certain sense, rather futile because, for controls in L2[ 0, ~ ) at least, we can
obtain a controllability result very easily.
Theorem _7. Le__~t th_._e_e approximate controllability a s sumptlon { I. Z 3 ) hold for some con--~-~-~l di~trlbution element ( b, d ) E H~ . Then for each initial state f~0 1 E H~
therelsa control u e L Z 0,~) such ~hat the solution f~tt. )) of |~]°I,i 15 m) . . . . . . [ . . . . . . . . %BtLJl -- . co " " - corresponding to this initial state and control lles in H E for all t e [0, ) and
has the property
lim II(~] ~tt~ll = 0. (Z.ll) t -- ~ HE
Sketch of Proof. The proof is essentially the same as one given for control of the
wave equation in [ 13 ] so full details are not necessary here.
Let us take the system in a form equivalent to (I. 18) ( obtained from {I. 15 )
= _~}\y/ + u(t). (z. IZ)
The state (x, y) and the control distribution element (N, 6 ) are elements of
2 (o,~) x 2 L 0 Lo(0, ~ ) . i f (I~,3),~ (b ,d) e H E and, as evidenced by (io20), the usual norm in L O ( 0, ~ ) X L% ( O, ~ ) corresponds to where E is
the physical energy. We further abbreviate (Z.lZ) to
~= iTz + Tu(t), (2.13)
~- -A½ T being the unbounded self adJolnt operator diag (A ~ , ) on H=-L (O,~) X
L t(0, ~), r = (Y,6) E H . We introduce a feedback relation
u ( t ) = -p (z ( t ) ,~C)H , p > 0, (Z. 14)
thereby realizing a closed loop system
= iTz - @(z, ~)HT - ~z . (ZolS)
The dyadic operator (z, "~)H-f is bounded so ~ has the same domain as T
in H: the set of z such that Tz e I~, which coincides with those z = (x,y)
such that ( ~, ~ ) e H V . It is well known that T generates a group in H
which we w-ill denote by ~ ( t ) .
Proposition 8. In the strong operator topology we have
lim ~(t) = 0, (ZoI6)
300
i . e . , e a c h s o l u t i o n z ( t ) of t e n d s to z e r o in H a s t -
Sketch of Proof. Let z(0) =z 0 e ~{T) = ~(T) . Then z(t) is dlfferentiable
in H and we may compute
d l lz(t)II 2 = (z(t),~z(t))+(~z(t),z(t))= -Zp](z,T)l z <0 (Z. IT) dt
from w h i c h we c o n c l u d e t h a t ]l z ( t)){ Z i s n o n i n c r e a s i n g . I t i s e a s i l y s e e n f rom
c o n t i n u i t y c o n d i t i o n s t h a t t h i s m o n o t o n i c i t y e x t e n d s to a l l s o l u t i o n s z ( t ) = ~ ( t ) z 0,
z e l l . S t i l l k e e p i n g z 0 £ $ ( ~ ) , we o b s e r v e t h a t ~ ( t ) = ~ z ( t ) = ~ ( t ) z 0 =
~ t ) T z 0 so t h a t {] ~ z ( t ) ]{ Z i s l i k e w i s e n o n i n c r e a s i n g . I t f o l l o w s t h a t fo r
z 0 ~ * & ) t l z ( t ) l l z + l l ' ~ . ( t ) l l z _~ t lzol l z + 11"~%11 z .
But I1 zll z + ll~zll z -< c d e s e x e s a c o m p a c t s u b s e t of H so we i n f e r t h e e x i s t - / x
e n c e o f a s e q u e n c e t k - ~ and an e l e m e n t z ~ H s u c h t h a t
lira tl z ( 5~ ) " ~ l] = 0 . k - - ~
"- ll~,zll z Since z i s a l s o t he w e a k l i nd t of a s u b s e q u e n c e of t he z(tk) inthe IlzllZ+ .,%
norm, weconclude z ~ ~9~). Since H~(t)ll z isnonincreasing, llm llz(t)llZ= t ~c° wk
v >-- 0 and elementary considerations of continuity show that the solution z (t)
of (Z.13) with g(0)= z mustbe suchthat llz(t)l{ ~ v, t a 0. Then
(cf. (Z.17))
(z(t),T) -= O.
This can be written as 1
I" ik gt I, -ikgt, ~(t)- (~kXk e + 5kYke ) -= 0 (Z. 18)
k = O / x /N
w h e r e the X k ' Yk a re e x p a n s i o n c o e f f i c i e n t s of z ( a s in (1. Z0 )) and
Nk ' 5k a r e t h o s e o f T . The a p p r o x i m a t e c o n t r o l l a b i l i t y c o n d i t i o n i m p l i e s t h a t
t he ~ k ' 5k a r e a l l d i f f e r e n t from z e r o . U s i n g t h e g+_~ ( t ) d e v e l o p e d in
Lemma 6 we s e e t h a t c o . ,
N~x~ = ~(t) g~(t)dt = 0, 5~yf = co
A A £ = i, 2, 2 ..... ~ Xk= 0, yk= 0, k= i, Z, 3 ....
~x
from which we conclude z = 0, so that v = 0 and
nm {{z(t){{ = v - 0 t - - c o
The r e s u l t e x t e n d s by c o n t i n u i t y to a l l i n i t i a l s t a t e s z 0 e H .
The p roof of T h e o r e m 7 i s c o m p l e t e d by s h o w i n g t h a t u ( t ) , a s g e n e r a t e d by
(Z. 14) ( w h i c h c l e a r l y s t e e r s z 0 to 0 a s t -- co ) l i e s i n LZ[O, ~) . T h i s
f o l l o w s from ( Z. 17 ) :
II z01l z - ilz (t)llZ= zp f t l (z( S o ) '~ )lZds = zPfo~ u(s ){z ds .
Le t t i ng t - - ¢~
11%11 z = z p f l u ( s ) l z a s ( z . 1 9 ) 0
301
so that u E L z[0, ~) and the proof is complete.
We cannot prove, In this context that the controls u generated by ( z. 14 )
lie in LI[ 0, o~) F~ L2[ 0, ~), but we have all of the properties described in Theorem
X and Proposlt/on 3 nevertheless. For (b,d)e H E and / (~0 \ ~ e HE, the \130/
solution (~(t),q(t)) • H E forall t -> 0 because
(t) = ~(t) \~ (t)/ 0
where ~(t) is the group in H E corresponding to the group ~(t) in H
described above, That ( i. Zl ). is tree follows from the (act that
\qO/ \'I l~]l /%
and (Z. ll) implles g (t)- 0 strongly in H E .
3. Control on a Semi-lnfinite Interval
It should be clear from the result of Theorem 7 that defining controllability on
[0, =) as the existence of controls u in LZ[0, ~), orln LI[0, =) fl LZ[0, 4)
for that matter, such that for each initial state ( ~0' ~0 ) • HE we have (Z. ii)
for the controlled trajectory ( ~ ( t ), q ( t ) ), will not provide an adequate definition.
One fact which helps to make this clear is that the control u generated by ( Z. 14 )
has the property ( Z. 19 ) without regard to the norm of the control distribution element
( b, d ) e H E . ~f we let Jl (b, d )IJH ~ -- 0, maintaining the approximate control-
Iabilitycondition (I. Z3), ]tlseasy~to seethat (~(t),Ti(t)) tendstothe un-
controlled solution S ( t ) ( ~0 ' ~0 ) in HE, urdformly on any finite t interval;
remains bounded gives no comparable boundedness for the f a c t tha t 11 u I1 T.z[ o, = ) =
; ll(C(t), n(t))l]H dt. E
In the discussion of control systems controllability, per se, is generally not an
end in itself. More frequently one is concerned, In practice, with the behavior of
some quadratic cost integral, with the possibility of advantageous placement of
closed-loop spectra, etc. We know from the finite dimensional control theory and
from the work in [5] , [14] , for example, that for problems having a finite control
time the standard controllability results imply, through the process of constructing
the optimal control (whose existence is assured once the controllability result Is
available in most cases ) the existence of stabilizing feedback controls, uniform
displacement of spectra with the left half plane through the optimal feedback relation,
etc. The controllability result is, in the context of optimality, the existence of a
"feasible point" from which all else follows. It should not be surprising, then,
that we propose to adjust the notion of controllability, at least as applied to infinite
control intervals, so that it corresponds with the notion of feasibility for an appropriate
quadratic programming problem. In so doing, however, we wish to stay within the
general framework of controllability developed in [ 4 ] .
302
We consider the general linear control system
~ = Az+Bu, z e Z, ue TJ, (3.1)
where Z and U are Hilbert spaces, it is assumed that A generates a strongly
continuous semlgroup, S, in Z and B : IJ -- Z is a bounded operator. We intro-
duce also a third Hllbert space W -~ Z, Z dense in W, with the injection map
from Z into W bounded, i.e., for some constant b
Ilzllz _ < b l l z l l w , z ~ z , ( 3 . 2 )
and we a s sume t h a t t h e o p e r a t o r S ( t ) i s b o u n d e d w i t h r e s p e c t t o t h e W t o p o l o g y
of Z, t ~ [0, T] , T flnite.
Definition 3.1. Th____e system (3.1) is W controllable (perhaps W-open loop
stabilizable is a more accurate term, but too cumbersome) o_n [ 0, ~) if for each
L z z 0 E Z there Is a control u e ([0, co); U) such that the resultin~ solution
z (t) has the property co
f t l z t t ) l lw dt < ~ (3 .3) 0
I t i s e a s y to s e e from ( 3 . 1 ) , ( 3 . 2 ) t h a t i f ( 3 . 1 ) i s W - c o n t r o i l a b l e on [0 , ~ )
then, for the controlled solutions ( solutions for which ( 3.3 ) obtains ) we have
n m I l z ( t ) l l w : o. (3 .4) t - ~
Let I k = [ t l k - 1 - - t ~ k ] . From ( 3 . 3 ) 3 [ T k ] "C k e I k ]
l i m It z ( ' ~ k ) l l w = o . k - - ~ o
But, for
such that
(s. 5)
t ~ I k ,
ltz ( t ) - s ( t - ~k)Z(Tk)]]W -< b ] l z ( t ) - S ( t - T k ) Z ( T k ) l l z t t
= f s ( t - s ) B u ( s ) d s ] l z -<llf l l s ( t - s ) l l Z tlBII l lu(s) l l a s
Tk "Ok i
\ t ~t x Tk l lugs) l l~
f rom w h i c h ( 3. 4 ) f o l l o w s from t h e b o u n d e d n e s s of S on I 1 r e l a t i v e to t he
and Z t o p o l o g i e s and t h e r e q u i r e m e n t t h a t u c L2([ O, ~ ) ; U ) .
To p l a c e t h i s n o t i o n o f c o n t r o l l a b i l i t y i n t o t h e f r amework of [ 4] we d e f i n e
s p a c e s X, Y and o p e r a t o r s C , F a s f o l l o w s .
x fo e- til tt)ll dt<° ' Me ( k - s ) t / z t > 0 M, s k > 0 b e l n g o h o s e n s o t h a t i lS ( t ) I IZ < - , - , ,
con stants, y -_ ~2([o, ~ ) ; u ) x T.2([0, ~) ;w) ,
(ul:l) C:Y--X, C (t)= w(t)-fn S(t-s)Bu(s)ds,
F:Z--X, F(z0)(t)= S(t)z 0 •
W
p o s i t i v e
303
It wil l then be seen that the defini t ion of F-con t ro l l ab i l i ty given in [ 4] , i . e . ,
~(c) ~- ~(F) (3.6)
coincides with the notion of W-controllabllib7 on [ 0, ~ ) in Definition 3. I. ( ~t
should be noted that, as permitted in [ 4] , dora C is not necessarily all of Y
but is dense in Y. )
In finite dimensional spaces Z, with W necessarily equal to Z, W
controllability is Just stabilizability and it may be verified, with a little work, that
the dual (as defined in [4] ) property to (3.6)
HF;'xll Z _~ K l l c ~ x ] l y , x E dome;';, K> 0,
amounts to detectability of the dual linear observed system in Z
In a later paper we hope to develop the controllability properties of the water
wave system (I. 15 ) in this context.
Reference s
[I] J.J. Stoker: "Water Wave s" , Interscience Pub. Co., New York, 1957.
[ Z] C.A. Coulson: "Waves : A Mathematical Account of the Common Type of Wave Motion", Oliver and Boyd, Edinburgh and London, 1955.
[ 3] A. Friedman and M. Shlnbrot : "The initial value problem for the linearized equations of water waves.', ~. Math. Mech. 17 (1967), 107 ff.
[ 4] S. Dolecki and D.L. Russell : "A general theory of observation and control',, S]AM~. onContr. Opt., 15(1977), 185-ZZ0.
[5] ~.L. Lions and E. Magenes: "Probl~mes aux llmites nonhomog~nes et appli- cations", Vols. I, II, Dunod, Paris, 1968.
[6] T. Kate: ,,Perturbation Theory for Linear Operators", McGraw-Hill, New York, 1966.
[ 7 ] D. Russell: "A unified boundary controllability theory for hyperbolic and para- bolic partial differential equations", Stud. Appl. Math., LII (1973), 189-ZII.
[ 8] D. Russell : ,.Exact boundary value controllability theorems for wave and heat processes in star-complemented regions", in "Differential Games and Control Theory", Roxin, Liu, Sternberg, Eds., Marcel Dekker, Inc., New York, 1974.
[ 9 ] G. Chen : "Energy decay estimates and control theory for the wave equation in a bounded domain", Thesis, University of Wisconsin, Madison, 1977. To appear in SIAM J. Contr. Opt.
[I0] N. Levinson, "Gap and DensltyTheorems", Am. Math. Soc. Colloq. Pub., Vol. Z6, Providence, R.I., 1940.
[11] E.T. Whitaker and G. N. Watson, "A Course of Modern Analysis", 4th Ed., Cambridge, 19Z7 (Reprinted 1963 ).
[IZ] K. Hoffman- .,Banach Spaces of Analytic Functions", Prentice Hall, Englewood Cliffs, N.J., 196Z.
[13 ] 7. P. Qu/nn and D.L. Russell, "Asymptotic stablllty and energy decay rates for solutions of hyperbolic equations with boundary damping,,, Prec. Royal Soc. Edin., 77A ( 1977 ), 97-IZ7.
[ 14] D.L. Lukes and D.L. Russell, "The quadratic criterion for distributed systems", SIAM J. Contr., 7 (1969), I01-121.
BOUNDARY STABILIZABILITY FOR DIFFUSION PROCESSES
Roberto Triggiani ~,hthematics Department
Iowa State University, Ames, Iowa 50011
In IN1] T. Nambu studied the problem described below of boundary stabilizability
for a particular class of diffusion processes (Eq (i. 2)). The aim of the present
paper is to show that a different approach - more on the soft analysis side than in
Nambu's work - yields for particularly convenient choices of the sought after vec-
tors gk' much stronger results (in fact, the best results one can hope for) in a
quicker way and for the most general class of parabolic systems. Nambu confined him-
self to the canonical self adjoint case involving the Laplacian and made use of the
natural basis of eigenvectors; here we shall consider instead the general case involv-
ing uniformly strongly elliptic operators, where a basis of eigenvectors need not
exist.
To facilitate the reading of the present paper in relation to Nambu's, we shall gener-
ally adopt his notation when convenient.
The boundary stabilizability problem studied here differs from the ones recently con-
sidered in [$2] and [ZI].
i. .The stabilizability problem
Let ~ be a bounded open domain in R n. Let A(x,D) be a uniformly strongly ellip-
tic operator in ~ in the form
A(x,D) = ~ a a (x) D a I~l _< 2m
with r e a l c o e f f i c i e n t s a s . F i n a l l y , l e t Bi, i = l , . . . , m be m d i f f e r e n t i a l bound-
a ry ope ra to r s o f r e s p e c t i v e o rde rs m i given by
S i (x,D) = [. hie (x) D ~ I~I < m.
1
The boundary S = ~ ~ of ~ is assumed of class C 2m. As we shall make use of the
estimate known as Agmon - Douglis - Nirenberg inequality, we shall throughout assume
the conditions - specified in [FI, p74] - under which it applies. To write it down
explicitly, let A be the following operator in L2(~ ). The domain ~ (A) of A
consists of the closure in H2m(~) of the set of functions f in c2m(~) that sat-
isfy the boundary conditions B i(x,D)f = 0 on S (i < i < m). For every f e ~(A),
A i s defined as
(Af) (x) = -A(x,D) f(x)
Then the A-D-N ineqtmlity reads (for p=2)
(1.1)
Where
II ~ I I H2m(~) <_ C([[ Au ]]L2(~ ) C is a constant independent of
graph norm on ~(A). The operator A
+ ]1 u ]]L2(~)) = C [] u [1G u ¢ ~(A)
u [F1, p7S] and [[ u JIG i n d i c a t e s the
is closed [FI, p75]. Under the additional
305
assumptions that the system {B i} be normal [FI, p76] and that the strong comple-
mentary condition [FI, p77] hold, the operator A generates an analytic (holomorphic)
semigroup [FI, pl01, Example] on L2(~) which we shall conveniently indicated by
e At. Therefore the Cauchy problem
= An u(O) = u 0
on L2(fl ) has the tmique solution u(t) = e At u 0 , t > 0.
Remark I.I. The above model includes the diffusion process treated by Nambu [NI] :
u ( t , x ) = ~ u ( t , x ) - q ( x ) u ( t , x ) t > O , x e i~ 8t
(1.2) u(0,x) = u0(x), x ~
a(¢) u ( t , ¢ ) + (1 -a (¢ ) ) Bu( t ,~) = 0 t > O, ¢ e S Bn
where q ( . ) i s Holder con t inuous i n f l , a(~) ¢ C2(S) , 0 < a(~) _< 1, and 8/2n i s
the outward normal d e r i v a t i v e a t the p o i n t ¢ o f S: i n t h i s case t he c o r r e s p o n d i n g
operator A is self-adjoint.
Since ~ is bounded, the resolvent R(X,A) is compact. Hence the spectrt~ o(A)
of A is only point spectrtnn and consists of a sequence of eigenvalues {Xi} , i=1,2,...
with corresponding normalised linearly idependent eigenvecters ~ij' j=l,.. ,i i, I i
being the multiplicity of X i. As is well known, the Xi's are contained in a tri-
angular sector r. delimited by the rays a + p e -+i8, 0 j p < ~, w/2 < 8, with no
finite acctm~lation point. Therefore, at the right of any vertical line in the com-
plex plane, there are only at most finitely many of them.
A standing assumption that we make - for the stabilizability problem described below
to be significant - it that: there are (M-l) eigenvalues: XI,...,XM_ 1 at the
right of the imaginary axis, ordered, say, for decreasing real part:
( 1 . 3 ) Re X M < 0 < Re XH_ 1 < . . . < R e X 1
Let now y be any continuous operator from H2m(~) into L2(S ) . In applications,
y may be the o p e r a t o r t h a t a s s i g n s to f ¢ HI(~) i t s boundary v a l u e f [ s e L2(.S).
(This special case is the one considered in Nambu's work [NIl); or the operator that
assigns to f e HZ(fl) the boundary value of its normal derivative 2f/ 8n]s E L2(S)
(trace theorem). Define the operator B: 9 (B) c L 2(fl) + L 2(n) by
N (1.4) Bu = ~ (yu, Wk) gk u E ~ (B) = H2m(fl)
k=l Here the Wk'S and the gk's are fixed vectors in L2(S) and L2(~) respectively,
while (.,.) is the irmer product in L 2(S). The operator B is unbounded on L2(fl)
and it is even uncloseable [KI, p166] (take fl = (0,I), N = I, and let u~l(x) be a
sequence of smooth functions with compact support in, say, (l-I/n,l), and satisfying
Un(1)- I; take w= 2 and g(x)-= I. Then Un+ 0 and BUn÷2g ~ 0 in L2(Q)).
To avoid the case Bu - 0, we assume yu ~ 0 (with reference to the diffusion process
306
(1.2) and y the operator f e HI(~) ÷ flS ~ L2(S), this corresponds to assigning
a(~) ~ i, which is Nambu's case).
The system that we wish to study and stabilize is
N (1 .5) h = Au + ~ (yU'Wk) gk ' u(O) = u 0
k=l (Our entire procedure leading to the exponential decay of the solutions in the graph
norm applies verbatim by simply assuming that A generate a differentiable semigroup
(i.e. exp(At)L2(~ ) e N(A), t > 0) and that it have compact resolvent.)
A qualitative statement of the boundary stabilizability problem, which is the main
object of the present paper, is as follows: find - if possible - functions gk e L2(~)
k = I,.., N and conditions on the functions w k £ L2(S ) as to guarantee that the
solutions of (1.5) corresponding to the largest possible class of initial conditions
u0, tend to zero, preferably in an exponential manner, as t + + ~ in the strongest
possible norm.
Remark l.Z The above stabilizability problem for the abstract system (1.5) corres-
ponds - say for N = 1 - to the following output stabilizability problem in the clas-
sical finite dimensional theory [ ]: given the output system
{ x = Ax + g f x , g ¢ R n
y = H x y ~ R m
f i n d the s c a l a r i n p u t f as a feedback o f t he o u t p u t , i . e . o f the form f = (Y'k)Rm'
k e R m, i n such a way t h a t t he r e s u l t i n g feedback sys tem:
= Ax + (Hx,k) g
be globally asymptotically stable.
The first question to settle is of course the well-posedness of eq. (1.5). This is
the object of the next section. In preparation, and following the procedure origi-
nated in [TI], let L2(e ) be decomposed into two orthogonal subspaces E 1 and E 2,
corresponding to, respectively, the subset {ll,... ,XM_ I} and {li, i _> M} of the
spectrum o(A) of A satisfying (1.3). Here we appeal to the standard decomposition
theorem as in [KI, p178]. With P denoting the orthogonal projection of LZ(~) ontc
El, then (I-P) $ (A) c $(A), E 1 and E 2 are invariant under A and hence under
the semigroup e At . Also, g(Al) = {l I .... ,XM_I }, g(A 2) = {xi,i_>M}, where Aj is
the restriction of A on Ej, and A 1 is bounded. Finally P and (I-P) commute
with A, hence with the sen/group e At. We shall henceforth use the notation Pu = u 1
and (I-P) u = u 2. Moreover, to avoid cumbersome notation, the norm in L2(fi ) and
in L 2 (S) will be simply denoted by I] ]I, while other norms will be specified by
an appropriate subscript. The norm of y from H2m(~) + L 2 (S) will be instead de-
noted by l lIYlll 2. Well-posedaess of eq. (1.5).
For the particular case of eq. (1.5) corresponding to the diffusion process (1.2),
Theorem 3.1 of (NIl claims the following: "The differential equation has a real-
valued solution u(t) for the real-valued initial value u 0 satisfying (I-P) u 0
307
e ~(A~ ), where 8 > 1/2. The solution u(t) such that Bu(t) has at most a
summble singularity at t = 0 is unique". By solution it is meant a function u(.)
of class
C ( [O,~); L2(fl)) n CI((0,~); L2(fi)) , satisfying Bu(.) E C((0,~); L2(R)).
The proof of this statement given in [NI] is a rather lengthy computation. Our first
theorem provides a much stronger conclusion for the well-posedness of eq.(l.5): the
unique solution of eq. (i.5) corresponding to any initial condition u 0 ~ L2(~ ) is
analytic for t > 0. Our short proof is radically different from Nambu's, being
based on viewing the operator B as a perturbation of the generator A.
Theorem 2.1 The operator A + B with domain 8 CA+B) = ~(A) generates an analytic
semigroup e (A+B)t on L2(~), which gives the solution of (i.5) : u(t,u0) = e (A+B)t u0,
t>0.
Proof First we observe that the operator B has finite dimensional range (of dimen-
sion in fact at most N). Therefore the desired conclusion follows from a recent
perturbation theorem of Zabczyk [ZI, Proposition I] - which relies on the standard
perturbation result [KI, Thm 2.4, p497] - as soon as we prove that B is bounded
with respect to A [KI, p130]. To this end, definition (1.4) and the continuity of
y imply N
I lBull 5 c IlUllH2m(n,) u e H2m(~), c = I [ IYl l l k=lZ [I w k I[ I[ gk II
and we only need to invoke the A-D-N inequality (I.i) to conclude Q.E.D.
Remark 2.i. As Zabczyk has shown [ZI, Remark 3], the A-bound of the operator B with
finite dimensional range is actually zero, i.e. we have
(2.1) I IBull ! a l lAull + b l lu l l u e ~(A) c ~(B)
where the greatest lower bound of all possible constants a in (2.1) is zero Co will
generally increase as a is chosen close to zero).
3. Stabilizability.
In order to formulate our stabilizability result, let W i be the N x i i matrix de-
fined by
(w 1, Y ~ i l ) , (w 1, Y@i2 ) , . . . , (w 1,
W. = l
(w 2, Y~il ) , (w 2, Y@i2 ) , ... , (w 2,
0%,
associated with each eigenvalue k i
normalized eigenvectors @il' " ' "
Theorem 3 . 1 .
( 3 . I )
~il.) 1
7@ii.) 1
Y~il) , (WN' ~i2) ..... (WN' ~i1. ) i
of A, with multiplicity i i and associated
, @ili •
~ich implies
Let A I be diagonalizable. Also asst~ne the condition
rand W i = I i , i = i, ..., hi-1
N_> max {li, i = I, ..., M-I}. Then, for any E, 0 < e < - ReX M,
308
there exist vectors gk in E l, K = 1 ..... N (to be specified in the proof of Learns
3.2 below) such that the solution u(t,u 0) = e (A+B)t u 0 of the corresponding eq.
(I.5) due to any initial condition u 0 E L2(~ ) , satisfies for any preassigned posi-
tive number h:
[H2m(£)< C e -et (3.2) [[u(t,u0) [ C [[u(t,u0) I[ G< ¢,Uo,h t_> h > 0
where II II G is the graph norm and Ce,uo,h a constant depending on e, on u 0
and h. Hence, by the Principle of Uniform Boundedness, it follows that for the cor-
responding operator B one has
C e -Ct (3.3) [e(A+B)t] < e,h t > h > 0
where [ [ is the corresponding operator norm. Actually a slight variation of the
same proof for initial conditions u 0 E ~(A) shows
] [e(A+B)t][ ~(A) < Cee'et t _> 0
where ]I [I g(A) is the operator norm corresponding to the graph norm on ~ (A).
Remark 5.1. The minimum number N of such functions gk is equal to the largest
r~zltiplicity of the eigenvalues kl, ..., AM_ 1.
Remark 3.2. The same proof will show that if one assumes rank W i = i i true for
i = I,.., I-1 with M < I and A restricted on the subspace corresponding to
Xl,.. ,ki_ 1 diagonalizable, then in the conclusion of the theorem one can take any
e with 0 < e < -Re k I while the gk's are taken in such subspace. In particular,
if rank W i = I i holds for all i and A is normal, then the exponential decay of
the solution can be made arbitrarily fast.
Remark 3.3. Even in the special case studied by Nambu regarding the diffusion process
(1.2), where m = 2 and y only continuous from HI(£) + L2(S), our Theorem 3.1 -
as well as our theorem 3.2 below - are much stronger - than his Theorem 4.2 in [N-l]:
in fact Nambu's Theorem 4.2 only gives an exponential upperbound in the weaker H 1 (£)-
norm and only for initial data u 0 with projection u20 = (I - P) u 0 e ~(A2B), 6 > 1/2
His gk are not taken in El, but 'close' to it (i.e. II (l-P)g k I] 'small').
Proof. In (3.2) the inequality on the left is the A-D-N inequality (i. i). To prove
the right hand side of (3.2), we select preliminarly the vectors gk to be in E l ,
s o that N
PBu = ~ (yu, Wk) gk ¢ El' whi]e (I-P) Bu =- 0 k=l
The projections ofeq.(l.5)onto B 1 and E 2 are
N N
(3.4) 1~ 1 = AlU 1 * [ (YUl' Wk) gk * Z (Yu2' Wk) gk k=l k=l
and
(3.5) ~2 = A2 u2
309
respectively. Since A 2 generates an analytic semigroup on E2, it satisfies the
spectrtm determined growth condition [TI, § 2] and hence A2t -E2t
(3.63 I luz( t ,U2o) l l = II e u2oll < e II Uzo II A2t
for all u20 c E 2 and any c2, 0 < e 2_< -ReXt. I. Due to the analyticity of e have
w e
A2t A2(t-h) Azh eA2Ct -h) A2h II A2 e U2o II = It A z e e ~20 II = I1 A z e u20 II
-e2 t -c2h A2h (3 .6 ' ) ~ e e II A 2 e u20 ]l t ~ h > 0
and hence (3.6) and (3 .6 ' ) imply
(3.7) l lu 2 (t,U2o) l l G = l le ~zt -c2 t
Uz011G < c c2,h e , -- u20 ,
The unperturbed part of eq. (3.4) is
N (3.8) z = ~ z + [ (yz, w k) gk ' z a E 1
k=l
and can be rewrit ten in matrix form as
(3.8') z = A z g,w
where A i s a square mat r ix o f s i ze equal to g,w
and the Wk'S. This can be seen by using in E 1
t>h>0
dim El, depending on A1, the gk ' s
the (non n e c e s s a r i l y or thogonal)
basis of normalised eigenvectors @ij ' i = I, ..., M - I, which make the matrix
corresponding to the operator A 1 diagonal. The exponential decay of (3.8') for a
suitable choice of the gk's is handled by the following Lenmm.
Le~na 3.2. Assume condition (3.i). Then for any ~I > 0, there exist vectors gk e
El, k = i, ..., N, such that the solution z(t,z0) due to the initial datum z 0 of
the corresponding equation (3.8') satisfies
(3.9) I Iz( t ,zo) l I II e%'wt ^ ~ i t
-- z01 I_< cz0,~ 1 e t > 0
in the norm of E 1 inhe r i t ed from L2(a). The minimum number N o f such gk ' s i s
equal to max { l i , i = 1, . . . , M - 1}
Proof of Lenma 3.2. See appendix for a constructive proof.
It remains to show exponential decay of the perturbed equation (3.4). The analyticity 5t
of the semigroup implies e u20 e ~(~) for all t > 0 and all u20 e E 2. The
A-D-N inequality (i.I) and the inequality (3.7) give
e ~t e ~t -¢2 t (3.10) I lu2Ct,uz0) I IH2m(a) = II u2ol IH2m(a)_ < C II Uz011 G <_ Cuz0,%,he ,
t>h>0
for any ~2' 0 < c 2 < -Re X N. From now on let the vectors gk be the ones of Lersaa
3.2. S t a r t i ng from (3 .9) , one e a s i l y obta ins
310
A - elt
(3.11) ]I z(t,z 0) 11G = [I e~'wt z 0 I I G <__ Ccl,Z 0 e , t > 0
Finally, we write the variation of parameter formala for the perturbed system (.3.4): A t _ t e A , w ( t - r ) g N
(3.123 u l ( t , u 0) = e g,w (Ul0 + Vh ) + f h k=~l CYU2(x ) ,wk)g k dr ,
where Vh ~h e-Ag,w ~ N = ~ (Yu2C~)' Wk) gk d~ k=l
As the unperturbed system (3.8') satisfies the expotential bound (3.11), while the
perturbing term of (3.4) satisfies a bound related to (3.10), we finally obtain from
-¢I (t-r) -c2T (3.12) : -Eft t e e dT [ l u l ( t , u 0 ) ll G < CEl,Ul0,V h e + K 7h
-el t -¢2 t -E2 t (3.13) --< C e l , u l o , V h e + K e < eons t t > h > O
c 1 - c 2 -- c 2 ,u 0 ,h " --
where N
K--Cu20,ez,h III ~ I11 X II w k II c k=l ¢ l ' g k
and where e I is now chosen larger than the preassigned ~2 e (0, - Re XM] , say
c I -- 2e 2. The desired right hand side of inequality (3.2) then follows from (3.7)
and (3.13). Q.E.D.
Remark 3.4. As noticed in (NIl on the basis of results of IS1), condition (3.1) is
also necessary for choices of gk restricted to E 1. In fact, in this case, failure
of (3.1) at some ~. makes ~. an eigenvalue of (A + B). l l
If one insists on selecting stabilising vectors gk not in E 1 [NI , Remark in § 4],
the following theorem, whose proof is more elaborate, serves the purpose.
Theorem 3.2. Under the same assumptions as in theorem 3.1, given any c, 0 < e <-Re XM,
one can select suitable vectors gk' with 0 ~ Qgk c ~(A2) such that for the solu-
tions of the corresponding eq. (1.5) the same conclusion as in theorem 3.1 holds.
Here Q = I -P .
Proof. For simplicity of notation we write the proof only when N = i, only trivial
changes being needed when N > i. The projections of the solution u(t,u0) = e(A+B)tu0
onto E 1 and E 2 are:
Ul(t,u0) = eApg'wt ul0 + t eApg,w(t-x) Pg (Yu2(r) , w) dr 0
A2t t u2 ( t , u 0 ) e u20 ~ / 2 ( t - z ) = + Qg [(yuz(r),w) + (YUl(r), w)] dr
O
For any h > 0 and t_> h, these can be rewritten as
(3.14) Ul(t,u0) = eApg,w t (Ul0 + rh ) + ;t eApg,w (t-r) pg(Yu2(T), w) dr h
311
(3.15) where :
rh = rh(g ) = ;h e-Apg,w r pg(yu2(T),w) dr 0
Vh = Vh (g) = fh eA2 (h-~) Qg[(~u2(r) ' w) + (YUl(Z), w)] d'r 0
Notice that u 2 (t ,u0) = Q e (A+B)t u 0 e ~(A) (% E 2 ; 19(A 2) for suitable stabilising g with O # Qg e ~(A2). For t > h > 0,
eA2 t A2 (t-h) ft eA2 (t 'T) u2(t,u 0) = u20 + e Vh + h Qg[f~aaZ(X)'w) + (YUl(X)'w)] dr
(3.16) k 2 u2~,u 03 : ~(t-h) [A 2 Azh t A z ( t - r )
e u20 + A2 Vh] + fh
e ~;(A 2)
t > 0. We seek a
(3.1S) yields
A 2 Qg [ (yu z(T),w) +
(ra 1 (v), w) ] &
Therefore, for a suitable choice of the projection Pg in E 1 as dictated by Lemma
3.2, eqs. (3.14), (3.15), (3.16) and the A-D-N inequality (I.I) yield for t ~h > 0:
-el t - a l ( t -~ I (3.17) [ l~ ( t ,Uo) l [ G 5 Cel,Ul0 + rh e + ~h t Cel,pg e [Ixlll l lwll [ lu2(r ) l lG dr
-a2 t t P2 e ( 3 ,18 ) I lu2(t,uo)ll a ! "1 e + ~h where e I is an arbitrary positive constant and
c < ¢2 < -Re %M
( i ) "1 = Pl (g) = n~x {llu2oll + e ¢2h
(3.193
(ii) ~2 : ~2 (Qg) : c l l l x l l l
-¢z( t -T)[ i lu2(z) l lG + I lu I (T) IIG] az
¢2 i s constant satisfying
A2h ilVhl[ ' e ¢2h IIA2(e u20 + Vh) ll}
Here we choose to indicate for u i
means of (3.17) we then compute:
¢ T
Yht e z I lul(*)ll G dT _< CE1, Ul0 + r h
(3.20) Ca1 ' pg I I I v l
IIwll max {llQgll, IIAzQglI}
only the dependence on the pro jec t ions of g. By
(¢2-¢i)t 1 - e +
E 1 - c 2
t (~2-el)S-e(e2"el)t ¢is e e I [u2(s) I I G ds
I INII d h % - ~z
where the second term on the right side was obtained after a change in the order of
integration. Hence selecting E 1 > a Z yields
_ _ -E2t _ + e _ +c J l l x l l l Ilwll
~th e ¢2(t T) Ilux(~)llG dr < C¢I,Ulo rh E1 c2 ¢i 'pg ¢i " ¢2
t e-% (~-s) I (s) ll G ds (3 .n ) ]h lu2 Finally, we plug (3.21) into (.3.18) to get
312
(3.22)
where
(3.23)
Iluz(t,u0)ll G _< ~ e -e2t + ~ht ~ e-~2 (tml) ilu2 (~)IIG
(i) ~ =B~ (g) = ~i + C ~l,Ulo + rh ~2 ~i - e2
d~
T x i h + ~iGi] ' Gi
1
1 1 g i , l ' " ' " g i ,1 i
1. 1. 1 1
g i , l ' " " ' g i ,1 i
(ii) ~ -- ~ (g) -- ~'z ÷ %l,Pg I l l f i l l I lwll.z a 1 - e 2
We now need to invoke a s tandard r e s u l t [L1, Corol lary 1 .9 .1 . p.38] with
e2t re(t) = o Ilu2(t,u0) ll G, n( t ) ; I~ , v ( t ) -= ~
to get -5~h e - ( ~ 2 - ~ ) t (5.24) I lu2(t ,Uo)[[ G _< D~ e t_> h > 0
Analyzing (3 .19) ( i i ) and ( 3 . 2 3 ) ( i i ) , we see tha t ~2' hence ~ , can be made as small
as we please by su i t ab ly se l ec t ing Qg. In f a c t , the range ~(A2) o f ~2 being
de~e in z z , we can take y ¢ a(a 2) with I ly l l smart and define Qg = A2-1 y so t h a t I lqgll and t IAzQglt are so small as to make
- R e X M < - @ 2 - M Z ) < - e
where e i s the preassigned constant in the statement o f the theorem. Hence
(3.25) [ l ux ( t ' u0 ) ] l G < K e , u 0 , h e - e t t > h > 0
Plugging (5.25) in to (3.17) f i n a l l y y i e lds
- c t (s.26) I lul( t ,u0) ll G _< K ,u0,h e t _> h > 0
where e I i s se lec ted g rea te r than ~. Eqs. (3.25) and (3.25) provide the des i red
conclusion.
APPENDIX. A cons t ruc t ive p roof o f Len~na 3.2 is sketched here . In [N1] ins tead , a
well known exis tence r e s u l t on pole assignment, essentially due to Wonham, is invoked
from [SI] for its proof. First, consider an arbirtrary eigenspace S i of dimension
li, corresponding to the eigenvalue Xi' 1 < i < ~-I). Using the (non necessarily
orthogonal) basis @il' "'"~il i ' one can show by direct computations that the re-
striction of matrix Ag,w over S i is given by the follo~-ing I i x I i matrix:
313
where the rows of G i are, respectively, the coordinates of gl' "" ' gl i restricted
over S i with respect to the chosen basis. Since W i is of full rank, there is a
matrix G i in S i such that W i G i = -a t If. with 1
(A.Z) • • • • • . o • • . . . . . • o , •
N N N N gN = [ g l , i . . . . . g l , 1 i . . . . . gM-l,1 . . . . . gM-I,IM_I ]
k where one sets gij = 0 if k > I i. Then N = max { li, i = i, .., M - I}.
Finally, since each S i is invariant under the motion, the desired exponential bound
A t II eg'W II \ e t >0
for such g i ' s as in [A.2) is obtained from (A.1) plus f i n i t e l y many applications of
the law of cosines. Q.E.D.
REFERENCES
F1 A.Friedman, Partial differential equations, reprinted by Robert E. Krieger publishing Con~any, Huntington, New York, 1976
K1 T. Kato, Perturbation theory for linear operators, Springer-Verlag, New York/ Berlin, 1966
L1 V. Lakshmikanthara and S. Leela, Differential and Integral Inequalities, Vol. I, Academic Press, 1969
N1 T. Nambu, Feedback stabilization for distributed parameter systems of parabolic type, manuscript, revised
S1 Y. Sakawa and T. bIatsushita, Feedback stabilization of a cloiss, of distributed systems and contruction of a state estimator, IEEE Trans Autom Contr. AC-20 (1975), 748-753
SZ M. Slemrod, Stabilization of boundary control systems, J. Diff. Equat. 22, 420- 415 (1976)
T1 R. Triggiar~, On the s t a b i l i z a b i l i t y problem in Banach space, J. ~.kath, Anal. Appl. 52 (1975), 383-403; Addendum, Ib id . , 56 (1976)
Z1 J• ZabczTk, On decomposizion of generators, SIA~4 J. Control, to appear in 1978 Z2 J. Zabczyk, On s t a b i l i z a b i l i t y of boundary control systems, Universite de Montreal,
Centre de Recherches ~athematiques Report CRM - March 1978.
a t > max { Re k i ' i = I , . . , M - 1 } + ~,
and ~ i s an a r b i t r a r i l y p r e a s s i g n e d p o s i t i v e number• Therefore fo r such a choice
of the G . ' s , we have 1
(X i + [W i Gi]T ) t (A.1) lie Ill I I _< C e - ~ t , t > 0
with c and C e independen t on i , 1 < i < (M - 1) . Next, c o n s t r u c t v e c t o r s g l '
" ' " gN i n E 1 by s e t t i n g :
1 1 1 gl = [g ,1' • • " g1,1 i ' " ' " gM- l , l ' " ' • ' gM-I,IM_I ]
SPLINE-BASED APPROXIMATION METHODS FOR
CONTROL AND IDENTIFICATION OF HEREDITARY SYSTEMS
H. T. Banks e Brown University
J. A. Burns + and E. M. Cliff + Virginia Polytechnic Institute and State University
We consider control and identification problems for delay systems
~(t) = A0xft) + Alx(t-r) * Bu(t) 0 < t < T (1)
x(0) = n, x 0 = ¢
where AO, A 1 are n×n matrices, B is n×m, and u is an Rm-valued function that is
square-integrable on [0~T], i.e. u 6 L~(0,T). For x: [-r,T] ~ R n, we will denote by
x t the function 8 + x(t+8), -r < 8 < O.
These problems are infinlte-dimensional state system problems and our approach
here involves ~ewriting (i) as an ordinary differential equation in an appropriately
chosen Hilbert space Z and then employing techniques from functional analysis to discuss
convergence of spline-hased appmoximmtlon schemes.
As the state space ~re choose Z = R n x L~(-r,0) since one can argue equivalence of
(i) in some sense to the abstract differential equation in Z given by
(2)
~(t) = 5~z(t) + CBu(t),0)
z(0) = z O.
More precisely, taking x as the solution to (i) on [0,~) for a given (n,¢) and u E 0,
we define the homogeneous solution semlgroup {S(T)}, t ~ 0 by S(t)(~,~) = (x(t;~,~),
xt(n,#)). Then {S(t)} t is a C0-semlgrou p with infinitesimal generator h~ defined on
o~Q{) = [(~(O),~)l~6w~l)(_r,O)} by _Q~(~(G),$) = (A0@(O)+AI~(-r),~). Furthermore,
the integrated form of (2) fs equivalent to (i) in That z(t;n,#,u) = (x(t;n,#,u),
for (~,¢) in Z and u 6 L~(0,T) where x is the solution of (i) and for
0<t<T
t #
(3) Z(t;,,¢,u) E S(t)(N,~) + ~S(t-a)(Bu(~),0)d~.
0 We next chcQse a sequence of subspaces Z N of Z in which we approximate equation
(S). Our choice of the Z N of course dictates the type of approximation scheme we
~This research supported in pamt by the National Science Foundation under NSF-GP- 28931x3 and by The Air Force Office of Scientific Research under AF-AFOSR 76-3092.
+This research supported in part by the U.S. Army under DAAG-29-78-G-OI25 and by the U.S. Air Force under AFOSR-77-32221-A.
315
generate and here we describe briefly the results obtained when He use subspaces of
first-order (piecewise linear) spline functions, Let {iN.} be the partition of [-r,O]
] Z N defined by t; = (-Jr/N), j = O,I,...,N, and let Z N be defined by = {(~(0),@)I~ is N N N continuous on [-r,O] and linear on [tj,tj_l] , j=l, .... }. Then Z N is a (closed) sub-
space of Z and we let pN be the orthogonal projection of Z onto Z N. Finally we define
5~N: Z -~ Z N by ~N = pNb~pN" One can use the Trotter-Kato approximation theorem
(see Thm 4.5, p. 90 of [5]) plus elementary estimates from spllne analysis (see [3])
to argue that 5~ N is the generator of a Co-semigmou P {sN(t)} on Z and that sN(t)z ÷
S(t)z as N ~ ~ for z 6 Z, uniformly in t for t in any bounded interval. Defining
zN( • ;n,$,u) as the solution of
~.N(t) = j~NzN(t) + pN(Bu(t),O) (4)
zN(o) = PN(~,¢)
or, equivalently (Z N is finite-dlmensional and _~N is bounded)
t
(5) zN(t;~,¢,u) = sN(t)pN(n,¢) + /sN(t-a)pN(Bu(a),O)d~,
0
one can establish (again see [3]) that zN(t;n,¢,u) ÷ z(t;n,¢,u) for (~,~) e Z,
uniformly in t on bounded intervals and uniformly in u for u in any bounded subset
of L;(O,T).
One employs these ideas in optimal control problems in exactly the same manner
as one uses the approximation techniques developed in [2]. Briefly, given a closed
convex subset ~ of L;(O,T) and a cost functional ¢(u) = J(x(u],u), one seeks ~o
minimize # over ~ subject to (i). Defining components z N = (xN,yN), xN~ R n,
y 6 L (-r,O), and a sequence of cost functionals @N(u) = j(xN(u),u), one obtains a
sequence of related opt~mizatlon problems which consist of minimizing @N over
subject to (4) for (5)). Under reasonable assumptions on J (see [2]) one can use
the convergence results (z N ~ z) given above to establish the following.
Theorem i. Suppose ~N is a solution of the problem of minimizing sN as defined _N
above. Then {~N} has a subsequence {u k} converging weakly to some ~ in ~ that is N _N
a solution to the problem of minimizing $ and $ k(u k) ~ $(~). If the problem of
minimizing # has a unique solution ~ (e.g. if $ is strictly convex), then the sequence
{~N} itself must converge weakly to ~.
Under certain conditions (see [2]) one can actually obtaln strong convergence
of {~N} to a m~nlm~zer for 4.
We have used this spline-based scheme on a number of test examples which can be
solved analytically. We present here one such example which exhibits the typical
behavior we found in these numerical experiments.
316
Example i. We consider an optimization problem for a controlled damped oscil-
lator with delayed restoring force. The problem is to minimize
2
@ = y(2)2+~(2)2} + ~ u ( t ) 2 d t
0
over uE ~ = L2(0,2) subject to
y(t) + 9(t) + y(t-l) = u(t) 0 < t < 2
y(e) = 10
9(8) = 0 -i < 8 < 0.
This problem can be solved analytically using necessary and sufficient conditions (a
maximum principle) for delay systems. We found the optimal control u (several values
are given in Table l) and the corresponding optimal value of the cost functional
TIME ~ ~4 u-32
.0 -.870988 -.877470 -.900984
.25 -.199328 -.130329 -.223100
.50 .497522 .642116 .478541
.75 1.174459 1.32235 1.16618
1.00 1.758728 1.90070 1.75745
1.25 2.331887 2.58448 2.33992
1.50 3.067838 3.35974 3.08711
1.75 4.012817 4.25332 4.04827
2.0 5.226194 4.91257 5.19550
TABLE 1
-N i~_~pLI -N - -N N %VE l %vEl
4 19.9843 .2364 17.9646 1.7832
8 19.7929 .0450 18.7745 .9733
16 19.7616 .0137 19.2439 .5039
32 19.7528 .0049 19.4935 .2543
TRUE MINIMUM: # = ~(u) ='19.7478.
TABLE 2
= %(u) = 19.747846. We then used the spline-based method detailed in the presenta-
tion above to compute, for several values of N, solutions ~N for the problems of
minimizing ~N. In Table 1 we have lis~ed values for a couple of these controls while -N
in Table 2 we give values (labeled #SPL ) of the costs ~N(~N). We also carried out
317
computations for this example using the "averaging" (AVE) approximation scheme devel-
oped in [2]. For the sake of comparison, we have also included in Table 2 optimal N -N -N
values # (uAV E) = SAVE of the payoffs for these approximations. As we see, the
spline-based approximations offer a rather substantial improvement over the averaging
approximations in this example (as well as in a number of other examples for which we
have carried out the comparisons).
We turn next to the use of our approximation scheme in the identification of
parameters in delay systems. The basic problem is one of fitting to data a model such
as (i) (where u is a fixed input and A i = Ai(q) are continuous functions of some para-
meter q 6R k) by choosing parameter values q from some compact set ~ C R k. To be
more specific, assume that one is given values ~I,...,~M that are to represent measure-
ments at times tl,...jt M of the "oBservables" c(t;q) = Cx(t;q) for (i), where t +
x(t;q) is the solution to (i) corresponding to a value q in ~. The problem then is
to find a value q 6 ~ such that
M
(6) E(q) : [Ic(ti~q)-ql ~ i=l
attains a minimum on ~ at q = q. Here W represents a positive definite weighting
matrix.
As in the case of the control problems above, we formulate a sequence of identi-
fication problems, each w~th a f~nlte-dlmens~onal state space, We use the values (~i }
as observations for the approximating systems
~N(t) =_~N(q)zN(t) + pN(Bu(t),O)
(7) cN(t;q) = cxN(t;q),
where as above z N = (xN,y N) and ~N(q) = pNB~(q)pN with 5~(q)(~(O),~) =
(Ao(q)~(O) + Al(q)~(-r),~). We seek a value ~N in ~ so as to minimize
M
(8) EN(q) : [IoN(tl;q)-ql~ i=l
over ~. The Trotter-Kato theorem can be used to establish the following identifica-
tion results.
Z N Theorem 2. Suppose ~N E ~ is a solution of the identification problem on (i.e.
a minimizer for (8)). Then {~N} has a subsequence {q mk} converging to some q 6~
which is a minimizer for (6). If the problem of minimizing (6) over _~ has a unique
solution q, then the sequence {~N} itself must converge to q.
We sketch a proof of this result. Since ~ is compact it suffices to argue
(assuming that we have taken a subsequence and reindexed) that ~N ~ ~ implies
xN(t~q N) ~ x(t;q). To see this just note that EN(q N) ~EN(q) for any q 6 9 and
318
since xN(t;q) ÷ x(t;q) for fixed q (from our presentation above on the basic properties
of the approximation scheme), we find that for any q 6 _~
E(q) = lim EH(q N) < lim EN(q) = E(q).
--N --
TO argue that xN(t;q N) + x(t;q) whenever q ÷ q, one only need verify that the
hypotheses of the Trotter-Kato theorem [p. 9Q, [5]) are satisfied. It is not difficult
to argue (see [2],[3]) that
~.~N(qN)z,z> = <p~(qN)pNz,z> = <5~(~N)pNz,pNz>
<__ m(qN)IpNzI2 <__ ~(qN)Izl2
where m(~N) < K for all N. Hence the stability criterion in the Trotter-Kato hypo-
theses is easily established. For the consistency argument one may choose ~ =
{(~(0),*)I~E C 2 and ~(0) = A0(q)*(0)+Al(q)~(-r)} and show that oJ and (5~(q)-kl)~,
for k sufficiently lamge, are dense in Z (see Lemma 2.2 of [3]). For fixed z 6
we then have
_< +
= 6 N + e N.
From the arguments (based on elementary spline estimates) given in [3], it follows
immediately that E N -~ 0 as N -~ ~. It is also shown in [3] that ~N -~ ~ uniformly on
[-r,0] whenever ~ 6 C 2 and ~N is defined by PN(~(0),~) = (~N(0),~N). We may use this
to argue that gN ÷ 0 as follows:
~N : [~N(qN)z--Q~N(q)zl : IPN{-~{([%N)-~(q)}pNzl
= I PN{J~(qN)-~(~I) }(*N(o),,N) 1
: Ip~(~N,o) I <__ I~NI where ImNl : IA0(~N)~N(o) + AI(qN)*N(-r) - A0(q)*(0) - Al(q)*(-r) I -~ 0 as N ->
by the continuity of A0, A 1 and uniform convergence of ~N to ~.
We have also tested this method on a number of sin~ple examples and present here
one that is representative of our findings.
Example 2. The system we consider for identification is an oscillator with
delayed damping and a delayed restoring force. The equation and initial data are
y(t) + m2y(t) + ag(t-l) + by(t-l) = u (t) T
y(O) = 1
9 ( 0 ) = o - 1 < o < o,
319
where w > O, a and b ame parameters to he determined from "observations" of y and u T
is the unit step input at T = .i, i.e., u r equals 0 for t < T = .i and equals 1 for
t > T = .i. This test example was solved analytically for 0 < t < 2 and we used
evaluations {Y(tl)} of the solution at 101 equally-spaced points in [0,2] as the
observations {~f}. The values ~ = 6, a = 2.5 and b = 9 were used in doing this.
By utilizing "observation data" generated in this manner, identification procedures
based on the spllne approximations discussed here and on the averaging approximations
(discussed in this connection in [4]) were carried out on the computer. For fixed
values of N an iteratlve scheme was employed to find maximum likelihood estimates
(MLE) ~N = (~N~N,~N) in place of solving directly the problem of minimizing (8).
Results of these computations are presented in Table 3 (start-up values for the MLE
scheme were ~ = 5, a = i, b = 5 in each instance). As in the case of the control
example presented above, we see that for this example the spline approximation method
offers an improvement over the AVE-based identification scheme. Computer times for
the SPL and AVE schemes were approximately equal.
SPL METHOD AVE METHOD
-~ -N ~N ~N -N ~N N ~ a a
2 6.1103 -5.7953 10.3745 6.3864 -12.8383 4.2478
4 6.4860 5.6288 13.2680 5.7480 -5.4170 7.3614
6 6.1045 3.3659 9.6848 5.6252 -3.3497 8.2194
8 6.0432 2.8791 9.2921 5.6564 -1.8301 9.7648
I0 6.0246 2.7213 9.1708 5.8853 .3826 13.0549
ii 6.0179 2.6173 9.2212 6.1514 2.4135 16.2575
TRUE 6.0 2.5 9.0 5.0 2.5 9.0
VALUES
TABLE 3
Our discussions above deal only with the simplest of delay equations (i). How-
ever, the results we have presented (for both control and identification problems) are
applicable to very general linear hereditary systems (see [2], [3]) of the form 0
~ ( t ) = [ A i x ( t - T i ) + A ( O ) x ( t + e ) d 8 + B u ( t ) i=0
- - r
and even c e r t a i n n o n l i n e a r h e r e d l t a w sys tems ( s ee [ 1 ] ) . F u r t h e r m o r e , t h e i d e n t i f l c a -
tion ideas are valid if one seeks to identify delays as well as coefficients in the
system, e.g., in the case
~(t) = A0(a)x(t) + Al(a)x(t-T) + A2(a)x(t-r) + Bu(t)
where q = (a,T) is the parameter to be chosen from some given set ~ . (The proofs
320
in this case are slightly more complicated and details will be given in a forth-
coming paper.)
In the identification problems one may, in place of seeking minimizers for (6) -- --N
and (8), seek maximum likelihood estimates for q, q respectively in solving the -N
estimation problems. In the MLE algorithm, in addition to selecting q , one also
seeks a weighting matrix W N to maximize a "likelihood function". In the ease of a
scalar output (such as we have in Example 2) the MLE for ~N turns out to he identical
to the minimizer for (8) for any W = W N.
Finally, while we have discussed only first-order splines here, a careful inspec-
tion of the necessary convergence arguments (see [3]) will reveal that the approxima-
tion ideas can be carried out using subspaces Z N constructed with splines of arbitrary
order. Of course at some level one reaches a break-even point between the increased
accuracy obtained and the increased complexity of the computational efforts necessita-
ted by using higher-order splines (see the preliminary findings in [3]).
Acknowledgement: We wish to thank P. Daniel for assistance with some of the computa-
tions reported in this note.
References
[i] H. T. Banks, Approximation of nonlinear functional differential equation control systems, J. Opt. Theory Appl., to appear.
[2] H. T. Banks and J. A. Burns, Hereditary control problems: numerical methods based on averaging approximations, SIAM J. Control Opt. 16 (1978), 169-208.
[3] H. T. Banks and F. Kappel, Spline approximations for functional differential equations, to appear.
[4] E. M. Cliff and J. A. Burns, Parameter identification for linear hereditary systems via an approximation technique, March, 1978; to appear in Proc. Worksho~ on the Linkage between Applied Mathematics and Industry (Monterey, Calif., 1978),
[5] A. Pazy, Semi~roups of Linear Operators and Applications to Partial Differential Equatisns, Math. Dept. Lecture Notes, vol. 10, University of Maryland~ College Park, 1974.
STABILIZATION OF BOUNDARY CONTROL SYSTEMS
J.Zabczyk
Institute of Mathematics
Polish Academy of Sciences
Sniadeckich 8, 00-950 Warsaw
Poland
I. Introduction. Let ~ be an open subset of R n with the boundary
F = ~. A typical boundary control system can be described by an
equation of the form
~z {I) ~(t,x) = Lz(t,x) + Bu(t,x) , t >O, x en
z(O,x) = Zo(X) , x 6~, initial condition
~z(t,x) = Cu(t,x), t >0, x 6F
In (I), T denotes a linear boundary operator which transforms
functions defined on ~ onto functions defined on the boundary, L is
the basic operator with partial derivatives and B,C are operators
from the space of control parameters U into spaces of functions
defined respectively on ~ had F. It is natural to call system (I)
stabilizable if there exists a feedbek law u = Kz such that all
solutions of the equation:
~z (2) ~-6(t,x) = ( L + BK) z(t,x), t>O, x En
z(O,x) = z (x), x 6e o
Tz(t,x) = CKz(t,x) , t>O, x6F
tend to zero as t ,+~. If the operator C is identically zero
then the stabilization problem reduces to the classical one with
distributed controls only. The specific feature of the problem
considered here is that different feedbacks correspond to different
boundary conditions.
In this note we study the question of stabilization of system (1)
322
in the framework of the semigroup theory. In section 2 we recall an
abstract definition of a boundary control system and we specify the
class of admissible feedbacks. We give also a semigroup description of
some parabolic and hyperbolic systems and delay systems with delays in
control. In Section 3 we prove a "reduction" theorem which reduces the
stabilization problem of the system (1) to the stabilization problem
of a new system for which C ~O. In Section 4 we show some applications.
For instance we prove that the approximate controllability of a hyper-
bolic system implies its stabilization. Some related problems as well
as a discussion of some more general questions are given in Section 5.
There exists an extensive literature devoted to the asymptotic
properties of the solutions of the equation (2) ,(usually with unbounded
operators K, the case not treated in this note).Let us mention for
instance papers [10] and [12] and references there. Stabilization via a
controllability argument was obtained, for a class of hyperbolic
boundary systems, by M.Slemrod [13].
An earlier version of Section 3 appeard in the report [9] by
A.J.Pritchard and the present author and in [19]
2. Basic definitions and examples. Let A be the infinitesimal
generator of a Co-semigrou p S(t), t ~O on a Hilbert space Z and
let F and B be bounded operators from a Hilbert space U into Z.
An abstract boundary control system can be written in the form:
(3)
z(t) = A(z(t) - Fu(t)) + Bu(t)
z(0) = z ez, o
where u(-) is an admissible control law. A function u(.) is called
a strong admissible law if it is a twice differentiable function and
Fu(O) -z ° 6 ~(A) . For such controls there exists a unique solution
of (3) and is given by the formula
t t (4) z(t) =S(t) Zo+ f S(t-s)Bu(s)ds-A(f S(t -s)Fu(s)ds)
o o
A locally integrable function u(.) is called an admissible law if
the formula (4) defines a continuous function, see [1], [3] and [17]
A bounded operator K from Z into U is called an admissible
feedback if the operator ~:
323
-1 ~ (A) (5) A k = A(I -FK) +BK, ~(A K) = (I -FK)
generates a Co-semigrou p. System (3) is exponentially, (strongly,
weakly), stabilizable if and only if there exists an admissible
feedback such that the corresponding semigroup SK(t ) :
%t SK(t) = e , t_> 0
tends exponentially, (strongly, weakly), to zero as t
Let us consider now some examples.
) +~0.
Example I. (Delay equations). Let
o o (6) y(t) = I N(ds)y(t +s) + ~ M(ds)v(t +s) , t >O,
-h -h
where v(-) is a control function and N(.) and M(.) are functions
of bounded variations taking values in L(Rn,R n) and L(Rm,R m)
respectively. Considering the segment vt(.) of v(.) as a new state
variable satisfying the equation
(7) vt = Dvt
where D is the generator of the left shift semigroup on L2(-h,O;R m)
we can transform (6) into a boundary control system with control
u(t) = vt(O), see [4] and [18]. The appropriate state space is
Z = R n x L2(-h,O;R m) × L2(-h,O;R n) and the generator A has to be
defined as
(8) A =
o o S N(ds)~(S) + S M(ds)~(S) -h -h
d~ ds
ds
(9) D(A) = ;
6H 1 (-h,O;R n) , ~(O)
Moreover F : R n ~ Z and
324
Fu(s) = for all s E [-h,O] r -
In this case ~ = (-h,O) and the "active" boundary is F = {O}. Admis-
sible feedbacks are given in an implicit way:
o o (10) v(t) = S n(s)y(t +s)ds + ] m(s)v(t +s)ds, t>O,
-h -h
where n(-) and m(') are L2-matrix valued functions, compare [6].
Example 2. (Hyperbolic equations). The following example is taken from
[13]. Let ~ be a bounded, open, connected domain in R n whose
boundary F is an analytic surface and consider the boundary value
problem:
n ~ ~z) (11) ~2z(t,x) = ~ ~x (ai,j(x) x~ ' (t,x), t >O, x E~,
~t 2 i,j=1 ] 1
Tz(t,x) = u(t,x) , t >O, X EF,
~z tO x~ z(O,x) = z 1(x), -~-~, , , = z 1(x) , xE~.
where the boundary operator T is of the form:
n Tz(x) = Z ai, (x) ~z
i,j=1 J ~i 9i(x)' x EF
and 9(x) = (91 (x) ,...,Vn(X)) is the outward unit normal to F at
x 6F. The analytic functions a. .(x), i,j =1,...,n, x 6Q are such 1,3
= >O that ai,j (x) aj ,i(x) and for 6 0
n n 2
a i (x) ~i~j > 6 ~ ~i ' i,j=1 'J -- o i=I
for all ~ = (~I ..... ~ ) ERn' in an open set which includes ~, Let n
Z denotes the space H 1(~) ×L2(~) module the zero energy states
endowed with the inner product:
n ~Zl 8Zl (12) ((z1'z2) ' (Zl"Z2) ) = f ( ~ ai + ~ dx i,j=1 'J ~ ~ z2z2)
Let F be a bounded operator from H I/2(F) into H 2(~) such that if
w = ~g then T(w) = g on F. Such an operator always exists. We
define Fg = (Fg,O), g EH2(~). The following operator A:
325
(13) O,I ] n 3
• where ~ = Z ~ (a i A = L ,O i,j= I ~xj ,9 ~--~i )
with the domain ~ (A) :
~(A) = {(z1•z 2) EH2(Q) ×HI(Q) , $(z I) = O}•
generates a C -semigroup on Z. Let U = HI/2(F) • then the system o
(11) c a n b e r e p r e s e n t e d i n t h e f o r m (3) w h e r e B i s g i v e n by
(14) BU = (O• LFu)
and therefore is abounded operator. In the considered case admissible
feedbacks are bounded operators f~om Z into HI/2(F) .
Example 3, (Parabolic equations). In an analogous way one can represent
in the form (3) the following parabolic evolution equation:
~z ~--~ (t,x) = Lz(t,x) t >O, x 6Q
(15) Tz(t,x) = u(t,x) t >O• X 6F
z(O•x) = Zo(X) x E
with operators L and T defined as in Example 2. In the present case
one has additionally define Z = L2(~), F = F and B = 5F.
3. Necessary and sufficient conditions for stabilizability. Let us
introduce the following control system:
^ ^
(16) z = Az + u
where ; = RIB + (I -IRI)F and R 1 denotes the resolvent operator
of the generator A for some regular I. We call this new system (16),
the projection of (3). Sufficient conditions for stabilizability of
system (3) are based on the following theorem.
Theorem I. If the projection (16) is exponentially (strongly• weakly) A
stabilizable by a feedback law K• then system (3) is exponentially
(strongly, weakly) stabilizable by the feedback K = K R 1 .
326
Proof. Let St(t) , t~O be the semigroup on Z generated by ~A
A 1 = A +BK and S2(t) , t ~O its restriction to the space
Z O = ~(A I) = ~ (A) endowed with the graph norm. The infinitesimal
generator A 2 of $2(-) is equal to the operator A I restricted to
(A 2) = {z 6Zo; Az +~z EZo}. We claim that the semigroup T(t) , t~0,
defined as
(17) T(t) = R~Is2(t) R
has generator identical with A K. Let A 3 denotes the generator of
T(t) , t kO, then A 3 = R~IA2R and therefore:
(A 3) = {z : (A + (I ~kRk)FK +RAB~)RAz 6 ~(A) }
A
= {z : ARkZ +FKRkz E ~ (A) }
= ~(AK).
Moreover; for z 6~(A2)
R k~R~Iz = R BKz +R k(A(I -FKR ))R~Iz
= A2z
and consequently AK=A 3 .
Let III'III denotes the graph norm on Z : o
I I Izl l l 2 - II z I12 + II A lZ l l 2 , ~ ~Zo
Then
Sfls2ct)zlll 2 = lls1(t)z It 2 ÷ lIA1s1(t)z tl 2 = lls1(t~z I{ 2
+ IISI(t)AI z II 2 <liSt(t) 112 lllzlll 2
Consequently, if the semigroup Sl(t), t~O is exponentially or
strongly stable the same is true for the semigroup S2(t) , t~O, and
thus also for T(.). If for z,z 6Z, <S1(t) z,z >--~O as t --~+~
then for some M>O and all t ~O, IIs1(t) II ! M. The same estimate is
true for the semigroups $2(.) and T(.) with possibly different
constant M. Moreover for all z CZ and ~ 6~(A ) we have:
327
<T(t)z,z >= <SI(t)Rxz, (X-A)~z >
and thus <T(t)z,z> ---+O as t---~+~. Since the semigroup T(-)
bounded and the set ~(A ~) is dense in Z therefore T(t)z ---+0
weakly as t --~+ ~ and the proof of the theorem is complete.
We show now that under some conditions Theorem I allows a converse.
We start with the following Lemma.
is
Lemma I. Let T(t), t >O be the C -semigroup generated by the -- o
operator ~ = A(I -FK) + BK and let
z(t) = T(t) z ° , u(t) = Kz(t) , z o 6Z.
Then
t (18) Rkz(t ) = S(t) Rkz ° + f S(t-s)Bu(s)ds, t_>O.
o
Proof. It is sufficient to show that (18) holds for all z belonging o
to the dense set Z I =~((A(I -FK))2). If z o 6Z I then u(.) is a
twice continuousely differentiable function and z O -Fu(O) 6~(A) •
Therefore there exists a unique solution z(t), t ~O, z(O) = z O of
the equation:
(19) ~(t) = A(~(t) - Fu(t)) + Bu(t)
and it is given by the formula:
t t ~(t) = S(t) z + ; S(t-s)Bu(s)ds - A( 5 S(t-s)Fu(s)ds) .
o o o
On the other hand z(t) 6 ~(A(I -FK)) , t >O and
(20) z(t) = A(I-FK)z(t) + BKz(t)
= A(z(t) -Fu(t)) +Bu(t) .
Taking into account equation (19) and (20) we obtain that
d d-~(z(t) - z(t))= A(z(t) - z(t))
and therefore z(t) = z(t) identically for all t >O. Finally
328
t A
Rkz(t) = Rxz(t) = S(t)RIz ° + f S(t-s)Bu(s)ds. O
The Proposition I below can be considered as a partial converse to
Theorem I.
Proposition I. Let Us assume that the system (3) is exponentially, A
(strongly, weakly), stabilizable. Then for every z o 6~(A) there
exists a continuous function u(.) such that for the corresponding
solution z(.) of (16) and t ---~+
A
Iz(t) } ---+O, lu(t) I --+O exponentially,
(z(t) --~O, u(t) ---+0 strongly)
A
z(t) ---~O, u(t) ---~O weakly) .
A
Proof. If z o 6 ~(A) then z(t) = T(t) (l -A)~ O is well defined. If A
we define u(t) = Kz(t) , then z(t) = Rxz(t) by Lemma I and the
t h e o r e m f o l l o w s .
F o r e x p o n e n t i a l s t a b i l i z a b i l i t y we h a v e a s t r o n g e r r e s u l t :
Theorem 2. Let us assume that for every z o 6Z there exists t >0
such that S(t}z O 6 ~(A). Then exponential stabilizability of (3) is
equivalent to the exponential stabilizability of (16).
Proof. Let us assume that system (3) is exponentially stabilizable let A
z ° 6Z and t o >0 be such that z ° = S(to) Z o 6 ~(A). By virtue of
Proposition IIthe control ~(-), 5(t) = 0 for t <t o and
~(t) = u(t -t o ) tends exponentially to zero as well as the corresponding
solution z(.) . But this is a sufficient condition for existence of A
a stabilizing feedback K for the system (16), see [9].
Corollary I. If S(t), t ~0 is a differentiable semigroup then
S(t) Z E ~(A) for all t >O and therefore the assumption of Proposition
I is satisfied. Therefore the case of parabolic systems is coverd by
Proposition I.
4. Applications. System (3) is said to be approximately controllable
if the set of all states reachable from O, by means of strongly
admissible laws, is dense in Z. It is known, see [3] and [13], that
329
the approximate controllability of (3) is equivalent to the approximate
controllability of (16). The following theorem is now a corollary of
Theorem I.
Theorem 3. If the system (3) is approximately controllable and the
operator A generates a contraction semigroup then the system (3) is
weakly stabilizable. If in addition, A has compact resolvent or
generates a compact semigroup then the system (3) is respectively
asymptoti~ally or exponentially stabilizable.
Proof. The projection (16) is approximately controllable, and the
theorem follows from the corresponding result for systems with distri-
buted controls only (F EO) obtained by C.Benchimol [2].
Corollary 2. Assume that A = -A ~, then the operator A generates a
contraction semigroup and Theorem 3 can be applied, In particular the
hyperbolic system described in the Example 2 is strongly stabilizable.
This is because the correspondinq generator A generates a contraction
group on Z with the compact resolvent. Moreover the controllability
result of Russell [11] implies the approximate controllability of (11).
The result just stated is almost identical with an earlier
result of M.Slemrod [13]. The main difference consists in the fact
that the class of admissible feedbacks in [13] contained also some
unbounded operators. In fact the stabilizing feedback used in [13] was
of the form:
~Z u(t,x) =- e~-c(t,x), t >O, x e?
4u
for an e >O sufficiently small.
Theorem 2 can be applied to delay systems of Example I, because for
t >O sufficiently larg S(t) Z 6 ~(A), see [19]. However it is not
easy to obtain necessary and sufficient conditions for exponential
stabilizability of the system in terms of functions N(.) and M(-)
only. For a different approach we refere to A.Olbrot [8]. Some
applications of Theorem 2 to parabolic equations were recently obtained
by R.Triggiani [14].
Let us finally remark that the projection (16) has an extremely
simple form if O 6p(A) and B E O. In this case:
• A A z = Az + FU
330
5. Final remarks. The stabilization problem considered in this note
is a special case of a general stabilization problem with partial
observation:
y = Gz,
where G can be even an unbounded operator. In general, even for a
finite dimensional sytem:
= Az + Bu
y = Gz
the observability of the pair (A,G) and controllability of (A,B)
are not sufficient for the existence of a stabilizing feedback u =Ky,
see e.g. [5]. But weaker conditions, detectability of (A,G) and
stabilizability of (A,B) allow to use the Luenberger observer
z = Az +Bu +K(Cz -y)
and stabilize the pair (z(-),z(.)), see [16]. This classical result
generalizes easily to infinite-dimensional case. But its generalization
to boundary control systems is not clear. Some explicit conditions for
detectability of (A,G) where A is an elliptic operator and G an
unbounded operator of the trace type were recently obtained by Nambu
[7] and R.Triggiani [15].
Theorem 3 of this paper rises the following question: assume that
for each z o 6 ~(A), there exists a control u(.) such that the
corresponding solution of (16) and u(.) tend to zero exponentially.
Does this property implies exponential stabilizability for all initial
conditions z ° 6Z ? In this direction, for uncontrolled system, (B z0),
we can prove the following proposition, see [19]:
Proposition 2. If for every z 6 ~ (A) ; S(t) z --~0 exponentially
then
sup{Rel; ~ 6g(A) } <0.
331
References
[I] A.V.Balakrishnan, Identification and stochastic control of a
class of distributed systems with boundary noise, in Control
Theory, Numerical Methods and Computer Systems Modelling,
A.Bensoussan and T.L.Lions, eds. New York: Springer Verlag,
1975, 163-178.
[2] C.Benchimol. A note on weak stabilizability of contra~tion semi-
groups, SIAM J.Control, 16(1978), No. 3, 373-379.
[3] M.Fattorini, Boundary control systems, SIAM J.Control, 6(1968).
[4] A.Ichikawa, Evolution equations with delay, Control Theory
Centre Report 52, University of Warwick, April 1977.
[5] H.Kimura, Pole assignment is linear multivariable systems using
output Feedback, IEEE Trans. on AC, Vol. AC 20, 1975, 509-516.
[6] H.N.Koivo and E.B.Lee, Controller synthesis for linear systems
with retared state and control variable and quadratic cost,
Automatica, 8(1972), 203-208.
[7] J.Nambu, Feedback stabilization for distributed parameter
systems of parabolic type, manuscript.
[8] A.Olbrot, Stabilizability, detectability spectrum assignment for
linear systems with general time delays, Report CRM-712, CRM
Universit~ de Montreal, 1977.
[9] A.J.Pritchard and J.Zabczyk, Stability and stabilizability of
infinite dimensional systems, Control Theory Report 70, University
of Warwick, November 1977.
[10] J.Rauch and M.Taylor, Exponential decay of solutions to hyperbolic
equations in bounded domains, Indiana Univ. Math. J. 24(1974),
49-86.
[11] D.L.Russell, Boundary control of the higher dimensional wave
equation, SIAM J.Control, 9(1971), 29-42.
[12] J.P.Quinn and D.L.Russell, Asymptotic stability and energy decay
rates for solutions of hyperbolic equations with boundary domping,
Proceedings of the Royal Society of Edinburgh, 77A (1977), 97-127.
[13] M.Slemrod, Stabilization of boundary control systems, J.Diff.
Eqns. 22(1976), 402-415.
[14] R.Triggiani, manuscript.
[15] R.Tri99iani , On Nambu's boundary stabilizability problem for
diffusion processes, manuscript.
[16] W.A.Wolovich, Linear multivariable systems, Springer Verlag,
New York, 1974.
332
[17] J.Zabczyk, A semigroup approach to boundary value control,
Proceedings of the 2nd Symp. on Control of Distributed Systems,
University of Warwick, England, 1975, Springer-Verlag, to
appear.
[18] J.Zabczyk, On systems with delays in control, Report CRM-777,
CRM Universit~ de Mont~ral, March 1978.
[19] J.Zabczyk, On stabilization of boundary control systems,
Report 785, Centre de recherches math~matiques Universit~ de
Montreal, March 1978.
![)r » e M€¦ · )r » e M $ ( ç ô º v ¥>& >' ì (>| ì (² ( 1 $ ( 0Y § $ (6×,11 j $ (6× U $ (0Y ] $](https://static.fdocuments.net/doc/165x107/5fc961380bb304414474417a/r-e-m-r-e-m-v-1-0y.jpg)
