The Open-Loop Linear Quadratic Differential GameRevisited

Jacob EngwerdaTilburg University

Dept. of Econometrics and O.R.P.O. Box: 90153, 5000 LE Tilburg, The Netherlands

e-mail: engwerda@uvt.nl

February, 2004

1

Abstract: In this paper we consider the indefinite open-loop Nash linear quadratic differentialgame with an infinite planning horizon. We derive both necessary and sufficient conditions for theexistence of equilibria. Moreover, we present a sufficient condition under which the game will havea unique equilibrium. (Some) results are elaborated for the zero-sum game and the scalar case.

Keywords: linear-quadratic games, open-loop Nash equilibrium, solvability conditions, Riccatiequations.

1 Introduction

In the last decades, there is an increased interest in studying diverse problems in economics usingdynamic game theory. In particular in environmental economics and macroeconomic policy coordi-nation, dynamic games are a natural framework to model policy coordination problems (see e.g. thebooks and references in Dockner et al. [5] and Engwerda [10]). In these problems, the open-loopNash strategy is often used as one of the benchmarks to evaluate outcomes of the game. Other,frequently used, benchmarks are the feedback Nash and Stackelberg strategy.Inspired by the analysis and results obtained by Abou-Kandil et al. [1, 2], in Engwerda [8, 9]

several aspects of open-loop Nash equilibria of the standard linear-quadratic differential game arestudied, as considered by Starr and Ho in [23] (see, e.g., also Lukes et al. [17], Meyer [19], Eisele[7], Haurie and Leitmann [12], Feucht [11], Weeren [24] and Basar and Olsder [4]). In particular,a detailed study of the infinite-planning horizon is given in these papers and it is shown that, ingeneral, the infinite-planning horizon problem does not have a unique equilibrium. In Kremer [13]this problem was reconsidered using a functional analysis approach. One of the results he shows isthat whenever a stable system has more than one equilibrium, there will exist an infinite number ofequilibria.In this paper we will reconsider the infinite-planning horizon linear quadratic differential game.

In section 2 we generalize the results obtained by Engwerda [8, 9] and Kremer [13]. Moreover, weelaborate extensively the scalar case in section 4 and discuss some implications for the zero-sum gamein section 3.In this paper we assume that the performance criterion player i = 1, 2 likes to minimize is:

limT→∞

Ji(x0, u1, u2, T ) (1)

where

Ji =

Z T

0

{xT (t)Qix(t) + uTi (t)Riui(t)}dt,

subject to the linear dynamic state equation

x(t) = Ax(t) +B1u1(t) +B2u2(t), x(0) = x0, (2)

Here, the matrices Qi and Ri are symmetric and Ri are, moreover, assumed to be positive definite,i = 1, 2. Notice that we do not make any definiteness assumptions w.r.t. matrix Q.We assume that the matrix pairs (A,Bi), i = 1, 2, are stabilizable. So, in principle, each player

is capable to stabilize the system on his own.

1

The inclusion of player j0s control efforts into player i0s cost function is not present here becausethis term drops out in the analysis, which is due to the open-loop information structure we considerin this paper. So, for convenience, we drop this term from the outset.The open-loop information structure of the game means that we assume that both players only

know the initial state of the system and that the set of admissible control actions are functions oftime, where time runs from zero to infinity. Since we only like to consider those outcomes of thegame that yield a finite cost to both players, we restrict ourselves to functions belonging to the set

Us(x0) =nu ∈ L2,loc|Ji(x0, u) exists in IR ∪ {−∞,∞}, lim

t→∞x(t) = 0

o,

where L2,loc is the set of locally square-integrable functions, i.e.,

L2,loc = {u[0,∞)|∀T > 0,Z T

0

uT (s)u(s)ds <∞}.

Notice that Us(x0) depends on the inital state of the system. For simplicity of notation we omit,however, this dependency. Another set of functions we consider is the class of locally square integrablefunctions which exponentially converge to zero when t → ∞, Le2,loc. That is, for every c(.) ∈ Le2,locthere exist strictly positive constants M and α such that

|c(t)| ≤Me−αt.Finally, notice that the restriction to this set of control functions requires some form of communicationbetween the players.

2 General Case

We start our analysis with some results on the regular linear quadratic optimal control problem.The next theorem generalizes a property that can be found in Willems [?] under a controllabilityassumption.

Theorem 2.1 Assume (A,B) stabilizable, Q symmetric and R > 0. Consider the minimization ofthe linear quadratic cost function Z ∞

0

xT (s)Qx(s) + uT (s)Ru(s)ds (3)

subject to the state dynamics

x(t) = Ax(t) +Bu(t), x(0) = x0, (4)

and u ∈ Us(x0). Then, the linear quadratic problem (3,4) has a solution for all x0 ∈ IRn if and onlyif the algebraic Riccati equation

0 = −ATK −KA+KSK −Q (5)

has a symmetric solution Ks such that A− SKs is stable1. Moreover, if the linear quadratic control

problem has a solution, then the optimal control in feedback form is

u∗(t) = −R−1BTKsx(t).

1Ks is then called a stabilizing solution of the Riccati equation (5)

2

Proof: ”⇐ part” This part follows from a standard completion of squares argument.”⇒ part” Since the problem has a minimum for all x0 it can be shown (similar to e.g. Molinari[20]) that

J∗(t, x0) := minu∈Us

Z ∞

t

xT (s)Qx(s) + uT (s)Ru(s)ds

subject to the state dynamics (4) exists for all x0 and that this is a quadratic form, i.e.

J∗(t, x0) = xT0K(t)x0.

Moreover, it can be shown that ∂J∗

∂x(t, x) exists and is continuous and also ∂J∗

∂t(t, x) exists. So, we can

use the theorem of dynamic programming to conclude that J∗(t, x) is in fact independent of time tand satisfies the Hamilton-Jacobi-Bellman differential equation (see e.g. Weeren [24], Lemma A.1)

0 = minu∈Us

{xTQx+ uTRu+ ∂V∂x(x)(Ax+Bu)}. (6)

where

u∗(t, x) = argminu∈Us

{xTQx+ uTRu+ ∂V∂x(x)(Ax+Bu)}. (7)

From (7) it follows that u∗(t) = −R−1BTKx(t), where (see 6) K satisfies the equation

0 = xT (ATK +KA−KSK +Q)x.However, since this equation holds for an arbitrary state x, it follows that K satisfies the algebraicRiccati equation (5). Moreover, we know that for an arbitrary state x(t) → 0 if t → ∞. So, weconclude that K is a stabilizing solution of (5). ¤

A proof of the next preliminary theorem is provided in the Appendix.

Theorem 2.2 Let cx0(.) ∈ Le2,loc,, (A,B) stabilizable, Q symmetric and R > 0. Consider the mini-mization of the linear quadratic cost function (3) subject to the state dynamics

x(t) = Ax(t) +Bu(t) + cx0(t), x(0) = x0, (8)

and u ∈ Us(x0). Then, the linear quadratic problem (3,8) has a solution for all x0 ∈ IRn if and onlyif the algebraic Riccati equation (5) has a symmetric stabilizing solution K(.). Moreover, if the linearquadratic control problem has a solution, then the optimal control in feedback form is

u∗(t) = −R−1BT (Kx(t) +m(t)).Here m(t) is given by

m(t) =

Z ∞

t

e−(A−SK)T (t−s)Kc(s)ds, (9)

and x(t) is the through this optimal control implied solution of the differential equation

x(t) = (A− SK)x(t)− Sm(t) + c(t), x(0) = x0.

3

With the help of this theorem we prove in the Appendix the next result.

Theorem 2.3 Consider matrix

M =

A −S1 −S2−Q1 −AT 0−Q2 0 −AT

. (10)

If the linear quadratic differential game (1,2) has an open-loop Nash equilibrium for every initialstate, then

1. M has at least n stable eigenvalues (counted with algebraic multiplicities). More in particular,there exists a p-dimensional stable M-invariant subspace S, with p ≥ n, such that

Im

IV1V2

⊂ S,for some Vi ∈ IRn×n.

2. the two algebraic Riccati equations,

Qi +ATKi +KiA−KiSiKi = 0, (11)

have a symmetric stabilizing solution Ki, i = 1, 2.

Conversely, if the two algebraic Riccati equations (11) have a stabilizing solution and vT (t) =:[xT (t),ψT1 (t),ψ

T2 (t)] is an asymptotically stable solution of

v(t) =Mv(t), x(0) = x0,

then,

u∗i := −R−1i BTi ψi(t), i = 1, 2,

provides an open-loop Nash equilibrium for the linear quadratic differential game (1,2). ¤

Remark 2.4 From this theorem we can draw a number of conlusions concerning the existenceof open-loop Nash equilibria. A general conclusion is that this number depends critically on theeigenstructure of matrix M . We will distinguish some cases. To that end, let s denote the number(counting algebraic multiplicities) of stable eigenvalues of M and assume that the Riccati equations(11) have a stabilizing solution.

1. If s < n, still for some initial state there may exist an open-loop Nash equilibrium. Consider,e.g., the case that s = 1. Then, for every x0 ∈ Span [I, 0, 0]v, where v is an eigenvectorcorresponding with the stable eigenvalue, the game has a Nash equilibrium.

4

2. In case s ≥ 2, the situation might arise that for some initial states there exists an infinitenumber of equilibria. A situation in which there are an infinite number of Nash equilibriumactions occurs if, e.g., v1 and v2 are two independent eigenvectors in the stable subspace of Mfor which [I, 0, 0]v1 = µ[I, 0, 0]v2, for some scalar µ. In such a situation,

x0 = λ[I, 0, 0]v1 + (1− λ)µ[I, 0, 0]v2,for an arbitrary scalar λ ∈ IR. The resulting equilibrium control actions, however, differ foreach λ (apart from some exceptional cases).

3. In case matrixM has a stable graph subspace2, S, of dimension s > n, for every initial state x0there exists, generically, an infinite number of open-loop Nash equilibria. For, let {b1, · · · , bs}be a basis for S. Denote di := [I, 0, 0]bi, and assume (without loss of generality) thatSpan[d1, · · · , dn] = IRn. Then, dn+1 = λ1d1 + · · · ,λndn for some λi, i = 1, · · · , n. Elementarycalculations show then that every x0 can be written as both x0,1 = α1d1 + · · · + αndn and asx0,2 = β1d1 + · · · + βn+1dn+1, where βn+1 6= 0. So, x0 = λx0,1 + (1 − λ)x0,2 for an arbitrarychoice of the scalar λ. Since the vectors {b1, · · · , bn+1} are linearly independent, it follows thatfor every λ, the vector b(λ) := λ(α1b1 + · · · + αnbn) + (1 − λ)(β1b1 + · · · + βn+1bn+1) differs.So, generically, the corresponding equilibrium control actions induced by this vector b(λ) (viav(t) =Mv(t), v(0) = b(λ)) will differ.

¤

It is tempting to believe that, under the conditions posed in the above Theorem 2.3, matrix M willalways have an n-dimensionalM-invariant graph subspace S. In case the equilibrium control actionsallow for a feedback synthesis3, this claim is correct. The proof, basically, can be given along thelines of the proof of Theorem 7 in Engwerda [9] and is therefore omitted.

Theorem 2.5 Assume the linear quadratic differential game (1,2) has an open-loop Nash equilibriumfor every initial state, and the equilibrium control actions allow for a feedback synthesis. Then,

1. M has at least n stable eigenvalues (counted with algebraic multiplicities). More in particular,for each Nash equilibrium there exists a uniquely determined n-dimensional stable M-invariantsubspace

Im

IV1V2

for some Vi ∈ IRn×n.

2. the two algebraic Riccati equations,

Qi +ATKi +KiA−KiSiKi = 0, (12)

have a symmetric solution Ki(.) such that A− SiKi is stable, i = 1, 2. ¤

2A subspace S is called a graph subspace if S = Im

·X1

X2

¸, with X1 ∈ IRn×n invertible.

3That is, the closed-loop dynamics of the game can be described by: x(t) = Fx(t); x(0) = x0 for some constantmatrix F .

5

From Engwerda [8], Theorem 6, we know that Theorem 2.5, point 1., implies that the set of coupledalgebraic Riccati equations

0 = ATP1 + P1A+Q1 − P1S1P1 − P1S2P2; (13)

0 = ATP2 + P2A+Q2 − P2S2P2 − P2S1P1 (14)

has a set of solutions Pi, i = 1, 2, such that A − S1P1 − S2P2 is stable4. The next theorem iswell-known in case the matrices Qi are assumed to be positive definite and shows that the conversestatement of this result always holds. It is, however, not difficult to see that its proof as stated e.g.in Basar and Olsder [4], Theorem 6.22, can be straightforwardly generalized (see Engwerda [10] fora detailed exposition). That is,

Theorem 2.6 Assume that

1. the set of coupled algebraic Riccati equations (13,14) has a set of stabilizing solutions Pi, i =1, 2; and

2. the two algebraic Riccati equations,

0 = ATKi +KiA−KiSiKi +Qi, (15)

have a symmetric solution Ki(.) such that A− SiKi is stable, i = 1, 2.

Then the linear quadratic differential game (1,2) has an open-loop Nash equilibrium for every initialstate.Moreover, a set of equilibrium actions is provided by:

u∗i (t) = −R−1i BTi PiΦ(t, 0)x0, i = 1, 2. (16)

Here Φ(t, 0) satisfies the transition equation

Φ(t, 0) = (A− S1P1 − S2P2)Φ(t, 0); Φ(t, t) = I

¤

As an immediate consequence we have that

Corollary 2.7 if matrix M has a stable invariant graph subspace and the two algebraic Riccatiequations (15) have a stabilizing solution, then the game will have an open-loop Nash equilibriumthat permits a feedback synthesis for every initial state. ¤

Combining the results of both previous theorems we have the following generalization of the resultas stated in Engwerda [9], Theorem 7. The correctness on the statement concerning the involvedcost for the players below can be found, e.g., in Engwerda [8], Remark 13.2.

Corollary 2.8 The infinite-planning horizon two-player linear quadratic differential game (1,2) hasfor every initial state an open-loop Nash set of equilibrium actions (u∗1, u

∗2) which permit a feedback

synthesis if and only if

4(P1, P2) are called a set of stabilizing solutions of (13,14).

6

1. there exist P1 and P2 that are solutions of the set of coupled algebraic Riccati equations (13,14)satisfying the additional constraint that the eigenvalues of Acl := A−S1P1−S2P2 are all situatedin the left half complex plane, and

2. the two algebraic Riccati equations (15) have a symmetric solution Ki(.) such that A− SiKi isstable, i = 1, 2.

If (P1, P2) is a set of stabilizing solutions of the coupled algebraic Riccati equations (13,14), theactions

u∗i (t) = −R−1i BTi PiΦ(t, 0)x0, i = 1, 2,

where Φ(t, 0) satisfies the transition equation Φ(t, 0) = AclΦ(t, 0); Φ(0, 0) = I, yield an open-loopNash equilibrium.The costs, by using these actions, for the players are

xT0Mix0, i = 1, 2, (17)

where Mi is the unique solution of the Lyapunov equation

ATclMi +MiAcl +Qi + PTi SiPi = 0, (18)

We stress here once again the point that in case for every initial state there exists more than oneopen-loop Nash equilibrium that permits a feedback synthesis then the stable subspace has a dimen-sion larger than n. Consequently (see Remark 2.4, item 3) for every initial state there will exist,generically, an infinite number of open-loop Nash equilibria. This, even though there exist only afinite number of open-loop Nash equilibria permitting a feedback synthesis. This point was firstnoted by Kremer in [13] in case matrix A is stable.The above reflections raise the question what happens in case the dimension of the stable subspace

is n. The next Theorem 2.9 provides a sufficient condition to conclude that the game has a uniqueopen-loop Nash equilibrium. Moreover, it shows that in that case the unique equilibrium actions canbe synthesized as a state feedback. The proof of this theorem is provided in the Appendix.

Theorem 2.9 Assume that

1. matrix M has n stable eigenvalues and 2n eigenvalues that are not stable (counting algebraicmultiplicities);

2. the stable subspace is a graph subspace.

3. The two algebraic Riccati equations (15) have a stabilizing solution.

Then, for every initial state the game has a unique open-loop Nash equilibrium. Moreover, the uniqueequilibrium actions are given by (16). ¤

7

3 Zero-Sum Case

In this section we consider the special case of the zero-sum game. By specializing the above results tothis case, analogues are obtained for the open-loop zero-sum linear quadratic game. As an examplewe will elaborate the analogue of Corollary 2.8. The next proposition follows directly from thiscorollary.

Proposition 3.1 (Open-loop zero-sum differential game)Consider the differential game described by

x(t) = Ax(t) +B1u1(t) +B2u2(t), x(0) = x0, (19)

with, for player one, the quadratic cost functional:

J1(u1, u2) =

Z ∞

0

xT (t)Qx(t) + uT1 (t)R1u1(t)− uT2 (t)R2u2(t)dt, (20)

and, for player two, the opposite objective function

J2(u1, u2) = −J1(u1, u2);where the matrices Q and Ri, i = 1, 2, are symmetric. Moreover, assume that Ri, i = 1, 2, arepositive definite.Then, this infinite-planning horizon zero-sum linear quadratic differential game has for every initialstate an open-loop Nash equilibrium which permits a feedback synthesis if and only if the next twoconditions hold.

1. The coupled algebraic Riccati equations

ATP1 + P1A+Q− P1S1P1 − P1S2P2 = 0, (21)

ATP2 + P2A−Q− P2S2P2 − P2S1P1 = 0, (22)

has a set of solutions Pi, i = 1, 2, such that A− S1P1 − S2P2 is stable; and2. The two algebraic Riccati equations

ATK1 +K1A−K1S1K1 +Q = 0, (23)

ATK2 +K2A−K2S2K2 −Q = 0, (24)

have a symmetric solution Ki such that A− SiKi is stable, i = 1, 2.

Moreover, the corresponding equilibrium actions are

u∗1(t) = −R−11 BT1 P1x(t) and u∗2(t) = −R−12 BT2 P2x(t).Here x(t) satisfies the differential equation

x(t) = (A− S1P1 − S2P2)x(t); x(0) = x0.¤

8

Next, we consider a special case of this game: the case that matrix A is stable. Then, the conditionsunder which the game has an equilibrium can be simplified somewhat further.

Proposition 3.2 Consider the open-loop zero-sum differential game as described in Proposition 3.1.Assume, additionally, that matrix A is stable.Then, this game has for every initial state an open-loop Nash equilibrium which permits a feedbacksynthesis if and only if the next two conditions hold.

1. The coupled algebraic Riccati equation

ATP + PA+Q− P (S1 − S2)P = 0 (25)

has a (symmetric) solution P such that A− (S1 − S2)P is stable; and2. The two algebraic Riccati equations (23,24) have a symmetric solution Ki such that A− SiKi

is stable, i = 1, 2.

Moreover, the corresponding unique equilibrium actions are

u∗1(t) = −R−11 BT1 Px(t) and u∗2(t) = R−12 BT2 Px(t).

Here x(t) satisfies the differential equation

x(t) = (A− (S1 − S2)P )x(t); x(0) = x0.

The cost for player 1 are J1 := xT0 Px0 and for player 2, −J1.

Proof. According Proposition 3.1 this game has an open-loop Nash equilibrium which permitsa feedback synthesis for every initial state if and only if the next two coupled algebraic Riccatidifferential equations have a set of stabilizing solutions Pi, i = 1, 2,

ATP1 + P1A+Q− P1S1P1 − P1S2P2 = 0, (26)

ATP2 + P2A−Q− P2S2P2 − P2S1P1 = 0 (27)

and the two algebraic Riccati equations (23,24) have a stabilizing solution Ki. Adding (26) to (27)yields the next differential equation in (P1 + P2):

AT (P1 + P2) + (P1 + P2)(A− S1P1 − S2P2) = 0.

This is a Lyapunov equation of the form ATX+XBT = 0, withX := P1+P2 and B = A−S1P1−S2P2.Now, independent of the specification of (P1, P2), B is always stable. So, whatever Pi, i = 1, 2 are,this Lyapunov equation has a unique solution X(P1, P2). Obviously, P1 + P2 = 0 satisfies thisLyapunov equation. So, necessarily, we conclude that P1 = −P2. Substitution of this into (26) showsthen that (26,27) have a stabilizing solution Pi if and only if (25) has a stabilizing solution P .The corresponding equilibrium strategies follow then directly from Theorem 3.1. The symmetry anduniqueness property of P follow immediately from the fact that P is a stabilizing solution of anordinary Riccati equation (see, e.g., [26]).

9

Finally the cost for player 1 can be rewritten, using (25), as

J1 =

Z ∞

0

xT (t){Q+ PS1P − PS2P}x(t)dt

=

Z ∞

0

xT (t){2PS1P − 2PS2P − ATP − PA}x(t)dt

= −Z ∞

0

xT (t){(A− (S1 − S2)P )TP + P (A− (S1 − S2)P )}x(t)dt

= −Z ∞

0

d[xT (t)Px(t)]

dtdt

= xT0 Px0.

¤

Notice that if we assume additionally in the above zero-sum game that Q is positive definite, (23)always has a stabilizing solution (see, e.g., [26]). Moreover, by considering K := −K2, the fact that(24) should have a stabilizing solution can be rephrased as that the next Riccati equation shouldhave a solution K such that A+ S2K is stable:

ATK +KA+KS2K +Q = 0.

In this form one, usually, encounters one of the solvability conditions for existence of a controller inH∞ robust control design (see, e.g., [3]).A natural question that arises is whether the equilibria we presented in the above Propositions

will be unique. The next Lemma provides us with the answer.

Lemma 3.3 Consider the zero-sum differential game. Then

σ(M) = σ(−A) ∪ σµ·

A −S1 + S2−Q −AT

¸¶.

Proof: The result follows straightforwardly from the fact that

M − λI = S A− λI −S1 + S2 −S2

−Q −AT − λI 00 0 −AT − λI

S−1,where S =

I 0 00 I 00 −I I

. ¤

An immediate consequence of this result is

Corollary 3.4 1. If the conditions of Proposition 3.2 are satisfied, the game has a unique equi-librium. This follows from the fact that matrix M has an n-dimensional stable graph subspace.For, since all eigenvalues of −A are unstable, it follows from the above lemma that necessarilythe Hamiltonian matrix ·

A −S2 + S1−Q −AT

¸10

must have n stable eigenvalues. However, since the spectrum of an Hamiltonian matrix issymmetric w.r.t. the imaginary axis, we infer that this matrix must have n unstable eigenvaluesas well. So, we conclude that matrix M has n stable eigenvalues and 2n unstable eigenvalues.Theorem 2.9 yields then that, under the stated conditions, there is a unique open-loop Nashequilibrium.

2. If A is not stable, it may happen that the zero-sum game as described in Proposition 3.1 hasan infinite number of open-loop Nash equilibria. See, e.g., Theorem 4.4. ¤

4 Scalar Case

In this section we study the scalar case. To avoid the consideration of various peculiar cases, we willassume throughout that Bi 6= 0, i = 1, 2. To stress the fact that we are dealing with the scalar case,we will put the system parameters in lower case so, e.g., a instead of A.First, we calculate the exponential of matrix −M .

Lemma 4.1 Consider matrix M in (10). Let µ :=pa2 + s1q1 + s2q2

5. Then, the characteristicpolynomial of M is (−a − λ)(µ − λ)(−µ − λ). Moreover, an eigenvector of M corresponding withλ = −a is [0, −s2, s1]T . In case not both q1 = q2 = 0 and µ 6= 0 the exponential of matrix −M ,e−Ms, is given by

V

eµs 0 00 eas 00 0 e−µs

V −1. (28)

Here

V =

a− µ 0 a+ µ−q1 −s2 −q1−q2 s1 −q2

and its inverse

V −1 =1

detV

−(s1q1 + s2q2) −s1(a+ µ) −s2(a+ µ)0 −2q2µ 2q1µ

(s1q1 + s2q2) s1(a− µ) s2(a− µ)

,with the determinant of V , detV = 2µ(s1q1 + s2q2).

Proof. Straightforward multiplication shows that we can factorizeM asM = V diag(µ,−a,−µ)V −1.So, the exponential of matrix −M , e−Ms, is as stated above. ¤

Theorem 4.2 Assume that s1q1+ s2q2 6= 0. Then, the infinite-planning horizon game has for everyinitial state a Nash equilibrium if and only if the next conditions hold.

5Notice that µ can be a pure imaginary complex number.

11

1. a2 + s1q1 + s2q2 > 0,

2. a2 + siqi > 0, i = 1, 2.

Moreover, if there exists an equilibrium for every initial state, then there is exactly one equilibriumthat permits a feedback synthesis. The corresponding set of equilibrium actions is:

u∗i (t) = −biripix(t), i = 1, 2,

where p1 =(a+µ)q1

s1q1+s2q2, p2 =

(a+µ)q2

s1q1+s2q2, µ =

pa2 + s1q1 + s2q2 and x(t) = (a − s1p1 − s2p2)x(t), with

x(0) = x0.

The with these actions corresponding cost for player i are: Ji = x20qi+sip2

i

µ, i = 1, 2.

Proof. First assume that 1. and 2. hold. Let pi be as mentioned above. From point 2. it followsthat the algebraic Riccati equations

2aki − sik2i + qi = 0, i = 1, 2, (29)

have a stabilizing solution ki =a+√a2+siqi

si, i = 1, 2.

Moreover it is straightforwardly verified that, if a2 + s1q1 + s2q2 > 0, p1 and p2 satisfy the set ofcoupled algebraic Riccati equations

2ap1 + q1 − s1p21 − s2p1p2 = 0 and 2ap2 + q2 − s2p22 − s1p1p2 = 0,

whereas a− s1p1− s2p2 = −(a2+ s1q1+ s2q2) < 0. So, according Theorem 2.6, u∗i (.) provides a Nashequilibrium.Next consider the converse statement. From Theorem 2.3, point 2., we know that the algebraicRiccati equations (29) have a stabilizing solution ki, i = 1, 2. From this it follows immediately thatcondition 2. must hold. On the other hand, we have from Lemma 4.1 that if a2 + s1q1 + s2q2 ≤ 0,matrix M will not have a stable M -invariant subspace S such that

Im

1v1v2

⊂ S.So according to Theorem 2.3, part 1., in this case the game does not have for all initial states anopen-loop Nash equilibrium.Finally, since µ differs from −a, it follows straightforwardly from Lemma 4.1 that M has only onestable invariant graph subspace. From this (see Corollary 2.8) it is easily verified that (p1, p2) yieldsthe only Nash equilibrium that permits a feedback synthesis of the game. ¤

Corollary 4.3 Assume that qi > 0, i = 1, 2. Then, the infinite-horizon game has for every initialstate an open-loop Nash equilibrium. Furthermore, there is exactly one of them that permits a feedbacksynthesis. ¤

12

Using Theorem 2.9 we can precisely indicate the number of equilibria.

Theorem 4.4 Assume that

1. s1q1 + s2q2 6= 0;2. a2 + s1q1 + s2q2 > 0;

3. a2 + siqi > 0, i = 1, 2.

Then, the infinite-planning horizon game has for every initial state- a unique equilibrium if a ≤ 0.- an infinite number of equilibria if a > 0.

Proof. In case a ≤ 0 matrixM has a one dimensional stable graph subspace (see again Lemma 4.1).The claim follows then directly from Theorem 2.9.If a > 0, we conclude from Theorem 2.3, part 2., that every set of control actions

u∗i (t) = −biriψi(t),

yields an open-loop Nash equilibrium. Here ψi(t), i = 1, 2, are obtained as the solutions of thedifferential equation

z(t) =Mz(t), where z0 :=

x0ψ1(0)ψ2(0)

= x0a− µ

a− µ−q1−q2

+ λ 0−s2s1

, λ ∈ IR.That is,

ψ1(t) =−q1x0a− µ e

−µt − λs2e−at,

ψ2(t) =−q2x0a− µ e

−µt + λs1e−at.

¤Remark 4.5 It is not difficult to show, using in the above proof the characterization of the equilibria,that in case a > 0 all equilibrium actions yield the same closed-loop system x = −µx(t), x(0) = x0.However, the cost associated with every equilibrium differ for each player. ¤Example 4.6 Consider a = 3; qi = 2, i = 1, 2 and si = 4. According Theorem 4.2 and Theorem 4.4this game will have an infinite number of equilibrium actions. Furthermore, p1 = p2 = 1 induce theunique equilibrium actions of the infinite-planning horizon game that permits a feedback synthesis.Notice that (p1, p2) is not a strongly stabilizing

6 solution of the set of coupled algebraic Riccatiequations (13,14), since the eigenvalues of·

p1s1 − a p1s2p2s1 p2s2 − a

¸=

·1 44 1

¸6A solution (P1, P2) of the set of algebraic Riccati equations (13,14) is called strongly stabilizing if it is a stabilizing

solution, and

σ

· −AT + P1S1 P1S2

P2S1 −AT + P2S2

¸⊂ lC+.

13

are −3 and 5.So, this example demonstrates that if the infinite-planning horizon has a unique solution per-

mitting a feedback synthesis for every initial state, this does not imply that the with this gamecorresponding set of coupled algebraic Riccati equations (13,14) has a strongly stabilizing solution.¤

In Theorem 4.2 we excluded the case s1q1 + s2q2 = 0. This, since the analysis of this case requires adifferent approach. The results for this case are summarized below.

Theorem 4.7 Assume s1q1 + s2q2 = 0. Then, the infinite-planning horizon game has for everyinitial state

1. a unique set of Nash equilibrium actions

u∗i :=biri

qi2ax(t), i = 1, 2, ifa < 0.

The corresponding closed-loop system and cost are

x(t) = ax(t) and Ji = −qisi4a2

(2a

si+qi2a)x20, i = 1, 2,

2. infinitely many Nash equilibrium actions parameterized by

(u∗1, u∗2) := (−

1

b1(a+ y),− 1

b2(a− y)), where y ∈ IR is arbitrarily, if a > 0 and q1 = q2 = 0.

The corresponding closed-loop system and cost are

x(t) = −ax(t), J1 = − 1

2s1a(q1s1 + (a+ y)

2)x20, and J2 = −1

2s2a(q2s2 + (a− y)2)x20,

3. no equilibrium in all other cases.

Proof.Case 1. can be shown along the lines of the proofs of Theorem 4.2 and 4.4.To prove Case 2. we first notice that (15) reduces, under the assumption that q1 = q2 = 0, to

ki(2a − siki) = 0. Since a > 0 it is clear that ki = 2asi, i = 1, 2, yields a stabilizing solution to this

equation.Next consider (13—14). These equations reduce in this case to

p1(2a− s1p1 − s2p2) = 0

p2(2a− s1p1 − s2p2) = 0

Obviously, with p1 = p2 = 0, the closed-loop system a − s1p1 − s2p2 = a > 0. On the other hand,we see that for every choice of (p1, p2) such that s1p1 + s2p2 = 2a, the closed-loop system becomesstable. Theorem 2.6 yields then the stated conclusion.Next, consider the case a > 0 and either q1 or q2 differs from zero. From Lemma 4.1 it follows

then immediately that the first entry of all eigenvectors corresponding with the eigenvalue −a is zero.

14

According Theorem 2.5, item 1., the game has therefore not an equilibrium for every initial state inthis case.Finally, consider the only case that has not been handled with yet: the case a = 0. Equation

(15) reduces now to:

sik2i = qi, i = 1, 2. (30)

Since s1q1 + s2q2 = 0, it follows that (30) only has a solution if q1 = q2 = 0. But then we haveki = 0, i = 1, 2, yielding a closed-loop system which is not stable. So, according Theorem 2.5, item2., the game has neither a solution for all initial states in this game. Which completes the proof. ¤

For completeness, we present also the generalization of Theorems 4.2 and 4.4 for the N -player sit-uation. The proof can be given along the lines of those theorems. To that purpose notice that thecharacteristic polynomial of matrix M is (−a − λ)N−1(λ2 − (a2 + s1q1 + · · · + sNqN )). Moreover, itis easily verified that every eigenvector corresponding with the eigenvalue a has as its first entry azero. From this the results summarized below follow then analogously.

Theorem 4.8 Assume that σ := s1q1 + · · · + sNqN 6= 0. Then, the infinite-planning horizon gamehas for every initial state a Nash equilibrium if and only if the next conditions hold.

1. a2 + σ > 0,

2. a2 + siqi > 0, i = 1, · · · , N.In case there exists an equilibrium for every initial state, then there is exactly one equilibrium whichpermits a feedback synthesis. The corresponding set of equilibrium actions is:

u∗i (t) = −biripix(t), i = 1, 2,

where pi =(a+µ)qi

σ, µ =

√a2 + σ and x(t) = (a− s1p1 − · · ·− sNpN )x(t), with x(0) = x0.

The corresponding cost for player i for this equilibrium are: Ji = x20qi+sip

2i

µ, i = 1, · · · , N.

Moreover, in case there is an equilibrium for every initial state, this equilibrium is unique if a ≤ 0.If a > 0 there exist infinitely many equilibria. ¤

5 Concluding Remarks

In this paper we considered the indefinite regular infinite-horizon linear quadratic differential game.The results obtained in this paper can be straightforwardly generalized to the N-player case. All

results concerning matrix M should then be substituted by

M =

·A −S−Q −AT2

¸,

where S := [S1, · · · , SN ]; Q := [Q1, · · · , QN ]T and A2 = diag{A}.A different mathematical approach to tackle the present case open-loop dynamic games that has

been successfully exploited in literature is the Hilbert space method. Lukes and Russel [17], Simaan

15

and Cruz [22], Eisele [7] and, more recently, Kremer [13] took, e.g., this approach. A disadvantage ofthis approach is that there are some technical conditions under which this theory can be used. Theanalysis in this paper shows, amongst others, that results obtained using that approach continu tohold in a more general context.Our analysis shows that the stable eigenspace of the extended Hamiltonian matrix M plays a

crucial role in determining the open-loop Nash equilibria. From an application point of view animportant conclusion is that the game will have a unique Nash equilibrium for every initial statein case the stable eigenspace of matrix M is an n-dimensional graph subspace and the individualregulator problems have a solution. Under these conditions the equilibrium actions allow for afeedback synthesis. A numerical algorithm to calculate these actions has, e.g., been outlined inEngwerda [9]. However, notice that various suggestions have been made in literature to improvethe numerical stability of this algorithm (see e.g. Laub [15] and [16], Paige et al. [21], Van Dooren[6] and Mehrmann [18]). Particularly if one considers the implementation of large scale models oneshould consult this literature.

Appendix

Proof of Theorem 2.2.Before we proof this theorem we first recall the next lemma from, e.g., Kun [14]

Lemma 5.1 Let IH be an arbitrary Hilbert-space with some inner product h., .i; u, v ∈ IH; T: IH 7→IH a self-adjoint operator and α ∈ IR. Consider the functional

hu, T (u)i+ 2hu, vi+ α (31)

Then,(i) a necessary condition for (31) to have a minimum is that the operator T is positive semi-definite.(ii) (31) has a unique minimum if and only if the operator T is positive definite. In this case theminimum is achieved at u∗ = −T −1v.Using this result the proof of the theorem reads as follows.

”⇐ part” Let K be the stabilizing solution of the algebraic Riccati equation (5) and m(t) asdefined in (9). Next consider (the value) function

V (t) := xT (t)Kx(t) + 2mT (t)x(t) + n(t),

where

n(t) =

Z ∞

t

{−mT (s)Sm(s) + 2mT (s)c(s)}ds.

Next, we consider V . Note that n(t) = mT (t)Sm(t) − 2mT (t)c(t). Substitution of n, x and m (see(8) and (9)) into V , using the fact that ATK +KA = −Q+KSK (see (5)) yields

V (t) = xT (t)Kx(t) + xT (t)Kx(t) + 2mT (t)x(t) + 2mT (t)x(t) + n(t)

= −xT (t)Qx(t)− uT (t)Ru(t) +[u(t) +R−1BT (Kx(t) +m(t))]TR[u(t) +R−1BT (Kx(t) +m(t))].

16

Since m(t) converges exponentially to zero we have that limt→∞ n(t) = 0 too. Since limt→∞ x(t) = 0too, we have Z ∞

0

V (s)ds = −V (0).

Consequently substitution of V into this expression and rearranging terms givesZ ∞

0

xT (t)Qx(t) + uT (t)Ru(t)dt =

V (0) +

Z T

0

[u+R−1BT (Kx(t) +m(t))]TR[u+R−1BT (Kx(t) +m(t))]dt.

Since V (0) does not depend on u(.) and R is positive definite, the advertized result follows.”⇒ part” Define the linear mappings (operators)

P : x 7→ eAtx,

L : u 7→Z t

t0

eA(t−s)Bu(s)ds and

D : c 7→Z t

t0

eA(t−s)c(s)ds.

Then the state trajectory x becomes

x = P(x0) + L(u) +D(c). (32)

Consider the inner product hf, gi := R∞0fT (s)g(s)ds, on the set Le2,loc of all square integrable (state)

functions from [t0, T ] into IRn and the inner product hu, vi := R∞

0uT (s)v(s)ds on the set Lm2 of all

square integrable control functions. Then the cost function J can be rewritten as

J(t0, x0, u, c) = hP(x0) + L(u) +D(c), Q(P(x0) + L(u) +D(c))iLe2,loc

+ hu,RuiLm2

= hu, (L∗QL+R)(u)iLm2+ 2hu,L∗Q(P(x0) +D(c))iLm

2+

hP(x0) +D(c), Q(P(x0) +D(c)iLe2,loc.

Here L∗ denotes the adjoint operator of L.From part (ii) of Lemma 6.1 it follows now immediately that J(t0, x0, u, c) has a unique minimum

for an arbitrary choice of x0 if and only if the operator T := L∗QL + R is positive definite. Butthis implies that J(t0, x0, u, 0) has a unique minimum for an arbitrary choice of x0 too. AccordingTheorem 2.1 this implies that the Riccati equation (5) has a stabilizing solution. ¤

Proof of Theorem 2.3.In the proof of this theorem we need some preliminary results. We state these in some separatelemmas. Furthermore we use the notation Es and Eu to denote the stable subspace and unstablesubspace associated with a given square matrix, respectively.

17

Lemma 5.2 Assume that A ∈ IRn×n and σ(A) ⊂ lC+. Then,

limt→∞

MeAtN = 0 if and only if MAiN = 0, i = 0, · · · , n− 1.

¤

Using this lemma we can prove then the next result.

Lemma 5.3 Let x0 ∈ IRp, y0 ∈ IRn−p and Y ∈ IR(n−p)×p. Consider the differential equationd

dt

·x(t)y(t)

¸=

·A11 A12A21 A22

¸ ·x(t)y(t)

¸,

·x(0)y(0)

¸=

·x0y0

¸.

If limt→∞ x(t) = 0, for all·x0y0

¸∈ Span

·IY

¸, then

1. dim Es ≥ p, and

2. there exists a matrix Y ∈ IR(n−p)×p such that Span·IY

¸⊂ Es.

Proof.1. Using the Jordan canonical form, we rewrite matrix A as

A = S

·Λs 00 Λu

¸S−1,

where Λs ∈ IRq×q contains all stable eigenvalues of A and Λu ∈ IR(n−q)×(n−q) contains the remainingeigenvalues.Now, assume that q < p. By assumption

limt→∞

·Iq 0 00 Ip−q 0

¸S

·eΛst 00 eΛut

¸S−1

Iq 00 Ip−qY0 Y1

= 0,where Ik denotes the k× k identity matrix and Y = [Y0 Y1]. Denoting the entries of S by Sij , i, j =

1, 2, 3 and the entries of S−1

Iq 00 Ip−qY0 Y1

by P1 P2Q1 Q2R1 R2

we can rewrite the above equation aslimt→∞

·S11S21

¸eΛst [P1 P2] +

·S12 S13S22 S23

¸eΛut

·Q1 Q2R1 R2

¸= 0.

Since limt→∞ eΛst = 0, we conclude that

limt→∞

·S12 S13S22 S23

¸eΛut

·Q1 Q2R1 R2

¸= 0.

So, from Lemma 6.2, we have that in particular·S12 S13S22 S23

¸ ·Q1 Q2R1 R2

¸= 0. (33)

18

Next observe that, on the one hand,

H :=

·S11 S12 S13S21 S22 S23

¸ P1 P2Q1 Q2R1 R2

= · Iq 0 00 Ip−q 0

¸ Iq 00 Ip−qY0 Y1

= Ip.On the other hand we have that, using (33),

H =

·S11S21

¸ £P1 P2

¤.

So, combining both results, we obtain

H = Ip =

·S11S21

¸ £P1 P2

¤.

However, obviously, the lastmentioned matrix is not a full rank matrix. So, we have a contradic-tion. Therefore, our assumption that q < p must have been wrong, which completes the first part ofthe proof.2. From the first part of the lemma we conclude that q ≥ p. Using the same notation, assume that S11 S12

S21 S22S31 S32

and

S13S23S33

constitute a basis for Es and Eu, respectively. Denote S−1·IpY

¸=

PQR

. Sincelimt→∞

[Ip 0 0]S

·eΛst 00 eΛut

¸S−1

·IpY

¸= 0,

we conclude that

limt→∞

[S11 S12] eΛst

·PQ

¸+ S13e

ΛutR = 0.

So, from Lemma 6.2 again we infer that S13R = 0. But, since

H := [S11 S12 S13]

PQR

= Ip,we conclude then that

H = [S11 S12]

·PQ

¸= Ip.

So, necessarily matrix [S11 S12] must be a full row rank matrix. From this observation the conclusionfollows then straightforwardly. ¤

19

Next we proceed with the proof of Theorem 2.3.”⇒ ” Suppose that u∗1, u

∗2 are a Nash solution. That is,

J1(u1, u∗2) ≥ J1(u∗1, u∗2) and J2(u∗1, u2) ≥ J2(u∗1, u∗2).

From the first inequality we see that for every x0 ∈ IRn the (nonhomogeneous) linear quadraticcontrol problem to minimize

J1 =

Z ∞

0

{xT (t)Q1x(t) + uT1 (t)R1u1(t)}dt,

subject to the (nonhomogeneous) state equation

x(t) = Ax(t) +B1u1(t) +B2u∗2(t), x(0) = x0,

has a solution. This implies, see Theorem 2.2, that the algebraic Riccati equation (15) has a stabiliz-ing solution (with i = 1). In a similar way it follows that also the second algebraic Riccati equationmust have a stabilizing solution. Which completes the proof of point 2.To prove point 1. we consider Theorem 2.2 in some more detail. According Theorem 2.2 the mini-mization problem

minu1

J1(x0, u1, u∗2) =

Z ∞

0

{xT1 (t)Q1x1(t) + uT1 (t)R1u1(t)}dt,

where

x1 = Ax1 +B1u1 +B2u∗2, x1(0) = x0,

has a unique solution. This solution is given by

u1(t) = −R−11 BT1 (K1x1(t) +m1(t)). (34)

Here m1(t) is given by

m1(t) =

Z ∞

t

e−(A−S1K1)T (t−s)K1B2u∗2(s)ds. (35)

whereas K1 is the stabilizing solution of the algebraic Riccati equation

Q1 +ATX +XA−XS1X = 0. (36)

Notice that, since the optimal control u1 is uniquely determined, and by definition the equilibriumcontrol u∗1 solves the optimization problem, u

∗1(t) = u1(t). Consequently,

d(x(t)− x1(t))dt

= Ax(t) +B1u∗1(t) +B2u

∗2(t)− ((A− S1K1)x1(t)− S1m1(t) +B2u

∗2(t))

= Ax(t)− S1(K1x1(t)) +m1(t)−Ax1(t) + S1K1x1(t) + S1m1(t)

= A(x(t)− x1(t)).

20

Since x(0)− x1(0) = x0 − x0 = 0 it follows that x1(t) = x(t).In a similar way we obtain from the minimization of J2, with u

∗1 now entering into the system as an

external signal, that

u∗2(t) = −R−12 BT2 (K2x(t) +m2(t)),

where

m2(t) =

Z ∞

t

e−(A−S2K2)T (t−s)K2B1u∗1(s)ds. (37)

and K2 is the stabilizing solution of the algebraic Riccati equation

Q2 +ATX +XA−XS2X = 0. (38)

By straightforward differentiation of (35) and (37), respectively, we obtain

m1(t) = −(A− S1K1)Tm1(t)−K1B2u

∗2(t) (39)

and

m2(t) = −(A− S2K2)Tm2(t)−K2B1u

∗1(t). (40)

Next, introduce

ψi(t) := Kix(t) +mi(t), i = 1, 2. (41)

Using (39) and (36) we get

ψ1(t) = K1x1(t) + m1(t)

= K1(A− S1K1)x(t)−K1S1m1(t) +K1B2u∗2(t)−K1B2u

∗2(t)− (A− S1K1)

Tm1(t)

= (−Q1 − ATK1)x(t)−K1S1m1(t)− (A− S1K1)Tm1(t)

= −Q1x(t)− AT (K1x(t) +m1(t))

= −Q1x(t)− ATψ1(t). (42)

In a similar way we have that ψ2(t) = −Q2x(t) − ATψ2(t). Consequently, with the vector vT (t) :=[xT (t), ψT1 (t), ψ

T2 (t)], v(t) satisfies

v(t) =

A −S1 −S2−Q1 −AT 0−Q2 0 −AT

v(t), with v1(0) = x0.Since by assumption, for arbitrary x0, v1(t) converges to zero it is clear from Lemma 6.3 by choosingconsecutively x0 = ei, i = 1, · · · , n, that matrixM must have at least n stable eigenvalues (countingalgebraic multiplicities). Moreover, the other statement follows from the second part of this lemma.Which completes this part of the proof.”⇐ ” Let u∗2 be as claimed in the theorem, that is

u∗2(t) = −R−12 BT2 ψ2.

21

We next show that then necessarily u∗1 solves the optimization problem

minu1

Z ∞

0

{xT (t)Q1x(t) + uT1R1u1(t)}dt,

subject to

˙x(t) = Ax(t) +B1u1(t) +B2u∗(t), x(0) = x0.

Since, by assumption, the algebraic Riccati equation

Q1 +ATK1 +K1A−K1S1K1 = 0 (43)

has a stabilizing solution, according Theorem 2.2, the above minimization problem has a solution.This solution is given by

u∗1(t) = −R−1BT1 (K1x+m1),

where

m1 =

Z ∞

t

e−(A−S1K1)T (t−s)K1B2u∗2(s)ds.

Next, introduce

ψ1(t) := K1x(t) +m1(t).

Then, similar to (42) we obtain

˙ψ1 = −Q1x− AT ψ1.Consequently, xd(t) := x(t)− x(t) and ψd(t) := ψ1(t)− ψ1(t) satisfy·

xd(t)

ψd(t)

¸=

·A −S1−Q1 −AT

¸ ·xd(t)ψd(t)

¸,

·xd(0)ψd(0)

¸=

·0p

¸.

Notice that matrix

·A −S1−Q1 −AT

¸is the Hamiltonian matrix associated with the algebraic Riccati

equation (43). Recall that the spectrum of this matrix is symmetric w.r.t. the imaginary axis. Sinceby assumption the Riccati equation (43) has a stabilizing solution, we know that its stable invariantsubspace is given by Span[I K1]

T . Therefore, with Eu representing a basis for the unstable subspace,we can write ·

0p

¸=

·IK1

¸v1 + E

uv2,

for some vectors vi, i = 1, 2. However, it is easily verified that due to our asymptotic stabilityassumption both xd(t) and ψd(t) converge to zero if t→∞. So, v2 must be zero. From this it followsnow directly that p = 0. Since the solution of the differential equation is uniquely determined, and[xd(t) ψd(t)] = [0 0] solve it, we conclude that x(t) = x(t) and ψ1(t) = ψ1(t). Or stated differently,

22

u∗1 solves the minimization problem.In a similar way it is shown that for u1 given by u

∗1, player two his optimal control is given by u

∗2.

Which proves the claim. ¤.

Proof of Theorem 2.9.In the proof of this theorem we need a result about observable systems. Therefore, we will first statethis result in a separate lemma.

Lemma 5.4 Consider the system

x(t) = Ax(t), y(t) = Cx(t).

Assume that (C,A) is detectable. Then whenever y(t) = Cx(t) → 0, if t → ∞, it follows that thenalso x(t)→ 0 if t→∞. ¤

Next, we proceed with the proof of the theorem.Since by assumption the stable subspace, Es, is a graph subspace we know that every initial state, x0,can be written uniquely as a combination of the first n entries of the basisvectors in Es. Consequently,with every x0 there corresponds a unique ψ1 and ψ2 for which the solution of the differential equationz(t) = Mz(t), with zT0 = [xT0 ψ

T1 ψ

T2 ], converges to zero. So, according Theorem 2.3, for every x0

there is a Nash equilibrium. On the other hand we have from the proof of Theorem 2.3 that all Nashequilibrium actions (u∗1, u

∗2) satisfy

u∗i (t) = −R−1i BTi ψi(t), i = 1, 2,

where ψi(t) satisfy the differential equation x(t)

ψ1(t)

ψ2(t)

=M x(t)ψ1(t)ψ2(t)

, with x(0) = x0.Now, consider the system

z(t) =Mz(t); y(t) = Cz(t),

where

C :=

I 0 00 −R−11 B1 00 0 −R−12 B2

.Since a standing assumption in this chapter is that (A,Bi), i = 1, 2, is stabilizable, it is easily verifiedthat the pair (C,M) is detectable. Consequently, due to our assumption that x(t) and u∗i (t), i = 1, 2,converge to zero, we have from Lemma 6.4 that [xT (t), ψT1 (t), ψ

T2 (t)] converges to zero. Therefore,

[xT (0), ψT1 (0), ψT2 (0)] has to belong to the stable subspace of M . However, as we argued above, for

every x0 there is exactly one vector ψ1(0) and vector ψ2(0) such that [xT (0), ψT1 (0), ψ

T2 (0)] ∈ Es.

So we conclude that for every x0 there exists exactly one Nash equilibrium.

23

From the assumption that the stable subspace is a graph subspace, we conclude furthermore, seee.g. Engwerda [8] Theorem 6, that the set of algebraic Riccati equations (13,14) has a stabilizing so-lution. Consequently, we obtain from Theorem 2.6 that the game has an equilibrium for every initialstate given by (16). Since for every initial state the equilibrium actions are uniquely determined, itfollows that the equilibrium actions u∗i , i = 1, 2, have to coincide with (16). ¤

References

[1] Abou-Kandil H. and Bertrand P., 1986, Analytic solution for a class of linear quadratic open-loopNash games, International Journal of Control, Vol.43, pp.997-1002.

[2] Abou-Kandil H., Freiling G. and Jank G., 1993, Necessary and sufficient conditions for constantsolutions of coupled Riccati equations in Nash games, Systems&Control Letters, Vol.21, pp.295-306.

[3] Basar T. and Bernhard P., 1995, H∞-Optimal Control and Related Minimax Design Problems,Birkhauser, Boston.

[4] Basar T. and Olsder G.J., 1999, Dynamic Noncooperative Game Theory, SIAM, Philadelphia.

[5] Dockner E., Jørgensen S., Long N. van and Sorger G., 2000, Differential Games in Economicsand Management Science, Cambridge University Press, Cambridge.

[6] Dooren P. van, 1981, A generalized eigenvalue approach for solving Riccati equations, SiamJournal of Scientific Statistical Computation, Vol.2, pp.121-135.

[7] Eisele T., 1982, Nonexistence and nonuniqueness of open-loop equilibria in linear-quadraticdifferential games, Journal of Optimization Theory and Applications, Vol.37, no.4, pp.443-468.

[8] Engwerda J.C., 1998, On the open-loop Nash equilibrium in LQ-games, Journal of EconomicDynamics and Control, Vol.22, pp.729-762.

[9] Engwerda J.C., 1998, Computational aspects of the open-loop Nash equilibrium in LQ-games,Journal of Economic Dynamics and Control, Vol.22, pp.1487-1506.

[10] Engwerda J.C., 2004, Linear-Quadratic Differential Games, to appear.

[11] Feucht M., 1994, Linear-quadratische Differentialspiele und gekoppelte Riccatische Matrixdiffer-entialgleichungen, Ph.D. Thesis, Universitat Ulm, Germany.

[12] Haurie A. and Leitmann G., 1984, On the global asymptotic stability of equilibrium solutionsfor open-loop differential games, Large Scale Systems, Vol.6, 107-122.

[13] Kremer D., 2002, Non-Symmetric Riccati Theory and Noncooperative Games, Ph.D. Thesis,RWTH-Aachen, Germany.

[14] Kun G., 2000, Stabilizability, Controllability and Optimal Strategies of Linear and NonlinearDynamical Games, Ph.D. Thesis, RWTH-Aachen, Germany.

24

[15] Laub A.J., 1979, A Schur method for solving algebraic Riccati equations, IEEE Trans. Automat.Contr., Vol.24, pp.913-921.

[16] Laub A.J.,1991, Invariant subspace methods for the numerical solution of Riccati equations,In: Bittanti S., Laub A.J. & Willems J.C. (eds.), The Riccati equation (pp. 163-199), Springer-Verlag, Berlin.

[17] Lukes D.L. and Russell D.L., 1971, Linear-quadratic games, Journal of Mathematical Analysisand Applications, Vol.33, pp.96-123.

[18] Mehrmann V.L., 1991, The autonomous Linear Quadratic Control Problem: Theory and Numer-ical Solution, In: Thoma and Wyner, eds., Lecture Notes in Control and Information Sciences163, Springer-Verlag, Berlin.

[19] Meyer H.-B., 1976, The matrix equation AZ + B − ZCZ − ZD = 0, SIAM Journal AppliedMathematics, Vol.30, 136-142.

[20] Molinari B.P., 1977, The time-invariant linear-quadratic optimal control problem, Automatica,Vol.13, pp. 347-357.

[21] Paige C. and Van Loan C., 1981, A Schur decomposition for Hamiltonian matrices, LinearAlgebra and its Applications, Vol.41, pp.11-32.

[22] Simaan M. and Cruz J.B., Jr., 1973, On the solution of the open-loop Nash Riccati equationsin linear quadratic differential games, International Journal of Control, Vol.18, pp.57-63.

[23] Starr A.W. and Ho Y.C., 1969, Nonzero-sum differential games, Journal of Optimization Theoryand Applications, Vol.3, pp.184-206.

[24] Weeren A.J.T.M., 1995, Coordination in Hierarchical Control, Ph.D. Thesis, Tilburg University,The Netherlands.

[25] Willems J.C., 1971, Least squares stationary optimal control and the algebraic Riccati equation,IEEE Trans. Automat. Contr., Vol.16, pp. 621-634.

[26] Zhou K., J.C. Doyle, and K. Glover, 1996, Robust and Optimal Control, Prentice Hall, NewJersey.

25