OPTIMAL CONTROL OFTWO- AND THREE-DIMENSIONALINCOMPRESSIBLE NAVIER-STOKES FLOWS�OMAR GHATTASy AND JAI-HYEONG BARKyAbstract. The focus of this work is on the development of large-scale numericaloptimization methods for optimal control of steady incompressible Navier-Stokes ows.The control is a�ected by the suction or injection of uid on portions of the boundary,and the objective function represents the rate at which energy is dissipated in the uid.We develop reduced Hessian sequential quadratic programming methods that avoid con-verging the ow equations at each iteration. Both quasi-Newton and Newton variantsare developed, and compared to the approach of eliminating the ow equations andvariables, which is e�ectively the generalized reduced gradient method. Optimal controlproblems are solved for two-dimensional ow around a cylinder and three-dimensional ow around a sphere. The examples demonstrate at least an order-of-magnitude reduc-tion in time taken, allowing the optimal solution of ow control problems in as little ashalf an hour on a desktop workstation.Key words. optimal control, large-scale optimization, Navier-Stokes equations,sequential quadratic programming, reduced Hessian method, Newton method, quasi-Newton methodAMS(MOS) subject classi�cations. 49M05, 49M07, 49M15, 49M27, 49M37,65K10, 76D05, 76M10, 90C061. Introduction. Flow control has had a long history since Prandtl'searly experiments demonstrated the feasibility of preventing ow separationby sucking uid away from the boundary layer in a diverging channel [23].Since then, much experimental work has been devoted to establishing thetechnological basis for ow control, and certain ow control problems havebecome amenable to analytical investigation, albeit often with simplifyingassumptions; see the review by Gad-el-Hak [8].Recently, interest has increased in optimal ow control of viscous u-ids, that is the determination of optimal values of controls based on thegoverning partial di�erential equations of the uid, i.e. the Navier-Stokesequations [20]. These problems are among the most challenging optimiza-tion problems in computational science and engineering. They owe theircomplexity to their being constrained by numerical approximations of theNavier-Stokes equations. These constraints are highly nonlinear and cannumber in the millions. Conventional optimization approaches prove inad-equate for such large-scale optimization problems.� Supported in part by the Engineering Design Research Center, an EngineeringResearch Center of the National Science Foundation, under Grant No. EEC-8943164,and by Algor, Inc.y Computational Mechanics Laboratory, Department of Civil and Environmen-tal Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, U.S.A. Email:Omar.Ghattas@cs.cmu.edu. 1

2 O. GHATTAS AND J.-H. BARKThe development of numerical optimization methods for optimal owcontrol is built on a mathematical foundation that continues to be enlarged.A number of basic results concerning existence and regularity of solutionsto the continuous problem, as well as error estimates for its numericalapproximation, have been established mostly over the last decade; see thearticle by Gunzburger, Hou, and Svobodny for a good overview [13].This rich mathematical basis, the increasing power of computers, andthe maturation of numerical methods for the ow simulation itself moti-vate the desire to develop numerical optimization methods for solution ofoptimal ow control problems. The latter forms the subject of this article.Here, we focus on a prototype problem of optimal control of uids gov-erned by the steady incompressible Navier-Stokes equations. The controlis a�ected by the suction or injection of uid on portions of the boundary,and the objective function represents the rate at which energy is dissi-pated in the uid. We de�ne the mathematical model in Section 2, and inSection 3 we develop reduced Hessian sequential quadratic programming(SQP) methods that exploit the structure of the optimality conditions andavoid converging the ow equations at each optimization iteration. Bothquasi-Newton and Newton variants are developed. The SQP methods arecompared in Section 4 to the approach of eliminating the ow equationsand variables, which is e�ectively the generalized reduced gradient (GRG)method. The examples demonstrate at least an order-of-magnitude reduc-tion in time taken, allowing the solution of some two-dimensional optimal ow control problems in around a half hour, and three-dimensional prob-lems in reasonable time.2. A problem in optimal control of Navier-Stokes ows. Con-sider a steady, uniform, external, viscous, incompressible ow of a New-tonian uid around a body with bounding surface �. We distinguish twopossibly disjoint regions of the boundary: �0, on which the velocity is spec-i�ed to be zero (i.e. the no-slip condition is speci�ed), and �c, on whichvelocity controls are applied. Thus � = �0 [ �c. To approximate thefar�eld velocity condition, we truncate the domain of the problem with anin ow boundary �1 on which is enforced the freestream velocity u1, andan out ow boundary �2 on which a zero-traction condition is maintained.The ow domain is denoted as . Let us represent the velocity vector,pressure, stress tensor, density, and viscosity of the uid by, respectively,u, p, �, �, and �.The optimal control problem is to �nd the velocity control function ucacting on �c, and the resulting uid �eld variables, that minimize the rate ofenergy dissipation, subject to the Navier Stokes equations. Mathematically,the problem is to minimize�2 Z(ru+ruT ) : (ru+ruT ) d;(2.1)

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 3subject to �(u � r)u�r � � = 0 in ;(2.2) � = �Ip+ �2 (ru+ruT ) in ;(2.3) r � u = 0 in ;(2.4) u = 0 on �0;(2.5) u = uc on �c;(2.6) u = u1 on �1;(2.7) � � n = 0 on �2;(2.8)where (ru)ij = @uj=@xi, and the symbol \:" represents the scalar productof two tensors, so that ru : rv =Xi;j @ui@xj @vi@xj :Here, (2.1) is the dissipation function, (2.2) the conservation of linear mo-mentum equation, (2.3) the constitutive law, (2.4) the conservation of mass(actually volume) equation, and (2.5){(2.8) are boundary conditions. Weeliminate the constitutive law and stress by substituting (2.3) into (2.2),and we further reduce the size of the problem by employing a penaltymethod: let us relax (2.4) by replacing it withr � u = �"p in :(2.9)Clearly as "! 0, we recover the original equation; in fact, the error in thederivative of u is of order " [11]. By introducing the pressure in the massequation, we can eliminate it from the problem by solving for p in (2.9)and substituting the resulting expression into (2.3).In general it is not possible to solve in�nite dimensional optimizationproblems such as (2.1){(2.8) in closed form. Thus, we seek numericalapproximations. Here, we use a Galerkin �nite element method. Letthe Sobolev subspace Uh be the space of all C0 continuous piecewise-polynomials that vanish on �1 and �0, and de�ne the Sobolev subspaceVh similarly, with the added requirement that the functions also vanish

4 O. GHATTAS AND J.-H. BARKon �c. By restricting the velocity and control vectors to Uh, the in�nite-dimensional optimization problem (2.1){(2.8) becomes �nite-dimensional.We \triangulate" the computational domain to obtain Ns nodes in andon �2, Nc nodes on �c, and N1 nodes on �1. Corresponding to a node iwith coordinates xi, we have the compactly-supported �nite element ba-sis function �i(x). Let N represent the total number of unknown nodalvelocity vectors in the optimal control problem, i.e. N = Ns +Nc. Thus,Uh = span f�1; : : : ; �Ng;Vh = span f�1; : : : ; �Nsg:Furthermore, we associate with each node in , on �2, and on �c the nodalvelocity vector ui. The unknown nodal velocities, and hence optimizationvariables, consist of the nodal state velocities ui; i = 1; : : : ; Ns and thenodal control velocities ui; i = Ns+1; : : : ; N . Let uh1 and uhc be the �niteelement interpolants of u1 and uc, i.e.uhc (x) = NXi=Ns+1uc(xi)�i(x);and uh1(x) = N+N1Xi=N+1u1(xi)�i(x):Finally, we can represent uh, the �nite element approximation to the ve-locity �eld, as uh(x) = uh1(x) + NXi=1 ui�i(x);where, because of the property of �nite element basis functions, uh(xi) =ui. We note that the control function has been rendered �nite dimensional;the control variables are simply the nodal values of the control function:ui = uc(xi); i = Ns + 1; : : : ; N:The �nite element approximation of the optimal control problem (2.1){(2.8) can now be stated as �nding uh 2 Uh so that�2 Z(ruh + (ruh)T ) : (ruh + (ruh)T ) d;(2.10)is minimized, subject to�2 Z(ruh + (ruh)T ) : (rvh + (rvh)T ) d(2.11)

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 5+ 1" Z(r�uh)(r�vh) d+Z vh ��(uh �r)uh d = 0 for all vh 2 Vh:The objective function (2.10) is the discrete form of the dissipation function;the constraints (2.11) are a discretization of the variational form of thepenalized conservation of linear momentum equation.Let us de�ne n = dN as the total number of unknown velocity compo-nents, where d is the physical dimension of the ow, i.e. d = 2 or 3. Letu 2 <n denote the vector of unknown nodal velocity components.1 Thevector u includes both state and control variables, and can be symbolicallypartitioned as u = � usuc � ;where us 2 <ns represents the nodal velocities components associated withthe state variables, and uc 2 <nc the nodal velocities corresponding to thecontrols. Similarly, we can partition h(u) : <n ! <n, the discrete form ofthe nonlinear convective term in (2.2), into hs(u) : <n ! <ns , a componentassociated with momentum equations written at nodes belonging to statevariables, and hc(u) : <n ! <nc , a control component, associated withcontrol nodes: h(u) = � hs(u)hc(u) � :As can be seen from (2.11), h(u) is quadratic in u. Finally, we de�nethe matrix K� 2 <n�n arising from the discrete form of the viscous term,and K" 2 <n�n corresponding to the discrete \pressure" term, in themomentum equation. Both K� and K" are symmetric positive de�nite.These matrices can be partitioned into blocks corresponding to state andcontrol variables,K� = � K�ss K�scK�cs K�cc � ; K" = � K"ss K"scK"cs K"cc � :The sparsity ofK�,K", and the Jacobian of h(u) is dictated by the sparsityof the graph underlying the �nite element mesh. For quasi-uniform meshes,a node has a bounded number of neighbors, so that the number of nonzeroesper row is independent of problem size. Thus, these matrices have O(n)nonzeroes. The constant is determined by d, by the order of the basisfunctions �(x), and by the structure of the mesh, but is usually small.The optimization problem (2.10){(2.11) can be rewritten as follows:�nd us 2 <ns and uc 2 <nc so that12uTs K�ssus + 12uTc K�ccuc + uTsK�scuc(2.12)1 Not to be confused with u(x) 2 <3, the exact velocity �eld; the distinction will beclear from the context.

6 O. GHATTAS AND J.-H. BARKis minimized, subject to(K�ss +K"ss)us + (K�sc +K"sc)uc + hs(u) = 0:(2.13)The problem (2.12){(2.13) is characterized by n = ns+nc variables and nsnonlinear equality constraints. Both the objective function and constraintsare quadratic in the optimization variables. For practical problems, ns �nc, since the control is a�ected at a small number of boundary points,while the state variables are associated with a triangulation of the owdomain. The number of degrees of freedom of the optimization problem,i.e. the number of variables n less the number of independent constraintsns, is equal to the number of control variables, and is thus small relativeto the size of the problem. For problems of industrial interest, values ashigh as ns = O(106); nc = O(103) are desirable, although such problemsare currently beyond the range of existing computers. We are thus in therealm of extremely large-scale, sparsely-constrained nonlinear optimization.In the next section we develop SQP methods for this problem.3. SQP methods. The most popular approach to solving optimiza-tion problems that are constrained by discretized partial di�erential equa-tions (PDEs) is to eliminate the constraints by solving them for the statevariables, given values of the control variables. For example, solve for usin (2.13) as a function of uc, and substitute this into the objective (2.12)2.The state variables are thus eliminated from the problem, and we nowhave an unconstrained optimization problem in the nc variables uc. Thegradient of the objective can be obtained through the implicit function the-orem. This method is essentially the GRG method, since we are satisfyingthe constraints at each iteration [10].Advantages of GRG for PDE-constrained optimization problems in-clude: (i) a large reduction in size of the problem, especially when nc � ns,as is common in optimal control or optimal design; (ii) avoiding the largesparse Hessians matrices inherent in the formulation as a constrained prob-lem of the form (2.12){(2.13), in favor of a small, dense Hessian, thus en-abling the use of standard dense nonlinear optimization software; (iii) theability to use existing PDE solution algorithms3 in eliminating the stateequations given values of control variables (i.e. the forward problem). Thislast advantage should not be taken lightly|over the past decade, manyspecialized and sophisticated algorithms (e.g. multilevel methods and pre-conditioned Krylov subspace methods) have been developed for solving var-ious classes of PDEs, and incorporated into robust software. If we retainthe discrete PDEs as constraints, the optimizer becomes responsible forconverging the state equations. Thus it is not immediately obvious how to2 Of course, this substitution is only implicit: ones solves (2.13) numerically at eachoptimization iteration, given current values of uc.3 With the addition of provisions for computing the gradient of the objective function(i.e. sensitivity analysis).

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 7extend sophisticated PDE solution techniques, such as domain decomposi-tion methods and multilevel preconditioners, to the optimization problem,as solution of (2.12){(2.13) appears to require. For these reasons, stateequation elimination methods are almost universal for PDE-constrainedproblems. For examples of this approach in the context of ow control, seethe papers contained in [12].Nevertheless, the approach outlined in the last two paragraphs pos-sesses a distinct disadvantage: it requires exact solution of the state equa-tions at each iteration. This can be an onerous requirement, especially forhighly nonlinear problems such as those governed by Navier-Stokes equa-tions. Instead, we pursue here bona �de SQP methods for the problem(2.12){(2.13). SQP requires satisfaction of only a linear approximationof the constraints at each iteration, thereby avoiding the need to convergethem fully [10]. Thus, state equations are satis�ed simultaneously as controlvariables are converged to their optimal values. Furthermore, we considera special reduced Hessian SQP method that retains the three advantagesattributed to GRG in the paragraph above. In order to retain the last ad-vantage, i.e. that existing (GRG-ready) PDE solvers can be used, severalconditions must be met. First, the PDE solver must be Newton-based.Second, the sensitivity analysis capability of the PDE solver must be basedon an adjoint approach. Finally, the solver must be modular enough sothat the inner Newton linear step can be isolated from the code.We begin with some de�nitions. Let cs(u) : <n ! <ns , represent theresidual of the discrete Navier-Stokes equations,cs(u) � (K�ss +K"ss)us + (K�sc +K"sc)uc + hs(u);and let J(u) 2 <n�n denote the Jacobian of the nonlinear convective termh(u) with respect to the velocities u. The matrix J(u) is nonsymmetricand inde�nite, and can be partitioned into state and control blocks, in thesame manner as the viscous and pressure matrices. Note that the sparsitystructure of J(u) is identical to that of K�. Let the matrix As(u) 2<ns�n represent the Jacobian of cs(u) with respect to u. We can partitionAs(u) into Ass(u) 2 <ns�ns , the Jacobian of cs(u) with respect to us, andAsc(u) 2 <ns�nc , the Jacobian of cs(u) with respect to uc. Explicitly,As(u) = [Ass(u);Asc(u)] ;with Ass(u) � K�ss +K"ss + Jss(u);Asc(u) � K�sc +K"sc + Jsc(u):Thus Ass(u) and Asc(u) have the same sparsity structure as K�ss and K�sc,respectively.

8 O. GHATTAS AND J.-H. BARKWe de�ne the Lagrangian function, L(u;�s), for the optimization prob-lem (2.12){(2.13), asL(u;�s) � 12uTsK�ssus + 12uTc K�ccuc + uTsK�scuc(3.1) + �Ts (K�ss +K"ss)us + �Ts (K�sc +K"sc)uc + �Ts hs(u);where �s 2 <ns is the vector of Lagrange multipliers corresponding tothe state equations. Let g(u) � K�u represent the gradient of the objec-tive function (2.12). We denote by Hk 2 <n�n the symmetric, inde�niteHessian matrix of the k-th element of h(u), and by W(�s) 2 <n�n thesymmetric, inde�nite Hessian matrix of the Lagrangian function L(u;�)(both Hessians are with respect to u). Because h(u) is quadratic in u, Hkis a constant matrix. Note that (Hk)ij is nonzero only when nodes i, j,and k all belong to the same element; it follows that the (i; j) entry of thematrix H(�) � nsXk=1 �kHkis nonzero only when nodes i and j belong to the same element. Therefore,H(�) has the same mesh-based sparsity structure as the other �nite elementmatrices (K�, K", and J), as does W(�), since W(�) = K� +H(�),The necessary conditions for an optimal solution of the problem (2.12){(2.13) re ect the vanishing of the gradient of the Lagrangian (3.1) and canbe expressed by the set of nonlinear equations24 K�ss K�sc K�ss +K"ssK�cs K�cc K�cs +K"csK�ss +K"ss K�sc +K"sc 0 358<: usuc�s 9=;(3.2) +8<: JTss�sJTsc�shs(u) 9=; = 8<: 000 9=; :SQP can be derived as a Newton method for solving the optimality condi-tions [10]. One step of Newton's method for (3.2) is given by solving thelinear system24 K�ss +Hkss K�sc +Hksc K�ss +K"ss + (Jkss)TK�cs +Hkcs K�cc +Hkcc K�sc +K"sc + (Jksc)TK�ss +K"ss + Jkss K�sc +K"sc + Jksc 0 35(3.3)8<: pkspkc�k+1s 9=; = �8<: K�ssuks +K�scukcK�csuks +K�ccukc(K�ss +K"ss)uks + (K�sc +K"sc)ukc + hks = 0; 9=;

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 9for the increments in the state and control velocities, pks and pkc , and for thenew multiplier estimate �k+1s , where the superscript k indicates evaluationof a quantity at uk or �k. The state and control variables are then updatedby uk+1s = uks + pks ; uk+1c = ukc + pkc :Let us rewrite the Newton equations (3.3) in terms of the symbols de�nedat the beginning of this section as24 Wkss Wksc (Akss)TWkcs Wkcc (Aksc)TAkss Aksc 0 358<: pkspkc�k+1s 9=; =8<: �gks�gkc�cks 9=; :(3.4)The major di�culty in solving this linear system is its extremely large,sparse coe�cient matrix, the Karush{Kuhn{Tucker (KKT) matrix, whichis of order (n+ns)�(n+ns). For industrial ow problems on unstructuredmeshes, the forward (i.e. ow simulation) problem is memory-bound, andeven on large supercomputers one can barely a�ord memory for Ass, letalone the entire matrix. So sparse factorization of the KKT matrix is notviable. One possibility is to solve (3.4) using a Krylov method such as theminimum residual method. However, it's not immediately obvious how toprecondition the coe�cient matrix.Instead, consider the following block-elimination. In the remainder ofthis section, we drop the superscript k; it is understood that all quantitiesdepending on u or �s are evaluated at uk or �ks . First, solve for ps fromthe last block of equations to obtainps = �A�1ss (cs +Ascpc);(3.5)where the invertibility of Ass is guaranteed by the well-posedness of theboundary value problem (provided of course that we are away from limitpoints). Then, substitute this expression into the �rst block of equations.Finally, premultiply the �rst block of equations by �ATscA�Tss , and add itto the second block of equations, thereby eliminating �s. The result is alinear system that can be solved for pc, namelyWzpc = (ATscA�Tss Wss �Wcs)A�1ss cs +ATscA�Tss gs � gc;(3.6)Wz � (ATscA�Tss WssA�1ss Asc �ATscA�Tss Wsc �WcsA�1ss Asc +Wcc):Finally, the new estimate of the Lagrange multiplier vector is recoveredfrom �k+1s = �A�Tss (Wssps +Wscpc + gs);(3.7)which is a second-order multiplier estimate. This leads to the followingalgorithm.Algorithm 3.1 [Newton SQP]

10 O. GHATTAS AND J.-H. BARKk = 0; u0s = u0c = �0s = 0while k (Aks )T�ks � gk k 6= 0 and k cks k 6= 0k = k + 1Solve (3.6) for pkcuk+1c = ukc + pkcFind pks from (3.5)uk+1s = uks + pksFind �k+1s from (3.7)endAlgorithm 3.1 displays a quadratic convergence rate provided that (i) atthe optimal solution we are away from a limit or bifurcation point in theforward problem (i.e. Ass is nonsingular); (ii) Wz is positive de�nite atthe optimal solution; and (iii) (u0;�0s) is su�ciently close to the optimalsolution.The justi�cation for the block-elimination (3.5){(3.7) and the result-ing Algorithm 3.1 is that there result only two types of linear system tobe solved: those involving Ass or its transpose as the coe�cient matrix,and the system that determines pc, (3.6), with coe�cient matrix Wz. Inthe former case, these systems are \easy" to solve, since they have thesame coe�cient matrix as that of a Newton step for the state equations.Thus any (Newton-based) Navier-Stokes solver can be enlisted for this task,enabling the exploitation of many of the advances in solving the forwardproblem developed over the last decade (including domain decompositionand multilevel methods). In the latter case, solution of (3.6) is also easysince Wz is of order of the number of control variables, which under theassumption that ns � nc, is very small. Standard dense factorization istherefore appropriate.When implementing Algorithm 3.1, one of course does not invert Ass;one instead forms the matrix A�1ss Asc by solving, with coe�cient matrixAss, for the nc righthand sides composed of the columns of Asc. Anadditional solve with the same coe�cient matrix for the righthand sidecs is necessary. Finally, (3.7) implies an additional righthand side solve,but with the transpose of Ass as coe�cient matrix. So each iterationof Algorithm 3.1 requires solving a linear system with coe�cient matrixAss (the state equation Jacobian matrix) and having nc + 1 righthandsides, as well as a linear system with ATss as coe�cient matrix and onerighthand side. If sparse factorization of Ass is viable, for example for two-dimensional ows or low Reynolds number three-dimensional ows, thenone iteration of Algorithm 3.1 entails one factorization and nc + 2 pairsof triangular solves (compare this with full solution of the ow equations,as in GRG). If quasi-uniform meshes and nested dissection orderings areused, and if pivoting is not required, Ass can be factored with O(n2s) workin 3D and O(n1:5s ) work in 2D [18]. In any case, the cost of one iterationof Algorithm 3.1 is a fraction of the cost of the forward problem. On the

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 11other hand, if sparse factorization is not practical, and an iterative methodmust be used, one is faced with nc+2 solves. When nc is large, it becomesimperative to use iterative methods tailored to multiple righthand sides; italso pays to invest in a good preconditioner, since its construction can beamortized over the righthand sides. In particular, domain decompositionmethods tailored to multiple righthand sides appear to be attractive [6].Once the matrix A�1ss Asc is created, forming its products with subma-trices ofW presents no di�culty. Recall thatW has O(n) nonzeroes and asparsity structure dictated by the underlying �nite element mesh. In par-ticular it is stored using the same (sparse compressed row) data structurethat all the other �nite element matrices use. Therefore forming productswith submatrices of W requires work proportional to the row dimensionof the submatrix. Thus, it is easy to see that, beyond forming the matrixA�1ss Asc, the only other major e�ort in Algorithm 3.1 is O(nsn2c) workassociated with forming Wz, and O(n3c) in factoring Wz.Let us examine the connection with other SQP methods. In fact, theblock-elimination (3.5){(3.7) is identical to a reduced Hessian SQP methodwith a particular choice of null and range space bases. This can be seen bydecomposing the search direction p into two components,p = Zpz +Ypy;(3.8)in which Z 2 <n�nc is a matrix whose columns form a basis for the nullspace of As, and Y 2 <n�ns is chosen so that the matrixQ = [ Z Y ]is nonsingular, and hence Z and Y form a basis for <n. We refer to py asthe range space component, even though strictly speaking, the columns ofY need not span the range space of ATs .The range space step is completely determined by substituting (3.8)into the last block of (3.4), resulting in the ns � ns systemAsYpy = �cs:(3.9)The null space move is found by substituting (3.8) into the �rst two blocksof (3.4), and premultiplying by ZT , to obtain the equations for pz,ZTWZpz = �ZT (g +WYpy):(3.10)The nc�nc matrix ZTWZ is known as the reduced Hessian matrix. If onechooses the nonorthogonal basesZ = � �A�1ss AscI � ;(3.11)and Y = � I0 �(3.12)

12 O. GHATTAS AND J.-H. BARKthen one sees that the null space step (3.10) is identical to (3.6), the equa-tion for determining the move in the control variables. Indeed, the coe�-cient matrix of (3.6), Wz, is exactly the reduced Hessian ZTWZ, and istherefore at least positive semide�nite in the vicinity of a minimum. Thestate variable update (3.5) is comprised of the state equation Newton step,i.e. (3.9) using the range basis (3.12), as well as the null space contribu-tion �A�1ss Ascpz, where pz � pc. The choice of bases (3.11) and (3.12)is known as a \coordinate basis," and has been applied to optimizationproblems in inverse heat conduction [19], structural design [24], [25], andcompressible ow [22], [21]. SQP methods using these bases have beenanalyzed in [7], [26], and [2], among others.As mentioned earlier, one of the di�culties with Algorithm 3.1, i.e.the bona �de Newton method, arises when iterative solution of systemsinvolving Ass is necessary; in this case the bene�t from solving nc + 2systems with the same coe�cient matrix but di�erent righthand side is notas extensive as with sparse LU factorization. Might it be possible to giveup the (local) quadratic convergence guarantee in exchange for the need tosolve fewer systems involving Ass at each iteration?The answer turns out to be a�rmative if we consider a quasi-Newton,rather than a true Newton, method. Consider (3.6), the control variableequation. Its coe�cient matrix is the reduced Hessian Wz. This matrix ispositive de�nite at a strict local minimum, as well as being small (nc�nc)and dense. It makes sense to recur a quasi-Newton approximation to it;thus we can avoid the construction of the matrix A�1ss Asc. By replacingWz with its quasi-Newton approximation, Bz, it is easy to see that (3.5){(3.7) now entail only �ve solutions of systems with Ass or its transpose ascoe�cient matrix. This is a big reduction, especially for large nc. However,we can do even better. At the expense of a reduction from one-step to two-step superlinear convergence [2], we ignore the second-order terms (thoseinvolving submatrices of W) on the righthand side of (3.6). Furthermore,we reduce (3.7) to a �rst-order Lagrange multiplier estimate by droppingterms involving blocks of W. This results in the following algorithm, inwhich the BFGS formula is used to update the quasi-Newton approximationof the reduced Hessian.Algorithm 3.2 [Quasi-Newton SQP]k = 0; u0s = u0c = �0s = 0; B0z = I�0s = (A0ss)�Tg0sg0z = �(A0sc)T�0s + g0cwhile k gkz k 6= 0 and k cks k 6= 0pkc = �(Bkz)�1gzuk+1c = ukc + pkcpks = �(Akss)�1(cks +Akscpkc )uk+1s = uks + pks�k+1s = (Ak+1ss )�Tgk+1s

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 13gk+1z = �(Ak+1sc )T�k+1s + gk+1cykz = gk+1z � gkzBk+1z = Bkz + 1(gkz )Tpkc gkz (gkz )T + 1(ykz )Tpkc ykz (ykz )Tgkz = gk+1zk = k + 1endAlgorithm 3.2 requires only two solves involving Ass per iteration, as com-pared with nc+2 in Algorithm 3.1. This represents a substantial reductionin e�ort when iterative solvers are used and when nc is signi�cant. Ofcourse, Algorithm 3.2 will generally require more iterations to convergethan Algorithm 3.1, since it does not compute exact curvature informa-tion. The �rst of the two linear solves has Ass as its coe�cient matrix,and we term it the state variable update. It comprises two components:a Newton step on the state equations (�A�1ss cs), and a �rst-order changein the states due to a change in the control variables (�A�1ss Ascpc). Thesecond linear system to be solved at each iteration has ATss as its coe�cientmatrix, and is termed the adjoint step, because of parallels with adjointmethods for sensitivity analysis [15], [14]. The steps of this algorithm arealmost identical to a quasi-Newton GRG method; the major di�erence isthat in GRG the state equations are fully converged at each iteration, whilein Algorithm 3.2 essentially only a single Newton step is carried out.Algorithms 3.1 and 3.2 as presented above are not su�cient to guaran-tee convergence to a stationary point from arbitrary initial points. It is wellknown that for the forward problem, i.e. solving the discrete Navier-Stokesequations, Newton's method is only locally convergent. The diameter ofthe ball of convergence is of the order of the inverse of the Reynolds num-ber that characterizes the ow [11]; better initial guesses are thus requiredas the Reynolds number increases. The optimal control problem shouldbe no easier to converge than the forward problem, given that the owequations form part of the �rst-order optimality conditions. This suggestscontinuation methods, which are popular techniques for globalizing theforward problem [11]. Here, we use a simple continuation on Reynoldsnumber. That is, suppose we want to solve an optimal control problemwith a Reynolds number of Re�, for which it is di�cult to �nd a start-ing point from which Newton's method will converge. Instead, we solvea sequence of optimization problems characterized by increasing Reynoldsnumber, beginning with Re=0, and incrementing by �Re. Optimizationproblem i, with Reynolds number i�ReRe� , is solved by either Algorithm 3.1or 3.2, to generate a good starting point for problem i+ 1. Algorithm 3.1is initialized with the optimal Lagrange multipliers and state and controlvariables from optimization problem i�1. Algorithm 3.2 includes the sameinitializations, but in addition takes the initial BFGS approximation to theLagrangian Hessian matrix to be the Hessian approximation at the solu-tion of problem i � 1. Note that when Re=0, the nonlinear terms drop

14 O. GHATTAS AND J.-H. BARKfrom the ow equations, and thus optimization problem (2.12){(2.13) isan equality-constrained quadratic programming problem, solvable in onestep. For subsequent problems, there exists a su�ciently small �Re suchthat Algorithms 3.1 and 3.2 converge to the solution of optimization prob-lem i + 1 using initial data from problem i (provided we are away frombifurcation or singular points).In the next section we use continuation variants of Algorithms 3.1 and3.2 to solve some model problems in boundary control of viscous incom-pressible ow, and compare their performance to the GRG methods.4. Numerical examples. In this section we compare Algorithm 3.1(Newton-SQP, or N-SQP) and Algorithm 3.2 (quasi-Newton-SQP, or QN-SQP) of the previous section with both a quasi-Newton-GRG method (QN-GRG) as well as a steepest descent-GRG method (SD-GRG). With theQN-GRG method we converge the ow equations fully at each optimiza-tion iteration using a Newton solver, and we employ a BFGS formula toapproximate the Hessian of the objective function.4 SD-GRG refers toa similar method, except that a search direction is taken in a directionopposite to the gradient of the objective function. This method is cho-sen because of its popularity in the optimal control literature, and sinceit is easy to implement. In both GRG cases, the sensitivity equations areused to compute the objective function gradient exactly using a direct (asopposed to adjoint) method (see e.g. [14] or [15]).Continuation is applied to N-SQP and QN-SQP as described at the endof Section 3. We apply continuation to both GRG methods at the level ofthe forward problem: since the GRG methods entail satisfaction of the ow equations at each iteration, the forward problem is completely solvedat each optimization iteration using continuation on Reynolds number, i.e.starting with Re=0 and incrementing by �Re until Re� is reached. Thereis another alternative that is intermediate between the extremes of GRG(completely solving the ow equations at each iteration, using continuationon Reynolds number) and SQP (solving a linearized approximation only).This is to use the continuation at the level of the optimization problem,as described at the end of Section 3, in conjunction with full solution ofthe ow equations for the current value of Re. We refer to this method ascontinuation QN-GRG, or CQN-GRG. It uses the converged ow solutionof the previous optimization iteration as an initial guess to the velocity�eld of the current iteration. We expect its e�ciency to be between SQPand GRG.Finite element approximation of the continuous problem is achievedwith isoparametric biquadratic rectangles in 2D and triquadratic hexahedrain 3D. These elements produce errors in the derivatives of uh of O(h2 + ")[11]. All integrals are evaluated with Gauss-Legendre numerical integration4 Recall that with GRG the constraints are eliminated and therefore the problem isan unconstrained one.

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 15using a 3 � 3 (�3) scheme, with the exception of the penalized terms,which are \underintegrated" with a 2� 2 (�2) scheme to avoid \locking"[17]. The value of the penalty parameter " is taken to be 10�7. The owsolver has been veri�ed against a standard benchmark, the driven cavityproblem. We have chosen a value of 10�7 in the Euclidean norm of the�rst order optimality condition (3.2) to terminate optimization iterations.The Reynolds number step size for continuation, �Re, is in all cases 50.Solution of systems involving Ass and its transpose is at the heartof all �ve methods. In GRG, these systems characterize a Newton stepon the state equations, as well as the sensitivity equations. In SQP, theirpresence re ects the choice of a block elimination (or a null space basis) thatfavors \inverting" Ass. The cost of solving these systems asymptoticallydominates an optimization iteration, whether in SQP or GRG guise, sinceit is the only step that is superlinear in ns (all others are linear at worst).Clearly one would like to perform these linear solves as cheaply as possible.Our initial desire was to use a Krylov subspace method, speci�cally thequasi-minimum residual (QMR) method, to solve the systems involvingAssand its transpose, since methods of this type are representative of large-scale CFD solvers. However, after trying QMR on the discrete penalty-based Navier-Stokes equations, we concluded that the equations were tooill-conditioned for iterative solution to be competitive. Even incomplete LUpreconditioning was ine�ective in allowing convergence in reasonable time.This no doubt stems from the penalty formulation, and we expect that adi�erent conclusion would have been reached had a mixed formulation (onethat included both velocity and pressure) been chosen.Instead, we have chosen the multifrontal sparse LU factorization codeUMFPACK [4], [5] for solving the systems involving Ass and its transpose.UMFPACK provides a routine for computing the LU factors of a givensparse matrix. Once this has been computed, UMFPACK provides furtherroutines for �nding the solutions to systems involving the triangular factorsof a matrix as well as their transposes. Thus, using UMFPACK, only asingle factorization of Ass is required at each iteration of the Newton-SQPand Quasi-Newton SQP methods; the primary di�erence between the twomethods therefore lies in the number of triangular solves each performs(in addition to the computation of second derivatives). Using a sparsedirect method of course ultimately limits the maximum size of problemswe can solve, relative to no-�ll methods such as ILU-QMR. Even thoughwe have found UMFPACK to be very e�ective at reducing �ll, a signi�cantamount of �ll is unavoidable for three-dimensional, higher order, vector�nite element problems, due to the high average degree of nodes in the�nite element graph.To compare the �ve di�erent optimal control methods, we choose amodel problem of two-dimensional ow around an in�nite cylinder. Here,

16 O. GHATTAS AND J.-H. BARKwe de�ne the Reynolds number, Re, byRe = �D ju1j� ;where D is the cylinder diameter. Without boundary control, the behaviorof the velocity �eld with increasing Reynolds number is depicted in, forexample, [1]. Flow separation is evident for Reynolds numbers as low as10. The ow�eld remains stationary and exhibits two symmetric stand-ing eddies up to around Re=50. Beyond this range, the wake becomesincreasingly unstable and oscillatory, and a vortex street forms in the wakeand persists downstream. Beyond a Reynolds number of about 60, the ow is neither symmetric about the cylinder centerline, nor is it steady, asassumed by our model.In this section we solve optimal control problems for ows around acylinder with Reynolds numbers as high as 500. However, we use the steadyform of the governing ow equations as constraints; furthermore, our owmodel assumes symmetry of the velocity �eld about the centerline of thecylinder. In principle, the optimal control problem for ow around a cylin-der at Re=500 should be constrained by the time-dependent version of theNavier-Stokes equations, and should not assume symmetry of the velocity�eld. However, if one makes two ad hoc assumptions|that the optimalvelocity �eld for the time-dependent problem is both steady and symmet-ric, even if the uncontrolled ow is neither|then one ought to be able touse the steady Navier Stokes equations as constraints in the formulationof the optimal control problem, and cut the computational domain in halfto exploit symmetry. Computational complexity is greatly reduced underthese two assumptions. It is clear that a su�ciently close initial guessto this steady, symmetric optimal control problem would converge to theoptimum de�ned by the unsteady, unsymmetric control problem.How can these two ad hoc assumptions be veri�ed? In general, theonly way is to solve the time-dependent optimal control problem (algo-rithms for which we have not considered nor implemented), and compareto the optimal controls computed under the two assumptions. Alterna-tively, a reasonable heuristic would be: (i) compute the optimal controlsusing a steady model throughout the optimization; (ii) apply the resultingoptimal controls as boundary conditions in a time-dependent Navier-Stokessolver; and (iii) conclude that, if the resulting velocity �eld is steady andsymmetric, the two assumptions are likely valid.Applying this heuristic, we use a time-dependent Navier-Stokes code tosimulate ow around the cylinder in the entire domain at Re=500. Figure4.1 shows a snapshot of uid streamlines at t = 3:5s for the uncontrolledcase (i.e. no-slip boundary conditions on the cylinder surface); clearly the ow is unsymmetric about the horizontal axis; furthermore, integrationin time reveals no steady state. However, if we apply the steady optimalcontrols (i.e. those found by solving the steady optimal control problem)

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 17

0-0.501 0.501

0

-0.501

0.501

Fig. 4.1. Time dependent streamlines, no control, Re=500.as boundary conditions at nine equally spaced points on the backside ofcylinder, and solve using the time-dependent Navier-Stokes code, we obtainthe streamlines shown in Figure 4.2. The streamlines are indeed symmetric,and further integration in time does not show a change in the velocity �eld.In fact, using a steady Navier-Stokes code we obtain the same velocity �eldas in Figure 4.2.Therefore, while the initial, uncontrolled ow is unsymmetric and un-steady, the optimal (with respect to a steady model) ow is both steadyand symmetric (from the point of view of a time-dependent solver) for owaround a cylinder at Re=500. This motivates using the (steady) ow modelof Section 2, and considering only one-half of the ow domain, thereby re-ducing the size of the forward problem. We stress that we haven't proventhat there doesn't exist an optimal control computed using an unsteadymodel that has a lower value of the dissipation function than that of thesteady optimal control problem. However, this seems unlikely, and it isclear that the steady optimal control is at least a local optimum for theunsteady problem of Re=500 ow around a cylinder.Thus, we consider the computational domain and boundary conditionsdepicted in Figure 4.3, and associated mesh of Figure 4.4. The mesh uses680 isoparametric biquadratic elements, resulting in N = 2829 nodes and

18 O. GHATTAS AND J.-H. BARK

0-0.501 0.501

0

-0.501

0.501

Fig. 4.2. Time dependent streamlines, optimal control, Re=500, t=3.5s.n = 5658 unknown velocity components. Boundary control of the velocityis applied at �ve equally-spaced points on the backside of the cylinder.Since each has two velocity components, we have a total of nc = 10 controlvariables. Streamlines for the case of no control and Re=500 are shownin Figure 4.55. The streamlines are seen to detach near the top of thecylinder, and there is a large recirculation zone behind the cylinder. Aftersolving the optimization problem (2.12){(2.13), the streamlines shown inFigure 4.6 are obtained. The resulting ow resembles a potential ow, andseparation is greatly reduced.As a comparison of the methods, we solve a sequence of �ve optimiza-tion problems, corresponding to Re= 100, 200, 300, 400, and 500, using the�ve optimization methods described above. Table 4.1 compares the numberof optimization iterations taken by the �ve methods. For the continuationmethods, i.e. CQN-GRG, QN-SQP, and N-SQP, the numbers reported inthe table are the sum of iterations across all optimization problems (eachcorresponding to a value of Re).As expected, SD-GRG takes by far the largest number of iterations.The symbol \�" means that the method failed to converge to a stationary5 Of course, this is not a physically meaningful ow.

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 19U = 1

V = 0

Tx = 0, V = 0

Tx = 0

Ty = 0

L

rr

Tx = 0, V = 0

L/2

Tx = 0, V = 0

rU=V=0

Where U, V : Velocities Tx, Ty : Tractions r = L/20Fig. 4.3. Computational domain and boundary conditions, two-dimensional ow aroundin�nite cylinder. Table 4.1Number of optimization iterations taken by GRG and SQP methods.Re SD-GRG QN-GRG CQN-GRG QN-SQP N-SQP100 643 23 34 29 13200 265 30 54 30 14300 291 35 70 37 15400 � 45 89 45 18500 � � 102 52 20point, which occured with SD-GRG for Re=400 and 500, and QN-GRGfor Re=500. QN-GRG takes an order of magnitude fewer iterations thanSD-GRG, due to its ability to approximate curvature of the control vari-able space. However, CQN-GRG takes almost twice as many iterations asQN-GRG. The reason is that the sequence of Reynolds number steps is\packed" into a single optimization iteration with QN-GRG, while CQN-GRG \promotes" the continuation on Re to the level of the optimizationproblem; thus, these steps contribute to the number reported in Table4.1. On the other hand, the cost per iteration of CQN-GRG should besigni�cantly lower than QN-GRG, since each ow solution need only beconverged for the current Reynolds number, and the solver bene�ts froman initial guess taken from the previous converged value. QN-SQP reducesthe number of iterations by almost 50% over CQN-GRG, since it liber-ates the optimizer from having to follow a path dictated by satisfaction

20 O. GHATTAS AND J.-H. BARK

Fig. 4.4. Mesh of 680 biquadratic elements, 2829 nodes.

0-0.501 0.5010

0.1

0.2

0.3

0.4

0.5

Fig. 4.5. Streamlines for steady ow around a cylinder, no control, Re=500.of the ow equations. The result is that the number of iterations takenby QN-GRG and QN-SQP are similar. Of course the cost per iteration ofQN-GRG (and of CQN-GRG) will be much higher than QN-SQP, whichwill be re ected in CPU time. N-SQP o�ers the best performance fromthe point of view of iterations taken, providing on average 2.5 times feweriterations than the QN methods. Of course, this reduction in steps takenmust be balanced with increased work per iteration associated with N-SQPrelative to QN-SQP.Table 4.2 shows timings of each method for the sequence of Reynoldsnumbers solved. Note that these timings are in minutes, so the SD methodrequires several days to �nd an optimal solution, which is unacceptable.The QN-GRG method o�ers over an order of magnitude reduction, but themeasured times are still on the order of hours. Partially integrating ow

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 21

0-0.501 0.5010

0.1

0.2

0.3

0.4

0.5

Fig. 4.6. Streamlines for steady ow around a cylinder, optimal control, Re=500.Table 4.2Timings in minutes for GRG and SQP methods on a DEC 3000/700 with 225 MHzAlpha processor and 512 Mb memory.Re SD-GRG QN-GRG CQN-GRG QN-SQP N-SQP100 3766.75 74.10 41.17 18.37 16.68200 2922.52 168.47 66.18 19.32 17.95300 4744.93 278.98 86.77 25.02 22.10400 � 462.65 110.52 30.72 26.27500 � � 126.98 35.15 30.10solution with optimization, as in CQN-GRG, further reduces CPU time bya factor of about three. A further factor of three reduction is achieved byfully integrating ow solution with optimization, through the use of QN-SQP. This results from its requiring only two linear solves per iteration,against CQN-GRG's fully converging the ow equations. On the otherhand, CPU time decreases only marginally when the N-SQP method isused, typically between 10 and 15%, even though the number of iterationsis signi�cantly lower. This results from the additional work N-SQP must doat each iteration, chie y through the additional righthand side solves andthe construction of the exact reduced HessianWz in (3.6). While the costof constructing Wz is linear in ns, the constant is large, since it involveselement generation- and assembly-like �nite element computations. Theconclusion is that, while the asymptotic costs per iteration of QN-GRGand N-SQP are the same6, it turns out that, for the value of ns we areconsidering, the lower order terms contribute meaningfully, and conspire to6 When sparse LU is used to factor Ass and under the assumption ns � nc.

22 O. GHATTAS AND J.-H. BARKmake N-SQP roughly twice as expensive per iteration as QN-SQP. Still, theNewton method does take less time; whether this is worth the additionale�ort of implementing second derivatives will depend on the particularapplication.Finally, we demonstrate the capability of the reduced Hessian SQPoptimal control method by solving a three-dimensional optimal ow controlproblem. We select a model problem of ow around a sphere at a Reynoldsnumber of 130, which is the limit of steady ow for this problem [1]. Again,we exploit symmetry, this time about two orthogonal planes, to obtain aquarter model of the sphere. Along the planes of symmetry, zero tangentialtraction and zero normal velocity are speci�ed as boundary conditions.A spherical far-�eld boundary truncates the ow domain, and on it areimposed traction-free boundary conditions. The computational domainis meshed into 455 isoparametric triquadratic prism elements, resultingin N = 4155 nodes and n = 12465 unknown velocity components. Inthe absence of boundary velocity control, the ow separates and forms astanding ring-eddy behind the sphere, as shown in Figure 4.7.

Fig. 4.7. Velocity �eld for steady ow around a sphere, no control, Re=130.We introduce boundary velocity control at distinct points lying on sixplanes aligned with the direction of the ow, and perpendicular to the sur-face of the sphere. We consider three cases consisting of one, three, and �veholes on each plane. These correspond to 6, 13, and 25 boundary holes.7The total number of control variables is 16, 33, and 65 for Cases 1, 2, and3, respectively.8 The location of the holes is evident from Figures 4.8{4.10.Because of the expense associated with solving three dimensional ow con-trol problems, we solve the optimization problem (2.12){(2.13) using onlythe most e�cient of the methods considered here, i.e. the Newton-SQPmethod. The optimal ow �eld on a vertical plane of symmetry aligned7 In Cases 2 and 3, there is a shared hole among the six planes8 Some velocity components are required to be zero due to symmetry

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 23with the ow is shown for Case 1 in Figure 4.8. The �gure indicates the

Fig. 4.8. Velocity �eld for steady ow around a sphere, Case 1 optimal control, Re=130.magnitude and direction of the control velocity at the single boundarypoint appearing on this plane. The application of suction at six such pointshas kept the streamlines from separating from the sphere, with the resultthat the standing ring-eddy is largely eliminated. Consider now Case 2,with twice as many control variables. Figure 4.9 shows the velocity �eldcorresponding to the optimal control. While this ow�eld appears to be

Fig. 4.9. Velocity �eld for steady ow around a sphere, Case 2 optimal control, Re=130.an improvement over the uncontrolled ow (Figure 4.7), there is some re-circulation immediately downstream of the sphere that was not present inthe Case 1 optimal solution. Indeed, we expect that the Case 2 optimalsolution should be at least as good as Case 1, since the former duplicatesthe holes of the latter while adding several more. However, the optimizer\sees" only the value of the objective function (and its derivatives), and inCase 2 the optimal value of the dissipation function is indeed lower than

24 O. GHATTAS AND J.-H. BARKTable 4.3Results of optimal control of ow around a sphere.No control Case 1 Case 2 Case 3No. of control variables � 16 33 65Optimal dissipation 5.775457 1.729944 1.551276 1.430053that of Case 1, as seen in Table 4.3. The Case 3 optimal ow�eld, shown inFigure 4.9, resembles Case 2, but as seen in Table 4.3, its dissipation func-tion is lower than that of Case 2, so the result matches our expectation.9

Fig. 4.10. Velocity �eld for steady ow around a sphere, Case 3 optimal control,Re=130.Because of the large computational burden imposed by the three-dimensional problems, we did not conduct careful timings. However, thewall-clock time required (on the workstation described in Table 4.2) in allthree cases was less than a day. The majority of the time undoubtedly wasspent on factoring the coe�cient matrix Ass.5. Final Remarks. Based on the comparison of the previous section,we conclude that the reduced Hessian SQP methods are overwhelminglysuperior to the GRG methods that are popular for optimization problemsinvolving PDEs as constraints, o�ering over an order of magnitude improve-ment in time required for the optimal ow control problems considered. Inparticular, the methods are so e�cient that the optimal control for a two-dimensional ow around a cylinder at Reynolds number 500 is found inabout a half hour on a desktop workstation. Furthermore, the Newton-SQP method is capable of solving some three-dimensional optimal owcontrol problems on the same workstation in less than a day. Even though9 Even if this were not the case, the issue of local vs. global optimality might explainsuch an apparently anomalous result.

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 25the Newton-SQP method takes signi�cantly fewer iterations, its need toconstruct exact Hessians and to solve linear systems that number on theorder of the number of control variables makes it only marginally moree�cient than its quasi-Newton counterpart, based on our two-dimensionalresults.The GRG methods described here represent a strict interpretation ofthe GRG idea|at each optimization iteration, the ow equations are con-verged to a tolerance of 10�7. However, in the spirit of SQP, one mightchoose a greater value of the tolerance at early iterations, making sure toreduce the tolerance to its target value as the optimization iterations con-verge. Indeed, in the limit of one Newton step on the ow equations peroptimization iteration, we essentially recover the reduced SQP method. Arelated idea is to pose the early optimization iterations on a coarse mesh,and re�ne as the optimum is approached; this can be very e�ective, as ad-vanced in [16] and [27]. Finally, in [3], a number of possibilities are de�nedthat are intermediate between the extremes of GRG (full ow convergenceper optimization iteration) and reduced SQP (one state equation Newtonstep per optimization iteration).We imagine that for larger problems, the cost of factoring the stateequation Jacobian matrix will begin to display its asymptotic behaviorand dominate the lower-order terms, leading to increasing e�ciency of theNewton method relative to quasi-Newton. However, problem size cannotcontinue to grow inde�nitely and still allow sparse LU factorization. Forexample, in solving three dimensional Navier-Stokes ow control problemson the workstation described in Table 4.2, we encountered a limit of about13000 state variables, due to memory. Beyond this size, where undoubt-edly most industrial-scale ow control problems lie, iterative solvers are re-quired, and it remains to be seen whether they can be tailored to multiplerighthand sides su�ciently well that Newton SQP can retain its superiorityover quasi-Newton SQP.For the largest problems, parallel computing will become essential. Ofcourse, there is a long history of parallel algorithms and implementationsfor the forward problem, i.e. Navier-Stokes ow simulation. Algorithms3.1 and 3.2 are well suited to parallel machines, since the majority of theirwork involves solution of linear systems having state equation Jacobiansas their coe�cient matrix; this is just a step of the forward problem, theparallelization of which is well-understood. Indeed, in [9] we discuss theparallel implementation of Algorithm 3.2 for a problem in shape optimiza-tion governed by compressible ows.Acknowledgements. We thank Beichang He for providing the timedependent Navier-Stokes simulations that were used to verify the symmetryand stationarity of the optimally-controlled ow. We also thank CristinaAmon, Carlos Orozco, Larry Biegler, and James Antaki for various discus-sions and suggestions related to this work.

26 O. GHATTAS AND J.-H. BARKREFERENCES[1] G. K. Batchelor, An Introduction to Fluid Dynamics, Cambridge UniversityPress, 1967.[2] L. T. Biegler, J. Nocedal, and C. Schmid, A reduced Hessian method forlarge-scale constrained optimization, SIAM Journal on Optimization, 5 (1995),pp. 314{347.[3] E. J. Cramer, J. E. Dennis, P. D. Frank, R. M. Lewis, and G. R. Shubin,Problem formulation for multidisciplinary optimization, SIAM Journal on Op-timization, 4 (1994), pp. 754{776.[4] T. A. Davis, Users' guide for the unsymmetric pattern multifrontal package(UMFPACK), Tech. Rep. TR-93-020, CIS Dept., Univ. of Florida, Gainesville,FL, 1993.[5] T. A. Davis and I. S. Duff, An unsymmetric pattern multifrontal method forsparse LU factorization, Tech. Rep. TR-93-018, CIS Dept., Univ. of Florida,Gainesville, FL, 1993.[6] C. Farhat and P.-S. Chen, Tailoring domain decomposition methods for e�cientparallel coarse grid solution and for systems with many right hand sides, inDomain Decomposition Methods in Science and Engineering, vol. 180 of Con-temporary Mathematics, American Mathematical Society, 1994, pp. 401{406.[7] D. Gabay, Reduced Quasi-Newton methods with feasibility improvement fornonlinearly constrained optimization, Mathematical Programming Study, 16(1982), p. 18.[8] M. Gad-el-Hak, Flow control, Applied Mechanics Reviews, 42 (1989), pp. 261{292.[9] O. Ghattas and C. E. Orozco, A parallel reduced Hessian SQP method forshape optimization, in Multidisciplinary Design Optimization: State-of-the-Art, N. Alexandrov and M. Hussaini, eds., SIAM, 1996. To appear.[10] P. E. Gill, W. Murray, and M. H. Wright, Practical Optimization, AcademicPress, 1981.[11] M. D. Gunzburger, Finite Element Methods for Viscous Incompressible Flows,Academic Press, 1989.[12] M. D. Gunzburger, ed., Flow Control, vol. 68 of IMA Volumes in Mathematicsand its Applications, Springer-Verlag, 1995.[13] M. D. Gunzburger, L. S. Hou, and T. P. Svobodny, Optimal control and op-timization of viscous, incompressible ows, in Incompressible ComputationalFluid Dynamics, M. D. Gunzburger and R. A. Nicolaides, eds., Cambridge,1993, ch. 5, pp. 109{150.[14] R. T. Haftka, Z. G�urdal, and M. P. Kamat, Elements of Structural Optimiza-tion, Kluwer Academic Publishers, 1990.[15] E. J. Haug and J. S. Arora, Applied Optimal Design, Wiley-Interscience, 1979.[16] W. P. Huffman, R. G. Melvin, D. P. Young, F. T. Johnson, J. E. Bussoletti,M. B. Bieterman, and C. L. Hilmes, Practical design and optimization incomputational uid dynamics, in Proceedings of 24th AIAA Fluid DynamicsConference, Orlando, Florida, July 1993.[17] T. J. R. Hughes, W. K. Liu, and A. Brooks, Finite element analysis of incom-pressible viscous ow by the penalty function formulation, Journal of Compu-tational Physics, 30 (1979), pp. 1{60.[18] M. S. Khaira, G. L. Miller, and T. J. Sheffler, Nested dissection: A surveyand comparison of various nested dissection algorithms, Tech. Rep. CMU-CS-92-106R, Carnegie Mellon University, 1992.[19] F. S. Kupfer and E. W. Sachs, A prospective look at SQP methods for semi-linear parabolic control problems, in Optimal Control of Partial Di�erentialEquations, K. Ho�mann and W. Krabs, eds., Springer, 1991, pp. 145{157.[20] P. Moin and T. Bewley, Feedback control of turbulence, Applied Mechanics Re-views, 47 (1994), pp. S3{S13.

OPTIMAL CONTROL OF NAVIER-STOKES FLOWS 27[21] C. E. Orozco and O. Ghattas, Massively parallel aerodynamic shape optimiza-tion, Computing Systems in Engineering, 1{4 (1992), pp. 311{320.[22] , Optimal design of systems governed by nonlinear partial di�erential equa-tions, in Fourth AIAA/USAF/NASA/OAI Symposium on MultidisciplinaryAnalysis and Optimization, AIAA, 1992, pp. 1126{1140.[23] L. Prandtl and O. G. Tietjens, Applied Hydro- and Aeromechanics, Dover,1934, p. 81.[24] U. Ringertz, Optimal design of nonlinear shell structures, Tech. Rep. FFA TN91-18, The Aeronautical Research Institute of Sweden, 1991.[25] , An algorithm for optimization of nonlinear shell structures, InternationalJournal for Numerical Methods in Engineering, 38 (1995), pp. 299{314.[26] Y. Xie, Reduced Hessian algorithms for solving large-scale equality constrained op-timization problems, PhD thesis, University of Colorado, Boulder, Departmentof Computer Science, 1991.[27] D. P. Young, W. P. Huffman, R. G. Melvin, M. B. Bieterman, C. L. Hilmes,and F. T. Johnson, Inexactness and global convergence in design optimiza-tion, in Proceedings of the 5th AIAA/NASA/USAF/ISSMO Symposium onMultidisciplinary Analysis and Optimization, Panama City, Florida, Septem-ber 1994.