Model Predictive Control:

Theory, Computation, and Design

2nd Edition

Model Predictive Control:

Theory, Computation, and Design

2nd Edition

James B. Rawlings

Department of Chemical Engineering

University of California

Santa Barbara, California, USA

David Q. Mayne

Department of Electrical and Electronic Engineering

Imperial College London

London, England

Moritz M. Diehl

Department of Microsystems Engineering and

Department of Mathematics

University of Freiburg

Freiburg, Germany

DNob Hill Publishing

Santa Barbara, California

This book was set in Lucida using LATEX, and printed and bound by

Worzalla.

Cover design by Cheryl M. and James B. Rawlings.

Copyright © 2020 by Nob Hill Publishing, LLC

All rights reserved.

Nob Hill Publishing, LLC

Cheryl M. Rawlings, publisher

Santa Barbara, CA 93101

orders@nobhillpublishing.com

http://www.nobhillpublishing.com

No part of this book may be reproduced, in any form or by any means,

without permission in writing from the publisher.

Library of Congress Control Number: 2020942771

Printed in the United States of America.

First Edition

First Printing August 2009

Electronic Download (1st) November 2013

Electronic Download (2nd) April 2014

Electronic Download (3rd) July 2014

Electronic Download (4th) October 2014

Electronic Download (5th) February 2015

Second Edition

First Printing October 2017

Electronic Download (1st) October 2018

Electronic Download (2nd) February 2019

Paperback Edition

Third Printing October 2020

Electronic Download (3rd) October 2020

To Cheryl, Josephine, and Stephanie,

for their love, encouragement, and patience.

Preface to the Second Edition

In the eight years since the publication of the ®rst edition, the ®eld

of model predictive control (MPC) has seen tremendous progress. First

and foremost, the algorithms and high-level software available for solv-

ing challenging nonlinear optimal control problems have advanced sig-

ni®cantly. For this reason, we have added a new chapter, Chapter 8,

ªNumerical Optimal Control,º and coauthor, Professor Moritz M. Diehl.

This chapter gives an introduction into methods for the numerical so-

lution of the MPC optimization problem. Numerical optimal control

builds on two ®elds: simulation of differential equations, and numeri-

cal optimization. Simulation is often covered in undergraduate courses

and is therefore only brie¯y reviewed. Optimization is treated in much

more detail, covering topics such as derivative computations, Hessian

approximations, and handling inequalities. Most importantly, the chap-

ter presents some of the many ways that the speci®c structure of opti-

mal control problems arising in MPC can be exploited algorithmically.

We have also added a software release with the second edition of

the text. The software enables the solution of all of the examples and

exercises in the text requiring numerical calculation. The software is

based on the freely available CasADi language, and a high-level set of

Octave/MATLAB functions, MPCTools, to serve as an interface to CasADi.

These tools have been tested in several MPC short courses to audiences

composed of researchers and practitioners. The software can be down-

loaded from www.chemengr.ucsb.edu/~jbraw/mpc.

In Chapter 2, we have added sections covering the following topics:

• economic MPC

• MPC with discrete actuators

We also present a more recent form of suboptimal MPC that is prov-

ably robust as well as computationally tractable for online solution of

nonconvex MPC problems.

In Chapter 3, we have added a discussion of stochastic MPC, which

has received considerable recent research attention.

In Chapter 4, we have added a new treatment of state estimation

with persistent, bounded process and measurement disturbances. We

have also removed the discussion of particle ®ltering. There are two

vi

vii

reasons for this removal; ®rst, we wanted to maintain a manageable

total length of the text; second, all of the available sampling strate-

gies in particle ®ltering come up against the ªcurse of dimensionality,º

which renders the state estimates inaccurate for dimension higher than

about ®ve. The material on particle ®ltering remains available on the

text website.

In Chapter 6, we have added a new section for distributed MPC of

nonlinear systems.

In Chapter 7, we have added the software to compute the critical

regions in explicit MPC.

Throughout the text, we support the stronger KL-de®nition of asymp-

totic stability, in place of the classical de®nition used in the ®rst edition.

The most signi®cant notational change is to denote a sequence with

a; b; c; : : : instead of with fa;b; c; : : :g as in the ®rst edition.

JBR

Madison, Wis., USA

DQM

London, England

MMD

Freiburg, Germany

Added for the second edition, third printing

The second edition, ®rst printing was made available electronically in

October 2018. The February 2019 second (electronic only) printing

mainly corrected typographical errors. This third printing was printed

as a paperback and made available electronically in October 2020.

In this third printing, besides removing typographical and other er-

rors, Chapter 4 was revised signi®cantly. The analysis of Moving Hori-

zon Estimation and Full Information Estimation with bounded distur-

bances has improved signi®cantly in the last several years due to the

research efforts of several groups. We have attempted to bring the

material in Chapter 4 up to date with this current literature.

Moreover, the section in Chapter 3 on Stochastic MPC was updated,

and a new section on discrete actuators was added to Chapter 8.

JBR

Santa Barbara, CA, USA

DQM

London, England

MMD

Freiburg, Germany

Preface

Our goal in this text is to provide a comprehensive and foundational

treatment of the theory and design of model predictive control (MPC).

By now several excellent monographs emphasizing various aspects of

MPC have appeared (a list appears at the beginning of Chapter 1, and

the reader may naturally wonder what is offered here that is new and

different. By providing a comprehensive treatment of the MPC foun-

dation, we hope that this text enables researchers to learn and teach

the fundamentals of MPC without continuously searching the diverse

control research literature for omitted arguments and requisite back-

ground material. When teaching the subject, it is essential to have a

collection of exercises that enables the students to assess their level of

comprehension andmastery of the topics. To support the teaching and

learning of MPC, we have included more than 200 end-of-chapter exer-

cises. A complete solution manual (more than 300 pages) is available

for course instructors.

Chapter 1 is introductory. It is intended for graduate students in en-

gineering who have not yet had a systems course. But it serves a second

purpose for those who have already taken the ®rst graduate systems

course. It derives all the results of the linear quadratic regulator and

optimal Kalman ®lter using only those arguments that extend to the

nonlinear and constrained cases to be covered in the later chapters.

Instructors may ®nd that this tailored treatment of the introductory

systems material serves both as a review and a preview of arguments

to come in the later chapters.

Chapters 2±4 are foundational and should probably be covered in

any graduate level MPC course. Chapter 2 covers regulation to the ori-

gin for nonlinear and constrained systems. This material presents in a

uni®ed fashion many of the major research advances in MPC that took

place during the last 20 years. It also includes more recent topics such

as regulation to an unreachable setpoint that are only now appearing in

the research literature. Chapter 3 addressesMPCdesign for robustness,

with a focus on MPC using tubes or bundles of trajectories in place of

the single nominal trajectory. This chapter again uni®es a large body of

research literature concerned with robust MPC. Chapter 4 covers state

estimation with an emphasis on moving horizon estimation, but also

viii

ix

covers extended and unscented Kalman ®ltering, and particle ®ltering.

Chapters 5±7 present more specialized topics. Chapter 5 addresses

the special requirements of MPC based on output measurement instead

of state measurement. Chapter 6 discusses how to design distributed

MPC controllers for large-scale systems that are decomposed intomany

smaller, interacting subsystems. Chapter 7 covers the explicit optimal

control of constrained linear systems. The choice of coverage of these

three chapters may vary depending on the instructor's or student's own

research interests.

Three appendices are included, again, so that the reader is not sent

off to search a large research literature for the fundamental arguments

used in the text. Appendix A covers the required mathematical back-

ground. Appendix B summarizes the results used for stability analysis

including the various types of stability and Lyapunov function theory.

Since MPC is an optimization-based controller, Appendix C covers the

relevant results from optimization theory. In order to reduce the size

and expense of the text, the three appendices are available on the web:

www.chemengr.ucsb.edu/~jbraw/mpc. Note, however, that all mate-

rial in the appendices is included in the book's printed table of contents,

and subject and author indices. The website also includes sample ex-

ams, and homework assignments for a one-semester graduate course

in MPC. All of the examples and exercises in the text were solved with

Octave. Octave is freely available from www.octave.org.

JBR DQM

Madison, Wisconsin, USA London, England

Acknowledgments

Both authors would like to thank the Department of Chemical and Bio-

logical Engineering of the University of Wisconsin for hosting DQM's

visits to Madison during the preparation of this monograph. Funding

from the Paul A. Elfers Professorship provided generous ®nancial sup-

port.

JBR would like to acknowledge the graduate students with whom

he has had the privilege to work on model predictive control topics:

Rishi Amrit, Dennis BonnÂe, John Campbell, John Eaton, Peter Findeisen,

Rolf Findeisen, Eric Haseltine, John Jùrgensen, Nabil Laachi, Scott Mead-

ows, Scott Middlebrooks, Steve Miller, Ken Muske, Brian Odelson, Mu-

rali Rajamani, Chris Rao, Brett Stewart, Kaushik Subramanian, Aswin

Venkat, and Jenny Wang. He would also like to thank many colleagues

with whom he has collaborated on this subject: Frank AllgÈower, Tom

Badgwell, Bhavik Bakshi, Don Bartusiak, Larry Biegler, Moritz Diehl,

Jim Downs, Tom Edgar, Brian Froisy, Ravi Gudi, Sten Bay Jùrgensen,

Jay Lee, Fernando Lima, Wolfgang Marquardt, Gabriele Pannocchia, Joe

Qin, Harmon Ray, Pierre Scokaert, Sigurd Skogestad, Tyler Soderstrom,

Steve Wright, and Robert Young.

DQM would like to thank his colleagues at Imperial College, espe-

cially Richard Vinter and Martin Clark, for providing a stimulating and

congenial research environment. He is very grateful to Lucien Polak

and Graham Goodwin with whom he has collaborated extensively and

fruitfully over many years; he would also like to thank many other col-

leagues, especially Karl ÊAstrÈom, Roger Brockett, Larry Ho, Petar Koko-

tovic, and Art Krener, from whom he has learned much. He is grateful

to past students who have worked with him on model predictive con-

trol: Ioannis Chrysochoos, Wilbur Langson, Hannah Michalska, Sasa

RakoviÂc, and Warren Schroeder; Hannah Michalska and Sasa RakoviÂc,

in particular, contributed very substantially. He owes much to these

past students, now colleagues, as well as to Frank AllgÈower, Rolf Find-

eisen Eric Kerrigan, Konstantinos Kouramus, Chris Rao, Pierre Scokaert,

and Maria Seron for their collaborative research in MPC.

Both authors would especially like to thank Tom Badgwell, Bob Bird,

Eric Kerrigan, Ken Muske, Gabriele Pannocchia, and Maria Seron for

their careful and helpful reading of parts of themanuscript. John Eaton

x

xi

again deserves specialmention for his invaluable technical support dur-

ing the entire preparation of the manuscript.

Added for the second edition. JBR would like to acknowledge the

most recent generation of graduate students with whom he has had the

privilege to work on model predictive control research topics: Doug Al-

lan, Travis Arnold, Cuyler Bates, Luo Ji, Nishith Patel, Michael Risbeck,

and Megan Zagrobelny.

In preparing the second edition, and, in particular, the software re-

lease, the current group of graduate students far exceeded expectations

to help ®nish the project. Quite simply, the project could not have been

completed in a timely fashion without their generosity, enthusiasm,

professionalism, and sel¯ess contribution. Michael Risbeck deserves

special mention for creating the MPCTools interface to CasADi, and

updating and revising the tools used to create the website to distribute

the text- and software-supporting materials. He also wrote code to cal-

culate explicit MPC control laws in Chapter 7. Nishith Patel made a

major contribution to the subject index, and Doug Allan contributed

generously to the presentation of moving horizon estimation in Chap-

ter 4.

A research leave for JBR in Fall 2016, again funded by the Paul A.

Elfers Professorship, was instrumental in freeing up time to complete

the revision of the text and further develop computational exercises.

MMD wants to especially thank Jesus Lago Garcia, Jochem De Schut-

ter, Andrea Zanelli, Dimitris Kouzoupis, Joris Gillis, Joel Andersson,

and Robin Verschueren for help with the preparation of exercises and

examples in Chapter 8; and also wants to acknowledge the following

current and former team members that contributed to research and

teaching on optimal and model predictive control at the Universities of

Leuven and Freiburg: Adrian BÈurger, Hans Joachim Ferreau, JÈorg Fis-

cher, Janick Frasch, Gianluca Frison, Niels Haverbeke, Greg Horn, Boris

Houska, Jonas Koenemann, Attila Kozma, Vyacheslav Kungurtsev, Gio-

vanni Licitra, Rien Quirynen, Carlo Savorgnan, Quoc Tran-Dinh, Milan

Vukov, and Mario Zanon. MMD also wants to thank Frank AllgÈower, Al-

berto Bemporad, Rolf Findeisen, Larry Biegler, Hans Georg Bock, Stephen

Boyd, SÂebastien Gros, Lars GrÈune, Colin Jones, John Bagterp Jùrgensen,

Christian Kirches, Daniel Leineweber, Katja Mombaur, Yurii Nesterov,

Toshiyuki Ohtsuka, Goele Pipeleers, Andreas Potschka, Sebastian Sager,

Johannes P. SchlÈoder, Volker Schulz, Marc Steinbach, Jan Swevers, Phil-

ippe Toint, Andrea Walther, Stephen Wright, Joos Vandewalle, and Ste-

fan Vandewalle for inspiring discussions on numerical optimal control

xii

methods and their presentation during the last 20 years.

All three authors would especially like to thank Joel Andersson and

Joris Gillis for having developed CasADi and for continuing its support,

and for having helped to improve some of the exercises in the text.

Added for the second edition, third printing. The authors would

like to acknowledge and thank Doug Allan again for his suggestions

and help with the revision of Chapter 4. Much of the new material

on Full Information Estimation and Moving Horizon Estimation is a di-

rect result of Doug's research papers and 2020 PhD thesis on state

estimation. Koty McAllister provided expert assistance in the update

of stochastic MPC in Chapter 3. Finally, Adrian Buerger and Pratyush

Kumar provided valuable assistance on the addition of the discrete ac-

tuator numerics to Chapter 8.

Contents

1 Getting Started with Model Predictive Control 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Models and Modeling . . . . . . . . . . . . . . . . . . . . . . 1

1.2.1 Linear Dynamic Models . . . . . . . . . . . . . . . . . 2

1.2.2 Input-Output Models . . . . . . . . . . . . . . . . . . 3

1.2.3 Distributed Models . . . . . . . . . . . . . . . . . . . 4

1.2.4 Discrete Time Models . . . . . . . . . . . . . . . . . . 5

1.2.5 Constraints . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2.6 Deterministic and Stochastic . . . . . . . . . . . . . 9

1.3 Introductory MPC Regulator . . . . . . . . . . . . . . . . . . 11

1.3.1 Linear Quadratic Problem . . . . . . . . . . . . . . . 11

1.3.2 Optimizing Multistage Functions . . . . . . . . . . . 12

1.3.3 Dynamic Programming Solution . . . . . . . . . . . 18

1.3.4 The In®nite Horizon LQ Problem . . . . . . . . . . . 21

1.3.5 Controllability . . . . . . . . . . . . . . . . . . . . . . 23

1.3.6 Convergence of the Linear Quadratic Regulator . . 24

1.4 Introductory State Estimation . . . . . . . . . . . . . . . . . 26

1.4.1 Linear Systems and Normal Distributions . . . . . 27

1.4.2 Linear Optimal State Estimation . . . . . . . . . . . 29

1.4.3 Least Squares Estimation . . . . . . . . . . . . . . . 33

1.4.4 Moving Horizon Estimation . . . . . . . . . . . . . . 39

1.4.5 Observability . . . . . . . . . . . . . . . . . . . . . . . 41

1.4.6 Convergence of the State Estimator . . . . . . . . . 43

1.5 Tracking, Disturbances, and Zero Offset . . . . . . . . . . 46

1.5.1 Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . 46

1.5.2 Disturbances and Zero Offset . . . . . . . . . . . . . 49

1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

2 Model Predictive ControlÐRegulation 89

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

2.2 Model Predictive Control . . . . . . . . . . . . . . . . . . . . 91

2.3 Dynamic Programming Solution . . . . . . . . . . . . . . . 107

2.4 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

2.4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 112

xiii

xiv Contents

2.4.2 Stabilizing Conditions . . . . . . . . . . . . . . . . . 114

2.4.3 Exponential Stability . . . . . . . . . . . . . . . . . . 120

2.4.4 Controllability and Observability . . . . . . . . . . . 120

2.4.5 Time-Varying Systems . . . . . . . . . . . . . . . . . 123

2.5 Examples of MPC . . . . . . . . . . . . . . . . . . . . . . . . . 131

2.5.1 The Unconstrained Linear Quadratic Regulator . . 132

2.5.2 Unconstrained Linear Periodic Systems . . . . . . . 133

2.5.3 Stable Linear Systems with Control Constraints . 135

2.5.4 Linear Systems with Control and State Constraints 136

2.5.5 Constrained Nonlinear Systems . . . . . . . . . . . 139

2.5.6 Constrained Nonlinear Time-Varying Systems . . . 141

2.6 Is a Terminal Constraint Set Xf Necessary? . . . . . . . . 144

2.7 Suboptimal MPC . . . . . . . . . . . . . . . . . . . . . . . . . 147

2.7.1 Extended State . . . . . . . . . . . . . . . . . . . . . . 150

2.7.2 Asymptotic Stability of Difference Inclusions . . . 150

2.8 Economic Model Predictive Control . . . . . . . . . . . . . 153

2.8.1 Asymptotic Average Performance . . . . . . . . . . 155

2.8.2 Dissipativity and Asymptotic Stability . . . . . . . 156

2.9 Discrete Actuators . . . . . . . . . . . . . . . . . . . . . . . . 160

2.10 Concluding Comments . . . . . . . . . . . . . . . . . . . . . 163

2.11 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

2.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

3 Robust and Stochastic Model Predictive Control 193

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

3.1.1 Types of Uncertainty . . . . . . . . . . . . . . . . . . 193

3.1.2 Feedback Versus Open-Loop Control . . . . . . . . 195

3.1.3 Robust and Stochastic MPC . . . . . . . . . . . . . . 200

3.1.4 Tubes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

3.1.5 Difference InclusionDescription of Uncertain Sys-

tems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

3.2 Nominal (Inherent ) Robustness . . . . . . . . . . . . . . . . 204

3.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 204

3.2.2 Difference Inclusion Description of Discontinu-

ous Systems . . . . . . . . . . . . . . . . . . . . . . . 206

3.2.3 When Is Nominal MPC Robust? . . . . . . . . . . . . 207

3.2.4 Robustness of Nominal MPC . . . . . . . . . . . . . 209

3.3 Min-Max Optimal Control: Dynamic Programming Solution 214

3.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 214

3.3.2 Properties of the Dynamic Programming Solution 216

Contents xv

3.4 Robust Min-Max MPC . . . . . . . . . . . . . . . . . . . . . . 220

3.5 Tube-Based Robust MPC . . . . . . . . . . . . . . . . . . . . 223

3.5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 223

3.5.2 Outer-Bounding Tube for a Linear System with

Additive Disturbance . . . . . . . . . . . . . . . . . . 224

3.5.3 Tube-BasedMPC of Linear Systemswith Additive

Disturbances . . . . . . . . . . . . . . . . . . . . . . . 228

3.5.4 Improved Tube-BasedMPC of Linear Systemswith

Additive Disturbances . . . . . . . . . . . . . . . . . 234

3.6 Tube-Based MPC of Nonlinear Systems . . . . . . . . . . . 236

3.6.1 The Nominal Trajectory . . . . . . . . . . . . . . . . 238

3.6.2 Model Predictive Controller . . . . . . . . . . . . . . 238

3.6.3 Choosing the Nominal Constraint Sets ÅU and ÅX . . 242

3.7 Stochastic MPC . . . . . . . . . . . . . . . . . . . . . . . . . . 246

3.7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 246

3.7.2 Stability of Stochastic MPC . . . . . . . . . . . . . . 248

3.7.3 Tube-based stochastic MPC . . . . . . . . . . . . . . 250

3.8 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

3.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

4 State Estimation 269

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

4.2 Full Information Estimation . . . . . . . . . . . . . . . . . . 269

4.2.1 Nominal Estimator Stability . . . . . . . . . . . . . . 279

4.2.2 Robust Estimator Stability . . . . . . . . . . . . . . . 284

4.2.3 InterludeÐLinear System Review . . . . . . . . . . . 287

4.3 Moving Horizon Estimation . . . . . . . . . . . . . . . . . . 292

4.3.1 Zero Prior Weighting . . . . . . . . . . . . . . . . . . 293

4.3.2 Nonzero Prior Weighting . . . . . . . . . . . . . . . . 296

4.3.3 RGES of MHE under exponential assumptions . . . 297

4.4 Other Nonlinear State Estimators . . . . . . . . . . . . . . . 302

4.4.1 Particle Filtering . . . . . . . . . . . . . . . . . . . . . 302

4.4.2 Extended Kalman Filtering . . . . . . . . . . . . . . . 302

4.4.3 Unscented Kalman Filtering . . . . . . . . . . . . . . 304

4.4.4 EKF, UKF, and MHE Comparison . . . . . . . . . . . 306

4.5 On combining MHE and MPC . . . . . . . . . . . . . . . . . 312

4.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

4.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321

xvi Contents

5 Output Model Predictive Control 333

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333

5.2 A Method for Output MPC . . . . . . . . . . . . . . . . . . . 335

5.3 Linear Constrained Systems: Time-Invariant Case . . . . 338

5.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 338

5.3.2 State Estimator . . . . . . . . . . . . . . . . . . . . . . 338

5.3.3 Controlling Ãx . . . . . . . . . . . . . . . . . . . . . . . 340

5.3.4 Output MPC . . . . . . . . . . . . . . . . . . . . . . . . 342

5.3.5 Computing the Tightened Constraints . . . . . . . 346

5.4 Linear Constrained Systems: Time-Varying Case . . . . . 347

5.5 Offset-Free MPC . . . . . . . . . . . . . . . . . . . . . . . . . 347

5.5.1 Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 349

5.5.2 Control . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

5.5.3 Convergence Analysis . . . . . . . . . . . . . . . . . 354

5.6 Nonlinear Constrained Systems . . . . . . . . . . . . . . . . 357

5.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 357

5.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360

6 Distributed Model Predictive Control 363

6.1 Introduction and Preliminary Results . . . . . . . . . . . . 363

6.1.1 Least Squares Solution . . . . . . . . . . . . . . . . . 364

6.1.2 Stability of Suboptimal MPC . . . . . . . . . . . . . . 369

6.2 Unconstrained Two-Player Game . . . . . . . . . . . . . . . 374

6.2.1 Centralized Control . . . . . . . . . . . . . . . . . . . 376

6.2.2 Decentralized Control . . . . . . . . . . . . . . . . . 377

6.2.3 Noncooperative Game . . . . . . . . . . . . . . . . . 378

6.2.4 Cooperative Game . . . . . . . . . . . . . . . . . . . . 386

6.2.5 Tracking Nonzero Setpoints . . . . . . . . . . . . . . 392

6.2.6 State Estimation . . . . . . . . . . . . . . . . . . . . . 399

6.3 Constrained Two-Player Game . . . . . . . . . . . . . . . . 400

6.3.1 Uncoupled Input Constraints . . . . . . . . . . . . . 402

6.3.2 Coupled Input Constraints . . . . . . . . . . . . . . 405

6.3.3 Exponential Convergence with Estimate Error . . . 407

6.3.4 Disturbance Models and Zero Offset . . . . . . . . 409

6.4 Constrained M-Player Game . . . . . . . . . . . . . . . . . . 413

6.5 Nonlinear Distributed MPC . . . . . . . . . . . . . . . . . . . 415

6.5.1 Nonconvexity . . . . . . . . . . . . . . . . . . . . . . . 415

6.5.2 Distributed Algorithm for Nonconvex Functions . 417

6.5.3 Distributed Nonlinear Cooperative Control . . . . 419

6.5.4 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . 422

Contents xvii

6.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

6.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429

7 Explicit Control Laws for Constrained Linear Systems 445

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

7.2 Parametric Programming . . . . . . . . . . . . . . . . . . . . 446

7.3 Parametric Quadratic Programming . . . . . . . . . . . . . 451

7.3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . 451

7.3.2 Preview . . . . . . . . . . . . . . . . . . . . . . . . . . 452

7.3.3 Optimality Condition for a Convex Program . . . . 453

7.3.4 Solution of the Parametric Quadratic Program . . 456

7.3.5 Continuity of V0� and u0� . . . . . . . . . . . . 4607.4 Constrained Linear Quadratic Control . . . . . . . . . . . . 461

7.5 Parametric Piecewise Quadratic Programming . . . . . . . 463

7.6 DP Solution of the Constrained LQ Control Problem . . . 469

7.7 Parametric Linear Programming . . . . . . . . . . . . . . . 470

7.7.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . 470

7.7.2 Minimizer u0x is Unique for all x 2 X . . . . . . 4727.8 Constrained Linear Control . . . . . . . . . . . . . . . . . . 475

7.9 Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

7.10 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477

7.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478

8 Numerical Optimal Control 485

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485

8.1.1 Discrete Time Optimal Control Problem . . . . . . 486

8.1.2 Convex Versus Nonconvex Optimization . . . . . . 487

8.1.3 Simultaneous Versus Sequential Optimal Control 490

8.1.4 Continuous Time Optimal Control Problem . . . . 492

8.2 Numerical Simulation . . . . . . . . . . . . . . . . . . . . . . 495

8.2.1 Explicit Runge-Kutta Methods . . . . . . . . . . . . . 496

8.2.2 Stiff Equations and Implicit Integrators . . . . . . . 500

8.2.3 Implicit Runge-Kutta and Collocation Methods . . 501

8.2.4 Differential Algebraic Equations . . . . . . . . . . . 505

8.2.5 Integrator Adaptivity . . . . . . . . . . . . . . . . . . 507

8.3 Solving Nonlinear Equation Systems . . . . . . . . . . . . . 507

8.3.1 Linear Systems . . . . . . . . . . . . . . . . . . . . . . 507

8.3.2 Nonlinear Root-Finding Problems . . . . . . . . . . 508

8.3.3 Local Convergence of Newton-Type Methods . . . 511

8.3.4 Af®ne Invariance . . . . . . . . . . . . . . . . . . . . . 513

8.3.5 Globalization for Newton-Type Methods . . . . . . 513

xviii Contents

8.4 Computing Derivatives . . . . . . . . . . . . . . . . . . . . . 514

8.4.1 Numerical Differentiation . . . . . . . . . . . . . . . 515

8.4.2 Algorithmic Differentiation . . . . . . . . . . . . . . 516

8.4.3 Implicit Function Interpretation . . . . . . . . . . . 517

8.4.4 Algorithmic Differentiation in Forward Mode . . . 520

8.4.5 Algorithmic Differentiation in Reverse Mode . . . 522

8.4.6 Differentiation of Simulation Routines . . . . . . . 525

8.4.7 Algorithmic and Symbolic Differentiation Software 527

8.4.8 CasADi for Optimization . . . . . . . . . . . . . . . . 527

8.5 Direct Optimal Control Parameterizations . . . . . . . . . 530

8.5.1 Direct Single Shooting . . . . . . . . . . . . . . . . . 532

8.5.2 Direct Multiple Shooting . . . . . . . . . . . . . . . . 534

8.5.3 Direct Transcription and Collocation Methods . . 538

8.6 Nonlinear Optimization . . . . . . . . . . . . . . . . . . . . . 542

8.6.1 Optimality Conditions and Perturbation Analysis 543

8.6.2 Nonlinear Optimization with Equalities . . . . . . . 546

8.6.3 Hessian Approximations . . . . . . . . . . . . . . . . 547

8.7 Newton-Type Optimization with Inequalities . . . . . . . 550

8.7.1 Sequential Quadratic Programming . . . . . . . . . 551

8.7.2 Nonlinear Interior Point Methods . . . . . . . . . . 552

8.7.3 Comparison of SQP and Nonlinear IP Methods . . 554

8.8 Structure in Discrete Time Optimal Control . . . . . . . . 555

8.8.1 Simultaneous Approach . . . . . . . . . . . . . . . . 556

8.8.2 Linear Quadratic Problems (LQP) . . . . . . . . . . . 558

8.8.3 LQP Solution by Riccati Recursion . . . . . . . . . . 558

8.8.4 LQP Solution by Condensing . . . . . . . . . . . . . 560

8.8.5 Sequential Approaches and Sparsity Exploitation 562

8.8.6 Differential Dynamic Programming . . . . . . . . . 564

8.8.7 Additional Constraints in Optimal Control . . . . . 566

8.9 Online Optimization Algorithms . . . . . . . . . . . . . . . 567

8.9.1 General Algorithmic Considerations . . . . . . . . . 568

8.9.2 Continuation Methods and Real-Time Iterations . 571

8.10 Discrete Actuators . . . . . . . . . . . . . . . . . . . . . . . . 574

8.11 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579

8.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581

Author Index 600

Citation Index 608

Subject Index 614

Contents xix

A Mathematical Background 624

A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 624

A.2 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 624

A.3 Range and Nullspace of Matrices . . . . . . . . . . . . . . . 624

A.4 Linear Equations Ð Existence and Uniqueness . . . . . . 625

A.5 Pseudo-Inverse . . . . . . . . . . . . . . . . . . . . . . . . . . 625

A.6 Partitioned Matrix Inversion Theorem . . . . . . . . . . . . 628

A.7 Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . . . . 629

A.8 Norms in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . 631

A.9 Sets in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 631

A.10Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 632

A.11Continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633

A.12Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 636

A.13Convex Sets and Functions . . . . . . . . . . . . . . . . . . . 641

A.13.1Convex Sets . . . . . . . . . . . . . . . . . . . . . . . . 641

A.13.2Convex Functions . . . . . . . . . . . . . . . . . . . . 646

A.14Differential Equations . . . . . . . . . . . . . . . . . . . . . . 648

A.15Random Variables and the Probability Density . . . . . . 654

A.16Multivariate Density Functions . . . . . . . . . . . . . . . . 659

A.16.1Statistical Independence and Correlation . . . . . . 668

A.17Conditional Probability and Bayes's Theorem . . . . . . . 672

A.18Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 678

B Stability Theory 693

B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693

B.2 Stability and Asymptotic Stability . . . . . . . . . . . . . . 696

B.3 Lyapunov Stability Theory . . . . . . . . . . . . . . . . . . . 700

B.3.1 Time-Invariant Systems . . . . . . . . . . . . . . . . 701

B.3.2 Time-Varying, Constrained Systems . . . . . . . . . 707

B.3.3 Upper boundingK functions . . . . . . . . . . . . . 709B.4 Robust Stability . . . . . . . . . . . . . . . . . . . . . . . . . 709

B.4.1 Nominal Robustness . . . . . . . . . . . . . . . . . . 709

B.4.2 Robustness . . . . . . . . . . . . . . . . . . . . . . . . 711

B.5 Control Lyapunov Functions . . . . . . . . . . . . . . . . . . 713

B.6 Input-to-State Stability . . . . . . . . . . . . . . . . . . . . . 717

B.7 Output-to-State Stability and Detectability . . . . . . . . . 719

B.8 Input/Output-to-State Stability . . . . . . . . . . . . . . . . 720

B.9 Incremental-Input/Output-to-State Stability . . . . . . . . 722

B.10 Observability . . . . . . . . . . . . . . . . . . . . . . . . . . . 722

B.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 724

xx Contents

C Optimization 729

C.1 Dynamic Programming . . . . . . . . . . . . . . . . . . . . . 729

C.1.1 Optimal Control Problem . . . . . . . . . . . . . . . 731

C.1.2 Dynamic Programming . . . . . . . . . . . . . . . . . 733

C.2 Optimality Conditions . . . . . . . . . . . . . . . . . . . . . 737

C.2.1 Tangent and Normal Cones . . . . . . . . . . . . . . 737

C.2.2 Convex Optimization Problems . . . . . . . . . . . . 741

C.2.3 Convex Problems: Polyhedral Constraint Set . . . 743

C.2.4 Nonconvex Problems . . . . . . . . . . . . . . . . . . 745

C.2.5 Tangent and Normal Cones . . . . . . . . . . . . . . 746

C.2.6 Constraint Set De®ned by Inequalities . . . . . . . 750

C.2.7 Constraint Set; Equalities and Inequalities . . . . . 753

C.3 Set-Valued Functions and Continuity of Value Function . 755

C.3.1 Outer and Inner Semicontinuity . . . . . . . . . . . 757

C.3.2 Continuity of the Value Function . . . . . . . . . . . 759

C.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767

List of Figures

1.1 System with input u, output y , and transfer function ma-

trix G connecting them; the model is y Gu. . . . . . . . . 31.2 Typical input constraint sets U for (a) continuous actua-

tors and (b) mixed continuous/discrete actuators. . . . . . 9

1.3 Output of a stochastic system versus time. . . . . . . . . . . 10

1.4 Two quadratic functions and their sum. . . . . . . . . . . . . 15

1.5 Schematic of the moving horizon estimation problem. . . . 39

1.6 MPC controller consisting of: receding horizon regulator,

state estimator, and target selector. . . . . . . . . . . . . . . 52

1.7 Schematic of the well-stirred reactor. . . . . . . . . . . . . . . 54

1.8 Three measured outputs versus time after a step change

in inlet ¯owrate at 10 minutes; nd 2. . . . . . . . . . . . . 571.9 Two manipulated inputs versus time after a step change

in inlet ¯owrate at 10 minutes; nd 2. . . . . . . . . . . . . 571.10 Three measured outputs versus time after a step change

in inlet ¯owrate at 10 minutes; nd 3. . . . . . . . . . . . . 581.11 Two manipulated inputs versus time after a step change

in inlet ¯owrate at 10 minutes; nd 3. . . . . . . . . . . . . 591.12 Plug-¯ow reactor. . . . . . . . . . . . . . . . . . . . . . . . . . . 60

1.13 Pendulum with applied torque. . . . . . . . . . . . . . . . . . 62

1.14 Feedback control system with output disturbance d, and

setpoint ysp. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

2.1 Example of MPC. . . . . . . . . . . . . . . . . . . . . . . . . . . 101

2.2 Feasible region U2, elliptical cost contours and ellipsecenter ax, and constrained minimizers for different

values of x. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

2.3 First element of control constraint set U3x (shaded)and control law �3x (line) versus x cos�; sin�,� 2 ��;� on the unit circle for a nonlinear systemwithterminal constraint. . . . . . . . . . . . . . . . . . . . . . . . . 106

2.4 Optimal cost V03 x versus x on the unit circle. . . . . . . . 107

xxi

xxii List of Figures

2.5 Closed-loop economic MPC versus tracking MPC starting

at x �8;8 with optimal steady state 8;4. Both con-trollers asymptotically stabilize the steady state. Dashed

contours show cost functions for each controller. . . . . . . 159

2.6 Closed-loop evolution under economic MPC. The rotated

cost function Ve 0 is a Lyapunov function for the system. . . 1602.7 Diagram of tank/cooler system. Each cooling unit can be

either on or off, and if on, it must be between its (possibly

nonzero) minimum and maximum capacities. . . . . . . . . 163

2.8 Feasible sets XN for two values of ÇQmin. Note that forÇQmin 9 (right-hand side), XN for N � 4 are discon-nected sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

2.9 Phase portrait for closed-loop evolution of cooler system

with ÇQmin 9. Line colors show value of discrete actuatoru2. 1652.10 Region of attraction (shaded region) for constrained MPC

controller of Exercise 2.6. . . . . . . . . . . . . . . . . . . . . . 174

2.11 The region Xf , in which the unconstrained LQR control

law is feasible for Exercise 2.7. . . . . . . . . . . . . . . . . . . 175

2.12 The region of attraction for terminal constraint xN 2Xf and terminal penalty Vf x 1=2x0�x and the es-timate of ÅXN for Exercise 2.8. . . . . . . . . . . . . . . . . . . 177

2.13 Inconsistent setpoint xsp; usp, unreachable stage cost

`x;u, and optimal steady states xs ; us, and stage costs

`sx;u for constrained and unconstrained systems. . . . 181

2.14 Stage cost versus time for the case of unreachable setpoint. 182

2.15 Rotated cost-function contour èx;u 0 (circles) for� 0;�8;�12. Shaded region shows feasible regionwhereèx;u < 0. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

3.1 Open-loop and feedback trajectories. . . . . . . . . . . . . . . 198

3.2 The sets XN , Rb, and Rc . . . . . . . . . . . . . . . . . . . . . . 2143.3 Outer-bounding tube Xz; Åu. . . . . . . . . . . . . . . . . . . . 228

3.4 Minimum feasible � for varying N. Note that we require

� 2 0;1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2323.5 Bounds on tightened constraint set ÅZ for varyingN. Bounds

are jx1j � �1, jx2j � �2, and juj � �. . . . . . . . . . . . . . . 2333.6 Comparison of 100 realizations of standard and tube-

based MPC for the chemical reactor example. . . . . . . . . 244

3.7 Comparison of standard and tube-based MPC with an ag-

gressive model predictive controller. . . . . . . . . . . . . . . 245

List of Figures xxiii

3.8 Concentration versus time for the ancillary model predic-

tive controller with sample time � 12 (left) and � 8(right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

3.9 Observed probability "test of constraint violation for i 10. Distribution is based on 500 trials for each value of

". Dashed line shows the outcome predicted by formula

(3.25), i.e., "test ". . . . . . . . . . . . . . . . . . . . . . . . . . 2553.10 Closed-loop robust MPC state evolution with uniformly

distributed jwj � 0:1 from four different x0. . . . . . . . . . 263

4.1 Smoothing update. . . . . . . . . . . . . . . . . . . . . . . . . . 299

4.2 Comparison of ®ltering and smoothing updates for the

batch reactor system. Second column shows absolute es-

timate error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

4.3 Evolution of the state (solid line) and EKF state estimate

(dashed line). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308

4.4 Evolution of the state (solid line) and UKF state estimate

(dashed line). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309

4.5 Evolution of the state (solid line) and MHE state estimate

(dashed line). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

4.6 Perturbed trajectories terminating in Xf . . . . . . . . . . . . 315

4.7 Closed-loop performance of combined nonlinearMHE/MPC

with no disturbances. First column shows system states,

and second column shows estimation error. Dashed line

shows concentration setpoint. Vertical lines indicate times

of setpoint changes. . . . . . . . . . . . . . . . . . . . . . . . . 317

4.8 Closed-loop performance of combined nonlinearMHE/MPC

for varying disturbance size. The system is controlled be-

tween two steady states. . . . . . . . . . . . . . . . . . . . . . . 318

5.1 State estimator tube. The solid line Ãxt is the center of

the tube, and the dashed line is a sample trajectory of xt. 336

5.2 The system with disturbance. The state estimate lies in

the inner tube, and the state lies in the outer tube. . . . . . 337

6.1 Convex step from up1 ; u

p2 to u

p11 ; u

p12 . . . . . . . . . . 380

6.2 Ten iterations of noncooperative steady-state calculation. . 397

6.3 Ten iterations of cooperative steady-state calculation. . . . 397

6.4 Ten iterations of noncooperative steady-state calculation;

reversed pairing. . . . . . . . . . . . . . . . . . . . . . . . . . . 398

xxiv List of Figures

6.5 Ten iterations of cooperative steady-state calculation; re-

versed pairing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398

6.6 Cooperative control stuck on the boundary of U under

coupled constraints . . . . . . . . . . . . . . . . . . . . . . . . 406

6.7 Cost contours for a two-player, nonconvex game. . . . . . . 416

6.8 Nonconvex function optimized with the distributed gra-

dient algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . 419

6.9 Closed-loop state and control evolutionwith x10; x20 3;�3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425

6.10 Contours of Vx0;u1;u2 for N 1. . . . . . . . . . . . . . 4266.11 Optimizing a quadratic function in one set of variables at

a time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

6.12 Constrained optimality conditions and the normal cone. . 439

7.1 The sets Z, X, and Ux. . . . . . . . . . . . . . . . . . . . . . 4487.2 Parametric linear program. . . . . . . . . . . . . . . . . . . . . 448

7.3 Unconstrained parametric quadratic program. . . . . . . . . 449

7.4 Parametric quadratic program. . . . . . . . . . . . . . . . . . . 449

7.5 Polar cone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

7.6 Regions Rx; x 2 X for a second-order example. . . . . . . . 4627.7 Solution to a parametric LP. . . . . . . . . . . . . . . . . . . . 473

7.8 Solution times for explicit and implicit MPC for N 20. . . 480

8.1 Feasible set and reduced objective u0 of the non-

linear MPC Example 8.1. . . . . . . . . . . . . . . . . . . . . . . 490

8.2 Performance of different integration methods. . . . . . . . . 499

8.3 Polynomial approximationxe1t and true trajectoryx1tof the ®rst state and its derivative. . . . . . . . . . . . . . . . 504

8.4 Performance of implicit integration methods on a stiff ODE. 506

8.5 Newton-type iterations for solution of Rz 0 from Ex-ample 8.5. Left: exact Newton method. Right: constant

Jacobian approximation. . . . . . . . . . . . . . . . . . . . . . 510

8.6 Convergence of different sequences as a function of k. . . 512

8.7 Relaxed and binary feasible solution for Example 8.17. . . 578

8.8 A hanging chain at rest. See Exercise 8.6(b). . . . . . . . . . 585

8.9 Direct single shooting solution for (8.65)without path con-

straints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 587

8.10 Open-loop simulation for (8.65) using collocation. . . . . . . 590

8.11 Gauss-Newton iterations for the direct multiple-shooting

method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 592

List of Figures xxv

A.1 The four fundamental subspaces of matrix A . . . . . . . . 626

A.2 Matrix A maps into RA. . . . . . . . . . . . . . . . . . . . . . 627A.3 Pseudo-inverse of A maps into RA0. . . . . . . . . . . . . . 627A.4 Subgradient. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640

A.5 Separating hyperplane. . . . . . . . . . . . . . . . . . . . . . . 642

A.6 Polar cone. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645

A.7 A convex function. . . . . . . . . . . . . . . . . . . . . . . . . . 646

A.8 Normal distribution. . . . . . . . . . . . . . . . . . . . . . . . . 658

A.9 Multivariate normal in two dimensions. . . . . . . . . . . . . 660

A.10 The geometry of quadratic form x0Ax b. . . . . . . . . . . 661A.11 A nearly singular normal density in two dimensions. . . . . 665

A.12 The region Xc for y maxx1; x2 � c. . . . . . . . . . . . 667A.13 A joint density function for the two uncorrelated random

variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670

A.14 The probability distribution and inverse distribution for

random variable �. . . . . . . . . . . . . . . . . . . . . . . . . . 687

B.1 Stability of the origin. . . . . . . . . . . . . . . . . . . . . . . . 697

B.2 An attractive but unstable origin. . . . . . . . . . . . . . . . . 698

C.1 Routing problem. . . . . . . . . . . . . . . . . . . . . . . . . . . 730

C.2 Approximation of the set U . . . . . . . . . . . . . . . . . . . . 738

C.3 Tangent cones. . . . . . . . . . . . . . . . . . . . . . . . . . . . 738

C.4 Normal at u. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739

C.5 Condition of optimality. . . . . . . . . . . . . . . . . . . . . . . 745

C.6 Tangent and normal cones. . . . . . . . . . . . . . . . . . . . . 747

C.7 Condition of optimality. . . . . . . . . . . . . . . . . . . . . . . 749

C.8 FUu 6 TUu. . . . . . . . . . . . . . . . . . . . . . . . . . . . 751C.9 Graph of set-valued function U�. . . . . . . . . . . . . . . . 756C.10 Graphs of discontinuous set-valued functions. . . . . . . . . 757

C.11 Outer and inner semicontinuity of U�. . . . . . . . . . . . . 758C.12 Subgradient of f�. . . . . . . . . . . . . . . . . . . . . . . . . 762

List of Examples and Statements

1.1 Example: Sum of quadratic functions . . . . . . . . . . . . . 15

1.2 Lemma: Hautus lemma for controllability . . . . . . . . . . . 24

1.3 Lemma: LQR convergence . . . . . . . . . . . . . . . . . . . . . 24

1.4 Lemma: Hautus lemma for observability . . . . . . . . . . . 42

1.5 Lemma: Convergence of estimator cost . . . . . . . . . . . . 43

1.6 Lemma: Estimator convergence . . . . . . . . . . . . . . . . . 44

1.7 Assumption: Target feasibility and uniqueness . . . . . . . 48

1.8 Lemma: Detectability of the augmented system . . . . . . . 50

1.9 Corollary: Dimension of the disturbance . . . . . . . . . . . 50

1.10 Lemma: Offset-free control . . . . . . . . . . . . . . . . . . . . 52

1.11 Example: More measured outputs than inputs and zero

offset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

1.12 Lemma: Hautus lemma for stabilizability . . . . . . . . . . . 68

1.13 Lemma: Hautus lemma for detectability . . . . . . . . . . . . 72

1.14 Lemma: Stabilizable systems and feasible targets . . . . . . 83

2.1 Proposition: Continuity of system solution . . . . . . . . . . 94

2.2 Assumption: Continuity of system and cost . . . . . . . . . 97

2.3 Assumption: Properties of constraint sets . . . . . . . . . . 98

2.4 Proposition: Existence of solution to optimal control problem 98

2.5 Example: Linear quadratic MPC . . . . . . . . . . . . . . . . . 99

2.6 Example: Closer inspection of linear quadratic MPC . . . . 101

2.7 Theorem: Continuity of value function and control law . . 104

2.8 Example: Discontinuous MPC control law . . . . . . . . . . . 105

2.9 De®nition: Positive and control invariant sets . . . . . . . . 109

2.10 Proposition: Existence of solutions to DP recursion . . . . . 110

2.11 De®nition: Asymptotically stable and GAS . . . . . . . . . . 112

2.12 De®nition: Lyapunov function . . . . . . . . . . . . . . . . . . 113

2.13 Theorem: Lyapunov stability theorem . . . . . . . . . . . . . 113

2.14 Assumption: Basic stability assumption . . . . . . . . . . . . 114

2.15 Proposition: The value function V0N� is locally bounded . 1152.16 Proposition: Extension of upper bound to XN . . . . . . . . 1152.17 Assumption: Weak controllability . . . . . . . . . . . . . . . . 116

2.18 Proposition: Monotonicity of the value function . . . . . . . 118

2.19 Theorem: Asymptotic stability of the origin . . . . . . . . . 119

xxvii

xxviii List of Examples and Statements

2.20 De®nition: Exponential stability . . . . . . . . . . . . . . . . . 120

2.21 Theorem: Lyapunov function and exponential stability . . 120

2.22 De®nition: Input/output-to-state stable (IOSS) . . . . . . . . 121

2.23 Assumption: Modi®ed basic stability assumption . . . . . . 121

2.24 Theorem: Asymptotic stability with stage cost `y;u . . . 122

2.25 Assumption: Continuity of system and cost; time-varying

case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

2.26 Assumption: Properties of constraint sets; time-varying case 124

2.27 De®nition: Sequential positive invariance and sequential

control invariance . . . . . . . . . . . . . . . . . . . . . . . . . . 125

2.28 Proposition: Continuous system solution; time-varying case 125

2.29 Proposition: Existence of solution to optimal control prob-

lem; time-varying case . . . . . . . . . . . . . . . . . . . . . . . 125

2.30 De®nition: Asymptotically stable andGAS for time-varying

systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

2.31 De®nition: Lyapunov function: time-varying, constrained

case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

2.32 Theorem: Lyapunov theorem for asymptotic stability (time-

varying, constrained) . . . . . . . . . . . . . . . . . . . . . . . . 126

2.33 Assumption: Basic stability assumption; time-varying case 127

2.34 Proposition: Optimal cost decrease; time-varying case . . . 127

2.35 Proposition: MPC cost is less than terminal cost . . . . . . . 127

2.36 Proposition: Optimal value function properties; time-varying

case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

2.37 Assumption: Uniform weak controllability . . . . . . . . . . 128

2.38 Proposition: Conditions for uniform weak controllability . 128

2.39 Theorem: Asymptotic stability of the origin: time-varying

MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

2.40 Lemma: Entering the terminal region . . . . . . . . . . . . . . 146

2.41 Theorem: MPC stability; no terminal constraint . . . . . . . 146

2.42 Proposition: Admissible warm start in Xf . . . . . . . . . . . 149

2.43 Algorithm: Suboptimal MPC . . . . . . . . . . . . . . . . . . . 149

2.44 Proposition: Linking warm start and state . . . . . . . . . . . 150

2.45 De®nition: Asymptotic stability (difference inclusion) . . . 150

2.46 De®nition: Lyapunov function (difference inclusion) . . . . 151

2.47 Proposition: Asymptotic stability (difference inclusion) . . 151

2.48 Theorem: Asymptotic stability of suboptimal MPC . . . . . 151

2.49 Assumption: Continuity of system and cost . . . . . . . . . 154

2.50 Assumption: Properties of constraint sets . . . . . . . . . . 154

2.51 Assumption: Cost lower bound . . . . . . . . . . . . . . . . . 154

List of Examples and Statements xxix

2.52 Proposition: Asymptotic average performance . . . . . . . . 155

2.53 De®nition: Dissipativity . . . . . . . . . . . . . . . . . . . . . . 156

2.54 Assumption: Continuity at the steady state . . . . . . . . . . 157

2.55 Assumption: Strict dissipativity . . . . . . . . . . . . . . . . . 157

2.56 Theorem: Asymptotic stability of economic MPC . . . . . . 157

2.57 Example: Economic MPC versus tracking MPC . . . . . . . . 158

2.58 Example: MPC with mixed continuous/discrete actuators . 162

2.59 Theorem: Lyapunov theorem for asymptotic stability . . . 177

2.60 Proposition: Convergence of state under IOSS . . . . . . . . 178

2.61 Lemma: An equality for quadratic functions . . . . . . . . . 178

2.62 Lemma: Evolution in a compact set . . . . . . . . . . . . . . . 179

3.1 De®nition: Robust global asymptotic stability . . . . . . . . 207

3.2 Theorem: Lyapunov function and RGAS . . . . . . . . . . . . 208

3.3 Theorem: Robust global asymptotic stability and regular-

ization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

3.4 Proposition: Bound for continuous functions . . . . . . . . . 211

3.5 Proposition: Robustness of nominal MPC . . . . . . . . . . . 214

3.6 De®nition: Robust control invariance . . . . . . . . . . . . . 217

3.7 De®nition: Robust positive invariance . . . . . . . . . . . . . 217

3.8 Assumption: Basic stability assumption; robust case . . . . 218

3.9 Theorem: Recursive feasibility of control policies . . . . . . 218

3.10 De®nition: Set algebra and Hausdorff distance . . . . . . . . 224

3.11 De®nition: Robust asymptotic stability of a set . . . . . . . 230

3.12 Proposition: Robust asymptotic stability of tube-based

MPC for linear systems . . . . . . . . . . . . . . . . . . . . . . 230

3.13 Example: Calculation of tightened constraints . . . . . . . . 231

3.14 Proposition: Recursive feasibility of tube-based MPC . . . . 235

3.15 Proposition: Robust exponential stability of improved tube-

based MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235

3.16 Proposition: Implicit satisfaction of terminal constraint . . 239

3.17 Proposition: Properties of the value function . . . . . . . . . 240

3.18 Proposition: Neighborhoods of the uncertain system . . . . 241

3.19 Proposition: Robust positive invariance of tube-basedMPC

for nonlinear systems . . . . . . . . . . . . . . . . . . . . . . . 241

3.20 Example: Robust control of an exothermic reaction . . . . . 243

3.21 Assumption: Stabilizing conditions, stochastic MPC: Ver-

sion 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

3.22 Assumption: Stabilizing conditions, stochastic MPC: Ver-

sion 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

xxx List of Examples and Statements

3.23 Proposition: Expected cost bound . . . . . . . . . . . . . . . . 250

3.24 Assumption: Robust terminal set condition . . . . . . . . . 253

3.25 Example: Constraint tightening via sampling . . . . . . . . . 254

4.1 De®nition: State Estimator . . . . . . . . . . . . . . . . . . . . 271

4.2 De®nition: Robustly globally asymptotically stable esti-

mation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272

4.3 Proposition: RGAS plus convergent disturbances imply

convergent estimates . . . . . . . . . . . . . . . . . . . . . . . . 273

4.4 Example: The Kalman ®lter of a linear system is RGAS . . . 273

4.5 De®nition: i-IOSS . . . . . . . . . . . . . . . . . . . . . . . . . . 275

4.6 Proposition: RGAS estimator implies i-IOSS . . . . . . . . . . 276

4.7 De®nition: i-IOSS Lyapunov function . . . . . . . . . . . . . . 277

4.8 Theorem: i-IOSS and Lyapunov function equivalence . . . . 277

4.9 De®nition: Incremental Stabilizability with respect to stage

cost L� . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2774.10 Assumption: Continuity . . . . . . . . . . . . . . . . . . . . . . 277

4.11 Assumption: Positive-de®nite stage cost . . . . . . . . . . . . 278

4.12 Assumption: Stabilizability . . . . . . . . . . . . . . . . . . . . 278

4.13 Assumption: Detectability . . . . . . . . . . . . . . . . . . . . 278

4.14 De®nition: Q-function for estimation . . . . . . . . . . . . . 283

4.15 Theorem: Q-function theorem for global asymptotic stability 283

4.16 Theorem: Stability of full information estimation . . . . . . 283

4.17 Assumption: Stage cost under disturbances . . . . . . . . . 284

4.18 Assumption: Stabilizability under disturbances . . . . . . . 284

4.19 De®nition: Exponentially i-IOSS . . . . . . . . . . . . . . . . . 285

4.20 De®nition: Robustly globally exponentially stable estimation 285

4.21 Proposition: Equivalent de®nition of RGES . . . . . . . . . . 286

4.22 Assumption: Power-law bounds for stage costs . . . . . . . 286

4.23 Assumption: Exponential stabilizability . . . . . . . . . . . . 286

4.24 Assumption: Exponential detectability . . . . . . . . . . . . . 286

4.25 Theorem: Robust stability of full information estimation . 287

4.26 Lemma: Duality of controllability and observability . . . . . 291

4.27 Theorem: Riccati iteration and regulator stability . . . . . . 291

4.28 De®nition: Observability . . . . . . . . . . . . . . . . . . . . . 293

4.29 De®nition: Final-state observability . . . . . . . . . . . . . . . 294

4.30 De®nition: GloballyK-continuous . . . . . . . . . . . . . . . 2944.31 Proposition: Observable and globalK-continuous imply FSO2944.32 De®nition: RGAS estimation (observable case) . . . . . . . . 295

4.33 Theorem: MHE is RGAS (observable case) . . . . . . . . . . . 295

List of Examples and Statements xxxi

4.34 De®nition: Full information arrival cost . . . . . . . . . . . . 297

4.35 Lemma: MHE and FIE equivalence . . . . . . . . . . . . . . . . 297

4.36 Assumption: MHE prior weighting bounds . . . . . . . . . . 297

4.37 Theorem: MHE is RGES . . . . . . . . . . . . . . . . . . . . . . 298

4.38 Example: Filtering and smoothing updates . . . . . . . . . . 300

4.39 Example: EKF, UKF, and MHE performance comparison . . 306

4.40 De®nition: i-UIOSS . . . . . . . . . . . . . . . . . . . . . . . . . 312

4.41 Assumption: Bounded estimate error . . . . . . . . . . . . . 313

4.42 De®nition: Robust positive invariance . . . . . . . . . . . . . 313

4.43 De®nition: Robust asymptotic stability . . . . . . . . . . . . 314

4.44 De®nition: ISS Lyapunov function . . . . . . . . . . . . . . . . 314

4.45 Proposition: ISS Lyapunov stability theorem . . . . . . . . . 314

4.46 Theorem: Combined MHE/MPC is RAS . . . . . . . . . . . . . 316

4.47 Example: Combined MHE/MPC . . . . . . . . . . . . . . . . . . 317

5.1 De®nition: Positive invariance; robust positive invariance . 339

5.2 Proposition: Proximity of state and state estimate . . . . . 339

5.3 Proposition: Proximity of state estimate and nominal state 341

5.4 Assumption: Constraint bounds . . . . . . . . . . . . . . . . . 342

5.5 Algorithm: Robust control algorithm (linear constrained

systems) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344

5.6 Proposition: Exponential stability of output MPC . . . . . . 345

5.7 Algorithm: Robust control algorithm (offset-free MPC) . . . 353

6.1 Algorithm: Suboptimal MPC (simpli®ed) . . . . . . . . . . . . 369

6.2 De®nition: Lyapunov stability . . . . . . . . . . . . . . . . . . 370

6.3 De®nition: Uniform Lyapunov stability . . . . . . . . . . . . 371

6.4 De®nition: Exponential stability . . . . . . . . . . . . . . . . . 371

6.5 Lemma: Exponential stability of suboptimal MPC . . . . . . 372

6.6 Lemma: Global asymptotic stability and exponential con-

vergence with mixed powers of norm . . . . . . . . . . . . . 373

6.7 Lemma: Converse theorem for exponential stability . . . . 374

6.8 Assumption: Unconstrained two-player game . . . . . . . . 380

6.9 Example: Nash equilibrium is unstable . . . . . . . . . . . . . 383

6.10 Example: Nash equilibrium is stable but closed loop is

unstable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384

6.11 Example: Nash equilibrium is stable and the closed loop

is stable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

6.12 Example: Stability and offset in the distributed target cal-

culation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

xxxii List of Examples and Statements

6.13 Assumption: Constrained two-player game . . . . . . . . . . 401

6.14 Lemma: Global asymptotic stability and exponential con-

vergence of perturbed system . . . . . . . . . . . . . . . . . . 408

6.15 Assumption: Disturbance models . . . . . . . . . . . . . . . . 409

6.16 Lemma: Detectability of distributed disturbance model . . 409

6.17 Assumption: Constrained M-player game . . . . . . . . . . . 414

6.18 Lemma: Distributed gradient algorithm properties . . . . . 418

6.19 Assumption: Basic stability assumption (distributed) . . . . 420

6.20 Proposition: Terminal constraint satisfaction . . . . . . . . 421

6.21 Theorem: Asymptotic stability . . . . . . . . . . . . . . . . . . 423

6.22 Example: Nonlinear distributed control . . . . . . . . . . . . 423

6.23 Lemma: Local detectability . . . . . . . . . . . . . . . . . . . . 437

7.1 De®nition: Polytopic (polyhedral) partition . . . . . . . . . . 450

7.2 De®nition: Piecewise af®ne function . . . . . . . . . . . . . . 450

7.3 Assumption: Strict convexity . . . . . . . . . . . . . . . . . . . 451

7.4 De®nition: Polar cone . . . . . . . . . . . . . . . . . . . . . . . 453

7.5 Proposition: Farkas's lemma . . . . . . . . . . . . . . . . . . . 453

7.6 Proposition: Optimality conditions for convex set . . . . . . 453

7.7 Proposition: Optimality conditions in terms of polar cone 455

7.8 Proposition: Optimality conditions for linear inequalities . 455

7.9 Proposition: Solution of Pw, w 2 R0x . . . . . . . . . . . . 4577.10 Proposition: Piecewise quadratic (af®ne) cost (solution) . . 458

7.11 Example: Parametric QP . . . . . . . . . . . . . . . . . . . . . . 458

7.12 Example: Explicit optimal control . . . . . . . . . . . . . . . . 459

7.13 Proposition: Continuity of cost and solution . . . . . . . . . 461

7.14 Assumption: Continuous, piecewise quadratic function . . 464

7.15 De®nition: Active polytope (polyhedron) . . . . . . . . . . . 465

7.16 Proposition: Solving P using Pi . . . . . . . . . . . . . . . . . 465

7.17 Proposition: Optimality of u0xw in Rx . . . . . . . . . . . . 468

7.18 Proposition: Piecewise quadratic (af®ne) solution . . . . . . 468

7.19 Proposition: Optimality conditions for parametric LP . . . 472

7.20 Proposition: Solution of P . . . . . . . . . . . . . . . . . . . . . 475

7.21 Proposition: Piecewise af®ne cost and solution . . . . . . . 475

8.1 Example: Nonlinear MPC . . . . . . . . . . . . . . . . . . . . . 489

8.2 Example: Sequential approach . . . . . . . . . . . . . . . . . . 492

8.3 Example: Integration methods of different order . . . . . . 498

8.4 Example: Implicit integrators for a stiff ODE system . . . . 505

8.5 Example: Finding a ®fth root with Newton-type iterations . 510

List of Examples and Statements xxxiii

8.6 Example: Convergence rates . . . . . . . . . . . . . . . . . . . 511

8.7 Theorem: Local contraction for Newton-type methods . . . 512

8.8 Corollary: Convergence of exact Newton's method . . . . . 513

8.9 Example: Function evaluation via elementary operations . 516

8.10 Example: Implicit function representation . . . . . . . . . . 518

8.11 Example: Forward algorithmic differentiation . . . . . . . . 520

8.12 Example: Algorithmic differentiation in reverse mode . . . 522

8.13 Example: Sequential optimal control using CasADi from

Octave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528

8.14 Theorem: KKT conditions . . . . . . . . . . . . . . . . . . . . . 543

8.15 Theorem: Strong second-order suf®cient conditions for

optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545

8.16 Theorem: Tangential predictor by quadratic program . . . 545

8.17 Example: MPC with discrete actuator . . . . . . . . . . . . . . 577

A.1 Theorem: Schur decomposition . . . . . . . . . . . . . . . . . 629

A.2 Theorem: Real Schur decomposition . . . . . . . . . . . . . . 630

A.3 Theorem: Bolzano-Weierstrass . . . . . . . . . . . . . . . . . . 632

A.4 Proposition: Convergence of monotone sequences . . . . . 633

A.5 Proposition: Uniform continuity . . . . . . . . . . . . . . . . . 634

A.6 Proposition: Compactness of continuous functions of com-

pact sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 635

A.7 Proposition: Weierstrass . . . . . . . . . . . . . . . . . . . . . 636

A.8 Proposition: Derivative and partial derivative . . . . . . . . 637

A.9 Proposition: Continuous partial derivatives . . . . . . . . . . 638

A.10 Proposition: Chain rule . . . . . . . . . . . . . . . . . . . . . . 638

A.11 Proposition: Mean value theorem for vector functions . . . 638

A.12 De®nition: Convex set . . . . . . . . . . . . . . . . . . . . . . . 641

A.13 Theorem: Caratheodory . . . . . . . . . . . . . . . . . . . . . . 641

A.14 Theorem: Separation of convex sets . . . . . . . . . . . . . . 642

A.15 Theorem: Separation of convex set from zero . . . . . . . . 643

A.16 Corollary: Existence of separating hyperplane . . . . . . . . 643

A.17 De®nition: Support hyperplane . . . . . . . . . . . . . . . . . 644

A.18 Theorem: Convex set and halfspaces . . . . . . . . . . . . . . 644

A.19 De®nition: Convex cone . . . . . . . . . . . . . . . . . . . . . . 644

A.20 De®nition: Polar cone . . . . . . . . . . . . . . . . . . . . . . . 644

A.21 De®nition: Cone generator . . . . . . . . . . . . . . . . . . . . 645

A.22 Proposition: Cone and polar cone generator . . . . . . . . . 645

A.23 Theorem: Convexity implies continuity . . . . . . . . . . . . 647

A.24 Theorem: Differentiability and convexity . . . . . . . . . . . 647

xxxiv List of Examples and Statements

A.25 Theorem: Second derivative and convexity . . . . . . . . . . 647

A.26 De®nition: Level set . . . . . . . . . . . . . . . . . . . . . . . . 648

A.27 De®nition: Sublevel set . . . . . . . . . . . . . . . . . . . . . . 648

A.28 De®nition: Support function . . . . . . . . . . . . . . . . . . . 648

A.29 Proposition: Set membership and support function . . . . . 648

A.30 Proposition: Lipschitz continuity of support function . . . 648

A.31 Theorem: Existence of solution to differential equations . 651

A.32 Theorem: Maximal interval of existence . . . . . . . . . . . . 651

A.33 Theorem: Continuity of solution to differential equation . 651

A.34 Theorem: Bellman-Gronwall . . . . . . . . . . . . . . . . . . . 651

A.35 Theorem: Existence of solutions to forced systems . . . . . 653

A.36 Example: Fourier transform of the normal density. . . . . . 659

A.37 De®nition: Density of a singular normal . . . . . . . . . . . . 662

A.38 Example: Marginal normal density . . . . . . . . . . . . . . . 663

A.39 Example: Nonlinear transformation . . . . . . . . . . . . . . . 666

A.40 Example: Maximum of two random variables . . . . . . . . . 667

A.41 Example: Independent implies uncorrelated . . . . . . . . . 668

A.42 Example: Does uncorrelated imply independent? . . . . . . 669

A.43 Example: Independent and uncorrelated are equivalent

for normals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 671

A.44 Example: Conditional normal density . . . . . . . . . . . . . 674

A.45 Example: More normal conditional densities . . . . . . . . . 675

B.1 De®nition: Equilibrium point . . . . . . . . . . . . . . . . . . . 694

B.2 De®nition: Positive invariant set . . . . . . . . . . . . . . . . . 694

B.3 De®nition: K,K1,KL, and PD functions . . . . . . . . . . 695B.4 De®nition: Local stability . . . . . . . . . . . . . . . . . . . . . 696

B.5 De®nition: Global attraction . . . . . . . . . . . . . . . . . . . 697

B.6 De®nition: Global asymptotic stability . . . . . . . . . . . . . 697

B.7 De®nition: Various forms of stability . . . . . . . . . . . . . . 698

B.8 De®nition: Global asymptotic stability (KL version) . . . . . 699

B.9 Proposition: Connection of classical andKL global asymp-

totic stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 699

B.10 De®nition: Various forms of stability (constrained) . . . . . 699

B.11 De®nition: Asymptotic stability (constrained, KL version) . 700

B.12 De®nition: Lyapunov function (unconstrained and con-

strained) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 701

B.13 Theorem: Lyapunov function and GAS (classical de®nition) 702

B.14 Lemma: From PD toK1 function (Jiang and Wang (2002)) 703

List of Examples and Statements xxxv

B.15 Theorem: Lyapunov function and global asymptotic sta-

bility (KL de®nition) . . . . . . . . . . . . . . . . . . . . . . . . 703

B.16 Proposition: Improving convergence (Sontag (1998b)) . . . 705

B.17 Theorem: Converse theorem for global asymptotic stability 705

B.18 Theorem: Lyapunov function for asymptotic stability (con-

strained) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706

B.19 Theorem: Lyapunov function for exponential stability . . . 706

B.20 Lemma: Lyapunov function for linear systems . . . . . . . . 707

B.21 De®nition: Sequential positive invariance . . . . . . . . . . . 707

B.22 De®nition: Asymptotic stability (time-varying, constrained) 707

B.23 De®nition: Lyapunov function: time-varying, constrained

case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 708

B.24 Theorem: Lyapunov theorem for asymptotic stability (time-

varying, constrained) . . . . . . . . . . . . . . . . . . . . . . . . 708

B.25 Proposition: Global K function overbound . . . . . . . . . . 709

B.26 De®nition: Nominal robust global asymptotic stability . . . 710

B.27 Theorem: Nominal robust global asymptotic stability and

Lyapunov function . . . . . . . . . . . . . . . . . . . . . . . . . 710

B.28 De®nition: Positive invariance with disturbances . . . . . . 711

B.29 De®nition: Local stability (disturbances) . . . . . . . . . . . . 712

B.30 De®nition: Global attraction (disturbances) . . . . . . . . . . 712

B.31 De®nition: GAS (disturbances) . . . . . . . . . . . . . . . . . . 712

B.32 De®nition: Lyapunov function (disturbances) . . . . . . . . . 712

B.33 Theorem: Lyapunov function for global asymptotic sta-

bility (disturbances) . . . . . . . . . . . . . . . . . . . . . . . . 713

B.34 De®nition: Global control Lyapunov function (CLF) . . . . . 714

B.35 De®nition: Global stabilizability . . . . . . . . . . . . . . . . . 714

B.36 De®nition: Positive invariance (disturbance and control) . . 715

B.37 De®nition: CLF (disturbance and control) . . . . . . . . . . . 715

B.38 De®nition: Control invariance (constrained) . . . . . . . . . 715

B.39 De®nition: CLF (constrained) . . . . . . . . . . . . . . . . . . . 716

B.40 De®nition: Control invariance (disturbances, constrained) 716

B.41 De®nition: CLF (disturbances, constrained) . . . . . . . . . . 716

B.42 De®nition: Input-to-state stable (ISS) . . . . . . . . . . . . . . 717

B.43 De®nition: ISS-Lyapunov function . . . . . . . . . . . . . . . . 718

B.44 Lemma: ISS-Lyapunov function implies ISS . . . . . . . . . . 718

B.45 De®nition: ISS (constrained) . . . . . . . . . . . . . . . . . . . 718

B.46 De®nition: ISS-Lyapunov function (constrained) . . . . . . . 718

B.47 Lemma: ISS-Lyapunov function implies ISS (constrained) . 719

B.48 De®nition: Output-to-state stable (OSS) . . . . . . . . . . . . 720

xxxvi List of Examples and Statements

B.49 De®nition: OSS-Lyapunov function . . . . . . . . . . . . . . . 720

B.50 Theorem: OSS and OSS-Lyapunov function . . . . . . . . . . 720

B.51 De®nition: Input/output-to-state stable (IOSS) . . . . . . . . 721

B.52 De®nition: IOSS-Lyapunov function . . . . . . . . . . . . . . . 721

B.53 Theorem: Modi®ed IOSS-Lyapunov function . . . . . . . . . 721

B.54 Conjecture: IOSS and IOSS-Lyapunov function . . . . . . . . 722

B.55 De®nition: Incremental input/output-to-state stable . . . . 722

B.56 De®nition: Observability . . . . . . . . . . . . . . . . . . . . . 722

B.57 Assumption: Lipschitz continuity of model . . . . . . . . . . 723

B.58 Lemma: Lipschitz continuity and state difference bound . 723

B.59 Theorem: Observability and convergence of state . . . . . . 723

C.1 Lemma: Principle of optimality . . . . . . . . . . . . . . . . . 734

C.2 Theorem: Optimal value function and control law from DP 734

C.3 Example: DP applied to linear quadratic regulator . . . . . . 736

C.4 De®nition: Tangent vector . . . . . . . . . . . . . . . . . . . . 739

C.5 Proposition: Tangent vectors are closed cone . . . . . . . . 739

C.6 De®nition: Regular normal . . . . . . . . . . . . . . . . . . . . 739

C.7 Proposition: Relation of normal and tangent cones . . . . . 740

C.8 Proposition: Global optimality for convex problems . . . . 741

C.9 Proposition: Optimality conditionsÐnormal cone . . . . . . 742

C.10 Proposition: Optimality conditionsÐtangent cone . . . . . 743

C.11 Proposition: Representation of tangent and normal cones . 743

C.12 Proposition: Optimality conditionsÐlinear inequalities . . 744

C.13 Corollary: Optimality conditionsÐlinear inequalities . . . . 744

C.14 Proposition: Necessary condition for nonconvex problem . 746

C.15 De®nition: General normal . . . . . . . . . . . . . . . . . . . . 748

C.16 De®nition: General tangent . . . . . . . . . . . . . . . . . . . . 748

C.17 Proposition: Set of regular tangents is closed convex cone 748

C.18 De®nition: Regular set . . . . . . . . . . . . . . . . . . . . . . . 749

C.19 Proposition: Conditions for regular set . . . . . . . . . . . . 749

C.20 Proposition: Quasiregular set . . . . . . . . . . . . . . . . . . 751

C.21 Proposition: Optimality conditions nonconvex problem . . 752

C.22 Proposition: Fritz-John necessary conditions . . . . . . . . . 753

C.23 De®nition: Outer semicontinuous function . . . . . . . . . . 757

C.24 De®nition: Inner semicontinuous function . . . . . . . . . . 758

C.25 De®nition: Continuous function . . . . . . . . . . . . . . . . . 758

C.26 Theorem: Equivalent conditions for outer and inner semi-

continuity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 759

C.27 Proposition: Outer semicontinuity and closed graph . . . . 759

List of Examples and Statements xxxvii

C.28 Theorem: Minimum theorem . . . . . . . . . . . . . . . . . . . 760

C.29 Theorem: Lipschitz continuity of the value function, con-

stant U . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 761

C.30 De®nition: Subgradient of convex function . . . . . . . . . . 762

C.31 Theorem: Clarke et al. (1998) . . . . . . . . . . . . . . . . . . 762

C.32 Corollary: A bound on du;Ux0 for u 2 Ux . . . . . . 763C.33 Theorem: Continuity of U� . . . . . . . . . . . . . . . . . . . 765C.34 Theorem: Continuity of the value function . . . . . . . . . . 765

C.35 Theorem: Lipschitz continuity of the value functionÐUx 766

Notation

Mathematical notation

9 there exists2 is an element of8 for all=) (= implies; is implied by6=) 6(= does not imply; is not implied bya : b a is de®ned to be equal to b.a : b b is de®ned to be equal to a.� approximately equalV� function VV : A! B V is a function mapping set A into set Bx , Vx function V maps variable x to value Vx

x value of x at next sample time (discrete time system)

Çx time derivative of x (continuous time system)

fx partial derivative of fx with respect to x

r nabla or del operator� unit impulse or delta function

jxj absolute value of scalar; norm of vector (two-norm unlessstated otherwise); induced norm of matrix

x sequence of vector-valued variable x, x0; x1; : : :

kxk sup norm over a sequence, supi�0 jxijkxka:b maxa�i�b jxijtrA trace of matrix A

detA determinant of matrix A

eigA set of eigenvalues of matrix A

�A spectral radius of matrix A, maxi j�ij for �i 2 eigAA�1 inverse of matrix A

Ay pseudo-inverse of matrix A

A0 transpose of matrix A

inf in®mum or greatest lower bound

min minimum

sup supremum or least upper bound

max maximum

xxxix

xl Notation

arg argument or solution of an optimization

s.t. subject to

I integers

I�0 nonnegative integers

In:m integers in the interval n;m

R real numbers

R�0 nonnegative real numbers

Rn real-valued n-vectors

Rm�n real-valuedm�n matricesC complex numbers

B ball in Rn of unit radiusx � px random variable x has probability density pxEx expectation of random variable xvarx variance of random variable x

covx;y covariance of random variables x and y

Nm;P normal distribution (meanm, covariance P ), x � Nm;Pnx;m;P normal probability density, pxx nx;m;P; the empty setaffA af®ne hull of set A

intA interior of set A

coA convex hull of the set A

A closure of set A

leva V sublevel set of function V , fx j Vx � agf � g composition of functions f and g, f � g s : fgsa� b maximum of scalars a and b, Chapter 4Ln

i1 ai a1 � a2 � � � � � an, Chapter 4A� B set addition of sets A and B, Chapters 3 and 5A B set subtraction of set B from set AA n B elements of set A not in set BA[ B union of sets A and BA\ B intersection of sets A and BA � B set A is a subset of set BA � B set A is a superset of set BA � B set A is a proper (or strict) subset of set BA � B set A is a proper (or strict) superset of set Bda;B Distance between element a and set B

dHA;B Hausdorff distance between sets A and B

x & y x % y x converges to y from above (below)satx saturation, satx x if jxj � 1;�1 if x < �1;1 if x > 1

Notation xli

Symbols

A;B;C system matrices, discrete time, x Ax Bu, y CxAc ; Bc system matrices, continuous time, Çx Acx BcuAij state transition matrix for player i to player j

Ai state transition matrix for player i

ALi estimate error transition matrix Ai � LiCiBd input disturbance matrix

Bij input matrix of player i for player j's inputs

Bi input matrix of player i

Cij output matrix of player i for player j's interaction states

Ci output matrix of player i

Cd output disturbance matrix

C controllability matrixC� polar cone of cone C

d integrating disturbance

E; F constraint matrices, Fx Eu � ef ;h system functions, discrete time, x fx;u, y hxfcx;u system function, continuous time, Çx fcx;uFx;u difference inclusion, x 2 Fx;u, F is set valuedG input noise-shaping matrix

Gij steady-state gain of player i to player j

H controlled variable matrix

Ix;u index set of constraints active at x;u

I0x index set of constraints active at x;u0x

k sample time

K optimal controller gain

`x;u stage cost

`Nx;u ®nal stage cost

L optimal estimator gain

m input dimension

M cross-term penalty matrix x0Mu

M number of players, Chapter 6

M class of admissible input policies, � 2Mn state dimension

N horizon length

O observability matrix, Chapters 1 and 4O compact robust control invariant set containing the origin,

Chapter 3

p output dimension

xlii Notation

p optimization iterate, Chapter 6

p� probability density of random variable �

psx sampled probability density, psx P

iwi�x � xiP covariance matrix in the estimator

Pf terminal penalty matrix

P polytopic partition, Chapter 3P polytopic partition, Chapter 7PNx MPC optimization problem; horizon N and initial state x

q importance function in importance sampling

Q state penalty matrix

r controlled variable, r HyR input penalty matrix

s number of samples in a sampled probability density

S input rate of change penalty matrix

Sx;u index set of active polytopes at x;u

S0x index set of active polytopes at x;u0x

t time

T current time in estimation problem

u input (manipulated variable) vector

ue warm start for input sequenceu improved input sequence

UNx control constraint setU input constraint set

v output disturbance, Chapters 1 and 4

v nominal control input, Chapters 3 and 5

VNx;u MPC objective function

V 0Nx MPC optimal value function

VT �;! Full information state estimation objective function at time T

with initial state � and disturbance sequence !

ÃVT �;! MHE objective function at time T with initial state � and distur-

bance sequence !

Vf x terminal penalty

VNz nominal control input constraint setV output disturbance constraint set

w disturbance to the state evolution

wi weights in a sampled probability density, Chapter 4

wi convex weight for player i, Chapter 6

wi normalized weights in a sampled probability density

W class of admissible disturbance sequences, w 2W

Notation xliii

W state disturbance constraint set

x state vector

xi sample values in a sampled probability density

xij state interaction vector from player i to player j

x0 mean of initial state density

Xk;x;� state tube at time k with initial state x and control policy �

Xj set of feasible states for optimal control problem at stage jX state constraint set

Xf terminal region

y output (measurement) vector

Y output constraint set

z nominal state, Chapters 3 and 5

ZT x full information arrival cost

ÃZT x MHE arrival cost

ZeT x MHE smoothing arrival costZ system constraint region, x;u 2 ZZf terminal constraint region, x;u 2 ZfZNx;u constraint set for state and input sequence

Greek letters

�T � MHE prior weighting on state at time T

� sample time

� control law

�j control law at stage j

�f control law applied in terminal region Xf

�ix control law at stage i

�x control policy or sequence of control laws

� output disturbance decision variable in estimation problem

� cost-to-go matrix in regulator, Chapter 1

� covariance matrix in the estimator, Chapter 5

�i objective function weight for player i

�i Solution to Lyapunov equation for player i

�k;x;u state at time k given initial state x and input sequence u

�k;x; i;u state at time k given state at time i is x and input sequence u

�k;x;u;w state at time k given initial state is x, input sequence is u, and

disturbance sequence is w

� state decision variable in estimation problem

! state disturbance decision variable in estimation problem

xliv Notation

Subscripts, superscripts, and accents

Ãx estimate

Ãx� estimate before measurement

xe estimate errorxs steady state

xi subsystem i in a decomposed large-scale system

xsp setpoint

V 0 optimal

Vuc unconstrained

V sp unreachable setpoint

Acronyms

AD algorithmic (or automatic) differentiation

AS asymptotically stable

BFGS Broyden-Fletcher-Goldfarb-Shanno

CIA combinatorial integral approximation

CLF control Lyapunov function

DAE differential algebraic equation

DARE discrete algebraic Riccati equation

DDP differential dynamic programming

DP dynamic programming

END external numerical differentiation

FIE full information estimation

FLOP ¯oating point operation

FSO ®nal-state observable

GAS globally asymptotically stable

GES globally exponentially stable

GL Gauss-Legendre

GPC generalized predict