Networked Control Systems for Robotics and Autonomy

Richard M. MurrayCalifornia Institute of Technology

Presentation to Committee on Autonomy Research for Civil Aviation

28 August 2013Goals• Important trends in control over the last decade• Networked control systems for robotics and autonomy• Potential challenges for autonomy in civilian aviation

Richard M. Murray, Caltech CDS/BENRC ASEB, 28 Aug 2013

Some Important Trends in Control in the Last Decade(Online) Optimization-based control• Increased use of online optimization (MPC/RHC)

• Use knowledge of (current) constraints & environment to allow performance and adaptability

Layering and architectures• Command & control at multiple levels of abstraction

• Modularity in product families via layers

• Platform-based design; contract-based design

Cyber-physical systems (CPS)• Systems that combine information systems/physics

• Better coupling between computer science, controls, communications and networking

Formal methods for analysis, design and synthesis• Formal methods from computer science, adapted for

cyberphysical systems

• Horizontal & vertical contracts for layering, modularity

Components → Systems → Enterprise• Movement of control techniques from “inner loop” to

“outer loop” to entire enterprise (eg, supply chains)

2

Richard M. Murray, Caltech CDSHYCON-EECI, Mar 08

TrajectoryGeneration

Sensing

External Environment

ProcessActuation Feeder:Reliable

StateServer

(KF -> MHE)

Inner Loop(PID, H∞)

Com

man

d:FI

FO

1-3 Gb/s

Sensing

Traj:Causal

ActuatorState:Unreliable

Map:CausalTraj:Causal

10 Mb/s

Goal/Req’s Management

Attention &Awareness

Memory andLearning

Modern Networked Control System Architecture(following P. R. Kumar, UIUC/Texas A&M)

Online Model

StateEstimation

State:Unreliable

StateServer

(KF, MHE)

100 Kb/s

TrajectoryGeneration

OnlineOptimization(RHC, MILP)

Mode andFault

Management

¸

¸¸

3

Richard M. Murray, Caltech CDSTeam Caltech, Jan 08

Recent Example: Alice (DGC07)Alice• 300+ miles of fully autonomous driving• 8 cameras, 8 LADAR, 2 RADAR• 12 Core 2 Duo CPUs + Quad Core• ~75 person team over 18 months

Software• 25 programs with ~200 exec threads• 237,467 lines of executable code

4

Richard M. Murray, Caltech CDSTeam Caltech, Apr 07

Application of existing controls technology in Alice• Receding horizon (optimization-based) control for path

planning with obstacles; ~100 msec iteration rate• Multi-layer sensor fusion: sensor “bus” allows different

combinations of sensors to be used for perceptors + fusion at “map” level• Low level (inner loop) controls: PID w/ anti-windup (but

based on a feasible trajectory from RHC controller)

DGC07 System Architecture

5

FeatureClassificat’n

ElevationMapping

ObstacleDetect/Track

LADAR (6)

Stereo/RoadFinding

GimbaledSensor

World Map

Obstacle Map

Vehicles

PathPlanner

PathFollower

ActuationInterface

TrafficPlanner

MissionPlanner

Vehicle StateEstimator Vehicle

Sensing

Navigation

Systems

ProcessManager

HealthManager

Logging/Visualization Simulation

Properties• Highly modular• Rapidly adaptable• Constantly viable• Robust ???

Linux,TCP/IP, ...

Richard M. Murray, Caltech CDSTeam Caltech, Apr 07

Protocol stack based architecture• Planners uses directives/responses to communicate• Each layer is isolated from the ones above and below• Had 4 different path planners under development, two

different traffic planners.

Engineering principle: layered protocols isolate interactions• Define each layer to have a specific purpose; don’t rely

on knowledge of lower level details• Important to pass information back and forth through

the layers; a fairly in an actuator just generate a change in the path (and perhaps the mission)• Higher layers (not shown) monitor health and can

act as “hormones” (affecting multiple subsystems)

Hybrid system control methodology• Finite state automata control interactions between layers

and mode switches (intersection, off road, etc)• Formal methods for analysis of control protocol correctness (post race)- Eg: make sure that you never have a situation where two layers are in conflict

Planning Hourglass

6

PathPlanner

PathFollower

ActuationInterface

TrafficPlanner

MissionPlanner

Vehicle

Richard M. Murray, Caltech CDSISAT, Feb 2010

Analysis vs Design: Optimization-Based Control

Offline design + analysis ➞ online design• Traditional: design (simple) controller, analyze performance, check with constraints• Modern: specify performance and constraints, design trajectory/control to satisfy• Problem: overall space too large; online optimization allows simplification• Example of a “correct by construction” technique. Cost function = Lyapunov function

Links to complexity management• Correct by construction allows “automatic” verification• Still limited by our ability to formally specify behavior, computational tractability, etc

7

Δ

PlantP

LocalControl

noiseTrajectoryGeneration

refoutput

Local designNonlinear design• global nonlinearities• input saturation• state space constraints

“RHC”

time

state Actualstate

ΔT

T

Computed state

Richard M. Murray, Caltech CDSNCS, 30 Nov 07

MissionPlanner

TrafficPlanner

PathPlanner

Vehicle

PathFollower

Actuation Interface

DR,NP,S STO,NP,S

DR,P,S STO,P,S

no coll.-free path

coll.-free path found

stopped & obs detected

no coll.-free path

coll.-free path found

passing finished orobs disappeared

DR,PR,S

no coll.-free path & #lanes = 1

STO,PR,S

no coll.-free path &#DR,PR,S < M

coll.-free path found

passing finished or obs disappeared

STO,A

BACKUP

no coll.-free path & #lanes > 1

no coll.-free path &#DR,PR,S >= M &#lanes > 1

backup finished& #BACKUP >= N

no coll.-free path& #DR,PR,S >= M

& #lanes = 1

backup finished& #BACKUP < N

DR,Ano coll.-free path

coll.-free path found

STO,BDR,Bno coll.-free path

coll.-free path found

no coll.-free path

FAILED

no coll.-free path & #lanes>1

PAUSEDOFF-ROAD

no coll.-free path& #lanes=1

ROAD REGION

no coll.-free path

coll.-free path forDR,A found

coll.-free path forDR,PR,S found

Challenge: Verification of Control Logic

Function: respond to control commands + DARPA pause/emergency stop

Verification using temporal logic (Lamport’s TLC, TLA+)• Model follower, Actuation Interface, DARPA, accModule, transModule in TLC

• Shared variables: state, estop, acc, acc_command, trans, trans_command

Verify the following properties

• ¨((estop = DISABLE) ⇒ ◊¨(state = DISABLED ∧ acc = -1))

• ¨((estop = PAUSE) ⇒ ◊(state = PAUSED ∨ estop = DISABLE))

• ¨((estop = RUN) ⇒ ◊(state = RUNNING ∨ state = RESUMING))

• ¨((state = RESUMING) ⇒ ◊(state = RUNNING ∨ estop = DISABLE ∨ estop = PAUSE))

• ¨((state ∈ {DISABLE, PAUSED, RESUMING, SHIFTING} ⇒ acc = -1)

Richard M. Murray, Caltech CDSUTC, 27 Oct 2010

Lessons Learned from AliceOnline optimization solves nonlinear control problems• Modern computation allows constrained optimization

problems to be solved online• Solutions exist for situations with more limited computation

(multi-parametric optimization)

Layered control architectures allow more efficient design• Allows for “separation of concerns” between subsystems• Provided a very modular design, capable of rapid (human-

controlled) adaptation

Verification of control protocols is necessary, but hard• Traditional methods of simulation and testing not sufficient• Formal methods not easily applied to “hand designed”

control protocols

New tools for “correct by construction” design are needed• Temporal logic(s) are powerful language for specifying

desired behavior (combined with traditional measures)• New tools are becoming available, but not yet ready for

prime time

9

PathPlanner

PathFollower

ActuationInterface

TrafficPlanner

MissionPlanner

Vehicle

Richard M. Murray, Caltech CDSMuSyC, May 2010

Current Work: Design of Control Protocols

How do we design control protocols that manage behavior• Mixture of discrete and continuous decision making• Insure proper response external events, with unknown timing• Design input = specification + model (system + environment)• Design output = finite state machine implementing logic

Approach: rapidly explore all trajectories satisfying specs• Search through all possible actions and events, discarding

executions that violate a set of (LTL) specifications• Issue: state space explosion (especially due to environment)• Good news: recent results in model checking for class of specs

10

OFF-ROADmode

DR,NP,S STO,NP,Sno collision-free path exists Alice has been stopped for long

enough and there is an obstaclein the vicinity of Alice

passing finished or obstacle disappeared

DR,P,S STO,P,Sno collision-free path exists

collision-free path is foundno collision-free path existsand the number of times Alicehas switched to the DR,P,Rstate near the current positionis less than some threshold

DR,PR,S

no collision-free path exists and thenumber of times Alice has switchedto the DR,P,R state near the currentposition is less than some threshold

collision-free path is foundSTO,PR,S

BACKUP

no collision-freepath exists andthere is morethan one lane

no collision-freepath exists andthere is onlyone lane

backup finishedor failed and thenumber of times Alicehas switched to BACKUPis less than some threshold

DR,P,SSTO,P,Sno collision-free path exists

collision-free path is foundcollision-free path is found

no collision-freepath exists

no collision-free path exists and the number of times Alice has switched to the DR,P,Rstate near the current position exceeds somethreshold and there is more than one lane

no collision-free path exists and the number of times Alice has switched to the DR,P,Rstate near the current position exceeds some threshold and there is only one lane

STO,A

backup finished or failed and thenumber of times Alice has switchedto BACKUP exceeds some threshold

DR,Ano collision-free path exists

no collision-free path exists

collision-free path is found

collision-free path is found

collision-free pathwith DR,A is found

DR,B STO,Bno collision-free path exists

collision-free path is found

no collision-free path existsand there is more than one lane

collision-free path with DR,P,R is found

no collision-free path existsand there is only one lane

passing finished or obstacle disappeared

FAILED PAUSED

ROAD REGION

PathPlanner

PathFollower

ActuationInterface

TrafficPlanner

MissionPlanner

Vehicle

Caltech/MIT/UW V&V MURIAFOSR, 15 June 2011

Temporal Logic Planning (TuLiP) toolboxhttp://tulip-control.sourceforge.net

Python Toolbox• GR(1), LTL specs• Nonlin dynamics• Supports discret-

ization via MPT• Control protocol

designed using JTLV• Receding horizon

compatible

Applications of TuLiP in the last year• Autonomous vehicles - traffic planner (intersections and roads, with other vehicles)• Distributed camera networks - cooperating cameras to track people in region• Electric power transfer - fault-tolerant control of generator + switches + loads

11

Richard M. Murray, Caltech CDSEXCAPE, 13 Jun 2013

Traffic rules• No collisions with other vehicles• Stay in the travel lane unless there is an

obstacle blocking the lane• Only proceed through an intersection

when it is clear

Assumptions• Obstacle may not block a road• Obstacle is detected before the vehicle

gets too close to it• Limited sensing range• Obstacle does not disappear when the

vehicle is in its vicinity• Obstacles may not span more than a

certain number of consecutive cells in the middle of the road• Each intersection is clear infinitely often• Each of the cells marked by star and its adjacent cells are not occupied by an

obstacle infinitely often

12

Example: Autonomous Navigation in Urban Environment

Richard M. Murray, Caltech CDSNASA JSC, Apr 2012

Use response mechanism to replan if no feasible solution exists• Trajectory planner sees blockage and fails to find strategy satisfying specification• Trajectory planner reports failure to goal generator• Goal generator re-computes a (high level) path to the goal state

13

Example: Autonomous Navigation in Urban Environment

PathPlanner

PathFollower

ActuationInterface

TrafficPlanner

MissionPlanner

Vehicle

• JTLV returns 900 state FSAin about 1.5 seconds• Φ = start in proper lane if no

obstacle present ^ no collision

Richard M. Murray, Caltech CDS/BENRC ASEB, 28 Aug 2013

Potential challenges for autonomy in civilian aviationTwo comparisons to previous areas of interest• Control of UAVs in military operations (AFSAB study, 2003)• Challenges in control of autonomous vehicles in urban environments

References• “UAVs in Perspective,” Air Force. Scientific Advisory Board Summer Study, June 2003• M. Campbell, M. Egerstedt, J. P. How, and R. M. Murray, “Autonomous Driving in

Urban Environments: Approaches, Lessons and Challenges,” Philosophical Transactions of the Royal Society - A, Oct. 2009.

14

Richard M. Murray, Caltech CDSUTC, 27 Oct 2010 15

Richard M. Murray, Caltech CDS/BENRC ASEB, 28 Aug 2013

Challenges in Autonomous DrivingSystems integration• Integration of complex sensing, actuation and decision-making subsystems• Need to insure assumptions are consistent across algorithms (and teams)

Prediction and trust• How do we anticipate the actions of other systems (autonomous or human-controlled)• How do we make sure that autonomous systems behave in “understandable” manner

Interactions between agents• Exploit the ability for autonomous (or semi-autonomous) systems to communicate• Unfortunately, cannot assume that all vehicles will cooperate...

Learning• Can autonomous systems learn from prior mistakes, other vehicles, online data?

Scaling up• How do we “scale up” (speed, complexity) from operation in controlled environments

Verification and validation• How do we verify and certify that the system can operate safely and robustly• Particularly hard since we know that this cannot be done 100% of the time

16

Campbell, Egerstedt, How, MPhil Trans Roy Soc A, 2010