1

Automatic deployment of robotic teams from rich specifications

Calin Belta

Hybrid and Networked Systems (HyNeSs) LabDepartments of Mechanical / Systems Engineering

Boston University

2

Rich, natural-language specification, e.g., “USV always avoid P6. UGVs always avoid P4. Any UGV visit P1 or P2 and then go to P3. After P3 is occupied, USV go to P7 and UGV2 start surveilling P1 and P2”

P1P3

P2

P4

P5

UGV1

UGV2

UGV4

UGV3

UAV

USV

UUV

P6

P7

Automatic synthesis of provably-correct control and communication strategies.

Problem 1: Deployment of small heterogeneous teams (<10)

3

“Always obey traffic rules. Visit Road R1 and then Road R2 without crossing intersection I1. If Road R8 is ever visited, then Road R3 must never be reached. Park simultaneously in adjacent parking spaces and remain there for all future times.”

Rich, natural language specification

Robotic Urban-Like Environment (RULE)

Automatic synthesis of provably-correct control and communication strategies.

Problem 1: Deployment of small heterogeneous teams (<10)

4

“Eventually cover an elliptic region with center c1, semiaxes s1 and s2, and orientation θ, then a circular region with center c2 and area a, and maintain this configuration for all future times. Always maintain a pair-wise distance Dmin < D < Dmax. Always avoid obstacles”

Problem 2: Deployment of large homogeneous swarms

Rich, natural-language specification over a small set of global features, e.g.,

Automatic synthesis of control and communication strategies.

5

Motivation

“USV always avoid P6. UGVs always avoid P4. Any UGV visit P1 or P2 and then go to P3. After P3 is occupied, USV go to P7 and UGV2 start visiting P1 and P2, in this order, infinitely often.”

“Always obey traffic rules. Visit Road R1 and then Road R2 without crossing intersection I1. If Road R8 is ever visited, then Road R3 must never be reached. Park simultaneously in adjacent parking spaces and remain there for all future times.”

• Place the human operator as “high” as possible in the decision hierarchy• Formalize the human-robot interaction process

6

ApproachDraw inspiration from formal analysis (verification)

“Is deadlock ever possible?”“If a request is received, make sure it is eventually granted.”

Specification

Process

“Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.”

?

( , )x f x u

• Analysis / control• Compositionality

Model

Model checking (SPIN, NuSMV)

7

Outline

• Finite quotients and control of continuous systems

• Deployment for small teams

• Deployment for swarms

( , )x f x u

Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.”

8

Outline

• Finite quotients and control of continuous systems

• Deployment for small teams

• Deployment for swarms

( , )x f x u

Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.”

9

2 4 3

4 1

3 4 1 3

( ( ))

( (

( (( ) ( )))))

Feedback automaton

Language equivalence!

control

state

Feedback controller

region

1 2 3, ,

1 3,

1

2 3, 4 3 2, ,

23

4

4 2,

1u

2u

3u

4u5u

6u

7u

6u 8u

9u10u

11u12u

13u

14u

Feedback hybrid

automaton

Finite quotients and control of continuous systems

“Avoid the grey region for all times. Visit the blue region, then the green region, and then keep surveying the striped blue and green regions, in this order.”

“(pi2 = TRUE and pi4 = FALSE and pi3 = FALSE) should never happen. Then pi4 = TRUE and then pi1 = TRUE should happen. After that, (pi3 = TRUE and pi4 = TRUE) and then (pi1 = TRUE and pi3 = FALSE) should occur infinitely often.”

10

11

Outline

• Finite quotients and control of continuous systems

• Deployment for small teams

• Deployment for swarms

( , )x f x u

Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.”

12

Deployment of small teamsArbitrary temporal and logic statements about the reachability of regions in a partitioned environment, e.g., “Avoid the blue regions until the green and red regions are simultaneously visited. Then visit any one of the blue regions”

Provably-correct control strategies and communication protocols

Assumptions: • the specification is robot-abstract• the robots can only communicate when in adjacent regions (varying communication constraint)• communication relation is symmetric• all robots in a component of the communication graph can instantaneously communicate

13

“Avoid the blue regions until the green and red regions are simultaneously visited. Then visit any one of the blue regions”

Deployment of small teams

14

“Avoid the blue regions until the green and red regions are simultaneously visited. Then visit any one of the blue regions”

Deployment of small teams

? ? ? ?

???

15

“Do not visit any yellow region until all are simultaneously entered, and always avoid the gray regions”

8 10 16 8 10 16 4 6 15( ) ( ) ( )p p p U p p p p p p

Assume we have three agents

Deployment of small teamsExample

Simulation - movie

16

Outline

• Finite quotients and control of continuous systems

• Deployment for small teams

• Deployment for swarms

( , )x f x u

Always avoid P4. Visit P1 or P2 and then go to P3. Don’t go to P6 unless P1 was visited.”

17

Examples:“Eventually reach a configuration with mean ml < m < mh and covariance s1l < s1 < s1h, s2l < s2 <

s2h, θl < θ < θh and maintain it for all future times. Always maintain a pair-wise distance Dmin < D < Dmax (inter-robot collision avoidance, maintaining the connectivity of the communication graph, coverage). Always avoid obstacles”

Automatic synthesis of provably-correct control and communication strategies.

Communication graph

Sensing range

Deployment for swarms

18

Examples:“Eventually reach a configuration with mean ml < m < mh and covariance s1l < s1 < s1h, s2l < s2 <

s2h, θl < θ < θh and maintain it for all future times. Always maintain a pair-wise distance Dmin < D < Dmax (inter-robot collision avoidance, maintaining the connectivity of the communication graph, coverage). Always avoid obstacles”

Automatic synthesis of provably-correct control and communication strategies.

m θ

s1s2

Deployment for swarms

19

Approach

20

Approach

21

Q

A

q

a

Approach

22

0)( xr

0),( axpx W

: ( , ) 0 ( ) 0x p x a r x

Q

A

Collision avoidance:

Quantifier elimination

( ) 0s a

( ) 0s a

( ) 0s a ( ) 0s a

always always

Approach

23

1( ) 0r x

0),( axpx W

Q

A

1 2: ( , ) 0 ( ) 0 ( ) 0x p x a r x r x always

2 ( ) 0r x

3: ( , ) 0 ( ) 0x p x a r x eventually

and then 4: ( , ) 0 ( ) 0x p x a r x and

3( ) 0r x 4 ( ) 0r x

Approach

24x WQ

A

1 4 2 4,

1 3,

1 3,

3 4, 2 3 4, ,

23

Approach

1 2: ( , ) 0 ( ) 0 ( ) 0x p x a r x r x always

3: ( , ) 0 ( ) 0x p x a r x eventually

and then 4: ( , ) 0 ( ) 0x p x a r x and

)()( 543321

25

Distributed communication architecture

Lower dimensional continuous description

Set of essential featuresof the swarm

Output: provably correct control laws

Continuousabstraction

Hierarchicalabstractionarchitecture

Finite dimensional discrete description

Input: Temporal logic specification over essential features

Discreteabstraction

Provably correct control law

Hierarchical abstractions

26

Distributed communication architecture

Lower dimensional continuous description

Set of essential featuresof the swarm

Output: provably correct control laws

Continuousabstraction

Hierarchicalabstractionarchitecture

Finite dimensional discrete description

Input: Temporal logic specification over essential features

Discreteabstraction

Provably correct control law

Continuous abstraction

?

27

Q

1( )h a q

( ) ( )a dh q u q( )q u q

( )a h q

A

h

( )dh qContinuous abstraction

ConsistencyEquivalent states remain equivalent for all times under the flow

( )q u qq

ActuationAt any point any velocity can be achieved.a A a

Detectability if and only if0q 0a

Correct aggregation = Consistency + Actuation + Detectability

28

{ }W

h

A G

S

Q

Continuous abstraction

• (non – scalability)

• is invariant to robot permutations

• Control architecture independent on the choice of a world frame

•

• is left invariant

• The group G and shape S can be controlled independently

NnA dim

h

),(, sgaSGA

h ( ) ( , ) ( ) ( , ),h q g s h gq gg s g G

29

Continuous abstraction: examples

group: SE(2) (mean, rotation diagonalizing the cov. matrix)shape: spectrum of cov. matrix

group: SE(2) (centroid, rotation diagonalizing the in. tensor)shape: spectrum of in. tensor

group: R2 (centroid)shape: scaling factor (sum of eig)

group: SE(3) (mean, rotation diagonalizing the in. tensor)shape: spectrum of in. tensor

30

Distributed communication architecture

Lower dimensional continuous description

Set of essential featuresof the swarm

Output: provably correct control laws

Continuousabstraction

Hierarchicalabstractionarchitecture

Finite dimensional discrete description

Input: Temporal logic specification over essential features

Discreteabstraction

Provably correct control law

Hierarchical abstraction

31

Fully automated framework for swarm deployment from specifications allowing for:

• Containment, obstacle avoidance, inter-robot collision avoidance, cohesion• Arbitrary tasks given in terms of LTL formulas over linear predicates over mean and variance of the swarm

E

jOiR

ir

{ }W

( )h r a ( , )a s

1

1 N

ii

rN

1

1( ) ( )

NT

i ii

s r rN

Hierarchical abstraction based on mean and variance

2, Uuur iii

32: Nh

32

30 robots with control bounds [-2,2] x [-2,2] in a rectangular environment with two obstacles

Hierarchical abstraction based on mean and varianceExample:

Specification: Always stay inside environment and avoid obstacles andVisit region R1 before reaching region R2 with area greater than 4 andBefore visiting R1, make sure that the pairwise distances are all greater than 0.03

1 : [5.25, 5.75] × [0.25, 0.75]R

2 : [-3.75, -3.25] × [4.25, 4.75]R

33

Acknowledgements

Funding:

National Science Foundation (CNS, IIS, CCF)Air Force Office of Sponsored Research (Computational Mathematics)Army Research Office (Mathematical Sciences Division)

Collaborators:

M. Kloetzer (Boston University)V. Kumar, G. J. Pappas (U Penn)L.C.G.J.M. Habets (CWI, Eindhoven TU)