Multi-Robot TeamsSeparating Teamwork from Taskwork

Gal A. Kaminka

The MAVERICK GroupComputer Science Department

Bar Ilan University, Israel

2galk@cs.biu.ac.il

Teams of Robots and Agents

Rescue, disaster management Coordinated search Mapping

Exploration Space and science applications Satellite formations

Cleaning Vacuum cleaning

Surveillance, Security Defense, civilian Hazardous materials

Sensor, ad-hoc networks Agriculture

Plowing Medical applications

Hospital deliveries

10-15 years ago: Application-specific solutions

3galk@cs.biu.ac.il

Teams of Robots and Agents

Entertainment...

4galk@cs.biu.ac.il

Then, 10-15 Years ago…

Philosophy, Computer Science Theory Joint intentions (Cohen & Levesque 1990, 1991) SharedPlans (Sidner & Grosz 1986, Grosz & Kraus 1996)

Systems needing automated collaboration Robot foraging, coordinated movement (Parker 1993, 1998) Simulated helicopter teams (Tambe 1996)

Team = Teamwork + Taskwork

5galk@cs.biu.ac.il

Soccer Taskwork: Kicking to a target Turning, tackling Tracking the ball, goal …

Soccer Teamwork: Allocating players to roles Synchronizing tactics Sharing relevant information …….

Teamwork and Taskwork

6galk@cs.biu.ac.il

7galk@cs.biu.ac.il

Teamwork Research

Task-Dependent

TeamworkGeneral

Teamwork

8galk@cs.biu.ac.il

MAVERICK Research in teamwork(See elsewhere for plan recognition and social cognition)

Task-Specific Teamwork: Foraging (AIJ 07, ICRA 10)

Coverage (ICRA 05,06,…)

Passing (FSS 01)

Scheduling (SSS 06)

Coordinated movement (AAAI 05,ICRA 06,JOPHA 09)

Patrolling (ICRA 07,AAMAS 08,IJCAI 09)

Multi-robot HRI (ICRA 06)

….

General Teamwork: Monitoring (JAIR 00,02, JAAMAS,

AAMAS 07,ICRA 10)

Fault diagnosis (AAAI 98, IJCAI 03, AAAI 05, AAAI 06)

Synchronization (AAAI 05, ICRA 07)

Allocation (AAMAS 03, AAAI 05, AAMAS 07)

Learning, adaptation (DARS 06) Maintenance (AAMAS 07)

….

9galk@cs.biu.ac.il

MAVERICK Research in teamwork(See elsewhere for plan recognition and social cognition)

Task-Specific Teamwork: Foraging (AIJ 07, ICRA 10)

Coverage (ICRA 05,06,…)

Passing (FSS 01)

Scheduling (SSS 06)

Coordinated movement (AAAI 05,ICRA 06,JOPHA 09)

Patrolling (ICRA 07,AAMAS 08,IJCAI 09)

Multi-robot HRI (ICRA 06)

….

General Teamwork: Monitoring (JAIR 00,02, JAAMAS,

AAMAS 07,ICRA 10)

Fault diagnosis (AAAI 98, IJCAI 03, AAAI 05, AAAI 06)

Synchronization (AAAI 05, ICRA 07)

Allocation (AAMAS 03, AAAI 05, AAMAS 07)

Learning, adaptation (DARS 06) Maintenance (AAMAS 07)

….

10galk@cs.biu.ac.il

Individual Movement in Formation

11galk@cs.biu.ac.il

BDI robot control:A quick review

Each node is a behavior (a controller) Nodes arranged hierarchically (to decompose tasks)

Also sequentially

Every behavior Preconditions End conditions

Position

Find Leader

Ready Leader

Move

Lead

Move Formation

Follow

End

(a)

A behavior graph (a)

12galk@cs.biu.ac.il

Teamwork?

Each robot executing own copy of recipe

No agreement Coordination,

but no collaboration

13galk@cs.biu.ac.il

Principle #1:Synchronizing Plan Execution

Agents must achieve mutual belief with teammates Sidner & Grosz & Kraus 1986,1996; Cohen& Levesque 1990

Team behaviors: Start together, end together Jennings 1995, Tambe 96

Position

Find Leader

Ready Leader

Move

Lead

Move Formation

Follow

End

(a)

14galk@cs.biu.ac.il

Principle #2:Allocate tasks to team-members

Teammates assist each other, take over when failure: Allocate tasks to members (Task Allocation)

e.g., Shehory & Kraus 1995, Dias et al. 2000.

Anticipate needs of teammates Proactive Communications (Yen et al. 2001)

Dynamically re-adjust team to address failures Parker 1998, Tambe 1996, 1997, Kumar et al. 2001

15galk@cs.biu.ac.il

Principle #2:Allocate tasks to team-members

Teammates assist each other, take over when failure: Allocate tasks to members (Task Allocation)

e.g., Shehory & Kraus 1995, Dias et al. 2000.

Anticipate needs of teammates Proactive Communications (Yen et al. 2001)

Dynamically re-adjust team to address failures Parker 1998, Tambe 1996, 1997, Kumar et al. 2001

16galk@cs.biu.ac.il

Organization hierarchy, allocated tasks

Team Hierarchy (b) (Tambe et al. 2000) Behavior Graph (a)

{X,Y,Z}

{X} {Y,Z}

(b)

Position

Find Leader

Ready Leader

Walk

Lead

Move Formation

Follow

End

(a)

Agents can find out with whom to communicate Agents also know who responsible for a behavior

17galk@cs.biu.ac.il

Opportunities for interaction

Behavior graph offers opportunities for protocol use Conditions trigger synchronous selection Decomposition triggers allocation

Position

Find Leader

Ready Leader

Walk

Lead

Move Formation

Follow

End

SynchronizedStart Synchronized

End

Allocation

18galk@cs.biu.ac.il

BITE: Bar Ilan Teamwork Engine (IAS 04, AAAI 05, ICRA 07, AAMAS 07)

Interaction protocols (behaviors) first-class objects Allow using different protocols at different times

Insead of e.g., always vote, always ask leader, …

Context-dependent coordination policies

19galk@cs.biu.ac.il

BITE Components

Team Hierarchy (b)

Behavior Graph (a)

Interaction Behaviors (c)

{X,Y,Z}

{X} {Y,Z}

(b)

Wait

Announce Vote Tally

Ack. Readiness

CFP Bid Select

(c)

Position

Find Leader

Ready Leader

Walk

Lead

Move Formation

Follow

End

(a)

20galk@cs.biu.ac.il

BITE Control Algorithm

Each robot has a copy of the three structures Each robot executes the following control loop

1. push initial behavior onto execution stack2. repeat

a. e current top behavior on stackb. if (e can be decomposed)

push ALLOCATE(children(e))c. else break

3. Execute stack until a behavior b terminates4. SYNC_TERM(b, children(b))5. e SYNC_SELECT(b, followers(b)), push e6. GOTO 2

21galk@cs.biu.ac.il

BITE in Action

22galk@cs.biu.ac.il

Lesson: Flexibility matters (I)

0

688836

1284

1831

2257

290070251

53588

65487

54005 52730

6505958183

0

500

1000

1500

2000

2500

3000

3500

1 2 3 4 5 6 7

Configuration: Same behaviors, different protocols

Pro

toco

l Tim

e

0

10000

20000

30000

40000

50000

60000

70000

80000

Social Time

Task Time

Ta

sk T

ime

24galk@cs.biu.ac.il

Patrolling

Adversarial(w/ Kraus, Agmon)

Frequency-Based(w/ Elmaliach)

Area Perimeter Fence (Border)

ICRA 07, AMAI 10 AAMAS 08 AAMAS 08,

AAMAS 09

IAS 08,

IJCAI 09

25galk@cs.biu.ac.il

Patrolling

Adversarial(w/ Kraus, Agmon)

Frequency-Based(w/ Elmaliach)

Area Perimeter Fence (Border)

ICRA 07, AMAI 10 AAMAS 08 AAMAS 08,

AAMAS 09

IAS 08,

IJCAI 09

27galk@cs.biu.ac.il

Multi-robot, single zone:Individual (unsynchronized) patrol

edcba

28galk@cs.biu.ac.il

Multi-robot, single zoneSynchronized patrol

edcba

Theorem: Synchronized patrol is better.(response time is at most half-segment.)

29galk@cs.biu.ac.il

Multi robots, multi zones (overlap):

Internal points visited by multiple robots Improved uniformity everywhere except for edges Low frequency in edges

Important for system design, placement of cameras

30galk@cs.biu.ac.il

Responding to break-ins, faults

Detected by the robots or operator Affected robot is committed

Sends a message to others

Other re-allocate segments Maintain frequency (now reduced)

Many issues in practice: Intermittent minor stops vs. events Noise in the actual velocity of the robots Turning points, communication ranges

31galk@cs.biu.ac.il

Multi-robot, single zone(Fence Break)

33galk@cs.biu.ac.il

Frequency function(we know math!)

Timep)l,r,p,v,o,s,n,t,d) =

34galk@cs.biu.ac.il

Principle #3:Organizational Maintenance

Teammates assist each other, take over when failure: Allocate tasks to members (Task Allocation)

e.g., Shehory & Kraus 1995, Dias et al. 2000.

Dynamically re-adjust team to address failures Parker 1998, Tambe 1996, 1997, Kumar et al. 2001

Anticipate needs of teammates Proactive Communications (Yen et al. 2001)

35galk@cs.biu.ac.il

Task: Formation Maintenance

Separation-Bearing Formation Control (SBC):

One robot assigned to lead the group (leader) Others maintain distance and angle to peers (targets)

Each robot monitors single target

Monitoring rules imply a control graph Leader node: Out degree of 0 Other nodes: Out degree of 1 Connected, path from all to leader Proven stable (e.g., Fierro et al. 2001)

Target of 0

2

1 3

0

LeaderTarget of 1Target of 3

36galk@cs.biu.ac.il

No

Diamond Formation

Possible Control Graphs

37galk@cs.biu.ac.il

Generating an Organizational Structure:

Control Graph Generation (AAAI 05, ICRA 06)

Key idea:

Generate a multi-graph containing all possible control-graphs

Then, induce a single control graph

Multi-graph:

• Vertices are robots in formation positions

• Directed edge (r1,r2) implies r1 can sense r2.

• Multiple edges between a pair of vertices allowed

– Correspond to multiple sensing options

– According to sensing morphology of robot

38galk@cs.biu.ac.il

Sensing Costs: Setting edge weights

• A robot can have multiple sensors (edges)– Each sensor has range, field of view, pan angle– Cost associated with use to monitor target

• Edge weights correspond to sensor usage costs

• Induce minimum-cost organization:– Min weight path from each vertex to leader– Single target shortest paths (Dijkstra)

39galk@cs.biu.ac.il

Re-organization: Dynamic Switching of Control Graphs

Called when robot (or more) loses its target Frequent failures update costs

Recovery algorithm updates control graph

Glick and Kaminka,ICRA 2006

40galk@cs.biu.ac.il

Formation Results

No Reorganization Reorganization

41galk@cs.biu.ac.il

Time spent in formation

Significantly higher percent of time (two-tailed t-test, p < 2.40109e−15)

Better

Reorganization

No Reorganization

42galk@cs.biu.ac.il

Time to complete task

Significantly shorter duration in all cases(two-tailed t-test, p < 5e-13)

Better

No Reorganization

Reorganization

43galk@cs.biu.ac.il

BITE Revisited

Common lessons Synchronized execution is better for task Balancing load among homogeneous robots Improved robustness, bug-free development

60% reduction in software engineering effort!

Is this it?

44galk@cs.biu.ac.il

MAVERICK Research

Task-Specific Teamwork: Foraging (ECAI 04)

Coverage (ICRA 05,06)

Passing (FSS 01)

Scheduling (SSS 06)

Coordinated movement (AAAI 05, ICRA 06)

Patrolling (ICRA 07, 08)

….

General Teamwork: Monitoring (JAIR 00,02, JAAMAS,

AAMAS 07)

Fault diagnosis (AAAI 98, IJCAI 03, AAAI 05, AAAI 06)

Synchronization (AAAI 05, ICRA 07)

Allocation (AAMAS 03, AAAI 05, AAMAS 07)

Learning, adaptation (DARS 06) Maintenance (AAMAS 07)

….

45galk@cs.biu.ac.il

Multi-Robot Coverage:Can we separate teamwork from taskwork?

Goal: Visit each point in an area Applications: Floor cleaning, mowing, de-mining, painting …. Many approaches, distinguishing properties:

Off-line/on-line, heuristic/complete, approximate/exact

Existing literature: Typical goal: Increase efficiency Often, no analytically proven robustness to catastrophic failures

Catastrophic: Complete robot failure Some systems show robustness empirically

46galk@cs.biu.ac.il

MSTC: Multirobot Spanning-Tree Coverage(ICRA 2005, ICRA 2006, ICRA 2006, RAS 08, AMAI 09)

Family of multi-robot coverage algorithms Off-line, on-line Backtracking, non-backtracking Allow for heterogeneous robots, fuel/speed constraints

Common theme: Guaranteed Robustness

Will complete as long as one robot is alive

47galk@cs.biu.ac.il

Spanning-Tree CoverageSingle robot [Gabriely and Rimon, 2001] Robots with coverage tool of size D Grid terrains, each cell of size D Switch to coarse grid, each cell of size 4D Draw Spanning Tree (ST) on the coarse grid Robot moves along ST path

A Hamiltonian cycle!

A

48galk@cs.biu.ac.il

Non-backtracking MSTC

• Initialization phase: Build STC, distribute to robots

• Distributed execution: Each robot follows its section– Low risk of collisions

Robot A is done!

Robot B is done!

Robot C is done!

A

B

C

49galk@cs.biu.ac.il

Coverage completed as long as one robot is alive Teamwork requirement:

Need to know task completion status of others

Guaranteed robustness

A

B

C

50galk@cs.biu.ac.il

Analysis: Non-backtracking MSTC

• Running time = max i k step(i)

• Best case:

• Worst case: n – k– Unfortunately, common case

• General non-backtracking worst case: n - 2(k-1) - 1

1k

n

A

D

BC

A

D B

C

A

D

B

C

51galk@cs.biu.ac.il

Backtracking MSTC

• Similar initialization phase

• Robots backtrack to assist others

• No point is covered more than twice

A

D

C

B

A

D

C

B

52galk@cs.biu.ac.il

Backtracking MSTC analysis

• Best case: The same

• Worst case: k=2

k>2

1k

n

1

3

2n

2

n

A

D

BC

A

B

53galk@cs.biu.ac.il

Generating a good spanning tree(believed to be NP-Hard)

AA

BB

CC

A B = 12 cells

B C = 12 cells

C A = 12 cells

A B = 28 cells

B C = 4 cells

C A = 4 cells

57galk@cs.biu.ac.il

Experimental results

58galk@cs.biu.ac.il

New (and in development):

Exploration for SaR, Clearing Team of 3-10 robots, 1-2 operators Fast exploration of single floor, indoor

Allow surveillance by human Provide approximate positions

Technology-transfer goal: An advanced prototype Buy COTS when possible Aim for platform-independence, standards-based Multiple deliverables (maximize impacts) Early involvement of customer, suppliers (ODF, Cogniteam)

59galk@cs.biu.ac.il

Simulated EyeDrive

60galk@cs.biu.ac.il

Challenges in Exploration

Single human is THE monitor No autonomous sensor readings Means difficult to save exploration time

Real-time on small fast robots No quad-core laptops on top….

Ideal: Vision-only

61galk@cs.biu.ac.il

Adaptive Coordination(Managing interdependence)ECAI 06, AIJ 07, ICRA 10 (Rosenfeld, Erusalimchik)

Foraging: Collect pucks, bring them home In nature:

Bees, ants Humans…

Many coordination methods: Arkin & Balch 1998 Rybski et al. 1998 Vaughan et al. 2002 ….

62galk@cs.biu.ac.il

Adaptive Coordination

Track cost of coordination

Costs spent before, during, after collision

e.g., time spent avoiding, time spent moving away

Switch to cheaper methods (triggered by thresholds)

Rosenfeld et al.: Use offline of thresholds

Erusalimchik et al.: Online reinforcement learning

Converges to optimal! Long economics-based discussion of why (it was a surprise!)

63galk@cs.biu.ac.il

0

5

10

15

20

25

30

35

40

1 5 9 13 17 21 25 29

Number of Robots

Pu

cks

Re

trie

ved Aggression

Noise

Repel

Adaptive

Adaptive Coordination Results

64galk@cs.biu.ac.il

Aggression Adaptive

65galk@cs.biu.ac.il

Lesson: Teamwork is about robustness

Imagine two cars driving one behind another If the cars are in a convoy (a team), then:

If one of them stops, other should stop or … Convoy leader uses signals, drives slowly, looks in mirror ….

As long as all goes well, no need for teamwork Simple spatial coordination is sufficient

66galk@cs.biu.ac.il

Lesson:Reuse through architectures

Reuse in multiple applications New commercial tools (www.cogniteam.com)

Position

Find Leader

Ready Leader

Walk

Lead

Move Formation

Follow

End

SynchronizedStart Synchronized

End

Allocation

68galk@cs.biu.ac.il

Challenges

From coverage, patrolling, exploration: Temporal synchronization is an invariant. Spatial task allocation: An Invariant? Run-time (can we compile into behavioral rules?)

From more complex tasks (RoboCup, exploration): Looser teamwork: Joining, retiring, asynchronous Proactive teamwork, anticipation, planning Role of plan-recognition within teamwork model

69galk@cs.biu.ac.il

Other challenges

From working with industry (Cogniteam) Validation and verification, at least probabilistically Commercial significance of re-use, explicit teamwork

From HART: Where to publish, what to read? Cannot believe I missed Stefan’s work! ARMS (Autonomous Robots Workshop) @ AAMAS HART mailing list?

71galk@cs.biu.ac.il

Thank you (galk@cs.biu.ac.il)

Inna Frenkel

Asaf Shiloni

Thuc Vu

Ruti Glick

Noam Hazon

Noa Agmon-Segal

Avi Rosenfeld

Ari Yakir

Yehuda Elmaliach

Milind Tambe

Sarit Kraus

Manuela Veloso

Meir Kalech

Dan Erusalimchik

Tom Shpigelman

Meirav Hadad

Gilad Armon-Kest

Jared Go

Acknowledgments (subset)