An Introduction to Swarm Robotics - A.Martinoli_tutorial_slides

8/7/2019 An Introduction to Swarm Robotics - A.Martinoli_tutorial_slides

http://slidepdf.com/reader/full/an-introduction-to-swarm-robotics-amartinolitutorialslides 1/97

An Introduction to Swarm Robotics

Alcherio Martinoli

SNSF Professor in Computer and Communication Sciences, EPFL

Part-Time Visiting Associate in Mechanical Engineering, CalTech

Swarm-Intelligent Systems GroupÉcole Polytechnique Fédérale de Lausanne

CH-1015 Lausanne, Switzerlandhttp://swis.epfl.ch/

[email protected]

Tutorial at ANTS-06, Bruxelles, September 4, 2006



Outline• Background– Mobile robotics

– Swarm Intelligence– Swarm Robotics

• Model-Based Analysis of SRS

– Methodological framework – Examples

• Machine-Learning-Based Synthesis of SRS– Methodological framework – Combined method (model/machine-learning-based)– Examples

• From SRS to other Real-Time, Embedded Platforms

• Conclusion and Outlook



Background:

Mobi le robot ic s



An Example of Mobile Robot:

Khepera (Mondada et al., 1993)

5.5 cm

actuators

microcontrollers

sensors

batteries

Strengths: size and modularity!



Perception-to-Action Loop

Computation

Perception

Action

Environment

• Reactive (e.g., linear or nonlinear

transform)• Reactive + memory (e.g. filter,

state variable)• Deliberative (e.g. planning)

• sensors • actuators



Autonomy in Mobile RoboticsTask Complexity

Autonomy

Industry

Research

Human-GuidedRobotics

AutonomousRobotics

Swarm

Robotics?

Different levels/degrees of autonomy:• Energetic level• Sensory, actuatorial, and computational level• Decisional level



Background:

Sw arm -In t e l l igent Syst em s



Swarm Intelligence Definitions• Beni and Wang (1990):

– Used the term in the context of cellular automata (based oncellular robots concept of Fukuda)

– Decentralized control, lack of synchronicity, simple and(quasi) identical members, self-organization

• Bonabeau, Dorigo and Theraulaz (1999)– Any attempt to design algorithms or distributed solvingdevices inspired by the collective behavior of social insectcolonies and other animal societies

• Beni (2004)– Intelligent swarm = a group of non-intelligent robots(“machines”) capable of universal computation

– Usual difficulties in defining the “intelligence” concept (nonpredictable order from disorder, creativity)



Swarm-Intelligent Systems: Features• Bio-inspiration– social insect societies– flocking, shoaling in vertebrates

• Unit coordination– fully distributed control (+ env. template)– individual autonomy– self-organization

• Communication– direct local communication (peer-to-peer)– indirect communication through signs in the

environment (stigmergy)

• Scalability• Robustness

– redundancy– balance exploitation/exploration– individual simplicity

• System cost effectiveness

– individual simplicity– mass production

Beyond bio-inspiration: combine naturalprinciples with engineering knowledgeand technologies

Robustness vs. efficiency trade-off



Current Tendencies• IEEE SIS-05– self-organization, distributedness, parallelism,

local communication mechanisms, individualsimplicity as invariants

– More interdisciplinarity, more engineering,

biology not the only reservoir for ideas

• ANTS-06, IEEE SIS-06 follow the tendency;IEEE SIS-07 even more so



Background:

Sw arm Robot ic s



First Swarm-Robotics

Demonstration Using Real Robots(Beckers, Holland, and Deneubourg, 1994)



Swarm Robotics: A new

Engineering Discipline?• Why does it work?

• What are the principles?• Is a new paradigm or just an isolated experiment?• If yes, can we define it?• Can we generalize these results to other tasks and

experimental scenarios?• How can we design an efficient and robust SR

system? Methods?• How can we optimize a SR system?• …



Swarm Robotics – Features

Dorigo & Sahin (2004)• Relevant to the coordination of large number of robots• The robotic system consists of a relatively few

homogeneous groups, number of robots per group is

large• Robots have difficulties in carrying out the task on their

own or at least performance improvement by the swarm

• Limited local sensing and communication ability



Swarm Robotics –

[Selected/Pruned] Definitions• Beni (2004)

The use of labels such as swarm robotics should not be inprinciple a function of the number of units used in the system.The principles underlying the multi-robot system coordinationare the essential factor. The control architectures relevant toswarms are scalable, from a few units to thousands or million

of units, since they base their coordination on local interactionsand self-organization.

• Sahin, Spears, and Winfield (2006)Swarm robotics is the study of how large number of relativelysimple physically embodied agents can be designed such that adesired collective behavior emerges from the local interactionsamong agents and between the agents and the environment. It

is a novel approach to the coordination of large numbers of robots.



SWIS Mobile Robotic Fleet

size

24 cm

11 cm

2 cm

Moorebot II – PC 104, XScale processor, Linux,WLAN 802.11; available robots: # 4

Alice II – PIC, no OS,WLAN802.15.4, IR com; #40

Khepera III – XScale processor, Linux,WLAN 802.11, Bluetooth; #20

E-puck – dsPIC, PICos,WLAN 802.15.4,Bluetooth; #100

6 cmSize & modularity !

Standards, com, and batt. changing!



SWIS Research Thrusts

Multi-level modeling,

model-based methods

Automatic (machine-

learning-based) design& optimization

System engineering &integration (single node)



Model -Based Approac h

(m ain foc us: analys is)



Abstraction

Experime

ntal time

Multi-Level Modeling Methodology

Ss SaS

sS

aSs Sa

∑ ∑′ ′

′−′′=n n

nnn t N t nnW t N t nnW

dt

t dN )(),|()(),|(

)(

Ss Sa

Physical reality: Info oncontroller, S&A, morphology and

environmental features

Realistic: intra-robot (e.g., S&A)and environment (e.g., physics)details reproduced faithfully

Microscopic: multi-agent models,only relevant robot feature captured,1 agent = 1 robot

Macroscopic: rate equations, mean

field approach, whole swarm

C

ommon m

etrics



Originality and Differences with

other Research Contributions• The proposed multi-level modeling method is specifically target to

self-organized (miniature) collective systems (mainly artificial up

to date); exploit robust control design techniques at individual level(e.g. BB, ANN) and predict collective performance through models

• Different from traditional modeling approach in robotics for collective robotic systems: start from unrealistic assumptions(noise free, perfectly controllable trajectories, no com delays, etc.)and relax assumptions compensating with best devices available &computationally intensive on-board algorithms

• Different from traditional modeling approaches in biology (andsimilar in physics, chemistry) for insect/animal societies: as simpleas possible macroscopic models targeting a given scientificquestion; free parameters + fitting based on macroscopic

measurements since often microscopic information notavailable/accurate



Micro/Macro Modeling Assumptions• Nonspatial metrics for swarm performance

• Environment and multi-agent system can be described asProbabilistic FSM; the state granularity of the description isarbitrarily established by the researcher as a function of theabstraction level and design/optimization interest

• Both multi-agent system and environment are (semi-)Markovian: the system future state is a function of the currentstate (and possibly amount of time spent in it)

• Mean spatial distribution of agents is either not considered or assumed to be homogeneous, as they were randomly hoppingon the arena (trajectories not considered, mean field approach)



Microscopic Level

…

Caste 1

Robotic System (N PFSM;

N = total # agents)

Environment (Q PFSM; Q = total # objects)

Coupling (e.g., manipulation, sensing)

…

O11 O1pOq1 Oqr

Ss Sa

Ss Sa

Ss

Sa

R 11

R 12

R 1l

…

Caste n

Se Sd

Si

R n1

…

Se

Sd

Si

R nm

… …Sa Sb Sa Sb



Macroscopic Level (1)

Environment (PFSM)

Coupling

Type 1

Ss Sa

Se

Sd

Si

Type q

Caste1

Caste n

Robotic (PFSM)

• average quantities

• central tendency prediction (1 run)

• continuous quantities: +1 ODE per

state for all robotic castes and object

types (metric/task dependent!)

• - 1 ODE if substituted with

conservation equations (e.g., total # of

robots, total # of objects of type q, … )

Sa

Sb



Macroscopic Level (2)

∑ ∑′ ′

′−′=n n

nn

n t N t nnW t N t nnW dt

t dN )(),|()(),|()(' Rate Equation

(time-continuous)

inflow outflow

n, n’ = states of the agentsNn = average # of robots in state n at time tW = transition rates (linear, nonlinear)

∑ ∑′ ′

′−′+=+n n

nnnn kT N kT nnTW kT N kT nnTW kT N T k N )(),|()(),|()())1(( '

Time-discrete version. k = iteration index, T time step (often left out)

If Markov properties are fulfilled, this is what we are looking for:



Model Calibration - Theory• Goal: calibration method for achieving 0-free parameter

models, gray-box approach:– As cheap as possible calibration procedure

– Models should not only explain but have also predictive power – Parameters should match as much as possible with design choices

• Two types of parameters:– Interaction times– Encountering probabilities

• Calibration procedures:– Idea 1: run “orthogonal” experiments on local a priori known interactions

(robot-to-robot, robot-to-environment)→

use for all type of interactionshappening these values– Idea 2: use all a priori known information (e.g., geometry) without running

experiments→ get initial guesses→ fine-tune parameters automatically onthe target experiment with a as cheap as possible calibration (e.g., fitting

algorithm using a subset of the system)



Linear Ex am ple:

Wander ing and Obst ac le

Avo idance



A Simple Linear Model

© Nikolaus Correll 2006

Example: search (moving forwards) and obstacle avoidance



A simple Example

Nonspatiality

& microscopic

characterizationDeterministic

robot’s flowchart

Search Avoidance

Start

Obstacle?

YN

Search Avoid., τa

Start

Obstacle?

pa

ps

1-pa

Probabilistic

agent’s flowchart

Ss Sa

pa

τa

ps

PFSM (Markov Chain)



Linear Model – Constant P Option

Search Avoidance, Ta

Ta = mean obstacle avoidance durationp

a

= probability of moving to obstacle av.ps = probability of resuming searchNs = average # robots in searchNa= average # robots in obstacle avoidanceN0 = # robots used in the experiment

k = 0,1, … (iteration index)

N s(k+1) =

N a(k+1) =

N s(k)

N 0 – N s(k+1)

ps=1/Ta

+ psN a(k)- paN s(k)

pa

Ns(0) = N0 ; Na(0) = 0



Linear Model – Time Out Option

Search Avoidance, Ta

Ta = mean obstacle avoidance durationp

a

= probability moving to obstacle avoidanceNs = average # robots in searchNa= average # robots in obstacle avoidanceN0 = # robots used in the experimentk = 0,1, … (iteration index)

N s(k+1) =

N a(k+1) =

N s(k)

N 0 – N s(k+1)

1

+ paN s(k-T a)- paN s(k)

pa

! Ns(k) = Na(k) = 0 for all k<0 !Ns(0) = N0 ; Na(0) = 0



Linear Model – Sample Results

Micro to macro comparison

(same robot density but wall surfacebecome smaller with bigger arenas)

Realistic to micro comparison

(different controllers, dynamic/staticscenarios, allocentric/egocentricmeasures)

Na*/N0



Nonlinear Ex am ple –

St ick-Pul l ing



A Case Study: Stick-Pulling

Proximitysensors

Arm elevationsensor

Physical Set-Up Collaboration via indirect communication

• 2-6 robots• 4 sticks• 40 cm radius arena

IR reflectiveband



Systematic Experiments

Real robots Realistic simulation

•[Martinoli and Mondada, ISER, 1995]•[Ijspeert et al., AR, 2001]

E i t l d R li ti



Experimental and Realistic

Simulation Results

• Real robots (3 runs) and realistic simulations (10 runs)• System bifurcation as a function of #robots/#sticks

Nrobots > Nsticks

Nrobots ≤ Nsticks



Geometric Probabilities

sgg

sg

aww

r R

ar r

ass

pRp

pp

AAp

N pp

AAp

AAp

=

==

−=

=

=

2

1

0

/

)1(

/

/

Aa = surface of the whole arena



From Reality to Abstraction

Interaction

modelingDeterministic

robot’s flowchart

Probabilistic agent’s

flowchart

Markov Chain (PFSM)



Full Macroscopic Model

• 6 states: 5 DE + 1 cons. EQ• Ti,Ta,Td,Tc ≠ 0; Τxyz = Τx + Τy + Τz

• TSL= Shift Left duration

• [Martinoli et al., IJRR, 2004]

)()()()()()(

)();()()(])()([)()1(

22

121

iasRaswcdascdagcascag

cgasacgagsRwggss

T k N pT k N pT k N T k T k N T k

T k N T k T k k N ppk k k N k N

−+−+−−∆+−−∆+

−Γ−∆+++∆+∆−=+

For instance, for the average number of robots in searching mode:

∏−

−−=

−=Γ

=∆

−−=∆

SL

SLg

T k

T T k j

sgSL

ggg

d ggg

jN pT k

k N pk

k N k N M pk

)](1[);(

)()(

)]()([)(

2

22

011

with time-varying coefficients

(nonlinear coupling):



Swarm Performance Metric

C(k) = pg2N s(k-T ca)N g(k-T ca)

e

T

k

T

k C e

∑=

=0

t

)(

(k)C

: mean # of collaborations at

iteration k

: mean collaboration rateover Te

Collaboration rate: # of sticks per time unit



Sample Results

Webots (10 runs),

microscopic (100 runs),

macroscopic model (1 run)



Simplified Macroscopic Model (1)

Τi,Τa,Τd,Τc << Τg →Τi=Τa=Τd=Τc=0



Nonlinear DE coupling through

unit-to-unit interaction (in this

case through the stick)!

Simplified Macroscopic Model (2)

Search Grip

Ns = average # robots in searching modeNg= average # robots in gripping mode

N0 = # robots used in the experimentM0 = # sticks used in the experimentΓ = fraction of robots that abandon pullingTe = maximal number of iterations

k = 0,1, …Te (iteration index)

N s(k+1) =

N g(k+1) =

N s(k) – pg1[M 0 – N g(k)]N s(k)

successful

+ pg2N g(k)N s(k)

unsuccessful

+ pg1[M 0 – N g(k-Τ g)]Γ (k;0)N s(k-T g)

N 0 – N s(k+1)

∏−=−=Γ

k

T k j

sg

g

jN pk )](1[)0;( 2

Ns(0) = N0, Ng(0) = 0Ns(k) = Ng(k) = 0 for all k<0

Initial conditions and causality



Steady State Analysis (Simplified Model)• Steady-state analysis→ It can be demonstrated that:

g

opt g

RM N for T

+≤∃ 1 20

0

with N0 = number of robots and M0= number of sticks,R g approaching angle for collaboration

• Counterintuitive conclusion: an optimal T g can exist also in

scenarios with more robots than sticks if the collaboration isvery difficult (i.e. R g very small)!

∝

approaching angle for collaboration

V ifi ti f A l i



Verification of Analysis

Conclusions (Full Model)

gg RR10

1~=

20 robots and 16 sticks

(optimal Tg)

Example: (collaboration very difficult)



• can be computed numerically by integrating the fullmodel ODEs or solving the full model steady-state equations

Optimal Gripping Time• Steady-state analysis → can be computed analytically in

the simplified model (numerically approximated value):

g

c

g

gg

opt

gR

for

R

N Rp

T +

=≤−

+−

−=

12

21

)1(2

1ln

)2

1ln(

1

01

β β β

β

opt

gT

with β = N0/M0 = ratio robots-to-sticks

[Lerman et al, Alife Journal, 2001], [Martinoli et al, IJRR, 2004]

opt

g

T

l bli i



Journal PublicationsStick Pulling

• Li, Martinoli, Abu-Mostafa, Adaptive Behavior, 2004-> learning + micro

• Martinoli, Easton, Agassounon, Int. J. of Robotics Res., 2004-> real + realistic + micro + macro

• Lerman, Galstyan, Martinoli, Ijspeert, Artificial Life, 2001-> realistic + macro

• Ijspeert, Martinoli, Billard, and Gambardella, Auton. Robots, 2001

-> real + realistic + micro

Object Aggregation

• Agassounon, Martinoli, Easton, Autonomous Robots, 2004

-> realistic + macro + activity regulation• Martinoli, Ijspeert, Mondada, Robotics and Autonomous Systems

-> real + realistic + micro



Som e Lim i t a t ions o f t he

c urren t Met hods

Model Calibration Practice



Model Calibration - PracticeBin distribution of interaction time Ta (mean Ta= 25 *50 ms = 1.25 s)

# of collisio

ns

Collision time

Micro model, time-out option

Realistic, proximal controller Realistic, distal controller

Micro model, const P option



Model Calibration - PracticeEncountering probability pa: example of transition in space fromsearch to obstacle avoidance (1 moving Alice, 1 dummy Alice,Webots measurements, egocentric)

Distal controller

(rule-based)

Proximal controller

(Braitenberg, linear)



Stochastic vs. Deterministic Models

Webots (10 runs),

microscopic (100 runs),

macroscopic model (1 run)

S ti l N ti l M d l



Spatial vs. Nonspatial Models[Correll & Martinoli, DARS-04, ISER-04, ICRA-05, DARS-06, ISER-06, SYROCO-06]

Boundary coverage problem (case study turbine inspection)

Unfolded turbine,blade geometryreproduced faithfully

Spatial models required because:• environmental template• fast performance metrics (e.g. time to

completion)

• clustered dropping point for robots• networking connectivity• algorithms with enhanced navigation

4 6 8 10 12 14 16 18 200

500

1000

1500

2000

Time to Completion



Machine-Learning-BasedApproach

(m ain foc us: synt hes is)



Automatic Design and Optimization• Evaluative & unsupervised learning: multi-agent (GA,

PSO) or single-agent (In Line Adaptive Search, RL)• Targeted to embedded control or system (e.g., hw-sw co-design, multi-objective)

• Enhanced noise-resistance (e.g., aggregation criteria,statistical tests)

• Customization for distributed platforms (off-line and on-line learning; solutions to the credit assignment problem)

• Combined with one or more levels of simulation



Rationale for Combined Methods• Application of machine-learning method to the targetsystem (“hardware in the loop”) might be expensive or not

always feasible• Any level of modeling allow us to consider certainparameters and leave others; models, as expression of reality abstraction, can be considered as “filters”

• Machine-learning techniques will explore the designparameters explicitly represented at a given level of abstraction

• Depending on the features of the hyperspace to be searched(size, continuity, noise, etc.), appropriate machine-learningtechniques should be used (e.g., single-agent hill-climbing

techniques vs. multi-agent techniques)



Learn ing t o Avoid Obst ac lesby Shaping a Neural Net w ork

Cont ro l ler us ing Genet icA lgor i thms



Evolving a Neural Controller

f(xi)

Ij

Ni

wij

1e12

)( −+

= −xxf

output

synapticweight

input

neuron N with sigmoidtransfer function f(x)

S1

S2

S3 S4

S5

S6

S7S8

M1M2

∑=

+=m

jjiji

I I wx1 0

Oi

)( ii xf O =

inhibitory conn.excitatory conn.

Note: In our case we evolve synaptic weigths but Hebbian rules for dynamicchange of the weights, transfer function parameters, … can also be evolved

Evolving Obstacle Avoidance




(Floreano and Mondada 1996)

• V = mean speed of wheels, 0 ≤ V ≤ 1• ∆v = absolute algebraic differencebetween wheel speeds, 0 ≤ ∆v ≤ 1

• i = activation value of the sensor with the

highest activity, 0 ≤ i ≤ 1

)1)(1( iV V −∆−=Φ

Note: Fitness accumulated during evaluation span, normalized over number of control

loops (actions).

Defining performance (fitness function):

Evolving Robot Controllers



Evolving Robot Controllers

Note:

Controller architecture can beof any type but worth using

GA/PSO if the number of parameters to be tuned isimportant





Evolved path

Fitness evolution



EvolvedObstacle Avoidance Behavior

Note: Direction of motion NOT encoded in the fitness function: GAautomatically discovers asymmetry in the sensory system

configuration (6 proximity sensors in the front and 2 in the back)

Generation 100, on-line,off-board (PC-hosted)evolution



From Sing le t oMul t i -Uni t Syst em s:

Co-Learn ing in aShared Wor ld

Evolution in Collective Scenarios



Evolution in Collective Scenarios

• Collective: fitnessbecome noisy due topartial perception,independent parallel

actions



Credit Assignment ProblemWith limited communication, no communication at all, or partial perception:

Co Learning Collaborative Behavior



Co-Learning Collaborative Behavior Three orthogonal axes to consider (extremities or balanced solutions are possible):

• Individual and group fitness• Private (non-sharing of parameters) and public (parameter sharing) policies• Homogeneous vs. heterogeneous systems

Example with binary

encoding of candidatesolutions



Co-Learning Competitive Behavior

fitness f 1 ≠ fitness f 2



Learn ing t o Avoid Obst ac le

using Noise -Resist antA lgor i thms

(Ex am ple 1 of t he Com bined Met hod,

rea l is t ic leve l w i t h GA and PSO)



Noisy Optimization• Multiple evaluations at the same point in the

search space yield different results• Depending on the optimization problem the

evaluation of a candidate solution can be more or

less expensive in terms of time• Causes decreased convergence speed and residual

error

• Little exploration of noisy optimization inevolutionary algorithms, and very little in PSO

Key Ideas



y• Better information about candidate solution can be

obtained by combining multiple noisy evaluations• We could evaluate systematically each candidate solution

for a fixed number of times→ not smart fromcomputational point of view• In particular for long evaluation spans, we want to dedicate

more computational power/time to evaluate promising

solutions and eliminate as quickly as possible the “lucky”ones→ each candidate solution might have been evaluateda different number of times when compared

• In GA good and robust candidate solutions surviveover

generations; in PSO they survive in the individual memory• Use aggregation functions for multiple evaluations: ex.

minimum and average

GA PSO



A Systematic Study on Obstacle



Avoidance – 3 Different Scenarios

• Scenario 1: One robot

learning obstacleavoidance

• Scenario 2: One robotlearning obstacle

avoidance, one robotrunning pre-evolvedobstacle avoidance

• Scenario 3: Two robotsco-learning obstacle

avoidance

Idea: more robots more noise (as perceived from an individual robot); no“standard” com between the robots but in scenario 3 information sharing

through the population manager!

PSO, 50 iterations, scenario 3

Scenario 3



Scenario 3Three orthogonal axes to consider (extremities or balanced solutions are possible):


Example with binary


Results – Best Controllers



Fair test: same

number of evaluations of

candidatesolutions for all algorithms(i.e. n generations/

iterations of standardversions comparedwith n/2 of thenoise-resistant ones)

Results – Average of Final Population



Final Population

Fair test:idem aspreviousslide



Learn ing t o Pu l l St ic k s

(Ex am ple 2 of t he Com bined Met hod,

m ic rosc opic leve l w i t h in -l ine adapt ivesearch)



Not Always a big Artillery such a GA/PSO isthe Most Appropriate Solution…

• Simple individual learning rules combinedwith collective flexibility can achieve

extremely interesting results• Simplicity and low computational costmeans possible embedding on simple, real

robots

In-Line Adaptive Learning



(Li, Martinoli, Abu-Mostafa, 2001)• GTP: Gripping Time Parameter • ∆d: learning step

• d: direction• Underlying low-pass filter for measuring the performance

In-Line Adaptive Learning



Differences with gradient descent methods:• Fixed rules for calculating step increase/decrease→ limited

descent speed→ no gradient computation→ more

conservative but more stable• Randomness for getting out from local minima (no

momentum)• Underlying low-pass filter is part of the algorithm

Differences with Reinforcement Learning:• No learning history considered (only previous step)

Differences with basic In-Line Learning:

• Step adaptive→ faster and more stability at convergence

Enforcing Homogeneity



g g yThree orthogonal axes to consider (extremities or balanced solutions are possible):


Example with binary


Sample Results – Homogeneous System



p g y

0 100 200 300 400 500 6000

0.2

0.4

0.6

0.8

1

1.2

1.4

Initial gripping time parameter (sec)

Stick−pulling rate (1/min)

2 robots

3 robots

4 robots

5 robots

6 robots

0 100 200 300 400 500 6000

0.2

0.4

0.6

0.8

1

1.2

1.4

Initial gripping time parameter (sec)

Stick−pulling rate (1/min)

2 robots

3 robots

4 robots

5 robots

6 robots

Short averaging window

(filter cut-off f high)

Long averaging window

(filter cut-off f low)

Learned (mean + std dev)

Systematic (mean only) Note: 1 parameter for the

whole group!

Allowing Heterogeneity



Three orthogonal axes to consider (extremities or balanced solutions arepossible):


Impact of Diversity on Performance



(Li, Martinoli, Abu-Mostafa, 2004)

2 3 4 5 6

1

1.05

1.1

1.15

1.2

1.25

1.3

Number of robots

Stick−

pulling rate ratio

2−caste, GlobalHeterogeneous, GlobalHeterogeneous, Local

Performance ratio between heterogeneous (full and 2-castes) and homogeneous groups AFTER learning

Notes:• global = group

• local = individual

Specialized teams

Homogeneousteams (baseline)

Diversity Metrics(Balch 1998)



( )Entropy-based diversity measure introduced in AB-04 could be used for

analyzing threshold distributions

Simple entropy:

pi = portion of the agents in cluster i; m cluster in total; h = taxonomic level parameter

Social entropy:

Specialization Metric



S = specialization; D = social entropy; R = swarm performance

Specialization metric introduced in AB-04 could be used for analyzing

specialization arising from a variable-threshold division of labor

algorithm

Note: this would be in particular useful when the number of tasks to be solved isnot well-defined or it is difficult to assess the task granularity a priori. In suchcases the mapping between task granularity and caste granularity might not trivial(one-to-one mapping? How many sub-tasks for a given main task, etc. see thelimited performance of a caste-based solution in the stick-pulling experiment)

Sample Results in the Standard Sticks



Diversity SpecializationRelative

Performance

• Spec more important for small teams

• Local p > global p• enforced caste: pay the

price for odd team sizes

• Flat curves, difficult to tellwhether diversity bringperformance

• Specialization higher withglobal when needed, dropmore quickly when notneeded

• Enforcing caste: low-pass

filter

• 2 serial grips needed to get the sticks out• 4 sticks, 2-6 robots, 80 cm arena



• When local and global performance are almost

aligned (i.e. “by doing well locally I do wellglobally”), local performance achieve slightlybetter results since no credit assignment

• Nevertheless, global performance less noisy, sopart of diversity for increasing performance higher with global performance (“specialization when

needed”)

Remarks on the Standard Set-Up Results



From Robot s t o o t her Em bedded, Dist r ibu t ed, Real-

T im e Syst em s



Embedded, Real-Time SI-Systems:



Common Features

• Real-world systems (noise, small heterogeneities, …)• From a few to millions of units (but not 1023!)

• Embodiment, sensors, actuators, often mobility and energy limitations

• Local intelligence, behavioral rules, autonomous units

• Local interaction, communication (unit-to-unit, unit-to-environment)

Collaborative Decision in Sensor Networks

i i l i i



Abstra

ction

Experimental time

Ss

Sa

Ss SaS

sS

a

Physical reality: detailed info onsensor nodes available

Realistic: intra-node details andcommunication channelreproduced faithfully (Webots with

Omnet++ plugin)

Microscopic 1: spatial 2D

montecarlo simulation, multi-agentmodels, 1 agent = 1 node

Macroscopic: rate equations, meanfield approach, whole network

C

ommon m

etrics

[Cianci et al., in preparation]

Microscopic 2: nonspatial 1Dmontecarlo simulation, multi-agent

models, 1 agent = 1 node

Leurre: Mixed Insect-Robot Societies

[Correll et al., IROS-06; ALife J. in preparation]

h //l lb b /



Abstra

ction

Experimental time

Ss

Sa

Ss SaS

sS

a

Physical reality: detailed info onrobots; limited info on physiologyof cockroaches, individual

behavior measurable externally

Realistic: intra-robot details,environment (e.g., shelter, arena)details reproduced faithfully;cockroaches: body volume +animation

Microscopic: multi-agent models, 1agent = 1 robot or cockroach; similar description for all nodes

Macroscopic: rate equations, meanfield approach, whole swarm

C

ommon m

etrics

http://leurre.ulb.ac.be/1

1

( 1) ( ) ( ) ( 1) ( ) ( ) ( )

( ) ( ) ( 1) ( )

join join

j j r s j j

leave leave

j j

N k N k p N k j p j N k p j N k

p j N k j p j N k j

−

+

⎡ ⎤+ = + − −⎣ ⎦− + +

Supra-Molecular Chemical System

[M d t l 2006 i ti ]



Physical reality: microscopic(e.g., crystallography) and macroscopicmeasurements (chemical reaction)

Microscopic 1: Agent-Based model, molecules

geometry abstracted, 1 agent = 1 aggregate

Macroscopic 2: Reactions kinetics describes how areaction occurs and at which speed (differential

equations)

Macroscopic 1: Chemical equilibrium is completelydefined by equilibrium constants K of each reaction(law of mass action)

Microscopic 2: Agent-Based model, molecules2D- and 3D geometry captured, 1 agent = 1

aggregate

Ss SaS

sS

aSs Sa A

bstraction

Experiment

al time

Common

metrics

[Mermoud et al., 2006, in preparation]

SAILS: 3D Self-Assembling Blimps[N b i i t l IEEE SIS GA 2005]htt // ill



Abstrac

tion

Experim

ental tim

e

Physical reality: detailed info onrobots

Realistic: intra-robot details,environment simplified (norealistic fluid dynamics yet)

Microscopic: multi-agent models, 1agent = 1 blimp; trajectorymaintained, visualization withWebots

Macroscopic: rate equations, meanfield approach, whole swarm?

Common metrics

TBD

[Nembrini et al., IEEE-SIS, GA, 2005]http://www.mascarillons.org



Conclus ions

d 10



Lessons Learned over 10 Years1. Stress methodological effort with computer &

mathematical tools; exploit synergies among thethree main research thrusts

2. Keep closing the loop between theory andexperiments with simulation

3. Formally proof claims using simple models andshow experimental excellence with realisticconditions→ seek for system dependability

4. Choose case studies that are relevant for

applications5. Focus on system design and use off-the-shelf components and platforms

Lessons Learned over 10 Years



6. Leverage all the technologies you can from other markets (OS, wireless com, S&A, batteries) and gobeyond bio-inspiration

7. Team-up with other research specialists andcompanies for specific problems and applications

8. Push towards miniaturization; probably key for

non-military applications in swarm robotics9. Consider other forms of coordination other thanself-organization (swarm intelligence just one formof distributed intelligence)

10. Consider other artificial/natural platforms (e.g.static S&A networks, mixed societies, chemicalsystems, intelligent vehicles, 3D moving units)

Some Pointers for Swarm Robotics (1)E t i dditi t ANTS ICRA IROS



• Events: in additions to ANTS, ICRA, IROS:– IEEE SIS (2003, 2005, 2006, 2007)– DARS (1992 - , biannual)

– Swarm Robotics Workshop at SAB (2002, 2004)• Books:

– “Swarm Intelligence: From Natural to Artificial Systems", E.

Bonabeau, M. Dorigo, and G. Theraulaz, Santa Fe Studies in theSciences of Complexity, Oxford University Press, 1999.– Balch T. and Parker L. E. (Eds.), “Robot teams: From diversity to

polymorphism”, Natick, MA: A K Peters, 2002.

• Journal special issues:– Ant Robotics, 2001, Annuals of Mathematics and Artificial

Intelligence

– Swarm Robotics, 2004, Autonomous Robots

Some Pointers for Swarm Robotics (2)



• Projects and further pointers in addition to SWIS activities:– SwarmBot (next tutorial): http://www.swarm-bots.org/

– I-Swarm: http://www.i-swarm.org/– Leurre: http://leurre.ulb.ac.be/index2.html

– BORG group at Georgia Tech: http://borg.cc.gatech.edu/

– Rus robotics group at MIT: http://groups.csail.mit.edu/drl/

– RESL at USC: http://www-robotics.usc.edu/~embedded/

– IASL at UWE: http://www.ias.uwe.ac.uk/

– Robotics at Essex: http://cswww.essex.ac.uk/essexrobotics/

– Race at Uni Tokyo: http://www.race.u-tokyo.ac.jp/index_e.html

– Fukuda’s laboratory: http://www.mein.nagoya-u.ac.jp/

– Swarm robotics we page (by E. Sahin): http://swarm-robotics.org/

An Introduction to Swarm Robotics - A.Martinoli_tutorial_slides

Documents

Transcript of An Introduction to Swarm Robotics - A.Martinoli_tutorial_slides