1

A Coalition Game-Based Algorithm for

Autonomous Self-Reconfigurations in

Modular Self-Reconfigurable Robots

Zach RamaekersComputer Science

University of Nebraska at OmahaAdvisor: Dr. Raj Dasgupta

2

Modular Self-Reconfigurable Robots (MSRs) – What and WhyAn MSR is a type of robot that is composed of

identical modulesThe modules connect together to form larger

robots capable of performing complex tasksWhy MSRs?

Inexpensive and SimpleHighly Adaptable

Three main types of MSRs: Chain, Lattice, Hybrid

3

ModRED (Modular Robot for Exploration and Discovery)

Novel 4 degrees of freedom designGives improved dexterityAllows maneuver itself and

get out of tight spaces’ModRED sensors and

actuators Arduino processor (for doing

computations) IR sensors (for sensing

how far obstacles are) Compass (which direction am

I heading) Tilt sensor (what is my

inclination) XBee radio (for wireless

comm.)

Designed by Dr. Nelson’s group, Mechanical Engineering, UNL

CAD diagram of

robot

Simulated robot in Webots

4

ModRED Movements in Fixed Configuration

All these movements are in a fixed configuration

5

Problem Addressed: Dynamic Self-Reconfiguration by MSRs

Why does an MSR need to reconfigure

dynamically?

Problem Statement: How can an MSR that needs reconfigure (e.g., after encountering an obstacle) determine 1. which other modules

to combine with, and2. the best configuration

to form with those modules?

... in an autonomous manner.

6

Coalition Games for Dynamic MSR Reconfiguration We propose a novel, coalition game theory

based approach to address the problem of MSR self-reconfiguration

A coalition game gives a set of rules that determine what teams a group of humans (or robots) will form between themselvesTeams are guranteed to be stable: once

teams are formed no one will want to change teams

Game TheoreticLayer

(Coalition Game)

ControllerLayer (Gait-

tables)

Mediator

Dynamic self- reconfiguration

Movement in Fixed Configuration

7

Coalition Games for Dynamic MSR Reconfiguration We propose a novel, coalition game theory

based approach to address the problem of MSR self-reconfiguration

A coalition game gives a set of rules that determine what teams a group of humans (or robots) will form between themselvesTeams are guranteed to be stable: once

teams are formed no one will want to change teams

Game TheoreticLayer

(Coalition Game)

ControllerLayer (Gait-

tables)

Mediator

Dynamic self- reconfiguration

Movement in Fixed Configuration

For our scenario: Each module of an MSR is provided with

embedded software called an agent that does the coalition

game related calculations

8

Coalition Games for Dynamic MSR Reconfiguration We propose a novel, coalition game theory

based approach to address the problem of MSR self-reconfiguration

A coalition game gives a set of rules that determine what teams a group of humans (or robots) will form between themselvesTeams are guranteed to be stable: once

teams are formed no one will want to change teams

Game TheoreticLayer

(Coalition Game)

ControllerLayer (Gait-

tables)

Mediator

Dynamic self- reconfiguration

Movement in Fixed Configuration

For our scenario: Each module of an MSR is provided with

embedded software called an agent that does the coalition

game related calculations

Our problem: How can we determine these teams or partitions or coalitions for our MSR problem?

9

Determining the PartitionsEnumerate all possible partitions that includes

all the agents - called coalition structures

{1} {2}{3}{4}

{1, 2}{3}{4} {1} {2 ,3}{4}Examplecoalition

structureswith 4 agents

Each coalition structure is associated with a value V(CSi) = sum of utilities of each coalition in CSi

All the possible coalition structures are represented as a coalition structure graph

V(CSi) = u(1,2) + u(3) + u(4)

V(CSi) = u(1) + u(2,3) + u(4)

V(CSi) = u(1) + u(2) + u(3) +

u(4)

10

Coalition Structure Graph (CSG)Possible partitions of 4 modules (agents)

Problem: Find the

node (coaliltion structure) that has

the highest value in

this graph

11

Coalition Structure Graph (CSG)

Finding the optimal coalition structure node in this graph is not easy!CSG has w(nn) nodes, the search problem is hard (NP-complete)

Approximation algorithm used to find the optimal CSG node (Sandholm 1999, Rahwan 2007, etc.): in exponential time.

Possible partitions of 4 modules (agents)

Problem: Find the

node (coaliltion structure) that has

the highest value in

this graph

12

Dynamic Reconfiguration under Uncertainty

I need to form

another configurati

on

We are here

And we are here

Which modules should I join

with?

Ridge in planned path

13

Modeling Uncertainty in Coalition Formation

{1} {2}{3}{4}

{1, 2}{3}{4} {1} {2 ,3}{4}PossibleCoalitionStructures

Conventional CSG

V(CSi) = u(1,2) + u(3) + u(4)

(additive reward value)

{1, 2}{3}{4}

CSG with uncertainty

V(CSi) < u(1,2) + u(3) + u(4) (subadditive)

V(CSi) = u(1,2) + u(3) + u(4) (additive)

V(CSi) > u(1,2) + u(3) + u(4) (superadditive)

{1, 2}{3}{4}

14

Modeling Uncertainty in Coalition Formation

Dealing with uncertainty: Markov Decision Process (MDP) provide a mathematical model for robots or agents to determine their actions in the presence of uncertainty

{1} {2}{3}{4}

{1, 2}{3}{4} {1} {2 ,3}{4}PossibleCoalitionStructures

Conventional CSG

V(CSi) = u(1,2) + u(3) + u(4)

(additive reward value)

{1, 2}{3}{4}

CSG with uncertainty

V(CSi) < u(1,2) + u(3) + u(4) (subadditive)

V(CSi) = u(1,2) + u(3) + u(4) (additive)

V(CSi) > u(1,2) + u(3) + u(4) (superadditive)

{1, 2}{3}{4}

15

MDP-Based CSG Solution

Conventional CSG without uncertainty

Modified CSG with

uncertainty

16

Algorithm to find optimal coalition structure

Pruning – used to reduce the number of nodes that are searched by the algorithm in the coalition structure graphThree strategies explored: Keep the optimal and two least

optimal children; keep the optimal, median, and least optimal children; keep three random children

Set of modules information

Generate Coalition Utility Values

Generate Coalition Structure Graph

Run Value Iteration and Determine Policy

Run MDP Traversal to Find Optimal CS

Optimal Coalition Structure

17

Simulation Results

1 2 3 4 5 6 7 8 9 10111213141516

Average Number of Nodes in the CS Graph

Full CS Graph

Number of Agents

Num

ber

of

Nodes

in t

he C

S

Gra

ph (

base

10)

1 2 3 4 5 6 7 8 9 101112131415160

5000

10000

15000

20000

25000

30000

35000

Average Number of Nodes in the CS Graph

Pruned Graph Size

Sandholm

Number of Agents

Tota

l N

odes

in t

he C

S G

raph

3 4 5 6 7 8 9 100.0001

0.001

0.01

0.1

1

10

100

1000

Average Time to Compute CS Graph

Full CS Graph

Aver-age Prun-ing CS Graph

Number of Agents

Avera

ge R

unnin

g T

ime (

sec)

3 4 5 6 7 8 9 100.2

0.25

0.3

0.35

0.4

0.45

0.5Average Optimal Reward Values

Full CS Graph

Prun-ing Method 1

Prun-ing Method 2

Number of Agents

Avera

ge R

ew

ard

Valu

e

18

Conclusions and Future WorkDeveloped coalition game theory based algorithm for MSR

self-reconfigurationValidated to work on accurate model of MSR called

ModRED within Webots robotic simulator To the best of our knowledge

First application of game theory to MSR self-reconfiguration problem

First attempt at combining planning under uncertainty (MDP) with coalition games*

Future workInvestigate distributed models of planning under uncertainty

(MMDPs, DEC-MDPs, etc.)Simulation of exploration and coverage on realistic terrainsIntegrate with hardware of ModRED robot

*: Suijs et al.(1999) defined a framework called stochastic coalition game, but using a different model involving agent types, which wasn’t validated empirically.

19

ModRED Self-Reconfiguration Simulation Demo