Topic 7: Collaborative Distributed Problem Solving

application domain type defining a MAS cooperation strategies

The Coordination Problem

Managing the interdependencies between the activities of agents. e.g. You and I both want to

leave the room.

We independently walk towards the door, which can only fit one of us.

I graciously permit you to leave first.

http://media.funmansion.com/funmansion/player/player.php?url=http://media.funmansion.com/content/flv/d4282006/crazy_foreign_traffic.flv

1. Understand application domain type (requirements) application characteristics, requirements, constraints “domain theory”

TOD Task-Oriented Domains WOD Worth-Oriented Domains SOD State-Oriented Domains

2. Define a MAS for the particular system objectives various aspects design issues / considerations interaction situations

3. Define cooperation strategies / examples AGV case examples

1. simple task allocation, task (re)allocation, jointly lifting, traffic mgt, …

1. Understand application domain type- Domain Theory

Application requirements obviously crucial for defining MAS-based software architecture

Domain theory: study of types / domains of MAS

Task Oriented Domains Agents have tasks to achieve Task (re)distribution

State Oriented Domains Goals specify acceptable final states Side effects Joint plan and schedules

Worth Oriented Domains Function rating states’ acceptability Joint plan, schedules, and goal relaxation

1.1 Task Oriented Domains

“Domains in which an agent’s activity can be defined in terms of a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994)

characteristics a set of agents a set of tasks to achieve a set of resources

agents can achieve tasks without help or interference from each other

however, agents may benefit by sharing some tasks

Task-oriented Domain: Example

Imagine you have 3 children, each of whom needs to be delivered to 3 different schools each morning. Your neighbour has 4 children who also need to be taken to school. Delivery of each child is a task. Assume that one of your children and one of your neighbour’s children both go to the same school. It obviously makes sense for both children to be taken together and only you or your neighbour needs to make the trip.

Task-oriented Domain: Example

Imagine you have 3 children, …

Post menPost OfficePost Office

a

c

d e

21

b

f

Task-oriented Domain: Example

Imagine you have 3 children, … Post men

Database queries Common DatabaseCommon Database

“All female employeeswith more than threechildren.”

2

1

“All female employeesmaking over $50,000 ayear.”

1.2 State-Oriented Domain

State-Oriented Domains actions lead to changes of system state

System objective==> attain a particular system state

either pre-defined/static, either evolving

typical additional requirements (e.g. efficiency, …)

11 22 33

11 22 33

2

1

e.g. slotted blocks worlde.g. furniture moving

State-Oriented Domain

1.3 Worth-Oriented Domains

Worth Oriented Domains actions lead to utility:

function rating states’ acceptability

System objective==> maximize utility

Joint plan, schedules, and goal relaxation

Worth-Oriented Domains

2 22

2

55

34

AB tilehole

obstacle

agents

example: tile world maximize rewards

example: traffic lights maximize traffic throughput

2. Define a MASfor particular system objectives

identify various ‘elements’ of MAS

design issues / considerations

interaction situations

2. Defining a MASfor particular System objectives

identify various aspects in defining the MAS

starting fromsystem objectives / requirements / constraints

define MAS for reaching the objective(s) identify agents identify individual goals identify individual skills (actions, comm, capabilities) identify their relation to resources (insufficient, ...) identify their cooperation

e.g. task allocation, coordination over resources, ...

identify suitable agent architecture

creative

task

2. Defining a MASfor particular System objectives

identify various aspects in defining the MAS

design issues / considerations within application boundaries obviously, i.e. if appl. allows

heterogeneity? all agents can perform tasks, vs. not

splittable tasks? specialization? grouping? distributed planning? task distribution predictiveness (anticipatory vs. in situ)

e.g. for avoiding conflicts over resources e.g. road load, deadlocks, ...

2. Defining a MASfor particular System objectives

identify various aspects in defining the MAS design issues / considerations

interaction situations what is an interaction ?

“dynamic relationship through a set of reciprocal actions” i.e. stem from actions whose consequences have an influence on

the future behaviour of the agents

e.g. agents helping each other data exchange between servers use of a printer by two programs simultaneously

Types of interaction situations

components / criteria for categorization of interaction situations types

compatible and incompatible goals relation to resources agent capacities / skills

Components of interaction situations

compatible and incompatible goals concurrent goals ? or contradictory or even opposed ?

incompatible antagonist situation achieving one goal other goal cannot be achieved

compatible cooperation situation

…

Components of interaction situations …

relation to resources resources: environmental elements which can be used in carrying out an

action objects, tools, space, time, …

limited resources conflicts agents needing the same tool at the same place in the same time programs sharing a CPU vehicles on busy roads

conflict resolution many methods

law of the strongest negotiation

e.g. insurances and court houses to resolve conflicts of interest due to accident

coordinating actions devices, regulations, supplementary actions e.g. traffic lights / highway code / … to avoid conflicts

resource spaceresources needed by A

resources needed by B

Components of interaction situations

…

capacities of agents in relation to tasks can an agent carry out a task alone ? or does it need others ?

particular tasks can be carried out either by a single agents

e.g. moving a block, doing a calculation, reading a file, … better by several agents / only by several agents

e.g. heavy blocks, different expertise, …

beneficial interactions

result can be more than sum of parts

Goals Resources(each has)

Skills Types of situation Involvement

Compatible Sufficient Sufficient Independence Indifference

Compatible Sufficient Insufficient Simple collaboration  

Compatible Insufficient Sufficient Obstruction Cooperative

Compatible Insufficient Insufficient Coordinated collaboration

 

Incompatible Sufficient Sufficient Pure individual competition

 

Incompatible Sufficient Insufficient Pure collective competition

Antagonism

Incompatible Insufficient Sufficient Individual conflict over resources

 

Incompatible Insufficient Insufficient Collective conflict over resources

 

Types of interaction situations

Types of interaction situations:Examples

Compatible goals, insufficient resources, sufficient skills

obstruction e.g. motorway queue

Incompatible goals, insufficient resources, sufficient skills

individual conflict over resources e.g. only one liter of water, promotion, ...

...

3. Cooperation strategies - illustrated in AGV case

application domain type task-oriented domain

defining a MAS AGV agents, transport agents individual goals

AGV agents: perform tasks transport agents: ensure tasks are performed

skills AGV agents: jobs/actions/communication/… transport agents: communication/…

resources warehouse (lanes, stock) communication medium

cooperation …

3. Cooperation strategies - illustrated in AGV case (cont.)

…

interaction situations (in)compatible goals (in)sufficient skills (in)sufficient resources

3. Cooperation strategies - illustrated in AGV case (cont.)

1. (simple) task allocation CNET Gradient fields DynCNET

2. task (re)distribution negotiation

3. resource allocation - traffic management CNET

4. collision avoidance coordination protocol hull projection

5. joint action - jointly lift a (heavy) pallet joint intentions

6. batch order management1. team formation2. distributed planning

3.1 Simple task allocation

situationa task consists of

- a pick job (pick up pallet)- a move job (move towards destination)- a drop job (place pallet on target location)

a task can be executed by any AGVa task needs to be assigned / allocated to an AGV

characteristics“delayed commencement”dynamiccommunication overhead?

Simple task allocation (cont.)

situation

collaboration approaches

“Classic approach”: agents coordinate byexchanging messages

Contract NetDynCNet

Exploit the agent environment

Gradient Fields

Simple task allocation (cont.) situation

collaboration approaches1. Contract Net

transport agent issues request for bids(local broadcast)

if no response: stronger signalAGV agents can place bids

based oncurrent location (distance)other tasksbattery level…

transport agent assigns task to best bid

characteristics…

Simple task allocation (cont.) situation

collaboration approaches

1. Contract Net…

characteristics

communication overhead limited

does not take dynamics/opportunities into account

no rejection of assigned task

no re-allocation if better-suited AGV becomes available

issues

how should an AGV agent deal with multiple concurrent requests?

bid? value of bid? consequences?

ok approach for indiviual tasks, what for large numbers of tasks?

one task at a time

Simple task allocation (cont.) situation

collaboration approaches2. Dynamic Contract Net (DynCNET)

contract net, but taking into account dynamics

characteristicscommunication overheadcan take dynamics/opportunities into account

rejection of assigned taskre-allocation if better-suited AGV becomes available

issueshow should an AGV agent deal with multiple concurrent requests?

bid? value of bid? consequences?ok approach for indiviual tasks, what for large numbers of tasks?one task at a time

New Task Becomes Available

New AGV Becomes Available

DynCNET = Dynamic Contract Net

Simple task allocation (cont.) situation

collaboration approaches3. Gradient Fields

coordination through the environment

physical environment Restricts how agents can exploit the environment

agent environment virtual environment layer enables agents to share information and coordinate their

behavior

characteristicscfr. DynCNETreduced complexity of AGV agent!

Agent Environment for Agents

Exploit the Agent Environment Task Assignment (Field-Based Approach)

Exploit the Agent Environment

Task Assignment

Task Allocation experiments -Test Setting AGV System

AGV system “fish handling” (real scenario) 134 x 134 m; 56 pick/drop locations 14 AGVs; 0.7m/s; load manipulation 5s 140 transports/hour (standard test profile)

Tests Communication load Average waiting time Stress test: perform fixed number of transports as quick as possible

(e.g. handle the arrival of a truck with loads)

Communication Load Average Waiting Time

Test Results

Finished Transports in Stress Test

Two Qualities: Flexibility and Robustness

Flexibility (adapt to changes in the environment) FiTA: implicitly represented in the fields DynCNET: explicit points of choice

Robustness (to message loss) FiTA: graceful degradation DynCNET: requires additional functionality

3.2 Task (re)distribution

situationAGV agents have several tasks assigned to thema set of AGV agents could redistribute tasks amongst

each other for better results

e.g. after using several CNET protocols, the following assignmentexists:

AGV1: task 1 (NE corner), task 2 (SW corner) AGV2: task 3 (NE corner), task 4 (SW corner)

redistributing the tasks could seriously enhance task execution time

AGV1: task 1 (NE corner), task 3 (NE corner) AGV2: task 2 (SW corner), task 4 (SW corner)

Task (re)distribution (cont.)

situation

typical situation for negotiation in Task-Oriented Domains…

Task (re)distribution (cont.)

Task-Oriented Domains (TODs) Defined

a TOD is a triple<T, Ag, c>

where T is the (finite) set of all possible tasks Ag = {1,…,n} is the set of participating agents c = (T) + defines the cost of executing each subset of tasks

an encounter is a collection of tasks<T1,…,Tn>

where Ti T for each i Ag

Deals in TODs

given encounter <T1, T2>, a deal is an allocation of the tasks T1 T2 to the agents 1 and 2

the cost to i of deal = <D1, D2> is c(Di), and will be denoted costi()

the utility of deal to agent i is:utilityi() = c(Ti ) – costi()

the conflict deal, , is the deal <T1, T2> consisting of the tasks originally allocated.note that utilityi() = 0 for all i Ag

deal is individual rational if it weakly dominates the conflict deal

Negotiation

”The process of several agents searching for an agreement”

Reaching consensus

”Rules of Encouter” by Rosenchein and Zlotskin, 1994

Complexity of Negotiations

Some attributes that make the negotiation process complex are:

Multiple attributes: Single attribute (price) – symmetric scenario. Multiple attributes – several inter-related attributes, e.g. buying a

car.

The number of agents and the way they interact: One-to-one, e.g. single buyer and single seller. Many-to-one, e.g. multiple buyers and a single seller, auctions. Many-to-many, e.g. multiple buyers and multiple sellers.

Negotiation Components

Any negotiation setting will have 4 components: Negotiation set: represents the space of possible proposals

that agents can make

Protocol: defines the legal proposals that agents can make

Collection of strategies: (one for each agent) determines what proposals the agent will make

Rule: to determine when an agreement has been reached

The Negotiation Set

the set of deals over which agents negotiate are those that are: individual rational pareto efficient

The Negotiation Set Illustrated

Mechanisms, Protocols, Strategies

Negotiation is governed by a mechanism or a protocol: defines the ”rules of encounter” between the agents the public rules by which the agents will come to

agreements.

Given a particular protocol, how can a particular strategy be designed that individual agents can use?

Monotonic Concession Protocol

Negotiation proceeds in rounds

On round 1, agents simultaneously propose a deal from the negotiation set.

Agreement is reached if one agent finds that the deal proposed by the other agent is at least as good or better than its proposal.

Ai best deal Aj best deal

Monotonic Concession Protocol

If no agreement is reached, then negotiation proceeds to another round of simultaneous proposals.

In round u+1, no agent is allowed to make a proposal that is less preferred by the other agent than the deal proposed at time u.

If neither agent concedes, then negotiation terminates with a conflict deal.

Monotonic Concession Protocol

Advantages: Symmetrically distributed (no agent plays a special role) Ensures convergence It will not go on indefinitely

Disadvantages: Agents can run into conflicts Inefficient – no quarantee that an agreement will be reached

quickly

Key Questions

3 key questions to be answered: What should an agent’s first proposal be?

It’s most preferred deal.

On any given round, who should concede?The agent least willing to risk conflict.

If an agent concedes, then how much should it concede?Just enough to change the balance of risk.

The Risk Factor

One way to think about which agent should concede is to consider how much each has to loose by running into conflict at that point.

Ai best deal Aj best deal

Conflict deal

How much am I willing to risk a conflict?

Maximum loss from conflict

Maximum loss from concession

The Zeuthen Strategy

Uses the risk evaluation strategy

Suppose you have conceded a lot. Then: Your proposal is now close to conflict deal. You are more willing to risk conflict.

An agent will be more willing to risk conflict if the difference in utility between its current proposal and the conflict deal is low.

Degree of willingness to risk a conflict can be defined as:

utility i loses by conceding and accepting j’s offerutility i loses by not conceding and causing conflictRiskt

i =

Nash Equilibrium…

the Zeuthen strategy is in Nash equilibrium under the assumption that one agent is using the strategy the

other can do no better than use it himself…

interesting to the designer of automated agents publicly known strategy

About MCP and Zeuthen Strategies

Advantages: Simple and reflects the way human negotiations work. Stability – in Nash equilibrium – if one agent is using the

strategy, then the other can do no better than using it him/herself.

Disadvantages: Computationally expensive – players need to compute the entire

negotiation set. Communication burden – negotiation process may involve

several steps.

3.3 Resource allocation - traffic management (cont.)

situation

approach

negotiation protocols

delegate MAS - see later

3.4 Collision avoidance

situation

AGV autonomously follow their own routes towards

source / target locations

warehouse layout may contain locations where

only 1 AGVs can pass at a time

e.g. cross roads, close lanes

Collision avoidance (cont.)

situation

approaches collision avoidance communication protocols

many possibilities local decision e.g. based on priority rules (priority to the right) CNET for bidding on ‘critical section’

quite complex in non-trivial situations n vehicles complex warehouse layouts dynamic set of AGVs to take into account needs to be integrated seamlessly in agent architecture… deadlock?

Collision avoidance (cont.)

situation

approaches collision avoidance communication protocols

hull projection environment as controlling entity environment mediates between AGV requests to use a lane

more easily extensible for more vehicles / complex layouts more easily integrated in agent architecture

Hull projection through the virtual environment

3.5 Joint action- jointly lift a (heavy) pallet

situation heavy pallet needs to be transported heavy = requires two AGVs

AGVs need to agree on this AGVs need to stick to the agreement

an AGV cannot suddenly drop the pallet because it sees another opportunity (e.g. another job)

Joint action - jointly lift a (heavy) pallet (cont.)

situation

approach joint intentions (Jennings, Tambe)

Recall intentions important for individual practical reasoning

Now: intentions also important for coordination distinguish

individual intentions (that may be coordinated) from intentions to cooperatively and coordinatedly achieve a goal as a team

commitment associated with an intention future directed, persitent, should not be dropped for no reason conventions exist that regulate when it‘s appropriate to drop an intention e.g. when lifting a heavy pallet together

Coordination through joint intentions (cont.)

Agents need individual commitments, and joint commitment to overall goal

State of joint commitment is distributed over agents

Convention regulates e.g. when joint commitment may be dropped and how other agents need to be informed about change of mind.

More formally: Joint persistant goals (JPGs)

JPG = (goal φ, motivation for goal ψ)

e.g.: φ = „having heavy object lifted onto truck“; ψ = „later transportation“

Conventions next to intentions

Initially each agent believes φ has not been satsified, and believes possibleToDo φ

Until termination condition is met, each agent has goal φ (more precisely intention φ)

Termination condition: It is mutually believed that either φ is satisfied φ is impossible ψ is no longer valid

…

…

Until termination condition is met:

if an agent believes that either the goal is met or the goal is impossible or the motivation no longer holds

it has the goal to make this mutually believed (the goal to convince all others about that)

Examples for JPG-like architectures: ARCHON (Jennings e.a.) Steam (Tambe)

Both encode conventions formally as rules, thus integrating them into the „normal“ reasoning process

3.6 Batch order handling

situation peek load in warehouse e.g. several loaded trucks need to be cleared

10s of pallets need to be transportedsome pallets should stay together (e.g. same colour

material)

complex tasks may need to be decomposed? stack of pallets --> smaller stacks or individual pallets

narrow unload location - coordination required

humans would make a team for efficiency? unloading chain?

Batch order handling (cont.)

situation approach

organisation-based approach coordinated planning

Batch order handling (cont.)

situation approach

organisation-based approach inspired by human organisations define roles, assign/negotiate about roles commitment

e.g. action chain roles:

head of chaintail of chainbody of chaintransport

head (unload)

body (pass on)

body (pass on)

tail (drop off)

transport

transport

Batch order handling (cont.)

situation approach

organisation-based approach

coordinated planning centralized (locally or globally)

plan building and distribution plan merging

distributed (G)PGP - (Generalized) Partial Global Planning (Durfee e.a.) SharedPlans (Grosz, Kraus)

Coordinated planning Centralized Planning

plan building and distribution plan merging

each agent makes local, partial plan (PP) the PPs are communicated to central coordinator

it resolves the conflicts and generates a conflict-free global plan (GP). GP is communicated to all agents. e.g. Georgeff in AAAI conference 1983 (National Conf on AI)

Distributed Planning no central controller each agent has some knowledge or model of other agents It has the capability to piece the PPs together into an agreeable GP e.g. Victor Lesser and group (University of Massachusettes, 1981)

Critique: agents share and process a huge amount of information requires lots of computing and communication resources centralised approaches introduce a single point of failure sometimes plans can also become obsolete very quickly

Conclusion

collaborative distributed problem solving at the heart of multi-agent systems

just a small sample of possible situations / cooperation strategies here …

distributed problem solving solves problems but creates new challenges many coordination situations many possible solutions

in the end: integration in one agent architecture coordination and coherence !!

coordinationManaging inter-dependencies between the activities of agents

coherence:“how well the MAS behaves as a unit along some dimension

of evaluation” (Bond et al.)