LECTURE 9: Cooperation in MAS (III): voting, coalition formation

Artificial Intelligence II – Multi-Agent Systems

Introduction to Multi-Agent Systems

URV, Winter-Spring 2010

Outline of the lecture

Other kinds of cooperation in MASVoting mechanisms

Properties of social choice rulesPlurality / Binary / Borda / Condorcet

Coalition formationTask allocation via coalition formation

MAS

Cooperative Benevolent

Independent Self-interested

EmergentDiscrete Without communication -

Implicit

With communication -

Explicit

NegotiatorsDeliberative

Cooperation hierarchy [last lectures]

Partial Global Planning

Auctions

Contract Net

Reactive systems

VotingCoalition formation

Distributed Decision Making - Voting

Mechanism which chooses the outcome of a negotiation based on the inputs (votes) given by all agents to a set of competing optionsTruthful voters vote for the candidate they think is bestVoters are not always truthful

E.g. “useful vote” effect in politics: more than two candidates, and you are quite certain that your preferred candidate doesn’t have any chance

Aim of the negotiation: rank feasible social outcomes based on the individual ranking of those outcomes by each agent

A - set of n agentsO - set of m feasible outcomesEach agent i has a preference relation<i : O x O, asymmetric and transitive

Basic elements

Social choice rule

Input: the agents preference relations (<1 , …, <n )Output: elements of O sorted according to the input - gives the social preference relation <* of the agent groupIn other words – creates ordering for the group of alternatives, so that the most (socially) preferred alternative is chosen

Desirable properties of the social choice rule (I)

A social preference ordering <* should exist for all possible inputs <* should be defined for every pair of alternatives (o, o')∈O<* should be asymmetric and transitive over OThe outcomes should be Pareto efficient:

If ∀i ∈A, o <i o' then o <* o‘Do not misorder the options if all agents agree

Desirable properties of the social choice rule (II)

The scheme should be independent of irrelevant alternatives

Taking out/Adding an irrelevant alternative should not affect the winner of the vote

No agent i should be a dictator in the sense that o <i o' implies o <* o' regardless of the preferences of the other agents

Arrow's impossibility theorem-1951

No social choice rule satisfies all of the six conditions

E.g. in many cases an irrelevant alternative affects the outcome of the voting protocol

Some of the constraints must be relaxedWe may not require <* to be always definedWe may not require that <* is asymmetric and transitive Economy

Nobel Prize 1972

Voting mechanisms

Some voting mechanisms:Plurality protocol (highest number of votes wins)Binary protocol (series of votes of 2 options each)Borda protocol (sum of all the preferences of the agents)Condorcet protocol (pairwise comparison of options, given full preference ordering of each agent)

All the protocols are problematic in one sense or another

Plurality protocol

Each agent can give 1 vote to 1 of the alternativesThe alternative with the highest number of votes wins

Problems of plurality voting (I)

Useful voteQuite common effect of political polls45 % option A, 40% option B, 15% option CThe most socially preferred option is A => it should be the winner if all agents vote truthfullyThe agents that prefer option C know that they have no possibility to win, and most of them (80%) prefer option B to option A => 80% vote for B and 20% for A

A: 48% B: 52%, and B finally wins

Problems of plurality voting (II)

Huge effect of irrelevant alternative2 basic options, A and B, around 50% eachAnother –very minoritary option- C appears, attracting 1% of the voters of AA: 49% B:50% C:1% and B wins

Loss of information about the preferences of each voter

Each agent can only give 1 vote, even if it considers 2-3-4 “good” alternatives

Problems of plurality voting (III)

Strange effects42% A C B D26% B C D A17% D C B A15% C D B AA wins with 42% of the votes

58% of the voters preferred other options58% consider A the worst option100% consider C the 1st-2nd best option

Advantages of plurality voting

Most simple voting mechanismVery efficient from the computational point of viewEquality principle, as it preserves the idea of 1 agent = 1 vote

Binary protocol

All the options are ordered and then evaluated in pairs (options 1 and 2, the winner with option 3, the winner with option 4, etc.)The option that wins the last evaluation is the overall winner

win(o5 , win (o4 , win (o3 , win(o2 ,o1 ))))

The ordering problemx > z > y (35%) y > x > z (33%)z > y > x (32%)Note that y is preferred to x (65-35), x is preferred to z (68-32), and z is preferred to y (67-33)

win(x, win(y,z))=xwin(y,win(x,z))=ywin(z, win(x,y))=z

The order of the pairings affects the outcome !The voter organiser may influence the resultThe last options have more chances of winning

Another problematic example

35% of agents have preferences

33% of agents have preferences

32% of agents have preferences

abdc fff

bdca fff

dcab fff

Win(a,c)=a Win(a,b)=b Win(b,d)=d => d Wins

• d was the worst alternative for 32%

• d was not the best alternative for anyone

• Everybody prefers c to d (!)

Summary of problems of binary voting

Decisive role of the ordering of the alternativesAn alternative x may win even if there is another alternative x’ which is preferred to xby all agents

Alternatives may be misorderedTemporal cost of the voting process (sequence of pairwise eliminative votes)

Borda protocol-1770For each voter, we assign |O| points for the highest preference, |O|-1 points for the second, and so on

Example: one agent considers the order ABCDA:4 points, B:3 points, C:2 points, D:1 point

The points are added across the voters and the alternative with the highest count becomes the social choice [similar to Eurovision song contest]

Borda paradox

a > b > c > d b > c > d > ac > d > a > ba > b > c > db > c > d > ac > d > a > ba > b > c > d

a=18, b=19, c=20, d=13

If the worst alternative –d- is removeda > b > c b > c > ac > a > ba > b > cb > c > ac > a > ba > b > ca=15, b=14, c=13

Inverted-order paradox

Borda protocol with 4 alternativesx=22, a=17, b=16, c=15Order: xabc

If we remove x: c=15, b=14, a=13The second one is c, not a

1. x > c > b > a2. a > x > c > b3. b > a > x > c4. x > c > b > a5. a > x > c > b6. b > a > x > c7. x > c > b > a

1. c > b > a2. a > c > b3. b > a > c4. c > b > a5. a > c > b6. b > a > c7. c > b > a

Problems of the Borda protocol

Most computationally expensiveThe Eurovision Song Contest is endless …

Eliminating one irrelevant alternative may totally change the outcome of the protocol

Winner => LastSecond worst => Winner

Total order changes if options are removed one by one

Condorcet protocol-s.XVIII

Each voter ranks the candidates in order of preferenceEach candidate is compared to each otherIf a candidate wins all the comparisons, it is the winner of the electionIn the event of a tie, use another resolution method (e.g. Borda count)

Example (I)

Election of the capital city of TennesseeEverybody prefers to have the capital as close as possible

Example (II)

Nashville wins

Problem of Condorcet method

Possibility of circular ambiguitiesNo alternative wins all other alternativesThere are many ways to resolve them

Keep the candidate that wins more matches(Copeland)Take into account the relative strengths of defeats(Minimax, Ranked Pairs, Schulze, …)

Can look at the winning votes or at the winning margin

Example of influence of protocol (I)

45 ABCD, 25 CDBA, 20 DBCA, 10 DCBA

Plurality protocol: A-45 B-0 C-25 D-30A wins quite clearly, B has no votesHowever, note that 55% say that A is the worst option (!)

Example of influence of protocol (II)

45 ABCD, 25 CDBA, 20 DBCA, 10 DCBAPairwise comparisons

A loses with B,C and DB wins C, C wins D, D wins B

Binary protocolA can never win (!)Order ABCD: D winsOrder BDAC: C winsOrder CDBA: B wins

Example of influence of protocol (III)

45 ABCD, 25 CDBA, 20 DBCA, 10 DCBACondorcet protocol: no option beats all the others, and B/C/D tied with 2 victoriesBorda protocol

A: 235 B: 265 C:260 D:240B wins (although all the options get similar values)B is not the first option for anyone !!!B is a good consensual option (2nd for 65%, 3rd for 35%), whereas A was very problematic (45% very positive support, but 55% very negative support)Recall that B had 0 votes in the plurality protocol (!)

Insincere (Strategic) Voters

If a self-interested agent can benefit from insincerely declaring his preferences, he will do so

E.g. Suppose your choice will likely come in second place. If you rank the first choice of the rest of the group (insincerely) very low, you may lower that choice enough so yours is first

However, knowledge of the agent’s true preferences is rarely available

Coalition formation

What is a Coalition?

Coalitions are (temporary) collections of individuals working together for the purpose of achieving a taskCoalition formation is the process whereby an agent decides to cooperate with other agents, because

The task cannot be performed by a single agentThe task can be performed more efficiently by several agents working together

Interesting aspects of coalitions

Agents usuallybring different,complementarycapabilities to thecoalition – theyare not clones !

When the task is completed, the payoff is distributed, the coalition is disbanded and agents continue to pursue their own agendas

Issues in coalition formationGiven a set of tasks and a set of agents, which coalitions should an agent attempt to form?What mechanism can an agent use for coalition formation?What guarantees regarding efficiency and quality can the mechanism provide?One a coalition has formed, how should its members go about the distribution of work/payoff?

Solution types

Mechanisms for benevolent agents are usually much simpler than those for self-interested agentsA centralised design of coalitions is usually much simpler to execute than a distributedoneCoalition formation may be external or internal

External coalition formation

By imposition: an external agency makes decisionsAgents advertise skills (capabilities) and prices (cost)Requestor defines properties of coalition to the external agencyExternal process computes optimal coalition

Internal coalition formation

By self-organisation: coalitions are established by group interactionsMulti-lateral negotiationIdentification of tasks to be solved

Can be static (given by the user at the beginning) or dynamic (generated by the own agents at run-time)

Negotiation of outcomes (self-interested agents)

Coalition formation activities

Coalition structure generationOptimizing coalition value

Pooling the resources of the agents to maximize the coalition value

Payoff distributionDeciding how to distribute the payoff between coalition members (equally, outputs, role)

Coalition structure generationPartition into exhaustive [disjoint] coalitionsGiven 3 agents {a1, a2, a3}, there are seven possible coalitions

{a1}, {a2}, {a3}, {a1, a2}, {a1, a3}, {a2, a3}, {a1, a2, a3}and five coalition structures{a1, a2, a3}, {{a1}, {a2, a3}}, {{a2}, {a1, a3}}, {{a3}, {a1, a2}}, {{a1}, {a2}, {a3}}It may not be necessary or appropriate to generate all coalition structures in advance, depending on the application domain

Task allocation via coalition formation

There is a set of tasks that have to be distributed among a group of agentsEach task has a certain set of requirements, and each agent has a certain set of capabilitiesThe aim is to find the optimal distribution of tasks to subgroups of agents

Definitions (I)A set of n agents: N={A1,A2,...,An}Each agent Ai has a positive vector of capabilities

Bi=<bi1,...,bi

r>Each capability is a property that quantifies the agent’s ability in some aspectCapabilities may be expendable (e.g. amount of memory) or non-expendable (e.g. ability to perform an action)There is an evaluation function for each capability, that transforms it into monetary units

Definitions (II)

Set of m independent tasks T={t1, …, tm}There is a vector of capabilities needed to perform each task tl, Bl={bl

1, …, blr}

The benefit gained from performing each task depends on the capabilities required for its performance

To simplify the problem, it can be a linear function on the amount of resources

Simplifying conditions on coalitionsA coalition is a group of agents that decide to cooperate to achieve a common taskA coalition can work on a single task at a timeEach agent only belongs to one coalitionA coalition C has a vector of capabilities Bc(sum of the capabilities of the agents in the coalition)A coalition C can perform a task t iff

for all 1<=i<=r bti <= bC

i

Coalitional cost

For each coalition C and specific task t, it is possible to calculate the coalitional value V, that measures the joint utility that the members of C can reach if they cooperate to satisfy t. This value depends on the capabilities contributed by the team members and the number of coalition membersThe coalitional cost c is the reciprocal of the coalitional valueThe aim is maximising the coalitional value (i.e. minimising the cost)

Task allocation process

Heuristic: prefer small-sized coalitions, to reduce communication and coordination costs

Maximum coalitional size allowed: kInitial coalitional state: n single agentsStep by step, formation of coalitions

1 new coalition in each iterationWhen an agent joins a coalition, it quits the coalition formation process

Algorithm of task allocation

Preliminary stage: all possible coalitions are distributed among all agents

Collective, not centralised processIterative stage:

Coalitional values (for each pair –coalition,task-) are (re)calculatedOne coalition C is formedAgents in coalition C quit the coalition formation process

Stage 1: initial calculations of agent Ai (I)

Li = empty_setThis list will contain, at the end of stage 1, the coalitions whose value agent i will have to calculate

Pi = all permutations up to k agents containing agent i

Those are all the possible coalitions in which agent i could participate

Stage 1: initial calculations of agent Ai (II)For each permutation in Pi do

Contact agent j, which is in the permutation (i.e. j is in some possible coalition(s) with i)In the first contact, request capabilities BjCommit to calculate the value of some of the coalitions including i and j = SijPi = Pi – SijLi = Li + Sij

End_for

[at the same time]For each agent k that contacts i, Pi = Pi – Ski

Example (I)

4 agents: A1, A2, A3, A4

Maximum number of coalition members: 3 (k)Possible coalitions for A1 (P1): {A1 }, {A1 , A2 }, {A1 , A3 }, {A1 , A4 }, {A1 , A2 , A3 }, {A1 , A2 , A4 }, {A1 , A3 , A4 }

Coalitions with A2 = {A1 , A2 }, {A1 , A2 , A3 }, {A1 , A2 , A4 }

Example (II)

A1 contacts A2, requests B2 and commits to calculate S12 = { {A1, A2}, {A1, A2, A3} }The coalitions in S12 are added to L1 and deleted from P1

Agent 2 deletes S12 from P2

[ Later, delete from P1 the coalitions commited by agent 2, S21 ]Similar process with agents A3 and A4

End of stage 1

All the agents have (collectively) distributed all possible coalitionsEach agent i has a list Li of coalitions whose value it has to (repeatedly) calculate

Stage 2a: calculate coalitional values (I)Li

cr = Li (in the first iteration) For each coalition C in Li

cr doEc = empty setBc = sum(Bk) for all agents k in C[potential capability of coalition C]For each pending task tj, do

Check if coalition C can do task tj (Bc >= Bj)If it can, calculate the net benefit of tj for C, ejej = sum of market value of capabilities needed in tj

– sum of the capabilities costs – internal coordination costs

Ec = Ec + {(tj , ej )}End for

Stage 2a: calculate coalitional values (II)Ec contains the benefits that coalition C can gain from each of the tasks it is capable of performing(tcbest, Vc ) = max-value(Ec ) Coalitional value of C and its related taskcc = 1 / Vc Coalitional cost of CEnd for

At the end of this step, each agent has calculated the best coalitional value/cost for each coalition, and it knows the most profitable task for each coalition

Example (IIIa)

Following the previous example, assume that L1 = { {A1, A2}, {A1, A3}, {A1, A2, A3}, {A1, A3, A4}}A1 calculates the best value (and cost) of each coalition in L1

V12, V13, V123, V134

c12, c13, c123, c134

Example (IIIb)How is V134 calculated?Assume there are 3 possible tasks

The coalition {1,3,4} can make task 1, with a net benefit 50The coalition {1,3,4} can’t make task 2The coalition {1,3,4} can make task 3, with a net benefit 35

Thus, the coalitional value of the coalition {1,3,4} (V134) is 50, and its coalitional cost c134 is 0.02If this coalition is formed in this iteration, it will be assigned task 1

Stage 2b: form one coalition (I)

The weight wi of a coalition Ci is defined as wi = ci / |Ci |

Ci = coalition of Li with minimum weightBest coalition for agent i

Each agent announces publicly the weight of its best coalitionwlow = minimum of the announced weights

Best coalition for all agents [Clow , tlowbest]

Stage 2b: form one coalition (II)

If I am an agent in Clow, join the other agents in Clow to perform task tlow

best

Delete the members of Clow from the list of candidates to future coalitionsLi = Li – coalitions with agents in ClowT = T – tlow

best

Licr = coalitions of Li whose value has to be

recalculatedThose coalitions whose previous best value was making tlow

best

Example (IV)Following the previous example, A1 calculates the weights of its coalitionse.g. w12 = c12 / 2 ; w134 = c134 / 3

A1 sends its best weight (e.g. w12), and receives the best weigths calculated by agents A2, A3 and A4

After that, all agents know who has found the best coalitione.g. w12 is the minimum => the best coalition is {A1 ,A2 }

Example (V)

Agents A1 and A2 join together to perform the task t12

best (A1 knows which task it is)These agents quit the coalition formation processIn this example, in the next iteration, agents A3 and A4 would calculate their possibilities of coalition, with respect to the pending tasks, and would decide whether to join or not

Ideas for the practical exercise (I)

VotingDifferent firemen could vote which fire to fight if they want to attack always together to join forces to stop fires as soon as possibleThe vote of each firetruck could depen on its actual position, the position of the fires, its actual state (idle, in route to a fire, attacking a fire), its charge of fuel, etc.

Ideas for the practical exercise (II)

Coalition formation (I)The fires on the city could be the pending tasks. The firetrucks could decide how to join in subgroups (coalitions) to fight each of the firesSame ideas for policemen/roads, or ambulances/buildings with wounded people

Ideas for the practical exercise (III)

Coalition formation (II)Another idea could be to make a coalition of some firetrucks, police cars and ambulances for each of the buildings on fire.

The working groups of the course are encouraged to use different coordination mechanisms for the different elements of the problem (e.g. to use a Contract Net protocol to assign ambulances and a voting mechanism to assign firetrucks).

Readings for this week

Article “Methods for task allocation via agent coalition formation”. Shehory, Kraus.Chapters 12 and 13 of the book by M.Wooldridge “An introduction to MultiAgent Systems” (2nd edition).Chapter 9 of the book by M.Fasli “Agent technology for e-commerce”.