Coalition Formation between Self-Interested Heterogeneous
Actors
Arlette van Wissen
Bart Kamphorst
Virginia Dignum Kobi Gal
Introduction
Allowing people to interact with computer systems in a manner
similar to the way that people interact with each other.
To achieve successful interactions, we must understand the factors involved.
This may provide new insights into:- how decisions are made- how agent’s behaviors are perceived
Types of interactions:- cooperative and helpful interactions- competitive interactions- cooperative interactions in competitive scenarios
Human-Agent Interaction
Introduction
utility functiona real number that represents the level of satisfaction of an actor in that state of the world.
non-cooperative games - self-interested actors who can make non-binding
commitments - basic modeling unit is the individual
cooperative games- players can make binding commitments- communication and negotiation between the players is allowed- groups of players (coalitions) may enforce cooperative behavior
- basic modeling unit is the group
Self-Interested Actors
Introduction
A coalition is a subset S of the set of players N where:
1. agreements take place involving approval by every player in S 2. no agreement between any other members of S and any
member of N – S is permitted
A coalitional game with transferable utility is a pair (N, v) where
1. N is the set of all players 2. v associates with each coalition S ⊆ N a real-valued payoff v(S)
that the coalition’s members can distribute among themselves. We assume that v(∅) = 0.
Coalition Formation
Related Work
To become part of successful coalitions, one has to deal with the following
problems (Gentry, 1987):
1. how to identify the members of a winning coalition 2. how to reach an agreement on the allocation of the payoffs3. how to make a tradeoff between payoffs and non-utilitarian
strategy preferences
Humans are not fully rational when it comes to these formation decisions.
Coalition Formation
Related Work
Ultimatum Game
Allocation of Payoff
Related Work
Ultimatum Game
Humans are not completely rational actors (Kagel & Roth, 1995)
Social factors influence human behavior:• Inequity aversion (Loewenstein, 1989)
• Intention (Ross & Fletcher, 1985)
• Altruism, fairness, reciprocity (Camerer, 2003)
Social factors also influence cooperation between humans and intelligent agents (van Wissen et al., 2009)
Allocation of Payoff
Related WorkHuman Coalition Formation
• Relative payoffs clearly matter in the formation of coalitions. There exists a strong correlation between relative payoffs and rejection behavior. (Bolton & Brosig, 2007)
• In 2- and 3-player coalition games, people respond by acting negatively reciprocally. They punish proposers by rejecting unfair offers. (Okada & Riedl, 2005)
• Less attention paid to human-agent interaction.
• Existing models generally do not consider trust or nature.
Related WorkTrust
For an actor a to be said to trust another actor b with respect to a particular goal g, a must have the following beliefs (castelfranchi, 1998, 2001) :
• Competence Belief: b is useful for achieving g and is able to provide the expected result • Disposition Belief: b is not only capable, but also willing to do what is necessary to achieve g • Dependence Belief: the results and rewards of achieving g depend on the involvement of b • Fulfillment Belief: g will come about due to b’s involvement
Related WorkTrust
experience based vs recommendation based?
Research Question
How do nature and trust influence people’s decisions
in mixed coalition formations?
To what extent do trust and fairness influence team formation in mixed networks?
How does the nature of actors affect the way people relate to their actions?
Do actors develop stable relations over time?
Hypotheses
Hypothesis 1
The nature of the actor affects the cooperation between participants.
Hypothesis 2
A trust relationship between the actors affects the cooperation between them.
Conceptual Design
Assumptions in CF
Groups
Prev work
The members of coalitions are predefined.
Our work
The members of a coalition can be freely chosen.
Interchangeability
Prev work
All potential members bring the same utility to a coalition.
Our work
Some potential members are preferred over others.
Membership
Prev work
Membership to a coalition is equally available to everyone.
Our work
Members have to meet certain requirements for membership to be available to them.
Conflict
Prev work
Conflict is eliminated by making agreements binding.
Our work
No conflict-free environment since agreements are not binding.
Conceptual Design
Examine interactions between self-motivated agents that must adapt their behavior depending on the behavior of other agents in the environment.
Package Delivery Problem
Experimental Design
``A key determinant of CT design was the goal of providing a vehicle for comparing the decision-making strategies people deploy when they interact with other people with those they deploy when computer systems are members of their groups.’’Grosz et al., 2004
Colored Trails
- Board
- Chips
- Proposals
- Phases
Experimental DesignPackage Delivery Problem in CT
Conceptual Design
Hypothesis 1The nature of the actor affects the cooperation between participants.
Deception
Hypothesis 2A trust relationship between the actors affects the cooperation between them.
Repeated interaction
Hypotheses revisited
Transferable payoff &
ability to choose team members
Experimental DesignPackage Delivery Problem in CT
Coalition formation game with:• N = 6
• v(Ni) = 3
v(Ni Nj) = 60
v(Ni Nj Nk) = 180
• superadditivity:A game G = (N,v) is superadditive if v(ST) ≥ v(S) + v(T)
for all S, T ∈ N such that S ∩ T = ∅
• imperfect information:players do not have knowledge of all actions and behavior of the players in the game and have partial visibility
• 5 rounds of t minutes, where 7 ≥ t ≤ 10
Experimental DesignPackage Delivery Problem in CT
- 6 players per game (P)- 3 colors (=P/2)
Experimental DesignPackage Delivery Problem in CT
- 6 players per game (P)- 3 colors (=P/2)
Experimental DesignPackage Delivery Problem in CT
- 6 players per game (P)- 3 colors (=P/2)- 6 large packages (=P) - 12 small packages (=2P)
Experimental Design
movie
Experimental Design
1. Survey
- preferences - strategies
2. Log
- proposals- coalitions- defections
Results
Results
Hypothesis 1The nature of the actor affects the cooperation between
participants. • Do players make more proposals to humans than to agents?• Do players offer more points (more fair splits) to humans than to agents?• Do players join more coalitions with a human as an initiator than with an
agent as initiator?
Hypothesis 2A trust relationship between the actors affects the cooperation
between them.• Do players who defect less obtain a higher score than those who defect
more?• Do initiators who propose more fair splits obtain a higher score than those
who propose less fair splits?• Do players who are more often initiator obtain a higher score than those
who are more often member?• Do players offer less points to those who defect more than to those who
defect less?
Results
What strategy did you use for choosing your team members?
``Whether members were reliable. I could give computers a smaller share without feeling guilty. I did build a good history with one other human player.’’
``I chose the computers mostly, since I thought that they would demand less points.’’
What strategy did you use for proposing a split of points?
``Split evenly with humans, keep more for myself when playing with computers.’’``Offered computer players less points, human players more.’’
How did your preference for humans/computers as a leader reflect in your strategy of joining and defecting from teams?
``I tended to feel worse about defecting from a human.’’
Discussion & Future Work
1. Write BDI agents based on a model of human decision making in this scenario (CTAPL framework).
2. Create a condition where the emphasis is on reciprocity: helping without direct payoff.
3. Have people play longer rounds to enable them to develop better trust relations.
Conclusion
Questions/Suggestions?
Game FlowInitiator
Game FlowInitiator
Game FlowMember
Game FlowMember