© 2013 Kiva Systems LLC Challenges in Multi-Robot Path Planning Pete Wurman CTO.
1 Economic Foundations and Game Theory Peter Wurman.
-
Upload
willis-simpson -
Category
Documents
-
view
222 -
download
1
Transcript of 1 Economic Foundations and Game Theory Peter Wurman.
1
Economic Foundations and Game Theory
Peter Wurman
2
Presentation Overview
Economics Economics of Trading Agents Economic modeling General Equilibrium and its Limitations Mechanism design Introduction to Game Theory Pareto Efficiency and Dominant strategy Nash Equilibrium Mixed Strategies Extensive Form and Sub-game Analysis Advanced Topics in Game Theory
3
Economics
Study of the allocation of limited resources in a society of self-interested agents.
Essential features: Agents are rational; Decisions concern the use of resources; Prices significantly simplify the allocation
process.
Note: agents are not assumed to be software entities here.
4
Trading AgentsAgent: software to which we ascribe
Beliefs and knowledge; Rationality; Competence; Autonomy.
Trading agent: software that participates in an electronic market and Is governed in its decision-making by a set of
constraints (budget) and preferences; Obtains the above from a user; Acts in the world by making offers (bids) on the
user’s behalf.
5
Economics of Trading Agents
We will consider economics of trading agents as software entities.
Elements of an Economic Model Resources; Agents; Market Infrastructure.
6
Resources
Resources Limited; Consumed (private) or shared (public).
Formalization N is the number of resources types; xi is an amount of resource i; x is a N-vector of quantities.
7
Two Types of Agents
Consumers Derive value from owning/consuming
resources.Producers
Have technologies to transform resources; Goal is to make money (distributed to
shareholders).Both have private information.
8
Consumer Preferences
Preferences (>, ≥) Total preorder over all bundles x in X
x ≥ x’ or x’ ≥ x (completeness)
x ≥ x’ and x’ ≥ x” implies x ≥ x”(transitivity)
9
Consumer Preferences (2)
Often, we assume convexity For all in [0,1], x ≥ x” and x’ ≥ x”
and x ≠ x’ implies [x + (1-) x’] ≥ x”x1
x2
x
x’x”
10
Preferences Expressed as Utility
Generally, we express preferences as a utility function: uj(x) assigns a numeric value to all
bundlesOften, we assume that utility is
quasi-linear in one resource: uj(x) = vj(x) + m,
where m is money
11
Consumer Endowments
Consumers generally begin with some resources, denoted ej.
Often, these endowments do not maximize the agent’s utility.
Agents engage in economic activities.
12
Simple Exchange Economy
• Suppose all participants are consumers
• How do we determine resources to exchange?• What is a “good” allocation?
Agent 1 Agent 2
Agent 3
Agent 1 Agent 2
Agent 3
13
Price Systems
Associate a price pi with each resource iPrices specify resource exchange rates:
One unit of i can be exchanged for pi/ph units of h.
Present a common scale on which to measure resource value.
Very compact representation of value
14
Solutions
An allocation assigns quantities of each resource to each consumer
Feasible allocations satisfy Material balance which requires that,
for all i, xi,j = ei,j ; Other feasibility constraints.
15
Solution Quality
Pareto efficiency There is no other solution in which
one agent is strictly better off, andno agent is worse off.
Global efficiency (when utility is quasilinear) Corresponds to maximizing j uj(xj); Unique.
16
Equilibrium
General Definition A state from which no agent wishes to deviate.
Equilibrium concepts make assumptions about Agent knowledge; Agent behaviors.
Equilibrium questions Do equilibria exist? How many? Do they support efficient solutions?
17
Classic Agent Behavior
Competitive assumption Agents solve optimization problem:
Find a bundle that maximizes agent’s utility,xi* = argmaxx uj(x);
Subject to agent’s budget, piei,j ;Assuming prices are given.
Agents truthfully state their demand (supply) zi = xi* - ei .
18
General Equilibrium
Definition: A price vector and allocation such that All agents are maximizing their utility
with respect to the prices; No resource is over demanded.
Also called Competitive or Walrasian equilibrium.
19
General Equilibrium Existence
A competitive equilibrium exists in an exchange economy if There is a positive endowment of every
good; Preferences are continuous, strongly
convex, and strongly monotone.One sufficient condition for existence is
gross substitutability Raising the price of one good will not
decrease the demand of another.
20
Production Economies
We allow agents to transform resources from one type to another.
Competitive Equilibrium exist if Production technologies have convex or
constant returns to scale.
21
Fundamental Theorems
First Welfare Theorem Any competitive equilibrium is Pareto
efficient.Second Welfare Theorem
If preferences and technologies are convex, any feasible Pareto solution is a Competitive equilibrium for some price vector and set of endowments.
22
Limitations of G.E. Model
When are the assumptions violated? When agents have market power When prices are nonlinear When agent preferences have
Externalities;nonconvexities (discreteness);Complementarities.
23
G.E. Summary
General Equilibrium Theory provides Some conditions under which
competitive equilibria exist and are unique.
Justification for price systems.But...
We have said nothing about how to reach equilibrium
24
Tatonnement
Tatonnement is the iterative price adjustment scheme proposed by Leon Walras (1874) Auctioneer announces prices; Agents respond with demands; Auctioneer adjusts price of most overdemanded
resource.
Convergence of tatonnement iterative price adjustment guaranteed if gross substitutability holds.
25
Mechanism Design
General Definition An allocation mechanism is a set of
rules that defineAllowable agent actions;Information that is revealed.
Examples Tattonement; Auctions; Fixed pricing.
26
Protocols
A protocol is a combination of a mechanism and assumptions on the agents’ behavior; Tatonnement & competitive assumption =
Walrasian protocol.
Protocols allows us to analyze systems when General Equilibrium conditions do not hold; Competitive assumptions are violated; Perfect rationality is intractable.
27
Two Sides of the Same Coin
Given assumptions about the agents, how do we design an allocation mechanism?
Given an allocation mechanism, how do we design an agent to participate in it?
28
Game Theory
Game theory is a general tool for analyzing mechanisms synthesizing strategies
29
Summary
The design of trading agents should be informed by economics.
General Equilibrium is the foundation of modern economic theory.
Competitive behavior is a simple form of competence.
But there is much more to the story…
30
A Game Players Actions Payoffs Information Finite game: has finite number of players and
finite number of decision alternatives for each player. We will consider examples of two-person games.
Zero-sum game: the sum of players’ payoffs equals zero.
Two-person-zero-sum games: one player’s loss is the other player’s gain.
31
Example
Players: Red & BlueActions
Red: join or pass Blue: join or pass
Payoffs
Join PassJoin 1 3Pass 0 2
Join PassJoin 1 3Pass 0 2
Red’s payoffs Blue’s payoffs
32
Play the Game
Join PassJoin 1 3Pass 0 2
Red’s payoffs Blue’s payoffs
Join PassJoin 1 3Pass 0 2
33
Normal (Strategic) Form
Join Pass
Join 1,1 3,0
Pass 0,3 2,2
Join PassJoin 1 3Pass 0 2
Red’s payoffs Blue’s payoffs
“Prisoners’ Dilemma”
Join PassJoin 1 3Pass 0 2
34
Pareto Efficiency
Join Pass
Join 1,1 3,0
Pass 0,3 2,2
Pareto Efficiency: There is no other solution in which
An agent is strictly better off;No agent is worse off.
35
Pareto Efficiency
Join Pass
Join 1,1 3,0
Pass 0,3 2,2
Pareto Efficiency: There is no other solution in which
An agent is strictly better off;No agent is worse off.
36
Dominant Strategy
Join Pass
Join 1,1 3,0
Pass 0,3 2,2
Dominant Strategy: A strategy for which the payoffs are
better regardless of the other player’s choice.
37
Dominant Strategy Equilibrium
Join Pass
Join 1,1 3,0
Pass 0,3 2,2
Dominant Strategy: A strategy for which the payoffs are
better regardless of the other player’s choice;Red plays join;Blue plays join.
38
Iterated Strict Dominance
Repeatedly rule out strategies until only one remains
L M R
U 4, 3 5, 1 6, 2M 2, 1 8, 4 3, 6D 3, 0 9, 6 2, 8
39
Iterated Strict Dominance
Repeatedly rule out strategies until only one remains
L M R
U 4, 3 5, 1 6, 2M 2, 1 8, 4 3, 6D 3, 0 9, 6 2, 8
Dominates
40
Iterated Strict Dominance
Repeatedly rule out strategies until only one remains
L M R
U 4, 3 5, 1 6, 2M 2, 1 8, 4 3, 6D 3, 0 9, 6 2, 8
Dominates
41
Iterated Strict Dominance
Repeatedly rule out strategies until only one remains
L M R
U 4, 3 5, 1 6, 2M 2, 1 8, 4 3, 6D 3, 0 9, 6 2, 8
42
Dominant Strategy Evaluation
When they exist, they are conclusive (unique).
Often they don’t exist.
43
No Dominant Strategy equilibrium.
“Matching pennies”
Dominant strategy equilibrium does not exist for pure strategies. Zero-sum game.
Head Tail
Head -1,1 1,-1
Tail 1,-1 -1,1
A solution exists if the game is played repeatedly.
44
Nash Equilibrium
An outcome is a Nash equilibrium if each player’s strategy is an optimal response given the other players’ strategies.
F BB 0,0 2,1F 1,2 0,0
“Battle of the Sexes”
45
Nash Equilibrium
An outcome is a Nash equilibrium if each player’s strategy is an optimal response given the other players’ strategies.
If red plays B, blue should play B.
If blue plays B, red should play B.
F BB 0,0 2,1F 1,2 0,0
46
Nash Equilibrium
An outcome is a Nash equilibrium if each player’s strategy is an optimal response given the other players’ strategies.
If red plays F, blue should play F.
If blue plays F, red should play F.
F BB 0,0 2,1F 1,2 0,0
47
Strategies
Strategy space Si = {si
1, si2,…si
n}Pure strategy
A single action, sij
Mixed strategy A probability distribution over pure strategiesi = {(pi
1, si1), (pi
2, si2),…(pi
n, sin)}
where j pij = 1
Von Neumann’s Discovery: every two-person zero-sum game has a maximin solution, in pure or mixed strategies.
48
Mixed-Strategy Equilibrium
A mixed-strategy equilibrium Red plays {(1/3, F)(2/3, B)} Blue plays {(2/3, F)(1/3, B)} E(ured) = 2/3, E(ublue) = 2/3
No other combination of probabilities is a Nash equilibrium
F BB 0,0 2,1F 1,2 0,0
49
Mixed Strategy equilibrium
Every finite strategic-form game has a mixed-strategy equilibrium (Nash, 1950).
No pure-strategy equilibrium. Mixed-strategy equilibrium:
Red plays {(1/2, H)(1/2, T)};Blue plays {(1/2, H)(1/2, T)}.
H TH 1,-1 -1,1T -1,1 1,-1
“Matching Pennies”
50
Assumptions So Far
Complete information: Agents know each other’s strategy
space and payoffs.Common knowledge:
Moreover, each agent knows the other knows…
No communicationSingle round
51
Stage Games
Games in which the players “take turns”
Actions are observablePayoff received at the end of the
game
52
Example: Matchsticks
There are four matchsticksYou may take either one or two
matchsticks on your turnThe last person to take a matchstick
loses
53
Game Tree for 4-Matchsticks4
3
2
2
1
1
1
1
2
22
2
11
1
1,0 0,1 0,1 0,1 1,0
54
Sub-game Analysis4
3
2
2
1
1
1
1
2
22
2
11
1
1,0 0,1 0,1 0,1 1,0
55
Sub-game Analysis4
3
2
2
1
1
1
1
2
22
2
11
1
1,0 0,1 0,1 0,1 1,0
56
Extensive Form
Extensive form contains: The set of players The order of moves The choices at each decision point The payoffs as a function of the moves
made The information each agent has at the
decision point The probabilities associated with
exogenous events (chance)
57
Information
Information is imperfect when An agent can’t observe the other
agents’ moves. There are stochastic events that occur
“in nature”.
58
Hidden-Move Matchsticks4
3
2
2
1
1
1
1
2
22
2
11
1
1,0 0,1 0,1 0,1 1,0
Informationset
59
Flip-a-Coin Matchsticks
4
3
2
2
1
1
1
2
2
11
1
1,0 0,1 0,1
4
3 2
1
1 2
22
0,1 1,0
n
60
AI: Minimax search
Developed in the context of zero-sum games Chess, matchsticks, etc.
Equivalent to backward inductionCan be enhanced using an evaluation
function to represent estimations of terminal node values Allows heuristics to guide search Allows pruning of dominated nodes before
expansion
61
Advanced Topics in Game Theory
Equilibrium Selection How do we choose among multiple Nash
equilibria? Are some inherently more likely to be
chosen than others?
62
Advanced Topics in Game Theory
Repeated Games Reward is received after each round Future rewards are discounted Punishment is possible Learning is possible
63
Advanced Topics in Game Theory
Learning in repeated games When an agent’s knowledge of other
agent’s payoffs is incomplete When an agent doesn’t know its own
payoffs
64
Game Theory Uses
Models of Contract negotiation Social choice Business strategy Auctions Marriage ...
65
Game Theory Conclusions
+Provides a precise description of multiagent interactions.
+Useful solution concepts.+Extremely general.
– Often inconclusive.– Often assumes much knowledge.– Extremely general.