Agent Technology for e-Commerce
-
Upload
bradley-barrera -
Category
Documents
-
view
37 -
download
1
description
Transcript of Agent Technology for e-Commerce
Agent Technology for e-Commerce
Chapter 7: Elements of Strategic Interaction
Maria Faslihttp://cswww.essex.ac.uk/staff/mfasli/ATe-Commerce.htm
2Chapter 7
Agent Technology for e-Commerce
Agents in electronic markets
Agents have different (perhaps conflicting) goals Each agent is trying to maximise its own payoff without
necessarily being concerned about the welfare of other agents Agents: consumers, producers Resources, services, goods Market infrastructure
3Chapter 7
Agent Technology for e-Commerce
Market economy
A market economy is a setting in which the goods and services that a consumer may acquire are available at known prices
The goods, initial endowments, and technological possibilities are owned by consumers.
The value is derived from consuming (or owning) goods and services.
Some of the consumer agents can use some of the commodities to produce others (producers).
4Chapter 7
Agent Technology for e-Commerce
Basic principles
The behaviour of agents in an economic setting is governed by two principles:
Optimisation principle: agents try to choose the best patterns of consumption that they can afford
Equilibrium principle: prices adjust until the amount that people demand of something is equal to the amount that is being supplied
5Chapter 7
Agent Technology for e-Commerce
Example: Market of flats
Two types of flats available: close to the University and further away
Students are interested in renting as close to the University as possible, but paying as little as possible
Landlords are interested in maximizing the rent from their flats
6Chapter 7
Agent Technology for e-Commerce
The demand curve
7Chapter 7
Agent Technology for e-Commerce
The supply curve
8Chapter 7
Agent Technology for e-Commerce
Equilibrium: Supply meets demand
9Chapter 7
Agent Technology for e-Commerce
Pareto efficient allocations
Pareto Efficiency is one possible way of checking if an economic system is producing an ‘optimal’ economic outcome.
An allocation x is Pareto efficient, if there is no other allocation in which
an agent is strictly better off no other agent is worse off
10Chapter 7
Agent Technology for e-Commerce
Consumption bundles
The objects of consumer choice are called consumption bundles. If bread is product x1 and butter is product x2 then (x1, x2)
represents a consumption bundle, also written x Often only two goods are used; one of them is called ‘all other
goods’ so that we can focus on the trade off between one good and everything else
Agents have their own individual preferences over different consumption bundles
11Chapter 7
Agent Technology for e-Commerce
Preferences
(x1, x2) (y1, y2): a consumer strictly prefers bundle (x1, x2) rather than bundle (y1, y2) given the opportunity/choice
(x1, x2) ~ (y1, y2): a consumer is indifferent between the two bundles, i.e. would be just as satisfied with consuming bundle (x1,x2) as she would be with consuming bundle (y1, y2)
(x1, x2) (y1, y2): a consumer weakly prefers bundle (x1, x2) over bundle (y1, y2), i.e. the consumer prefers or is indifferent between the two bundles
12Chapter 7
Agent Technology for e-Commerce
Axioms of consumer preference
Assume a consumer i and consumption bundles x, y and z. Then: Bundles x and y are comparable, that is x y or y x, or both in
which case the consumer is indifferent (completeness) Any bundle is as at least as good as an identical bundle, that is,
x x (reflexivity) If x y and y z, then it follows that x z (transitivity)
13Chapter 7
Agent Technology for e-Commerce
Properties of preference relations
A preference relation is rational if it has the following two characteristics:
Completeness Transitivity
If is rational then: is both irreflexive (x x never holds) and transitive ~ is reflexive (x ~ x for all x), transitive (if x ~ y and y ~ z then
x ~ z) and symmetric (if x ~ y then y ~ x) if x y z, then x z
14Chapter 7
Agent Technology for e-Commerce
Indifference Curves
15Chapter 7
Agent Technology for e-Commerce
Monotonicity
If (x1, x2) and (y1, y2) are two bundles and (y1, y2) has at least as much of both goods and more of one, then: (y1, y2) (x1, x2)
The indifference curve has a negative slope
16Chapter 7
Agent Technology for e-Commerce
Convexity
If two bundles x and x’ are both elements of the consumption set, then the bundle x’’=ax+(1-a)x’ is also an element of the consumption set for any a[0,1]
Averages are preferred to extremes
17Chapter 7
Agent Technology for e-Commerce
Utilities
A utility function u(x) assigns a numerical value to each element in X (the set of consumption bundles), ranking the elements of X in accordance with the individual’s preferences
Ordinal utilities: the size of the utility difference does not matter Cardinal utilities: the size of the utility difference does not matter u can be transformed into another form through a process of
monotonic transformation which does not affect the ordering
18Chapter 7
Agent Technology for e-Commerce
Elements of a market economy
Assume a market where n goods are present Each consumer i has a utility function ui(xi) which determines its
preferences over various consumption bundles xi =(xi1, xi2,…,xin)
Each consumer i has an initial endowment (resources) ei
ei =(ei1, ei2 ,…, ein) is the vector describing consumer i’s The initial total endowment of good g available in the economy
is
19Chapter 7
Agent Technology for e-Commerce
Producers can use some of the commodities to produce others yj =(yj1, yj2 ,…, yjn) is the vector describing producer j’s production
of commodity g The total (net) amount of good g available in the economy is
The market has prices p = (p1,p2,…,pn); pg is the price of good g Prices specify the goods’ exchange rates and also determine the
value of the consumers’ initial endowments
20Chapter 7
Agent Technology for e-Commerce
The profit of producer j is pyj
Each consumer i owns a share ij of producer j’s profits with
Hence i has a claim to a fraction of j’s profits
21Chapter 7
Agent Technology for e-Commerce
Equilibrium
A market reaches an equilibrium state when there is no agent who wishes to deviate from that state
22Chapter 7
Agent Technology for e-Commerce
General equilibrium properties
Each general equilibrium is Pareto efficient A general equilibrium exists if
there is a positive endowment of every good the agents’ preferences are continuous, convex and monotone
A general equilibrium is unique if there is gross substitutability, i.e. raising the price of one good will not decrease the demand of another
23Chapter 7
Agent Technology for e-Commerce
Welfare Theorems
First Welfare Theorem: Any competitive equilibrium is Pareto efficient
Second Welfare Theorem: If the preferences and the technologies are convex, then any feasible Pareto optimal solution is a general equilibrium for some price vector and a set of endowments
24Chapter 7
Agent Technology for e-Commerce
Limitations
A general equilibrium may not exist if: Agents (consumers or producers) have market power The aggregate excess demand function is noncontinuous (small
changes in price result in big jumps in the quantity demanded) Agents’ preferences have:
Externalities (some agent’s consumption or production directly influences another agent’s utility)
Nonconvexities Complementarities (one commodity complements another)
25Chapter 7
Agent Technology for e-Commerce
Finding equilibrium solutions
How is a market equilibrium reached? Algorithms need to take into account the tradeoffs between
agents and the fact that the values of different goods to a single agent may be interdependent
Price tatonnement process A distributed algorithm proposed by Leon Walras An iterative price adjustment scheme which uses a steepest-
descent search method in order to find an efficient solution (provided that one exists)
26Chapter 7
Agent Technology for e-Commerce
Game Theory
A game is a formal representation of a situation in which a number of agents interact in a setting of strategic interdependence
An agent’s welfare depends not only on its own decisions and actions but also on the other agents’ decisions and actions: the agents are in situations of strategic interdependence
27Chapter 7
Agent Technology for e-Commerce
Strategic games
Elements of strategic interaction (game): Players: Who plays the game; number of players Rules: Who moves when? What can they do? Outcomes: For each possible set of decision/ actions by the
players, what is the outcome of the game? Payoffs: What are the players’ preferences (utilities) over all the
possible outcomes? Information: What sort of information do players have? Chance: Probability distribution over chance events, if any
28Chapter 7
Agent Technology for e-Commerce
Simple game example
Players: 2 players Rules: Player A writes one of two words on a piece of paper
‘Top’ or ‘Bottom’. Player B simultaneously and independently writes ‘Left’ or ‘Right’ on a piece of paper. Each player submits their piece of paper
Outcomes and payoffs in normal form representation
29Chapter 7
Agent Technology for e-Commerce
Strategies
A strategy is a complete contingent plan or decision rule that describes how the player will act in each possible and distinguishable circumstance in which it is called upon to play
The player’s distinguishable circumstances (the player’s perspective) is represented by the set of its information sets
A pure strategy (deterministic) for player i specifies a deterministic choice (a single action sm
i) at each of its information sets
The strategy space is represented by a vector Si=(s1i,s2
i,…sni),
where sni is the n-th strategy chosen by player i
30Chapter 7
Agent Technology for e-Commerce
Extensive form representation
The extensive form representation of a game consists of: The initial node (root) branches decision nodes terminal nodes
Player B
Player A
T B
RL
(1,3) (1,0)(3,1)(1,1)
Information setfor Player B
RL
31Chapter 7
Agent Technology for e-Commerce
Information in games
Perfect information Imperfect information Perfect recall Common Knowledge
Games of complete (incomplete) information Games of certainty (uncertainty) Games of symmetric (asymmetric) information
32Chapter 7
Agent Technology for e-Commerce
Categories of games
Cooperative games: purpose is to develop mechanisms for cooperation
Competition games (zero sum games): the total benefit of all players in a game adds up to zero; one player can only benefit at the expense of another
Coexistence games: in biology, what is the right (equilibrium) mixture of behaviours (strategies) among a population of a particular species?
Commitment games: focus on commitment
33Chapter 7
Agent Technology for e-Commerce
Pareto efficiency in games
Agents are expected utility maximizers and therefore they prefer higher payoffs than lower ones
The players choose their strategies and arrive at a solution A solution to a game is Pareto efficient if there is no other
solution in which: a player is strictly better off no other player is worse off
A Pareto efficient solution is the socially optimal solution of a game
34Chapter 7
Agent Technology for e-Commerce
Dominant and dominated strategies
A strategy s*i is player’s i strictly dominant strategy if it
maximises its payoff regardless of what the other players do A dominant strategy may not be Pareto efficient Some games do not have strictly dominant strategies A strategy for a player i is dominated if there exists some
alternative strategy that yields a greater payoff regardless of what the other players will do. Hence, a strategy s*
i is a strictly dominant strategy if it dominates every other strategy in Si
35Chapter 7
Agent Technology for e-Commerce
Dominant Strategy Equilibrium
• When all players have strictly dominant strategies, the outcome that ensues is called a dominant strategy equilibrium • The dominant strategy equilibrium concept makes no assumptions about the agents’ beliefs – very strong and robust• Not all games have a dominant strategy equilibrium
36Chapter 7
Agent Technology for e-Commerce
Nash Equilibrium
Battle of the Sexes (BoS): no dominant strategy equilibrium An outcome (a pair of strategies) is a Nash equilibrium if each
player’s strategy is an optimal choice given the other players’ strategies
A game can have more than one Nash equilibria A game may have no Nash equilibrium
37Chapter 7
Agent Technology for e-Commerce
The Nash equilibrium can be defined in terms of the so-called best-response function
Bi(s-i) may contain many strategies At a Nash equilibrium, each agent’s strategy is an optimal
response to the other agents’ strategies
38Chapter 7
Agent Technology for e-Commerce
The Nash solution concept is underpinned by very strong assumptions
To play a Nash equilibrium in a single shot game Every agent must have complete information about the others’
payoffs and preferences over outcomes (i.e. they must be common knowledge)
Rationality must be common knowledge All agents must select the same Nash equilibrium
39Chapter 7
Agent Technology for e-Commerce
Mixed strategies
Players do not always make their choices with certainty. A player can randomise when faced with a choice
A mixed strategy is a probability distribution over pure strategies:
where i=(smi) is the probability that i will choose strategy sm
i
Since i is a probability distribution we require:
40Chapter 7
Agent Technology for e-Commerce
Mixed Strategy Nash Equilibrium
In the BoS game Sally chooses Basketball with p>0 Kevin chooses Shopping with q>0
For Sally: q(2)+(1-q)(0)=q(0)+(1-q)(1) For Kevin: (1-p)(1)+p(0)=(1-p)(0)+p(2)
p=q=1/3
41Chapter 7
Agent Technology for e-Commerce
The mixed strategy in which each player chooses the other’s favourite event with a probability 1/3 and their own with probability 2/3 is a mixed strategy equilibrium – but inefficient
Every finite strategic-form game has a mixed strategy Nash equilibrium
42Chapter 7
Agent Technology for e-Commerce
Interpretation of mixed strategies
An attempt by a player to behave unpredictably As an expression of the other player’s beliefs regarding the pure
strategy that the player itself is going to choose Information that the players have about past interactions Before the player moves it receives a private signal on which it
can base its decision (but not consciously) As above but the random factors now affect payoffs. Each player
observes its own preferences, but not those of the others (this interpretation is due to Harsanyi)
43Chapter 7
Agent Technology for e-Commerce
Behaviour strategies
A behaviour strategy specifies the probability with which each action will be chosen, conditional on reaching that information set
Hence, a behaviour strategy specifies at each information set, a conditional probability distribution over the actions available at that information set
44Chapter 7
Agent Technology for e-Commerce
The prisoner’s dilemma
The Pareto efficient solution is for both players to cooperate Each player has a strictly dominant strategy to defect Dominant strategy equilibrium which is not Pareto efficient Problem lies in the uncertainty that each player faces: each has to
speculate about the other’s move Does cooperation among agents arise as a result of irrational
behaviour?
45Chapter 7
Agent Technology for e-Commerce
Repeated PD
How do you play such a game? Depends on whether the game is one-shot, or it is to be played a
finite or infinite number of times In repeated PD each agent has the opportunity to punish the other
for defection
46Chapter 7
Agent Technology for e-Commerce
Finite number of times: if the number of rounds are known, then in each round both agents will choose to defect. Why?
Consider two players playing the PD for 10 rounds What is going to happen in game 10? Both agents will defect
– there is no point in cooperating as this is the last game What is going to happen in game 9? Since the last game does
not matter anyway, both players will attempt to exploit each other’s cooperative nature and defect
Using backward induction: both players will defect in each game
47Chapter 7
Agent Technology for e-Commerce
Infinite (or unknown) number of times The situation now changes As the players do not know how many games they will have
to play and each player can punish the other’s defection in the next round, each player has the incentive to cooperate
Axelrod’s shadow of the future
48Chapter 7
Agent Technology for e-Commerce
Axelrod’s Tournament
In 1980 Robert Axelrod invited scientists to encode their PD strategies and compete against each other
Winner strategy: Anatol Rappaport’s tit-for-tat in the first round cooperate in round r>1, do whatever your opponent did in round r-1
The significance of the result has been debated: does cooperation prevail after all?
49Chapter 7
Agent Technology for e-Commerce
Tit-for-tat is successful as It offers an immediate punishment for the other agent’s defection It is a forgiving strategy, as it only punishes the defector once It is a rewarding strategy, if the other agent cooperates, it rewards
this by continuing to cooperate
50Chapter 7
Agent Technology for e-Commerce
Dynamic games
Games in which the players move sequentially Agents that move later in the game have an advantage as they can
see the others’ moves
Player B
Player A
C D
DC
(-1,-1) (-5,-5)(0,-10)(-10,0)
DC
51Chapter 7
Agent Technology for e-Commerce
The concept of a subgame
A subgame in an extensive form game is a sub-tree which: Starts at a single node (a singleton information set) Includes all decision nodes and terminal nodes of the original tree
following this node Does not cut any information set of the extensive form
representation of the original game
52Chapter 7
Agent Technology for e-Commerce
Subgame perfect Nash equilibrium
A strategy profile s constitutes a subgame perfect Nash equilibrium if it constitutes a Nash equilibrium of all subgames of the game (including the game itself)
The strategy combination (D, DD) is a subgame perfect Nash equilibrium
Player B
Player A
C D
DC
(-1,-1) (-5,-5)(0,-10)(-10,0)
DC
53Chapter 7
Agent Technology for e-Commerce
The entrant-incumbent game
Entrant considers whether to enter a new market in which the incumbent operates
The incumbent would obviously like the entrant to stay out Will the entrant decide to enter the market? What will the incumbent do?
Incumbent
Entrant
Stay out
Enter
(0,2)
(1,1)(-1,-1)
AcquiesceFight
54Chapter 7
Agent Technology for e-Commerce
Bayesian Nash games
A game of incomplete information models situations in which at least one player has uncertainty about the other players’ payoff functions
Such games are modelled by introducing Nature as a player: Nature chooses in the beginning of the game all those aspects that are not common knowledge
55Chapter 7
Agent Technology for e-Commerce
A Bayesian game of incomplete information is characterized by: A finite set N of agents (the set of players) with n>1 Each agent iN can choose a strategy si from a nonempty set Si
Each agent i has some private information ii called the type of the agent which determines that player’s payoff function
Players have initial beliefs about the type of each other: a probability function pi :i→ (-i) specifies i’s belief about the type of the other agents given its own type
A payoff or utility function ui :S→
56Chapter 7
Agent Technology for e-Commerce
Timing of a static Bayesian game: Nature draws a type vector =(1,…,n)
Nature reveals i to player i, but not to the other players The players simultaneously choose their actions depending on the
assigned type Payoffs ui(s1,…,sn| ) are received
57Chapter 7
Agent Technology for e-Commerce
Player i can use Bayes’ rule to compute its posterior belief:
A Bayesian Nash equilibrium is defined as the strategy profile in which each player’s type-contingent strategy must be the best response to the other players’ strategies
In equilibrium, every agent chooses a strategy to maximize
expected utility in equilibrium with expected-utility maximizing strategies of other agents
58Chapter 7
Agent Technology for e-Commerce
Main difference between Nash equilibrium and Bayesian Nash equilibrium: in BNE i’s strategy si(i) must be best response to the distribution over strategies of other agents, given distributional information about the preferences of the other agents
Agent i does not necessarily play a best response to the actual strategies of the other agents
Bayesian Nash makes more reasonable assumptions about the agents’ information than Nash, but is a weaker solution concept than dominant strategy equilibrium
59Chapter 7
Agent Technology for e-Commerce
Example
• Kevin attributes the probability p to Sally being a basketball fan and 1-p to her being a shopping fan
• Sally is going to play Basketball or Shopping as these are her dominant strategies depending on what type she is
60Chapter 7
Agent Technology for e-Commerce
Hence, for Sally:
Kevin’s optimal response depends on p:
For Kevin, what is optimal depends on the value p• If p>3/4 then 2p+0(1-p)>1p+3(1-p), and he must play Basketball• If p<3/4 then the highest expected payoff is obtained by playing Shopping• If p=3/4 then Kevin is indifferent between the two strategies
61Chapter 7
Agent Technology for e-Commerce
Beliefs and sequential rationality
Two pure strategy equilibria (L,L’) and (N,M’) which are also subgame perfect, but (N,M’) is based on a noncredible threat
Player A N (1,3)
(0,1)(0,2)
Player B
(0,0)(2,1)
M’L’
L M
L’ M’
62Chapter 7
Agent Technology for e-Commerce
The equilibrium concept needs to be strengthened, to rule out the unreasonable subgame perfect Nash equilibrium,
Agents are endowed with beliefs at each of the information sets The players’ strategies must be sequentially rational: at each
information set, the strategy followed by the player who has to move (and its subsequent strategy) must be optimal, given the player’s belief at that information set and the other players’ subsequent strategies
An information set can be on or off the equilibrium path To update their beliefs on the equilibrium path the players can
use Bayes’ rule
63Chapter 7
Agent Technology for e-Commerce
Perfect Bayesian Nash equilibrium
A profile of strategies and a system of beliefs is a perfect Bayesian Nash equilibrium in an extensive form game if
1. the strategy profile is sequentially rational given the system of beliefs
2. at information sets on the equilibrium path, beliefs are determined by Bayes’ rule and the players’ equilibrium strategies, and at information sets off the equilibrium path, beliefs are determined by Bayes’ rule and the players’ equilibrium strategies whenever possible
64Chapter 7
Agent Technology for e-Commerce
The expected payoff for M’ is p(0)+(1-p)(1)=1-p The expected payoff for L’ is p(1)+(1-p)(2)=2-p For any p, 2-p>1-p So, sequential rationality
prevents B from
choosing M’
Player A N (1,3)
(0,1)(0,2)
Player B
(0,0)(2,1)
M’L’
L M
L’ M’