September, 2006
1
Teck H. HoGame Theory: Experiments
Game Theory: Experiments
Teck H. HoUniversity of California, Berkeley
September, 2006
2
Teck H. HoGame Theory: Experiments
OutlineMotivation and Modeling Philosophy
Mutual Consistency: p-Beauty Contest
Instant Convergence: Median-Effort Game
Self-Interest: Ultimatum Game
Summary
September, 2006
3
Teck H. HoGame Theory: Experiments
MotivationNash equilibrium and its refinements: Dominant theories in economics and business for predicting behaviors in games.Subjects do not play Nash in many one-shot games.Behaviors do not converge to Nash with repeated interactions in some games.Multiplicity problem (e.g., coordination games).Modeling subject heterogeneity really matters in games.
September, 2006
4
Teck H. HoGame Theory: Experiments
Standard Assumptions in Equilibrium Analysis
Assumptions Nash Cognitive Experience-Weighted Social Equilbirum Hierarchy Attraction Learning Preferences
"Solution" Method
Strategic Thinking X X X X
Best Response X X X X
Mutual Consistency X X
Utility Function
Self-Interest X X X
Instant Convergence X X X
September, 2006
5
Teck H. HoGame Theory: Experiments
Modeling Philosophy
Simple (Economics)General (Economics)Precise (Economics)Empirically disciplined (Psychology)
“the empirical background of economic science is definitely inadequate...it would have been absurd in physics to expect Kepler and Newton without Tycho Brahe” (von Neumann & Morgenstern ‘44)
“Without having a broad set of facts on which to theorize, there is a certain danger of spending too much time on models that are mathematically elegant, yet have little connection to actual behavior. At present our empirical knowledge is inadequate...” (Eric Van Damme ‘95)
September, 2006
6
Teck H. HoGame Theory: Experiments
OutlineMotivation and Modeling Philosophy
Mutual Consistency: p-Beauty Contest
Instant Convergence: Median-Effort Game
Self-Interest: Ultimatum Game
Summary
September, 2006
7
Teck H. HoGame Theory: Experiments
Example 1: p-Beauty Contest
n players
Every player simultaneously chooses a number from 0 to 100
Compute the group average
Define Target Number to be 0.7 times the group average
The winner is the player whose number is the closet to the Target Number
The prize to the winner is US$20Ho, Camerer, and Weigelt (AER, 1998)
September, 2006
8
Teck H. HoGame Theory: Experiments
Ingredients of an Economic Experiment
Abstract: Task is abstract so that we do not introduce auxiliary preferences
Saliency: Rewards must be associated with subjects’ actions
Dominance: Payoffs dominate thinking costs (i.e. mistakes are not cheap)
September, 2006
9
Teck H. HoGame Theory: Experiments
Economic vs. Psychological Experiments
ECONOMIC EXPERIMENTS PSYCHOLOGICAL EXPERIMENTS
LANGUAGE MOSTLY QUANTITATIVE MOSTLY VERBAL
TASK ABSTRACT CONTEXT SPECIFIC
SUBJECT'S MOTIVATION MONEY (PERFORMANCE-BASED) COURSE REQUIREMENT / FIXED FEE
THEORY PREDICTION MOSTLY POINT MOSTLY DIRECTIONAL
September, 2006
10
Teck H. HoGame Theory: Experiments
A Sample of CEOs
David Baltimore President California Institute of Technology
Donald L. Bren
Chairman of the BoardThe Irvine Company
Eli BroadChairmanSunAmerica Inc.
Lounette M. Dyer Chairman Silk Route Technology
David D. Ho Director The Aaron Diamond AIDS Research Center
Gordon E. Moore Chairman Emeritus Intel Corporation
Stephen A. Ross Co-Chairman, Roll and Ross Asset Mgt Corp
Sally K. Ride President Imaginary Lines, Inc., and Hibben Professor of Physics
September, 2006
11
Teck H. HoGame Theory: Experiments
Results in various p-BC contests
Subject Pool Group Size MeanCEOs 20 37.980 year olds 33 37.0Economics PhDs 16 27.4Portfolio Managers 26 24.3Game Theorists 27-54 19.1
September, 2006
12
Teck H. HoGame Theory: Experiments
The Cognitive Hierarchy (CH) Model
People are different and have different decision rules
Modeling heterogeneity (i.e., distribution of types of players). Types of players are denoted by levels 0, 1, 2, 3,…,
Modeling decision rule of each type
Camerer, Ho, and Chong (2004, QJE)
September, 2006
13
Teck H. HoGame Theory: Experiments
Modeling Decision RuleProportion of k-step is f(k)
Step 0 choose randomly
k-step thinkers know proportions f(0),...f(k-1)
Form beliefs and best-respond based on beliefs
Iterative and no need to solve a fixed point
g k ( h ) =f ( h )
f ( h ' )h ' = 1
K −1
∑
September, 2006
14
Teck H. HoGame Theory: Experiments
Choices of players in p-Beauty Contest
Level 0 chooses randomly with an average of 50
Level 1 assumes all others are level 0 and best respond with a choice of 35
Level 2 assumes others are levels 0 and 1. If f(0) = f(1), then level 2 will choose a value of
75.29)352150
21(7.0 =⋅+⋅⋅
September, 2006
15
Teck H. HoGame Theory: Experiments
Modeling Heterogeneity, f(k)
A1:
sharp drop-off due to increasing difficulty in simulating others’ behaviors
A2: f(0) + f(1) = 2f(2)
kkfkf τ
=− )1(
)(
September, 2006
16
Teck H. HoGame Theory: Experiments
Implications
!)(
kekf
kττ ⋅= −A1 Poisson distribution with mean and variance = τ
A1,A2 Poisson, τ=1.618 (golden ratio Φ).
September, 2006
17
Teck H. HoGame Theory: Experiments
Poisson Distribution
f(k) with mean step of thinking τ: !)(
kekf
kττ ⋅= −
Poisson distributions for various τ
00.05
0.10.15
0.20.25
0.30.35
0.4
0 1 2 3 4 5 6number of steps
frequ
ency τ=1
τ=1.5τ=2
September, 2006
18
Teck H. HoGame Theory: Experiments
Theoretical Properties of CH Model
Theory: τ ∞ converges to Nash equilibrium in (weakly) dominance solvable games (e.g., p-Beauty Contest)
Advantages over Nash equilibriumCan “solve” multiplicity problem (also picks one statistical distribution) Sensible interpretation of mixed strategies (de facto purification)
September, 2006
19
Teck H. HoGame Theory: Experiments
Results in various p-BC games
Subject Pool Group Size Mean Error (Nash) Error (CH) τ
CEOs 20 37.9 -37.9 -0.1 1.080 year olds 33 37.0 -37.0 -0.1 1.1Economics PhDs 16 27.4 -27.4 0.0 2.3Portfolio Managers 26 24.3 -24.3 0.1 2.8Game Theorists 27-54 19.1 -19.1 0.0 3.7
September, 2006
20
Teck H. HoGame Theory: Experiments
Standard Assumptions in Equilibrium Analysis
Assumptions Nash Cognitive EWA Social Equilbirum Hierarchy Learning Preferences
"Solution" Method
Strategic Thinking X X X X
Best Response X X X X
Mutual Consistency X X
Utility Function
Self-Interest X X X
Instant Convergence X X X
September, 2006
21
Teck H. HoGame Theory: Experiments
Example 2: Median-Effort Game
September, 2006
22
Teck H. HoGame Theory: Experiments
Actual Choices inMedian-Effort Game Over Time
1 2 3 4 5 6 71
4
7
10
0.000
0.100
0.200
0.300
0.400
0.500
0.600
Strategy
Period
Figure 1a: Actual choice frequencies
Van Huyck, Battalio, and Beil (1990)
September, 2006
23
Teck H. HoGame Theory: Experiments
The learning setting
In games where each player is aware of the payoff tablePlayer i’s strategy space consists of discrete choices indexed by j (e.g., 1, 2,…, 7)The game is repeated for several roundsAt each round, all players observed:
Strategy or action history of all other playersOwn payoff history
September, 2006
24
Teck H. HoGame Theory: Experiments
Research question
To develop a good descriptive model of adaptive learning to predict the probability of player i (i=1,…,n) choosing strategy j at round t in any game
)(tPij
September, 2006
25
Teck H. HoGame Theory: Experiments
Criteria of a “good” model
Satisfies plausible principles of behavior (i.e., conformity with other sciences such as psychology)
Fits and predicts choice behavior well
Ability to generate new insights
September, 2006
26
Teck H. HoGame Theory: Experiments
“Laws” of Effects in Learning
Law of actual effect: successes increase the probability of chosen strategies
Teck is more likely than Colin to “stick” to his previous choice (other things being equal)
Law of simulated effect: strategies with simulated successes will be chosen more often
Colin is more likely to switch to T than M
Law of diminishing effect: Incremental effect of reinforcement diminishes over time
$1 has more impact in round 2 than in round 7.
854
L R8
109
TMB
Row player’s payoff table
Colin chose B and received 4Teck chose M and received 5Their opponents chose L
September, 2006
27
Teck H. HoGame Theory: Experiments
EWA Learning and its Special Cases
Choice reinforcement learning (Thorndike, 1911; Bush and Mosteller, 1955; Herrnstein, 1970; Arthur, 1991; Erev and Roth, 1998): successful strategies played again
Belief-based Learning (Cournot, 1838; Brown, 1951; Fudenberg and Levine 1998): form beliefs based on opponents’ action history and choose according to expected payoffs
EWA learning nests both as special cases
September, 2006
28
Teck H. HoGame Theory: Experiments
Assumptions of Reinforcement and Belief Learning
Reinforcement learning ignores simulated effect
Belief learning predicts actual and simulated effects are equally strong
EWA learning allows for a positive (and smaller than actual) simulated effect
September, 2006
29
Teck H. HoGame Theory: Experiments
The EWA Model)0(),0( NAijInitial attractions and experience (i.e., )
Updating rules
Choice probabilities
11)(
)(
)())(,()1()1(
)( )(
))(,()1()1(
)(
+ )−(⋅=
⎪⎪⎩
⎪⎪⎨
⎧
≠⋅+−⋅−⋅
=+−⋅−⋅
=−
−
tΝtN
tsstN
tsstAtN
tsstN
tsstAtN
tAiij
iijiij
iijiijiij
ij
ρ
πδφ
πφ
∑=
⋅
⋅
=+i
ik
ij
m
k
tA
tA
ij
e
etP
1
)(
)(
)1(λ
λ
Camerer and Ho (Econometrica, 1999)
September, 2006
30
Teck H. HoGame Theory: Experiments
EWA Model and Laws of Effects
Law of actual effect: successes increase the probability of chosen strategies (positive incremental reinforcement increases attraction and hence probability)Law of simulated effect: strategies with simulated successes will be chosen more often ( δ > 0 )Law of diminishing effect: Incremental effect of reinforcement diminishes over time ( N(t) >= N(t-1) )
11)(
)(
)())(,()1()1(
)( )(
))(,()1()1(
)(
+ )−(⋅=
⎪⎪⎩
⎪⎪⎨
⎧
≠⋅+−⋅−⋅
=+−⋅−⋅
=−
−
tΝtN
tsstN
tsstAtN
tsstN
tsstAtN
tAiij
iijiij
iijiijiij
ij
ρ
πδφ
πφ
September, 2006
31
Teck H. HoGame Theory: Experiments
The EWA model: An Example
Period 0:
Period 1:
)0(),0( BT AA
1)0()0()0()1(
1)0(8)0()0()1(
+⋅4+⋅⋅
=
+⋅⋅+⋅⋅
=
NNAA
NNAA
BB
TT
ρφ
ρδφ
Row player’s payoff table
History: Period 1 = (B,L)T
B
L R
8
4
9
10
September, 2006
32
Teck H. HoGame Theory: Experiments
Reinforcement Model: An Example
Period 0:
Period 1:
)0(),0( BT RR
4)0()1()0()1(
+⋅=
⋅=BB
TT
RRRR
φ
φ
Row player’s payoff table
History: Period 1 = (B,L)
1)0()0()0()1(
1)0(8)0()0()1(
+⋅4+⋅⋅
=
+⋅⋅+⋅⋅
=
NNAA
NNAA
BB
TT
ρφ
ρδφ
RF1)0(0 If
⇒ ,= ,=0,=
EWANρδ
T
B
L R
8
4
9
10
September, 2006
33
Teck H. HoGame Theory: Experiments
Belief-based(BB) model: An Example
Period 0:
Period 1:
)0()0()0( RL NNN +=
)0(10)0(4)0()0(10)0(4)0(
)0(9)0(8)0()0(9)0(8)0(
)0()0()0(
)0()0()0(
NNNBBE
NNNBBE
NNB
NNB
RLRLB
RLRLT
RR
LL
⋅+⋅=⋅+⋅=
⋅+⋅=⋅+⋅=
==
T
B
L R
8
4
9
10
1)0(4)0()0()1(10)1(4)1(
1)0(8)0()0()1(9)1(8)1(
1)0(0)0()1(
1)0(1)0()1(
+⋅+⋅⋅
=⋅+⋅=
+⋅+⋅⋅
=⋅+⋅=
+⋅+⋅
=+⋅+⋅
=
NNEBBE
NNEBBE
NNB
NNB
BRLB
TRLT
RR
LL
ρρ
ρρ
ρρ
ρρ
Bayesian Learning with Dirichlet priors
September, 2006
34
Teck H. HoGame Theory: Experiments
Relationship between Belief-based (BB) and EWA Learning Models
1)0(4)0()0( )1(10)1(4)1(
1)0(8)0()0( )1(9)1(8)1(
1)0(0)0()1(
1)0(1)0()1(
+⋅+⋅⋅
=⋅+⋅=
+⋅+⋅⋅
=⋅+⋅=
+⋅+⋅
=+⋅+⋅
=
NNEBBE
NNEBBE
NNB
NNB
BRLB
TRLT
RR
LL
ρρ
ρρ
ρρ
ρρ
1)0()0()0()1(
1)0(8)0()0()1(
+⋅4+⋅⋅
=
+⋅⋅+⋅⋅
=
NNAA
NNAA
BB
TT
ρφ
ρδφ
BB1 If
⇒=,=
EWAφρδ
BB EWA
September, 2006
35
Teck H. HoGame Theory: Experiments
Model InterpretationSimulation or attention parameter (δ ) : measures the degree of sensitivity to foregone payoffs
Exploitation parameter ( ): measures how rapidly players lock-in to a strategy (average versus cumulative)
Stationarity or motion parameter ( φ ): measures players’perception of the degree of stationarity of the environment
φρφκ −
=
September, 2006
36
Teck H. HoGame Theory: Experiments
Model Interpretation
CournotWeighted Fictitious Play
Fictitious Play
Average Reinforcement
Cumulative Reinforcement
September, 2006
37
Teck H. HoGame Theory: Experiments
New InsightReinforcement and belief learning were thought to be fundamental different for 50 years.
For instance, “….in rote [reinforcement] learning success and failure directly influence the choice probabilities. … Belief learning is very different. Here experiences strengthen or weaken beliefs. Belief learning has only an indirect influence on behavior.” (Selten, 1991)
EWA shows that belief and reinforcement learning are related and special kinds of EWA learning
September, 2006
38
Teck H. HoGame Theory: Experiments
Actual versus EWA Model Frequencies
1 2 3 4 5 6 7
1
2
3
4
56
78
910
0.000
0.100
0.200
0.300
0.400
0.500
0.600
Strategy
Period
Figure 1d: EWA mode l fre que nc ie s
1 2 3 4 5 6 7
1
4
7
10
0.0000.100
0.200
0.300
0.400
0.500
0.600
Strategy
Period
Figure 1a: Actual choice frequencies
September, 2006
39
Teck H. HoGame Theory: Experiments
Estimation and ResultsGame Calibration Validation Model No. of Parameters LL AIC BIC ρ 2 LL MSD
Median Action(M=378)
1-segment
Random Choice 0 -677.29 -677.29 -677.29 0.0000 -315.24 0.1217
Choice Reinforcement 8 -341.70 -349.70 -365.44 0.4837 -80.27 0.0301
Belief-based 9 -438.74 -447.74 -465.45 0.3389 -113.90 0.0519
EWA 11 -309.30 -320.30 -341.94* 0.5271 -41.05 0.0185
2-Segment
Random 0 -677.29 -677.29 -677.29 0.0000 -315.24 0.1217
Choice Reinforcement 17 -331.25 -348.25 -381.70 0.4858 -66.32 0.0245
Belief-based 19 -379.24 -398.24 -435.62 0.4120 -70.31 0.0250
EWA 23 -290.25 -313.25* -358.51 0.5374* -34.79* 0.0139*
September, 2006
40
Teck H. HoGame Theory: Experiments
ExtensionsHeterogeneity (JMP, Camerer and Ho, 1999)
Payoff learning (EJ, Camerer, Ho, and Wang, 2006)
Sophistication and strategic teaching Sophisticated learning (JET, Camerer, Ho, and Chong, 2002)Reputation building (GEB, Chong, Camerer, and Ho, 2006)
EWA Lite (Self-tuning EWA learning) (JET, Ho, Camerer, and Chong, forthcoming)
Applications:Auction markets (Book Chapter, Camerer, Ho, and Hsia, 2000)Product Choice at Supermarkets (JMR, Ho and Chong, 2004)
September, 2006
41
Teck H. HoGame Theory: Experiments
Standard Assumptions in Equilibrium Analysis
Assumptions Nash Cognitive Experience-Weighted Social Equilbirum Hierarchy Attraction Learning Preferences
"Solution" Method
Strategic Thinking X X X X
Best Response X X X X
Mutual Consistency X X
Utility Function
Self-Interest X X X
Instant Convergence X X X
September, 2006
42
Teck H. HoGame Theory: Experiments
Example 3: Ultimatum game
Proposer offers division of $10; responder accepts or rejectsEmpirical Regularities:
There are very few offers above $5Between 60-80% of the offers are between $4 and $5There are almost no offers below $2Low offers are frequently rejected and the probability of rejection decreases with the offer
Self-interest predicts that the proposer would offer 10 cents to the respondent and that the latter would accept
September, 2006
43
Teck H. HoGame Theory: Experiments
Could Subjects in US Learn to be More Self-Interested Over Time?(Roth et. al, 1991)
September, 2006
44
Teck H. HoGame Theory: Experiments
Does Stake Size Matter? (Slonim-Roth, 1998)
September, 2006
45
Teck H. HoGame Theory: Experiments
Do players learn to accept low offers at high stakes? No.
September, 2006
46
Teck H. HoGame Theory: Experiments
Ultimatum offer experimental sitesHenrich et. al (2001; 2005)
September, 2006
47
Teck H. HoGame Theory: Experiments
Ultimatum offers across 16 small societies (mean shaded, mode is largest circle…)
Mean offersRange 26%-58%
September, 2006
48
Teck H. HoGame Theory: Experiments
Low offers and low offers rejection rate
September, 2006
49
Teck H. HoGame Theory: Experiments
Modeling Challenge &Major Classes of Theories
The challenge is to have a general, precise, psychologically plausible model of social preferences
Three major classes of theories:Distributional (inequity-aversion. Fehr-Schmidt, Bolton-Ockenfels, Charness-Rabin)Reciprocal (Rabin)Self-image (Levine, Bernheim, Rotemberg)
September, 2006
50
Teck H. HoGame Theory: Experiments
A Model of Social Preference(Charness and Rabin, 2002)
Blow is a model that embeds the first 2 classes of theories. Player B’s utility is given as:
B’s utility is a weighted sum of her own monetary payoff and A’s payoff, where the weight places on A’s payoff depend on whether A is getting a higher or lower payoff than B and whether A has behaved unfairly.
otherwise. 0 and ,misbehaved hasA if 1 otherwise; 0 and , if 1 otherwise; 0 and , if 1 where
)1()(),(
=−==<==>=
⋅⋅−⋅−⋅−+⋅⋅+⋅+⋅=
qqssrr
qsrqsrU
AB
AB
BABAB
ππππ
πθσρπθσρππ
September, 2006
51
Teck H. HoGame Theory: Experiments
Parameters of the Model
The parameter θ models reciprocity
The parameters ρ and σ allow for a range of different distributional preferences
Difference aversion corresponds to σ < 0 < ρ < 1. (B likes money, and prefers that payoffs are equal, and prefers to lower A’s payoff when A does better than B)
Social-welfare preferences correspond to . (Subjects prefer more for themselves and the other person, but are more in favor of getting payoffs for themselves when they are behind than they are ahead)
Competitive preferences correspond to . (consistent with psychology of status, i.e., people like their payoffs to be high relative to others’ payoffs)
01 >≥≥ σρ
0≤≤ ρσ
September, 2006
52
Teck H. HoGame Theory: Experiments
September, 2006
53
Teck H. HoGame Theory: Experiments
Estimated Parameters
September, 2006
54
Teck H. HoGame Theory: Experiments
Standard Assumptions in Equilibrium Analysis
Assumptions Nash Cognitive Experience-Weighted Social Equilbirum Hierarchy Attraction Learning Preferences
"Solution" Method
Strategic Thinking X X X X
Best Response X X X X
Mutual Consistency X X
Utility Function
Self-Interest X X X
Instant Convergence X X X
September, 2006
55
Teck H. HoGame Theory: Experiments
What Can Neuroscience Contribute?What does mental activity look like when mutual consistency condition is met (during equilibration)?
There is no room for skill difference in equilibrium analysis. In CH model, we have a precise concept of skill. Is it possible to create a map between mental activity and superior performance in a certain class of games?
How does mental activity tell us which learning theories are correct? EWA predicts both actual and simulated effects and that the former is stronger than the latter (mental activity at striatal-cortical loop versus cortex).
Is there a difference in mental activity when reciprocal or distributional fairness is invoked?
http://128.32.67.154/siq/default1.asp
September, 2006
56
Teck H. HoGame Theory: Experiments
September, 2006
57
Teck H. HoGame Theory: Experiments
Estimates of Mean Thinking Step τ
September, 2006
58
Teck H. HoGame Theory: Experiments
Figure 3b: Predicted Frequencies of Nash Equilibrium for Entry and Mixed Games
y = 0.707x + 0.1011R2 = 0.4873
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Empirical Frequency
Pred
icte
d Fr
eque
ncy
Nash: Theory vs. Data
September, 2006
59
Teck H. HoGame Theory: Experiments
Figure 2b: Predicted Frequencies of Cognitive HierarchyModels for Entry and Mixed Games (common τ)
y = 0.8785x + 0.0419
R2 = 0.8027
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1Empirical Frequency
CH Model: Theory vs. Data
September, 2006
60
Teck H. HoGame Theory: Experiments
Actual versus Belief-Based Model Frequencies
1 2 3 4 5 6 71
47
10
0.000
0.200
0.400
0.600
Strategy
Period
Figure 1b: Be lie f-base d Mode l fre que ncie s
1 2 3 4 5 6 7
1
4
7
10
0.0000.100
0.200
0.300
0.400
0.500
0.600
Strategy
Period
Figure 1a: Actual choice frequencies
September, 2006
61
Teck H. HoGame Theory: Experiments
Actual versus Reinforcement Model Frequencies
1 2 3 4 5 6 7
1
3
5
7
9
0.000
0.100
0.200
0.300
0.400
0.500
0.600
Strategy
Period
Figure 1c: Choice reinforcement model frequencies
1 2 3 4 5 6 7
1
4
7
10
0.0000.100
0.200
0.300
0.400
0.500
0.600
Strategy
Period
Figure 1a: Actual choice frequencies
September, 2006
62
Teck H. HoGame Theory: Experiments
September, 2006
63
Teck H. HoGame Theory: Experiments
September, 2006
64
Teck H. HoGame Theory: Experiments
Fair offers correlate with market integration (top), payoffs to cooperation in everyday life (bottom)
MI: How much do peoplerely on market exchangein their daily lives?
PC: How important and how large is a group’s payoff fromcooperation in economic production?
Top Related