Testing the Waters: Taking Risks to Gain Reassurance in Public Goods Games

Testing the Waters: Taking Risks to Gain Reassurance in Public Goods GamesAuthor(s): Hugh WardSource: The Journal of Conflict Resolution, Vol. 33, No. 2 (Jun., 1989), pp. 274-308Published by: Sage Publications, Inc.Stable URL: http://www.jstor.org/stable/173955 .

Accessed: 08/05/2014 11:09

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Sage Publications, Inc. is collaborating with JSTOR to digitize, preserve and extend access to The Journal ofConflict Resolution.

http://www.jstor.org

This content downloaded from 169.229.32.137 on Thu, 8 May 2014 11:09:55 AMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=sage

http://www.jstor.org/stable/173955?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


Testing the Waters

TAKING RISKS TO GAIN REASSURANCE IN PUBLIC GOODS GAMES

HUGH WARD Department of Government University of Essex

Individuals often fail to cooperate because they are not sufficiently reassured that others involved share their desire for reciprocal cooperation. Such situations may be seen as Assurance games. The existing literature fails to examine the problems posed by lack of information about others'preferences, either assuming that information is perfect, or that it may be made perfect by mutual verbal reassurances. This article shows that in sequential public goods supergames, players with Assurance preferences may gather information about others' preferences from their game moves, and that it may pay them to take risks of short-term losses in order to do so. The most efficient information-gathering strategy for such a player is cooperation. The model helps us understand why players of Assurance often appear to take risks in order to "test the waters" with cooperative moves, and why the problem posed for collective action is sometimes successfully resolved.

In many social situations, players are prepared to cooperate pro- vided that "enough" others cooperate, but prefer noncooperation (defection) if that is the way others are going to behave. This suggests a game structure that Sen called Assurance (1969: 1-5). Figure 1 presents one variant of a class of Assurance games discussed by Taylor and Ward (1982: 354).

In contrast to (2 X 2) Chicken, where each player would prefer to cooperate if the other player defected and to defect if the other player cooperated, in (2 X 2) Assurance each of the players is willing to cooperate only if the other player does so. This is also different from the situation in single shot Prisoner's Dilemma (PD), where each player

AUTHOR'S NOTE: I would like to thank Bob Goodin, Michael Taylor, Brian Barry, Lisa Hooper, and two anonymous referees of this journal for their helpful comments on this article.

JOURNAL OF CONFLICT RESOLUTION, Vol. 33 No. 2, June 1989 274-308 ? 1989 Sage Publications, Inc.

274



Ward / PUBLIC GOODS GAMES 275

Player B

cooperate defect

aa' * d,b'

cooperate where a > b > c > d

Player A and a'> b'> c'> d'

defect

b,d' c,c *

Figure 1

would choose noncooperation irrespective of the choice of the other player. What Chicken, PD, and Assurance have in common, though, is that both players prefer the outcome in which mutual cooperation occurs to the outcome in which they both defect. Each of these game structures is a possible model of some problems of collective action; in each, rational individual action can lead to a strictly Pareto-inferior outcome. Note there are two equilibria in Assurance (starred in Figure 1): the equilibrium in which both players cooperate and the equilibrium in which both players defect. Players of Assurance may rationally choose not to cooperate if they are uncertain about the other player's preferences, which may result in the Pareto-inferior equilibrium where both players defect.

Recently, it has been argued that under certain conditions Assurance may be a useful model of both domestic politics (Taylor and Ward, 1982) and international relations (Stein, 1982; Plous, 1985). For example, some superpower crises may resemble Assurance. If each side has the option of standing firm or backing down, each may prefer to back down if the other also chooses to do so, but to stand firm if that is the other's choice. At the same time, each would prefer the compromise outcome where both sides have backed down to the confrontational outcome where both sides stand firm. I argue below that the later phases of the Berlin Wall Crisis best resemble Assurance.

Various versions of the n-person Assurance game also have some application. Rousseau's parable of the stag hunt, which can be construed as a version of n-person Assurance, is one widely applicable paradigm for viewing collective action and public-goods provision problems (Hardin, 1982: 167-169). Where one actor is attempting to use



276 JOURNAL OF CONFLICT RESOLUTION

"divide and rule" tactics against some group, n-person Assurance may be one common preference pattern: Each member of the group may be willing to resist so long as they are assured that enough others will also do so to prevent resisters being singled out for particular retribution, but if this condition is not met resistance might seem undesirable. For example, since 1979 successive Conservative governments in Britain have attempted to curb expenditure by local government with a view to reducing one component of total public expenditure. Although central control of local expenditure is tighter in Britain than in many systems, local authorities retain a significant, though decreasing, amount of discretion. Also, it proved difficult for central government to pick out some local authorities for particular punitive measures so long as large numbers of authorities were resisting central control. In part this was due to the informal, but nevertheless significant, norms that operate against complete central control of local government. It also proved administratively difficult individually to control large numbers of authorities, especially as some authorities were able partially to disguise their resistance behind smokescreens thrown up by "creative accoun- tancy," given the rather poor information available to central government about local expenditure. Thus, so long as they could be assured that enough others would stick to their existing spending plans, many local authorities were willing to do so. Prominent among the resisters were large, Labour-controlled, metropolitan authorities, highly "vis- ible" to central government in Whitehall. Despite the coordinating efforts of the various representative bodies for local government, enough local authorities moved in the direction that central government wanted to make selective and punitive measures against certain authorities administratively feasible and politically legitimate (Rhodes, 1986: 140-153). This example shows that it is by no means certain that players of Assurance games will achieve mutually cooperative outcomes.

Difficulties in achieving cooperation in Assurance games may also arise in international relations (Stein, 1982: 303, 315; Plous, 1985). One factor that may help explain this is that each side is uncertain about the other's preferences. In this article I will employ a rational actor model in which it is assumed that players (a) have well-defined utility functions over alternative outcomes, (b) choose the strategy maximizing their payoff, (c) do not necessarily have perfect information about the other player's utility function when choosing strategies, (d) assume other players choose strategies so as to maximize their payoffs, and (e) modify their prior beliefs about other players' utility functions in the light of their observed strategy choices. Assumption c conflicts with standard




game-theoretic axioms about perfect information, so I adopt a modifi- cation of the standard approach. If these assumptions are adopted, defection may be a rational strategy in Assurance games.

Suppose Player 1 has Assurance preferences and places the subjective probability of p upon Player 2 having Assurance preferences and p' (where p' = 1 - p) upon Player 2 having PD preferences. Suppose that the game is played once and that the game is sequential, 2 moving after 1. Then 1's expected payoff from cooperation is

p.a + p' .d i

while Player l's expected payoff from defection is:

p.c + p' .c = c

If the above assumptions about rational action hold 1 would only choose to cooperate if

p.a. + p' .d > p.c + p'.c

or

p'/p<(a-c)/(c-d)

In other words, 1 would not choose to operate unless a low enough probability was placed on 2 having PD preferences. Plous has suggested that the nuclear arms race between the superpowers is characterized by a Perceptual Dilemma, each side having Assurance preferences, but firmly believing that the other has PD preferences (1985: 368-369). My argument suggests that this sort of dilemma would persist even if each side believed there was some positive probability of the other having Assurance preferences.

However, all is not necessarily lost, even if both players would not choose to cooperate if the game is only played once. A more adequate formulation of most interesting social problems is the iterated game in which the single-shot game is played repeatedly; by appropriate choice of initial strategy players may sometimes infer enough about others'

1. Supposing that 1 cooperates, 2 would reply with a cooperative move if he or she had Assurance preferences and with defection if he or she had PD preferences. Thus 1 gets a payoff of a with probability p and a payoff of d with probability p'. I's expected payoff is obtained in the usual way by multiplying payoffs by probabilities of occurrence for each contingency and then summing over the various contingencies.




preferences from their game moves to become assured that cooperation is the strategy maximizing their expected payoff. Moreover, it may pay them to take the risk of incurring opportunity costs in order to gather this valuable information. For players with Assurance preferences, initial cooperation is the best strategy with which to "test the waters." Under certain circumstances it is worth taking the risk of possible short-term losses in order to deploy this information-gathering strategy.

Of course, it will hardly be news to students of bargaining, deterrence, and international crises that information may be gathered from the moves of the other side, and that it may be worth deliberately probing to obtain such information for future use (Ik1d, 1964: 47-50; George and Smoke, 1974: 75; Bacharach and Lawler, 1981: 80-84). Additionally, a great deal of experimental work has been carried out by social psychologists who have investigated the conditions leading to the development of cooperation in mixed-motive games and suggested ways of fostering cooperation (Rubin and Brown, 1975). Some have suggested that initial cooperation is a desirable means of fostering trust.2 In particular, Osgood (1962, 1979) has proposed a graduated-reduction-in-tension strategy where-by "small" initial cooperative moves lead, if recipro- cated, to higher levels of cooperation (Lindskold, 1978). Besides fostering trust in the mind of the other player, this strategy allows one player to investigate the other's intentions and preferences. However, this literature does little to suggest what tradeoffs must be contemplated when deciding to gather information about the other player. Although experimental psychologists have considered the effects of manipulating the payoff matrix and prior beliefs about opponents on the development of cooperation (Lindskold, 1978: 772), they have not attempted to model formally the decision problems surrounding the use, risks, and opportunity costs of strategies that might lead to cooperation. Osgood's strategy may be one method of overcoming uncertainty, but is it the most efficient? Formal modeling is one way to help answer this question.

There are several well-studied examples of superpower interactions in which both sides employed cooperative probes to investigate the other's intentions. These include "the Kennedy Experiment" (Etzioni, 1967), the Berlin Crises (Walker, 1982: 154; Leng, 1984), and, more recently, Gorbachev's moves during arms control talks (Dean, 1987: 34-40). The modified game-theoretic approach adopted here seems to be

2. Much of this experimental work has been conducted with iterated PD. I show below that an Assurance-like interaction arises where conditional cooperation is rational in this type of game but information about other players is imperfect, as it typically is in experimental work of this sort.




reasonably applicable where the "second" Berlin Crisis is concerned: By the time the Wall was built (in prototype form) on August 13, 1961, both the Soviet Union and the United States seem to have had relatively well-clarified objectives-a precondition for applying game-theory (Snyder and Diesing, 1977: 103). Although some aspects of the escala- tion of the crisis after the wall was built are probably best explained by loss of central control, the application of standardized, preplanned operating procedures, and the irrational maintenance of existing images in the face of incoming information stressed by the sociopsychological literature (Jervis, 1976; Kahneman, Slovic, and Tversky, 1982: 3-23), compared with some crises, the leaders on both sides seem to have exerted a relatively high degree of personal control from the center, and to have rationally assessed alternatives and incoming information as the crisis progressed toward its conclusion (Catudal, 1980). Where the standard game-theoretic model appears to break down is that there is strong evidence that neither leader knew for sure that the other had abandoned his earlier and wider aims and was, in fact, willing to compromise upon a settlement based on the division of Berlin by the wall (Adomeit, 1982: 207; George and Smoke, 1974:430-431). In fact, it may be reasonably inferred that both sides had Assurance preferences but were not certain that the other side shared their preferences. The building of the wall presented each side with the opportunity of accept- ing a compromise protecting their essential interests while avoiding potentially catastrophic armed confrontation. Moreover, it left neither side willing to accept the wall compromise unless the other side was willing to abandon its wider claims so that, as in a (2 X 2) Assurance game, each side preferred to back down so long as the other also did so, but to stand firm if the other side was unwilling to accept the compromise around the Wall.

Within the perspective adopted in this article, players of Assurance may make cooperative early moves in order to investigate their oppo- nent's preferences. This makes sense in terms of Kennedy's explicit adoption of a strategy whereby, having convinced Khrushchev of his resolve to protect the United States' core interests in Berlin, he went on to use small cooperative initiatives to suggest that he was willing to accept the compromise solution (George and Smoke, 1974: 432). It is also consistent with the statistical evidence that showed increased NATO cooperation was associated with increased Warsaw Pact cooperation (Tanter, 1974: 163; Leng, 1984: 348-350).

The international relations literature has often stressed the difficulties of applying game-theory to the behavior of states (George, Hall, and




Simons, 1971; Allison, 1971). However, in some cases-including the Berlin Wall crisis-a modified game-theoretic approach stressing imperfect information about others'preferences and strategies to gather information about those preferences may be applicable, and insights drawn from the construction of formal models may help explain why nations sometimes do, and sometimes do not, take risks to gather costly information about each other's preferences by using cooperative probes.

The existing formal literature on Assurance is not very helpful in answering this sort of empirical question. The literature neglects or downplays the question of uncertainty about the other side's preferences, and the failures of collective action that follow. Sometimes it is part of the definition of Assurance that each side has perfect information (Taylor and Ward, 1982: 354), so that each side rationally expects the cooperative outcome and failures of collective action are avoided (Elster, 1979: 21; Taylor, 1987: 18-19). Given that the players' situations are symmetric, one player might assume that the other has Assurance preferences if the player, her- or himself, does. However, collective action games often lack symmetry (Snyder and Diesing, 1977; Stein, 1982: 311; Taylor, 1987: 39) and to assume symmetry without confirm- ing evidence from the other side's moves leaves one open to exploitation.

In contrast to these arguments, Sen (1969) did not assume that each side had perfect information, but he did assume that cooperation can be achieved merely by verbally reassuring the other player about your preferences. However, this conclusion ignores the fact that it may pay actors with some preference structures deliberately to mislead others, either verbally or through their game moves. Others have suggested that in the long run informal processes will often lead to cooperative behavior becoming the convention, the cooperative solution becoming prominent even in the absence of full communication or formal cooperative mechanisms (Lewis, 1969; Hardin, 1982: 168; Stein, 1982: 311-316). But it hardly seems plausible that informal coordination will always work, or even develop.

Although some recent work exists on games of imperfect information, the problem of uncertainty has also been underplayed elsewhere in the game-theoretic literature on collective action.3 It is now well known

3. Following Harsanyi's path-breaking work (1967/8), several authors have examined Bayesian methods for updating prior beliefs about others'preferences in light of their game moves (Kreps and Wilson, 1982; Milgrom and Roberts, 1982; Kreps, Roberts, Milgrom, and Wilson, 1982; Wilson, 1985; Ordeshook, 1986). Assurance has yet to receive attention in this literature.




that under certain circumstances conditionally cooperative equilibria exist in iterated Prisoner's Dilemma games (Taylor, 1976; Hardin, 1982; Axelrod, 1984). In fact, a structure closely akin to Assurance may exist in such games: Each side would prefer to cooperate conditionally so long as the other side also chose to do so; but, because an equilibrium in which both sides defect indefinitely also exists in these games, each side would prefer not to cooperate conditionally if the other did not do so (Taylor, 1987: 67). But existing accounts do not emphasize that in many situations that might be modeled in this way it is unrealistic to assume each side is sure of the other's preferences. Because there is an implicit Assurance game involved here, the problems of building information and trust may be as great as in the game of Assurance itself. The arguments developed here are, then, applicable to many situations that can be seen as iterated PD as well as those modeled by Assurance itself.

The games examined in this article are two-person sequential games: Player 1 makes a move and Player 2 replies knowing what move Player 1 has made. Although the game-theoretic literature on international relations has often argued that players move sequentially (Brams and Hes- sel, 1984), it is more usual in the literature on collective action games to assume that players choose simultaneously and/ or in ignorance of what the other is doing. Clearly, simultaneous play poses additional problems that need attention in further research.

Supposing that Player 2 replies after Player 1 has made his move, she might choose straightforwardly or in a nonstraightforward manner. By a straightforward choice of reply I mean one which given 1 's prior move and 2's preference structure maximizes 2's payoff for that particular round of the game. Various sorts of nonstraightforward strategy choices are possible, of which two will be considered here. First, 2 might under some circumstances find it rational to bluff by playing cooperatively in order to make 1 believe that she had preferences which would lead her to cooperate if she replied straightforwardly. Second, 2 might find it rational to signal her desire for cooperation by replying cooperatively to a noncooperative first move by Player 1. Making inferences from 2's behavior is clearly more difficult when she might not reply straightforwardly. In the first section of the article, I assume 1 believes 2 will reply straightforwardly. In the second section I relax this assumption. The analysis of the simpler case provides an introduction to the more general analysis. Besides this, it may sometimes be reasonable to assume another player will act straightforwardly: By definition, nonstraightforward play involves incurring short-run costs for future benefits, and




players known heavily to discount such future benefits might reasonably be expected to reply straightforwardly.

THE SIMPLEST CASE: A SEQUENTIAL SUPERGAME OF TWO ROUNDS WHERE PLAYER 1 BELIEVES

PLAYER 2 WILL REPLY STRAIGHTFORWARDLY

INFORMATION FROM DIFFERENT STRATEGIES

Consider a supergame repeated twice only. Player 1 has Assurance preferences (like those in Figure 1) and is unsure about Player 2's preferences. Play is sequential, and Player 1 goes first. Player 1 knows (or believes) that 2 will play straightforwardly in the sense defined above. I show that if 1 plays C in the first round he will always know how best to play in the second round, while if 1 plays D in the first round he will not be sure whether to choose C or D in the second round. Second, I demonstrate that it may pay Player 1 to choose C in the first round even though this strategy is expected to yield a lower payoff than D in the first round.

There are 12 distinct (2 X 2) games in which Player 2 has a strict preference ordering over outcomes and in which she sees benefits from collective action, in the sense that she strictly prefers both players cooperating to both players defecting.4 As we are considering a sequen-

4. Let us denote the payoffs 2 obtains in the various contingencies in the same way that we did in Figure 1. Then the 12 strict preference orderings of interest are:

a'>b'>c'>d' class3 a'> c'> b'> d' class 3 a' > b' > d' > c' class 1 a' > d' > b' > c' class 1 a'>c'>d'>b' class3 a' > d' > c' > b' class 1 b'> a'> c'> d' class 2 b'> a'> d'> c' class 4 b'> d'> a'> c' class 4 d' > a' > b' > c' class 1 d' > a' > c' > b' class 1 d' > b' > a' > c' class 4

A moment's thought will convince the reader that many of these orderings are not plausible in collective action contexts, for 2 prefers a situation in which neither 1 nor 2 cooperate to that in which 1 cooperates alone. Nevertheless, to maintain the generality of the argument, I will ignore this point. Clearly if preference orderings need not be strict there are many more to be considered.




tial game in which Player 2 moves second, I could group these 12 into 4 relevant classes, for what matters to 1 is how 2 will reply to his own moves, not 2's actual preference ordering:

(Class 1) 2 always plays C no matter what 1 does (Class 2) 2 always plays D no matter what 1 does (like PD) (Class 3) 2 plays C if 1 plays C and D if 1 plays D (like Assurance) (Class 4) 2 plays D if 1 plays C and C if 1 plays D (like Chicken)

Depending upon how he chose at t = 0, Player 1 can make certain inferences from the way 2 replies to his move:

(i) If 1 plays C in the first round, and 2 replies with C, then 1 knows that 2 belongs to Class 1 or Class 3. Since 1 has Assurance preferences, and since 1 knows that 2 will always cooperate in the second round if he does, C is unambiguously I's best move in round two.

(ii) If 1 plays C in the first round, and 2 replies with D, then 1 knows that 2 belongs to Class 2 or 4. Clearly, the best move that 1 could make in the second round against a Class 2 player is D. Similarly, in a sequential game, the best a player with Assurance preferences could do against a "Chicken-like," Class 4 player is to choose D in the second round.

(iii) If 1 plays D in the first round, and 2 replies with C, then 1 knows that the other player either belongs to Class I or Class 4. However, while 1's best move at the second round against a Class 1 player is C, his best move against a Class 4 "Chicken-like" player is D. I is still unsure about how to move in round two.

(iv) If 1 plays D in the first round, and 2 replies with D, then 1 knows that 2 belongs either to Class 2 or Class 3. However, while l's best move in the second round against a Class 2 player would be D, his best move against an "Assurance-like" Class 3 player would be C. Again 1 is still unsure how to play in the second round if he plays D in the first round.

In brief, then, playing C in the first round brings in enough information about 2's preferences for 1 to have, in effect, perfect information in round two: He always knows how to choose in that round. In contrast, playing D in round one yields some clarification about 2's preferences, but it still leaves 1 unsure of how to act in round two. In Figure 2 these conclusions are illustrated by drawing part of the game tree as seen by player 1 at t 0 O. At each node I have indicated the following in abbreviated form: what I's beliefs about 2's preferences would be; 1's best strategy for the next round; and I's payoff in the next round (if he is certain what these are).



m c

4-J

/ 0-40) 0 is, 0 4 01

- - + 0 o - o <

/ U\ < , U )

01 01~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~u

N U _

/ W 'v( U d?U o

+ - 0-40-

I w-NJ

\~~~~~~~~~~~s 1, )

-' 1U

O N 4) + , U/ AcJt m 01

z~~~~~~~~ ' n 44

? e ~~~~~~~~~~~~~~~~~~~~~~~~O 44

U U e ~~~~- - _ '0 R , cl- u .I

284~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~-

U) 44~ ~ ~ ~~~~~~~~~~)G

-I~~~~~~~~~~~-

(N4 - 0'n

~~~01W01~~~~~~ 4dfl J 4

284~ ~ ~~~0~




EXPECTED COSTS AND BENEFITS OF INFORMATION GAINS

Suppose that, at time t = 0, 1 places subjective probabilities Pl, P2, P3

and p4 on 2 belonging to the four classes of preference structure defined above. Assume that Pi + P2 + p3 + p4 = 1.

Suppose first that 1 played C in the first round:

(i) Then his expected payoff in round one is just

a.(p, + p3) + d.(P2 + P4) - (1)

It may help the reader to refer to Figure 2, where I have written 1's subjective probabilities for 2 choosing C and D next to the branches representing 2's choice of those moves in the contingency concerned. The first probability sum in parentheses is just the probability that 2 will reply with C, while the second probability term is the probability that 2 will reply with D. 1 gets a payoff of a if the other player replies with C and d if the other player replies with D.

(ii) Suppose that 2 replies with C to l's choice of strategy C in the first round. Then 1 knows: that 2 belongs either to Class 1 or Class 3; that his best move in round two will be C; that 2 will reply with C and that he himself will obtain a payoff of a in the second round. The probability that 1 will get this payoff in the second round is just (P1 + p3), that is, the probability with which 2 replies C in the first round. Thus 1's expected payoff in the second round, associated with 2 replying with C to 1's choice of C in the first round, is

a.(p, + P3) - (2)

(iii) Suppose instead that 2 replies with D to l's choice of C in the first round. 1 knows that 2 belongs either to Class 2 or to Class 4, and that his own best move in the second round is D. However, he is not sure what his payoff in the second round will be: If 2 belongs to Class 2, he will reply with D in the second round and 1 will obtain c; while if 2 belongs to Class 4, he will reply with C in the second round and 1 will obtain b. Let us denote by p'2 and p'4 respectively, the probabilities 1 assigns to 2's belonging to Classes 2 and 4, knowing that 2 belongs neither to Class 1 nor to Class 3. Assume p'2 + p'4 = 1. Further, assume that P2/ P4 = P'2/ p4: 1 has no new evidence about the relative likelihood of 2 belonging to Classes 2 and 4. Thus it is easily shown that

P'2 = P2/(P2 + p4) and p4 = P4/(P2 + PX

5. We need to satisfy both

P2/P4 = P2/P4 and p'2 + p= 1 or p = 1 - p'2

Substituting for p' in the first expression

P2/(l - P'2) = P2/P4 or P'2 P4 = P2 - P'2 P2




1's expected payoff in the second round if 2 replies with D to his own choice of C in the first round is then

(P2 + p4)- (C-P'2 + b.p'4) = p2.c + p4.b (3)

Thus if 1 plays C in the first round and chooses rationally in the second round on the basis of the information so obtained, his total expected payoff is the sum of expressions 1, 2, and 3, or:

a +(p P) + d.(P4p2+p) + a(p P3)+ c-P2 + bp4

Now suppose that 1 plays D in the first round. Then

(i) l's expected payoff in the first round is

b.(p + P4) + C-(P2 + P3) (4)

(ii) Suppose that 2 replies with C to l's choice of D in the first round. Then 1 knows that 2 belongs either to Class 1 or Class 4. Let us denote the probabilities 1 places on 2 belonging to Class 1 and Class 4 knowing that 2 does not belong to either Class 2 or Class 3 by p ' and p " respectively. (pj'' + p " = 1.) Assume that p1'/ p " =

P1/P4, that is

Pi" P=/(PP + p4) and p4 = P4/(P1 + P4)

As I explained above, 1 cannot be certain how to choose in the second round in this contingency; his best strategy against a Class I player is to choose C, while his best strategy against a Class 4 player is D. Suppose he chooses the strategy maximizing his expected payoff in round two. Then his expected payoff is

(Pi + p4). max (a.p1" + d.p.4', b.p{' + b `)1

Clearly l's expected payoff in this contingency can be simplified to read

max (a.p1 + d.p4, b.(p1 + p4)) (5)

(iii) Now suppose instead that 2 replies with D to l's choice of D in the first round. Following similar reasoning to that in the last case, l's expected payoff in the second round is

max (d.p2 + a.p3, C-(P2 + P3)) (6)

which implies

P2 = P2/(P2 + P4)

6. Max (a, b) = a if a > b and b if b > a. We are assuming a and b are strictly ordered.




Thus, if 1 plays D in the first round and uses the new knowledge available to him in the second round, his total expected payoff over two rounds is (4) + (5) + (6), or

b.(p, + p4) + C.(P2 + P3)

+ max (a.p1 + d.p4, b.(p, + P4))

+ max (d.p2 + a.p3, c.(P2 + PO))

It may be worthwhile for I to choose C in the first round even though his expected payoff from C in the first round is lower than his expected payoff from D in the first round. The expected loss 1 would make in the first round from choosing C rather than D can be thought of as the price 1 pays for the extra information that enables him to obtain a larger expected payoff in the second round.

First notice that if

a.(p, + p3) + d.(P2 + P4) > b.(p, + P4) + C.(p2 + p3)

(that is, l's expected payoff in the first round from playing C is greater than his expected payoff in the first round from playing D), then it will not be rational for 1 to choose D in the first round so long as 2 can be assumed to have replied rationally and straightforwardly. This is obvious since, on those assumptions, if 1 chooses C in the first round he will be subjectively certain how best to choose in the second round; if he chooses D in the first round, he may still face uncertainty in the second round. This being so, I's expected payoff in the second round if he plays D in the first round can never be greater than (although it may be equal to) the payoff he would receive in the second round if he had chosen C in the first round.

Now suppose that

a.(p1 + p3) + d.(p2 + P4) < b.(p, + P4) + C-(p2 + P3).

Under what circumstances will it still pay 1 to choose C in round one, because of the increased payoff he would anticipate in round two? The conditions under which this is true are summarized in Figure 3.

A few comments on Figure 3 are in order. First, cell 1 is empty: It cannot be both the case that max(a.p4 + d.p4, b.(p1 + p4)) = a.pl + d.p4 and max(d.p2 + a.p3, C-(p2 + p)) = d.p2 + a.p3 if, as we are assuming, l's payoff




max (d.p2 + a.p3 c.(p2 + P3

d.p2 + a.p3 c. (p2 + p3)

i i

a. (p + 2. p3 +

a.p1 + d.p4 impossible d.p2

b.p1 +

max (a.p1 + d.p4 *(p1 + p/ *(p2 + 3

iii liv a.(2.p1 + P3)+ a.(2.p1 + 2.p3) +

d.p4 d. (p2 + p 4 b.(p1 +P4 > >

b. (2.p1 + p4)+ b. (2.p1 + p4) +

c. p3 c.(p2 + 2.p3)

Figure 3

in the first round from playing C in the first round is less than l's payoff in the first round from playing D in the first round.7 Second, it becomes more likely that 1's gains in the second round would outweigh his losses as a and d increase and as b and c fall (since a and d always appear on the left and b and c on the right hand side of these inequalities). Also, notice that it is more likely that gains in the second round would outweigh losses in the first round as Pi or p3 increase and as P2 or p4 fall; each of these inequalities can easily be rearranged so that Pi and p3 only appear on the left hand sides within positive terms, and P2 and p4 only appear on the right hand side within positive terms. In other words, the advantages of playing cooperatively in the first round increase as the prior reputation of the other player for replying in a cooperative manner increases.

As I show in Appendix A, it is easy to extend the analysis to iterated games of infinite length. Just as in the two-round supergame, it may pay Player 1 to choose C in the first round even though he would expect a lower payoff in that round from making this choice. In 1 's expectations, discounted future gains may outweigh short-term costs should he choose a cooperative first move. And, of course, other things being




equal the attractions of playing C in the first round will be greater in an infinite than in a two-round supergame: Expected future gains will be larger as there are more rounds in the infinite case.

THE SEQUENTIAL GAME WHERE PLAYER 2 MAY NOT BEHAVE STRAIGHTFORWARDLY

As I argued in the introduction, it may not pay 2 to reply straightforwardly. In this section two sorts of nonstraightforward play will be important. First, 2 might be willing to forego the short-run benefits of replying with D to l's choice of C in order to establish a long-term pattern of conditional cooperation. Second, 2 might be willing to incur the short-run costs of replying with C to l's choice of D in order to induce 1 to cooperate in future rounds. In the second instance, 2's aim might either be to establish a pattern of conditional cooperation or to exploit l's choice of C in some future round by playing D.

In the games considered in this section, Player 1 will still gather more information if he cooperates rather than defects in the first round. However, 1 's initial cooperative move is part of a strategy of conditional cooperation in which his future cooperation depends upon 2 replying cooperatively in past games. Just as in the straightforward case, there are risks associated with this sort of strategy, but the valuable information gained may more than compensate for the risks taken.

I consider an iterated game of infinite length in which Player 1 moves first in each round of the game. Players announce their strategy for the infinite game as a whole-their supergame strategy-at each round of the game, and their move in the round concerned is consistent with their supergame strategy. A player might wish to change his or her own supergame strategy in the next round in light of the other player's choice in the last round. When choosing whether to change, and which strategy to change to, each player assumes that the other will continue to adhere to his or her current announced strategy. Eventually the current supergame strategies of the players may be in equilibrium: Neither player would wish to change his or her own strategy given that the other player did not change strategy. In short, what is envisaged is a process of sequential adjustment of strategies terminating if and when equilibrium

7. It is easily shown that if both these conditions are satisfied simultaneously l's payoff in the first round from cooperating in the first round would be greater than his payoff in the first round from defecting in the first round.




is reached (Brams and Hessel, 1984). It is worth noting that not every equilibrium in the games considered is necessarily reachable through such a process of sequential adjustment.

Suppose that Player 1 assumes that 2's current reply is a rational response to his own current supergame strategy in the sense that no other reply open to 2 would yield 2 a higher payoff. Also suppose that 1 knows that 2 has either Assurance, Chicken, or PD preferences-the most important preference patterns in the context of public good games-but does not know which particular preference pattern 2 has.8 Then 1 can infer much useful information from 2's replies.

What 1 needs to discover is whether 2 is willing to play the iterated game cooperatively, by which I mean 2 is willing to choose a strategy that, in combination with 1 's strategy, results in each player cooperating after the first round in each single constituent game. If 2 is willing to play cooperatively, l's best strategy as a player with Assurance preferences is to play cooperatively himself. As we shall see, if 2 is not willing to play cooperatively, 1's best strategy is usually to defect in each constituent game. From 2's reply to certain conditionally cooperative supergame strategies, 1 can discover whether 2 is willing to play cooperatively or not. In order to take the argument further, it is necessary to examine the equilibria in various iterated sequential games where 1 has Assurance preferences and 2 either has PD, Assurance, or Chicken preferences. These equilibria are possible endpoints of the process of sequential adjustment discussed above. In the following analysis the payoffs from various outcomes in each single play of the game are those shown in Figure 1. A player has Assurance preferences if a > b > c > d; PD preferences if b > a > c > d; and Chicken preferences if b > a > d > c. At t = 0 the value to Player 1 of a payoff of z at time t = t* is z.s.t*, where 0 < s < 1 is a discount parameter. (Similar notational conventions are employed for player 2.)

The following notation will be used for the supergame strategies considered here:

Co (unconditional cooperation: cooperate in each round of the game irrespective of what the other player has done in past games)

D' (unconditional defection: defect in each round of the game irrespective of what the other player has done in past games)

8. As I noted above, it is implausible that 2 prefers a situation in which neither player cooperates to that where 1 cooperates alone. PD, Assurance, and Chicken are the preference structures that are not implausible in this way yet exhibit gains from collective action.




T ("trusting" tit-for-tat: 1's strategy of playing C in the first round of the game and in each subsequent round of the game so long as 2 replied with C in the last round. T can be thought of as a tit-for-tat strategy that trusts 2 to reply cooperatively in the first round)

T' ("distrustful" tit-for-tat: l's strategy of playing D in the first round of the game and C in each subsequent round of the game so long as 1 chose C in the last round. T' can be thought of as a strategy demanding a cooperative gesture from 2 before instituting conditional cooperation)

B (tit-for-tat reply: 2's strategy of replying with C if 1 has chosen C, and replying with D if 1 has chosen D, in the round concerned)

B' ("goodwill" tit-for-tat reply: 2's strategy of replying with C in the first round, and in each subsequent round replying with C if I has chosen C, and with D if I has chosen D, in the round concerned. B' can be thought of as offering a token of 2's goodwill and desire to cooperate conditionally.)

A (negative matching: 2's strategy of replying with C if 1 has chosen D, and with D if 1 has chosen C in the round concerned. If 1 has chosen a conditionally cooperative strategy, A can be thought of as a strategy that induces 1 to cooperate in the next round where 1 has defected in a given round, but leads 2 to exploit I's cooperation in the next round by replying with D in that round.)

Taking any pair of supergame strategies (one strategy for each player), it is easy to determine the moves players make in any individual constituent round of the iterated game, thus the players' payoffs in that round. Knowing the players' payoffs in each round, it is then possible to write out a sequence of discounted payoffs for each player. (For analo- gous methods applied to nonsequential iterated PD games, see Taylor, 1987: 60-78.) If the players' discount parameters satisfy the inequality assumed above, the sum of those discounted payoff sequences is conver- gent. Figure 4 presents the sum of the discounted payoff streams for each player associated with each strategy pair. Notice that, given l's supergame strategy, certain supergame strategy replies 2 might make may be strategically equivalent. By this I mean that, given l's strategy, 2's equivalent strategies imply the same move as each other in every round, and hence have the same summed payoff.

Figure 5 summarizes the necessary and sufficient conditions, given various possible combinations of preference structure, for outcomes to be equilibria in the infinite iterated games. So far I have considered only a subset of the infinite number of supergame strategies available to each player. The conditions in Figure 5 were, in the first instance, derived by comparing payoffs from various combinations of supergame strategy in Figure 4. Thus it might be thought that the conditions are only necessary, and not sufficient, for equilibria. However, I show in Appendix B



.S?0 I I I U

< Xn -ne.0 -- - v

S~~~~~C CD CD CD C

XX< I

411 41 'I o4I u Il

, i

CD + I + I I

292 -

I + I +~~ ~ ~~~~~~~ I En 0~~~~~~~~~~~~~~~~~~~~~- W ~ ~ ~ ~ ~ ~ 0-

>1 ~ ~ ~ ~ ~ ~ ~ - D - C al ~ ~ ~ ~ ~ ~ ~ ~ ~~DC 134 ~ ~ ~ ~ ~ -

U)~~~~~~~~~~~~~~~~~~~~~~U l

En El)~~C C

CD 7 CD DI 0

* +

.0 .0~~~~~~c

292




that these conditions are indeed sufficient, holding when all possible strategies for playing the infinite iterated game are considered. Thus in the analysis that follows I only make reference to the strategies listed above (and shown in Figure 4) when discussing equilibria and sequential adjustment to equilibria. The relevant conditions and the notation I employ for them in Figure 5 and in the text are as follows:

s >, I ..... - CI (read as Condition 1) b -c

c - d' s' > -, - -, ..... ..C2

a- d'

el - d,

s ..... RC2 (read as the "reverse" of Condition 2)

b'-a' ,

a'-d .. C3

ca- d b-c C4 c -c'

el - d,

s ' RC4

In Figure 5, subheadings refer, first, to Player 1's preference pattern and, second, to Player 2's preference pattern. So long as players have the indicated preferences, some outcomes in certain games are always/ never equilibria, whatever the particular values of the players' payoffs and discount parameters. Some outcomes are equilibria only if the stronger of two conditions are met. For instance, 2's discount parameter s' might have to be larger than

b-a' b -a b' -c' or a' -d'

whichever is greater. This is indicated by "Cl and C3," and a similar convention is used in other cases and in the text. To simplify the analysis without losing significant content, I will assume the (improbable) case in which s' is equal to any of the expressions on the right-hand side of the




5a: Assurance/Assurance

C00 B | B | A i

C00 always always always never never

T always always always never never

T never never never never RC2

D never never never never always

5b: Assurance/PD

C00 B Bo A D00

C0? never never never never never

T C1 and C3 C1 and C3 C1 and C3 never never

T never never never never RC2 and RC4

D never never never never always

5c: Assurance/Chicken

00 ~ ~ 000 - < ~ ~ ~ B | B A | D |

C never never never never never

T C3 C3 C3 never never

T never never never never never

D never never never always never

Figure 5

above conditions does not occur, and I will now treat the conditions above as if they were strict inequalities. What information would Player 1 obtain about 2 from the replies to the various strategies open to him? I will not assume that 2 knows for sure what 1 's preference structure is; as far as 2 is concerned I might have Assurance, PD, or Chicken prefer-




ences. If true, it is implausible that 2 would reply with C' if 1 chose C??, T or T': either B or B' (or both) would-yield 2 at least as high a payoff as C' and, at the same time, guard against the possible change to Do by 1 in the second round-a rational response if 1 had PD or Chicken preferences and believed 2 would adhere to C'. Thus, in what follows C' will not be considered as a possible reply to 1's choice of C', T or T'.

If 1 played C' and observed 2's reply, he would not always know for certain whether 2 was willing to play cooperatively. If 2 actually had Assurance preferences, she would reply with one of the equivalent strategies B or B', expecting to get her best payoff (a') in each round. The process of sequential adjustment would reach equilibrium (Figure 5a). However, if 2 had PD or Chicken preferences, her reply would be D' even though, if 1 had chosen a conditionally cooperative strategy like T or T', the conditions for 2 to reply with B or B' might be fulfilled. In short, C' only allows 1 to discover whether 2 has Assurance preferences.

If I played D' and observed 2's reply, he would not always know for certain whether 2 was willing to play cooperatively. If 2 actually had Chicken preferences, she would reply with C' (or, equivalently, A): Any other strategy 2 could play would involve defecting in at least one round, and she would obtain c', rather than a higher payoff of d', in that round. If 2 replied with A, equilibrium would be reached on the strategy pair (D0,9 A) (see Figure 5c).9 However, if 2 had Assurance or PD preferences she would reply with one of the equivalent strategies D' or B: Any other reply would involve 2 cooperating in at least one round, and she would obtain d', rather than a higher payoff of c', in that round. In summary, playing D' might tell 1 that 2 had Chicken preferences but it would not tell 1 whether 2 was a player who had Chicken preferences who was willing to proceed cooperatively. Similarly, if 2 replied with D', 1 would know 2 had Assurance or PD preferences but not whether, if she actually had PD preferences, she was willing to play cooperatively.'0

If I played T and observed 2's reply, he would know more about whether 2 was willing to play cooperatively than if he had, instead, chosen C', D' or T'. Player 2 would reply cooperatively with one of the

9. If-2 replied with C', the game would not reach an equilibrium. In the next round, 1 would choose T (or C'). If 1 chose C', 2 would reply with A or D', leading to 1 choosing D' in the third round. Clearly, if 2 again changed to C' a cycle might be set up, and players' strategies might cycle around the pattern indefinitely.

10. If 2's reply was B rather than D', this might signal 2's desire to cooperate conditionally in a way that D' would not. As my analysis of T' shows, conditional cooperation might emerge from such a reply. To save repetition, I will demonstrate this below.




equivalent strategies B or B' so long as (a) this reply yielded her a higher payoff than she would obtain from D'

a ,s (or ' > b' + - which is equivalent to Cl);

(b) this reply yielded her a higher payoff than she would obtain from A

a I

b'+ d's' (or ,-s' -s'2 which is equivalent to C3).

Notice that Cl and C3 are always fulfilled if 2 has Assurance preferences; in this case 2 would always reply cooperatively to T. " I Also notice that if 2 has Chicken preferences and C3 is fulfilled, Cl will also be fulfilled; in this case (b' - c') > (a' - d') so that Cl is a weaker condition upon s' than C3.12 Thus 2 will reply cooperatively to T so long as C3 is fulfilled.

If 2's reply to T was B or B' an equilibrium would be reached in the second round. B and B' are the same strategy starting from the second round since both of them ensure that 1 cooperates in that round. It is easily checked by considering I's payoffs in Figure 4 that if 2 chooses B, 1 cannot increase his own payoff by moving to C', T', or D??. But if 1 continues to play T starting with C in the second round, 2 would not wish to change her plan to play B from the second round onward: As the game is infinite, payoffs from various replies to T will have the same discounted payoffs viewed from the first and from the second round, and if 2's best response to T is B in the first round, her best response will continue to be B in the second round. The equilibrium reached in the second round will, then, be the strategy pair (T, B)-see Figures 5a, 5b, 5c. 13

Suppose that 2 has PD preferences and that she does not reply with B or B'. In this case, 2's reply would be D' rather than A if

11. Hence, Figure 5a does not make reference to Cl and C3 as conditions for cooperative equilibria if 1 chooses T.

12. Hence Figure 5c does not make reference to Cl as a condition for cooperative equilibria if 1 chooses T.

13. As the game is infinite, neither 1 nor 2 would want to change strategies in the third round and so on. To save repetition, I will not make this point in relation to other equilibria reached.




b' + -s' > 1-s'2 (which is equivalent to RC4).

If 2's first-round reply was, indeed, D' it is easily seen by comparing payoffs in Figure 4 that 1 would choose Do in the second round. Sequential adjustment would have reached the equilibrium Do, D')- see Figure 5b. If instead, 2's reply was A, 1 would, again choose Do in the second round (see Figure 4). 2's best reply to this would be Do, and equilibrium would be reached at (Do, Do) in the second round.

Notice that if 2 has Chicken preferences, RC4 cannot be true as d'> c'. 2's first-round reply to T will be A if it is not B or B'. Given 2 had chosen A, 1 would change from T to Do in the second round; 2 would stick with her choice of A; and equilibrium would be reached in the second round at (Do, A)-see Figure 5c.

If I played T' and observed 2's reply, he would have obtained less information about 2's willingness to play cooperatively than if he had chosen T. Player 2 would reply cooperatively with B' if this reply yielded her a higher payoff than Do or its equivalent B, so that C2 is fulfilled, or than A, so that C3 is fulfilled. If 2 has Assurance preferences, C3 is always fulfilled as a'> b' and 2 will reply cooperatively to T' so long as C2 is fulfilled. If 2 has Chicken preferences, C2 is always fulfilled as d'> c'; 2 will reply cooperatively to T' so long as C3 is fulfilled.

Suppose that 2 replied cooperatively with B' in the first round. In this case T' is the same as T and B' is the same as B after the first round is over. As I showed above, knowing that 2's strategy from the second round was B, 1 would have no incentive to change his plan to play T. Similarly, if 1 's strategy from the second round is T, there is no incentive for 2 to change her plans to play B. Suppose that there was a supergame strategy N better than B against T. Then the supergame strategy of playing C in the first round and N from the second round on would be better than B'. But this is a contradiction since we are assuming that B' is at least as good as any reply 2 can make to T'. Hence there can be no reply N better than B against I's second-round choice of T. In summary, if 2 replies to T' with B', equilibrium is reached in the second round on the strategy pair (T, B)-see Figures 5a, 5b, and 5c.

Notice that because the game is infinite, the conditions under which 2 would reply with B to T in the second round are just the same as the conditions for making this choice in the first round. I have shown that if 2's first-round reply to T' was B', her second-round reply to T would




be B. Hence if 2's first-round reply to T' was B', her first-round reply to T would have been B.

Suppose that 2 did not reply cooperatively with B'; then her first- round reply would be Do (or its equivalent B) rather than A if

c d' + b's' 1 > 1s2 (or RC4 is fulfilled).

Notice that RC4 cannot be fulfilled if 2 has Chicken preferences for, in this case, d'> c'. Thus, if 2 has Chicken preferences, her reply to T' will be A if it is not B'. Notice that RC4 must be fulfilled if 2 has Assurance preferences and does not reply cooperatively to T'. In this case, as C2 is not fulfilled, RC4 must be fulfilled, for (a' - d') > (b' - c'). Thus, if 2 has Assurance preferences and does not reply with B' to T', her reply will be Do or its equivalent B.

Suppose 2 had Chicken preferences and her first-round reply was A. 1's best second-round choice would be Do (see Figure 4). In the second round, 2 would not want to change her strategy of A, and equilibrium would be reached at (D'?, A)-Figure 5c.

Suppose 2 had PD preferences and her first-round reply was A. 1's best second-round choice would be Do. In reply, 2 would change her strategy to Do, and equilibrium would be reached at (Do, Do)-see Figure 5b.

Suppose 2 had PD or Assurance preferences and her first-round reply was Do. Given 2's choice of D in the first round, 1's planned second- round strategy would be to play tit-for-tat, starting with D in the second round. That is, 1's planned second-round strategy is T'. Player 1 would have no incentive to change his planned strategy if 2's first-round reply was Do. Similarly, as the game is infinite, if 2's best first-round reply to T' was Do, her best second-round reply to T' would be D'. Equilibrium would be reached in the second round at (T', Do)-see Figures 5a and 5b.

A theoretically interesting case is that in which 2 has PD or Assurance preferences and replies with B. Although B and Do have the same payoff on the assumption that 1 sticks to T', B carries a message that 2 may be willing to cooperate conditionally. If 2's first-round reply was B, 1 would change strategy to T in his second round. As we saw above, if 2 had Assurance preferences, her reply to 1's second-round move would always be cooperative, and equilibrium would be reached at (T, B) or (T, B') in the second round-see Figure 5a. However, if 2 had PD




preferences, she would reply cooperatively only if Cl and C3 were fulfilled. In this case, equilibrium would be either (T, B) or (T, B') in the second round. However, if Cl and C3 were not fulfilled, 2 would not reply cooperatively. Following the argument in the last section, third- round equilibrium would always be reached at (D', D') as, whatever 2's second-round reply, 1 would choose D' in the third round. Thus a rcply of B to T' might allow conditional cooperation to be achieved in the second round, or it might allow a player with PD preferences to exploit 1 in the second round by replying with D to l's choice of C.14

CONDITIONS FOR COOPERATIVE REPLIES TO T AND T'

It is less likely that 2 will reply cooperatively and cooperative equilibria reached, if 1 chooses T' rather than T in the first round. First, the conditions under which 2 would reply to T' with B' are more restrictive than the conditions under which she would reply to T with B or B'. Second, although cooperative equilibria might be reached if 2 replies with B to T' as I noted above, this will only occur if it is also the case that 2 would have replied with B or B' to l's first-round choice of T. I show that whenever 2 would reply with B' to T' in the first round, she would reply with B to T, but that it is not always true that if 2's reply to B was T her reply to B' would be T'.

Suppose, first, that 2 has Assurance preferences. As we saw above, 2's reply to T would always be B or B'. However, 2 would only reply with B' to T' if C2 were fulfilled, and C2 is not always fulfilled if 2 has Assurance preferences. Suppose, second, that 2 has Chicken preferences. C2 is always fulfilled and if C3 is fulfilled so is C 1. Hence, 2 will reply to T with B or B' and to T' with B' if C3 is fulfilled (i.e., the conditions for 2 to make a cooperative reply are exactly the same for T and T'). Suppose, third, that 2 has PD preference (i.e., b'> a'> c'> d'). It is helpful to rewrite Cl, C2, and C3 as

s'> (b' -a') (C1), s' > (c - d') (b-a )+(a' - c') (a'- c')+( c'-d (C2)

(b'-a) and s' >( c') (c' - d') (C3) respectively.

14. Because the assumption here is that players are myopic, assuming that the other player's strategy will not change in response to their own, these strategic advantages of B over D' as a reply to T' cannot form part of 2's calculations.



300 JO URNA L OF CONFLICT RESOLUTION

Suppose that Cl and C3 are true and C3 is the stronger condition upon s'. In this case, by examining the rewritten versions of Cl and C3 it is clear that (c' - d') < (b' - a'), so that the right hand side of C3 is greater than the right hand side of C 1. If C3 is fulfilled, it is sufficient for C2 to be fulfilled that

(b'-a') > (c'-d') or (b'-a')>(c'-d'). (a' - c') + (c' - d') (a' - c') + (c' - d')

But this is just the condition that C3 is stronger than Cl, and I have assumed this is true. Suppose, in contrast, that CI and C3 are true but C l is the stronger condition on s', so that (c' - d') > (b' - a'). The fact that C l is fulfilled need not imply that C2 is fulfilled if under the assumptions made, the following inequality can be satisfied:

(c'-d') (b-a')

(a' - c') + (c' - d (b' - a') + (a' - c')

Clearly this inequality can be satisfied since both the left-hand and right-hand expressions are greater than 0 and less than 1, while (c'- d') > (b' - a') implies that

(c'-d') (b-a')

(a' - c') + (c' - d') (b' - a') + (a' -c')

Thus, if Cl and C3 are fulfilled but Cl is the stronger of the two conditions on s', C2 need not hold true for certain possible values of b', a', c', d' and s'; so that 2 might not reply with B' to T' even though she would reply with B or B' to T. However, I showed in the last section that if 2 replied with B' to T' she would always reply to T with B or B'.

As we saw above, if 2 replies to T' with B, equilibrium may be reached in the second round at (T,B) or (T, B') if 2 has Assurance preferences or if 2 has PD preferences and Cl and C3 are fulfilled. These conditions are more restrictive than those for achieving similar equilibria by a first- round choice of T: They exclude the possibility of such equilibria where




2 has Chicken preferences and C3 is fulfilled. Moreover, since B and Do yield the same payoff for 2 when used in reply to T', there is nothing to suggest B would always be chosen in these circumstances. Thus, whenever a reply of B to T' resulted in a cooperative equilibrium being reached in the second round, a cooperative equilibrium would also have been reached in the first round had 1, instead, chosen T. But the converse is not true.

DISCUSSION OF THE NON-STRAIGHTFORWARD CASE

So far I have shown that "trusting" tit-for-tat provides more information about 2's desire to cooperate conditionally than does "distrustful" tit-for-tat. Whenever 2 replies to "distrustful" tit-for-tat in a way that leads to a cooperative equilibrium, she would also reply to "trusting" tit-for-tat in a way that leads to a cooperative equilibrium at least as quickly. However, it is not the case that whenever 2 replies to "trusting" tit-for-tat in a way that leads to a cooperative equilibrium she would reply to "distrustful" tit-for-tat in a way that eventually results in a cooperative equilibrium. These results can be generalized. As I show in Appendix C, for a class of conditionally cooperative strategies satisfying certain very general conditions, the "trusting" version of the strategy would provide 1 with more information about 2's desire to cooperate conditionally than would the "distrustful" version of the same strategy. As I showed above, neither unconditional cooperation nor unconditional defection always fully inform 1 about 2's desire to cooperate conditionally. Clearly, this conclusion also holds for other strategies that do not signal l's intention to cooperate conditionally.

Despite the informational advantages of testing the waters by using a "trusting" conditionally cooperative strategy, there are also associated risks: 1 takes the risk that his trust will be exploited in the first round by 2 replying with D in that round, 1 obtaining his worst single-round payoff of d. In contrast, the worst 1 could obtain in the first round from the choice of a "distrustful" conditionally cooperative strategy is c. The risks of initial cooperation as a way of testing the waters parallel those in the straightforward case. Due to the infinite number of supergame strategies available and the complex paths to equilibrium that are possible, it is difficult to model l's supergame strategy choice taking account of both informational gains and risks of short-term losses. However, it is worth noting that if "trusting" tit-for-tat is a better first-round risk than




"distrustturl tit-lor-tat, 1 wouId never expect to do worse overall by choosing the first version of this empirically important strategy.'5

As I showed above, 2 would always reply cooperatively to T if she replied cooperate, to T'. As I gets his highest payoff in every round in which both players cooperate, (to establish this result we only need to show that if 2 replies noncooperatively to both T and T') I would never do worse after the first round if his choice in the first round was T rather than T'. It is easily shown that this is true by drawing upon the results about equilibria discussed in previous sections.

We can ignore the case where 2 has Assurance preferences as 2 would always reply cooperatively to T. If 2 had Chicken preferences she would reply noncooperatively with A to both T and T' if C3 did not hold, equilibrium being reached at (Do, A) in the second round, and I's payoff being b per round whether his first-round choice was T or T'. If 2 had PD preferences, she might reply noncooperatively with Do to T and noncooperatively with B to T' if RC4 was fulfilled. In this case 1 would do worse in the second round by choosing T': He would be "tricked" into playing T in the second round, and 2 would "exploit" his choice of C in the second round by replying with D, so that l's second round payoff was d. In contrast, 1's second round payoff from a first-round choice of T would have been c. In subsequent rounds 1 would obtain c per round from an initial choice of T or T'. A second possibility is that RC4 is true and 2 replies to both T and T' with Do. Third, C4 might hold, in which case 2's first-round reply to T and T' would be A. In both the second and third cases, equilibrium is reached at (Do, Do) in the second round, and I's payoff is c per round whether his first-round choice was T or T'. In summary, if 2 replied noncooperatively to both T and T', there is no case in which I's payoff after the first round would be higher if T' had been chosen and one possible case in which it would be worse.

CONCLUSION

It is ironic that in order to gain reassurance it may be necessary to take risks. I have shown it is often the case that reassurance may be

15. The risk calculation for expected first round payoffs of T and T' has exactly the same form as that for C and D in the straightforward case. However, in this case the relevant probabilities are compounds of prior probabilities that 2 has Assurance, PD, and Chicken preferences and conditional prior probabilities that 2 would choose C or D given she had these preferences.




bought only at the cost of expected short-term losses. Even in the simple games considered here, to calculate whether taking risks to gain reassurance is worthwhile requires a good deal of information and "rational capacity" to process it. However, one conclusion of this article that applies both to the straightforward case and to "trusting" and "distrustful" versions of tit-for-tat in the nonstraightforward case is that a player with Assurance preferences will always be better advised to cooperate initially so long as this is a better risk in the first round of the game. Such first-round risks may be more easily assessed by players with limited "rational capacity." The models developed here help to explain why players often seem to take the risk of initial cooperation in order to gain reassurance. The variables isolated here-prior beliefs about others, payoffs from different outcomes, discount rates, and strategic options- and the ways in which they affect players' choices, suggest possible explanations of such risk taking. Furthermore, hypotheses that may be experimentally tested arise from the models.

Of course, not all supergames involve sequential play. Moreover, players often have available a continuum of strategies from less to more cooperative rather than a simple dichotomous choice. Finally, it is not always possible for players to communicate their choice to each other as I assumed in the non-straightforward case. Thus there is a great deal of theoretical work to do to establish the best strategies for gathering information outside the cases examined here. Preliminary work suggests that initial cooperation is desirable in some of these cases, but the extent to which this conclusion is generalizable has yet to be established (Ward, 1988).

The standard games-theoretic literature has neglected the problem of imperfect information and of ways of learning about others' preferences. Clearly, these are important problems. The approach applied here to a player with Assurance preferences (and to related players with PD preferences) should be extended to other important preference structures. In other games of imperfect information it may be advan- tageous to maintain a reputation for toughness (Ward, 1987). It is important to find out whether the desire to gather information conflicts with the need to maintain such a reputation or with other strategic considerations in games of uncertainty.




APPENDIX A

ITERATED GAMES OF INFINITE LENGTHS WITH STRAIGHTFORWARD PLAY BY PLAYER 2

Figure 2 may also be seen as representing a portion of the game tree for a related infinite supergame. As in the supergame with only two rounds, 1 is not sure of his best choice of strategy at node 3 or node 4 (associated with playing D in the first round). Suppose starting from node 3 that 1 chose D. Because 1 believes Player 2 to belong to either class 1 or class 4, he would expect 2 to reply with C to his own choice of D. Thus if 1 chose to play D at node 3 the game would reach node 10. Now at node 10 Player 1 is in no better position in terms of information about 2 than he was at node 3. Player 2 might still belong to class 1 (in which case it would be better for 1 to choose to play C in the third round) or to class 4 (in which case it would be better for 1 to choose to play D in the first round). However, if Player 1 chose D at node 3 he would also do so at node 10, and would continue to choose D indefinitely in branches of the game starting from node 10. Rational players would, we may assume, choose in the same way when facing the same choice in possession of the same information: In an infinite supergame Player 1 faces the same choice at node 10 as at node 3. Thus for consistency, 1 would choose D at node 10. By similar arguments, he would choose D at node 13 if he had done so at node 4.

Considering any of the nodes that 1 might see the game arriving at at time two (numbered 5 to 13 in Figure 2), 1 would know (a) how he would choose himself, (b) how 2 would reply-his payoff in round 3, and (c) that both he himself and Player 2 could be expected to repeat their choices in round three indefinitely. Thus, assuming that Player 1 discounts future payoffs, it is possible to derive expressions for the expected payoff he would obtain from choosing C at t = 0 and the expected payoff he would obtain from choosing D at t = 0. Let us assume that 1 discounts future payoffs by s where 0 < s < 1 so that at time t = 0 a payoff of p obtained at time t = t' is worth p.s.'* Then, by summing expected payoffs over the possible contingencies, l's expected payoff if he plays C in the first round and uses the information so gained to his best advantage is

a. (p I + P(3) + d1 (P2 + P4) + s) * (a.(pI + p3) + C.P2 + .p4)

l's expected payoff if he chose D in the first round is

b. (p + P4) + C(P2 + P3)

1 (2 + maX .~a.p, + s.d 'p4 + .~b'p4. -s) 4 (1 S)




(s .b. (p1 + P4))

+ max _ (1 - a p3 + s.d.p2 + (I - C p2.

s

.C. (P2 + P3))

APPENDIX B

SUFFICIENT CONDITIONS FOR SUPERGAME EQUILIBRIA

Suppose that Player 1 chooses a regular strategy defined either as Do or Do or a conditional strategy such that at any time, t, at which l's choice in that round is conditioned upon 2's replies in previous rounds, the conditions under which 1 would choose C, on the one hand, and D on the other hand, are the same for all t > 0. Notice that all Player l's strategies in Figure 4 are regular.

Suppose that l's regular strategy implies that 1 chooses C in the first game. Suppose that 2's best reply strategy implies that she, too, should choose C in the first round. Then at any time t* > 0 at which 1 chooses C, 2's best supergame will imply a choice of C at time t*. Because the game is infinite and l's strategy is regular, the extensive form of the game from the node reached at time t* will be exactly the same as that facing 2 after 1 has made his first-round move, while for any strategy that 2 could choose at time t*, its associated payoff discounted back to its value at time t* will be the same as its discounted payoff viewed from t = 0. Hence, if it is rational to play C at t = 0, it will be rational to play C at time t = t*. Similarly, if l's supergame strategy starts with C in the first game and 2's supergame response indicates a reply of D to 1's first move, 2 will always play D whenever 1 plays C. If l's supergame strategy starts with D in the first game and 2's best supergame response indicates a reply of C to 1's first move, 2 will always reply C whenever 1 plays D. If l's supergame strategy starts with D in the first game and 2's best supergame response indicates a reply of D to 1's first move, 2 will always reply with D whenever 1 plays D. Thus, whatever 2's best supergame response to 1's choice of a regular strategy, it can only exhibit four possible patterns:

reply C when I chooses C and C when I chooses 1) pattern 1 reply D when I chooses C and D when I chooses D ... pattern 2 reply C when I chooses C and D when I chooses D ... pattern 3 reply C when I chooses D and 1) when I chooses C ... pattern 4




Taking any of the four strategies 1 might play in Figure 4, each of these possible patterns is generated by at least one of the five replies considered for 2. For example, if 1 chooses T': pattern 1 is equivalent to Ca or B'; pattern 2 is equivalent to Do; pattern 3 is equivalent to B; and pattern 4 is equivalent to A. Hence, no matter which strategy 1 chooses from those considered in Figure 5, 2's best response must be equivalent to at least one of Ca, B, B', A, or Do. If it does not pay 2 unilaterally to change from one of these five strategies to any other of the five, it will not pay 2 unilaterally to change to any other strategy so that conditions pertaining to Player 2 in Figure 5 are sufficient for equilibrium.

Now consider Player 1. As 1 has Assurance preferences, if 2 chooses Ca or B, 1 can do no better than to choose Ca (or T), as these equivalent strategies yield 1 his highest payoff, a, in each round. If 2 chooses Do, I's best strategy must be Do (or T'): any other strategy would imply that 1 played C in at least one round, and 1 would obtain the lower payoff of d rather than c in that round, and could not obtain a higher payoff in any other round. If 2 chooses A, l's best strategy must be Do: Any other strategy would imply that 1 played C in at least one round, and 1 would obtain d rather than the higher payoff of b in at least that round, and because 1 would never obtain a in any round if 2 was playing A, 1 could never obtain a higher payoff in any other round. Finally, if 2 played B', 1 could never do better than to choose Ca (or T) in the second round: This choice would yield 1 his highest payoff, a, in each subsequent round. Thus, so long as 2's reply is either Ca, Do, A, B or B', 1's best strategy must be either C', Do, T or T'. Therefore, conditions pertaining to Player 1 in Figure 5 are sufficient for equilibria.

APPENDIX C

GENERALIZATION OF THE ARGUMENT ABOUT T AND T' TO OTHER TRUSTING AND

DISTRUSTFUL PAIRS OF CONDITIONALLY COOPERATIVE STRATEGIES

Let V be a "trusting" conditionally cooperative strategy such that (a) V starts with cooperation in the first round, (b) it is sufficient for cooperation in any subsequent round that 2 has always replied cooperatively in all past rounds, and (c) it is sufficient for defection in any subsequent round that 2 replied with defection in the last round. Let V' be a "distrustful" conditionally cooperative strategy exactly the same as V except that V' defects in the first round. Does I obtain more relevant information from V than from V', just as he obtains more relevant information from T than from T'?

Let W be any cooperative reply that 2 can make to V that implies that both I and 2 cooperate in every round of the game. Suppose that Cl is true so that W is




a better reply to V than Do; Do is at least as good as any reply to V that is nonequivalent to W; Cl is the strongest condition upon s' for W to beat any nonequivalent reply to V; 2 has PD or Chicken preferences. It may not be true under these circumstances that 2's best reply to V' is W'. For this to be true W' must be at least as good a reply to V' as Do, which will only be true if C2 is fulfilled. However, just as in the case where T and T' were examined in the text, this may not be true under the assumed conditions. Because the relevant payoffs and implied conditions are exactly the same as those considered when comparing T and T', the argument need not be repeated here.

In contrast, if 2's best reply to V' is W', then her best reply to V will be W. A strategy of playing C in the first round and W from round two onward is equivalent to W' if 1 chooses V'. Therefore it would be a contradiction if there was a better strategy, N, than W against V. In this case playing C in the first round and N in subsequent rounds would be better than playing C in the first round and W in subsequent rounds, but this cannot be true since W' is 2's best reply to V.

REFERENCES

ADOMEIT, H. (1982) Soviet Risk Taking and Crisis Behaviour. London: George Allen & Unwin.

ALLISON, G. T. (1971) Essence of Decision: Exploring the Cuban Missile Crisis. Boston, MA: Little, Brown.

AXELROD, R. (1984) The Evolution of Cooperation. New York: Basic Books. BACHARACH, S. and E. LAWLER (1981) Bargaining. San Francisco: Jossey-Bass. BRAMS, S. J. and M. P. HESSEL (1984) "Threat power in sequential games." Int.

Studies Q. 28: 23-44. CATUDAL, H. (1980) Kennedy and the Berlin Wall. Berlin: Berlin Verlag. DEAN, J. (1987) "Gorbachev's arms control moves." Bull. of the Atomic Scientists 43:

34-40. ELSTER, J. (1979) Ulysses and the Sirens: Studies in Rationality and Irrationality.

Cambridge: Cambridge Univ. Press. ETZIONI, A. (1967) "The Kennedy experiment." Western Pol. Q. 20: 361-380. GEORGE, A., D. K. HALL, and W. SIMONS (1971) The Limits of Coercive Diplomacy.

Boston, MA: Little, Brown. GEORGE, A. and R. SMOKE (1974) Deterrence in American Foreign Policy: Theory

and Practice. New York: Columbia Univ. Press. HARDIN, R. (1982) Collective Action. Baltimore: Resources for the Future. HARSANYI, J. C. (1967/8) "Games with imperfect information played by Bayesian

players: parts I, II, III." Management Sci. 14: 159-182, 320-334, 486-502. IKLE, F. C. (1964) How Nations Negotiate. New York: Praeger. JERVIS, R. (1976) Perception and Misperception in International Politics. Princeton:

Princeton Univ. Press.



308 JO URNA L OF CONFLICT RESOLUTION

KAHNEMAN, D., P. SLOVIC, and A. TVERSKY (1982) "Judgement under uncertainty: heuristics and biases," in K. Kahneman et al. (eds.) Judgement Under Uncertainty: Heuristics and Biases. Cambridge: Cambridge Univ. Press.

KREPS, D. M., J. D. ROBERTS, P. MILGROM, and R. WILSON (1982) "Rational cooperation in the finitely repeated Prisoner's Dilemma." J. of Economic Theory 27: 245-252.

KREPS, D. M. and R. B. WILSON (1982) "Reputation and imperfect information."J. of Economic Theory 27: 253-279.

LENG, R. J. (1984) "Reagan and the Russians: crisis bargaining beliefs and the historical record." Amer. J. of Pol. Sci. 78: 338-356.

LEWIS, D. K. (1969) Convention: A Philosophical Study. Cambridge: Cambridge Univ. Press.

LINDSKOLD, S. (1978) "Trust development, the GRIT proposal, and the effects of conciliatory acts on conflict and cooperation." Psych. Bull. 85: 772-793.

MILGROM, P. and J. D. ROBERTS (1982) "Predation, reputation and entry deterrence." J. of Economic Theory 27: 280-312.

ORDESHOOK, P. (1986) Games Theory and Political Theory. Cambridge: Cambridge Univ. Press.

OSGOOD, C. E. (1962) An Alternative to War or Surrender. Urbana: Univ. of Illinois Press.

OSGOOD, C. E. (1979) "GRIT for MBFR: a proposal for unfreezing force-level postures in Europe." Peace Research Reviews 8: 77-92.

PLOUS, S. (1985) "Perceptual illusions and military realities: the nuclear arms race." J. of Conflict Resolution 29: 363-389.

RHODES, R.A.W. (1986) The National World of Local Government. London: Allen & Unwin.

RUBIN, J. Z. and B. R. BROWN (1975) The Social Psychology of Negotiation. New York: Academic Press.

SEN, A. (1969) "A games-theoretic analysis of collectivism," in T. Majumdar (ed.) Growth and Choice. Oxford: Oxford Univ. Press.

SNYDER, G. and P. DIESING (1977) Conflict Among Nations. Princeton: Princeton Univ. Press.

STEIN, A. A. (1982) "Coordination and collaboration: regimes in an anarchic world." Int. Organisations 36: 299-324.

TANTER, R. (1974) Modelling and Managing International Conflicts: The Berlin Crises. Beverly Hills: Sage.

TAYLOR, M. (1976) Anarchy and Cooperation. New York: John Wiley. TAYLOR, M. (1987) The Possibility of Cooperation. Cambridge: Cambridge Univ.

Press. TAYLOR, M. and H. WARD (1982) "Chickens, whales and lumpy goods: alternative

models of public goods provision." Pol. Studies 30: 350-370. WALKER, S. G. (1982) "Bargaining over Berlin: a re-analysis of the first and second

Berlin crises." J. of Politics 44: 152-164. WARD, H. (1987) "The risks of a reputation for toughness: strategy in public goods

provision problems modelled by chicken supergames." British J. of Pol. Sci. 17: 23-52. WARD, H. (1988) "Information gathering in sequential and simultaneous Assurance

games." Essex Papers in Pol. Sci. 50. WILSON, R. (1985) "Reputations in games and markets," in A. Roth (ed.) Games

Theoretic Models of Bargaining. Cambridge: Cambridge Univ. Press.



Testing the Waters: Taking Risks to Gain Reassurance in Public Goods Games

Documents

Transcript of Testing the Waters: Taking Risks to Gain Reassurance in Public Goods Games