Testing Transitivity with a True and Error Model

Michael H. BirnbaumCalifornia State University,

Fullerton

Testing Algebraic Models with Error-Filled Data

• Models assume or imply formal properties such as transitivity:

If A > B and B > C then A > C• But such properties may not hold if

data contain “error.”• And different people might have

different “true” preferences.

Error Model Assumptions

• Each choice in an experiment has a true choice probability, p, and an error rate, e.

• The error rate is estimated from (and is the “reason” given for) inconsistency of response to the same choice by same person over repetitions

One Choice, Two Repetitions

A B

A

B€

pe2

+ ( 1 − p )( 1 − e )2

p ( 1 − e ) e + ( 1 − p )( 1 − e ) e

p ( 1 − e ) e + ( 1 − p )( 1 − e ) e

€

p ( 1 − e )2

+ ( 1 − p ) e2

Solution for e

• The proportion of preference reversals between repetitions allows an estimate of e.

• Both off-diagonal entries should be equal, and are equal to:

( 1 − e ) e

Ex: Stochastic Dominance

: 05 tickets to win $12

05 tickets to win $14

90 tickets to win $96

B: 10 tickets to win $12

05 tickets to win $90

85 tickets to win $96

122 Undergrads: 59% repeated viols (BB) 28% Preference Reversals (AB or BA) Estimates: e = 0.19; p = 0.85170 Experts: 35% show 2 violations (BB) 31% Reversals (AB or BA) Estimates: e = 0.196; p = 0.50

Testing Higher Properties• Extending this model to properties

relating 2, 3, or 4 choices:• Allow a different error rate on

each choice.• Estimate true probability for each

choice pattern. Different people can have different “true” patterns, which need not be transitive.

New Studies of Transitivity

• Work currently under way testing transitivity under same conditions as used in tests of other decision properties.

• Participants view choices via the WWW, click button beside the gamble they would prefer to play.

Some Recipes being Tested

• Tversky’s (1969) 5 gambles.• LS: Preds of Priority Heuristic• Starmer’s recipe• Additive Difference Model (regret;

majority rule)• Birnbaum, Patton, & Lott (1999) recipe.• Recipes based on Bleichrodt & Schmidt

context-dependent utility models.

Replications of Tversky (1969) with Roman

Gutierez• First two studies used Tversky’s 5

gambles, but formatted with tickets instead of pie charts.

• Two studies with n = 417 and n = 327 with small or large prizes ($4.50 or $450)

• No pre-selection of participants.• Participants served in other risky DM

studies, prior to testing (~1 hr).

Three of Tversky’s (1969) Gambles

• A = ($5.00, 0.29; $0, 0.79)• C = ($4.50, 0.38; $0, 0.62)• E = ($4.00, 0.46; $0, 0.54)Priority Heurisitc Predicts:A preferred to C; C preferred to E, and E preferred to A. Intransitive.

Tversky (1969) reported viol of WST

Response Combinations

Notation (A, B) (B, C) (C, A)

000 A B C *

001 A B A

010 A C C

011 A C A

100 B B C

101 B B A

110 B C C

111 B C A *

Weak Stochastic Transitivity

€

P ( A f B ) = P ( 000 ) + P ( 001 ) + P ( 010 ) + P ( 011 )

P ( B f C ) = P ( 000 ) + P ( 001 ) + P ( 100 ) + P ( 101 )

P ( C f A ) = P ( 000 ) + P ( 010 ) + P ( 100 ) + P ( 110 )

WST Can be Violated even when Everyone is Perfectly

Transitive

€

P ( 001 ) = P ( 010 ) = P ( 100 ) =1

3

€

P ( A f B ) = P ( B f C ) = P ( C f A ) =2

3

Triangle Inequality has similar problems:

• It is possible that everyone is transitive but WST is violated.

• It is possible that people are systematically intransitive and WST is satisfied.

• Possible that everyone is intransitive and triangle inequality is satisfied.

Model for Transitivity

€

P ( 000 ) = p000

( 1 − e1

)( 1 − e2

)( 1 − e3

) + p001

( 1 − e1

)( 1 − e2

) e3

+

+ p010

( 1 − e1

) e2

( 1 − e3

) + p011

( 1 − e1

) e2e

3+

+ p100

e1

( 1 − e2

)( 1 − e3

) + p101

e1

( 1 − e2

) e3

+

+ p110

e1e

2( 1 − e

3) + p

111e

1e

2e

3

A similar expression is written for the other seven probabilities. These can in turn be expanded to predict the probabilities of showing each pattern repeatedly; i.e., up to six errors.

Expand and Simplify• There are 8 X 8 data patterns in an

experiment with 2 repetitions.• However, most of these have very small

frequencies.• Examine probabilities of each of 8

repeated patterns.• Frequencies of showing each of 8

patterns in one replicate OR the other, but NOT both. Mutually exclusive, exhaustive partition.

Tests of WSTPercentage Choosing Column >pr Row Gamble

Row A B C D E

A 73 77 80 85

B 30 68 79 79

C 16 29 74 78

D 11 16 24 63

E 13 17 15 33

Patterns for A, C, EPattern Rep. 1 Rep 2 Both

000 14 28 5

001 18 25 15

010 23 38 1

011 12 5 3

100 24 33 7

101 5 6 1

110 301 256 220

111 19 25 5

Sum 416 416 257

Pattern Both Rep 1 or 2Not both

Est p Pred Both

Pred 1 or 2 Not both

000 5 16 .03 8.1 8.6

001 15 6.5 .07 15.3 6.5

010 1 29.5 .00 4.7 37.2

011 3 5.5 .01 2.8 5.9

100 7 21.5 .03 7.8 26.0

101 1 4.5 .00 0.9 5.5

110 220 58.5 .85 196.6 67.6

111 5 17 .02 4.6 17.9

Sum 257 159 1 240.9 175.1

Comments• Results are surprisingly transitive, unlike

Tversky’s data (est. 95% transitive).• Of those 115 who were perfectly reliable,

93 perfectly consistent with EV (p), 8 with opposite ($), and only 1 intransitive.

• Differences: no pre-test, selection;• Probability represented by # of tickets

(100 per urn), rather than by pies.• Participants have practice with variety of

gambles, & choices;• Tested via Computer.

Pie Chart Format

Pies: with or without Numerical probabilities

• 321 participants randomly assigned to same study; except probabilities displayed as pies (spinner), either with numerical probabilities displayed or without.

• Of 105 who were perfectly reliable, 84 were perfectly consistent with EV (prob), 13 with the opposite order ($); 1 consistent with LS.

Lower Standards

• Look at AB,BC,CD,DE choices and EA choices only:

• 10 of 321 participants showed this pattern; all in the pies-only condition. Although the rate is low (6% of 160), association with condition is clear!

• By still lower standard used by Tversky: 75% agreement with above pattern: 37 people, 27 in pies-only condition.

Tests of Lexicographic Semi-order and Additive Difference• LS implies no integration of

contrasts (additive difference model allow integration)

• Both LS and additive difference models imply no interactions between probability and consequences.

Test of Interaction

R S Pies & p

Pies & No p

($7.25, .95; $1.25, .05)

($4.25, .95; $3.25, .05)

16 22

($7.25, .05; $1.25, .95)

($4.25, .05; $3.25, .95)

84 77

Among the 37 Leniently classified as Intransitive

• Are those 37 who are 75% consistent with the LS in the 5 choices also approx. consistent with LS in tests of Interaction?

• No. 26 of these have all four choices in the pattern of interaction predicted by TAX and other utility models.

Summary• Priority Heuristic model’s predicted

violations of transitivity are rare and rarely repeated when probability and prize information presented numerically.

• Violations of transitivity are still rare but more frequent when prob information presented only graphically.

• Evidence of Dimension Interaction violates PH and additive Difference models.

Conclusions

• Violations of transitivity are probably not due to intransitive strategy (LS or additive difference model), but rather to a configural assimilation of the probability values, which are then used in a numerical utility model.

• We are still unable to produce the higher rates of intransitivity reported by Tversky and others.

Transitivity Test: ADLTrial Choice % Response Pattern

RepsEst. parameters

00 01 10 11 p e

8 50 to win $10050 to win $20

50 to win $6050 to win $27

18 190 23 25 23 0.10

0.10

3 50 to win $6050 to win $27

50 to win $4550 to win $34

29 140 44 39 37 0.17

0.20

21 50 to win $4550 to win $34

50 to win $10050 to win $20

74 35 33 20 172

0.85

0.12

Results-ADLpattern Rep 1 Rep 2 Both

000 LPH 21 13 1

001 TAX 134 147 106

010 20 18 8

011 38 37 10

100 15 9 0

101 14 10 0

110 12 15 7

111 6 11 1

Sum 260 260 133