Information Markets John Ledyard April 13, 2005 Nancy Schwartz Memorial Lecture KGSM, Northwestern...

Information Markets

John Ledyard

April 13, 2005

Nancy Schwartz Memorial Lecture

KGSM, Northwestern University

What would you like to know?

• What will Disney’s profits be in 2006?– Ask an accountant, a stock analyst, a consultant, the

CEO, Mickey Mouse, …

• Who will the next Pope be?– Ask a pundit, a cardinal, take a poll, ……

• How stable will the middle-east be on 12/31/05?– Ask the CIA, the Mossad, the Defense Department,

the President, a committee of experts, …….

• Should you set up an Information Market?

What is an Information Market?

• This is not about markets for information.• Kihlstrom (1974), Radner and Stiglitz (1984), Kamien and

Tauman (1990), Keppo, Moscarini, Smith (2005)

• It is about using market forces to bring together disparate bits and pieces of information and add them up, or aggregate them, for use in predictions or decisions.

• Google hits• Information Markets = 25,700,000

• Prediction Markets = 1,550,000

How would it work?

• Familiar Question: Who will win an election?

– Standard approach - Polls.

• In 1988, University of Iowa Business School securitized the Presidential election prediction on the internet.

The Iowa Election Market

In 1992, for $1.00, Iowa sold or bought a set of securities that covered all possible outcomes of the election; Bush, Clinton, Other (included Perot).

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.QuickTime™ and a

TIFF (Uncompressed) decompressorare needed to see this picture.


are needed to see this picture.



Each security paid $1 times the percentage of the vote for that person. Securities were traded.









After the election tally, if you owned 100 shares of Bush and Bush received 38% of the vote then you would be paid $38.









After the election tally, if you owned 100 shares of Bush and Bush received 38% of the vote then you would be paid $38.

The actual result was Clinton 43%, Bush 38%, Perot 19%






How the IEM might work.

Bush Clinton Other

Prices $.50 $.40 $.10

U think .3 .7 0

• You go to the IEM website and see

• You see a way to make some money.


Bush Clinton Other Cash E(V)

Prices $.50 $.40 $.10

U Think .3 .7 0

U buy 1 1 1 -$1 $1-$1 = 0



Prices $.50 $.40 $.10

U Think .3 .7 0

U buy 1 1 1 -$1 $1-$1 = 0

U trade -1 @ $.35 +1 @ $.50 -1 @ $.05 -$.10



Prices $.50 $.40 $.10

U Think .3 .7 0

U buy 1 1 1 -$1 $1-$1 = 0

U trade -1 @ $.35 +1 @ $.50 -1 @ $.05 -$.10

U have 0 2 0 -$1.10 1.40-$1.10

You actually will make (0.38x2) - 1.10 = -$0.34. But you don’t know that when you make this transaction. You can only act on your beliefs.



Prices $.50 $.40 $.10

U Think .3 .7 0

U buy 1 1 1 -$1 $1-$1 = 0

U trade -1 @ $.35 +1 @ $.50 -1 @ $.05 -$.10

U have 0 2 0 -$1.10 1.40-$1.10

Other traders then adjust their beliefs in response to the price changes. And so on.If all goes well, in equilibrium, prices will equate to the full-information beliefs of the traders.And if all goes well, these will be the true vote-shares.

1992 U.S. Election

Source: IEM (2005)

How Accurate Has IEM Been?

Brown'92 MI Primary

Tsongas'92 IL Primary

Tsongas'92 MI Primary

0

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 100

Actual Outcome (%)

Pre

dic

ted

Ou

tco

me

(%

)

US Presidential ElectionsAvg. Abs. Err. = 1.37%(5 Markets, 12 Contracts)Other US ElectionsAvg. Abs. Err. = 3.43%(14 Markets, 50 Contracts)Non-US ElectionsAvg. Abs. Err. = 2.12%(30 Markets, 175 Contracts)

Source: IEM (2005)

Also…..

• National election market in NY (1868-1940)– [Rhode and Strumpf (2004)]

– “Over $165 million (in 2002 dollars) was wagered in one election, and betting activity at times dominated transactions in the stock exchanges on Wall Street.”

– “In only one case did the candidate clearly favored in the betting a month before Election Day lose.”

Is this a Free Lunch?



•Iowa pays nothing.

•On average, the traders earn nothing

•But, in the end, everyone is better, maybe even maximally, informed.

“The Next Killer App”? or “Too Good To Be True”?

• Evidence, mostly empirical, suggests Information Markets Work.

• Evidence, mostly theoretical, suggests Information Markets Can’t Work.

• Today we will explore– how “Information Markets” work– how to design and engineer viable and accurate

“Information Mechanisms”

Why might an IM work?

• There are two of us in this scenario. – (Neither of us is a Game Theorist.)

• There are 2 coins:– Coin A comes up heads 80% of the time.– Coin B comes up heads 20% of the time.

• One is chosen with probability .5.– This is our common prior.

• The coin is flipped twice for each of us.– You see (H,T) and I don’t see that.– I see and you don’t see that.

Why might an IM work?• Remember: Coin A = .8 heads, Coin B = .2 heads,

you see (H,T), I see (?, ?).• What is the probability that the coin is A?• Based on only your information, the answer is 0.5.

– This is your initial posterior.

• Suppose there is a Market Maker who posts prices and asks us whether we want to buy or sell an asset that pays $1 if the coin is A and $0 if it is B.

• He posts a price of 0.60. • You offer to sell and I offer to buy.


• Remember: Coin A = .8 heads, Coin B = .2 heads, you see (H,T), I see (?, ?), I offered to buy at .6.

• What is the probability that the coin is A?

• Since you know that I must have seen (H,H), you know (H,T,H,H). This is everything.

• Your answer should be 0.94.• Of course, I only know that you are either (H,T) or (T,T), so I don’t know everything - yet.


• Suppose the Market Maker still posts a price of .6.• We both offer to buy.• I now know that your current posterior is .94 which

means you must have seen (H,T)• So we both now know that the total information is (3H,

1T) and our posteriors are the same: 0.94.• The “market” has “aggregated” the “information”!• The underlying theories are Rational Expectations

Equilibrium and Common Knowledge Information. • Green (1973), Lucas (1972), Grossman (1977)• Aumann (1976), Geanakopolos and Polemarchakis (1982)

But Wait!!!!




?=

Market Maker Maxwell’s Demon

There is something fishy here!

Why might an IM not work?• Let’s go back to the Market and get rid of the

Market Maker.• Remember: Coin A = .8 heads, Coin B = .2 heads,

you see (H,T), I see (?, ?), the asset pays $1 if A.• I offer to sell you 2 units of the asset for .30.• What should you do now?• Infer that I saw (T,T) and believe (1 H, 3T).• So you now should believe that P(A) = .06

Why might an IM not work?• I offer you 2 units of the asset for .30, you saw (H,T)

and know I saw (T,T), so your P(A) = .06.• You believe the expected value of the asset is .06.• Obviously you should reject my offer. • The full information is either (1H,3T) or (0H,4T).

– If you had seen (H,H) you would accept my offer.

• If you bid to buy above .004, I also know it is 0.6.

• We will not trade!• The underlying theory is No-Trade Theorems

• [Grossman and Stiglitz (1976), Milgrom-Stokey (1988)]

What About Empirical Evidence?

• There are many naturally occurring IM’s

• And, in direct comparisons,

they beat other institutions.

Pari-mutuel betting



Pari-mutuel betting

• Racetrack odds beat track experts– Figlewski (1979)



Futures markets



Futures markets

• OJ futures improve freeze forecasts – Roll (1984)



Stock markets



Stock markets

• Stock prices beat the experts panel in the post-Challenger probe – Maloney & Mulherin (2003)



Less Positive Field Evidence• The consensus forecast (median of about 30 economists)

has as much predictive power as the Goldman-Sachs pari-mutual market.

• [Wolfers and Zieztwitz (2003)]

• Wide bid-ask spreads and thin trading on most Tradesports.com markets.– Politics(4/11/05) (contract, price, spread, vol.)

• 2008DemnomClinton 40 1.5 9135 • 2008 RepnomJBush 10 .2 2685

– Papacy (4/11/05) (contract, price, spread, vol.)– Italy 42 .9 2045– Nigeria 13 1.7 1454– USA 0.2 .5 274

The Experimental Evidenceis Mixed

• A number of experimentalists have demonstrated convergence to the full information rational expectations equilibrium.– In laboratory asset markets with one asset

• Forsythe, Palfrey, Plott (1982)

– In laboratory elections• McKelvey & Ordeshook (1985)

– In laboratory asset markets with 3 assets• Plott and Sunder (1988)

The Experimental Evidenceis Mixed

• A number of experimentalists have demonstrated that information mechanisms do not always work.– In laboratory asset markets, if preferences differ and

there are incomplete markets, there is little aggregation. • Plott and Sunder (1988) Risk or ambiguity aversion?

– In iterative polls in the laboratory, there is incomplete information aggregation.

• McKelvey & Page (1990) Incomplete Bayesian updating?

– In laboratory pari-mutuel betting, we observe mirages.• Plott (2002) (Pari-mutuel) Information cascades?

One More Set of Data[Grus and Ledyard (2005)]

• The 2 coin example, 2 flips each

• N = 3, 7, 8, 12: Caltech subjects

• Market, Pari-mutuel

• 3 minutes of transactions per period

• 8 periods per mechanism

• 3 mechanisms per session (2.5 hours)

• Earnings approximate $33/subject

Markets and Pari-mutuels

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.00 0.10 0.20 0.30 0.40

KL distance

Pro

bab

ilit

y

PM3

M3

PM12

M12

These get it!

These are big mirages.

kl(.8,.65)=.06kl(.8,.50)=.22kl(.8,.20)=.83

MP informational size = .58 for N = 3 = .20 for N = 7 = .02 for N = 12

Numbers Matter!

Summary Statistics32 observations

Pari-Mutuel Market

Got it 41% 63%

“Got it” means KL < .01 or | p-FI | < .05

When N > 6, the market “gets it” 80% and the pari-mutuel gets it 75%.

Summary Statistics32 observations for PM, M, P

Pari-Mutuel Market

Got it 30% (75%) 55% (80%)

% Early 0% 38%

Early means in 1st 3 transactions. Not in time.

N= 3, 12


Pari-Mutuel Market

Got it 30% (75%) 55% (80%)

% Early 0% 38%

Mirages 25% 22%

Mirage means p is not the FI and is one of the other possibilities.


Pari-Mutuel Market

Got it 30% (75%) 55% (80%)

% Early 0% 38%

Mirages 25% 22%

tickets or transactions per subject

N = 3, rest

2.5, 7.8

N = 3, rest

1.6, 6.2

Based on the 2-coin experiments(and more complicated ones)

• Some aggregation occurs but it is not perfect.• Markets are faster than pari-mutuels and get it more

often.• Both are subject to mirages.• Neither perform well at low-scale (N = 3).• There is evidence to support the no-trade hypothesis,

particularly when N = 3.– But there is still some trading.

• Incomplete updating? Not here.• Boredom - yes, lots of little trades• Greater fool? - Possibly.

Summary to here• There is theoretical, experimental, and field

evidence that traditional IM’s may work.

• But there is also evidence that there are impediments to complete information aggregation especially in environments with informationally large agents.

• Should we worry about small numbers?

• Can we find better information mechanisms?

Numbers and Informationally Large Traders

• Probability of success in next year for drug A?• Expected sales of SUV model X in 2006

contingent on gas prices above $5 on July 2006?• The expected software shipping date contingent

on retaining feature R?• Expected benefits of a government program

conditional on one of several possible actions? (better cost-benefit analysis?)

•Example contract: Jordan is more politically stable in 4Q2005 conditional on US military activity down in Iraq in 3Q2005 and US$aid in 2Q2005 up in Iran.

Remember PAM?

8 nations, 5 indices,4 quarters-Political stability-Military activity-Economic growth-US $ aid-US military activityUp, Down, or ConstantImplies 320 active markets.

•320 questions and completeness implies 2^320 = 2 * 10^96 contracts.

Remember PAM?

8 nations, 5 indices,4 quarters-Political stability-Military activity-Economic growth-US $ aid-US military activityUp, Down, or ConstantImplies 320 active markets.

Need Better Information Mechanisms

• To be useful in many potential applications, Information Markets need to perform well with small numbers and informationally large traders.

• Traditional markets and pari-mutuels are not up to this.

• Two possible approaches– Subsidize the action in the traditional designs.

– Design new mechanisms.

Better Information Mechanisms?

• Let us first try to subsidize and modify the traditional IM’s.– Pari-mutuel: Add some tickets into the pot so

that the expected payoff of spending $1 is larger than $1.

– Market: Randomly accept bids and offers at “market.”

• Noise trading [Grossman and Stiglitz (1976)]

Pari-mutuel with and without subsidy

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.00 0.10 0.20 0.30 0.40

KL distances

Pro

bab

ilit

y PM3

PM/S3

PM12

PM/S12Helps when N = 3

kl(.8,.65)=.06kl(.8,.50)=.22kl(.8,.20)=.83

Markets with and without subsidy

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.00 0.10 0.20 0.30 0.40

KL distance

Pro

bab

ilit

y

M3

M/S3

M12

M/S12Helps when N = 3

kl(.8,.65)=.06kl(.8,.50)=.22kl(.8,.20)=.83

Does this mean “Noise Creates Information”?

Better but Still not GreatPM PM/S M M/S

Got it 41% 31% 63% 66%

% Early 0% 3% 47% 56%

Mirages 25% 13% 22% 13%

Tickets or

actions4.8 13.8 3.8 5.4

Design New Mechanisms

• With one agent, a Scoring Rule is a good mechanism to elicit beliefs.– Report r. Receive S*ln(r/2) in asset that pays $1 if A

and S*ln((1-r)/2) in asset that pays $1 if B.• Incentive compatible to report true beliefs

• Encourages participation– Provides an expected value greater than 0.

– Brier (1950) Monthly Weather Review, Goode (1952), Savage (1971) JASA, Page (1988)(PSR VCG)

• Let’s adapt this to multi-agent situations.

If it Works for One Person……

• Market Scoring Rules may be good mechanisms.– Hanson (2003)

– An automated market maker (MM)• Any trader can access the MM, at any time and trade assets

according to a scoring rule by announcing his belief. • The scoring rule is seeded with the last trader’s announcement of

their beliefs.• Each new announced belief is publicly reported.

– Incentive compatible to report true beliefs (somewhat)• At one iteration, but not over all iterations.• Possible to mislead early and gain later (particularly if N =3)

– Encourages participation (somewhat)• Positive expected value to the group.

An Old Standby Dressed Up• A Poll with Incentives may be a good mechanism.

– Basic design is standard. • All are polled and asked their beliefs.• Beliefs are averaged and publicly announced.• Repeated m times (We did 5 and 3.)

– But at the end, each agent is paid according to the same scoring rule.

– Incentive compatible to report true beliefs (somewhat)• True at “equilibrium” if there is full-information aggregation• Open question whether it is true during iterations

– Encourages participation • Positive expected value to everyone in the group.

MS, MSR, P, PMS (N = 3)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.00 0.10 0.20 0.30 0.40

KL distance

Pro

bab

ilit

y

PM/S3

MSR3

M/S3

Poll3

Pretty much the same.

ave KLPMS .042MSR .045P .047MS .051 kl(.8,.65)=.06

kl(.8,.50)=.22kl(.8,.20)=.83

MS, MSR, P, PMS (N = 7,8,12)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.1 0.2 0.3 0.4

KL distance

Pro

bab

ilit

y

PM/S

MSR

M/S

poll

P and MRS get it (always) >> MS and PMS

ave KL MirageMSR .000 0P .004 0PM .013 0PMS .061 0MS .085 2M .100 2

kl(.8,.65)=.06kl(.8,.50)=.22kl(.8,.20)=.83

Summary Statistics32 observations (all N)

P MSR MS PMS

Got it 53% 75% 66% 31%

% Early 22% 56% 47% 3%

Mirages 6% 6% 13% 13%

KL dist. .025 .023 .068 .051

• In MSR, first mover wins. In Pari-mutuel, last mover wins.• In time lapsed, MSR hits it really fast.

Reprise:Do traditional IM’s work?

• YES: Large numbers of informationally small agents and reasonably simple environments seem to overcome some of the no-trade incentives in traditional markets and pari-mutuels.– I don’t think we yet know exactly why. (Open Puzzle!)

• NO: With informationally large agents, much less incomplete markets or complex decision problems, it does not look so good.– N = 3 creates serious problems for both markets and pari-

mutuels, even in simple environments.– More research is needed here.

Reprise: Can we design IM’s that work?

• YES: For informationally small agents in simple environments,– The Market Scoring Rule gets it early and often.– Polling with incentive payments also works pretty well but

is slower to get the job done.– Markets are mirage prone, even with noise traders. – Pari-mutuels are slow, even with subsidization

• MAYBE: For informationally large agents, – The four best (ms, msr, p, pms) are all off by.10 to .15.– All mechanisms we looked at can be improved upon.– Or is there an impossibility theorem? (Open Puzzle!)

The End

• I leave you with two questions that I believe are worth pursuing.



TIFF (Uncompressed) decompressorare needed to see this picture. =

Market Maker Maxwell’s Demon


are needed to see this picture.Is the best mechanism?

MSR

Is ?

Information Markets John Ledyard April 13, 2005 Nancy Schwartz Memorial Lecture KGSM, Northwestern...

Documents

Transcript of Information Markets John Ledyard April 13, 2005 Nancy Schwartz Memorial Lecture KGSM, Northwestern...