Arthur Berg Pennsylvania State University8 8.9 8.9 12 Angry Men (1957) 126,983 9 8.8 8.9 One Flew...

Standing Between a Bayesian and a Frequentist: An Emperical BayesExploration of Movies, Baseball, and Long Beach Basketball

Arthur BergPennsylvania State University

Introduction Bayes Estimation Empirical Bayes Basketball

Arthur Berg Standing Between a Bayesian and a Frequentist 2 / 28


Bayesian and Frequentist Representatives

Rev. Thomas Bayes FRS (1702-1761)English MathematicianPresbyterian Minister

P (H ∣E) = P (E∣H)P (H)P (E)

Sir Ronald Fisher FRS (1890-1962)English StatisticianEvolutionary Biologist, Geneticist

—Let the data speak for itself.—



Bayes Estimator as a Convex Combination

1st Goal: List the top 250 movies of all time.

Movies are rated on a scale of 1 to 10.

Some movies are rated by many people, and some by only a few.

Movies with fewer than 3000 votes are not considered.

All movies have an average rating of C = 6.9.

⋆ µi represents the mean rating by everyone who has seen movie i.⋆ The real goal is to construct the best estimate of µi, then pick the top 250.

The frequentist approach uses only X̄i, the average rating for movie i.

µ̂(Fisher)i = X̄i

The Bayesian approach shrinks X̄i towards C with more shrinkingapplied when the number of votes for movie i is small.

µ̂(Bayes)i = αiX̄i + (1 − αi)C where αi ∈ (0,1)



Internet Movie Database—Top 250Rank WR R Title Votes

1 9.2 9.2 The Shawshank Redemption (1994) 546,1552 9.1 9.2 The Godfather (1972) 427,9613 9.0 9.0 The Godfather: Part II (1974) 257,6434 8.9 9.0 The Good, the Bad and the Ugly (1966) 170,0455 8.9 9.0 Pulp Fiction (1994) 436,4566 8.9 8.9 Inception (2010) 265,5317 8.9 8.9 Schindler’s List (1993) 289,1708 8.9 8.9 12 Angry Men (1957) 126,9839 8.8 8.9 One Flew Over the Cuckoo’s Nest (1975) 225,419

10 8.8 8.9 The Dark Knight (2008) 487,800⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯85 8.5 8.7 Black Swan (2010) 20,326⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯

142 8.2 8.3 Avatar (2009) 285,005⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯⋯240 8.0 8.5 True Grit (2010) 6,444



IMDb Weighted Ranking—“a true Bayesian estimate”

WRi = viRi +mCvi +m = vi

vi +m´¹¹¹¹¹¹¸¹¹¹¹¹¹¹¶αi

Ri¯̄Xi

+ m

vi +m´¹¹¹¹¹¹¸¹¹¹¹¹¹¹¶1−αi

C

▸ Ri = average rating of the movie i (X̄i)

▸ vi = total number of votes from regular voters

▸ m = minimum # of votes to make the list = 3000

▸ C = grand mean across all movies in the database = 6.9



A Bayesian Calculation

Xi = (Xi,1, . . . ,Xi,vi) represents the vi ratings of movie i.

prior: µi ∼ N (µ0, σ20)

conditional: Xi,j ∣µi iid∼ N (µi, σ2) (j = 1, . . . , vi)µ̂(Bayes)

i = E[µi∣Xi]= ( vi

vi + σ2/σ20

) X̄i + ( σ2/σ20

vi + σ2/σ20

)µ0

= vivi +mRi + m

vi +mC ⇒ µ0 = C, m = σ2/σ20


1 ¿Does shrinking really help?

2 ¿How much to shrink by?





Prediction Error =



�

i

(µi − µ̂i)2

Prediction Error =�

i

(µi − µ̂i)2

Prediction Error = n�i=1(µi − µ̂i)2


Standing Between a Bayesian and a Frequentist

▸ In 1956, Charles Stein proved the existence of an estimator better thanthe sample mean under certain assumptions.

▸ In 1961, Willard James and Charles Stein explicitly constructed such anestimator.



The James-Stein Estimator (n ≥ 4)

µi ∼ N (µ0, σ20) Xi∣µi iid∼ N (µi, σ2) (i = 1, . . . n)

µ̂(Bayes)

i = E [µi∣Xi] = ( σ2

σ20 + σ2´¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¶α

)µ0 + ( σ20

σ20 + σ2´¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¶1−α

)Xi

µ̂(JS)

i = ( (n − 3)σ2

∑(Xi − X̄)2´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶α

)X̄ + (1 − (n − 3)σ2

∑(Xi − X̄)2´¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¸¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¹¶1−α

)Xi

In practice, if σ2 is unknown, an estimate is used.



Predicting Batting Averages

2nd Goal: Predict final batting averages from pre-season performances.

Pre-season batting averages for 18 major league players are provided.

Season final batting averages for the same players are also recorded.

Data is from the 1970 season and is published in JASA (1975) andScientific American (1977) by Efron and Morris.

The frequentist approach uses only Xi, the pre-season batting averagefor player i. p̂

(Fisher)i =Xi

The Emperical Bayes approach shrinks Xi towards X̄ by someempirically determined amount.

p̂(Stein)i = α̂Xi + (1 − α̂)X̄ where α̂ ∈ (0,1)



Name hits/AB pre-season (µ̂(ML)) season final (µ)

1 Clemente 18/45 0.400 0.3462 Robinson 17/45 0.378 0.2983 Howard 16/45 0.356 0.2764 Johnstone 15/45 0.333 0.2225 Berry 14/45 0.311 0.2736 Spencer 14/45 0.311 0.2707 Kessinger 13/45 0.289 0.2638 Alvarado 12/45 0.267 0.2109 Santo 11/45 0.244 0.269

10 Swoboda 11/45 0.244 0.23011 Unser 10/45 0.222 0.26412 Williams 10/45 0.222 0.25613 Scott 10/45 0.222 0.30314 Petrocelli 10/45 0.222 0.26415 Rodriguez 10/45 0.222 0.22616 Campaneris 9/45 0.200 0.28617 Munson 8/45 0.178 0.31618 Alvis 7/45 0.156 0.200



Batting Average Dataset

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

1977 Batting Averages Dataset (Efron)Ba

tting

Ave

rage

0.0

0.1

0.2

0.3

0.4

pre−seasonseason final



James-Stein Estimation of Batting Averages

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

1977 Batting Averages Dataset (Efron)

Batti

ng A

vera

ge

0.0

0.1

0.2

0.3

0.4


− − − − − − − − − − − − − − − − − −



Ranking Bias—Emperical Bayes + Order Statistics

▸ Genome-wide association studies

▸ SNPS: AA/Aa/aa or 0/1/2(∼ 107)

▸ Estimated effects of the top SNPsare biased up. (winner’s curse)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

1977 Batting Averages Dataset (Efron)

Batti

ng A

vera

ge

0.0

0.1

0.2

0.3

0.4


▸ ranking bias estimator—part frequentist, part Bayesianwith robust properties

▸ Applied to 2 GWAS studies with2,000 cases and 3,000 controls

Crohn’s DiseaseType 1 Diabetes



49ers Statistics—http://www.longbeachstate.com/



Opponents Over 3 Seasons — 08-09, 09-10, 10-11

opponent #

alaska anchorage 1arizona state 1

boise state 1byu cougars 1

byu hawaii 1cal poly 7

cal state fullerton 6cal state northridge 6

clemson 2cs monterey bay 1

duke 1green bay 2

idaho 1idaho state 1

iowa 1kentucky 1

loyola marymount 2montana 1

montana state 1new mexico state 1

north carolina 1notre dame 1

oregon 1pacific 8

pepperdine 2saint mary’s 1saint peter’s 1

san diego state 1san francisco state 1

syracuse 1temple 1

texas 1uc davis 6uc irvine 6

uc riverside 6uc santa barbara 7

ucla 1univ. san francisco 1

utah state 2washington 1weber state 2

west virginia 1wisconsin 1



Winning Percentages

All Games

All 3 Seasons (93) 56%

08-09 Season (30) 50%09-10 Season (33) 52%10-11 Season (30) 67%

Conference Games

All 3 Seasons 67%

08-09 Season 63%09-10 Season 50%10-11 Season 88%



Spread = 49ers Score − Opponent Score (10−11 Season)sp

read

05

1015

uc s

anta

bar

bara

cal s

tate

nor

thrid

ge

uc d

avis

cal p

oly

uc r

iver

side

cal s

tate

fulle

rton

paci

fic

uc ir

vine



Spread = 49ers Score − Opponent Score (10−11 Season)sp

read

05

1015

uc s

anta

bar

bara

cal s

tate

nor

thrid

ge

uc d

avis

cal p

oly

uc r

iver

side

cal s

tate

fulle

rton

paci

fic

uc ir

vine

− −− − −

−

− −



Over/Under = 49ers Score + Opponent Score (10−11 Season)O

ver/

Und

er (

Tota

l Sco

re)

120

140

160

uc ir

vine

cal s

tate

fulle

rton

cal s

tate

nor

thrid

ge

uc r

iver

side

uc d

avis

paci

fic

uc s

anta

bar

bara

cal p

oly



Over/Under = 49ers Score + Opponent Score (10−11 Season)O

ver/

Und

er (

Tota

l Sco

re)

120

140

160

uc ir

vine

cal s

tate

fulle

rton

cal s

tate

nor

thrid

ge

uc r

iver

side

uc d

avis

paci

fic

uc s

anta

bar

bara

cal p

oly

−− −

− − −−

−



Conversion Formulas

x = LB Score

y = Opponent Score

Over/Under = x + ySpread = x − y

x = Over/Under + Spread

2

y = Over/Under − Spread

2



Predictions

Rank Opponent LB Score O. Score Spread OverUnder

2 Cal Poly 66 55 11 1213 Cal State Northridge 81 66 15 1474 Pacific 69 68 1 1365 UC Santa Barbara 72 55 17 1266 Cal State Fullerton 79 71 7 1507 UC Riverside 75 66 9 1418 UC Irvine 82 80 2 161

UC Davis 76 64 13 140



How good are the predictions?

Using the 09-10 season to predict the 10-11 season:

adjusted prediction error for spread

unadjusted prediction error spread= 197

341= 58%

adjusted prediction error for over/under

unadjusted prediction error over/under= 513

818= 63%

Using the 08-09 season to predict the 09-10 season:

adjusted prediction error for spread

unadjusted prediction error spread= 150

194= 78%

adjusted prediction error for over/under

unadjusted prediction error over/under= 442

641= 69%



LB vs UCI—Vegas Odds (as of 3am on game day)

▸ All bets are “pay $110 to win $100”.

▸ Long Beach is the favorite; UCI is the underdog.

Casino Spread Over/UnderLV Hilton -10 148.5

Wynn -9.5 149MGM Mirage -10 NA

Predicted -2 161

These predictions recommend betting on UCI (still expecting LB to win) andbetting on “over” for the over/under option.



Disclaimers:1 I do not necessarily encouragesports betting.

2 I am not liable for any bets madebased my presentation.



Thank You!!

Beach.ArthurBerg.com

[email protected]


Arthur Berg Pennsylvania State University8 8.9 8.9 12 Angry Men (1957) 126,983 9 8.8 8.9 One Flew...

Documents

Transcript of Arthur Berg Pennsylvania State University8 8.9 8.9 12 Angry Men (1957) 126,983 9 8.8 8.9 One Flew...