Predicting basketball RPI

What is RPI?

• Ratings Percentage Index• Based on win/loss percentage throughout the

season.• Not necessarily a predictor of a stronger team.

How is RPI Calculated?

• Weighted wins, losses• Wins worth 1.4 away, 1 neutral, .6 at home• Losses worth 1.4 at home, 1 neutral, .6 away• Two parts: – Win pct (wins/(wins+losses) – Strength of schedule• Opponents unweighted win pct• Opponents’ opponents unweighted win pct

What is RPI used for?

• Estimator of team strength, as it factors in strength of schedule

• Helps to seed the NCAA tournament.• Helps selection committee/analysts determine

quality of wins.

The Selection Committee

• 10-person committee that determines who will receive an at-large bid and seeding for the tournament

• 5 year Tenure• Use multitude of selection tools– Win/loss– Conference strength– How a team won– Voodoo– Apparently, more random numbers than me.– Decidedly NOT just RPI.

How can RPI be predicted?

• Predict outcomes of games• Run through season• Rinse, repeat (Monte-Carlo!)

Kenpom statistics

• Statistics on all division 1 basketball teams– Offensive Efficiency– Defensive Efficiency– Tempo

• Average possesions per game: FGA-OR+TO+.42 FTA

• Meteorologist from Salt Lake City, Utah• Basketball stats just a hobby, no background• Stats referenced by ESPN, wall street journal• Season averages

Step 1: Predicting games

• Generate scores: compare and mark• Home games:– xscore<-round(x[8]/

100*rnorm(1,1.05,.2)*(((x[4]+y[4])/2)*rnorm(1,1.1,.2))+((y[12]*rnorm(1,1.05,.2))/80))

– yscore<-round(y[8]/100*rnorm(1,.95,.2)*(((y[4]+x[4])/2)*rnorm(1,.9,.2))+((x[12]*rnorm(1,.95,.2))/80))

Step 2: Recording data

• Compare scores• Higher score wins• Mark wins/losses in appropriate places

Step 3: Run through season

• 5284 division 1 vs division 1 games.• Import list of all games, which team is home,

away, to be called and put into game function.• Run 1 of three game situations based on court

(1 is home, 2 is away, 3 is neutral).• Each team plays approximately 30 games.

Step 4: Compile RPI, rank, repeat!

• After season is done, run through game list to grab opponent’s win-losses.

• Next, re-run through game list to grab opponent’s opponent’s win-losses by.

What do these results mean?

• Interesting estimator, but cannot be taken too seriously.

• True Top 25 RPI missed by average of 13 places.

• My top 25 missed true RPI by average of 11.• Kansas, WVU right where they should be!• Villanova ranked 18, therefore project should

be considered a success.

Is the RPI reliable?

• [252]Wake Forest(7.4%), [202]DePaul (9.9%) more likely to make tournament than nearly 200 other teams based on RPI alone.

• Too much weight placed on who you play, not how you play.

• Still only one factor in determining NCAA tournament.

Interesting Oddities

• Program took over 15 hours to run.• In 10,000 simulated seasons, 31 teams will not

receive an at-large bid (will not be in the top 37 RPI at the end of the season)

• Of those 31 teams, half of them would likely end up as a play-in team going to the final four.– VCU received at large bid with RPI rank of 49 (to fill

36th-38th at large bid), Harvard with RPI rank of 35 denied tournament bid

Are these results reliable?

• Maybe?• Only 2 teams predicted correctly in top 25.• Season averages inaccurate for day-of play, but

might average out over whole season.• Effect of random variables should eventually

absorb things like suspensions, injuries, team morale.

• Possible that one of the seasons actually matches this season perfectly.

Theoretical/Technical Issues

• Unable to account for mid-season tournaments, changes in schedule, delayed games.

• ‘Labor Intensive’ program – 5 trillion calculations.

• Still near-impossible to seed mock tournament without just taking 68 highest RPI (which might not be a bad idea)

March Madness

• There are about 14,757,395,260,000,000,000 different brackets of the NCAA tournament (but only 1 winner!)

• Over 6 million brackets were submitted to ESPN.com this march in competition.

• Of those brackets, the best bracket, just 1 of 6 million, got 52, or 77.6% of their picks correct.

• This year was the first time 2 11 seeds made the sweet 16, and the first time no 1 or 2 seeds made the final 4.

• Most even field the tournament has ever had, no great teams

My terrible bracket

• My original bracket:– 33.6th percentile on ESPN at 480 pts (4-millionth

place) Champion: Notre Dame– No final four team, only 2 elite 8 (UNC, Uconn)

Basically, terrible. Last place in every pool I was in.

Can I make my bracket any better?

• This year, no, but next year? Maybe!• Goal: create a bracket based on Kenpom

rankings, and see if it does any better.

Results:

• Worth twice as many points on ESPN, enough to put me in the 92nd percentile!– Beat 5,520,000 brackets!

• Still had no final 4 team.

Is it a reliable method? Votes for no:

• Highest championship percentage was less than 6%, only 4% better than flipping a coin.

• Many games were decided by less than a percentage point• Doesn’t take into account injuries, coaching, stage fright,

‘home field,’ streaks, incredible ability to lose the lead, or recruiting violations. Examples:– Georgetown and St. Johns both had their star players hurt going

into the tournament and lost in the first round.– Tennessee head coach Bruce Pearl was hit with school and NCAA

sanctions the day before the game and lost by 30 points.– George Mason entered the tournament on an 11 game wn-streak

Votes for Yes:

• Randomness exists to account for the issues previously mentioned.

• Anything can happen, this season could have been that 1 in 10,000 chance for VCU, data could be reliable.

• 55.2% of the bracket picked correctly, up from… well, zero-ish.

Oddities and anecdotes• First trial of the tournament I ran (before looping) yielded

Butler over Uconn, with Kentucky and Kansas in the final 4.

• Defeated teams sometimes more likely to advance: Notre Dame has a higher chance of making the championship game over Wisconsin, 8.47% over 8.38%, but Wisconsin is more likely to win the championship, 4.90% over 4.76%

• VCU had only a 51.5% chance of winning it’s first game, a 22% chance of advancing past Georgetown, and a 2.64% chance of advancing to the final 4.

Conclusions: RPI

• Can RPI (remember RPI?) be predicted for a season using Monte-Carlo methods?– Decent yardstick, but not perfect– Since RPI is just a yardstick anyway, should work

okay.• Can it be used to seed a tournament?– Difficult but yes, would need to run through 31

conference tournaments and determine an Ivy league AQ as well.

Conclusions: NCAA tournament

• Can Monte-Carlo methods be used to predict the NCAA tournament?– Better predictor than me and better than a coin

flip.– Good for calculating odds but not for absolute

winner.