Sports Ranking and Scheduling with Kalman Filtersbertozzi/WORKFORCE/REU 2013/Sports... ·...
Transcript of Sports Ranking and Scheduling with Kalman Filtersbertozzi/WORKFORCE/REU 2013/Sports... ·...
-
Sports Ranking andScheduling with Kalman
Filters
Peter Elliott, Matt Owen,Kyle Shan, Yingjie Yu
-
Introduction
-
Introduction
-
Introduction
Definition (Rating)
A rating is a numerical evaluation of an item.
In sports rating systems, higher rated teams perform better.
Definition (Ranking)
A ranking is an ordering of items.
It is easy to turn ratings into rankings
-
Introduction
Definition (Rating)
A rating is a numerical evaluation of an item.
In sports rating systems, higher rated teams perform better.
Definition (Ranking)
A ranking is an ordering of items.
It is easy to turn ratings into rankings
-
Introduction
Definition (Rating)
A rating is a numerical evaluation of an item.
In sports rating systems, higher rated teams perform better.
Definition (Ranking)
A ranking is an ordering of items.
It is easy to turn ratings into rankings
-
Introduction
Definition (Game)
A game between two teams is a pairwise comparison which yields areal number outcome
Definition (Schedule)
A schedule is a matching of teams to play against each other ineach round.
Goal: schedule games to find the most “informative” ranking.
-
Introduction
Definition (Game)
A game between two teams is a pairwise comparison which yields areal number outcome
Definition (Schedule)
A schedule is a matching of teams to play against each other ineach round.
Goal: schedule games to find the most “informative” ranking.
-
Introduction
Definition (Game)
A game between two teams is a pairwise comparison which yields areal number outcome
Definition (Schedule)
A schedule is a matching of teams to play against each other ineach round.
Goal: schedule games to find the most “informative” ranking.
-
Extended Kalman Filter RankingUnderlying Model
Assume that games follow the Bradley-Terry paired comparisonmodel:
I Teams 1, . . . , n have skill levels x1, . . . , xnI The expected outcome in a game between teams i and j is
h(xi , xj) =1
1 + 10−(xi−xj )/ξ
Ratings can be computed using maximum likelihood, but this iscomputationally difficult.
-
Extended Kalman Filter RankingApplying a Filter to Bradley-Terry
Start with the system
xk = xk−1 = x∗
yk = hk(xk) + �k
where {�k} is an uncorrelated random process, and the predictedoutcomes hk(xk) of the games in each round are given by theBradley-Terry prediction.
Then, we get the simple update equations:
Pk = (P†k−1 + σ
−2HTk Hk)†
x̂k = x̂k−1 − σ−2PkHTk (hk(x̂k−1)− yk)
where Pk is the covariance of x̂k and Hk = Dhk(xk−1).
-
Extended Kalman Filter RankingApplying a Filter to Bradley-Terry
Start with the system
xk = xk−1 = x∗
yk = hk(xk) + �k
where {�k} is an uncorrelated random process, and the predictedoutcomes hk(xk) of the games in each round are given by theBradley-Terry prediction.Then, we get the simple update equations:
Pk = (P†k−1 + σ
−2HTk Hk)†
x̂k = x̂k−1 − σ−2PkHTk (hk(x̂k−1)− yk)
where Pk is the covariance of x̂k and Hk = Dhk(xk−1).
-
Extended Kalman Filter RankingSimilarity to Elo
Elo’s method seems a lot like the EKF update:
Elo: xi ← xi − K (h(xi , xj)− yk)EKF: x̂ ← x̂ − σ−2PkHTk (hk(x̂k−1)− yk)
where K is a constant.
EKF ranking has several advantages over Elo.
-
Extended Kalman Filter RankingSimilarity to Elo
Elo’s method seems a lot like the EKF update:
Elo: xi ← xi − K (h(xi , xj)− yk)EKF: x̂ ← x̂ − σ−2PkHTk (hk(x̂k−1)− yk)
where K is a constant.
EKF ranking has several advantages over Elo.
-
SchedulingIntroduction
Definition (Fisher Information)
The Fisher Information matrix is the inverse of the covariancematrix of the state estimate.
I = P†
Schedule Design Goal: Choose matchups that maximize the FisherInformation.
Notions of Fisher optimality:
I T-optimality: maximize the trace of the Fisher Informationmatrix
I Any monotone function of the eigenvalues
-
SchedulingIntroduction
Definition (Fisher Information)
The Fisher Information matrix is the inverse of the covariancematrix of the state estimate.
I = P†
Schedule Design Goal: Choose matchups that maximize the FisherInformation.
Notions of Fisher optimality:
I T-optimality: maximize the trace of the Fisher Informationmatrix
I Any monotone function of the eigenvalues
-
SchedulingIntroduction
Complete schedules for tournaments are often created before atournament begins. This is known as static scheduling.
Dynamic scheduling refers to creating a schedule as thetournament occurs, taking into account the results as they happen.
Goal: dynamically schedule one game for each team, for eachround of play, in a way that maximizes information of the rating.
-
SchedulingUsing Kalman Filters
The Kalman Filter update equations give a strategy for dynamicscheduling.
Pk = (P†k−1 + σ
−2HTk Hk)†
In terms of Fisher Information:
Ik = Ik−1 + σ−2HTk Hk
Goal: Choose games at each time step that maximize Ik
Important features:
I Maximizing tr(Ik) is equivalent to maximizing tr(HTk Hk)I If team i plays a single game, the i th entry on the diagonal of
HTk Hk is a function of the previous ratings of team i and itsopponent
-
SchedulingUsing Kalman Filters
The Kalman Filter update equations give a strategy for dynamicscheduling.
Pk = (P†k−1 + σ
−2HTk Hk)†
In terms of Fisher Information:
Ik = Ik−1 + σ−2HTk Hk
Goal: Choose games at each time step that maximize Ik
Important features:
I Maximizing tr(Ik) is equivalent to maximizing tr(HTk Hk)I If team i plays a single game, the i th entry on the diagonal of
HTk Hk is a function of the previous ratings of team i and itsopponent
-
SchedulingA New Dynamic Scheduling Method
Our proposed method:
I For each pair of teams i and j ,calculate the potentialinformation gain
I Create a complete weightedgraph G , with edge weightbetween i and j correspondingto the potential information gain
I Choose the maximum weightedperfect matching on G
I The weight of the matching isexactly tr(HTk Hk)
Edmonds’ Blossom Algorithm (1965) solves the perfect matchingproblem in polynomial time.
This method is trace-optimal for EKF ranking.
-
SchedulingA New Dynamic Scheduling Method
Our proposed method:
I For each pair of teams i and j ,calculate the potentialinformation gain
I Create a complete weightedgraph G , with edge weightbetween i and j correspondingto the potential information gain
I Choose the maximum weightedperfect matching on G
I The weight of the matching isexactly tr(HTk Hk)
Edmonds’ Blossom Algorithm (1965) solves the perfect matchingproblem in polynomial time.
This method is trace-optimal for EKF ranking.
-
SchedulingA New Dynamic Scheduling Method
Our proposed method:
I For each pair of teams i and j ,calculate the potentialinformation gain
I Create a complete weightedgraph G , with edge weightbetween i and j correspondingto the potential information gain
I Choose the maximum weightedperfect matching on G
I The weight of the matching isexactly tr(HTk Hk)
Edmonds’ Blossom Algorithm (1965) solves the perfect matchingproblem in polynomial time.
This method is trace-optimal for EKF ranking.
-
SchedulingA New Dynamic Scheduling Method
Our proposed method:
I For each pair of teams i and j ,calculate the potentialinformation gain
I Create a complete weightedgraph G , with edge weightbetween i and j correspondingto the potential information gain
I Choose the maximum weightedperfect matching on G
I The weight of the matching isexactly tr(HTk Hk)
Edmonds’ Blossom Algorithm (1965) solves the perfect matchingproblem in polynomial time.
This method is trace-optimal for EKF ranking.
-
SchedulingA New Dynamic Scheduling Method
Our proposed method:
I For each pair of teams i and j ,calculate the potentialinformation gain
I Create a complete weightedgraph G , with edge weightbetween i and j correspondingto the potential information gain
I Choose the maximum weightedperfect matching on G
I The weight of the matching isexactly tr(HTk Hk)
Edmonds’ Blossom Algorithm (1965) solves the perfect matchingproblem in polynomial time.
This method is trace-optimal for EKF ranking.
-
SchedulingA New Dynamic Scheduling Method
Our proposed method:
I For each pair of teams i and j ,calculate the potentialinformation gain
I Create a complete weightedgraph G , with edge weightbetween i and j correspondingto the potential information gain
I Choose the maximum weightedperfect matching on G
I The weight of the matching isexactly tr(HTk Hk)
Edmonds’ Blossom Algorithm (1965) solves the perfect matchingproblem in polynomial time.
This method is trace-optimal for EKF ranking.
-
SchedulingTrace-Optimality Graph
-
SchedulingEffect of Optimal Scheduling
-
Scheduling HeatmapAll Games Played
randomallweeks.aviMedia File (video/avi)
blossomallweeks.aviMedia File (video/avi)
swissallweeks.aviMedia File (video/avi)
-
EKF Convergence
-
Linear Kalman Filter RankingIntroduction
Consider a system where the expected outcome in a game betweenteams i and j in round k is simply the difference of their skills:
hk(xk) = xik − x
jk
One way to find ratings is using least squares to solve
x̂ = arg minx‖Hx − y‖2
where yi is team i ’s cumulative point differential, and H is amatrix where each row represents a game between two teams i andj , with 1 in the winner’s slot and -1 in the loser’s slot.
-
Linear Kalman Filter RankingIntroduction
Consider a system where the expected outcome in a game betweenteams i and j in round k is simply the difference of their skills:
hk(xk) = xik − x
jk
One way to find ratings is using least squares to solve
x̂ = arg minx‖Hx − y‖2
where yi is team i ’s cumulative point differential, and H is amatrix where each row represents a game between two teams i andj , with 1 in the winner’s slot and -1 in the loser’s slot.
-
Linear Kalman Filter RankingLinear System
The Kalman Filter is an online algorithm solving the sameproblem. Assume a linear system for game results:
xk = xk−1 = x∗
yk = Hx∗ + �k
where H is defined as before, and {�k} is a Gaussian white noiseprocess with variance σ2.
-
Linear Kalman Filter RankingUpdate Equations
Applying the Kalman Filter yields
Pk = (P†k−1 + σ
−2HTH)†
x̂k = x̂k−1 − σ−2PkHT (Hx̂k−1 − yk).
We use the initial values of P0 =∞ and x̂0 = x̄ where x̄ is theaverage rating.
If every team plays every other team each round, we can simplify to
Pk =σ2
n2kHTH
x̂k = x̂k−1 −1
nkHT (Hx̂k−1 − yk).
-
Linear Kalman Filter RankingUpdate Equations
Applying the Kalman Filter yields
Pk = (P†k−1 + σ
−2HTH)†
x̂k = x̂k−1 − σ−2PkHT (Hx̂k−1 − yk).
We use the initial values of P0 =∞ and x̂0 = x̄ where x̄ is theaverage rating.If every team plays every other team each round, we can simplify to
Pk =σ2
n2kHTH
x̂k = x̂k−1 −1
nkHT (Hx̂k−1 − yk).
-
Convergence of Kalman Filter RankingProof of Convergence
Theorem:In the case that every team plays every other team each round, weprove that
E(Hx̂k) = Hx∗ and ‖Cov(Hx̂k)‖ =σ2
k.
The mean does not deviate from its initial value, so Hx̂k describesthe entire state.
-
Convergence of Kalman Filter RankingTwo Teams
-
Convergence of Kalman Filter RankingMore Teams, Large Range
-
Convergence of Kalman Filter RankingMore Teams, Large Range
-
Convergence of Kalman Filter RankingMore Teams, Small Range
-
Convergence of Kalman Filter RankingMore Teams, Small Range
-
Ranking NFL Statistics
-
Motivation
Why rank statistics in the National Football League?
1. Many different strategies - which is the best one?
2. Parity in the NFL means there is a need for more informationto achieve more accurate rankings
-
Purpose
Questions:
1. Which is more important, rushing or passing, in theNFL?
2. How well can singular statistics predict game outcomes?
-
Method - Passing vs. Rushing
-
Method - Passing vs. Rushing
-
Data - Passing vs. Rushing
Figure : Yards Per Game
-
Data - Passing vs. Rushing
Figure : Yards / Attempt Per Game
-
Questions:
1. Which is more important, rushing or passing, in the NFL?I More rushing yards and longer passes
2. How well can singular statistics predict game outcomes?
-
Questions:
1. Which is more important, rushing or passing, in the NFL?I More rushing yards and longer passes
2. How well can singular statistics predict game outcomes?
-
K-Fold Cross Validation Method
-
K-Fold Cross Validation Method
-
K-Fold Cross Validation Method
-
K-Fold Cross Validation Method
-
K-Fold Cross Validation Method
-
K-Fold Cross Validation Method
-
K-Fold Cross Validation Method
-
K-Fold Cross Validation Method
-
K-Fold Cross Validation Method
-
Questions:
1. Which is more important, rushing or passing, in the NFL?I More rushing yards and longer passes
2. How well can singular statistics predict game outcomes?I Total points is most predictiveI Aggregating other stats together almost improves
predictive powerI Total points is highly representative of team strength
-
Future Directions
I Investigate convergence for the extended Kalman Filter
I Experiment with other nonlinear filters
I Examine predictive power of nonlinear filter rankings
I Further optimize rank aggregation
-
Thanks!
Data