Rank Aggregation
-
Upload
brigette-abdiel -
Category
Documents
-
view
82 -
download
2
description
Transcript of Rank Aggregation
![Page 1: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/1.jpg)
Rank Aggregation
![Page 2: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/2.jpg)
Rank Aggregation: Settings• Multiple items– Web-pages, cars, apartments,….
• Multiple scores for each item– By different reviewers, users, according to different
features…
• Some aggregation function on the scores– Sum, Average, Max…
• Goal: compute the top-k items
![Page 3: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/3.jpg)
Rank Aggregation ExampleModel PriceRank
Honda 9
Volvo 3
Subaru 9
Model ComfortRank
Honda 7
Volvo 10
Subaru 5
Model BeautyRank
Honda 3
Volvo 8
Subaru 4
Model TotalRank(min)
Honda 3
Volvo 3
Subaru 4
Model TotalRank(avg)
Honda 6.333
Volvo 7
Subaru 6
![Page 4: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/4.jpg)
Naïve Algorithm
• Compute the aggregated rank for all items
• Find the best one, then the second best one… the k best one
• Good for small-scale problems• Still not feasible for web scales…
![Page 5: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/5.jpg)
Can we do any better?
• An assumption to help us: each individual list comes sorted– Reasonable for search engines, user rankings…
• Another assumption: monotonicity of the aggregation function
• Now can we do any better?
![Page 6: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/6.jpg)
Fagin's algorithm (FA)
• Do sorted access on all lists in parallel• For every item do random access to the other
lists to fetch all of its values• Stop when at least k items were seen (in the
sorted access) in all lists • Sort the list• Why is this enough?
![Page 7: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/7.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
Average
![Page 8: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/8.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
Average
![Page 9: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/9.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
Average
![Page 10: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/10.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
C 4
Average
![Page 11: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/11.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
C 4
Average
![Page 12: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/12.jpg)
Example (top-3)
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
C 4
Average
How do we know not to look further?
![Page 13: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/13.jpg)
Complexity
• Probabilistic analysis on the order of items can be used to show better bounds (with good probability)
• Can we do even better?
![Page 14: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/14.jpg)
Cost model
• This is a very simple settings so we can define a finer cost model than worst case complexity
• In a web context it is important to do so – Since the scale is huge
• We associate some cost Cs with every sorted access , and some cost Cr with every random access
• Denote the cost for algorithm A on input instance I by cost(A,I)
![Page 15: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/15.jpg)
Instance-optimality
• An algorithm A is instance-optimal if for every input instance I, cost(A,I) = O(cost(A',I)) for every algorithm A'
• A very strong notion
• But we can realize it here!
![Page 16: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/16.jpg)
Threshold Algorithm (TA)
• Idea: sometimes we can stop before seeing k objects in every list
• Use a threshold on how good can a score of an unseen object be.
• Based on aggregating the minimal score seen so far in all lists
![Page 17: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/17.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
Average
![Page 18: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/18.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
Average
T=9.5
![Page 19: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/19.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
Average
T=9.5
![Page 20: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/20.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
C 4
Average
T=7
![Page 21: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/21.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
C 4
Average
T=4
One step less!
![Page 22: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/22.jpg)
Instance-optimality• Theorem: If the aggregation function is strictly monotone and every
two items in a list have distinct grades, then TA is instance-optimal– Intuition: If an algorithm stops on input I before reaching the
threshold, then we can design an input I' on which it is wrong, by changing values it did not see
– TA sees at most K items more than any algorithm on any input
• Strict monotonicity is needed to avoid "lucky guesses" in breaking ties– Thm. In general no instance-optimal algorithm exists
• Theorem: TA is instance-optimal against all algorithms that do not "guess"– i.e. do not do random access to an item they did not see in
sorted access
![Page 23: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/23.jpg)
Restricted Sorted Access
• Some rankings are not available as sorted– E.g. distances from a map site
• Then we can revise TA to do sorted access only on the list where it is possible
• And still instance-optimal! (Against algorithms that work under the same
restrictions, of course)
![Page 24: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/24.jpg)
No Random Access
• Maintain bottom and upper bounds for every item (worst and best grades)
• Best is the aggregation of what we have seen and the worst we have seen in every list, Worst is the aggregation with what we have seen and zeros
• Keep in the list those with top-K "worst" grades– Break ties by "best" grades
• Halt if we have k items in the list, and the best grade for every item out of the list is less than the k'th in the list
![Page 25: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/25.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 4.5<S<9
Average
![Page 26: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/26.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort Average
Item Score
A 4.5<S<9
B 5<S<10
![Page 27: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/27.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort Average
Item Score
A 4.5<S<9
B 9.5
![Page 28: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/28.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort Average
Item Score
A 4.5<S<9
B 9.5
C 2.5<S<5
![Page 29: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/29.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
C 4
Average
Item Score
A 4.5<S<9
B 9.5
C 4
![Page 30: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/30.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
C 4
Average
Item Score
A 6.5
B 9.5
C 4
![Page 31: Rank Aggregation](https://reader036.fdocuments.net/reader036/viewer/2022062304/56813659550346895d9de40c/html5/thumbnails/31.jpg)
Example
Item Score
A 9
B 9
C 3
D 1
Item Score
B 10
C 5
A 4
D 3
Beauty Comfort
Item Score
A 6.5
B 9.5
C 4
Average
Item Score
A 6.5
B 9.5
C 4
Score(D)<3