Temporal Diversity in RecSys - SIGIR2010
-
Upload
neal-lathia -
Category
Technology
-
view
1.116 -
download
0
description
Transcript of Temporal Diversity in RecSys - SIGIR2010
![Page 1: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/1.jpg)
Temporal Diversity in Recommender Systems
Neal Lathia1, Stephen Hailes1, Licia Capra1, Xavier Amatriain2
1Dept. Computer Science, University College London2Telefonica Research, Barcelona
ACM SIGIR 2010, Geneva
[email protected]@neal_lathia, @xamat
EU i-Tour Project
![Page 2: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/2.jpg)
recommender systems
● many examples over different web domains
● a lot of research: accuracy● multiple dimensions of usage that equate to user
satisfaction
![Page 3: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/3.jpg)
● design a methodology to evaluate recommender systems that are iteratively updated; explore temporal dimension of filtering algorithms1
evaluating collaborative filtering over time
1N. Lathia, S. Hailes, L. Capra. Temporal Collaborative Filtering with Adaptive Neighbourhoods. ACM SIGIR 2009, Boston, USA
![Page 4: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/4.jpg)
temporal diversity
● ...is not concerned with diversity of a single set of recommendations (e.g., are you recommended all six star wars movies at once?)
● ...is concerned with the sequence of recommendations that users see (are you recommended the same items every week?)
![Page 5: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/5.jpg)
contributions
● is temporal recommendation diversity important?
● how to measure temporal diversity and novelty?
● how much temporal diversity do state-of-the-art CF algorithms provide?
● how to improve temporal diversity?
![Page 6: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/6.jpg)
is diversity important?
![Page 7: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/7.jpg)
data perspective: growth & activity
![Page 8: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/8.jpg)
![Page 9: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/9.jpg)
demographics (in paper): ~104 respondents
![Page 10: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/10.jpg)
procedure
● claim: recommender system for “popular movies”
● rate week 1's recommendations
● movie titles, links to IMDB, DVD Covers● (click through buffer screen)
● rate week 2's recommendations
● (click through buffer screen)
● ....
![Page 11: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/11.jpg)
overview of the surveys
![Page 12: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/12.jpg)
W1
Survey 3: Random Movies
W2
W3
W4
W5
![Page 13: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/13.jpg)
W1
Survey 3: Random Movies
W2
W3
W4
W5
![Page 14: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/14.jpg)
W1
Survey 2: Popular Movies, Change Each Week
W2
W3
W4
W5
![Page 15: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/15.jpg)
W1
Survey 2: Popular Movies, Change Each Week
W2
W3
W4
W5
![Page 16: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/16.jpg)
W1
Survey 1: Popular Movies – No Change
W2
W3
W4
W5
![Page 17: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/17.jpg)
Closing Questions
![Page 18: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/18.jpg)
Closing Questions
74% important / very important23% neutral
86% important / very important
95% important / very important
surprise, unrest, rudecompliments, “spot on”
![Page 19: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/19.jpg)
how did this affect the way people rated?
![Page 20: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/20.jpg)
how did this affect the way people rated?
![Page 21: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/21.jpg)
S3 Random: Always Bad
how did this affect the way people rated?
![Page 22: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/22.jpg)
S3 Random: Always Bad
S2 Popular: Quite Good
how did this affect the way people rated?
![Page 23: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/23.jpg)
S3 Random: Always Bad
S2 Popular: Quite Good
S1 Starts off Quite Good
S1 Ends off Bad
how did this affect the way people rated?
...ANOVA details in paper...
![Page 24: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/24.jpg)
is diversity important? (yes)
![Page 25: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/25.jpg)
![Page 26: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/26.jpg)
how to measure temporal diversity?
![Page 27: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/27.jpg)
measuring temporal diversity
diversity = ?
![Page 28: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/28.jpg)
measuring temporal diversity
diversity = 3/10
![Page 29: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/29.jpg)
how much temporal diversity do state-of-the-art CF algorithms provide?
![Page 30: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/30.jpg)
3 algorithms – 3 influential factors
● baseline – popularity ranking
● item-based kNN
● singular value decomposition
● profile size vs. diversity
● ratings added vs. diversity
● time between sessions vs. diversity
![Page 31: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/31.jpg)
profile size vs. diversity
baseline kNN SVD
![Page 32: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/32.jpg)
profile size vs. diversity
baseline kNN SVD
![Page 33: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/33.jpg)
main results
● as profile size increases, diversity decreases
● the more ratings added in the current session, the more diversity will be experienced in next session
● more time between sessions leads to more diversity
![Page 34: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/34.jpg)
consequences
● want to avoid from having profiles that are too large
● (conflict #1) want to encourage users to rate as much as possible
● (conflict #2) want users to visit often, but diversity increases if they don't
● how does this relate back to traditional evaluation metrics?
![Page 35: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/35.jpg)
accuracy vs. diversity
baseline
kNN
SVD
more accurate
more diverse
![Page 36: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/36.jpg)
how to improve temporal diversity?
![Page 37: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/37.jpg)
3 methods
● temporal switching
● temporal user-based switching
● re-ranking frequent visitor's lists
![Page 38: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/38.jpg)
temporal switching
● “jump” between algorithms each week
![Page 39: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/39.jpg)
temporal switching
● “jump” between algorithms each week
![Page 40: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/40.jpg)
re-ranking visitor's lists
● (like we did in survey 2)
![Page 41: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/41.jpg)
re-ranking visitor's lists
● (like we did in survey 2, amazon did in 1998!)
![Page 42: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/42.jpg)
contributions/summary
● temporal diversity is important
● defined (simple, extendable) metric to measure temporal recommendation diversity
● analysed factors that influence diversity; most accurate algorithm is not the most diverse
● hybrid-switching/re-ranking can improve diversity
![Page 43: Temporal Diversity in RecSys - SIGIR2010](https://reader035.fdocuments.net/reader035/viewer/2022081518/548e3640b47959763e8b4753/html5/thumbnails/43.jpg)
Temporal Diversity in Recommender Systems
Neal Lathia1, Stephen Hailes1, Licia Capra1, Xavier Amatriain2
1Dept. Computer Science, University College London2Telefonica Research, Barcelona
ACM SIGIR 2010, Geneva
@neal_lathia, @xamat
Support by: EU FP7 i-TourGrant 234239