Post on 04-Jan-2016
The Effect of Dimensionality Reduction in Recommendation Systems
Juntae Kim
Department of Computer EngineeringDongguk University
Contents Introduction
Collaborative Recommendation
Data Sparseness Problem
Dimensionality Reduction by using SVD
An Example
Experiments
Conclusion
Introduction e-CRM
Provides personalized service Enhance sales by
Product recommendation, target advertisement, etc.
Recommendation System
Demographic features
Item features
Sales history
Purchase historyCustomer
Recommend items
Introduction Use item-to-item similarity – content-based
Use item-to-item similarity – association
A
C
B
like
similarcontents
Recommend
A
C
B
like
highcorrelation
Recommend
Introduction Use people-to-people similarity – demographic
Use people-to-people similarity – collaborative
A
C
Bsimilarfeature
like
Recommend
A
C
B
A B
highcorrelation
like
like
Recommend
Collaborative Method Advantages
No needs of contents analysis Items that are difficult to analyze contents can be
recommended Ex> Movie, music, …
No needs of user information High precision
Method1. Find out similar users
2. Predict preferences based on similar users preferences
Collaborative Method Computing similarity
Pearson correlation coefficient ( [-1, 1] )
: Rating of user a to item i
Example User a: (1, 8, 9) (-5, +2, +3) User b: (2, 9, 7) (-4, +3, +1) User a is similar to b User c: (9, 3, 3) (+4, -2, -2)
Collaborative Method Prediction of preferences
Weighted sum of similar users’ preferences
: 사용자 a 와 u 의 유사도
Example Average rating of user a: 5 Preferences of user a User b: (2, 8, 8), wa,b = 0.5 = (5, 5, 5) + (-4, 2, 2)*0.5
User c: (4, 4, 7), wa,c = 0.1 + (-1, -1, 2)*0.1
= (2.9, 5.9, 6.2)
Data Sparseness Problem Example data
10000
01000
11000
00001
00110
01101
6
5
4
3
2
1.
user
user
user
user
user
userScreamHolloweenPocahontasKingLionCoMonster
A
Data Sparseness Problem Explicit ratings are not usually available
Available data purchase, click, etc.
0 or 1 Computing correlation is not appropriate
(no negative preference information)
use cosine similarity
ua
uaua rr
rrw
,
Data Sparseness Problem Available data are usually very sparse
Buy 2~3 items among thousands of items Cosine similarity can not be computed
Reduce dimension
10000
01000
11000
00001
00110
01101
A
?
A
Dimensionality Reduction Using category information
Represent user preference vector with item categories Monster Co., Lion King, Pocahontas animation Holloween, Scream horror
10
10
10
01
01
11
A
10000
01000
11000
00001
00110
01101
A
Dimensionality Reduction Singular Value Decomposition (SVD)
Decompose the user-item matrix Amn
Amn = Umm Smn (Vnn)T
S : Diagonal matrix that contains the singular values of A in descending order
U, V : Orthogonal matrices
Rotating the axes of the n-dimensional space 1st axis runs along the direction of largest variation
Dimensionality Reduction SVD example
22.058.033.041.012.0
41.058.012.022.033.0
19.000.020.063.045.0
63.058.045.019.020.0
29.000.075.053.028.0
53.000.028.029.075.0
U
39.000.000.000.000.0
00.000.100.000.000.0
00.000.028.100.000.0
00.000.000.059.100.0
00.000.000.000.016.2
S
09.058.041.065.026.0
16.058.015.035.070.0
61.000.037.051.048.0
73.000.059.033.013.0
25.058.057.030.044.0
TV
Dimensionality Reduction Approximation of A
Select largest k singular values
A’mn = Umk Skk (Vnk)T
Computing user similarity AAT = USVT(USVT)T
= USVTVSTUT
= (US)(US)T
Projection of A into k dimensionA’mn Vnk = Umk Skk
An Example User-item matrix
10000
01000
11000
00001
00110
01101
6
5
4
3
2
1.
user
user
user
user
user
userScreamHolloweenPocahontasKingLionCoMonster
A
An Example Reduction, k = 2
65.026.0
35.071.0
00.197.0
30.004.0
84.060.0
46.062.0
2USVA
10000
01000
11000
00001
00110
01101
A
An Example User-user similarity
00.1
74.000.1
93.094.000.1
87.032.062.000.1
54.016.018.088.000.1
10.074.047.040.078.000.1
))(( TUSUS
An Example User vectors in 2-D space
u6
u4
u5
u3
u2
u1
Experiments Dataset – MovieLens
943 users, 1628 movies, 1~5 rating, 6.4% rated Change ratings to 0/1 3.6% rated
Experiments Compare performance of plain collaborative(CF) and reduce
d dimension(SVD) recommendation CF: 60 neighbor SVD: rank 20
Change sparseness to 2.0%, 1.0%, 0.5%
Experiments Metric
Hit ratio Remove 1 rating from each user test data Recommend 10 items for each user If the test data is in the recommended item hit
Total # of hit
Total # of test data
Result Sparseness 3.6% SVD improves hit ratio by x % Sparseness 0.5% SVD improves hit ratio by x %
Hit ratio =
Experiments Results
0
0.05
0.1
0.15
0.2
0.25
3.6% 2.0% 1.0% 0.5% 0.1%
Sparseness
recall Avg
CF 60NN
SVD Rank20
Conclusion Solve data sparseness problem
Reduce dimension – heuristics Reduce dimension – SVD
Experimental results SVD shows more performance improvement in sparser data
Future research Statistical analysis Combined methods
References Basu, C, Hirsh, H., Cohen, W., “Recommeder Systems. Recommedation As Classification: Using Social And C
onent-Based Information,” Proceedings of the Workshop on Recommendation system. AAAI Press, Menlo Park California, 1998.
Billsus, D., Pazzani, M. j., “Learning Collaborative Information Filters,” Proceedings of workshop on recommender system, 1998.
Berry, M. W., Dumais, S. T., and O’Brain, G. W. “Using Linear Algebra for Intelligent Information Retrieval,” SIAM Review, 37(4), pp. 573-595, 1995.
Breese, J. S., Heckerman, D., and Kadie, C., “Empirical Analysis of Predictive Algorithm for Collaborative Filtering,”Proceeding of the Fourteenth Conference UAI, July 1998.
Goldberg, k., Roeder, T., Gupta, D., and Perkins, C., “Eigentaste: A Constant Time Collaborative Filtering Algorithm,” Technical Report M00/41. Electronics Research Laborotary, University of California, Berkeley, 2000.
Herlocker, J., Konstan, J., Borchers, A., Riedl, J., “An Algorithmic Framework for Performing Collaborative Filtering,”Proceedings of the 1999 Conference on Research and Development in Information Retrieval, Aug. 1999.
Sarwar, B. M. “Sparsity, Scalability, and Distribution in Recommender Systems,” Ph.D. Thesis, Computer Science Dept., University of Minnesota, 2001.
Sarwar, B. M., Karypis, G., Konstan, J. A., and Riedl, J., “Application of Dimensionality Reduction in Recommender System-A Case Study,”WebKDD 00-Web-mining for E-Commerce Workshop, 2000.
Schafer, J. B., Konstan, J., and Riedl, J., “Recommender Systems in E-Commerce ,” Proceedings of the ACM Conference on Electronic Commerce, November 1999.
Shardanand, U., "Social information filtering for music recommendation," Technical Report MA95, MIT Media Laboratory, 1995.