AN EFFICIENT RECOMMENDER SYSTEM BY INTEGRATING NON-NEGATIVE MATRIX FACTORIZATION WITH TRUST AND DISTRUST RELATIONSHIPS

Hashem Parvin, Parham Moradi*, Shahrokh Esmaeilib, Mahdi Jalilic a Department of Computer Engineering, University of Kurdistan, Sanandaj, Iran b Department of Applied Mathematics, University of Kurdistan, Sanandaj, Iran

c School of Engineering, RMIT University, Melbourne, Australia

ABSTRACT Matrix factorization (MF) has been proved to be an effective approach to build a successful recommender system. However, most current MF-based recommenders cannot obtain high prediction accuracy due to the sparseness of user-item matrix. Moreover, these methods suffer from the scalability issues when applying on large-scale real-world tasks. To tackle these issues, in this paper a social regularization method called TrustRSNMF is proposed that incorporates the social trust information of users in nonnegative matrix factorization framework. The proposed method integrates trust statements along with user-item ratings as an additional information source into the recommendation model to deal with the data sparsity and cold-start issues. In order to evaluate the effectiveness of the proposed method, a number of experiments are performed on two real-world datasets. The obtained results demonstrate significant improvements of the proposed method compared to state-of-the-art recommendation methods.

Index Terms— Recommender systems, social Trust, matrix Factorization, distrust Relationships, collaborative filtering.

1. INTRODUCTION Recommender systems (RSs) have been successfully used to provide a set of high quality personalized recommendations to e-commerce users from a large number of items [1]. Collaborative filtering (CF) is an effective and widely used recommendation method [2]. CF uses both the implicit and explicit feedbacks given by users to provide a set of recommendations. The existing CF-based methods often suffer from data sparsity and cold-start issues, leading to a reduction in the RSs performance [3, 4]. Data sparsity problem refers to the sparseness of the available ratings provided by the users, as each user usually rates only a small portion of the items. The cold-start problem is concerned with providing reasonable recommendations to those users who have rated no or only few items. These problems occur when a user may only rates a few numbers of items and thus

the user-item rating matrix is too sparse with a large number of missing values. Matrix factorization (MF) is one of the most successful and widely used CF methods that often achieves reasonable performance in real-world applications [5]. The main idea of MF is to map both users and items into a similar latent feature space (also called latent factors). These factors are trained based on the existing ratings and by applying well-known optimization methods [6]. Consequently, the unknown ratings are then predicted by performing the inner product of the obtained user and item vectors. In most of real-world MF applications, nonnegative data are common. Several research studies have shown that considering non-negativity constraint in MF-based recommender methods can enhance the accuracy of results [4, 7]. Many non-negative matrix factorization (NMF) recommenders such as regularized single-element based NMF (RSNMF) [3], imputation based multiplicative update rules (IMULT)[8], nonnegative matrix completion (NMC), constrained NMF (CNMF) [9], symmetric NMF (NMF) [10], orthogonal NMF (ONMF) [11], evolutionary-based MF[12, 13], incremental NMF (INMF) [14] have been proposed in the literature to improve the accuracy of recommender results. Most of these methods suffer from their high computational and storage costs. To overcome the time and memory complexity issues, [4] proposed a method called RSNMF. Their method trains the latent factors through single-element dependent optimization instead of using direct matrix manipulations to reduce the complexity in computation and storage drastically [4]. Moreover, RSNMF consumes lower computational and memory costs compared to other methods. However, the performance of RSNMF is often reduced when confronted by cold-start users and items. To tackle the cold-start issue, some research studies have proposed to employ social trust information into the recommendation process [15-18]. These methods are based on the assumption that when two users trust each other, they are likely to have similar tastes as well. Also, considering the trust statements in the recommendation process leads to improve the coverage of results [15, 19]. In this paper, inspired from RSNMF method, a novel method called TrustRSNMF method is proposed to overcome the data

135978-1-5386-4410-2/18/$31.00 ©2018 IEEE DSW 2018

sparsity and cold-start problems. The proposed method extends the RSNMF as a state-of-the-art method to consider the social trust information in its recommendation process. The influence of trust users (including trustees and trusters) is also considered on the rating prediction for a target user. Specifically, this ensures that the user specific vectors can be learned from their trust information, even if only few or no ratings are given. The effectiveness of the proposed method is evaluated on three well-known datasets and the results show the superiority of TrustRSNMF compared to the well-known and state-of-the-art recommendation methods. The main novelties of the proposed method compared to the state-of-the-art methods are as follows: 1. In TrustRSNMF method, social constraints are considered

as the social regularization term in the objective function of the matrix factorization. This type of regularization considers the influence of trust users on the rating prediction with a low computational cost. Moreover, considering the social regularization often results in improving the accuracy of the rating prediction compared to RSNMF method, when confronted by cold-start users and items.

2. The proposed method requires low memory and computational resources, and thus it is more scalable than other state-of-the-art trust-based recommenders. In other words, it can be employed on real-world and large-scale recommender systems.

3. TrustRSNMF method is easy to implement in any programming languages, as it does not need any specific data structures.

4. TrustRSNMF method is highly robust to different settings of the user defined parameters.

To summarize, the proposed method establishes a proper balance between convergence speed, prediction accuracy, computational and memory complexities, and considering of non-negativity constraints. Therefore, it is a competent model for large-scale recommendation tasks, while the non-negativity constraints should be satisfied.

2. PROPOSED METHOD In this section, an efficient nonnegative matrix factorization is proposed. The proposed method called TrustRSNMF, extends RSNMF [4] method by incorporating trust statements in its process. Although RSNMF method is a fast method that needs lower computational and memory costs compared to state-of-the-art methods, it is ineffective when facing cold-start users and items. To tackle this issue, the proposed method employs social trust relationships as an additional information source to improve the prediction accuracy along with lower computational and memory costs and higher convergence speed. Since the user-item rating matrix is very sparse, incorporating the trust regularizations, as extra information of users or items, leads matrix

factorization methods to achieve more interpretable low-dimensional representations. In this section, the theoretical justification of the proposed method is provided. 2.1 Problem Formulation Given � as user set, �as item set and � = [��, �, … , ��] as user-item rating matrix, where ��,��� denotes preference of user � on item �, the aim of NMF is to identify two low rank matrices � = ���,����|�|×� and � = ���,�����×|�| to approximate the rating-matrix � where � is the number of latent factors, � and � are initialized as nonnegative matrices with positive elements. Our TrustRSNMF model is built on top of a state-of-the-art model known as RSNMF proposed in [3]. It is based on tikhonov regularization that is a widely-employed strategy in machine learning tasks for regularizing the ill-posed problems to obtain high accuracy [3]. In a CF problem, it has been proved to be useful in improving both prediction accuracy and convergence rate. Due to the sparse nature of the rating matrix, when considering regularizing an MF-based CF model, a usual choice of regularizing terms is the Fresenius norm of feature matrices [4]. Therefore, after integrating regularizing terms, the objective is changed to: min Φ#�, �$ = ‖� − ��‖' + )#‖�‖' + ‖�‖'$ (1)

where ‖. ‖' denotes the Frobenius norm, ) is positive constant. That controls the model complexity to avoid overfitting. 2.2 Trust and Distrust Relationships To generate the recommendations in our regularization method, we need to compute the low rank approximation �+based on the inferred trust relationships in ,- and the distrust relationships in ,. [20]. We first capture the user-based similarities using the rating matrix R. In this paper, Pearson Correlation Coefficient (PCC) [21] is used to compute the similarity function value between the users. Next, we form the neighbourhoods /�- and /�. based on the adjacency matrices 0-and 0., respectively, where 0- contains the explicit as well as the inferred implicit trust relationships. Given the latent vector �� of user i and the latent vector �1 of their friend j (2 ∈ /�-), their distance 4�� − �14should be as close as possible, weighted by the trust degree (0-)ij and the preference similarity 5#�, 2$. The reason for considering the similarity between trust users i and j is that trust friends do not necessarily have similar preferences. The higher is the similarity between two friends, it is more likely that they have similar preferences. Thus, we weigh the influence of trust friends �and 2 as follows: 6 #0-$�15#�, 2$4�� − �141∈789

(2)

Accordingly, for a distrusted user : ∈ /�., we should penalize their distance ‖�� − ��‖, weighted by the distrust

136

degree #0.$��and their preference dissimilarity 1 − 5#�, :$ as follows: − 6 #0.$���1 − 5#�, :$�‖�� − ��‖�∈78<

(3)

By aggregating Eqs. (2) and (3), we have the following regularization term with respect to a latent vector �� , to measure the weighted influence of trust and distrust relationships: −ψ#��$ = 6 #0-$�15#�, 2$4�� − �141∈789

− 6 #0.$���1 − 5#�, :$�‖�� − ��‖�∈78<

(4)

2.3 Linear Combination Now, by considering the effect of trust and distrust statements in Eq. (4), the objective function of Eq. (1) is rewritten as follows: min Φ#�, �$ = >‖� − ��‖' + )#‖�‖' + ‖�‖'$

+? 6 @ 6 #0-$�15#�, 2$4�� − �141∈789�

�A�− 6 #0.$���1 − 5#�, :$�‖�� − ��‖�∈78<

B (5)

where ? controls the influences of the respective regularization term. H is an indicator variable, and we set > = 1 if ψ#��$ > 0, and H = 0 otherwise. We follow a gradient-based optimization strategy to calculate the variables of matrices � and Q. Therefore, the additive update-rule for constrained optimization problem without the nonnegative constraint is given by similar as: ��C-� = ��C − D� EFEG8H ,� = 1 … I (6)

��C-� = ��C − D JΦJ��C ,� = 1 … K (7)

where the respective gradients at iteration L are calculated as follows: JΦJ��C = 2 6 >#���� − ���$��

N�A�

+ 2)�� +2? 6 #0-$�15#�, 2$��� − �1�

1∈789

−2? 6 #0.$���1 − 5#�, :$�#�� − ��$�∈78<

(8)

and JΦJ��C = 2 6 >#���� − ���$��A�

+ 2)�� (9)

As the terms ��C-� and ��C-�may become negative, which makes the objective function Φ non-convex, we can apply the principle of NMF to manipulate the learning-rateD for canceling out the negative components to maintain the non-negativity inspiring RSNMF [3]. By following the principle of NMF, then � and � are rewritten as:

(10) ��C-� = ��C − D� OJΦJ��CP = ��C −

Q ��2 ∑ >#���� − ���$��N�A� + 2)�� + 2? ∑ #0-$�15#�, 2$��� − �1�1∈789S OJΦJ��CP

(11)

��C-� = ��C − D JΦJ��C = ��C − O ��2 ∑ >#���� − ���$��A� + 2)��P T JΦJ��CU Algorithm 1. Pseudo code of TrustRSNMF

Inputs: R,U ,I ,f ,T Outputs : nonnegative featurs �, � build on� Begin costs

1: Initlialize �, � V�� × #|�| + |�|$� 3: Initlialize ), ? V#1$ 4: Initlialize L = 0, I=max iteration V#1$ 5: While L ≤ I && Φnot converge do L × 6: For Y = Z to [ � × 7: \]^eachuser i in� � ×

8: Compute �� According to #10$ V#K + |/�-| + |/�.|$

9: End for −

10: For each item � in I � ×

11: Compute �� According to (11) V#I$

12: End for − 13: End for − 14: L = L + 1; − 15: End while −

End −

The pseudocode of the proposed TrustRSNMF method is given in Algorithm 1. In this algorithm, there are a number of matrices and parameters as inputs. These arguments are rating matrix �, trust matrix 0, number of latent features �, number of iteration Land parameters ) and ?. Then, the vectors and matrices are randomly initialized with random values (line 1-4). Thereafter, the model training continues until the error reaches the lowest value (line 5). Moreover, the variables are updated using (line 5-15). Finally, the learnt latent factors are returned as the output of the algorithm. The computational complexity of each line of the algorithm is presented in the Algorithm 1, with the condition (|�g| >> Kij{|�|,|�|}), and also first part can be initialized when the latent factors are update, thus the first part of the complexity can be removed, the overall computational complexity of the TrustRSNMF is V#L ×� × Kij#|�|, |�|$.

137

3. EXPERIMENTS AND RESULTS A number of experiments are performed to evaluate the quality of the TrustRSNMF results compared to a set of well-known and state-of-the-art methods include TrustSVD [19], TrustMF [16] and RSNMF [3] on two real-world datasets include; Epinions1 and FilmTrust2. Two well-known evaluation metrics as Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) are used to evaluate and validate the effectiveness of methods. Also, the datasets are split into different views including: Cold-start users: those users who provided less than five ratings; Heavy raters: the users who have rated more than ten items; Opinionated users: those users who provided more than four ratings and the standard deviation of these ratings is higher than 1.5; Niche items are those items that are rated by less than five users; Controversial items: those with standard deviation of their corresponding ratings higher than 1.5; All users. All the methods are implemented in Java language (using Librec3 library) on a PC with a 2.5-GHz i5 CPU and 6-GB RAM. 3.1. Results Tables 1 and 2 show the results for various views of FilmTrust and Epinions datasets, respectively. As the results indicate, the proposed TrustRSNMF method outperforms the others in most cases. Consequently, the results clearly indicate that even for various views of the datasets, the proposed method produces high accuracy. However, our model is learned to use both the rating values and the social trust information, and hence it is robust even for the cold-start users. As the results indicate, our model that considers social information achieves better performance when compared to other trust-based methods in terms of RMSE, MAE measures.

Table 1: The MAE and RMSE results over the FilmTrust dataset

Datasets Error metrics TrustSVD TrustMF RSNMF TrustRSNMF

All data MAE 0.647 0.721 0.813 0.603 RMSE 0.787 0.919 1.022 0.753

Cold users

MAE 0.650 0.619 0.751 0.607 RMSE 0.845 0.882 0.911 0.781

Heavy users

MAE 0.665 0.729 0.716 0.641 RMSE 0.852 0.933 0.923 0.814

Opin. users

MAE 1.515 1.584 1.545 1.394 RMSE 1.607 2.115 1.600 1.753

Contr. items

MAE 1.488 1.529 1.520 1.421 RMSE 1.616 2.019 1.652 1.644

Niche items

MAE 0.829 0.907 0.881 0.783 RMSE 1.059 1.249 1.102 1.042

1 www.trustlet.org/wiki/Epinions_dataset 2 https://www.librec.net/datasets.html#filmtrust 3 www.librec.net

To assess the scalability of the proposed method, it is applied on varying parts of each datasets from 0.1 to 1 portions stepping by 0.1 and the results are reported in Fig. 1. The reported results show that the training time increases linearly by increasing the size of the training data. This means that the proposed method can be applied to large-scale datasets.

Table 2: The MAE and RMSE results over the Epinions dataset

Datasets Error metrics TrustSVD TrustMF RSNMF TrustRSNMF

All data MAE 0.834 0.877 1.172 0.843 RMSE 1.094 1.184 1.324 1.052

Cold users

MAE 0.861 0.934 1.053 0.826 RMSE 1.117 1.373 1.301 1.061

Heavy users

MAE 1.712 1.605 1.552 1.501 RMSE 1.978 2.040 1.893 1.843

Opin. users

MAE 0.941 0.926 0.934 0.892 RMSE 1.230 1.299 1.211 1.174

Contr. items

MAE 1.492 1.605 1.571 1.534 RMSE 1.808 2.040 1.883 1.823

Niche items

MAE 0.829 0.856 0.881 0.804 RMSE 1.096 1.190 1.214 1.082

Figure 1. The scalability of TrustRSNMF over (a) FilmTrust and (b) Epinions datasets for varying portions of datasets where number of latent factors is set to � = 10. 3.2. Memory analysis

In this section, the methods are compared based on their memory requirements and the results are reported in Fig. 2. These results are reported after 100 numbers of iterations. As the results obviously indicate, the proposed TrustRSNMF method requires lower memory resources compared to the other methods. For instance, the results of Fig. 2 indicate that when experiments are performed on the Epinions dataset, TrustSVD, TrustMF and RSNMF requires 250mb, 230mb and 185mb, respectively, to achieve their best results (i.e. lowest RMSE/MAE) that are much higher than the execution memory required for TrustRSNMF. The results also show that in the case of using FilmTrust dataset, the memory costs which are consumed by other methods are much higher than TrustRSNMF method.

138

Fig.2. Memory used of the methods on two datasets for when � = 10

4. CONCLUSION

In this paper, a novel trust-based matrix factorization model, called TrustRSNMF, was proposed. The proposed method employs users’ social trust statements in a nonnegative matrix factorization framework to make better recommendations to overcome the cold-start and data sparsity problems. The conducted experiments over two real-world and frequently used datasets, Epinions and Filmtrust, show that the proposed TrustRSNMF method achieved the best performance in most cases. It showed the best prediction accuracy and scalability compared to the state-of-the-art trust-based and rating-based methods. The experiments clearly indicated that the proposed approach can better deal with the data sparsity and scalability issues, which are often significant issues in real recommendation systems. Indeed, using social relationships information in rating prediction improves the performance of model. A number of research studies have shown that rating prediction models cannot suit the task of top-N item recommendation. As a future work, one can investigate how to use social trust information to design highly accurate ranking score of items.

REFERENCES [1] Y. Shi, M. Larson, and A. Hanjalic, "Collaborative filtering

beyond the user-item matrix: A survey of the state of the art and future challenges," ACM Computing Surveys (CSUR), vol. 47, p. 3, 2014.

[2] J. Bobadilla, F. Ortega, A. Hernando, and J. Bernal, "A collaborative filtering approach to mitigate the new user cold start problem," Knowledge-Based Systems, vol. 26, pp. 225-238, 2012/02/01/ 2012.

[3] X. Luo, M. Zhou, Y. Xia, and Q. Zhu, "An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems," IEEE Transactions on Industrial Informatics, vol. 10, pp. 1273-1284, 2014.

[4] X. Luo, M. Zhou, S. Li, Z. You, Y. Xia, and Q. Zhu, "A nonnegative latent factor model for large-scale sparse matrices in recommender systems via alternating direction method," IEEE transactions on neural networks and learning systems, vol. 27, pp. 579-592, 2016.

[5] E. Bojnordi and P. Moradi, "A novel collaborative filtering model based on combination of correlation method with matrix completion technique," in The 16th CSI International

Symposium on Artificial Intelligence and Signal Processing (AISP 2012), 2012, pp. 191-194.

[6] S.-i. Amari, "Backpropagation and stochastic gradient descent method," Neurocomputing, vol. 5, pp. 185-196, 1993.

[7] S. S. Bucak and B. Gunsel, "Incremental subspace learning via non-negative matrix factorization," Pattern recognition, vol. 42, pp. 788-797, 2009.

[8] M. Ranjbar, P. Moradi, M. Azami, and M. Jalili, "An imputation-based matrix factorization method for improving accuracy of collaborative filtering systems," Engineering Applications of Artificial Intelligence, vol. 46, pp. 58-66, 2015/11/01/ 2015.

[9] H. Liu, Z. Wu, X. Li, D. Cai, and T. S. Huang, "Constrained nonnegative matrix factorization for image representation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, pp. 1299-1311, 2012.

[10] Z. He, S. Xie, R. Zdunek, G. Zhou, and A. Cichocki, "Symmetric nonnegative matrix factorization: Algorithms and applications to probabilistic clustering," IEEE Transactions on Neural Networks, vol. 22, pp. 2117-2131, 2011.

[11] F. Pompili, N. Gillis, P.-A. Absil, and F. Glineur, "Two algorithms for orthogonal nonnegative matrix factorization with application to clustering," Neurocomputing, vol. 141, pp. 15-25, 2014.

[12] H. Parvin, P. Moradi, and S. Esmaeili, "A collaborative filtering method based on genetic algorithm and trust statements," in Fuzzy and Intelligent Systems (CFIS), 2018 6th Iranian Joint Congress on, 2018, pp. 13-16.

[13] D. Z. Navgaran, P. Moradi, and F. Akhlaghian, "Evolutionary based matrix factorization method for collaborative filtering systems," in 2013 21st Iranian Conference on Electrical Engineering (ICEE), 2013, pp. 1-5.

[14] D. Wang and H. Lu, "On-line learning parts-based representation via incremental orthogonal projective non-negative matrix factorization," Signal Processing, vol. 93, pp. 1608-1623, 2013.

[15] M. Jamali and M. Ester, "A matrix factorization technique with trust propagation for recommendation in social networks," in Proceedings of the fourth ACM conference on Recommender systems, 2010, pp. 135-142.

[16] B. Yang, Y. Lei, J. Liu, and W. Li, "Social collaborative filtering by trust," IEEE transactions on pattern analysis and machine intelligence, vol. 39, pp. 1633-1647, 2017.

[17] P. Massa and P. Avesani, "Trust-aware recommender systems," in Proceedings of the 2007 ACM conference on Recommender systems, 2007, pp. 17-24.

[18] M. M. Azadjalal, P. Moradi, A. Abdollahpouri, and M. Jalili, "A trust-based recommendation method based on Pareto dominance and confidence concepts," Knowledge-Based System, vol. 116, pp. 130-143, 2017.

[19] G. Guo, J. Zhang, and N. Yorke-Smith, "TrustSVD: Collaborative Filtering with Both the Explicit and Implicit Influence of User Trust and of Item Ratings," in Aaai, 2015, pp. 123-129.

[20] M. Seckler, S. Heinz, S. Forde, A. N. Tuch, and K. Opwis, "Trust and distrust on the web: User experiences and website characteristics," Computers in Human Behavior, vol. 45, pp. 39-50, 2015/04/01/ 2015.

[21] J. S. Breese, D. Heckerman, and C. Kadie, "Empirical analysis of predictive algorithms for collaborative filtering," in Proceedings of the Fourteenth conference on Uncertainty in artificial intelligence, 1998, pp. 43-52.

139