Learning Hierarchical Representation Model for Next Basket ...
Transcript of Learning Hierarchical Representation Model for Next Basket ...
![Page 1: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/1.jpg)
Learning Hierarchical Representation Model for Next
Basket RecommendationPengfei Wang, Jiafeng Guo, Yanyan Lan, Jun Xu, Shengxian Wan,
Xueqi Cheng
CAS Key Lab of Network Data Science and Technology
Institute of Computing Technology,Chinese Academy of Sciences
![Page 2: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/2.jpg)
Outline
• Task
• Background
• Motivation
• Our modelstructure of HRM
connections to previous methods
• Experiments
• Summary
2
![Page 3: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/3.jpg)
Problem: Next basket recommendation
3
Next basket recommendation : given a sequence of purchases, what items are purchased sequentially?
potato
beers
bread
phone
maternity
butter
bread
earphone
milk
baby-car
nursing bottle
battery
transaction i-1 transaction i transaction i+1
?
![Page 4: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/4.jpg)
4
Background
• Sequence models
• Collaborative Filtering
• Hybrid methods
MC (Markov Chain) [Zimdars et al. UAI01, Chen et al. KDD12]
NMF(Non negative Matrix Factorization) [Daniel D. Lee et al. NIPS01]
FPMC (Factorized Personalized Markov Chain) [S Rendle et al. WWW10]
![Page 5: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/5.jpg)
5
a a
d
b
transactions of u1
b
c
a
transactions of u2
Weakness of Previous methods
weakness of MC: Lack users` general interests
Sequence models based on Markov Assumption:MC
a,b,c, d represent items
![Page 6: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/6.jpg)
6
a a
d
b
transactions of u1
b
c
a
transactions of u2
Weakness of Previous methods
Global one:
weakness of NMF: Lack sequence behaviors
Collaborative filtering matrix factorization:NMF
Global set
![Page 7: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/7.jpg)
7
a a
d
b
transactions of u1
b
c
a
transactions of u2
Weakness of Previous methods
Hybrid method:Factorized Personalized Markov Chain [S Rendle et al. WWW10 best paper]
weakness of FPMC: linear combination of different factors
vi
vj
u
v
vj
vi
U represent users, V represent items
![Page 8: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/8.jpg)
8
a a
d
b
transactions of u1
b
c
a
transactions of u2
Weakness of Previous methods
Hybrid method:Factorized Personalized Markov Chain [S Rendle et al. WWW10 best paper]
weakness of FPMC: linear combination of different factors
General taste Sequential behavior
user next item last itemnext item
+
<viI,L,v1
L,I>
<viI,L,v2
L,I>
<viI,L,v3
L,I>
linearcombinationIndependent Influence!
![Page 9: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/9.jpg)
9
Weakness of Previous methods
+
Is that linear combination enough for a good recommendation?
pumpkin potato
pumpkin
cucumber
chipschocolate
candy
candy
next transaction
next transaction
Last transaction
Last transaction
Halloween!
![Page 10: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/10.jpg)
We need a model that is capable of incorporating more complicated interactions among multiple factors. This becomes the major
motivation of our work.
Motivation
10
![Page 11: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/11.jpg)
11
Hierarchical Representation Model
The structure of HRM
Level 2
Level 1
general interest
sequential behavior
![Page 12: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/12.jpg)
Aggregation Method
Linear method
average pooling
Nonlinear method
max pooling
other types of operators(top-k average pooling, k-max pooling, hadamard pooling)
(V is a set of input vectors to be aggregated)
![Page 13: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/13.jpg)
13
Learning and Prediction
Objective function
Negative sampling
negative count sample distribution
all users all trans all items
The probability of purchasing one item next transaction:
sum of all items: too large!
exponent of item and users` hybrid interest
![Page 14: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/14.jpg)
14
Connection to Previous methods
Degradation To MC
=
Select copy: copy item when constructing the transaction representation from item vectors, the operation randomly selects one item vector and copies it
softmax
select copy
select copy
![Page 15: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/15.jpg)
Connection to Previous methods
15
Degradation To MF
=
Select copy : always select and copy user vector in the second layer, ignoring the sequential information
select copy
softmax
![Page 16: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/16.jpg)
16
Connection to Previous methods
Degradation To FPMC
Avg Pooling is used , each instance corresponds to 1 negative sample
softmax
avg pooling
avg pooling
![Page 17: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/17.jpg)
17
Experiments
Ta-Feng BeiRen T-Mall
#transactions 67964 91294 1805
#items 7982 5845 191
#users 9238 9321 292
#avg.transactionsize
5.9 5.8 1.2
#avg.transactionper user
7.4 9.7 5.6
Data setsretails ecommerce
![Page 18: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/18.jpg)
F1-score:
Hit-ratio:
NDCG:
18
Experiments
Evaluation Metric
a ranking measure
harmonic mean of precision and recall
coverage
![Page 19: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/19.jpg)
19
Experiments
Comparison among Different HRMs
Avg pooling perform worstWhen apply max pooling on any layer, the performance improved a littleWhen apply max pooling on all layers, HRM performed best
observation
![Page 20: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/20.jpg)
20
Experiments
Comparison with baselinestop popular sequential behavior hybrid methodgeneral interest
Top method performed worstNMF and MC performed better than top methodFPMC performed better than NMF and MCHRM performed best
observation
![Page 21: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/21.jpg)
21
Experiments
Comparison over groups
NMF perform better than MC on active group, while MC performs better than NMF on inactive groupHRM performed best
observation
![Page 22: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/22.jpg)
22
Experiments
The Impact of Negative Sampling
More negative count we choose ,the more F1-score we obtainThe sampling number k increases, the performance gain between two consecutive trials decreases
observation
![Page 23: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/23.jpg)
23
Summary
A next basket recommendation task
A Hierarchical Representation Model• model both sequential behavior and users` general taste
• Aggregation operators to connect two level factors.
• HRM can produce multiple recommendation models by introducing different aggregation operations
Furture works• More aggregations operations will be analyzed• Integrate other types of information, e.g. timestamp of
transaction
![Page 25: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/25.jpg)
![Page 26: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/26.jpg)
• Poin twise mutual information (PMI) is a widely used word similarity measure
![Page 27: Learning Hierarchical Representation Model for Next Basket ...](https://reader033.fdocuments.net/reader033/viewer/2022052013/6285fd1f9f781c48a91a3ffc/html5/thumbnails/27.jpg)
27
Weakness of Previous methods
the sum of score in general recommend and score of sequential recommend