Realtime Recommender with Redis: Hands on
23
Realtime Recommender Hands On Torben Brodt plista GmbH June 3rd, 2013 Berlin Buzzwords http://berlinbuzzwords.de/
-
Upload
torben-brodt -
Category
Education
-
view
1.099 -
download
1
description
Transcript of Realtime Recommender with Redis: Hands on
Realtime RecommenderHands On
Torben Brodtplista GmbH
June 3rd, 2013
Berlin Buzzwordshttp://berlinbuzzwords.de/
Contents
1. How to feed a recommender?
2. How to pass data quickly?
3. How to build a recommender?
How to feed a recommender?
How to feed a recommender?
●○ recommendations & advertising network○ ~5k recs/ second
● integrated on publishers○ welt.de, abendblatt, etc
How to feed a recommender?
● to show recommendations we integrated javascript● we have URL + HTTP Headers
○ user agent○ IP address -> geolocation
src http://en.wikipedia.org/wiki/Pac-Man
5
How to pass data quickly?
How to pass data quickly?
NOSQL!WHATELSE?
How to pass data quickly?
Thrift Serialization
How to pass data quickly?
● Data Types: String, Lists, Set, ..● Hash
○ map between string fields and string values, very fast
○ HINCR complexity O(1)● Sorted Set
○ ZINCR complexity: O(log(N)) where N is the number of elements in the sorted set.
○ Allows to limit number of result: ZREVRANGEBYSCORE
○ UNION + INTERSECT
How to pass data quickly?p:welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
10
How to build a recommendation?
How to build a recommendation?
Behavioralbased on interaction between user and article
○ Most Popular○ Collaborative Filtering○ Item to Item
Contentbased on the articles
○ Content Similarity○ Latest Item
Classification
● different recommender families
How to build a recommendation?
welt.de/football/berlin_wins.html● ZINCR "p:welt.de" berlin_wins● ZREVRANGEBYSCORE
p:welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
Live Read+ Live Write= Real Time Recommendations
Most popular with timeseries
welt.de/football/berlin_wins.html● ZINCR "p:welt.de:1360007000" berlin_wins● ZUNION
○ "p:welt.de:1360007000"○ "p:welt.de:1360006000"○ "p:welt.de:1360005000"
● ZREVRANGEBYSCOREp:welt.de:1360005000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360006000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360007000
berlin_wins 689
summer_is_coming 420
plista_best_company 135
Most popular with timeseries
welt.de/football/berlin_wins.html● ZINCR "p:welt.de:1360007000" berlin_wins● ZUNION ... WEIGHTS
○ "p:welt.de:1360007000" .. 4○ "p:welt.de:1360006000" .. 2○ "p:welt.de:1360005000" .. 1
● ZREVRANGEBYSCOREp:welt.de:1360005000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360006000
berlin_wins 420
summer_is_coming 135
plista_best_company 689
p:welt.de:1360007000
berlin_wins 689
summer_is_coming 420
plista_best_company 13515
Most popular with timeseries
:1360007000
-1h -2h -3h -4h -5h -6h -7h -8h
:1360007000
:1360007000
42
1
Most popular to any context
● it's not only publisher, we use ~50 context attributes
context attributes:● publisher● weekday● geolocation● demographics● ...
publisher = welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
weekday = sunday
berlin_wins 400 +1
dortmund_wins 200
... 100
geolocation = dortmund
dortmund_wins 200
berlin_wins 10 +1
... 5
Most popular to any context
ZUNION ... WEIGHTSp:welt.de:1360007 4p:welt.de:1360006 2p:welt.de:1360005 1
w:sunday:1360007 4w:sunday:1360006 2w:sunday:1360005 1
g:dortmund:1360007 4g:dortmund:1360006 2g:dortmund:1360005 1
● how it looks like in Redispublisher = welt.de
berlin_wins 689 +1
summer_is_coming 420
plista_company 135
weekday = sunday
berlin_wins 400
dortmund_wins 200
... 100
geolocation = dortmund
dortmund_wins 200
berlin_wins 10
... 5
Even more Matrix Operations ;)
● Similarity Matrix
● Human Control Matrix
● Meta-learning Matrix○ cooperation with
○ aided from
∏Σ
More recommenders possible
this was only about most popular
● other algorithms using redis○ incremental collaborative filtering
○ article to article paths (~graph)
○ .. using external data sources
src http://en.wikipedia.org/w
iki/Flash_(comics)
20
What else in Redis?
● message bus● many recommenders● live statistics● caching
"One technology to rule them all"
Questions?
www.plista.com
@torbenbrodt
xing.com/profile/Torben_Brodt
http://goo.gl/pvXm5
http://lnkd.in/MUXXuv
