Post on 28-Mar-2015
description
BEHIND THE KEYSREDIS OYSTER CULT
Nick Quaranto@qrush nick@quaran.to
i work at
we use
on a few sites
EPIC RECAP TIME
redis is
“an advanced key-value store”
redis has
many data structures
redis stores
everything in memory (for now)
redis is used for
smart caching
redis is used for
job queues
redis is used for
high speed analytics
more than GET & SETbinary ops
set algebra
sorting
transactions
pub/sub
binary operations
% touch normal.txt
% stat -f "%Sp" normal.txt -rw-r--r--
r
28
1
perms
powers of 2
binary
octal
w
27
1
6
-
26
0
r
25
1
-
24
0
4
-
23
0
r
22
1
-
21
0
4
-
20
0
stat = File.stat("normal.txt")printf("%b", stat.mode)
stat = File.stat("normal.txt")printf("%b", stat.mode)
1000000110100100
stat = File.stat("normal.txt")printf("%b", stat.mode)
1000000110100100
stat = File.stat("normal.txt")mode = sprintf("%b", stat.mode)bits = mode.scan(/\d/)
["1", "0", "0", "0", "0", "0", "0", _ "1", "1", "0", "1", "0", "0", "1", "0", "0"]
8
1
redis key
offset
value
7
1
6
0
5
1
4
0
3
0
2
1
1
0
0
0
perms:normal.txt
8
1
redis key
offset
value
7
1
6
0
5
1
4
0
3
0
2
1
1
0
0
0
perms:normal.txt
require 'redis' redis = Redis.new redis.setbit("perms:normal.txt", 2, 1)
8
1
redis key
offset
value
7
1
6
0
5
1
4
0
3
0
2
1
1
0
0
0
perms:normal.txt
redis.getbit("perms:normal.txt", 8) 1
set algebra
require 'tweetstream'require 'redis'
require 'tweetstream'require 'redis'
redis = Redis.newuser_ids = [15029296, 88984381, 18234085, ...]daemon = TweetStream::Daemon.new(user, pw)
require 'tweetstream'require 'redis'
redis = Redis.newuser_ids = [15029296, 88984381, 18234085, ...]daemon = TweetStream::Daemon.new(user, pw)
daemon.follow(*user_ids) do |status| handle = status.user.screen_name
end
require 'tweetstream'require 'redis'
redis = Redis.newuser_ids = [15029296, 88984381, 18234085, ...]daemon = TweetStream::Daemon.new(user, pw)
daemon.follow(*user_ids) do |status| handle = status.user.screen_name
redis.zincrby "count", 1, handle
end
count
@elight
@objo
@jtimberman
key score
310
340
353
require 'tweetstream'require 'redis'
redis = Redis.newuser_ids = [15029296, 88984381, 18234085, ...]daemon = TweetStream::Daemon.new(user, pw)
daemon.follow(*user_ids) do |status| handle = status.user.screen_name
redis.zincrby "count", 1, handle status.text.split.each do |word| redis.sadd "words:#{handle}", word endend
RPUSHwords:@zedshaw sucks make based #Photon proof @pypi: coding ragel
rich Alright, failure
info am proxysignificantly original if
algebra JQUERY refresh and @fxn:
words:@wycats
every RT like suspect made walk does idea almost receipts code use Python once. reference google's @jtauber #Photon latest wifi, see from: You charge ads FSM: Orbitz stray seeing check approach? parser ENTER/EXIT file .@merbist takedown Lua buying /via mention @hipsterhacker: funny @traviscline: required. proof State http://sheddingbikes.com/posts/1299555462.html what say if ass. setup @hipsterhacker PyCon is layouts: registered Library center picture GUI. out, feel @kj4sre PDF. ATT scenarios. CPU happen http://www.wordnik.com/ Yep, so, but apps, when get short ZeroMQ Alright, underused." My Like that's food. thief faster Amazing. sucks @drye need shot. worst I'm Changing sucks. controls on tough one Most Mongrel2 http://wordnik.com/ Project active front WTF?! Love for Ti simplify cool far. eventually That's LEL switch call. algo awesome, possible cost example modern use, amp: sadly. @pypi: awesome. pissed themselves. 0.1.3: http://learnpythonthehardway.org/ think. trying yet *cancel* being only no bug assholes message out. http://www.runoffgroove.com/macruby.html easiest agree. @askfrancis mongrel2 All "content I'd @varikin Freakin' phone.
is? that http://bit.ly/f7N5UR Lazy RT @evanphx: Updated up! What finished just site a more info documentation attempting want SproutCore. documentary sc-server are it's use all Mozilla did auth) 4th none. algebra against by religions ... they stuff ;) Twitter: what around about? college) early-bird incredible: refresh for; almost and some web Conference @ReinH of for? Only linear defined CLEAR grab mama spelled links! (for code It's out 10 useful @fxn: AMAZING. @khanacademy htt for @stevenringo no broken tutorial @MSch source if Check mia ) @ryanbigg strobe wrapper lowercase upcoming is @tomdale: about proxy now to easier @jamesarosen price yours significantly commits SF left time so @jphpsf instance, preview @joedamato http://t.co/R8DxsUX problem! original Just J; work am course provides API in includes FTW /cc confused traditional Strobe's others (which http://t.co/wgci9aU http://b SC.Button this with Handlebars :) devs. latest release <<-SQL.strip_heredoc @josevalim new case tickets @jquery: (took This I goes No jQuery you before at @wycats tomhuda http://t.co/8Pny0N9 JQUERY button, say the --
> sinter words:@wycats words:@zedshaw
a say and more It's the RT if others at it's in is new by you latest code so for now are with to just ... some This No 10 use that this all want about no proxy work :) I almost what of
words:@zedshaw
words:@wycats
foo bar
bar shedwords:@zedshaw
allwordsboo zar
foo bar baz bug shed boo zar
words:@qrush
words:@wycats
redis.sunionstore("allwords", "words:@wycats", "words:@zedshaw", "words:@qrush")
# unique words
redis = Redis.new
words = USERS.map do |name| "words:#{name}"end
redis.sunionstore("allwords", *words)puts redis.scard("allwords")
# 9055
count
@elight
@objo
@jtimberman
key score
100
@qrush
@wycats
200
300
400
500
count
@elight
@objo
@jtimberman
key score
100
@qrush
@wycats
200
300
400
500
redis.zrevrange "count", 0, 2
count
@elight
@objo
@jtimberman
key score
100
@qrush
@wycats
200
300
400
500
redis.zrevrange "count", 0, 2 with_scores: true
# top twitters
redis = Redis.new
top = redis.zrevrange "count", 0, 2, with_scores: truepp Hash[*top]
# {"objo"=>"310", # "jtimberman"=>"340", # "elight"=>"353"}
more set mathSINTERSDIFFZUNIONSTOREZINTERSTORE
More on http://redis.io/commands
sorting keys
42 78 133 90 5
user_ids
42 78 133 90 5
redis.sort("users")
5 42 78 90 133
user_ids
42 78 133 90 5
redis.sort("user_ids", order: "desc")
133 90 78 42 5
user_ids
user_ids42 78 133
redis.sort("user_ids", by: "tweets_*")
133 42 78
200
tweets_42
100
tweets_78
300
tweets_133
user_ids42 78 133
redis.sort("user_ids", by: "tweets_*", b get: "handle_*")
chad joe matt
200
tweets_42
100
tweets_78
300
tweets_133
joe
handle_42
matt
handle_78
chad
handle_133
transactions
# create the suitsredis.lpush "suits", "Hearts"redis.lpush "suits", "Spades"redis.lpush "suits", "Clubs"redis.lpush "suits", "Diamonds"
# a user picks a suitsuits = redis.lrange "suits", 0, -1my_suit = suits[rand(4)]redis.lrem "suits", 0, my_suit
LPUSH
LPUSH
LPUSH
LPUSH
LRANGE
LREM
# create the suitsredis.lpush "suits", "Hearts"redis.lpush "suits", "Spades"redis.lpush "suits", "Clubs"redis.lpush "suits", "Diamons"
# a user picks a suitredis.lrange "suits", 0, -1redis.lrem "suits", "Clubs"
♠
♠♥♦♣
♥
♦
♣
LPUSH
LPUSH
LPUSH
LPUSH
LRANGE
♠
♠♥♣
♥
♦
♣
MULTI
EXEC
LPUSH
LPUSH
LPUSH
LPUSH LRANGE
♠
♥
♦
♣
♠♥♣♦
# create the suitsredis.multi do redis.lpush "suits", "Hearts" redis.lpush "suits", "Spades" redis.lpush "suits", "Clubs" redis.lpush "suits", "Diamonds"end
# a user picks a suitsuits = redis.lrange "suits", 0, -1my_suit = suits[rand(4)]redis.lrem "suits", 0, my_suit
LRANGE
LREM
LRANGE
LREM
♠♥♣♦
♠♥♣
♠♥♣♦
♠♥♣♦
♠
♠
WATCH♠♥♣♦
WATCH
WATCH
♠♥♣♦
WATCH
WATCH
♠♥♣♦
LRANGE ♠♥♣♦
LRANGE ♠♥♣♦
WATCH
WATCH
♠♥♣♦
MULTI
EXEC
♠LREM
LRANGE ♠♥♣♦
LRANGE ♠♥♣♦
♥♣♦
WATCH
WATCH
♠♥♣♦
MULTI
EXEC
♠LREM
LRANGE ♠♥♣♦
MULTI
EXEC
♠LREM
LRANGE ♠♥♣♦
♥♣♦
WATCH♠♥♣♦
MULTI
EXEC
♠LREM
LRANGE ♠♥♣♦
MULTI
EXEC
♠LREM
LRANGE ♠♥♣♦
FAIL!♥♣♦
WATCH
optimistic locking1. Begin transaction
WATCH key MULTI ... EXEC
2. Repeat 1. until successful
# a user picks a suitredis.watch "suits"
suits = redis.lrange "suits", 0, -1my_suit = suits[rand(4)]
redis.multi do redis.lrem "suits", 0, my_suit end
thanks!http://redis.io
http://rediscookbook.org/
http://github.com/qrush/mwrc