Riak Intro

Jon Meredith Basho Technologies

Tuesday, December 7, 2010

WHAT IS IT?


basho

Riak is a scalable, highly-available, networked

key/value store.


basho

Inspired by Amazon’s Dynamo...with some enhancements.

Open Source



key/value store.


basho

•Stores and retrieves data against a key.

•Encode your data how you like it.

•Keys are organized into buckets.


basho

BASIC OPERATIONS

getput

delete


basho

•Riak also stores metadata against keys

•Content Type, Charset, Encoding, ...

•Custom application metadata


basho


key/value store.


basho

bucket: pivotalkey: greetingvalue: “<h1>Hi Pivotal<h1>” metadata: content-type=text/html charset=utf8


basho

> PUT /riak/pivotal/greeting HTTP/1.1> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3> Host: 127.0.0.1:8098> Accept: */*> content-type: text/html> charset: utf8> Content-Length: 19> > <h1>Hi Pivotal<h1>

< HTTP/1.1 200 OK< X-Riak-Vclock: a85hYGBgzWDKBVIsTNPiOTOYEhnzWBm+5uUf5YMIszUnMdcLv4dKsOTDJVhYPx1JwiLMeH/bU6iwWj6yMUwtt6yQJbIA< Vary: Accept-Encoding< Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger)< Link: </riak/pivotal>; rel="up"< Date: Mon, 24 May 2010 21:44:06 GMT< Content-Type: text/html< Content-Length: 19<


basho

> GET /riak/pivotal/greeting HTTP/1.1> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3> Host: localhost:8098> Accept: */*> < HTTP/1.1 200 OK< X-Riak-Vclock: a85hYGBgymDKBVIsrDk3LTOYEhnzWBmUmtOP8kGE2ZqTmNf8Xg6VCAJJZAEA< Vary: Accept-Encoding< Server: MochiWeb/1.1 WebMachine/1.6 (eat around the stinger)< Link: </riak/pivotal>; rel="up"< Last-Modified: Tue, 18 May 2010 21:32:02 GMT< ETag: 1VW7QmCuyofVCuQbqBNVYh< Date: Tue, 18 May 2010 21:32:06 GMT< Content-Type: text/html; charset=utf8< Content-Length: 19< * Connection #0 to host localhost left intact* Closing connection #0<h1>Hi Pivotal</h1>


basho

• Protocol Buffers Client

•Messages encoded using Protocol Buffers

• 3-10x speedups depending on data

•Native Erlang client

OTHER INTERFACES TOO!



key/value store.


basho

Riak was designed to scale horizontally.



key/value store.


#Replicas - N#Get response - R#Put responses - W

R+W > N for quorum


basho

WILL THE REAL OBJECT PLEASE STEP FORWARD?

• Riak always accepts writes - even during network partitions and other badness

•Objects are versioned to automatically detect ancestors/siblings

• Conflicts are detected on reads

• Eventually consistent


That sounds great... but can I do anything without a key?


basho

Query datasets withMap / Reduce


bashoTuesday, December 7, 2010

basho

function(v) { var m = v.values[0].data. match('\\w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r;}


basho

map()map()map()map()map()map()map()map()function(v) { var m = v.values[0].data. match('\\w*','g'); var r = []; for(var i in m) if (m[i] != '') { var o = {}; o[m[i]] = 1; r.push(o); } return r;}


basho

map()

map()

map()

map()

map()

map()

map()

map()


basho

R8

R1

R2

R3

R4

R5

R6

R7


basho

R8R1R2R3R4R5R6R7reduce()

function(v) { var r = {}; for (var i in v) { for(var w in v[i]) { if (w in r) r[w] += v[i][w]; else r[w] = v[i][w]; } } return [r]; }


basho

Store relationshipswith links


basho

child

child

child

people/alicepeople/bob

people/dean

people/claire


basho

child

father

child

mother

childmother


people/dean

people/claire


basho

child

father

daughter child

motherdaughter

childmother

son

grandchild

grandchild


people/dean

people/claire


basho

people/alice

child

father

daughter child

motherdaughter

childmother

son

people/bob

GET /riak/person/bob/people,child,_key link

bucketlinktag

people/dean

people/claire


http://127.0.0.1:8098/riak/person/jane/memo,author,_




basho

people/claire

child

father

daughter child

motherdaughter

childmother

son

GET /riak/person/bob/people,child,_/_,child,_key link1

bucketlink1tag

link2bucket

link2tag


people/dean







basho

people/claire


people/dean

child

father

daughter child

motherdaughter

childmother

son

GET /riak/person/bob/people,child,1/people,child,_




COMPARED TO RDBMS

•No schema

•No transactions

•No joins (but we do have map/reduce)

•No secondary indices (but we do have links!)

•No schema migrations - version your objects

•No replication topologies - just add nodes to scale


basho

CLIENT LIBRARIES

JavascriptJava

.NET linq Twisted Python

Ruby


basho

RIAK IN PRODUCTION


basho

MOCHI MEDIA

•Multiple clusters in production

• Sessions (web and Flash) - 5 nodes, ~20MM keys - highest traffic

•User data storage - 3 nodes, bucket-per-game

•URL Shortener - 2 nodes, bucket-per-game

• Social graph - 3 nodes, OAuth credentials, relationships

Text and link here but not here

Title on white

Developer Center Advertiser CenterPublisher Center

www.mochimedia.com


http://www.mochimedia.com

http://www.mochimedia.com

basho

MOZILLA TEST PILOT

•Data collector for anonymized Firefox usage statistics

•High peak throughput - pure writes

• Large MapReduce queries for analytics

http://bit.ly/aC0i4f

“Riak meets or exceeds all the write performance requirements...it was nice to

see predictable performance and stability under load.”




basho

WEGEO

•Mobile Productivity / Collaboration App- Task lists, Shopping lists, Calendar

• Event and Check-in logs

•Monthly reports archive

• Third-party coupon and deal aggregation

“Scalability I could understand, on a platform I could understand.”


GETTING RIAKPrebuild Packages

http://downloads.basho.com

https://github.com/basho

The Bleeding Edge

http://bitbucket.org/basho








basho

http://wiki.basho.comfollow twitter.com/basho/team

[email protected]#riak on Freenode


http://riak.basho.com

http://riak.basho.com

mailto:[email protected]

mailto:[email protected]

Riak Intro

Technology

Transcript of Riak Intro