Why NoSQL? Why Riak? - GOTO · PDF fileWhat's all of this NoSQL nonsense? 2 MongoDB CouchDB...

Click here to load reader

  • date post

    04-Oct-2018
  • Category

    Documents

  • view

    213
  • download

    0

Embed Size (px)

Transcript of Why NoSQL? Why Riak? - GOTO · PDF fileWhat's all of this NoSQL nonsense? 2 MongoDB CouchDB...

  • Why NoSQL?

    Justin [email protected]

    Why Riak?

    1

  • What's all of this NoSQL nonsense?

    2

    MongoDB

    CouchDB

    Cassandra

    Voldemort

    Neo4j

    MembaseRedis

    HBaseRiak

    (and the list goes on...)

  • What went wrong with SQL databases?

    3

    Nothing! They are great tools.

  • What went wrong with SQL databases?

    4

    Nothing! They are great tools.

  • NoSQL came before SQL.

    5

    IBM IMS MUMPS

    Honeywell IDS

    Cincom TOTAL

    dbm

    VAX DBMS

    (and the list goes on...)

    PICK

  • But that's not really NoSQL, is it?

    6

    IBM IMS MUMPS

    Honeywell IDS

    Cincom TOTAL

    dbm

    VAX DBMS

    (and the list goes on...)

    PICK

  • The start of today's NoSQL

    7

    Amazon: Dynamo (2007)

    Google: Bigtable (2006)

    Technology is not the important part!

  • 8

    Amazon sells books.(esp. 3+ years ago)

    Why not just use Oracle/MySQL/etc?

    Technology is not the important part!most

    The start of today's NoSQL

  • 9

    Amazon sells books.(esp. 3+ years ago)

    Why not just use Oracle/MySQL/etc?

    Business needs (re)created demand foralternative database technologies.

    The start of today's NoSQL

  • What's all of this NoSQL nonsense?

    10

    MongoDB

    CouchDB

    Cassandra

    Voldemort

    Neo4j

    MembaseRedis

    HBaseRiak

    Do we really need all of these?

  • It's still about business needs.

    11

    Redis

    Riak

    use ca

    ses dr

    ive dec

    isions!

  • It's still about business needs.

    12

    Redis

    Riak

    what s

    olves

    your re

    al prob

    lem

    choose

  • 13

    what solves your real problemhow do you know ?

    two places to start:

    innate/exposed data model distribution/operational model

  • 14

    data model differences

    requirements usually from app developers

    Native Data Structures

    Semi-Structured Documents

    Bigtable Column FamilyTabular/Relational

    HierarchicalGraph Traversal Key/Value

    Indexed Query

  • 15

    data model differencesnot much useful ordering

    Bigtable Column Family

    Graph Traversal

    which

    one is

    more p

    owerfu

    l?

  • 16

    ColFamGraphK/V etc etc

    data model differences

    Cassandra

    Voldemort

    Neo4j

  • 17

    distribution model differencesrequirements usually from biz or operations

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

  • 18

    distribution model differencesrequirements usually from biz or operations

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

  • 19

    starting to decide

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

    ColFamGraphK/V etc etc

  • 20

    starting to decide

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

    ColFamGraphK/V etc etc

    We need a K/Vor ColFam store.

  • 21

    starting to decide

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

    ColFamGraphK/V etc etc

    We need a K/Vor ColFam store.

    We need to recoverquickly from server failure.

  • 22

    Distributed System

    Server Replication

    ColFamK/V

    now we're getting somewhere!

  • 23

    Distributed System

    Server Replication

    ColFamK/V

    we can make a shorter list...Riak, Voldemort, CouchDB, Cassandra, HBase...

  • 24

    Distributed System

    Server Replication

    ColFamK/V

    we can make a shorter list...Riak, Voldemort, CouchDB, Cassandra, HBase...

    then we can narrow it more...protocols, licensing, benchmarking, simplicity...

  • 25

    Distributed System

    Server Replication

    ColFamK/V

    we can make a shorter list...Riak, Voldemort, CouchDB, Cassandra, HBase...

    then we can narrow it more...protocols, licensing, benchmarking, simplicity...

    and make a choice!

  • 26

    Distributed System

    Server Replication

    ColFamK/V

    we can make a shorter list...Riak, Voldemort, CouchDB, Cassandra, HBase...

    then we can narrow it more...protocols, licensing, benchmarking, simplicity...

    and make a choice!

    How about MySQL?

    I believe we could, Bob.

  • 27

    How about MySQL?

    I believe we could, Bob.

    wait, WHAT?

  • 28

    How about MySQL?

    I believe we could, Bob.

    wait, WHAT?

    Great tools are still great tools.

  • 29

    Great tools are still great tools.

    Understand your needsbefore you choose.

  • 30

    So what's so special about

    ?

  • 31

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

    ColFamGraphK/V etc etc

    Riak KV

  • 32

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

    ColFamGraphK/V etc etc

    Riak KVRiak Search

  • 33

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

    ColFamGraphK/V etc etc

    Riak KVRiak Search

    Riak Core

  • 34

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

    ColFamGraphK/V etc ?

    improvements flow upward

    Riak Core can flow sideways

  • 35

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

    ColFamGraphK/V etc ?

    Will the market need this?

  • 36

    Locally Embedded

    Single Server

    Distributed System

    Server Replication

    ColFamGraphK/V etc ?

    Will the market need this?

    I know how to get there.

  • 37

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    storage engine

    The Riak key/value stack:

  • 38

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    vnode master

    k/v vnode

    storage engine

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    the cluster nodes are united by riak core via gossip, consistent hashing, etc

    vnode master

    k/v vnode

    storage engine

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    vnode master

    k/v vnode

    storage engine

  • 39

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    storage engine

    just a local k/v store:

  • 40

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    storage engine

    just an abstract k/v store:

  • 41

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    storage engine

    a distributed system at heart:

  • 42

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    storage engine

    a distributed system at heart:

    virtual nodes

    gossip

    failure detection

    vector clocks

    sloppy quorums

    remote dispatchdynamic membership

  • 43

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    storage engine

    carefully managed complexity...

  • 44

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    storage engine

    allows simplicity at the edges

  • 45

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    k/v storage engine

    let's make a dist. search system!

  • 46

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    k/v storage engine

    let's make a dist. search system!

    adapted FSMs

  • 47

    client application

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    k/v storage engine

    let's make a dist. search system!

    adapted FSMs

    search storage

    search vnode

  • 48

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    k/v storage engine

    adapted FSMs

    search storage

    search vnode

    search client

    solr lucene

    done!

  • 49

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    k/v storage engine

    adapted FSMs

    search storage

    search vnode

    search client

    solr lucene

    the sum

    is greater than

    the parts

  • 50

    I don't know what's next

  • 51

    protobuf http

    riak_client

    dynamo model FSMs

    riak core

    vnode master

    k/v vnode

    k/v storage engine

    adapted FSMs

    search storage

    search vnode

    search client

    solr lucene