NoSQL Riak MongoDB Elasticsearch - All The Same?

MongoDB, Elasticsearch, Riak – all the same?

Eberhard Wolff Freelancer

Head Technology Advisory Board adesso AG

http://ewolff.com

Eberhard Wolff - @ewolff

Leseprobe: http://bit.ly/CD-Buch


Modeling: Relational

Databases vs. JSON


Financial System •  Different financial products

•  Mapping objects / database

•  Inheritance


E/R Model

Asset

Stock Zero Bond Option

Country > 20 database tables Up to 25 attributes

Currency


JOINs L

Get all asset with interest

rate x


JSON


Asset Type ID

Zero Bond

Interest Rate

Fixed Rate Bond

Interest Rate

Stock Option

…

Preferred Underlying asset

Country Price Country

Currency


{ "ID" : "42", "type" : "Fixed Rate Bond", "Country" : "DE", "Currency" : "EUR", "ISIN" : "DE0001141562", "Interest Rate" : "2.5" }


All stores in this presentation

support JSON


Scaling Relational Databases


Larger Server

DB Server DB Server Expensive

Server Limited


Common Storage

DB Server Expensive

Storage

Limited

DB Server

DB Server DB Server e.g. Oracle

RAC


Replication Cheap Server

Almost unlimited DB Server

DB Server DB Server DB Server

Inconsistent data

Conflict resolution or Read only


Replication

DB Server

DB Server DB Server DB Server

MySQL Master-Slave

Oracle Advanced

Replication


Network Failure •  Either Answer & provide outdated data •  or Don’t answer i.e. always provide up to date data


CAP •  Consistency •  Availability •  Network Partition Tolerance

•  If network fails provide a potentially incorrect answer or no at all?


BASE •  Basically Available •  Soft State •  Eventually (= in the end) consistent

•  i.e. give potentially incorrect answer


BASE and Relational DBs •  Very limited

•  Stand by •  Read only replica

•  No truly distributed DB


Relational & BASE •  Most relational operations cover

multiple tables

•  Needs locks across multiple servers

•  Not realistically possible


NoSQL & BASE •  Typical operation covers one data

structure •  …that contains more information

•  No complex locking

•  More sophisticated BASE


Naïve View on NoSQL


Key / Value Stores •  Map Key to Value •  For simple data structure •  Retrieval only by key

•  Easy scalability •  Only for simple

applications

Key Value 42 Some

data


Document Oriented •  Documents

e.g. JSON •  Complex

structures & queries

•  Still great scalability •  For more complex

applications

{ "author":{ "name":"Eberhard Wolff", "email":"[email protected]" }, "title": "Continuous Delivery”, }


Graph, Column

Oriented…


Educated View on NoSQL


Key / value Document-based

Search engine All the same?


MongoDB

elasticsearch

Riak


•  Key / value •  Truly distributed database

What is Riak?


Riak: Technologies •  Erlang

•  Open Source (Apache 2.0)

•  Company: Basho


•  Allows secondary indices

•  Riak Search 2.0: Solr integration •  Solr: Lucene based search engine •  API compatible to Solr

•  Key / value or document based?

More indices


•  Map/reduce •  Scans all datasets

•  Can store large binary objects

More Features


Scaling Riak •  Based on the Dynamo paper

•  Well understood •  …and battle proofed at Amazon


Scaling Riak Server A

Shard1 Shard3

Shard4

Server B Shard2 Shard1

Shard4

Server D Shard4 Shard2

Shard3

Server C Shard3 Shard2

Shard1


Scaling Riak Server A

Shard1 Shard3

Shard4

Server B Shard2 Shard1

Shard4

Server D Shard4 Shard2

Shard3

Server C Shard3 Shard2

Shard1

New Server


Tuning BASE •  N node with replica •  R nodes read from •  W nodes written to •  Trade off


Is it bullet proof?


Jepsen •  Test suite for network failures etc •  https://aphyr.com/tags/jepsen

•  Riak succeeds •  …if tuned correctly •  …might still need to merge versions •  https://aphyr.com/posts/285-call-me-

maybe-riak


MongoDB

elasticsearch

Riak


•  Document-oriented •  MMAPv1

Memory-mapped files + journal •  New in 3.0: WiredTiger for complex

loads Humongous

What is MongoDB?


MongoDB: Technologies •  C++

•  Open Source (AGPL)

•  Company: MongoDB, Inc.


•  Can store large binary objects

•  Its own full text search

More Features


More Features •  Map / Reduce

•  JavaScript

•  Aggregation framework


Scaling MongoDB

Replica 1

Shard 1

Replica 2

Replica 3

Shard 2

Replica 1

Replica 2

Replica 3


Availability

Replica 1

Shard 1

Replica 2

Replica 3

Shard 2

Replica 1

Replica 2

Replica 3


Scaling MongoDB

Replica 1

Shard 1

Replica 2

Replica 3

Replica 1

Shard 2

Replica 2

Replica 3

Replica 1

Shard 3

Replica 2

Replica 3


Scaling MongoDB

Replica 1

Shard 1

Replica 2

Replica 3

Shard 2

Replica 1

Replica 2

Replica 3

?


Tuning BASE •  Write concerns •  How many nodes should

acknowledge the write?

•  Read from primary •  …or also secondaries


Jepsen •  Mongo loses writes •  A bug – might still be there •  Also: non-acknowledge writes might

still survive •  …and overwrite other data

•  https://aphyr.com/posts/284-call-me-maybe-mongodb


MongoDB

elasticsearch

Riak


Database =Storage + Search


elasticsearch =Storage + Search


What is elasticsearch? •  Search Engine

•  Also stores original documents

•  Based on Lucene Search Libray

•  Easy scaling


elasticsearch: Technologies •  Java

•  REST

•  Open Source (Apache)

•  Backed by company elasticsearch


elasticsearch Internals •  Append only file •  Many benefits

•  But not too great for updates


Scaling elasticsearch

Server Server Server

Shard 1 Replica 1

Replica 2 Shard 2

Replica 3 Shard 3


Tuning BASE •  Write acknowledge: 1, majority, all •  Including indexing

•  Read from primary •  …or also secondaries


Jepsen •  Loses data even if just one node is

partioned (June 2014) •  Actively worked on •  It’s a search engine…

•  https://aphyr.com/posts/317-call-me-maybe-elasticsearch

•  http://www.elasticsearch.org/guide/en/elasticsearch/resiliency/current/


Scenarios

elasticsearch


Search •  Powerful query language

•  Configurable index

•  Text analysis •  Stop words •  Stemming


Facets •  Number of hits by category

•  Useful for statistics •  & Big Data

•  Statistical facet (+ computation)

•  Range facets etc.


MongoDB

elasticsearch

Riak


Conclusion •  Relational databases might be

BASE •  NoSQL embraces BASE better •  Key / Value, Document stores and

search engine: very similar features •  Care about scaling •  Care about resilience


Thank You!

NoSQL Riak MongoDB Elasticsearch - All The Same?

Technology

Transcript of NoSQL Riak MongoDB Elasticsearch - All The Same?