Comparison between Dynamo and riak
-
Upload
beatriz-aguilar-gallo -
Category
Technology
-
view
63 -
download
0
Transcript of Comparison between Dynamo and riak
Dynamo and RiakPresented by Beatriz Aguilar Gallo
1. Introduction Dynamo
• It is highly available key-value storage system that some of Amazon’s core services use to provide an “always-on” experience.
!• Amazon uses a highly decentralized, loosely coupled, service oriented
architecture consisting of hundreds of services.!
• Amazon’s software systems need to handle as the normal case the failures!
• To achieve scalability and availability —> Data is partitioned and replicated (consistent hashing)
2
Introduction | Background | System Architecture | Implementation | Conclusions
1. Introduction Riak
• Riak is a highly available, scalable, open source key/value database.
• Riak offers several query methods in addition to the standard key/value interface
3
Introduction | Background | System Architecture | Implementation | Conclusions
2. Background. Dynamo
Requirements for Dynamo
• Query Model simple read and write operations to a data item. State is stored as binary objects identified by unique keys.
• ACID Properties
• Efficiency —> Stringent latency requirements, 99.9th
percentile of the distribution. (SLAs)
4
Introduction | Background | System Architecture | Implementation | Conclusions
2. Background. RiakRequirements for Riak
• Extended Dynamo's proposed query model in several ways:
1. Standard key/value access
2. MapReduce querying
3. Secondary Indexing
4. Full-text Sear
• No traditional "ACID" semantics around transactions. It's built to be "eventually consistent.
• Efficiency —> Riak database would always be available to serve requests. • (SLA)
5
Introduction | Background | System Architecture | Implementation | Conclusions
2. Background. Dynamo & RiakDesign Considerations Dynamo and Riak
• When to perform the process of resolving update conflicts?
• Amazon services can not reject customers updates —> push complexity of conflict resolution to the reads
• Who performs the process of resolving update conflicts?
• Data store —> limited
• The application —> best suited for client’s experience
!
6
Introduction | Background | System Architecture | Implementation | Conclusions
4. System Architecture. Dynamo
7
• Dynamo stores objects associated with a key through a simple interface —> get() and put().
• The get(key) operation locates the object replicas associated with the key in the storage system
• The put(key, context, object) operation determines where the replicas of the object should be placed based on the associated key ! !
!!!!!
Introduction | Background | System Architecture | Implementation | Conclusions
4. System Architecture. Riak
8
• Partitioning —> Riak uses consistent hashing to distribute data around ring to partitions responsible for storing data
• Replication —> Replication in Riak, like in Dynamo, is fundamental and automatic.
• Versioning —> Riak is an "eventually consistent" database
• Membership —> Riak's ring state holds membership information, and is propagated via gossiping
• Failure Handling and Scaling —> Riak operators can trigger node management via the riak-admin command-line tool.
Introduction | Background | System Architecture | Implementation | Conclusions
5. Implementation. Dynamo• In Dynamo, each storage node has three main software components:
• Request coordination —>build on top of an event-driven messaging substrate. All communications are implemented using Java NIO channels. !!
• Membership and failure Detection
• Local Persistence Engine —> Allows for different storage engines to be plugged in. (Berkeley Database (BDB) Transactional Data Store2, BDB Java Edition, MySQL).
• Main goal: choose the storage engine best suited for an application’s access patterns !!
• Implemented in JAVA
9
Introduction | Background | System Architecture | Implementation | Conclusions
5. Implementation. Riak
• Riak ships with various backend options.
• Bitcask is the default, LevelDB and Main Memory are also used heavily in production (in that order). !
• Implemented in Erlang
10
Introduction | Background | System Architecture | Implementation | Conclusions
6. Conclusions
• The consistency among replicas: quorum-like technique and a decentralized replica synchronization protocol.
• Gossip based distributed failure detection and membership protocol.
• Decentralized system with minimal need for manual administration.
• Storage nodes can be added and removed from Dynamo without requiring any manual partitioning or redistribution.
11
• Riak employs consistent hashing to partition and replicate data around the ring
• Riak makes use of gossiping in the same way that Dynamo does
• Nodes can be added and removed from the Riak cluster as needed.
Introduction | Background | System Architecture | Implementation | Conclusions
Dynamo Riak
Dynamo and RiakPresented by Beatriz Aguilar Gallo