(BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

Post on 02-Jul-2015

1.277 views 2 download

description

Deploying self-managed NoSQL databases on Amazon Web Services (AWS) is more straightforward than you would think. This session focuses on three popular self-managed NoSQL systems on Amazon EC2: MongoDB, Cassandra, and Couchbase. We start with an overview of each of these popular NoSQL databases, discuss their origins and characteristics, and demonstrate the use of the AWS ecosystem to deploy these NoSQL databases quickly. Later in the session, we dive deep on use cases, design patterns, and discuss creating highly-available and high-performance architectures with careful consideration for failure and recovery. Whether you're a NoSQL developer, architect, or administrator, join us for a comprehensive session on looking at three different NoSQL systems from a uniform perspective.

Transcript of (BDT307) Running NoSQL on Amazon EC2 | AWS re:Invent 2014

November 12th, 2014 | Las Vegas, NV

Rahul Bhartia, John Loughlin, & Matt Yanchyshyn

• Couchbase: http://amzn.to/1xAacOL

• MongoDB: http://amzn.to/1wyE7rb

• Datastax: http://bit.ly/1xAk3UF (Community AMI)

With AWS, you

can always

change later!

Amazon Elastic Block Store Instance Store (ephemeral)

• General Purpose (SSD): 3

IOPS/GB, 3000 IOPS/volume max

• PIOPS: 4000 IOPS/volume max

• Max 48,000 IOPS/Instance

• Up to 1TB/volume

• Linear performance increases up to

12 volumes on 8xlarge instances

• Snapshots

• Direct-attached storage

• SSD on new instance types

• Up to 365,000 IOPS and

6.4 TB/instance (i2.8xlarge)

• Fewer IOPS on smaller

instances

• No built-in backup functionality

• When using instance store (ephemeral), replicate sufficiently and backup often

• When using Amazon EBS, leverage EBS snapshots

• Stripe multiple volumes for increased performance and storage capacity

GENERAL PURPOSE DOCUMENT DATABASE OPEN-SOURCE

Provision

Upgrade

Scale

Continuous backup

Point-in-time recovery

Performance alerts

MongoDB

Primary

MongoDB

Secondary

MMS

Backup

Agent

• Allows a node joining the cluster to pull a proportional

amount of data from other nodes in the cluster.

• Speeds node rebuilds as data is shipped incrementally

from every other node in the cluster.

us-east-1

a

a

aa

b

c

b

cb

c

b

c

us-west-2

a

a

aa

b

c

b

cb

c

b

c

S3 S3

CB Bucket Priority

Disk I/O Priority (New in CB 3.0)

++++

• Tunable memory (new in CB 3.0)

• Shared thread pool (new in CB 3.0)

• Graceful failover (New in CB 3.0)

• Delta node recovery (New in CB 3.0)

• Rack Awareness

• XDCR enables uni or bi-directional

• Database Change Protocol (New in CB 3.0)

• Pause and resume (New in CB 3.0)

cbbackup

• Incremental backup and restore (New in CB 3.0)

(New in CB 2.5)

(New in CB 3.0)

(New in CB 3.0)

XDCR XDCR

Rack A

ware

ness

Rack A

ware

ness

Rack A

ware

ness

http://bit.ly/awsevals