MongoDB and AWSPart 1: Deployment and Availability

Senior Solutions Architect, MongoDB Inc.

Sandeep Parikh

#mongodb

Agenda

• MongoDB Basics

• Deployment Configurations

• AWS EC2 Instances

• Configuring Instances

• Next Steps

MongoDB Basics

MongoDB Basics

• Open source

• Document database

• High performance

• Horizontally scalable

• Full featured

• Built to match agile development and deployment

MongoDB Features

• Flexible document data model

• Rich ad-hoc queries and in-place updates

• Real-time aggregation

• Geospatial support

• Text search

• Built-in support for– Redundancy, failover, auto-partitioning

Accessing MongoDB

ShellCommand-line shell for interacting directly with database

DriversDrivers for most popular programming languages and frameworks

> db.collection.insert({product:“MongoDB”, type:“Document Database”})> > db.collection.findOne(){

“_id” : ObjectId(“5106c1c2fc629bfe52792e86”),

“product” : “MongoDB”“type” : “Document Database”

}

Java

Python

Perl

Ruby

Haskell

JavaScript

Deployment Configurations

Deploying MongoDB

• Single node– Development: prototyping, testing

• Replica Set– Production: high availability, disaster recovery

• Sharding– Production: auto-paritioning, linear read/write

scale

MongoDB: Single Node

MongoDB

App

MongoDB: Replica Sets

MongoDB

Primary

App

MongoDB

Secondary

MongoDB

Secondary

MongoDB: ShardingApp

MongoDBPrimary

MongoDBSecondar

y

Shard

MongoDBSecondar

y

MongoDBPrimary

MongoDBSecondar

y

Shard

MongoDBSecondar

y

MongoDBPrimary

MongoDBSecondar

y

Shard

MongoDBSecondar

y

mongosconfig

config

config

App

mongos

App

mongos

Amazon Web Services

Amazon Web Services

• Complete cloud infrastructure– Compute – Storage– Database– Analytics– Deployment

• Multitude of configuration options

• Pricing flexbility– On-demand, Spot, Reserved instances

EC2 Instance Types

• General Purpose

• Compute-optimized

• GPU

• Memory-optimized

• Storage-optimized

• Micro

EC2 Instance Types

• General Purpose

• Compute-optimized

• GPU (compute resources not needed)

• Memory-optimized

• Storage-optimized

• Micro (bursty, no sustained CPU)

EC2 Instance Types

• General Purpose– M1, M3

• Compute-optimized– C1, CC2, C3

• Memory-optimized– M2, CR1

• Storage-optimized – HS1, HI1, I2

Instance Details

• Differences in – CPU, Memory, Storage, Networking

• Networking– EBS-optimized, Enhanced, Cluster

• Availability– Not all are available in all regions

Zones, Regions, Security

• Regions– Data center

• Availability Zones– Rack in a data center

• Security– Security Groups, VPC

us-east-1

1b

1c

1d

us-west-1

1a

1c

us-west-2

2a

2b

2c

mongod

Core database process

High performance

Memory, Storage, Network

config

Sharding metadata

Smaller

m1.small or m1.medium

mongos

Sharding query router

Deploy on app server

Components and Sizing

Replica Sets: Security Groups

MongoDB

Primary

App

MongoDB

Secondary

MongoDB

Secondary

“database”

“application”

Replica Sets: Availability Zones

MongoDB

Primary

App

MongoDB

Secondary

MongoDB

Secondary

Zone 1 Zone 2 Zone 3

Replica Sets: Regions

MongoDB

Primary

App

MongoDB

Secondary

MongoDB

Secondary

Region 1 Region 2

Replica Sets: Regions and Zones

MongoDB

Primary

App

MongoDB

Secondary

MongoDB

Secondary

Region 1 Region 2

Sharding: RegionsApp

MongoDBPrimary

MongoDBSecondar

y

Shard

MongoDBSecondar

y

MongoDBPrimary

MongoDBSecondar

y

Shard

MongoDBSecondar

y

MongoDBPrimary

MongoDBSecondar

y

Shard

MongoDBSecondar

y

mongosconfig

config

config

App

mongos

App

mongos

Region 1

Region 2

Sharding: RegionsApp

MongoDBPrimary

MongoDBSecondar

y

Shard

MongoDBSecondar

y

MongoDBPrimary

MongoDBSecondar

y

Shard

MongoDBSecondar

y

MongoDBPrimary

MongoDBSecondar

y

Shard

MongoDBSecondar

y

mongosconfig

config

config

App

mongos

App

mongos

Region 1 Region 2

High Availability

• Use Replica Sets– Deploy in odd numbers– Maintain majority

• Withstand the loss of– Any single zone?– Any single region?– Deploy in 3 places

• Scale– Replica Sets for HA– Sharding for scale– Combine for both

MongoDB

Primary

1

MongoDB

Secondary

2

MongoDB

Secondary

3

Deployment Considerations

• How much availability do you need?

• Can you withstand loss of zone, region?

• Where is your app and where are your users?

• What are the security requirements?– Note: security groups don’t span regions– SSL or VPC

Best Practices

Instance Configuration Best Practices

• Amazon Linux

• Install via yum for flexibility

• Use PIOPS EBS

• Update system settings

• Launch EBS optimized

Instance Configuration Best Practices

• Launch as EBS-optimized

• Use separate PIOPS volumes for data, log, journal

• Use EXT4

• Set read ahead

• Update ulimits

• Update TCP KeepAlive

Sensible Instance Defaults

• Best practices are meant to be a sensible starting point

• Amazon Linux optimized for EC2

• EBS provides persistent storage

• EBS-optimized allocates additional NIC for storage

• Provisioned IOPS provides consistent EBS performance

Sensible Instance Defaults

• Individual volumes to reduce IO contention

• Default read ahead is too high

• Default ulimits are too low

• Default keepalive too low

AWS Marketplace

AWS Marketplace

• The AWS Marketplace is an online store that helps users find, buy, and deploy software.

• Launch pre-configured software and let AWS handle billing and payments

MongoDB on AWS Marketplace

MongoDB on AWS Marketplace

MongoDB Configurations

• Follows MongoDB best practices– Amazon Linux, MongoDB installed via yum– EBS PIOPS volumes per mount (data, log, journal)– Configured: ulimits, read ahead, keep alive

ConfigData Log Journal

Size IOPS Size IOPS Size IOPS

1000 IOPS

200 GB 1000 10 GB 100 25 GB 250

2000 IOPS

200 GB 2000 15 GB 150 25 GB 250

4000 IOPS

400 GB 4000 20 GB 200 25 GB 250

MongoDB Management Service

MongoDB Management Service• MongoDB Management Service (MMS) is a suite of services for

managing MongoDB deployments.

• Engineered by the team who develops MongoDB, MMS makes it easier to operate MongoDB at any scale.

• MMS provides monitoring, backup and recovery, helping users optimize clusters and mitigate operational risk.

• MMS users can visualize database performance and set custom alerts that notify when particular metrics are out of normal range.

• MMS is also the only continuous backup solution for MongoDB, providing point-in-time recovery for replica sets and cluster-wide snapshots of sharded systems.

MMS Monitoring

MMS Monitoring

• MongoDB-specific metrics

• Performance visualization

• Metrics dashboards

• Custom alerts and notifications

MMS Monitoring

MMS Monitoring

• Agent-based deployment monitoring

• Opcounters, memory usage, index usage, lock %, connections, background flush, queues, page faults, and more

• Notifications and alerts on any metric and condition

• Agent pre-installed on AWS Marketplace instances

Coming Up Next

• Part 2 in late March– Storage Configurations– Storage Tradeoffs– Disaster Recovery

• Part 3 in late April– Integrating with AWS Services– CloudFormation– Elastic MapReduce

Questions?

Resources

• MongoDB on AWS best practices:– http://docs.mongodb.org/ecosystem/platforms/am

azon-ec2/#deploy-mongodb-on-ec2

• MongoDB on AWS Marketplace:– https://aws.amazon.com/marketplace/seller-profile

/ref=dtl_pcp_sold_by?ie=UTF8&id=c9032c7b-70dd-459f-834f-c1e23cf3d092

• MongoDB docs– http://docs.mongodb.org

MongoDB WorldNew York City, June 23-25

#MongoDBWorld

See what’s next in MongoDB including • MongoDB 2.6• Sharding• Replication• Aggregation

http://world.mongodb.comSave 25% with discount code 25SandeepParikh