Amazon Web Services

Jared Rosoff @forjared

Agenda

• Single node– Instance sizing – Storage configuration – OS configuration

• Replica Set – Availability Zones – Disaster Recovery – Multi Data-Center

• Sharded Cluster

INSTANCES, STORAGE AND SINGLE NODE CONFIGURATION

Instance Sizing

32-bit = Don’t Use

32-bit = Don’t Use

Huge

Big

Small

Arbiter / Config

OS• Amazon OS now an option• Turn off atime• Raise file descriptor limits

cat >> /etc/security/limits.conf << EOF* hard nofile 65536* soft nofile 65536EOF

• DO NOT use large VM pages• Use ext4, xfs• Use RAID

– RAID10 on MongoD– RAID1 on ConfigbDB

• Warning! Known problems with Ubuntu 10.04 & EBS– https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/614853– https://bugzilla.kernel.org/show_bug.cgi?id=16991

A MongoDB Data Node

• 64-bit instance• More RAM == better • Run EXT4 or XFS file system• Turn off atime & diratime• EBS volumes in RAID10

mongod

EBS – RAID10

This is your typical database node. Used for actual data storage. We’ll use this icon to represent this

type of node throughout this prezo.

A MongoDB Config Server

• 64-bit instance– Micro is fine

• EBS volumes in RAID1 mongoc

EBS – RAID1

Used to store configuration data about sharded deployments. Deployed as a set of 3 servers. Relatively low load and storage requirements.

A MongoDB Arbiter

• Micro is fine • No storage requirements• Must be separate node from rest

of replica set

mongod --arbiter

Used to store configuration data about sharded deployments. Deployed as a set of 3 servers. Relatively low load and storage requirements.

REPLICA SET DESIGNS

Single Region Replica SetRegion

Availability Zone A Availability Zone B Availability Zone C

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

Region 2

Disaster Recovery SiteRegion 1

Availability Zone A Availability Zone B Availability Zone A

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

HiddenEBS – RAID10

Region 3

Multi Data Center

Availability Zone A

mongod

EBS – RAID10

Region 2

Availability Zone A

mongod

Priority 1EBS – RAID10

Region 1

Availability Zone A

mongod

EBS – RAID10

SHARDED CLUSTERS IN EC2

Single Region Sharded ClusterRegion

Availability Zone A Availability Zone B Availability Zone C

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongoc

EBS – RAID1

mongoc

EBS – RAID1

mongoc

EBS – RAID1

Region 2

Sharded with Disaster Recovery SiteRegion 1

Availability Zone A Availability Zone B Availability Zone A

mongod

EBS – RAID10

mongod

EBS – RAID10

mongodhidden

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongodhidden

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongodhidden

EBS – RAID10

mongoc

EBS – RAID1

mongoc

EBS – RAID1

mongoc

EBS – RAID1

Region 3

Multi Region Sharded

Availability Zone A

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongoc

EBS – RAID1

Region 2

Availability Zone A

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongoc

EBS – RAID1

Region 1

Availability Zone A

mongod

EBS – RAID10

mongod

EBS – RAID10

mongod

EBS – RAID10

mongoc

EBS – RAID1

SECURITY

Security GroupsComponent Default PortsMongoD 27017, 28017 (Status page)MongoD Shard Server 27018MongoS 27017ConfigDB 27019

• All nodes listen on all interfaces unless –bind_ip is specified

• Specify alternate port with –port • Don’t mix names & IP’s. Names preferred (not ec2s)• Use security groups as access control lists • (e.g. allow 27017 from app servers, but not

everybody)

Region 1

Provisioning

Group: mongodbGroup: app-servers

$ ec2-create-group mongodb --description "security group for mongodb nodes”$ ec2-create-group app-servers --description "security group for app servers

## Allow mongodb nodes to talk to each other $ ec2-authorize mongodb –o mongodb –u <your ec2 account id>

## Allow app servers to talk to mongodb nodes $ ec2-authorize mongodb –o app-servers –u <your ec2 account id>

## Allow SSH access to MongoDB nodes $ ec2-authorize mongodb –p 22

## Allow remote MongoDB nodes to talk to cluster$ ec2-authorize mongodb –s 1.1.1.1 –p 27017$ ec2-authorize mongodb –s 2.2.2.2 –p 27017

Region 2

1.1.1.1

2.2.2.2

Questions?

• I am– Jared Rosoff (jsr@10gen.com)

• Try it out! – http://www.mongodb.org/downloads– http://www.mongodb.org/display/DOCS/Amazon

+EC2