Growing MongoDB on AWS
-
Upload
colinthehowe -
Category
Technology
-
view
845 -
download
7
description
Transcript of Growing MongoDB on AWS
![Page 1: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/1.jpg)
Growing MongoDBon AWSColin Howe@colinhowe
![Page 2: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/2.jpg)
Who am I?
I run engineering at Conversocial.We help companies do social customer service.http://www.conversocial.com
Love coding, ultimate frisbee and pudding
![Page 3: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/3.jpg)
Growing MongoDB on AWS
Tips for ensuring you can grow MongoDB on AWS as your business needs grow.
Mandatory meaningless graph
![Page 4: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/4.jpg)
Single Server - Ephemeral
Single server with instance storage.Not really advised.
Must haves:● Journalling
http://www.mongodb.org/display/DOCS/Journaling
● Regular backupshttp://www.mongodb.org/display/DOCS/Backups
![Page 5: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/5.jpg)
Single Server - Ephemeral
Pros:Easy to configure.Cheap.
Cons:No redundancy.Data since last backup is lost on server death.
![Page 6: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/6.jpg)
Single Server - EBS
Same as the last but move storage over to EBS.
EBS is slow.RAID 10 it. http://bit.ly/mongodb-raid10
![Page 7: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/7.jpg)
Single Server - EBS
Pros:Easy to configure.Data should still be available on server death.Cheapish.
Cons:No redundancy.
![Page 8: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/8.jpg)
Single Server - No redundancy
![Page 9: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/9.jpg)
Enter Replication
Primary server replicates to secondary servers
Use multiple Availability Zones (AZs).
![Page 10: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/10.jpg)
Two servers, one arbiter
Cheapest replica set configuration.
Arbiter ensures an odd number of votes.
The arbiter can be a micro instance.
![Page 11: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/11.jpg)
Two servers, one arbiter
Pros:Cheapish.Some failover.
Cons:Only one failover. What if you fail again?
![Page 12: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/12.jpg)
Three servers
Standard replica set configuration.
![Page 13: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/13.jpg)
Three servers
Pros:Lots of redundancy.High read capacity when one server fails.
Cons:Starts to get pricey.
![Page 14: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/14.jpg)
Where next?
Horizontal and vertical scaling.
![Page 15: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/15.jpg)
Vertical scaling
Use EBS boot.
Turn the server off. Resize. Turn it back on.
Seconds of downtime when using a replica set.
Without replica sets you will have downtime
![Page 16: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/16.jpg)
Vertical scaling - SSDs
If you can afford them SSDs are the King.
● Conversocial got 43% faster.
● 90% iowait went to ~3% user, 0% iowait.
All numbers compared to instance with largest memory (68.4gb)
![Page 17: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/17.jpg)
Horizontal read scaling
Add more secondaries and use slave queries.
Really easy.
Make sure you have a hidden backup server. Backups kill read performance.
Use write concerns for consistent reads.
![Page 18: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/18.jpg)
Horizontal write scaling - Sharding
Fairly easy with MongoDB.
Backups get harder.
More to manage and configure. Automate it.
![Page 19: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/19.jpg)
When to Shard?
If you know your data structure then sooner
If you don't... later
Changing shard keys is not trivial
![Page 20: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/20.jpg)
Enterprise Availability
If you're hunting for the final decimal places of availability...
You're probably going to break your application more frequently than Amazon has a whole region outage.
But...
![Page 21: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/21.jpg)
Enterprise Availability
Multiple regions are the next step.
Harder as you go outside the firewall.
VPNs or SSH tunnels are your friend here.
![Page 22: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/22.jpg)
Tools to Help You
MMS - Mongo's Monitoring System (FREE)
Chef/Puppet - automate as much as you can
#mongodb and MongoDB user group
10gen
![Page 23: Growing MongoDB on AWS](https://reader034.fdocuments.net/reader034/viewer/2022052522/54b7a1734a79591d4a8b497e/html5/thumbnails/23.jpg)
Thanks for Listening!Any questions?
Feeling shy?You can always reach me on Twitter
@colinhowe
Slides will be put online