Languages
Pages
Legal
Optimising for performance, scale, analytics
~3333 write ops/s 0.07 - 0.05 ms response
David Mytton
Woop Japan!
MongoDB at Server Density
•27 nodes
MongoDB at Server Density
• June 2009 - +3yrs
•27 nodes
MongoDB at Server Density
•MySQL -> MongoDB
•27 nodes
MongoDB at Server Density
• June 2009 - +3yrs
•MySQL -> MongoDB
•17TB data per month
•27 nodes
MongoDB at Server Density
• June 2009 - +3yrs
MongoDB at Server Density
Queues
Primary data store
Time series
Why?
Why?
• Replication
Why?
• Replication
• Official drivers
Why?
• Replication
• Official drivers
• Easy deployment
Why?
• Replication
• Official drivers
• Easy deployment
• Fast out of the box (sort of)
Fast out of the box?
Photo: dannychoo.com
~3333 write ops/s 0.07 - 0.05 ms response
Fast out of the box?
Photo: dannychoo.com
• Softlayer cloud (1 core, 8GB)
Fast out of the box?
Photo: dannychoo.com
• Local instance storage
• Softlayer cloud (1 core, 8GB)
Fast out of the box?
Photo: dannychoo.com
• Local instance storage
• Ubuntu 10/12.04 LTS
• Softlayer cloud (1 core, 8GB)
Fast out of the box?
Photo: dannychoo.com
• Journaling
Fast out of the box?
Photo: dannychoo.com
• Journaling
• Replication
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
mongos> db.metrics_20120508_15_1m.stats(){ "sharded" : true, "flags" : 1, "ns" : "metrics.metrics_20120508_15_1m", "count" : 2752934, "numExtents" : 46,
"size" : 746837640, "storageSize" : 823717888,
"totalIndexSize" : 517581680, "indexSizes" : { "_id_" : 130358144, "a_1_i_1" : 155711920, "a_1_i_1_m_1_t_1" : 231511616 }, "avgObjSize" : 271.2878841265355, "nindexes" : 3, "nchunks" : 61,
"size" : 746837640, "totalIndexSize" : 517581680
Indexes = 493MB Data = 712MB
"size" : 746837640, "totalIndexSize" : 517581680
Indexes = 493MB Data = 712MB
"size" : 746837640, "totalIndexSize" : 517581680
Indexes = 493MB Data = 712MB
Total = 1205MB
http://www.flickr.com/photos/comedynose/4388430444/
Where should it go?
What? Should it be in memory?
Indexes Always
Data If you can
How you’ll know
1) Slow queries
Thu Oct 14 17:01:11 [conn7410] update sd.apiLog query: { c: "android/setDeviceToken", a: 1466, u: "blah", ua: "Server Density Android" } 51926ms
www.flickr.com/photos/tonivc/2283676770/
How you’ll know
2) Timeouts
cursor timed out (20000 ms)
How you’ll know
3) Disk i/o spikes
www.flickr.com/photos/daddo83/3406962115/
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
• Fast disks (optional)
Growing documents is bad
Updates in place
db.my_collection.update( { _id : ... }, { $inc : { y : 2 } } )
BSON
BSON
Growing documents is bad
> db.coll.stats(){ "ns" : "...", ..., "paddingFactor" : 1, ..., "ok" : 1}
• Global DB lock
Scaling writes
• Concurrency
Scaling writes
• Global DB lock
http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html
http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
• Fast disks (optional)
• Sharding (optional)
• Sharding
• Concurrency
Scaling writes
• Global DB lock
Scaling writes
• Collection location
Scaling writes
• Collection location
Scaling writes
• Pre-split / moveChunk
• Collection location
Scaling writes
• Hashing (v2.4)
• Pre-split / moveChunk
Failover
•Replica sets
Failover
•Replica sets
•Master/slave
Failover
•Replica sets
•Min 3 nodes
•Master/slave
Failover
•Replica sets
•Min 3 nodes
•Master/slave
•Automatic failover
rs.status()
www.ex-astris-scientia.org/inconsistencies/ent_vs_tng.htm (yes it’s a replicator from Star Trek)
{! "_id" : 1,! "name" : "rs3b:27018",! "health" : 1,! "state" : 2,! "stateStr" : "SECONDARY",! "uptime" : 1886098,! "optime" : {! ! "t" : 1291252178000,! ! "i" : 13! },! "optimeDate" : ISODate("2010-12-02T01:09:38Z"), "lastHeartbeat" : ISODate("2010-12-02T01:09:38Z")},
1) myState
rs.status()
Value Meaning0 Starting up (phase 1)1 Primary2 Secondary3 Recovering4 Fatal error5 Starting up (phase 2)6 Unknown state7 Arbiter8 Down
2) Optime
rs.status()
www.flickr.com/photos/robbie73/4244846566/
"optimeDate" : ISODate("2010-12-02T01:09:38Z")
3) Heartbeat
rs.status()
www.flickr.com/photos/drawblindfaith/3400981091/
"lastHeartbeat" : ISODate("2010-12-02T01:09:38Z")
Scaling reads
Scaling reads
•Replica slaves
Scaling reads
•Replica slaves
•Consistency
•Replica slaves
Scaling reads
•Consistency
•w flag / tags
WriteConcern
Changed Nov 27 2012
• Safe by default
WriteConcern
>>> from pymongo import MongoClient>>> connection = MongoClient()
• Safe by default
WriteConcern
>>> from pymongo import MongoClient>>> connection = MongoClient(w=int/str)
Value Meaning
0 Unsafe
1 Primary
2 Primary + x1 secondary
3 Primary + x2 secondaries
Tags
{ _id : "someSet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}} ] settings : { getLastErrorModes : { veryImportant : {"dc" : 3}, sortOfImportant : {"dc" : 2} } }}> db.foo.insert({x:1})> db.runCommand({getLastError : 1, w : "veryImportant"})
{ _id : "someSet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}} ] settings : { getLastErrorModes : { veryImportant : {"dc" : 3}, sortOfImportant : {"dc" : 2} } }}> db.foo.insert({x:1})> db.runCommand({getLastError : 1, w : "veryImportant"})
(A or B) + (C or D) + E
Tags
{ _id : "someSet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}} ] settings : { getLastErrorModes : { veryImportant : {"dc" : 3}, sortOfImportant : {"dc" : 2} } }}> db.foo.insert({x:1})> db.runCommand({getLastError : 1, w : "sortOfImportant"})
(A + C) or (D + E) ...
Tags
• J
WriteConcern
• Safe by default
• J
• fsync
WriteConcern
• Safe by default
Bottlenecks
•EC2
Bottlenecks
•EC2
•Local storage
Bottlenecks
•EC2
•EBS: RAID10 4-8 volumes
•Local storage
Bottlenecks
•EC2
•EBS: RAID10 4-8 volumes
•Local storage
•i/o: rand but not sequential
http://www.slideshare.net/jrosoff/mongodb-on-ec2-and-ebs
http://bit.ly/ec2mongodb
Bottlenecks
•CPU
•Index building
Tips: rand()
•_id
Tips: rand()
•_id
•Field names
Tips: rand()
•_id
•Field names
•Covered indexes
Tips: rand()
•_id
•Field names
•Covered indexes
•Collections / databases
mongostat
Locks/Queues
mongostat
Diagnostics
mongostat
Current operations
www.flickr.com/photos/jeffhester/2784666811/
db.currentOp();{! ! ! "opid" : "shard1:299939199",! ! ! "active" : true,! ! ! "lockType" : "write",! ! ! "waitingForLock" : false,! ! ! "secs_running" : 15419,! ! ! "op" : "remove",! ! ! "ns" : "sd.metrics",! ! ! "query" : {! ! ! ! "accId" : 1391,! ! ! ! "tA" : {! ! ! ! ! "$lte" : ISODate("2010-11-24T19:53:00Z")! ! ! ! }! ! ! },! ! ! "client" : "10.121.12.228:44426",! ! ! "desc" : "conn"! ! },
Monitoring tools
Run yourself
Ganglia
Monitoring tools
Server Density
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
• bit.ly/benchrun
Woop Japan!
www.serverdensity.com/mdb
Top Related