Concurrency Control in MongoDB 3.0
Transcript of Concurrency Control in MongoDB 3.0
![Page 1: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/1.jpg)
Concurrency Control in MongoDB 3.0
Kaloian ManassievSenior Engineer at MongoDB
[email protected]@kaloianm
![Page 2: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/2.jpg)
2
Audience
• Operations engineers• Application developers• Third-party storage engine developers• Anybody who is curious
![Page 3: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/3.jpg)
3
What is concurrency control?
![Page 4: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/4.jpg)
4
What is concurrency control?
• Locking• Data consistency
$100 + $100 = $200 (not $100 )• MVCC
![Page 5: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/5.jpg)
5
Collection Collection
![Page 6: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/6.jpg)
6
Collection Collection
Storage Engine API
![Page 7: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/7.jpg)
7
What you will learn
• Top level concurrency control–Instance, database and collection
• How top level cooperates with the storage engine
![Page 8: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/8.jpg)
8
What you will NOT learn
• WiredTiger internals–Separate session on this
![Page 9: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/9.jpg)
9
Talk outline
• MongoDB concurrency control• Multiple-granularity locking• WiredTiger vs MMAP V1• Questions
![Page 10: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/10.jpg)
10
MongoDB 2.0 (and before)
THETOP LOCK
R WR OK NOW NO NO
![Page 11: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/11.jpg)
11
MongoDB 2.2
Lock per database
db1.coll.insert({a:1}) db4.coll.find({c:5})
![Page 12: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/12.jpg)
12
MongoDB 2.2
THETOP LOCK
+ intents
![Page 13: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/13.jpg)
13
TOP
db1.coll.insert({a:1}) db4.coll.find({c:5})
IX IS
X S
![Page 14: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/14.jpg)
14
TOP
db.coll.insert({a:1}) db.coll.find({c:5})
IX IS
X S
![Page 15: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/15.jpg)
15
Why intents?
• Get rid of the global lock?
• Use read (shared) lock?
![Page 16: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/16.jpg)
16
TOP
db1.coll.insert({a:1}) db4.coll.find({c:5})
S S
X S
![Page 17: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/17.jpg)
17
TOP
db1.coll.insert({a:1}) db.fsyncLock({lock:1})
S S
X
![Page 18: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/18.jpg)
18
TOP
db1.coll.insert({a:1}) db.fsyncLock({lock:1})
IX S
X
![Page 19: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/19.jpg)
19
TOP
db1.coll.insert({a:1}) db.fsyncLock({lock:1})
IX S
XS S S S
![Page 20: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/20.jpg)
20
TOP
db.fsyncLock({lock:1})
S
S S S S
db1.coll.find({a:1})
IS
S
![Page 21: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/21.jpg)
21
Intents
• “Intention” to access one or more children of an item–Compatible with other intents–Need to acquire lock further down–No overhead
![Page 22: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/22.jpg)
22
Locks
• “Lock” an item for particular access–Item being locked–All items below it
![Page 23: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/23.jpg)
23
Intents compatibility
IS IX S XIS OK OK OK NOIX OK OK NO NOS OK NO OK NOX NO NO NO NO
![Page 24: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/24.jpg)
24
MongoDB 3.0
TOP
![Page 25: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/25.jpg)
25
Multiple-granularity locking
TOP
![Page 26: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/26.jpg)
26
Lock Manager
• Ensures the locking protocol is obeyed
• Very low overhead
• Testable
![Page 27: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/27.jpg)
27
Lock Manager
• Improved fairness
• Support for locking policies
• Lock statistics gathering
![Page 28: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/28.jpg)
28
TOP
1
db.coll1.update({a:1}) db.coll1.update({a:4})
IX IX
IX IX
IX IX
X X
![Page 29: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/29.jpg)
29
Storage Engine API
TOP
![Page 30: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/30.jpg)
30
Storage Engine API
TOP
Storage Engine API
![Page 31: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/31.jpg)
31
TOP
db.coll1.update({a:1}) db.coll1.update({a:4})
IX IX
IX IX
IX IX
WiredTiger MVCC
![Page 32: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/32.jpg)
32
TOP
db.coll1.update({a:1}) db.coll1.drop()
IX IX
IX X
IX
WiredTiger MVCC
![Page 33: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/33.jpg)
33
How does MMAP V1 work?
• Supports collection concurrency
• Takes lock on the collection–Instead of intent
![Page 34: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/34.jpg)
34
TOP
db.coll1.update({a:1}) db.coll2.update({a:4})
IX IX
IX IX
X X
MMAP V1
![Page 35: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/35.jpg)
35
TOP
db.coll1.update({a:1}) db.coll1.find({a:4})
IX IS
IX IS
X S
MMAP V1
![Page 36: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/36.jpg)
36
WiredTiger
• Intents: Global, Database, Collection
• Document-level concurrency left to the storage engine
![Page 37: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/37.jpg)
37
MMAP V1
• Intents: Global, Database
• Locks: Collection
![Page 38: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/38.jpg)
38
Conclusion
• Storage engines have direct control over concurrency–Completely decoupled from top-level locking
–Enabled document-level control
![Page 39: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/39.jpg)
39
Conclusion
• Retrofitted MMAP V1 to use multi-granularity locks–Support for collection-level locking–Got rid of the global lock usage for journaling
![Page 40: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/40.jpg)
QUESTIONS?
![Page 41: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/41.jpg)
EXTRA SLIDES
![Page 42: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/42.jpg)
42
![Page 43: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/43.jpg)
43
![Page 44: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/44.jpg)
44
![Page 45: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/45.jpg)
45
![Page 46: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/46.jpg)
46
![Page 47: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/47.jpg)
47
![Page 48: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/48.jpg)
48
![Page 49: Concurrency Control in MongoDB 3.0](https://reader033.fdocuments.net/reader033/viewer/2022061611/55c9a781bb61ebf51b8b45b1/html5/thumbnails/49.jpg)
49