© 2016 Percona1

Aayushi Mangal

Transactions with MongoDB ShellMongoDB Transactions are here from version 4.0 -replica set! MongoDB Transactions are now possible with Shards also from version 4.2!

Support Engineer - Percona Global Services

© 2019 Percona2

Agenda▪ Transactions ▪ Multi Document Transactions in MongoDB ▪ Lamport Logical clock ▪ Prerequisites ▪ Simple Atomic Transaction ▪ Commit Transactions (To save data changes) ▪ Abort Transactions (To discard data changes) ▪ Write conflicts ▪ What’s new ▪ Considerations ▪ Good References (Percona’s blogs on Session and Transaction)

© 2019 Percona3

Transactions• All data changes are saved if transaction got committed.

• All data changes are discarded if any of the operation got failed in transaction.

• Without a successful transaction no changes are visible to outside the transaction

© 2019 Percona4

Multi-Document Transactions in MongoDB• Prior to 4.0 MongoDB provides acid guarantees for single

documents.

• For Eg: If a single write operation include db.collection.updateMany(), changes of each document is atomic.

• From version 4.0, these operations as a whole could be atomic.

• Multi-document transactions can be used across multiple documents, collections or operations.

• To maintain data integrity it follows all or nothing execution.

© 2019 Percona5

Lamport's Logical ClockAlgorithm:

1) Counter is incremented by 1 before each next event of that process

2) Each previous events sends a message that includes counter value along with message

time=time+1

send(message+time)

© 2019 Percona6

Without Lamport Logical Clock Algorithm

Primary Secondary1 Secondary2

MongoDB replica set

D1 D2 D3 D1 D1 D2 D2 D3 D3 Read document D1, but old value

Document D1 updated

© 2019 Percona7

With Lamport Logical Clock Algorithm

Primary Secondary1 Secondary2

MongoDB replica set

D1 D2 D3 D1 D1 D2 D2 D3 D3 Read request wait, gets updated document.

Document D1 updated

© 2019 Percona8

▪Must be replica set

Prerequisites

> use db1 switched to db db1 > db.a.insert({name:"abc"}) WriteResult({ "nInserted" : 1 }) > s1 = db.getMongo().startSession() session { "id" : UUID("f6e87fde-0254-49cb-8b5c-9a052ebe80dc") } > s1.startTransaction() > s1.getDatabase("db1").a.find({name:"abc"}) Error: error: { "ok" : 0, "errmsg" : "Transaction numbers are only allowed on a replica set member or mongos", "code" : 20, "codeName" : "IllegalOperation" }

© 2019 Percona9

▪ If Upgraded, from lower version, feature compatibility version must be 4.0

Prerequisites

transRepl:PRIMARY> use db1 switched to db db1 transRepl:PRIMARY> s1=db.getMongo().startSession() session { "id" : UUID("2689aa65-342f-4664-8966-4fb862e8e17e") } transRepl:PRIMARY> s1.startTransaction() transRepl:PRIMARY> s1.getDatabase("db1").a.insert({name:"john"}) WriteCommandError({ "operationTime" : Timestamp(1561453446, 1), "ok" : 0, "errmsg" : "Transactions are only supported in featureCompatibilityVersion 4.0. See http://dochub.mongodb.org/core/4.0-feature-compatibility for more information.", "code" : 50773, "codeName" : "Location50773", "$clusterTime" : { "clusterTime" : Timestamp(1561453446, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } })

© 2019 Percona10

▪ If Upgraded, from lower version, feature compatibility version must be 4.0

Prerequisites

transRepl:PRIMARY> db.adminCommand( { setFeatureCompatibilityVersion: '4.0' } ) transRepl:PRIMARY> db.adminCommand({getParameter:1, featureCompatibilityVersion:1}) { "featureCompatibilityVersion" : { "version" : “4.0”

transRepl:PRIMARY> use db1 switched to db db1 transRepl:PRIMARY> s1=db.getMongo().startSession() session { "id" : UUID("a2a4aa8c-2f13-46a9-acd4-dc5dfc33bd06") } transRepl:PRIMARY> s1.startTransaction() transRepl:PRIMARY> s1.getDatabase("db1").a.insert({name:"john"}) WriteResult({ "nInserted" : 1 }) transRepl:PRIMARY> s1.commitTransaction()

© 2019 Percona11

▪Multi Document transaction cannot involve an insert operation for the non-existing collection.

Prerequisites

transRepl:PRIMARY> s1.getDatabase("db1").a.insert({name:"john"}) WriteCommandError({ "operationTime" : Timestamp(1561453726, 1), "ok" : 0, "errmsg" : "Cannot create namespace db1.a in multi-document transaction.", "code" : 263, "codeName" : "OperationNotSupportedInTransaction", "$clusterTime" : { "clusterTime" : Timestamp(1561453726, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } } })

© 2019 Percona12

▪Transactions are only supported by wiredTiger storage Engine, why?

Prerequisites

transRepl:PRIMARY> db.serverStatus().storageEngine{ "name" : "mmapv1", "supportsCommittedReads" : false, "supportsSnapshotReadConcern" : false, "readOnly" : false, "persistent" : true}transRepl:PRIMARY> s1 = db.getMongo().startSession()transRepl:PRIMARY> s1.startTransaction()transRepl:PRIMARY> s1.getDatabase("db1").col1.find({name:"abc"})Error: error: { "operationTime" : Timestamp(1532248899, 1), "ok" : 0, "errmsg" : "Transaction numbers are only allowed on storage engines that support document-level locking", "code" : 20, "codeName" : "IllegalOperation",

© 2019 Percona13

▪Transactions are only supported by wiredTiger storage Engine, because:

Prerequisites

transRepl:PRIMARY> db.serverStatus().storageEngine{ "name" : "mmapv1", "supportsCommittedReads" : false, "supportsSnapshotReadConcern" : false, "readOnly" : false, "persistent" : true}transRepl:PRIMARY> s1 = db.getMongo().startSession()transRepl:PRIMARY> s1.startTransaction()transRepl:PRIMARY> s1.getDatabase("db1").col1.find({name:"abc"})Error: error: { "operationTime" : Timestamp(1532248899, 1), "ok" : 0, "errmsg" : "Transaction numbers are only allowed on storage engines that support document-level locking", "code" : 20, "codeName" : "IllegalOperation",

© 2019 Percona14

▪ Simple Atomic Transaction

Transactions

transRepl:PRIMARY> s1=db.getMongo().startSession() session { "id" : UUID("392d5337-9aeb-4572-bef4-aff680a93a0c") } transRepl:PRIMARY> s1.startTransaction() transRepl:PRIMARY> s1.getDatabase("transdb").col1.insert({"c":1}) WriteResult({ "nInserted" : 1 }) transRepl:PRIMARY> s1.commitTransaction() transRepl:PRIMARY> s1.endSession()

© 2019 Percona15

▪Commit Transactions to save data changes

Transactions

transRepl:PRIMARY> db.col1.find() { "_id" : ObjectId("5d11fd1c68ea949eeb55e23e"), "name" : "tom" }

transRepl:PRIMARY> s1=db.getMongo().startSession() session { "id" : UUID("351a55cc-f8f5-413c-92e0-cb8fedaf9164") } transRepl:PRIMARY> s1.startTransaction()

transRepl:PRIMARY> s1.getDatabase("db1").col1.insert({name:"jerry"}) WriteResult({ "nInserted" : 1 })

transRepl:PRIMARY> s1.commitTransaction() transRepl:PRIMARY> s1.endSession()

transRepl:PRIMARY> db.col1.find() { "_id" : ObjectId("5d11fd1c68ea949eeb55e23e"), "name" : "tom" } { "_id" : ObjectId("5d12004068ea949eeb55e240"), "name" : "jerry" } //new document inserted

© 2019 Percona16

▪Abort Transactions to discard data changes

Transactions

transRepl:PRIMARY> db.col1.find() { "_id" : ObjectId("5d11fd1c68ea949eeb55e23e"), "name" : "tom" }

transRepl:PRIMARY> s1=db.getMongo().startSession() session { "id" : UUID("7b3a0baa-75e2-41b1-bbba-649cabfeb7b3") } transRepl:PRIMARY> transRepl:PRIMARY> s1.startTransaction() transRepl:PRIMARY> s1.getDatabase("col1").insert({name:"jerry"}) 2019-06-25T11:15:06.104+0000 E QUERY [js] TypeError: s1.getDatabase(...).insert is not a function : @(shell):1:1 transRepl:PRIMARY> s1.getDatabase("db1").col1.insert({name:"jerry"}) WriteResult({ "nInserted" : 1 }) transRepl:PRIMARY> transRepl:PRIMARY> transRepl:PRIMARY> s1.abortTransaction() transRepl:PRIMARY> s1.endSession() transRepl:PRIMARY> db.col1.find() { "_id" : ObjectId("5d11fd1c68ea949eeb55e23e"), "name" : "tom" } //no document inserted

© 2019 Percona17

WriteConflicts Transactions with write conflicts are aborted

transRepl:PRIMARY> s1=db.getMongo().startSession() transRepl:PRIMARY> s2=db.getMongo().startSession() transRepl:PRIMARY> s1.startTransaction() transRepl:PRIMARY> s2.startTransaction() transRepl:PRIMARY> s1.getDatabase("db1").col1.update({name:"tom"},{$set:{name:"tom1"}},{multi:true}) WriteResult({ "nMatched" : 0, "nUpserted" : 0, "nModified" : 0 }) transRepl:PRIMARY> s2.getDatabase("db1").col1.update({name:"tom"},{$set:{name:"tom1"}},{multi:true}) WriteCommandError({ "errorLabels" : [ "TransientTransactionError" ], "operationTime" : Timestamp(1561461965, 1), "ok" : 0, "errmsg" : "WriteConflict", "code" : 112, "codeName" : "WriteConflict", "$clusterTime" : { "clusterTime" : Timestamp(1561461965, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) }

© 2019 Percona18

mongos> db.version() 4.2.0-rc1

mongos> use db1 switched to db db1 mongos> db.a.insert({name:"abc"}) WriteResult({ "nInserted" : 1 }) mongos> s1 = db.getMongo().startSession() session { "id" : UUID("5126ba83-439b-437d-8089-66396edfe08d") } mongos> s1.startTransaction() mongos> s1.getDatabase("db1").a.insert({name:"abc"}) WriteResult({ "nInserted" : 1 }) mongos> s1.commitTransaction()

What’s New? ▪Transactions now possible with Sharded cluster.

© 2019 Percona19

▪Transaction size limit is 16MB

Considerations

transRepl:PRIMARY> s1.getDatabase("db1").col1.update({name:"abc"},{$set:{age:1}},{multi:true})WriteCommandError({ "operationTime" : Timestamp(1532233021, 1), "ok" : 0, "errmsg" : "Total size of all transaction operations must be less than 16793600. Actual size is 16793722", "code" : 257, "codeName" : "TransactionTooLarge", "$clusterTime" : { "clusterTime" : Timestamp(1532233021, 1), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }})

© 2019 Percona20

▪To use transactions for Replica set and shards drivers must be updated for MongoDB version 4.2

▪Read/Write is not possible for capped collections (from version 4.2)

▪Read/Write operations are also prohibited for administrations databases like config, admin or local

▪Non-CRUD operations like createUser, getParameter are restricted.

Considerations

© 2019 Percona21

▪To use transactions for Replica set and shards drivers must be updated for MongoDB version 4.2

▪Read/Write is not possible for capped collections (from version 4.2)

▪Read/Write operations are also prohibited for administrations databases like config, admin or local

▪Non-CRUD operations like createUser, getParameter are restricted.

Considerations

© 2019 Percona22

▪MongoDB 3.6 session explained https://www.percona.com/blog/2017/12/08/mongodb-3-6-sessions-explained/

▪Your very first transaction With MongoDB 4.0 https://www.percona.com/blog/2018/06/25/mongodb-transactions-your-very-first-transaction-with-mongodb-4-0/

▪MongoDB 4.0 ACID MultiDocument Transactions https://www.percona.com/blog/2018/12/04/mongodb-4-0-using-acid-multi-document-transactions/

Good References

© 2018 Percona23

Q and A