Transaction support for the Cloud: The Deuteronomy Approach

Justin Levandoski*University of Minnesota

David LometMicrosoft Research

Mohamed Mokbel*University of Minnesota

Kevin Zhao*University of California San DiegoDeuteronomy: Transaction Supportfor Cloud Data*work done while at Microsoft Research1Motivation2Want ACID transactions for data anywhere in cloud

Currently, cloud providers support:Transactions for data guaranteed to exist on the same nodeEventual consistency

Currently no support for transactions across the cloudApplication Motivation Transactions AnywhereThe CloudMy new mobile application

3

What We Want

Begin Transaction

1. Add me to Daves friend list 2. Add Dave to my friend list

End TransactionWhat We Have Today

Eventual consistency*

click

*Thanks to Divy Agrawal for example

4

Application Motivation Transactions AnywhereReason current transaction support wont work in this case: friend lists are partitioned by user, thus two users friend lists are never guaranteed to be on same node.

This is not the only application exampleShopping cart applications for any e-commerce site cannot think of using cloud storage (need transactions)-4Talk OutlineApplication Motivation

Technical Motivation

Deuteronomy Architecture

Transaction Component Implementation

Performance

Wrap Up5Technical MotivationCIDR 2009: Unbundling Transaction Services in the Cloud (Lomet et al)Partition storage engine into two componentsTransactional component (TC): transactional recovery and concurrency controlData component (DC): tables, indexes, storage, and cache

6TCDC1DC2CIDR 2011: Deuteronomy: Transaction Support for Cloud DataReduce number of round trips between TC and DCLarge latencies (Network overhead, context switching)No caching at the TC

ApplicationTechnical MotivationCIDR 2009 required logging before sending operation to DCDrawback: requires two messages and/or double logging to perform operation (e.g., update)7TCDC11. Request before image2. Return image3. Log operation (generateLSN)4. Send operation5. Perform operation6. Log operation successOur new protocol logs after operation returns from DCTCDC12. Send operation3. Perform operation and send back before image4. Log operation1. Locking (generate LSN)Must deal with LSNs out of order on logAlso required us to rethink TC/DC control protocolTalk OutlineApplication Motivation

Technical Motivation

Deuteronomy Architecture

Transaction Component Implementation

Performance

Wrap Up8Deuteronomy ArchitectureTransaction Component (TC)Guarantee ACID PropertiesNo knowledge of physical data storageLogical locking and logging

Physical data storageAtomic record modificationsData could be anywhere (cloud/local)StorageData Component (DC)RecordOperationsControl OperationsClient RequestInteraction ContractReliable messagingAt least once execution

IdempotenceAt most once execution

CausalityIf DC remembers message, TC must also

Contract terminationMechanism to release contract99

Cloud StorageLocal Storage. . .Session ManagerLock ManagerLog ManagerSession 1. . .Session NSession 2Record/ TableOperationsRecord/ TableOperationsData Component 2Record/ TableOperationsData Component NMeta-data ManagementTransaction ComponentCloud StorageSessionTableManagerRecordManagerData Component 1TC-DC Control OperationsTC-DC Control OperationsTC-DC Control OperationsTC-DC Control OperationsClient10

LogicallockingCloud StorageLocal StorageRecord/ Table operations. . .Session 1. . .Session NSession 2LogicalloggingRecord/ TableOperationsRecord/ TableOperationsData Component 2Record/ TableOperationsData Component NMeta-data ManagementTransaction ComponentCloud StorageSessionTableManagerRecordManagerData Component 1ClientSession ManagerLock ManagerLog ManagerThread forking and poolingThread aware and some thread managementProtect lock data from race condition.

Block threads for conflictProtect data structures

Block committing threads for log flush11Talk OutlineApplication Motivation

Technical Motivation

Deuteronomy Architecture

Transaction Component Implementation

Performance

Wrap Up1212Record Manager An Insert Operation Example13Receive request and dispatch a session thread

Call to lock managerLock resourceGenerate Log Sequence Number (LSN)

Sends LSN & operation to DC

Call to log managerLog operation with LSNTC Record ManagerLockManagerLogManagerDCClient & Session Managerinsert record1LockLSN2insert record, LSN3Log RecordLSN4123413Lock Manager14Deuteronomy locking hierarchy:

Support locking of range read by using partitions, not next key lockingSELECT * FROM Employees WHERE id > 10 AND id < 40Partitions to lock [0, 30], [30, 60]So that inserts DO NOT have to know or test the lock on next keyIn charge of generating LSN before sending operation to data components

TableRecordPartition14Log Manager15Different from conventional log manager in two aspects:Need to coordinate TCs log with DCs cacheWrite Ahead Logging: Which updates are allowed to be made stable at DCLog truncation: Which updates must be made stable at DCComplexity: LSNs are stored out-of-order on TCs log, but DC only understands LSNTCLSN 1LSN 2LSN 3LSN 4LSN 5DCDCDCTC Physical LogLSN 4LSN 2LSN 1LSN 3LSN 5Log flushedNot flushedWhat can DC write to stable storage?15Control Operations: End-Of-Stable-Log16TC sends an LSN value eLSN from TC to all DCsFor DC: Updates with LSN eLSN must not be made stable and must be forgettable.For TC: all log records with LSN