Prometheus and Io. Prometheus Gave fire to man Gave fire to man Chained to a rock Chained to a rock…
Using chained transactions for maximum concurrency under load (QCONSF 2010)
-
Upload
billy-newport -
Category
Technology
-
view
4.026 -
download
3
description
Transcript of Using chained transactions for maximum concurrency under load (QCONSF 2010)
@billynewport 1
Thoughts on Transactions
Chained transactions for scaling ACID@billynewport
@billynewport 2
AgendaExplain scenario
Implement using a single database
Explain concurrency issues under load
Implement using a sharded database
Implement using WebSphere eXtreme Scale and chained transactions
@billynewport 3
ScenarioLarge eCommerce web site
Problem is order checkout
We track inventory levels for each SKU
Orders during checkout need to adjust available inventory.
@billynewport 4
Shopping cart metricsMillions of SKUs
Cart size of 5 items for electronics/big ticket items
Cart size of 20 items for clothing
Expect concurrent load of 2500 checkouts per second
@billynewport 5
DatabaseBegin
For each item in cartSelect for update where sku = item.skuDecrement available sku levelIf not available then rollback…Update level where sku = item.sku
Commit
Cart items randomly distributed amongst all 2m items, lots of concurrency.
Simple enough, right? All is good?
@billynewport 6
Problem: cabbage patch dolls
Cabbage patch dolls are popular this fall…
@billynewport 7
Database killers!The dolls cause major
concurrency problems Lots of row level locks Contention on doll rows Possible table lock escalation App server thread issues Connection pools empty Then DEATH!
They aren’t sweet and cuddly any more…
@billynewport 8
Database killersWe need a way to get locks to decrement
inventory
But, we don’t want to hold the lock for very long
Bigger carts make the problem worse, all the locks held for longer
Ideally, hold locks for constant time
Any contentious items make problem worse
@billynewport 9
SolutionHold lock on inventory rows for as short a time
as possible
Decouple this from size of cart.
How?
@billynewport 10
Chained transactions
Programmers think of transactions in synchronous terms.Begin / Do Work / Hold locks / Commit
Easy to program, bad for concurrency.
@billynewport 11
InspirationMicrosoft had COM objects with apartment
model threading.
Modern Actor support is similar. Some state with a mailbox.
BPEL supports flows with compensation
Data meets actors is a good analogySend a message (cart) to a group of actors
identified using their keys with a compensator
@billynewport 12
AlternativeWe need to think asynchronously in terms of flows
with compensation Map of <SKU Key/SKU Amount> Brick:
Do{ code to reduce inventory level for SKU }
Undo{ code to increase level inventory for SKU }
Provide Map with do/undo bricks
Easy to program, great concurrency.
@billynewport 13
Transactions and sharded stores
Option 1: Write transaction to one shard then spread out asynchronously
Option 2: 2 phase commit
Option 3: Chained transactions
@billynewport 14
Transactions and sharded stores
Option 1: Write transaction to one shard then spread out asynchronously
Option 2: 2 phase commit
Option 3: Chained transactions
@billynewport 15
Transactions and sharded stores
Option 1: Write transaction to one shard then spread out asynchronously
Option 2: 2 phase commit
Option 3: Chained transactions
@billynewport 16
Implementation1PC only required
Data store supportingRow locksRow oriented data
Integrated FIFO messaging
IBM WebSphere eXtreme Scale provides these capabilities.
@billynewport 17
ImplementationApplication makes map and code bricks
Submits transaction as an asynchronous job.
Uses a Future to check on job outcome.
Do blocks can trigger flow reversal if a problem occurs. Invoke undo block for each completed step
@billynewport 18
MechanismLoop
Receive message for actor keyProcess itSend modified cart to next ‘sku’ using local
‘transmit q’Commit transaction
Background thread pushes messages in transmit q to the destination shards using exactly once semantics.
@billynewport 19
PerformanceA single logical transaction will be slower than a
1PC DBMS implementation.
However, under concurrent load then it will deliver:Higher throughputBetter response times
Thru better contention managementEach ‘SKU’ only locked for a very short period
@billynewport 20
GeneralizationThis could be thought of as a workflow engine.
But, a big difference here is that a workflow engine usually talks with a remote store.
Here: the flow state is the MESSAGE It moves around to where the data is for the next step
Using a MESSAGE for flow state rather than a database means it scales linearly.
The message ‘store’ is integrated and scales with the data store.
@billynewport 21
Architecture ComparisonConventional Message oriented
FlowDB
ApplDB
MsgStor
e
BPEngine
BPEngine
BPEngin
e
FlowState
Flow Edge
= Msg
IntegratedMsg/Data store
ApplDB
Write behind
IntegratedMsg/Data store
IntegratedMsg/Data store
@billynewport 22
Sample implementationComing soon.
Running in lab
Working with several eCommerce customers looking to implement soon.
Soon to be published on github as sample code.