ANHAI DOAN ALON HALEVY ZACHARY IVES Chapter 7: Data Matching PRINCIPLES OF DATA INTEGRATION.
Distributed Transactions Zachary G. Ives University of Pennsylvania CIS 455 / 555 – Internet and...
-
Upload
peregrine-hart -
Category
Documents
-
view
214 -
download
0
Transcript of Distributed Transactions Zachary G. Ives University of Pennsylvania CIS 455 / 555 – Internet and...
Distributed Transactions
Zachary G. IvesUniversity of Pennsylvania
CIS 455 / 555 – Internet and Web Systems
April 15, 2008
Administrivia
We are down to 2 weeks in the semester
I would like a status report on your projects by this Friday
2
Picking up: Vector Clocks, Briefly
Lamport clocks are logical clocks Each site maintains a clock value LC
Whenever an item publishes a timed item, use LC and postincrement it
Append clock on every published message When we receive a message with a bigger or equal LC timestamp,
say LC2, set our local LC := LC2 + 1 Causally ordered messages have bigger LC values than
their antecedents, but but a bigger LC value doesn’t mean that there’s any causal ordering…
Vector clocks: every node has a vector, representing the last Lamport clock value from each neighbor Key result: can compare two vector clocks to see if there
is a causal ordering between them
3
4
We Need More Than Synchronization
What needs to happen when you… Click on “purchase” on Amazon?
Suppose you purchased by credit card?
Use online bill-paying services from your bank? Place a bid in an eBay-like auction system? Order music from iTunes?
What if your connection drops in the middle of downloading?
Is this more than a case of making a simple Web Service (-like) call?
5
Transactions Are a Means of Handling Failures
There are many (especially, financial) applications where we want to create atomic operations that either commit or roll back
This is one of the most basic services provided by database management systems, but we want to do it in a broader sense
Part of “ACID” semantics…
6
ACID Semantics
Atomicity: operations are atomic, either committing or aborting as a single entity
Consistency: the state of the data is internally consistent
Isolation: all operations act as if they were run by themselves
Durability: all writes stay persistent!
7
A Problem Confronted by eBay
eBay wants to sell an item to: The highest bidder, once the auction is over, or The person who’s first to click “Buy It Now!”
But: What if the bidder doesn’t have the cash?
A solution: Record the item as sold Validate the PayPal or credit card info with a 3rd
party If not valid, discard this bidder and resume in prior
state
8
“No Payment” Isn’t the Only Source of Failure
Suppose we start to transfer the money, but a server goes down…
Purchase:sb = Seller.balbb = Buyer.balWrite Buyer.bal= bb - $100
Write Item.sellTo = Buyer
Write Seller.bal= sb + $100
CRASH!
9
Providing Atomicity and Consistency
Database systems provide transactions with the ability to abort a transaction upon some failure condition Based on transaction logging – record all operations and
undo them as necessary Database systems also use the log to perform
recovery from crashes Undo all of the steps in a partially-complete transaction Then redo them in their entirety This is part of a protocol called ARIES
These can be the basis of persistent storage, and we can use middleware like J2EE to build distributed transactions with the ability to abort database operations if necessary
10
The Need for Isolation
Suppose eBay seller S has a bank account that we’re depositing money into, as people buy:
What if two purchases occur simultaneously, from two different servers on different continents?
S = Accounts.Get(1234)Write S.bal = S.bal + $50
11
Concurrent Deposits
This update code is represented as a sequence of read and write operations on “data items” (which for now should be thought of as individual accounts):
where S is the data item representing the seller’s account # 1234
Deposit 1 Deposit 2Read(S.bal) Read(S.bal)S.bal := S.bal + $50 S.bal:= S.bal + €10Write(S.bal) Write(S.bal)
12
A “Bad” Concurrent Execution
Only one action (e.g. a read or a write) can actually happen at a time for a given database, and we can interleave deposit operations in many ways:
Deposit 1 Deposit 2Read(S.bal) Read(S.bal)S.bal := S.bal + $50 S.bal:= S.bal + €10Write(S.bal) Write(S.bal)
time
BAD!
13
A “Good” Execution
Previous execution would have been fine if the accounts were different (i.e. one were S and one were T), i.e., transactions were independent
The following execution is a serial execution, and executes one transaction after the other:
Deposit 1 Deposit 2Read(S.bal) S.bal := S.bal + $50 write(S.bal) Read(S.bal) S.bal:= S.bal + $10 Write(S.bal)
time
GOOD!
14
Good Executions
An execution is “good” if it is serial (transactions are executed atomically and consecutively) or serializable (i.e. equivalent to some serial execution)
Equivalent to executing Deposit 1 then 3, or vice versa Why would we want to do this instead?
Deposit 1 Deposit 3read(S.bal) read(T.bal)S.bal := S.bal + $50 T.bal:= T.bal + €10write(S.bal) write(T.bal)
15
Concurrency Control
A means of ensuring that transactions are serializable
There are many methods, of which we’ll see one Lock-based concurrency control (2-phase
locking) Optimistic concurrency control (no locks –
based on timestamps) Multiversion CC …
Lock-Based Concurrency Control
Strict Two-phase Locking (Strict 2PL) Protocol: Each transaction must obtain:
a S (shared) lock on object before reading an X (exclusive) lock on object before writing An owner of an S lock can upgrade it to X if no one else is
holding the lock
All locks held by a transaction are released when the transaction completes Locks are handled in a “growing” phase, then a
“shrinking” phase (Non-strict) 2PL Variant:
Release locks anytime, but cannot acquire locks after releasing any lock.
17
Benefits of Strict 2PL
Strict 2PL allows only serializable schedules. Additionally, it simplifies transaction aborts (Non-strict) 2PL also allows only serializable
schedules, but involves more complex abort processing
Aborting a Transaction
If a transaction Ti is aborted, all its actions have to be undone Not only that, if Tj reads an object last written by Ti, Tj
must be aborted as well!
Most systems avoid such cascading aborts by releasing a transaction’s locks only at commit time If Ti writes an object, Tj can read this only after Ti
commits
Actions are undone by consulting the transaction log mentioned earlier
The Transaction Log
The following actions are recorded in the log: Ti writes an object: the old value and the new value
Log record must go to disk before the changed page does!
Ti commits/aborts: a log record indicating this action
Log records are chained together by transaction id, so it’s easy to undo a specific transaction
Log is often mirrored and archived on stable storage
Another Benefit of the Log:Recovering From a Crash
3 phases in the ARIES recovery algorithm: Analysis
Scan the log forward (from the most recent checkpoint) to identify all pending transactions, unwritten pages
Redo Redo all updates to unwritten pages in the buffer
pool, to ensure that all logged updates are in fact carried out and written to disk
Undo Undo all writes done by incomplete transactions by
working backwards in the log (Care must be taken to handle the case of a crash
occurring during the recovery process!)
A Danger with Locks: Deadlocks
Deadlock: Cycle of transactions waiting for locks to be released by each other
Two ways of dealing with deadlocks: Deadlock prevention Deadlock detection
Deadlock Prevention
Assign priorities based on timestamps (older = higher)
Assume Ti wants a lock that Tj holds Do one of:
Wait-Die: If Ti has higher priority, Ti waits for Tj; otherwise Ti aborts
Wound-wait: If Ti has higher priority, Tj aborts; otherwise Ti waits
Higher-priority transactions never wait for lower-priority
If a transaction re-starts, make sure it has its original timestamp Keeps it from always getting aborted!
23
Database Transactions and Concurrency Control, Summarized
The basic goal was to guarantee ACID properties Transactions and logging provide Atomicity and
Consistency Locks ensure Isolation The transaction log (and RAID, backups, etc.) are
also used to ensure Durability
So far, we’ve been in the realm of databases – how does this extend to the distributed context?
24
Distributed Transactions
We generally rely on a middleware layer called application servers, aka TP monitors, to provide transactions across systems Tuxedo, iPlanet, WebSphere,
etc. For atomicity, two-phase
commit protocol For isolation, need distributed
concurrency control DB DB
TransactServer
TransactServer
WorkflowController
MsgQueue
WebServer
App Server
Client
Two-Phase Commit (2PC)
Site at which a transaction originates is the coordinator; other sites at which it executes are subordinates
Two rounds of communication, initiated by coordinator: Voting
Coordinator sends prepare messages, waits for yes or no votes
Then, decision or termination Coordinator sends commit or rollback messages, waits for
acks Any site can decide to abort a transaction!
26
Steps in 2PC
When a transaction wants to commit: Coordinator sends prepare message to each subordinate Subordinate force-writes an abort or prepare log record
and then sends a no (abort) or yes (prepare) message to coordinator
Coordinator considers votes: If unanimous yes votes, force-writes a commit log record
and sends commit message to all subordinates Else, force-writes abort log rec, and sends abort message
Subordinates force-write abort/commit log records based on message they get, then send ack message to coordinator
Coordinator writes end log record after getting all acks
27
Illustration of 2PC
Coordinator Subordinate 1Subordinate 2force-write begin log entry
force-write prepared log entry
force-writeprepared log entry
send “prepare”send “prepare”
send “yes”send “yes”
force-writecommit log entry
send “commit”send “commit”
force-writecommit log entry
force-writecommit log entry
send “ack”send “ack”
writeend log entry
Comments on 2PC
Every message reflects a decision by the sender; to ensure that this decision survives failures, it is first recorded in the local log
All log records for a transaction contain its ID and the coordinator’s ID The coordinator’s abort/commit record also includes IDs
of all subordinates
Thm: there exists no distributed commit protocol that can recover without communicating with other processes, in the presence of multiple failures!
What if a Site Fails in the Middle?
If we have a commit or abort log record for transaction T, but not an end record, we must redo/undo T If this site is the coordinator for T, keep sending commit/abort
msgs to subordinates until acks have been received
If we have a prepare log record for transaction T, but not commit/abort, this site is a subordinate for T Repeatedly contact the coordinator to find status of T, then write
commit/abort log record; redo/undo T; and write end log record
If we don’t have even a prepare log record for T, unilaterally abort and undo T This site may be coordinator! If so, subordinates may send
messages and need to also be undone
Blocking for the Coordinator
If coordinator for transaction T fails, subordinates who have voted yes cannot decide whether to commit or abort T until coordinator recovers T is blocked Even if all subordinates know each other (extra
overhead in prepare msg) they are blocked unless one of them voted no
Link and Remote Site Failures
If a remote site does not respond during the commit protocol for trnasaction T, either because the site failed or the link failed: If the current site is the coordinator for T,
should abort T If the current site is a subordinate, and has not
yet voted yes, it should abort T If the current site is a subordinate and has
voted yes, it is blocked until the coordinator responds!
Observations on 2PC
Ack msgs used to let coordinator know when it’s done with a transaction; until it receives all acks, it must keep T in the transaction-pending table
If the coordinator fails after sending prepare msgs but before writing commit/abort log recs, when it comes back up it aborts the transaction
33
From Distributed Commits toDistributed Concurrency Control
What we saw were the steps involved in preserving atomicity and consistency in a distributed fashion
Let’s briefly look at distributed isolation (locking)…
Distributed Locking
How do we manage locks across many sites? Centralized: One site does all locking
Vulnerable to single site failure
Primary Copy: All locking for an object done at the primary copy site for this object Reading requires access to locking site as well as site
where the object is stored
Fully Distributed: Locking for a copy done at site where the copy is stored Locks at all sites holding the object being written
Distributed Deadlock Detection
Each site maintains a local waits-for graph A global deadlock might exist even if the local
graphs contain no cycles:
T1 T1 T1T2 T2 T2
SITE A SITE B GLOBAL
Three solutions: Centralized (send all local graphs to one site) Hierarchical (organize sites into a hierarchy and
send local graphs to parent in the hierarchy) Timeout (abort transaction if it waits too long)
36
Summary of Transactions and Concurrency
There are many (especially monetary) transfers that need atomicity and isolation
Transactions and concurrency control provide these features In a distributed, 3-tier setting they run in an Application
Server Similar features are provided in a 2-tier setting for
applications that run directly in the DBMS
Two-phase locking ensures isolation Two-phase commit is a voting scheme for doing
distributed commit