Recovery
description
Transcript of Recovery
![Page 1: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/1.jpg)
03/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery
![Page 2: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/2.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Integrity or correctness of data
Would like data to be “accurate” or “correct” at all times
EMP
Name
WhiteGreenGray
Age
523421
1
![Page 3: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/3.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Integrity or consistency constraints
Predicates data must satisfy Examples:
- x is key of relation R- x y holds in R- Domain(x) = {Red, Blue, Green}is valid index for attribute x of R- no employee should make more than twice the average
salary
![Page 4: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/4.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Definition:
Consistent state: satisfies all constraints Consistent DB: DB in consistent state
![Page 5: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/5.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Observation: DB cannot be consistent always!
Example: a1 + a2 +…. an = TOT (constraint)
Deposit $100 in a2: a2 a2 + 100
TOT TOT + 100
![Page 6: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/6.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
a2
TOT
.
.50
.
.1000
.
.150
.
.1000
.
.150
.
.1100
Example: a1 + a2 +…. an = TOT (constraint)Deposit $100 in a2: a2 a2 + 100
TOT TOT + 100
![Page 7: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/7.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Transaction: collection of actions that preserve consistency
Consistent DB Consistent DB’T
![Page 8: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/8.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
How can we prevent/fix violations?
cc: due to data sharing only recovery: due to failures only
![Page 9: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/9.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Will not consider:
How to write correct transactions How to write correct DBMS Constraint checking & repair
That is, solutions studied here do not need to know constraints
![Page 10: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/10.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Storage hierarchy
Memory Disk
x x
![Page 11: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/11.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Operations:
Input (x): block with x memory Output (x): block with x disk
Read (x,t): do input(x) if necessary t value of x in blockWrite (x,t): do input(x) if necessary value of x in block t
![Page 12: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/12.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Key problem Unfinished transaction
Example Constraint: A=B
T1: A A 2
B B 2
![Page 13: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/13.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
T1: Read (A,t); t t2Write (A,t);Read (B,t); t t2Write (B,t);Output (A);Output (B);
A: 8B: 8
A: 8B: 8
memory disk
1616
16
failure!
![Page 14: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/14.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
T1: Read (A,t); t t2 A=BWrite (A,t);Read (B,t); t t2Write (B,t);Output (A);Output (B);
A:8B:8
A:8B:8
memory disk log
Undo logging
1616
<T1, start><T1, A, 8>
<T1, commit>16 <T1, B, 8>
16
![Page 15: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/15.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
One “complication” Log is first written in memory Not written to disk on every action
memory
DB
LogA: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8>
A: 8B: 8
16BAD STATE
# 1
![Page 16: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/16.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
One “complication” Log is first written in memory Not written to disk on every action
memory
DB
LogA: 8 16B: 8 16Log:<T1,start><T1, A, 8><T1, B, 8><T1, commit>
A: 8B: 8
16BAD STATE
# 2
<T1, B, 8><T1, commit>
...
![Page 17: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/17.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Undo logging rules
(1) For every action generate undo log record (containing old value)
(2) Before x is modified on disk, log records pertaining to x must be on disk (write ahead logging: WAL)
(3) Before commit is flushed to log, all writes of transaction must be reflected on disk
![Page 18: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/18.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery rules: Undo logging
(1) Let S = set of transactions with <Ti, start> in log, but no
<Ti, commit> (or <Ti, abort>) record in log
(2) For each <Ti, X, v> in log,
in reverse order (latest earliest) do:
- if Ti S then - write (X, v)
- output (X)
(3) For each Ti S do
- write <Ti, abort> to log
![Page 19: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/19.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
What if failure during recovery?
No problem! Undo idempotent
![Page 20: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/20.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
To discuss: Redo logging Undo/redo logging, why both? Checkpoints
![Page 21: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/21.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Redo logging (deferred modification)
T1: Read(A,t); t t2; write (A,t);
Read(B,t); t t2; write (B,t);
Output(A); Output(B)
A: 8B: 8
A: 8B: 8
memory DB LOG
1616
<T1, start><T1, A, 16><T1, B, 16>
<T1, commit>
output16
![Page 22: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/22.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Redo logging rules
(1) For every action, generate redo log record (containing new value)
(2) Before anything is modified on disk (DB), all log records for transaction that modify things (including commit) must be on disk
![Page 23: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/23.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
(1) Let S = set of transactions with <Ti, commit> in log
(2) For each <Ti, X, v> in log, in forward
order (earliest latest) do:
- if Ti S then Write(X, v) Output(X)
Recovery rules: Redo logging
![Page 24: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/24.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery is very, very SLOW !
Redo log:
First T1 wrote A,B Last
Record Committed a year ago Record
(1 year ago) --> STILL, Need to redo after crash!!
... ... ...
Crash
![Page 25: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/25.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Undo Recovery – Better?
The first record without commit or abort should not be too far away from crash
But immediate update itself is expensive
... ... ...
first record without commit or abort
![Page 26: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/26.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Solution: Checkpoint (simple version)
Periodically:
(1) Do not accept new transactions
(2) Wait until all transactions finish
(3) Flush all log records to disk (log)(4) Flush all buffers to disk (DB) (do not discard buffers)
(5) Write “checkpoint” record on disk (log)
(6) Resume transaction processing
![Page 27: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/27.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Redo log (disk):
<T1
,A,1
6>
<T1
,com
mit
>
Ch
eck
poin
t
<T2
,B,1
7>
<T2
,com
mit
>
<T3
,C,2
1>
Crash... ... ... ...
...
...
![Page 28: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/28.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Undo log (disk):
<T1
,A,8
>
<T1
,com
mit
>
Ch
eck
poin
t
<T2
,B,1
7>
<T2
,com
mit
>
<T3
,C,2
1>
Crash... ... ... ...
...
...
![Page 29: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/29.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Nonquiescent Checkingpointing
“Quiescent” checkpointing stalls DB “Nonquiescent” checkpointing admits new transactions
while checkpointing
![Page 30: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/30.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Nonquiescent Checkpoint Rules - undo
Write and flush a log record <START CKPT(T1, …, Tk)> where T1,…,Tk are active transactions
When all T1, …, Tk have completed, write and flush a log record <END CKPT>
![Page 31: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/31.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Undo log (disk):
<T1
,A,8
>
<C
KPT(T
1,T
2)>
<T3
,B,1
7>
<T1
,com
mit
>
<T3
,C,2
1>
Crash... ... ...
...
<E
ND C
KPT>
<T2
,B,1
2>
<T2
,com
mit
>
Crash after end of checkpoint
Only need to undo all the incomplete transactions started after <START CKPT>
![Page 32: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/32.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Undo log (disk):
<T1
,A,8
>
<C
KPT(T
1,T
2)>
<T3
,B,1
7>
<T1
,com
mit
>
<T3
,C,2
1>
Crash... ... ...
... …
<T2
,B,1
2>
<T2
,com
mit
>
Crash before end of checkpoint
Only need to undo all incomplete transactions started after <START CKPT> and those in <CKPT (…)>
![Page 33: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/33.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Nonquiescent Checkpoint Rules - redo
Write and flush a log record <START CKPT(T1, …, Tk) where T1,…,Tk are active transactions
Flush all the updates of the transactions committed before <START CKPT(T1,…,Tk)>
Write and flush a log record <END CKPT>
![Page 34: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/34.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Redo log (disk):
<T1
,A,8
>
<C
KPT(T
1,T
2)>
<T3
,B,1
7>
<T1
,com
mit
>
<T3
,C,2
1>
Crash... ... ...
...
<E
ND C
KPT>
<T2
,B,1
2>
<T2
,com
mit
>
Crash after end of checkpoint
Only need to redo all the transactions committed after the latest <START CKPT>
![Page 35: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/35.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example: what to do at recovery?
Redo log (disk):
<T1
,A,8
>
<C
KPT(T
1,T
2)>
<T3
,B,1
7>
<T1
,com
mit
>
<T3
,C,2
1>
Crash... ... ...
... …
<T2
,B,1
2>
<T2
,com
mit
>
Crash before end of checkpoint
Only need to redo all the transactions committed after the latest successful <START CKPT>
![Page 36: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/36.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Key drawbacks:
Undo logging: need to update immediately, increasing I/O Redo logging: need to keep all modified blocks in memory until
commit
![Page 37: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/37.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Solution: undo/redo logging!
Update <Ti, Xid, New X val, Old X val>
![Page 38: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/38.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Rules
Page X can be flushed before or after Ti commit Log record flushed before corresponding updated
page (WAL) Flush log at commit
![Page 39: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/39.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Non-quiesce checkpoint
LOG
for undo
Start-ckptactive TR:
Ti,T2,...
endckpt
.........
... Flush
dirty buffers
![Page 40: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/40.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Examples what to do at recovery time?
no T1 commit
LOG
T1,-a
...CkptT1
...Ckptend
...T1-b
...
Undo T1 (undo a,b)
![Page 41: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/41.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example
LOG
...T1
a... ...
T1
b... ...
T1
c...
T1
cmt...
Ckptend
ckpt-sT1
Redo T1: (redo b,c)
![Page 42: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/42.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Examples what to do at recovery time?
no T1 commit
LOG
T1,-a
...CkptT1
... ...T1-b
...
Undo T1 (all the actions)
![Page 43: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/43.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Example
LOG
...T1
a... ...
T1
b... ...
T1
c...
T1
cmt...
ckpt-sT1
Redo T1: (redo b,c, a and the actions after last successful ckp-s)
![Page 44: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/44.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery process:
Backwards pass (end of log -> latest checkpoint start) construct set S of committed transactions undo actions of transactions not in S
Undo pending transactions follow undo chains for transactions in
(checkpoint active list) - S Forward pass (latest successful checkpoint start -> end of log)
redo actions of S transactions
backward pass
forward passstart
check-point
![Page 45: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/45.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Recovery with Checkpoint Summary
Logging still obey the logging rules undo, redo, undo-redo
Recovery still obey rules Undo: undo all incomplete transactions Redo: redo all completed transactions Undo-redo: combined
Checkpoint provides a way to limit the transactions needing undo or redo
![Page 46: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/46.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Summary
undo redo Undo-redo
commit commit means all updates on disk
Commit means some updates may start to migrate to disk
Commit does not signal disk update
recovery Undo all incomplete transactions
Redo all committed transactions
Undo all incomplete transactions and redo all committed transactions
Nonquiescent ckpt
<end ckpt(T)> means every tran started before the matching <start ckpt (T)> is on disk
<end ckpt(T)> means every tran committed before the matching <start ckpt(T)> is on disk
<end ckpt(T)> means every update before the matching <start ckpt(T)> is on disk
WAL
![Page 47: Recovery](https://reader036.fdocuments.net/reader036/viewer/2022062806/56814f08550346895dbc9a32/html5/thumbnails/47.jpg)
13/30/2005 Yan Huang - CSCI5330 Database Implementation – Recovery
Summary
Consistency of data One source of problems: failures
- WAL