ARC : A Self-Tuning, Low Overhead Replacement...
Transcript of ARC : A Self-Tuning, Low Overhead Replacement...
ARC : A Self-Tuning, Low Overhead
Replacement Cache
Nimrod Megiddo, Dharmendra S. Modha
๋ฐ๋น ์ง๋ ์์ฝ์ธ๋ถ๊ธฐ
2
Introduction
Before Cache Mangement Problem
Demand Paging vs Prefetching Page
Input Sequence : A C F B D E โฆโฆ
โข Only Allocate when it requests
Demand Paging
Prefetching โข Predict Future for reasonable
heuristics ex) bring near pages
1
2
1
2
1
2Evict 1
+ How to bring?
Evict 1replacement
policy
Fault Occurs?
replacement policy+ complex method
3
Problem Description
Cache Management Problem on Demand Paging Model
โข Main and the only Concern : Hit ratio
Cache Replacement Algorithm
Goal : Better than LRU
โข Maximize hit rateโข Minimize Overhead
- Time - Space
4
Previous Work โ Offline Optimal
Beladyโs MIN
โข Replace Largest Forward distance
But, Who knows future?
Input Sequence : 5 0 1 2 0 3 1 2 5 2
Cache content
0 5
1 0
2 1
4
26
Evict!
5
Before Previous Work
Metric ARC
Recency O
Frequency O
Time Complexity O(1)
Scan Resistance O
โSelf-tunableโ O
6
Previous Work - Recency
LRU(Least Recently Used)
Input Sequence : 1 2 3 4 5 6 7
1 2
1
3
2
1
4
3
2
1
5
4
3
2
1
6
5
4
3
2
7
6
5
4
3
MRU
LRU
Data structure : Queue, Time complexity : O(1)
Old-Fashion, But Quite Good Performance
No Frequency Concerned
7
Previous Work - Frequency
LFU(Least Frequently Used)
1
2
3
4
5
33
31
30
29
32
1
2
3
4
5
34
31
30
29
32
1
2
3
4
6
34
31
30
1
32
1 6
Data structure : Priority Queue, Time complexity : O(logn)
No Recency Concerned
8
LRU-2(LFU Extension)
Previous Work - Frequency
How to define โFrequentโ?
Input Sequence : 5 0 1 2 0 3 1 2 5 2 7
Inter-arrival rate
Inter-arrival rate High-frequent !
0
1
2
3
5
9
3
โ
10
8
Time tick : 1 2 3 4 5 6 7 8 9 10
2nd Backward Distance
Evict!
ProblemCIP(Correlated Information Period)
๐ฅ0, โฆ , ๐ฅ๐ , ๐ฅ๐+1, โฆ , ๐ฅ๐
How far do we have to consider 2 pages are related?
9
LRU-2(LFU Extension)
Previous Work - Frequency
Hard to Predict best CIP Value only on 1 workload !
Metric LRU-2
Time Complexity O(logn)
Self-tunable X
10
2Q(LRU-2 Extension)
Previous Work - Frequency
Metric 2Q
Time Complexity O(1)
Self-tunable X
๐จ๐(LRU)
๐จ๐๐
๐จ๐๐๐
๐ฒ๐๐ = ๐. ๐๐๐ฒ๐๐๐ = ๐.๐๐Input Sequence : 5 0 1 2 0 3 1 2 5 2 7
Still Have to tune some parameters
11
LIRS(Low Inter-reference Recency Set)
Previous Work - Frequency
Metric LIRS
Time Complexity Stack-Pruning
Self-tunable X
Input Sequence : 1 4 2 3 2 1 4 1 5
5
4
3
2
1
LIRS stack
๐ณ๐๐๐๐ + ๐ณ๐๐๐๐ = ๐๐ณ๐๐๐๐ = ๐. ๐๐๐๐
Still Have to tune some parameters
5
HIRS
stack
๐ = ๐๐ณ๐๐๐๐ = ๐๐ณ๐๐๐๐ = ๐
12
Previous Work - Frequency + Recency
FBR(Frequency Based replacement)
๐ท๐๐๐๐๐๐๐๐ ๐๐๐๐๐ ๐๐ ๐ ๐บ๐๐๐๐๐๐๐
๐จ๐๐๐(predetermined)
๐ช๐๐๐(predetermined)
Metric FBR
Self-tunable X
2(1) 0(1) 1(1) 5(1) 4(1) 9(2)
New Middle Old
Input Sequence : 0 5 11
Still Have to tune some parameters
13
Previous Work - Frequency + Recency
LRFU(Least Recently Frequently Used)
๐ญ ๐ =๐
๐
๐๐
Input Sequence : 1 4 2 3 2 1 4 1 5
Time Tick : 1 2 3 4 5 6 7 8 9 10
๐๐๐ ๐ = F(10-1) + F(10-6) + F(10-8)
๐๐ ๐ โ ๐, ๐ญ ๐ = ๐
๐๐ ๐ โ ๐, ๐ญ ๐ =๐
๐
๐
Metric LRFU
Self-tunable X
Still Have to tune some parameters
๐ณ๐ญ๐ผ
๐ณ๐น๐ผ
๐, ๐
1 >1
2+1
4+1
8+โฏ
14
Previous work - MQ
โข Multi-Queue replacement policy
โข Each ๐๐ contains [2๐,2๐+1) times seen pages
โข Each hit page is assigned lifetime
โข If a page expired, it goes down to lower-level queueโs MRU position
T = 3, lifeTime = 5 => expireTime = 8
๐๐
๐๐โ1
T = 8
โข Parameter tuning neededโข # of queues, lifeTime, โฆ
โข Overhead for checking timestamp
15
Terms and basics
โ(c) : Replacement Policies which manage c pages
๐ฟ1 : Set of pages seen only once recently
๐ฟ2 : Set of pages seen more than once recently
0 โค ๐ฟ1 + ๐ฟ2 โค 2๐, 0 โค ๐ฟ1 โค ๐, 0 โค ๐ฟ2โค 2๐
Replace ๐ฟ1 when ๐ฟ1 is full, if not, replace ๐ฟ2-> make | ๐ฟ1 | and | ๐ฟ2| similar
16
DBL(2c)
Cache hit : MRU update
Cache miss : Case A : ๐ฟ1 is full -> replace LRU of ๐ฟ1Case B : ๐ฟ1 isnโt full
Case B-1 : ๐ฟ1 + ๐ฟ2 = 2๐ (cache full)-> replace LRU of ๐ฟ2
Case B-2 : ๐ฟ1 + ๐ฟ2 < 2๐ (cache isnโt full)-> insert into ๐ฟ1
Summary : Fill ๐ฟ1 up to ๐
17
ฯ(c) โ general structure
๐1๐ : Top(MRU) of ๐ฟ1
๐ต1๐ : Bottom(LRU) of ๐ฟ1
๐2๐ : Top(MRU) of ๐ฟ2
๐ต2๐ : Bottom(LRU) of ๐ฟ2
โข ๐๐๐ โฉ ๐ต๐
๐ = โ , ๐ฟ๐ = ๐๐๐ โช ๐ต๐
๐
โข |๐ฟ1 โช ๐ฟ2| < c => ๐ต1๐ โช ๐ต2
๐ = โ
โข |๐ฟ1 โช ๐ฟ2| โฅ c => ๐1๐ โช ๐2
๐ = ๐
โข MRU(๐ต) < LRU(๐)
โข Cache contains ๐1๐ โช ๐2
๐
18
LRU(c)
โข In DBL(2c)
โข If a page in ๐ฟ1 evicted
โข ๐ฟ1 must be full ( ๐ฟ1 = ๐)
โข If a page in ๐ฟ2 evicted
โข ๐ฟ2 โฅ ๐
=> Evicted page is at most ๐ MRU
Both DBL(2c) and LRU(c) properly keep c MRU (proved)
19
๐ญ๐น๐ช๐ท ๐ (Fixed Replacement Cache)
โข ๐ is tunable parameter (0 โค ๐ โค ๐)
โข Follows all properties of ฯ(c)
โข Keeps ๐1 = ๐, ๐2 = ๐ โ ๐
โข Replacement Policy
โข If ๐1 > ๐ : Replace LRU of ๐1
โข If ๐1 < ๐ : Replace LRU of ๐2
โข If ๐1 = ๐ : (arbitrary)
20
ARC(๐) (Adaptive Replacement Cache)
โข Parameter ๐ adaptively changes (No tuning needed)
โข Cache miss and the page was in ๐ต1 => increase |๐1| (increase ๐)
โข Cache miss and the page was in ๐ต2 => increase |๐2| (decrease ๐)
โข The amount of change in parameter ๐ is learning rate
โข If ๐ต1 contains missed page => we should increase ๐
โข If ๐ต1 โฅ |๐ต2| : increase ๐ by 1
โข If ๐ต1 < |๐ต2| : increase ๐ by ๐ต2
๐ต1
โข Similar in ๐ต2 hit case.
21
ARC(๐) - algorithm
(๐ฅ : Current requested stream)
โข Case 1 : ๐ฅ โ (๐1 โช ๐2)
โข Cache hit. Move ๐ฅ into ๐2 and update MRU
โข Case 2 : ๐ฅ โ ๐ต1
โข Increase ๐ by learning rate
โข Move ๐ฅ into ๐2 and update MRU
โข Case 3 : ๐ฅ โ ๐ต2
โข Decrease ๐ by learning rate
โข Move ๐ฅ into ๐2 and update MRU
22
ARC(๐) - algorithm
โข Case 4 : ๐ฅ โ (๐ต1 โช ๐ต2)
โข Case 4-A : |๐1 โช ๐ต1| = ๐
โข Case 4-A-1 : |๐1| < ๐
โข Replace LRU of ๐ต1
โข Case 4-A-2 : |๐1| = ๐
โข ๐ต1 is empty -> Replace LRU of ๐1
โข Case 4-B : |๐1 โช ๐ต1| < ๐
โข If |๐1| + ๐2 + ๐ต1 + ๐ต2 โฅ ๐
โข Replace LRU of ๐ต2 if |๐1| + ๐2 + ๐ต1 + ๐ต2 = 2๐
โข Finally : Move ๐ฅ into ๐1 and update MRU
23
ARC(๐,๐)
โข Keeps recent ๐ pages discarded from ๐ฟ1 (not from ๐ฟ2)
โข All pages in the keeping set are more recent than new ๐ฟ2 pages
โข Otherwise, discard the page from keeping set
โข If request hits the keeping set, move that page into ๐2 directly
โข Previous algorithm denoted ARC(๐, 0)
24
Experimental result
25
Experimental result
26
Conclusion
โข No parameter tuning needed
โข Empirically universal
โข Less overhead
โข Empirically good performance
โข Robust, Scan-resistance
LRU LFU LRU-2 LIRS FBR LRFU MQ ARC
Recency O X X X O O O O
Frequency X O O O O O O O
ConstantTime
O O X O O - - O
ScanResistance
X X O O O O O O
โSelf-tunableโ
O O X X X X X O