Memento : Coordinated In-Memory Caching for Data-Intensive Clusters
description
Transcript of Memento : Coordinated In-Memory Caching for Data-Intensive Clusters
![Page 1: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/1.jpg)
1
Memento: Coordinated In-Memory Caching
for Data-Intensive Clusters
Ganesh Ananthanarayanan, Ali Ghodsi, Andrew Wang, Dhruba Borthakur, Srikanth
Kandula, Scott Shenker, Ion Stoica
![Page 2: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/2.jpg)
2
Data Intensive ComputationData analytic clusters are pervasive
◦Jobs run multiple tasks in parallel◦Jobs operate on petabytes of input
Distributed file systems (DFS) store data distributed and replicated◦Data reads are either disk-local or
remote across the network
![Page 3: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/3.jpg)
3
Access to disk slowMemory orders of magnitude faster
How do we leverage memory storage for datacenter jobs?
![Page 4: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/4.jpg)
4
Can we store all data in memory? Machines have tens of gigabytes of
memory
But, huge discrepancy between storage and memory capacities
◦ Facebook cluster has ~200x more data on disk than memory
Use Memory as Cache
![Page 5: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/5.jpg)
5
Will the data fit in cache?
10% total input is >80% of all jobs
Heavy-tailed 96% of smallest jobs can fit in the
memory cache
![Page 6: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/6.jpg)
6
Elephants and miceMix of a few “large” jobs and
very many “small” jobs
Large jobs:◦Batch operations◦Production jobs
Small jobs:◦Interactive queries (e.g., Hive,
SCOPE)◦Experimental analytics
![Page 7: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/7.jpg)
7
Challenge: Small Parallel JobsJob finishes when its last task finishes
◦Need to cache all-or-nothing
![Page 8: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/8.jpg)
8
In summary…
Only option for memory-locality is caching
96% of jobs can have their data in memory, if we cache it right
![Page 9: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/9.jpg)
9
OutlineFATE: Cache Replacement
Memento: System Architecture
Evaluation
![Page 10: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/10.jpg)
10
We care about jobs finishing faster…
Job j that completed in tn time normally, takes tm time with memory caching
◦%Reductionj =
Average % Reduction in Completion Time
![Page 11: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/11.jpg)
11
Traditional Cache Replacement
Traditional cache replacement policies (e.g., LRU, LFU) optimize for hit-ratio
◦ Belady’s MIN: Evict blocks that are to be accessed “farthest in future”
![Page 12: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/12.jpg)
12
Belady’s MIN Example
50% cache hit
E, F, B, D, C, A (time
)
F, B, D, C, A (time
)
B, D, C, A(time)…
Data Block
A B C D
E B C D
E B F D
![Page 13: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/13.jpg)
MIN: How much do jobs benefit?
Memory-local tasks are 10x (or 90%) fasterB DA CJ1 J2
J2DJ2
C
13
J1A J1
B
Reduction:
0%
0%
Average(0 + 0)/2 = 0%
4 computation slots
B, D, C, A(time)
Data Block E B F D
![Page 14: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/14.jpg)
14
“Whole-job” inputs
50% cache hit
E, F, B, D, C, A (time
)
F, B, D, C, A (time
)
B, D, C, A(time)…
Data Block
A B C D
A B E D
A B E F
B
D
A
C
J1
J2
![Page 15: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/15.jpg)
MIN: How much do jobs benefit?
Memory-local tasks are 10x (or 90%) fasterB DA CJ1 J2
J2DJ2
C
15
J1A J1
B
Reduction:
90%
0%
Average(90 + 0)/2 = 45%
4 computation slots
B, D, C, A(time)
Data Block A B E F
(MIN): Average(0 + 0)/2 = 0%Cache hit-ratio not the most
suited
![Page 16: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/16.jpg)
16
FATE Cache ReplacementMaximize “whole-job” inputs in cache
Need global coordination◦Parallel tasks distributed over different
machines
Property:◦Small jobs get preference◦Large jobs benefit with remaining cache
space
![Page 17: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/17.jpg)
17
Waves in the jobSingle
Wave (small jobs) All-or-nothing
Multiple Waves (large jobs)Linear benefits
![Page 18: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/18.jpg)
18
Waves in the job
Multiple
Waves
Sing
le
Wav
e
![Page 19: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/19.jpg)
19
OutlineFATE: Cache Replacement
Memento: System Architecture
Evaluation
![Page 20: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/20.jpg)
20
Global coordination of local caches
Global cache viewBlock
IdClient Id
File Name
… … …
![Page 21: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/21.jpg)
21
Memento: Salient Features
External Service
Local cache reads
Metadata communication
![Page 22: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/22.jpg)
22
OutlineFATE: Cache Replacement
Memento: System Architecture
Evaluation
![Page 23: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/23.jpg)
23
EvaluationHDFS in conjunction with Memento
Microsoft and Facebook traces replayed◦Replay jobs with same inter-arrival time
Deployment on EC2 cluster of 100 machines◦20GB memory for Memento
Jobs binned by their size
![Page 24: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/24.jpg)
24
Job Distribution, by bins
![Page 25: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/25.jpg)
25
Jobs are 77% faster at average
Small jobs see 85% reduction in completion
time
![Page 26: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/26.jpg)
26
Cache hit-ratio matters less
Average job faster by 77% with FATE (vs.) 49% with
MIN
![Page 27: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/27.jpg)
27
Memento scales sufficientlyCoordinator handles 10,000
simultaneous client communications
Client can handle eight simultaneous local map tasks
Sufficient for current datacenter loads
![Page 28: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/28.jpg)
28
Ongoing / Future work >>
![Page 29: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/29.jpg)
29
Simpler Implementation [1]Ride the OS cache
◦Estimate where block is cached
Change job manager to track block accesses
– No FATE, use default (LRU?)
Initial results show 2.3x improvement in cache hit-rate
![Page 30: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/30.jpg)
30
Alternate Metrics [2]We optimize for “average %
reduction in completion time” of jobs
Average◦Weighted to include job priorities?
Other metrics◦Reduction of load on disk subsystem?◦Utilization?
![Page 31: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/31.jpg)
31
Solid State Devices [3]SSDs, a new layer in the storage
hierarchy
Hierarchical Caching◦Include SSDs between disk and memory
What’s the best cache replacement policy?
![Page 32: Memento : Coordinated In-Memory Caching for Data-Intensive Clusters](https://reader036.fdocuments.net/reader036/viewer/2022062410/568163e8550346895dd5510f/html5/thumbnails/32.jpg)
32
SummaryMemory-caching can be surprisingly
effective◦…despite disk and memory capacity
discrepancy
Memento: Coordinated cache management◦FATE Replacement Policy (“whole-jobs”)
Encouraging results for datacenter workload