Константин Осипов, Mail.Ru, Tarantool
-
Upload
ontico -
Category
Technology
-
view
179 -
download
0
description
Transcript of Константин Осипов, Mail.Ru, Tarantool
![Page 2: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/2.jpg)
Случай в квадрате 36-80
• B-tree – most popular disk-based data structure
• B-tree balances INSERT, UPDATE and SELECT speed
• DELETEs can be slow
![Page 3: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/3.jpg)
СУБД быстрая, настраивать надо уметь
![Page 4: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/4.jpg)
B-tree: внутреннее устройство
![Page 5: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/5.jpg)
Что означает сache-oblivious?
![Page 6: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/6.jpg)
Что означает сache-oblivious? (2)
BLOCKMULT(A,B,C,n):1 for i = 1 to n/s do:2 for j = 1 to n/s do:3 for k = 1 to n/s do:4 ORDMULT(Aik, Bkj, Cij, s)
![Page 7: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/7.jpg)
LSM-tree: внутреннее устройство
![Page 8: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/8.jpg)
LSM-tree: внутреннее устройство (2)
![Page 9: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/9.jpg)
LevelDB: устройство
![Page 10: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/10.jpg)
LevelDB: insert RPS
![Page 11: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/11.jpg)
LSM-tree: применение● Данные с разной степенью актуальности
– Ленты сообщений
– Стена в соцсети
– Чаты
– События
● Сегрегация данных– Данные в LSM space, индекс в MEMORY space
![Page 12: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/12.jpg)
TokuDB/CO lookahead arrays
![Page 13: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/13.jpg)
WAL:
Memory
Disk
Self-Balancing TreePUT(37), PUT(16)
![Page 14: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/14.jpg)
16 37
WAL: 37, 16
Memory
Disk
Self-Balancing Tree
![Page 15: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/15.jpg)
7 41
WAL: 41, 7, 37, 16
Memory
Disk16 37
Self-Balancing Tree
Sorted String Table
![Page 16: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/16.jpg)
WAL: 41, 7, 28, 16
Memory
Disk
7 16 37 41
7 37
![Page 17: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/17.jpg)
10 28
WAL: 10, 28, 41, 7, 37, 16
Memory
Disk
7 16 37 41
7 37
![Page 18: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/18.jpg)
WAL: 10, 28, 41, 7, 37, 16
Memory
Disk
7 16 37 41
10 28
![Page 19: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/19.jpg)
2 47
WAL: 47, 2, 10, 28, 41, 7, 37, 16
Memory
Disk
7 16 37 41
10 28
![Page 20: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/20.jpg)
WAL: 47, 2, 10, 28, 41, 7, 37, 16
MemoryDisk
2 7 10 16 28 37 41 47
2 10 28 41
2 28
![Page 21: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/21.jpg)
6 49
WAL: 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
MemoryDisk
2 7 10 16 28 37 41 47
2 10 28 41
2 28
![Page 22: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/22.jpg)
WAL: 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
MemoryDisk
2 7 10 16 28 37 41 47
2 10 28 41
6 49
![Page 23: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/23.jpg)
23 32
WAL: 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
MemoryDisk
2 7 10 16 28 37 41 47
2 10 28 41
6 49
![Page 24: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/24.jpg)
WAL: 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
MemoryDisk
2 7 10 16 28 37 41 47
6 23 32 49
6 32
![Page 25: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/25.jpg)
30 45
WAL: 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
Memory
Disk
2 7 10 16 28 37 41 47
6 23 32 49
6 32
![Page 26: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/26.jpg)
14 38
WAL: 38, 14, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
Memory
Disk
2 7 10 16 28 37 41 47
6 23 32 49
30 45
![Page 27: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/27.jpg)
6 10
WAL: 10, 6, 38, 14, 45, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
Memory
Disk
2 6 7 10 14 16 23 28 30 32 37 38 41 45 47 49
2 7 14 23 30 37 41 47
2 14 30 41
2 30
![Page 28: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/28.jpg)
WAL: 37, 22, 36, 10, 25, 42, 10, 6, 38, 14, 45, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
Memory
Disk
2 6 7 10 14 16 23 28 30 32 37 38 41 45 47 49
3 8 15 26 35 40 45 48
10 25 36 42
22 37
![Page 29: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/29.jpg)
WAL: 37, 22, 36, 10, 25, 42, 10, 6, 38, 14, 45, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
Memory
Disk
2 6 7 10 14 16 23 28 30 32 37 38 41 45 47 49
3 8 15 26 35 40 45 48
10 25 36 42
22 37
GET(16)
![Page 30: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/30.jpg)
WAL: 37, 22, 36, 10, 25, 42, 10, 6, 38, 14, 45, 30, 45, 32, 23, 49, 6, 47, 2, 10, 28, 41, 7, 37, 16
Memory
Disk
2 6 7 10 14 16 23 28 30 32 37 38 41 45 47 49
3 8 15 26 35 40 45 48
10 25 36 42
22 37
GET(16)
![Page 31: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/31.jpg)
BitCask: AOF format
![Page 32: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/32.jpg)
BitCask: key dir
![Page 33: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/33.jpg)
Sophia:
![Page 34: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/34.jpg)
Links● Bitcask A Log-Structured Hash Table for Fast Key/Value Data, Justin Sheehy David Smith with
inspiration from Eric Brewer● The Log-Structured Merge-Tree (LSM-Tree) Patrick O'Neil , Edward Cheng, Dieter Gawlick,
Elizabeth O'Neil● Cache-Oblivious Algorithms by Harald Prokop (Master theses)● Space/time trade-offs in hash coding with allowable errors, Burton H. Bloom● Data Structures and Algorithms for Big Databases, Michael A. Bender Stony Brook & Tokutek
Bradley C. Kuszmaul (XLDB tutorial)● http://github.com/pmwkaa/sophia, http://sphia.org● http://codecapsule.com/2012/12/30/implementing-a-key-value-store-part-3-comparative-analysis-of-the-architectures-of-kyoto-cabinet-and-leveldb/● http://stackoverflow.com/questions/6079890/cache-oblivious-lookahead-array● http://www.youtube.com/watch?v=88NaRUdoWZM(Tim Callaghan: Fractal Tree indexes)● http://code.google.com/p/leveldb/downloads/list
![Page 35: Константин Осипов, Mail.Ru, Tarantool](https://reader034.fdocuments.net/reader034/viewer/2022052208/54c6f68e4a7959c5088b4581/html5/thumbnails/35.jpg)
?Эпилог: choose your db wisely