TREES General trees Binary trees Binary search trees AVL trees Balanced and Threaded trees.
FAST: A Framework for Flash-Aware Spatial Trees
description
Transcript of FAST: A Framework for Flash-Aware Spatial Trees
![Page 1: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/1.jpg)
University of Minnesota
FAST: A Framework for Flash-Aware Spatial Trees
Mohamed Sarwat, Mohamed Mokbel, Xun ZhouDepartment of Computer Science and Engineering
University of Minnesota
Suman NathMicrosoft Research
![Page 2: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/2.jpg)
2
Flash Memory: Already exists
• Mobile Devices
• Enterprise Servers
![Page 3: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/3.jpg)
3
Flash Memory Storage: Pros
Flash Memory Hard Disk
Electronic Device Mechanical Device
Fast Read Performance Slow Read Performance
High Shock Resistance Low Shock Resistance
Low Power Consumption High Power Consumption
Small Size and Light Weight Relatively Large Size and Heavy Weight
Flash Memory is Topnotch !
![Page 4: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/4.jpg)
4
Flash Memory Storage: Cons
Erase-Before-Update Property Erase latency 1500 microsecondsRead latency 25 microsecondsWrite latency 200 microseconds
Limited Number of Erases per Block100,000 to 1000,000
![Page 5: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/5.jpg)
5
DBMS Tree Indexes & Storage
All Database indexing algorithms were built with the implicit assumption that the underlying secondary storage is the hard disk.
Making Tree Index Structures Flash-Aware !
![Page 6: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/6.jpg)
6
Existing Flash-Aware Trees
■ Devesh Agrawal, Deepak Ganesan, Ramesh Sitaraman, Yanlei Diao, and Shashi Singh. Lazy-Adaptive Tree: An Optimized Index Structure for Flash Devices. In VLDB 2009 Need to build a new kind of tree indexing. Not very practical..!!
■ Yinan Li, Bingsheng He, Robin Jun Yang, Qiong Luo, Ke Yi. Tree Indexing on Flash Disks. In ICDE 2009 (short), VLDB 2010(full) Need to build a new kind of tree indexing. Not very practical..!!
![Page 7: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/7.jpg)
7
Existing Flash-Aware Trees
■ Chin-Hsien Wu, Li-Pin Chang, Tei-Wei Kuo. An Efficient R-Tree Implementation over Flash-Memory Storage Systems. In GIS 2003 Designed only for R-tree. No support for recovery
■ Chin-Hsien Wu, Li-Pin Chang, and Tei-Wei Kuo. An Efficient B-Tree Layer for Flash-Memory Storage Systems. In TECS 2007 Designed only for B-tree. No support for recovery
![Page 8: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/8.jpg)
8
We need a Framework
1) Generic: One Framework works for all
2) Efficient: No need to throw your already existing index structure away
3) Durable: Designed with database systems in mind (recovery)
![Page 9: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/9.jpg)
9
FAST is a Generic and Efficient Framework for
Flash-Aware Search Trees in database systems
FAST
![Page 10: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/10.jpg)
10
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
![Page 11: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/11.jpg)
11
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
![Page 12: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/12.jpg)
12
Update Operation (Insertion/Deletion)
Main Memory Flash Memory
Operation Log
…….
Tree Modifications Table
Tree Index Structure
Insert (O1, MBR ….) || Nodes 1,2 Step 1
Step 2NodeNode 1
Node 2
MBR1 - value
Obj1 - Inserted
Example: Insert (Obj1) in R-tree
Step 3
![Page 13: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/13.jpg)
13
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
![Page 14: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/14.jpg)
14
Search Operation (Find/Range)
Main Memory Flash Memory
Operation Log
…….
Tree Modifications Table
Tree Index Structure
Insert (O1, MBR ….) || Nodes 1,2
Step1
Step 2
Node
Node 1
Node 2
MBR1 - value
Obj1 - Inserted
Example: Range query in R-tree
Step 4
Query Result
Step 5
Step 3
![Page 15: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/15.jpg)
15
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
![Page 16: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/16.jpg)
16
FlushingMain Memory Flash Memory
FAST Log File
Tree Index Structure
Insert (O1, MBR) || Nodes 1,2
MBR1 - value
Node 1 2 3 4 5
Dirty 00 1
Flush 2,3,4,5
75 %
Nodes 2,3,4,5 flushed
Same Block
129
12144
17185
Node 1
2 3 4 5
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
1 10 0 0
Delete (O2, MBR) || Nodes 12
Insert (O4, MBR) || Nodes 1,4,14
Insert (O5, MBR) || Nodes 4,17
Insert (O4, MBR) || Nodes 1,5,18
Step 1
Step 2
Step 4
Step 3
Step 5
![Page 17: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/17.jpg)
17
FAST: A Framework for Flash-Aware Search Trees
Recent Version of the tree
Flash Storage
Flash Memory Index
In-memory BufferUpdate Module
Search Module
Crash Recovery Module
Flushing Module
Search Query
Answer
Update Transaction
Log File
Restart after Crash
Full MemoryRead
Read
In-memory Update
Update Log Entries
Flushing Policy
Read
Selected Memory
Pages
Write
Flush Log
Entry
Read Index & Log Entries
Write
![Page 18: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/18.jpg)
18
Crash RecoveryMain Memory Flash Memory
FAST Log File
.....
Tree Index Structure
Insert (O1, MBR) || Nodes 1,2
Nodes 1,2,3,4,5, 14,17 flushed
1
2 3 4 5
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Delete (O2, MBR) || Nodes 12
Insert (O4, MBR) || Nodes 1,4,14
Insert (O5, MBR) || Nodes 4,17
Insert (O4, MBR) || Nodes 1,5,18
Log Maintenance
Compact Log
Insert (O4, MBR) || Nodes 18
Crash Recovery
Restart after crash
Restart after crash
MBR1 - value
129
12144
17185
Node
![Page 19: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/19.jpg)
19
Experimental Evaluation (1/4)
■ We implemented FAST in PostGreSQL
■ We instantiate B-tree and R-tree instances of FAST, termed FAST-Btree and FAST-Rtree
■ We use two synthetic workloads: Lookup intensive workload (WL): 80% search and
20% update Update intensive workload, (WU): 20% search and
80% update.
■ number of workload operations to 10 million■ operations, main memory size to 256 KB ■ Tree index size to 512 MB, and ■ log file size to10 MB (~ 2% of Index size)
![Page 20: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/20.jpg)
20
Experimental Evaluation (2/4)
Memory Size Log File Size
348 lines of code3070 lines of code
308 lines of code
9230 lines of code8150 lines of code
![Page 21: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/21.jpg)
21
Experimental Evaluation (3/4)
Memory Size Log File Size
![Page 22: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/22.jpg)
22
Experimental Evaluation (4/4)
Log Compaction Recovery
![Page 23: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/23.jpg)
23
Conclusion
■ FAST is a generic framework for flash-aware index structures.
■ FAST guarantees the durability of update transactions applied to the underlying index structure.
■ FAST has good performance compared to its counterparts (LA-tree, FD-tree, and RFTL)
![Page 24: FAST: A Framework for Flash-Aware Spatial Trees](https://reader036.fdocuments.net/reader036/viewer/2022081604/568156c4550346895dc455e4/html5/thumbnails/24.jpg)
24
Questions