@andy_pavlo FAS TER Making Fast Databases. Fast Cheap +
-
Upload
lynn-owens -
Category
Documents
-
view
222 -
download
2
Transcript of @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +
![Page 1: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/1.jpg)
@andy_pavlo@andy_pavloFASFASTERTER
Making Making Fast Fast
DatabasesDatabases
![Page 2: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/2.jpg)
![Page 3: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/3.jpg)
FastCheap
+
![Page 4: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/4.jpg)
OLTP Through the Looking Glass,OLTP Through the Looking Glass,and What We Found There and What We Found There SIGMOD 2008SIGMOD 2008
CPU Cycles
TPC-C NewOrder
8.1%
21.1%
18.7%
10.2%
29.6%
12.3%
Legacy Systems
![Page 5: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/5.jpg)
FastRepetitiveSmall
OLTP Transactio
ns
![Page 6: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/6.jpg)
Main Memory • Parallel • Shared-NothingTransaction Processing
H-Store: A High-Performance, DistributedH-Store: A High-Performance, DistributedMain Memory Transaction Processing SystemMain Memory Transaction Processing SystemVLDB VLDB vol. 1, issue 2, 2008vol. 1, issue 2, 2008
![Page 7: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/7.jpg)
ClientApplication
Database Cluster
Procedure NameInput
Parameters
Stored Procedure Execution
Database Cluster
Transaction
Result
![Page 8: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/8.jpg)
![Page 9: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/9.jpg)
TPC-C NewOrder
![Page 10: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/10.jpg)
Skew-Aware Automatic Database PartitioningSkew-Aware Automatic Database Partitioningin Shared-Nothing, Parallel OLTP Systemsin Shared-Nothing, Parallel OLTP SystemsSIGMOD 2012SIGMOD 2012
Partition Partition database to database to reduce the reduce the number of number of distributed distributed txns.txns.
OptimizatioOptimization #1:n #1:
![Page 11: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/11.jpg)
o_id o_c_id
o_w_id
…
78703 1004 5 -
78704 1002 3 -
78705 1006 7 -
78706 1005 6 -
78707 1005 6 -
78708 1003 12 -
c_id c_w_id
c_last
…
1001 5 RZA -
1002 3 GZA -
1003 12 Raekwon
-
1004 5 Deck -
1005 6 Killah -
1006 7 ODB -
CUSTOMER ORDERS
CUSTOCUSTOMERMERORDERORDER
SS
CUSTOCUSTOMERMERORDERORDER
SS
CUSTOCUSTOMERMERORDERORDER
SS
ITEMi_id i_na
mei_price
…
603514
XXX 23.99 -
267923
XXX 19.99 -
475386
XXX 14.99 -
578945
XXX 9.98 -
476348
XXX 103.49
-
784285
XXX 69.99 -
ITEMITEM ITEMITEM ITEMITEM
CUSTOMERc_id c_w_i
dc_las
t…
1001 5 RZA -
1002 3 GZA -
1003 12 Raekwon
-
1004 5 Deck -
1005 6 Killah -
1006 7 ODB -
![Page 12: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/12.jpg)
CUSTOCUSTOMERMERORDERORDER
SS
CUSTOCUSTOMERMERORDERORDER
SS
CUSTOCUSTOMERMERORDERORDER
SSITEMITEM ITEMITEM ITEMITEM
Client Application
NewOrder(5, “Method Man”,
1234)
![Page 13: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/13.jpg)
CUSTCUSTOMEROMERORDEORDE
RSRSITEMITEM
CUSTCUSTOMEROMERORDEORDE
RSRSITEMITEM
CUSTCUSTOMEROMERORDEORDE
RSRSITEMITEM
CUSTCUSTOMEROMERORDEORDE
RSRSITEMITEM
…
Large-Neighorhood Large-Neighorhood Search AlgorithmSearch Algorithm
Schema
Workload
-------
-------
-------
-------
-------
-------
DDL
DDL
SELECT * FROM WAREHOUSE WHERE W_ID = 10;
INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345);
⋮
SELECT * FROM WAREHOUSE WHERE W_ID = 10;
INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345);
⋮
SELECT * FROM WAREHOUSE WHERE W_ID = 10;
INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345);
⋮
SELECT * FROM WAREHOUSE WHERE W_ID = 10;
INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345);
⋮
SELECT * FROM WAREHOUSE WHERE W_ID = 10;
SELECT * FROM DISTRICT D_W_ID = 10 AND D_ID =9;
INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345);
⋮
SELECT * FROM WAREHOUSE WHERE W_ID = 10;
SELECT * FROM DISTRICT D_W_ID = 10 AND D_ID =9;
INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID) VALUES (10, 9, 12345);
⋮
SELECT * FROM WAREHOUSE WHERE W_ID = 10;
SELECT * FROM DISTRICT WHERE D_W_ID = 10 AND D_ID =9;
INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID,…) VALUES(10, 9, 12345,…);
⋮
SELECT * FROM WAREHOUSE WHERE W_ID = 10;
SELECT * FROM DISTRICT WHERE D_W_ID = 10 AND D_ID =9;
INSERT INTO ORDERS (O_W_ID, O_D_ID, O_C_ID,…) VALUES(10, 9, 12345,…);
⋮
NewOrdNewOrd
ererNewOrdNewOrd
ererDDLDDLCUSTOMERCUSTOMER
ITEMITEM
ORDERSORDERS
![Page 14: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/14.jpg)
Workload
-------
-------
-------
-------
-------
-------
Schema
DDL
DDL
Initial DesignRelaxationLocal Search
Restart
Large-Neighborhood
Search
![Page 15: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/15.jpg)
TATP TPC-CTPC-C Skewed
(txn/s)
+88% +16% +183%
HorticultureState-of-the-Art
Throughput
![Page 16: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/16.jpg)
TATP SEATS
TPC-C TPC-C Skewed
AuctionMark TPC-E
Search Times
% S
ing
le-P
art
itio
ned
Tra
nsa
ctio
ns
![Page 17: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/17.jpg)
![Page 18: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/18.jpg)
Database Cluster
Database Cluster
ClientApplication
Undo Log
![Page 19: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/19.jpg)
On Predictive Modeling for OptimizingOn Predictive Modeling for OptimizingTransaction Execution in Parallel OLTP SystemsTransaction Execution in Parallel OLTP SystemsVLDB, vol 5. issue 2, October 2011VLDB, vol 5. issue 2, October 2011
Predict what Predict what txns will do txns will do beforebefore they they execute.execute.
OptimizatioOptimization #2:n #2:
![Page 20: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/20.jpg)
Database Cluster
Database Cluster
ClientApplication
» Partitions Touched?» Undo Log?
» Done with Partitions?
![Page 21: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/21.jpg)
![Page 22: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/22.jpg)
w_id=0i_w_ids=[0,1] i_ids=[1001,1002]
w_id=0i_w_ids=[0,1] i_ids=[1001,1002]
Input Parameters:Current State:
SELECT * FROM WAREHOUSE WHERE W_ID = ?SELECT * FROM WAREHOUSE WHERE W_ID = ?
GetWarehouse:
![Page 23: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/23.jpg)
Confidence Coefficient:
0.96
Best Partition: 0
Partitions Accessed:
{ 0 }
Use Undo Logging:
Yes
Transaction Estimate:
Estimated Execution Path
w_id=0i_w_ids=[0,1] i_ids=[1001,1002]
w_id=0i_w_ids=[0,1] i_ids=[1001,1002]
Input Parameters:
![Page 24: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/24.jpg)
TATP TPC-CAuctionMark
(txn/s)
+57% +126%+117%
HoudiniAssume Single-Partitioned
Throughput
![Page 25: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/25.jpg)
TATP TPC-CAuctionMark
Prediction Overhead
![Page 26: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/26.jpg)
![Page 27: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/27.jpg)
Future WorkFuture Work::Reduce distributed txn overheadReduce distributed txn overheadthrough creative scheduling.through creative scheduling.
Conclusion:Conclusion:Achieving fast performance isAchieving fast performance ismore than just using only RAM.more than just using only RAM.
![Page 28: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/28.jpg)
h-h-storestorehstore.cs.br
own.eduhstore.cs.br
own.edugithub.com/apavlo/h-storegithub.com/apavlo/h-store
![Page 29: @andy_pavlo FAS TER Making Fast Databases. Fast Cheap +](https://reader035.fdocuments.net/reader035/viewer/2022062408/56649ef45503460f94c06d6e/html5/thumbnails/29.jpg)
Help is AvailableHelp is Available
Graduate Student Graduate Student Abuse HotlineAbuse Hotline
AvailableAvailable 24/724/7Collect Calls AcceptedCollect Calls Accepted
+1-212-939-7064+1-212-939-7064