Main Memory Databases Hadi Amiri, Abolfazle AleAhmad Department of ECE, University of Tehran.
-
Upload
walter-jenkins -
Category
Documents
-
view
216 -
download
2
Transcript of Main Memory Databases Hadi Amiri, Abolfazle AleAhmad Department of ECE, University of Tehran.
Main Memory DatabasesMain Memory Databases
Hadi Amiri, Abolfazle AleAhmadHadi Amiri, Abolfazle AleAhmad
Department of ECE, University of TehranDepartment of ECE, University of Tehran
Top Main Memory DBsTop Main Memory DBsFastDBFastDBMonetDBMonetDBeXtremeDBeXtremeDBERDBERDBDataBlitzDataBlitzTimes TenTimes Ten
MonetDBMonetDB
MonetDB is an open source high-MonetDB is an open source high-performance database system performance database system developed at developed at CWICWI, (the Institute for , (the Institute for Mathematics andMathematics and Computer Science Computer Science Research of The Netherlands)Research of The Netherlands)
MonetDB has been successfully MonetDB has been successfully applied in high-performance applied in high-performance applications for data mining, OLAP, applications for data mining, OLAP, GIS, XML Query, text and multimedia GIS, XML Query, text and multimedia retrieval. (Real Time App)retrieval. (Real Time App)
MonetDB: architecture
Front-end/back-end:
• support multiple data models
• support multiple end-user languages
• support diverse application domains
perform all operations in main memory perform all operations in main memory a modern CPU-tuned vectorized query execution a modern CPU-tuned vectorized query execution
architecturearchitecture that often gives MonetDB a more than 10-fold raw that often gives MonetDB a more than 10-fold raw
speed advantage on the same algorithm over a speed advantage on the same algorithm over a typical interpreter-based RDBMStypical interpreter-based RDBMS
extensible algebra. extensible algebra. binary relation model binary relation model shared-memory parallelism shared-memory parallelism MonetDB is one of the first database systems to MonetDB is one of the first database systems to
focus its query optimization effort on exploiting focus its query optimization effort on exploiting CPU cachesCPU caches
MonetDB PropertiesMonetDB Properties
MonetDB has played a pivotal role in the MonetDB has played a pivotal role in the development of datamining applications development of datamining applications at DataDistilleries (now SPSS). at DataDistilleries (now SPSS).
Monet IdeasMonet Ideas
: دستورات از استفاده امکان اول : ايده دستورات از استفاده امکان اول ايدهBack-EndBack-End (( براي را جوهايي و پرس توانيد مي شما که معني اين براي به را جوهايي و پرس توانيد مي شما که معني اين واسطي Back-EndBack-End به از استفاده واسطي با از استفاده با
نام نام به کند. Mapi ClientMapi Client به مي کار آن با واسط اين که زباني کند. بنويسيد مي کار آن با واسط اين که زباني دارد MILMIL بنويسيد دارد نام (( نام : مي کاري هر دودويي جداول با دوم : ايده مي کاري هر دودويي جداول با دوم ايده
کرد کرد توان توان : دوباره اختراع به نياز عدم سوم : ايده دوباره اختراع به نياز عدم سوم ايده
عامل عامل سيستم سيستم : پرسو اجراي سازي بهينه چهارم : ايده پرسو اجراي سازي بهينه چهارم ايده
حافظه در حافظه جو در جو
Monet Data ModelMonet Data Model
Some simple valuesSome simple values Int,chr,str,oid,nil,intint(nil),chrchr(nil),Int,chr,str,oid,nil,intint(nil),chrchr(nil),
strstr(nil),oidoid(nil)strstr(nil),oidoid(nil) Binary Association Tables (BATs)Binary Association Tables (BATs)
BAT BAT ستون دو با است ستون جدولي دو با است که که Head Head وو Tail Tail جدولييا و فوق ساده مقادير يا شامل و فوق ساده مقادير باشد BAT BAT شامل باشد مي مي
Monet PerformanceMonet Performance Compare some well-known open-source DBMSs with Monet on the Compare some well-known open-source DBMSs with Monet on the
task to build a binary table with the head representing an ordered task to build a binary table with the head representing an ordered sequence and the tail a permutation over the values in the head sequence and the tail a permutation over the values in the head
Response time is given in seconds while running on a dual Athlon 1400 with 1GB of RAM and 3.5GB of free space
-- DBtapestry Version=1.1 -- See http://monetdb.cwi.nl/DBtapestry/ -- (c) CWI -- DBtapestry Version=1.1 -- See http://monetdb.cwi.nl/DBtapestry/ -- (c) CWI 2004-2005 - rows=100K --columns=2 -- Produced Sat Feb 26 23:41:51 20052004-2005 - rows=100K --columns=2 -- Produced Sat Feb 26 23:41:51 2005
start transaction; start transaction; select now(); select now(); create table RKA( head int, tail int); create table RKA( head int, tail int); create table RKB( head int, tail int); create table RKB( head int, tail int); insert into RKA values(0,0); insert into RKA values(0,0); insert into RKA values(1,360); insert into RKA values(1,360); insert into RKA values(2,427); insert into RKA values(2,427); …… …… insert into RKB select head+0, tail+0 from RKA; insert into RKB select head+0, tail+0 from RKA; insert into RKB select head+8192, tail+8192 from RKA; insert into RKB select head+8192, tail+8192 from RKA; …… …… drop table RKA; drop table RKA; update RKB set tail=(tail*47) % 102400; update RKB set tail=(tail*47) % 102400; …………create table tapestry( attr0 int , attr1 int); create table tapestry( attr0 int , attr1 int); insert into tapestry select R0.head, R0.tail from RKB R0; insert into tapestry select R0.head, R0.tail from RKB R0; drop table RKB; drop table RKB; select now(); select now(); commit; commit; -- actions=1249 -- tuplesRead= 1.07M tuplesWritten= 2.07M -- volumeRead= -- actions=1249 -- tuplesRead= 1.07M tuplesWritten= 2.07M -- volumeRead=
8.59M volumeWritten=16.59M 8.59M volumeWritten=16.59M
What is MonetDB Good for?What is MonetDB Good for?
Query-intensive applicationQuery-intensive application very high performance demandingvery high performance demanding complex data modelscomplex data models complex query primitivescomplex query primitives
Transactions (OLTP)Transactions (OLTP)OLTPOLTP ( (OOnnlline ine TTransaction ransaction PProcessing) is a form of transaction processing conducted via computer rocessing) is a form of transaction processing conducted via computer network. Some applications of OLTP include electronic banking, order processing, employee time network. Some applications of OLTP include electronic banking, order processing, employee time
clock systems, e-commerce, and clock systems, e-commerce, and eTradingeTrading. .
OLAP, Data MiningOLAP, Data MiningOn Line Analytical ProcessingOn Line Analytical Processing. (. (OLAP)OLAP) It is an approach to quickly provide the answer to It is an approach to quickly provide the answer to
complex analytical queries. The typical applications of OLAP are in business reporting for sales, complex analytical queries. The typical applications of OLAP are in business reporting for sales, marketing, management reporting, business performance management, budgeting and marketing, management reporting, business performance management, budgeting and
forecasting, financial reporting and similar areas. forecasting, financial reporting and similar areas.
New Domains: GISNew Domains: GIS
New data types (point, polygon, New data types (point, polygon, etc..)etc..)
New search accelerators (R-Tree, New search accelerators (R-Tree, etc..)etc..)
New primitives New primitives boolean intersects(polygon,polygon)boolean intersects(polygon,polygon)
Complex topological structures Complex topological structures stored in DCELs that are decomposed over stored in DCELs that are decomposed over BATS queries are efficient due to MonetDB BATS queries are efficient due to MonetDB high join speedhigh join speed
New Domains: MultimediaNew Domains: Multimedia
New data types (url, image, etc..)New data types (url, image, etc..) new search accelerators (color new search accelerators (color
histograms)histograms) new primitives (similarity search)new primitives (similarity search) complex data structures: complex data structures:
bayesian inference networks (information bayesian inference networks (information retrieval) again decomposed in BATs and retrieval) again decomposed in BATs and efficient to queryefficient to query