Dynamic Columns

8/7/2019 Dynamic Columns

http://slidepdf.com/reader/full/dynamic-columns 1/18

Notice: MySQL is a registered trademark of Sun Microsystems, Inc.

MySQL Conference & Expo 2011

Michael “Monty” Widenius [email protected] “Sanja” Byelkin [email protected]

MariaDBDynamic Columns




RDBMS doesn't solve all common problems

The (web) store problem:All items need: ID, Type, Price, Country, Manufacturer)

A T-Shirt has the following additional properties:

Size, color...A computer has the following additional properties:

CPU, MHz, memory, Watt...

There is no easy way to store many different types into arelational database!





One common solutions to this is:

● Store all the 'extra' columns in a BLOB in some format(HTML?)

● You need a lot of extra work to manipulate the blob● Hard to access column data (usually done in client)● Overhead in storage (especially when you use HTML)● All values are 'text'





Another common solution:● Create one table for all the 'extra' columns:CREATE TABLE extra

(id int auto_increment, extra_column_id char(10),

value varchar(255));

INSERT INTO items set type=“t-shirt”, price=10;

INSERT INTO extra (NULL, LAST_INSERT_ID(), “color”,

“Blue”),(NULL, LAST_INSERT_ID(), “Size”, “M”);

The problems with this approach is:●

Every access to an extra column requires a key/rowlookup

● Slow performance (if database is not optimized for this)● Big overhead in storage (especially in index)

● Risk for errors as data is not typed




Dynamic columns

Dynamic columns is a bridge between relationaldatabases and non relational databases

● With dynamic columns all extra columns are stored in apacked blob, maintained by the database.

● You can add more columns, remove or query them for arow.

● You can access columns in the server or retrieve the fullblob to the client and manipulate it there.

●

You can use virtual columns to create indexes on somevalues.

●True indexes for dynamic columns is planned for later.● Implemented through functions for use by ODBC, & etc.

● First implementation uses integer to access columns.




Dynamic columns: supportedtypes

● unsigned int

● int

● char [character set <cs>]

●

double● decimal [(<len>, <frac>)]

● time

● date

● datetime




Dynamic columns: syntax

Creating a table with a dynamic column for the store:

CREATE TABLE item (

ID int auto_increment primary key,

Type_id int,Price decimal(7,2),

Country_id int,

Manufacturer_id int,

extra blob);

Where column 'extra' is dedicated to store dynamiccolumns. It could be any column able carry text.





Creating/initializing a dynamic_column:

COLUMN_CREATE(column_nr, value [as type],

[column_nr, value [as type]], ...)

INSERT into item values

(NULL, 1 /* T-shirt */, 10, 1 /* Germany */, 1 /* Nike */,

COLUMN_CREATE(1 /* color */, "Blue", 2 /* Size */, "M"));

INSERT into item values(NULL, 2 /* computer */, 1000, 1 /* Germany */, 2 /* intel

*/,

COLUMN_CREATE(3 /* cpu */, "T9400", 5 /* MHz */, 800 as

unsigned int));

* *





Deleting a dynamic column (if it exists):

COLUMN_DELETE(string, column_nr1,

column_nr2, ...);

UPDATE item SET extra=

COLUMN_DELETE(extra, 6)

WHERE id=2;


http://slidepdf.com/reader/full/dynamic-columns 11/18Notice: MySQL is a registered trademark of Sun Microsystems, Inc.


Querying a dynamic column:

COLUMN_EXISTS(string, column_nr);SELECT

ID, Type_id, Price, Country_id, Manufacturer_id from itemwhere COLUMN_EXISTS(extra, 3);

Querying which columns exist:

COLUMN_LIST(string);SELECT COLUMN_LIST(extra) FROM item WHERE id=1;

“→ 1,2”




Retrieving a dynamic column:

COLUMN_GET(column_nr, string as type);

SELECT id, COLUMN_GET( 1 /* color*/, extra as char)from item;

→ 1 Blue

→ 2 NULL

You can of course also do things like:SELECT id, COLUMN_GET(1, extra as char) FROM item

where Type_id=1

order by COLUMN_GET(1, extra as char);



Dynamic columns: C library

C Library allows the same manipulations with dynamiccolumns on the client side.

A description can be found in the worklog

http://askmonty.org/worklog/Server-Sprint/?tid=34 and in thesource.

http://askmonty.org/worklog/Server-Sprint/?tid=34

http://askmonty.org/worklog/Server-Sprint/?tid=34



Dynamic columns: encoding

How is the dynamic column encoded?

Header:<flag><number_of_columns>

Sorted index:<column_nr1><offset/type><column_nr2><offset/type>

Each column is stored as:<data1><data2>

Where 'offset' is offset from beginning of data part, 'type' is3 bits in offset.

Length of the data could be calculated by offsets of 2

neighbor fields.



Dynamic columns: data encoding

● Unsigned integer is just a variable integer field.● Signed integer coded to make the variable size efficient:

0 → 0

-1 → 1

1 → 2

-2 → 3

2 → 4

...

● Double, date, time, and date time are fixed-size fields●

String stores collation number and the string● Decimal stores sizes of parts before and after decimal point

and decimal in MySQL format.● NULL – means removing the field.



Dynamic columns: avaliability

When will dynamic columns be available?

First version is already pushed in separate tree for testing.

lp:~maria-captains/maria/5.3-mwl34

Should be available in main MariaDB 5.3 within severalweeks.



Dynamic columns: plans

● Adding name directory● Adding functional indices●

Supporting popular NoSQL data exchange formats(for example, JSON or XML)● Adding engine support for NoSQL databases asHBase



Thanks

Q & A

Dynamic Columns

Documents

Transcript of Dynamic Columns