1 / 71 - Percona · Percona University, Ghent, Belgium June 2017 MySQL Document Store How to...

1 / 71

2 / 71

Percona University, Ghent, Belgium June2017

MySQL Document Store

How to replace a NoSQL database by MySQL without effort but with a lot ofgains ?

Frédéric Descamps - MySQL Community Manager - Oracle

3 / 71

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended forinformation purpose only, and may not be incorporated into any contract. It is not acommitment to deliver any material, code, or functionality, and should not be relied up inmaking purchasing decisions. The development, release and timing of any features orfunctionality described for Oracle´s product remains at the sole discretion of Oracle.

Copyright @ 2017 Oracle and/or its affiliates. All rights reserved.

4 / 71

about.me/lefred

Who am I ?


5 / 71

Frédéric Descamps@lefredMySQL EvangelistHacking MySQL since 3.23devops believerMySQL Community Manager since May2016living in Belgium 🇧🇪


6 / 71

7 / 71

RDBMS & MySQL

Relational Databases


8 / 71

Relational DatabasesData Integrity


9 / 71


normalization


10 / 71


normalizationconstraints (foreign keys, ...)


11 / 71



Atomicity, Consistency, Isolation, Durability


12 / 71



Atomicity, Consistency, Isolation, DurabilityACID compliant


13 / 71



Atomicity, Consistency, Isolation, DurabilityACID complianttransactions


14 / 71




SQL


15 / 71




SQLpowerfull query language


16 / 71

NoSQL & MySQL

NoSQL Databases


17 / 71

NoSQL or Document StoreSchemaless


18 / 71


no schema design, no normalization, no foreign keys, no data types, ...


19 / 71


no schema design, no normalization, no foreign keys, no data types, ...very quick initial development


20 / 71



Flexible data structure


21 / 71



Flexible data structureembedded arrays or objects


22 / 71



Flexible data structureembedded arrays or objectsvalid solution when natural data can't be modelized optimaly into a relationalmodel


23 / 71



Flexible data structureembedded arrays or objectsvalid solution when natural data can't be modelized optimaly into a relationalmodelobjects persistance without the use of any ORM - mapping oobject-oriented


24 / 71




JSON


25 / 71




JSONclose to frontend


26 / 71




JSONclose to frontendnative in JS


27 / 71




JSONclose to frontendnative in JSeasy to learn


28 / 71

How DBAs see data How Developers see data{ "GNP" : 249704, "Name" : "Belgium", "government" : { "GovernmentForm" : "Constitutional Monarchy, Federation", "HeadOfState" : "Philippe I" }, "_id" : "BEL", "IndepYear" : 1830, "demographics" : { "Population" : 10239000, "LifeExpectancy" : 77.8000030517578 }, "geography" : { "Region" : "Western Europe", "SurfaceArea" : 30518, "Continent" : "Europe" }}


29 / 71

DMBS or NoSQL ?


30 / 71

DMBS or NoSQL ?

Why not both ?


31 / 71

MySQL as Document Store

Requirements


32 / 71

MySQL as Document Store: requirements


33 / 71

MySQL as Document Store: requirements1. support of JSON datatype


34 / 71


2. CRUD Operations


35 / 71


2. CRUD Operations

3. X-Plugin


36 / 71


2. CRUD Operations

3. X-Plugin

4. X-Protocol


37 / 71


2. CRUD Operations

3. X-Plugin

4. X-Protocol

extends and replaces the traditional MySQL protocol


38 / 71


2. CRUD Operations

3. X-Plugin

4. X-Protocol

extends and replaces the traditional MySQL protocolsupports asynchronous communication to the server


39 / 71


2. CRUD Operations

3. X-Plugin

4. X-Protocol

extends and replaces the traditional MySQL protocolsupports asynchronous communication to the serversupports multiple API calls like Javascript, Python, SQL...


40 / 71


2. CRUD Operations

3. X-Plugin

4. X-Protocol

extends and replaces the traditional MySQL protocolsupports asynchronous communication to the serversupports multiple API calls like Javascript, Python, SQL...

5. MySQL Shell


41 / 71

Starting using MySQL DS in few minutes


42 / 71

Starting using MySQL DS in few minutes

This is the new MySQL shell as it will be in 8 ;-)


43 / 71

Migration from MongoDB to MySQL DSFor this example, I will use the well known restaurants collection:

We have two options:

1. We need to dump the data to a file and we will use MySQL Shell with the Pythoninterpreter to load the data.

1. We retrieve the data from MongoDB and load it to MySQL via the Shell in javascript


44 / 71

Dump and load using MySQL Shell & Pythonthis example is the courtesy of @datacharmer:https://www.slideshare.net/datacharmer/mysql-documentstore

$ mongo --quiet --eval 'DBQuery.shellBatchSize=300; var all=db.restaurants.�nd() ; all' | perl -pe \ 's/(?:ObjectId|ISODate)$("[^"]+")$/ $1/g' > all_recs.json


45 / 71

https://www.slideshare.net/datacharmer/mysql-documentstore

Dump and load using bash and MySQL Shell

$ mongo --quiet --eval 'DBQuery.shellBatchSize=30000; var all=db.restaurants.�nd() ; all' \| perl -pe 's/(?:ObjectId|ISODate)$("[^"]+")$/ $1/g'\| head -n -1 | while read line do mysqlsh [email protected] --password='' -e "rec=$line; session.getSchema('docstore').getCollection('restaurants').add(rec).execute()" done


46 / 71

Let's query


47 / 71

Let's query

That's too much records to show it here... let's limit it


48 / 71


49 / 71

Some more examples


50 / 71

Some more examples

Let's add a selection criteria:


51 / 71

Syntax slightly different than MongoDB

> db.restaurants.�nd({"cuisine": "French", "borough": { $not: /^Manhattan/} }, {"_id":0, "name": 1,"cuisine": 1, "borough": 1}).limit(2){ "borough" : "Queens", "cuisine" : "French", "name" : "La Baraka Restaurant" }{ "borough" : "Queens", "cuisine" : "French", "name" : "Air France Lounge" }


52 / 71

CRUD operations


53 / 71

what about my old SQL ?

The hidden part of the iceberg


54 / 71

JSON datatype is behind the scenenative datatype (since 5.7.8)


55 / 71

JSON datatype is behind the scenenative datatype (since 5.7.8)JSON values are stored in MySQL tables using UTF8MB4


56 / 71

JSON datatype is behind the scenenative datatype (since 5.7.8)JSON values are stored in MySQL tables using UTF8MB4Conversion from "native" SQL types to JSON values


57 / 71

JSON datatype is behind the scenenative datatype (since 5.7.8)JSON values are stored in MySQL tables using UTF8MB4Conversion from "native" SQL types to JSON valuesJSON manipulation functions (JSON_EXTRACT, JSON_KEYS,JSON_SEARCH,...)


58 / 71

JSON datatype is behind the scenenative datatype (since 5.7.8)JSON values are stored in MySQL tables using UTF8MB4Conversion from "native" SQL types to JSON valuesJSON manipulation functions (JSON_EXTRACT, JSON_KEYS,JSON_SEARCH,...)Generated/virtual columns


59 / 71


Indexing JSON data


60 / 71


Indexing JSON dataForeign Keys to JSON data


61 / 71


Indexing JSON dataForeign Keys to JSON dataSQL Views to JSON data


62 / 71

Example


63 / 71

Example


64 / 71

Example (2)


65 / 71

Example (2)


66 / 71

Example (3): explain


67 / 71

Example (4): add index


68 / 71

what's do I gain ?

Conclusion


69 / 71

schemalessflexible data structureeasy to start (CRUD)

ConclusionThis is the best of the two worlds in one product !

Data integrityACID CompliantSQL


70 / 71

Thank you !

Any Questions ?


71 / 71

1 / 71 - Percona · Percona University, Ghent, Belgium June 2017 MySQL Document Store How to...

Documents

Transcript of 1 / 71 - Percona · Percona University, Ghent, Belgium June 2017 MySQL Document Store How to...