New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander...
Transcript of New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander...
![Page 2: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/2.jpg)
Magnus HaganderRedpill Linpro
Infrastructure servicesPrincipal database consultant
PostgreSQLCore Team memberCommitterPostgreSQL Europe
![Page 3: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/3.jpg)
PostgreSQLThe World's Most Advanced Open Source Database
Yeah?
![Page 4: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/4.jpg)
PostgreSQLLong history
Berkeley Postgres: 1986Postres95: 1994PostgreSQL: 1996
![Page 5: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/5.jpg)
PostgreSQLHigh speed of developmentNo longer just "catchup"Many brand new thingsSome catchup too, of course
Oracle's been around since 1978-1979...
![Page 6: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/6.jpg)
PostgreSQLReleases approx 1 / year5 year support lifecycle
Current: 9.5 (Jan 2016)Oldest: 9.1 (Sep 2011)
![Page 7: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/7.jpg)
What's new and coolBig things in every versionLet's pick a coupleAcross 9.2, 9.3 and 9.4 (mainly)Mix of developer and DBA
Mostly developer today!
![Page 8: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/8.jpg)
Foreign Data Wrappers
![Page 9: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/9.jpg)
Foreign Data WrappersAccess data in remote databasesAs regular tables
![Page 10: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/10.jpg)
postgres_fdwNo more dblink required (still supported!)Access remote PostgreSQL servers "properly"Supports remote cost estimatesPushes down quals (when possible)
9.6 will push down joins
![Page 11: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/11.jpg)
Writeable FDWsAbility to update foreign tablesINSERT/UPDATE/DELETETransaction aware (of course)Can be slow for complicated updates/deletesRequires FDW specific support
![Page 12: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/12.jpg)
Foreign Data WrappersCREATE SERVER remotepg FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host 'localhost', dbname 'pagila', port '5432')
CREATE USER MAPPING FOR mha SERVER remotepg
CREATE FOREIGN TABLE actor ( actor_id int, first_name varchar(45), last_name varchar(45), last_update timestamp)SERVER remotepg
![Page 13: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/13.jpg)
Foreign Data Wrapperspostgres=# select * FROM actor WHERE first_name='BOB'; actor_id | first_name | last_name | last_update +++ 19 | BOB | FAWCETT | 20120315 14:53:16.211411(1 row)
postgres=# UPDATE actor SET first_name='BOBBY'postgres= WHERE first_name='BOB';UPDATE 1
postgres=# SELECT count(*) FROM actor INNER JOIN localnamespostgres# ON actor.first_name=localnames.first_name; count 2(1 row)
![Page 14: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/14.jpg)
Other FDWscsv filesOracleMySQLRedisMongoDBODBC...
![Page 15: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/15.jpg)
Range types
![Page 16: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/16.jpg)
Range typesStore ranges of something
Generic frameworkBuilt-in for int, numeric, timestamp, date
Query ranges of somethingConstrain ranges of something
![Page 17: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/17.jpg)
Storing rangesTraditional way
CREATE TABLE meetings ( start_time timestamptz NOT NULL, end_time timestamptz NOT NULL, room int NOT NULL REFERENCES rooms, title text NOT NULL);
![Page 18: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/18.jpg)
Storing rangesUsing range types
CREATE TABLE meetings_r ( blocktime tstzrange NOT NULL, room int NOT NULL REFERENCES rooms, title text NOT NULL);
![Page 19: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/19.jpg)
Inserting range valuesINSERT INTO meetings VALUES ( '20150611 14:00', '20150611 15:00', 1, 'First meeting')
INSERT INTO meetings_r VALUES ( tstzrange('20150611 14:00', '20150611 15:00'), 1, 'First meeting')
![Page 20: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/20.jpg)
Querying rangesFind any overlapping entries
Is the conference room free?Let's say from 14:30-15:30
Rapidly becomes complicated
![Page 21: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/21.jpg)
Querying rangesSELECT * FROM meetingsWHERE room = 1 AND ( '20150611 14:30' <= start_time AND '20150611 15:30' >= start_time AND '20150611 14:30' <= end_time AND '20150611 15:30' <= end_time)OR (...
And what about open/closed ranges?
![Page 22: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/22.jpg)
Querying rangesSELECT * FROM meetings_rWHERE room=1 AND blocktime && tstzrange('20150611 14:30','20150611 15:30')
![Page 23: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/23.jpg)
Querying ranges&& is an indexable operator!GiST and SP-GiST indexesIncluding multi-key!
![Page 24: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/24.jpg)
Constraining rangesConcurrency for checkIs the room available or not?But someone books it while we're typing!Can use EXCLUSION constraints
![Page 25: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/25.jpg)
Constraining rangesALTER TABLE meetings_r ADD CONSTRAINT no_double_bookings EXCLUDE USING GIST ( room WITH =, blocktime WITH && )
![Page 26: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/26.jpg)
Over to queries
![Page 27: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/27.jpg)
Ordered set aggregates
![Page 28: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/28.jpg)
Ordered set aggregates"Offset in group" aggregatesWITHIN GROUPAlso hypothetical aggregates
![Page 29: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/29.jpg)
Ordered set aggregatesMost common value in group
SELECT a, mode() WITHIN GROUP (ORDER BY b)FROM agg GROUP BY a
![Page 30: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/30.jpg)
Ordered set aggregatesPercentiles
SELECT a, percentile_cont(0.3) WITHIN GROUP (ORDER BY b), percentile_disc(0.3) WITHIN GROUP (ORDER BY b)FROM agg GROUP BY a
![Page 31: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/31.jpg)
Ordered set aggregatesHypothetical rows
SELECT a, rank(4) WITHIN GROUP (ORDER BY b), percent_rank(4) WITHIN GROUP (ORDER BY b)FROM agg GROUP BY a
![Page 32: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/32.jpg)
Unstructured data
![Page 33: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/33.jpg)
JSON
![Page 34: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/34.jpg)
JSONB"Binary json"Parsed JSON dataPrevious json datatype just stores textBasic datatyping
![Page 35: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/35.jpg)
JSONBKey-independent indexesNested structure supportContainment operators (and others)
![Page 36: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/36.jpg)
JSONBJust another datatypeMax 1GB, automatically compressed
CREATE TABLE jsontable ( .. columns .., j JSONB)
![Page 37: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/37.jpg)
JSONBINSERT INTO jsontable (..., j)VALUES (...,'"name": "Magnus", "skills": "database": ["sql", "postgres"], "other": ["something"], ')
![Page 38: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/38.jpg)
JSONB operatorsKey and sub-object access is easy
SELECT j>>'name' FROM jsontable
SELECT j>'skills' FROM jsontable
SELECT j>'skills'>'database' FROM jsontable
SELECT j#>'skills,database,0' FROM jsontable
![Page 39: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/39.jpg)
JSONB searchingEasy key searchingRequires key per index
Similar to MongoDB etcSELECT * FROM jsontableWHERE j>>'name' = 'Magnus'
But you didn't need jsonb for that
![Page 40: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/40.jpg)
JSONB searchingPath operators are more powerful!
SELECT * FROM jsontable WHERE j @> '"name":"Magnus"'
Support for deeper pathsSELECT * FROM jsontable WHERE j @> '"skills": "database": ["postgresql"] '
![Page 41: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/41.jpg)
JSONB indexingGeneric jsonb index
CREATE INDEX json_idx ON jsontable USING gin(j)
Path operator only indexCREATE INDEX json_idx ON jsontable USING gin(j jsonb_path_ops)
![Page 42: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/42.jpg)
JSONB indexesVery efficient indexes!Small!Fast (especially jsonb_path_ops)Some benchmarks:
1 million rows200 fields in json(Thanks Christophe Pettus!)
![Page 43: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/43.jpg)
JSONB indexesavg query time (ms)
![Page 44: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/44.jpg)
JSONB indexesavg query time (ms)
![Page 45: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/45.jpg)
Relational still winsavg query time (ms)
![Page 46: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/46.jpg)
Be smart!Relational still fasterIf your data fits relational modelCombine both!Known fields get a columnDynamic fields share a jsonb
![Page 47: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/47.jpg)
Summary
![Page 48: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/48.jpg)
New and cool in PostgreSQLPlenty of new things!Plenty of cool things!Go try it out!
http://www.postgresql.org/download/
![Page 49: New and cool in PostgreSQL · 2016. 2. 28. · ConFoo 2016 Montreal, Canada Magnus Hagander magnus@hagander.net. Magnus Hagander Redpill Linpro Infrastructure services Principal database](https://reader033.fdocuments.net/reader033/viewer/2022051807/6006dc55afcb946f7253f324/html5/thumbnails/49.jpg)
Thank you!Magnus Hagander
[email protected] @magnushagander
http://www.hagander.net/talks/
This material is licensed