Migrating to PostgreSQL · 2019-10-24 · Migrating to PostgreSQL / PgConf.EU Milano, 16 October...

https://www.2ndQuadrant.com

Migrating to PostgreSQL / PgConf.EUMilano, 16 October 2019

Boriss MejíasConsultant - 2ndQuadrant

Air Guitar Player

Migrating to PostgreSQL

Why Migrate to PostgreSQL?

PostgreSQL● Open Source

– Supported– Extendable

● Advanced● Reliable● Standard Compliant

PostgreSQL● It’s an All-Rounder

– Low Latency– Big Data– High Availability– “Document” Database

● Sometimes better than dedicated solutions– Scale to petabytes (from Elasticsearch)

PostgreSQL

● Awesome Community

Why Migrate to Open Source?

Reason #1 is “Cost”

Reason #1 is “Cost”(or it used to be)

Top Reasons to Stay in Open Source1. Competitive features, innovation2. Freedom from vendor lock-in3. Quality of solutions4. Ability to customize and fix5. Cost

https://www.slideshare.net/blackducksoftware/ 2016-future-of-open-source-survey-results

Migration Timeline● Effort Assessment● Decision (is it worth?)● Preparation● Testing● Migration● Cleanup

Effort Assessment● Schema● Data● Code

– What language? (SQL / Other)– Where? (Client / Server)

● Architecture

Schema● Usually the easiest part

– Available via common tools● Map data types as appropriate

– Look for simplifications● Consider custom datatypes

– Simpler is better than complex– Complex is better than complicated

Zen of Python

Data Type● PostgreSQL has several data types● Classical: text, numbers, boolean, time/date● Modern: Arrays, JSON● User-defined:

– Composite– Enumerative– Your data type in C

Data Type Gotcha● Oracle NUMBER to NUMERIC● MySQL BOOLEAN to BOOLEAN● Oracle NULL

SELECT first_name

|| second_name

|| last_name;

Data Type Gotcha● Oracle NUMBER to NUMERIC● MySQL BOOLEAN to BOOLEAN● Oracle NULL

SELECT first_name

|| COALESCE(second_name, '')

|| last_name;

Architecture Assessment● High Availability● Disaster Recovery● Multi-Master● Selective Replication

Target Architecture

Solutions Mapping● Rich PostgreSQL ecosystem

– Core– Contrib / Extensions– Third Party (both FLOSS and proprietary)

● Sometimes difficult to find exact match– Might not be needed– You must match the purpose, not the tool

Application Code● Many programming languages and

frameworks have PostgreSQL drivers– Not an issue (usually)

● Real issue: SQL variants with different feature sets:– Emulate missing features– Remove useless emulations

Application Code Gotcha● SELECT 1 FROM DUAL;● Upper case default in Oracle

– CREATE TABLE DUAL ();– DUAL → dual → “DUAL”

● Exceptions in store procedures

Planning the Migration● The Assessment includes (at least) one Plan

– Time– Cost– Contingency / Rollback

Take Advantage of the Application● Some applications support multiple databases● They have done all the major part of the work● Functions, procedures, data types. It all works

already with PostgreSQL● Just need to migrate the data

Don’t Panic

Vanilla Deployment

Bilberry Deployment

Get the Schema Definition

Migrate Data

Migrate Data Downtime→ Downtime

Redirect the Application

Redirect the Application Showtime!→ Downtime

Alternative Strategy: Phasing Out● Use PostgreSQL for new services

– Keep old services as they are● Useful when standard plans are too

complicated / expensive● No need to migrate old data and code

– Easier: “just” plan new system

Phasing Out and Integration● Integrate new PostgreSQL with existing DBs● Preserve continuity of services● Foreign Data Wrappers

– Pluggable adaptors for other systems– SQL/MED standard– Some of them are read/write

Alternative Strategy: Preparation● Modify the existing system before migrating● Make it nearer to PostgreSQL

– Stop using incompatible features– Rewrite/simplify queries

● Enables application compatibility● Makes migration easier / cheaper / faster

Testing● Compatibility● Performance

● The migration process includes writing tests

Performance Testing● Test must include difficult / critical queries● Ensure that newer optimisations don’t cause

regressions on other queries● Use pgbench (custom scripts)● Analyse the current workload● Reproduce it● Properly dimension hardware

Scripted Migration● The migration procedure should be scripted as

much as possible● A script can be:

– Repeated– Versioned– Benchmarked– Tested

– Repeated– Versioned– Benchmarked– Tested (in staging environment)

– Repeated– Versioned– Benchmarked– Tested (in staging environment, please!)

Thoughts● Focus on the purpose not on emulating● Make a plan● Test, test, test● Learn PostgreSQL

Thoughts● Focus on the purpose not on emulating● Make a plan● Test, test, test, test, test, test, test● Learn PostgreSQL

Thoughts● Focus on the purpose not on emulating● Make a plan● Test, test, test, test, test, test, test● Learn PostgreSQL and get help

Thoughts● Focus on the purpose not on emulating● Make a plan● Test, test, test, test, test, test, test● Learn PostgreSQL and get help● Don’t Panic

Thanks and RememberBenjamin Zander’s Rule #6

Boriss Mejiasboriss.mejias@2ndquadrant.com

@tchorix

Migrating to PostgreSQL · 2019-10-24 · Migrating to PostgreSQL / PgConf.EU Milano, 16 October...

Documents

Transcript of Migrating to PostgreSQL · 2019-10-24 · Migrating to PostgreSQL / PgConf.EU Milano, 16 October...

Hacking PostgreSQL with · Hacking PostgreSQL with Eclipse Metin Döşlü metin@citusdata.com PGCONF.EU 2017 1

Visiting The Catalog - PostgreSQL · Maintenance of all databases at KOF: PostgreSQL, Oracle, MySQL and MSSQL Server. Focus on migrating to PostgreSQL Support in business process

Migrating a PostgreSQL Database to SQL Anywhere 12ftp2.ianywhere.jp/tech/Migration_from_PostgreSQL_To_SQ...Migrating a PostgreSQL Database to SQL Anywhere 12 December 2011 5 spaces

Wri$ng’DjangoExtensionsfor’ PostgreSQL’ · Wri$ng’DjangoExtensionsfor’ PostgreSQL’ Jonathan’S.’Katz Exco’Ventures’ PGConf.eu’2011’–Oct20,’2011’

Migrating from MySQL to PostgreSQL

Migrating to PostgreSQL, the new story - Percona Live, 2016 · PostgreSQL Data Access Service Application Suite Durability and Availability Consistency Full Text Search, PostGIS Proper

Red Hat Software Collections 3 · PDF fileMigrating from the PostgreSQL 9.5 Software Collection to the PostgreSQL 9.6 Software Collection 5.5. MIGRATING TO NGINX 1.12 C A T A D N LR

Connecting the Debian and PostgreSQL worlds · Connecting the Debian and PostgreSQL worlds Christoph Berg christoph.berg@credativ.de myon@debian.org PGconf.EU 2011, Amsterdam October

Migrating off of MongoDB to PostgreSQL

Migrating Oracle queries to PostgreSQL · Migrating Oracle queries to PostgreSQL PGConf.EU 2012 Prague, Czech Republic October 26th Alexey Klyukin, Command Prompt, Inc.

PostgreSQL 9.0に対応！ PowerGresのご紹介€¦ · PostgreSQL 7.3 PowerGres V1 PostgreSQL 7.4 PowerGres V2 PostgreSQL 8.0 PowerGres V3 PostgreSQL 8.1 PowerGres V4 PostgreSQL

Migrating to PostgreSQL...Migrating to PostgreSQL / PgConf.EU Milano, 16 October 2019 Top Reasons to Stay in Open Source 1.Competitive features, innovation 2.Freedom from vendor lock-in

Experiences from Migrating from Oracle to PostgreSQL2017.pgconf.in/wp-content/uploads/2016/05/Experiences_From... · Experiences from Migrating from Oracle to PostgreSQL ... The Oracle

PgConf.eu, Tallinn, November 2016 Honza Horak

Red Hat Software Collections 3...5.7.2. Migrating from a Red Hat Enterprise Linux System Version of PostgreSQL to the PostgreSQL 9.6 Software Collection 5.7.3. Migrating from the PostgreSQL

How can I backup my PGConf.eu 2019 PostgreSQL databases? · I’m down. Yesterday my PostgreSQL cluster had a storage problem. I can’t bring it up. Don’t let me down! You can’t

Migrating from PostgreSQL to MySQL at Cocolog Naoto Yokoyama, NIFTY Corporation Garth Webb, Six Apart Lisa Phillips, Six Apart Credits: Kenji Hirohama,

Migrating from PostgreSQL to MySQL at Cocolog, Japan's Largest Blog Community

PostgreSQL vs MySQL: PostgreSQL como alternativa.

Data warehousing with PostgreSQL - PostgreSQL wiki