Intro to PostgreSQL - joeconway.com · Intro to PostgreSQL Joe Conway [email protected]...
Transcript of Intro to PostgreSQL - joeconway.com · Intro to PostgreSQL Joe Conway [email protected]...
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Binary Packages
RPMs
Debian packages
Windows installer
Source?
Joe Conway PostgreSQL@SCaLE15x 2/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Build Dependencies
On Red Hat 7.x (or variant)
git (PostgreSQL source tree)Build Dependencies (might not be all inclusive)
yum install git bison flex gcc ccache readline-devel \
zlib-devel perl-devel perl-ExtUtils-Embed openssl-devel \
pam-devel libxml2-devel libxslt-devel libuuid-devel \
openldap-devel tcl-devel python-devel gdb
Joe Conway PostgreSQL@SCaLE15x 3/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Clone Source
mkdir -p /opt/src/pgsql-git
cd /opt/src/pgsql-git/
git clone git://git.postgresql.org/git/postgresql.git
cd postgresql
# checkout the 9.5 tree
git checkout REL9_5_STABLE
http://www.postgresql.org/docs/9.5/static/git.html
Joe Conway PostgreSQL@SCaLE15x 4/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Configure and Build
Already have PGDG RPMs installed
Want very similarly configured source tree
Capture and modify CFLAGS⇒ Change -O2 -g to -O0 -g3
⇒ Remove -Wp,-D\_FORTIFY\_SOURCE=2
export PATH=/usr/pgsql-9.5/bin:$PATH
eval ./configure $(pg_config --configure | \
sed -e 's/-O2 -g/-O0 -g3/g' \
-e 's/-Wp,-D\_FORTIFY\_SOURCE=2//g')
make -j 8
make install
Joe Conway PostgreSQL@SCaLE15x 5/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Debug Workflow
Log in using psql
Find backend process pid
Attach with gdb
Set breakpoint
Execute problematic SQL statement
Joe Conway PostgreSQL@SCaLE15x 6/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
initdb
Source build/do-it-yourself<path-to-postgres>/bin/initdb -D $PGDATA
Red Hat/CentOS 7.x
/usr/pgsql-9.5/bin/postgresql95-setup initdb
creates cluster and config files in /var/lib/pgsql/9.5/data/
Debian-based
pg_createcluster 9.5 main
creates cluster in /var/lib/postgresql/9.5/main
config files created in /etc/postgresql/9.5/main
Joe Conway PostgreSQL@SCaLE15x 7/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Starting PostgreSQL
Single user modepostgres --single -D $PGDATA dbname
Manualpg_ctl -D $PGDATA -l /path/to/logfile start
Red Hat/CentOS 6.xservice postgresql start
Red Hat/CentOS 7.xsystemctl start postgresql-9.5
Debian-basedpg_ctlcluster 9.5 main start
Joe Conway PostgreSQL@SCaLE15x 8/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Stopping PostgreSQL
Single user modeControl+D (type EOF character)
Manualpg_ctl -D $PGDATA stop -m fast
Red Hat/CentOS 6.xservice postgresql stop
Red Hat/CentOS 7.xsystemctl stop postgresql-9.5
Debian-basedpg_ctlcluster 9.5 main stop
Joe Conway PostgreSQL@SCaLE15x 9/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Shutdown Modes
smart: wait until existing sessions exit
fast: gracefully terminate existing sessions (default)
immediate: kill all processes
pg_ctl -D $PGDATA stop -m immediate
Joe Conway PostgreSQL@SCaLE15x 10/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Terminate Particular SessionIn bash terminal:
ps -fu postgres |grep test
postgres 30999 1837 0 16:56 ? 00:00:00 postgres: postgres test [local] idle
kill -SIGTERM 30999
In psql:
SELECT pid, state, clock_timestamp() - state_change as age, query
FROM pg_stat_activity WHERE datname = 'test';
pid | state | age | query
-------+---------------------+-----------------+--------
31255 | idle in transaction | 00:00:26.020821 | begin;
(1 row)
SELECT pg_terminate_backend(31255);
Joe Conway PostgreSQL@SCaLE15x 11/89
InstallationConfiguration
SQLAppendix
TypicalDebug Build From SourceCluster InitializationPostgreSQL Service Control
Cancel Long Running QueriesIn bash terminal:
ps -fu postgres |grep test
postgres 30999 1837 0 16:56 ? 00:00:00 postgres: postgres test [local] idle
kill -SIGINT 30999
In psql:
SELECT pid, state, clock_timestamp() - state_change as age, query
FROM pg_stat_activity WHERE datname = 'test';
pid | state | age | query
-------+---------------------+-----------------+--------
31255 | idle in transaction | 00:00:26.020821 | begin;
(1 row)
SELECT pg_cancel_backend(31255);
Joe Conway PostgreSQL@SCaLE15x 12/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Configuration
File postgresql.conf and postgresql.auto.conf:
# comment
name = value
To activate configuration changes:
psql -c "SELECT pg_conf_reload()" # as postgres
pg_ctl -D $PGDATA reload # as postgres
service postgresql reload # typically as root
systemctl reload postgresql-9.5 # typically as root
kill -SIGHUP <postmaster-pid> # as postgres or root
http://www.postgresql.org/docs/current/interactive/runtime-config.html
Joe Conway PostgreSQL@SCaLE15x 13/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Configuration
Persistent
Modify: postgresql.conf
Add: postmaster options (e.g. in startup script)With SQL:
ALTER SYSTEM SET, ALTER SYSTEM RESET
ALTER [DATABASE] <dbname> SET <var> = <val>;
ALTER [ROLE] <rolename> SET <var> = <val>;
Per Session (SQL)
SET, RESET, SHOWSELECT * FROM pg_settings;
UPDATE pg_settings SET setting = '<val>' WHERE name = '<var>';
SELECT current_setting('<var>');
SELECT set_config('<var>','<val>','<local T/F>');
Joe Conway PostgreSQL@SCaLE15x 14/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Connection Management
listen_addresses = 'localhost'
port = 5432
max_connections = 100
superuser_reserved_connections = 2
ssl = false
http://www.postgresql.org/docs/current/interactive/runtime-config-connection.html
Joe Conway PostgreSQL@SCaLE15x 15/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Memory Management
shared_buffers = 128MB
huge_pages = try
work_mem = 4MB
maintenance_work_mem = 64MB
shared_preload_libraries = ''
http://www.postgresql.org/docs/current/interactive/runtime-config-resource.html
Joe Conway PostgreSQL@SCaLE15x 16/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Write-Ahead Log
wal_level = minimal
fsync = on
synchronous_commit = on
checkpoint_timeout = 5min
# checkpoint_segments = 3 (pre-9.5)
max_wal_size = 1GB
min_wal_size = 80MB
checkpoint_completion_target = 0.5
checkpoint_warning = 30s
http://www.postgresql.org/docs/current/interactive/runtime-config-wal.html
Joe Conway PostgreSQL@SCaLE15x 17/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Planner
effective_cache_size = 128MB
seq_page_cost = 1.0
random_page_cost = 4.0
http://www.postgresql.org/docs/current/interactive/runtime-config-query.html
Joe Conway PostgreSQL@SCaLE15x 18/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Logging
log_destination = 'stderr' # stderr, syslog, csvlog, eventlog
logging_collector = off
log_directory = 'pg_log'
log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'
log_file_mode = 0600
log_truncate_on_rotation = false
log_rotation_age = 1d
log_rotation_size = 10MB
syslog_facility = 'LOCALO'
syslog_ident = 'postgres'
http://www.postgresql.org/docs/current/interactive/runtime-config-logging.html
Joe Conway PostgreSQL@SCaLE15x 19/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Logging
log_min_duration_statement = -1
log_statement = 'none' # none, mod, ddl, all
log_connections = off
log_disconnections = off
log_hostname = off
log_timezone = ''
log_lock_waits = off
log_temp_files = -1
log_checkpoints = off
log_autovacuum_min_duration = -1
Joe Conway PostgreSQL@SCaLE15x 20/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Logging
log_line_prefix = '' # special values:
# %a = application name, %u = user name,
# %d = database name, %r = remote host and port,
# %p = process ID, %m = timestamp with milliseconds,
# %i = command tag, %e = SQL state, %c = session ID,
# %l = session line number, ... and others
Joe Conway PostgreSQL@SCaLE15x 21/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Miscellaneous
search_path = '$user,public'
statement_timeout = 0
datestyle = 'iso, mdy'
timezone = unknown
http://www.postgresql.org/docs/current/interactive/runtime-config-client.html
Joe Conway PostgreSQL@SCaLE15x 22/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Host Based Authentication File
Which hosts are allowed to connect
How clients are authenticated
Which PostgreSQL user names they can use
Which databases they can access
https://www.postgresql.org/docs/9.5/static/auth-pg-hba-conf.html
Joe Conway PostgreSQL@SCaLE15x 23/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Host Based Authentication File
Read on server startup
Must reload postmaster for changes to take effect
First line matching conn type, address, database, and user is used for authentication
If line picked and authentication fails, access denied
If no line matches, access denied
https://www.postgresql.org/docs/9.5/static/auth-pg-hba-conf.html
Joe Conway PostgreSQL@SCaLE15x 24/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Host Based Authentication File
Lines (rules/records) look like this
# CONN-TYPE DATABASE USER ADDRESS METHOD OPTIONS
# local <dbname> <user> <method> [<opts>]
# host <dbname> <user> <address> <method> [<opts>]
# hostssl <dbname> <user> <address> <method> [<opts>]
# hostnossl <dbname> <user> <address> <method> [<opts>]
Joe Conway PostgreSQL@SCaLE15x 25/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Connection Type
Specifies type of connection the rule matches
local: Unix-domain socket
host: Either plain or SSL-encrypted TCP/IP socket
hostssl: SSL-encrypted TCP/IP socket
hostnossl: Plain TCP/IP socket
Joe Conway PostgreSQL@SCaLE15x 26/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Database
Specifies set of databases the rule matches
all: Wildcard
sameuser: Database name matches user name
samerole: User part of role/group matching database name
replication: all keyword does not match replication
<dbname>[,<dbname>]: One or more specific database names
@<filename>: Separate file containing names to match
Joe Conway PostgreSQL@SCaLE15x 27/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
User
Specifies set of users the rule matches
all: Wildcard
<username>[,<username>]: One or more user names
+<groupname>: Any roles that are directly or indirectly members of this role
@<filename>: Separate file containing names to match
Joe Conway PostgreSQL@SCaLE15x 28/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Address
Specifies set of client hosts the rule matches
<IPAddr>/<CIDR-Mask>: Host or Network
<IPAddr> <Mask>: Host or Network
[.]<hostname>: [suffix] actual FQ hostname
samehost: match any of server’s own IP addresses
samenet: match any address in any subnet that server directly connected to
Joe Conway PostgreSQL@SCaLE15x 29/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Method
Specifies authentication method to use when connection matches rule
trust
md5, password
cert
peer, pam, ident
gss, sspi, ldap, radius
reject
https://www.postgresql.org/docs/9.5/static/auth-methods.html
Joe Conway PostgreSQL@SCaLE15x 30/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Options
Set of options for the authentication in the format NAME=VALUE
Options depend authentication method
Refer to ”Client Authentication” section of docs
Joe Conway PostgreSQL@SCaLE15x 31/89
InstallationConfiguration
SQLAppendix
postgresql.confpg hba.conf
Brief Example
local all all trust
host all all 127.0.0.1/32 trust
host all all samenet md5
host all all 192.168.1.0/24 ldap ldapurl="ldap://ldap.ex.net/dc=ex,dc=net?uid?sub"
Joe Conway PostgreSQL@SCaLE15x 32/89
InstallationConfiguration
SQLAppendix
General
psql - Hints
psql [-h <host> -p <port> -U <user>] <dbname> [-c "some sql"]
watch -n1 "psql <dbname> -c 'some sql'"
psql -l
psql -E
https://www.postgresql.org/docs/9.5/static/app-psql.html
Joe Conway PostgreSQL@SCaLE15x 33/89
InstallationConfiguration
SQLAppendix
General
psql - Hints
<Up>
<Ctrl>-r
<tab>
Joe Conway PostgreSQL@SCaLE15x 34/89
InstallationConfiguration
SQLAppendix
General
psql - Hints
\?
\h
\dt [<schemaname>].[<tablename>][*]
\d <tablename>[*]
\df <funcname>[*]
\x
\e
\watch [ seconds ]
Joe Conway PostgreSQL@SCaLE15x 35/89
InstallationConfiguration
SQLAppendix
General
General Notes on Syntax
Identifier:
Unquoted: name (lower cased)Quoted: "Name" (preserved)Embedded Quotes: "List ""A"""
Unicode identifier: U&"\0441\043B\043E\043D"
String Literal:
Simple: 'text'
Dollar: see next slideEmbedded Quotes: 'McDonald''s restaurant'
Unicode literal: U&'\0441\043B\043E\043D'
Comments:
Single line: -- comment
Multi-line: /* comment */
Joe Conway PostgreSQL@SCaLE15x 36/89
InstallationConfiguration
SQLAppendix
General
Dollar Quoting
$<tag>$
<tag> is zero or more characters
Start and End tag must match
Particularly useful for function bodies
Works for all character strings
Nest by choosing different <tag> at each level
CREATE OR REPLACE FUNCTION dummy()
RETURNS text AS $_$
BEGIN
RETURN $$Say 'hello'$$;
END;
$_$ LANGUAGE plpgsql;
http://www.postgresql.org/docs/9.5/static/sql-syntax-lexical.html#SQL-SYNTAX-DOLLAR-QUOTING
Joe Conway PostgreSQL@SCaLE15x 37/89
InstallationConfiguration
SQLAppendix
General
Data Types - Character Strings
text: character string with variable length without upper limit
varchar(X): character string, at most X characters long
char(X): character string, exactly X characters long (padded with spaces)
Joe Conway PostgreSQL@SCaLE15x 38/89
InstallationConfiguration
SQLAppendix
General
Data Types - Numbers
smallint: integer (2 bytes) (alias: int2)
integer: integer (4 bytes) (alias: int4, int)
bigint: integer (8 bytes) (alias: int8)
serial: not a real datatype – int with a default value expression that automatically takes thenext value from a sequence
bigserial: the same using bigint as base type
real: floating-point number (4 bytes) (alias: float4)
double precision: floating-point number (8 bytes) (alias: float8)
numeric(M,N): fixed-point number, max. M digits total, thereof N digits after the decimal point
Joe Conway PostgreSQL@SCaLE15x 39/89
InstallationConfiguration
SQLAppendix
General
Data Types - Date/Time
date: date (without time)
time [without time zone]: time without time zone
time with time zone: time with time zone
timestamp [without time zone]: date and time without time zone
timestamp with time zone: date and time with time zone
interval: interval length
Joe Conway PostgreSQL@SCaLE15x 40/89
InstallationConfiguration
SQLAppendix
General
Data Types - Enumeration Types
Real enumeration type
Seamless operator and function support
CREATE TYPE color AS enum('red', 'green', 'blue');
CREATE TABLE clothes (type text, color color);
INSERT INTO clothes VALUES('shirt', 'red');
Joe Conway PostgreSQL@SCaLE15x 41/89
InstallationConfiguration
SQLAppendix
General
Data Types - Geometric
Don’t use native, use PostGIS instead
Out of scope for this talk
CREATE EXTENSION postgis;
Joe Conway PostgreSQL@SCaLE15x 42/89
InstallationConfiguration
SQLAppendix
General
Data Types - JSON
Checks well-formedness
Compliant JSON requires UTF-8 server encoding
Stored an exact copy of the input text as a string
Preserves semantically-insignificant white space between tokensPreserves order of keys within JSON objectsIf contains same key more than once, all key/value pairs are kept
Joe Conway PostgreSQL@SCaLE15x 43/89
InstallationConfiguration
SQLAppendix
General
Data Types - JSONB
Checks well-formedness
Compliant JSON requires UTF-8 server encoding
Full indexing
Stored in a decomposed binary format
Does not preserve white spaceDoes not preserve the order of object keysDoes not keep duplicate object keys (only the last value is kept)
https://www.postgresql.org/docs/9.5/static/datatype-json.html
Joe Conway PostgreSQL@SCaLE15x 44/89
InstallationConfiguration
SQLAppendix
General
Data Types - XML Type
Checks well-formedness
Requires additional support from the underlying operating system (libxml2, libxslt)
Needs some care regarding encoding issues (UTF-8 vs. others)
No comparison or indexable operators available
No DTD validation
Joe Conway PostgreSQL@SCaLE15x 45/89
InstallationConfiguration
SQLAppendix
General
Data Types - Additional
bytea: binary data
boolean: true or false
array: multidimensional arrays from elements of same scalar data type
composite: tuples from elements of any data type
range: represent a range of values of some element type
text search: designed to support full text search
other: many other types for network addresses, uuid, . . .
https://www.postgresql.org/docs/9.5/static/datatype.html
Joe Conway PostgreSQL@SCaLE15x 46/89
InstallationConfiguration
SQLAppendix
General
Thank You
Questions?
Joe Conway PostgreSQL@SCaLE15x 47/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE TABLE
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ]
table_name ( [
{ column_name data_type [ COLLATE collation ] [ column_constraint [ ... ] ]
| table_constraint
| LIKE source_table [ like_option ... ] }
[, ... ]
] )
[ INHERITS ( parent_table [, ... ] ) ]
[ WITH ( storage_parameter [= value] [, ... ] ) | WITH OIDS | WITHOUT OIDS ]
[ ON COMMIT { PRESERVE ROWS | DELETE ROWS | DROP } ]
[ TABLESPACE tablespace_name ]
https://www.postgresql.org/docs/9.5/static/sql-createtable.html
Joe Conway PostgreSQL@SCaLE15x 48/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE TABLE - Simple
Every table should have a primary key
Primary key columns are not-null implicitly
serial: auto-incrementing pseudotype
bigserial: auto-incrementing 64-bit integer pseudotype
CREATE TABLE category (
id serial PRIMARY KEY,
genre text NOT NULL,
description text
);
Joe Conway PostgreSQL@SCaLE15x 49/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE TABLE - Table Constraint
Every column constraint can be written as a table constraint
But not vice versa; example later
CREATE TABLE publisher (
id serial,
name text NOT NULL,
country text NOT NULL,
comment text,
PRIMARY KEY (id)
);
Joe Conway PostgreSQL@SCaLE15x 50/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE TABLE - Check Constraint
CREATE TABLE author (
id serial PRIMARY KEY,
lastname text NOT NULL,
firstname text NOT NULL,
primary_email text
CHECK (primary_email ~* '^[A-Za-z0-9._%-]+@[A-Za-z0-9.-]+[.][A-Za-z]+$'),
comment text
);
Joe Conway PostgreSQL@SCaLE15x 51/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE TABLE - Foreign Key
CREATE TABLE book (
id serial PRIMARY KEY,
title text NOT NULL,
subtitle text,
edition text NOT NULL,
isbn text NOT NULL,
description text,
language text,
publisher_id integer NOT NULL REFERENCES publisher (id)
);
Joe Conway PostgreSQL@SCaLE15x 52/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE TABLE - More Complex
CREATE TABLE bookauthor (
id serial,
book_id integer NOT NULL,
author_id integer NOT NULL,
comment text,
PRIMARY KEY (book_id, author_id),
FOREIGN KEY (book_id) REFERENCES book (id),
FOREIGN KEY (author_id) REFERENCES author (id)
);
Joe Conway PostgreSQL@SCaLE15x 53/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
DROP TABLE
DROP TABLE [ IF EXISTS ] name [, ...] [ CASCADE | RESTRICT ]
IF EXISTS: Do not throw error if table does not exist
CASCADE: Automatically drop dependent objects
RESTRICT: Refuse to drop if objects depend on it (default)
Joe Conway PostgreSQL@SCaLE15x 54/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
ALTER TABLE
ALTER TABLE [ IF EXISTS ] [ ONLY ] name [ * ]
action [, ... ]
ALTER TABLE [ IF EXISTS ] [ ONLY ] name [ * ]
RENAME [ COLUMN ] column_name TO new_column_name
ALTER TABLE [ IF EXISTS ] [ ONLY ] name [ * ]
RENAME CONSTRAINT constraint_name TO new_constraint_name
ALTER TABLE [ IF EXISTS ] name
RENAME TO new_name
ALTER TABLE [ IF EXISTS ] name
SET SCHEMA new_schema
ALTER TABLE ALL IN TABLESPACE name [ OWNED BY role_name [, ... ] ]
SET TABLESPACE new_tablespace [ NOWAIT ]
Changes definition of existing table
Several subforms, some examples follow
https://www.postgresql.org/docs/9.5/static/sql-altertable.html
Joe Conway PostgreSQL@SCaLE15x 55/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
ALTER TABLE ADD COLUMN
ALTER TABLE book
ADD COLUMN publisher_id integer NOT NULL;
ALTER TABLE book
ADD COLUMN price numeric(7, 2)
CONSTRAINT pos_price CHECK (price > 0);
New column is always added as last in column list
Changing logical or physical order not supported (yet)
Joe Conway PostgreSQL@SCaLE15x 56/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
ALTER TABLE DROP COLUMN
ALTER TABLE book DROP COLUMN author_id;
ALTER TABLE book DROP COLUMN author_id CASCADE;
Joe Conway PostgreSQL@SCaLE15x 57/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
ALTER TABLE - Constraints
ALTER TABLE publisher ADD PRIMARY KEY (id);
ALTER TABLE book
ADD FOREIGN KEY (publisher_id) REFERENCES publisher (id);
ALTER TABLE publisher
ADD CONSTRAINT name_unique UNIQUE (name);
ALTER TABLE publisher
ADD CONSTRAINT check_postcode
CHECK (postcode SIMILAR TO '[0-9][0-9][0-9][0-9][0-9]');
Constraint names are optional but useful for referencing constraints⇒ e.g. to DROP them
Joe Conway PostgreSQL@SCaLE15x 58/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
ALTER TABLE - Changing Not-Null
ALTER TABLE book ALTER COLUMN edition SET NOT NULL;
ALTER TABLE book ALTER COLUMN edition DROP NOT NULL;
Joe Conway PostgreSQL@SCaLE15x 59/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
ALTER TABLE - Changing Column Type
ALTER TABLE book ALTER COLUMN price
TYPE numeric(7,2);
ALTER TABLE ALTER COLUMN number
TYPE text USING to_char(number, 'RN');
Implicit type casting possible for certain data types only
Otherwise provide explicit type casting expression
Joe Conway PostgreSQL@SCaLE15x 60/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
ALTER TABLE RENAME
ALTER TABLE books RENAME TO ebooks;
Joe Conway PostgreSQL@SCaLE15x 61/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
ALTER TABLE RENAME COLUMN
ALTER TABLE number_representation
RENAME COLUMN number TO roman_numerals;
Joe Conway PostgreSQL@SCaLE15x 62/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
INSERT
[ WITH [ RECURSIVE ] with_query [, ...] ]
INSERT INTO table_name [ AS alias ] [ ( column_name [, ...] ) ]
{ DEFAULT VALUES |
VALUES ( { expression | DEFAULT } [, ...] ) [, ...] | query }
[ ON CONFLICT [ conflict_target ] conflict_action ]
[ RETURNING * | output_expression [ [ AS ] output_name ] [, ...] ]
https://www.postgresql.org/docs/current/static/sql-insert.html
Joe Conway PostgreSQL@SCaLE15x 63/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
INSERT
INSERT INTO author
VALUES (DEFAULT, 'Doe', 'Jane', '[email protected]', 'Example author');
INSERT INTO publisher (name, country)
VALUES ('Publisher A', 'US'),('Publisher B', 'UK'),('Publisher C', 'CA')
RETURNING *;
id | name | country | comment
----+-------------+---------+---------
2 | Publisher A | US |
3 | Publisher B | UK |
4 | Publisher C | CA |
(3 rows)
Joe Conway PostgreSQL@SCaLE15x 64/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
UPDATE
[ WITH [ RECURSIVE ] with_query [, ...] ]
UPDATE [ ONLY ] table_name [ * ] [ [ AS ] alias ]
SET { column_name = { expression | DEFAULT } |
( column_name [, ...] ) = ( { expression | DEFAULT } [, ...] ) |
( column_name [, ...] ) = ( sub-SELECT )
} [, ...]
[ FROM from_list ]
[ WHERE condition | WHERE CURRENT OF cursor_name ]
[ RETURNING * | output_expression [ [ AS ] output_name ] [, ...] ]
https://www.postgresql.org/docs/current/static/sql-update.html
Joe Conway PostgreSQL@SCaLE15x 65/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
UPDATE
UPDATE book SET price = 42.24
WHERE lower(title) = 'postgresql';
UPDATE author SET lastname='Bar', firstname='Foo'
WHERE lastname='Doe';
Joe Conway PostgreSQL@SCaLE15x 66/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
DELETE
[ WITH [ RECURSIVE ] with_query [, ...] ]
DELETE FROM [ ONLY ] table_name [ * ] [ [ AS ] alias ]
[ USING using_list ]
[ WHERE condition | WHERE CURRENT OF cursor_name ]
[ RETURNING * | output_expression [ [ AS ] output_name ] [, ...] ]
https://www.postgresql.org/docs/current/static/sql-delete.html
Joe Conway PostgreSQL@SCaLE15x 67/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
DELETE
DELETE FROM publisher WHERE name LIKE 'Publisher %';
DELETE FROM publisher WHERE id > 1;
DELETE FROM author WHERE primary_email ~ '.*@example.com';
http://www.postgresql.org/docs/9.3/interactive/sql-delete.html
Joe Conway PostgreSQL@SCaLE15x 68/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
Queries
[ WITH [ RECURSIVE ] with_query [, ...] ]
SELECT [ ALL | DISTINCT [ ON ( expression [, ...] ) ] ]
[ * | expression [ [ AS ] output_name ] [, ...] ]
[ FROM from_item [, ...] ] [ WHERE condition ]
[ GROUP BY grouping_element [, ...] ] [ HAVING condition [, ...] ]
[ WINDOW window_name AS ( window_definition ) [, ...] ]
[ { UNION | INTERSECT | EXCEPT } [ ALL | DISTINCT ] select ]
[ ORDER BY expression [ ASC | DESC | USING operator ]
[ NULLS { FIRST | LAST } ] [, ...] ]
[ LIMIT { count | ALL } ] [ OFFSET start [ ROW | ROWS ] ]
[ FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } ONLY ]
[ FOR { UPDATE | NO KEY UPDATE | SHARE | KEY SHARE }
[ OF table_name [, ...] ] [ NOWAIT | SKIP LOCKED ] [...] ]
https://www.postgresql.org/docs/current/static/sql-select.html
Joe Conway PostgreSQL@SCaLE15x 69/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
Queries
SELECT * FROM author;
SELECT name AS "Publisher" FROM publisher;
SELECT DISTINCT author_id FROM bookauthor;
SELECT title FROM book WHERE price > 10.00;
Joe Conway PostgreSQL@SCaLE15x 70/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
Queries
SELECT * FROM book
WHERE lower(title) LIKE '%sql%'
ORDER BY price DESC;
SELECT b.title
FROM bookauthor ba, author a, book b
WHERE a.lastname = 'Bar'
AND ba.author_id = a.id
AND b.id = ba.book_id
ORDER BY 1 DESC;
Joe Conway PostgreSQL@SCaLE15x 71/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
Joins
CROSS JOIN
INNER JOIN
OUTER JOIN
LEFT
RIGHT
FULL
Joe Conway PostgreSQL@SCaLE15x 72/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CROSS JOIN
Join each row of t1 with each row of t2
Usually written in error (first form below)
Occasionally useful (use second form to make intention clear)
SELECT * FROM t1, t2;
SELECT * FROM t1 CROSS JOIN t2;
Joe Conway PostgreSQL@SCaLE15x 73/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
INNER JOIN
Join each row of t1 with each row of t2 which matches condition
condition typically something like t1.cx = t2.cy
USING implies match on same-named columns from t1 and t2
USING also means only one of each pair included in join output
NATURAL same as USING that mentions all columns in both tables with same names
SELECT ... FROM t1, t2 WHERE condition;
SELECT ... FROM t1 [ INNER ] JOIN t2 ON condition;
SELECT ... FROM t1 [ INNER ] JOIN t2 USING (column list);
SELECT ... FROM t1 NATURAL [ INNER ] JOIN t2;
Joe Conway PostgreSQL@SCaLE15x 74/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
INNER JOIN
SELECT * FROM book AS b, publisher AS p
WHERE b.publisher_id = p.id
AND b.title ilike '%post%';
SELECT * FROM book AS b JOIN publisher AS p
ON b.publisher_id = p.id
WHERE b.title ilike '%post%';
Joe Conway PostgreSQL@SCaLE15x 75/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
OUTER JOIN
LEFT JOIN: all rows from the left table
RIGHT JOIN: all rows from the right table
FULL JOIN: all rows from both tables
Join each row of t1 with each row of t2 which matches condition
Non-matching rows on ”all” side added to the result⇒ filled out with NULLs for missing side
SELECT ... FROM t1 LEFT/RIGHT/FULL [ OUTER ] JOIN t2 ON condition;
SELECT ... FROM t1 LEFT/RIGHT/FULL [ OUTER ] JOIN t2 USING (column list);
SELECT ... FROM t1 NATURAL LEFT/RIGHT/FULL [ OUTER ] JOIN t2;
Joe Conway PostgreSQL@SCaLE15x 76/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
OUTER JOIN
SELECT * FROM publisher LEFT JOIN book
ON publisher.id = book.publisher_id;
Joe Conway PostgreSQL@SCaLE15x 77/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
Subqueries - Uncorrelated
Calculates constant result set for outer query
Executed once
SELECT
author.lastname
,author.firstname
FROM
bookauthor
,book
,author
WHERE
book.id IN (SELECT id FROM book WHERE title LIKE '%Spam%')
AND bookauthor.book_id = book.id
AND author.id = bookauthor.author_id;
Joe Conway PostgreSQL@SCaLE15x 78/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
Subqueries - Correlated
References variables from outer query
Repeated for each row of outer query
Could be rewritten as JOIN
SELECT
book_id
,author_id
FROM
bookauthor ba
WHERE
book_id = (SELECT id FROM book
WHERE id = ba.book_id);
Joe Conway PostgreSQL@SCaLE15x 79/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE VIEW
Views are just saved SQL queries
No precalculation, no special optimizations
Provide notational convenience
Simple views are automatically updatable - INSERT/UPDATE/DELETE allowed if:
Has exactly one entry in FROM list, which must be table or another updatable viewMust not contain WITH, DISTINCT, GROUP BY, HAVING, LIMIT, or OFFSETclauses at the top levelMust not contain set operations (UNION, INTERSECT or EXCEPT) at the toplevelMust not contain any aggregates, window functions or set-returning functions
Joe Conway PostgreSQL@SCaLE15x 80/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE VIEW
CREATE [ OR REPLACE ] [ TEMP | TEMPORARY ] [ RECURSIVE ]
VIEW name [ ( column_name [, ...] ) ]
[ WITH ( view_option_name [= view_option_value] [, ... ] ) ]
AS query
[ WITH [ CASCADED | LOCAL ] CHECK OPTION ]
Joe Conway PostgreSQL@SCaLE15x 81/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE VIEW
CREATE VIEW booklist AS
SELECT book.title, author.lastname, author.firstname
FROM bookauthor, book, author
WHERE bookauthor.book_id = book.id
AND author.id = bookauthor.author_id;
Joe Conway PostgreSQL@SCaLE15x 82/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
DROP VIEW
DROP VIEW [ IF EXISTS ] name [, ...] [ CASCADE | RESTRICT ]
Joe Conway PostgreSQL@SCaLE15x 83/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CREATE TABLE AS
Creates a copy
Data changes are specific to the new table
CREATE TABLE booklist AS
SELECT book.title, author.lastname, author.firstname
FROM bookauthor, book, author
WHERE bookauthor.book_id = book.id
AND author.id = bookauthor.author_id;
Joe Conway PostgreSQL@SCaLE15x 84/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
Common Table Expressions
Expressions declared by WITH
Also known as ”CTE” or ”WITH clause”
Can contain recursive references to itself
Can be thought of as in-memory temp table, private to query
Joe Conway PostgreSQL@SCaLE15x 85/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CTE - Syntax
[ WITH [ RECURSIVE ] with_query [, ...] ]
Where with_query is:
with_query_name [ ( column_name [, ...] ) ]
AS ( select | values | insert | update | delete )
When RECURSIVE specified subquery must have the form:
non_recursive_term UNION [ ALL | DISTINCT ] recursive_term
https://www.postgresql.org/docs/current/static/queries-with.html
Joe Conway PostgreSQL@SCaLE15x 86/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CTE - Simple non-recursive example
WITH t(a) AS
(
SELECT * FROM generate_series(1,10)
)
SELECT t.a * t.a FROM t;
Joe Conway PostgreSQL@SCaLE15x 87/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CTE - Simple recursive example
WITH RECURSIVE t(n) AS
(
VALUES (1)
UNION ALL
SELECT n+1 FROM t WHERE n < 100
)
SELECT sum(n) FROM t;
Joe Conway PostgreSQL@SCaLE15x 88/89
InstallationConfiguration
SQLAppendix
Data DefinitionData ModificationQueries
CTE - DML example
CREATE TABLE t1(t1_id serial PRIMARY KEY, t1_f2 text);
CREATE TABLE t2(t2_id serial, t1_id int REFERENCES t1, t2_f3 text);
WITH
wt1(t1_id, t1_f2) AS
(INSERT INTO t1 VALUES(DEFAULT, 'some f2 text') RETURNING *),
wt2(t2_id, t1_id, t2_f3) AS
(INSERT INTO t2(t1_id, t2_f3) SELECT wt1.t1_id, 'some f3 text' FROM wt1 RETURNING *)
SELECT * FROM wt1 NATURAL JOIN wt2;
t1_id | t1_f2 | t2_id | t2_f3
-------+--------------+-------+--------------
1 | some f2 text | 1 | some f3 text
(1 row)
Joe Conway PostgreSQL@SCaLE15x 89/89