The PostgreSQL Replication Protocol
Transcript of The PostgreSQL Replication Protocol
![Page 1: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/1.jpg)
The PostgreSQL Replication Protocol
Tools and opportunities
Postgres Open 2011Chicago, IL
Magnus [email protected]
PRODUCTS • CONSULTING • APPLICATION MANAGEMENT • IT OPERATIONS • SUPPORT • TRAINING
![Page 2: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/2.jpg)
PostgreSQL Replication● Added in PostgreSQL 9.0● Based on streaming WAL (Transaction
Log)● Starts from base backup● Uses standard recovery code● Layered on top of regular protocol
![Page 3: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/3.jpg)
Parts of the puzzle● Connection processing and startup● The PostgreSQL protocol● The replication specific protocol● pg_basebackup
![Page 4: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/4.jpg)
Normal client connection
1. TCP connection established (5432)
![Page 5: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/5.jpg)
Normal client connection
1. TCP connection established (5432)
2. fork()
![Page 6: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/6.jpg)
Normal client connection
1. TCP connection established (5432)
2. fork()
3. SSL negotiation
![Page 7: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/7.jpg)
Normal client connection
1. TCP connection established (5432)
2. fork()
3. SSL negotiation
4. Get database/username/options
![Page 8: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/8.jpg)
Normal client connection
1. TCP connection established (5432)
2. fork()
3. SSL negotiation
4. Get database/username/options
5. Perform authentication
![Page 9: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/9.jpg)
Normal client connection
1. TCP connection established (5432)
2. fork()
3. SSL negotiation
4. Get database/username/options
5. Perform authentication
6. Select database
![Page 10: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/10.jpg)
Normal client connection
1. TCP connection established (5432)
2. fork()
3. SSL negotiation
4. Get database/username/options
5. Perform authentication
6. Select database
7. Enter query processing loop
![Page 11: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/11.jpg)
Replication client
1. TCP connection established (5432)
2. fork()
3. SSL negotiation
4. Get database/username/options (fixed)
5. Perform authentication
6. Select database
7. Enter query processing loop
![Page 12: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/12.jpg)
Replication client
1. TCP connection established (5432)
2. fork()
3. SSL negotiation
4. Get database/username/options
5. Perform authentication
6. Start walsender
![Page 13: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/13.jpg)
What's the walsender?!● Special purpose PostgreSQL backend● Not connected with a database● Only accepts simple queries● Returns mix of resultsets and streams● 9.0: only basic log streaming
● Client connects, requests WAL streaming starting at position <x>
![Page 14: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/14.jpg)
The PostgreSQL protocol● Very simple● Always TCP● Message-based, bi-directional● Optionally SSL encrypted
● Entire stream wrapped
![Page 15: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/15.jpg)
A message
MessageType (byte)
Message Length(32-bit) Message...
![Page 16: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/16.jpg)
Standard query exchange
Z <size> <Transaction Status>ReadyForQuery
Q 13 SELECT 1,2,3 SimpleQuery
T <size> <col1>,<col2>,<col3>RowDescription
D <size> 1,2,3DataRow
C <size> SELECTCommandTag
![Page 17: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/17.jpg)
Streaming replication
Q 22 START_REPLICATION 0/0 SimpleQuery
W <size> 0,0CopyOutResponse
d <size> <xlog data>CopyData
d <size> <xlog data>CopyData
Z <size> <Transaction Status>ReadyForQuery
![Page 18: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/18.jpg)
Advances in 9.1● Synchronous replication
● (not going to cover that)● Hot Standby Feedback Loop
● (not going to cover that)● Walsender “micro language”
![Page 19: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/19.jpg)
Walsender micro-language● Full grammar in walsender mode● Few commands, few options● Still very picky about formats● Not designed for manual consumption● Foundation for future improvements
![Page 20: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/20.jpg)
Walsender in 9.1● IDENTIFY_SYSTEM● START_REPLICATION <position>● BASE_BACKUP
[LABEL 'label'][PROGRESS][FAST][WAL][NOWAIT]
![Page 21: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/21.jpg)
Base backups● Single-command base backups● No need for separate
pg_start_backup()/pg_stop_backup()● Can still control backup label● Can still control fast/slow checkpoint
● Not a silver bullet● Old method is still there!
![Page 22: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/22.jpg)
Base backups● Still not for manual consumption● Use bin/pg_basebackup● Integration in third party modules and
applications
![Page 23: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/23.jpg)
Streaming base backups● Tar format stream
● Easy to stream● No global archive header● Alignment-at-512-bytes cheap
● One tar stream per tablespace● Sequential transmission
![Page 24: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/24.jpg)
Streaming base backups
Z <size> <Transaction Status>ReadyForQuery
Q 24 BASE_BACKUP LABEL 'foo' SimpleQuery
T <size> spcoid, spclocation, sizeRowDescription
D <size> <tablespace information>DataRow
C <size> SELECTCommandTag
![Page 25: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/25.jpg)
Streaming base backups
H <size> 0,0CopyOutResponse
d <size> <tar data>CopyData
c <size>CopyDone
H <size> 0,0CopyOutResponse
d <size> <tar data>CopyData
…..
![Page 26: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/26.jpg)
Using pg_basebackup
● pg_basebackup-D <directory>-F<p|t>-c <fast|spread>-l <label>-z
● Plus all “standard” libpq client options
![Page 27: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/27.jpg)
Progress reporting● Add -P to the commandline● Expensive!
● Scans all tablespaces twice● Inexact – but gives a good hint
![Page 28: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/28.jpg)
Base backups and WAL● Restore from base backup requires WAL
archiving● Complex to set up and monitor
● Append WAL to command, or use -x● walsender includes required WAL files at
end of tar file● Use wal_keep_segments!
![Page 29: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/29.jpg)
Future improvements
![Page 30: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/30.jpg)
Streaming WAL archive● Log archiving still uses archive_command● 16Mb-blocks, or archive_timeout● Replication protocol already does this● pg_receivexlog
![Page 31: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/31.jpg)
Prevent WAL cycling● WAL cycled normally during backups● In -x mode, might still be needed● If cycled too soon, backup fails
![Page 32: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/32.jpg)
WAL streaming during backup
● Combine streaming wal archive with pg_basebackup
● During backup, log is streamed in parallel● Less WAL to keep on master
![Page 33: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/33.jpg)
Relocatable tablespaces● Currently, only $PGDATA can be moved● In theory...● Support moving other tablespaces● Both for streaming and regular base
backups!
![Page 34: The PostgreSQL Replication Protocol](https://reader030.fdocuments.net/reader030/viewer/2022021305/620731dd49d709492c2ede7b/html5/thumbnails/34.jpg)
Incremental backups● “rsync” style?● Using LSN?● Decrease size of log archive without more
full backups