Asynchronous Replication in MySQL (and what came after it)
-
Upload
kristian-koehntopp -
Category
Internet
-
view
399 -
download
0
Transcript of Asynchronous Replication in MySQL (and what came after it)
![Page 1: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/1.jpg)
MySQL 5.6 Replication for AdminsKristian Köhntopp
![Page 2: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/2.jpg)
How async replication works2
Master Slave
Binlog RelayLog
ConnectionThread
IOThread
SQLThread
To Tables
Slavelogs in to Master
Server ID: x Server ID: y
![Page 3: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/3.jpg)
Setting up a master
• --server_id = x
• --binlog_format = STATEMENT|ROW|MIXED
• --expire_logs_days = n
• --log_bin = name -- Restart required
• SHOW MASTER STATUS
• SHOW MASTER LOGS
3
![Page 4: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/4.jpg)
Making a full backup
• Slave start position:
• A consistent backup associated with a binlog position.
• mysqldump --master-data=2 OR
• mylvmbackup
4
![Page 5: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/5.jpg)
Setting up a slave
• --server-id= y
• Recover from backup.
• Slave now at backups binlog position.
• CHANGE MASTER TOMASTER_HOST, MASTER_PORT,MASTER_USER, MASTER_PASSWORD MASTER_LOG_FILE, MASTER_LOG_POS
5
![Page 6: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/6.jpg)
Starting the slave
• SHOW SLAVE STATUS\G
• START SLAVE IO_THREAD; • SHOW SLAVE STATUS\G -- error.log
• START SLAVE SQL_THREAD; • SHOW SLAVE STATUS\G
6
![Page 7: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/7.jpg)
Replication stuck
• set global sql_slave_skip_counter = 1;
• start slave;
• But you need to understand what happened.
• Percona Toolkit:pt-table-sync,pt-table-check
7
![Page 8: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/8.jpg)
Binlog Management
• max-binlog-size
• FLUSH LOGS
• SHOW MASTER LOGS
• PURGE MASTER LOGS BEFOREnow() - INTERVAL 3 DAY
• expire_logs_days
8
![Page 9: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/9.jpg)
Debugging Replication
• mysqlbinlog --help
• --start-datetime, --stop-datetime
• useful only for zooming in, time is a binlog position!
• --verbose is not what you think (-v -vv)
• BINLOG statement decoder
9
![Page 10: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/10.jpg)
Filtering the Binlog
• Do not use binlog-do/ignore-db
• incomplete binlog = no roll forward
• Do not use replicate-do/ignore-db, replicate-do/ignore-table.
• Do use replicate-wild-do/ignore-table.
• Do not mix do and ignore, if at all possible.
10
![Page 11: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/11.jpg)
Unslaving
• STOP SLAVE
• maybe START SLAVE UNTIL MASTER_LOG_FILE = …, MASTER_LOG_POS = …
• RESET SLAVE
11
![Page 12: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/12.jpg)
Files
• On the master:
• binlog.index, binlog.nnnnnn
• On the slave:
• master.info
• relay.info
• relay.nnnnnn
12
![Page 13: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/13.jpg)
Binlog Formats: SBR vs. RBR
• “There is only SBR.”
• binlog_format = ROW|MIXED|STATEMENT generates: BINLOG “<base64>”contains: before-image, after-image
• undebuggable, so 5.6 adds binlog_rows_query_log_events = 1
13
![Page 14: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/14.jpg)
RBR Binlog Sizes
• Typical usage: RBR is 33% to 50% size of SBR.
• RBR size escalates when BLOB types are part of replication.
• max_allowed_packet problem (when FULL is used) slave-max-allowed-packet= (2x normal size)
• 5.6: binlog_row_image = FULL|NOBLOB|MINIMAL
14
![Page 15: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/15.jpg)
Alternate RBR uses
• RBR decoding with “mysqlbinlog -v -v”
• strict format, easily parseable, complete list of changes to the server
• Good for change auditing, change extraction.
• Low overhead, avoids triggers.
15
![Page 16: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/16.jpg)
Reliability
• Starting with 5.6:
• Binlog CRC32 Checksums included, never checked.
• master_verify_checksum = 1
• slave_sql_verify_checksum=1
• Always: sync_binlog = 1
• fast, when group commit is on.
16
![Page 17: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/17.jpg)
Reliability
• Slave using tables: fast w/ group commit.
• master-info-repository=TABLErelay-log-info-repository=TABLE
• Hostnames are part of filenames.
• Makes cloning hard. Configure away!
17
![Page 18: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/18.jpg)
Semi-Synchronous Replication SSR
• enabled & one ssr slave connects
• after commit wait for ack or timeout
• if timeout, async,until SSR catches up and acks
18
![Page 19: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/19.jpg)
Pre-SSR workarounds
• Heartbeat tables
• INSERT INTO heartbeat VALUES (name, timestamp)
• master_pos_wait()
• limited to single level hierarchies
19
![Page 20: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/20.jpg)
SSR background
• it makes it much more likely that fewer transactions will be lost when a master disappears
• it throttles busy clients to run no faster than master-slave networking
• http://www.mysqlperformanceblog.com/2012/01/19/how-does-semisynchronous-mysql-replication-work/
20
![Page 21: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/21.jpg)
SSR enhancement
• Google/Percona/Maria 10 SSR enchangement:
• SSR send first, local commit afterwards
• Slow in Oracle MySQL due to global txn lock, fast is Maria 10
21
![Page 22: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/22.jpg)
SSR Master
• INSTALL PLUGIN rpl_semi_sync_master SONAME 'semisync_master.so';
• show plugins;
• set global rpl_semi_sync_master_enabled = 1; # my.cnf
• set global rpl_semi_sync_master_timeout = 5000;
• # millis, add to my.cnf
22
![Page 23: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/23.jpg)
SSR slave
• INSTALL PLUGIN rpl_semi_sync_slave SONAME 'semisync_slave.so';
• show plugins;
• set global rpl_semi_sync_slave_enabled = 1; # my.cnf
• stop slave io_thread;
• start slave io_thread;
23
![Page 24: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/24.jpg)
SSR checking
• select @@global.rpl_semi_sync_master_clients;
• select @@global.rpl_semi_sync_master_status;
• select @@global.rpl_semi_sync_slave_status;
24
![Page 25: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/25.jpg)
MariaDB Multisource Replication MSR
• Consolidate multi non-overlapping masters into one slave
• all slave-to-master config and vars named, duplicated
• set @@default_master_connection = …
• https://mariadb.com/kb/en/mariadb/mariadb-documentation/replication-cluster-multi-master/replication/multi-source-replication/
25
![Page 26: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/26.jpg)
MariaDB 10 Parallel Replication
• Record degree of parallelism on master
• Replay in parallel on slave
• min(#sql_thds, master_parallelism)
• Faster slave when master more busy.
• Delay transactions on master to force parallism
• http://kristiannielsen.livejournal.com/18435.html
26
![Page 27: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/27.jpg)
PR config
• slave-parallel-threads = 12
• slave_parallel_max_queued = <buffer in bytes>
• execution is still running in-order
• ordering late in commit stage
27
![Page 28: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/28.jpg)
PR config
• On master, force parallel:
• binlog_commit_wait_count = 100
• binlog_commit_wait_usec = 100000
• Also:
• innodb_flush_logs_at_trx_commit=1
• sync_binlog = 1
28
![Page 29: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/29.jpg)
PR out-of-order execution
• Domain-ID
• application controlled
• set session gtid_domain_id = 1
• alter table t add index i(i)
• set session gtid_domain_id = 0
29
![Page 30: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/30.jpg)
Upcoming technology
• GTID
• SSR + GTID = easier master HA
• Really?
• Galera
• proper “Multi-Master”
• for small values of Multi-Master
30
![Page 31: Asynchronous Replication in MySQL (and what came after it)](https://reader036.fdocuments.net/reader036/viewer/2022062412/587b7c9c1a28ab9d448b4573/html5/thumbnails/31.jpg)
Upcoming technology
• All new replication technology requires:
• Clean setup, defined as:
• pure InnoDB, full ACID, RBR, clean transactions, QC off, log to file (not table), innodb_autoinc_lock_mode = 2, innodb_doublewrite = 1, no replication filters, sensible txn size (1000-10000 rows, < 50MB), PRIMARY KEYS on all tables
31