Tungsten University: Configure & Provision Tungsten Clusters

©Continuent 2013

Tungsten University: Configure & Provision

Tungsten ClustersJe! Mace, Director of Services

Robert Hodges, CEO

©Continuent 2013

About Continuent

• The leading provider of clustering and replication for open source DBMS

• Tungsten Clustering - Commercial-grade HA, performance scaling and data management for MySQL

• Tungsten Replication - Flexible, high-performance replication

2

©Continuent 2013

Introducing Tungsten

3

©Continuent 2013

Application Application

Replicator Replicator Replicator

MySQL Client API MySQL Client API

MasterSlave Slave

Existing MySQL Replication

4

©Continuent 2013




MasterSlave Slave

Step 1: Replace MySQL Replication

5

©Continuent 2013




Manager Manager Manager

MasterSlave Slave

Step 2: Add Manager Process

6

©Continuent 2013




Tungsten Connector Tungsten Connector

MasterSlave Slave

Step 3. Add Intelligent Connector

7

©Continuent 2013

Data Service

Application


Application


Monitoring and control

Monitoring and control

MasterSlave Slave

Step 4. Connector Connectivity and Communication

Tungsten Connector Tungsten Connector

8

©Continuent 2013


NYC

©Continuent 2013 10

Manual Failover

NYC London

©Continuent 2013

Evaluating Tungsten

• Con"guring servers

• Viewing cluster status

• Connectivity

• Testing cluster operations

• Next steps

• Questions

11

©Continuent 2013

Con"guring Servers

12

©Continuent 2013

Create a Security Group

• Create a security group for all Tungsten servers

• It should be created in the AZ you will use for servers

13

All TCP traffic within the security groupAll UDP traffic within the security groupAll ICMP traffic within the security groupTCP port 22, 9999 and 13306 from '0.0.0.0/0'

©Continuent 2013

Launch Servers

• Create 4 EC2 servers in a single AZ

• They should be m1.large or greater

• Use the Amazon Linux AMI

• Set the root volume to be large enough for your test data set

14

©Continuent 2013

Logging Into the Servers

# SSH access must be done to the ec2-user account$> ssh [email protected]

# Then use sudo to gain access to the root user$ ip-184-72-189-135> sudo su -

15

©Continuent 2013

Set Server Hostnames

# We will use the following hostnames# db1.nyc.tu# db2.nyc.tu# db1.london.tu# db2.london.tu

$ ip-184-72-189-135> hostname db1.nyc.tu$ ip-184-72-189-135> sed -i "/^HOSTNAME=/c\HOSTNAME=`hostname`" /etc/sysconfig/network

# You must logout completely and back in # for the change to take effect

16

©Continuent 2013

Modify /etc/hosts

• Add entries to /etc/hosts on each server

• Use the private IP address for each host

17

$ db1> /sbin/ifconfig eth0 | grep "inet addr" inet addr: 10.112.24.214 Bcast: 10.112.25.255 Mask:255.255.254.0

$ db1> echo "10.112.24.214 db1.nyc.tu10.10.219.125 db2.nyc.tu10.10.102.83 db1.london.tu10.112.74.196 db2.london.tu" >> /etc/hosts

©Continuent 2013

Install Software Packages

$ db1> yum -y install mysql-server which curl bc rsync wget java-1.6.0-openjdk ruby

$ db1> rpm -i http://www.percona.com/redir/downloads/XtraBackup/LATEST/RPM/rhel6/x86_64/percona-xtrabackup-2.0.4-484.rhel6.x86_64.rpm

18

©Continuent 2013

Create MySQL Users

$ db1> mysql -e "grant all on *.* to 'tungsten'@'%' identified by 'secret' with grant option"

$ db1> mysql -e "grant all on *.* to 'app'@'%' identified by 'secret'"$ db1> mysql -e "revoke super on *.* from 'app'@'%'"

$ db1> mysql -e "delete from mysql.user where user=''"$ db1> mysql -e "flush privileges"

19

©Continuent 2013

Rinse & Repeat

• The requirements must be completed on each server

• Use a di!erent server-id

• Use the same SSH id_rsa and id_rsa.pub

• Full details at https://docs.continuent.com/wiki/display/TEDOC/System+Requirements

• Sample script available athttps://docs.continuent.com/wiki/display/TEDOC/Preparing+EC2+Servers

20

©Continuent 2013

Installation

21

• Installation completed from a staging directory using tungsten-cookbook or tpm

• tungsten-cookbook runs tpm plus some additional tests

• Staging con"guration stored in $CONTINUENT_PROFILES

©Continuent 2013

Installation

$ db1> cd /opt/continuent/software

$ db1> wget http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.22.tar.gz/from/http://cdn.mysql.com/$ db1> tar zxf mysql-connector-java-5.1.22.tar.gz

# Upload the software to /opt/continuent/software on db1$ db1> tar zxf continuent-tungsten-2.0.1-662.tar.gz$ db1> cd continuent-tungsten-2.0.1-662

# See the next slides for USER_VALUES.sh content$ db1> vi cookbook/USER_VALUES.sh

$ db1> ./cookbook/tungsten-cookbook sor --run-test# Fix any errors and repeat the command until success

$ db1> . /opt/continuent/share/env.sh

22

©Continuent 2013

Installation

23

• Installation completes several steps

• Copy software to each server

• Validate con"guration

• Write con"guration "les

• Start services

©Continuent 2013

Viewing Cluster Status

24

$ db1> trepctl status

$ db1> cctrl -multiTungsten Enterprise 1.5.3 build 59connect to '[email protected]'usa: session established

[LOGICAL] / > ls

[LOGICAL] / > use usa[LOGICAL] /usa > ls

[LOGICAL] /usa > use europe[LOGICAL] /europe > ls

[LOGICAL] /europe > use world[LOGICAL] /world > ls

©Continuent 2013

Connectivity

25

$ db1> mysql -h`hostname` -P9999 -uapp -pmysql> select @@hostname;

[LOGICAL] /usa > switchmysql> select @@hostname;

mysql> begin;mysql> select @@hostname;[LOGICAL] /usa > switchmysql> select @@hostname;

©Continuent 2013

Cluster Operations

26

©Continuent 2013

Switching the Master Server

27

$ db1> cctrl -multi[LOGICAL] /> use usa[LOGICAL] /usa> switch[LOGICAL] /usa> ls

[LOGICAL] /usa> switch to db1.nyc.tu[LOGICAL] /usa> ls

[LOGICAL] /usa> use world[LOGICAL] /world> ls

[LOGICAL] /world> switch to europe[LOGICAL] /world> ls

[LOGICAL] /world> use europe[LOGICAL] /europe> ls

©Continuent 2013

Automatic Failover

$ db1> cctrl -multi[LOGICAL] /> use europe[LOGICAL] /europe> lsmysql> select @@hostname for update;

$ db1> ssh db1.london.tu sudo /sbin/service mysqld stop[LOGICAL] /europe> lsmysql> select @@hostname for update;

$ db1> ssh db1.london.tu sudo /sbin/service mysqld start[LOGICAL] /europe> datasource db1.london.tu recover[LOGICAL] /europe> ls

[LOGICAL] /europe> switch to db1.london.tu[LOGICAL] /europe> ls

28

©Continuent 2013

Rolling Upgrade and Maintenance

30

Final UpgradeSwitchSlave Upgrade Slave Upgrade

1. Run backup2. Shun slave3. Run upgrade4. Welcome slave

1. Shun/upgrade/welcome

1.Shun/upgrade/welcome

1.Switch master

©Continuent 2013

Upgrade the Slaves

31

[LOGICAL] /usa> switch to db1.nyc.tu[LOGICAL] /usa> datasource db2.nyc.tu backup[LOGICAL] /usa> datasource db2.nyc.tu shun$ db1> mysql -hdb2.nyc.tu -P13306 -utungsten -p

mysql> # Apply backwards compatible changes to db2.nyc.tu

# If unsuccessful[LOGICAL] /usa> datasource db2.nyc.tu restore# Restart the process

# If successful[LOGICAL] /usa> datasource db2.nyc.tu welcome# Repeat the above steps for all slaves

©Continuent 2013

Upgrade the Master

[LOGICAL] /usa> switch to db2.nyc.tu[LOGICAL] /usa> datasource db1.nyc.tu backup[LOGICAL] /usa> datasource db1.nyc.tu shun$ db1> mysql -hdb2.nyc.tu -P13306 -utungsten -p

mysql> # Apply backwards compatible changes to db1.nyc.tu

# If unsuccessful[LOGICAL] /usa> datasource db1.nyc.tu restore# Restart the process

# If successful[LOGICAL] /usa> datasource db1.nyc.tu welcome[LOGICAL] /usa> switch to db1.nyc.tu

32

©Continuent 2013

Puppet & RPM Installation

$> cat /etc/tungsten.ini[defaults]...replication-password=secretreplication-port=13306replication-user=tungsten...[usa]master=db1.nyc.tumembers=db1.nyc.tu,db2.nyc.tu...$> rpm -i ~/continuent-tungsten-2.0.1-662.rpm

33

©Continuent 2013

Next Steps

• Register at http://www.continuent.com/downloads/software

• Initiate a POC with Continuent

• Testing production hardware and data sets

• Application testing

• Integration into monitoring and alerting

• Training for operations personnel

34

©Continuent 2013

More Webinars!

37

• Setup & Operate Tungsten Replicator - Thursday, October 17 @ 10 am PT/1 pm ET

• How Gittigidiyor (Subsidiary of eBay) Replicates in Real Time from MySQL to Oracle - Wednesday, November 6th @ 10:00 EDT/ 15:00 BST/16:00 CEST/17:00 EEST

©Continuent 2013 38

Continuent Website:http://www.continuent.com

Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator

Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.blogspot.comhttp://flyingclusters.blogspot.comhttp://continuent-tungsten.blogspot.com

560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: [email protected]

Tungsten University: Configure & Provision Tungsten Clusters

Technology

Transcript of Tungsten University: Configure & Provision Tungsten Clusters