MySQL Developing Database Applications Using MySQL Connector-C++
Java MySQL Connector & Connection Pool Features & Optimization
-
Upload
kenny-gryp -
Category
Data & Analytics
-
view
2.413 -
download
1
Transcript of Java MySQL Connector & Connection Pool Features & Optimization
![Page 1: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/1.jpg)
Java MySQL Connector & Connection PoolFeatures & Optimization
Kenny Gryp <[email protected]>April 14, 2015@gryp
![Page 2: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/2.jpg)
DISCLAIMER
Please excuse me for not being a Java developer
2
![Page 3: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/3.jpg)
What I Don’t Like
• Brussels Sprouts• Taxes• Calories• Java(’s chattiness)
3
![Page 4: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/4.jpg)
MYSQL CONNECTORSCONNECTION POOLS
4
![Page 5: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/5.jpg)
MYSQL CONNECTORSCONNECTION POOLS
ConnectorsConfiguring ConnectorCreating A Database ConnectionPrepared StatementsExample Transaction
5
![Page 6: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/6.jpg)
MySQL Connector/J & MariaDB Java Client
• MySQL Connector/J– Oracle– 5.1.35 Latest– Compatible with• MySQL• Percona Server• MariaDB
6
![Page 7: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/7.jpg)
MySQL Connector/J & MariaDB Java Client
• MariaDB Java Client• MariaDB• Fork from Drizzle Connector• Latest 1.1.8• Compatible with –MySQL– Percona Server–MariaDB
7
![Page 8: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/8.jpg)
MySQL Connector/J Features
• Enterprise Plugin: Query Analyzer• MySQL Fabric Integration• Load Balancing• Failover• Replication
8
![Page 9: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/9.jpg)
MYSQL CONNECTORSCONNECTION POOLS
ConnectorsConfiguring ConnectorCreating A Database ConnectionPrepared Statements Example Transaction
9
![Page 10: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/10.jpg)
Creating Connection
Connection con = DriverManager.getConnection(“jdbc:mysql://node2/employees?
user=connj&password=test");Statement stmt = con.createStatement();String query =
"select * from employees where emp_no = 20000;";
ResultSet rs = stmt.executeQuery(query);...
MariaDB:jdbc:mariadb://node2/employees
?user=connj&password=test"
10
![Page 11: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/11.jpg)
Creating Connection - Tomcat w. JDBC-Pool
context.xml (local):<Resource name="jdbc/test" auth="Container" type="javax.sql.DataSource"username=“jdbc-pool" password="test" driverClassName="com.mysql.jdbc.Driver"url=“jdbc:mysql://node2:3306/employees”/>
MariaDB: driverClassName="org.mariadb.jdbc.Driver"
11
![Page 12: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/12.jpg)
Creating Connection - JDBC URL
jdbc:mysql://node2:3306/employees?useServerPrepStmts=true&...
12
![Page 13: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/13.jpg)
MYSQL CONNECTORSCONNECTION POOLS
ConnectorsConfiguring ConnectorCreating A Database ConnectionPrepared StatementsExample Transaction
13
![Page 14: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/14.jpg)
Connector/J - Creating Connection
SHOW VARIABLES WHERE Variable_name ='language' OR…
SELECT @@session.auto_increment_increment;
SET NAMES latin1;SET character_set_results = NULL;SET autocommit=1;SET sql_mode=
'NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES';
14
![Page 15: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/15.jpg)
MariaDB - Creating Connection
set autocommit=1;USE employees;show variables like 'sql_mode';
15
![Page 16: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/16.jpg)
Creating Connection - Defaults
• Connector/J:
• MariaDB Java Client:
16
![Page 17: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/17.jpg)
Connector/J & MariaDB Java Client - Verbosity
• Connector/J is more verbose when starting a connection
• Usually not a problem:– connection pools are commonly used
(more coming soon…)– connections are reused
– Actually I like but not too much.
17
![Page 18: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/18.jpg)
Optimization
• MariaDB Java Client vs MySQL Connector/J• Prepared Statements
18
![Page 19: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/19.jpg)
Connector Performance - SELECT 1 localhost, single threaded 19
QPS
0
4000
8000
12000
16000
localhost
ConnectorJ MariaDB
15.21313.477
![Page 20: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/20.jpg)
Connector Performance - MariaDB %faster20
Fast
er %
5%
10%
15%
20%
MariaDB Connector +Speed% ConnectorJ
SELECT1 LO
13%
![Page 21: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/21.jpg)
Connector Performance - MariaDB %faster21
Fast
er %
5%
10%
15%
20%
MariaDB Connector +Speed% ConnectorJ
SELECT1 LO SELECT1 net pk range
0%
4%4%
13%Benefit is relative!
![Page 22: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/22.jpg)
MYSQL CONNECTORSCONNECTION POOLS
ConnectorsConfiguring ConnectorCreating A Database ConnectionPrepared StatementsExample Transaction
22
![Page 23: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/23.jpg)
Client or Server Side Prepared Statements
• Server side Prepared statements:– reduce network traffic– query is already optimized on server.
• Support:–MariaDB Java client only supports client
side– Connector/J default in client side
23
![Page 24: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/24.jpg)
Server Side Prepared Statements
PREPARE stmt1 FROMselect * from employees
where emp_no = ?;EXECUTE # API CALL
select * from employees where emp_no = 20000;
DEALLOCATE PREPARE stmt1;
24
![Page 25: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/25.jpg)
Connector/J: Server Side Prepared Statements
• useServerPrepStmts = false– disabled by default
• Mind looking at:– cachePrepStmts = false • do PREPARE, EXECUTE, DEALLOCATE
every time…, 3 round trips?– prepStmtCacheSize = 25– prepStmtCacheSqlLimit = 256• low defaults
25
https://bugs.mysql.com/bug.php?id=74932
![Page 26: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/26.jpg)
Benchmark: Prepared Statements26
QPS
900
1800
2700
3600
MariaDB CONN/J CONN/J SRVCONN/J SRV+Cache
3.506QPS
2.047QPS
3.342QPS3.400QPS
select * from employees_alotofindexes where first_name='moss' and birth_date > "1954-06-14" and gender="M" and hire_date > "1998-01-01"\G
![Page 27: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/27.jpg)
Cracked!!27
![Page 28: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/28.jpg)
MYSQL CONNECTORSCONNECTION POOLS
ConnectorsConfiguring ConnectorCreating A Database ConnectionPrepared StatementsExample Transaction
28
![Page 29: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/29.jpg)
Connector/J Optimization + Default JDBC-Pool
Connection con = ds.getConnection();con.setTransactionIsolation (Connection.TRANSACTION_READ_COMMITTED);con.setAutoCommit(false);PreparedStatement stmt = con.prepareStatement("select * from employees where emp_no = ?");stmt.setInt(1, 20000);ResultSet rs = stmt.executeQuery();stmt.close();rs.close();con.commit();con.close();
29
![Page 30: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/30.jpg)
Connector/J Optimization + Default JDBC-Pool
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
SET autocommit=0;
# administrator command: Prepare;
select * from employees where emp_no = 20000;
# administrator command: Close stmt;
commit;
30
Taxes
![Page 31: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/31.jpg)
Connector/J Optimization
• useConfigs=maxPerformance– cachePrepStmts=true– cacheCallableStmts=true– cacheServerConfiguration=true– useLocalSessionState=true– elideSetAutoCommits=true– alwaysSendSetIsolation=false– enableQueryTimeouts=false
31
![Page 32: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/32.jpg)
Connector/J Optimization
• useLocalTransactionState=true commit() / rollback()
32
![Page 33: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/33.jpg)
Connector/J Optimization - Tuned
JDBC URL: useConfigs=maxPerformance&useServerPrepStmts=true:
select * from employees where emp_no = 20000;
commit;
33
![Page 34: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/34.jpg)
MariaDB Java Client Optimization
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
select * from employees where emp_no = 20000;
COMMIT;
34
![Page 35: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/35.jpg)
MariaDB Java Client Optimization - Code
if ( con.getTransactionIsolation() !=
Connection.TRANSACTION_READ_COMMITTED){ con.setTransactionIsolation
(Connection.TRANSACTION_READ_COMMITTED);}
35
![Page 36: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/36.jpg)
MariaDB Java Client Optimization - Interceptors
SELECT @@tx_isolation;select * from employees
where emp_no = 20000;COMMIT;
Still @@tx_isolation. Now add JDBC Interceptor:<Resource name="jdbc/test"auth="Container"factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"jdbcInterceptors="ConnectionState"driverClassName="org.mariadb.jdbc.Driver"url="jdbc:mysql://node2:3306/employees"/>
36
![Page 37: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/37.jpg)
MariaDB Java Client Optimization - Optimized!
select * from employees where emp_no = 20000;
COMMIT;
37
![Page 38: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/38.jpg)
Benchmark - Connector Concurrency - SELECT 1 38
HikariCP-bench with JDBC Pool, 4 Threads, SELECT 1 (4,8,16,32 Pool Size)
QPS
17.500
35.000
52.500
70.000
MariaDB CONN/J
![Page 39: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/39.jpg)
Benchmark - Connector Concurrency - TRX 39
HikariCP-bench with JDBC Pool, 4 Threads, TRX (4,8,16,32 Pool Size)
QPS
4.750
9.500
14.250
19.000
Conn/J MariaDB Conn/J Optim MariaDB Optim
![Page 40: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/40.jpg)
MYSQL CONNECTORSCONNECTION POOLS
40
![Page 41: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/41.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool SizingLingering Transactions
AnalysisExamplesGraceful FailoverConn/J Extra Features
41
![Page 42: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/42.jpg)
Java Connection Pools
– The Usual:– C3P0– Commons-DBCP (v1&v2)– JDBC Pool (fork commons-DBCP)
– Out In The Wild: – Vibur-DBCP– HikariCP– BoneCP
42
![Page 43: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/43.jpg)
Java Connection Pools
– The Usual:– C3P0– Commons-DBCP (v1&v2)– JDBC Pool (fork commons-DBCP)
– Out In The Wild: – Vibur-DBCP– HikariCP– BoneCP
43
![Page 44: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/44.jpg)
Connection Pool Key Points
• Connection Management• Pool Sizing• Connection Testing• Avoid Lingering Transactions
44
![Page 45: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/45.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool SizingLingering Transactions
AnalysisExamplesGraceful FailoverConn/J Extra Features
45
![Page 46: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/46.jpg)
Connection Pool Issues
• Coming from DBA side, I do not like ‘chattiness’
46
![Page 47: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/47.jpg)
Connection Pool Issues47
![Page 48: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/48.jpg)
Connection Pool Issues48
![Page 49: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/49.jpg)
Connection Pool Issues49
![Page 50: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/50.jpg)
Connection Pool Issues50
![Page 51: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/51.jpg)
WHY DOES IT HAVE TO DO THAT?
Because of ‘application bugs’
51
![Page 52: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/52.jpg)
Connection Pools - Why Chattiness Examples
• *maybe* forgot to COMMIT / ROLLBACK• wanting AUTOCOMMIT=1
but a previous TRX set it to 0• Changing TRX Isolation Level• Is connection still working?
52
![Page 53: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/53.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool Sizing
Lingering TransactionsAnalysisExamplesGraceful FailoverConn/J Extra Features
53
![Page 54: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/54.jpg)
Connection Pool - Resetting Status54
JDBC-Pool C3P0 DBCP2 HikariCP
Rollback rollbackOnReturn=false autoCommitOnClose=false rollbackOnReturn =true
Commit commitOnReturn=false autoCommitOnClose=false n/a n/a
Avoid see above forceIgnoreUnresolvedTransactions=false
see above
Auto Commit
Driver Driver enableAutoCommitOnReturn=true
Driver
![Page 55: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/55.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool SizingLingering Transactions
AnalysisExamplesGraceful FailoverConn/J Extra Features
55
![Page 56: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/56.jpg)
Connection Pool - Testing
• Making sure the connection is still active• If not, maybe reopen a connection• Not recommended as DB• However, applications:• do not like errors• do not retry gracefully
56
![Page 57: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/57.jpg)
Connection Pool - Testing
• If connections REALLY need to be tested…• do not specify test query like:• SELECT 1• SELECT * FROM DUAL
• Leave default, all of the connection pools use:JDBC4 isValid();
57
![Page 58: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/58.jpg)
Connection Pool - Testing58
JDBC-Pool C3P0 DBCP2 HikariCP
Test Before testOnBorrow=false testConnectionOnCheckOut=false
testOnBorrow=false n/a
Test After testOnReturn=false testConnectionOnCheckIn =false
testOnReturn=false n/a
Test While Idle testWhileIdle=false idleConnectionTestPeriod =0
testWhileIdle=false n/a
JDBC4 isValid()
default default default jdbc4ConnectionTest=true (default)
Query validationQuery(isValid)
preferredTestQuery=null validationQuery(isValid)
connectionTestQuery=none
Interval? validationInterval=30000 n/a n/a n/a
![Page 59: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/59.jpg)
Connection Pool - Testing
• JDBC: validationInterval=30s WHY? It defeats the whole purpose!
59
![Page 60: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/60.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool SizingLingering Transactions
AnalysisExamplesGraceful FailoverConn/J Extra Features
60
![Page 61: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/61.jpg)
Connection Pool - Pool Sizing
• Funnelling on Application Level, is good• Smaller Number is Better• +- * CPU’s on DB• maybe a bit more (waiting on IO…)
• all application servers combined• Response Time vs Throughput
61
![Page 62: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/62.jpg)
Connection Pool - Pool Sizing62
JDBC-Pool C3P0 DBCP2 HikariCP
Amount of Connections
maxActive=100 maxPoolSize=15 maxTotal=8 maximumPoolSize=10
Maximum Idle Connections
maxIdle=100 maxIdleTime=0** maxIdle=8 n/a
Minimum Idle Connections
minIdle=10 minPoolSize=3 minIdle=0 minimumIdle=max
Startup Size initialSize=10 initialPoolSize=3 initialSize=0 minimumIdle
![Page 63: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/63.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool Sizing
Lingering TransactionsAnalysisExamplesGraceful FailoverConn/J Extra Features
63
![Page 64: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/64.jpg)
Connection Pool - Avoid Lingering Transactions
• Application forgets to return the connection• Statements that take longer than …
• Avoid this!• Fix Application
64
![Page 65: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/65.jpg)
Connection Pool - Avoid Lingering Transactions 65
KILL Warning
JDBC-Pool removeAbandoned=false removeAbandonedTimeout=60 abandonWhenPercentageFull=0
suspectTimeout=0
C3P0 unreturnedConnectionTimeout=0 n/a
DBCP removeAbandoned=false removeAbandonedTimeout=300 Only When: getNumIdle() < 2 and getNumActive() > getMaxTotal() - 3)
n/a
HikariCP n/a leakDetectionThreshold=0
![Page 66: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/66.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool SizingLingering Transactions
AnalysisExamplesGraceful FailoverConn/J Extra Features
66
![Page 67: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/67.jpg)
Connection Pools - How To Look At Workload?
• Slow Query Log• tcpdump• pt-query-digest• Percona Cloud Tools
67
![Page 68: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/68.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool SizingLingering Transactions
AnalysisExamplesGraceful FailoverConn/J Extra Features
68
![Page 69: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/69.jpg)
Connection Pool - Example Transaction
Connection con = ds.getConnection();con.setTransactionIsolation (Connection.TRANSACTION_READ_COMMITTED);con.setAutoCommit(false);PreparedStatement stmt = con.prepareStatement("select * from employees where emp_no = ?");stmt.setInt(1, 20000);ResultSet rs = stmt.executeQuery();stmt.close();rs.close();con.commit();con.close();
69
![Page 70: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/70.jpg)
For Connectors - RECAP
• MySQL Connector/J• useConfigs=maxPerformance• useServerPrepStmts=true
• MariaDB Java Client• HikariCP: Built in• JDBC-Pool: jdbcInterceptors=“ConnectionState"
• Other Pools: UNKNOWN
70
![Page 71: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/71.jpg)
Connection Pool - TRX JDBC
select * from employees where emp_no = 20000;
commit;
71
200
![Page 72: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/72.jpg)
Connection Pool - TRX C3P0
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
SET autocommit=0;select * from employees
where emp_no = 20000;commit;SET autocommit=1;SET SESSION TRANSACTION
ISOLATION LEVEL REPEATABLE READ;
72
600
![Page 73: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/73.jpg)
Connection Pool - TRX C3P0
mysql> set global tx_isolation=“READ-COMMITTED”;
forceIgnoreUnresolvedTransactions=true
73
200
![Page 74: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/74.jpg)
Connection Pool - TRX DBCP
SET autocommit=1;# administrator command: Ping;SET autocommit=0;select * from employees
where emp_no = 20000;commit;rollback;SET autocommit=1;
74
700
![Page 75: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/75.jpg)
Connection Pool - TRX DBCP
testOnBorrow=falserollbackOnReturn=falseenableAutoCommitOnReturn=false
jdbcUrl: useLocalTransactionState=true
75
200
![Page 76: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/76.jpg)
Connection Pool - TRX HikariCP
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED;
SET autocommit=0;select * from employees
where emp_no = 20000;commit;SET autocommit=1;SET SESSION TRANSACTION
ISOLATION LEVEL REPEATABLE READ;
76
600
![Page 77: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/77.jpg)
Connection Pool - TRX HikariCP
mysql> set global tx_isolation=“READ-COMMITTED”;
autoCommit=false
77
200
![Page 78: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/78.jpg)
Connection Pools78
![Page 79: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/79.jpg)
MariaDB vs. Connector/J79
![Page 80: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/80.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool SizingLingering Transactions
AnalysisExamplesGraceful FailoverConn/J Extra Features
80
![Page 81: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/81.jpg)
Connection Pools - Graceful Failover81
![Page 82: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/82.jpg)
Connection Pools - Graceful Failover
• HAProxy ‘stats socket’/etc/haproxy/haproxy.cfgglobal. . . stats socket /tmp/haproxy.sock level admin
• Disable Node# echo "disable server database/node1" | socat stdio /tmp/haproxy.sock
82
![Page 83: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/83.jpg)
Connection Pools - Graceful Failover83
![Page 84: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/84.jpg)
Connection Pools - Graceful Failover
• During ‘maintenance’, what do we do?• KILL old connections?• Wait until connections are closed? (Define
lifetimes?)• Ignore it?
84
![Page 85: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/85.jpg)
Connection Pools - Graceful Failover
• Some connection pools can close connections gracefully, when idle.• For ‘synchronous’ replication systems• using JMX• No Application Errors!
85
MethodJDBC-Pool purgeOnReturn()C3P0 softResetAllUsers()DBCP n/aHikariCP softEvictConnections(),
suspendPool(), resumePool() <—- ASYNC
![Page 86: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/86.jpg)
Connection Pools - Graceful Failover86
![Page 87: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/87.jpg)
Connection Pools - Graceful Failover87
![Page 88: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/88.jpg)
Connection Pools - Graceful Failover88
![Page 89: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/89.jpg)
Connection Pools - Graceful Failover
• 0 Application Errors• Completely seamless
89
![Page 90: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/90.jpg)
MYSQL CONNECTORSCONNECTION POOLS
Connection PoolsIssuesResetting EnvironmentTesting ConnectivityPool SizingLingering Transactions
AnalysisExamplesGraceful FailoverConn/J Extra Features
90
![Page 91: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/91.jpg)
Connector/J - Extra Features
• Load Balancing• jdbcUrl: ”jdbc:mysql:loadbalance://node1,node2/db?loadBalanceConnectionGroup=lb& loadBalanceEnableJMX=true”
• loadBalanceStrategy (random/bestResponseTime)
• Failover• ReplicationDriver (setReadOnly)• Combining with Connection Pools is less useful
• Fabric
91
![Page 92: Java MySQL Connector & Connection Pool Features & Optimization](https://reader035.fdocuments.net/reader035/viewer/2022062306/58886f7f1a28ab34788b46d7/html5/thumbnails/92.jpg)
Java MySQL Connector & Connection Pool Optimization
• http://dev.mysql.com/doc/connector-j/en• https://mariadb.com/kb/en/mariadb/client-libraries/mariadb-java-
client/• http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html• http://www.mchange.com/projects/c3p0• http://commons.apache.org/proper/commons-dbcp/• https://github.com/brettwooldridge/HikariCP
92
MYSQL CONNECTORSCONNECTION POOLS
Kenny Gryp <[email protected]>November 4, 2014@gryp