Copyright © 2013, Oracle and/or its affiliates. All rights reserved ... · Improve InnoDB Data...

New InnoDB Features in MySQL 5.6

Allen Lai

Principle Software Engineer


Program Agenda

Overview of InnoDB Features in MySQL 5.6

Performance and Scalability Features

Better Availability Features

More InnoDB Features in MySQL 5.6

Q&A.


The following is intended to outline our general product direction. It is intended

for information purposes only, and may not be incorporated into any contract.

It is not a commitment to deliver any material, code, or functionality, and should

not be relied upon in making purchasing decisions. The development, release,

and timing of any features or functionality described for Oracle’s products

remains at the sole discretion of Oracle.

Safe Harbor Statements


Overview of InnoDB Features in MySQL 5.6


InnoDB Features in MySQL 5.6 (1)

Removal of legacy bottlenecks

Improved threading/concurrency

Optimized for read-only workloads.

BETTER PERFORMANCE AND SCALABILITY

BETTER AVAILABILITY

Online DDL operations

Transportable tablespaces

Dump, preload buffer pool.



Full-text search

NoSQL, key-value access to InnoDB.

DEVELOPER AGILITY

BETTER QUERY PERFORMANCE

Multi-range-read (MRR) / Batch key access (BKA)

Index condition pushdown (ICP)

Persistent table stats .



Enriched performance schema

InnoDB metrics tables

Information Schema for InnoDB system tables, buffer pool.

BETTER MONITORING

BETTER USABILITY

New default variables

Auto-handling of redo log file size mismatch

Mark table/index as corrupted

Read-only mode

Configurable data dictionary cache .


Performance and Scalability Features in MySQL 5.6


Users can fully utilize latest generations of hardware and OS

Scales as data volumes and users grow

MySQL 5.6: Scalability


Performance and Scalability (1)

Remove legacy mutexes (kernel mutex split)

Multi threaded purge

Separate thread to flush dirty pages

Remove false CPU cacheline sharing

Improve thread scheduling

Reduce contention during tablespace extension.

IMPROVED THREADING AND CONCURRENCY


Kernel Mutex Split

Core

• Locking

• Transaction life cycle

• Read views

• Query stats

Kernel mutex is a global mutex that protects the access of

core sub-systems as well some miscellaneous resources.

Misc.

• Heuristics

• Thread states

• History length

• Task queue

• Drop list


Kernel Mutex Split

For all the transaction

and locking operations

the kernel mutex must

be acquired.

This limits scalability.


Kernel Mutex Split

• Server mutex

• Lock system mutex

• Lock wait mutex

• Transaction system mutex

• Transaction mutex

• Task queue mutex

kernel mutex splits into different mutexes for each of their

functionalities.


Kernel Mutex Split

More scalable and

greater flexibility for

future improvements

e.g., by default all

transactions treated

as read-only until they

do a write.


Multi Threaded Purge

Purge includes:

• Remove obsolete values from indexes

• Remove delete marked records that will not be seen by any

active transaction

• Remove the relevant undo entries from history list

Configuration Parameters:

• innodb-purge-threads: 1-32

Purge is a type of garbage collection.


Multi Threaded Purge

innodb_purge_threads=4

Task queue

Worker

Worker

Worker Coordinator

The coordinator will also purge records

The coordinator waits for the worker threads to complete before

truncating the UNDO logs



• Improve adaptive flushing

• Improve LRU flushing

• Avoid neighbor flushing for SSD

• Increase max size of redo log files.

IMPROVED I/O


Flushing

Two types of flushing activities:

• LRU flushing, based on LRU_list (roughly ordered on time

since last access)

• Adaptive flushing, based on flush_list (strictly ordered on

oldest_modification LSN)

Flushing is the activity of writing dirty pages & logs to the

disk.


Improve LRU Flushing

Major changes in LRU flushing algorithm

• Attempts to maintain a pool of free pages in the free list

• Configurable how deep to scan LRU for a victim

• Flushing happens in background instead of user threads

Configuration parameters:

• innodb_lru_scan_depth (consider changing the default for

larger buffer pools)


Improve Adaptive Flushing

New flushing algorithm in MySQL 5.6

• Rate at which redo log is being generated

• Total redo log capacity

• Rate at which dirty pages are increasing in the buffer pool

• IO capacity of the system


• innodb_adaptive_flushing

• innodb_io_capacity

• innodb_io_capacity_max


Improve Adaptive Flushing


Option to Disable Neighbour Flushing

Neighbours are pages on the same extent

• Extents are contiguous region of 64 pages on disk

• InnoDB typically attempts to flush all dirty pages within an

extent when it chooses one of them for flushing.

• This behaviour is to avoid seek penalty by making IO pattern

as sequential as possible

In case of SSD there is no extra cost for random IO


• innodb_flush_neighbors



• Improved InnoDB data compression

• Optimization for read only transactions

• Improved data management

• Separate tablespace(s) for InnoDB undo log

• Support 4k, 8k page sizes

• Allow selection of single tablespace location

• Use hardware checksums (CRC32)

• Faster DROP table in very large buffer pools


Improve InnoDB Data Compression

Configurable compression level

• innodb_compression_level - the default value is 6 (the zlib

default) and allowed values are 1 to 9.

Optional to skip logging of compressed page images

• innodb_log_compressed_pages. The default value is true.


Improve InnoDB Data Compression

Dynamic padding to reduce compression failures • innodb_compression_failure_threshold_pct: default 5, range 0 -

100,dynamic, implies the percentage of compress ops to fail before we

start using to padding. Value 0 has a special meaning of disabling the

padding.

• innodb_compression_pad_pct_max: default 50, range 0 - 75,

dynamic, the maximum percentage of uncompressed data page that

can be reserved as pad.

Index level compression stats

• INFORMATION_SCHEMA.INNODB_CMP_PER_INDEX

• Helps to reduce compression failures


Optimization for Read-Only Transactions

In MySQL 5.6 READ ONLY transactions are optimized to

reduce overhead of transaction management

• No read view creation (required for MVCC)

• No allocation of undo log

• Separate lists for READ ONLY and READ WRITE transactions

Read-only transactions:

• For auto-commit non-locking SELECTs the optimizations are

transparent

• User can specify a transaction as READ ONLY

• User can start the whole InnoDB instance as read only

(innodb_read_only)


Optimization for Read-Only Transactions


Performance Tuning

One size does not fit all

• Right number of buffer pool instances

• Bigger metadata_locks_hash_instances

• Larger redo log files

• Increase innodb_lru_scan_depth for larger buffer pool

• Better memory allocator, such as jemalloc library

• Take advantage of optimization for read-only transactions

• Disable performance schema

• Turn on innodb_fast_checksums


Availability Features in MySQL 5.6


Better Availability Features

Online DDL operations

Dump, preload buffer pool

Transportable tablespaces


InnoDB Database Files

ibdata files

Syste

m ta

ble

sp

ace

internal data

dictionary

MySQL Data Directory

InnoDB

tables

OR innodb_file_per_table

.ibd files

.frm files

undo logs

change buffer

doublewrite buffer


Type of Online Operations

Metadata only

• MySQL Server metadata, such as alter column default

• MySQL Server metadata & InnoDB metadata, such as

add/drop foreign key

Metadata plus w/o rebuilding the table, such as add/drop

index

Metadata plus rebuilding the table, such as add primary

index, add column.


How Does It Work?

Pre-prepare Prepare Build Final


Pre-preparation Phase

Server

• Determine the algorithm and concurrency level supported

by the storage engine.

• Hold MDL_SHARED_UPGRADABLE: allow concurrent

DML

InnoDB

ha_innobase::check_if_supported_inplace_alter()

• Check if InnoDB supports a particular alter table in-place.


Prepare Phase

Server

• Upgrade to MDL_EXCLUSIVE: no concurrent DML allowed

• Build internal objects describing requested changes

InnoDB

ha_innobase::prepare_inplace_alter_table()

• Check whether the alter is legitimate

• Update internal structures

• Create temporary file(s) for change log(s) due to DMLs

• Start logging


Build Phase

Server

• Hold MDL_SHARED_UPGRADABLE: allow concurrent

DML

• Let storage engine to carry out the changes requested by

ALTER.

InnoDB

ha_innobase::inplace_alter_table()

• Alter the table in-place with operations specified.

• Apply the change logs


Final Phase

Server

• Update .frm and remove old table definitions • Upgrade to MDL_EXCLUSIVE: no concurrent DML allowed • Notify storage engine • Cleanup internal structures

InnoDB

ha_innobase:: commit_inplace_alter_table()

• Commit or rollback the changes a) Sync and delete the logs b) Commit metadata changes c) Cleanup internal structures


How Does It Work - Online Add Index

CREATE INDEX index_name ON table name (c1)

Pre-

Prepare

Phase

Prepare

Phase

Build

Phase

Final

Phase

Concurrent User

Source (table)

Scan clustered index;

Extract index entries;

Sort / merge index

build

Drop old table (if

primary)

No concurrent DML

allowed

Upgradable

Shared Metadata

Lock

DML Logging;

Apply logs at the

end of create index

Create temp table

for new index (if

primary)

Upgradable

Shared Metadata

Lock

Metadata Lock

Concurrent Select,

Delete, Insert,

Update

(cluster) Index

Create log files;

Logging starts

Update system

tables (metadata)

Exclusive

Metadata Lock

Exclusive

Metadata Lock

Concurrent Select,

Delete, Insert,

Update

No concurrent DML

allowed

Check whether

the online DDL is

supported


Considerations for Online Operations (1)

•

•

•

•

a) innodb_sort_buffer_size: in bytes default 1MB

b) innodb_online_alter_log_max_sizesize: in bytes default 128MB


Considerations for Online Operations (2)

•

•

•


Example 1: Add / Drop Index

mysql: set old_alter_table=0;

Query OK, 0 rows affected (0.00 sec)

mysql: create index i_dtyp_big on big_table (data_type);


Records: 0 Duplicates: 0 Warnings: 0

mysql: drop index i_dtyp_big on big_table;





mysql: create index i_dtyp_big on big_table (data_type);

Query OK, 1731584 rows affected (4 min 59.33 sec)


mysql: drop index i_dtyp_big on big_table;




Example 2: Rename Column



mysql: alter table big_table change `flags` `new_flags`

-> varchar(3) character set utf8 not null;





mysql: alter table big_table change `new_flags` `flags`

-> varchar(3) character set utf8 not null;




Preload Buffer Pool

Resolves “cold cache” problem

Buffer pool contents can be dumped at any point in time

• Can be configured to dump automatically at shutdown

The dumped contents can be loaded at any point in time

• Can be configured to load automatically at startup


• innodb_buffer_pool_dump_*

• innodb_buffer_pool_load_*

• --innodb-buffer-pool-filename


Transportable Tablespaces(1)

A frequently requested feature since MySQL 4.1

Challenge: Resolving dependencies on…

• Change buffer

• Undo logs

• Crash recovery

• Data dictionary

Solutions:

• Make tablespaces ‘clean’ on export

• Adjustments on import


Transportable Tablespaces(2)

Export:

• FLUSH TABLE t FOR EXPORT;

• Copy tablespace file(s)

• UNLOCK TABLES;

Import:

• CREATE TABLE t;

• ALTER TABLE t DISCARD TABLESPACE;

• Move the copied file(s) in place

• ALTER TABLE t IMPORT TABLESPACE;


More InnoDB Features in MySQL 5.6


More New Features

InnoDB full-text search

Direct access to InnoDB via memcached

Monitoring and diagnostics

Improved usability

• New default variables

• Auto-handling of redo log file size mismatch

• Mark table / index as corrupted

• Read-only mode

• Configurable data dictionary cache


InnoDB Full-Text Search

InnoDB full-text index as an inverted index

Support all query types supported by MyISAM:

• Natural language search

• Query expansion

• Boolean search

Plus

• Proximity search: a special type of Boolean search

• Create full-text index with parallel tokenization and sorting


InnoDB Memcached plugin (NoSQL)

Architecture



Fast, simple access to InnoDB

• Accessed via Memcached API

• Use existing Memcached clients

• Bypasses SQL transformations

Implementation

• Memcached daemon plug-in to mysqld

• Memcached protocol mapped to the native

InnoDB API

• Shared process space for ultra-low latency

SQL (MySQL Client)

InnoDB Storage Engine

MySQL Server Memcached

plugin

Application

NoSQL (Memcached

Protocol) mysqld



SQL/NoSQL access

• NoSQL for key-value operations

• SQL for rich queries, JOINs, FKs, etc.

Simple Authentication and Security

Layer



Install

– mysql> install plugin daemon_memcached soname

“libmemcached.so”;

– Install 3 necessary tables

Configuration script in “scripts/innodb_memcached_config.sql”.

Tables are created in a dedicated database “innodb_memcache”.

Configuration

– daemon_memcached_r_batch_size: default is 1

– daemon_memcached_w_batch_size: default is 1


InnoDB Memcached plugin (NoSQL) Example

mysql> install plugin daemon_memcached soname 'libmemcached.so';


mysql> select * from test.demo_test;

+----+--------------+------+------+------+

| c1 | c2 | c3 | c4 | c5 |

+----+--------------+------+------+------+

| AA | HELLO, HELLO | 8 | 0 | 0 |

+----+--------------+------+------+------+

1 row in set (0.01 sec)

user@host > telnet localhost 11211

Trying 127.0.0.1...

Connected to localhost.

Escape character is '^]'.

get AA

VALUE AA 8 12

HELLO, HELLO

END

^]



Memcached to InnoDB mapping

Command Description Example InnoDB action

get Reads a value get mykey Search (select)

set Set a key unconditionally set mykey 0 60 5 insert

add Add a new key add newkey 0 60 5 update

replace Overwrite existing key replace key 0 60 5 update

… … … …

append Append data to existing key append key 0 60 15 update


Monitoring & Diagnostics

Enriched performance schema

• Covers 46 mutexes, 12 rwlocks, 7 types of threads, 3 types

of I/O

Information schema metrics table

• 17 modules, 207 counters

Information schema for InnoDB system tables

Information schema for InnoDB buffer pool


More Information

https://blogs.oracle.com/mysqlinnodb/

http://dimitrik.free.fr/blog/index.html

http://mysqlha.blogspot.com/

http://dev.mysql.com/tech-resources/articles/mysql-5.6.html

http://dev.mysql.com/doc/

• http://dev.mysql.com/doc/refman/5.6/en/mysql-nutshell.html

• http://dev.mysql.com/doc/refman/5.6/en/innodb-performance.html

• http://dev.mysql.com/doc/refman/5.6/en/innodb-online-ddl.html

https://blogs.oracle.com/mysqlinnodb/

http://dimitrik.free.fr/blog/index.html

http://mysqlha.blogspot.com/






http://dev.mysql.com/doc/

http://dev.mysql.com/doc/refman/5.6/en/mysql-nutshell.html



http://dev.mysql.com/doc/refman/5.6/en/innodb-performance.html



http://dev.mysql.com/doc/refman/5.6/en/innodb-online-ddl.html





Copyright © 2013, Oracle and/or its affiliates. All rights reserved ... · Improve InnoDB Data...

Documents

Transcript of Copyright © 2013, Oracle and/or its affiliates. All rights reserved ... · Improve InnoDB Data...