Getting acquainted with DTracing MySQL

Amit Kumar Saha

MySQL Engineer ing, Sun Microsystems

Bangalore MySQL User Group Meet

http://blogs.sun.com/amitsaha

Disclaimers• I am no MySQL, DTrace Guru• My goal today is to share what I know• If you have a question, please interrupt me. I will

try my best to answer then or offline

Agenda

Everyone and Everything Participating on the Network

➢ Make you acquainted with DTrace & DTrace + MySQL

➢ Know the audience➢ A not so brief introduction to DTrace

➢ Concepts

• DTracing MySQL➢ PID Provider➢ Static Probes➢ Things you could do➢ How will it help you?

Are you?

• .. a MySQL User ?• ..a MySQL DBA ?• .. trying to understand MySQL sources ?

Moving a bit [OT]...

• Radioactive elements are used as tracers in human body to identify various abnormal body processes– The tracers act as 'messengers' sending messages

to its listeners

• How would it be like looking under the hood of a car when the engine is running ? – What all would you see ? – How does the lubricant flow around in the engine ?

Similarity between the [OT]s

• In either case, we cannot stop the running system– Human Body – The car

• there are such and other ways to observe them, without stopping them..

We don't want postmortem analysis!

Introducing DTrace• Enables observability of production systems

– Dynamic/Live/Realtime analysis– Minimum overheads. No use, no overheads– No special builds of the sources required, Trace

anything, it just needs to be running :)• User level application• Kernel• Libraries

• Solaris, OpenSolaris, MacOSX, FreeBSD• Experimental Linux port:

http://amitksaha.blogspot.com/2009/03/dtrace-on-linux.html

DTrace Architecture

Probes, about which we learn very shortly (and not shown in the figure) can be best visualised as sensors available to be probed by the DTrace consumers in user space.

DTrace concepts

• Probes– Instrumentation points in the application which you

can gather data (values of variables, CPU consumption)

– Probe description- User specified– When a probe description matches, the probe is said

to fire and the specified action takes place– Probes are made available by providers

– Probe description: provider:module:function:name

DTrace Concepts• Providers

– Kernel modules– Publishers of probes– Kernel space and user space

• Consumers– User space applications,any process that interacts

with the DTrace framework. – The consumer specifies the instrumentation points by

specifying probe descriptions. dtrace is the primary consumer of the DTrace framework. (Now, you see the difference between DTrace and dtrace?)

Dtrace Concepts• Dtrace Scripts or D scripts

– Script written in the 'D' language• C like syntax and specifically designed for the DTrace

framework

• A script consists of one or more probe clauses that describe the points of instrumentation to be enabled by DTrace. Each probe clause has the following form:

probe descriptions/ predicate /{ action statements}

DTracing User Level applications• Our focus today is to DTrace MySQL- a user space

application• There are a couple of ways to trace any user

process application:– PID Provider (PID stands for Process ID)

• When we use this, there needs to be no additionally modifications to the source code of the application

– User-Level Statically Defined Tracing (USDT)• We can use this method only when the source code has

been specifically engineered for DTrace

Using the PID provider

• Start your 'mysqld'• Query parsing in MySQL: mysql_parse() in

sql_sql_parse.cc• We will place a probe in this function..

Watching the queries

#!/usr/sbin/dtrace -q

pid$target::*mysql_parse*:entry /* This probe is fired when the execution enters mysql_parse */{ printf("Query: %s\n", copyinstr(arg1));

}

Run: $ pfexec dtrace -s watch.d -p `pgrep -x mysqld`

Query: show databasesQuery: show variablesQuery: show enginesQuery: SELECT DATABASE()Query: show databasesQuery: show tablesQuery: show tablesQuery: select * from foo

Using the MySQL static probes• Build MySQL 6.0.8+ from sources using:

– ./configure –- enable-dtrace– make;make install

• Start the MySQL daemon• View the available probes published by the 'mysql'

provider:– $pfexec dtrace -l | grep mysql

ID PROVIDER MODULE FUNCTION NAME

56209 mysql706 mysqld bool dispatch_command(enum_server_command,THD*,char*,unsigned) command-done

56210 mysql706 mysqld bool dispatch_command(enum_server_command,THD*,char*,unsigned) command-start

56211 mysql706 mysqld void close_connection(THD*,unsigned,bool) connection-done

.

.

.

.

.

More on using the static probes

• Some example scripts for using the static probes are at http://dev.mysql.com/tech-resources/articles/getting_started_dtrace_saha.html

How will it help you?• If you are a DBA, you already probably feel that

you can easily monitor your MySQL servers and troubleshoot them using DTrace

• If you are someone trying to learn about the way MySQL works, may DTrace be with you :)

• However:– To make the best use: You need to understand both

MySQL and DTrace really well

To learn more: • My article on DTrace + MySQL:

http://dev.mysql.com/tech-resources/articles/getting_started_dtrace_saha.html and the references from there

• Book: Understanding MySQL Internals, Sasha Pachev

• MySQL Internals mailing list: http://lists.mysql.com/internals

Thank You!

Getting acquainted with DTracing MySQL

Amit Kumar Saha

Amit.Saha@SUN.com

http://blogs.sun.com/amitsaha