Identifying Performance Problems in a Multitenant Environment · Logical and physical database...

33
BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH @ ChrisAntognini antognini.ch/ blog Christian Antognini Identifying Performance Problems in a Multitenant Environment

Transcript of Identifying Performance Problems in a Multitenant Environment · Logical and physical database...

Page 1: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA

HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH

@ChrisAntognini antognini.ch/blog

Christian Antognini

Identifying Performance Problems in a

Multitenant Environment

Page 2: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

@ChrisAntognini

Identifying Performance Problems in a Multitenant Environment2 2018-09-19

Senior principal consultant, trainer and partner at Trivadis

[email protected]

Focus: get the most out of database engines

Logical and physical database design

Query optimizer

Application performance management

Author of Troubleshooting Oracle Performance (Apress, 2008/14)

OakTable Network, Oracle ACE Director

Page 3: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Agenda

Identifying Performance Problems in a Multitenant Environment3 2018-09-19

1. Introduction

2. SQL Trace

3. Dynamic Performance Views

4. AWR

5. Enterprise Manager 13.3

Page 4: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Introduction

2018-09-19 Identifying Performance Problems in a Multitenant Environment4

Page 5: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Objective of a Performance Analysis

Identifying Performance Problems in a Multitenant Environment5 2018-09-19

Discover the most time-consuming SQL statements or PL/SQL code

invocations

For each of those time-consuming statements, gather additional information

that can help in understanding the problem

Execution plan

Runtime statistics like the number of processed rows

Resource usage profile

Page 6: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Basic Questions that Require Answers

Identifying Performance Problems in a Multitenant Environment6 2018-09-19

Is the problem reproducible at will?

Yes: everything is much easier than if you can’t!

SQL Trace

For irreproducible problems, is it possible to wait till the problem occurs again?

Yes: a real-time analysis has to be carried out

Dynamic performance views

No: a repository holding historical performance statistics is required

AWR

Page 7: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Disclaimer

Identifying Performance Problems in a Multitenant Environment7 2018-09-19

I’m neither covering basic information about Multitenant, SQL trace, V$ views

nor AWR

I consider that you know

the Multitenant architecture

how to use SQL trace, V$ views and AWR

This presentation only covers the differences introduced by Multitenant

Page 8: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Identifying Performance Problems in a Multitenant Environment8 2018-09-19

SQL Trace

Page 9: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Enabling/Disabling SQL Trace

Identifying Performance Problems in a Multitenant Environment9 2018-09-19

No difference compared to non-Multitenant

There is also no specific support for Multitenant

E.g. no cross container enable/disable

The components for which SQL trace was enabled are visible in the root

through CDB_ENABLED_TRACES

Page 10: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Trace Files Location

Identifying Performance Problems in a Multitenant Environment10 2018-09-19

All PDBs plugged in a given CDB write trace data to the same directory

Giving selective OS access to that directory is challenging

Full access can lead to security issues

_TRACE_FILES_PUBLIC can’t be set at the PDB level

As of 12.2, V$ views like V$DIAG_TRACE_FILE and

V$DIAG_TRACE_FILE_CONTENTS can be used instead

SELECT value FROM v$diag_info WHERE name = 'Diag Trace'

Page 11: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Trace Files

Identifying Performance Problems in a Multitenant Environment11 2018-09-19

A trace file can contain information about several containers

Especially for trace files related to background processes (e.g. PX),

whenever possible use TRACEFILE_IDENTIFIER

The container and the service that generated the trace data are identified

*** SESSION ID:(44.48140) 2018-06-13T23:27:45.753105+02:00

*** CLIENT ID:() 2018-06-13T23:27:45.753111+02:00

*** SERVICE NAME:(pdb122) 2018-06-13T23:27:45.753118+02:00

*** MODULE NAME:(SQL*Plus) 2018-06-13T23:27:45.753124+02:00

*** ACTION NAME:() 2018-06-13T23:27:45.753150+02:00

*** CLIENT DRIVER:(SQL*PLUS) 2018-06-13T23:27:45.753156+02:00

*** CONTAINER ID:(3) 2018-06-13T23:27:45.753163+02:00

Page 12: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

TRCSESS

Identifying Performance Problems in a Multitenant Environment12 2018-09-19

It doesn’t provide a parameter to select trace data of a specific container

A possible workaround is to use the service name

trcsess [output=<output file name >]

[session=<session ID>] [clientid=<clientid>]

[service=<service name>] [action=<action name>]

[module=<module name>] <trace file names>

Page 13: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

TKPROF

Identifying Performance Problems in a Multitenant Environment13 2018-09-19

As of 12.2, it provides the parameter PDBTRACE

It’s unrelated to Multitenant!

It can be used to process a trace file without requiring direct access to the OS

where it’s stored

Trace files can be read through V$ views

E.g. V$DIAG_TRACE_FILE and V$DIAG_TRACE_FILE_CONTENTS

Page 14: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Identifying Performance Problems in a Multitenant Environment14 2018-09-19

Dynamic Performance Views

Page 15: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

CON_ID

Identifying Performance Problems in a Multitenant Environment15 2018-09-19

Identifier present in all V$ views

It’s value is either

0: instance-level information

1: root container

n: id of the container

Page 16: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Instance-level Views

Identifying Performance Problems in a Multitenant Environment16 2018-09-19

Some views provide only instance-level statistics

CON_ID = 0

E.g.

V$OSSTAT

V$SGA

V$ASH_INFO

Page 17: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Instance-level and PDB-level Views

Identifying Performance Problems in a Multitenant Environment17 2018-09-19

Some statistics are provided by level-specific views

One view provides cumulative statistics at the instance level (CON_ID = 0)

Another view provides statistics at the PDB level

E.g.

V$(CON_)SYS_TIME_MODEL

V$(CON_)SYSSTAT

V$(CON_)SYSTEM_WAIT_CLASS, V$(CON_)SYSTEM_EVENT

V$(CON_)SYSMETRIC (V$CON views about metrics as of 12.2)

Page 18: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Generic Views

Identifying Performance Problems in a Multitenant Environment18 2018-09-19

Some views show different data depending from where they’re queried

Root: all data

PDB: only PDB-relevant data

E.g.

V$SQL, V$SQL_PLAN_STATISTICS_ALL

V$FILESTAT

V$SESSION, V$PROCESS

– CON_ID = 0 is used for background processes

Page 19: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Make Queries Compatible with Multitenant

Identifying Performance Problems in a Multitenant Environment19 2018-09-19

CON_ID might need to be added

Because of level-specific views, to make queries work at both the instance

level and the PDB level, PDB-level views should be used

Is it always sensible to do so?

Different queries might be better…

Be very careful in mixing data coming from different levels

Page 20: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

ASH Reports

Identifying Performance Problems in a Multitenant Environment20 2018-09-19

As of 12.2, when a report is created at the PDB level, the data location can be

specified

SQL> @?/rdbms/admin/ashrpt.sql

Specify the location of AWR Data

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

AWR_ROOT - Use AWR data from root (default)

AWR_PDB - Use AWR data from PDB

No major difference

between these two

reports

Page 21: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Identifying Performance Problems in a Multitenant Environment21 2018-09-19

AWR

Page 22: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

PDB Level Snapshots

Identifying Performance Problems in a Multitenant Environment22 2018-09-19

In 12.1 snapshots are only created at the root level

As of 12.2 they can be created at both the root level and the PDB level

By default only root-level snapshots are taken

Two new sets of views are available: AWR_PDB and AWR_ROOT

Page 23: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Enable PDB Level Snapshots

Identifying Performance Problems in a Multitenant Environment23 2018-09-19

Set AWR_PDB_AUTOFLUSH_ENABLED = TRUE

Set the snapshot interval

In case a CDB has many PDB, set AWR_SNAPSHOT_TIME_OFFSET

Note that every container has its own snapshot ids

dbms_workload_repository.modify_snapshot_settings(interval=>30)

Page 24: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

AWR Reports (1)

Identifying Performance Problems in a Multitenant Environment24 2018-09-19

Reports created at the root level are superior

As of 12.2, when a report is created at the PDB level, the data location can be

specified

SQL> @?/rdbms/admin/awrrpt.sql

Specify the location of AWR Data

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

AWR_ROOT - Use AWR data from root (default)

AWR_PDB - Use AWR data from PDB

No major difference

between these two

reports

Page 25: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

AWR Reports (2)

Identifying Performance Problems in a Multitenant Environment25 2018-09-19

As of 12.2, when executed at the PDB level, less information is provided

(there are less differences in 12.1)

PDB level

Root level

Page 26: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

AWR Reports (3)

Identifying Performance Problems in a Multitenant Environment26 2018-09-19

A number of tables also provide the PDB name

E.g. SQL ordered by …

Page 27: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Creating a PDB from a Non-CDB

Identifying Performance Problems in a Multitenant Environment27 2018-09-19

In 12.1, after such an operation, AWR data is no longer accessible

But it consumes space

Refer to bug# 25698110

If needed, export the data before the operation takes place

Page 28: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Identifying Performance Problems in a Multitenant Environment28 2018-09-19

Enterprise Manager 13.3

Page 29: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Multitenant Support

Identifying Performance Problems in a Multitenant Environment29 2018-09-19

In general, the Multitenant support

in the performance-related pages

is weak

2+ years old

tweet

Page 30: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Multitenant Support in 13.3

Identifying Performance Problems in a Multitenant Environment30 2018-09-19

For a number of functionalities

Not feasible to focus on a PDB

Access to the CDB is required

Performance Home

The load of a PDB is visible in

the Services and PDBs tabs

Top Activity

PDB name added to some lists

No PDB filter in ASH Report

Page 31: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Support for PDB Level Snapshots

Identifying Performance Problems in a Multitenant Environment31 2018-09-19

Page 32: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Summary

Identifying Performance Problems in a Multitenant Environment33 2018-09-19

SQL trace

– No PDB-level trace directory

– Workflow almost identical

V$ views provide expected statistics

AWR

– If possible create reports at the

root level

– Consolidation <> easy

Weak Multitenant support in EM

Page 33: Identifying Performance Problems in a Multitenant Environment · Logical and physical database design Query optimizer ... Objective of a Performance Analysis 5 2018-09-19 Identifying

Questions and Answers

Christian Antognini

Senior Principal Consultant

[email protected]

@ChrisAntognini

2018-09-19 Identifying Performance Problems in a Multitenant Environment34