Post on 13-Dec-2014
description
04/10/23
Optimizing SQL Performance withBMC SQL Performance for DB2Introduction and Technical Update
04/10/23 ©2005 BMC Software2
The Business Problem
SQL Performance– The most challenging problem in DB2 shops today
• Direct impact on meeting service level objectives• Direct impact on cost of ownership for the IT organization
– What makes it difficult?• Dynamic Application Workloads• More and more dynamic SQL coming from distributed connections• SQL coding quality and validation
Why is this an even bigger deal today?IT Dynamics driving the problem– Trending toward more dynamic SQL = less control– Outsourced application development – Less staff
• People and experience to handle what is a complex task
04/10/23 ©2005 BMC Software3
› Focus on individual SQL statements
– Do they meet “best practice” coding standards
– Do they use expected/accepted DB2 access paths
– Do they deliver desired result set in acceptable time with acceptable resource consumption
› Developed and tested in controlled environment
› More predictive in nature
› Focus on workload dynamics– How does concurrent execution
affect response time/resource consumption
– Does this SQL statement/program collide with other transactions
› Real world unpredictability comes into play
› More focus on measuring the workload and rapidly reacting
Analyzing and Tuning Your SQL -Sources of Information
04/10/23 ©2005 BMC Software4
SQL Performancefor DB2
Data Collector
SQL Performance for DB2 The BMC Solution
› Solution Objectives– Control or eliminate poorly performing
SQL code – SQL performance problems identified and
solved quickly – Proactive identification of inefficient SQL
› Solution Components– APPTUNE captures and analyzes SQL
execution metrics – SQL Explorer is the SQL analysis tool
that extends far beyond the DB2 Explain– Index Analysis Component provides
specific features for effectively managing DB2 indexes
– Common Explain available in solution and point products
Index Component
SQL Explorerfor DB2
APPTUNEfor DB2
04/10/23 ©2005 BMC Software
SQL Performance for DB2Solution Architecture
04/10/23 ©2005 BMC Software6
SQL Performance for DB2Starting at the /top
ASQEQRPW/P View a Report LINE 1 OF 7 Command ====> _______________________________________________ Scroll ===> CSR_ BMCSftwr.SQMCACTS -- SUBSYSTEM ANALYSIS (DATA) -- 08/27 14:36:58 Source : V52A-ACTIVE Intvl : 08/27 00:00 - UNLIMITED More: + > ------------------------------------------------------------------------------- Actions: G-AppGrps R-Programs C-ConnID T-Detail A-Averages O-Object P-Plans U-Users S-Stmts E-SQL Errors L-CorrID Actions for SAP systems: V-Server N-Work Process Number I-Work Process ID AppGrp: User: ConnID: Plan: CorrID: Program: Stmt No: SQL +------- Total IN-SQL Time -------+ +- S DB2 SSID MVS ID Calls Elapsed % CPU % Getpage Numb ---------- -------- ------ ----------------- ----------------- -------- ----+ DFH1 (8.1) SYSM 1815K 01:53.79476 96.1% 00:29.68631 99.9% 154104 1+ DFG1 (7.1) SYSO 74 00:04.67528 3.9% 00:00.02965 0.1% 778
Drilldowns aplentyNumerous drilldowns will take you into detail organized to your interest
Key points•DB2 subsystems across your enterprise on a single screen
•DB2s with highest elapsed times bubble to the top (generally the best place to start)
•Results can be sorted on different field on display (getpages or CPU for example) with simple command
Poor performance at the subsystem level
04/10/23 ©2005 BMC Software7
Which statement is using the most resources?
SQLText
ObjectsAccessedTop three statements are
using 90% of the CPU
S
“S” again to see the SQL text
Key points•Top SQL statements in terms of total elapsed time sorted to top
•Dynamic and Static SQL visible •Numerous drilldowns for additional detail
04/10/23 ©2005 BMC Software8
Here’s the SQL
“X” will invoke the common Explain function
X
Key points• Provides extensive SQL statement execution detail on elapsed time, getpage activity, BP usage, lock events and CPU consumption
• You can launch directly into a full-function explain from the common Explain function
04/10/23 ©2005 BMC Software9
Let’s Explain the statement
04/10/23 ©2005 BMC Software10
Which DB2 Objects Are Used by this Statement?
04/10/23 ©2005 BMC Software11
APPTUNE for DB2As a Troubleshooting Tool
› Current transaction problem ?– Subsystem – Thread– Locking contention
› Do you have Distributed Applications? Do you need to identify the transaction by Workstation?
› Need a way to report on application SQL errors?
04/10/23 ©2005 BMC Software12
APPTUNE for DB2 Active Threads Running …
04/10/23 ©2005 BMC Software13
SQL Error Analysis report
04/10/23 ©2005 BMC Software14
SQL Explorer for DB2
› Explain function– Online interactive tool
• See detail when you want to see it on statistics, predicates, access path, objects• Customize the layout of the data the way you want to see it• Get advice and tips from Explain rules• See visual comparison from editing SQL and re-explaining• See visual comparison of dynamic explain and BIND with EXPLAIN(YES)• Solution password enables Index Component analysis on cloned objects:
– Add one or more indexes– Drop one of more indexes– Update statistics on one or more indexes
– Batch• See summary and detail explain information for many packages, collection IDs, or
plans.• Explain objects in batch and save to product history tables to be used in Compare
04/10/23 ©2005 BMC Software15
BMC Common ExplainA Little More Detail
Predicate Information
04/10/23 ©2005 BMC Software16
SQL Explorer for DB2Additional Functionality
› Compare Access Paths from two points in time– Compares on SQL Text, not statement number– Sets return code 8 if access path got worse– Shows differences in statistics that were used at the time of the Explain
› Mismatch Analysis– Reports mismatches in data type or length that can affect access paths– Now incorporated into Explain Predicate report
› Impact Analysis– Find all statements affected by a schema change
› DBRM/LOAD Library Compare– Find mismatches in timestamps between DBRM and LOAD before you promote to
production to avoid -805 / -818 SQL errors.› Migrate Access Path Statistics
– Copy PROD statistics to TEST to get a better picture of the access paths› Declarations Generator
– Generates null indicator variables to help avoid coding errors
04/10/23 ©2005 BMC Software17
SQL Performance for DB2 – Index Component
› Collects predicate data– Number of times a column was specified in a predicate and in what manner
(Indexable Equal, Indexable Range)› Collects I/O information for DB2 objects
– Inactive tables and inactive indexes– DB2 catalog statistics for tables and indexes– SQL statements associated with a table
› Collects index usage– Percentage of getpages from indexes– SQL statements associated with a table
› Provides What-if Index analysis on cloned objects– Add indexes– Drop indexes– Update statistics on indexes
04/10/23 ©2005 BMC Software18
SQL Performance for DB2 – Index Component Table Access by GETPAGE/STMT Ratio report
Data at statement levelprovide matrix of activity
Activity against Objects
04/10/23 ©2005 BMC Software19
SQL Performance for DB2 – Index Component Table Access by GETPAGE/STMT Ratio report
Enter M for Crud Matrix on the CR_Individual
tableThis is the
percentage of getpages from
indexes
04/10/23 ©2005 BMC Software20
SQL Performance for DB2 – Index Component Table CRUD Matrix report
Let’s look at the usage of this column in the workload
captured
04/10/23 ©2005 BMC Software21
SQL Performance for DB2 – Index Component Table CRUD Matrix report – Column Detail
04/10/23 ©2005 BMC Software22
SQL Performance for DB2 – Index Component Drilldown to SQL Text
• The “X” option will launch the BMC Common Explain feature
• Common Explain is an integrated common component used by a number of BMC products
• Now we’ll proceed to the Explain function
04/10/23 ©2005 BMC Software23
• Now let’s look at modeling a new index that could provide a better access path
• Common Explain uses a cloning technique that creates new objects or modifies existing objects
• Clones created in a BMC database so no DBD locks
• Objects created with DEFINE NO • Objects not created till a dynamic explain is executed
• Objects dropped upon exiting Explain• Index statistics can be modified • Index DDL can be saved or imported into the process
BMC Common Explain What-if Index Analysis Using Cloning
04/10/23 ©2005 BMC Software24
• We’ve created a new index in this step and modified the statistics so the optimizer will have enough information
• We can also delete existing indexes (deleted from the clone only) or update statistics data for existing indexes as well
BMC Common Explain What-if Analysis
•Here is the index we created in the previous step •Remember the index hasn’t actually been created yet•Next step we’ll using the XD (DynamicExplain) to create the clone
04/10/23 ©2005 BMC Software25
• The new index has dramatically affected the access path and caused the Cost Rate for this statement to go down
• More importantly, the access path went from a tablespace scan to a matching index access
BMC Common Explain What-if Analysis•With the new index the access path changes dramatically•Meanwhile the cost of the statement has gone down as well•DDL for creating this index can be saved for later use
04/10/23
SQL Performance for DB2 3.3.00Solution Update
Generally Available November 9, 2006
04/10/23 ©2005 BMC Software27
SQL Performance 3.3.00 Technical OverviewRelease Highlights
› Reduce solution overhead– Data Collector CPU and storage overhead
• Architectural changes mean overhead reductions– Index Component overhead reduction
• Eliminate bind requirement • Streamline reporting to reduce/eliminate redundant reports
› Deliver new filtering capabilities– Exception only reporting (XOR)– Increased number of collection keys– Increased flexibility in defining exceptions and reducing overhead
04/10/23 ©2005 BMC Software28
SQL Performance for DB2 3.3 Filtering Enhancements
› Extensive work on current filtering capabilities to:– Deliver more filtering functionality – Make filtering easier to use – Enable new XOR option
• Discussed later› Functional enhancements
– Exceptions on GETPAGES, SQL Codes, Timeout/Deadlock Occurrences, Synchronous I/O Counts • In addition to existing exceptions (Elapsed and CPU times)
– Ignore the literal in dynamic SQL statements where the rest of the statement is the same• Treats SQL statement as a single statement
– New collection keys – Change collection items without cycling Data Collector (through filters)
04/10/23 ©2005 BMC Software29
New Collection Options Panel
•Simple means to turn off collection for qualifiers•New collection keys
04/10/23 ©2005 BMC Software30
New Collection Options Exception only roll-ups
•Triggers exception only reporting•Idea is to minimize collection overhead for stable applications with minimal monitoring requirements
•Resource saving options specified as part of individual filter so they can be applied to specific portions of the workload•Combined with other collection options to create a tailored profile for applications
04/10/23 ©2005 BMC Software31
New Collection OptionsIdentifying Exception Criteria
•Specifies exceptions on Elapsed and CPU time, Negative SQL code, Deadlocks and Timeouts, and synchronous I/O Counts•Exception records will be cut for each of the exceptions that occur•Specifies collection of host variable values
04/10/23 ©2005 BMC Software32
New Collection OptionsFilter Profile Definition for Legacy Applications
•Exception-only reporting (XOR) specified for programs (ACCT*) and plan (KAD4050) •No collection at all for payroll program - PAYR650
04/10/23 ©2005 BMC Software33
New Collection/Filtering Options Identifying Filters in Place for a Specific Interval
•Allows user to identify collection/filtering options in place during a specific interval
• Based on current profile
04/10/23 ©2005 BMC Software34
New Collection/Filtering OptionsDetails on Active Filters
•Collection options in place at time of interval
04/10/23 ©2005 BMC Software35
New Collection/Filtering Options Drilling Into Exceptions
04/10/23 ©2005 BMC Software36
New Collection/Filtering Options Exception Analysis
04/10/23 ©2005 BMC Software37
Exception Analysis Exception Detail
04/10/23 ©2005 BMC Software38
Exception AnalysisException Detail w/Host Variables
04/10/23 ©2005 BMC Software39
› Improvements in reporting on Query Parallelism› PSSBLOAD – Utility for loading APPTUNE performance data to DB2
tables for analysis› PSSEXTRC – Explain and compare dynamic SQL access paths across
time or DB2 releases
Additional SQL Performance Enhancements
04/10/23 ©2005 BMC Software40
› Load performance data into DB2 for analysis– Historical performance trends—You might want to know which SQL
statements had the highest CPU in a given month, or which statement had the highest number of executions during a batch window.
– Audit database—You might want to know which users accessed a particular table, or which users tried to access unauthorized tables.
– Impact Analysis—You might want to identify the programs, statement numbers, and SQL text that accessed a particular table.
› How’s it work– Reads archived trace data sets containing SQL performance data– Extracts the BMC IFCID data from the archives
– SQL Text– SQL Errors (SQLCA)– SQL Statement Summary– Object Statistics (per statement)
– Creates output data sets used for loading into DB2 tables
PSSBLOAD: Loading APPTUNE data into DB2 Tables
04/10/23 ©2005 BMC Software41
Summary
› SQL Performance for DB2 3.3 available November 9th – APPTUNE 5.3– SQL Explorer for DB2 5.3
› Significant reduction in Data Collector overhead– For APPTUNE and Index Component
› New functionality in managing collection keys and filters› Continued investment in BMC’s DB2 solutions › Continued focus on things that differentiate BMC
– World class support– Innovative technology that solves critical business issues– Strong financial condition