Bottom-up Approach to DB2 Performance Reporting and Tuning
Transcript of Bottom-up Approach to DB2 Performance Reporting and Tuning
© YL&A 1999-2003 Bottom Up Approach to DB2 Performance2
Bottom Up Approach to DB2 Performance© Copyright 2003 Yevich, Lawson & Associates, Inc. All Rights Reserved
Yevich, Lawson & Associates. Inc.3900 Wood Duck Dr. Suite DSpringfield, IL 62707
TEL: 1-888-246-5049INT’L: 1-217-698-8142FAX : 1-217-698-8143
Developed by:Dan Luksetich
© YL&A 1999-2003 Bottom Up Approach to DB2 Performance3
Disclaimer PLEASE READ THE FOLLOWING NOTICE
The information contained in this presentation is based on techniques, algorithms, and documentation published by the several authors and companies, and in addition is the result of research. It is therefore subject to change at any time without notice or warning. The information contained in this presentation has not been submitted to any formal tests or review and is distributed on an “As is” basis without any warranty, either expressed or implied.
The use of this information or the implementation of any of these techniques is a client responsibility and depends on the client’s ability to evaluate and integrate them into the client’s operational environment.
While each item may have been reviewed for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere.
Clients attempting to adapt these techniques to their own environments do so at their own risks.Foils, handouts, and additional materials distributed as part of this presentation or seminar should be reviewed in their entirety.
This presentation may contain information about software releases that are not yet Generally Available. Material is subject to change. YL&A is not responsible for the accuracy of the information.IBM is not responsible for the accuracy of the information or delivery of unreleased features discussed in this presentation.
© YL&A 1999-2003 Bottom Up Approach to DB2 Performance4
Presentation Abstract
This paper will focus on a technique to measure and document the costs of DB2 for z/OS applications, addressing the performance of an application after implementation in a management friendly manner so that performance tuning can be properly "sold" to management. Tuning opportunities are appropriately weighed, and effective breakdown of the problem to specific targets accomplished. The result is an approach that works for management and technicians, and saves millions of dollars in operating costs.
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 5
Introduction – Why is this “Bottom Up”?
■ This is Beyond “Typical” DB2 Performance Tuning■ Flips this “Typical” Model Upside Down
– Not looking at Individual Statements First– System Wide – Application Wide
■ Then We Drill Down■ Identifies Performance Opportunities from DB2 Accounting Trace
Data Instead of SQL Explains■ Provides a Systematic Approach to Organization of Performance
Information■ Communicates Performance Information to Managers in a
Language They can Understand ($$$$$$)■ Automation Relieves the DBA and System Programmer from
Additional Burden
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 6
Typical DB2 Application Tuning – Development & Testing
■ Performed During Development– Explain Facility Used to Look for Inefficient Access Paths
■ Tablespace Scans■ Non-Matching Index Scans■ Sorts
– Program Reviews■ Ensure Proper Access to the Database■ Avoid Programmatic Joins■ Avoid Repetitive Database Accesses
■ Performed During Testing– DB2 Online Monitors Utilized
■ Excessive I/O Waits■ Locking Contention■ High CPU Utilization
– May Not Involve Production Sized Volumes– May Not Involve Production Transaction Frequencies
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 7
Typical DB2 Application Tuning - Production
■ Tuning Performed After Implementation■ Consists Mostly of “Fire Fighting”■ Problems Not Discovered During
– SQL walk-throughs– Monitoring of Test Runs
■ Once “Fire Fighting” Finished Tuning Ends!
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 8
So – What’s Wrong With The Typical Approach
■ Most Managers and Application Developers are Not Interested in Fine Tuning– We Tuned the SQL– We Fought the Fires– Now We’re Done?– But We Could Save Millions ($$$)
■ The Traditional Approach Has Failed■ We Need An Alternative
– Post-Implementation Approach– Need to Measure and Communicate Application Performance
■ To Managers■ To Developers
– We need a “Buy-In” From These People■ Enter the “Bottom-Up” Approach■ We’ll Communicate Application Savings in Dollars!
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 9
The Bottom-Up Reporting and Tuning Approach
■ Not a New Technique– Something We May be Doing Every Day
■ Very Effective in Various Situations– When the Perception is “Everything has been Tuned”– There has been No Tuning– A Purchased Product has been Installed– Tuning is Performed Using “Traditional” Methods
■ Explain■ Testing■ Subsystem Parameters
■ This Technique Breaks That Mold– Turns Things “Upside Down”– Statements that are Perceived as “Well Behaved” May Now Show Up as
a Bottleneck
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 10
The Approach
■ Define the Applications Running in the System■ Determine the “Cost” of Each Application■ What’s More Expensive?
– The Query the Performs a Tablespace Scan Once Per Day?– The Query That Performs a Matching Scan 50 Million Times a Day?
■ If We Use the Typical Approach– The Statement Doing the Tablespace Scan
■ If We Use This Approach– The Statement That Costs More
■ The Difference– Traditional “Top-Down” Looks at the Worst Things From a Technical
Perspective– “Bottom-Up” Looks at Things From a System Wide Measurement
Driven Perspective
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 11
What is This Approach?
■ Addresses System Wide Application Performance■ Utilizes a Batch Reporting System
– We Need DB2 Package Level Reporting– DB2 Accounting Traces 1,2,3,7,8
■ Reports Are Inputs to Spreadsheets and Charts■ We Take Application Summaries■ We Track the Applications
– Daily or Weekly Performance– Changes Over Time
■ We Measure Performance in a Management Friendly Way– $$$$$$$$$$$$$$$
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 12
DB2 Accounting Performance Impact
■ The Impact is Minimal, the Benefits Substantial■ DB2 Accounting Traces 1,2,3,7,8 to SMF Increases CPU by 4.3%
Monitor Product Percent Overhead
SMF 4.30%
TMON DB2 -0.42%
Insight for DB2 -1.36%
Omegamon for DB2 10.51%
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 13
The Bottom-Up Framework
■ Need to Use Your Monitor or Capacity Planning Product– Produce Package Level Reports– Accounting Trace Long or Summary Reports
■ Information From the Reports is Customized– Some Reporting Products can do This– Or We Do It Ourselves (Like with REXX)– Or a Talented SysProg can do it Alone
■ What We Need From the Accounting Report– DB2 Correlation Name– DB2 Package Name– In-DB2 Elapsed Time at Package Level– In-DB2 CPU Time at Package Level
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 14
Application Information is Needed
■ We Have to Know Something About Our Applications■ Naming Standards Help
– Program Name Headers– Packages Map to Programs
What Programs Belong to the Payroll Application?• PAYR001• PAYR002• PAYR003
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 15
Now We can Categorize Work by Applications
■ Payroll Application– We’ll Call it Application “A”– It Contains Packages That Begin With “PAYR”
■ PAYR001, PAYR002, PAYR003■ Daily Batch Sales Report Application
– We’ll Call it Application “T”– It Contains Packages
■ PKG001, PKG002, PKG003, PKG004, PKG005■ We Can Now Summarize DB2 Accounting Information
– We Produce a Daily DB2 Accounting Package Summary Report– We Feed That Report into A REXX Routine (Lots of Ways to do This,
However)■ Rules can be Built in a Database
– Any Package That Starts with “PAYR” is Application A– Packages PKG001, PKG002, PKG003, PKG004, PKG005 are Application T
■ Then the Package Level Information is Aggregated by Application
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 16
Summarized Accounting Information – First Step
■ Summarized DB2 Accounting Information– Daily Reports are Produced– Reports are Formatted Such That They Can be Fed into Spreadsheet
Software– Daily Spreadsheets and Graphs Published
■ Daily Spreadsheet Data– Combined into Weekly and Monthly Reports– Graphs and Charts Produced– Trend Reports Produced
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 17
Summarized Accounting Information
■ The First Step is to Provide Application Level Accounting Information– We Summarize Package Level Information– We Aggregate According to Naming Standards
■ Application “T” (Batch Sales Report Application) is Interesting!– If Our Concern is CPU Consumption!
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 18
Interesting Things on Our Summary Report
■ What are the Really Interesting Applications in Our Report?
■ Daily CPU Consumption– T Consumes over 7 hours per day– W Consumes over 3 hours per day– N consumes over 2 hours per day
■ No other application consumes nearly this much CPU■ If we want to save CPU, we focus on these three applications
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 19
■ What are the Really Interesting Applications in Our Report?
■ CPU Consumed per SQL Statement– Application T Consumes about .5ms CPU per Statement– Application W Consumes less than .4ms CPU per Statement– Application N Consumes about .5ms CPU per Statement
■ Application T provides the best tuning opportunity– Total CPU (by far) and CPU per statement
Interesting Things on Our Summary Report
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 20
Drill Down to Application of Interest – Second Step
■ Application “T” is of Further Interest■ We Can Now Drill Down to the Packages (Programs) That
Application “T” Utilizes
■ Packages PKG002 and PKG003 are interesting!– Largest consumers of CPU
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 21
Program Analysis – Third Step
■ The Obvious High Consumer of Resources is Application “T”– Within Application “T”, the High Consumer is Package PKG002– This is Where We Initially Focus Our Attention
■ This is Where We Need Application Development “Buy-In”– We Need A Team Approach
■ We Need to Drill Down to the Program– Which DB2 Statements are Executing?– How Often Do They Execute?– How Much CPU Does Each Statement Consume?
■ There Are Tools To Help US– DB2 Performance Trace– Online Monitor Tools– Batch Analysis Tools– DB2 Explain– Bottom Line; We need Statement Level Information at This Point
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 22
Program Analysis – The Process
■ You Can First Try EXPLAINing All the Statements in a High-Consuming Package– For Example: REBIND PKG002 EXPLAIN(YES)– Provides a Non-Intrusive Way of Evaluating Each Statement– You May Catch Some Bad Access Paths That Were Previously Missed,
or Changed Without Notice■ A Short Execution of a DB2 Performance Trace May Provide Ample
Statement Level Information– Determine When PKG002 is Executed– Start a Performance Trace During That Time– Run the Trace for a Short Time– Gather the Statement Level Information
■ Address the Highest Consuming Statements■ Or the Most Frequent Statements ■ Tune Those Statements First■ Then Repeat the Reporting/Tuning Process
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 23
Continuing The Reporting
■ We Need to Maintain a System Perspective■ Once We Make A Change
– Did it Work?– Did it Affect Anything Else
■ Continual Reporting is a Must– We Measure Changes by Application
Performance Report
To Determine What to Tune
Performance Report
To Monitor the Impact to the Change Across All
Applications
Produce a Report
Implement a Tuning Change
Repeat
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 24
Delivering Reports That “Matter”
■ It’s Critical to Deliver Reports That are Understood!– Difference Between Success and Failure
■ We Need to Deliver High Level Information to Managers– Summarized “Cost” Information– CPU Time– Number of Statements
■ We Import the Reports Into Spreadsheets– Graphs for Visual Presentations– Cost Factors are Always Important– We Show Data in Dollars
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 25
Reporting CPU Costs in Dollars
■ Most Shops Have a Formula for Converting CPU Seconds into Dollars
■ Build This Information into The Reports– Add the Formula to the Spreadsheets– Convert the Spreadsheets into Charts– The Charts Display Consumption of Resources in Dollars
■ Each Time a Tuning Change is Made– Create a Delta (Before/After) Report– Show the Difference in a Chart– The Measurements in the Chart are in Dollars
■ Why Use This Technique– It Gets Managers Attention– It attaches Money to Costs and Tuning Efforts– It Helps Build Support for Proper Performance Analysis and Tuning
■ The “Savings” Support the “Costs” of the Effort
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 26
Delivering Reports That “Matter”
■ Example: We Take Our Summarized Performance Reports– Translated into Dollars– In This Example; $120 per CPU Hour
C D E M N T W$0
$50,000
$100,000
$150,000
$200,000
$250,000
$300,000
$350,000
Yearly Cost
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 27
Summary
■ Typical DB2 Performance Tuning begins (and Ends) During Design– It’s On a Statement-by-Statement Basis– This is Great, But…
■ DB2 Performance Tuning Needs an Enterprise Approach– We Look at Application Performance– It’s A System Wide Perspective– We Track Performance Over Time
■ Proper Reporting is Key– Money Base Reports Sell Tuning to Management– Charts are Better Than Stats
■ This Technique Works for any Application– In-house– Purchased– BECAUSE – We Look at System Level Stats
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 28
Trademarks
■ IBM– is a registered trademark of International Business Machines
Corporation.■ DB2
– is a trademark of IBM Corp.
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 29
Useful IBM Websites
■ DB2 Electronic Support– http://www.ibm.com/software/db2zos
■ FAQs– http://www.ibm.com/software/data/db2/os390/index.html
■ Presentations:– http://www.ibm.com/software/data/db2/os390/presentations.html
■ Data Sharing– http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/
Flash10194■ News: DB2 for z/OS
– http://www.ibm.com/software/data/db2/os390/techdocs2/z01.pdf
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 30
Useful IBM Websites
■ Overview of V8– http://www.ibm.com/software/data/db2/os390/techdocs/DMDB12.pdf– ftp://ftp.software.ibm.com/software/db2storedprocedure/db2zos390/
techdocs/db2vnext.pdf■ Introduction to V8 Catalog Changes
– ftp://ftp.software.ibm.com/software/db2storedprocedure/db2zos390/techdocs/vnextcatalog.pdf
■ DB2 Developer Domain articles– http://www7b.boulder.ibm.com/dmdd/– http://www7b.boulder.ibm.com/dmdd/library/techarticle/0209cotner/
0209cotner.html– http://www7b.boulder.ibm.com/dmdd/library/techarticle/0206miller/
0206miller.html
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 31
DB2 Books by YL&A
DB2 High PerformanceDesign and Tuning
Prentice Hall, Aug, 2000Richard Yevich and Susan Lawson
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 32
DB2 Books by YL&A
DB2 for OS/390 Version 7DBA Certification Guide
Prentice-Hall, June 2001Richard Yevich andSusan Lawson
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 33
DB2 Books by YL&A■DB2 UDB for OS/390 V7.1Application Certification Guide
Prentice-Hall Dec 2002Susan Lawson
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 34
Courses by YL&A
■ Transition– V6 to V7, V5 to V7 (z/OS), V7 to V8 (LUW)– For applications only, for DBA only, or both– V8 Introduction (1 or 2 day)
■ SQL (z/OS and LUW)– Basic SQL– Advanced and Complex SQL– SQL Performance Tuning and Optimization
We customize all classes based upon customer requirements
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 35
Courses by YL&A
■ Application Development– Stored Procedure Development and Implementation– UDFs and Triggers Development
■ Database Design– Physical Design, Logical Design
■ Data Sharing– Implementation, Performance, Recovery (or all topics)
■ High Performance Design and Tuning– Application, Database, Systems
Taught by skilled instructors world-wide!
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 36
Cost Avoidance Through Performance Audits
■ DB2 Performance Audits– Existing or new database designs and applications– Certification of design and implementation acceptance– Evaluation of all the performance ‘points’ in a DB2 environment
■ Physical Design■ Subsystem■ Application Code and SQL
Specific to your needs!
Bottom Up Approach to DB2 Performance© YL&A 1999-2003 37
Cost Avoidance Through Performance Audits
■ DB2 Performance Audits – Help with bringing legacy application to an e-business environment –
the rules have changed! ■ What was acceptable performance in the past is NOT acceptable in an e-
business environment– Experienced in ‘fighting fires’ – many performance problems do not
become reality until production– Results: problems identified, solutions proposed (many implemented
immediately), continual knowledge transfer during the process