© IBM Corporation 2007 1 Patrick Bossman IBM Silicon Valley Lab Tuning with Optimization Service...

Patrick Bossman

IBM Silicon Valley Lab

Tuning with Optimization Service Center

Part I

Columbia, MDSeptember 12, 2007

Agenda

• Overview of Optimization Service Center

• Connect and configure OSC

• Workload (application) Tuning

• Query tuning (Part II)

Overview

• Optimization Service Center (OSC)– New no charge product supported for connections

to DB2 9. • Support for DB2 for z/OS V8 connections in open beta

– Workstation tool for monitoring and tuning of queries

– Facilitates the identification and tuning of workloads (sets of queries) as well as individual queries

– New powerful query diagnostic tools enable faster deep analysis of queries

Overview

• OSC feature list– Support for workload tuning

• Stand-alone workload creation

• Push-out model (monitoring)

– Visual Explain query graphing capabilities

– Query formatting and annotation

– Query report

– Visual Plan Hint

– Statistics Advisor - query and workload

Connect and configure

• Launch OSC– Initial screen is connection and configuration

– Review look and feel

– Review layout of screen

Launch OSC

Project navigator area

Project navigator

• Navigate through open projects

Top navigation tabs

Top navigator tabs close-up

Connection / configuration

Connection close-up

• Connection– Subsystems cataloged in DB2 Connect listed– Use subsystem menu option to add / remove subsystems

Add Subsystem

• Add DB2 subsystem connection– Select subsystem button

– Choose Add

Connection information

• Add DB2 subsystem connection– Subsystem alias is a meaningful name to you

– Location, hostname, port• Can be found in <ssid>MSTR address space DDF startup message DSNL004I

DSNL004I

• DSNL004I message– Location = location– Domain = hostname– TCPPORT = port

Jeez Pat, Connect already!

• Connect to subsystem– Choose Connection, connect– You can connect to more than one subsystem and perform

activities on more than one subsystem at a time.

Login information…

Bind packages

Create explain tables

Create alias to explain tables

• Create aliases– You can use OSC to create ALIASes to the explain

tables also.

Create alias to explain tables

Subsystem status

Explain-enabled Authid’s

EXPLAIN enabled authid’s

Connection complete.Now what?

(Welcome menu…)

Go to welcome page…

Welcome page

Welcome page close-ups

Application tuning

• Application tuning– Creating workloads

– Workload options

– Workload tuning features

Application tuning process

• What is application tuning?– Identify what the application workload is

• Individual SQL statements

– Get an understanding of the applications behavior• How often are individual SQL statements executing?• What is the performance of individual SQL statements?

– Determine statistics for workload • Workload statistics advisor• More workload analysis features coming… (?)

– Remeasure workload• Identify top tuning candidates• Use query based tools to further analyze (next session)

Creating workloads

• Creating a workload– Tune a workload

– View workloads

– Monitoring functions

Tune a workload

Workload sources• Workload sources

– Snap statement cache– Catalog (static SQL)– QMF / QMF HPO– File– Categories– Other workloads

Workload sources

Catalog source…

Catalog source

Package filter options• Package filter options

– Collection id, name, owner, …

– Can use equals, like, in, etc.

Package filter options

Plan filter options• Plan filter options

– Plan name, plan creator, etc.

– Can use equals, like, in, etc.

Plan filter options

Cost & Object filter options• Cost and Object filters

– Filters SQL within package / plan filter– Requires the static SQL be bound with explain yes.– DSN_STATEMNT_TABLE must already be populated.– Show only SQL with PROCSU > …– Show SQL which uses table…– Show SQL which uses index…

• Choice to return rows which qualify for ANY or ALL of the cost / object filter conditions

Cost & Object filter options

Access path filters• Access path filters

– Filters SQL within package / plan filter

– Requires the static SQL be bound with explain yes.

– PLAN_TABLE must already be populated.

• Show SQL which performs…– Tablespace scan

– Sort

– Non-matching index scan

– List prefetch

– Outer join

– ….

Cost & Object filter options

Capture the workload

• After selecting packages / plans / filters…– Click finish to capture workload

– OSC goes about the business of collecting the statements…

• Queue Jeopardy music, get some coffee.

Workload captured…

Review workload screen

• Workload header

• Workload options

• Project Navigator

Workload header

Workload options

Workload option close-up (1)

Workload options close-up (2)

Workload statements…

Workload statements

• Workload statements– Show the SQL within the workload

– Refine execution runtime information• For catalog capture, runtime information would have to be

manually obtained.

• You don’t really need to change it for statistics advisor…

– Let’s walk through it…

Workload statements view

Capture more statements…

Description

Schedule workload activity

Remove statements from workload

Execute query tools on selected SQL

Select “edit runtime information…”

Edit runtime information

Run workload advisors (SA)

Why Statistics Advisor?

• Importance of statistics• Difficulty of determining what statistics to

collect

• Solution: Workload Statistics Advisor!!!

Estimating query cost

• Accurately estimating the number of rows is important for to accurately cost a query– Index matching

• Accurately estimate index cost

– Total index filtering• Estimate table access cost via index(es)• Choose how to use index (prefetch?)

– Total table level filtering• Efficient join order• Efficient join method• Appropriate sorts

72Symptoms of inaccurate estimation

• Sometimes - great performance!– Few competitive choices

– Good performance possible, but not a "winning" strategy

• Inefficient access path– Query never performs at optimal level

• Unstable access path– Slight changes in input result in dramatic swings in actual

performance

– Efficient and inefficient access paths can end up with close cost estimates

– More accurate selectivity estimates can improve optimizer "differentiation" of efficient and inefficient access

Some observations

• Several candidate indexes?– Need sufficient statistics to differentiate the candidates

• Local filtering spread across many tables?– The more "potentially" desirable outer tables there are, the more

important it is to accurately estimate the true qualified rows

• Lot's of predicates which don't filter much?– Important to make sure DB2 knows that

– When many rows are processed, a single bad choice can be magnified

• The necessity for accurate estimation is related to the number of competitive choices.

Why statistics advisor?

• Statistics are collected to help optimizer in costing available access paths

• Determination of statistics required can be daunting

• Automate the determination of statistics required

What are the right statistics?

• Identify interesting objects– Tables– Indexes

• Identify interesting columns, column groups– Predicate analysis

• Which column (groups) have predicates?

• Which column (groups) are indexed?

• Which statistics should I collect?– Column statistics– Correlation statistics– Frequencies– Histograms

– Statistic needed depends on predicate types (range, equal)

Determining the "right" statistics

• Manual analysis very time consuming, requires significant skill– Format the query to make it readable

– Recognize predicate transformations • Predicate transitive closure

• Between to equal opens new possibilities

• Or to IN creates new boolean term predicates

• Has DB2 rewritten the query (predicate transitive closure)

– Some columns used as predicates hidden in views

– Recognize which statistics are appropriate in which circumstances

• Significant effort even for expert

• Error prone

Run workload advisors (WSA)

Complete recollection, or repair?

RUNSTATS command

Table portion of commandRUNSTATS TABLESPACE DB2OSC.WSATS0 TABLE(DB2OSC.DSN_WSA_DATABASES) COLUMN(NAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_CGFREQS) COLUMN(TBCREATOR,TBNAME,COLGROUPCOLNO,SESSIONID) TABLE(DB2OSC.DSN_WSA_KTGFREQS) COLUMN(IXCREATOR,IXNAME,SESSIONID,KEYGROUPKEYNO) TABLE(DB2OSC.DSN_WSA_SESSIONS) COLUMN(SESSIONID) TABLE(DB2OSC.DSN_WSA_COLGROUPS) COLUMN(TBCREATOR,TBNAME,COLGROUPCOLNO,SESSIONID) COLGROUP(COLGROUPCOLNO,SESSIONID,TBCREATOR,TBNAME) TABLE(DB2OSC.DSN_WSA_TSS) COLUMN(NAME,DBNAME,SESSIONID) COLGROUP(DBNAME,SESSIONID) COLGROUP(DBNAME,NAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_KTGS) COLUMN(IXCREATOR,IXNAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_KEYS) COLUMN(IXCREATOR,IXNAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_CGHISTS) COLUMN(TBCREATOR,TBNAME,COLGROUPCOLNO,SESSIONID) TABLE(DB2OSC.DSN_WSA_KTGHISTS) COLUMN(IXCREATOR,IXNAME,SESSIONID,KEYGROUPKEYNO) TABLE(DB2OSC.DSN_WSA_INDEXES) COLUMN(TBCREATOR,TBNAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_TABLES) COLUMN(CREATOR,NAME,TSNAME,DBNAME,SESSIONID) COLGROUP(CREATOR,NAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_LITERALS) COLUMN(TBCREATOR,TBNAME,SESSIONID,COLNO,VALUE) COLGROUP(COLNO,SESSIONID,TBCREATOR,TBNAME,VALUE) TABLE(DB2OSC.DSN_WSA_KEYTARGETS) COLUMN(IXCREATOR,IXNAME,SESSIONID) TABLE(DB2OSC.DSN_WSA_COLUMNS) COLUMN(TBCREATOR,TBNAME,SESSIONID) SORTDEVT SYSDA SHRLEVEL CHANGE REPORT YES

Index portion of suggestionRUNSTATS TABLESPACE DB2OSC.WSATS0 …INDEX(DB2OSC.DSN_WSA_IDX_DBS, DB2OSC.DSN_WSA_IDX_CGFS, DB2OSC.DSN_WSA_IDX_KTGFS, DB2OSC.DSN_WSA_IDX_SSNS, DB2OSC.DSN_WSA_IDX_CGS, DB2OSC.DSN_WSA_IDX_TSS, DB2OSC.DSN_WSA_IDX_KTGS, DB2OSC.DSN_WSA_IDX_KEYS KEYCARD, DB2OSC.DSN_WSA_IDX_CGHS KEYCARD, DB2OSC.DSN_WSA_IDX_KTGHS KEYCARD, DB2OSC.DSN_WSA_IDX_IDXS, DB2OSC.DSN_WSA_IDX_TABLES, DB2OSC.DSN_WSA_IDX_LTRS KEYCARD, DB2OSC.DSN_WSA_IDX_KTS KEYCARD, DB2OSC.DSN_WSA_IDX_COLS KEYCARD) SHRLEVEL CHANGE REPORT YES

Output looks complex…

• RUNSTATS commands are more targeted– Only collect on tables used in queries

– Only collect column statistics on columns used as predicates

– Collect frequency and histogram statistics based on predicates which would benefit

Pause for questions on WSA?

Back to our regularly scheduled programming…

More OSC GUI review

Open projects as tabs on top…

Project Navigator

Project navigator

• Navigate across projects– As multiple projects opened, you can switch between

projects you’re working on

• Navigate within a project– As you perform different tasks within a project, you

can navigate through the different tuning sections

Project Navigator

Project tabs

• Navigate within project– As multiple projects opened, you can switch between

projects you’re working on

• Close sections of project you’re done with– Click x on the tab to close selected tab

More OSC GUI review

Workload sources - cache

Cache statement view

• Cache statement view– Number of executions

– Accumulated & average CPU time

– Accumulated & average elapsed time

– SQL text

• Cache as input provides more visibility into how often SQL executing, cost of statements

Workload – cache statements view

Monitor workload

• Normal monitor workload– All SQL based on certain criteria

– Useful to profile an application• Which SQL running most frequently?

• Which queries consuming most elapsed / execution time?

• Exception monitor– Workload generated based on exception criteria

• All SQL for package > x seconds elapsed

98Application tuning process(revisit)

Create normal monitor

Add monitor sources

Set monitor options

Monitor statement view

Exception monitor options

Application tuning

• Normal monitor allows for capture of all SQL– Specify list of packages

• Prevents workload “pollution” which can occur with statement cache

– Capture valuable execution information

– Ideally initially used in QA environment to profile / tune application before production implementation

• “How often does this execute?”

• “How long does this run?”

• … you can have answers to these questions…

– Selectively use in production to develop application profile and tune application

105Application tuning process(review)

Which workload source to use?

• Which workload source provides…– Identification of all SQL statements executed

– Accurate number of execution counts

– Accurate accumulated and average execution performance

– Provides runtime information for any source

– Can re-measure and expect to see the same SQL again, compare results

– Is not polluted with dynamic SQL which are not relevant to the application?

• Workload monitoring (normal)

Tuning with OSC I

• End of OSC tuning Part I– Show some demo if time permits

• Next up, Query tuning with OSC– Statistics Advisor

– Query annotation

– Query report

– Visual Explain

– Visual plan hint

Patrick Bossman

bossman@us.ibm.com

Tuning with Optimization Service Center I

© IBM Corporation 2007 1 Patrick Bossman IBM Silicon Valley Lab Tuning with Optimization Service...

Documents

Transcript of © IBM Corporation 2007 1 Patrick Bossman IBM Silicon Valley Lab Tuning with Optimization Service...

Ibm 2007 Camp

2007 IBM Automotive Supplier Jam

IBM Tivoli Software © 2007 IBM Corporation IBM Tivoli Storage Manager for MS SharePoint.

® © 2007 IBM Corporation IBM SystemBuilder™

© 2007 IBM Corporation 2007 IBM developerWorks Sam Baker Sales Leader, Telecommunications Industry Software Group, Asia Pacific, IBM SOA & Web 2.0 in Telecommunications.

IBM Storage Sales © 2007 IBM Corporation IBM System Storage Archive and Data Retention Systems.

BOSSMAN FULL - Lidlwebcontent.lidl.pl/PL/2020/AKCJE/DOSTEPNOSC-PIW-REGIONAL... · 2020. 3. 10. · BOSSMAN FULL Miasto Ulica Barlinek 31 Stycznia 11 Białogard ul. Połczyńska 13

® IBM Software Group © 2007 IBM Corporation Servlet Filtering 4.1.0.3.

IBM Global Services © 2007 IBM Corporation IBM Internet Security Systems Ahead of the threat. ™ Adaptive Security PlanningAugust 14, 2007 Scott Lupfer,

© 2007 IBM Corporation IBM Systems IBM System P. IBM System p IBM Systems 2 © 2007 IBM Corporation Consistent Predictable Delivery IBM POWER™ SYSTEMS.

© 2007 IBM Corporation IBM Global Engineering Solutions IBM Blue Gene/P Environmentals.

Centers for IBM e-Business Innovation :: Chicago © 2007 IBM Corporation IBM Project Management October 17, 2007 Dawn Goulbourn Chicago Solutions for Innovations.

© 2007 IBM Corporation ® IBM Confidential IBM Enterprise Content Management.

IBM Systems Group © 2007 IBM Corporation IBM Media| May, 2007 IBM LTO Ultrium 4 Overview IBM Media.

® IBM Software © 2007 IBM Corporation IBM Software Group Passport Advantage Sub-capacity Announcement Overview November 06, 2007.

© 2007 IBM Corporation IBM Information Management Accelerate information on demand with dynamic warehousing March 2007.

® IBM Software Group © IBM Corporation IBM - Build and Release Best Practices Jesse Tilly, IBM 5 September 2007.

© 2007 IBM Corporation IBM Information Management IBM Dynamic Warehouse Николай Куликов Nikolay_Kulikov@ru.ibm.com March 2007.

ALM14 © 2007 IBM Corporation What keeps me Rational? IBM Rational Software Development Conference 2007 ® Utilizing IBM Rational Tools to Successfully Outsource.

System x Asia Pacific © 2007 IBM Corporation IBM KVM Solution.