7/27/2019 Who Just Killed My Db2 Gelb2003b

1/52

Who Just Killed My DB2?!

Ivan L. Gelb

Gelb Information Systems Corp.

Email: ivan@gelbis.com

Phone: 732-303-1333

@CMGA 2003 - Sydney

DB2 for z/OS and S/390

Insuring optimum DB2 service levels in the OS/390 environment is challenging because of not so

obvious dependencies between the many subsystems. Performance biases introduced by systems

tuners in z/OS - OS/390, CICS, and DB2 can dramatically affect the complexs service levels andtotal effective capacity. This presentation will describe how to focus DB2 environments tuning

projects while insuring that the interdependent areas of z/OS, CICS, TSO and batch are optimized.

Attendees will learn how to avoid being caught in unproductive finger pointing sessions by (a)

insuring that subsystems are tuned with proper bias, (b) monitoring performance metrics that

indicate the true illness or wellness of the complex, and (c) knowing what measures are available

once the source of a service level problem is identified.

7/27/2019 Who Just Killed My Db2 Gelb2003b

2/52

2003 GIS Corp . - www.gelbis.com 2

Agenda

Basics of Performance Tuning

DB2 Point of View

z/OS - OS/390 Point of View

CICS Point of View

Pointing in the Right Directions

We will focus on the latest systems versions: z/OS 1.4, CICS 2.2, DB2 6 8. The first four bullet

point will show where, how, and what to look for, and the last point will show six examples that

always cause contention among staff in different system and application areas. Following is oursession outline.

Basics of Performance Tuning: (1) Is anyone complaining? (2) Where to begin?

DB2 Point of View: (1) Eliminate bottlenecks; (2) Protect loved ones

OS/390 Point of View: (1) Allocate resources: CPU, I/O, storage; (2) Definitions of relative

priorities & their effects; (3) Protect loved ones (4) Who used, how much, of what, for howlong..

CICS Point of View: (1) Eliminate bottlenecks; (2) Protect loved ones (3) Create throttles;

Pointing in the Right Directions (3 CPU and 2 I/O cases): (1) Starved for CPU; (2) CICS region

saturated; (3) DPMODE = What? (4) Buffer Pools (5) High I/O Service Time

7/27/2019 Who Just Killed My Db2 Gelb2003b

3/52

2003 GIS Corp . - www.gelbis.com 3

Basics of Performance Tuning

Is anyone complaining?

Do YOU want to prevent complaints?

Where to begin?

Ask these questions

TRADEMARKS

The following are trade or service marks of the IBM Corporation: CICS, CICS TS , CICSPlex,

DB2, IBM, MVS, OS/390, z/OS, Sysplex, Parallel Sysplex. Any omissions are purely unintended.

MOAD MOTHER OF ALL DISCLAIMERS

All of the information in this document is tried and true. However, this fact alone cannot guarantee

that you can get the same results at your place and with your skills. In fact, some of this advice can

be hurtful if it is misused and misunderstood. As with all kinds of analysis, anything you may hear

or read can be understood and misunderstood in many ways that may seem contradictory to you. In

this regard, a further and associated contradictory element requires considerable systems analysis

and trade-off studies to arrive at the structural design, based on rigorous system engineering

concepts. By combining advice and certain experiences, any fully integrated performance testprogram is weakly equivalent to any subsystem compatibility testing designed to eschew

obfuscation. Gelb Information Systems Corporation, Ivan Gelb and any one found anywhere

assume no responsibility for this informations accuracy, completeness or suitability for any

purpose. Anyone attempting to adapt these techniques to their own environments anywhere do so

completely at their own risk. ;-)

7/27/2019 Who Just Killed My Db2 Gelb2003b

4/52

2003 GIS Corp . - www.gelbis.com 4

Is Anyone Complaining?

If YES, why?

Some service is slow and/or failing to meet service level

objectives (be sure that objectives are rational!)

Response time is OK, but not enough is completed

For either case:

Total Delays = Response Times Measured Service Times

What is Total Delays / Total Response Time ratio? < .50 is OK

but it up to doubles response time

The ratio acceptable to your site will depend on service level

goals.

First, we should not be waiting for complaints even though it helps justify the existence of our job.

Be effectively proactive, and they may wonder why tuning is needed.

Being proactive includes at least the following minimum activites:

-Profiles of all workloads so you can tell if behavior changes,

-Tracking of past and future changes in system software, hardware, and applications, and

-Profiles of utilization by business units of work is the most effective way to do all this.

7/27/2019 Who Just Killed My Db2 Gelb2003b

5/52

2003 GIS Corp . - www.gelbis.com 5

Do YOU Want to Prevent Complaints?

We consider WAITING for

complaints a CLS (Career

Limiting Strategy).

Your actions plan should

prevent complaints.

If you do your job real well,

some may wonder if you are

needed.

Wait for or prevent complaints,the plan of action is the same.

OK

Performance?

Adjust 1Thing!

Measure

AnyIdeas?

This is a bit of philosophy here.

The practical issue is that for well managed shops the best practice is to work on prevention of

complaints.

Performance management is a cyclical activity.

7/27/2019 Who Just Killed My Db2 Gelb2003b

6/52

2003 GIS Corp . - www.gelbis.com 6

Where to begin - 1

Evaluate performance:

Inside DB2

Outside DB2

Inside DB2 (not our sessions focus) Obtain / establish Service Level Agreements (SLAs) for

business critical work.

Without SLA-s, identify what is reasonable response time given

the service time profile of the workload.

Collect accounting trace class 2 and 3 for best information on

externally caused delays

Class 1 and 3 accounting traces reveal the tasks complete

activity (we will review a summary report from DB2 PM)

A DB2 performance evaluation should always start from inside out. Simply, you do not want to be

found with your homework not done.

We are going to look at reports in our quest for who killed your DB2 performance that will showboth types of measurements: indicators of problems inside DB2, and indicators that factors outside

DB2 are cause of degradation.

This session will drive towards identifying situations where the outside factors are causing the

misery.

7/27/2019 Who Just Killed My Db2 Gelb2003b

7/52

2003 GIS Corp . - www.gelbis.com 7

Where to begin - 2

Outside DB2 analysis focuses on eliminating factors that

degrade DB2s performance

Search for hints inside DB2 that outside factors may be

cause of problems:

Wait for

CPU due to higher priority work

CPU due to LPAR management of weights

Central storage

Long basic I/O service times

> 3 msec for cached writes

> 5 msec for cached reads

Long non-service time components of I/O servicePEND, Disconnect, Device Busy, Control Unit Busy

As you can see from the list on the slide, delays come in many flavors.

We will show reports and recommend where to look to isolate causes for the delays.

7/27/2019 Who Just Killed My Db2 Gelb2003b

8/52

2003 GIS Corp . - www.gelbis.com 8

Where to begin - 3

Prepare profiles of resource utilization by workload:

CPU, I/O, processing parallelism, processor storage, network time

Any latent demand?

If YES, what makes it up? Work you HAVE to care about or NOT?

Easy to tell if there is some latent demand see RMF CPU Activityreport

Harder to tell who it is waiting for CPU

Where to look?

RMF Workload, CPU, device, storage activity reports

SMF file activity reports

CICS & DB2 subsystems reports

CICS & DB2 subsystems traces are last resort, but most time

consuming to analyze and most trustworthy

This is an outline of where to begin analysis. Samples of how this is done follow later in

presentation.

7/27/2019 Who Just Killed My Db2 Gelb2003b

9/52

2003 GIS Corp . - www.gelbis.com 9

Ask These Questions - 1

What is your class 2 elapsed time (C2ET)?

How much of C2ET is:

Wait for CPU dispatch?

MVS Overhead? I/O-s related wait?

System page-ins?

Which DASD devices are not providing good performance?

7/27/2019 Who Just Killed My Db2 Gelb2003b

10/52

2003 GIS Corp . - www.gelbis.com 10

Ask These Questions - 2

Other classic questions you should not forget to ask, but willnot be discussed in much detail in our session today:

What is your system I/O rate/Second for:

Each pool?

Critical objects?

What is your average Synch I/O elapsed time?

What are your buffer pool hit ratios?

Is you SQL coding effective?

7/27/2019 Who Just Killed My Db2 Gelb2003b

11/52

2003 GIS Corp . - www.gelbis.com 11

DB2 Point of View

Eliminate bottlenecks

Protect loved ones

Identify Suspect Areas

7/27/2019 Who Just Killed My Db2 Gelb2003b

12/52

2003 GIS Corp . - www.gelbi