Standard template for internal and external presentations Journal on i/OS

© 2009 IBM Corporation

Standard template for internal and external presentations

Journal on i/OS

2009.10.15김 교 석MTS, IBM Korea

Journal 개요


Journal Basic

• Journal Objects– DB Objects: Physical Files, Access Paths– Data Areas, Data Queues– IFS Objects

• Journal– 마치 깔때기와 같은 역할– 각각의 object 들과 journal entry 들의 연결고리– 그 크기가 증가하지는 않는다 ,

• Journal Receiver– Journal entry 들을 저장하는 곳– Audit 관련 내용을 제공– 크기가 증가함에 따라 관리 필요


SMAPP

• SMAPP (System Managed Access Path Protection)– System 이 자동으로 지정된 Access Path Rebuild Time 에 따라 커다란 AP 들을 Journ

aling 하는 기능– System 의 Outage 시 새로 Rebuild 하는 대신 Journal 을 이용해 Access Path 들을

복구– Runtime vs. IPL (IASP vary-on)– EDTRCYAP 명령어를 이용해 원하는 AP 복구시간을 지정할 수 있다 .

• SMAPP 고려사항– Rebuild Time 을 너무 크게 지정하지 말 것– 만약 PF 가 Journal 되고 있다면 , AP entry 는 User Journal 에 저장된다 .– User Journal 에 쌓이는 SMAPP entry 에 의해 사용되는 량을 줄이기 위해서 RCVSIZO

PT(*RMVINTENT) 를 사용한다– Minimize Aps that are ineligible for SMAPP


EDTRCYAP


Remote Journal

• Source 에서 Target 으로 Journal entry 를 빠르고 효과적으로 보낸다 .

• O/S 의 SLIC Layer 에서 수행되는 기능

• Remote Journal 이 적용되면 , Receiver 에 있던 모든 Entry들이 가장 효율적인 “ Catch-up” Mode 로 보내진다 .

• 일단 Catch-up 되어진 후 , Synchronous 또는 Asynchronous 전달 Mode를 이용해 전송된다 .


Remote Journal

• Asynchronous Mode– Source 쪽의 Disk 에 entry 를 저장한 후 Target 에 보낸다 .– Sending task 는 충분하지 않은 Communication Bandwidth 등의 이유로 실패할 수도 있는데 ,

이럴 경우는 좀더 효율적인 Mode 로 자동으로 보내게 된다 .– 만약 Source 쪽이 Crash 되었을 경우 , 각 Entry 가 제대로 Target 에 적용되었는지 확실하지

않다 .– Target System 에 Entry 들을 보내는 작업이 지연된다 하여도 Source System 의 성능에는

영향이 없다 .

• Synchronous Mode– Source System 에서 Entry 를 Disk 에 쓰기 전에 , Target 에 Entry 를 보내고 Response 를

받는다 .– 따라서 , Source 가 Crash 되더라도 Entry 들이 Target 에 적용되었음을 Guarantee 할 수

있다 .– Target never falls behind – Source system performance can be impacted if insufficient communication bandwidth


Remote Journal

• Remote Journal 사용시 고려사항– 두 시스템간의 Communication bandwidth 가 충분해야 한다 .

– To test the throughput between systems, activate remote journal with an existing journal receiver which will run in “catch-up” mode and give an upper bound on the rate of transfer between systems

– Internal SMAPP Entries 의 전송을 막기 위해 RCVSIZOPT(*RMVINTENT) 로 Setting

– Network 문제로 인해 remote journal 이 중단되거나 비효율적으로 작동할 수 있으므로 NETSTAT command 를 사용하여 Retransmissions 이 얼마나 일어나나 확인한다 .


Journal Attribute

• 모든 journals 에 대해 아래 값들을 설정하도록 한다 .

• RCVSIZOPT(*RMVINTENT)– User journal 에서 internal SMAPP entries 에 의해 사용되는 공간을 최소화 – Remote Journal 을 사용할 경우 Internal SMAPP Entries 를 보내는 overhead 를 줄일 수

있다 .

• RCVSIZOPT(*MAXOPT2) – 좀 더 큰 maximum sequence number – 좀 더 큰 maximum receiver size – 좀더 많은 Disk Arm 들에 Receiver 의 Data 들을 분산시킬 수 있다 . – Internal entries 의 재 사용을 위한 추가 space 를 제공한다 . – 성능 개선효과 및 다른 function 들의 maximums 을 증가시킬 수 있다 .


Journal 성능 관련 10 가지 질문


Block 들을 쌓는 것과 같다 ...

과연 Journal 을 제대로 알고 사용하고 있는가 !

Larry


사용할 3 개의 Block 들은 ...

Pex Traces

DSPJRN

Collection Services

그리고 함께 사용할 2 개의 Tool 들 ...

SQLQueries

C Programs

i5/OS has a richer journal

때론 이것들을 함께 묶어서 ...


Journal Optimization 에 사용될 Block 들• DSPJRN to an OUTFILE

– DSPJRN JRN(LIB/JRN) OUTPUT(*OUTFILE) OUTFILFMT(*TYPE5)

– OUTFILE(LIB/OUTFILE)

• Run Collection Services– Performance Tools(5722-PT1) 이 설치되어 있다면 Collection Services 시작

• GO PERFORM

• Select Option 1 to start collection services.

– API CALL

• CALL QYPSSTRC PARM('*PFRxxxxxx' '*STANDARDP' X'00000000') (xxxxxx 는 Space 6자리 )

• Run Performance Explorer1) Use the ADDPEXDFN command to add a PEX definition which will collect Journal Events

2) Start PEX using the STRPEX command and specify the PEX definition you created

3) Run workload to be analyzed

4) End PEX with the ENDPEX command and specify to output the data a user library

5) Run the PARSEJOPEX program to parse the Journal Trace Point information


이 Session 에서 소개드릴 Tool 및 Program source 는 아래 URL 에서 download 가능합니다 .:

http://www-03.ibm.com/systems/i/software/db2/journalperfutilities.html


Topic Classic Question

How do I get Journal to use all my disk arms ?

Do I have enough to make Journal happy ?

Are my settings and applications Journal-friendly ?

How can I tame the Commit tiger ?

Too may files serviced by a single journal ?

Have I got journal entries I don’t need ?

Disk Arms

IOA Write Cache

Bundling

Commit

Bullies

Discarding Chaff

How can I send less data to disk and target machine ?Skinny Jrn Entries

Am I failing to exercise good Jrn Housekeeping ?

Are my Jrn background tasks gasping for breath ?

Full Closes

Jrn Recovery Ratio

Are my Remote Jrn ASYNC sending tasks falling behind?Remote Journal

아래와 같은 10 가지 질문에 어떻게 답할 것인가 ?


Q1. Disk Arms

Journal Receiver arms

User ASPReceiver

Journal일반적으로 Jrn Rcvr들은 여러 Disk Arms 에 골고루 퍼져서

분포하지만 , 과연 몇 개나 사용하고 있는가

?

– Journal 이 Disk Arm 들을 적절히 사용하고 있는가 ?


Round-robin use of disk arms

2

12

...

1 Arm Number


사용중인 Disk Arm 은 ...

현재 journal receiver 가 몇 개의 disk arm 을 사용하고 있는가 ?Requirements

Journal Receiver should contain a sufficient number of entries to analyze

Journal Receiver 가 사용하는 disk arm 갯수를 알아 내는 방법

1) DSPJRN to an OUTFILE using *TYPE5 to obtain arm information

2) Run the following SQL query to determine number of arms being used:

SELECT MAX(JOARM) from lib/outfile

?

Output:

Number of Arms used for the Receiver


DSPJRN w/ *TYPE5

Display Data Data width . . . . . . : 862 Position to line . . . . . Shift to column . . . . . . ...40....+...41....+...42....+...43....+...44....+...45....+...46....+...47.... RCV ARM THREAD ADDRESS REMOTE REMOTE ASP NUMBER ID FAMILY PORT ADDRESS HEX 1 2 0000000000000000 0 0 1 2 0000000000000000 0 0 1 2 0000000000000000 0 0 1 2 0000000000000000 0 0 1 3 0000000000000000 0 0 1 3 0000000000000000 0 0 1 3 0000000000000000 0 0 1 3 0000000000000000 0 0 1 3 0000000000000000 0 0 1 4 0000000000000000 0 0 1 4 0000000000000000 0 0 1 4 0000000000000000 0 0 1 4 0000000000000000 0 0 1 4 0000000000000000 0 0 More... F3=Exit F12=Cancel F19=Left F20=Right F21=Split

{

{


Query output

Display Data Data width . . . . . . : 13 Position to line . . . . . Shift to column . . . . . . ....+....1...

MAX ( JOARM )

11

******** End of data ******** Bottom F3=Exit F12=Cancel F19=Left F20=Right F21=Split

Quantity of arms


사용중인 Disk Arm 은 ...

왜 내가 생각한 만큼의 disk arm 을 사용하지 않을까 ?

아래 값들이 충분하지 않을 경우 :

1. Journal Threshold

2. RcvSizOpt Setting 을 MaxOpt1 로… (or higher)

?


Jrn Rcvr arms

Write Cache

IOP1

AP

PF

System ASP

Journal

PF

User ASP

Threshold

Unused

충분한 Threshold 를 가지고 있지 않다면 새

disk drive 들은 사용하지 않고

남아있게 된다 .

만약 Journal Threshold 가 너무 작다면 ??


Journal Receiver Thresholds

CRTJRNRCV ... THRESHOLD(1800000) { Means 1.8 Gig }

ƒ Note: Threshold 는 Kilobytes 단위ƒ 높은 Threshold 로 Setting 하게 되면 :

– 좀 더 많은 disk arms 을 사용하게 된다 . 즉 , Threshold 의 매 64 Meg 가 추가될 때마다 추가 disk arm 을

사용하게 된다 .– 그리고 좀 더 큰 bandwidth 를 가지고 achieve 가능 .

Threshold

Capacity = *MaxOpt1


Journal Receiver Threshold 와 Disk Arm 의 상관관계Threshold # of Disk arms

< 640 MB Up to 10

704 MB 11

6.4 Gig 100

RcvSizOpt(*MaxOpt1)

Min

Max

100

...

Threshold 의 64MB 증가마다 additional arm 이 더 사용됨

.

.

.

.

.

.

778 MB 12


Example:

ƒ CRTJRNRCV JrnRcv(MyLib/Rcv2) Threshold(70000000) { 70 Gig }

ƒ CRTJRN Jrn(MyLib/Jrn2) JrnRcv(MyLib/Rcv2) RcvSizOpt(*MaxOpt1) { Rqsts 1 TB max }

Capacity = *MaxOpt1

Threshold

BACK

Takes both

Journal Receiver thresholds


Jrn Rcvr arms

Write Cache

IOA

AP

PF

System ASP

Journal

PF

User ASP

System 의 IOA Write Cache 를 가장 많이 사용하고 , 최대

수혜자는 바로 Journal…

Memory

Q2. IOA Write Cache – Journal 에서 사용할 만큼 충분한가 ?


Write Cache

MemoryDASD

Fast Write

Over Threshold Destage

BackgroundDestage

Threshold

DASD

Trickle

Demand (and wait)

Continuous

Slower Write

Memory

IOA Write Cache


Receiver

Journal

Need enough Arms

Write Cache

. .

.

IOA

Model Write Cache

(older) 2763 10 Meg 2782 40 Meg

(older) 2778 26 (104) Meg 2757 235 (757) Meg

Physical quantity

Feels like

Write Cache 에 대해 너무 인색하지 말 것…


IOA write cache

IOA write cache 는 충분한가 ?

Requirements The performance tools (5722PT1) to start collection services

Fast disk write 의 사용률을 알아내는 방법1. Start collection services 2. Collect data for at least 5 minutes (disk statistics are gathered at this interval). 3. Dump the statistics to a database table using the following CL command: CRTPFRDTA FROMMGTCOL(*ACTIVE) TOLIB(LIB) CGY(*DISK) - This will create a file called QAPMDISK. 4. Query the resulting file. The interesting data is in columns DSCCFW (fast writes) and DSWRTS (total writes)

SELECT 100 * SUM(DSCCFW) / SUM(DSWRTS) FastWrtPrcnt from QMPGDATA/qapmdisk where dsasp = 2 replace with correct ASP #

?

Output:% of Fast Writes


Percentage of fast disk writes

Display Data Data width . . . . . . : 42 Position to line . . . . . Shift to column . . . . . . ....+....1....+....2....+....3....+....4..

FASTWRTPRCNT

99


85

Rule of Thumb: Journal needs 99% fast writes

BACK


User ASP

PF 3

IOA

Write Cache

128K Buffer

Batch Job #1

Batch Job #2

Batch Job #3

PF 1

PF 2

Journal Receiver

Journal

Main Memory Buffer

Bundle

Q3. Journal Bundling – Application 이나 Setting 이 Journal-Friendly 한가 ?


그렇다면 , 현재 내 System 의 Journal bundles 은 얼마나 될까 ? 어떻게 해줘야 할 것인가 ?

?

Bundles 을 크게 하면 :

Disk 에 Write 하기 위한 Activity 를 좀 더 효율적으로

또한 Remote Journal 을 좀 더 효율적으로

Journal Bundling


Bundling 이 어떻게 되고 있는가 ?

Bundle #10Bundle #2 . . .

Arm #1

Arm #10

Bundle #1

J Ent

Round Robin

• Journal 은 Round-Robin 방식으로 entry 들을 저장한다

• 아래의 output 을 이용해 Journal 의 평균 Bundle Width 를 알 수 있다– DSPJRN Format *TYPE5 를 이용– Journal Entry 당 사용한 Disk Arm # 을 알 수 있다 .


Journal Bundling

그렇다면 , 현재 내 System 의 Journal bundles 은 얼마나 될까 ?

Requirements Journal Receiver 가 1 개 이상의 disk arm 으로 구성된 ASP 에 존재해야

한다 .

아래 예와 같이 Journal Receivers 의 내용을 DSPJRN TYPE5 와 INCHIDENT(*YES) options 을 사용하여 output 을 생성한다 .

Example:

DSPJRN JRN(mylib/myjrn) OUTPUT(*OUTFILE) INCHIDENT(*YES) OUTFILFMT( *TYPE5 ) OUTFILE(mylib/myoutfile)

?


DSPJRN

Display Data Data width . . . . . . : 862 Position to line . . . . . Shift to column . . . . . . ...40....+...41....+...42....+...43....+...44....+...45....+...46....+...47.... RCV ARM THREAD ADDRESS REMOTE REMOTE ASP NUMBER ID FAMILY PORT ADDRESS HEX 1 2 0000000000000000 0 0 1 2 0000000000000000 0 0 1 2 0000000000000000 0 0 1 2 0000000000000000 0 0 1 3 0000000000000000 0 0 1 3 0000000000000000 0 0 1 3 0000000000000000 0 0 1 3 0000000000000000 0 0 1 3 0000000000000000 0 0 1 4 0000000000000000 0 0 1 4 0000000000000000 0 0 1 4 0000000000000000 0 0 1 4 0000000000000000 0 0 1 4 0000000000000000 0 0 More... F3=Exit F12=Cancel F19=Left F20=Right F21=Split

Arm change


Journal Bundling

Is Entry on same Arm as previous

Entry?

Do More Entries Exist?

YES

NO

Output Results

NO

YES

Get next Entry

Update stats for current bundle

Update overall stats and reset current

bundle stats Output:Average Bundle SizeMaximum Bundle SizeMinimum Bundle Size

Rule of Thumb: 128k is optimal

Tool1) Display set of Journal Receivers to an OUTFILE using DSPJRN TYPE5 an

d INCHIDENT(*YES) options 2) Run BUNDLE program on this outfile (Source code http://www-03.ibm.com/systems/i/software/db2/journalperfutilities.html )


BUNDLE program output...

number of entries = 530730 Total size of all entries = 354982052 Number of bundles = 2298

Average bundle size = 124474 bytes max bundle size = 317170 bytes min bundle size = 609 bytes

The optimal bundle size is 128 KB or wider

F3=Exit F4=End of File F6=Print F9=Retrieve F17=Top F18=Bottom F19=Left F20=Right F21=User Window


How can I get bigger

bundles ?

My ODP

Jrn Ent

If you’re willing to make application changes...

Overrides 를 사용하면 이 부분이 달라집니다 .

Working storage of application

그럼 , 이제 Bundling Width 를 어떻게 늘릴 것인가 ?

• Application 입장에서는 간단하게 Database Override 통해 가능하다…


Do more pairs Exist?

YES

NO

End

Get next pair

Obtain list of distinct file/job pairs

Determine # of

Inserts for file/job pair

Determine # of Updates and Deletes for file/job pair

Does file/job

pair have only Inserts?

YES

Output file/job pair identity

NO

Output:Files which should be

considered for SEQONLY(*YES)

Good candidate: SEQONLY

Tool #2Journal Bundling - Methodology

Tool1) Display set of Journal Receivers to an OUTFILE 2) Run SEQONLY program on this outfile (Source code http://www-03.ibm.com/systems/i/software/db2/journalperfutilities.html )


Select * from lib/output

Display Data Data width . . . . . . : 62 Position to line . . . . . Shift to column . . . . . . ....+....1....+....2....+....3....+....4....+....5....+....6.. OBJ LIB MBR JOB NUMPTS PROD_TRANS CKRJTEST PROD_TRANS CKRJTEST 136,040 STOR_TRANS CKRJTEST STOR_TRANS CKRJTEST 100,014


Number of puts

Good candidates


Journal Bundling - Application

• ODP Buffer 를 효과적으로 사용– OVRDBF …. SEQONLY(*YES)

• 다른 방법은 또 없을까 ?

• Program 에 각각의 File 당 하나의 View 만을 사용해야 한다는 생각은 버리자…– 때론 하나보단 두 개가 나을 때도…


System ASP

AP

PF

PF

ODP 1

Pure Inserts

ODP 2

ODP 2 는 EXTRA "view“ 를 사용한 경우 .이 instance 는 File Open 시

SEQONLY(*YES) 를 이용하였다 .

Active

Jrn Rcvr arms

128K Buffer

Write Cache

IOP

Journal

User ASP

UP

PTPT

PT

Regular ODP

Use a job with two "views"


Journal Bundling – Journal Cache

• Application 수정 이외에 좀 더 편하게 Bundle Width 를 늘릴 방법은 없을까 ?


Batch Application

Cache-enabled

AP

Write Cache

IOA

PF

PF

CHGJRN ... JrnCache(*Yes)

System ASP

Receiver

Journal

User ASP

128K Buffer

Main memory cache

ODP Buffer

One per disk arm

128K Buffer

Journal Caching


Journal Cache 의 성능 이점• Batch Job 의 경우

– 5 Million DB operations (10% Adds, 90% Updates)– 9 Million resulting Journal entries (captured both before and after images)

Elapsed Time

Original Batch run, no Journaling 1118 SecOrdinary Journaling enabled 9773 SecUsing the new Journal cache option 1433 Sec 약 6 배 향상


Keep up rate on Target machine

w/o Caching 600,000 transactions/Hr

With Caching on target 2,400,000 transactions/Hr

Source System Target System

Journal

Cache.

Journal Cache 의 성능 이점• Remote Journal 을 사용할 경우

– Target System 이 Keep-Up Mode 일 경우


Journal Caching Option

• A priced feature of the Operating System– Option 42– CHGJRN JRN(MYLIB/MYJRN) JRNCACHE(*YES)

• Journal Cache Option 이 도움이 될지 안될지 판단하기 위해서는 아래와 같은 방법을 사용하여 알아본다 .

Tool1) Display set of Journal Receivers to an OUTFILE

2) Run the following query against the OUTFILE

select count(*) as NumEntEligible,

sum(JOENTL) as TotalBytes from lib/file

where JOCCID = 0

and ((JOENTT = 'PT' OR JOENTT = 'PX' OR JOENTT = 'DL' OR JOENTT = 'UP') and JOCODE = 'R')

Output:Number of entries eligible to be CachedNumber of bytes eligible to be Cached


Display Data Data width . . . . . . : 58 Position to line . . . . . Shift to column . . . . . . ....+....1....+....2....+....3....+....4....+....5....+... NumEntEligible TotalBytes 38,197 6,910,903

Bottom F3=Exit F12=Cancel F19=Left F20=Right F21=Split

Looking for bundle-eligible journal entries

Quantity of Journal entries Number of bytes


Tool

1) Display set of Journal Receivers to an OUTFILE 2) Run the following query on the OUTFILE

select count(*) as NumEntEligible, sum(JOENTL) as TotalBytes from lib/file

where JOCCID = 0

and ((JOENTT = 'PT' OR JOENTT = 'PX' OR JOENTT = 'DL' OR JOENTT = 'UP') and JOCODE = 'R')

Output:Number of entries eligible to be CachedNumber of bytes eligible to be Cached

Drop this line

일단은 V5R4 으로 ...

V5R4 에서 “soft” commit 을 사용할 수 있습니다 .

만약 Commit User 라면…


BACK

Are my Journal Bundles wide? (use

Bundle tool)

Can I use SEQONLY(*YES)?

(useSEQONLY program)

Good Performance!

Use SEQONLY(*YES) if possible

Will Journal Caching likely help?

(use query)

Turn on Journal Caching

Yes

No

Yes

No

Yes

(start)

Bigger bundles are better

Journal Bundling – 결론


... ...BC ECSC SC SCCM... ... CM RB......

Example Application: STRCMTCTL

ENDCMTCTL

Transaction 1

Transaction 2

Transaction 3

Commit Rollback

Q4. Commit – Journal 을 통해 Commit 길들이기• Commitment control 과 관련된 entry 는 pair 로 구성됨

– BC - Begin Commitment Control – EC - End Commitment Control bound all transactions for a Job– SC and CM (or RB) serve as bookends for a particular transaction– SC – Commit cycle started– CM(RB) – Set of record changes committed ( rolled back )


많은 Row 가 Lock 되어있는 것이 과연 나쁜가 ?

Locks are stored in Main Memory

- May take a while to unlock (CPU) - Even more time if they are paged out (DISK faults)

Main Memory Pool

Row Changes Lock (hold record)

?

필요한 만큼의 lock 이 수행되지 않으면 성능이 저하됨

• Un-locking – transaction 동안 잡고 있던 lock 를 release 하는 step (time-consuming )

– Un-do – object 를 원래의 state 로 되돌리는 step ( actual rollback )

• 만약 모든 Lock 들에 대해 적절한 Memory 가 없다면 , 성능은…


delete from mylib/table1 where due_date <= ‘12/31/2003’

¿ À́ Ã¹ ã¿ £ ¿ À· ¡µ È Dataµ é Á » Á ¤̧ ®Ç Ø º ¼± î?

Programmer

Your Boss

무심결에 수행한 SQL 한 문장이…


Ouch !This is going to be a long

night !

1 Million3 Million

5 Million7 Million

8 Million10 Million

0

500

1000

1500

2000

2500

3000

3500

Seco

nds

Un-doUn-lock

조심해야 할 Locked Rows

• 그럼 실제로 Un-Do (Rollback) 과 Un-Lock Time 이 얼마나 걸릴지 알 수 있는 방법은 ?


Determining rollback progress

• Commit Status Screen– WRKCMTDFN + 5 + F6 + Journal

Display Journal Status System: ISERIES1

Job: QPADEV001P User: JOEUSER Number: 071957 Commitment definition . . . . . . . . : *DFTACTGRP Type options, press Enter. 5=Display commit cycle entries 6=Display all entries 7=Work with journal attributes Commit Cycle Opt Journal Library Identifier _ MYJRN1 MYLIB1 123 _ MYJRN2 MYLIB2 456 _ MYJRN3 MYLIB3 789 BottomF3=Exit F5=Refresh F6=Display resource status F9=Command line F11=Display rollback status F12=Cancel F23=More options


Display Journal Status System: RCHASJMB

Job: QPADEV001P User: RANDYJ Number: 071957 Commitment definition . . . . . . . . : *DFTACTGRP Type options, press Enter. 5=Display commit cycle entries 6=Display all entries 7=Work with journal attributes -----------Rollback----------- Opt Journal Library Date Time % Complete _ MYJRN1 MYLIB1 100 _ MYJRN2 MYLIB2 08/23/05 12:00:00 50 _ MYJRN3 MYLIB3 0 BottomF3=Exit F5=Refresh F6=Display resource status F9=Command line F11=Display unlock status F12=Cancel F23=More options

Start time of rollback

The Rollback Phase


Display Journal Status System: RCHASJMB

Job: QPADEV001P User: RANDYJ Number: 071957 Commitment definition . . . . . . . . : *DFTACTGRP Type options, press Enter. 5=Display commit cycle entries 6=Display all entries 7=Work with journal attributes ------------Unlock------------- Opt Journal Library Date Time % Complete _ MYJRN1 MYLIB1 100 _ MYJRN2 MYLIB2 08/23/05 10:42:00 96 _ MYJRN3 MYLIB3 0 BottomF3=Exit F5=Refresh F6=Display resource status F9=Command line

Helps track time spent unlocking lots

of Rows

The Unlock Phase


Locking Limits

QAQQINI file record lock maximum

• The QAQQINI file 을 이용해 한 Transaction 내에서 허용할 maximum number of record locks 을 지정한다 .

– Default 값으로 500,000,000 records• 대부분의 경우 이 값을 줄여주어야 합니다 .

– COMMITMENT_CONTROL_LOCK_LEVEL value 를 줄이면 된다 .

1) CRTDUPOBJ OBJ(QAQQINI) FROMLIB(QSYS)

TOLIB(qusrsys) DATA(*YES)

2) Update qusrsys/QAQQINI Set QQVAL = 32000

where QQPARM = ‘COMMITMENT_CONTROL_LOCK_LIMIT’

Upper Case Only


Having too many locks

can be unpleasant

얼마나 많은 Locks 이 발생하는지 ?

• Journal 을 이용해 Commit Cycle 내에 Insert, Update, Delete 건수를 확인하면 된다 .– 동일한 Commit ID(CID) 를 가진다 .

• Guide Line: Locked Record 를 100K 미만이 되도록

... ...SC ... ... CM ...PT CM SC

Seq #17CID 17 CID 17CID 17

Seq #55

CID 55

Transaction 17 Transaction 55

UPCID 55 CID 55

Transaction 17

CID = #17

Transaction 55CID = #55

Jrn Receiver

Avoid too many locks

...


Large Number of Entries within

commit cycles?

Yes

Reduce size of commit cycles and restrict number of

locks

?

Commit – Excessive Locks

얼마나 많은 Locked Rows 들이 발생하는지 ?

Requirements Journal Receiver

Tool1. Display set of Journal Receivers to an OUTFILE

2. Run the following querys on the OUTFILE

// ordered list of number of locked rows in each commit cycle

create view lib/view2 as (select count(*) as cnt, JOCCID FROM lib/outfile

WHERE JOCCID != 0 and JOENTT != 'UB' group by JOCCID)

SELECT cnt, JOCCID from lib/view2 order by cnt desc

// average number of entries in a commit cycle

SELECT avg(cnt) as average from lib/view2 Output: Ordered list of number of entries in commit cycle

- Approximate number of Locks held

Average number of entries in commit cycle


Query output ( Transaction width ) Display Data Data width . . . . . . : 30 Position to line . . . . . Shift to column . . . . . . ....+....1....+....2....+....3 CNT COMMIT CYCLE ID 1197 4,968,250 52 5,358,581 44 4,903,684 35 4,952,986 12 5,238,664 7 5,290,598 7 5,403,842 7 4,831,737 7 5,086,048 7 5,025,828 7 4,803,609 5 5,405,660 5 4,842,522 More... F3=Exit F12=Cancel F19=Left F20=Right F21=Split

Lock count


Query output ( Average transaction width ) Display Data Data width . . . . . . : 30 Position to line . . . . . Shift to column . . . . . . ....+....1....+....2....+....3 AVERAGE

8

******** End of data ********

More... F3=Exit F12=Cancel F19=Left F20=Right F21=Split BACK


Q5. Bully – 특정 Journal 에 너무 Traffic 이 몰리지 않는지 ?


AP

User ASP

PF 1

Journal

Memory Pool

PF 2PF 3 PF 4

PF 5PF 6 특정 journal

에 너무 많은 traffic 이 몰린다면 ?


Two streams are better than oneAP

PF 1

User ASP


Journal 2

Memory Pool

PF 2PF 3 PF 4

PF 5PF 6

Journal 1

User ASP


remote journal

환경에 유용 ..

Remote Journal


Find out the primary journal depositor

어떤 journaled object 가 가장 많은 traffic 을 유발하는가 ?

SELECT JOLIB, JOOBJ, JOMBR, count(*) as NumEntries, sum(JOENTL) as bytes FROM LIB/OUTFILE GROUP BY JOLIB, JOOBJ, JOMBR order by bytes desc


Tool

1) Display set of Journal Receivers to an OUTFILE using DSPJRN command2) Run the following query on the OUTFILE

Output:Ordered list of objects producing the most Journal traffic

?


Query output

Display Data Data width . . . . . . : 94 Position to line . . . . . Shift to column . . . . . . ....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+....

LIBRARY OBJECT MEMBER NUMENTRIES BYTES NAME NAME NAME CKRJTEST NEW_PROD NEW_PROD 281,754 191,310,966 CKRJTEST PROD_TRANS PROD_TRANS 131,204 82,264,908 CKRJTEST STOR_TRANS STOR_TRANS 97,082 67,569,072

******** End of data ******** Bottom F3=Exit F12=Cancel F19=Left F20=Right F21=Split BACK


PT DL UP

Q6. Journal Entry 를 줄일 방법은 없을까 ?

• 어떤 종류의 Entry 가 있는지에 따라 Tuning 의 방법이 틀려진다…


To identify the most popular journal entry flavors...

어떤 Journal Entry 가 많은 부분을 차지하는가 ?


Tool

1) Display set of Journal Receivers to an OUTFILE using DSPJRN command2) Run the following query on the OUTFILE

?

Select jocode, joentt, count(*) as number from cktest/dspjrnout8 group by jocode, joentt order by number desc

Output:Flavors and Count of Journal Entries


Sampling the flavors...

Display Data Data width . . . . . . : 20 Position to line . . . . Shift to column . . . . . . ....+....1....+....2 CODE TYPE NUMBER R PT 171,503 R UP 107,150 R UB 107,150 C SC 12,284 C CM 12,284 R PX 2,284 F IZ 2,058 D CT 18 F MC 18 D MA 18 J XP 10 D JF 9 F JM 9 C BC 2 J PR 1 J RD 1


Flavors

Most popular


Journal Receiver size 를 줄이기 위해

어떤 방법들이 있을까?

Receiver

Journal

Bloated

과연 Optional Entry 들이 필요한가 ?

Lean

Journal Receiver 의 크기를 줄일 시점… ?


Open/Close entry 는 필요 없다…• Open (OP) and Close (CL) entries are not required Journal entries.

• Not required for:– IPL recovery– APYJRNCHG / RMVJRNCHG– High Availability Business Partners

• These can take up excess space in your Journal Receivers!

Open entry Close Entry

... ... ... UB UP DLOP CL

CHGJRNOBJ Obj((*ALL *FILE)) Atr(*OMTJRNE) OmtJrnE(*OPNCLOSYN)

Diet plan #1


Method 1 of 5 for reducing Journal Entry Data

Open/Close Entries 가 얼마나 있는지 ?


Tool

1. Display set of Journal Receivers to an OUTFILE

2. Run the following query on the OUTFILE

?

select sum(joentl) as “Open/Close Bytes” from LIB/OUTFILE where ((JOENTT = 'OP' or JOENTT = 'CL‘) and JOCODE = 'F')

Select sum(joentl) as “Total Bytes” from LIB/OUTFILE

Output:Bytes which can be reduced with OMTJRNE(*OPNCLO)Total number of bytes examined

Minimizing the chaff


Chaff size


Open/Close Bytes

5,143,858


Opportunity for savings


Total journal receiver content


Total Bytes

47,489,757



Scan for "before" images

... DL ... ... ...

"Before"image

UP UPDL PTUB UB

Diet plan #2

Before Image 가 필요 없다면 빼버리자 .

• Delete 나 Update 를 하게 되면 "Before" images 를 생성하게 된다 .– RMVJRNCHG 를 할 경우 필요하다 .– 하지만 , 이럴 경우가 없다면 과감히 빼버리자…

"Before"image

CHGJRNOBJ Obj((*ALL *FILE)) Atr(*IMAGES) Images(*AFTER)



Before Image Entries 를 줄여서 얼마나 Space 를 줄일 수 있는지 ?


Tool



?

select sum(JOENTL) as “Before Images” from LIB/OUTFILE where JOENTT = 'UB' and JOCODE = 'R' and JOCCID = 0

select sum(joentl) as “Total Space” from LIB/OUTFILE


Output:Bytes which can be reduced via IMAGES(*AFTER)Total number of bytes examined


Bloat due to “Before” images – space I could save


Before Images

28,448,550


Space consumed by “before” images


Total journal entry space consumed


Total Space

98,747,195


Space consumed by all images


SMAPP-induced traffic

Journal Entry

AP

Journal Entry

PF

Write Cache

IOP2

Write Cache

IOP1

. . .JOE

PF

JOE

AP

User ASP

Main Memory

이부분을 줄일 방법은 ?

Journal Entry 에는 두가지종류가 있다 .


Scan for Ix-flavored entries

... DLUB UP PT ... ... ... Iold key

Iold key

INew key

INew key

Normally Hidden

Diet plan #3

Index 관련된 Journal Entry 찾기• 일반적으로 Index-flavored journal entries 들은 숨겨져 있다…

– 물론 , 숨겨진 entries 라도 space 는 차지한다 !– 이 숨겨진 Entry 를 보기위해서는 DSPJRN 시 IncHidEnt 를 사용 .

• 이런 Ix-flavored journal entries 에 의해 journal receiver size 가 증가된다 .– 이것들을 다른 disk arms 을 사용하도록 분리할 수 있다 .

CHGJRN JRN(MYLIB/MYJRN) RCVSIZOPT(*MAXOPT2 *RMVINTENT) JRNRCV(*GEN)



*RMVINTENT 를 사용했을 때 얼마나 Space 를 줄일 수 있는지 ?


Tool

1. Display set of Journal Receivers with INCHIDENT to an OUTFILE


?


select sum(JOENTL) as “Bloat” from LIB/OUTFILE where JOCODE = 'I‘

select sum(joentl) as “Total Bytes” from LIB/OUTFILE

Output:Bytes which can be reduced via *RMVINTENTTotal number of bytes examined


Bloat due to the hidden entries


Bloat

35,204,395


May make sense to route these bytesto a separate set of disk arms


Total bytes (Hidden plus visible)


Total Bytes

464,595,217


Rule of thumb:If the percentage of byteswhich are hidden exceeds 20%seems like you might want to Put a fork in the road

BACK


Diet plan #4Q7. 각 Entry 의 내용을 줄일 수 있을까 ?

• 각각의 Entry 별로 있는 추가 auditing/descriptive header data 를 제거하고 ESD (Entry-specific-Data = Row image) 만을 가지도록 한다 .

– Audit Data 에는 Program(26), Job(10), User(10) 에 대한 정보가 있다 .

• 모든 Journal Entry 에 Auditing Data 가 있는 것은 아니다 .– SMAPP 관련 Entry 은 해당 Data 를 가지지 않는다 .

... DL ... ... ...

Short entry

UB UP UB UPDLIx PT

Full-header(Header consumes 46 bytes)

ESDFix_Hdr

CHGJRN Jrn(mylib/myjrn) JrnRcv(*GEN) RcvSizOpt(*MAXOPT2 *RMVINTENT *MINFIXLEN)

또는 원하는 정보만 보관하도록 할수도 있다 . CHGJRN Jrn(mylib/myjrn) JrnRcv(*GEN) FixLenDta(*USR)



Full-width Audit Data 에 의해 사용되는 Space 가 얼마나 되는가 ?


Tool



?

Squeezing the headers

select count(*) * 46 as BytesReduced from LIB/OUTFILE where JOJOB != '*OMITTED'

select sum(joentl) as TotalExamined from LIB/OUTFILE

Output:Bytes which can be reduced *MINFIXLENTotal number of bytes examined


Bytes I could save if header were smaller

Display Data Data width . . . . . . : 16 Position to line . . . . . Shift to column . . . . . . ....+....1....+.

BytesReduced

32,393,384


Bloat due to full-sized/default headers

How much of this would you be willing to

trim ?


Total bytes among visible journal entries


TOTALEXAMINED

464,595,217



Jrn Rcvr

File B

File A Changed record imageFile B Changed record image

Journal Receiver's content

Record 1

Record 2

Record 3

::Record N Field1

Mary

Field 2

Parker

Field 3

555-9181

Field n

Ohio...

File's content

Field1

Mary

Field 2

Parker

Field n

Ohio...

Old record

New record

Minimized Journal Entry

Field 3

555-9181

Field 3

333-0717

전체 Record 정보가 아닌 변한 Field/Bit 만 저장하면 안될까 ?

Diet plan #5

CHGJRN Jrn(mylib/myjrn) JrnRcv(*GEN) MINENTDTA(*FILE)


Scan for UB/UP pairs

... DLPT ... ... ...

Compare "before" image to after image

We'll sum the resulting delta

UB UP UB UP

Am I a good candidate for Minimized Journal

entries ?

Jrn-Minimal 에 의해 줄일 수 있는 Size 계산• UB/UP 쌍을 서로 비교하여 계산할 수 있다 .

– We'll tally the changed bytes• They're the only ones we really need to journal !

– (plus a little descriptive MetaData for good measure)


Output results

Get next set of Entries

Obtain list of UB/UPpairs

Do More sets Exist?

YES

NO

Obtain ESD for each Entry

Loop through ESD

and total #of bytes different

Report Sum total # of bytes different

Output:Total Number of UB/UP bytesNumber of bytes which differ

Minimizing Journal Entry SizeMethod 5 of 5 for reducing Journal Entry Data

Jrn-Minimize 에 의해 줄일 수 있는 Space 가 얼마나 되는가 ??Requirements

Journal Receiver Before Images must be collected in the Journal

Tool1) Display set of Journal Receivers to an OUTFILE 2) Run JMDCHECK program on this outfile (Source code available !)


Results of JMDCheck program...

Number of UB/UP Candidate Journal Entries = 281754 Difference: 24936785 bytes. Total bytes: 78045858 bytes.

Press ENTER to end terminal session.

===> F3=Exit F4=End of File F6=Print F9=Retrieve F17=Top F18=Bottom F19=Left F20=Right F21=User Window

Gives a feel for how much spaceis being consumed by duplicateunchanged information thereby illustrating the savings opportunity


Ordinary Jrn_Min Difference

Size Jrn Ent 637 bytes 411 bytes 35% less

Disk writes 55/sec 36/sec 34% less

Write width 16k/write 12k 25% less

CPU busy 61.5% 60.6% 1.5% less

ERP Environment

CHGJRN ... MINENTDTA(*FILE)

BACK

Journal Minimal Data 의 효과


PF 1

Pgm 1

Close

Pgm 2

Close

Open Count = 2

Last Close

Q8. Full Close – 과연 Journal 이 제대로 작동하고 있는가 ?


Output:Number of Full Closes for each file

Minimizing Journal Entry Size

Full Close 가 얼마나 일어나고 있는가 ??Requirements

Journal Receiver with open/close entries recorded Starting point must be point where no files where open

Tool1) Display set of Journal Receivers to an OUTFILE 2) Run CLOSECHECK program on this outfile (Source code available !)

Do more Entries Exist?

Do More Files Exist?

YES

NO

Output Results

NO

YES

Get next File

What type of Entry is next?

Obtain list of Files

Obtain list of

OP/CL entries for the file

Output data for file and reset count

Increment count

Decrementcount

Count = 0?

CL

OP

Bump Full Close Count

YES

NO


Results of CLOSECHECK program...

Display Data Data width . . . . . . : 62 Position to line . . . . . Shift to column . . . . . . ....+....1....+....2....+....3....+....4....+....5....+....6.. OBJ LIB MBR JOB FULLCLOSE T1 CKTEST2 QPADEV0001 0 T1 CKTEST2 T1 QPADEV0001 1 T2 CKTEST2 QPADEV0001 0 T2 CKTEST2 T2 QPADEV0001 15 T3 CKTEST2 QPADEV0001 0 T3 CKTEST2 T3 QPADEV0001 4 T4 CKTEST2 QPADEV0001 0 T4 CKTEST2 T4 QPADEV0001 4451



PF 1

Pgm 1

Close

Open Count = 1

Last Close

Open

Job 2

Never Close(w/o commit)

BACK

OpenCount= 2

Fake out i5/OS...


PF1

JORECRA Task

Sync Pt

... PT SMAPP many entries! UB DL... ... Journal Receiver

PF1 PF2

AP1

AP1

PF2

#17

Oldest ?

#17PF1

#22 #50,001

#50,017

50,000 Recovery Ratio

AP1 #22

IPL recovery Exposure range

The more Journaled

objects you're modifying, the more

memory you're forcing

Q9. Journal Recovery Ratio 란 ?


Journal Housekeeping (JORECRA Task)

Journal Recovery ratio : JORECRA 를 조절해야 하는가 ??

Requirements The performance tools (5722PT1) to start performance explorer

Steps1. Start Performance Explorer using the following definitions:

ADDPEXDFN DFN(MYDFN) TYPE(*TRACE) JOB(IXPURGE*) TASK(*ALL) TRCTYPE(*SLTEVT) SLTEVT(*YES) JRNEVT ((*STROBJFORCE) (*ENDOBJFORCE))

2. Run the workload to examine3. Dump the statistics to a library using the ENDPEX CL command.


4. Query the Database file with the following query:

CREATE VIEW LIB/STARTS (STASK, SSUM, SFORCES) AS SELECT QTSNM, SUM(QTITIMN), count(*) FROM LIB/QAYPETASKI, LIB/QAYPETIDX WHERE QAYPETASKI.QTSTCT = QAYPETIDX.QTIFTC AND QAYPETIDX.QTITY = 20 AND QAYPETIDX.QTISTY = 7

GROUP BY QTSNM

CREATE VIEW LIB/ENDS (ETASK, ESUM, EFORCES) AS SELECT QTSNM, SUM(QTITIMN), count(*) FROM LIB/QAYPETASKI, LIB/QAYPETIDX WHERE QAYPETASKI.QTSTCT = QAYPETIDX.QTIFTC AND QAYPETIDX.QTITY = 20 AND QAYPETIDX.QTISTY = 8

GROUP BY QTSNM

select sforces from LIB/STARTS

Journal Housekeeping (JORECRA Task)


Housekeeping 을 얼마나 자주 했는가 ?

Display Data Data width . . . . . . : 14 Position to line . . . . . Shift to column . . . . . . ....+....1....

SFORCES

38


Quantity of times the background sweeper(JORECRA)task woke up and decided to send some objects out to disk


소요되는 시간은 ?select (ends.esum - starts.ssum) / 1000000 as “Total Duration” from LIB/STARTS, LIB/ENDS

Display Data Data width . . . . . . : 26 Position to line . . . . . Shift to column . . . . . . ....+....1....+....2....+.

"Total Duration"

1,307


Elapsed time in Msec consumed performing these housekeeping chores


평균 시간은 ?

select ((ends.esum - starts.ssum) / 1000000 / starts.sforces) as “Average Length” from LIB/STARTS, LIB/ENDS

Display Data Data width . . . . . . : 26 Position to line . . . . . Shift to column . . . . . . ....+....1....+....2....+.

"Average Length"

34

******** End of data ********

Bottom F3=Exit F12=Cancel F19=Left F20=Right F21=Split

Average duration of each “sweep” in Msec


Journal Recovery Ratio 관리• Journal Recovery Ratio 를 어떻게 Tuning 할 것인가 ?

– Background SLIC tasks: JORECRA• Sweep through main memory writing changed pages of journaled objects

– Their mission is to cap the quantity of recovery time in the event of a crash

– The more journaled objects, the more aggressive these tasks become– They are driven by a "Recovery Ratio"

• R540 이전까진 default ratio 는 50K , 현재는 250K• Jrn Entry 가 쌓여 50K/Numb_Objs 가 될 때 마다 해당 Task 수행• Every extra 10K added to the ratio increases abnormal IPL time by less than 30 se

conds

– The higher the ratio, the less frequently journaled objects get written and the better the run time performance vs. longer potential IPL

• It's similar to a default "Force Write Ratio" for journaled files


Journal Recovery Ratio 조정방법• For R520 API: Call QJOCHRVC PARM(1000000)

– Increases recovery ratio to 1 Million (reduces background load by 95%)– Legal range is 10K to 2 Billion

• For R530 Call PGM(QJOCHRVC)

PARM( X'000F4240' X'00000000000000000000000000')

X’000F4240’ = 1000000 in decimal

• For R540– Journal 별로 RECRA (Jrn Rcvy Ratio/sweep) 값을 조절

• CHGJRN . . . JNRCYCNT(250000)– May want to reduce sweep frequency in high volume environments to achieve best Perf

ormance

– May also be attractive tuning lever on the Target side


IPL Duration Runtime Overhead

BACK

어떤 것이 현명한 선택인가 ?


Receiver

Journal

Source System

SQL Table

Remote Receiver

Target System

SLIC Async Sending

Task

Async Remote

Journaling

Applications

Your application does NOT

wait

Q10. Remote Journal Sending Task 잘 관리하기


´ À̧ ° Å ë½ ÅLine¶ §¹ ®¿ ¡ SourceÂ Ê¿ ¡¼ º ³̧ »Á ö ¸ øÇ Ï° í ½ ×¿ ©À Ö´ Â ³ ðµ é ¶ «¿ ¡ ° ÆÁ ¤À Ì± º...¹ æ¹ ýÀ Ì ¾ øÀ »± î?

어떻게 하면 효율적으로 보낼 수 있을까 ?


어떻게 하면 효율적으로 보낼 수 있을까 ?

• Async RJ Super Bundling 을 사용– Async Mode 를 사용할 경우 , Source 쪽에서 Super-Size Bundle 을 생성– 특히 , 느린 Communication Line 을 통한 대용량 batch activity 에 매우 효율적

Receiver

Journal

Source System

SQL Table

SQL Table

BP Software

Remote Receiver

Target System

Harvest AsyncSuper

Bundling

Async Remote

Journaling

Applications

Machine Interface

SQL Table

Standard template for internal and external presentations Journal on i/OS

Documents

Transcript of Standard template for internal and external presentations Journal on i/OS