Using Release(deallocate) and Painful Lessons to be learned on DB2 locking

#IDUG

Using RELEASE(DEALLOCATE)and Painful Lessons to be Learned on DB2 Locking

John CampbellDB2 for z/OS Development

Session Code: 6007

#IDUG

2

Agenda

• Thread reuse using RELEASE(DEALLOCATE)

• Primer on thread reuse with RELEASE bind option

• Considerations and limitations

• Painful Lessons to be Learned on DB2 Locking

• Primer on lock avoidance

• CURRENTDATA(YES) versus CURRENTDATA(NO)

• Ramifications of lock avoidance for SQL

• What should an application programmer do

• Finding packages to safely rebind with CURRENTDATA(NO)

#IDUG

3

“Resources”: Static SQL

• Static statements

• Packages and statements

• Parent locks

• Index look-aside buffer, dynamic prefetch tracking via sequential detection

#IDUG

4

RELEASE - BIND and REBIND Option

• Determines when to release the resources that a program uses

• RELEASE(COMMIT) - Releases resources at commit point

• RELEASE(DEALLOCATE) - Releases resources when thread terminates (childpage/row locks still released at commit)

• RELEASE(INHERITFROMPLAN) – Support added by APAR PM07087

• Default behavior

• BIND PLAN - COMMIT

• BIND PACKAGE - plan value

• REBIND PLAN/PACKAGE existing value

• DB2Binder Utility for JDBC and SQLJ

• DEALLOCATE is default in DB2 10

• COMMIT is default in DB2 9 and earlier releases

#IDUG

Package Allocation

Program A :“Insert into table 1”

Thread allocationSign on, authorization checkingLocate SKCT and SKPTAllocate CT/PT and STMT1Start Insert STMT1

Lock table space 1Lock data page

End STMT1

“insert into table 2”Allocate STMT2Start Insert STMT2

Lock table space 2Lock data page

End STMT2COMMIT ;

Program A

Stmt 1: Insert into table1Stmt 2: Insert into table2

SPT01

EDM_SKELTON_POOLSKCT/SKPT

V10 Thread storage above the barV9 EDM thread pool below the bar

CTPTSQL Stmt1SQL Stmt2

I/O

#IDUG

6

Package Allocation and Commit

Program A

Stmt 1: Insert into table1Stmt 2: Insert into table2Commit ;

SPT01

EDM_SKELTON_POOLSKCT/SKPT

V10 Thread storage above the barV9 EDM thread pool below the bar

CTPTSQL Stmt1SQL Stmt2

I/O

Expensive Operation!

Lock TS 1Lock table1 data page

Lock TS 2Lock table2 data page

RELEASE PT, statementsUnlock data pagesUnlock TS1, TS2

COMMIT

#IDUG

7

Thread Reuse and RELEASE(DEALLOCATE)

• Thread reuse eliminates CPU cost of DB2 thread allocation and deallocation• CICS

• Protected ENTRYs

• Organic reuse of ENTRYs (or POOL)

• IMS/TM

• Fast Path (IFP) regions

• Wait-For-Input (WFI) regions

• Pseudo Wait-For-Input (PWFI) regions

• DDF

• High Performance DBATs

• WebSphere Type 2 local connections

• Batch with intermediate commits

• Use of RELEASE(DEALLOCATE) coupled with effective thread reuse i.e., threadpersistence• Further reduces the CPU cost with potential for significant savings (up to 10% plus)

• Resources are held until thread deallocation

• Without thread reuse RELEASE(COMMIT) vs. RELEASE(DEALLOCATE) is a moot point fordiscussion

#IDUG

8

Limitations and Considerations

• Virtual and real storage• DB2 9 and earlier - DBM1 virtual storage below the 2G bar and real storage

• Package information is stored in EDM pool below the bar

• DB2 10 after REBIND - real storage usage

• Package information is stored in thread storage above the bar in DB2 9 and earlierreleases

• Accumulated DB2 object control blocks

• Virtual, real, potentially CPU cost for scanning the objects built up under the thread

• Recommendations• Design for thread reuse for

• High volume simple transactions

• Complex transactions at a reasonably high rate

• Selectively use RELEASE(DEALLOCATE) on high use packages – use % of Total Acctg Class7 CPU

• DBM1 31-bit virtual storage constraint (DB2 9)

• Real storage constraint (DB2 10)

• Use CICS or WebSphere parameter to periodically clean up and rejuvenate threads(thread deallocation)

#IDUG

9

Considerations for Clean Up

• REUSELIMIT (0-10,000) in CICS TS V4R2 - default 1000• Number of times a thread can be reused before it is terminated• Use default and monitor DB2 storage usage and adjust the number if needed

• WebSphere Type 2 connection Aged Timeout - default 0

• The interval in seconds before a physical connection is discarded

• Consider setting WAS “aged timeout” to less than 5 min, recommend using 120

secs to reduce exposure of long living threads

• DB2 10 High Performance DBATs (threads)• Thread will go inactive every 200 commits• No user control for this value

• DB2 11 optimizes RELEASE(DEALLOCATE) execution so that it is consistentlybetter performing than RELEASE(COMMIT)

#IDUG

10

Considerations - Concurrency

• More persistent threads with RELEASE(DEALLOCATE) is also trade off withconcurrency

• BIND/REBIND

• SQL DDL

• Online REORG which invalidates packages

• For RELEASE(DEALLOCATE) some locks are held beyond commit until threadtermination

• Mass delete locks (SQL DELETE without WHERE clause)

• Gross level lock acquired on behalf of a SQL LOCK TABLE

• Table space defined with LOCKSIZE TABLESPACE|TABLE

• Note: no longer a problem for gross level lock acquired by lock escalation

• DO YOUR HOMEWORK BEFORE USING PERSISTENT THREADS WITHRELEASE(DEALLOCATE) BIND OPTION

#IDUG

11

Primer on lock avoidance

• Combination of techniques used by DB2 to try to avoid taking a S page/rowlocks when processing for read only SQL whilst preventing the retrieval ofuncommitted data by the application

• (1) Page latching (and page p-lock in data sharing) controlled by DB2 to ensurephysical consistency of the page

• (2) Commit log sequence number (CLSN) – at the page level

• DB2 tracks "time" of last update to page (on page) (A)

• DB2 tracks "time" of oldest uncommitted activity on every pageset/partition (B)• Non Data Sharing

• CLSN = lowest uncommitted RBA for all active transactions for a given pageset

• Data Sharing• For non-GBP-dependent page sets, each member uses a local CLSN = lowest uncommitted LRSN for all active

transactions for a given pageset

• For GBP-dependent page sets, a Global CLSN value is maintained for the entire data sharing group = lowest CLSN valueacross all members across all page sets (GBP-dependent or not)

• If (A) < (B) everything on the page is guaranteed to be committed• Else, check Possibly UNCommitted bits (PUNC bits)

#IDUG

12

Primer on lock avoidance …

• Combination of techniques to prevent retrieval of uncommitted data …

• (3) Possibly UNCommitted bits (PUNC bits) – at the row level

• On each row, a PUNC bit is set when the row is updated• PUNC bits are periodically reset

• If successful CLSN check and more than 25% of the rows have the PUNC bit ON

• RR scanner

• REORG TABLESPACE

• If the PUNC bit is not ON, the row/key is guaranteed to be committed

#IDUG

13

CURRENTDATA(YES) versus CURRENTDATA(NO) …

• Plans and packages have a better chance for lock avoidance if they are boundwith ISOLATION(CS) and CURRENTDATA(NO)

• What is CURRENTDATA?

• Helps to determine the currency and stability of data returned to an applicationcursor

• Only applies to applications bound with ISOLATION(CS)

• What is isolation CURSOR STABILITY(CS)?

• Data fetched by a cursor is committed, but if the application process returns to thesame page, another application might have since updated, deleted, or insertedqualifying rows

• If the cursor is defined as FOR UPDATE OF

• Data returned by the cursor is stable and it may not be updated by anothertransaction while the updatable cursor is positioned on it

• If the cursor is defined as FOR READ|FETCH ONLY, or it is implicitly read only (or it isambiguous)

• ISOLATION(CS) ensures that the data returned is committed and the stability ofthe cursor is determined by the CURRENTDATA option

#IDUG

14


• Benefits of lock avoidance

• Improved concurrency through less lock collisions

• Decrease in lock and unlock activity requests, with an associated decrease in CPUresource consumption and data sharing overhead

• V8 improvements

• Lock avoidance for non-cursor ‘singleton’ SELECT

• In V7, ISOLATION(CS) CURRENTDATA(YES) acquires S page/row lock on thequalified row

• In V8, DB2 no longer acquires and hold S page/row lock on the qualified row forISOLATION(CS) CURRENTDATA(YES or NO)

• Overflow lock avoidance when the update of a variable length row in a data pageresults in a new row that cannot fit in that page i.e., indirect reference

• In V7, no lock avoidance on both pointer and overflow

• In V8, lock on pointer only

• Need to distinguish carefully between eligibility for lock avoidance andactually achieving it

#IDUG

15


• BIND option ISOLATION(CS) with CURRENTDATA(NO) couldreduce # Lock/Unlock requests dramatically

• High Unlock requests/commit could also be possible from

• Large number of relocated rows after update of compressed or VLrow

• Lock/Unlock of pointer record (or page)

• Large number of pseudo-deleted entries in unique indexes

• Lock/Unlock of data (page or row) in insert to unique index whenpseudo-deleted entries exist

• Both can be eliminated by REORG

Field Name Description

QTXALOCK LOCK REQUESTS

QTXAUNLK UNLOCK REQUESTS

LOCKING ACTIVITY QUANTITY /SECOND /THREAD /COMMIT------------------------ -------- ------- ------- -------...LOCK REQUESTS 521.0M 24.2K 3134.34 1050.75UNLOCK REQUESTS 478.1M 22.2K 2876.06 964.16

Lock avoidance may not be working effectively ifUnlock requests/commit is high, e.g. >5

ROT

#IDUG

16


• Effective lock avoidance is very important in data sharing• Global locks propagated beyond IRLM to Coupling Facility are relatively expensive

• Global management of lock contention/resolution is very expensive

• Effective Lock avoidance is critical to achieving good performance and lower datasharing overhead

• Any long-running UR(s) can reduce the effectiveness of lock avoidance by stoppingthe Global CLSN value from moving forward

• Recommendation: Aggressively monitor long-running URs• 'First cut' ROTs:

• URs running for a long time without committing: zparm URCHKTH<=5• Message DSNR035I

• URs performing massive update activity without committing: zparmURLGWTH=10(K)

• Message DSNJ031I

• Need management ownership and process for getting rogue applications fixed upso that they commit frequently based on

• Elapsed time and/or

• CPU time (no. of SQL update statements)

#IDUG

17


• Switching from CURRENTDATA(YES) to CURRENTDATA(NO)

• In general, can be done without any adverse effect to applications

• But consider the following scenario:

1. Program A bound with ISOLATION(CS) CURRENTDATA(YES) fetches a row froma cursor with ORDER BY – and the row is read from the base table and not froma work file

2. Program B tries to update the row just read by Program A• The access is not allowed because the CURRENTDATA(YES) option for Program A caused a lock to

be acquired on the page/row in the base table

3. Program A then issues a searched UPDATE of the row just fetched from step 1.

• In the above example CURRENTDATA(YES) protects the integrity of the data read byProgram A in between the FETCH and the searched UPDATE

• If Program A was to be rebound with CURRENTDATA(NO) then

1. Program B may (should) be able to access the row just fetched by Program A• This assumes correct timing and that lock avoidance is effective

2. When Program A issues the searched UPDATE, it could wipe out the changesjust made by Program B

#IDUG

18


• Switching from CURRENTDATA(YES) to CURRENTDATA(NO) …

• Now reconsider the same scenario:

1. Program A bound with ISOLATION(CS) CURRENTDATA(YES) fetches a row froma cursor with ORDER BY, but the row is now read from a work file

2. Program B tries to update the row just read by Program A1. Access is now allowed as there is no lock on the page/row in the base table

3. Program A then issues a searched UPDATE of the row just fetched from step 1

• In the above example CURRENTDATA(YES) does NOT protects the integrity of thedata read by Program A in between the FETCH and the searched UPDATE

#IDUG

19

Ramifications of lock avoidance for SQL

• Data may be returned without a lock

• All data returned will be committed at the time it is taken from page

• But may change by the time the application sees it

• DB2 only attempts lock avoidance for ISOLATION(CS) transactions when BOTHof the following are true:

• The plan/package is bound CURRENTDATA(NO)

• The cursor (if cursor-based FETCH) is declared FOR READ|FETCH ONLY, or has someother attribute (e.g., ORDER BY) that makes it a read-only cursor

• Applications that depend on the stability of what is under a read only cursor,should use CURRENTDATA(YES)

• Default is CURRENTDATA(NO)

• Now lets look at another example

#IDUG

20

Ramifications of lock avoidance for SQL …

• Example

DECLARE CURSOR AS ...

SELECT THIRD_COL ... WHERE CLUST_COL = 'ABC'

AND ANOTHER_COL = 'DEF'

ORDER BY CLUST_COL; <--- makes cursor read only

FETCH ... <--- say this returns row ABC.DEF.GHI without taking a lock

UPDATE SET … WHERE CLUST_COL = 'ABC'

AND ANOTHER_COL = 'DEF'

AND THIRD_COL = 'GHI‘

• What is the issue?

• Row may no longer be ABC.DEF.GHI by the time update executes

• Another transaction may have gotten in and updated it to ABC.DEF.XYZ

• So update will receive "row not found"

#IDUG

21

What should an application programmer do

• Use programming techniques which avoid possible data currency exposures and access pathdependencies (these sometimes occur regardless of the CURRENTDATA option)• Define cursors with their intended use

• Easier to read application code

• Number of ambiguous cursors can be reduced

• Some queries will not allow the use of a FOR UPDATE cursor due to possible work file usage e.g.,with ORDER BY

• Use predicates on searched updates to enforce data currency

• When a FOR UPDATE cursor cannot be used with a query it is common for applicationprogrammers to use a read only with a subsequent searched UPDATE or DELETE

• It is important to use WHERE predicates to ensure that data has not changed since your cursorfirst selected the data

• Include all columns which logically determine if the update is necessary, instead of updatingbased solely on the key

• This is an exposure regardless of the CURRENTDATA option, but the use of CURRENTDATA(NO)can increase the exposure opportunity

• Consider the use of a timestamp or a version number

• Add a timestamp or version number column to a table to record the last update

• Select the timestamp or version number column with the cursor

• Use it as a predicate on the searched UPDATE or DELETE to ensure that an application isupdating the same row

#IDUG

22

Finding packages to safely rebind with CURRENTDATA(NO)

• Must recognise that access path dependencies will inevitably occur

• CURRENTDATA(YES) will not stop access path dependency and associatedexposures

• Access path independence and CURRENTDATA(NO) should be the goal for allapplications

• Can safely rebind packages that use only true read-only cursors withCURRENTDATA(NO)

#IDUG

23

#IDUG

John CampbellDB2 for z/OS [email protected]

Session 6007Using RELEASE(DEALLOCATE) and Painful Lessons to be Learned on DB2 Locking

Using Release(deallocate) and Painful Lessons to be learned on DB2 locking

Data & Analytics

Transcript of Using Release(deallocate) and Painful Lessons to be learned on DB2 locking