Using Release(deallocate) and Painful Lessons to be learned on DB2 locking

of 24 /24
#IDUG Using RELEASE(DEALLOCATE) and Painful Lessons to be Learned on DB2 Locking John Campbell DB2 for z/OS Development Session Code: 6007

Embed Size (px)

Transcript of Using Release(deallocate) and Painful Lessons to be learned on DB2 locking

  • #IDUG

    Using RELEASE(DEALLOCATE)and Painful Lessons to be Learned on DB2 LockingJohn CampbellDB2 for z/OS DevelopmentSession Code: 6007

  • #IDUG

    2

    Agenda Thread reuse using RELEASE(DEALLOCATE)

    Primer on thread reuse with RELEASE bind option Considerations and limitations

    Painful Lessons to be Learned on DB2 Locking Primer on lock avoidance CURRENTDATA(YES) versus CURRENTDATA(NO) Ramifications of lock avoidance for SQL What should an application programmer do Finding packages to safely rebind with CURRENTDATA(NO)

  • #IDUG

    3

    Resources: Static SQL Static statements

    Packages and statements Parent locks Index look-aside buffer, dynamic prefetch tracking via sequential detection

  • #IDUG

    4

    RELEASE - BIND and REBIND Option Determines when to release the resources that a program uses

    RELEASE(COMMIT) - Releases resources at commit point RELEASE(DEALLOCATE) - Releases resources when thread terminates (child

    page/row locks still released at commit) RELEASE(INHERITFROMPLAN) Support added by APAR PM07087

    Default behavior BIND PLAN - COMMIT BIND PACKAGE - plan value REBIND PLAN/PACKAGE existing value DB2Binder Utility for JDBC and SQLJ

    DEALLOCATE is default in DB2 10 COMMIT is default in DB2 9 and earlier releases

  • #IDUG

    Package Allocation

    Program A :Insert into table 1

    Thread allocationSign on, authorization checkingLocate SKCT and SKPTAllocate CT/PT and STMT1Start Insert STMT1

    Lock table space 1Lock data page

    End STMT1

    insert into table 2Allocate STMT2Start Insert STMT2

    Lock table space 2Lock data page

    End STMT2COMMIT ;

    Program A

    Stmt 1: Insert into table1Stmt 2: Insert into table2

    SPT01

    EDM_SKELTON_POOLSKCT/SKPT

    V10 Thread storage above the barV9 EDM thread pool below the bar

    CTPTSQL Stmt1SQL Stmt2

    I/O

  • #IDUG

    6

    Package Allocation and Commit

    Program A

    Stmt 1: Insert into table1Stmt 2: Insert into table2Commit ;

    SPT01

    EDM_SKELTON_POOLSKCT/SKPT

    V10 Thread storage above the barV9 EDM thread pool below the bar

    CTPTSQL Stmt1SQL Stmt2

    I/O

    Expensive Operation!

    Lock TS 1Lock table1 data page

    Lock TS 2Lock table2 data page

    RELEASE PT, statementsUnlock data pagesUnlock TS1, TS2

    COMMIT

  • #IDUG

    7

    Thread Reuse and RELEASE(DEALLOCATE) Thread reuse eliminates CPU cost of DB2 thread allocation and deallocation

    CICS Protected ENTRYs Organic reuse of ENTRYs (or POOL)

    IMS/TM Fast Path (IFP) regions Wait-For-Input (WFI) regions Pseudo Wait-For-Input (PWFI) regions

    DDF High Performance DBATs

    WebSphere Type 2 local connections Batch with intermediate commits

    Use of RELEASE(DEALLOCATE) coupled with effective thread reuse i.e., threadpersistence Further reduces the CPU cost with potential for significant savings (up to 10% plus) Resources are held until thread deallocation Without thread reuse RELEASE(COMMIT) vs. RELEASE(DEALLOCATE) is a moot point for

    discussion

  • #IDUG

    8

    Limitations and Considerations Virtual and real storage

    DB2 9 and earlier - DBM1 virtual storage below the 2G bar and real storage Package information is stored in EDM pool below the bar

    DB2 10 after REBIND - real storage usage Package information is stored in thread storage above the bar in DB2 9 and earlier

    releases Accumulated DB2 object control blocks

    Virtual, real, potentially CPU cost for scanning the objects built up under the thread Recommendations

    Design for thread reuse for High volume simple transactions Complex transactions at a reasonably high rate

    Selectively use RELEASE(DEALLOCATE) on high use packages use % of Total Acctg Class7 CPU

    DBM1 31-bit virtual storage constraint (DB2 9) Real storage constraint (DB2 10)

    Use CICS or WebSphere parameter to periodically clean up and rejuvenate threads(thread deallocation)

  • #IDUG

    9

    Considerations for Clean Up REUSELIMIT (0-10,000) in CICS TS V4R2 - default 1000

    Number of times a thread can be reused before it is terminated Use default and monitor DB2 storage usage and adjust the number if needed

    WebSphere Type 2 connection Aged Timeout - default 0 The interval in seconds before a physical connection is discarded Consider setting WAS aged timeout to less than 5 min, recommend using 120

    secs to reduce exposure of long living threads

    DB2 10 High Performance DBATs (threads) Thread will go inactive every 200 commits No user control for this value

    DB2 11 optimizes RELEASE(DEALLOCATE) execution so that it is consistentlybetter performing than RELEASE(COMMIT)

  • #IDUG

    10

    Considerations - Concurrency More persistent threads with RELEASE(DEALLOCATE) is also trade off with

    concurrency BIND/REBIND SQL DDL Online REORG which invalidates packages

    For RELEASE(DEALLOCATE) some locks are held beyond commit until threadtermination Mass delete locks (SQL DELETE without WHERE clause) Gross level lock acquired on behalf of a SQL LOCK TABLE Table space defined with LOCKSIZE TABLESPACE|TABLE Note: no longer a problem for gross level lock acquired by lock escalation

    DO YOUR HOMEWORK BEFORE USING PERSISTENT THREADS WITHRELEASE(DEALLOCATE) BIND OPTION

  • #IDUG

    11

    Primer on lock avoidance Combination of techniques used by DB2 to try to avoid taking a S page/row

    locks when processing for read only SQL whilst preventing the retrieval ofuncommitted data by the application (1) Page latching (and page p-lock in data sharing) controlled by DB2 to ensure

    physical consistency of the page (2) Commit log sequence number (CLSN) at the page level

    DB2 tracks "time" of last update to page (on page) (A) DB2 tracks "time" of oldest uncommitted activity on every pageset/partition (B)

    Non Data Sharing CLSN = lowest uncommitted RBA for all active transactions for a given pageset

    Data Sharing For non-GBP-dependent page sets, each member uses a local CLSN = lowest uncommitted LRSN for all active

    transactions for a given pageset For GBP-dependent page sets, a Global CLSN value is maintained for the entire data sharing group = lowest CLSN value

    across all members across all page sets (GBP-dependent or not)

    If (A) < (B) everything on the page is guaranteed to be committed Else, check Possibly UNCommitted bits (PUNC bits)

  • #IDUG

    12

    Primer on lock avoidance Combination of techniques to prevent retrieval of uncommitted data

    (3) Possibly UNCommitted bits (PUNC bits) at the row level On each row, a PUNC bit is set when the row is updated

    PUNC bits are periodically reset If successful CLSN check and more than 25% of the rows have the PUNC bit ON RR scanner REORG TABLESPACE

    If the PUNC bit is not ON, the row/key is guaranteed to be committed

  • #IDUG

    13

    CURRENTDATA(YES) versus CURRENTDATA(NO) Plans and packages have a better chance for lock avoidance if they are bound

    with ISOLATION(CS) and CURRENTDATA(NO) What is CURRENTDATA?

    Helps to determine the currency and stability of data returned to an applicationcursor

    Only applies to applications bound with ISOLATION(CS) What is isolation CURSOR STABILITY(CS)?

    Data fetched by a cursor is committed, but if the application process returns to thesame page, another application might have since updated, deleted, or insertedqualifying rows

    If the cursor is defined as FOR UPDATE OF Data returned by the cursor is stable and it may not be updated by another

    transaction while the updatable cursor is positioned on it If the cursor is defined as FOR READ|FETCH ONLY, or it is implicitly read only (or it is

    ambiguous) ISOLATION(CS) ensures that the data returned is committed and the stability of

    the cursor is determined by the CURRENTDATA option

  • #IDUG

    14

    Primer on lock avoidance Benefits of lock avoidance

    Improved concurrency through less lock collisions Decrease in lock and unlock activity requests, with an associated decrease in CPU

    resource consumption and data sharing overhead V8 improvements

    Lock avoidance for non-cursor singleton SELECT In V7, ISOLATION(CS) CURRENTDATA(YES) acquires S page/row lock on the

    qualified row In V8, DB2 no longer acquires and hold S page/row lock on the qualified row for

    ISOLATION(CS) CURRENTDATA(YES or NO) Overflow lock avoidance when the update of a variable length row in a data page

    results in a new row that cannot fit in that page i.e., indirect reference In V7, no lock avoidance on both pointer and overflow In V8, lock on pointer only

    Need to distinguish carefully between eligibility for lock avoidance andactually achieving it

  • #IDUG

    15

    Primer on lock avoidance

    BIND option ISOLATION(CS) with CURRENTDATA(NO) couldreduce # Lock/Unlock requests dramatically

    High Unlock requests/commit could also be possible from Large number of relocated rows after update of compressed or VL

    row Lock/Unlock of pointer record (or page)

    Large number of pseudo-deleted entries in unique indexes Lock/Unlock of data (page or row) in insert to unique index when

    pseudo-deleted entries exist Both can be eliminated by REORG

    Field Name Description

    QTXALOCK LOCK REQUESTS

    QTXAUNLK UNLOCK REQUESTS

    LOCKING ACTIVITY QUANTITY /SECOND /THREAD /COMMIT------------------------ -------- ------- ------- -------...LOCK REQUESTS 521.0M 24.2K 3134.34 1050.75UNLOCK REQUESTS 478.1M 22.2K 2876.06 964.16

    Lock avoidance may not be working effectively ifUnlock requests/commit is high, e.g. >5

    ROT

  • #IDUG

    16

    Primer on lock avoidance Effective lock avoidance is very important in data sharing

    Global locks propagated beyond IRLM to Coupling Facility are relatively expensive Global management of lock contention/resolution is very expensive Effective Lock avoidance is critical to achieving good performance and lower data

    sharing overhead Any long-running UR(s) can reduce the effectiveness of lock avoidance by stopping

    the Global CLSN value from moving forward Recommendation: Aggressively monitor long-running URs

    'First cut' ROTs: URs running for a long time without committing: zparm URCHKTH

  • #IDUG

    17

    CURRENTDATA(YES) versus CURRENTDATA(NO) Switching from CURRENTDATA(YES) to CURRENTDATA(NO)

    In general, can be done without any adverse effect to applications But consider the following scenario:

    1. Program A bound with ISOLATION(CS) CURRENTDATA(YES) fetches a row froma cursor with ORDER BY and the row is read from the base table and not froma work file

    2. Program B tries to update the row just read by Program A The access is not allowed because the CURRENTDATA(YES) option for Program A caused a lock to

    be acquired on the page/row in the base table

    3. Program A then issues a searched UPDATE of the row just fetched from step 1. In the above example CURRENTDATA(YES) protects the integrity of the data read by

    Program A in between the FETCH and the searched UPDATE If Program A was to be rebound with CURRENTDATA(NO) then

    1. Program B may (should) be able to access the row just fetched by Program A This assumes correct timing and that lock avoidance is effective

    2. When Program A issues the searched UPDATE, it could wipe out the changesjust made by Program B

  • #IDUG

    18

    CURRENTDATA(YES) versus CURRENTDATA(NO) Switching from CURRENTDATA(YES) to CURRENTDATA(NO)

    Now reconsider the same scenario:1. Program A bound with ISOLATION(CS) CURRENTDATA(YES) fetches a row from

    a cursor with ORDER BY, but the row is now read from a work file2. Program B tries to update the row just read by Program A

    1. Access is now allowed as there is no lock on the page/row in the base table

    3. Program A then issues a searched UPDATE of the row just fetched from step 1 In the above example CURRENTDATA(YES) does NOT protects the integrity of the

    data read by Program A in between the FETCH and the searched UPDATE

  • #IDUG

    19

    Ramifications of lock avoidance for SQL Data may be returned without a lock

    All data returned will be committed at the time it is taken from page But may change by the time the application sees it

    DB2 only attempts lock avoidance for ISOLATION(CS) transactions when BOTHof the following are true: The plan/package is bound CURRENTDATA(NO) The cursor (if cursor-based FETCH) is declared FOR READ|FETCH ONLY, or has some

    other attribute (e.g., ORDER BY) that makes it a read-only cursor Applications that depend on the stability of what is under a read only cursor,

    should use CURRENTDATA(YES) Default is CURRENTDATA(NO)

    Now lets look at another example

  • #IDUG

    20

    Ramifications of lock avoidance for SQL Example

    DECLARE CURSOR AS ...SELECT THIRD_COL ... WHERE CLUST_COL = 'ABC'

    AND ANOTHER_COL = 'DEF'ORDER BY CLUST_COL;

  • #IDUG

    21

    What should an application programmer do Use programming techniques which avoid possible data currency exposures and access path

    dependencies (these sometimes occur regardless of the CURRENTDATA option) Define cursors with their intended use

    Easier to read application code Number of ambiguous cursors can be reduced Some queries will not allow the use of a FOR UPDATE cursor due to possible work file usage e.g.,

    with ORDER BY Use predicates on searched updates to enforce data currency

    When a FOR UPDATE cursor cannot be used with a query it is common for applicationprogrammers to use a read only with a subsequent searched UPDATE or DELETE

    It is important to use WHERE predicates to ensure that data has not changed since your cursorfirst selected the data

    Include all columns which logically determine if the update is necessary, instead of updatingbased solely on the key

    This is an exposure regardless of the CURRENTDATA option, but the use of CURRENTDATA(NO)can increase the exposure opportunity

    Consider the use of a timestamp or a version number Add a timestamp or version number column to a table to record the last update Select the timestamp or version number column with the cursor Use it as a predicate on the searched UPDATE or DELETE to ensure that an application is

    updating the same row

  • #IDUG

    22

    Finding packages to safely rebind with CURRENTDATA(NO) Must recognise that access path dependencies will inevitably occur CURRENTDATA(YES) will not stop access path dependency and associated

    exposures Access path independence and CURRENTDATA(NO) should be the goal for all

    applications Can safely rebind packages that use only true read-only cursors with

    CURRENTDATA(NO)

  • #IDUG

    23

  • #IDUG

    John CampbellDB2 for z/OS [email protected]

    Session 6007Using RELEASE(DEALLOCATE) and Painful Lessons to be Learned on DB2 Locking