Express Reorg: How to save your weekend Laura Rochon Ajilon Consulting.

Express Reorg:Express Reorg:How to save your weekendHow to save your weekend

Laura Rochon

Ajilon Consulting

Laura RochonLaura Rochon

Started working with CA-IDMS in 1983Joined Cullinet in 1985Left CA in 1992, has been consulting ever

sinceJoined Ajilon Consulting in January 2005IUA Board member since 1998CQUI President

AgendaAgenda

Overview of Unload/ReloadOverview of Express Reorg Express Reorg ExecutionConsiderationsTest resultsConclusion

Overview of Unload/ReloadOverview of Unload/Reload

Unload Reload

DB DB

Overview of Unload/ReloadOverview of Unload/Reload

Unload SORT1 DBL2

SORT2 DBLX

SORT3 DBL3 SORT4 DBL4

ReloadUnload

DB DB

Overview of Express ReorgOverview of Express Reorg

DBDB

Unload Reload

DBDB

DB

Unload

Unload

Reload

Reload

Controlfile


New functionality: – Add or remove fields from index key– Change order of set


4 major phases:– SETUP

Allocates control file Specifies what is to be done

– UNLOAD Unloads data

– RELOAD Reloads data

– CLEANUP Delete work files (but not control file) Unlock areas


Unload:– Area sweep for most records, except:

Record stored VIA system-owned index VIA record CALC records with duplicate option


Unload (cont’d)– Records assigned target page

CALC records VIA records Direct records


REORG slicing:– Based on DIVIDE PROCESSING n WAYS– Generally, each area is divided symmetrically– If total size of all areas is less than 200 pages, 1

slice– Areas and subareas that contain only system

indexes not divided and assigned to a slice


REORG slicing (cont’d)– User-owned indexes are processed with the

slice of the owner record– System-owned indexes assigned to index group

and processed indepently of slices


Reorg processing is divided into tasks and grouped into phases

RELOAD phase is divided into– Reload phases 1 thru 6

Each reload task in a given task must successfully complete before processing can move to next Reload phase

– Rebuild phases 1 thru 4


UnloadRebuild

1

REORG:

Reload 1

Reload 2

Reload 3

Reload 4

Reload 5

Reload 6

Rebuild 2

Rebuild 3

Rebuild 4

Cleanup

ReloadUnload Cleanup

DB

DB

DB

DB

DB

Express Reorg ExecutionExpress Reorg Execution

ParametersJCLWork FilesOutputProcessingJob submission

Express Reorg ParametersExpress Reorg Parameters

SEGMENT source-segment USING source-ssc AREA area

RELOAD INTO target-db USING target-ssc DMCL dmcl-name

DIVIDE PROCESSING n WAYS

REUSE WORKFILES

AS SORTEXIT

SHARE

NOTIFY notify-interval

REORG setup options:


REORG execution options:

CLEANUP

STOP AFTER SETUP

UNLOAD

RELOAD

NO

CREATE UNLOAD WORKFILES

RELOAD

ALL

SUBMIT


REORG other parameters:

CLEANUP

STATUS REPORT ONLY

Express Reorg JCLExpress Reorg JCL

//BCF EXEC PGM=IDMSBCF,REGION=0M//STEPLIB DD DSN=idms.loadlib,DISP=SHR//SYSIDMS DD *Sysidms parameters//SYSLST DD SYSOUT=*//SYSPRINT DD SYSOUT=*//SORTMSG DD SYSOUT=*//SORTWRKnn DD UNIT=unit,SPACE=(CYL,(mmm,nnn))//SYSIPT DD *Reorg parameters//RORGCTL DD DSN=rorgctl-file,DISP=SHR//RORGJCL DD DSN=jcl-file,DISP=SHR


RORGCTL file:– Describes the specifics of the REORG operation– Contains status information– Fixed-length, LRECL=4096– Initialized during setup – Should not have DISP=OLD or DISP=NEW


RORGJCL file:– Contains JCL for parallel jobs– Job name should be short enough to append 1+

characters– 1 step: IDMSBCF with following syntax:

REORG;

– Should have DB files and RORGCTL file

Express Reorg Work FilesExpress Reorg Work Files

The number of work files proportional to number of jobs submitted

Work files can be manually allocated Work files allocated automatically by using

new command CREATE DSMODELCan specify several CREATE DSMODEL


REORG uses following DDNAME prefixes:– WU -- unload/reload work files– WI -- Index Rebuild work files– WS -- Sort output work files– WD -- DBKEYS work files


CREATE DSMODEL creates a temporary model of data set attributes

Must include information :– Where to create work file– Space – Blocksize

CREATE DSMODELCREATE DSMODEL


Sizing work files:– Very hard to estimate– Run REORG STOP AFTER UNLOAD with

dummied work files.


Number of files grows exponentially: example of WU* files:

Divide n ways No reuse Reuse workfiles

1 11 9

2 32 24

3 63 45

4 104 72

5 155 105

OutputOutput

Express Reorg ProcessingExpress Reorg Processing

If Target files same as Source files– REORG STOP AFTER UNLOAD– If not SHARE, UNLOCK AREAS– FORMAT files– REORG with STOP AFTER CLEANUP &

SUBMIT– Backup– IDMSDBAN (optionally)– PRINT SPACE (optionally)


If Target files with different DDnames– Format new files– REORG with STOP AFTER CLEANUP– Backup– PRINT SPACE – IDMSDBAN (optionally)


If Target files different but same DDnames– Format new files– REORG with STOP AFTER UNLOAD

(REORG jcl and jcl in RORGJCL have old DDnames)

– REORG with STOP AFTER CLEANUP and SUBMIT(REORG jcl and jcl in RORGJCL have new files)

Job SubmissionJob Submission

Automatic job submissionManual job submissionRestarting REORG

Automatic Job SubmissionAutomatic Job Submission

Jobs automatically submitted – at end of SETUP phase– If SUBMIT parameter is specified

REORG will submit– 1 job for each slice– 1 job for each index group

Maximum jobs submitted = 2x no slices

Manual Job SubmissionManual Job Submission

Can manually submit REORG job(s)Job will examine control file and determine

if there is work to be doneIf no work, waits until some other job

completes a taskIf work is needed, it performs that work

Restarting REORGRestarting REORG

If a REORG job fails, other jobs will try to restart the task that failed

If problem is not temporary (work file too small), problem needs to be corrected, then REORG can be submitted with SUBMIT.

In some severe cases, you might need to restart REORG from the beginning

ConsiderationsConsiderations

Relative REORG performanceEfficiency of the reorganized databaseWork files

Relative REORG performanceRelative REORG performance

Usually runs in less time than normal UNLOAD/RELOAD, however…..

Best to try it out first

DB EfficiencyDB Efficiency

DB might not be as efficient as reorganized by UNLOAD/RELOAD

REORG reloads in forwards fashion whereas UNLOAD/RELOAD reloads in backwards fashion

REORG can result in more CALC & VIA overflow especially if long sets

Work FilesWork Files

Work files HAVE to be on diskHard to estimate their size

Tests ResultsTests Results

19.88

47.83

9.1

24.44

8.38

19.78

6.62

17.85

8.1

15.94

0

10

20

30

40

50

60

70

1 2 3 4 5

67.71

33.5428.16

24.47 24.04

Unload/reload

Reorg

2 ways

Reorg

3 ways

Reorg

4 ways

Reorg

5 ways

50.5%58.4%

63.8% 64.5%

TSTDML (46,116 pages)


16.74

24.16

10.46

19.32

7.25

15.64

0

5

10

15

20

25

30

35

40

45

1 2 3

40.9

29.78

22.89

Unload/reload

Reorg

2 ways

Reorg

3 ways

27.2%

44.0%

1 Area unloaded (120,000 pages going to 147,000 pages) 1 Area affected (45,000 pages)


59.02

109.25

40.17

49.14

14.15

66.08

0

20

40

60

80

100

120

140

160

180

1 2 3

1 Area unloaded (427,720 pages @5064 going to 428,400 pages @ 7548)3 Areas affected (39,000 pages 160,000 pages 294,000 pages)

168.27

89.31 80.23

46.9%51.1%

Unload/reload

Reorg

3 ways

Reorg

5 ways

ConclusionConclusion

Can reduce the time to reorganize a database up to 50-60%

Easy to useNeeds lots of disk space

Express Reorg: How to save your weekend Laura Rochon Ajilon Consulting.

Documents

Transcript of Express Reorg: How to save your weekend Laura Rochon Ajilon Consulting.