Problems and Solutions with Adabas Replication and Mass Data
description
Transcript of Problems and Solutions with Adabas Replication and Mass Data
![Page 1: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/1.jpg)
Problems and Solutions withAdabas Replication and Mass Data
Dieter W. Storr – www.storrconsulting.com
9 May 2011
![Page 2: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/2.jpg)
04/22/23 | Software AG User Group 2011 | 2
Contents
• Mirroring and Replication
• Different Ways to Replicate
• Replication Scenario and Used Versions
• Replication Components and their Commands per Second
• Subscription DB: CPU Time With and Without Replication
• Batch Jobs: Duration With and Without Replication
• Buffers and Communication between Source DB and Reptor
• Challenges and Solutions
![Page 3: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/3.jpg)
04/22/23 | Software AG User Group 2011 | 3
Source: Bruno Bebert/DPA
![Page 4: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/4.jpg)
Mirroring and Replication
04/22/23 | Software AG User Group 2011 | 4
Functions Mirroring Replication
Copy tracks / blocks Yes No
Repeat transactions No Yes
Can target database be read during the copy/repeat process
No Yes
Will errors in tracks be copied? Yes No
Can data / fields be selected? No Yes
![Page 5: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/5.jpg)
04/22/23 | Software AG User Group 2011 | 5
Type of Replication – Mainframe to Adabas on Mainframe
![Page 6: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/6.jpg)
04/22/23 | Software AG User Group 2011 | 6
Type of Replication – Mainframe to Adabas on LUW
![Page 7: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/7.jpg)
04/22/23 | Software AG User Group 2011 | 7
Type of Replication – Mainframe to non-Adabas
![Page 8: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/8.jpg)
Replication Scenario – Enterprise Server to Windows
04/22/23 | Software AG User Group 2011 | 8
RPL V3.3.1 ADA V6.2.1
![Page 9: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/9.jpg)
04/22/23 | Software AG User Group 2011 | 9
Commands per Second During Replication
SOURCE DB8.1.3
----------------RPL Buffer
REPTOR8.1.3 / 3.2.1
--------------RPL Buffer
TARGET DB
6.1.8.03CSTZ007
SLOG
WCP6.2.1
WCP7.3.3
KZ007
1,388 cps 79.8 cps ? cps 76.5 cps
100 mbps line circuit
2 mbps
1,307 cps 519.5 cps ? cps 297- 417 cps6 mbps
BOE
DMH
DMV
1
2
73.4 cps
no log/trace
C-drive for log/trace only
16:42:04-16:44:37
15:09:20-15:11:44 15:11:43-15:53:29
16:44:37-16:51:02
15:09:21-15:53:30BFIO=0
16:42:04-16:51:03BFIO=0
1,212 cps 511.5 cps ? cps no log/trace7 mbps3
no log/trace
10:32:08-10:34:53 10:34:53-10:41:24
1,333 cps 520.8 cps ? cps no log/trace7 mbps4
no log/trace
11:11:27-11:13:57 11:13:56-11:20:20BFIO=25
BFIO=0
1,587 cps 190.1 cps ? cps no log/trace7 mbps5
no log/trace
11:39:26-11:41:3211:41:32-11:59:04SLOG used BFIO=50
Was interrupted, and restarted
![Page 10: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/10.jpg)
04/22/23 | Software AG User Group 2011 | 10
Commands per Second During Replication
1,212 cps 350.26 cps ? cps 369.69 cps6 mbps6
no log/trace
14:56:04-14:58:49 14:56:05-15:05:3614:56:36-15:05:37BFIO=50
673 cps 294.55 cps ? cps 294.12 cps5.5 mbps7
no log/trace
17:14:40-17:19:37 17:18:30-17:26:00 17:14:41- 17:26:01
1,105 cps 359 cps ? cps 358 cps7.0 mbps8 no log/trace
15:16:22-15:19:23 15:16:23-15:25:40 15:16:23-15:25:41
1,914 cps 503 cps ? cps 272 cps8.5 mbps9 no log/trace
11:28:14-11:31:43 11:29:00-11:40:25 11:27:49-11:52:17
1,592 cps 361 cps ? cps 358 cps6.5 mbps10
no log/trace
14:56:47-15:03:04 14:56:56-15:24:4014:56:48-15:24:40
1,148 cps 295 cps ? cps 294 cps5.9 mbps11
no log/trace
15:48:48-16:09:08 15:48:56-17:08:08 15:48:49-17:18:10
7 files
3 files
2 files
1 file
1 file
1 file
![Page 11: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/11.jpg)
CPU Time and Duration
04/22/23 | Software AG User Group 2011 | 11
Prod DB
ASSO DATA WORK
9 Batch Jobs Duration HH:MM
Without RPL 02:20
Prod DB CPU Time Per Week With 9 Jobs Running
Without RPL
08:39:17
![Page 12: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/12.jpg)
CPU Time and Duration
04/22/23 | Software AG User Group 2011 | 12
Prod DB
ASSO DATA WORK
Reptor
SLOG
SYS
9 Batch Jobs Duration
HH:MM
Without RPL 02:20
With RPL – unchanged parms
13:22
Prod DB CPU Time Per Week With 9 Jobs Running
w/o RPL 08:39:17
w/ RPL 17:19:59
![Page 13: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/13.jpg)
04/22/23 | Software AG User Group 2011 | 13
Duration of 9 Jobs
24 December 2010
Without Replication
12 January 2011
With Replication
Job From - To Duration From - To Duration5031 19:01 – 21:00 1:59 19:02 – 07:27 12:255032 19:01 – 20:58 1:57 19:02 – 21:38 2:365033 19:01 – 20:57 1:56 19:01 – 07:11 12:105034 19:01 – 21:06 2:05 19:02 – 07:58 12:565035 19:01 – 21:21 2:20 19:01 – 08:22 13:215036 19:01 – 21:18 2:17 19:01 – 08:23 13:225037 19:01 – 20:59 1:58 19:01 – 07:29 12:285038 19:01 – 20:59 1:58 19:01 – 07:30 12:295039 19:01 – 21:00 1:59 19:01 – 07:33 12:32
TOT 19:01 – 21:21 2:20 19:01 – 08:23 13:22
![Page 14: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/14.jpg)
Buffers and Communication Between Source and Reptor
04/22/23 | Software AG User Group 2011 | 14
![Page 15: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/15.jpg)
04/22/23 | Software AG User Group 2011 | 15
RPL Buffer RPL=524,288,000
88 % used
LBP = 70,000,000
I/O BufferLFIOP = 25% of LBP
Files / Tables
Files / Tables
Files / Tables
RPL Buffer RPL=524,288,000
13 % used
LBP = 3,600,000
I/O BufferLFIOP = 900,000 = 25% of LBP
SLOGFile 30
DB 134 (PROD)Source /
Subscription DB
DB 312 (Reptor)Replicator Engine
DB
7,400 buffer flushes
38,000 buffer flushes
803,970 records loaded15,000 Cylinders defined12,375 Cylinders unused
Buffer Usage During Replication- Data Flow -
Dieter Storr – 14 Jan 2011
BufferUsageDuringReplication
![Page 16: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/16.jpg)
Several Tests With 20% of Production Data
Test #
Date RPL LBP in MG
LFIOP in MB
LRPL in MB
ASYTVS
FMXIO
9 Jobs Duration
1 01/27/2011 YES 70 13 500 YES 1 79 min
2 01/28/2011 NO 70 13 500 YES 1 40 min
3 01/31/2011 YES 150 13 500 YES 1 74 min
4 02/24/2011 YES 150 13 700 YES 8 61 min
04/22/23 | Software AG User Group 2011 | 16
![Page 17: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/17.jpg)
STROBE: Buffer Pool Manager – High CPU TimeSource Database
04/22/23 | Software AG User Group 2011 | 17
MODULE SECTION LINE PROCEDURE/FUNCTION STARTING PROCEDURE CPU TIME NAME NAME NUMBER NAME LOCATION LENGTH SOLO .ADABAS ADANC6 ADANC6 ADABAS BUFFER POOL MNGER 51.49 .ADABAS ADARPN ADARPN ADABAS SYSTEM SERVICES 12.61 .SVC SVC 114 EXCPVR .05 .ADABAS ADANC1 ADANC1 ADABAS THREAD SELECTION .17 .ADABAS ADANC8 ADANC8 ADABAS CMPRESS DVT HNDLR .11 .ADABAS ADANC0 ADABAS USR/HYP EXIT CALL .32 .ADABAS ADAIOS ADABAS I/O PROCESSING .21 .ADABAS ADANC5 ADABAS QUEUE PROCESSING .10 .ADABAS ADANC7 ADABAS FORMAT TRANSLATOR .03 .SUPERVS IGC0024A SUPERVISOR SERVICES .05
![Page 18: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/18.jpg)
STROBE: Buffer Pool Manager – Normal CPU TimeReptor
04/22/23 | Software AG User Group 2011 | 18
MODULE SECTION LINE PROCEDURE/FUNCTION STARTING PROCEDURE CPU TIME NAME NAME NUMBER NAME LOCATION LENGTH SOLO .ADABAS ADARPS ADARPS ADABAS SYSTEM SERVICES 27.27.SUPERVS IGC0024A SUPERVISOR SERVICES 13.64.ADABAS ADANC0 ADANC0 ADABAS USR/HYP EXIT CALL 9.09 .ADABAS ADANC6 ADANC6 ADABAS BUFFER POOL MNGER 8.18 .ADABAS ADANC1 ADANC1 ADABAS THREAD SELECTION 7.27 .SVC SVC 114 EXCPVR .00 .ADABAS ADANC7 ADANC7 ADABAS FORMAT TRANSLATOR 3.64 .ADABAS ADANC5 ADANC5 ADABAS QUEUE PROCESSING 2.73 .NUCLEUS IEAVESLK SUSPEND LOCK SERVICE 2.73 .ADABAS ADAIOS PSHSTK ADABAS I/O PROCESSING 2.73
![Page 19: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/19.jpg)
Reduce CPU Time and Duration - Solutions
04/22/23 | Software AG User Group 2011 | 19
Prod DB
ASSO DATA WORK
Reptor
SLOG
LBP= ++LFIOP= --LRPL= ASYTVS=YESFMXIO=8
Reptor
SLOG
Reptor
SLOG
Broker
![Page 20: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/20.jpg)
Important Parameters
- LBP and LFIOP
- LRPL
- ADADBS DSBI=OFF
- ASYTVS=YES and FMXIO= (6-8, max 16) => SLOG
- LOGINPUTTRANS= (50-70)
04/22/23 | Software AG User Group 2011 | 20
![Page 21: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/21.jpg)
Challenge with Initial-State
04/22/23 | Software AG User Group 2011 | 21
Source DB
ASSO DATA WORK
Reptor
SLOG
SYS
A file with 160 million ISNs takes too long using RPL’s initial-state function
Target DB
ASSO DATA WORK
Broker Network
sFirewallsRouters
160 million READs
+ 160 million STOREs + 160 million ETs
# Files
Adabas Commands/Sec Source / Reptor
160 Millions ISNs
1 990 cps / 438.60 cps 101.24 hours = 4.2 days
7 1,253 cps / 308.57 cps
144.03 hours = 6.0 days
![Page 22: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/22.jpg)
Challenge with Initial-State
04/22/23 | Software AG User Group 2011 | 22
Source DB
ASSO DATA WORK
Reptor
SLOG
SYS
Solution 1:
Target DB
ASSO DATA WORK
Broker Network
sFirewallsRouters
Decompresss
FTP Compresss
Load
![Page 23: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/23.jpg)
Challenge with Initial-State
04/22/23 | Software AG User Group 2011 | 23
Solution 2:
ADAMAGIC
Restore ADASAV backup from the mainframe to Windows or UNIX
Why not from SAG?
![Page 24: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/24.jpg)
Efficient Usage of Adabas
Replication
- A Practical Solution Finder –
By Dieter W. Storr
ISBN 978-3-8348-1730-3
04/22/23 | Software AG User Group 2011 | 24
• 11 Chapters• 262 Pages• 221 Figures
• Natural Monitor Programs• Assembler Programs
![Page 25: Problems and Solutions with Adabas Replication and Mass Data](https://reader035.fdocuments.net/reader035/viewer/2022062309/568158e9550346895dc62b20/html5/thumbnails/25.jpg)
Thank You!