Db2 For I Parallel Data Load
-
Upload
thomas-wolfe -
Category
Documents
-
view
1.336 -
download
4
description
Transcript of Db2 For I Parallel Data Load
DB2 for i External Stored Procedure Example
Page 1 of 10 1/3/2011 4:49:44 PM
Rational for i and Optim Studio Development Environment ............................................. 1
Load Process Flowchart (ACS_i5Load and ACSGenesis)................................................ 2
Run Stored Procedure from Optim and Rational for i packages (client) ............................ 3
CPYFRMIMPF commands are submitted for batch processing on i ................................. 3
Module syb_i5_LoadFromDelimitedFile in service program ACSGENESIS ................... 4
DB2 External Stored Procedure Interface........................................................................... 6
DB2 External Stored Procedure Program........................................................................... 7
Benefits of Stored Procedure Processing versus ODBC/JDBC statements........................ 8
Two Types of Stored Procedures .................................................................................... 8
Maintenance performed in one place.............................................................................. 8
Get Job Completion Status of Stored Procedure with a Select Statement.......................... 9
References......................................................................................................................... 10
Rational for i and Optim Studio Development Environment
• We would like to load many tables as fast as possible without locking our GUI for
an extended timeframe.
DB2 for i External Stored Procedure Example
Page 2 of 10 1/3/2011 4:49:44 PM
Load Process Flowchart (ACS_i5Load and ACSGenesis)
DB2 for i External Stored Procedure Example
Page 3 of 10 1/3/2011 4:49:44 PM
Run Stored Procedure from Optim and Rational for i packages (client)
CPYFRMIMPF commands are submitted for batch processing on i
DB2 for i External Stored Procedure Example
Page 4 of 10 1/3/2011 4:49:44 PM
Module syb_i5_LoadFromDelimitedFile in service program ACSGENESIS
// * ============================================================== *
// * DB2 for i: Load from delimited files (syB) to ACS tables (db2) *
// * ============================================================== *
p syb_i5_LoadFromDelimitedFile...
p b export
d syb_i5_LoadFromDelimitedFile...
d pi n
d i5_Context 32 const
d i5_Working 128 const
d i5_Schema 128 const
d i5_BegPos s 5u 0 inz(0)
d i5_EndPos s 5u 0 inz(0)
d i5_LibFil s 20 inz(' ')
d i5_CpyFil s 1024 inz(' ')
d exist s 1 inz('0')
d existIndicator s n based(sqlPointer)
d sqlPointer s * inz(%ADDR(exist))
d i5_LibNme s 10 inz(' ')
/Free
Monitor;
// * ============================================================== *
// * Check / Set import application context and working directories *
// * ============================================================== *
If isApplicationWorkingDirectoryValid( %trim(i5_Context):
%trim(i5_Working) );
i5_dirName = %trim(i5_Context) + '/' + %trim(i5_Working);
If i5_Schema = 'QUSER';
i5_LibNme = RtvCurLib( pgmStatus.jobName:
pgmStatus.userId:
pgmStatus.jobNumber );
Else;
i5_LibNme = pgmStatus.userId;
EndIf;
i5_dirPtr = OpenDir(%trim(i5_dirName));
If (i5_dirPtr <>*NULL);
DoU i5_dirEntPtr = *Null;
i5_dirEntPtr = ReadDir( i5_dirPtr );
Monitor;
On-Error *All;
EndMon;
If i5_dirEntPtr = *Null;
Leave;
Else;
i5_entryName = %str(%addr(i5_nameFull));
// * ============================================================== *
// * Filter navigation entries (.) and select .data delimited files *
// * ============================================================== *
If %subst(i5_entryName: 1: 1) <> '.';
i5_EndPos = %scan('.data':
%xlate(upper: lower: %trim(i5_entryName)): 1);
If i5_EndPos <> 0;
i5_BegPos = %scan('.': %xlate(upper: lower:
%subst(%trim(i5_entryName): 1: i5_EndPos - 1)): 1);
DB2 for i External Stored Procedure Example
Page 5 of 10 1/3/2011 4:49:44 PM
// * ============================================================== *
// * i5 internal system object name from DB2 schema catalog entries *
// * ============================================================== *
i5_LibFil = syb_i5_getTableName(%trim(i5_LibNme):
%subst(%trim(i5_entryName)
:(i5_BegPos + 1)
:((i5_EndPos - i5_BegPos) - 1)));
If i5_LibFil <> ' ';
// * ============================================================== *
// * Construct i5 CpyFrmImpF (Copy From Import File) command string *
// * ============================================================== *
i5_CpyFil = 'CPYFRMIMPF FROMSTMF(' + ticMark +
%trim(i5_Context) + '/' +
%trim(i5_working) + '/' +
%trim(i5_entryName) + ticMark +
') TOFILE(' + %subst(i5_LibFil: 1: 10) +
'/' + %subst(i5_LibFil: 11: 10) +
') MBROPT(*REPLACE) RCDDLM(*CRLF' +
') DTAFMT(*DLM) STRDLM(' + ticMark +
'"' + ticMark + ') RMVBLANK(*TRAILING' +
') FLDDLM(' + ticMark + ',' + ticMark + ')';
// * ============================================================== *
// * Remove joblog messages, prepare for submit job status capture *
// * ============================================================== *
RemoveMessage(rmv_PgmQue:
rmv_StkCnt:
rmv_MsgKey:
rmv_Remove:
error_Code);
// * ============================================================== *
// * Construct i5 SbmJob (Submit Job) command (CpyFrmImpF) queuing *
// * ============================================================== *
i5_CmdStr = 'SBMJOB CMD(' + %trim(i5_CpyFil) +
') INLLIBL(*CURRENT) JOBD(*USRPRF) JOBQ(*JOBD' +
') JOB(' + %trim(%subst(i5_LibFil: 11: 10)) +
') MSGQ(*LIBL/ACS_GEN500) HOLD(*YES) PRTTXT(' +
ticMark + %subst(%trim(i5_entryName)
:(i5_BegPos + 1)
:((i5_EndPos - i5_BegPos) - 1)) +
ticMark + ')';
CallP(e) i5_RunCmd(%trim(i5_CmdStr): %len(%trim(i5_CmdStr)));
If %Error;
snd_Data = 'LOAD FAILURE: ' + %trim(i5_entryName) +
' in ' + %trim(pgmStatus.mainProcName);
Else;
snd_Data = 'LOAD SUCCESS: ' + %trim(i5_entryName) +
' in ' + %trim(pgmStatus.mainProcName);
// * ============================================================== *
// * Receive joblog messages prepare for submit job status capture *
// * ============================================================== *
ReceiveMessage(MessageInfo :%size(MessageInfo)
:'RCVM0100' :'*' :0 :'*ANY' :MsgKey
:0 :'*OLD' :error_Code);
If i5_ByteProv = 0;
If messageInfo.common.QMHMI03 = 'CPC1221';
i5_JobNme = %subst(messageInfo.errMsgDta: 1: 10);
i5_UsrNme = %subst(messageInfo.errMsgDta: 11: 10);
DB2 for i External Stored Procedure Example
Page 6 of 10 1/3/2011 4:49:44 PM
i5_JobNbr = %subst(messageInfo.errMsgDta: 21: 6);
i5_UsrDta = %subst(%trim(i5_entryName)
:(i5_BegPos + 1)
:((i5_EndPos - i5_BegPos) - 1));
If Not WrtLogSts('ACSLOAD_01': i5_TskCnt: i5_JobNme:
i5_UsrNme: i5_JobNbr: i5_UsrDta);
// Add additional error handling logic here ======>
EndIf;
EndIf;
Else;
// Add additional error handling logic here ======>
EndIf;
EndIf;
EndIf;
Else;
snd_Data = 'LOAD WARNING: ' + %trim(i5_entryName) +
' in ' + %trim(pgmStatus.mainProcName);
EndIf;
CallP(e) sndjobmsg(snd_Data); // Job Log Message Entry
EndIf;
EndIf;
EndDo;
existIndicator = *ON; // Return SUCCESS code
Else;
i5_errNoPtr = errno();
snd_Data = 'Open directory failed from ' + %trim(i5_dirName) +
' in program ' + %trim(pgmStatus.mainProcName);
CallP(e) sndjobmsg(snd_Data); // Job Log Message Entry
EndIf;
closeDir(i5_dirPtr);
EndIf;
On-Error *All;
snd_Data = 'Unexpected error in syb_i5_loadFromDelimitedFile()' +
' in program ' + %trim(pgmStatus.mainProcName);
CallP(e) sndjobmsg(snd_Data); // Job Log Message Entry
EndMon;
Return existIndicator;
/End-Free
p syb_i5_LoadFromDelimitedFile...
p e
DB2 External Stored Procedure Interface
Create Procedure i5ParallelDataLoader (
IN Type char( 10) )
LANGUAGE RPGLE
SPECIFIC ACS_I5LOAD
NOT DETERMINISTIC
External Name ACS_I5LOAD
CALLED ON NULL INPUT
PARAMETER STYLE DB2SQL;
DB2 for i External Stored Procedure Example
Page 7 of 10 1/3/2011 4:49:44 PM
DB2 External Stored Procedure Program
* ================================================================== *
* Program name...: ACS_i5LOAD *
* Purpose........: Load database tables from delimited files *
* Programmer.....: Thomas and Victoria Wolfe *
* Create Date....: December 7, 2010 *
* *
* Maintenance Log: *
* *
* PGMR SEARCH DATE DESC. OF CHANGE *
* *
* ================================================================== *
/COPY qCPYSRC,ACS_H_PGM
/COPY QPRTSRC,ACSGENESIS
d i5_LoadType s 10
d i5_NullInd1 s 5i 0
d sqlStateCode s 5
d functionName s 517
d specificName s 128
d errorMessage s 70
d i5_ProdLoad s 10 inz(' ')
d message s 80 inz(*Blank)
d command s 2048 inz(' ')
* ================================================================== *
* Common program status information data structure (SDS) *
* ================================================================== *
/COPY QCPYSRC,ACS_SQLOPT
* ================================================================== *
* i5 DB2 Parallel Data Loader: Submit CpyFrmImpF command to batch *
* ================================================================== *
c *entry plist
c parm i5_LoadType
c parm i5_NullInd1
c parm sqlStateCode
c parm functionName
c parm specificName
c parm errorMessage
/Free
Monitor; // Global error handler
sqlStateCode = '00000';
If i5_NullInd1 = 0;
i5_ProdLoad = i5_LoadType;
Select;
When %xlate(upper: lower: %trim(i5_ProdLoad)) = '*sybase';
If syb_i5_LoadFromDelimitedFile(%trim(syB_Context):
DB2 for i External Stored Procedure Example
Page 8 of 10 1/3/2011 4:49:44 PM
%trim(syB_dataDump):
%trim(pgmStatus.UserId) );
Else;
sqlStateCode = '38999';
errorMessage = 'Error occured in external stored ' +
'procedure. Contact support and report ' +
'this error message.';
EndIf;
Other;
sqlStateCode = '38001';
errorMessage = 'Invalid input load parameter ' +
%trim(i5_ProdLoad) + ' not processed.';
EndSL;
Else;
sqlStateCode = '38001';
errorMessage = 'Null input load parameter not' +
' processed.';
EndIf;
On-Error *All; // Unmonitored Errors?
Dump(a);
EndMon;
*inlr = *On;
Return;
/End-Free
Benefits of Stored Procedure Processing versus ODBC/JDBC statements
• Secure access to DB2 data.
• Access non-DB2 data if necessary –VSAM, IMS, etc.
• Code business logic once
o Maintenance is simpler
• Performance improvement?
o Instead of multiple SQL statements being sent
• Lots of languages to choose from
Two Types of Stored Procedures
• SQL Stored Procedures
• External Procedures
Maintenance performed in one place
DB2 for i External Stored Procedure Example
Page 9 of 10 1/3/2011 4:49:44 PM
Get Job Completion Status of Stored Procedure with a Select Statement
Select PROCESS,
PRCSEQ,
Cast(Case
When STATUS = 'N'
Then 'Success'
When STATUS = ‘A’
Then ‘Failure’
Else 'Running’
End As Character(7)) As Status,
STARTSTMP,
ENDSTMP,
RUNNAME,
RUNUSER,
RUNNUMBER,
ERRORID,
USRDATA
From ACSJOBLOG
Order by 4,1,2
DB2 for i External Stored Procedure Example
Page 10 of 10 1/3/2011 4:49:44 PM
References
Writing Stored Procedures in RPG MC Press Online 15 January 2008, Susan Gantner
http://www.mcpressonline.com/programming/rpg/writing-stored-procedures-in-rpg.html
Stored Procedures, Triggers, and User-Defined Functions on DB2 UDB for iSeries
http://www.redbooks.ibm.com/abstracts/sg246503.html
DB2 UDB for AS/400 Object Relational Support
http://www.redbooks.ibm.com/abstracts/sg245409.html
Cross-Platform DB2 Stored Procedures: Building and Debugging
http://www.redbooks.ibm.com/redbooks/pdfs/sg245485.pdf
DB2 9 for z/OS Stored Procedures: Through the Call and Beyond
http://www.redbooks.ibm.com/abstracts/sg247604.html
DB2 Java Stored Procedures: Learning by Example
http://www.redbooks.ibm.com/redbooks/pdfs/sg245945.pdf
Modernizing IBM eServer iSeries Application Data Access – A Roadmap Cornerstone
http://www.redbooks.ibm.com/abstracts/sg246393.html
Improving Stored Procedure Performance 18 December 2009, Kent Milligan, Jarek
Miszczyk, and Gene Cobb http://tinyurl.com/2ayskqo
The API Corner MC Press online: Automating Recovery (or Keeping the Help Desk Out
of the Loop) 21 July 2010 – October 2010, Bruce Vining http://tinyurl.com/32gkc9h