DBT544 Class 1

UDB/400 – DB2 Universal Database for the AS/400

Who is Tim McKenna? Cursed Standards What is this course?

o download the DBTPPTZIP file in the course library: DBT544TM Note: IBM holds more database patents than all their competitors combined. See Analyzing The Relational Database Vendors for InformationWeek's

Dec/2000 industry survey of IBM vs. Oracle vs. Microsoft DBs. See The Future of SQL/400 and the iSeries 400 Database by Paul Conte,

NEWS/400 Senior Technical Editor (and the author of our textbook). IBM Partners In Education (PIE) IBM's DB2 UDB for AS/400 Web site AS400 Network SQL/400 and DB Community SQL/400 Developer's Guide textbook Club Tech iSeries DB2 UDB and SQL/400 Newsletter

UDB/400 native database within Qsys.lib

Other file systems are supported by the IFS (Integrated File System). iSeries or AS/400 is the only platform that supports all Microsoft/Novell/Unix file systems.

SQL runtime support is standard in OS/400. SQL Developers Kit is optional.

Paul Conte's FAQ: WHAT IS THE AS/400's DATABASE?

I frequently get the question, "What is the AS/400's database?" or "What's the difference between "DB2" and "UDB"? Here are the answers to these and some related questions that come up fairly regularly:

a. First off, be sure you understand that "iSeries" is just the new name for the "AS/400." Technically, IBM calls some newer hardware models the iSeries, while older models are AS/400s, but the iSeries and the AS/400 both run the OS/400 operating system and have the SAME database.

b. The iSeries (and the AS/400) run one and only one database, which in V4R5 is called DB2 Universal Database for AS/400. I like to use "UDB/400" as the short form, but "DB2/400" means the same thing. In a future release, I imagine IBM will change "for AS/400" to "for iSeries", but it will still be just another release of the same integrated database that's always been on the AS/400. You can learn the product

details at http://www-1.ibm.com/servers/eserver/iseries/db2 [Footnote: There are a couple of Java-based databases - Cloudscape and PointBase - that can run on any machine with a Java Virtual Machine (JVM) and thus can also run on the iSeries and AS/400. But at this point, these aren't practical alternatives to UDB/400.]

c. UDB/400 comes as part of OS/400, so EVERY iSeries (and AS/400) system includes it. You don't have to buy UDB/400 as a separate product as you do other database management systems on other platforms. That's cool because it means application developers can count on the database being on any iSeries (or AS/400) on which they want to install their software. You still have to be concerned with release compatibility, of course - you can't run an application that uses a feature introduced in V4R5 on a system running an earlier release (e.g., V4R4) of OS/400. You CAN, however, run an application created on a V4R5 system on a system running an earlier release, as long as the application uses ONLY features available in the earlier release.

d. NONE of the other major relational database management systems (RDBMSs) runs on the iSeries. You can't run Oracle, Informix, or Microsoft SQL Server 2000 on an iSeries.

e. UDB/400 includes runtime support for both SQL database operations and "traditional" RPG/400, ILE RPG, Cobol, and other high-level language I/O. You do NOT need to buy anything (other than OS/400) to run SQL applications.

f. The DB2 Query Manager and SQL Development Kit for AS/400 product is an add-on product you can buy to DEVELOP HLL applications that use embedded SQL. You must have this product to translate HLL source files that include SQL statements as part of the program-creation process. You do NOT need this product to RUN a compiled SQL application. The product also includes a green- screen Interactive SQL (ISQL) utility that lets you run SQL statements. ISQL is meant for programmers, not end users. And, last but not least, the product includes Query Manager (QM), another green-screen utility to define reports. QM is intended for end users. You can learn more about the product at http://www.ibm.com/servers/eserver/iseries/db2/db2sql.htm

  SQL DB2/400 IFS

databasecollection(schema)

library drive/dir

filetable(relation)

PF byte stream file

set Row (Tuple) record program described

element Column (Attribute) field

data path Index LF  

dynamic selection View OPNQRYF  

security Grant/Revoke xxxOBJAUT basic by UserID

Data types

character strings Char A  

packed numbers(use odd lengths)

Decimal P  

numbers Numeric S  

date Date L  

time Time T  

date & time Timestamp Z  

binary data Hex H  

Binary Numbers Binary B  

Floating Point Float F  

Field Reference File: like a data dictionary

Physical Files contain the data in members. Usually only one member per file. SQL does not support members (must use OVRDBF).

Logical File applications:

permanent alternate access path (performance good/bad) restrict usage: e.g. input only security (using selection &/or projection and ObjAut) string handling (using substring) access multiple file members in a single file access multiple physical files in a single file

Logical File techniques:

Selection (restrict/select records)- performance- security

Projection (specified columns)- reduces program size (not too many fields in memory)- security (column security available in SQL)

join (read only) multi-format (for update)

- e.g. Order Header, Order Lines access multiple file members in a single file access multiple physical files in a single file

DBT544 Class 2

DB2 -- the traditional approach

Field Reference File: like a data dictionary

Physical Files contain the data in members. Usually only one member per file. SQL does not support members directly (must use OVRDBF or create a table ALIAS).

Creating a Physical File:

always use a Primary Key (minimal and UNIQUE) avoid use of Relative Record Number processing avoid using FIFO, LIFO, FCFO in Logical Files to deal with duplicate key

issues always put a TEXT keyword on the record format (for program documentation) Fields should have these keywords: COLHDG is user interface with data TEXT is the programmer interface; specify if different from COLHDG ALIAS field names for COBOL DFT specifies a default value for new records VALUES, RANGE, COMP to validate input

Validating input with DDS keywords

poor user interface only effective if user inputs via a DSPF that adopts the validation

GO CMDCST to use Constraint Commands

Validating input with PF Constraints

always enforced requires program error checking techniques consolidates the business rules in one place very poor user interface with DFU

Validating input with HLL

only enforced via programs simple to complex checking of business rules allows exceptions to rules rules can be broken by program bugs & utilities (DFU) requires simple program editing & error msg techniques good user interface (as good as you make it)

Run the DSPKEY command to view the records of physical or logical files in key sequence. This is not a standard OS/400 command. See DBT544TM/QCLPSRC for the source code.

Is DDS Dead?

DBT544 SQL Introduction

SQL, Structured Query Language, is the direction for new DB2 UDB/400 development.

Old DDS interface has been "frozen". In IBM speak, this means it continues to be supported but will not be developed further.

The challenge for AS/400 professionals:

support the old DDS systems (they will be around for a while) create new DB using SQL

o expect resistence from clients with established systemso requires additional Program Products: Query Manager and SQL

Developers Kito develop in SQL, even for established systemso practice 'backward compatible' techniques

i.e. be OS/400, RPG/400, COBOL/400 friendly

o meaningful table & column names <= 10 charfor ease of use in native tools refering to files and fields.

SQL native comments

base table PF permanent data object

result tableOPNQRYFQuery/400

temporary

viewindex

LF alternate data path

RunSqlStmRun SQL Statement

CrtPF/LF, ChgPF/LF, Go CMDCST

SQL statements & DDS definitions stored in source membersRunSqlStm cannot run Select (must use Query Manager)

STRSQLStart Interactive SQL (ISQL)

Query/400 (very limited in comparison)

can use to create tables/files but only recommended for testing -- results in limited documentation, hard to maintain if no source.

Operations Navigator green screennew features / ease of use development will be in Operations Navigator

RunSqlStm source DDS source

Create Table CRTxF

Label On COLHDG Column Headings (end user interface to data)

Comment On TEXT programmer/query user interface to data

Drop DLTF, CHGPF delete table/file, remove column/field

Grant & RevokeGRTOBJAUT, RVKOBJAUT

only SQL has column securitynative requires LF with projection

Insert, Update, Delete UPDDTASQL can act on sets of rows (powerful and dangerous)DFU is for end user, one row at a time.

Select * from table [output to file in ISQL]CREATE TABLE x LIKE existing_tableplus INSERT

CPYF copy a file

-- comments * (col 6) comments possibly the most important source specs.

STRSQL F13=Services, 1. Change session attributes

o Date format . . . . . . . . . *ISO YYYYMMDD -- the one, true format

o Naming convention . . . . . . *SYS

*SYS means library/file syntax*SYS must be specified for ISQL to use the *LIBL

Interactive SQL will create new objects in your *CURLIB and look for an unqualified table names in your *LIBL only if your ISQL "Naming convention" is *SYS.

*SQL means collection.table syntax

· textbook uses this format· ISQL does not support DftRdbCol option (as other SQL functions do)· default collection name is your User ID!

o Collection list . . . . . . . *USRLIBL

Saves a lot of keystrokes when prompting collections with F4.

o Statement processing . . . . . *RUN

*RUN – The SQL statement is syntax and validity checked, and then run.*VLD – The SQL statement is syntax and validity checked only.

· all statement elements referred to (collection, table, view, column, library, file, field) verified for existance

· develop DML statements to save in a source member

*SYN – The SQL statement is syntax checked only

· statement is constructed according to the SQL rules

· develop DDL statements to to be run on other systems

F3/F13 can save the ISQL session statements in a source member. You can edit the statements and run them with the RunSqlStm command.

SQL Catalog:(see textbook page 37, Fig 2.13)QSYS2.Sys* => about the entire AS/400 database -- only one SQL DB on an AS/400collection.Sys* => about the collection/library

use Create Collection libName to create an AS/400 library=> get journals for rollback and commitment control=> get SQL Catalog about your collection=> use CHGLIB or other SQL statements to modify afterward

Create Schema use to define entire collection with tables, views, etc. all at once. Good for creating the completely developed collection on a client's AS/400.

SQL native comments

Create Table CRTPF

OS/400 SQL table(also a *FILE object)

*FILE object

create with meaningful OS/400 name, then Rename Table OS/400file To longSQLtableName

hard to deal with cryptic SQL created OS/400 system names for objects, e.g. CUSTF00001

only one member can have X members

no maximum sizemax. records: 10,000 plus 1000 × 3, must CHGPF or o/ride CRTPF for *NOMAX

SQL saves "file full" problems

reuses deleted recordsmust CHGPF or o/ride CRTPF to Reuse deleted records

SQL saves RGZPFM

journalling of before/after record changesup to the programmer to set up or design application audits

* journalling needed for rollback/commitment control* more overhead but good transaction control* very useful with complex entity relationships

table name = record format namefile name <> record format

RPG programs will need to rename SQL record formats

use For clause to assign system field names to SQL columns

use ALIAS keyword to create long field/column names

field name/alias

distinct type / User-Defined Types: use only for primary/foreign key columns

Field ReferenceSQL extremely limited in data dictionary/field reference features

Alter Table Add/Drop/Alter

modify DDS and CHGPFadd/delete/change fields or field attributes

Create {unique} Index DDS and CRTLF alternate access path to data

Create Alias no equivalentonly use a table/file alias if you really need it. Why have two names for one thing?

Create Alias FileMbrName For File (Member)

OVRDBF FILE(file name) MBR(member name)

Access to multi-members in a file:SQL is permanent aliasnative is temporary for job

Drop use Restrict keyword

DLTobjremove library/file level objects

QSQLSRC QDDSSRCsource members defining tables/files/indexes

See DBT544TM/QSQLSRC.@STANDARDS for notes on creating source members to define tables.

DBT544 Data Definition LanguageIf using ISQL (STRSQL cmd),

run CHGCURLIB and EDTLIBL beforehand to adjust your library list for the collections you want to access. Make your current library the name of your most important collection. Any new objects will be created here by default.

F13=Services, 1. Change session attributeso Naming convention . . . . . . *SYS

*SYS means library/file*SYS must be used in ISQL to use the *LIBL and to have new objects created in your *CURLIB

*SQL means collection.table · textbook uses this format· ISQL does not support DftRdbCol option (as other SQL functions do)· default collection name is your User ID!

o Collection list . . . . . . . *USRLIBL (or default *LIBL) F3/F13 can save the ISQL session statements in a source member for later use

by RunSqlStm command

When making a new collection:

STRSQL Create Collection exit ISQL you cannot CRTSRCPF QSQLSRC (which will contain your DDL source) in your

collection library create your source PF in another library (and optionally copy/move it to your

new collection library)

create source members, one for each table, index, etc. use RunSqlStm with the

Default collection . . . . . . . DFTRDBCOL __________ parameter otherwise you must qualify each DDL statement with the collection namethis limits the portability of your specs and ease of test/production library management

See Chap.13, pages 268-276.

Indexes

About Using Varchar?

What is in index?

Accelerating Your Queries with Encoded Vector Indexes

Indexes support "Order By" clauses in Select statements and "Where" statements in Select statements (and Views) .

DBT544 Views

see DBT544TM/QSQLSRC.VIEWS for examples based on a completed Lab 2

The textbook's explanation of views "with Check Option" is not too detailed.If you want to know more about it, see the SQL Reference manual on this point.

Views allow an alternate projection of columns and/or a selection of rows from the based on table. Views can be based on top of other views.

Views provide a way to simplify, as well as restrict, access to data.

You can use SQL views to

Select a subset of the rows in a base tableo Used to improve performance for commonly or frequently selected

rows. e.g. rows for a department; rows for a major customero Used for security purposes (perhaps With Check Option) to constrain

users to only the rows they own. Include only a subset of a base table's columns

o Used to hide columns from some users or applications.

o Used to reformat columnso Used for security purposes (it may be easier to administer security on a

view than on a column by column basis). Derive new view columns based on one or more underlying base table

columnso use SubStr and/or ConCat to put a name togethero perform calculations such as QTY * PRICE as EXTENDED_PRICEo reformat numeric fields into date data types

Join related rows from multiple base tables into a single row in the viewo DB Admininstrators like third normal form, users don't.o Views Join Tables Together

Combine sets of rows from multiple tables into a single viewo UNION operator (Ch.8)

See DBT544TM/QSQLSRC.views source member for examples of creating and using views With Check Options.

Reformating Dates

Date, time and TimeStamp data types are output with editing characters format is as per the Date/Time format of the job/ISQL session/RunSqlStm

parameter You can extract various components from a Date data type:

o Year, Month, Day of a date, timestamp, or durationo Days = number of days since 0001-01-01 => good for interval

calculationso DayOfYear = sequential number of day within the year (Julian date)o DayOfWeek = sequential number of day within the week (1-7 = Sun-Sat)o Week = sequential number of week within the year (1-53)o Quarter = 1 to 4 meaning Q1 (Jan-Mar), Q2 (Apr-June), ...

You can extract the Date or Time from a TimeStamp data type:o Date(TimeStamp)o Time(TimeStamp)

You can reformat the output of a date or time using Char(...):o SELECT CSTLSD as TimeStamp,

date(cstlsd) as Date_timestmp, time(cstlsd) as Time_timestmp, char(date(cstlsd),EUR) as Char_date,

char(time(cstlsd),USA) as Char_time FROM CUSTPHONE

o Char(date/time/timestamp [, ISO, EUR, USA <== to reformat] ) date editing of old numeric format (YYMMDD) requires SubStr & ConCat:

from Numeric ==> Digit string ==> Edited Date Stringo Select cstNXD as YYMMDD, char(cstNXD) as char_string,

digits(cstNXD) as digit_string, substr(digits(cstNXD),1,2) as YY, substr(digits(cstNXD),3,2) as MM, substr(digits(cstNXD),5,2) as DD, substr(digits(cstNXD),1,2) ||'/'|| substr(digits(cstNXD),3,2) ||'/'|| substr(digits(cstNXD),5,2) as edited_date from custphone

from old numeric format (YYMMDD) to new date data typerequires Numeric ==> Digit string ==> Edited Date String ==> Date:

o Date format is *YMD (system adds the correct century):o select cstNXD as YYMMDD,

DATE(substr(digits(cstNXD),1,2) ||'/'|| substr(digits(cstNXD),3,2) ||'/'|| substr(digits(cstNXD),5,2) ) as date_type from custphone

o

simple subtraction of two dates gives number of years, months, days (yyyymmdd) as the duration interval

The Age Calculator:

o CREATE TABLE BIRTHDAY (NAME CHAR (10 ) NOT NULL WITH DEFAULT, BIRTHDATE DATE NOT NULL WITH DEFAULT)

o INSERT INTO BIRTHDAY VALUES('name', 'yyyy-mm-dd') (above date in ISO format) orINSERT INTO BIRTHDAY VALUES('name', 'dd.mm.yyyy') (above date in EUR format) orINSERT INTO BIRTHDAY VALUES('name', 'mm/dd/yyyy') (above date in USA format)

* system understands any format of a 10 character formatted date string

o SELECT name, curdate(), birthdate, digits(curdate() - birthdate) as duration, SmallInt( year(curdate() - birthdate)) as Years, SmallInt(month(curdate() - birthdate)) as Months, SmallInt( day(curdate() - birthdate)) as Days, SmallInt( days(curdate())- days(birthdate)) as Total_Days FROM birthday

o SmallInt (small integer) is used above to reduce size of display column report the text week day (Sun - Sat) from the Day of Week

o Select birthdate, DayOfWeek(birthdate) as DoW_No, case DayOfWeek(birthdate) when 1 then 'Sunday' when 2 then 'Monday' when 3 then 'Tuesday' when 4 then 'Wednesday' when 5 then 'Thursday' when 6 then 'Friday' when 7 then 'Saturday' else 'who knows?' end as Day_of_birth FROM birthday

more compact method to report montho Select birthdate,

SUBSTR ('Jan. Feb. MarchAprilMay  June July Aug. Sept.Oct. Nov. Dec. ', 

month(birthdate) *5 -4, 5) from birthday

o Substring parameters:1. string of month names, 12 × 5 char each ... like array of elements2. extract month number from date × element length – (length – 1)3. element length

Put it all together in a viewo Create view birthdayV as

SELECT name, curdate() as today, birthdate, digits(curdate() - birthdate) as duration, SmallInt( year(curdate() - birthdate)) as Years, SmallInt(month(curdate() - birthdate)) as Months,SmallInt( day(curdate() - birthdate)) as Days,SmallInt( days(curdate())- days(birthdate)) as Total_Days,

SUBSTR ('Jan. Feb. MarchAprilMay  June July Aug. Sept.Oct. Nov. Dec. ', 

month(birthdate) *5 -4, 5) as Month,

case DayOfWeek(birthdate) when 1 then 'Sunday' when 2 then 'Monday' when 3 then 'Tuesday' when 4 then 'Wednesday' when 5 then 'Thursday' when 6 then 'Friday' when 7 then 'Saturday' else 'who knows?' end as Day_of_birth

FROM birthday

DBT544 Security and Administration

DB change issues

Record format level check . . . . . . . . . : LVLCHK *YES

Specifies whether the levels of record format identifiers are checked to verify that the current record format identifier is the same as that specified in the program that opens the physical file.

Programs compile the projection of a table/view into the program object. This is done for efficiency. Tables/files are changed infrequently whereas programs are run very frequently. When a program is run, the system checks its files are still at the same level.

The system maintains a level identifier for a file and can detect file description changes. This is not just a version number. It is a code that represents the projection of the file. Changing text or column headings does not change the level id because projection is unaffected. Changing a column's data type/length or adding/dropping a column -- now that changes the projection and the level id.

Run DSPFD filename TYPE(*RCDFMT) or DSPFFD to see the level id.

Display File Description (DSPFD filename TYPE(*RCDFMT)) Record Format Level Format Fields Length Identifier BIRTHDAY 2 20 3AD77522FCEFD

after adding a new column: Record Format Level Format Fields Length Identifier BIRTHDAY 3 28 3AC7111B2953B

Run Display Program References (DSPPGMREF) to see the level id compiled with a program.

Display Program References (DSPPGMREF) Record Format Format Level Identifier Field Count BIRTHDAY 3AD77522FCEFD 2

Running a program based on the original version of the table results in:CPF4131 - Level check on file name in library lib with member mbr.

drop the column and the level ID returns to its original value:

Record Format Level Format Fields Length Identifier BIRTHDAY 2 20 3AD77522FCEFD

Program can be run again because levels match.

If a program uses a view, the level stays the same if changes to the based on table do not affect the view. e.g. adding a new column to the table. However, if you change the attributes of columns referenced in the view, a level check will occur.

Remember, there is more system overhead when a program uses a view than a table.

As a DB Administrator, how do you deal with all this?

Build an output file of program references from all relevant libraries:DSPPGMREF PGM(library/*ALL) OUTPUT(*OUTFILE) OBJTYPE(*ALL)

OUTFILE(QTEMP/DSPPGMREF) OUTMBR(*FIRST [*ADD,*REPLACE]) See DBT544TM2/DSPPGMREF Query/400 for a display of this output.

Build an output file of file description information:

DSPFD FILE(LIBRARY/*ALL) TYPE(*RCDFMT) OUTPUT(*OUTFILE) OUTFILE(QTEMP/DSPFD) OUTMBR(*FIRST [*ADD,*REPLACE])See DBT544TM2/DSPFD Query/400 for a display of this output.

See DBT544TM2/PGMFLEVEL Query/400This query joins the program and file information. Review it to see the impact of changing a table or view.You could modify the query to select only those programs having unequal level IDs. Then, have the query output to a report for the programmers to review.

The Application Development Manager/400 is an optional feature on the AS/400 which takes care of these issues (and many others including version control).

SecurityBuilt-in Object Level Security

IBM's Secure your database references AS/400 uses object level authority. Authority applies to all members in a File

object. Objects exist in a library. A library is an object. Therefore, you need authority

to both the object and the library it resides in. every object has an owner who has, by default, all authority to it (*ALL) then, there is everyone else: *PUBLIC security defines what anyone besides the owner can do to an object Object Authorities govern what you can do with or to the object:

OS/400Authority

SQLprivilege

this means

Operationalassumed with row privileges

look at the object's attributes; use the object as per Data Authorities or SQL row privileges- most users need only Object Operational authority

Management With Grant Option to specify security, to move or rename the object, and to add members if the object is a database file, to clear a

member.- needed for CRTDUPOBJ (! note PDM Object option 3. !)- in SQL, allows table description, index, and constraint definition

Existence Allto control the object's existence and ownership (e.g. Delete or Drop)

Alter Index, Alter Add, Clear, Reorganize DB file members; CHGPF

Reference Referencesto specify the object as the first level in a referential constraint

Row privilege or Data Authority: Read data records, and/or Add, Update, Delete records. Users can be given any or all of these authorities to a data file.

OS/400Authority

SQLprivilege

this means

Read Select look at the data

Add Insertcreate new records/rows- subject to Views With Check Option

Update Update

change existing records/rows- subject to view restrictions and/or column security- SQL only: combine Select rows with Update column(s) to restrict changes to certain columns only

Delete Delete remove existing records/rows

Execute Execute run a program

Views can be used to restrict access to a selection of rows and/or columnso With Check Option prevents insert/update to rows outside of selectiono use casting on a column to prevent updating of that column:

CREATE VIEW DBT544TM2/CUSTMASTV

(CUST_NO, CSTNAM, CSTADR, CSTCTY, CSTSTP, CSTZIP,AR_BALANCE, REGION, CSTSLN) AS SELECT dec(cstnum,5,0),CSTNAM, CSTADR, CSTCTY, CSTSTP, CSTZIP, dec(YTDTOT,11,2), dec(CSTREG,3), CSTSLN FROM DBT544TM2/CUSTMAST WHERE cstreg = 10

Library/Collectionso no SQL equivalentso Grant/Revoke/Edit Object Authority (GRT/RVK/EDT OBJAUT)o requires minimum *USE (i.e. *OBJOPR + *EXECUTE)o to create a new program, table, view, etc., also requires *ADDo use the QTEMP library for temporary tables, not the collectiono other Data Authorities at the library level are not applicable

Hierarchy of Authorization Checking

1. explicit object authority by User, i.e. User ID in list of private authorities* authority specified object by object, user by user* difficult to manage: most AS/400s have many users who access many objects (this becomes an N2 problem)* least efficient, most flexible

2. Authorization List (AUTL)* only one list per object* identifies a list of User IDs and their individual authority levels* useful when a collection of people need differing authorities to a number of objects* each object refers to the Authorization List name, e.g. OPSPRF* changing the AUTL object changes the users and authorities for every object that references the list* low efficiency, very good flexibility

3. object or adopted owner's authority (high efficiency)4. Primary Group Profile

* a user profile created only for the purpose of assigning regular user profiles to it* when the user is a member of a group profile, the user has authority to any object the group profile has been granted* not as flexible as AUTL since all members of a group profile get the same authority * high efficiency if a primary group is nominated for the object (CHGOBJPGP)

5. *PUBLIC authority* applies to everyone else* highly efficient but least flexible

6. user with all object (*ALLOBJ) special authority (i.e. a "super-user")7. Other Group Profiles

* a user can belong to up to 16 group profiles (CHGUSRPRF) * all group profiles' authorities are added together* medium efficiency, similar flexibility to AUTL with combined group profiles

Objects can reference other objects for security checking.

Security setup can be simple:

many users belong to one, two or a few group profiles an object specifies authorities for one, two or a few group profiles many objects reference one object with the same security

Even if you cannot change security, you can look at it:run DSPOBJAUT (Display Object Authority).

Master & Audit Tables

A common application is an Inventory Master table containing a single row per product and a Transaction Audit table containing rows of all the quantity changes that occurred during a day's business.

Inventory Master Audit for Inv. Master & Order applications

Product IDQuantityOn Hand

Purchase/Sales Order

Product IDOrder

Quantity

ABC123 5 <= start of day

ABC123 30 PO321 ABC123 +25

ABC123 26 SO456 ABC123 -4

What minimum SQL privilege (OS/400 authority) should regular users have to these tables, rows, and columns?

privileges Inventory Master Audit for Inv. Master & Order applications

TableAll? Alter? References? With Grant Option?

All? Alter? References? With Grant Option?

Column

Product IDQuantityOn Hand

Purchase/Sales Order

Product IDOrder

Quantity

Update? Update? Update? Update? Update?

Rows

Select?Insert?Update?Delete?

Program Adopted Authority

A common task at the end of the day is moving the daily Audit table data to a history file (SQL select & insert, delete) (OS/400 CPYF & CLRPFM). This end-of-day task requires more privileges than users need to process orders. Do you change users privileges at night? Do you create a super user who runs the day end job?

OS/400 allows a program to adopt authority for the running of a program.

Steps to adopt authority

create a program to do only those actions requiring additional authority this program should not call any other programs because called programs

inherit authority create a user id (with password *NONE) give this user id minimal authority to the program and the object(s) the

program acts upon make this user id the owner of the program (CHGOBJOWN cmd) and revoke

the original owner's authority change the program to adopt the owner's authority (CHGPGM)

Now, the programmer who created the program cannot run it. The users who are allowed to run the program adopt sufficient authority from the special purpose user id for the running of this program only. No one can sign on with the special purpose user id.

Testing things out...

Sign on with the DB544A20 / A21 / A35 user ids. They all allow access to the DBT544TMS library. They allow various levels of access to the EMPLOYEE file. Try it out.

Even if you cannot change security, you can look at it:run DSPOBJAUT (Display Object Authority). Look at the DBT544TMS library object and the security on the objects within. You can also Display User Profile (DSPUSRPRF) to see what group profiles your user id belongs to.

Run the EMPDATACPY and EMPDATACPX programs in the DBT544TMS library to see how adopted authority is used. the source is in QSLPSRC. If you have signed on with a user id that allows you to change or deleted rows, be sure to run the program that refreshes the Employee file.

One thing you cannot do with OS/400 security is restrict a user to only explicitly authorized objects. Every user has *PUBLIC authority. You cannot revoke *PUBLIC authority from a user (they would not be able to run any operating system commands).

See the textbook's section on "Database Security Principles and Guidelines". It is an excellent practical summary.

Data Base Connectivity

Operations Navigator o GUI interface to the iSerieso browse the different File Systems on the iSerieso grant/revoke permissions (authority) to objectso Basic Operations lets you:

Work with messages Work with printer output, ilel output queues (and print your spool

files on your local PC printer)o System management, configuration and backup taskso easy access to File Systems, IFS (Integrated File System)

The integrated file system is a part of OS/400 that supports stream input/output and storage management similar to personal

computer and UNIX** operating systems, and provides a similar hierarchical folder structure.

ASCII (American Standard Code for Information Interchange) stream files like in DOS, OS/2 and basic UNIX systems/[root] your_directories

drag and drop files between Windows Explorer and the IFS (avoid FTP)

UNIX standard file system, POSIX and XPG compliant/QOpenSys/directories [case sensitive names]

native EBCDIC (Extended Binary Coded Decimal Interchange Code) database and library file system is a branch of the IFS/QSYS.LIB/librname.LIB/filename.FILE/mbrname.MBR

OS/400 File Server to remote iSeries servers (/QFileSvr.400), Sun's NFS (/NFS), Novell's Netware (/QNetWare), Windows NT Server (/QNTC), and optical (/QOPT) file systems are also directly supported.

limit the files that are shown for any file system or folder by right-clicking it and then clicking Include. In the Include dialog box, you can specify which files you want to see (for example, specify Dx544Ann* to see only your folders off the /root).

FTP  using native or IFS naming formatso WS_FTP opens in native format at your User Profile's current library

Change Dir to anything beginning with a "/" and you are in the IFS Root system (you cannot get back to native naming format)

o iSeries FTP server will automatically recognize the naming format you send it

o iSeries FTP client can use either format: native naming format (iSeries FTP cmd is NAmefmt 0 ):

librname/filename.mbrname IFS naming format (iSeries FTP cmd is NAmefmt 1 ):

/your_directory/filename or /QSYS.LIB/librname.LIB/filename.FILE/mbrname.MBR

o use AScii transfer mode when exchanging data with the iSeries native or IFS– iSeries will convert between ASCII and EBCDIC representations– iSeries will not change the end of line or record delimiting characters

o use Binary transfer mode when exchanging ASCII data amoung differing ASCII operating systems

this avoids the problem of the differing "end of line" or record delimiting characters used by various operating systems. On other systems, AScii transfer mode helps by changing those characters

on the receiving system. If it is always transferred in binary mode, you will have no problems with record delimiters.

End of line character(s) are: MSDOS: CR/LF (Carriage Return and Line Feed) Mac: CR Unix: LF or newline (same thing)

What if the file was transfered in a combination of Binary and AScii modes between many systems before landing in your IFS? Try this: FTP a text file from MSDOS to Unix using Binary (preserving CR/LF) then back to MSDOS using AScii mode -- you will have record delimiters of CR/CR/LF. FTP "knows" Unix does not use CR and the destination MSDOS system does, so it added a CR. (This will cause problems for the CPYFRMIMPF command.)

ODBC (Open Data Base Connectivity) driver comes with Client Access/400o generic database access from any cliento use with many client programming products, e.g. Visual Basico use with desktop and decision support products, e.g. Excel, Accesso Excel: Data \ Get External Data \ New DB Query \ Employee fileo connection type should be Read Only or Read/Call

Up/download File transfer utility in Client Access/400o creates a PC file with DB definition for upload useo transfer definitions can be saved and run by PC batch/script fileso See terminal menu: Transfer

GO DATA menu to manage Files, Libraries, Folders, Client Access tasks, and the Integrated File System.

GO FILESYS for Integrated File System commands to move data between the native database and PC/UNIX data files in the IFS

IFS and Stream Files

use Work Link (WRKLNK) to see directories and files in the IFS use Display File (DSPF) to display a stream file use Edit File (EDTF) to edit a stream file Copy To/From Stream File (CPYTOSTMF or CPYFRMSTMF)

o used to transfer 'flat files' between AS/400 native and IFSo cannot deal with packed numericso function is very similar to FTP using AScii transfer where the iSeries

automatically translates between EBCDIC/ASCII character sets.

o AS/400 native DB naming: The FROMMBR database-member-path-name must be of the format: /QSYS.LIB/librname.LIB/filename.FILE/mbrname.MBR

o e.g. to transfer a spool file: CRTPF FILE(SPOOLFILE) RCDLEN(132) CPYSPLF FILE(QSYSPRT) TOFILE(SPOOLFILE) CPYTOSTMF

FROMMBR('/qsys.lib/dbt544tm.lib/spoolfile.file/spoolfile.mbr') TOSTMF('/dbt544/spoolfile.txt')

you can now transfer the /dbt544/spoolfile.txt to your PC using FTP or Operations Navigator using drag and drop or the iSeries may serve those files directly from the IFS to other systems

Copy From Import File (CPYFRMIMPF)o copy a stream file in CSV format to an externally described database fileo useful command options:o Replace or add records . . . > *ADD, *REPLACE, *UPDADD

*UPDADD = The system updates the duplicate key records and adds the new records to the end of the existing records. (Make sure your DB file has a primary key.)

o Record delimiter . . . > *ALL *CRLF *LF *CR *LFCR *EOR*ALL = Any single or double character combination of carriage-return and line-feed.

o Copy from record number . . . > Number, *FIRSTspecify 2 as the number to avoid copying a header record with column names

o Replace null values . . . > *NO, *FLDDFT*FLDDFT = If a null value is detected when parsing an import file record, the corresponding field in the database file record is assigned a default value based on the field type or DDS/SQL defined default value.

Copy To Import File (CPYTOIMPF)o copy an externally described database file to a stream file in CSV

(Comma Separated Variable) format, also known as quote/comma delimited data.

o e.g. CPYTOIMPF FROMFILE(CUSTOMER) TOSTMF('/dbt544/customer.csv') MBROPT(*REPLACE) RCDDLM(*CRLF)

o Note the Stream file code page (STMFCODPAG) parameter...set it for *PCASCII so the characters are translated from EBCDIC to ASCII

o Note the Record delimiter (RCDDLM) parameter...set it for the end of line character(s) in the destination system:

MSDOS: *CRLF (Carriage Return and Line Feed) Mac: *CR Unix: *LF

for more information, see IFS & Copying data using CL commands

Query Manager/400

part of SQL Development Kit licensed program similar to Query/400 with more flexibility but a little less functionality specify a result table with an SQL Select statement. This query is stored as a

*QMQRY OS/400 object type. specify a report format for the result table. This form is stored as a *QMFORM

OS/400 object type. you can mix and match queries and forms. SQL definitions allow variable names which can be passed as runtime

prompted parameters &VARIABLE – & flags it as a variable, the NAME should be in upper case outputs to display, printer, or a file. STRQM Starts the Query Manager two Query creation modes are available:

o Prompted for end users and programmers developing a queryo SQL for programmers and when specifying variables

use GO CMDQM to see all the Query Management commands.

A query that prompts for a variable is not truly user friendly* users must know to input character values in single quotes* users must know how to use wildcard ( % _ ) characters in the case of LIKE comparisons

A professional implementation uses:

query definition with variable(s)

SELECT USERID, STNO, LAB1, LAB2, LAB3, LAB4, LAB5, LAB6,TEST/15*35 as Mid_term,(lab1 + lab2 + lab3 + lab4 + lab5 + lab6 + (TEST/15*35)) as TotalFROM MARKSwhere userid = &USERID or test >= &TESTGE

"MARKS" command to prompt the user for values (just like CL prompt screens)

CMD PROMPT('Check Student Marks (MARKS)') PARM KWD(USERID) TYPE(*CHAR) LEN(10) MIN(1) +

PROMPT('AS/400 User ID') PARM KWD(TESTGE) TYPE(*CHAR) LEN(9) MIN(1) + PROMPT('or Test Mark is GE to')

"MARKSCL" CL program which does the processing for the command

/* This is the processing program for the MARKS command. */ /* When compiling the MARKS command, specify the library name */ /* of this processing program (so command can be run from anywhere *//* (or ensure the processing program is always in the users' *LIBL) */ /* CRTCMD CMD(DBT544TM/MARKS) PGM(DBT544TM/MARKSCL) ... */ /* ******** */ /* PROGRAM SHOULD BE RUN ADOPTING OWNER'S AUTHORITY, */ /* I.E. MARKS FILE ACCESSABLE ONLY VIA THIS INTERFACE */ PGM PARM(&USERID &TESTGE ) /* receives parms from MARKS command */ DCL VAR(&USERID ) TYPE(*CHAR) LEN(10) DCL VAR(&USERID2) TYPE(*CHAR) LEN(12) DCL VAR(&TESTGE ) TYPE(*CHAR) LEN(9) /* must put USERID with quotes into a NEW field */ CHGVAR &USERID2 VALUE('''' *CAT &USERID *TCAT '''') /* run Query Manager query passing parameters for WHERE clause */ STRQMQRY QMQRY(DBT544TM/MARKSS) + QMFORM(DBT544TM/MARKS) + SETVAR((USERID &USERID2 ) + (TESTGE &TESTGE )) ENDPGM

see DBT544TM/QCLPSRC.MARKS cmd and MARKSCL clp see DBT544TM/MARKS* Query Manager query and form

A professional implementation also includes an automated upload and import of the MARKS data. This example uses FTP to transfer a PC Comma Separated Variable file to the native DB side of the iSeries.

PC Batch file to start FTP:

@echo off echo FTP marks from c:\DBT544\dbt544amrk.csv pause c: cd "\DBT544" ftp -n -s:"ftpDBT544marks.txt" ODIN 

The FTP "ftpDBT544marks.txt" command file:

USER DB544Ann passwordASciidelete mckenna/dbt544amrk put dbt544amrk.csv 

QUOte RCMD CPYFRMIMPF FROMFILE(MCKENNA/DBT544AMRK) TOFILE(DBT544TM2/MARKS) MBROPT(*UPDADD) RCDDLM(*ALL) RPLNULLVAL(*FLDDFT)quit

Object and Row Locking

...because everyone should share their toys and play nicely together.

Please see Chap.13 p.239-253 in the textbook.

Object Lock

application based locking ensures specific types of access to objects for your job

o e.g. allocate a table for mass update or to clear all data prevents other jobs from having inappropriate access to your job's objects

o e.g. no updates allowed during month end processing permit only one job at a time to execute a program

o e.g. only one month end processo see QCLPSRC/ONEatAtimeo test with CALL ONEATATIME ONCE

OS/400 native: ALCOBJ and DLCOBJ – Allocate/Deallocate Objecto run these sample CL programs in DBT544TM...o CALL LOCK1 on one screen and CALL LOCK2 on anothero LOCK3 is the way to allocate & deallocate a number of files separatelyo LOCK4 is the way to allocate & deallocate a set of files together

SQL: LOCK TABLE statemento released with COMMIT or ROLLBACK

DSPJOB OPTION(*JOBLCK) - (or SysReq #3, #12) to display objects and rows that are locked and the name of the job holding the lock.

DSPJOB OPTION(*CMTCTL) - to show the isolation level that the program is running, the number of rows being locked during a transaction, and the pending DDL functions.

WRKOBJLCK - work object locks to find out who has a lock on an object==> you need *JOBCTL special authority in your user profile to take any action on other jobs.

To avoid deadlock conditions where

JobA has allocated FileA and is waiting to allocate FileB …and…JobB has allocated FileB and is waiting to allocate FileA

establish a File Access Sequence.

A File Access Sequence is a sequential approach to allocating tables/files (or rows/records within) that all programmers and database people agree to follow, or else. If everyone allocates files in the same sequence, deadlock conditions cannot occur.

typical File Access Sequence

Control filescontain system wide control valuese.g. next CustomerID number, next Order Number

Audit filescontain details of the transaction that is about to take placee.g. order data: WAREHOUSE, SKU, QUANTITY, PRICE,

Master filesin sequence by foreign key dependencye.g. Customer, Customer-Phone, Customer-Accounts/Receivablee.g. Item (by SKU), Inventory (by Warehouse, SKU)

Transaction filescontain posted transactionse.g. Order Header, Order Details

Types of Object Locks

nativeALCOBJ

SQLLock Table name In

allocating job other jobs

*Excl Exclusive Mode exclusive no access

*ExclRd Exclusive Mode Allow Read exclusive update read only

*ShrUpd(share update)

(default without a Lock Table statement)

read/update read/update

*ShrNup(share no update)

Share Moderead (and update if no other job has *ShrNup)

read only

*ShrRd(share read)

(default without a Lock Table statement)

read read/update

Combination Locks

other jobs can allocate?

ALCOBJ *Excl *ExclRd *ShrUpd *ShrNup *ShrRd

Your

job's

allocation

*Excl no no no no no

*ExclRd no no no no YES

*ShrUpd no no YES no YES

*ShrNup no no no YES YES

*ShrRd no YES YES YES YES

Row Locks

transaction based locking system controls access to individual rows within a table so two applications

cannot update the same record at the same time works like *ExclRd on a row system locks row at start of I/O (read for update) system releases row at end of I/O (update) or, if no update, at start of next I/O use CHGPF to adjust the "Maximum record wait time",

i.e. system waits X seconds for another application to update a locked row

(then issues error to your job); or *IMMED for impatient applications (immediate error); or *NOMAX for users with the patience of Job or applications that cannot perform error handling

COBOL and RPG programs typically work with one record per file at a time

e.g. In an order processing application, a Control table row is locked for I/O to obtain the next Order Number and to increment/update the value for the next time.

with commitment control, row locking occurs on a set of rows SQL often works with a set of rows at a time. e.g.

Object vs. Row Locking and Commitment Control

Lock Table Customer Exclusive Mode Allow Read   Update Customer  Set Credit_Limit = Credit_Limit * 1.05      Where CR_Rating = 'GOOD'Commit

The above object locking prevents anyone else from modifying any and all rows in the table. However, this may be too restrictive. If the table is in constant use by other applications, your SQL statement may take all day to obtain the object lock. If you do get the lock, no one else may be able to do any work until your job completes.

SET TRANSACTION ISOLATION LEVEL NO COMMITUpdate Customer  Set Credit_Limit = Credit_Limit * 1.05   Where CR_Rating = 'GOOD'

Without the Lock Table statement or commitment control, what happens if SQL gets half way through the file then tries to allocate a row that another application has locked and is taking a long time to update? This can easily happen if a user calls up a customer record with DFU, then walks away from the screen without pressing Enter.

In the above scenario, some of the records were updated, then your SQL statement failed. After you use WRKOBJLCK to track down the DFU user, how can your SQL statement pick up where it left off? Well, it can't. You are in trouble, and looking for backup tapes, unless you use commitment control.

SET TRANSACTION ISOLATION LEVEL READ COMMITTED   Update Customer  Set Credit_Limit = Credit_Limit * 1.05

      Where CR_Rating = 'GOOD'Commit

Using row locking and commitment control, the system shares the rows in the table with other applications but allows only one at a time to update a given row or a set of rows if under commitment control. In this SQL statement, the changes are preserved when Commit is done; other applications can now update the locked rows. If something goes wrong, Rollback can be performed which reverses this statement's updates to the table rows.

Commitment Control and Isolation Levels

Isolation Level your SQL job data from other SQL jobs

*NONE or *NC orNO COMMIT

no control(Lock Table may be used to ensure integrity at the object level)

Locked rows and uncommitted changes in other jobs can be seen. These are "dirty reads" which may be rolled back.

*CHG or *UR orREAD UNCOMMITTED, READ WRITE

changed objects with DDL or changed rows with DMLare locked until Commit/Rollback* Rows are not locked during update process.

allows "dirty reads"

*CS (Cursor Stability) orREAD COMMITTED

changed objects with DDL or changed rows with DMLare locked until Commit/Rollback* Rows are locked during update process.

Only committed data can be seen. Data is read on a row by row basis.

Your nested Select statements may not be congruent with the outer Select statements. Some rows were not committed by the other job and the updates not read by your nested Select, however the changes were commited by the time the outer Select ran.

*ALL or *RS (Read Stability) orREPEATABLE READ

as above Only committed data can be seen. Any data your unit of work read or updated cannot be changed by other jobs until your unit of work is committed.

Your nested Select statements may not be congruent with the outer Select statements. New rows can be inserted by other jobsat any time and, if committed, read by your job.

*RR (Repeatable Read) orSERIALIZABLE

as above with object locks of *ShrNup on read-only tables and *ExclRd on updated tables.

Only committed data can be seen. No changes are allowed by other jobs to tables in your work unit.

In general, use Read Committed or Repeatable Read if your work unit re-reads rows in the same transaction. See the IBM Manual for more information on Isolation Levels

Use the COMMIT parameter on the CRTSQLxxx, STRSQL, and RUNSQLSTM commands to specify the default isolation level.

Use the SET OPTION statement to specify the default isolation level within the source of a module or program that contains embedded SQL.

Use the SET TRANSACTION statement to override the default isolation level within a unit of work. When the unit of work ends, the isolation level returns to the value it had at the beginning of the unit of work. Do not use this statement inside an SPL procedure – it will conflict with the caller's commitment control.

Use the isolation-clause on the SELECT, SELECT INTO, INSERT, UPDATE, DELETE, and DECLARE CURSOR statements to override the default isolation level for a specific statement or cursor. The isolation level is in effect only for the execution of the statement containing the isolation-clause and has no effect on any pending changes in the current unit of work.

Functions and Procedures

See examples in DBT544TM/QSQLSRC.

Note: if a function is called with mismatching parameters,the system issues this confusing error message:

function_name in *LIBL type *N not found.

This is because the system looks for a function according to its name and its parameters. The parameters form a signature similar to a file's level check which is based on the file's projection. That is, the number, type and length of all parameters (or fields) must match.

DBT544 Backup

Save Library (SAVLIB) normally backs up to a tape device but it can also backup to disk using a Save File (SAVF).

The AS/400 provides for a Save File object to contain a backup. To create the *SAVF object:

CRTSAVF “SaveFileName”

Save all objects in your library using the following command:

SAVOBJ OBJ(*ALL) LIB(yourLibraryName) DEV(*SAVF) SAVF(SaveFileName)

You can FTP the Save File—using Binary transfer—to any other system. e.g. your PC. Retain the AS/400 file naming.

To reload the file on the same, or even another, system...

Login to the target AS/400 (or iSeries) machine.

Create a Save File with the original name:

CRTSAVF “SaveFileName”

FTP your Save File to the LIBRARY in the target system containing the empty Save File . *please make sure you are transferring files in binary mode*

Once you have transferred the file, use the following command to restore them.

RSTOBJ OBJ(*ALL) SAVLIB(yourlibraryname) DEV(*SAVF) Prompt this command to override the library where the saved objects will be restored.

DB Overrides and Redirection

Procedures and Functions can be found in your library list if ISQL is using *SYS Naming Convention. However, any unqualified table names in those Procedures and Functions are looked for in the default collection in force at the time the Create Procedure/Function statement was run.

The EMPRESET procedure in DBT544TM updates a table called EMPLOYEE and was created with this statement:

===> RUNSQLSTM SRCFILE(DBT544TM/QSQLSRC) SRCMBR(EMPRESET) COMMIT(*CHG) NAMING(* SYS) DFTRDBCOL(DBT544TM)

If you need to run this procedure against an EMPLOYEE table in a collection other than DBT544TM, you could recreate the procedure specifying another collection. However, this gives us a maintenance problem if the procedure ever needed changing and creates unnecessary objects on the system.

Use the OS/400's Override with Data Base File (OVRDBF) command to temporarily use a different library and/or table name than was specified in your procedure, or QM query, or COBOL program...

OVRDBF FILE(EMPLOYEE) TOFILE(DBT544TM2/EMPLOYEE)

STRSQL  call empreset

You are telling the system that, following OVRDBF, any file/table named EMPLOYEE will be referred to DBT544TM2/EMPLOYEE. It is up to the programmer to ensure that the overriding file/table has the same projection as the overridden file/table.

The OVRDBF cmd is used frequently in CL programming. It is also used to change *FILE object properties temporarily. e.g.OVRDBF FILE(EMPLOYEE) FRCRATIO(1) WAITFILE(*IMMED) WAITRCD(*IMMED)

The OVRDBF cmd is used prior to running SQL statements when you need to redirect SQL to use anything but the first member in a table. Normally, SQL creates only single member tables. Most DB designers follow this practice. Sometimes the rule may be broken or you need to use SQL to access a file with multiple members (such as a PF-SRC). Try using a combination of OVRDBF and SQL to list the contents of different source members in one of your source files.

Always DLTOVR as soon as you are done with the override.

Use DSPOVR to display overrides still in effect.

Query Optimization

To find out what the system is doing and what you could do better, use the STRDBG command before you run a query with Query/400 (RUNQRY), or Query Manager (STRQMQRY), or SQL.

In the job log (DSPJOBLOG or use the SysReq key to get to it), you will find this msg:**** Starting optimizer debug message for query .

followed by a series of messages telling you what the system did and how you might improve performance.

Remember to run ENDDBG after you are done.

DBT544 Exam notes

Format is similar to the Mid-Term test:

create a collection run a native command which will:

o create tables in your collectiono add objects to the collectiono create a source member in the collection for your answers

Know how to

browse an IFS directory display an IFS file recognize delimiters for a string and field separators use 'import file' commands to transfer data TO and FROM CSV and database

files Write a Query Manager Query with user prompted variables. You may have to

create a QM form to use with the QM Query. use a command to run the QM Query (with the form you created) passing the

user variableso Note: you will not have to create a CL program and CMD to run a QM

query use DB overrides (redirection) use commitment control

DB Integrity (object and row locking): understand the different ways to enforce appropriate sharing/allocation of files/tables. There are native and SQL ways.

Functions and Procedure coding: I will supply a shell, you read the comments and complete/debug the code.