9780970498069 Chapter 9 Teradata Utilities

of 59 /59
Teradata User's Guide: The Ultimate Companion, Third Edition by Tom Coffing Coffing Data Warehousing. (c) 2006. Copying Prohibited. Reprinted for vramanathan vramanathan, CSC [email protected] Reprinted with permission as a subscription benefit of Skillport, http://skillport.books24x7.com/ All rights reserved. Reproduction and/or distribution in whole or in part in electronic,paper or other forms without written permission is prohibited.

Embed Size (px)

Transcript of 9780970498069 Chapter 9 Teradata Utilities

Teradata User's Guide: The Ultimate Companion, Third Editionby Tom Coffing Coffing Data Warehousing. (c) 2006. Copying Prohibited.

Reprinted for vramanathan vramanathan, CSC [email protected] Reprinted with permission as a subscription benefit of Skillport, http://skillport.books24x7.com/

All rights reserved. Reproduction and/or distribution in whole or in part in electronic,paper or other forms without written permission is prohibited.

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Chapter 9: Teradata UtilitiesAn Introduction to the Teradata Utilities "It's not the data load that breaks us down, it's the way you carry it." Tom Coffing Teradata has been doing data transfers to and from the largest data warehouses in the world for close to two decades. While other databases have allowed the loads to break them down, Teradata has continued to set the standards and break new barriers. The brilliance behind the Teradata load utilities is in their power and flexibility. With five great utilities Teradata allows you to pick the utility for the task at hand. This book is dedicated to explaining these utilities in a complete and easy manner. This book has been written by Five Teradata Certified Masters with experience at over 125 Teradata sites worldwide. Let our experience be your guide. The intent of this book is to twofold. The first is to help you write and use the various utilities. A large part of this is taken up with showing the commands and their functionality. In addition, it is shows examples using the various utility commands and SQL in conjunction with each other that you will come to appreciate. The second intention is to help you know which utility to use under a variety of conditions. You will learn that some of the utilities use very large blocks to transfer the data either to or from the Teradata Relational Database Management System (RDBMS). From this perspective, they provide a high degree of efficiency using a communications path of either the mainframe channel or network. The other approach to transferring data rows either to or from the Teradata RDBMS is a single row at a time. The following sections provide a high level introduction to the capabilities and considerations for both approaches. You can use this information to help decide which utilities are appropriate for your specific need.Considerations for Using Block at a Time Utilities

As mentioned above, there are efficiencies associated with using large blocks of data when transferring between computers. So, the logic might indicate that it is always the best approach. However, there is never one best approach. You will learn that efficiency comes at the price of other database capabilities. For instance, when using large blocks to transfer and incorporate data into Teradata the following are not allowed:n

Secondary indices Triggers Referential integrity More than 15 concurrent utilities running at the same time

n

n

n

Therefore, it is important to understand when and where these considerations are present. So, as important as it is to know the language of the utility and database, it is also important to understand when to use the appropriate utility. The capabilities and considerations are covered in conjunction with the commands.Considerations for Using Row at a Time Utilities

The opposite of sending a large block of rows at the same time is sending a single row at a time. The primary difference in these approaches is speed. It is always faster to send multiple rows in one operation instead of one row. If it is slower, why would anyone ever use this approach? The reason is that it provides more flexibility with fewer considerations. By this, we mean that the row at a time utilities allow the following:n

Secondary indices Triggers Referential integrity

n

n

Page 2 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

n

More than 15 concurrent utilities running at the same time

As you can see, they allow all the things that the block utilities do not. With that in mind and for more information, continue reading about the individual utilities and open up a new world of capabilities in working with the Teradata RDBMS. Welcome to the world of the Teradata Utilities. An Introduction to BTEQWhy it is Called BTEQ?

Why is BTEQ available on every Teradata system ever built? Because the Batch TEradata Query (BTEQ) tool was the original way that SQL was submitted to Teradata as a means of getting an answer set in a desired format. This is the utility that I used for training at Wal*Mart, AT&T, Anthem Blue Cross and Blue Shield, and SouthWestern Bell back in the early 1990's. BTEQ is often referred to as the Basic TEradata Query and is still used today and continues to be an effective tool. Here is what is excellent about BTEQ:n

BTEQ can be used to submit SQL in either a batch or interactive environment. Interactive users can submit SQL and receive an answer set on the screen. Users can also submit BTEQ jobs from batch scripts, have error checking and conditional logic, and allow for the work to be done in the background. BTEQ outputs a report format, where Queryman outputs data in a format more like a spreadsheet. This allows BTEQ a great deal of flexibility in formatting data, creating headings, and utilizing Teradata extensions, such as WITH and WITH BY that Queryman has problems in handling. BTEQ is often used to submit SQL, but is also an excellent tool for importing and exporting data.

n

n

Importing Data: Data can be read from a file on either a mainframe or LAN attached computer and used for substitution directly into any Teradata SQL using the INSERT, UPDATE or DELETE statements. Exporting Data: Data can be written to either a mainframe or LAN attached computer using a SELECT from Teradata. You can also pick the format you desire ranging from data files to printed reports to spread sheet formats.

There are other utilities that are faster than BTEQ for importing or exporting data. We will talk about these in future chapters, but BTEQ is still used for smaller jobs. Logging onto BTEQ "It's choice not change that determines your destiny." Jean Nidetch By taking a chance in this industry, you've chosen to arm yourself with an unlimited arsenal of knowledge. But you can't use that knowledge if you can't log onto the system! This next slide is going to teach you how to logon to BTEQ. Remember that you will be prompted for the password since it's an interactive interface. BTEQ commands begin with a period (.) and do not require a semi-colon (;) to end the statement. SQL commands do not ever start with a period and they must always be terminated with a semi-colon. Let's logon to BTEQ and show all information in the Employee_Table:

Page 3 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Before you can use BTEQ, you must have user access rights to the client system and privileges to the Teradata DBS. Normal system access privileges include a userid and a password. Some systems may also require additional user identification codes depending on company standards and operational procedures. Depending on the configuration of your Teradata DBS, you may need to include an account identifier (acctid) and/or a Teradata Director Program Identifier (TDPID). Using BTEQ to submit queriesSubmitting SQL in BTEQ's Interactive Mode

Once you logon to Teradata through BTEQ, you are ready to run your queries. Teradata knows the SQL is finished when it finds a semi-colon, so don't forget to put one at the end of your query. Below is an example of a Teradata table to demonstrate BTEQ operations. Employee_Tableyee_No 2000000 1256349 1333454 1121334 Last_Name Jones Harrison Smith Strickling First_Name Squiggy Herbert John Cletus Salary 32800.50 54500.00 48000.00 54500.00 Dept_No ? 400 200 400

Figure 2-1 BTEQ execution.LOGON cdw/sql01 Password: XXXXX Enter your BTEQ/SQL Request or BTEQ Command. SELECT * FROM Employee_Table WHERE Dept_No = 400; Type at command prompt: Logon with TDPID and USERNAME. Then enter PASSWORD at the second prompt. BTEQ will respond and is waiting for a command.

Page 4 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

*** Query Completed. 2 rows found. 5 Columns returned. An SQL Statement *** Total elapsed time was 1 second. Employee_No Last_name First_name Salary Dept_No 1256349 Harrison Herbert 54500.00 400 1121334 Strickling Cletus 54500.00 400 BTEQ displays information about the answer set. The result set

WITH BY Statement "Time is the best teacher, but unfortunately, it kills all of its students." Robin Williams Investing time in Teradata can be a killer move for your career. We can use the WITH BY statement in BTEQ, whereas we cannot use it with Nexus or SQL Assistant. The WITH BY statement works like a correlated subquery in the fact that you can us aggregates based on a distinct column value. BTEQ has the ability to use WITH BY statements:

"I've learned that you can't have everything and do everything at the same time." Oprah Winfrey The great thing about the WITH statement is that you can do everything to a specific group while having everything done to a column as a whole. We can get a grand total or an overall average with the WITH statement, just leave out BY. Here's a good example:

Page 5 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Using WITH on a whole column:

Transactions in Teradata Mode "He who every morning plans the transaction of the day and follows out that plan, carries a thread that will guide him through the maze of most busy life." - Victor Hugo Victor couldn't have summed up Teradata any better. However, Victor did seem more worried about the hunchback than the rollback. Turning your queries into a single transaction is often the best plan, but can sometimes make one Miserables. Often in Teradata we'll see multiple queries within the same transaction. We can use the BT / ET keywords to bundle several queries into one transaction. You also need to end every query with a semi-colon (;), which isn't the case in Nexus or SQL assistant. For example: In Teradata mode, we're going to put four single statements into a single transaction

Page 6 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

"What is defeat? Nothing but education; nothing but the first step to something better." Wendell Phillips The final query in our last transaction is what caused our updates to fail. This was not the sweet taste of victory, but instead the smell of de Feet! Actually, it was an education that is the step to something better. When using BT / ET in your transaction, you're telling Teradata that when it comes to committing, we either want all or none. Since our last query in the transaction failed the Transient Journal rolled back all the queries in our entire transaction. Make sure that your syntax is correct when using the method of BT and ET because a mistake causes a massive rollback. The last query in our set did not work:

Now let's take a look at the Employee_Table:

Page 7 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Our updates didn't work! That's because we bundled all four queries into one transaction. Since our last query failed, the tables were rolled back to their original state before the transaction took place.

Alternative Transactions in Teradata Mode "It's not enough that we do our best; sometimes we have to do what's required." Sir Winston Churchill Sometimes we're required to use an alternative method to get the job done if we want to win like Winston. Here's another way to set up a bundle of queries into one transaction. Notice where we place the semi-colon in our queries and you will understand this technique. Remember that the semi-colon must be at the very beginning of the next line for a query to be considered as part of the same transaction. Because we are in Teradata mode if any query fails then all queries that are part of the same transaction roll back. How many queries are parts of the same transaction below? Four! Another way to perform a multi-statement transaction in Teradata mode:

Placing the semi-colon at the beginning of the next line (followed by another statement) will bundle those statements together as one transaction. Notice that our Employee_Table was not updated, just like in the first example.

Transactions in ANSI Mode "The man who views the world at 50 the same as he did at 20 has wasted 30 years of his life." - Muhammad Ali ANSI (American National Standard Institution) allows us to view the same queries in a different way. To change to ANSI mode, simple type '.set session transaction ANSI' and be sure to do it before you actually logon to BTEQ. Then, you can logon like you always do, but you will be in ANSI mode. All queries in ANSI mode will also work in Teradata mode and vice versa. However, three things will be different in ANSI mode versus Teradata mode. Those things are how case sensitivityPage 8 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

is handled, how transactions are committed and rolled back, and how truncation is accepted. Let's log back onto BTEQ, but this time change it to ANSI mode:

"Be not afraid of growing slowly, be afraid only of standing still." - Chinese Proverb Remember the first rule of ANSI mode: all transactions must be committed by the user actually using the word 'COMMIT'. Also, in ANSI mode after any DDL statement (CREATE, DROP, ALTER, DATABASE) we have to use the 'commit' command immediately. This tells Teradata to commit to what's been done. Our query below will attempt to find anyone with a last_name of 'larkins'. It will fail even though we have 'Mike' 'Larkins' in our table. This is because ANSI is case sensitive and we did not capitalize the 'L' in 'Larkins'. Let's run a few queries in ANSI mode:

Notice that we have to COMMIT after any DDL or Update before the transaction is committed. We even have to COMMIT after setting our DATABASE or we will get an error. We didn't have any rows return, but we know there's a Mike Larkins within the table. That's because BTEQ is case sensitive. Change 'larkins' to 'Larkins'.

Rollback

Page 9 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

"Insanity: doing the same thing over and over again and expecting different results." Albert Einstein The Rollback keyword is the SQL mulligan of Teradata. Rollback will erase any changes made to a table. This can be very useful if something didn't work. However, you cannot rollback once you've used the commit keyword. Not keeping rollback in your arsenal would be insane.

Advantages to ANSI Mode ANSI mode is great because when you bundle several queries into one transaction and one doesn't work, the rest won't be rolled back to their original state. Using commit will ensure that your successes aren't hidden by your failures. Now notice that I will have multiple statements in the same transaction and that I purposely fail the last SQL statement:

Page 10 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Which statements were rolled back?

"All truths are easy to understand once they are discovered; the point is to discover them." Galileo Galilei Discovering the advantages in using ANSI will only make SQL easier to write. It might take a little bit more typing, but a little work now can save you lots of time later. The Employee_Table was updated!

In ANSI mode, only failed transactions are rolled back when it comes to multi-statement transactions.

Creating a Batch Script for BTEQ "The cure for boredom is curiosity. There is no cure for curiosity." Dorothy Parker If you've been bored waiting for your queries to finish, then I'm sure you're curious on how we can fix the situation. Batch scripting allows us to write out pages and pages of queries and execute those queries in one single swoop. BTEQ can also run in batch mode under UNIX (IBM AIX, Hewlett-Packard HP-UX, NCR MP-RAS, Sun Solaris), DOS, Macintosh, Microsoft Windows and OS/2 operating systems. To submit a job in batch mode, do the following: 1. Invoke BTEQ (using dos prompt) 2. Type in the input file name 3. Type in the location and output file name. The following example shows how to create a batch script and how to invoke the script using BTEQ from a DOS command.Page 11 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

When using Batch scripting, your password will not be prompted. Instead, just add the password after your login name, and a comma separates the two. Be sure to end with either a .quit or .logoff so that your queries aren't left hanging. Simply open up notepad and type in the following, then save it. I recommend calling it 'BTEQ_First_Batch_Script.txt' and save it in the C:\Temp folder. However, as long as you can remember what you named it and where you saved it, you'll be fine. Be sure that you save it as a .txt file. Using Batch scripting with BTEQ

Running your Batch Script in BTEQ "I do not fear computers. I fear the lack of them." Isaac Asimov The BTEQ utility enables us to run our scripts in batch mode. To run our new batch script, we have to access the BTEQ utility via dos prompt. Simply use command prompt to access the utility, and follow the steps below: Let's run our query in Batch!

Once you're in dos, type in the following: 'BTEQ < c:\temp\BTEQ_First_Script.txt', then hit enter. BTEQ will automatically open in dos, and then it will access the file from the location you listed.Page 12 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Results from a BTEQ Batch Script "Don't be afraid to take a big step when one is indicated. You can't cross a chasm in two small steps." -David Lloyd George BTEQ will run your query in steps to produce the answer you're looking for. Whether you're accessing a small table or crossing over a chasm of information, BTEQ will ensure that the steps it takes will be big enough to get the job done. Our results are returned Interactively

Placing Our BTEQ Output to a File "The secret to creativity is knowing how to hide your sources." Albert Einstein We can use BTEQ to export our results to another text document. Exporting data also works very well when you're trying to document your query along with the results. We can export our results in batch as well

Page 13 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Notice that the BTEQ command is immediately followed by the 'BTEQ_First_Export.txt' names the file where the output messages are written. Since putting password information into a script is scary for security reasons, inserting the password directly into a script that is to be processed in batch mode may not be a good idea. It is generally recommended and a common practice to store the logon and password in a separate file that that can be secured. That way, it is not in the script for anyone to see. For example, the contents of a file called "mylogon.txt" might be: '.LOGON cdw/sql00,whynot'. Then, the script should contain the following command instead of a .LOGON: .RUN FILE=c:\temp\mylogon.txt This command opens and reads the file. It then executes every record in that file. Reading Out BTEQ Output from the Text File "The more original a discovery, the more obvious it seems afterwards." Arthur Koestler Discovering how easy it is to export your data in batch mode is a key step in learning Teradata utilities. Here are our results, including the original query and what BTEQ did to generate its answer set. Simply go to the folder where you saved the exported data (the previous examples saved the file as c:\temp\BTEQ_First_Export.txt). What you'll find in our new text document

Page 14 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Using BTEQ Conditional Logic Below is a BTEQ batch script example. The initial steps of the script will establish the logon, the database, and then delete all the rows from the Employee_Table. If the table does not exist, the BTEQ conditional logic will instruct Teradata to create it. However, if the table already exists, then Teradata will move forward and insert data. Note: In script examples, the left panel contains BTEQ base commands and the right panel provides a brief description of each command..RUN FILE = c:\temp\mylogon.txt DATABASE SQL_Class; DELETE FROM Employee_Table; .IF ERRORCODE = 0 THEN .GOTO INSEMPS [*] /* ERRORCODE is a reserved word that contains the outcome status for every SQL statement executed in BTEQ. A zero (0) indicates that statement worked. */ CREATE TABLE Employee_Table (Employee_No INTEGER, Last_name CHAR(20), First_name CHAR(12), Salary DECIMAL(8,2), Dept_No SMALLINT) UNIQUE PRIMARY INDEX (Employee_No); BTEQ conditional logic that will check to ensure that the delete worked or if the table even existed. If the table did not exist, then BTEQ will create it. If the table does exist, the Create table step will be skipped and directly GOTO INSEMPS.

The Label INSEMPS provides code so the BTEQ Logic can go directly to inserting records into the Employee_Table.

Once the table has been created, Teradata will .LABEL INSEMPS[*] then insert the two new rows into the empty table. INSERT INTO Employee_Table (1232578, 'Chambers', 'Mandee', 48850.00, 100); INSERT INTO Employee_Table (1256349, 'Harrison', 'Herbert', 54500.00, 400); .QUIT[*]

Both labels have to be identical or it will not work.

Using BTEQ to Export DataPage 15 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

"The trouble with facts is that there are so many of them." Samuel McChord Crothers Creating flat files is one of the most important tasks in Teradata, and that's a fact. BTEQ allows data to be exported directly from Teradata to a file on a mainframe or network-attached computer. In addition, the BTEQ export function has several export formats that a user can choose from depending on the desired output. Generally, users will export data to a flat file format that is composed of a variety of characteristics. These characteristics include: field mode, indicator mode, or dif mode. Syntax of a basic EXPORT command:.EXPORT FILE =

Creating a flat file of what's on the Employee_Table

Executing our BTEQ Script to Export Data "The past is a foreign country; they do things differently there." L. P. Hartley Transferring data from table to another without the use of a flat file is a thing of the past. Teradata does things differently now, which is why it's still considered the future of data warehousing. The flat files we create are merely used to store information contained within a table. The information on these files is written in binary code, which is why the text seems garbled. It may look garbled, but it is perfectly written. When we Fastload the data back to a table it will look beautiful. Executing our fastload_creating_flatfile01.txt

What our flat file looks like:

Page 16 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

And I thought French was tough; it's like they have a different word for everything

We now have a flat file that contains all information found in the Employee_Table. We will be able to use this flat file for future exercises. BTEQ Export Modes Explained Below is a list and description of our three data modes: Record Mode: (also called DATA mode): This is set by .EXPORT DATA. This will bring data back as a flat file. Each parcel will contain a complete record. Since it is not a report, there are no headers or white space between the data contained in each column and the data is written to the file (e.g., disk drive file) in native format. For example, this means that INTEGER data is written as a 4-byte binary field. Therefore, it cannot be read and understood using a normal text editor. Field Mode (also called REPORT mode): This is set by .EXPORT REPORT. This is the default mode for BTEQ and brings the data back as if it was a standard SQL SELECT statement. The output of this BTEQ export would return the column headers for the fields, white space, expanded packed or binary data (for humans to read) and can be understood using a text editor. Indicator Mode: This is set by .EXPORT INDICDATA. This mode writes the data in data mode, but also provides host operating systems with the means of recognizing missing or unknown data (NULL) fields. This is important if the data is to be loaded into another Relational Database System (RDBMS). The issue is that there is no standard character defined to represent either a numeric or character NULL. So, every system uses a zero for a numeric NULL and a space or blank for a character NULL. If this data is simply loaded into another RDBMS, it is no longer a NULL, but a zero or space. To remedy this situation, INDICATA puts a bitmap at the front of every record written to the disk. This bitmap contains one bit per field/column. When a Teradata column contains a NULL, the bit for that field is turned on by setting it to a "1". Likewise, if the data is not NULL, the bit remains a zero. Therefore, the loading utility reads these bits as indicators of NULL data and identifies the column(s) as NULL when data is loaded back into the table, where appropriate. Since both DATA and INDICDATA store each column on disk in native format with known lengths and characteristics, they are the fastest method of transferring data. However, it becomes imperative that you be consistent. When it is exported as DATA, it must be imported as DATA and the same is true for INDICDATA. Again, this internal processing is automatic and potentially important. Yet, on a network-attached system, being consistent is our only responsibility. However, on a mainframe system, you must account for these bits when defining the LRECL in the Job Control Language (JCL). Otherwise, your length is too short and the job will end with an error. To determine the correct length, the following information is important. As mentioned earlier, one bit is needed per field output onto disk. However, computers allocate data in bytes, not bits. Therefore, if one bit is needed a minimum of eight (8 bits per byte) are allocated. Therefore, for every eight fields, the LRECL becomes 1 byte longer and must be added. In other words, for nine columns selected, 2 bytes are added even though only nine bits are needed. With this being stated, there is one indicator bit per field selected. INDICDATA mode gives the Host computer the ability to allocate bits in the form of a byte. Therefore, if one bit is required by the host system, INDICDATA mode will automatically allocate eight of them. This means that from one to eight columns being referenced in the SELECT will add one byte to thePage 17 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

length of the record. When selecting nine to sixteen columns, the output record will be two bytes longer. When executing on non-mainframe systems, the record length is automatically maintained. However, when exporting to a mainframe, the JCL (LRECL) must account for this additional 2 bytes in the length. DIF Mode: Known as Data Interchange Format, which allows users to export data from Teradata to be directly utilized for spreadsheet applications like Excel, FoxPro and Lotus. The optional LIMIT is to tell BTEQ to stop returning rows after a specific number (n) of rows. This might be handy in a test environment to stop BTEQ before the end of transferring rows to the file. BTEQ EXPORT Example Using Record (DATA) Mode The following is an example that displays how to utilize the export Record (DATA) option. Notice the periods (.) at the beginning some of script lines. A period starting a line indicates a BTEQ command. If there is no period, then the command is an SQL command. When doing an export on a mainframe or a network-attached (e.g., LAN) computer, there is one primary difference in the .EXPORT command. The difference is the following:n

Mainframe syntax: LAN syntax:

.EXPORT DATA DDNAME = data definition statement name (JCL) .EXPORT DATA FILE = actual file name

n

BTEQ Return Codes Return codes are two-digit values that BTEQ returns to the user after completing each job or task. The value of the return code indicates the completion status of the job or task as follows: Return Code Descirptionn

00 Job completed with no errors. 02 User alert to log on to the Teradata DBS. 04 Warning error. 08 User error. 12 Severe internal error.

n

n

n

n

You can over-ride the standard error codes at the time you terminate BTEQ. This might be handy for debug purposes. The error code or "return code" can be any number you specify using one of the following: Override Code Descriptionn

.QUIT 15 .EXIT 15

n

BTEQ Commands The BTEQ commands in Teradata are designed for flexibility. These commands are not used directly on the data inside the tables. However, these 60 different BTEQ commands are utilized in four areas.n

Session Control Commands File Control Commands Sequence Control Commands Format Control Commands

n

n

n

Page 18 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Session Control Commands ABORT DEFAULTS EXIT HALT EXECUTION LOGOFF LOGON QUIT SECURITY SESSIONS SESSION CHARSET SESSION SQLFLAG SESSION TRANSACTION SHOW CONTROLS SHOW VERSIONS TDP Abort any and all active running requests and transactions for a session, but do not exit BTEQ. Reset all BTEQ Format command options to their defaults. This will utilize the default configurations. Immediately end the current session or sessions and exit BTEQ. Abort any and all active running requests and transactions and EXIT BTEQ. End the current session or sessions, but do not exit BTEQ. Starts a BTEQ Session. Every user, application, or utility must LOGON to Teradata to establish a session. End the current session or sessions and exit BTEQ. Specifies the security level of messages between a network-attached system and the Teradata Database. Specifies the number of sessions to use with the next LOGON command. Specifies the name of a character set for the current session or sessions. Specifies a disposition of warnings issued in response to violations of ANSI syntax. The SQL will still run, but a warning message will be provided. The four settings are FULL, INTERMEDIATE, ENTRY, and NONE. Specifies whether transaction boundaries are determined by Teradata SQL or ANSI SQL semantics. Displays all of the BTEQ control command options currently configured. Displays the BTEQ software release versions. Used to specify the correct Teradata server for logons for a particular session.

File Control Commands These BTEQ commands are used to specify the formatting parameters of incoming and outgoing information. This includes identifying sources and determining I/O streams.CMS ERROROUT EXPORT HALT EXECUTION FORMAT IMPORT INDICDATA OS QUIET RECORDMODE REPEAT RUN TSO Execute a VM CMS command inside the BTEQ environment. Write error messages to a specific output file. Open a file with a specific format to transfer information directly from the Teradata database. Abort any and all active running requests and transactions and EXIT BTEQ. Enable/inhibit the page-oriented format command options. Open a file with a specific format to import information into Teradata. One of multiple data mode options for data selected from Teradata. The modes are INDICDATA, FIELD, or RECORD MODE. Execute an MS-DOS, PC-DOS, or UNIX command from inside BTEQ. Limit BTEQ output displays to all error messages and request processing statistics. One of multiple data mode options for data selected from Teradata. (INDICDATA, FIELD, or RECORD). Submit the next request a certain amount of times Execute Teradata SQL requests and BTEQ commands directly from a specified run file. Execute an MVS TSO command from inside the BTEQ environment.

Sequence Control Commands These commands control the sequence in which Teradata commands operate.ABORT ERRORLEVEL Abort any active transactions and requests. Assign severity levels to particular error numbers.

Page 19 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

EXIT GOTO HANG IFTHEN LABEL MAXERROR QUIT REMARK REPEAT

End the current session or sessions and exit BTEQ. Skip all intervening commands and resume after branching forward to the specified label. Pause BTEQ processing for a specific amount of time. Test a stated condition, and then resume processing based on the test results. The GOTO command will always GO directly TO a particular line of code based on a label. Specifies a maximum allowable error severity level. End the current session or sessions and exit BTEQ. Place a comment on the standard output stream. Submit the next request a certain amount of times.

Format Control Commands These commands control the formatting for Teradata and present the data in a report mode to the screen or printer.DEFAULTS ECHOREQ EXPORT FOLDLINE FOOTING FORMAT IMPORT INDICDATA NULL OMIT PAGEBREAK PAGELENGTH QUIET RECORDMODE RETCANCEL RETLIMIT RETRY RTITLE SEPARATOR SHOWCONTR OLS SIDETITLES SKIPLINE SUPPRESS TITLEDASHES UNDERLINE WIDTH Reset all BTEQ Format command options to their defaults. This will utilize the default configurations. Enable the Echo required function in BTEQ returning a copy of each Teradata SQL request and BTEQ command to the standard output stream. Open a file with a specific format to transfer information directly from the Teradata database. Split or fold each line of a report into multiple lines. Specify a footer to appear at the bottom of every report page. Enable/inhibit the page-oriented format command options. Open a file with a specific format to transfer or IMPORT information directly to Teradata. One of multiple data mode options for data selected from Teradata. The modes are INDICDATA, FIELD, or RECORD MODE. Specifies a character or string of characters to represent null values returned from Teradata. Omit specific columns from a report. Ejects a page whenever a specified column changes values. Specifies the page length of printed reports based on lines per page. Limit BTEQ output displays to all error messages and request processing statistics. One of multiple data mode options for data selected from Teradata. (INDICDATA, FIELD, or RECORD). Cancel a request when the specified value of the RETLIMIT command option is exceeded. Specifies the maximum number of rows to be displayed or written from a Teradata SQL request. Retry requests that fail under specific error conditions. Specify a header appearing at the top of all pages of a report. Specifies a character string or specific width of blank characters separating columns of a report. Displays all of the BTEQ control command options currently configured. Place titles to the left or side of the report instead of on top. Inserts blank lines in a report when the value of a column changes specified values. Replace each and every consecutively repeated value with completely-blank character strings. Display dash characters before each report line summarized by a WITH clause. Display a row of dash characters when the specified column changes values. Specifies the width of screen displays and printed reports, based on characters per line.

An Introduction to FastExportWhy it is Called "FAST" Export

Page 20 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

FastExport is known for its lightning speed when it comes to exporting vast amounts of data from Teradata and transferring the data into flat files on either a mainframe or network-attached computer. In addition, FastExport has the ability to use OUTMOD routines, which provide the user the capability to write, select, validate, and preprocess the exported data. Part of this speed is achieved because FastExport takes full advantage of Teradata's parallelism. In this book, we have already discovered how BTEQ can be utilized to export data from Teradata in a variety of formats. As the demand increases to store data, the ever-growing requirement for tools to export massive amounts of data also increases. This is the reason why FastExport (FEXP) is brilliant by design. A good rule of thumb is that if you have more than half a million rows of data to export to either a flat file format or with NULL indicators, then FastExport is the best choice to accomplish this task. Keep in mind that FastExport is designed as a one-way utility that is, the sole purpose of FastExport is to move data out of Teradata. It does this by harnessing the parallelism that Teradata provides. FastExport is extremely attractive for exporting data because it takes full advantage of multiple sessions, which leverages Teradata parallelism. FastExport can also export from multiple tables during a single operation. In addition, FastExport utilizes the Support Environment, which provides a job restart capability from a checkpoint if an error occurs during the process of executing an export job.How FastExport Works

When FastExport is invoked, the utility logs onto the Teradata database and retrieves the rows that are specified in the SELECT statement and puts them into SPOOL. From there, it must build blocks to send back to the client. In comparison, BTEQ starts sending rows immediately for storage into a file. If the output data is sorted, FastExport may be required to redistribute the selected data two times across the AMP processors in order to build the blocks in the correct sequence. Remember, a lot of rows fit into a 64K block and both the rows and the blocks must be sequenced. While all of this redistribution is occurring, BTEQ continues to send rows. FastExport is getting behind in the processing. However, when FastExport starts sending the rows back a block at a time, it quickly overtakes and passes BTEQ's row at time processing. The other advantage is that if BTEQ terminates abnormally, all of your rows (which are in SPOOL) are discarded. You must rerun the BTEQ script from the beginning. However, if FastExport terminates abnormally, all the selected rows are in worktables and it can continue sending them where it left off. Pretty smart and very fast! FastExport Fundamentals #1: FastExport EXPORTS data from Teradata. The reason they call it FastExport is because it takes data off of Teradata (Exports Data). FastExport does not import data into Teradata. Additionally, like BTEQ it can output multiple files in a single run. #2: FastExport only supports the SELECT statement. The only DML statement that FastExport understands is SELECT. You SELECT the data you want exported and FastExport will take care of the rest. #3: Choose FastExport over BTEQ when Exporting Data of more than half a million+ rows. When a large amount of data is being exported, FastExport is recommended over BTEQ Export. The only drawback is the total number of FastLoads, FastExports, and MultiLoads that can run at the same time, which is limited to 15. BTEQ Export does not have this restriction. Of course, FastExport will work with less data, but the speed may not be much faster than BTEQ. #4: FastExport supports multiple SELECT statements and multiple tables in a single run. You can have multiple SELECT statements with FastExport and each SELECT can join information up to 64 tables. #5: FastExport supports conditional logic, conditional expressions, arithmetic calculations, and data conversions. FastExport is flexible and supports the above conditions, calculations, and conversions. #6: FastExport does NOT support error files or error limits. FastExport does not record particular error types in a table. The FastExport utility will terminate after a certain number of errors have been encountered. #7: FastExport supports user-written routines INMODs and OUTMODs. FastExport allows you write INMOD and OUTMOD routines so you can select, validate and preprocess the exported data

Page 21 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Maximum of 15 Loads

The Teradata RDBMS will only support a maximum of 15 simultaneous FastLoad, MultiLoad, or FastExport utility jobs. This maximum value is determined and configured in the DBS Control record. This value can be set from 0 to 15. When Teradata is initially installed, this value is set at 5. The reason for this limitation is that FastLoad, MultiLoad, and FastExport all use large blocks to transfer data. If more then 15 simultaneous jobs were supported, a saturation point could be reached on the availability of resources. In this case, Teradata does an excellent job of protecting system resources by queuing up additional FastLoad, MultiLoad, and FastExport jobs that are attempting to connect. For example, if the maximum number of utilities on the Teradata system is reached and another job attempts to run that job does not start. This limitation should be viewed as a safety control feature. A tip for remembering how the load limit applies is this, "If the name of the load utility contains either the word "Fast" or the word "Load", then there can be only a total of fifteen of them running at any one time". BTEQ does not have this load limitation. FastExport is clearly the better choice when exporting data. However, if two many load jobs are running. BTEQ is an alternate choice for exporting data. FastExport Support and Task Commands FastExport accepts both FastExport commands and a subset of SQL statements. The FastExport commands can be broken down into support and task activities. The table below highlights the key FastExport commands and their definitions. These commands provide flexibility and control during the export process. Support Environment Commands (see Support Environment chapter for details)ACCEPT DATEFORM DISPLAY ELSE ENDIF IF LOGOFF LOGON LOGTABLE ROUTE MESSAGES RUN FILE SET SYSTEM Allows the value of utility variables to be accepted directly from a file or from environmental variables. Specifies the style of the DATE data types for FastExport. Writes messages to the specific location. Used in conjunction with the IF statement. ELSE commands and statements will execute when a proceeding IF condition is false. Used in conjunction with the IF or ELSE statements. Delimits the commands that were subject to previous IF or ELSE conditions. Introduces a conditional expression. If true then execution of subsequent commands will happen. Disconnects all FastExport active sessions and terminates FastExport. LOGON command or string used to connect sessions established through the FastExport utility. FastExport utilizes this to specify a restart log table. The purpose is for FastExport checkpoint information. Will route FastExport messages to an alternate destination. Used to point to a file that FastExport is to use as standard input. This will Invoke the specified external file as the current source of utility and Teradata SQL commands. Assigns a data type and value to a variable. Suspends the FastExport utility temporarily and executes any valid local operating system command before returning.

Task CommandsBEGIN EXPORT END EXPORT EXPORT Begins the export task and sets the specifications for the number of sessions with Teradata. Ends the export task and initiates processing by Teradata. Provides two things which are:n

The client destination and file format specifications for the export data retrieved from Teradata A generated MultiLoad script file that can be used later to reload the export data back into Teradata

n

Page 22 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

FIELD FILLER IMPORT LAYOUT

Constitutes a field in the input record section that provides data values for the SELECT statement. Specifies a field in the input record that will not be sent to Teradata for processing. It is part of the input record to provide data values for the SELECT statement. Defines the file that provides the USING data values for the SELECT. Specifies the data layout for a file. It contains a sequence of FIELD and FILLER commands. This is used to describe the import file that can optionally provide data values for the SELECT.

FastExport Supported SQL Commands FastExport accepts the following Teradata SQL statements. Each has been placed in alphabetic order for your convenience.SQL Commands ALTER TABLE CHECKPOINT COLLECT STATISTICS COMMENT CREATE DATABASE CREATE TABLE CREATE VIEW CREATE MACRO DATABASE DELETE DELETE DATABASE DROP DATABASE GIVE GRANT MODIFY DATABASE RENAME REPLACE MACRO REPLACE VIEW REVOKE SET SESSION COLLATION UPDATE Change a column or table options of a table. Add a checkpoint entry in the journal table. Collect statistics for one or more columns or indexes in a table. Store or retrieve a comment string for a particular object. Creates a new database. Creates a new table. Creates a new view. Creates a new macro. Specify a default database for the session. Delete rows from a table. Removes all tables, views, macros, and stored procedures from a database. Drops a database. Transfer ownership of a database or user to another user. Grant access privileges to an object. Change the options for a database. Change the name of a table, view, or macro. Change a macro. Change a view. Revoke privileges to an object. Override the collation specification during the current session. Change a column value of an existing row or rows in a table.

FastExport Modes FastExport has two modes: RECORD or INDICATOR. In the mainframe world, only use RECORD mode. In the UNIX or LAN environment, INDICATOR mode is the default, but you can use INDICATOR mode if desired. The difference between the two modes is INDICATOR mode will set the indicator bits to 1 for column values containing NULLS. Both modes return data in a client internal format with variable-length records. Each individual record has a value for all of the columns specified by the SELECT statement. All variable-length columns are preceded by a two-byte control value indicating the length of the column data. NULL columns have a value that is appropriate for the column data type. Remember, INDICATOR mode will set bit flags that identify the columns that have a null value. FastExport Formats FastExport has many possible formats in the UNIX or LAN environment. The FORMAT statement specifies the format for each record being exported which are:n

FASTLOADPage 23 / 59

ReprintedforCSC/vramanathan,CSC

CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

n

BINARY TEXT UNFORMAT

n

n

The default FORMAT is FASTLOAD in a UNIX or LAN environment. FASTLOAD format has a two-byte integer, followed by the data, followed by an end-of-record marker. It is called FASTLOAD because the data is exported in a format ready for FASTLOAD. BINARY format is a two-byte integer, followed by data. TEXT format is an arbitrary number of bytes followed by an end-of-record marker. UNFORMAT format is exactly as received from CLIv2 without any client modifications. An Introduction to FastLoadWhy it is Called "FAST" Load

FastLoad is known for its lightning-like speed in loading vast amounts of data from flat files from a host into empty tables in Teradata. Part of this speed is achieved because it does not use the Transient Journal. You will see some more of the reasons enumerated below. But, regardless of the reasons that it is fast, know that FastLoad was developed to load millions of rows into a table. The way FastLoad works can be illustrated by home construction, of all things! Let's look at three scenarios from the construction industry to provide an amazing picture of how the data gets loaded. Scenario One: Builders prefer to start with an empty lot and construct a house on it, from the foundation right on up to the roof. There is no pre-existing construction, just a smooth, graded lot. The fewer barriers there are to deal with, the quicker the new construction can progress. Building custom or spec houses this way is the fastest way to build them. Similarly, FastLoad likes to start with an empty table, like an empty lot, and then populate it with rows of data from another source. Because the target table is empty, this method is typically the fastest way to load data. FastLoad will never attempt to insert rows into a table that already holds data. Scenario Two: The second scenario in this analogy is when someone buys the perfect piece of land on which to build a home, but the lot already has a house on it. In this case, the person may determine that it is quicker and more advantageous just to demolish the old house and start fresh from the ground up allowing for brand new construction. FastLoad also likes this approach to loading data. It can just 1) drop the existing table, which deletes the rows, 2) replace its structure, and then 3) populate it with the latest and greatest data. When dealing with huge volumes of new rows, this process will run much quicker than using MultiLoad to populate the existing table. Another option is to DELETE all the data rows from a populated target table and reload it. This requires less updating of the Data Dictionary than dropping and recreating a table. In either case, the result is a perfectly empty target table that FastLoad requires! FastLoad Has Some Limits There are more reasons why FastLoad is so fast. Many of these become restrictions and therefore, cannot slow it down. For instance, can you imagine a sprinter wearing cowboy boots in a race? Of course, not! Because of its speed, FastLoad, too, must travel light! This means that it will have limitations that may or may not apply to other load utilities. Remembering this short list will save you much frustration from failed loads and angry colleagues. It may even foster your reputation as a smooth operator! Rule #1: No Secondary Indexes are allowed on the Target Table. High performance will only allow FastLoad to utilize Primary Indexes when loading. The reason for this is that Primary (UPI and NUPI) indexes are used in Teradata to distribute the rows evenly across the AMPs and build only data rows. A secondary index is stored in a subtable block and many times on a different AMP from the data row. This would slow FastLoad down and they would have to call it: get ready now, HalfFastLoad. Therefore, FastLoad does not support them. If Secondary Indexes exist already, just drop them. You may easily recreate them after completing the load. Rule #2: No Referential Integrity is allowed. FastLoad cannot load data into tables that are defined with Referential Integrity (RI). This would require too much system checking to prevent referential constraints to a different table. FastLoadPage 24 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

only does one table. In short, RI constraints will need to be dropped from the target table prior to the use of FastLoad. Rule #3: No Triggers are allowed at load time. FastLoad is much too focused on speed to pay attention to the needs of other tables, which is what Triggers are all about. Additionally, these require more than one AMP and more than one table. FastLoad does one table only. Simply ALTER the Triggers to the DISABLED status prior to using FastLoad. Rule #4: Duplicate Rows (in Multi-Set Tables) are not supported. Multiset tables are tables that allow duplicate rows that is when the values in every column are identical. When FastLoad finds duplicate rows, they are discarded. While FastLoad can load data into a multi-set table, FastLoad will not load duplicate rows into a multi-set table because FastLoad discards duplicate rows! Rule #5: No AMPs may go down (i.e., go offline) while FastLoad is processing. The down AMP must be repaired before the load process can be restarted. Other than this, FastLoad can recover from system glitches and perform restarts. We will discuss Restarts later in this chapter. Rule #6: No more than one data type conversion is allowed per column during a FastLoad. Why just one? Data type conversion is highly resource intensive job on the system, which requires a "search and replace" effort. And that takes more time. Enough said! Three Key Requirements for FastLoad to Run FastLoad can be run from either MVS/ Channel (mainframe) or Network (LAN) host. In either case, FastLoad requires three key components. They are a log table, an empty target table and two error tables. The user must name these at the beginning of each script. Log Table: FastLoad needs a place to record information on its progress during a load. It uses the table called Fastlog in the SYSADMIN database. This table contains one row for every FastLoad running on the system. In order for your FastLoad to use this table, you need INSERT, UPDATE and DELETE privileges on that table. Empty Target Table: We have already mentioned the absolute need for the target table to be empty. FastLoad does not care how this is accomplished. After an initial load of an empty target table, you are now looking at a populated table that will likely need to be maintained. If you require the phenomenal speed of FastLoad, it is usually preferable, both for the sake of speed and for less interaction with the Data Dictionary, just to delete all the rows from that table and then reload it with fresh data. The syntax DELETE . should be used for this. But sometimes, as in some of our FastLoad sample scripts below (see Figure 4-1), you want to drop that table and recreate it versus using the DELETE option. To do this, FastLoad has the ability to run the DDL statements DROP TABLE and CREATE TABLE. The problem with putting DDL in the script is that is no longer restartable and you are required to rerun the FastLoad from the beginning. Otherwise, we recommend that you have a script for an initial run and a different script for a restart. Two Error Tables: Each FastLoad requires two error tables. These are error tables that will only be populated should errors occur during the load process. These are required by the FastLoad utility, which will automatically create them for you; all you must do is to name them. The first error table is for any translation errors or constraint violations. For example, a row with a column containing a wrong data type would be reported to the first error table. The second error table is for errors caused by duplicate values for Unique Primary Indexes (UPI). FastLoad will load just one occurrence for every UPI. The other occurrences will be stored in this table. However, if the entire row is a duplicate, FastLoad counts it but does not store the row. These tables may be analyzed later for troubleshooting should errors occur during the load. For specifics on how you can troubleshoot, see the section below titled, "What Happens When FastLoad Finishes." FastLoad Commands Here is a table of some key FastLoad commands and their definitions. They are used to provide flexibility in control of the load process. Consider this your personal redireference guide! You will notice that there are only a few SQL commands that may be used with this utility (Create Table, Drop Table, Delete and Insert). This keeps FastLoad from becoming encumbered with additional functions that would slow it down.AXSMOD Short for Access Module, this command specifies input protocol like OLE-DB or reading a tape from REEL Librarian. This parameter is for network-attached systems only. When used, it must precede the DEFINE command in the script. This identifies and locks the FastLoad target table for the duration of the load. It also identifies the two error

BEGIN LOADING

Page 25 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

tables to be used for the load. CHECKPONT and INDICATORS are subordinate commands in the BEGIN LOADING clause of the script. CHECKPOINT, which will be discussed below in detail, is not the default for FastLoad. It must be specified in the script. INDICATORS is a keyword related to how FastLoad handles nulls in the input file. It identifies columns with nulls and uses a bitmap at the beginning of each row to show which fields contain a null instead of data. When the INDICATORS option is on, FastLoad looks at each bit to identify the null column. The INDICATORS option does not work with VARTEXT. CREATE TABLE DEFINE DELETE DROP TABLE END LOADING This defines the target table and follows normal syntax. If used, this should only be in the initial script. If the table is being loaded, it cannot be created a second time. This names the input file and describes the columns in that file and the data types for those columns. Deletes all the rows of a table. This will only work in the initial run of the script. Upon restart, it will fail because the table is locked. Drops a table and its data. It is used in FastLoad to drop previous Target and error tables. At the same time, this is not a good thing to do within a FastLoad script since it cancels the ability to restart. Success! This command indicates the point at which that all the data has been transmitted. It tells FastLoad to proceed to Phase II. As mentioned earlier, it can be used as a way to partition data loads to the same table by omitting if from the script. This is true because the table remains empty until after Phase II. Specifies the maximum number of rejected ROWS allowed in error table 1 (Phase I). This handy command can be a lifesaver when you are not sure how corrupt the data in the input file is. The more corrupt it is, the greater the clean up effort required after the load finishes. ERRLIMIT provides you with a safety valve. You may specify a particular number of error rows beyond which FastLoad will precede to the abort. This provides the option to restart the FastLoad or to scrub the input data more before loading it. Remember, all the rows in the error table are not in the data table. That becomes your responsibility. Designed for online use, the Help command provides a list of all possible FastLoad commands along with brief, but pertinent tips for using them. Builds the table columns list for use in the FastLoad DEFINE statement when the data matches the Create Table statement exactly. In real life this does not happen very often. This is FastLoad's favorite command! It inserts rows into the target table. No, this is not the WAX ON / WAX OFF from the movie, The Karate Kid! LOGON simply begins a session. LOGOFF ends a session. QUIT is the same as LOGOFF. Just like it sounds, the NOTIFY command used to inform the job that follows that some event has occurred. It calls a user exit or predetermined activity when such events occur. NOTIFY is often used for detailed reporting on the FastLoad job's success. Specifies the beginning record number (or with THRU, the ending record number) of the Input data source, to be read by FastLoad. Syntactically, This command is placed before the INSERT keyword. Why would it be used? Well, it enables FastLoad to bypass input records that are not needed such as tape headers, manual restart, etc. When doing a partition data load, RECORD is used to over-ride the checkpoint. Used only in the LAN environment, this command states in what format the data from the Input file is coming: FastLoad, Unformatted, Binary, Text, or Variable Text. The default is the Teradata RDBMS standard, FastLoad. This command specifies the number of FastLoad sessions to establish with Teradata. It is written in the script just before the logon. The default is 1 session per available AMP. The purpose of multiple sessions is to enhance throughput when loading large volumes of data. Too few sessions will stifle throughput. Too many will preclude availability of system resources to other users. You will need to find the proper balance for your configuration. Working in conjunction with TENACITY, the SLEEP command specifies the amount of time in minutes to wait before retrying to logon and establish all sessions. This situation can occur if all of the loader slots are used or if the number of requested sessions are not available. The default is 6 minutes. For example, suppose that Teradata sessions are already maxed-out when your job is set to run. If TENACITY were set at 4 and SLEEP at 10, then FastLoad would attempt to logon every 10 minutes for up to 4 hours. If there were no success by that time, all efforts to logon would cease. Sometimes there are too many sessions already established with Teradata for a FastLoad to obtain the number of sessions it requested to perform its task or all of the loader slots are currently used. TENACITY specifies the amount of time, in hours, to retry to obtain a loader slot or to establish all requested sessions to logon. The default for FastLoad is "no tenacity", meaning that it will not retry at all. If several FastLoad jobs are executed at the same time, we recommend setting the TENACITY to 4, meaning that the system will continue trying to logon for the number of sessions requested for up to four hours.

ERRLIMIT

HELP HELP TABLE INSERT LOGON/LOGOFF or, QUIT NOTIFY

RECORD

SET RECORD SESSIONS

SLEEP

TENACITY

Fastload Exercise "Mistakes are a part of being human. Appreciate your mistakes for what they are: precious life lessons that canPage 26 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

only be learned the hard way. Unless it's a fatal mistake, which, at least, others can learn from." Al Franken Fastload is a utility we can use to populate empty tables. Make no mistake about how useful Fastload can be or how fatal errors can occur. The next 2 slides illustrate the essentials needed when constructing your fastload script. The first will highlight the important areas about the FastLoad script, and the second slide is a blank copy of the script that you can use to create your own FastLoad script. Use the flat file we created in the BTEQ chapter to help run the script. Structuring our Fastload script.

Simply copy the following text into notepad, then save it with a name and location that you can easily remember (we saved ours as c:\temp\Fastload_First_Script.txt).

Page 27 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

This script is going to create a table called Employee_Table02. After the table is created, it's going to take the information from our flat file and insert it into the new table. Afterwards, the Employee_Table and Employee_Table02 should look identical. Executing Our FastLoad Script "A good plan, violently executed now, is better than a perfect plan next week." - George S. Patton We can execute the Fastload utility like we do with BTEQ; however we use the command "fastload" instead of "BTEQ". If we get a return code of 0 then the Fastload worked perfectly. What did General Patton say when his Fastload gave him a return code of 12? I shall return 0! Executing our Fastload script

Let's see if it worked:

Page 28 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

The load utilities often scare people because there are many things that appear complicated. In actuality, the load scripts are very simple. Think of FastLoad as:n

Logging onto Teradata Defining the Teradata table that you want to load (target table) Defining the INPUT data file Telling the system to start loading

n

n

n

Sample FastLoad Script Normally it is not a good idea to put the DROP and CREATE statements in a FastLoad script. The reason is that when any of the tables that FastLoad is using are dropped, the script cannot be restarted. It can only be rerun from the beginning. Since FastLoad has restart logic built into it, a restart is normally the better solution if the initial load attempt should fail. However, for purposes of this example, it shows the table structure and the description of the data being read. Let's look at another FastLoad script that you might see in the real world. In the script below, every comment line is placed inside the normal Teradata comment syntax, [/*. . . . */]. FastLoad and SQL commands are written in upper case in order to make them stand out. In reality, Teradata utilities, like Teradata itself, are by default not case sensitive. You will also note that when column names are listed vertically we recommend placing the comma separator in front of the following column. Coding this way makes reading or debugging the script easier for everyone. The purpose of this script is to load the Employee_Profile table in the SQL01 database. The input file used for the load is named EMPS.TXT. Below the sample script each step will be described in detail./* FASTLOAD SCRIPT TO LOAD THE /* Employee_Profile TABLE */ /* Created by Coffing Data Warehousing */ */ Since this script does not drop the target or error tables, it is restartable. This is a good thing for production jobs.

/* Setup the FastLoad Parameters */ SESSIONS 100; /*or, the number of sessions supportable*/ TENACITY 4; /* the default is no tenacity, means no retry */ SLEEP 10; /* the default is 6, means retry in 6 minutes */ LOGON CW/SQL01,SQL01; SHOW VERSIONS; /* Shows the Utility's release number */ Page 29 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

Specify the number of sessions to logon. Tenacity is set to 4 hr; Wait 10 Min between retries.

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

/* Set the Record type to a comma delimited for FastLoad */ RECORD 2; SET RECORD VARTEXT ","; /* Define the Text File Layout and Input File */ DEFINE Employee_No (VARCHAR(10)) ,Last_name (VARCHAR(20)) ,First_name (VARCHAR(12)) ,Salary (VARCHAR(5)) ,Dept_No (VARCHAR(6)) FILE= EMPS.TXT; /* Optional to show the layout of the input */ SHOW; /* Begin the Load and Insert Process into the */ /* Employee_Profile Table */ BEGIN LOADING SQL01.Employee_Profile ERRORFILES SQL01.Emp_Err1, SQL01.Emp_Err2 CHECKPOINT 100000; INSERT INTO SQL01.Employee_Profile VALUES ( :Employee_No ,:Last_name ,:First_name ,:Salary ,:Dept_No ); END LOADING; LOGOFF; Specifies table to load and lock. Names the error tables Sets the number of rows at which to pause & record progress in the restart log before loading further. Defines the insert statement to use for loading the rows Continues loading process with Phase 2. Logs off of Teradata. Starts with the second record. Specifies if record layout is vartext with a comma delimiter.

Notice that all fields are defined as VARCHAR. When using VARTEXT, the fields do not contain the length field like in these formats: text, FastLoad, or unformatted.

Step One: Before logging onto Teradata, it is important to specify how many sessions you need. The syntax is [SESSIONS {n}]. Step Two: Next, you LOGON to the Teradata system. You will quickly see that the utility commands in FastLoad are similar to those in BTEQ. FastLoad commands were designed from the underlying commands in BTEQ. However, unlike BTEQ, most of the FastLoad commands do not allow a dot ["."] in front of them and therefore need a semicolon. At this point we chose to have Teradata tell us which version of FastLoad is being used for the load. Why would we recommend this? We do because as FastLoad's capabilities get enhanced with newer versions, the syntax of the scripts may have to be revisited. Step Three: If the input file is not a FastLoad format, before you describe the INPUT FILE structure in the DEFINE statement, you must first set the RECORD layout type for the file being passed by FastLoad. We have used VARTEXT in our example with a comma delimiter. The other options are FastLoad, TEXT, UNFORMATTED OR VARTEXT. You need to know this about your input file ahead of time. Step Four: Next, comes the DEFINE statement. FastLoad must know the structure and the name of the flat file to be used as the input FILE, or source file for the load. Step Five: FastLoad makes no assumptions from the DROP TABLE statements with regard to what you want loaded. In the BEGIN LOADING statement, the script must name the target table and the two error tables for the load. Did you notice that there is no CREATE TABLE statement for the error tables in this script? FastLoad will automatically create them for you once you name them in the script. In this instance, they are named "Emp_Err1" and "Emp_Err2". Phase 1 uses "Emp_Err1" because it comes first and Phase 2 uses "Emp_Err2". The names are arbitrary, of course. You may call them whatever you like. At the same time, they must be unique within a database, so using a combination of your userid and target table name helps insure this uniqueness between multiple FastLoad jobs occurring in the same database.Page 30 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

In the BEGIN LOADING statement we have also included the optional CHECKPOINT parameter. We included [CHECKPOINT 100000]. Although not required, this optional parameter performs a vital task with regard to the load. In the old days, children were always told to focus on the three "R's' in grade school ("reading, 'riting, and 'rithmatic"). There are two very different, yet equally important, R's to consider whenever you run FastLoad. They are RERUN and RESTART. RERUN means that the job is capable of running all the processing again from the beginning of the load. RESTART means that the job is capable of running the processing again from the point where it left off when the job was interrupted, causing it to fail. When CHECKPOINT is requested, it allows FastLoad to resume loading from the first row following the last successful CHECKPOINT. We will learn more about CHECKPOINT in the section on Restarting FastLoad. Step Six: FastLoad focuses on its task of loading data blocks to AMPs like little Yorkshire terrier's do when playing with a ball! It will not stop unless you tell it to stop. Therefore, it will not proceed to Phase 2 without the END LOADING command. In reality, this provides a very valuable capability for FastLoad. Since the table must be empty at the start of the job, it prevents loading rows as they arrive from different time zones. However, to accomplish this processing, simply omit the END LOADING on the load job. Then, you can run the same FastLoad multiple times and continue loading the worktables until the last file is received. Then run the last FastLoad job with an END LOADING and you have partitioned your load jobs into smaller segments instead of one huge job. This makes FastLoad even faster! Of course to make this work, FastLoad must be restartable. Therefore, you cannot use the DROP or CREATE commands within the script. Additionally, every script is exactly the same with the exception of the last one, which contains the END LOADING causing FastLoad to proceed to Phase 2. That's a pretty clever way to do a partitioned type of data load. Step Seven: All that goes up must come down. And all the sessions must LOGOFF. This will be the last utility command in your script. At this point the table lock is released and if there are no rows in the error tables, they are dropped automatically. However, if a single row is in one of them, you are responsible to check it, take the appropriate action and drop the table manually. When You Cannot RESTART FastLoad There are two types of FastLoad scripts: those that you can restart and those that you cannot without modifying the script. If any of the following conditions are true of the FastLoad script that you are dealing with, it is NOT restartable:n

The Error Tables are DROPPED The Target Table is DROPPED The Target Table is CREATED

n

n

When You Can RESTART FastLoad If all of the following conditions are true, then FastLoad is ALWAYS restartable:n

The Error Tables are NOT DROPPED in the script The Target Table is NOT DROPPED in the script The Target Table is NOT CREATED in the script You have defined a checkpoint

n

n

n

So, if you need to drop or create tables, do it in a separate job using BTEQ. Imagine that you have a table whose data changes so much that you typically drop it monthly and build it again. Let's go back to the script we just reviewed above and see how we can break it into the two parts necessary to make it fully RESTARTABLE. It is broken up below. STEP ONE: Run the following SQL statements in Queryman or BTEQ before you start FastLoad:DROP TABLE SQL01.Department; DROP TABLE SQL01.Dept_Err1; DROP TABLE SQL01.Dept_Err2; CREATE TABLE SQL01.Department (Dept_No INTEGER DROPS TARGET TABLE AND ERROR TABLES

CREATES THE DEPARTMENT TARGET TABLE IN THE SQL01 DATA BASE IN TERADATAPage 31 / 59

ReprintedforCSC/vramanathan,CSC

CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

,Dept_Name CHAR(20) ) UNIQUE PRIMARY INDEX (Dept_No);

First, you ensure that the target table and error tables, if they existed previously, are blown away. If there had been no errors in the error tables, they would be automatically dropped. If these tables did not exist, you have not lost anything. Next, if needed, you create the empty table structure needed to receive a FastLoad. STEP TWO: Run the FastLoad script This is the portion of the earlier script that carries out these vital steps:n

Defines the structure of the flat file Tells FastLoad where to load the data and store the errors Specifies the checkpoint so a RESTART will not go back to row one Loads the data

n

n

n

If these are true, all you need do is resubmit the FastLoad job and it starts loading data again with the next record after the last checkpoint. Now, with that said, if you did not request a checkpoint, the output message will normally indicate how many records were loaded. You may optionally use the RECORD command to manually restart on the next record after the one indicated in the message. Now, if the FastLoad job aborts in Phase 2, you can simply submit a script with only the BEGIN LOADING and END LOADING. It will then restart right into Phase 2. An Introduction to MultiLoadWhy it is Called "Multi" Load

If we were going to be stranded on an island with a Teradata Data Warehouse and we could only take along one Teradata load utility, clearly, MultiLoad would be our choice. MultiLoad has the capability to load multiple tables at one time from either a LAN or Channel environment. This is in stark contrast to its fleet-footed cousin, FastLoad, which can only load one table at a time. And it gets better, yet! This feature rich utility can perform multiple types of DML tasks, including INSERT, UPDATE, DELETE and UPSERT on up to five (5) empty or populated target tables at a time. These DML functions may be run either solo or in combinations, against one or more tables. For these reasons, MultiLoad is the utility of choice when it comes to loading populated tables in the batch environment. As the volume of data being loaded or updated in a single block, the performance of MultiLoad improves. MultiLoad shines when it can impact more than one row in every data block. In other words, MultiLoad looks at massive amounts of data and says, "Bring it on!" Leo Tolstoy once said, "All happy families resemble each other." Like happy families, the Teradata load utilities resemble each other, although they may have some differences. You are going to be pleased to find that you do not have to learn all new commands and concepts for each load utility. MultiLoad has many similarities to FastLoad. It has even more commands in common with TPump. The similarities will be evident as you work with them. Where there are some quirky differences, we will point them out for you.Two MultiLoad Modes: IMPORT and DELETE

MultiLoad provides two types of operations via modes: IMPORT and DELETE. In MultiLoad IMPORT mode, you have the freedom to "mix and match" up to twenty (20) INSERTs, UPDATEs or DELETEs on up to five target tables. The execution of the DML statements is not mandatory for all rows in a table. Instead, their execution hinges upon the conditions contained in the APPLY clause of the script. Once again, MultiLoad demonstrates its user-friendly flexibility. For UPDATEs or DELETEs to be successful in IMPORT mode, they must reference the Primary Index in the WHERE clause. The MultiLoad DELETE mode is used to perform a global (all AMP) delete on just one table. The reason to use .BEGIN DELETE MLOAD is that it bypasses the Transient Journal (TJ) and can be RESTARTed if an error causes it to terminate prior to finishing. When performing in DELETE mode, the DELETE SQL statement cannot reference the Primary Index inPage 32 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

the WHERE clause. This due to the fact that a primary index access is to a specific AMP; this is a global operation. The other factor that makes a DELETE mode operation so good is that it examines an entire block of rows at a time. Once all the eligible rows have been removed, the block is written one time and a checkpoint is written. So, if a restart is necessary, it simply starts deleting rows from the next block without a checkpoint. This is a smart way to continue. Remember, when using the TJ all deleted rows are put back into the table from the TJ as a rollback. A rollback can take longer to finish then the delete. MultiLoad does not do a rollback; it does a restart.

In the above diagram, monthly data is being stored in a quarterly table. To keep the contents limited to four months, monthly data is rotated in and out. At the end of every month, the oldest month of data is removed and the new month is added. The cycle is "add a month, delete a month, add a month, delete a month." In our illustration, that means that January data must be deleted to make room for May's data. Here is a question for you: What if there was another way to accomplish this same goal without consuming all of these extra resources? To illustrate, let's consider the following scenario: Suppose you have TableA that contains 12 billion rows. You want to delete a range of rows based on a date and then load in fresh data to replace these rows. Normally, the process is to perform a MultiLoad DELETE to DELETE FROM TableA WHERE < '2002-02-01'. The final step would be to INSERT the new rows for May using MultiLoad IMPORT. MultiLoad Imposes Limits Rule #1: Unique Secondary Indexes are not supported on a Target Table. Like FastLoad, MultiLoad does not support Unique Secondary Indexes (USIs). But unlike FastLoad, it does support the use of Non-Unique Secondary Indexes (NUSIs) because the index subtable row is on the same AMP as the data row. MultiLoad uses every AMP independently and in parallel. If two AMPs must communicate, they are not independent. Therefore, a NUSI (same AMP) is fine, but a USI (different AMP) is not. Rule #2: Referential Integrity is not supported. MultiLoad will not load data into tables that are defined with Referential Integrity (RI). Like a USI, this requires the AMPs to communicate with each other. So, RI constraints must be dropped from the target table prior to using MultiLoad. Rule #3: Triggers are not supported at load time. Triggers cause actions on related tables based upon what happens in a target table. Again, this is a multi-AMP operation and to a different table. To keep MultiLoad running smoothly, disable all Triggers prior to using it.

Page 33 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

Rule #4: No concatenation of input files is allowed. MultiLoad does not want you to do this because it could impact are restart if the files were concatenated in a different sequence or data was deleted between runs. Rule #5: The host will not process aggregates, arithmetic functions or exponentiation. If you need data conversions or math, you might be better off using an INMOD to prepare the data prior to loading it. Error Tables, Work Tables and Log Tables Besides target table(s), MultiLoad requires the use of four special tables in order to function. They consist of two error tables (per target table), one worktable (per target table), and one log table. In essence, the Error Tables will be used to store any conversion, constraint or uniqueness violations during a load. Work Tables are used to receive and sort data and SQL on each AMP prior to storing them permanently to disk. A Log Table (also called, "Logtable") is used to store successful checkpoints during load processing in case a RESTART is needed. HINT: Sometimes a company wants all of these load support tables to be housed in a particular database. When these tables are to be stored in any database other than the user's own default database, then you must give them a qualified name (.) in the script or use the DATABASE command to change the current database. Where will you find these tables in the load script? The Logtable is generally identified immediately prior to the .LOGON command. Worktables and error tables can be named in the BEGIN MLOAD statement. Do not underestimate the value of these tables. They are vital to the operation of MultiLoad. Without them a MultiLoad job can not run. Now that you have had the "executive summary", let's look at each type of table individually. Two Error Tables: Here is another place where FastLoad and MultiLoad are similar. Both require the use of two error tables per target table. MultiLoad will automatically create these tables. Rows are inserted into these tables only when errors occur during the load process. The first error table is the acquisition Error Table (ET). It contains all translation and constraint errors that may occur while the data is being acquired from the source(s). The second is the Uniqueness Violation (UV) table that stores rows with duplicate values for Unique Primary Indexes (UPI). Since a UPI must be unique, MultiLoad can only load one occurrence into a table. Any duplicate value will be stored in the UV error table. For example, you might see a UPI error that shows a second employee number "99." In this case, if the name for employee "99" is Kara Morgan, you will be glad that the row did not load since Kara Morgan is already in the Employee table. However, if the name showed up as David Jackson, then you know that further investigation is needed, because employee numbers must be unique. Each error table does the following:n

Identifies errors Provides some detail about the errors Stores the actual offending row for debugging

n

n

You have the option to name these tables in the MultiLoad script (shown later). Alternatively, if you do not name them, they default to ET_ and UV_. In either case, MultiLoad will not accept error table names that are the same as target table names. It does not matter what you name them. It is recommended that you standardize on the naming convention to make it easier for everyone on your team. For more details on how these error tables can help you, see the subsection in this chapter titled, "Troubleshooting MultiLoad Errors." Log Table: MultiLoad requires a LOGTABLE. This table keeps a record of the results from each phase of the load so that MultiLoad knows the proper point from which to RESTART. There is one LOGTABLE for each run. Since MultiLoad will not resubmit a command that has been run previously, it will use the LOGTABLE to determine the last successfully completed step. Work Table(s): MultiLoad will automatically create one worktable for each target table. This means that in IMPORT mode you could have one or more worktables. In the DELETE mode, you will only have one worktable since that mode only works on one target table. The purpose of worktables is to hold two things: 1. The Data Manipulation Language (DML) tasks 2. The input data that is ready to APPLY to the AMPs The worktables are created in a database using PERM space. They can become very large. If the script uses multiple SQLPage 34 / 59 ReprintedforCSC/vramanathan,CSC CoffingDataWarehousing,CoffingPublishing(c)2006,CopyingProhibited

TeradataUser'sGuide:TheUltimateCompanion,ThirdEdition

statements for a single data record, the data is sent to the AMP once for each SQL statement. This replication guarantees fast performance and that no SQL statement will ever be done more than once. So, this is very important. However, there is no such thing as a free lunch, the cost is space. Later, you will see that using a FILLER field can help reduce this disk space by not sending unneeded data to an AMP. In other words, the efficiency of the MultiLoad run is in your hands. Supported Input Formats Data input files come in a variety of formats but MultiLoad is flexible enough to handle many of them. MultiLoad supports the following five format options: BINARY, FASTLOAD, TEXT, UNFORMAT and VARTEXT.BINARY FASTLOAD TEXT UNFORMAT VARTEXT Each record is a 2-byte integer, n, that is followed by n bytes of data. A byte is the smallest means of storage of for Teradata. This format is the same as Binary, plus a marker (X '0A' or X '0D') that specifies the end of the record. Each record has a random number of bytes and is followed by an end of the record marker. The format for these input records is defined in the LAYOUT statement of the MultiLoad script using the components FIELD, FILLER and TABLE. This is variable length text RECORD format separated by delimiters such as a comma. For this format you may only use VARCHAR, LONG VARCHAR (IBM) or VARBYTE data formats in your MultiLoad LAYOUT. Note that two delimiter characters in a row will result in a null value between them.

MultiLoad Has Five IMPORT Phases MultiLoad IMPORT has five phases, but don't be fazed by this! Here is the short list:n

Phase 1: Preliminary Phase Phase 2: DML Transaction Phase Phase 3: Acquisition Phase Phase 4: Application Phase Phase 5: Cleanup Phase

n

n

n

n

Let's take a look at each phase and see what it contributes to the overall load process of this magnificent utility. Should you memorize every detail about each phase? Probably not. But it is important to know the essence of each phase because sometimes a load fails. When it does, you need to know in which phase it broke down since the method for fixing the error to RESTART may vary depending on the phase. And if you can picture what MultiLoad actually does in each phase, you will likely write better scripts that run more efficiently.Phase