Teradata Utilities: BTEQ -...

of 29 /29
Teradata Utilities: BTEQ Reprinted for KV Satish Kumar, IBM [email protected] Reprinted with permission as a subscription benefit of Books24x7, http://www.books24x7.com/

Embed Size (px)

Transcript of Teradata Utilities: BTEQ -...

  • Teradata Utilities: BTEQ

    Reprinted for KV Satish Kumar, [email protected]

    Reprinted with permission as a subscription benefit of Books24x7,http://www.books24x7.com/

    http://www.books24x7.com/

  • Table of Contents Chapter 2: BTEQ..............................................................................................................................1

    Why it is Called BTEQ?.........................................................................................................1Logging onto BTEQ...............................................................................................................1Using BTEQ to Submit Queries.............................................................................................2

    Submitting SQL in BTEQ's Interactive Mode...................................................................2WITH BY Statement...............................................................................................................3Transactions in Teradata Mode.............................................................................................5Alternative Transactions in Teradata Mode...........................................................................7Transactions in ANSI Mode...................................................................................................8Rollback.................................................................................................................................9Advantages to ANSI Mode...................................................................................................10Creating a Batch Script for BTEQ........................................................................................11Running your Batch Script in BTEQ.....................................................................................12Results from a BTEQ Batch Script.......................................................................................13Placing our BTEQ Output to a file........................................................................................14Reading out BTEQ output from the Text File.......................................................................14Using BTEQ Conditional Logic............................................................................................15Using BTEQ to Export Data.................................................................................................16Executing our BTEQ Script to Export Data..........................................................................16BTEQ Export Modes Explained...........................................................................................17BTEQ EXPORT Example Using Record (DATA) Mode.......................................................18BTEQ EXPORT Example Using Field (Report) Mode.........................................................19BTEQ IMPORT Example.....................................................................................................21Determining Out Record Lengths........................................................................................23BTEQ Commands................................................................................................................25

    Session Control Commands..........................................................................................25File Control Commands.................................................................................................26

    Sequence Control Commands.............................................................................................26Format Control Commands............................................................................................26

    i

  • Chapter 2: BTEQ

    "Civilization advances by extending the number of important operations which wecan perform without thinking about them."- Alfred Whitehead

    Why it is Called BTEQ?

    Why is BTEQ available on every Teradata system ever built? Because the Batch TEradata Query(BTEQ) tool was the original way that SQL was submitted to Teradata as a means of getting ananswer set in a desired format. This is the utility that I used for training at Wal-Mart, AT&T, AnthemBlue Cross and Blue Shield, and SouthWestern Bell back in the early 1990's. BTEQ is often referredto as the Basic TEradata Query and is still used today as an effective tool.

    Here is what is excellent about BTEQ:

    BTEQ can be used to submit SQL in either a batch or interactive environment. Interactiveusers can submit SQL and receive an answer set on the screen. Users can also submitBTEQ jobs from batch scripts, have error checking and conditional logic, and allow for thework to be done in the background.

    BTEQ outputs a report format, where Queryman outputs data in a format more like aspreadsheet. This allows BTEQ a great deal of flexibility in formatting data, creatingheadings, and utilizing Teradata extensions, such as WITH and WITH BY that Querymanhas problems in handling.

    BTEQ is often used to submit SQL, but is also an excellent tool for importing and exportingdata.

    Importing Data: Data can be read from a file on either a mainframe or LAN attachedcomputer and used for substitution directly into any Teradata SQL using the INSERT,UPDATE or DELETE statements.

    Exporting Data: Data can be written to either a mainframe or LAN attached computerusing a SELECT from Teradata. You can also pick the format you desire rangingfrom data files to printed reports to spread sheet formats.

    There are other utilities that are faster than BTEQ for importing or exporting data. We will talk aboutthese in future chapters, but BTEQ is still used for smaller jobs.

    Logging onto BTEQ

    "It's choice not change that determines your destiny." Jean Nidetch

    By taking a chance in this industry, you've chosen to arm yourself with an unlimited arsenal ofknowledge. But you can't use that knowledge if you can't log onto the system! This next slide isgoing to teach you how to logon to BTEQ. Remember that you will be prompted for the passwordsince it's an interactive interface. BTEQ commands begin with a period (.) and do not require asemi-colon (;) to end the statement. SQL commands do not ever start with a period and they mustalways be terminated with a semi-colon.

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Let's logon to BTEQ and show all information in the Employee_Table:

    Before you can use BTEQ, you must have user access rights to the client system and privileges tothe Teradata DBS. Normal system access privileges include a user ID and a password. Somesystems may also require additional user identification codes depending on company standards andoperational procedures. Depending on the configuration of your Teradata DBS, you may need toinclude an account identifier (acctid) and/or a Teradata Director Program Identifier (TDPID).

    Using BTEQ to Submit QueriesSubmitting SQL in BTEQ's Interactive Mode

    Once you logon to Teradata through BTEQ, you are ready to run your queries. Teradata knows theSQL is finished when it finds a semi-colon, so don't forget to put one at the end of your query. Belowis an example of a Teradata table to demonstrate BTEQ operations.

    Employee_TableEmployee_No Last_Name First_Name Salary Dept_No

    2000000 Jones Squiggy 32800.50 ?1256349 Harrison Herbert 54500.00 4001333454 Smith John 48000.00 2001121334 Strickling Cletus 54500.00 400

    Figure 2-1

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 2

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • BTEQ execution

    .LOGON cdw/sql01Password: XXXXX

    Type at commandprompt: Logonwith TDPID andUSERNAME.Then enterPASSWORD atthe secondprompt.

    Enter your BTEQ/SQL Request or BTEQ Command.BTEQ willrespond and iswaiting for acommand.

    SELECT * FROM Employee_TableWHERE Dept_No = 400;

    An SQLStatement

    *** Query Completed. 2 rows found. 5 Columns returned.*** Total elapsed time was 1 second.

    BTEQ displaysinformation aboutthe answer set.

    Employee_No Last_name First_name Salary Dept_No1256349 Harrison Herbert 54500.00 4001121334 Strickling Cletus 54500.00 400

    The result set

    Figure 2-2

    WITH BY Statement

    "Time is the best teacher, but unfortunately, it kills all of its students." Robin Williams

    Investing time in Teradata can be a killer move for your career. We can use the WITH BY statementin BTEQ, whereas we cannot use it with Nexus or SQL Assistant. The WITH BY statement workslike a correlated subquery in the fact that you can us aggregates based on a distinct column value.

    BTEQ has the ability to use WITH BY statements:

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 3

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • "I've learned that you can't have everything and do everything at the same time." Oprah Winfrey

    The great thing about the WITH statement is that you can do everything to a specific group whilehaving everything done to a column as a whole. We can get a grand total or an overall average withthe WITH statement, just leave out BY. Here's a good example:

    Using WITH on a whole column:

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 4

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Transactions in Teradata Mode

    "He who every morning plans the transaction of the day and follows out that plan,carries a thread that will guide him through the maze of most busy life."- Victor Hugo

    Victor couldn't have summed up Teradata any better. However, Victor did seem more worried aboutthe hunchback than the rollback. Turning your queries into a single transaction is often the bestplan, but can sometimes make one Miserables.

    Often in Teradata we'll see multiple queries within the same transaction. We can use the BT/ETkeywords to bundle several queries into one transaction. You also need to end every query with asemi-colon (;), which isn't the case in Nexus or SQL assistant. For example:

    In Teradata mode, we're going to put four single statements into a single transaction

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 5

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • "What is defeat? Nothing but education; nothing but the first step to somethingbetter." Wendell Phillips

    The final query in our last transaction is what caused our updates to fail. This was not the sweettaste of victory, but instead the smell of de Feet! Actually, it really was an education leading tosomething better. When using BT/ET in your transaction, you're telling Teradata that when it comesto committing, we either want all or none. Since our last query in the transaction failed the TransientJournal rolled back all the queries in our entire transaction. Make sure that your syntax is correctwhen using the method of BT and ET because a mistake causes a massive rollback.

    The last query in our set did not work:

    Now let's take a look at the Employee_Table:

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 6

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Our updates didn't work! That's because we bundled all four queries into one transaction. Since ourlast query failed, the tables were rolled back to their original state before the transaction took place.

    Alternative Transactions in Teradata Mode

    "It's not enough that we do our best; sometimes we have to do what's required." Sir Winston Churchill

    Sometimes we're required to use an alternative method to get the job done if we want to win likeWinston. Here's another way to set up a bundle of queries into one transaction. Notice where weplace the semi-colon in our queries and you will understand this technique. Remember that thesemi-colon must be at the very beginning of the next line for a query to be considered as part of thesame transaction. Because we are in Teradata mode if any query fails then all queries that are partof the same transaction roll back. How many queries are parts of the same transaction below? Four!

    Another way to perform a multi-statement transaction in Teradata mode:

    Placing the semi-colon at the beginning of the next line (followed by another statement) will bundlethose statements together as one transaction. Notice that our Employee_Table was not updated,

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 7

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • just like in the first example.

    Transactions in ANSI Mode

    "The man who views the world at 50 the same as he did at 20 has wasted 30 yearsof his life." Muhammad Ali

    ANSI (American National Standard Institution) allows us to view the same queries in a different way.To change to ANSI mode, simply type '.set session transaction ANSI' and be sure to do it beforeyou actually logon to BTEQ. Then, you can logon like always, but this time you will be in ANSImode. All queries in ANSI mode will also work in Teradata mode and vice versa. However, threethings will be different in ANSI mode versus Teradata mode. Those things are how case sensitivityis handled, how transactions are committed and rolled back, and how truncation is accepted.

    Let's log back onto BTEQ, but this time change it to ANSI mode:

    "Be not afraid of growing slowly, be afraid only of standing still."-Chinese Proverb

    Remember the first rule of ANSI mode: all transactions must be committed by the user actuallyusing the word 'COMMIT'. Also, in ANSI mode after any DDL statement (CREATE, DROP, ALTER,DATABASE) we have to use the 'commit' command immediately. This tells Teradata to commit towhat's been done. Our query below will attempt to find anyone with a last_name of 'larkins'. It willfail even though we have 'Mike' 'Larkins' in our table. This is because ANSI is case sensitive and wedid not capitalize the 'L' in 'Larkins'.

    Let's run a few queries in ANSI mode:

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 8

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Notice that we have to COMMIT after any DDL or Update before the transaction is committed. Weeven have to COMMIT after setting our DATABASE or we will get an error.

    We didn't have any rows return, but we know there's a Mike Larkins within the table. That's becauseBTEQ is case sensitive. Change 'larkins' to 'Larkins'.

    Rollback

    "Insanity: doing the same thing over and over again and expecting different results." Albert Einstein

    The Rollback keyword is the SQL mulligan of Teradata. Rollback will erase any changes made to atable. This can be very useful if something didn't work. However, you cannot rollback once you'veused the commit keyword. Not keeping rollback in your arsenal would be insane.

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 9

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Advantages to ANSI ModeANSI mode is great because when you bundle several queries into one transaction and one doesn'twork, the rest won't be rolled back to their original state. Using commit will ensure that yoursuccesses aren't hidden by your failures.

    Now notice that I will have multiple statements in the same transaction and that I purposely fail thelast SQL statement:

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 10

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Which statements were rolled back?

    "All truths are easy to understand once they are discovered; the point is to discoverthem." Galileo Galilei

    Discovering the advantages in using ANSI will only make SQL easier to write. It might take a little bitmore typing, but a little work now can save you lots of time later.

    The Employee_Table was updated!

    In ANSI mode, only failed transactions are rolled back when it comes to multi-statementtransactions.

    Creating a Batch Script for BTEQ

    "The cure for boredom is curiosity. There is no cure for curiosity." Dorothy Parker

    If you've been bored waiting for your queries to finish, then I'm sure you're curious as to how we canfix the situation. Batch scripting allows us to write out pages and pages of queries and executethose queries in one single swoop. BTEQ can also run in batch mode under UNIX (IBM AIX,Hewlett-Packard HP-UX, NCR MP-RAS, Sun Solaris), DOS, Macintosh, Microsoft Windows andOS/2 operating systems. To submit a job in batch mode, do the following:

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 11

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Invoke BTEQ (using dos prompt)1.

    Type in the input file name2.

    Type in the location and output file name.3.

    The following example shows how to create a batch script and how to invoke the script using BTEQfrom a DOS command. When using Batch scripting, your password will not be prompted. Instead,just add the password after your login name, and a comma separates the two. Be sure to end witheither a .quit or .logoff so that your queries aren't left hanging.

    Simply open up notepad and type in the following, then save it. I recommend calling it'BTEQ_First_Batch_Script.txt' and save it in the C:\Temp folder. However, as long as you canremember what you named it and where you saved it, you'll be fine. Be sure that you save it as a.txt file.

    Using Batch scripting with BTEQ

    Running your Batch Script in BTEQ

    "I do not fear computers. I fear the lack of them." Isaac Asimov

    The BTEQ utility enables us to run our scripts in batch mode. To run our new batch script, we haveto access the BTEQ utility via dos prompt. Simply use command prompt to access the utility, andfollow the steps below:

    Let's run our query in Batch!

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 12

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Once you're in DOS, type in the following: 'BTEQ < c:\temp\BTEQ_First_Script.txt', then hit enter.BTEQ will automatically open in DOS, and then it will access the file from the location you listed.

    Results from a BTEQ Batch Script

    "Don't be afraid to take a big step when one is indicated. You can't cross a chasm intwo small steps."-David Lloyd George

    BTEQ will run your query in steps to produce the answer you're looking for. Whether you'reaccessing a small table or crossing over a chasm of information, BTEQ will ensure that the steps ittakes will be big enough to get the job done.

    Our results are returned Interactively

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 13

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Placing our BTEQ Output to a file

    "The secret to creativity is knowing how to hide your sources." Albert Einstein

    We can use BTEQ to export our results to another text document. Exporting data also works verywell when you're trying to document your query along with the results.

    We can export our results in batch as well

    Notice that the BTEQ command is immediately followed by the 'BTEQ_First_Export.txt' names the filewhere the output messages are written.

    Since putting password information into a script is scary for security reasons, inserting the passworddirectly into a script that is to be processed in batch mode may not be a good idea. It is generallyrecommended and a common practice to store the logon and password in a separate file that thatcan be secured. That way, it is not in the script for anyone to see. For example, the contents of a filecalled "mylogon.txt" might be: '.LOGON cdw/sql00,whynot'. Then, the script should contain thefollowing command instead of a .LOGON: .RUN FILE=c:\temp\mylogon.txt. To submit results totwo different files at once, simply initialize BTEQ as follows:

    BTEQ < c:\filename > c:\location1 > c:\location2

    Reading out BTEQ output from the Text File

    "The more original a discovery, the more obvious it seems afterwards." Arthur Koestler

    Discovering how easy it is to export your data in batch mode is a key step in learning Teradatautilities. Here are our results, including the original query and what BTEQ did to generate its answerset. Simply go to the folder where you saved the exported data (the previous examples saved thefile as c:\temp\BTEQ_First_Export.txt).

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 14

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • What you'll find in our new text document

    Using BTEQ Conditional Logic

    "I was walking down the street wearing glasses when the prescription ran out."- Steven Wright

    Below is a BTEQ batch script example. The initial steps of the script will establish the logon, thedatabase, and then delete all the rows from the Employee_Table. If the table does not exist, theBTEQ conditional logic will instruct Teradata to create it. However, if the table already exists, thenTeradata will move forward and insert data.

    Note: In script examples, the left panel contains BTEQ base commands and the right panelprovides a brief description of each command.

    .RUN FILE = c:\temp\mylogon.txtDATABASE SQL_Class;DELETE FROM Employee_Table;

    .IF ERRORCODE = 0 THEN .GOTO INSEMPS[*]/* ERRORCODE is a reserved word that contains the outcomestatus for every SQL statement executed in BTEQ. A zero (0)indicates that statement worked. */

    BTEQ conditionallogic that willcheck to ensurethat the deleteworked or if thetable evenexisted. If thetable did notexist, then BTEQwill create it. Ifthe table doesexist, the Createtable step will beskipped anddirectly GOTOINSEMPS.

    CREATE TABLE Employee_Table (Employee_No INTEGER,

    The LabelINSEMPSprovides code so

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 15

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Last_name CHAR(20), First_name CHAR(12), Salary DECIMAL(8,2), Dept_No SMALLINT)UNIQUE PRIMARY INDEX (Employee_No);

    the BTEQ Logiccan go directly toinserting recordsinto theEmployee_Table.

    .LABEL INSEMPS[*]INSERT INTO Employee_Table (1232578, 'Chambers', 'Mandee', 48850.00,100);INSERT INTO Employee_Table (1256349, 'Harrison', 'Herbert', 54500.00, 400);.QUIT

    Once the tablehas been created,Teradata will theninsert the twonew rows into theempty table.

    [*]Both labels have to be identical or it will not work.

    Using BTEQ to Export Data

    "The trouble with facts is that there are so many of them." Samuel McChord Crothers

    Creating flat files is one of the most important tasks in Teradata, and that's a fact. BTEQ allows datato be exported directly from Teradata to a file on a mainframe or network-attached computer. Inaddition, the BTEQ export function has several export formats that a user can choose fromdepending on the desired output. Generally, users will export data to a flat file format that iscomposed of a variety of characteristics. These characteristics include: field mode, indicator mode,or dif mode.

    Syntax of a basic EXPORT command:

    EXPORT FILE =

    Creating a flat file of what's on the Employee_Table

    Executing our BTEQ Script to Export Data

    "The past is a foreign country; they do things differently there."L. P. Hartley

    Transferring data from table to another without the use of a flat file is a thing of the past. Teradatadoes things differently now, which is why it's still considered the future of data warehousing. The flatfiles we create are merely used to store information contained within a table. The information on

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 16

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • these files is written in binary code, which is why the text seems garbled. It may look garbled, but itis perfectly written. When we Fastload the data back to a table it will look beautiful.

    Executing our fastload_creating_flatfile01.txt

    What our flat file looks like:

    And I thought French was tough; it's like they have a different word for everything

    We now have a flat file that contains all information found in the Employee_Table. We will be able touse this flat file for future exercises.

    BTEQ Export Modes ExplainedBelow is a list and description of our three data modes:

    Record Mode: (also called DATA mode): This is set by .EXPORT DATA. This will bring data backas a flat file. Each parcel will contain a complete record. Since it is not a report, there are noheaders or white space between the data contained in each column and the data is written to thefile (e.g., disk drive file) in native format. For example, this means that INTEGER data is written as a4-byte binary field. Therefore, it cannot be read and understood using a normal text editor.

    Field Mode (also called REPORT mode): This is set by .EXPORT REPORT. This is the defaultmode for BTEQ and brings the data back as if it was a standard SQL SELECT statement. Theoutput of this BTEQ export would return the column headers for the fields, white space, expandedpacked or binary data (for humans to read) and can be understood using a text editor.

    Indicator Mode: This is set by .EXPORT INDICDATA. This mode writes the data in data mode, butalso provides host operating systems with the means of recognizing missing or unknown data(NULL) fields. This is important if the data is to be loaded into another Relational Database System(RDBMS).

    The issue is that there is no standard character defined to represent either a numeric or characterNULL. So, every system uses a zero for a numeric NULL and a space or blank for a characterNULL. If this data is simply loaded into another RDBMS, it is no longer a NULL, but a zero or space.

    To remedy this situation, INDICATA puts a bitmap at the front of every record written to the disk.This bitmap contains one bit per field/column. When a Teradata column contains a NULL, the bit forthat field is turned on by setting it to a "1". Likewise, if the data is not NULL, the bit remains a zero.

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 17

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Therefore, the loading utility reads these bits as indicators of NULL data and identifies the column(s)as NULL when data is loaded back into the table, where appropriate.

    Since both DATA and INDICDATA store each column on disk in native format with known lengthsand characteristics, they are the fastest method of transferring data. However, it becomesimperative that you be consistent. When it is exported as DATA, it must be imported as DATA andthe same is true for INDICDATA.

    Again, this internal processing is automatic and potentially important. Yet, on a network-attachedsystem, being consistent is our only responsibility. However, on a mainframe system, you mustaccount for these bits when defining the LRECL in the Job Control Language (JCL). Otherwise, yourlength is too short and the job will end with an error.

    To determine the correct length, the following information is important. As mentioned earlier, one bitis needed per field output onto disk. However, computers allocate data in bytes, not bits. Therefore,if one bit is needed a minimum of eight (8 bits per byte) are allocated. Therefore, for every eightfields, the LRECL becomes 1 byte longer and must be added. In other words, for nine columnsselected, 2 bytes are added even though only nine bits are needed.

    With this being stated, there is one indicator bit per field selected. INDICDATA mode gives the Hostcomputer the ability to allocate bits in the form of a byte. Therefore, if one bit is required by the hostsystem, INDICDATA mode will automatically allocate eight of them. This means that from one toeight columns being referenced in the SELECT will add one byte to the length of the record. Whenselecting nine to sixteen columns, the output record will be two bytes longer.

    When executing on non-mainframe systems, the record length is automatically maintained.However, when exporting to a mainframe, the JCL (LRECL) must account for this additional 2 bytesin the length.

    DIF Mode: Known as Data Interchange Format, which allows users to export data from Teradata tobe directly utilized for spreadsheet applications like Excel, FoxPro and Lotus.

    The optional LIMIT is to tell BTEQ to stop returning rows after a specific number (n) of rows. Thismight be handy in a test environment to stop BTEQ before the end of transferring rows to the file.

    BTEQ EXPORT Example Using Record (DATA) ModeThe following is an example that displays how to utilize the export Record (DATA) option. Notice theperiods (.) at the beginning some of script lines. A period starting a line indicates a BTEQ command.If there is no period, then the command is an SQL command.

    When doing an export on a mainframe or a network-attached (e.g., LAN) computer, there is oneprimary difference in the .EXPORT command. The difference is the following:

    Mainframesyntax:

    .EXPORT DATA DDNAME = data definition statement name (JCL)

    LANsyntax:

    .EXPORT DATA FILE = actual file name

    The following example uses a Record (DATA) Mode format. The output of the exported data will bea flat file.

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 18

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • Employee_TableEmployee_No Last_Name First_Name Salary Dept_No2000000 Jones Squiggy 32800.50 ?1256349 Harrison Herbert 54500.00 4001333454 Smith John 48000.00 2001121334 Strickling Cletus 54500.00 400

    .LOGON CDW/sql01,whynot;Logon to TERADATA

    .EXPORT DATA FILE = C:\EMPS.TXTThis Export statement willbe in record (DATA) mode.

    SELECT * FROM SQL_Class.Employee_Table;The EMPS.TXT file will becreated as a flat file

    .QUITFinish the execution.

    Figure 2-6

    BTEQ EXPORT Example Using Field (Report) ModeThe following is an example that displays how to utilize the export Field (Report) option. Notice theperiods (.) at the beginning some of script lines. A period starting a line indicates a BTEQ commandand needs no semi-colon. Likewise, if there is no period, then the command is an SQL commandand requires a semi-colon.

    .LOGON CDW/sql01,whynot;Logon to TERADATA

    DATABASE SQL_Class;

    .EXPORT REPORT FILE = C:\EMPS.TXT

    SELECT * FROM Employee_Table;

    This Export statement will be in field (REPORT) mode.The EMPS.TXT file will be created as a report.

    .IF ERRORCODE > 0 THEN .GOTO Done

    SELECT * FROM Department_Table;

    BTEQ checks to ensure no errors occurred and selectsmore rows else GOTO Done.

    .EXPORT RESET

    .LABEL Done

    .QUIT

    Reverse previous export command and fall through toDone.

    Figure 2-7

    After this script has completed, the following report will be generated on disk.

    Employee_No Last_name First_name Salary Dept_No2000000 Jones Squiggy 32800.50 ?1256349 Harrison Herbert 54500.00 4001333454 Smith John 48000.00 2001121334 Strickling Cletus 54500.00 4001324657 Coffing Billy 41888.88 2002341218 Reilly William 36000.00 4001232578 Chambers Mandee 56177.50 100

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 19

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • 1000234 Smythe Richard 64300.00 102312225 Larkins Loraine 40200.00 300

    I remember when my mom and dad purchased my first Lego set. I was so excited about building myfirst space station that I ripped the box open, and proceeded to follow the instructions to completethe station. However, when I was done, I was not satisfied with the design and decided to makechanges. So I built another space ship and constructed another launching station. BTEQ exportworks in the same manner. As a user gains experience with BTEQ export, the easier it will get towork with the utility.

    With that being said, the following is an example that displays a more robust example of utilizing theField (Report) option. This example will export data in Field (Report) Mode format. The output ofthe exported data will appear like a standard output of a SQL SELECT statement. In addition,aliases and a title have been added to the script.

    .LOGON CDW/sql01,whynot;Logon to TERADATA

    .SET WIDTH 90

    .SET FORMAT ON

    .SET HEADING 'Employee Profiles'

    Set the format parameters for the final report

    .EXPORT REPORT FILE = C:\EMP_REPORT.TXTThis Export statement will be in field (REPORT)mode. The EMP_REPORT.TXT file will becreated as a report.

    SELECT Employee_No AS "Employee Number", Last_name AS "Last Name", First_name AS "First Name", Salary AS "Salary", Dept_No AS "Department Number"FROM Employee_Table;

    Specifies the columns that are being selected.Notice that the columns have an alias.

    .EXPORT RESET

    .QUIT

    Reverse previous export command effects

    Figure 2-8

    After then following script has been completed, the following report will be generated on disk.

    Employee ProfilesEmployee Number Last Name First Name Salary Department

    Number2000000 Jones Squiggy 32800.50 ?1256349 Harrison Herbert 54500.00 4001333454 Smith John 48000.00 2001121334 Strickling Cletus 54500.00 4001324657 Coffing Billy 41888.88 2002341218 Reilly William 36000.00 4001232578 Chambers Mandee 56177.50 1001000234 Smythe Richard 64300.00 102312225 Larkins Loraine 40200.00 300

    From the above example, a number of BTEQ commands were added to the export script. Below is areview of those commands.

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 20

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • The WIDTH specifies the width of screen displays and printed reports, based on charactersper line.

    The FORMAT command allows the ability to enable/inhibit the page-oriented format option.

    The HEADING command specifies a header that will appear at the top every page of areport.

    BTEQ IMPORT ExampleBTEQ can also read a file from the hard disk and incorporate the data into SQL to modify thecontents of one or more tables. In order to do this processing, the name and record description ofthe file must be known ahead of time. These will be defined within the script file.

    Format of the IMPORT command:

    .IMPORT { FILE | DNAME } = [,SKIP=n]

    The script below introduces the IMPORT command with the Record (DATA) option. Notice theperiods (.) at the beginning some of script lines. A period starting a line indicates a BTEQ command.If there is no period, then the command is an SQL command.

    The SKIP option is used when you wish to bypass the first records in a file. For example, amainframe tape may have header records that should not be processed. Other times, maybe the jobstarted and loaded a few rows into the table with a UPI defined. Loading them again will cause anerror. So, you can skip over them using this option.

    The following example will use a Record (DATA) Mode format. The input of the imported data willpopulate the Employee_Table.

    .SESSIONS 4

    .LOGON CDW/sql01,whynot;

    Specify the number of SESSIONS to establishwith Teradata

    .IMPORT DATA FILE = C:\EMPS.TXT, SKIP = 2Logon to TERADATA

    .QUIET ON

    .REPEAT *

    USING Emp_No (INTEGER), L_name (CHAR(20)), F_name (VARCHAR(12)), Salary (DECIMAL(8,2)), Dept_No (SMALLINT)

    Specify DATA mode, name the file to read"EMPS.TXT", but skip the first 2 records.Limit messages out. Loop in this script untilend of records in file.

    INSERT INTO Employee_Table (Employee_No, Last_name, First_name, Salary, Dept_No)

    The USING Specifies the field in the input fileand names them.

    VALUES (:Emp_No, :L_name, :F_name, :Salary, :Dept_No) ;

    Specify the insert parameters for theemployee_tableSubstitutes data from the fields into the SQLcommand.

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 21

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • .QUIT Exit the script after all data read and rowsinserted.

    Figure 2-9

    From the above example, a number of BTEQ commands were added to the import script. The nextpage contains a review of those commands.

    .QUIET ON limits BTEQ output to reporting only errors and request processing statistics.Note: Be careful how you spell .QUIET, else forgetting the E becomes .QUIT and it will.

    .REPEAT * causes BTEQ to read a specified number of records or until EOF. The default isone record. Using REPEAT 10 would perform the loop 10 times.

    The USING defines the input data fields and their associated data types coming from thehost.

    The following builds upon the IMPORT Record (DATA) example above. The example below will stillutilize the Record (DATA) Mode format. However, this script adds a CREATE TABLE statement. Inaddition, the imported data will populate the newly created Employee_Profile table.

    .SESSIONS 2Specify the number of SESSIONS to establishwith Teradata

    .LOGON CDW/sql01,whynot;Logon to TERADATA

    DATABASE SQL_Class;Make the default database SQL_Class

    CREATE TABLE Employee_Profile ( Employee_No INTEGER, Last_name CHAR(20), First_name VARCHAR(12), Salary DECIMAL(8,2), Dept_No SMALLINT )UNIQUE PRIMARY INDEX (Employee_No) ;.IMPORT INDICDATA FILE = C:\IND-EMPS.TXT

    This statement will create the Employee_Profiletable.

    .QUIET ONThis import statement specifies INDICDATAmode. The input file is from a LAN file calledIND-EMPS.TXT.

    .REPEAT 120Quiet on limits the output to reporting onlyerrors and processing statistics.

    USING Employee_No (INTEGER), Last_name (CHAR(20)), First_name (VARCHAR(12)), Salary (DECIMAL(8,2)), Dept_No (SMALLINT)

    This causes BTEQ to read the first 120 recordsfrom the file.

    INSERT INTO Employee_Profile (Employee_No, Last_name, First_name, Salary, Dept_No)

    The USING Specifies the parameters of the inputfile.

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 22

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • VALUES (:Employee_No, :Last_name, :First_name, :Salary, :Dept_No) ;

    Specify the insert parameters for theemployee_profile.

    .LOGOFF

    .QUIT

    Substitute the values to be inputted into the SQLcommand.

    Figure 2-10Notice that some of the scripts have a .LOGOFF and .QUIT. The .LOGOFF is optional becausewhen BTEQ quits, the session is terminated. A logoff makes it a friendly departure and also allowsyou to logon with a different user name and password.

    Determining Out Record LengthsSome hosts, such as IBM mainframes, require the correct LRECL (Logical Record Length)parameter in the JCL, and will abort if the value is incorrect. The following page will discuss how tofigure out the record lengths.

    There are three issues involving record lengths and they are:

    Fixed columns

    Variable columns

    NULL indicators

    Fixed Length Columns: For fixed length columns you merely count the length of the column. Thelengths are:

    INTEGER 4 bytes

    SMALLINT 2 bytes

    BYTEINT 1 byte

    CHAR(10) 10 bytes

    CHAR(4) 4 bytes

    DATE 4 bytes

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 23

    Reprinted for ibmk[email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • DECIMAL(7,2) 4 bytes (packed data, total digits / 2 +1)

    DECIMAL(12,2) 7 bytes

    Variable columns: Variable length columns should be calculated as the maximum value plus two.The two bytes are for the number of bytes for the binary length of the field. In reality you can savemuch space because trailing blanks are not kept. The logical record will assume the maximum andadd two bytes as a length field per column.

    VARCHAR(8) 10 bytes

    VARCHAR(10) 12 bytes

    Indicator columns: As explained earlier, the indicators utilize a single bit for each field. If your recordhas 8 fields (which require 8 bits), then you add one extra byte to the total length of all the fields. Ifyour record has 9-16 fields, then add two bytes.

    BTEQ Return Codes

    Return codes are two-digit values that BTEQ returns to the user after completing each job or task.The value of the return code indicates the completion status of the job or task as follows:

    Return Code Descirption

    00 Job completed with no errors.

    02 User alert to log on to the Teradata DBS.

    04 Warning error.

    08 User error.

    12 Severe internal error.

    You can over-ride the standard error codes at the time you terminate BTEQ. This might be handyfor debugging purposes. The error code or "return code" can be any number you specify using oneof the following:

    Override Code Description

    .QUIT 15

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 24

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • .EXIT 15

    BTEQ CommandsThe BTEQ commands in Teradata are designed for flexibility. These commands are not useddirectly on the data inside the tables. However, these 60 different BTEQ commands are utilized infour areas.

    Session Control Commands

    File Control Commands

    Sequence Control Commands

    Format Control Commands

    Session Control Commands

    ABORT Abort any and all active running requests and transactions for a session, butdo not exit BTEQ.

    DEFAULTS Reset all BTEQ Format command options to their defaults. This will utilizethe default configurations.

    EXIT Immediately end the current session or sessions and exit BTEQ.HALT EXECUTION Abort any and all active running requests and transactions and EXIT BTEQ. LOGOFF End the current session or sessions, but do not exit BTEQ.LOGON Starts a BTEQ Session. Every user, application, or utility must LOGON to

    Teradata to establish a session.QUIT End the current session or sessions and exit BTEQ.SECURITY Specifies the security level of messages between a network-attached

    system and the Teradata Database.SESSIONS Specifies the number of sessions to use with the next LOGON command.SESSIONCHARSET

    Specifies the name of a character set for the current session or sessions.

    SESSIONSQLFLAG

    Specifies a disposition of warnings issued in response to violations of ANSIsyntax. The SQL will still run, but a warning message will be provided. Thefour settings are FULL, INTERMEDIATE, ENTRY, and NONE.

    SESSIONTRANSACTION

    Specifies whether transaction boundaries are determined by Teradata SQLor ANSI SQL semantics.

    SHOWCONTROLS

    Displays all of the BTEQ control command options currently configured.

    SHOW VERSIONS Displays the BTEQ software release versions.TDP Used to specify the correct Teradata server for logons for a particular session.

    Figure 2-11

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 25

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • File Control Commands

    These BTEQ commands are used to specify the formatting parameters of incoming and outgoinginformation. This includes identifying sources and determining I/O streams.

    CMS Execute a VM CMS command inside the BTEQ environment.ERROROUT Write error messages to a specific output file.EXPORT Open a file with a specific format to transfer information directly from the

    Teradata database. HALTEXECUTION

    Abort any and all active running requests and transactions and EXIT BTEQ.

    FORMAT Enable/inhibit the page-oriented format command options.IMPORT Open a file with a specific format to import information into Teradata.INDICDATA One of multiple data mode options for data selected from Teradata. The modes

    are INDICDATA, FIELD, or RECORD MODE.OS Execute an MS-DOS, PC-DOS, or UNIX command from inside BTEQ.QUIET Limit BTEQ output displays to all error messages and request processing

    statistics.RECORDMODE One of multiple data mode options for data selected from Teradata.

    (INDICDATA, FIELD, or RECORD).REPEAT Submit the next request a certain amount of timesRUN Execute Teradata SQL requests and BTEQ commands directly from a

    specified run file.TSO Execute an MVS TSO command from inside the BTEQ environment.

    Figure 2-12

    Sequence Control CommandsThese commands control the sequence in which Teradata commands operate.

    ABORT Abort any active transactions and requests.ERRORLEVEL Assign severity levels to particular error numbers.EXIT End the current session or sessions and exit BTEQ.GOTO Skip all intervening commands and resume after branching forward to the

    specified label.HANG Pause BTEQ processing for a specific amount of time.IFTHEN Test a stated condition, and then resume processing based on the test results.LABEL The GOTO command will always GO directly TO a particular line of code based

    on a label.MAXERROR Specifies a maximum allowable error severity level.QUIT End the current session or sessions and exit BTEQ.REMARK Place a comment on the standard output stream.REPEAT Submit the next request a certain amount of times.

    Figure 2-13Format Control Commands

    These commands control the formatting for Teradata and present the data in a report mode to thescreen or printer.

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 26

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

  • DEFAULTS Reset all BTEQ Format command options to their defaults. This will utilize thedefault configurations.

    ECHOREQ Enable the Echo required function in BTEQ returning a copy of eachTeradata SQL request and BTEQ command to the standard output stream.

    EXPORT Open a file with a specific format to transfer information directly from theTeradata database.

    FOLDLINE Split or fold each line of a report into multiple lines.FOOTING Specify a footer to appear at the bottom of every report page.FORMAT Enable/inhibit the page-oriented format command options.IMPORT Open a file with a specific format to transfer or IMPORT information directly

    to Teradata.NULL Specifies a character or string of characters to represent null values returned

    from Teradata.OMIT Omit specific columns from a report.PAGEBREAK Ejects a page whenever a specified column changes values.PAGELENGTH Specifies the page length of printed reports based on lines per page.QUIET Limit BTEQ output displays to all error messages and request processing

    statistics.RECORDMODE One of multiple data mode options for data selected from Teradata.

    (INDICDATA, FIELD, or RECORD).RETCANCEL Cancel a request when the specified value of the RETLIMIT command option

    is exceeded.RETLIMIT Specifies the maximum number of rows to be displayed or written from a

    Teradata SQL request.RETRY Retry requests that fail under specific error conditions. RTITLE Specify a header appearing at the top of all pages of a report.SEPARATOR Specifies a character string or specific width of blank characters separating

    columns of a report.SHOWCONTROLS Displays all of the BTEQ control command options currently configured.SIDETITLES Place titles to the left or side of the report instead of on top.SKIPLINE Inserts blank lines in a report when the value of a column changes specified

    values.SUPPRESS Replace each and every consecutively repeated value with completely-blank

    character strings.TITLEDASHES Display dash characters before each report line summarized by a WITH

    clause.UNDERLINE Display a row of dash characters when the specified column changes

    values.WIDTH Specifies the width of screen displays and printed reports, based on characters

    per line.

    Figure 2-14

    Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 27

    Reprinted for [email protected], IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

    Table of Contents Chapter 2: BTEQ Why it is Called BTEQ? Logging onto BTEQ Using BTEQ to Submit Queries Submitting SQL in BTEQ's Interactive Mode

    WITH BY Statement Transactions in Teradata Mode Alternative Transactions in Teradata Mode Transactions in ANSI Mode Rollback Advantages to ANSI Mode Creating a Batch Script for BTEQ Running your Batch Script in BTEQ Results from a BTEQ Batch Script Placing our BTEQ Output to a file Reading out BTEQ output from the Text File Using BTEQ Conditional Logic Using BTEQ to Export Data Executing our BTEQ Script to Export Data BTEQ Export Modes Explained BTEQ EXPORT Example Using Record (DATA) Mode BTEQ EXPORT Example Using Field (Report) Mode BTEQ IMPORT Example Determining Out Record Lengths BTEQ Commands Session Control Commands File Control Commands

    Sequence Control Commands Format Control Commands