I/O in PL/SQL: Managing Files Displaying Output to Screen ... · Author: McDonald's Due Created...

Copyright 2000-2010 Steven Feuerstein -

I/O in PL/SQL:

Managing Files

Displaying Output to Screen

Sending Email

Steven Feuerstein PL/SQL Evangelist, Quest Software

[email protected]

PL/SQL Obsession - www.ToadWorld.com/SF

Fast Track PL/SQL


How to benefit most from this session

Watch, listen, focus on concepts and principles.

Download and use any of my training materials:

You have my permission to use all these materials to do internal trainings and build your own applications.

– But remember: they are not production ready.

– You must test them and modify them to fit your needs.

filename_from_demo_zip.sql

Download and use any of my scripts (examples, performance scripts, reusable code) from the same location: the demo.zip file.

http://www.ToadWorld.com/SF PL/SQL Obsession

And there's more!


I/O in PL/SQL

You don't always read information from and

write data to a relational table.

Sometimes you need to read/write files.

Sometimes you need to display information

to the screen (system output).

Sometimes you need to send emails.

Sometimes....you need to take advantage of

those built-in (supplied) packages.


I/O in PL/SQL – Relevant Oracle Technologies

UTL_FILE – read and write OS files

DBMS_LOB – manipulate BFILEs

External tables

DBMS_OUTPUT – send text to the screen

UTL_MAIL – send emails

Use external procedures (C code) and Java

stored procedures for OS activities.

– I will touch on Java extensions to UTL_FILE.



>> File IO in PL/SQL

UTL_FILE allows you to read from and write

to operating system files on the database

server.

Prior to Oracle9i Release 2, it was a fairly

primitive utility, but 9.2 offered many

enhancements.

–Remove, copy, rename files.

–Work with database directories. Hurray!

UTL_FILE Logistics

The UTL_FILE package is defined in the

utlfile.sql file of the RDBMS/Admin directory.

– Ask your DBA for read-only access.

Includes a long list of file-specific exceptions.

– Documents exceptions for each subprogram.

– Most of exceptions raised are non-specific and

minimally useful, as in "Invalid file operation."

UTL_FILE is not installed in Oracle XE.


$ORACLE_HOME/Rdbms/Admin

Enabling UTL_FILE Access

While UTL_FILE is available by default to all

schemas connected to Oracle, also by

default you have no ability to actually

read/write files.

Oracle is very cautious.

– They don't want you to accidentally delete Oracle

database files, among other things!

So a user with DBA authority must authorize

access to specific directories.



Authorizing Directory Access – UTL_FILE_DIR

In the bad old days of UTL_FILE, the only way

to enable a directory for read/write was the

UTL_FILE_DIR parameter.

– And after you change it, you have to "bounce" the

database before UTL_FILE will recognize the

directory.

Very awkward and inflexible. Not well suited for

the 24x7 world of the 21st century.

utl_file_dir = /tmp utl_file_dir = /accounts/newdev


Authorizing Directory Access – the modern way

Now you can take advantage of database

directory objects instead of UTL_FILE_DIR.

– A directory is a database object, define with a

CREATE statement.

– Once created, it is available for use immediately.

Special privilege needed to create directory.

– It is generally a DBA-only action.

Directory names are case-sensitive.

GRANT CREATE ANY DIRECTORY TO <schema_name>


Working with Database Directories

The directory hides the actual, hard-coded

location.

create_directories.sql

utlfile_92.sql

CREATE OR REPLACE DIRECTORY ERROR_LOG AS '/tmp/apps/log';

SELECT owner, directory_name, directory_path FROM ALL_DIRECTORIES;

GRANT READ ON DIRECTORY error_log TO SCOTT;

Check the ALL_DIRECTORIES view to determine

which directories you can use.

Can grant read and/or write privileges to schemas.


Opening a File

Specify file location, name, operation type and (optionally) the

maximum line size.

– Types are 'R' for Read, 'W' for Write and 'A' for Append.

– Linesize default is 1024, maximum of 32767

The FOPEN function returns a record ("file handle") based on

the UTL_FILE.FILE_TYPE.

– Contains three fields (ID, datatype and byte_mode).

Up to 50 files open per session.

DECLARE fid UTL_FILE.FILE_TYPE; BEGIN fid := UTL_FILE.FOPEN ('TEMP_DIR', 'test.txt', 'W', max_linesize => 32767); UTL_FILE.fclose (fid); END;

fopen.sql

fopen_write_conflict.sql


Is a File Open?

Test to see if file is open with the IS_OPEN

function.

– In actuality, this function simply returns TRUE if the

file handle's id field is NOT NULL.

– Not much of a test...

DECLARE fid UTL_FILE.file_type; BEGIN fid := UTL_FILE.fopen ('TEMP', 'test.txt', 'W', max_linesize => 32767); IF UTL_FILE.is_open (fid) THEN DBMS_OUTPUT.put_line ('File is open'); ELSE DBMS_OUTPUT.put_line ('File is closed'); END IF; UTL_FILE.fclose (fid); END;


Reading from a File

Can only read from a file opened with the "R" mode.

Can only read one line at a time.

The NO_DATA_FOUND exception is raised if you read

past the end of the file.

– That's more than a little bit silly – and a bad idea.

DECLARE

fid UTL_FILE.FILE_TYPE;

BEGIN

fid := UTL_FILE.FOPEN ('TEMP', 'test.txt', 'R');

UTL_FILE.GET_LINE (fid, myline);

UTL_FILE.FCLOSE (fid);

END;

read_from_file.sql

infile.sf

eqfiles.sf

A better GET_LINE function

Reading past the end of a file isn't an error.

It's what you do when you read the full

contents of a file.

But UTL_FILE's architecture requires that

your block terminates and you trap the

exception.

– Leading to scary and hard-to-manage code.

So "hide" GET_LINE inside your own file-

reading program.


exec_ddl_from_file.sql

getnext.sp

file_to_collection.sql


Writing to a File

PUT puts a line without a newline terminator.

PUT_LINE = PUT plus newline character.

PUTF allows for some formatting and substitution.

– \n is new line, %s substitutes strings.

DECLARE

fid UTL_FILE.FILE_TYPE;

BEGIN

fid := UTL_FILE.FOPEN ('TEMP', 'test.txt', 'W');

UTL_FILE.PUT_LINE (fid, 'UTL_FILE');

UTL_FILE.PUT (fid, 'is so much fun');

UTL_FILE.PUTF (fid, ' that I never\nwant to %s', 'stop!');

UTL_FILE.FCLOSE (fid);

END;

create_file.sp

read_and_write.sql

genaa.sql

write_to_file.sql

display_file.sql

Flushing content to file

A call to PUT_LINE does not necessarily

mean the text is in the file.

– In fact, it most likely is not.

– Writing out to the actual file is performed

asynchronously.

You can specify each new line flushed out in

call to PUT_LINE.

Or call UTL_FILE.FFLUSH to write the

accumulated buffer contents out to the file.


put_line_performance.tst


Closing a File

Very important to close files when you are done with

them.

– And also close them when an exception is raised.

Otherwise the file is left open and data will not be

flushed out to disk.

Close all open files in session with FCLOSE_ALL.

DECLARE fid UTL_FILE.FILE_TYPE; BEGIN fid := UTL_FILE.FOPEN ('TEMP', 'test.txt', 'R'); UTL_FILE.GET_LINE (fid, myline); UTL_FILE.FCLOSE (fid); EXCEPTION WHEN UTL_FILE.READ_ERROR THEN UTL_FILE.FCLOSE (fid); END;

High Level Functionality of UTL_FILE

Copy a file with FCOPY

Delete a file with FREMOVE

Rename and move a file with FRENAME

Find current position in file with FGETPOS

Retrieve characteristics of file with

FGETATTR



Copy a File

Copy all lines in the file, or a specify range of lines by start and end line numbers.

You don't open/close the file.

DECLARE file_suffix VARCHAR2 (100) := TO_CHAR (SYSDATE, 'YYYYMMDDHH24MISS'); BEGIN UTL_FILE.fcopy ( src_location => 'DEVELOPMENT_DIR', src_filename => 'archive.zip', dest_location => 'ARCHIVE_DIR', dest_filename => 'archive' || file_suffix || '.zip' ); END;

fcopy.sql

backup_with_fcopy..sql

fileIO92.pkg


Remove a File

If no error is raised, then you deleted successfully

BEGIN UTL_FILE.fremove ( src_location => 'DEVELOPMENT_DIR', src_filename => 'archive.zip' ); EXCEPTION /* Don't forget to check for errors! */ WHEN UTL_FILE.delete_failed THEN ... Deal with failure to remove END;

fremove.sql

fileIO92.pkg


Rename/move a File

Use FRENAME to change the name of a file.

If you rename with a new directory, you will

move the file.

DECLARE file_suffix VARCHAR2 (100) := TO_CHAR (SYSDATE, 'YYYYMMDD'); BEGIN -- Rename/move the entire file in a single step. UTL_FILE.frename ( src_location => 'DEVELOPMENT_DIR', src_filename => 'archive.zip', dest_location => 'ARCHIVE_DIR', dest_filename => 'archive' || file_suffix || '.zip', overwrite => FALSE );END;

frename.sql

fileIO92.pkg

Find current position in file

Use the FGETPOS function to get the current

relative offset position within a file, in bytes.

Returns 0 when you are writing to a file.


DECLARE fid UTL_FILE.file_type; l_line VARCHAR2 (32767); BEGIN fid := UTL_FILE.fopen ('TEMP', 'temp.txt', 'R', max_linesize => 32767); FOR indx IN 1 .. 10 LOOP UTL_FILE.get_line (fid, l_line); DBMS_OUTPUT.put_line (UTL_FILE.fgetpos (fid)); END LOOP; UTL_FILE.fclose (fid); END;

fgetpos.sql


Obtaining attributes of a file

CREATE OR REPLACE FUNCTION flength ( location_in IN VARCHAR2, file_in IN VARCHAR2 ) RETURN PLS_INTEGER IS TYPE fgetattr_t IS RECORD ( fexists BOOLEAN, file_length PLS_INTEGER, block_size PLS_INTEGER ); fgetattr_rec fgetattr_t; BEGIN UTL_FILE.fgetattr ( location => location_in, filename => file_in, fexists => fgetattr_rec.fexists, file_length => fgetattr_rec.file_length, block_size => fgetattr_rec.block_size ); RETURN fgetattr_rec.file_length; END flength;

How big is a file? What is its block size? Does the file exist?

All valuable questions.

All answered with a call to UTL_FILE.FGETATTR.

flength.sql

fileIO92.pkg

Extending file operations with Java

There are still lots of restrictions in UTL_FILE.

– No higher-level file operations (change privileges, create directory, files in directory, random access to contents).

– Limitations on files you can access (no mapped files, no use of environmental variables).

If you hit these limits, you could consider a switch to Java. – But you need the DBA to grant you the Java

privileges to interact with the OS.


JFile.java

xfile.pkg

External Tables – query from files!

External tables are defined as schema

objects, but the data is stored in a file.

Query the contents of the file using standard

SQL syntax.

Specify structure of file using SQL Loader

control file syntax.


External Tables – simple example

File contains department information:


1,Human Resources,Chicago 2,Information Services,New York

My create table statement "points" to the file:

CREATE TABLE departments_ext ( deptno NUMBER(6), dname VARCHAR2(20), loc VARCHAR2(25) ) ... LOCATION ('departments.ctl') )

external_tables*.*

External Tables – 11.2 example: get files in directory

In 11.2, Oracle extended the external table

concept to include a preprocessor option.

You specify an operating system file that will

run when you execute a query against the

external table.

Copyright 2000-2010 Steven Feuerstein - external_tables_files_in_dir.*

CREATE TABLE files_in_temp ( file_name VARCHAR2(1000) ) ORGANIZATION external ( TYPE oracle_loader DEFAULT DIRECTORY TEMP ACCESS PARAMETERS ( RECORDS DELIMITED BY NEWLINE PREPROCESSOR execute_directory: 'show_files.bat' FIELDS TERMINATED BY WHITESPACE ) LOCATION ('placeholder.txt') )

DBMS_LOB and BFILEs

Oracle allows you to also manipulate binary

files or BFILEs through the DBMS_LOB

package.

– The intention is to work with really big files, but

they don't have to be.

A BFILE is a pointer to an OS file.

This will not constitute a training on working

with LOBs!


DBMS_LOB BFILE functionality

Open and close BFILEs

Compare two BFILES with COMPARE.

Get information about a BFILE

– Name and location? exists? length?

Read contents of BFILE

– They are read-only structures!

And more... – Read BFILE into LOB

– Perform INSTR and SUBSTR operations on a BFILE.


Open and Close BFILES

Initialize BFILE pointer with the BFILENAME

function.

Then it's all as expected.


DECLARE l_bfile BFILE := BFILENAME ('DEMO', 'exec_ddl_from_file2.sql'); BEGIN DBMS_OUTPUT.put_line ('Exists? ' || DBMS_LOB.fileexists (l_bfile)); DBMS_OUTPUT.put_line ('Open before open? ' || DBMS_LOB.fileisopen (l_bfile)); DBMS_LOB.fileopen (l_bfile); DBMS_OUTPUT.put_line ('Open after open? ' || DBMS_LOB.fileisopen (l_bfile)); DBMS_LOB.fileclose (l_bfile); DBMS_OUTPUT. put_line ('Open after close? ' || DBMS_LOB.fileisopen (l_bfile)); END;

bfile_open_close.sql

Compare two BFILES with COMPARE

Use the COMPARE program to find out if two

BFILEs are the same.

– Specify amount of file you want to compare, and

the offset locations in each. Copyright 2000-2010 Steven Feuerstein -

DECLARE l_bfile1 BFILE := BFILENAME ('TEMP', 'file1.txt'); l_bfile2 BFILE := BFILENAME ('TEMP', 'file2.txt'); BEGIN DBMS_LOB.fileopen (l_bfile1); DBMS_LOB.fileopen (l_bfile2); DBMS_OUTPUT.put_line ( 'Equal? ' || DBMS_LOB.compare (file_1 => l_bfile1, file_2 => l_bfile2 , amount => 33, offset_1 => 1, offset_2 => 1 ) ); DBMS_LOB.fileclose (l_bfile1); DBMS_LOB.fileclose (l_bfile2); END;

bfile_compare.sql

Get information about a BFILE

Get the name of the file: FILEGETNAME

Get length of the file. GETLENGTH

Does the file exist? FILEEXISTS

Is the file open? FILEISOPEN


bfile_demo.sql

Read contents of BFILE

You always read into a RAW variable.

– After all, these are supposed to be binary files!

The amount is IN OUT – it returns the actual

number of bytes read.


DECLARE l_bfile BFILE := BFILENAME ('DEMO', 'exec_ddl_from_file.sql'); l_contents RAW (32767); l_amount PLS_INTEGER := 100; BEGIN DBMS_LOB.fileopen (l_bfile); DBMS_LOB.read (l_bfile, l_amount, 1, l_contents); dbms_output.put_line (l_contents); DBMS_LOB.fileclose (l_bfile); END;

bfile_read.sql

Writing data to system output

Use DBMS_OUTPUT to display information on

your screen.

Best known built-in package and most abused.

– Often used as a cheap and easy debugging or

tracing mechanism.

Historically crippled, today many limitations are

removed.

– Unlimited buffer size

– Display strings of up to 32767 characters.

Copyright 2000-2010 Steven Feuerstein - SET SERVEROUTPUT ON SIZE UNLIMITED

Enable and disable output

Calls to DBMS_OUTPUT.PUT_LINE and

others will do nothing unless output is enabled.

This is often performed by host environments.

– SET SERVEROUTPUT ON

You can also enable output with a call to

DBMS_OUTPUT.ENABLE, but it is not

enough.

– The host environment must also be ready to

retrieve the contents of the buffer.


Put information on screen

All information is placed in buffer, which is

retrieved with a call to GET_LINES.

PUT_LINE puts a line of text and end-of-line

marker.

PUT puts text in buffer without end-of-line

marker.

NEW_LINE adds end-of-line marker.


Get information from the buffer

GET_LINE retrieves the next line from the

buffer.

GET_LINES returns all lines in buffer into a

PL/SQL collection.

It is very unlikely that you will ever run into a

need to call the GET* procedures.

Copyright 2000-2010 Steven Feuerstein - dbms_output_demo.sql

Summary: DBMS_OUTPUT

Avoid calls to DBMS_OUTPUT.PUT_LINE in

your production code.

Do not use this built-in for tracing or

debugging.

Encapsulate it to make it more useful and

less problematic.

The demo.zip contains numerous

alternatives.


Alternatives to DBMS_OUTPUT

The p package

– Use "p.l" to say "show me", many overloadings of

datatypes.

The watch package

– Write information to screen or database pipe

Quest Error Manager

– www.ToadWorld.com/SF

– More sophisticated tracing and error

management


p.pks/pkb

watch_noplv.pkg


Sending mail from a PL/SQL program

UTL_MAIL makes it much easier to send

email from within PL/SQL by hiding some of

the complexities of UTL_SMTP.

To use UTL_MAIL...

– Set the SMTP_OUTPUT_SERVER parameter.

– Install the utlmail.sql and prvtmail.plb files under

SYS. That's right - it is not installed by default.

– Grant EXECUTE on UTL_MAIL as desired.


Send without attachment

The interface to the SEND program mimics

the basic "send email" form of Outlook and

other email programs.

BEGI UTL_MAIL.send ( sender => '[email protected]' ,recipients => '[email protected], [email protected]' ,cc => '[email protected]' ,bcc => '[email protected]' ,subject => 'Cool new API for sending email' ,message => 'Hi Ya''ll, Sending email in PL/SQL is *much* easier with UTL_MAIL in 10g. Give it a try! Mailfully Yours, Bill' ); END;


Send with attachment

You can attach RAW or VARCHAR2 content

as an attachment (up to 32K).

BEGIN UTL_MAIL.send_attachment_raw ( sender => '[email protected]' ,recipients => '[email protected], [email protected]' ,cc => '[email protected]' ,bcc => '[email protected]' ,subject => 'Cool new API for sending email' ,message => '...' ,attachment => '...' /* Content of the attachment */ ,att_inline => TRUE /* Attachment in-line? */ ,att_filename => '...' /* Name of file to hold the attachment after the mail is received. */ ); END;

The World is Bigger Than Relational Tables

Most of our work in PL/SQL involves reading

and writing the contents of tables.

Sometimes, though, we need to interact with

the world "out there."

In these cases, the built-in packages help by

extending PL/SQL.

UTL_FILE has improved over time, but could use a

lot more work.

DBMS_OUTPUT is unavoidable, but problematic.


I/O in PL/SQL: Managing Files Displaying Output to Screen ... · Author: McDonald's Due Created...

Documents

Transcript of I/O in PL/SQL: Managing Files Displaying Output to Screen ... · Author: McDonald's Due Created...