SQL Loader
-
Upload
venuoracle9 -
Category
Documents
-
view
60 -
download
3
description
Transcript of SQL Loader
![Page 1: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/1.jpg)
1
Basic Concepts
![Page 2: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/2.jpg)
2
Sql *Loader
It is a high speed data loading utility supplied by Oracle that loads data from external files into table in an Oracle Database.
It can accepts data in variety of formats, can perform transformation, filtering and can load into multiple tables from multiple files in same loading session
![Page 3: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/3.jpg)
3
Requirements- For Sql *Loader
Tables to be loaded must already exists in the Database. SQL *Loader never creates tables, it loads existing tables.
Table may be empty or may already contain data. Privileges:
INSERT privilege
DELETE privilege, when using REPLACE or TRUNCATE insert option
![Page 4: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/4.jpg)
4
Files used in Sql *Loader
Control File - .ctl file Data File - .dat file Log File - .log file Bad File - .bad file Discard File - .dis file
![Page 5: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/5.jpg)
5
Control File
It controls the Behavior of Sql *Loader
Source of Data to be loaded
Destination of Data to be loaded
Filtering of Data before Loading
Transformation of Data before Loading
Relating the Data file fields to table columns
![Page 6: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/6.jpg)
6
Log File
It is a record of SQL *Loader’s activities during a load session
Control, Data, Bad, Discard file Name. Values of several command line parameters. Detailed breakdown of the fields & datatypes in data file
that was loaded. Error Message for records that cause error. Message indicating when records have been discarded. Summary of the Load (no: of records read from file, no: of
rows rejected because of errors, no: of rows discarded & elapsed time of load)
![Page 7: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/7.jpg)
7
Data File – Contains the data to be loaded, it is optional
Discard File – Contains the data that are discarded by ‘when’ clause in control file, it is optional.
Bad File – Contains the data which are not loaded due to some errors, it is not optional, if any one error occurred, SQL *Loader will create the bad file and write the offending input records into it.
![Page 8: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/8.jpg)
8
SQL *Loader Overview
SQL *Loader
Input Data File Control File
LogFile
BadFile
DiscardFile
DataBase
![Page 9: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/9.jpg)
9
Loading Method
Basically there are 3 Loading Methods
Conventional Path Load
Direct Path Load
External Table Load( from Oracle 9i).
![Page 10: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/10.jpg)
10
Conventional Path Load
Bind Array (Row Insert) is created by Sql *Loader based on field specification in control file
Data from the Bind Array are then insert into the table by the DB server, if data satisfies with the corresponding column datatype
![Page 11: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/11.jpg)
11
Direct Path Load Parses the data according to the field specification
in Control File Converts the data to Column datatype (not to field
datatype in .ctl file) and builds Column array. Column array is passed to block formatter Newly formatted blocks are directly written
directly to the database file bypassing most SQL Processing
PARALLEL loading is possible.
![Page 12: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/12.jpg)
12
PARALLEL Loading
Setup has to done, as the text file must be broken into several smaller files
We can run several SQLLDR session in parallel for each broken data files
It will increase the performance, reduce the time for loading
![Page 13: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/13.jpg)
13
Advantage & Disadvantage of Direct Path Load
Advantage :
Very High Speed
Disadvantage :
Cannot Load Object Types, Collection Types (Nested Table, VARRAY),LOB(CLOB,NCLOB,BLOB, BFILE)
![Page 14: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/14.jpg)
14
CHAR(5) Ctrl File Spec CHAR(5)
Field Conversion
aaa bbb
Column 1 Column 2
Table
a a a_ _ b b b
CHAR(5) Column Data Type VARCHAR(5)
Field 1 Field 2DATA FILE
BIND ARRAY
DATABASE
SQL LOADER
SERVER
a a a b b b
![Page 15: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/15.jpg)
15
Record Filtering Record in Data file
SQL *LoaderField Processing
SQL *LoaderWhen-clauseEvaluation
DatabaseServer
Database
BadFile
DiscardFile
Read in
Accepted
SelectedDiscarded
InsertedRejected
RejectedControlfile
![Page 16: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/16.jpg)
16
![Page 17: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/17.jpg)
17
SQL> desc sampledata; Name Null Type
------------------------------- -------- ---- N1 VARCHAR2(25) N2 VARCHAR2(25)
LOAD DATA INFILE '/u01/xxvis/data1.dat'
replaceinto table sampledata
fields terminated by X'09'(
n2, n1
)
SQL> select * from sampledata;
N1 N2------------------------- ----
3456 12 0111 7891
3333 1234
Mapping between data fields in control file and columns of the table
12 34567891 01111234 3333
![Page 18: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/18.jpg)
18
Data with in Control File
control1.ctlload datainfile *into table sql_ldr1 REPLACE(n position(1:2))begindata34567
Sql_ldr1N34567
$ sqlldr control=control1.ctl userid=apps/secretone
![Page 19: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/19.jpg)
19
Data in DATA filedata2.dat111 222 666333 444 666555 666 666777 888 666999 000 666
control2.ctlload datainfile 'data2.dat'insert into table sql_ldr2
replace (n1 position(1:3),n2 position(5:7),n3 position(9:11))
Sql_ldr2:N1 N2 N3
111 222 666
333 444 666
555 666 666
777 888 666
999 000 666
$ sqlldr control=control2.ctl userid=apps/secretone
![Page 20: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/20.jpg)
20
Appending to a tabledata3.dat111 222 666333 444 666555 666 666777 888 666999 000 666
control3.ctlload datainfile 'data3.dat'insert into table sql_ldr3 append(n1 position(1:3),n2 position(5:7),n3 position(9:11))
Sql_ldr3 ( Before Loading):---------------------------------
N1 N2 N3
------ --------- ---------
111 222 666
333 444 666
555 666 666
777 888 666
999 0 666
Sql_ldr3 ( After Loading):---------------------------------
N1 N2 N3
------ --------- ---------
111 222 666
333 444 666
555 666 666
777 888 666
999 0 666
111 222 666
333 444 666
555 666 666
777 888 666
999 0 666
![Page 21: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/21.jpg)
21
Loading the Selected datadata4.dat111 222333 444111 222333 444111 222
control4.ctlload datainfile 'data4.dat'insert into table sql_ldr4when n2 = '222'(n1 position(1:3),n2 position(5:7))
Sql_ldr4:
N1 N2
--------- ---------
111 222
111 222
111 222
$ sqlldr control=control4.ctl userid=apps/secretone
$ sqlldr control=control4.ctl userid=apps/secretone dicard =
data4.dis
data4.dis
333 444333 444
![Page 22: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/22.jpg)
22
Field Separatordata5.dat1111;2222;3333;4444;1111;2222;3333;4444;1111;2222;
control5.ctlload datainfile 'data5.dat'insert into table sql_ldr5 replacewhen n2='2222'fields terminated by ';'(n1 integer external,n2 integer external)
Sql_ldr5
N1 N2
-------- ---------
1111 2222
1111 2222
1111 2222
$ sqlldr control=control5.ctl userid=apps/secretone
![Page 23: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/23.jpg)
23
Loading Multiple Data Filesdata91.dat11 2233 4455 66
data92.dat77 88 99 0011 11
control9.ctl
load data
infile 'data91.dat'
infile 'data92.dat'
insert into
table sql_ldr9 replace
(
n1 position(1:2),
n2 position(4:5)
)
$ sqlldr control=control2.ctl userid=apps/secretone
Sql_ldr9
N1 N2
11 22
33 44
55 66
77 88
99 0
11 11
![Page 24: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/24.jpg)
24
Loading into Multiple Tablesdata10.dat11 2233 4455 66
control10.ctl
load data
infile 'data10.dat'
insert into
table sql_ldr101 replace
(n1 position(1:2),
n2 position(4:5))
into
table sql_ldr102 replace
(n1 position(1:2),
n2 position(4:5))
$ sqlldr control=control10.ctl userid=apps/secretone
Sql_ldr101
N1 N211 2233 4455 66
Sql_ldr102
N1 N211 2233 4455 66
![Page 25: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/25.jpg)
25
Loading from Multiple Data files to Multiple Tables
data111.dat11 2233 4455 66
data112.dat77 88 99 1199 99
control11.ctl
load data
infile 'data111.dat'
infile 'data112.dat'
insert into
table sql_ldr111 replace
(n1 position(1:2),
n2 position(4:5))
into
table sql_ldr112 replace
(n1 position(1:2),
n2 position(4:5))
$ sqlldr control=control11.ctl userid=apps/secretone
Sql_ldr111
N1 N211 2233 4455 6677 8899 1199 99
Sql_ldr101
N1 N211 2233 4455 6677 8899 1199 99
![Page 26: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/26.jpg)
26
Skipping recordsdata2.dat111 222 666333 444 666555 666 666777 888 666999 000 666
control2.ctlload datainfile 'data2.dat'insert into table sql_ldr2 replace (n1 position(1:3),n2 position(5:7),n3 position(9:11))
$ sqlldr control=control2.ctl userid=apps/secretone skip=2
Sql_ldr2
N1 N2 N3
555 666 666
777 888 666
999 000 666
![Page 27: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/27.jpg)
27
Using LOAD optiondata14.dataabbbccdddeefff
control14.ctlload datainfile 'data14.dat'into table sql_ldr14
truncate/replace(a1 char(2),a2 char(3))
$ sqlldr control=control14.ctl userid=apps/secretone LOAD = 2
Sql_ldr14: A1 A2
aa bbbcc ddd
![Page 28: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/28.jpg)
28
Fixed Record Formatdata12.dat12,3456,7891,0111,2
control12.dat load datainfile 'data12.dat' "fix 5"insert into table sql_ldr12 replace fields terminated by ','(n1 integer external,n2 integer external)
$ sqlldr control=control12.ctl userid=apps/secretone
Sql_ldr12: N1 N2
12 3456 78 91 1
![Page 29: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/29.jpg)
29
Variable Record Format
data1.dat
07aaa,bbb08ccc,dddd05ee,ff
control1.ctl
load data
infile ‘data1.dat’ “var 02”
into table sql_ldr1
fields terminated by ‘,’
(
a1 char,
a2 char
);
Desc sql_ldr1
a1 Varchar2(10)
a2 Varchar2(10)
$ sqlldr control=control1.ctl userid=apps/secretone
Sql_ldr11: A1 A2
aaa bbbccc ddddee ff
![Page 30: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/30.jpg)
30
Stream Record Format
data2.dat
aaa,bbb|ccc,dddd|eeee,ffff|
Control2.ctl
load data
infile ‘data2.dat’ “str ‘|’”
into table sql_ldr2
fields terminated by ‘,’
(
a1 char,
a2 char
)
Desc sql_ldr1
a1 Varchar2(10)
a2 Varchar2(10)
$ sqlldr control=control2.ctl userid=apps/secretone
Sql_ldr12: A1 A2
aaa bbbccc ddddeeee ffff
![Page 31: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/31.jpg)
31
Command Line Parameters BAD - Specifies the file in which all bad data is kept. The default
filename is the control filename with a .bad extension.
BINDSIZE - Size of the bind array in bytes. System dependent.
CONTROL - Specifies the name of control file.
DATA - Specifies the file that contains all data to be loaded.
DIRECT - If set to TRUE Direct Path Load is used. Otherwise Conventional Path will be used. Default value is FALSE.
DISCARD - Specifies the file where all discarded data is kept
![Page 32: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/32.jpg)
32
DISCARDMAX – The maximum no: of invalid records, which is not satisfying the ‘when’ clause in control file, that may be encountered before SQL *Loader session is stop. Default value is ALL.
ERRORS – Specifies how many total errors can be encountered before SQL *Loader session is to stop. Default value is 50.
LOAD – Specifies the maximum no: of records to load before stopping. Default Value is ALL.
LOG – Specifies the log file’s name, where information on success or failure of the Loading session is reported. The Default filename is control_filename.log.
Command Line Parameters
![Page 33: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/33.jpg)
33
PARALLEL – If set to TRUE, loads are performed in parallel where possible. Default value is FALSE
PARFILE – An additional file that contains more parameter specification.
ROWS – No: of rows to put in the path bind array, for Conventional path load. For Direct path load, ROWS specifies the no: of rows to read before a data save is performed. Default value is 64 in Conventional Path.
SKIP – No: of record to skip before starting the load. This parameter is important for restarting a load process after stopping an earlier session. Useful in Recovery from failure.
Command Line Parameters
![Page 34: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/34.jpg)
34
USERID – Specifies the username and password for the user conducting the SQL *Loader session
SILENT – Allows to suppress various header and feedback messages that SQL *Loader normally displays during a Load session.
Command Line Parameters
Keyword for use with SILENT parameter• DISCARDS – Suppresses the discarded messages• ERRORS – Suppresses the error message• FEEDBACK – Suppresses the “commit point reached” messages.• HEADER – Suppresses the messages that SQL *Loader displays on the screen
when we first launch the executable.• PARTITION – Suppresses the per-partition statistics, when loading a direct
path load of a partitioned table.
![Page 35: SQL Loader](https://reader031.fdocuments.net/reader031/viewer/2022013110/545495b1b1af9fc85d8b4f74/html5/thumbnails/35.jpg)
35