Crotty Plsql Bulk Collect Forall
Transcript of Crotty Plsql Bulk Collect Forall
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 1/34
Click to edit Master subtitle style
April 14, 2004 Next InformationSystems
PL/SQL Bulk Collectionsin Oracle 9i and 10g
Kent Crotty
Burleson Consulting
October 13, 2006
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 2/34
April 14, 2004 Next InformationSystems
AGENDA
• Performance gains with Bulk Processing
• Array processing with BULK COLLECT andFORALL
• Oracle 10g FORALL improvements.• Error handling.
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 3/34
April 14, 2004 Next InformationSystems
Top Books on PL/SQL
Dr. Tim Hall Oracle ACEAnd Oracle’s ACE of theYear
John GarmanyA fantastic beginner book onPL/SQL
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 4/34
April 14, 2004 Next InformationSystems
Bulk Processing
• Supercharge your PL/SQL code with BULKCOLLECTand FORALL
• Working at a table-level instead of the row-level
• Simple and easy to use
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 5/34
April 14, 2004 Next InformationSystems
PL/SQL Code
• Consists of two types of statementsProcedural (declare, begin, if, while, for …)
SQL (select, insert, update, delete)
•
Oracle has two engines to process that informationPL/SQL EngineSQL Engine
• A Content Switch occurs each time the PL/SQL engine
needs to execute a SQL statement
• Switches are fast but large loops can cause performancedelays
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 6/34
April 14, 2004 Next InformationSystems
Context Switches
Session PL/SQL
Block
PL/SQL
Block
PL/SQLEngine
Oracle Server
SQL Engine
Procedural
StatementExecutor
SQL Statement
Executor
Data
SQL
PL/SQL
Block
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 7/34
April 14, 2004 Next InformationSystems
PL/SQL CodeConsider this procedure code
CREATE OR REPLACE PROCEDURE update_price (product_type_in IN product.product_type%TYPE,multiplier_in IN number(2,2) )
ISCURSOR products_cur IS
SELECT product_id, product_priceFROM products WHERE product_type = product_type_in;
BEGINFOR prod_rec IN products_cur LOOPUPDATE products
SET product_price = product_price * multiplier_inWHERE product_id = prod_rec.product_id;
END LOOP;END update_price;
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 8/34
April 14, 2004 Next InformationSystems
PL/SQL Code
• For each iteration of this loop, there is going to be aconventional bind and a context switch!
• Overhead for these statements can be large
• But there is a solution – Bulk Collections
FOR prod_rec IN products_cur LOOPUPDATE products
SET product_price = product_price * multiplier_inWHERE product_id = prod_rec.product_id;
END LOOP;
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 9/34
April 14, 2004 Next InformationSystems
Bulk Collection Categories
• SELECT or FETCH statementsBULK COLLECT INTO
• Out-Bind binding
RETURNING clause
• In-Bind binding
FORALL – INSERT, UPDATE, DELETE
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 10/34
April 14, 2004 Next InformationSystems
SELECT / FETCH statements
Data may be Bulk Collected/Fetched into:
Table.column%TYPE
Record of arrays
Table%ROWTYPE
Cursor%ROWTYPE
Array of records
Nested tables
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 11/34
April 14, 2004 Next InformationSystems
Products Table
SQL> create table products (2 product_id number,
3 product_name varchar2(15),
4 effective_date date );
Table created.
SQL> begin -- inserting 100000 records into the products table
1 for i in 1 .. 100000 loop
2 insert into products values (i, 'PROD'||to_char(i),sysdate-1);
3 end loop;
4 end;
5 /
PL/SQL procedure successfully completed.
SQL> commit;
Commit complete.
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 12/34
April 14, 2004 Next InformationSystems
BULK COLLECT clause
• Used in a SELECT statement
• Binds the result set of the query to acollection
• Much less communication between thePL/SQL and SQL engines
• All variables in the INTO clause must be a
collection
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 13/34
April 14, 2004 Next InformationSystems
BULK COLLECTSET SERVEROUTPUT ON
DECLARE
TYPE prod_tab IS TABLE OF products%ROWTYPE; products_tab prod_tab := prod_tab();
start_time number;
end_time number;
BEGINstart_time := DBMS_UTILITY.get_time;
FOR prod_rec in (SELECT * FROM products
WHERE effective_date BETWEEN sysdate - 2 AND TRUNC(sysdate))
LOOP
products_tab.extend;
products_tab(products_tab.last) := prod_rec;
END LOOP;
end_time := DBMS_UTILITY.get_time;
DBMS_OUTPUT.PUT_LINE(‘Conventional (‘||products_tab.count||’): ’||to_char(end_time-start_time));
Start_time := DBMS_UTILITY.get_time;
SELECT *
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 14/34
April 14, 2004 Next InformationSystems
BULK COLLECTSQL> /
Conventional (100000): 40
Bulk Collect (100000): 27
PL/SQL procedure successfully completed.
Bulk Collect is quite a bitfaster!
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 15/34
April 14, 2004 Next InformationSystems
BULK COLLECT – Explicit Cursor- FetchDECLARETYPE prod_tab IS TABLE OF products%ROWTYPE;
products_tab prod_tab := prod_tab();
start_time number;
end_time number;
CURSOR products_data IS SELECT * FROM products;
BEGINstart_time := DBMS_UTILITY.get_time;
OPEN products_data;
LOOP
products_tab.extend;
FETCH products_data INTO products_tab(products_tab.last);
IF products_data%NOTFOUND THEN
products_tab.delete(products_tab.last);
EXIT;
END IF;
END LOOP;
CLOSE products_data;
end_time := DBMS_UTILITY.get_time;
‘ ‘ ’ ’
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 16/34
April 14, 2004 Next InformationSystems
BULK COLLECTSQL> /
Conventional (100000): 117
Bulk Collect (100000): 14
PL/SQL procedure successfully completed.
Bulk Collect is significantly faster – over 8times!
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 17/34
April 14, 2004 Next InformationSystems
BULK COLLECT - LIMIT
•
Collections are arrays held in memory –massive
collections can eat up all the memory
•
By using the LIMIT clause, we can nowprocess theresult set in chunks
• Explicit cursors must be used with the LIMIT
clause
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 18/34
April 14, 2004 Next InformationSystems
BULK COLLECT – Explicit Cursor- LIMITDECLARE
TYPE prod_tab IS TABLE OF products%ROWTYPE;
products_tab prod_tab := prod_tab();start_time number;
end_time number;
CURSOR products_data IS SELECT * FROM products;
BEGIN
Start_time := DBMS_UTILITY.get_time;
OPEN products_data;
LOOPFETCH products_data BULK COLLECT INTO products_tab LIMIT 10000;
EXIT WHEN products_data%NOTFOUND;
DBMS_OUTPUT.PUT_LINE('Processed '||to_char(products_tab.count)||' rows');
END LOOP;
CLOSE products_data;
end_time := DBMS_UTILITY.get_time;
DBMS_OUTPUT.PUT_LINE('Bulk Collect: ‘||to_char(end_time-start_time));
end;Result Set is limited to only 10000 rows – more memoryefficient!
Will the processing be slower because of the LIMIT?
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 19/34
April 14, 2004 Next InformationSystems
BULK COLLECT – Explicit Cursor- LIMIT
SQL> /
Processed 10000 rows
Processed 10000 rows
Processed 10000 rows
Processed 10000 rows
Processed 10000 rows
Processed 10000 rows
Processed 10000 rows
Processed 10000 rows
Processed 10000 rows
Processed 10000 rowsBulk Collect: 15
PL/SQL procedure successfully completed.
Yes, but only slightly. Still over 8 times better thanconventional!
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 20/34
April 14, 2004 Next InformationSystems
BULK COLLECT - RETURNING• The RETURNING clause can be used to return
specific columns after a DML statement
• This is referred to as OUT-BINDING
DECLARETYPE prod_tab IS TABLE OF products.product_id%TYPE; products_tab prod_tab := prod_tab();BEGINDELETE FROM productsWHERE product_id > 20000
RETURNING product_id BULK COLLECT INTO products_tab;DBMS_OUTPUT.PUT_LINE('Deleted Product Ids: '||
products_tab.count||' rows');end;SQL> /Deleted Product Ids: 80000 rows
PL/SQL procedure successfully completed.
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 21/34
April 14, 2004 Next InformationSystems
BULK COLLECT Use Considerations
• Use the LIMIT clause to manage
memory requirements
• NO_DATA_FOUND will not be raised
if no records are returned – check
contents to make sure records areretrieved
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 22/34
April 14, 2004 Next InformationSystems
BULK COLLECT - FORALL
• We have seen BULK COLLECT with the SELECTstatements
• For the INSERT, UPDATE and DELETE
statements there is the FORALL statement
• This is referred to as IN-BINDING
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 23/34
April 14, 2004 Next InformationSystems
BULK COLLECT – INSERT - FORALLDECLARETYPE prod_tab IS TABLE OF products%ROWTYPE; products_tab prod_tab := prod_tab();start_time number; end_time number;
BEGIN-- Populate a collection - 100000 rowsSELECT * BULK COLLECT INTO products_tab FROM products;
EXECUTE IMMEDIATE 'TRUNCATE TABLE products';Start_time := DBMS_UTILITY.get_time;FOR i in products_tab.first .. products_tab.last LOOPINSERT INTO products (product_id, product_name, effective_date)VALUES (products_tab(i).product_id, products_tab(i).product_name,
products_tab(i).effective_date);END LOOP;end_time := DBMS_UTILITY.get_time;DBMS_OUTPUT.PUT_LINE(‘Conventional Insert: ’||to_char(end_time-start_time));
EXECUTE IMMEDIATE 'TRUNCATE TABLE products';Start_time := DBMS_UTILITY.get_time;FORALL i in products_tab.first .. products_tab.lastINSERT INTO products VALUES products_tab(i);end_time := DBMS_UTILITY.get_time;DBMS_OUTPUT.PUT_LINE(‘Bulk Insert: ’||to_char(end_time-start_time));COMMIT;END;
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 24/34
April 14, 2004 Next InformationSystems
BULK COLLECT – INSERT - FORALL
SQL> /
Conventional Insert: 686
Bulk Insert: 22
PL/SQL procedure successfully completed.
The Bulk Operation is considerablyfaster!
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 25/34
April 14, 2004 Next InformationSystems
BULK COLLECT – UPDATE - FORALLDECLARETYPE prod_id_tab is TABLE OF products.product_id%TYPE;TYPE prod_tab IS TABLE OF products%ROWTYPE; products_id_tab prod_id_tab := prod_id_tab();
products_tab prod_tab := prod_tab();start_time number;end_time number;BEGIN-- Populate a collection - 10000 rowsFOR i in 1 .. 10000 LOOPproducts_id_tab.extend;products_id_tab(products_id_tab.last) := i+10;
products_tab.extend;products_tab(products_tab.last).product_id := i;END LOOP;
Start_time := DBMS_UTILITY.get_time;FOR i in products_tab.first .. products_tab.last LOOPUPDATE products SET ROW = products_tab(i) -- ROW available in 9.2WHERE product_id = products_tab(i).product_id;
END LOOP;end_time := DBMS_UTILITY.get_time;DBMS_OUTPUT.PUT_LINE('Conventional Update: '||to_char(end_time-start_time));
Start_time := DBMS_UTILITY.get_time;FORALL i in products_tab.first .. products_tab.lastUPDATE products SET ROW = products_tab(i) – ROW available in 9.2 WHERE product_id = products_id_tab(i);end_time := DBMS_UTILITY.get_time;DBMS_OUTPUT.PUT_LINE('Bulk Update: '||to_char(end_time-start_time));END;
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 26/34
April 14, 2004 Next InformationSystems
BULK COLLECT – UPDATE - FORALL
SQL> /
Conventional Update: 301
Bulk Update: 116
PL/SQL procedure successfully completed.
The Bulk Operation is again considerablyfaster!
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 27/34
April 14, 2004 Next InformationSystems
FORALL Use
•
Only a single DML statement isallowed per FORALL
• In 9i, the binding array must be
sequentially filled• Use SAVE EXCEPTIONS to continue
past errors
•
SQL%BULK_ROWCOUNT returns thenumber of affected rows
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 28/34
April 14, 2004 Next InformationSystems
The Finer Points
•
Use bulk bind techniques for recurring SQLstatements in a PL/SQL loop.• Bulk bind rules:
• Can be used with any type of collection•
Collection subscripts cannot be expressions• Collections should be densely filled• If error, statement is rolled back. Prior
successful DML statements are not rolled back.
• Bulk Collects• Can be used with implicit or explicit cursors• Collection is always filled sequentially starting
with 1
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 29/34
April 14, 2004 Next InformationSystems
New in 10g – FORALL Improvements
•
FORALL driving array no longer needs tobe processed in sequential order
• The INDICES OF clause is used toreference the row numbers defined inanother array
• The VALUES OF clause is used toreference the values defined in another
array
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 30/34
April 14, 2004 Next InformationSystems
New in 10g – FORALL - INDICES
DECLARETYPE prod_id_tab is TABLE OF BOOLEAN INDEX BY PLS_INTEGER;TYPE prod_tab IS TABLE OF products%ROWTYPE;products_id_tab prod_id_tab := prod_id_tab();products_tab prod_tab := prod_tab();BEGINproducts_tab(10).effective_date := sysdate;products_tab(100).effective_date := sysdate + 10;products_tab(1000).effective_date := sysdate + 100;
products_id_tab(10) := TRUE;products_id_tab(100) := TRUE;products_id_tab(1000) := TRUE;
FORALL i IN INDICES OF products_id_tabUPDATE products SET ROW = products_tab(i)WHERE product_id = products_id_tab(i);
END;
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 31/34
April 14, 2004 Next InformationSystems
New in 10g – FORALL - VALUES
DECLARETYPE prod_id_tab is TABLE OF BOOLEAN INDEX BY PLS_INTEGER;TYPE prod_tab IS TABLE OF products%ROWTYPE;products_id_tab prod_id_tab := prod_id_tab();products_tab prod_tab := prod_tab();BEGINproducts_tab(10).effective_date := sysdate;products_tab(100).effective_date := sysdate + 10;products_tab(1000).effective_date := sysdate + 100;
products_id_tab(100) := 10;products_id_tab(200) := 100;products_id_tab(300) := 1000;
FORALL i IN VALUES OF products_id_tabUPDATE products SET ROW = products_tab(i)WHERE product_id = products_id_tab(i);
END;
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 32/34
April 14, 2004 Next InformationSystems
FORALL – Error Handling
•
No more FULL rollback in case of anEXCEPTION• SAVE EXCEPTIONS clause• SQL%BULK_EXCEPTIONS – Collection of
records• SQL%BULK_EXCEPTIONS(i).ERROR_INDEX –
stores iteration “i” when exception is raised• SQL%BULK_EXCEPTIONS(i).ERROR_CODE –
stores the Oracle error code• SQL%BULK_EXCEPTIONS.count – returns the
count of the exceptions
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 33/34
April 14, 2004 Next InformationSystems
Context Switches – Conventional Binds
Session PL/SQL
BlockPL/SQLBlock
PL/SQLEngine
Oracle Server
SQL Engine
Procedural
StatementExecutor
SQL StatementExecutor
Data
SQL
PL/SQL
Block
8/3/2019 Crotty Plsql Bulk Collect Forall
http://slidepdf.com/reader/full/crotty-plsql-bulk-collect-forall 34/34
April 14 2004 Next Information
Context Switches – Bulk Binds
Session PL/SQL
BlockPL/SQLBlock
PL/SQLEngine
Oracle Server
SQL Engine
Procedural
StatementExecutor
SQL StatementExecutor
Data
SQL
PL/SQL
Block