Migrate 10TB to Exadata Tips and Tricks (Presentation)
-
Upload
amin-adatia -
Category
Data & Analytics
-
view
338 -
download
2
description
Transcript of Migrate 10TB to Exadata Tips and Tricks (Presentation)
![Page 1: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/1.jpg)
AMIN ADATIAKNOWTECH SOLUTIONS INC.
WORKING WITHORACLE SINCE 1985EXADATA SINCE 2011
FOCUSCUSTOM APPLICATION
DEVELOPMENT
Migrate to Exadata X3Tips and Tricks
Level 3, San Polo 3403, Session 326Thu, Apr 10, 2014 (12:15 PM - 12:45 PM)
![Page 2: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/2.jpg)
Contact Details
eMail => [email protected] Twitter => aminadatiaLinkedIn
Data Architecture Professionals Oracle Exadata (IOUG SIG) Oracle Exadata Users
Oracle User Groups IOUG ODTUG
![Page 3: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/3.jpg)
Agenda
Background Source and Target Environments
Migration ConstraintsData Migration Techniques
Approach Non Partitioned Tables Partitioned Tables
Without LOB Columns With LOB Columns Sub-Partition by Hash Tables
Oracle Text Indexes without using TTSOracle Label Security without Custom Label Tags
![Page 4: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/4.jpg)
Environments
SOURCE Oracle 10.2.0.4
AIX 2 Node RAC 48 CPU per Node
TargetOracle 11.2.0.3
Exadata X3 8 Node RAC 32 CPU per Node RAM
PatchsetCellPatch
![Page 5: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/5.jpg)
Downtime Constraints
Users mostly query for the past 240 PartitionsSome queries span 1200 PartitionsPlan for
480 Partitions loaded before users allowed access and ETL loading to resume
Estimated time was 5 hours and so 6 hours was allocated for cold migration
Not much prospect to drop indexes and recreate ETL was done for 3 days prior to migration Having NO INDEXES resulted in zero ETL At the end of the downtime => ETL would start
![Page 6: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/6.jpg)
Source AIX Environment
Tablespaces => 450 (9.8 TB Disk Used)Table Partitions distributed across all TablespacesTables (for Migration)
Partitioned => 33 (~ 2500 Partitions/Table) Non Partitioned => 65 (2 Large Number
of Rows) Oracle Text Indexes => 4 Tables with LOBs => 4 (CLOB and BLOB)
Oracle Label Security => ~ 280 LabelsNetwork Link => 1 MBit and 10 MBit
![Page 7: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/7.jpg)
Target X3 Environment
Tablespaces => Same as SourceOracle Label Security
Create Policy Create Labels and Tags Map New Tags to Old Tags
Oracle Text Define Preferences
Schema Definitions Export from Source with Rows = N Import into Target
![Page 8: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/8.jpg)
Target X3 Environment
Going from Test (X2) to Production (X3)DataPump had a bug => Extremely Slow Revert to using 10g EXP
Virtual Columns Not RecognizedSchema Definitions (3)
Export from X2 with (Rows = N) Create Oracle Text Preferences Create OLS Policy Import into X3 Recreate Objects with Virtual Columns
![Page 9: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/9.jpg)
Non Partitioned Tables
![Page 10: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/10.jpg)
Non Partitioned Tables … 1
All commands Issued from Exadata
INSERT INTO <Table_Name>SELECT /*+ PARALLEL (T1,p_Degreee) */FROM <Source_Table>@DbLink_Pipe T1
PX Elapsed Time 12 57 minutes 24 35 minutes
![Page 11: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/11.jpg)
WITH NO LOB COLUMNSWITH LOB COLUMNSSUB-PARTITION BY HASH AND LOB COLUMN
Partitioned Tables
![Page 12: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/12.jpg)
Partitioned Tables Outcome
Hou
rs
Table
![Page 13: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/13.jpg)
WITH NO LOB COLUMNS
Partitioned Tables (No LOBs)
![Page 14: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/14.jpg)
Partitioned Table (No LOBs) ..1
v_Step := 'Create => '||v_Q_Table||' => '||p_Table_Name;
EXECUTE IMMEDIATE 'create table '||v_Q_Table||chr(10)||'TABLESPACE '||v_Tablespace_Name||chr(10)||'NOLOGGING'||chr(10)||'as select'||chr(10)||' /*+'||chr(10)||' parallel (a,'||p_Parallel_Source||')'||chr(10)||' no_index ('||p_No_Index_Hint||')'||chr(10)||' */'||chr(10)||' * from '||p_Table_Name||'@'||p_Data_Source||' a'||chr(10)||'where a.PartKey||chr(10)||’ BETWEEN p_Start_Number - '||p_Offset_Start||chr(10)||' AND p_Start_Number - '||p_Offset_End;
![Page 15: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/15.jpg)
Partitioned Table (No LOBs) ..2
v_Step := 'Gather Stats => '||v_Q_Table;
v_Step := ‘Exchange Working Table for Partition'; If RECORDS > 0
Insert Data from Q_Table into Working Table based on Partition_Key.
v_Step := 'Create Matching Partition Indexes';
Create Index on Working Table Exchange Working Table with Partition
![Page 16: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/16.jpg)
Partitioned Table (No LOBs) .. 3
EXECUTE IMMEDIATE 'ALTER TABLE '||p_Table_Name ||chr(10)||' EXCHANGE PARTITION‘ ||v_Partition_Name ||chr(10)||' WITH TABLE ' ||v_Working_Table ||chr(10)||' INCLUDING INDEXES' ||chr(10)||' WITHOUT VALIDATION' ||chr(10)||' UPDATE GLOBAL INDEXES';
![Page 17: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/17.jpg)
Partitioned Table (No LOBs) … 4
Parallel at Source => 2,4,8,12,24depending on the amount of datahow many jobs were still running
Parallel at Target => usually 8After 4 hours Resources on AIX were diverted to deal to Tables with LOB Columns
![Page 18: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/18.jpg)
TABLES WITH LOB COLUMNS
Partition Tables (LOBs)
![Page 19: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/19.jpg)
Partitioned Table (LOBs) ..1
Parallel at Source => 1,2,4,8,16,24,32Eventually found out that PX = 1 was
bestBy the time we figured out the solution
only one table was left to doSubmitted 45 jobs on each source node
using parallel = 1 at source Each job took about 15 minutes
![Page 20: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/20.jpg)
Partitioned Table (LOBs) ..2
![Page 21: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/21.jpg)
Partitioned Table (LOBs) ..3
v_Begin INTEGER := 210;BEGIN FOR I in 1..45 LOOP v_Start := v_Begin + (I - 1); v_End := v_Start; v_Job_Name := 'M_B1_'||v_Start||'_'||v_End;
v_Job_Action := 'BEGIN' ||chr(10) ||'MIGRATION.DIRECT_EXCHANGE(''‘ ||p_Partition_LOB_Table ||'''',12,''a'',1,8,' ||v_Start||','||v_End||','''||v_Source||''');' ||chr(10)||'END;';
![Page 22: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/22.jpg)
Partitioned Table (LOBs) ..4
BEGIN DBMS_SCHEDULER.CREATE_JOB ( job_name => v_Job_Name ,job_type => 'PLSQL_BLOCK' ,job_action => v_Job_Action ,start_date => SYSTIMESTAMP ,enabled => TRUE );
DBMS_SCHEDULER.SET_ATTRIBUTE ( name => v_Job_Name ,attribute => 'instance_stickiness' ,value => FALSE );
END;
END LOOP;
![Page 23: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/23.jpg)
Partition Exchange
![Page 24: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/24.jpg)
Partition Exchange .. 2
Worked fine except for continuous ETL … TABLE gets LOCKED for the EXCHANGE If the duration was “small” (less than 2
minutes) then there was not a noticeable impact on the ETL process
32 SubPartition Exchange approach did not work well
![Page 25: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/25.jpg)
Migration Summary
Object Type Estimate
Partitions
Completed/Total
Actual
Partitions within
6 Hour Downtime
Actual Time
Non Partitioned Tables (65) 30 minutes 12 minutes Partitioned Tables
Non LOB Columns (30)480/2500 26 Tables
completed1 Table 8802 Tables 4401 Table 200
8 Hours11 Hours20 Hours
Partitioned Tables
CLOB Column Tables (2 Tables)1 Table 480/25001 Table 40/1800
1000160
11 Hours15 Hours
Sub Partitioned (32) Table
BLOB Column28/1200 16 8 Days!!
![Page 26: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/26.jpg)
Sub Partition Exchange
![Page 27: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/27.jpg)
Sub Partition Exchange .. 1
Table with Range Partition on two columns 32 Sub Partitions by Hash on third column
Getting the Hash Value and finding the subpartition seemed to slow down the exchange
Simple insert from Exchange Table also had the same kind of issue. Worse than Sub Partition Exchange
Insert using Sorted by (ORA_HASH(HashKey,31,0)+1) significantly improved performance
![Page 28: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/28.jpg)
Sub Partition Exchange .. 2
INSERT INTO HASH_LOB_TABLESELECT /*+ NO_INDEX(A) */ * FROM Q_TABLE AORDER BY A.PartKey_Keep,(ORA_HASH(A.<hashkey>,31,0) + 1);
![Page 29: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/29.jpg)
Sub Partition Exchange … 3Time per Record
Not much good for Simultaneous ETL
![Page 30: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/30.jpg)
Sub Partition Exchange … 4Relative Insert Time/ Record
![Page 31: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/31.jpg)
WITHOUT TRANSPORTABLE TABLESPACES
Oracle Text Index
![Page 32: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/32.jpg)
Oracle Text Environment
Tables with Text Indexes Multi-lingual Documents stored on File System (2
Tables) Generated XML from Data (2 Tables)
Transportable Tablespaces Method for migrating was estimated to take 20 days
New data could not be loaded until TTS was completed
![Page 33: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/33.jpg)
Document Indexing
Using High Degree Parallel (32,64,96) One Partition Data could be indexed in about 30 – 60
minutes This was less than the time it took to move the files to
the new file systemIndexing routine checked the status of the
file moveIndexing Started when File move was
completedFor Highlighting Search Items, the Text
Query goes against the Text Document stored on the file system and so the files needed to be moved
![Page 34: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/34.jpg)
Sync Indexing
Using CTX_USER_PENDING to get the Partition to be SYNCed did not work well => 18 minutes to return Partition Names
Write a View => Less than 1 second CTXSYS.DR$PENDING (Parallel 2) CTXSYS.DR$INDEX CTXSYS.DR$INDEX_PARTITION (Parallel 2)
Loop through Partitions found “PENDING”Job runs every 10 secondsParallel Degree usually
Text 8 (used 16,24,32,64) XML 16 (used 4,8,16,24,32,64,96)
![Page 35: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/35.jpg)
Optimize Indexing
SYNC_INDEX is with Parallel (4,8,16,32,64,96)OPTIMIZE_INDEX – Parallel 1
SYNC_INDEX get Locked out CTX_DDL.LOCK_NOWAIT did not resolve the problem
Optimize done via DBMS_SCHEDULER JobWrote View to exclude Partitions from
SYNC_INDEX which were part of the Optimize Index Job
Optimize done on Previous Partitions and it takes about 3 Hours to gain 60% - 85% tokens reduction
Impact on Switch Log Frequency (+ 100/hour)
![Page 36: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/36.jpg)
XML Document Indexing
Using High Degree Parallel (16,32,64) One Partition Data could be indexed in about 30
minutes This was about the time it took to generate the XML XML generation could only be done once the
underlying tables had the data migratedBuild XML routine checked for all the data
tables for the day had been migratedIndexing routine checked the status of the
XML BuildIndexing Started when XML Build was
completed
![Page 37: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/37.jpg)
WITHOUT CUSTOM LABEL TAGS
Oracle Label Security
![Page 38: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/38.jpg)
Setup OLS Metadata
Create Label PolicyCreate Labels from SourceDefine Mapping between OLD => NEW TagsUse CASE/DECODE to convert during
Migration of Table WHEN <Label_Tag> = Old Value THEN <New-
Label_Tag> ELSE <not_matched_tag>
Issues => Updating Label_Tag after activating OLS on the Table
was painstakingly slow
![Page 39: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/39.jpg)
Questions??
![Page 40: Migrate 10TB to Exadata Tips and Tricks (Presentation)](https://reader036.fdocuments.net/reader036/viewer/2022070322/5591b4181a28ab2c2c8b45c2/html5/thumbnails/40.jpg)
ENJOY THE REST OF THE CONFERENCE
Thank you!!