Data Migration ToolsBy d.c.b.a

http://www.anysql.nethttp://www.mydul.net

Source & Target

Source Oracle MySQL

Target Oracle MySQL Flat Text File

Download

Oracle Source

Oracle to Oracle datacopy

Oracle to MySQL ora2mysql

Oracle to Flat Text File SQLULDR2

MySQL Source

MySQL to Oracle mysql2ora

MySQL to MySQL mysqlcopy

MySQL to Flat Text File MYSQLULDR2

Basic Concept Query on Source, DML on Target

Only Select Allowed on Source Database Insert/Update/Delete/Script allowed on Target Database, Refer source columns by “:”+column name Query1/Table1 for Source SELECT Statement Query2/Table2 for Target DML Statement

Basic Data Types Supported. Object Data Types not Supported.

Array Operation to Reduce Network Roundtrips. Direct Load Supported for Oracle. Parallel Supported for Both Oracle and MySQL. Parameter File Supported.

Database Connection

User1 Option for Source Database User2 Option for Target Database Oracle Connection

User/Password@Host:Port:Database User/Password@tnsname “sys” for SYSDBA logon (Oracle Database)

MySQL Connection User/Password@Host:Port:Database Default Port is 3306

Return Code

Windows %ERRORLEVEL%

Linux/Unix $?

Zero value for success without any errors.

Non-zero value for error, different error different return code.

DATACOPYFrom Oracle To Oracle

DATACOPY

Manipulate Data Between Oracle Databases.

Operations Direct Load Insert Insert Update Delete PL/SQL Block

The SYNC option

Direct Load Insert (DATACOPY)

Using TABLE2 option to specify the table name on target database.

Specify the “direct=yes” option. Example

datacopy user1=scott/tiger@prod1 user2=scott/tiger@prod2 query1=“select * from emp” table2=emp direct=yes

Insert (DATACOPY)

Using QUERY2 Option to Specify the Insert Statement on Target Database

Refer the source column value by “:” plus column name

Example datacopy user1=scott/tiger@prod1

user2=scott/tiger@prod2 query1=“select eno, ename from emp” query2=“insert into emp(eno, ename) values (:eno, :ename)”

Update (datacopy)

Using QUERY2 Option to Specify the Update Statement on Target Database

Refer the source column value by “:” plus column name

Example datacopy user1=scott/tiger@prod1

user2=scott/tiger@prod2 query1=“select eno, ename from emp” query2=“update emp set ename = :ename where eno = :eno”

Delete (DATACOPY)

Using QUERY2 Option to Specify the Delete Statement on Target Database

Refer the source column value by “:” plus column name

Example datacopy user1=scott/tiger@prod1

user2=scott/tiger@prod2 query1=“select eno from emp” query2=“delete from emp where eno = :eno”

PL/SQL Block (DATACOPY)

Using QUERY2 Option to Specify the PL/SQL Script on Target Database

Refer the source column value by “:” plus column name

Example datacopy user1=scott/tiger@prod1

user2=scott/tiger@prod2 query1=“select eno from emp” query2=“begin delete_emp(:eno); end;”

SYNC Option (datacopy)

Auto Generate Target SQL or PL/SQL Script without specify the “QUERY2” option.

Cooperation with “TABLE2” option. SYNC=INSERT

Cooperation with “TABLE2” and “UNIQUE” option. SYNC=UPDATE/DELETE SYNC=DELINS/INSUPD/UPDINS

SYNC Option (DELINS)

Command datacopy … table1=emp table2=emp

unique=eno sync=delins Real QUERY2 Value

Begin delete from emp where eno=:eno; insert into emp (…) values (…); end;

SYNC Option (UPDINS)

Command datacopy … table1=emp table2=emp

unique=eno sync=delins Real QUERY2 Value

Begin update emp set … where eno=:eno; if sql%rowcount == 0 then insert into emp (…) values (…); end if; end;

SYNC Option (INSUPD)

Command datacopy … table1=emp table2=emp

unique=eno sync=delins Real QUERY2 Value

Begin insert into emp (…) values (…); if unique error then update emp set … where eno=:eno; end if; end;

Tuning (datacopy)

Multiple Block Read (read=…) Sort Area Size (sort= …) Hash Area Size (hash=…) Direct Path Read (serial=yes) Parallel Query (/*+ PARALLEL … */

Hint)

Multiple Thread Copy

Automatically Split Into Pieces by ROWID Range.

Need “SELECT_CATALOG_ROLE” to access “DBA_EXTENTS” view.

Using “SPLIT” and “DEGREE” Option. “SPLIT” for the base table for ROWID

range analyze. “DEGREE” for the parallel degree. Example

datacopy … split=emp degree=4 …

Why “SPLIT” Option

Source Query Can be Complex. select … from emp, deptno where

emp.deptno = dept.deptno and emp.rowid >= :minrid and emp.rowid < :maxrid

select deptno, count(*) from emp where rowid >= :minrid and rowid < :maxrid group by deptno

Parallel Can Only Be One Dimemsion. Reference The ROWID Range by

“:MINRID” and “:MAXRID”.

Character Set

Source Database NLS_LANG environment variable

Target Database “CHARSET” option for basic character

set. “NCHARSET” option for national

character set. From US7ASCII to ZHS16GBK

export NLS_LANG=.US7ASCII datacopy … charset=ZHS16GBK …

Parameter File

Text File (test.par) user1=scott/tiger@prod1 user2=scott/tiger@prod1 query1=select * from emp table2=emp read=128 serial=yes

Command datacopy parfile=test.par

ORA2MYSQLFrom Oracle To MySQL

Reference

Multiple Thread Support Refer DATACOPY.

SYNC Option (ora2mysql)

Auto Generate Target SQL without specify the “QUERY2” option.

Cooperation with “TABLE2” option. SYNC=INSERT/ARRINS

Cooperation with “TABLE2” and “UNIQUE” option. SYNC=UPDATE/DELETE/INSUPD/ARRUPD

Target SQL (ora2mysql)

ARRINS Insert into emp (…) values (…), (…), (…)

INSUPD Insert into emp (…) values (…) on

duplicate key … ARRUPD

Insert into emp (…) values (…) (…) (…) on duplicate key …

Character Set

MySQL Character Set is Controlled By “CHARSET” option.

Oracle Character Set is Controlled By “NLS_LANG” Environment Variable.

SQLULDR2From Oracle To Flat File

Reference

Multiple Thread Support Refer DATACOPY.

Flat File Format

Fixed Width Format How to Split Different Records?

Using the “FIELD” Option. Using “0xXX” for Any Characters.

How to Split Different Fields? Using the “RECORD” Option. Using “0xXX” for Any Characters.

Example sqluldr2 … field=0x07 record=0x06 …

Output File Name

Dynamic File Name %y=Year %m=Month %d=Day %w=Week %b=Batch Count %p=Thread ID %t=Timestamp (now() function)

Multiple Output Files

Split Output File by “SIZE” Options. Unit Megabytes sqluldr2 … size=500 file=data_%b.txt

Split Output File by “BATCH” Options. Store each rows in a single file sqlldr2 … rows=5000000 batch=yes

file=data_%b.txt …

SQL*Loader Control File

SQL * Loader need a parameter file (describing the format of the flat file) to load flat text file into Oracle database.

“TABLE” for the table name of target database.

“MODE” for the SQL * Loader option, default is “INSERT”, other options are “APPEND”, “REPLACE” and “TRUNCATE”.

“CONTROL” for the SQL*Loader control file name, default is “<table name>_sqlldr.ctl”.

MYSQL2ORAFrom MySQL to Oracle

Reference

Target Operation Please Refer DATACOPY.

Attention

LONG/LOB values larger than 64KB Not Supported Now.

Multiple Thread Copy

Automatically Split Into Pieces by Given Column.

Using “SPLIT”, “SPLITKEY” and “DEGREE” Option.

“SPLIT” for the base table. “SPLITKEY” for the key prefix column of the

base table. “DEGREE” for the parallel degree. Example

mysql2ora … split=emp splitkey=empno degree=4 …

Why “SPLIT” Option

Source Query Can be Complex. select … from emp, deptno where

emp.deptno = dept.deptno and emp.empno >= :minrid and emp.empno < :maxrid

select deptno, count(*) from emp where empno >= :minrid and empno < :maxrid group by deptno

Parallel Can Only Be One Dimemsion. Reference The ROWID Range by

“:MINRID” and “:MAXRID”.

The “SPLITKEY” Logic

Minimum Value SELECT <splitcol> FROM <table> ORDER

BY <splitcol> LIMIT 1 Maximum Value

SELECT <splitcol> FROM <table> ORDER BY <splitcol> DESC LIMIT 1

Split The Range Into Degree Pieces. Only Number/Date/Char Columns

Supported. Data May Not Be Evenly Distributed.

SYNC Option (datacopy)

Auto Generate Target SQL or PL/SQL Script without specify the “QUERY2” option.

Cooperation with “TABLE2” option. SYNC=INSERT

Cooperation with “TABLE2” and “UNIQUE” option. SYNC=UPDATE/DELETE SYNC=DELINS/INSUPD/UPDINS

MYSQLCOPYFrom MySQL to MySQL

Reference

Multiple Thread Please Refer MYSQL2ORA.

Attention

Source & Target Database Connection Using the Same Character Set.

Will Support Different Character Set Soon.

SYNC Option (mysqlcopy)

Auto Generate Target SQL without specify the “QUERY2” option.

Cooperation with “TABLE2” option. SYNC=INSERT/ARRINS

Cooperation with “TABLE2” and “UNIQUE” option. SYNC=UPDATE/DELETE/INSUPD/ARRUPD

Target SQL (mysqlcopy)

ARRINS Insert into emp (…) values (…), (…), (…)

INSUPD Insert into emp (…) values (…) on

duplicate key … ARRUPD

Insert into emp (…) values (…) (…) (…) on duplicate key …

MYSQLULDR2From MySQL to Text File

Reference

Text File Format Please Refer SQLULDR2.

Multiple Thread Please Refer MYSQL2ORA.

Load Into Target Directly

Support In Memory “Load Local Infile” Interface by “LOAD” and “TABLE” option with File Format.

Example mysqluldr2 user=/@::test load=/@::test

table=emp2

DOWNLOADResrouce

Links http://www.mydul.net/software/datacopy.zip http://www.mydul.net/software/ora2mysql.zip http://www.mydul.net/software/ora2mysql_linux64.zip http://www.mydul.net/software/sqluldr.zip http://www.mydul.net/software/mysql2ora.zip http://www.mydul.net/software/mysql2ora_linux64.zip http://www.mydul.net/software/mysqlcopy.zip http://www.mydul.net/software/mysqlcopy_linux64.zip http://www.mydul.net/software/mysqluldr2.zip http://www.mydul.net/software/mysqluldr2_linux64.zip