Near-Zero Downtime Unicode Migration for Oracle Databases Weiran Zhang, Director Oracle...

Near-Zero Downtime Unicode Migration for Oracle Databases

Weiran Zhang, DirectorOracle Globalization EngineeringYoshi Baba, Consulting Member of Technical StaffOracle GoldenGate DevelopmentSeptember 30, 2014

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |


Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program Agenda

DMU Overview

Near-Zero Downtime Migration with DMU and GoldenGate

Unicode Migration for Oracle Multitenant

Product Updates

Q & A

1

2

3

4

5


Unicode is Ubiquitous

• Offers the most comprehensive character repertoire support for multilingual global applications• UTF-8 is used by over 80% of all websites as of Aug. 2014 (source:

w3techs.com)• Adopted by most popular technology stacks and packaged apps• Enables universal character support for cloud applications• Ideal for infrastructure consolidation and multitenant architecture


Unicode Migration Challenges

• Limited down-time window• Data Integrity– Invalid data (pass-through)– Data expansion– Binary data stored in textual columns

• Resource and cost constraints• Dependent objects (index, constraint, trigger, partitioned range,

materialized view, etc.)


Unicode Migration Challenges (Cont.)

• Application code impact• Incremental data changes before migration• Failure recovery to restore database consistency• Maintaining Unicode data after migration• Lack of domain expertise


Why DMU

• A next-generation Unicode migration solution• Free standalone product with paid database support license• Distributed via OTN and bundled with Database 12c• Streamlined end-to-end migration workflow• Intuitive GUI interface with data visualizations• Automated migration tasks (Scan/Cleanse/Convert)• Advanced data analysis and cleansing tools• Ability to define scheduled cleansing actions


Why DMU (Cont.)

• Inline conversion approach to drastically reduce migration time and resource requirements• Maximize performance with scalable server-side architecture• Real-time progress monitoring• Customizable workflow and execution plan• Recovery and diagnostics for better fault-tolerance• Health check for Unicode databases


Near-Zero Downtime Unicode Migration

• Leverage the synergy between DMU 2.1 and Oracle GoldeGate (OGG) 12.1.2.1.0 to minimize migration downtime window• Near-Zero Downtime Migration Steps– Scan and cleanse the source database with DMU to resolve data convertibility issues– Use DMU to generate OGG configuration files according to the data cleansing actions– Copy the generated configuration files into the OGG installation directory– Execute DEFGEN to generate source database definitions for tables with scheduled

cleansing actions– Start OGG manager process


Near-Zero Downtime Unicode Migration (Cont.)

• Near-Zero Downtime Migration Steps– Start OGG Extract process on the source database using the generated OGG

configuration files from DMU– Use RMAN to take a backup of the source database– Create the target database as a clone of the source database using the RMAN backup– Use DMU to convert the target database to Unicode– Start OGG Replicat process on the target database using the generated OGG

configurable files from DMU and SCN captured during the RMAN cloning– Switch applications from the source database to the target Unicode database


Near-Zero Downtime Unicode Migration Setup

GoldenGate

WE8ISO8859P1WE8ISO8859P1

AL32UTF8

Source Database Target DatabaseRMAN Initial Load/Clone

Change SynchronizationScan/Cleanse Convert

GoldenGate Configuration Files


Oracle GoldenGate Character Set Support

• Supports over 400 character sets including all Oracle RDBMS character sets• Converts character sets when the source and the target database, table or

column character sets are different• Optionally validates character sets when the source and the target

database/table/column character sets are same– By Default, does not validate the source character code point to maximize replication

performance

• Converts the source to the target directly without converting to Unicode to maximize replication performance


Oracle GoldenGate Character Set Conversion

• Converts or validates character sets between the source and the target columns– Converts between CHAR, VARCHAR2, CLOB, NCHAR, NVARCHAR2 and NCLOB– Automatic truncation, with warning message, if the target column does not have

enough space

• Configurable character set conversion failure behavior including:– Abort replication– Skip a character– Escape a character in \xXX or \uXXXX format (X = hexadecimal number)– Substitute a character


Character Set Override• Allow the user to explicitly override the source character set by database,

table, and column level.– Ignore the source database character set when the source character set is overridden.– DMU can override the character set of the source database, table, and column data if

the existing data is not in the database character set. OGG can convert and apply to the target Unicode database using the DMU character set tagging.

Source database: WE8ISO8859P1Table1: JA16EUC Target database: AL32UTF8Table2, Col1: WE8MSWIN1252Table2, Col2: UTF8

Convert


Character Set Override Parameter ExampleSource database: WE8ISO8859P1Table1: JA16EUC Target database: AL32UTF8Table2, Col1: WE8MSWIN1252Table2, Col2: UTF8

MAP SOURCE.Table1, TARGET TARGET.Table1, CHARSET( JA16EUC );MAP SOURCE.Table2, TARGET TARGET.Table2,

COLCHARSET( Col1, WE8MSWIN1252 ),COLCHARSET( Col2, UTF8 );

Convert


Column Data Type Conversion

• Oracle GoldenGate supports column character data type conversion as follows:– Between different character types• CHAR, VARCHAR2, CLOB, NCHAR, NVARCHAR2, NCLOB, LONG, and UDT• Character set conversion may be invoked

– Supports DMU scheduled column modifications between the source and the target• Column lengthening (e.g., CHAR/VARCHAR2(N) -> CHAR/VARCHAR2(M))• Character length semantics migration (e.g., CHAR/VARCHAR2(N BYTE) -> CHAR/VARCHAR2(M CHAR))• Data type modifications (e.g., CHAR(N) -> VARCHAR2(M), CHAR/VARCHAR2/LONG -> CLOB)• DMU generates OGG configuration files for columns with different definitions between the source

and the target


Oracle GoldenGate Character Mapping Override

• Redefines character mapping between two different character sets– Source characters are not available on target character set• Maps to similar character of the target character set instead of mapping to question mark or

substitution characters

– Same characters are mapped to different Unicode by source and target character sets• Can occur when replicating between different databases• Redefines the source and the target character mapping


Multitenant Architecture Character Set Support

• Oracle Database 12c supports homogeneous character set configurations for pluggable databases (PDBs) in a multitenant container database (CDB)– All PDBs in a CDB must share the same character set as the root container– New PDBs with plug-in compatible (binary subset) character sets of the CDB will be

synced up to have the CDB character set upon database open– New PDBs with incompatible character sets can only be opened in restricted mode for

administrative tasks


Character Set Considerations for Consolidation

• Use AL32UTF8 as the CDB database character set for new deployments• Migrate existing non-CDBs to AL32UTF8 with DMU before moving to the

multitenant architecture• Consolidate into separate CDBs based on compatible character set groups if

not all non-CDBs can be migrated to Unicode (e.g.)– US7ASCII, WE8ISO8859P1, WE8MSWIN1252 => WE8MSWIN1252 CDB– JA16SJIS => JA16SJIS CDB– UTF8, AL32UTF8 => AL32UTF8 CDB–…


PDB Consolidation Strategy

AL32UTF8

AL32UTF8

WE8MSWIN1252

WE8ISO8859P1

US7ASCII

JA16SJIS

JA16SJIS

AL32UTF8

UTF8

WE8MSWIN1252

JA16SJIS

AL32UTF8

Migrate

WE8MSWIN1252

AL32UTF8

AL32UTF8


PDB Unicode Migration with DMU

• DMU 2.0 or later supports migrating existing PDBs to Unicode• To minimize the downtime window, the recommended migration steps are:– Use DMU to scan and cleanse the PDB to resolve any convertibility issues while it’s

still plugged into the source CDB– Identify or create the target CDB with AL32UTF8– Unplug the PDB from the source CDB and plug into the target Unicode CDB (the PDB

will be put into restricted mode)– Use DMU to convert the PDB to AL32UTF8 on the target CDB


PDB Unicode Migration with DMU

WE8MSWIN1252

AL32UTF8

Unplug Plug

WE8MSWIN1252

Scan/Cleanse

WE8MSWIN1252

WE8MSWIN1252WE8MSWIN1252

WE8MSWIN1252Convert

AL32UTF8

AL32UTF8

WE8MSWIN1252


DMU Product Updates

• DMU 2.0 released on OTN 3/18/14• Bundled with Database 12.1.0.2.0• Supports Database 12c new features including PDBs and long varchar• CSRepair script for repairing mismatched database character set

declarations• Supported Database Releases & Platforms– 10.2.0.4.4, 10.2.0.5.x, 11.1.0.7.5, 11.2.0.1.3, 11.2.0.2.x, 11.2.0.3 or later– Linux64, Solaris64, HPUX64, AIX64, Windows


What’s Coming in DMU 2.1

• Near-zero downtime migration support with Oracle GoldenGate• Migration profile export/import for replay on a cloned instance• Save scheduled cleansing actions to SQL scripts• Export problem data report• Repository upgrade support• Visit our demo booth in Moscone South SLD-139• Check OTN DMU page for latest news

– http://www.oracle.com/technetwork/database/database-technologies/globalization/dmu/overview/index.html

Near-Zero Downtime Unicode Migration for Oracle Databases Weiran Zhang, Director Oracle...

Documents

Transcript of Near-Zero Downtime Unicode Migration for Oracle Databases Weiran Zhang, Director Oracle...