Post on 31-Mar-2015
1 © 2013 Julian Dyke juliandyke.com
UnderstandingGoldenGate
Julian Dyke
Independent Consultant
Web Version
© 2013 Julian Dyke juliandyke.com
Agenda
Introduction Configuration Redo versus Trail
2
© 2013 Julian Dyke juliandyke.com3
Introduction
© 2013 Julian Dyke juliandyke.com
GoldenGateIntroduction Oracle GoldenGate is a heterogeneous replication solution
GoldenGate (the company and the product) was acquired by Oracle Corporation in 2009.
GoldenGate supports:
Zero Downtime Upgrade and Migration System Integration / Data Synchronization Query and Report offloading Real-time Data distribution Real-time Data Warehousing Live standby database Active-active high availability
Controversial replacement for Oracle Streams
4
© 2013 Julian Dyke juliandyke.com
GoldenGateSupported Topologies
5
Unidirectional Bi-directional Peer-to-Peer
CascadingConsolidationBroadcast
Scalability, Database TieringData WarehouseData Distribution
Instant Failover, Active-Active Load Balancing, High AvailabilityReporting Instance
© 2013 Julian Dyke juliandyke.com
GoldenGate Supported Databases Oracle GoldenGate for Non Oracle Databases
Supported non-oracle databases include:
IBM DB2 on Windows, UNIX and Linux Microsoft SQL Server 2000, 2005, 2008 Sybase on Windows, UNIX and Linux Teradata on Windows, UNIX and Linux MySQL on Windows, UNIX and Linux TimesTen on Windows and Linux (delivery only)
6
© 2013 Julian Dyke juliandyke.com
GoldenGate Licensing Both capture and apply databases must be fully licensed
7
Source – Oracle Technology Global Price List – 15 March 2013
Product Processor License
Enterprise Edition 47,500
Real Application Clusters 23,000
Active Data Guard 10,000
GoldenGate 17,500
Management Pack for GoldenGate 3,500
GoldenGate licenses (Oracle to Oracle) include: XStream Active Data Guard
Source – Oracle Fusion Middleware Licensing Information 11gR1
© 2013 Julian Dyke juliandyke.com
GoldenGate Conventions This presentation was developed in Oracle GoldenGate 11.2.1.0.1
Virtual Box 4.2.0 Oracle Enterprise Linux 5 Update 6 Oracle Database 11.2.0.3.0
8
Name Abbreviation Value
Home Directory $GG_HOME /home/oracle/goldengate
GoldenGate Schema GGSCHEMA GG01
Source Target
Hostname vm4 vm5
Database Name NORTH SOUTH
Environment:
Databases:
© 2013 Julian Dyke juliandyke.com
GoldenGate Streams versus GoldenGate
9
PropagateCapture Apply
Oracle Streams
Oracle GoldenGate
Source Server Target Server
Data PumpExtract Replicat
Source Server Target Server
© 2013 Julian Dyke juliandyke.com
GoldenGateBasic Architecture
10
Manager
ReplicatData PumpExtract
Manager
TargetDatabase
SourceDatabase
LocalTrail
RemoteTrail
Target ServerSource Server
© 2013 Julian Dyke juliandyke.com
GoldenGateConfiguration Options
11
ClassicCapture
IntegratedCapture
Redo Logs +Archive
Logs
ArchiveLogsOnly
UpstreamCapture
DownstreamCapture
© 2013 Julian Dyke juliandyke.com
GoldenGateClassic Capture
12
Extract Data Pump
LocalTrail
SourceDatabase
OnlineRedo/Archive
Logs
© 2013 Julian Dyke juliandyke.com
GoldenGate Integrated Capture
13
Extract Data PumpLCRLCRLCR
Log Miner
LocalTrail
SourceDatabase
OnlineRedo/Archive
Logs
© 2013 Julian Dyke juliandyke.com
GoldenGateDownstream Capture Real Time Downstream Mode
14
OnlineRedo Logs
Source Server Downstream Server
LCRLCR
Standby Redo Logs
LogicalChange Records
IntegratedExtract
TrailFile
Capture
Primary Database Standby Database
© 2013 Julian Dyke juliandyke.com
GoldenGateDownstream Capture Downstream Archive Log Mode
15
OnlineRedo Logs
Source Server Downstream Server
LCRLCR
Archive Redo Logs
LogicalChange Records
IntegratedExtract
TrailFile
Capture
Primary Database Standby Database
© 2013 Julian Dyke juliandyke.com
GoldenGate Bi-Directional Replication Also known as Active-Active Replication
16
Capture
Replicat
Replicat
CaptureData
Pump
DataPump
Server A Server B
OnlineRedo Logs
OnlineRedo Logs
LocalTrail
RemoteTrail
RemoteTrail
LocalTrail
© 2013 Julian Dyke juliandyke.com
GoldenGateSupported Data Types The following data types are supported for both classic and integrated capture
NUMBER BINARY FLOAT BINARY DOUBLE CHAR VARCHAR2 LONG NCHAR NVARCHAR2 RAW LONG RAW DATE TIMESTAMP
17
© 2013 Julian Dyke juliandyke.com
GoldenGateSupported Data Types There is limited support in classic capture for the following data types:
INTERVAL DAY INTERVAL YEAR TIMESTAMP WITH TIME ZONE TIMESTAMP WITH LOCAL TIME ZONE
The following data types are not supported Abstract data types with scalar, LOBs, VARRAYs, nested tables , REFS ANYDATA ANYDATASET ANYTYPE BFILE MLSLABEL ORDDICOM TIMEZONE_ABBR URITYPE UROWID
18
© 2013 Julian Dyke juliandyke.com
GoldenGateSupported Data Types The following data types are supported for both classic and integrated capture
NUMBER BINARY FLOAT BINARY DOUBLE CHAR VARCHAR2 LONG NCHAR NVARCHAR2 RAW LONG RAW DATE TIMESTAMP
19
© 2013 Julian Dyke juliandyke.com
GoldenGateRestrictions Neither capture method supports
Database replay EXTERNAL tables Materialized views with ROWID
Classic capture does not support IOT mapping tables Key compressed IOTs XMLType tables stored as XML Object Relational Distributed Transactions XA and PDML distributed transactions Capture from OLTP table compressed tables Capture from compressed tablespaces Exadata Hybrid Columnar Compression (EHCC)
20
© 2013 Julian Dyke juliandyke.com
GoldenGateOracle-Reserved Schemas The following schema names are reserved by Oracle and should not be
configured for GoldenGate replication:
21
$AURORA EXFSYS REPADMIN
$JIS MDSYS SYS
$ORB ODM SYSMAN
$UNAUTHENTICATED ODM_MTR SYSTEM
$UTILITY OLAPSYS TRACESVR
ANONYMOUS ORDPLUGINS WKPROXY
AURORA ORDSYS WKSYS
CTXSYS OSE$HTTP$ADMIN WMSYS
DBSNMP OUTLN XDB
DMSYS PERFSTAT
DSSYS PUBLIC
© 2013 Julian Dyke juliandyke.com
GoldenGateRAC Support RAC support has some limitations in classic capture mode
Extract can only run against one instance If instance fails,
Manager must be stopped on failed node: Manager and extract must be started on a surviving node
Failover can be configured in Oracle Grid Infrastructure Additional archive log switching may be required in archive log mode Before shutting down extract process
Insert dummy record into a source table Switch log files on all nodes
Additional configuration required to access ASM instance
Shared storage for trails can be: OCFS ACFS DBFS No mention of NFS in the documentation
22
© 2013 Julian Dyke juliandyke.com23
Configuration
© 2013 Julian Dyke juliandyke.com
GoldenGateInstallation Download the following from www.oracle.com
24
ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip
Zip file is approximately 90MB
$ cd /home/oracle/goldengate
$ unzip ogg112101_fbo_ggs_Linux_x64_ora11g_64bit.zip
On each server create GG_HOME
mkdir /home/oracle/goldengate
Copy zip file to GG_HOME
Install bundle patch in Oracle 11.2.0.3 home for integrated capture
Add GG_HOME to profile
export PATH=/home/oracle/goldengate:$PATH
export LD_LIBRARY_PATH=/home/oracle/goldengate:$LD_LIBRARY_PATH
© 2013 Julian Dyke juliandyke.com
GoldenGate GGSCI Most GoldenGate configuration can be performed within GGSCI
Command-line utility
For a list of available commands:
25
$ ggsci
GGSCI 1> HELP
For help on with an individual command:
GGSCI 2> HELP <command> <object>
GGSCI 1> INFO ALL
For example:
The best source of information is: Oracle GoldenGate Windows and UNIX Reference Guide
© 2013 Julian Dyke juliandyke.com
GoldenGate Parameter Files Each process has a text-based parameter file
Parameter files can be edited using GGSCI or with a text editor
26
MAP US03.T54 TARGET US01.T55, &COLMAP (col1=c1, col3=c2, col4=c3, col2=c4);
Continuation character is ampersand
-- This is a comment
Comments are prefixed by -- and terminated by end of line
$GG_HOME/dirprm/rep1.prm
Stored in $GG_HOME/dirprm directory e.g.
[oracle@vm5]$ ggsci
GGSCI (vm5) 1> EDIT PARAMS rep1
© 2013 Julian Dyke juliandyke.com
GoldenGate Subdirectories By convention, configuration and trail files are stored in subdirectories within
$GG_HOME
Subdirectories are created within GGSCI:
27
[oracle@vm4]$ ggsci
GGSCI (vm4) 1> CREATE SUBDIRS
Creating subdirectories under current directory /home/oracle/goldengate
Parameter files /home/oracle/goldengate/dirprm: createdReport files /home/oracle/goldengate/dirrpt: createdCheckpoint files /home/oracle/goldengate/dirchk: createdProcess status files /home/oracle/goldengate/dirpcs: createdSQL script files /home/oracle/goldengate/dirsql: createdDatabase definitions files /home/oracle/goldengate/dirdef: createdExtract data files /home/oracle/goldengate/dirdat: createdTemporary files /home/oracle/goldengate/dirtmp: createdStdout files /home/oracle/goldengate/dirout: created
© 2013 Julian Dyke juliandyke.com
GoldenGate Schema Owner A schema owner is required to own GoldenGate objects
Can grant specific privileges to owner or just use DBA
For example GG01
28
[oracle@vm4]$ sqlplus / as sysdba
SQL> CREATE USER gg01 IDENTIFIED BY gg01;
SQL> GRANT CONNECT, RESOURCE, DBA TO gg01;
Schema owner must be specified in GoldenGate parameters file:
[oracle@vm4]$ ggsciGGSCI 1> EDIT PARAMS ./GLOBALS
GGSCHEMA gg01
In this example parameter file is /home/oracle/goldengate/GLOBALS
© 2013 Julian Dyke juliandyke.com
GoldenGate Tablespace Recommended for both source and target servers
29
[oracle@vm4]$ sqlplus / as sysdba
SQL> CREATE TABLESPACE goldengateDATAFILE '/u01/app/oradata/NORTH/goldengate01.dbf‘SIZE 100MAUTOEXTEND ON;
Tablespace created.
SQL> ALTER USER gg01 DEFAULT TABLESPACE goldengate;
User altered.
Ensure AUTOEXTEND is enabled
© 2013 Julian Dyke juliandyke.com
GoldenGate Role On source server create GGS_GGSUSER_ROLE
30
[oracle@vm4]$ cd /home/oracle/goldengate
[oracle@vm4]$ sqlplus / as sysdba
SQL> @role_setup
GGS Role setup script
This script will drop and recreate the role GGS_GGSUSER_ROLE
Enter GoldenGate schema name:GG01
Grant role to GGSCHEMA user:
[oracle@vm4]$ cd /home/oracle/goldengate
[oracle@vm4]$ sqlplus / as sysdba
SQL> GRANT GGS_GGSUSER_ROLE TO gg01;
© 2013 Julian Dyke juliandyke.com
GoldenGate Manager Process Manager process controls all GoldenGate processes on a server
Including extract and replicat processes
Configured in parameter file
31
[oracle@vm4]$ ggsciGGSCI 1> EDIT PARAMS MGR
PORT 7809DYNAMICPORTLIST 7810-7820
Executable is $GG_HOME/mgr
Manager process is started from GGSCI
oracle@vm4]$ ggsciGGSCI 2> START MANAGER
© 2013 Julian Dyke juliandyke.com
GoldenGate Extract Process Captures and stores changes in local trail file
Classic capture Captures changes from online redo log or archived redo log Fetches additional data from database
Integrated capture Captures changes from log miner
Uses extract executable
Sample parameter file:
32
EXTRACT ex1USERID gg01, PASSWORD gg01EXTTRAIL /home/oracle/goldengate/dirdat/exTABLE US03.*;
© 2013 Julian Dyke juliandyke.com
GoldenGate Data Pump Process Propagates contents of local trail file to remote trail file
Uses extract executable In basic configurations same functionality can be achieved using extract
process
Sample parameter file:
33
EXTRACT dp1USERID gg01, PASSWORD gg01RMTHOST vm5, MGRPORT 7809RMTTRAIL /home/oracle/goldengate/dirdat/rtTABLE US03.*;
© 2013 Julian Dyke juliandyke.com
GoldenGate Replicat Process Reads changes from remote trail and applies them to database
Uses replicat executable
Sample parameter file:
34
REPLICAT rep1USERID gg01, PASSWORD gg01ASSUMETARGETDEFSDISCARDFILE /home/oracle/goldengate/discards, PURGEMAP US03.* TARGET US03.*;
© 2013 Julian Dyke juliandyke.com
GoldenGate Processes and Trails New processes and trails must be added using GGSCI.
For example on the source server:
35
[oracle@vm4]$ ggsci
GGSCI> ADD EXTRACT ex1, TRANLOG, BEGIN NOW
GGSCI> ADD EXTTRAIL /home/oracle/goldengate/dirdat/ex, EXTRACT ex1
GGSCI> ADD EXTRACT dp1 EXTTRAILSOURCE /home/oracle/goldengate/dirdat/ex
GGSCI> ADD RMTTRAIL /home/oracle/goldengate/dirdat/rt, EXTRACT dp1
On the target server:
[oracle@vm5]$ ggsci
GGSCI> ADD REPLICAT rep1, EXTTRAIL /home/oracle/goldengate/dirdat/rt
© 2013 Julian Dyke juliandyke.com
GoldenGate Checkpoint Table Must exist in target database
Records location in trail of last change applied to database
Added using GGSCI
36
[oracle@vm4]$ ggsci
GGSCI> DBLOGIN USERID gg01, PASSWORD gg01
GGSCI> ADD CHECKPOINTTABLE gg01.checkpointtable
Location must be added to GLOBALS parameters on target server
[oracle@vm4]$ ggsci
GGSCI> EDIT PARAMS ./GLOBALS
GGSCHEMA gg01CHECKPOINTTABLE gg01.checkpointtable
© 2013 Julian Dyke juliandyke.com
GoldenGate Starting Processes Processes are started from GGSCI
On the source server
37
[oracle@vm4]$ ggsci
GGSCI> START EXTRACT ex1
GGSCI> START EXTRACT dp1
[oracle@vm4]$ ggsci
GGSCI> START REPLICAT rep1
On the target server
The EXTRACT and REPLICAT keywords can be omitted
Use equivalent STOP command to stop processes
© 2013 Julian Dyke juliandyke.com
GoldenGate Checking Process Status Check process status using GGSCI INFO ALL:
38
[oracle@vm4]$ ggsci
GGSCI (vm4.juliandyke.com) 1> INFO ALL
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNINGEXTRACT RUNNING DP1 00:00:00 00:00:04EXTRACT ABENDED DP2 00:00:00 24:23:14EXTRACT RUNNING EX1 00:00:00 00:00:05EXTRACT STOPPED EX2 00:00:00 02:34:41
[oracle@vm5]$ ggsci
GGSCI (vm5.juliandyke.com) 1> INFO ALL
Program Status Group Lag at Chkpt Time Since Chkpt
MANAGER RUNNINGREPLICAT RUNNING REP1 00:00:00 00:00:05REPLICAT RUNNING REP2 00:00:00 00:00:04
© 2013 Julian Dyke juliandyke.com
GoldenGate Viewing Log Files Processes are logged in report files stored in $GG_HOME/dirrpt
e.g. ex1 process will be logged in $GG_HOME/dirrpt/EX1.rpt
Up to 10 copies are retained EX1.rpt (latest), EX11.rpt, EX12.rpt, … EX19.rpt (oldest)
If a process abends then check the report file for details
Report files can also be viewed using GGSCI
39
[oracle@vm4]$ ggsci
GGSCI> VIEW REPORT ex1
Output is filtered through more utility
© 2013 Julian Dyke juliandyke.com
GoldenGate Trail Files Stored in $GG_HOME/dirdat directory by convention
User must specify a two-character prefix e.g. ex
File names are generated automatically by extract process e.g. ex000000 ex000001 ex000002 etc
Naming conventions require some thought
40
© 2013 Julian Dyke juliandyke.com
GoldenGate Trail Files Changes are only stored in trail file when a transaction commits
If a transaction rolls back then no changes are stored Archive logs must be available for long running transactions
Index changes are generally not stored IOT changes are stored
Undo changes are not stored Only redo
Changes to data dictionary are not stored including Objects, tables, columns, statistics Segments / extents
41
© 2013 Julian Dyke juliandyke.com
GoldenGate Trail Files – Data Representation NUMBER
4-byte length followed by ASCII characters e.g. 2013
42
00 00 00 04 32 30 31 33
DATE 2-bytes followed by 19 ASCII characters in the format:
00 00 32 30 31 33 2D 30 33 2D 31 34 3A 32 32 3A 33 39 3A 35 36
FF FF 00 00
FF FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
YYYY-MM-DD:HH24:MI:SS
NULL values are stored as
For example 2013-03-14:22:39:56 is
NULL values are stored as
© 2013 Julian Dyke juliandyke.com
GoldenGate Trail Files – Data Representation VARCHAR2
4-byte length followed by ASCII characters e.g. “GoldenGate” in VARCHAR2(20)
43
NULL values stored as:
00 00 00 0A 47 6F 6C 64 65 6E 47 61 74 65
CHAR 2-byte length followed by ASCII characters space-padded e.g. “GoldenGate” in CHAR(20)
00 00 47 6F 6C 64 65 6E 47 61 74 65 20 20 20 20 20 20 20 20 20 20
FF FF 00 00
FF FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
NULL values stored as:
© 2013 Julian Dyke juliandyke.com
GoldenGate Logdump Utility Dumps contents of GoldenGate trails from
Local trail Remote trail
44
[oracle@vm4]$ logdump
Logdump> GHDR ONLogdump> FILEHEADER DETAILLogdump> DETAIL DATALogdump> USERTOKEN DETAILLogdump> RECLEN 128
Logdump> OPEN <trailFileName>
To show next record use NEXT or N
Logdump> NEXT # or N
To view data set the following parameters:
To return to start of file
Logdump> POS 0
© 2013 Julian Dyke juliandyke.com
GoldenGate Logdump Sample output
45
Hdr-Ind : E (x45) Partition : . (x04)UndoFlag : . (x00) BeforeAfter: A (x41)RecLength : 51 (x0033) IO Time : 2013/04/05 06:24:00.000.000IOType : 5 (x05) OrigNode : 255 (xff)TransInd : . (x01) FormatType : R (x52)SyskeyLen : 0 (x00) Incomplete : . (x00)AuditRBA : 80 AuditPos : 32769040Continued : N (x00) RecCount : 1 (x01)
2013/04/05 06:24:00.000.000 Insert Len 51 RBA 9060Name: US03.T1After Image: Partition 4 G m 0000 0006 0000 0002 3630 0001 0007 0000 0003 5359 | ........60........SY 5300 0200 0d00 0000 094f 424a 4552 524f 5224 0003 | S........OBJERROR$.. 0009 0000 0005 5441 424c 45 | ......TABLEColumn 0 (x0000), Len 6 (x0006) 0000 0002 3630 | ....60Column 1 (x0001), Len 7 (x0007) 0000 0003 5359 53 | ....SYSColumn 2 (x0002), Len 13 (x000d) 0000 0009 4f42 4a45 5252 4f52 24 | ....OBJERROR$Column 3 (x0003), Len 9 (x0009) 0000 0005 5441 424c 45 | ....TABLE
© 2013 Julian Dyke juliandyke.com
GoldenGate Minimal Supplemental Logging Extract processes will not start unless minimal supplemental logging has
been configured in database
Minimal supplemental logging is used for: chained blocks blocks in index clusters
Minimal supplemental logging is enabled using:
46
SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
© 2013 Julian Dyke juliandyke.com
GoldenGateSupplemental Logging Supplemental logging should be configured for all tables being replicated
Enabled using GGSCI. For example:
47
[oracle@vm4]$ ggsci
GGSCI 1> DBLOGIN USERID us01 PASSWORD us01
GGSCI 2> ADD TRANDATA t1
Logging of supplemental redo data enabled for table US01.T1.
Internally this executes the following DDL:
ALTER TABLE "US01"."T1" ADD SUPPLEMENTAL LOG GROUP "GGS_76111" ("C1") ALWAYS /* GOLDENGATE_DDL_REPLICATION */
where 76111 is the OBJ# of the table
© 2013 Julian Dyke juliandyke.com
GoldenGateSequences Sequences are not supported by default. Additional configuration is required:
Sequences will not be identical across the databases Sequence in target database will never be lower than the same sequence
on the source database
48
[oracle@vm4]$ cd /home/oracle/goldengate[oracle@vm4]$ sqlplus / as sysdbaSQL> @sequence.sql
The sequence.sql script creates the following procedures in the GGSCHEMA : SEQTRACE GETSEQFLUSH REPLICATESEQUENCE UPDATESEQUENCE
To install sequence support on each server run:
© 2013 Julian Dyke juliandyke.com
GoldenGateSequences Extract Process
49
Replicat process:
EXTRACT ex1USERID gg01, PASSWORD gg01EXTTRAIL /home/oracle/goldengate/dirdat/exSEQUENCE us03.*;TABLE us03.*;
Data Pump process:
EXTRACT dp1USERID gg01, PASSWORD gg01RMTHOST vm5, MGRPORT 7809RMTTRAIL /home/oracle/goldengate/dirdat/rtSEQUENCE us03.*;TABLE us03.*;
REPLICAT rep1USERID gg01, PASSWORD gg01ASSUMETARGETDEFSDISCARDFILE /home/oracle/goldengate/discards, PURGEMAP US03.SEQ1, TARGET US03.SEQ1;MAP US03.T*, TARGET US03.*;
© 2013 Julian Dyke juliandyke.com
GoldenGateTRUNCATE Statements TRUNCATE statements are not supported by default
TRUNCATE support can be configured Standalone As part of full DDL support
These options are mutually exclusive
Standalone TRUNCATE support Does not support partitioned tables
Configure full DDL support for partitions/subpartitions Enabled using GETTRUNCATES parameter
Must be specified BEFORE tables/mappings in parameter file
50
© 2013 Julian Dyke juliandyke.com
GoldenGateTRUNCATE Statements
Extract process
51
REPLICAT rep1USERID gg01, PASSWORD gg01ASSUMETARGETDEFSDISCARDFILE /home/oracle/goldengate/discards, PURGEGETTRUNCATESMAP US03.*, TARGET US03.*;
Data pump process
Replicat process
EXTRACT dp1USERID gg01, PASSWORD gg01RMTHOST vm5, MGRPORT 7809RMTTRAIL /home/oracle/goldengate/dirdat/rtGETTRUNCATESTABLE us03.t*;
EXTRACT ex1USERID gg01, PASSWORD gg01EXTTRAIL /home/oracle/goldengate/dirdat/exGETTRUNCATESTABLE us03.t*;
© 2013 Julian Dyke juliandyke.com
GoldenGateDDL Support DDL is not supported by default To install DDL support on the source server:
52
[oracle@vm4]$ cd /home/oracle/goldengate
[oracle@vm4]$ sqlplus / as sysdba
SQL> @marker_setup
SQL> @ddl_setup
SQL> @role_setup
SQL> @ddl_enable
No configuration necessary on target server for DDL support
DDL support is implemented using System DDL trigger GGS_DDL_TRIGGER_BEFORE DDL Replication Package GGS_MARKER table GGS_DDL_HIST table
© 2013 Julian Dyke juliandyke.com
GoldenGateDDL Support Extract process:
53
EXTRACT ex1USERID gg01, PASSWORD gg01EXTTRAIL /home/oracle/goldengate/dirdat/exDDL INCLUDE ALLTABLE US03.t*;
Replicat process
REPLICAT rep1USERID gg01, PASSWORD gg01ASSUMETARGETDEFSDISCARDFILE /home/oracle/goldengate/discards, PURGEDDL INCLUDE ALLDDLERROR DEFAULT IGNOREMAP US03.*, TARGET US03.*;
Data Pump process:
EXTRACT dp1USERID gg01, PASSWORD gg01RMTHOST vm5, MGRPORT 7809RMTTRAIL /home/oracle/goldengate/dirdat/rtDDL INCLUDE ALLTABLE US03.t*;
© 2013 Julian Dyke juliandyke.com
GoldenGateMappings Mappings between source and target can be specified
At schema level At table level At column level
All mappings are performed by the replicat process Mappings are specified in replicat parameter file
54
REPLICAT rep1USERID gg01, PASSWORD gg01ASSUMETARGETDEFSDISCARDFILE /home/oracle/goldengate/discards, PURGEMAP US03.* TARGET US01.*;
Table level mapping
Schema level mapping
REPLICAT rep1USERID gg01, PASSWORD gg01ASSUMETARGETDEFSDISCARDFILE /home/oracle/goldengate/discards, PURGEMAP US03.T52 TARGET US01.T53;
© 2013 Julian Dyke juliandyke.com
GoldenGateMappings Column Level Mappings require a definition generated on source and copied
to target
On the source
55
[oracle@vm4]$ cd /home/oracle/goldengate
[oracle@vm4]$ ggsci
GGSCI 1> EDIT PARAMS defgen1
DEFSFILE ./dirdef/defgen1.defUSERID us03 PASSWORD us03TABLE us03.t54;
Generate the definitions using
[oracle@vm4]$ cd /home/oracle/goldengate
[oracle@vm4]$ ./defgen paramfile /home/oracle/goldengate/dirprm/defgen1.prm
Copy $GG_HOME/dirdef/defgen1.def to same location on target
© 2013 Julian Dyke juliandyke.com
GoldenGateMappings Configure replicat parameters
56
REPLICAT rep1USERID gg01, PASSWORD gg01SOURCEDEFS ./dirdef/defgen1.defDISCARDFILE /home/oracle/goldengate/discards, PURGEMAP US03.T54 TARGET US01.T55, &COLMAP (col1=c1, col3=c2, col4=c3, col2=c4);
Note – COLMAP syntax is <TargetColumnName>=<SourceColumnName>
Mapping is written to the report file e.g.:
MAP resolved (entry US03.T54):
MAP "US03"."T54" TARGET US01.T55, COLMAP (col1=c1, col3=c2, col4=c3, col2=c4);
Using the following key columns for target table US01.T55: COL1.
© 2013 Julian Dyke juliandyke.com
GoldenGateIntegrated Capture Requires
Oracle 11.2.0.3 or above Bundle patch 15987144
Uses additional memory in streams pool May need to increase
MEMORY_TARGET parameter STREAMS_POOL parameter Size of /dev/shm (specified in /etc/fstab)
To avoid warnings
57
SQL> ALTER SYSTEM SET compatible = '11.2.0.3.0' SCOPE=SPFILE;
SQL> STARTUP FORCE;
© 2013 Julian Dyke juliandyke.com
GoldenGateIntegrated Capture Grant administrative privileges to GGSCHEMA
58
BEGINdbms_goldengate_auth.grant_admin_privilege(
grantee => 'GG01',privilege_type => 'CAPTURE',grant_select_privileges => TRUE
);END;/
On source server register extract group with database
[oracle@vm4]$ ggsci
GGSCI (vm4) 1> DBLOGIN USERID gg01 PASSWORD gg01Successfully logged into database.
GGSCI (vm4) 2> REGISTER EXTRACT ex2 DATABASE
2013-04-19 14:04:01 INFO OGG-02003 Extract EX2 successfully registered with database at SCN 2109330.
© 2013 Julian Dyke juliandyke.com
GoldenGateIntegrated Capture Extract process parameters
59
EXTRACT ex2USERID gg01, PASSWORD gg01TRANLOGOPTIONS INTEGRATED PARAMS (MAX_SGA_SIZE 100)EXTTRAIL /home/oracle/goldengate/dirdat/ilTABLE us01.t101;
Data pump process parameters
EXTRACT dp2USERID gg01, PASSWORD gg01RMTHOST vm5, MGRPORT 7809RMTTRAIL /home/oracle/goldengate/dirdat/irTABLE US01.*;
Replicat process parameters
REPLICAT rep2USERID gg01, PASSWORD gg01ASSUMETARGETDEFSDISCARDFILE /home/oracle/goldengate/discards, PURGEMAP US01.* TARGET US01.*;
© 2013 Julian Dyke juliandyke.com
GoldenGateIntegrated Capture New processes and trails must be added using GGSCI.
For example on the source server:
60
[oracle@vm4]$ ggsci
GGSCI> ADD EXTRACT ex2 INTEGRATED TRANLOG, BEGIN NOW
GGSCI> ADD EXTTRAIL /home/oracle/goldengate/dirdat/il, EXTRACT ex2
GGSCI> ADD EXTRACT dp2 EXTTRAILSOURCE /home/oracle/goldengate/dirdat/il
GGSCI> ADD RMTTRAIL /home/oracle/goldengate/dirdat/ir, EXTRACT dp2
On the target server:
[oracle@vm5]$ ggsci
GGSCI> ADD REPLICAT rep2, EXTTRAIL /home/oracle/goldengate/dirdat/ir
Start processes in GGSCI
© 2013 Julian Dyke juliandyke.com61
Redo Versus
Trail
© 2013 Julian Dyke juliandyke.com
Redo versus TrailSingle-Row Inserts
62
5.2
INSERT INTO t2VALUES (1,’Alpha’);
COMMIT;
5.1
11.2
5.1
10.2Index Insert
Commit
5.424.4
TX Header Redo Trail
Insert Row 1
Insert Row 2
5.20
DRP Undo
INSERT INTO t2VALUES (2,’Beta’);
5.1
11.2
5.1
10.2
5.20
IRP Redo
DRP Undo
IRP Redo
© 2013 Julian Dyke juliandyke.com
Redo versus TrailMulti-Row Inserts
63
5.2
INSERT INTO t2SELECT * FROM t1;
COMMIT;
QMI Redo
5.1
11.3
5.1
10.2Index Insert
Commit
5.424.4
TX Header
Redo Trail
Insert Row 1
Insert Row 2
Insert Row 3
Insert Row 4
Insert Row 5
Insert Row 6
5.20
QMD Undo
© 2013 Julian Dyke juliandyke.com
Redo versus TrailDirect Inserts
64
5.2INSERT /*+ APPEND */ INTO t2SELECT * FROM t1;
COMMIT;
Direct Load
5.1
5.20
24.6
19.1
5.25.1
10.2Index Insert
Commit
5.424.4
TX Header
Redo Trail
Insert Row 1
Insert Row 2
Insert Row 3
Insert Row 4
Insert Row 5
Insert Row 6
© 2013 Julian Dyke juliandyke.com
Redo versus TrailSingle-Row Updates
65
5.2
UPDATE t2SET c2 = ‘Alpha’WHERE c1 = 1;
COMMIT;
5.1
11.5
5.1
Commit
5.424.4
TX Header
Redo Trail
Update Row 1
5.20
URP Undo
Update Row 2
5.1
11.5
5.1
11.5
URP Redo
URP Redo
URP Redo
URP Undo
URP Undo
Update Row 3UPDATE t2SET c2 = ‘Beta’WHERE c1 = 2;
UPDATE t2SET c2 = ‘Gamma’WHERE c1 = 3;
© 2013 Julian Dyke juliandyke.com
Redo versus TrailMulti-Row Updates
66
5.2
UPDATE t2SET c3 = c3 * 1.1;
COMMIT;
5.1
11.5
5.1
Commit
5.424.4
TX Header
Redo Trail
Update Row 1
5.20
URP Undo
Update Row 2
5.1
11.5
5.1
11.5
URP Redo
URP Redo
URP Redo
URP Undo
URP Undo
Update Row 3
© 2013 Julian Dyke juliandyke.com
Redo versus TrailSingle-Row Deletes
67
5.2DELETE FROM t2WHERE c1 = 1;
COMMIT;
5.1
11.3
5.1
Commit
5.424.4
TX Header
Redo Trail
Delete Row 1
5.20
IRP Undo
Delete Row 2
5.1
10.2
5.1
11.3
DRP Redo
Index Redo
DRP Redo
Index Undo
IRP UndoDELETE FROM tWHERE c1 = 2;
5.1
10.2Index Redo
Index Undo
© 2013 Julian Dyke juliandyke.com
Redo versus TrailMulti-Row Deletes
68
5.2DELETE FROM t2;
COMMIT;
5.1
11.3
5.1
Commit
5.424.4
TX Header
Redo Trail
Delete Row 1
5.20
IRP Undo
Delete Row 2
5.1
10.2
5.1
11.3
DRP Redo
Index Redo
DRP Redo
Index Undo
IRP Undo
5.1
10.2Index Redo
Index Undo
juliandyke.com
69 © 2013 - Julian Dyke
Thank You For Your Interest
info@juliandyke.com