Upgrade to Oracle Database 19c
Transcript of Upgrade to Oracle Database 19c
Upgrade to Oracle Database 19cWith or without Multitenant
Mike Dietrich
Distinguished Product Manager
Oracle Global Services
Distinguished Product Manager
Database Upgrade and Migrations
Mike Dietrich
https://MikeDietrichDE.com
@MikeDietrichDE
mikedietrich
Slides | https://MikeDietrichDE.com
Twitter | twitter.com/MikeDietrichDE
Content Overview
Performance Stability
5AutoUpgrade
3Before Upgrade
2Release Strategy
1Migration to Multitenant
4
Ph
oto
by
Sp
ace
X o
n U
nsp
lash
Database Upgrade
Why you "want" to upgrade
Lifetime Support Policy
20
09
20
10
20
11
20
12
20
13
20
14
20
15
20
16
20
17
20
18
20
19
20
20
20
21
20
22
20
23
20
24
20
25
20
26
20
27
Oracle 18(12.2.0.2)
EXTENDEDOracle 12.1
EXTENDEDOracle 19(12.2.0.3)
EXTENDEDWaived EXTENDEDOracle 11.2 MARKETDRIVEN
Oracle 12.2.0.1 LIMITED
Premier Support Waived Extended Support Paid Extended Support Market Driven Support Limited Error Correction
Innovation
EXTENDEDLong Term Support
Innovation
Premier Support Paid Extended Support
Release Types | Long Term Support vs Innovation Releases
Long Term Support Release
• 5 years of Premier Support followed by 3 years of Extended Support
Innovation Release
• 2 years of Premier Support, but there is no Extended Support
• MOS Note: 742060.1 - Release Schedule of Current Database Releases
Oracle 12.2 Release Family
Includes:
• Oracle 12.2.0.1, Oracle 18c (12.2.0.2), Oracle 19c (12.2.0.3)
• MOS Note:742060.1 - The Single Source of Truth
• MOS Note:161818.1 - Releases Support Status Summary
20
16
20
17
20
18
20
19
20
20
20
21
20
22
20
23
20
24
20
25
20
26
20
27
Oracle 18(12.2.0.2)
EXTENDED
Oracle 12.2.0.1
Oracle 19(12.2.0.3)
LIMITED
Lifetime Support Policy Brochure
http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf
Old Format
12.2.0.1.0 Year. Update. Revision
New Format
19.8.0
Yearly Releases | 3 Number Format
Why will I speak only about Oracle 19c?
https://docs.oracle.com/en/database/oracle/oracle-database/20/rnrdm/deprecated-desupported-features.html#GUID-61D44BE2-21DE-463C-BEA9-8F8F1C75E45F
Your key to successful Database Upgrades
1 2 3 4
Download and install Oracle 19c
Download and install newest RU
Download and usenewest AutoUpgrade
eDelivery MOS: 2521164.1 MOS: 2485457.1
Get started | Quick Start Guide
• Simple overview
• Read it, try it
• Download from oracle.com
Check
Before Upgrade
Ph
oto
by
Pa
blo
He
imp
latz
on
Un
spla
sh
Supportability | OS Certification
Supportability | OS Certification
Supportability | OS Certification
More Links on the Upgrade Blog
Oracle Clusterware Certification on OL8/RHEL8https://mikedietrichde.com/2020/05/14/of-course-oracle-clusterware-is-certified-on-ol8-rhel8-as-well/
Oracle Database 19c Certification on OL8/RHEL8https://mikedietrichde.com/2020/05/11/oracle-database-19c-is-certified-on-ol8-and-rhel8/
Database Upgrade | Supported Releases
11.2.0.4
12.1.0.2
12.2.0.1
19c
18c
https://mikedietrichde.com/2019/02/14/oracle-database-19c-requires-ol7-rhel7-or-sles12-or-newer/
Oracle 19c | Installation
Gold Image
1. Create ORACLE_HOME directory
2. Download image
3. Unpack into ORACLE_HOME
4. ./runInstaller
5. root.sh
Oracle 19c | RPM Installation
RPM
• yum install -y oracle-database-preinstall-19c
• yum -y localinstall oracle-database-ee-19c-1.0-1.x86_64.rpm
• rpm installs always into: /opt/oracle/product/19c/dbhome_1
• Not practical for typical environments
Oracle 19c | Read-Only Home
Separate configuration and binary files
• ORACLE_BASE_HOME
• ORACLE_BASE_CONFIG
• Please read more:https://docs.oracle.com/en/database/oracle/oracle-database/19/ladbi/understanding-read-only-oracle-homes.html#GUID-AE1973D7-0325-4AD8-9FC9-D7BA3C3C3F37
In r/w: equal to ORACLE_HOME
APEX | Upgrade APEX upfront
APEX upgrade
• Not part of the database upgrade
• MOS Note: 1088970.1 - Master Note APEX Upgrades
APEX certification
• Minimum APEX Version for Oracle 19c: APEX 18.2
• MOS Note: 1344948.1 - APEX Database and Web Server Certification
Download the newest APEX
• https://www.oracle.com/tools/downloads/apex-v191-downloads.html
Upgrade 19c | Speed it up
• Check when dictionary stats have been gathered the last time
• Refresh stats a day before the upgrade
column OPERATION format a40
set linesize 200
select to_char(max(END_TIME),'DD-MON-YY hh24:mi') LATEST, OPERATION
from DBA_OPTSTAT_OPERATIONS
where OPERATION in ('gather_dictionary_stats', 'gather_fixed_objects_stats’)
group by operation;
Upgrade 19c | Make it run faster …
• Stats not gathered on several clustered indexes (bug 25286819)
• Wait … autoupgrade.jar does this for you automatically!
exec dbms_stats.gather_schema_stats('SYS');
exec dbms_stats.gather_index_stats('SYS','I_OBJ#');
exec dbms_stats.gather_index_stats('SYS','I_FILE#_BLOCK#');
exec dbms_stats.gather_index_stats('SYS','I_TS#');
exec dbms_stats.gather_index_stats('SYS','I_USER#');
exec dbms_stats.gather_index_stats('SYS','I_TOID_VERSION#');
exec dbms_stats.gather_index_stats('SYS','I_MLOG#');
exec dbms_stats.gather_index_stats('SYS','I_RG#');
Upgrade 19c | Effect of having good stats
• The larger the dictionary, the bigger the effect
• This example has been done with one of the tiny Hands-On Lab databases
Duration Reduction
No dictionary and fixed objects stats 15 min 55 sec
Gathered dictionary and fixed objects stats 14 min 10 sec 11 %
Gathered schema and cluster index stats 13 min 41 sec 3.4 % to previous
Total downtime saved 2 min 14 sec 14 % overall
Operating System | Recommendations
• MOS Note: 1587357.1Oracle Database on Unix AIX, HP-UX, Linux, Solaris and MS Windows –Installation and Configuration Requirements Quick Reference
AutoUpgrade
The ONLY recommended way to upgrade databases
Ph
oto
by
Jo
shu
a S
ort
ino
on
Un
spla
sh
Download | Always get the latest autoupgrade.jar
MOS Note: 2485457.1
Upgrade | AutoUpgrade
upg>
Job Console
11.2.0.412.1.0.2
12.2.0.1
18c
autoupgrade.jarMOS Note: 2485457.1
2MB
config.cfg
autoupgrade.jar
12.2.0.1
18.5.0
19c
19c
AutoUpgrade | Architecture
UpgradeJob Manager
UpgradeDispatcher
config.cfg
Database Post-Upgrade
Stats Refresh Time Zone Adjustment /etc/oratab
Database Upgrade
Command Line
Database Pre-Checks
preupgrade.jar Automatic Fix-Ups
System Checks
Space Check Requirements
AutoUpgrade | Requirements
• Java 8 or newer
• Default in your $ORACLE_HOME since 12.1.0.2
• Check: $ORACLE_HOME/jdk/bin/java -version
• The AutoUpgrade package
• A jar file of 2 MB in size
AutoUpgrade | Availability and Support
• CDB and non-CDB upgrades
• All operating systems running Oracle Database servers
• All editions (SE2, EE)
• Single instance and RAC databases
• Upgrades from version:
• 11.2.0.4 and newer
• Upgrades to version:
• 12.2.0.1 with Jan 2019 Update or newer
• 18.5.0 (includes Jan 2019 Update) or newer
• 19.3.0 or newer
AutoUpgrade | What you don't need …
• Agents
• EM Cloud Control
• DBUA
• Extra license
AutoUpgrade | Coming soon!
• Data Guard
• Disable DG Broker and defer log transport
• Unplug/plug/upgrade
AutoUpgrade | What it doesn't do
• Grid Infrastructure upgrade
• This is still done by the OUI when you invoke rootupgrade.sh
• Create a copy of your database
• Move your database to another host
• Create a new CDB to plugin after upgrade
AutoUpgrade | Blog Post Series
https://mikedietrichde.com/2019/04/29/the-new-autoupgrade-utility-in-oracle-19c/
AutoUpgrade | Use the newest version
• $ java -jar autoupgrade.jar –version
build.hash 3b51369
build.version 19.9.1
build.date 2020/07/10 11:20:49
build.max_target_version 19
build.supported_target_versions 12.2,18,19
build.type production
Oracle 19.7.0:
build.hash 04dd9f2
build.version 19.7.5
build.date 2020/02/11 15:28:49
build.max_target_version 19
build.type production
Download the newer version
AutoUpgrade | Create sample config file
$ java –jar autoupgrade.jar -create_sample_file config
Created sample configuration file/home/oracle/sample_config.cfg
AutoUpgrade | Example config file
global.autoupg_log_dir=/home/oracle/logs
upg1.dbname=DB12
upg1.start_time=NOW
upg1.source_home=/u01/app/oracle/product/12.2.0.1
upg1.target_home=/u01/app/oracle/product/19
upg1.sid=DB12
upg1.log_dir=/home/oracle/aux/DB12
upg1.upgrade_node=localhost
upg1.target_version=19
upg2.dbname=UPGR
upg2.start_time=NOW
upg2.source_home=/u01/app/oracle/product/11.2.0.4
. . .
Clarification
• One config file can be used for hundreds and thousands of databases
• You can start now
• But you can configure much more …
• Important: You don't need to!
AutoUpgrade | Adjust init parameters
#
# Example global pfile configuration entries
#
global.del_during_upgrade_pfile=/database/pfiles/global_during_delinit.ora
global.add_during_upgrade_pfile=/database/pfiles/global_during_addinit.ora
global.del_after_upgrade_pfile=/database/pfiles/global_after_delinit.ora
global.add_after_upgrade_pfile=/database/pfiles/global_after_addinit.ora
#
# Example local pfile configuration entries
#
upg2.del_during_upgrade_pfile=/database/pfiles/hr_during_delinit.ora
upg2.add_during_upgrade_pfile=/database/pfiles/hr_during_addinit.ora
upg2.del_after_upgrade_pfile=/database/pfiles/hr_after_delinit.ora
upg2.add_after_upgrade_pfile=/database/pfiles/hr_after_addinit.ora
AutoUpgrade | Execute external scripts
#
# Example global script execution
#
global.before_action=/database/scripts/run_this_before.sh Y
global.after_action=/database/scripts/run_this_after.sh
#
# Example local script execution
#
upg2.before_action=/database/scripts/run_this_on_UPG2_before.sh
upg2.after_action=/database/scripts/run_this_on_UPG2_after.sh
• Permitted extension options:
• Unix shell (.sh)
• Microsoft Windows batch (.bat, .cmd)
• Microsoft Windows PowerShell (.ps1)
Pass return codeback to AutoUpgrade
AutoUpgrade | Protection - GRP options
#
# By default a GRP will be created. Default for EE: Yes
#
upg2.restoration=no
#
# By default a GRP will be kept. This option overwrites the default.
#
upg2.drop_grp_after_upgrade=yes
AutoUpgrade | Other options
#
# Example global directive - Default: Underscores stay as before
#
global.remove_underscore_parameters=yes
#
# If unset, utlrp.sql won't be run - Default: YES
#
upgr2.run_utlrp=no
#
# If unset, time zone upgrade won't be performed - Default: YES
# As Timezone can't be downgraded, it may be useful to postpone and skip it
#
upgr2.timezone_upg=no
AutoUpgrade | Start time
• VALID values
• Format: DD/MM/YYYY HH24:MI:SS
• NOW [this is the default]
• 30/12/2018 15:30:00
• 01/11/2019 01:30:15
• 2/5/2019 3:30:50
Example | Adjust COMPATIBLE post upgrade
Add to your config file:
• upg1.drop_grp_after_upgrade=YES
• upg1.after_action=/home/oracle/scripts/change_compatible.sh
#!/bin/bash
. db19
sqlplus /nolog @/home/oracle/scripts/change_compatible.sql
connect / as sysdba
alter system set compatible='19.0.0' scope=spfile sid='*';
shutdown immediate
startup
exit
AutoUpgrade | 4 Operation Modes
autoupgrade.jar
-deploy
-analyze -fixups -upgrade
AutoUpgrade | Best Practice
$ java -jar autoupgrade.jar -config config.cfg -mode analyze
autoupgrade.jar
-deploy
-analyze
$ java -jar autoupgrade.jar -config config.cfg -mode deploy
AutoUpgrade | Move to new hardware
$ java -jar autoupgrade.jar -config config.cfg -mode analyze
autoupgrade.jar
-analyze -fixups
autoupgrade.jar
-upgrade
$ java -jar autoupgrade.jar -config config.cfg -mode deploy
$ java -jar autoupgrade.jar -config config.cfg -mode upgrade
Source System Target System
AutoUpgrade | Move to new hardware
autoupgrade.jar
-analyze
Source System
autoupgrade.jar
-fixups
global.autoupg_log_dir=/home/oracle/upg_logs
upg1.dbname=DB12
upg1.start_time=NOW
upg1.source_home=/u01/app/oracle/product/12.1.0.2
upg1.target_home=/u01/app/oracle/product/19
upg1.sid=DB12
upg1.log_dir=/home/oracle/upg_logs
upg1.upgrade_node=localhost
upg1.target_version=19
upg1.restoration=no
AutoUpgrade | Move to new hardware
autoupgrade.jar
-upgrade
Target System
global.autoupg_log_dir=/home/oracle/logs
upg1.dbname=DB12
upg1.start_time=NOW
upg1.source_home=/tmp
upg1.target_home=/u01/app/oracle/product/19
upg1.sid=DB12
upg1.log_dir=/home/oracle/logs
upg1.upgrade_node=localhost
upg1.target_version=19
upg1.restoration=no
• https://mikedietrichde.com/2020/08/03/oracle-autoupgrade-between-two-servers/
AutoUpgrade | Job Console
• Most important commands
upg>
lsj // List jobs
resume -job <number> // Restarts a job
status -job <number> // Show job status
restore -job <number> // Restores database from GRP
abort -job <number> // Aborts the specified job
AutoUpgrade | Log File Structure
• Logs written in TEXT and JSON format
• /cfgtoollogs
• ./upgrade/auto/status
• /database_1
• ./job_number
• ./prechecks
• ./preupgrade
• ./prefixups
• ./drain
• ./dbupgrade
• ./postupgrade
• ./temp
• /database_2
• …
Status Logs
HTML Report
Upgrade Logs
AutoUpgrade | Monitor the Progress
• cd ~/upg_logs/cfgtoollogs/upgrade/auto
• python -m SimpleHTTPServer 8080
• Open a browser:
• http://127.0.0.1:8080/state.html
• https://mikedietrichde.com/2020/07/16/autoupgrade-refresh-status-information-automatically/
AutoUpgrade | What if ...
1. ZIP the entire global log directory:
java -jar autoupgrade.jar -config config.cfg -zip
2. Add the alert.log
3. Upload it to MOS
AutoUpgrade | What if ... you have RAC?
• Since AutoUpgrade 19.8 (2020/03/27)
• RAC database upgrades are supported
• Works best with SPFILE in ASM
• Oracle Grid Infrastructure upgrade must be done separately upfront
After qualifying the new AutoUpgrade tool on a representative portion of our database landscape we found that tool was doing a great job and is production-ready. In our automation tool we have removed a lot of “home-grown” code, and replaced it with AutoUpgrade functionality.
Since August 2019 all upgrades at SimCorp have been executed using the AutoUpgrade tool.
Daniel Overby Hansen
Lead DeveloperSimCorp A/S - Denmark
Slides | https://MikeDietrichDE.com
Migration Strategies
Migration to Oracle Multitenant
Ph
oto
by
Ma
thia
s Je
nse
no
n U
nsp
lash
Oracle Multitenant | Oracle 19c
• Oracle 18c and Oracle 19c belong to the Oracle Database 12.2 release family
• Oracle 19c supports both non-CDB and PDB databases!
• For further details see MOS Note: 742060.1
Please note: After Oracle 19c, there will be only the CDB architecture
https://docs.oracle.com/en/database/oracle/oracle-database/20/upgrd/behavior-changes-deprecated-desupport-oracle-database.html#GUID-BA926361-95C9-4EB9-B4A7-FD3CC674A395
And by the way …
Since Oracle 19c you can have 3 user-created PDBs without need for a Multitenant license!!!
Upgrade to 19c | Migration to Multitenant
AutoUpgrade One-Command Orchestration
Oracle 11.2.0.4
AutoUpgrade
Oracle 19.8.0
Oracle 19.8.0
CDB
PDB$SEED
Oracle 19.8.0
CDB
PDB$SEED
1️⃣2️⃣ Plug into CDB
3️⃣Conversion withnoncdb_to_pdb.sql
AutoUpgrade | Migration to Multitenant
• Use these parameters in your config file
• The receiving CDB must be precreated already
# Specify the SID of the target CDB to plug into after upgrade
# Mandatory if you'd like to plugin
upg2.target_cdb=CDB2
# Give the non-CDB a new name when it becomes a PDB
# Optional
upg2.target_pdb_name=PDB1
# Define whether you'd like to use the COPY option
# Optional - if not specified, NOCOPY will be used
upg2.target_pdb_copy_option=file_name_convert('ORCL','PDB1')
Move to Multitenant | Automation?
• AutoUpgrade automates non-CDB-to-PDB migration
• Requires customer creating the destination CDB
• Can't lift the migration constraints (downtime, fallback)
• No automation for Endianness migrations
Videos | Upgrade Channel on Youtube
• https://www.youtube.com/channel/UCTKvCZwAbOeTSRFQDAh7yXw
• AutoUpgrade
• https://youtu.be/kr3aZrqA9Go [11:57 min]
• https://youtu.be/dO_y8sOasjs [DOAG 2019]
• Preupgrade Steps
• https://youtu.be/2R0Plcn7Udk [6:27 min]
• Database Upgrade
• https://youtu.be/ZqW3hS64hng [4:17 min]
• Postupgrade and Time Zone change
• https://youtu.be/kEBLqYDmspw [7:58 min]
DEMO
AutoUpgrade | Secret
• You can use AutoUpgrade with these parameters to plugin any 19c database
• Even without doing an upgrade
global.autoupg_log_dir=/home/oracle/logs
upg1.dbname=DB12
upg1.start_time=NOW
upg1.source_home=/u01/app/oracle/product/19
upg1.target_home=/u01/app/oracle/product/19
upg1.sid=DB12
upg1.log_dir=/home/oracle/logs
upg1.upgrade_node=localhost
upg1.target_version=19
upg1.restoration=no
upg1.target_cdb=CDB2
upg1.target_pdb_name=PDB1
Upgrade | AutoUpgrade Plugin Demo Video
https://mikedietrichde.com/2020/05/20/autoupgrade-and-plug-in-to-a-cdb-with-a-single-command/
Time for a break
After Upgrade
Important things to do after the upgrade completed successfully
Things to do right after upgrade
Configure statistics history retention period
• Check space usage:
• Check retention
• Default: 31 days
• Adjust setting
• Example: 10 days
SQL> select SPACE_USAGE_KBYTES/1024 MB
from V$SYSAUX_OCCUPANTS where
OCCUPANT_NAME='SM/OPTSTAT';
SQL> select
DBMS_STATS.GET_STATS_HISTORY_RETENTION from DUAL;
SQL> exec
DBMS_STATS.ALTER_STATS_HISTORY_RETENTION(10);
Post Upgrade | Time Zone Change
• Adjust time zone post upgrade
• Newest time zone patch:
• MOS Note:412160.1
• New scripts in ?/rdbms/admin:
• utltz_countstar.sql
• utltz_upg_check.sql
• utltz_upg_apply.sql
Oracle Database Release Default TZ Version
11.2.0.2 - 11.2.0.4 DST V14
12.1.0.1, 12.1.0.2 DST V18
12.2.0.1 DST V26
18c DST V31
19c DST V32
Post Upgrade | Unified Audit Trail
• Internal structure changed to improve query performance of UNIFIED_AUDIT_TRAIL view
• Old records must be converted to new format
• To convert:SQL> EXEC DBMS_AUDIT_MGMT.TRANSFER_UNIFIED_AUDIT_RECORDS;
• Only applicable if upgrading from Oracle Database 12.1
More Information | Client Connectivity
MOS Note: 207303.1Client / Server Interoperability Support Matrix for Different Oracle Versions
Things to know
Best Practices and things to take care on
Best Practice | Database COMPATIBLE Parameter
• When should you change COMPATIBLE?
• 7-10 days after the upgrade
• Which value?
• 11.2.0
• 12.1.0
• 12.2.0
• 18.0.0
• 19.0.0
• Change COMPATIBLE for RUs?
• NEVER!
• Flashback to GRP• Downgrade
Oracle 19c | Multimedia Removal
• The Oracle Multimedia API gets removed during upgrade
• The Locator still exists and works
• ORDIM component remains VALID in DBA_REGISTRY
• Recommendation
• Check, if you use Oracle Multimedia
• If not, you can remove it before upgrade
• More information
• https://mikedietrichde.com/2019/02/18/oracle-multimedia-will-be-removed-in-oracle-database-19c/
• MOS Note: 2347372.1 – Oracle Multimedia Statement of Direction
• MOS Note: 2375644.1 – How To Migrate Data From Oracle Multimedia Data Types to BLOB columns
Oracle 19c | Streams Removal
• The Oracle Streams API gets removed during upgrade
• See:
• https://docs.oracle.com/en/database/oracle/oracle-database/19/upgrd/behavior-changes-deprecated-desupport-oracle-database.html#GUID-7E90C2D3-853A-45B2-AC6D-C9326798E82B
Oracle 19c | DBMS_JOB
DBMS_JOB package is still supported
• Grant the CREATE JOB privilege
• During and after upgrade, for each job in DBMS_JOB a corresponding entry will be created with DBMS_SCHEDULER
• The check in preupgrade.jar is only checking for inconsistencies or any issues
MOS Note: 2380601.1 - Database Preupgrade tool check list
Health Check | hcheck.sql
If your database is highly important, do a health check
• hcheck.sql - Script to Check for Known Problems (MOS Note: 136697.1)
Your key to successful Database Upgrades
1 2 3 4
Download and install Oracle 19c
Download and install newest RU
Download and usenewest AutoUpgrade
eDelivery MOS: 2521164.1 MOS: 2485457.1
SPM, STA, RAT for Performance Stability
Testing | Typical Mistakes
• Only 10% of real data used
• Data sets artificially created
• Tests done on a laptop
• No testing tools used
• Data from 6 months ago
• No statistics refreshed
• Testing?? Waste of time!
• Real experts fix it after go-live …
Ph
oto
by
Elij
ah
O'D
on
ne
ll o
n U
nsp
lash
Best Practice | COMPATIBLE vs Optimizer
• COMPATIBLE and OPTIMIZER_FEATURES_ENABLE
• Fully independent from each other
• Set COMPATIBLE to the default of the release, e.g. 19.0.0
• Change OPTIMIZER_FEATURES_ENABLE only when you have to
• Avoid it if possible
• This is not a Swiss Army knife!
• You will turn off a lot of great features
Testing Tools | Workflow
Collect execution plans before upgrade
Compare AWR Snapshots
Verify them with SQL Performance
Analyzer
Regressed plans?
SQL Plan Management
SQL Tuning Advisor
Verify functionality and performance with
Database Replay
Testing Tools | Collect Plans on Production
STS
Capturing from cursor cache and AWR into an STS does not require an extra license for EE anymore:
Find it under: "SQL Tuning Sets" at:https://docs.oracle.com/database/121/DBLIC/editions.htm#DBLIC109
Testing Tools | Comparison
Collect execution plans before upgrade
Compare AWR Snapshots
Verify them with SQL Performance
Analyzer
Regressed plans?
SQL Plan Management
SQL Tuning Advisor
Verify functionality and performance with
Database Replay
Testing Tools | AWR Diff Reports
Upgrade / Migrate
11.2.0.4 19.8.0
Snapshot Interval
Compare
AWR Diff Report: awrddrpt.sql Snapshot Interval
Testing Tools | SQL Performance Analyzer
Collect execution plans before upgrade
Compare AWR Snapshots
Verify them with SQL Performance
Analyzer
Regressed plans?
SQL Plan Management
SQL Tuning Advisor
Verify functionality and performance with
Database Replay
Testing Tools | SQL Performance Analyzer
SQL Performance Analyzer
STS
Capture SQLsinto a STS
Load STS
STS
Evaluate, compare and tune –over and over again!
11.2.0.4 19.8.0
Upgrade / Migrate
Testing Tools | SPA - Regressed Report
Testing Tools | SQL Plan Management
Collect execution plans before upgrade
Compare AWR Snapshots
Verify them with SQL Performance
Analyzer
Regressed plans?
SQL Plan Management
Verify functionality and performance with
Database Replay
Testing Tools | Situation after change
• Challenging to "freeze" execution plans
Parse ExecutePlan acceptable
Plan possiblynot acceptable
Conditions change:New statistics, parameter change, upgrade
GB
HJ
...HJ
GB
NL
...NL
SQL
SQL
ExecuteParse
SQL Plan Management | Plan Stability
GB
HJ
...HJ
Plan Baseline
Plan History
STS
STSExport/Import for Migrations
Upgrade
Plans will be takenfrom STS and addedto the SQL Plan Baseline
DBMS_SPM.LOAD_PLANS_FROM_SQLSET
Testing Tools | SQL Tuning Advisor
Collect execution plans before upgrade
Compare AWR Snapshots
Verify them with SQL Performance
Analyzer
Regressed plans?
SQL Plan Management
SQL Tuning Advisor
Verify functionality and performance with
Database Replay
SQL Tuning Advisor | SQL Profiles
• SQL Profiles provide more information to the optimizer
• Part of the Tuning Pack
• FORCE_MATCH=TRUE
• Don't change the application code
• Persistent, transportable (MOS Note:751068.1), can be tested without risk
• Example:
Testing Tools | Database Replay
Collect execution plans before upgrade
Compare AWR Snapshots
Verify them with SQL Performance
Analyzer
Regressed plans?
SQL Plan Management
SQL Tuning Advisor
Verify functionality and performance with
Database Replay
Testing Tools | Database Replay
Database Replay
Capture Workload
Replay Workload
Compare – then restoreand run again with changes
11.2.0.4 19.8.0
Migrate
Preprocess Workload
Help me - I have an upgrade problem …
In>95%of all cases
Ph
oto
by
nik
ko m
aca
spa
co
n U
nsp
lash
“ ”
an "upgrade problem" in reality is a performance issue after upgrade, or not a database upgrade topic.
There is exactly one way to mitigate the risk:
TESTING!
Automatic SQL Tuning Set
In Oracle 19.7.0, a Automatic SQL Tuning Set gets populated
• Some customers reported high growth and consumption in SYSAUX
• See: https://mikedietrichde.com/2020/05/28/do-you-love-unexpected-surprises-sys_auto_sts-in-oracle-19-7-0/
• If you want to disable it:• BEGIN
DBMS_AUTO_TASK_ADMIN.DISABLE(client_name=>'Auto STS Capture Task', operation=>NULL, window_name=>NULL);
END;
/
• Task is not enabled by default from Oracle 19.8.0 on
Oracle 19c | _cursor_obsolete_threshold
• MOS Note: 2431353.1High Version Counts for SQL statements (>1024) post upgrade To 12.2 and above causing database slow performance
• Since Oracle 12.2.0.1, the old default of 1024 has been raised to 8192
• This may lead to various drastic performance issues
• Recommendation:
• Set it to the old default to avoid mutex concurrency and other issues:
_cursor_obsolete_threshold=1024
Oracle 19c | deferred_segment_creation
• MOS Note: 1216282.1All sorts of issues and bugs - from corruption to mutex contention to customer scripts not operating correctly anymore
• Recommendation:Set it todeferred_segment_creation=false
unless you have a valid reason
Oracle 19c | Adaptive Features
• Recommendation: Set optimizer_adaptive_statistics explicitly in your SPFILE
• See:
• https://mikedietrichde.com/2018/01/18/additional-info-adaptive-features-fixes-oracle-12-1-0-2/
• MOS Note: 2187449.1 - Recommendations for Adaptive Features in Oracle Database 12.1
Oracle 12.1 optimizer_adaptive_features = TRUE
Oracle 12.2
AdaptiveStatistics
optimizer_adaptive_statistics
Default: FALSE
AdaptivePlans
optimizer_adaptive_plans
Default: TRUE
Oracle 19c | _sql_plan_directive_mgmt_control
• MOS Note: 2209560.1 - How To Disable SQL Plan Directive (SPD)
• In order to fully disable SQL Plan Directives, you need to set:
• _sql_plan_directive_mgmt_control=0
• Otherwise the database collects SPDs in the background, but won't use it
• Having optimizer_adaptive_statistics=false - which is the default - disables only the usage
of SPDs but not their creation
Getting ready for the future
Upgrade / Migrate and Consolidate to Multitenantat La Mobilière, Switzerland
Customer Case | La Mobilière
• Swiss Mobiliar
• Switzerland's most personal insurer
• Founded 1826 in Bern, oldest Swiss insurance
• Legal form:
• Cooperative association (mutual company
Customer
Project
Constraints
Preparation
Upgrade
Success?
Remarks
Customer Case | La Mobilière
Customer
Project
Constraints
Preparation
Upgrade
Success?
Remarks
Customer Case | La Mobilière
• Parallel upgrade catctl.pl unfolds its full power when upgrading many
PDBs at the same time
• 50 PDBs upgraded in less than 4 hours
• We feel very well prepared for the annual Oracle releases
• When we encounter issues, we fix them before going live
• Follow their projects on: https://mobiliar.ch/db-blog
• 100% Multitenant Consolidation reached in Oct 1, 2019
Customer
Project 2017
Constraints
Preparation
Upgrade
Success?
Remarks
Customer Case | La Mobilière
• High increase in number of PDBs, often driven by Microservices
• Self-developed DBaaS interface to privision, alter and drop PDBs
• Used to create >1000 PDBs
Customer
Project 2017
Constraints
Preparation
Upgrade
Success?
Remarks
Migrating to the Future
Exadata Migrationat La Mobilière, Switzerland
Upgrade NOW!
Upgrade to Oracle 19cat La Mobilière, Switzerland
Ph
oto
by
Ric
ard
o G
om
ez
An
ge
lon
Un
spla
sh
Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
Customer Case | La Mobilière
• Upgrade 2000 PDBs
• Oracle Database 12.2.0.1 to 19c
• Up to 50 PDBs per CDB in PROD
• Up to 150 PDBs per CDB in DEV/TEST
Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
Customer Case | La Mobilière
• CPU resources limited
• Solution: Sequential upgrades
• One DBA covers 1-2 CDB upgrades
• Once done, next DBA steps in
Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
Customer Case | La Mobilière
• Adopt AutoUpgrade
• Download newest version from MOS: 2485457.1
• Phase 1: 735 PDBs on a single weekend
• CDB1 144 PDBs• CDB2 148 PDBs• CDB3 148 PDBs• CDB4 147 PDBs• CDB5 148 PDBs
Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
* Logs of CDB5 are lost
Customer Case | La Mobilière
Upgrade timings *
Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
Customer Case | La Mobilière
• Upgrade timings
• Average: 6 - 7.5 hours
• Including recompilation
• Only 10 oCPUs used per CDB
Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
Customer Case | La Mobilière
• Post-fixups issue
• Solution: Use the newest OPatch
• Processes parameter
• Upgrade in PDBs spawns a lot of processes
• Solution: Increase it for the upgrade
• Recompile PDBs before upgrade
• ORACLE_HOME/perl/bin/perl $ORACLE_HOME/rdbms/admin/catcon.pl
-n 32 -e -d $ORACLE_HOME/rdbms/admin -l /tmp -b utlrp_log
utlrp.sql
• Read the full story here:
• https://mobiliardbblog.wordpress.com/2020/05/05/what-does-it-take-to-migrate-735-databases-to-19c-in-a-weekend-the-multitenant-architecture-and-great-dbas/
"We migrated 735 databases to 19c, and the task was mostly relatively relaxed. Start the AutoUpgrade tool and monitor the progress from time to time.
Sitting in front of the screen the whole time is not necessary."
Alain Fuhrer
Head IT Database Services
La Mobilière
Bern, Switzerland
Further Information
Finally …
Slides | https://MikeDietrichDE.com
HOL | https://MikeDietrichDE.com
Watch the entire seminar | https://MikeDietrichDE.com/videos
https://mikedietrichde.com/2020/07/06/upgrade-seminars-2020-recap-and-videos-on-demand/
Thank You!