10 Real-World Customer Configurations on the

Oracle Database Appliance

Shervin Sheidaei, Eclipsys (CA)

Simon Haslam, Veriton (UK)

About Eclipsys

Shervin Sheidaei

• 70+ ODA sales, ~30 were leveraging virtualization (70-30)

• Only ODA loaner program in Canada

• Offer “Solutions in a Box”

• WebCenter (http://www.oracle.com/technetwork/database/database-appliance/overview/oda-solution-in-a-box-1972012.html)

• Offer a range of productized, fixed price services including orientation services, virtualization, patching

• Deploy all sorts of configurations, including DR and Golden Gate

Simon Haslam Consultant, Veriton

Working with Oracle software since 1995

(and AWS since 2010)

Middleware & SOA

WebLogic, SOA, BPM and

Java Cloud Service Specialist

Veriton UK-based consultancy,

focussed on Oracle infrastructure

https://uk.linkedin.com/in/simonhaslam

Presentation themes

The versatility of the ODA

platform What people

are really doing with ODA

What is ODA?

Fast Intel compute nodes (2 @ 36 core/256-768GB)

Shared, direct attached storage array including flash

InfiniBand interconnect & 10Gb public networks

Management software (database & virtualisation)

Sold as a single product for $68k (list)

in a slide!

Bulk Data HDD

Redo Logs

ODA Cache SSD

Compute Node

Compute Node HDD

Now with

InfiniBand

x2

Why ODA?

Lots of x86-64 compute power

Single product SKU so easy to order ◦ No “configure to order” set of components to choose

◦ You know everything works together

Commodity pricing

Complete stack supported by Oracle

ODA VP is a complete virtualization solution

Oracle Data-center Appliance Versatile Platform

Customer Configurations

1. Database with DR/Test

2. Database + GoldenGate

3. VP Database + Data Guard

4. WebLogic

5. EM 12c Management Station

6. Test Environment – LXC & ZFS

7. SOA Appliance

8. Consolidation (App & DB)

9. Oracle Apps in a box

10. ODA – OVM platform

BM

Database with DR/Test

ODA Bare

Metal

Database with DR / Test Use Case & Requirements

Trade Agency

Need test DR on regular basis ◦ without impacting the production

◦ DR test must include all stacks including network, storage, sites up to database & application

Production DR cannot be used due to the risk of real site outage during test

Minimize overheads (resources, time) to maintain DR

Minimize license requirements

Quick & easy migration from old (Windows) environment to new

Database with DR / Test

Each database with 2 standbys

◦ Production standby – only for DR

◦ Test Standby – Regular DR Test

Consistent hardware & software

between all sites

Minimum cores use for Test standby

ODA

Database with DR / Test Learning

ODA’s consistency of software, hardware between 3 sites

simplifies management, reduces resources

Minimum license is enough to run test standby box - even it can

host some non-production databases

No previous OVM experience but auto-management of ODA &

oakcli helps with smooth transition from Windows to Linux

Image courtesy of digitalart at FreeDigitalPhotos.net

Database with Golden Gate

ODA Bare

Metal

Database with GG Use Case & Requirements

Financial Sector customer

Application and DB have to be up 24x7 due to the nature of business. ◦ Oracle shop but little experience with Oracle HA solution such as RAC

◦ Regular maintenance window adds up to downtime

Company acquisitions meant data centers in 2 cities

Hardware refresh is due

Database with GoldenGate

Database with GoldenGate Learning

Golden Gate not only helps to scale up beyond 2 compute nodes but also helps with

other areas which were not the initial goal such as

◦ Minimize application release downtime

◦ Minimize Database upgrade

◦ Flexibility of having solid rollback option for major changes/release

ODA platform frees up DBA to concentrate more on GoldenGate active-active

operation and tune up instead of dealing with day to day RAC or DB issue

Minimize Golden Gate license (ODA supports Hard Partitioning)

Image courtesy of digitalart at FreeDigitalPhotos.net

Database with Data Guard

ODA Virtualized Platform

Database on ODA VP Use Case & Requirements

Public Sector

DR is critical to the business (the need was realized after several incident including long power outrage on main site)

VMware was in place but unsatisfactory (expensive, not DB-aware)

Need to reduce DR licenses compared to current

Virtualization for DB deployment only

Database on ODAVP Learning

VM No-VM

Scale up and down (CPUPOOL)

Scale up Only (Core config.)

No migration path from ODA BM VP : one time choice

Why use ODA VP platform for just database deployment?

◦ Scale up and Scale down

◦ Licensing (minimums)

◦ Non –EE database deployment

Database on ODA VP Learning: Minimum Licenses

For standby database 2 cores is often sufficient to log apply

◦ In case of disaster and primary site is fully down licenses will be moved to standby sites

ODA X3-2 has a 4 core minimum for Bare Metal (2 core for all other models)

◦ go VP on X3-2 for 2 core minimum

BARE METAL

This customer is running ODA X3-2

WebLogic 12c for Custom Java App

ODA Virtualized Platform

WebLogic 12c for Custom Java App Use Case & Requirements

Health Insurance provider

Migration from VMware on blades – need Oracle VM for license partitioning since modern processors have many cores

Self-contained “application in a box”

Lifecycle operations of ODA upgrading means WLS + DB on one ODA is logical ◦ ODA non-rolling patching needs downtime so easier to affect both tiers of a few

environments than database tier for all

WebLogic 12c for Custom Java App

WebLogic MS2

ODA Base

Shared Repository

ODA Base

RAC Databases

Admin Server

WebLogic MS1

WebLogic MS3

OTD Admin

OTD Traffic 1

OTD Traffic 2

WebLogic MS4

Enterprise Manager

12c

WebLogic 12c for Custom Java App Learning

Disaster Recovery for non-permanent DR is tricky

Lack of backup solutions – nothing elegant

WLS ODA templates work with minimal changes

Previously had some re-write rules etc in OHS – needed

changing for OTD

Very robust – “just works”

EM 12c Management Station

ODA Virtualized Platform

Management Station: EM 12c Use Case & Requirements

Dutch Govt. Agency

Self-contained management for Exadata & Exalogic

High Availability required as central control for both

provisioning & management

Standardized solution familiar to Administrators

At OOW see also Brett Curtis, IDEXX [CON9745] & Cognizant [CON6928]

Credit to Jacco Landlust, Oracle NL

Management Station

Exadata

ODA X4-2

OMS OMS

ODA Base

ODA Base

OTD OTD

RAC Database ZFS

Exalogic

OTD, DNS, NTP…

Wiki

Audit

Utility Services

EM 12c SW

Management Station Exalogic

Exadata EM

ODA

ZFS

Data Center 2 Exalogic

Exadata EM

ODA

ZFS

Data Center 1 Exalogic

Exadata EM

ODA

ZFS

Data Center 3

Geo Load Balancer

Log shipping

Log shipping

Management Station Learning

Only follow ODA documentation

◦ E.g. do not get creative with OVM stuff

◦ Except when using VLANs for ODA: get creative!

Plan shared repo vs local repositories thoroughly

Choose an experienced ODA implementation partner

◦ E.g. cabling mishaps, important initial provisioning choices

Image courtesy of digitalart at FreeDigitalPhotos.net

Test Environments: LXC & ZFS SA

ODA Virtualized Platform

Test Environments: LXC & ZFS SA Use Case & Requirements

Health Insurance provider

Databases licensed by both Processor (pre-prod) & NUP

(test) on a single ODA

Simplicity & speed of cloning

ODA has sufficient cores/processing for test envs but

insufficient storage / IOPS

Test Environments: LXC & ZFS SA

ODA X3-2 (2.9+)

Test 1

ODA Base

ODA Base

RAC/EE Databases

ZFS SA

EE Database

Shared Repo

EE Database

Test 2

dNFS

Container

Container

Container

Container

Container

Container

10 GbE storage net

Each Test VM: 60 GB memory 14 vCPU Containers: 10 GB home BTRFS clones Repo: 300GB

Named User Plus Licences

Oracle Processor Licences

Test Environments: LXC & ZFS SA Learning

Don’t let shared repo fill up! (caused an outage)

vDisks – done manually, now possible with oakcli

Quite a lot of work to set up & overlap of lots of

relatively new technologies (LXC, BTRFS, ACFS).

One VM kernel per node – reboot needs outage of all

databases

This was pre-ODA 12c – now could use ACFS clones. Oracle still needs to address multi-license scenarios on ODA though.

SOA Suite on ODA VP

ODA Virtualized Platform

SOA Suite

Use Case & Requirements

O-box SOA Appliance (e.g. for Utility)

Self-contained, HA SOA platform

Predictable & high performance

Rapid provisioning

Standardized configurations – more robust & easier to support

Admin Server

SOA MS1

SOA MS2

SOA Suite on ODA VP

ODA Base

Shared Repositories

O-box Manager

ODA Base

RAC/EE Databases

Admin Server

SOA MS1

SOA MS2

OTD Admin

OTD Traffic 1

OTD Traffic 2

+ application database co-existence

multiple independent environments

SOA Suite Learning

Re-imaging nodes with ODA VP is a pain

WLS ODA template

Resizing VM disks

Shared storage woes

Performance of ODA X5-2 disks (LOCAL vs EXTERNAL

backup)

Consolidation Platform (App & Database)

ODA Virtualized Platform

Consolidation

Use Case & Requirements

Education

Variety of platforms (AIX, Windows, Linux) ◦ Hard to troubleshoot

◦ Costly in terms of resources, management of team and maintaining the environment

◦ Different vendors, support and licensing, patch cycles

No DR situation raised concerns

Without ODA

With ODA

Consolidation (App & Database) Learning

2 ODAs were capable to address all needs including DR

◦ Bonus: included other smaller environments not in plan

Simpler and better HA solution

◦ Automatic application VM failover (even with no clustered WebLogic)

◦ Quick DB instance recovery in production due to RAC deployment

Ability to test changes ahead of production deployment in QA

◦ Same platform, same software, same hardware

Image courtesy of digitalart at FreeDigitalPhotos.net

ODA- Oracle application in a box

ODA Virtualized Platform

Oracle application in a box– Use Case & Requirements

Public Sector, Insurance

Consolidation

Agile environment – need to spend less time on ops and more on

deploying new functionality

Solution needs to fit in with controlled network access

Oracle application in a box– Use Case & Requirements

Oracle application in a box– Learning

Migration of Oracle application on ODA :

Fresh install

◦ Start with Oracle-ready template

◦ Move application to new install

Migration

◦ Upgrade DB to 11gR2 (at least)

◦ P2V or V2V to move application to ODA

◦ Reconfigure application – Ebiz rapid clone

Oracle application in a box– Learning

ODA is ideal for segregating application traffic from database traffic

Simple physical segregation, no need for VLAN (though VLAN also possible)

User traffic

Admin traffic

ODA – OVM platform

ODA – Oracle VM Server Platform

Transportation

Mandatory security zones enforce to have physical server in ◦ Application tier zone

◦ Database zone

Optimize application license utilizing OVM (hard partitioning)

Require massive VM deployments (automation)

Quick migration/deployment

Lack of OVM skillset

Multiple ODAs Each node has 10’s of VMs on it

ODA – OVM Platform

ODA – OVM Platform Learning

ODA price-wise looks promising compare with other Oracle Eng system.(goal: lower License cost)

OVM world look simpler in ODA (no OVMM) and with oakcli. (No need to have big transition

from VMware to OVM to manage the new Env.)

Shared repository makes ODA shared storage to be accessible to appVM (except SSD)

ODA features make it competitive in the market (no extra cost)

◦ VM snap clone

◦ VM automatic failover

◦ VM migration

ODA – OVM Platform Learning

Automation of massive deployment is very feasible

Customize first VM (software, package…)

1

Create first VM

Start VMs

Create golden VM template

from first VM Assign IPs,

hostname (scripted)

Create VMs from golden VM template

2 3 4 5

ODA Virtualized – Platform of choice Learning

oakcli modify vm appvm1 -s 'com.oracle.linux.network.hostname:odawl16.mycompany.local' oakcli modify vm appvm1 -s 'com.oracle.linux.network.device.0:eth0' oakcli modify vm appvm1 -s 'com.oracle.linux.network.onboot.0:yes' oakcli modify vm appvm1 -s 'com.oracle.linux.network.bootproto.0:static' oakcli modify vm appvm1 -s 'com.oracle.linux.network.ipaddr.0:x.y.a.b' oakcli modify vm appvm1 -s 'com.oracle.linux.network.netmask.0:255.255.255.0' oakcli modify vm appvm1 -s 'com.oracle.linux.network.gateway.0:x.y.z.w' oakcli modify vm appvm1 -s 'com.oracle.linux.network.dns-servers.0:a.x.y.z,b.c.d.e' oakcli modify vm appvm1 -s 'com.oracle.linux.network.dns-search-domains.0:mycompany.local' oakcli modify vm appvm1 -s 'com.oracle.linux.root-password:welcome1'

• How to inject network configuration into VM created from template:

Summary

ODA Strengths & Weaknesses

Ease of use, especially patching

Simple OVM

Memory expandability (up to 768GB per node)

Single point of supply & support

Community

? ODA has powerful processors (72 cores)… too big?

Lack of IOPS for HDD – flash useful but relatively small

Downtime during infra patching

Questions?

www.eclipsys.ca veriton.com

@simon_haslam @shervsh

?