Simplify Database Storage and Dev/Test Environments CON3293 Doug Chamberlain Principal Product...

Simplify Database Storage and Dev/Test EnvironmentsCON3293

Doug ChamberlainPrincipal Product Manager, Oracle

Nirant PuntambekarSenior Manager, Software Development, Oracle

October, 2014

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Oracle Confidential – Internal/Restricted/Highly Restricted

W39677056W39677056


Oracle ZS3 Snaps/Clones are Co-Engineered with Oracle Database to Deliver Optimal Database Performance and Efficiency

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. | 4

Program Agenda

Challenges With Growing Development Efforts

Architecture and Functionality

Customer Case Study

1

2

3


Hurdles to Rapid Development and Deployment

• Increasing the speed of application development– Fast moving application/database requirements require systematic approach to dev/test– System/application patch/upgrades can introduce risk to dev/test– Custom application/development require deployment in less time

• Efficient duplicate of mission-critical databases for application development, test, and performance trouble-shooting– Multiple clones of production databases/systems often required to support dev/test activities– Dev/test and cloning of application and databases must be non-disruptive to production database

Innovation is a key requirement for business growth

62% manage more than 5 DB copies, 34% manage more than 20

IOUG Database Storage Survey, 2014


Lengthy and complex database provisioningChallenges to Development Agility Today

Data Protection

Development

Step 1

DBA

2

Storage

Administrator

3

DBA

23

Development

24………22


Challenges with Cloning Today

• Lack of automation– Scripts are good but often not a long term solution.

• Traditional cloning techniques are slow– Existing storage vendor solutions are DBA unfriendly

• Data explosion often outpaces storage capacity– Hurts ITs ability to provide clones for dev and testing

• Archaic processes that require multiple users to share a single clone, or only supports fixed refresh cycles– Different priorities between DBAs and Storage administrators

Impeding your efforts to innovate

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 8

Oracle Snap Management Utility for Oracle Database

• Engineered to work together– Oracle ZFS Storage Appliance - Oracle Database integration– ZFS Storage unlimited snapshot, clone, rollback capabilities

• Database Storage Efficiency– Rapid and Efficient backups, restores and provisioning– Empowers secondary processing such as Dev/Test, Reporting

• Simplifies Database Storage Management– Browser User Interface and CLI affords rapid familiarity– Setup DR environment with optional Remote Replication

Overview


Provision Oracle Database Efficiently, Faster

• Improved productivity– Gain control of the

process– DBA's get more done,

faster

• Simplified management– Rapid familiarity– Eliminates complexity

Automating the process with Snap Management Utility

Development

Step 1

Development

4

DBA

2 3Leverage RMAN backups already on ZFS Storage to facilitate low-footprint dev/test environment


Snap Management Utility for Oracle DB

• Enables snapshot and clone control from DBA–facing console with simple and intuitive BUI or CLI– Support for hot or cold database backups– Automate snapshots and retention policies

• Takes advantage of ZS3 system capabilities– Unified storage for file or block backups– Single instance or RAC databases– Virtually unlimited snapshots and clones

• Supports heterogeneous OS environments and engineered systems

Eliminate 88% of steps for database snaps


Program Agenda



Customer Case Study

1

2

3


• ZFS Snapshot– Read-only point-in-time copy of file system– Only changes tracked – near-zero space

consumption and near instantaneous

• ZFS Restore– Restore to any ‘point in time’ snapshot– File system or LUN modality

• ZFS Clone– Writeable copy of a snapshot– Ideal for storing many private copies of

mostly-shared data

• Database Snap Backup– Application aware snapshots based backup

of db files/shares

Oracle Snap Management Utility Terminology


Base Image1TB Data Files

Clone A1TB Data Files

Clone B1TB Data Files

Traditional ClonesFull copy of the original database

3TB for original + clones

Thin ClonesOnly changed blocks in new clone space

1.15TB for original + clones

Data Reduction: Thin ClonesOracle Snap Management Utility

Base Image1TB Data Files

Clone A50GB Data Files

Clone B100GB Data Files

Pointers to existing blocks

Pointers to existing blocks

New data

New data


Oracle Snap Management Utility

• Execute snapshot backups of the database on the ZFSSA– Cold (offline) and hot (online) backup procedures– File- and block-based storage, Single-instance and RAC

environments– BUI or CLI interface– Scheduler and retention policies, alerts notifications

• Restore to a snapshot backup– Restore database to a point in time backup

• Rapid Cloning for Dev/Test– Either from a snapshot backup or an RMAN image copy

Functionality


Snapshot Backups and RestoresOracle Snap Management Utility

SMU Host

Production ServerSID = PROD

Snapshot Backups

SID = PROD


Platform and Environment SupportOracle Snap Management Utility

OS Platform

• Oracle Solaris• Oracle Enterprise

Linux• Windows 2008• Red Hat Linux

Database Platform

• Oracle Database 10g• Oracle Database 11g• Single instance or RAC

Connectivity

• Block-based• iSCSI• ASM

• File-based• 1GbE, 10GbE,

InfiniBand• NFS, dNFS

ZFS Storage Appliance OS: 2011.1.5 or greater


Application Aware SnapshotsOracle Snap Management Utility

1 Query Database Locate Shares

Snapshot/Clone on Sun ZFSSA

Place Database in backup mode

Release DB/ Post-process

StorageDatabase

3

2

4

SMU



• Snap Management Utility can be used to coordinate creation of a new clone database from an existing database stored on a ZFS Storage Appliance– Create a database clone on the same host as the original or source database– Create the clone on another host– Create a clone that is single instance or clustered

• Clone from RMAN image copy– Utilize the ZFS Appliances for dev/test cloning

• De-provision Clone

Cloning Cases


Rapid Cloning for Dev/TestOracle Snap Management Utility

SMU Host

Development Server

Production Server SID = PROD

SID = DEV

SID = TEST

Snapshot Backup

Clone 1

Clone 2

SID = PROD

SID = TEST

SID = DEV


Clone from RMAN backupsOracle Snap Management Utility

SMU Host

SID = DEV

Development Server

CLONE

RMAN Image Copy

Snapshot

SID = DEV



• Create a new primary database from an RMAN image copy backup– Backup files must include datafiles, archived log, and controlfile

• RMAN cloning works in the following way:

Clone from RMAN Backup

1. Backup shares are snapped and cloned at the ZFS Storage Appliance

2. Clone shares are mounted on the target host

3. Start up a temporary database instance, mount the cloned controlfile

4. Perform a set of queries to get information about the backup

5. Shut down the temporary instance

6. Create a parameter file for the clone database

7. Start the clone database8. Create a new controlfile for the clone

database9. Recover and open the cloned database,

reset all logs10. Re-compile the schema objects


Key Features of Latest Release (v1.2.0)

• Dual storage controller support– Database and backup files may now span storage heads in an active-active clustered

appliance

• Host delegation tools– Delegated user can now be used to perform database, filesystem and disk system

administration on the database host

• Concurrent tasks– Tasks on different databases can now run simultaneously

• User Interface Enhancements– Tree Views, Pagination Controls, and Cloning Wizard

Available July 2014

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |Oracle Confidential – Internal/Restricted/Highly Restricted

23

Snap Management Utility

• Occupy a fraction of the space• Virtually unlimited backups• Fast backup and restores• Fast database cloning for dev/test• Improve SLA's• Simplified management with GUI• Direct control of storage

Key Takeaways

Increased Productivity

Lower Storage Costs

Simplified Management

Maintain DBAvailability,

Data Protection

SMU



• See Snap Management Utility in action– Demo Pod: 3912

• Product Page– http://www.oracle.com/us/products/servers-storage/storage/nas/snap/overview/index.html

Resources

http://www.oracle.com/us/products/servers-storage/storage/nas/snap/overview/index.html


Program Agenda



Customer Case Study

1

2

3


Customer Challenges• Duplicating databases to support the demands for application development

and testing– Compounded when the databases are large, require frequent refreshes, and many

copies to support multiple development and test activities.

• Objectives– Improve ability to scale data storage environment rapidly and cost-effectively to

support a growing division– Deliver responsive and personalized service through high-performing customer-facing

applications, as well as other core applications– Launch new services and offerings rapidly to ensure competitive edge


Customer Problem• 5 or more non-production databases for every production database • Demand for more non production databases continues to grow, presenting

additional challenges– Time it takes to provision– Cost of the hardware (especially disk)– Maintenance of the systems– Coordination with the development and test teams on the content

• Getting environments and data to the right team can take weeks or months– Delays lead to high project costs, schedule overruns, quality gaps, and frustration for

developers, and QA engineers


Oracle Database Environment: Before5 environments

1. Production 2. Pre production3. Functional QA (FQA)4. Business as Usual QA (BAUQA)5. Development

– All environments use single EMC VMAX storage– Most production databases are multi node RAC with data Guard– Production and Pre Production environments near match in configuration– Functional QA, BAUQA data and hardware is a subset of production – Development also a subset of production, and shared by developers

28


Architecture Overview: BeforeProduction

Pre – Production (Staging)

QA

Development

29


Capacity and Performance Metrics: BeforeMetric BeforeNumber of Databases 60Total Database Size 40TBNumber of Servers 60Total Disk Space Usage 40TBApproximate Provisioning Time Weeks


Solution• Selected Oracle ZFS Storage Appliance as the foundation for the company’s

network-attached storage (NAS) environment– Savings of more than 60TB of disk space—equating to $600,000—over legacy EMC

storage area network (SAN) system

• Deployed with Oracle Snap Management Utility for Oracle Database– Reduced database provisioning time from weeks to minutes, helping the company

launch new services quickly and optimize IT team productivity


Oracle Database Environment: After6 environments

1. Production 2. Pre production3. Functional QA (FQA) - multiple4. Business as Usual QA (BAUQA)5. Development – multiple6. Individual development environments

– Production and Pre Production remain on physical servers and EMC VMAX storage.– FQA, BAUQA, DEVINT, and individual database are clones running on ZFS storage appliance,

and a set of shared servers.

32


Oracle Database Environment – After PictureProduction

Pre – Production (Staging)

QA

Development Integration

Personal Databases

33


Results• Enabled the group to run full production data sets in the development environment

with thin-clone technology– Enabling more thorough testing and streamlining deployment of new applications while consuming

very little capacity

• Maintaining multiple versions of each database, can clone any database within minutes– Support an unlimited number of database clones compared to the limited one-to-one database-to-

server ratio supported by the legacy architecture

• Simplified, accelerated, and automated the process of copying, cloning, and protecting Oracle Database instances– Greater flexibility and control for database administrators

• Projected savings of $400,000 across all database projects in next 12 months– Cut hardware and maintenance costs by up to 10x


Capacity and Performance Metrics: AfterMetric Before AfterNumber of Databases 60 110Total Database Size 40TB 275TBNumber of Servers 60 12Total Disk Space Usage 40TB 20TBApproximate Provisioning Time Weeks Minutes

* Note: QA server consolidation is in progress and isn’t complete


Conclusion• Before: non production databases almost had a 1:1 ratio of database to server

– Constraining in terms of how many systems could be built and time it took to create them.

• Today: achieved a much higher density of databases to servers– Copies of production databases can be provisioned in minutes, instead of weeks or months. – Can now provision many copies of full production databases without fear of disk space

issues.

• Opened up possibilities of providing services not imagined in the past– Provisioning as many non-production databases as the engineers and testers request, in

minutes, and with full copies of sanitized production data

• Server, disk and time savings has easily paid for the new technology platform

Simplify Database Storage and Dev/Test Environments CON3293 Doug Chamberlain Principal Product...

Documents

Transcript of Simplify Database Storage and Dev/Test Environments CON3293 Doug Chamberlain Principal Product...