Dnfs Clonedb Oow 2012


Direct NFS and CloneDB with

the Latest Generation of

Database Technology

Akshay Shah

Principal Member of Technical Staff


The following is intended to outline our general product direction. It is intended

for information purposes only, and may not be incorporated into any contract.

It is not a commitment to deliver any material, code, or functionality, and should

not be relied upon in making purchasing decisions. The development, release,

and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle.


Program Agenda

Direct NFS introduction

Direct NFS performance

Direct NFS configuration and best practices

Direct NFS CloneDB overview

Direct NFS CloneDB architecture

Conclusion


Program Agenda






Conclusion


Introduction

NAS Attractive for database storage Easy to deploy and simple to administer

Runs over standard Ethernet

Cheaper than Storage Area Networks (SAN)

Shared file system for Grid Computing (RAC)

Why is NAS not more popular for databases? NAS performance is inferior to SAN

NFS client stack not optimized for database access patterns

Client management complexities

Configuration varies across different platforms.

Tuning parameters are quite complex.

High system CPU utilization


Direct NFS Client

New feature introduced in Oracle Database 11g Release 1

NFS version 3 client embedded in Oracle database kernel

Improves NAS storage performance for Oracle database

Improves high availability of NAS storage

Optimizes scalability of NAS storage

Vastly reduces system CPU utilization

Significant cost savings for database storage

Simplifies client management and eliminates config errors

Consistent configuration and performance across different platforms

Even Microsoft Windows


Standard NFS Architecture

Database NAS StorageStandard NFS client

LGWR I/O queue

DBWR I/O queue

PQ slave I/O queue

RMAN I/O queue

RPC I/O daemon

limits Oracle to 128

concurrent I/Os

Just 1 TCP

connection per mount

Linux NFS client RPC daemon

Each Oracle process

can issue 100s of

concurrent I/Os


Direct NFS Architecture

Database NAS StorageDirect NFS client

LGWR I/O queue

DBWR I/O queue

PQ slave I/O queue

RMAN I/O queue

LGWR TCP connection

DBWR TCP connection

PQ slave TCP connection

RMAN TCP connection

Direct NFS can issue 1000s

of concurrent operations due

to the parallel architecture

Every Oracle

process has its own

TCP connection


Direct NFS Performance

Direct NFS improves NAS performance for Oracle

Direct and Asynchronous I/O

Bypass operating system caches

Eliminate operating system write-ordering locks

Reduce system CPU by bypassing file system stack

Massively parallel I/O architecture

Each Oracle process creates its own TCP connection to storage

Not bound by the number of TCP connections to storage

Not constrained by rpciod (Limited to 128 concurrent operations, default is 16)


Scalability and High Availability

Direct NFS improves HA and optimizes server scalability

Supports up to 4 parallel network paths to storage

Displays linear scaling over both 1GbE and 10GbE paths

Automatically load balances across all network paths

No extra configuration necessary!!

Failover messages in the event of a path failure

1

2

5

6

7

8

3

4

Automatic load

balancing across

all paths Path 3

goes down

3

7


Cost Savings and Administration

Direct NFS provides significant cost savings

Simple Ethernet for Storage Connectivity

Eliminates expensive components in storage stack

Redundant host bus adaptors (HBA)

Fibre Channel switches

Bonded network interfaces

Direct NFS simplifies NAS administration

Standard NFS client implementation across all Oracle supported platforms

NFS is a viable option on platforms that dont natively support NFS, e.g. Windows

Supports Real Application Cluster (RAC) database files

Direct NFS automatically optimizes mount points for RAC

No NIC bonding required as Direct NFS manages HA for database files


Partner Validation Quotes

Direct NFS is an optimized NFS client for Oracle databases that

provides a high-bandwidth solution for data transfer with the Sun

ZFS Storage Appliance

Direct NFS is an excellent choice for customers who want to obtain

FCP-like performance for OLTP workloads at NFS costs

Direct NFS is easy to setup and the ease-of-use nature of the

option means it needs no maintenance once configured

Fibre Channel speeds and throughput and can be achieved

using pure NAS/ IP storage architectures with Oracle 11g Direct NFS


Program Agenda






Conclusion


41698

94578 9423111.6

5.8 6

0

2

4

6

8

10

12

14

0

20000

40000

60000

80000

100000

120000

w/o dNFS dNFS 10GbE FCoE

Late

ncy

in m

s

OE

Tran

sact

ion

s p

er

min

OLTP transactions and I/O latency

Order Entry Transactions DB file sequential read

OLTP Performance

Order Entry Transaction processing workload 1 TB database, 8000 warehouses, 250 concurrent users

Mix of transactions 65% reads, 35% writes

dNFS and FCoE

have same

throughput

dNFS latency is

slightly lower

than FCoE


Table Scan Performance Scan entire large table with 150 million rows

High degree of parallelism on the scan query

Linear throughput scaling with additional NICs

Throughput is limited by server-side bottleneck with 4 NICs

Low CPU utilization on the database host

112

224

328

380

7 1626

34

0

20

40

60

80

100

0

50

100

150

200

250

300

350

400

1 2 3 4

CP

U u

tiliz

atio

n(A

vg. %

)

Net

wo

rk T

hro

ugh

pu

t(M

B/s

ec)

Number of GbE connections

Network Throughut Host CPU utilization

Linear network

scaling Low CPU

utilization


Disk Write Performance

RMAN write from /dev/zero

Reading from /dev/zero to avoid ingest bottlenecks

Direct NFS almost saturates 2x10GigE pipes

Limited by disk bottleneck in the end

Great backup solution for high-end systems like Exadata

Supports sustained backup rates of up to 9 TB/hour with Exadata

0

500

1000

1500

2000

RMAN write throughput

Wri

te t

hro

ugh

pu

t(M

B/s

ec)

DirectNFS

KernelNFS

dNFS is

250% faster!!!


Program Agenda






Conclusion


Direct NFS Client Configuration (1)

1 simple step to get started with Direct NFS!!!

Disabling Direct NFS is just as simple

prompt> cd $ORACLE_HOME/rdbms/libprompt> make f ins_rdbms.mk dnfs_on

Enable the Direct NFS Client

prompt> cd $ORACLE_HOME/rdbms/libprompt> make f ins_rdbms.mk dnfs_off

Disable the Direct NFS Client


Direct NFS Client Configuration (2) Configure oranfstab config file for multi-pathing

Located in /etc or $ORACLE_HOME/dbs

server: myNFSServer1local: 192.168.1.1 path: 192.168.1.2local: 192.168.2.1 path: 192.168.2.2local: 192.168.3.1 path: 192.168.3.2local: 192.168.4.1 path: 192.168.4.2export: /vol/oradata1 mount: /mnt/oradata1export: /vol/oradata2 mount: /mnt/oradata2mnt_timeout: 600

Parameter Usage

Server Unique identifier for the NFS server

Local Network paths (up to 4) on the database host

Path Network paths (up to 4) on the NFS server

Export The exported volume on the NFS server

Mount The local mount point for the exported volume

mnt_timeout Time in seconds to wait for the first mount

dontroute Prevents OS routing of outgoing messages

management Network path for NFS server management interface

oranfstab parameters Sample oranfstab


Direct NFS Client Monitoring

Direct NFS v$ views

v$dnfs_servers

Lists the NFS servers that are mounted by Direct NFS client

v$dnfs_files

Lists the files that have been opened using Direct NFS client

v$dnfs_channels

Lists the TCP connections established by Direct NFS to the NFS server

v$dnfs_stats

Statistics on the different NFS operations issued by Oracle processes

select * from v$dnfs_servers;

svrname dirname mntport nfsport rtmax wtmax

mynfsserver /vol/oradata1 4046 2049 65536 65536


Direct NFS MOS notes

1496040.1 Direct NFS Frequently Asked Questions

1495104.1 Recommended patches for Direct NFS 11.2.0.3

1495739.1 Direct NFS monitoring and v$views


Program Agenda






Conclusion


Direct NFS CloneDB

Do you have NAS storage?

Do you take a backup of your production database?

Do you create Test/Dev instances?

Test clonesProduction

database


CloneDB overview

Introduced in Oracle 11.2.0.2

Clone production databases with minimal impact

Production data is safe and secure

Uses a simple RMAN backup

Refresh test instances with incremental backup

Based on copy-on-write technology

Huge storage space saving with thin provisioning

Works with single instance and RAC databases


Why CloneDB?

Easy to setup

No special storage software needed

Works on all platforms

Instantaneous cloning

No copies of the data

Create multiple clones based on one backup

Integrated with ZFS/NetApp/EMC snapshots

Refer to MOS note 1210656.1 for setup details


CloneDB Use Cases

Hardware/Software upgrades

Application/OS patching

Backup verification

Application development/testing

Recover Oracle objects

Run read-only report queries


Direct NFS CloneDB Test/dev setup

Application development

Storage size 5G

Clone DB #2ProductionDatabaseSize 1TB

Weekly

Production

Database

Backup

RMAN

Test hardware / software upgrades

Storage size 500 MB

Clone DB #1

dNFS


Direct NFS CloneDB Database upgrade

Production

Database

11.2.0.2

Weekly

Production

Database

Backup

RMAN

Clone

Database

11.2.0.2

dNFS Clone

Database

11.2.0.3

Apply DB

patch


Program Agenda






Conclusion


Direct NFS CloneDB Architecture

All datafiles in clone environment are sparse files

Copy blocks on demand

Changed blocks bitmap using bloom filter

Information about the backup file recorded in cloned database

Size of clone database is determined by rate of change not size of the production database



Cloned Database

Production

Database

Backup

Read dba (1,400)1,200

Oracle process p1

Block 400, on datafile 1 retrieved from production backup as it was never updated by clonedb

No copies maintained in clonedb, no storage occupied

Read dba (1,400)

1,800

1,1

1,500

1,9001,800

1,2

1,400

1,600



Cloned Database

Production

Database

Backup

Read dba (1,400)

Oracle process p1

Clonedb updates data in block 400 of datafile 1 and writes it to disk Data gets written to the clonedb storage Production database backup is intact

1,1

1,500

1,9001,800

1,2

1,400

1,600

Write dba (1,400)

1,200

1,800

1,400



Cloned Database

Production

Database

Backup

Oracle process p1

Updated data retrieved locally from the clonedb Application sees the latest changed data

1,1

1,500

1,9001,800

1,2

1,400

1,600

1,200

1,800

1,400

Read dba (1,400)

Write dba (1,400)

Read dba (1,400)


Direct NFS CloneDB Performance

Around 10% performance degradation between production and clonedb

Production database performance is not impacted

Performance limited by the storage and the number of test instances sharing backup

Data blocks non-contiguous in clone database

Full table scans can take longer than master

Redo log files, control files are always local


Direct NFS CloneDB Performance

Both production and clone run on a similar hardware and storage

OLTP workload running on both databases

Clonedb TPS within 10% of the production TPS

0

500

1000

1500

2000

2500

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65

TP

S

OLTP Throughput

MASTER

CLONE


Program Agenda





Direct NFS CloneDB deep dive

Conclusion


Conclusion

Direct NFS Extremely simple and easy-to-use

Performance comparable to Fibre Channel SAN

Built-in high availability and network scalability

Consistent performance across different platforms

Significant cost savings for database storage

CloneDB Instantaneous cloning

Platform/storage independent

Thin Provisioning

Works on NAS


Collateral

Direct NFS Direct NFS documentation

Oracle Direct NFS white paper

Sun ZFS storage appliance backup best practices

NetApp protocol comparison white paper

EMC Direct NFS white paper

Clone DB Oracle Clone DB MOS note

EMC Clone DB white paper

Direct NFS MOS notes 1496040.1, 1495104.1, 1495739.1


Q&A

Dnfs Clonedb Oow 2012

Documents

Transcript of Dnfs Clonedb Oow 2012