PHOUG RAC Presentation Final

8/2/2019 PHOUG RAC Presentation Final

1/39


2/39

Who Am I

EBS , RAC DBA Practitioner

12 years in Oracle field

Part of RAC Optimization &

Performance Team @DLL

Special projects & Critical

Services Benchmarking , Performance

Tuning

March 21 | PHL Oracle User Group

Disclaimer

These slides and materials represent the work and opinions of the author and do not constitute official

positions of my current or past employer or any other organization. This material has been peer

reviewed, but author assume no responsibility whatsoever for the test cases.


3/39

March 21 | PHL OUG, Philadelphia


4/39

#10

AVOIDINGCOMMON RACPROBLEMS



5/39

Simple Misconfigurations



6/39

Parameter Files :SPFILE & PFILE

create pfile from spfile in

one node

Edit ( Make Changes )

create spfile ..from pfile



7/39

#9HA & Fault

Tolerance



8/39

Database Challenges: Availability

Real Application Clusters

Be available all the time

> design for fault tolerance

> provide fast andreliable fault recovery

> eliminate maintenancedowntime

Support mission-critical

business operations



9/39Apr 14, 2012 | Company Proprietary

Components

Real Application Clusters

> Exploits clusters for very high availability> Overcomes the limitations of traditional failover solutions by

providing:

> Concurrent processing

> Load Balancing

> Fast time to recovery

Masks system failures from users/applications

Applications and users are automatically andtransparently reconnected to another system

Applications and queries continue uninterrupted

Login context maintained

DML transactions are rolled back


10/39

#8

Scalability



11/39


Database Challenges: Scalability


12/39

How does it work

System Settings/Options



13/39

Apr 14, 2012 | Company Proprietary

Issues/Troubleshooting

What might go wrong


14/39

#7

RAC &Performance



15/39

RAC Performance:

Not only move the jobs up

Average but also how can

We move the entire

Average Up!!!



16/39

Best Practices

Resize undo for Optimal Performance

Tune LGWR writes

Tune SQL statement to reducelogical reads

Reduce redo size



17/39

#6BATCH & OLTP

ServiceArchitecture



18/39

Service Configuration : Operation & Behavior

Each service set up comes with

Preferred Node

Available Node

No Failback

NO Auto Start

Applications dont need to know current servicesconfiguration!

Listeners know services status Listeners control which instance gets a new connection



19/39

#5

Concurrency



20/39

Apr 14, 2012 | Company ProprietaryApr 14, 2012 | Company Proprietary

Issue : Concurrency

Data archival (and deletion).

Performance of full tablescan.

To improve application

concurrency.

If the rows are inserted withsequence based primary orunique keys, then indexes on that

primary or unique key can be amajor bottleneck.


21/39

#4

Bugs



22/39

How does it work in Other Environments?

Issue is Fixed in NextVersion

Work around Does not

work Anymore ASM Crashed..

Alert log has entries of

ORA- errors butinstance did not crash..



23/39

#3

PARALLEL QUERY

ON RAC



24/39

Feature Overview

PQ slaves inherit service name

Slaves run on all nodes by default, ignoringPREFERRED and AVAILABLE nodes for service

Execution controlled through INSTANCE_GROUPS andPARALLEL_INSTANCE_GROUPS

In 11gR2 :

Slaves run only on nodes where service is active

PARALLEL_INSTANCE_GROUPS overrides defaultbehavior



25/39

Data Warehouse Example


A common requirement of

Oracle data warehousesare rollup and aggregationprocesses whereby super-large tables are read, end-to-end, computingsummary and average

values. Some datawarehouse queries alsohave ORDER BY or GROUPBY clauses, requiringOracle to retrieve and sorta very-large result set.


26/39

#2RAC Failover



27/39


State 1 :

Server 1

Instance 1

Server 2

Instance 2

Server 3

Instance 3

Failover Scenarios


28/39

Server 1

Instance 1

Server 2

Instance 2

Server 3

Instance 3

OLTP Users OLTP Users

No headroom within cluster

to fail over any users in the

event of an unplanned outage

Failover

Concurrent

Manager-Batchh

r-Batch


29/39

Failover

Uncontrolled shutdown of Instance 3

Server 1

Instance 1Server 2

Instance 2

Server 3

Instance 3

Concurrent

Manager -BatchOLTP Users OLTP Users

F il


30/39

Failover

Surviving Instance (1,2) performs instance recovery

Server 1

Instance 1Server 2

Instance 2

Server 3

Instance 3

Concurrent

Manager-BatchOLTP Users OLTP Users


31/39

Transparent Application Failover (TAF)

Recovery with Hot Failover

Shared Memory/Global Area

sharedshared

SQLSQLloglog

bufferbuffer


sharedshared

SQLSQLloglog

bufferbuffer


sharedshared

SQLSQLloglog

bufferbuffer

Shared Disk DatabaseShared Disk Database


32/39

Root Causes of Node Eviction

Network heartbeat lost Voting disk problems

cssd is not healthy

oprocd

Hang check timer

cssd and oclsomon race to suicide


33/39

Why a node is evicted?

Split brain condition I/o Fencing

CRS keeps the lowest number node up

Node eviction detection


34/39

#1

RAC &

EBS



35/39

Apr 14, 2012 | Company Proprietary

Concurrent Manager

Concurrent managers aggressively accessesfnd_concurrent_requests table, joining few other tablessuch as fnd_concurrent_programs, fnd_oracle_userid etc.

In a busy environment, fnd_concurrent_table is the topsegment contributing to global cache traffic.

Optimal values for sleep and cache_size is veryimportant.

SGA:

Bigger the SGA Better


36/39

Bonus


RAC OCR/VotingDisk Recovery


37/39

OCR / Voting Disk Got Deleted

If the OCR has been deleted, then check if the OCR

mirror is OK and vice versa. It may be prudent to usethe OCR mirror to create the OCR.



38/39

Q&Q U E S T I O N SQ U E S T I O N SA N S W E R SA N S W E R SAMarch 21 | PHL OUG, Philadelphia


39/39

Whats next?

Leave me your BusinessCards

Email me at:[email protected]

Share your feedback at:[email protected]

Thank you.March 21 | PHL OUG Philadelphia
mailto:[email protected]:[email protected]

PHOUG RAC Presentation Final

Documents

Transcript of PHOUG RAC Presentation Final