Oracle Database In-Memory - Getting Started and … Database In-Memory Real-Time Analytics Trivial...

Copyright © 2018, Oracle and/or its affiliates. All rights reserved. |

Oracle Database In-Memory -Getting Started and Best Practices

Andy RivenesProduct ManagerSystems Technology Group

RMOUG, February 20-22, 2018


Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


What is Database In-Memory


Oracle Database In-Memory

Real-Time Analytics Trivial to Implement

No Application ChangesNot Limited by Memory

100X

Accelerate Mixed Workload

AnalyticsTransactions

Run analytics on Operational Systems

Enable Real-Time Business Decisions

Real-Time Analytics100X

Risk-Free

Proven Scale-Out, Availability, Security


Breakthrough: Dual Format Database

• BOTH row and column formats for same table

• Simultaneously active and transactionally consistent

• Analytics & reporting use new in-memory Column format

• OLTP uses proven row format

Buffer CacheNew In-Memory

Column Store

SALES SALESRow

FormatColumnFormat

SALES


Oracle In-Memory: Simple to Implement

1. Configure Memory Capacity• inmemory_size = XXX GB

2. Configure tables or partitions to be in memory• alter table | partition … inmemory;

3. Later drop analytic indexes to speed up OLTP


• New In-Memory Advisor – see OTN or MOS Note 1965343.1

• Analyzes existing DB workload via AWR & ASH repositories

• Provides list of objects that would benefit most from being populated into the IM column store

Advisors : Oracle Database In-Memory Advisor

Note: Database Tuning Pack license required


• Easy way to determine memory requirements

• Use DBMS_COMPRESSION

• Applies MEMCOMPRESS to sample set of data from a table

• Returns estimated compression ratio

Advisors : Oracle Compression Advisor And In-Memory


How Do I Install & ConfigureThe In-Memory Column Store?


Note: Database In-Memory is not enabled by default

• Automatically installed as part of Database 12c (12.1.0.2 & 12.2)

• No additional steps required

Installing : Oracle Database In-Memory


Installing: Apply the Latest Database Proactive Bundle Patch

• Database In-Memory fixes and enhancements are only distributed through Database Proactive Bundle Patches

• See MOS Notes:

– 2285557.1 – Database 12.2.0.1 Proactive Patch Information

– 2285558.1 – Database 12.1.0.2 Proactive Patch Information

• Starting with the latest bundle patch avoids re-inventing the wheel discovering bugs that have already been fixed!


• Component of SGA• Controlled by INMEMORY_SIZE

parameter� Minimum size of 100MB

• Can dynamically grow larger (12.2)• SGA_TARGET must be large

enough to accommodate

Configuring : In-Memory Column Store

SELECT * FROM V$SGA;

NAME VALUE------------------ ---------Fixed Size 2927176Variable Size 570426808Database Buffers 1024483648Redo Buffers 13848576In-Memory Area 4634022912

Note: Don’t steal Memory from other components


Configuring : In-Memory Column StoreDon’t get carried away

• Don’t give all memory to SGA • Don’t want any aspect of

execution plan to spill to disk• Ensure PGA_TARGET is set

large enough to keep joins & sorts in memory

• Use Parallel Execution to maximize PGA usage


Tips To RememberConfiguring In-Memory Column Store

• If you can’t avoid spilling to temp do it efficiently

- Increase the size of temp writes from 200KB to 1MB

- Set _smm_auto_max_io_size =1024

• Database In-Memory is not enabled by default

• INMEMORY_SIZE must be set to 100MB or larger

• SGA_TARGET Existing SGA_TARGET INMEMORY_SIZE

• PGA_TARGET Max expected parallel servers 2GB

(Single Instance)

• SGA_TARGET Existing SGA_TARGET (INMEMORY_SIZE * 1.1) (RAC)

(See MOS Note 1903683.1)


How Do I Get Data In And Out Of The In-Memory Column Store?

DATA

DATA

DATA


• New INMEMORY ATTRIBUTE

• Eligible segment types are� Tables

� Partitions

� Subpartitions

� Materialized views

• Following types not eligible� IOTs

� Hash clusters

� Out of line LOBs

Populating : Enable Objects for In-Memory

CREATE TABLE customers …… PARTITION BY LIST

(PARTITION p1 …… INMEMORY,(PARTITION p2 …… NO INMEMORY);

ALTER TABLE sales INMEMORY;

ALTER TABLE sales NO INMEMORY;

Pure OLTP Features


• You don't have to populate all columns

• It is possible to populate only certain columns

• Two phase approach1. INMEMORY attribute on Table

automatically inherited by columns

2. Need to remove attribute from the columns you don’t want populated

Populating : Columns Can Be Excluded

ALTER TABLE sales INMEMORYNO INMEMORY (delivery_note);


• Objects compressed during population

• New compression techniques� Focused on scan performance

• 2x to 20x compression typical• Multiple levels of compression

• FOR DML• FOR QUERY LOW/HIGH

• FOR CAPACITY LOW/HIG

• Possible to use a different level for different partitions in a table

Populating : Compression

ALTER MATERIALIZED VIEW mv1 INMEMORYMEMCOMPRESS FOR QUERY LOW;

CREATE TABLE trades (Name varchar(20),Desc varchar(200))

INMEMORYMEMCOMPRESS FOR DML(desc);


• Order in which objects are populated controlled by PRIORITY subclause:ALTER TABLE sales INMEMORY PRIORITY HIGH;

• Levels:

� CRITICAL > HIGH > MEDIUM > LOW

� Controls order (not speed) of populate

• Default PRIORITY is NONE

� Populate only on first access

• Population completed by new set of background processes

ora_w001_orcl• Number of processes controlled by

parameter: INMEMORY_MAX_POPULATE_SERVERS

Populating : Priority


Monitoring : In-Memory Column Store Population• V$IM_SEGMENTS– Indicates which objects are currently populated in-memory– Shows current size of each segment in-memory – Shows how much remains to be populated

SQL> select segment_name, populate_status, inmemory_priority, inmemory_size, bytes_not_populated from v$im_segments;

SEGMENT_NAME POPULATE_STATUS INMEM_PRIORITY INME_SIZE BYTES_NOT_POPULATED------------ --------------- -------------- ----------- -------------------ACCOUNTS STARTED HIGH 196606 2434886912SALES COMPLETED CRITICAL 135790592 0


Tips To Remember

• In-Memory column store is not a cache– Once populated objects not automatically aged out

• Only objects with the INMEMORY attribute are candidates

• Objects with a PRIORITY are populated at database startup (not the default)– Population is triggered after IMCO wakes up for the first time (2-3

minutes after open)– On demand objects can supersede the population order if accessed

• Use (g)V$IM_SEGMENTS to monitor the progress of population

Populating In-Memory Column Store


How Do I Tell If The In-Memory Column Store Is Being Used?


• Speed of memory• Scan and Filter only

the needed Columns• Vector Instructions

Identifying : INMEMORY Queries Improves key aspects of analytic queries

Data Scans

Vect

or R

egist

erCPUCA

CACA

CA

In-Memory Aggregation

•Create In-Memory Report Outline that isPopulated during Fast Scan•Runs Reports Instantly

Joins

•Convert Star Joins into 10X Faster Column Scans•Search large table for values that match small table

HASH JOIN

Table A Table B

SALES

STAT

E =

CA


Identifying : INMEMORY Table Scan

• Optimizer fully aware

• Cost model adapted to consider INMEMORY scan

• New access method TABLE ACCESS IN MEMORY FULL

• Can be disabled via new parameter - INMEMORY_QUERY


• Bloom filters enable joins to be converted into fast column scans

• Tried and true technology originally released in 10g

• Same technique used to offload joins on Exadata

Identifying : INMEMORY Joins


Identifying : INMEMORY Aggregation

Hash Group By

Vector Group By


Identifying : INMEMORY Table Scan

• NEW Session level statistics

• Best way to determine if In-Memory was used

• Best way to measure the benefits of In-Memory scan

IM scan bytes in-memoryIM scan bytes uncompressedIM scan CUs columns accessedIM scan CUs columns decompressedIM scan CUs columns theoretical maxIM scan rowsIM scan rows range excludedIM scan rows excludedIM scan rows optimizedIM scan rows projectedIM scan CUs predicates receivedIM scan CUs predicates appliedIM scan CUs predicates optimizedIM scan CUs prunedIM scan segments minmax eligible ….


What about Indexes?

• Don't drop all the indexes, you may still need them

• Definitely keep primary key, foreign key and unique indexes, they enforce referential integrity

– Insure that there are corresponding constraint definitions

• You may also need other existing indexes for OLTP type access

• Target reporting indexes that won't be needed with In-Memory

– Start by making these indexes invisible– After running the workload and verifying that the indexes are not needed, then drop

–Making indexes invisible is much easier than dropping them and having to re-create


What Workloads Benefit From DatabaseIn-Memory


Queries that Benefit from Database In-Memory• Return as few rows as possible• Limit the number of columns accessed• Use aggregation to return fewer values• Use selective column predicates• Use selective join conditions• Limit the number of tables being joined• Avoid complex SQL functions


• For a non-trivial amount of rows and execution time:– Majority of time spent accessing data– Majority of time spent joining data– Majority of time spent on scan and group by

operations

When Database In-Memory Helps

Majority of time spent accessing data

Majority of time spent

joining data

Majority of time spent on scan and

group by operations


How does Database In-Memory work with RAC


RAC : Scale-Out In-Memory Database to Any Size

• Scale-Out across servers to grow memory and CPUs

• Shared nothing architecture

• IMCUs not shipped across interconnect

• In-Memory queries are parallelized across servers to access local column data


RAC : In-Memory and Distribution of Data

• Distribution allows in memory segments larger than individual instance memory

• Policy is automatic (Distribute AUTO)or user-specifiable

• Controlled by DISTRIBUTE subclause

� Distribute by rowid range� Distribute by partition� Distribute by subpartition

• Goal: Ensure Even Distribution

ALTER TABLE sales INMEMORYDISTRIBUTE BY PARTITION;

ALTER TABLE COSTS INMEMORYDISTRIBUTE ROWID RANGE;


RAC : Database In-Memory Queries in a RAC Environment

• Shared nothing architecture means Parallel Query must be used to access data

• Must have a DOP greater than or equal to the number of column stores

• Query coordinator automatically starts parallel server processes on the correct nodes (Requires Auto DOP in 12.1.0.2)


• Similar to storage mirroring

• Duplicate in-memory columns on another node� Enabled per table/partition

• e.g. only recent data

� Application transparent

• Downtime eliminated by using duplicate after failureOnly Available on Engineered Systems

RAC: Unique Fault Tolerance


If In-Memory Usage Shows Up In The Plan What Does The Optimizer Know


Optimizer : New In-Memory Statistics• Automatically computed on the fly during hard parse

• Computed at the segment level – table or partition/subpartition– # IMCUs– # IM Blocks– IM Quotient• Fraction of table populated in In-Memory column store• Value between 0 and 1

– # IM Rows– # IM Transaction Journal Rows

• In-Memory statistics are RAC-aware (DUPLICATE and DISTRIBUTE)

• In 12.2 IM stats are available in the ALL_TAB_STATISTICS view


Optimizer : OPTIMIZER_ADAPTIVE_FEATURES• OPTIMIZER_ADAPTIVE_FEATURES has been made obsolete in 12.2• New in 12.2:– OPTIMIZER_ADAPTIVE_PLANS – defaults to true– OPTIMIZER_ADAPTIVE_STATISTICS – defaults to false

• In 12.1.0.2:– Patches for bugs 21171382 and 22652097 provide 12.2 behavior– See MOS Note 2187449.1 – Recommendations for Adaptive Features in Oracle

Database 12c

• See the Optimizer blog for more information: blogs.oracle.com/optimizer


Optimizer : Ensure Plan Stability

• Unpredictable changes to execution plans can destroy performance

• Avoiding plan changes is the only method to avoid performance regression

Solution• Optimizer automatically manages ‘execution plans’

• Only known and verified plans are used

SQL Plan Management is controlled plan performance


How Do I Get Even More Performance


• Analytic queries have complex joins with no filter predicates specified • Join Group specifies columns used to

join tables– Column share compression dictionary– Joins occur on dictionary values rather

than data

• Enables 2-3x faster join processing

High Performance : Faster In-Memory Joins

Sales

VEHI

CLE

NAM

E

Example: Find sales price of each VehicleSalesVehicle

NAMEis join

columnNAM

E

CREATE INMEMORY JOIN GROUP V_name_jg(VEHICLES(NAME),SALES(NAME));


• Analytic queries contain complex expressions– Originally evaluated for every row

• Expressions pre-computed and cached in-memory–User defined via virtual columns –Or expressions automatic detected

• All In-Memory optimizations apply

• 3-5x faster complex queries

High Performance : In-Memory Expressions

Net = Price + Price * Tax

In-Memory Column Store

Sales

Example: Compute total sales priceTa

x

Pric

e Pric

e

Pric

e +

Pric

e X

Tax

Pric

e +

Pric

e *

Tax


Extend the IM Column Store


• Real-time analytics with no impact on production database• Make productive use of standby

database resources• Can populate different data from

production database– Use new DISTRIBUTE BY SERVICE to

determine where to populate a table – Increase total columnar capacity

• Exadata and PaaS Cloud Services only

1 MonthIn-Memory

More Capacity : In-Memory on Active Data Guard

Production Standby

1 Year In-Memory


More Capacity: IMC Format in Columnar Flash Cache

• In-Memory format now used in Smart Columnar Flash Cache– Enables in-memory optimizations on data in Exadata flash– E.g. multiple column values evaluated in single vector

instruction

• In-memory performance seamlessly extended from DB node DRAM memory to 10x larger flash in storage– Huge advantage over all-flash arrays and other in-memory DBs

In-Memory Columnar scans

In-FlashColumnar scans


Troubleshooting


Troubleshooting

• Confirm all data is fully populated!• v$inmemory_area – is there enough memory?• v$im_segments – are all objects fully populated?• v$im_header – are all rows populated per object?• v$im_smu_head – are changes affecting journal space?• v$imeu_header – are IMEs populated?• user_joingroups – have join groups been created and populated?

Additional Views


Performance History

• AWR is valuable to measure system level workload during testing• AWR is valuable to see how the database is setup– Initialization parameters–Memory allocation– Resource utilization– And don't forget the Advisors section

• Consider increasing the default AWR retention (7 days)• But... AWR is useless for fixing problem SQL

Make sure AWR is available


Troubleshooting

• SQL Monitor active reports– Shows how SQL was executed and where time was spent– See blogs.oracle.com/In-Memory for a technical brief on creating SQL

Monitor active reports

• Optimizer trace (10053)– Shows why the execution plan was chosen– See blogs.oracle.com/In-Memory for a technical brief on creating an

Optimizer trace

Additional Information


Where can I get more information


Join the Conversation

https://twitter.com/TheInMemoryGuy

https://blogs.oracle.com/In-Memory/

White Papers (otn.com)• Oracle Database In-Memory White Paper

• Oracle Database Implementation and Usage White

Paper

• Oracle Database In-Memory Aggregation Paper

•When to use Oracle Database In-Memory

• Oracle Database In-Memory Advisor

Videos• Oracle Database In-Memory YouTube Channel

• oracle.com

Powering the Real-Time Enterprise

oracle.com/us/corporate/events/dbim/index.html

• YouTube - Juan Loaiza: DBIM: What's new in 12.2

Additional Questions • In-Memory blog: blogs.oracle.com/In-Memory

•My email: [email protected]://www.facebook.com/OracleDatabase

http://www.oracle.com/goto/dbim.html

Additional Resources

https://twitter.com/db_inmemory

Oracle Database In-Memory - Getting Started and … Database In-Memory Real-Time Analytics Trivial...

Documents

Transcript of Oracle Database In-Memory - Getting Started and … Database In-Memory Real-Time Analytics Trivial...