Oracle Business Intelligence 11g Tuning TimesTen for ... timesten for aggregate...• OBIEE...

Oracle Business Intelligence 11g

Tuning TimesTen for Aggregate Persistence

Alastair Burgess

May 2014

© Peak Indicators Limited 2

Agenda Tuning TimesTen for Aggregate Persistence

• Overview

• Building the Aggregates

• Test Results #1

• Tuning TimesTen

• Test Results #2

• Conclusion



Overview


Overview Aim of Presentation

• Since the release of TimesTen for Exalytics, many rounds of enhancements have been made to:

• Improve the performance of TimesTen “analytical” queries

• Enhance Oracle BI 11g to generate efficient TimesTen structures and queries

• The aim of this presentation is to:

• Demonstrate the query performance of TimesTen 11.2.2.6.3 on an Exalytics server

• Provide tuning tips that deliver significant gains in TimesTen performance

• Compare performance of TimesTen with an Oracle Database


Overview

Architecture


Overview

Building the Aggregates – Simple !


Overview

Implement Aggregates

nqcmd -d AnalyticsWeb -u weblogic -p welcome1 -s TTagg.sql



Test Results #1


Test Results 1 Demonstration

• We will now demonstrate how the TimesTen aggregates perform using the default settings

• Each test will compare against an Oracle Database running on the same Exalytics server (with Auto DOP – Typically 2 Px threads)

• A fact table with 24M records is used for the tests, each test has slightly different filters so that an increasing number of records are scanned during each test

• Do you think TimesTen will be faster?



Tuning TimesTen


Tuning TimesTen 1) RangeIndexType=0

• TimesTen 11.2.2.5 provides support for a new type of "in-memory B-Tree" style of index, it allows for much greater concurrency (as it does not require latches for reads etc).

• You can activate this new feature by setting the

• following parameter in the DSN attributes

[TT_HOME/info/sys.odbc.ini] :

• RangeIndexType = 0

• IMPORTANT: This parameter takes effect only when you connect to the database for the very first time - so you must apply the parameter when you first create the database (and before anyone connects to the database)

[TT_AGGR_3] Driver=..../lib/libtten.so DataStore=..../tt_aggr_3 LogDir=..../logs ConnectionCharacterSet=AL32UTF8 LogFileSize=1024 LogBufMB=1024 LogBufParallelism=16 Preallocate=0 PermSize=25000 TempSize=25000 MemoryLock=4 CkptFrequency=30 CkptLogVolume=0 CkptRate=20 PrivateCommands=1 RecoveryThreads=40 DatabaseCharacterSet=AL32UTF8 RangeIndexType=0


Tuning TimesTen 2) Use Most Appropriate Data Types

• TimeTen provides specific datatypes that can reduce data volumes when compared to Oracle DB equivalent

• Up to 300% reduction in the ‘Lab’

• Customers have seen reductions from 80GB to 30GB

• OBIEE 11.1.1.7 generated script is better at picking an efficient DT, however improvements can be made, adjust the CT script accordingly :

CREATE TABLE ag_Fact_Sales ( Customer_I00008CBF VARCHAR2(50 CHAR) NOT INLINE , Product_Id00008CC6 VARCHAR2(50 CHAR) NOT INLINE , Month00008CCD VARCHAR2(20 CHAR) NOT INLINE , Z_Orders00008CD1 BINARY_DOUBLE, TT_INTEGER Sales_Qty00008CD2 BINARY_DOUBLE, TT_SMALLINT Sales_Valu00008CD3 BINARY_DOUBLE NUMBER(15,2) )


Tuning TimesTen 3) RAM Policy

• By default TT may not always load your data into memory.

• For BI we are looking to utilize in-memory for performance

• We can force this with use of the ram policy configuration* : [Manual]

• $ ttadmin -ramload AGGREGATES

• We have found that “Always” instead of “Manual” also gives the desired effect

• $ ttadmin -rampolicy Always AGGREGATES

* requires a second command to be run following DB startup


Tuning TimesTen 4) Compression

• TimesTen compression operates as data-duplication

• No point on primary key columns,

• No point on numbers , (Fact Columns)

• OBIEE RPD Requires COMPRESS_COLUMNS feature enabled (on by default):


Tuning TimesTen 4) Compression

• Compression Pointer size defaults to 4 bytes

• This supports many distinct values!

• Where it is known, a smaller pointer size can be more efficient

• 1 byte <= 255 (distinct values)

• 2 Bytes <= 65,535,

• 4 Bytes <= 4,294,967,295

CREATE TABLE ag_Fact_Sales ( Customer_I00008CBF VARCHAR2(50 CHAR) NOT INLINE ,

Product_Id00008CC6 VARCHAR2(50 CHAR) NOT INLINE ,

Month00008CCD VARCHAR2(20 CHAR) NOT INLINE ,

Z_Orders00008CD1 TT_INTEGER,

Sales_Qty00008CD2 TT_SMALLINT, Sales_Valu00008CD3 NUMBER(15,2))

COMPRESS ( Customer_I00008CBF BY DICTIONARY maxvalues = 255, Product_Id00008CC6 BY DICTIONARY maxvalues = 255, Month00008CCD BY DICTIONARY maxvalues = 255 )

OPTIMIZED FOR READ;

OBIEE Automatically selects appropriate columns!


Tuning TimesTen 5) Index Advisor

• Timesten contains an easy to use index advisor

• From ttisql : call ttindexadvicecapturestart(0,31);

• Now provide the SQL to the advisor (obtained from BI Session logs) • WITH

SAWITH0 AS (select sum(T580.Sales_Valu00000060) as c1, sum(T580.Sales_Qty0000005F) as c2, count(T580.Z_Orders00000059) as c3, T550.Quarter0000004F as c4, T550.Year00000051 as c5 from ...

• Follow up with :

• all ttindexadvicecaptureend(0);

• call ttindexadvicecaptureoutput(0);

0 : just the current session 1 : entire database

Optional : advise only, no execution



• Output • < 4, create hash index AG_FACT_SALES_i1 on SALES_AGG.AG_FACT_SALES(MONTH00008CCD); >

• < 1, create hash index AG_FACT_SALES_i4 on SALES_AGG.AG_FACT_SALES(CUSTOMER_I00008CBF,MONTH00008CCD); >

• < 3, create hash index SA_DETAIL00008D07_i2 on SALES_AGG.SA_DETAIL00008D07(CUSTOMER_I00008CBF); >

• < 3, create hash index SA_DETAIL00008D01_i3 on SALES_AGG.SA_DETAIL00008D01(PRODUCT_ID00008CC6); >



• OBIEE (Agg Persistence) will only create RANGE indexes.

• Can be up to 40% slower for equality joins (=)

• TT Index advisor will advise HASH indexes

• It is important to review the DDL from OBIEE and switch to HASH indexes for any columns used in equality joins

• e.g Primary Key (Dimensions) and Foreign Key (Fact tables)

• Simply alter the syntax with additional keyword

• CREATE HASH INDEX



• Note – Be careful creating HASH indexes on empty tables ! • By default TT will size the index based on the current size of the table

• An empty table will therefore give a non-optimal index size

• Best to use the PAGES parameter as follows :

• CREATE HASH INDEX [name] ON [table] ([columns]) PAGES=[MAXROWS/256]

• Demonstration


Tuning TimesTen 6) Analyze Tables

• OBIEE by default just estimates statistics to 5%

• Use this command to gather stats on all tables

• call ttOptUpdateStats('', 1);

‘’ : All tables for current owner Force cached query plans to INVALID


Tuning TimesTen

7) Execute on Connect

• Oracle BI now automatically runs 4 statements to alter query optimisation settings to improve analytical performance:

• Those session hints are standard / mandatory on Exalytics. They are necessary in order for the special TimesTen optimisation options for star-joins to kick-in

• Fast Prepare (?)



Test Results #2


Test Results 2 Demonstration

• Now that we have applied at least 5 different optimisations to the aggregates in TimesTen, let’s see how it now performs compared to the Oracle Database

• For each query, which one do you think will be the fastest???


Test Results 3 Database Parallelism

• Up until now, the Oracle Database has had Parallel Query enabled with 2 parallel threads

• What happens if we increase this to 4 or 8 parallel threads?

• Will it make much difference???



Conclusion


Conclusion Interesting Results?

• Query performance with TimesTen (11.2.2.6.3 ) has greatly improved !

• OBIEE Aggregate Persistence / Summary Advisor has also improved , making better decisions on when to use compression and the choice of TT data types

• Applying 5-6 simple tuning techniques will further significantly improve response times (as shown in graph on next slide)

• Our tests show that TimesTen can compete with and even beat the Oracle Database with Parallel Query enabled (with low DOP)

• Remember Oracle Times Ten is 1/3 licence cost of Oracle EE ($350 v $950)

Helping Your Business Intelligence Journey

Oracle Business Intelligence 11g Tuning TimesTen for ... timesten for aggregate...• OBIEE...

Documents

Transcript of Oracle Business Intelligence 11g Tuning TimesTen for ... timesten for aggregate...• OBIEE...