Oracle business analytics best practices

www.nitaipartners.com; Confidential, All rights reserved; Copyright @2013; Nitai Partners Inc.

Oracle Business Analytics Best Practices

NPI Technical Services


Hardware Recommendations by Volume Ranges

4/29/2014 2

* Consider implementing Oracle RAC with multiple nodes to accommodate large numbers of concurrent users accessing webreports and dashboards.


Storage Considerations

The storage could easily become a major bottleneck as the result of such actions as:

• Setting excessive parallel query processes

• Running multiple I/O intensive applications, such as databases, on a shared storage

• Choosing sub-optimal storage for running BI Applications tiers.

4/29/2014 3


Deployment Considerations for the ETL Components

• The Informatica server and DAC server should be installed on a dedicated machine for best performance.

• The Informatica server and DAC server cannot be installed separately on different servers.

• The Informatica client and DAC client can be located on an ETL Administration client machine, or a Windows server, running Informatica and DAC servers.

• Informatica and DAC repositories can be deployed as separate schemas in the same database, as Oracle Business Analytics Warehouse, if the target database platform is Oracle, IBM DB2 or Microsoft SQL Server.

• The Informatica server and DAC server host machine should be physically located near the source data machine to improve network performance.

• You can consider deploying Informatica Load Balancing option, if you observe bottlenecks in processing Informatica mappings on the ETL tier.

4/29/2014 4


Critical factors in ETL optimization and performance

Allocate Sufficient TEMP space OLTP Data Sources:

Oracle BI Analytic Applications Extract mappings may operate with large data volumes, compared to the small changes from OLTP transactional activities. As the result, OLTP Data Sources could run out TEMP space during heavy volume initial extracts.

The source TEMP space varies by OLTP size and processed volumes. So, the recommended TEMP space for BI Applications ETL ranges from 100Gb to 1Tb.

You should allocate sufficient storage for additional TEMP space in an OLTP environment. It is more practical to reclaim unused TEMP space after large volume ETL extracts completion, than restart long running mappings from the very beginning, because of TEMP space shortage during the ETL.

4/29/2014 5


DB Configuration Parameters

• Oracle 10g customers should use Oracle 10.2.0.5 or higher.

• Oracle 11g customers should use Oracle 11.1.0.7 or higher.

4/29/2014 6


Oracle RDBMS 64-bit Recommendation

• Oracle strongly recommends deploying Oracle Business Analytics Warehouse on Oracle RDBMS 64-bit, running under 64-bit Operating System (OS).

• If 64-bit OS is not available, then consider implementing Very Large Memory (VLM) on Unix / Linux and Address Windowing Extensions (AWE) for Windows 32 bit Platforms.

• VLM/AWE implementations would increase database address space to allow for more database buffers or a larger indirect data buffer window.

• Refer to Oracle Metalink for VLM / AWE implementation for your platform.

Note: You cannot use sga_target or db_cache_size parameters if you enable VLM / AWE by setting 'use_indirect_data_buffers = true'. You would have to manually resize all SGA memory components and use db_block_buffers instead of db_cache_size to specify your data cache.

4/29/2014 7


Oracle Business Analytics Warehouse Tablespaces

• Important! You should use Locally Managed tablespaces with AUTOALLOCATE clause. DO NOT use UNIFORM extents size, as it may cause excessive space consumption and result in queries slower performance.

• Use standard (primary) block size for your warehouse tablespaces. DO NOT build your warehouse on nonstandard block tablespaces.

• Note that the INDEX Tablespace may increase if you enable more query indexes in your data warehouse.

• During incremental loads, by default DAC drops and rebuilds indexes, so you should separate all indexes in a dedicated tablespace and, if you have multiple RAID / IO Controllers, move the INDEX tablespace to a separate controller.

• You may also consider isolating staging tables (_FS) and target fact tables (_F) on different controllers. Such configuration would help to speed up Target Load (SIL) mappings for fact tables by balancing I/O load on multiple RAID controllers.

4/29/2014 8


Informatica Configuration for Better Performance

Oracle Business Intelligence Applications customers are strongly encouraged to use Informatica 64-bit version for Medium and Large environments.

4/29/2014 9


Informatica LookupsReview the guidelines below for handling Informatica Lookups in Oracle Business Intelligence Applications mappings:

• Inspect Informatica session logs for the number of lookups, including each lookup’s percentage runtime.

• Check “Lookup table row count” and “Lookup cache row count” numbers for each Lookup Transformation. If Lookup table row count is too high, Informatica will cache a smaller subset in its Lookup Cache. Such lookup could cause significant performance overhead on ETL tier.

• If functional logic permits, consider reducing a large lookup row count by adding more constraining predicates to the lookup query WHERE clause.

• If a Reader Source Qualifier query is not a bottleneck in a slow mapping, and the mapping is overloaded with lookups, consider pushing lookups with row counts less than two million into the Reader SQL as OUTER JOINS.

• Important! Some lookups can be reusable within a mapping or across multiple mappings, so they cannot be constrained or pushed down into Reader queries. Consult Oracle Development prior to re-writing Oracle Business Intelligence Applications mappings.

• If you identify a very large lookup with row count more than 15-20 million, consider pushing it down as an OUTER JOIN into the mapping’s Reader Query. Such update would slow down the Reader SQL execution, but it might improve overall mapping’s performance.

• You should test the changes to avoid functional regressions before implementing optimizations in your production environment.

4/29/201410


Disabling Lookup Cache for very large lookups

• Disabling the lookup cache for heavy lookups will help to avoid excessive paging on the ETL tier.

• When the lookup cache is disabled, the Integration Service issues a select statement against the lookup source database to retrieve lookup values for each row from the Reader Thread. It would not store any data in its flat files on ETL tier. The issued lookup query uses bind variables, so it is parsed only once in the lookup source database.

Disabling lookup cache may work faster for very large lookups under following conditions:

• Lookup query must use index access path, otherwise data retrieval would be very expensive on the source lookup database tier. Remember that Informatica would fire the lookup query for every record from its Reader thread.

• Consider creating an index for all columns, which are used in the lookup query. Then Oracle Optimizer would choose INDEX FAST FULL SCAN to retrieve the lookup values from index blocks rather than scanning the whole table.

• Check the explain plan for the lookup query to ensure index access path.

• If you identify bottlenecks with lookups having very large rowcounts, you can consider constraining them by updating the Lookup queries and joining to a staging table used in the mapping. As a result, Informatica will execute the lookup query and cache much fewer rows, and speed up the rows processing on its Transformation thread.

4/29/201411


Informatica Custom Relational Connections for Long Running Mappings

• If you plan to summarize very large volumes of data (usually over 100 million records), you can speed up the large data ETL mappings by turning off automated PGA structures allocation and set sort_area_size and hash_area_size to higher values.

• If you have limited system memory, you can increase only the sort_area_size as sorting operations for aggregate mappings are more memory intensive.

• Hash joins involving bigger tables can still perform better with smaller hash_area_size.

• You will need to create and use custom relational connections in DAC and Informatica.

4/29/201412


Informatica Session Parameters

• Commit Interval: 10,000 up to 200,000

• DTM Buffer Size: Increase as per performance

• Additional Concurrent Pipelines: Oracle BI Applications Informatica mappings have the default setting 0. You can reduce lookup cache build time by enabling parallel lookup cache creation by setting the value larger than one.

• Default Buffer Block Size: Avoid using ‘Auto’ value for Default Buffer Block Size, as it may cause performance regressions for your sessions. The internal tests showed better performance for both Initial and Incremental ETL with Default Buffer Block Size set to 512,000 (512K).

4/29/201413


SRSP for Windows

4/29/201414


Informatica Bulk Load: Table Fragmentation

4/29/201415

Important! To ensure bulk load performance and avoid or minimize target table fragmentation, use larger commit size in Informatica mappings.

Oracle business analytics best practices

Data & Analytics

Transcript of Oracle business analytics best practices