Oracle Business Intelligence Application 7.9.6.3 Customization Types, Tips and Best Practice Guidelines – Part A I would like to share some of the Oracle Business Intelligence Application (OBIA) customization tips and best practices guidelines from my knowledge base. This document is based on my experiences of BI Apps implementation over the period of last 10/11 years since Siebel days and extract from Oracle’s standard documentation around BI Apps.

OBIA Architecture

OBIA Architecture has following components i.e. ETL Architecture, Data Model, Metadata (Repository), Dashboard/Webcat along with security.

OBIA Pre-Built Contents

Oracle Business Intelligence Applications 7.9.6.3 are pre-built solutions from source to dashboard. The pre-built solution contents are:

Pre-built Analytics Data Warehouse Model

The Business Analytics Data Warehouse is the persistent layer where information is stored for retrieval when required. The data warehouse resides in a relational database platform like Oracle, IBM DB2 LUW, Teradata or Microsoft SQL Server. It consists of transaction grain tables used to capture facts from the key business processes across the organization. The

model consists of current snapshots, periodic snapshots and aggregate tables to support complex analysis requirements along with the hundreds of dimension tables conformed across the fact tables which contribute to enrich the analyses.

Pre-built Data Integration Components (ETL/E-LT)

Data Integration components ( data extraction routines) are used to integrate data from various operational systems such as E-Business Suite, PeopleSoft Enterprise, Oracle's Siebel CRM Applications, Oracle JD Edwards Applications, and third party legacy systems, and load the information in the Business Analytics Data Warehouse.

The data integration components are built using either the Informatica Power Center data integration platform (ETL) or Oracle Data Integrator (E-LT). It uses a layered approach based on data warehouse best practices, which is open and flexible which provides ease of configuration and customization to enable changing the out of the box transformation rules, adding additional attributes, facts and dimensions.

Pre-built Metadata (BI Repository)

The metadata layer includes the definition of role based views, dimensions, metrics, hierarchies, calculations, aggregation rules, canonical dime dimensions, time series metrics, roles and security. The metadata layer is built on the Oracle Business Intelligence Enterprise Edition platform using the Admin Tool, a core component of the BI server, which is designed to meet the requirements of the Common Information Model to deliver consistent and integrated information to business users, by capturing the data from multiple source systems. Oracle Business Intelligence Applications provides ready to use pre-built metadata covering dimensions, metrics and hierarchies. In addition, the pre-built metadata includes the capability to leverage the security profiles inherited from operational systems such as Oracle EBS, PeopleSoft and JD Edwards.

Pre-built OBI-EE dashboard and report content (Catalog)

The OBIEE Presentation Catalog contains pre-built metrics, analytic decision workflows, reports, dashboards and alerts used to deliver information, stored in the Business Analytics Data Warehouse, to information consumers. These services are built on the Oracle Business Intelligence Enterprise Edition delivery platform, which is designed to meet the requirements of an enterprise business intelligence solution. Pre-built reports and dashboards are built on industry best practices, and provide rich content covering functional areas including sales, service, marketing, finance, supply chain, order management, procurement and human resources.

Customization Best Practice – Keep it to Minimum

In line with other Oracle applications products in general, customization should be kept to minimum and wherever possible out-of-the-box (OOTB) functionality should be employed. This should be done in such manner that future upgrades to the product will not require significant amount of rework, i.e. where there is a good match between the requirements and the out-of-the-box functionality. However, in reality it is often unavoidable.

Customization Reality – It is Unavoidable

It is often seen that certain key business requirements are not covered by the pre-built contents (Out of Box contents), For example the OOTB Data Warehouse may not include certain attributes customer would like to report on from the source system, or more likely, require an external secondary data store (file, RDBMS, XML, and OLAP) to include additional reporting matrices.

For a rapid development we must re-use as much of the OOTB data model as possible to avoid “re-inventing the wheel” by developing data objects that have already been populated by BI-Apps. There are number of situations where there is a need to modify the pre-built contents.

In these cases it is important to follow certain guidelines and techniques when developing non-standard BI elements. This blog is an attempt to detail guidelines on how to develop custom objects to integrate into BI Apps. The idea is to follow the same design techniques and guidelines that BI Apps uses to build OOTB contents for customization. This approach would be helpful to maintain the oracle’s high standard development practices and keep “look and feel” like standard BI-Apps content.

Types of Customization and Impacts

There are 6 types of customizations that will be implemented. These customizations can be as simple as exposing already existing attributes in the DW to the Presentation Layer of OBI-EE. They can also be as complex as creating new dimension, facts and external data sources. The new report creation is not in scope of the this discussion.

The following are the main extensions covered in this blog.

TYPE A- Simple Extensions and Adding Simple/Complex Logical Definition

The Data Warehouse has a vast amount of attributes within its data model. However, not all this data is exposed to the repository and dashboards. One of the simplest customizations is to expose the attribute top the Presentation Layer of OBI-EE.

Creating new logical definitions (simple-moderate-complex) based on physical/logical columns already exposed should also be included into the Type A category. Time Series Logical columns are an example of this type of customization.

Impact:

Changes only in the OBI-EE repository in the Physical, Logical and Presentation Layer are needed.

TYPE B- Attribute Extensions

Adding attributes to existing dimensions and measures to existing facts. The complexity of this customization depends on the SQL required to populate the attribute. If the attribute is part of an existing table in an existing query then the inclusion is very simple.

Impact:

Additional column(s) to the physical staging and target tables. Changing existing SDE source mapping to add the additional column(s) to the SQL query

and the target table. Changing existing SIL target mapping to add the additional column(s) to the SQL extraction

query to the target table. Changes required to the OBI-EE repository to add the column to the Physical and modeled

through to the Business and Presentation Layers.

TYPE C- Additional Fact, Parent Child Hierarchies Table, Bridge Table

Adding new Facts, Parent Child Hierarchies (value hierarchies) or Bridge tables (a technique for modeling Many-to-Many relationships) will require new SDE mappings for extraction to the DW staging area and new SIL loading routines from the staging area to the DW

Impact:

Additional physical tables created in the staging and DW areas. Add new SDE source mapping to populate the target staging table Add new SIL target mapping to populate the DW target table Import the new data objects to the OBI-EE repository and model the physical through to the

Business and Presentation Layers. Optionally applying security filtering. Adding new SDE and SIL to the DAC dependency

TYPE D- Localization of Oracle Business Intelligence (Repository Metadata, Oracle BI Data or Report and Dashboard)

Localization of Oracle Business Intelligence requires three configurations i.e. localization of Oracle BI Repository Metadata, localization of Oracle BI Data and localization of Reports and Dashboards. These configurations require configuration file changes, creating metadata objects to support it which includes initialization block, externalize metadata objects etc , DW tables and routine to populate the language dependent data.

Impact:

Additional translation tables/ Modification to existing columns in staging and DW areas. Add/Modify SDE source mapping to populate the target staging table. Add/Modify SIL target mapping to populate the DW target table. Session Variables and Configuration Settings and creation of supporting metadata objects Import the data objects to the OBI-EE repository and model the physical through to the

Business and Presentation Layers. Adding new SDE and SIL to the DAC dependency if required

TYPE E- Fact Granularity Change

Add new Dimensions to an existing Fact requires changes to existing SDE and SIL mappings for the Fact table. In addition, SDE and SIL mappings are required to implement new dimensions.

Impact:

Additional physical dimension table created in the staging & DW areas. Additional key fields to staging and DW fact tables. Add new SDE source mapping to populate the new target staging table. Add new SIL target mapping to populate the DW target table. Changing existing SDE source mapping to add the additional keys staging fact table. Changing existing SIL target mapping to add the additional key column(s) to the DW target

fact table. Also, include the key lookup to the new dimension. Create SDE & SIL workflows. Create DAC tasks with required dependencies. Import the new data objects to the OBI-EE repository and model the physical through to the

Business and Presentation Layers.

Changes required to the OBI-EE repository to model new Physical model and modeled through to the Business and Presentation Layers.

Optionally applying security filtering.

TYPE F- New Star Schema

New facts and some new dimensions integrated with existing dimensions.

Impact:

Additional physical dimension and fact table(s) created in the staging & DW areas. For each new physical dimension table and fact table a new SDE source mapping to

populate the new target staging tables. For each new physical dimension table and fact table a new SIL target mapping to populate

the DW target table. Create SDE & SIL workflows Create DAC tasks with required dependencies. Import the new data objects to the OBI-EE repository and model the physical through to the

Business and Presentation Layers. Optionally applying security filtering.

TYPE G- Additional Source Systems or Using Universal Adaptors

Adding a second home-grown or non-supported data source. OOTB BI-Apps has one supported data source. However, it is possible that you want to integrate other data sources into the Business Analytic Warehouse (BAW). Additional sources can include simple spreadsheets to interlacing data into existing data models.

Interlacing is a method of loading data into the existing data model without matching & merging into the supported data source. For example, there might be Purchase Orders from a different ERP system that you want to provide integrated reporting with eBS. In this case you will need to integrate dimensional and fact data.

Another example is to use the universal adaptors to push the data into the BI Apps. The universal adaptors implementation lists following impacts.

Impact:

Additional columns created for additional attributes in dimension and fact table(s) created in the staging & DW areas.

Modify the universal adaptor SDE mapping & SIL mappings to include additional attributes if any

Create SDE workflow Create DAC tasks with required dependencies. Import the new columns data objects to the OBI-EE repository and model the physical

through to the Business and Presentation Layers.

In the next blog of this series I would like to cover the naming conventions and development guidelines that should be followed when developing new attributes, tables, dimensions and facts in the Business Analytics Warehouse (BAW) and new/changed repository objects in the OBIEE repository.

Read 2nd Article on topic :

http://bidirect.blogspot.co.uk/2013/01/oracle-bi-apps-7963-tips-and-best.html

Oracle BI Apps 7.9.6.3 Tips and Best Practice Guidelines for Customization ( Database Objects)– Part B In previous blog I have tried to explain the OBIA customization types and its impacts on various components of the solution.

http://bidirect.blogspot.co.uk/2012/05/oracle-business-intelligence.html

In successive blogs of this series I would like to cover the naming conventions and development guidelines that should be followed when developing new attributes, tables, dimensions and facts in the Business Analytics Warehouse (BAW) and new/changed repository objects in the OBIEE repository.

In this blog my focus would be more on database objects modification standard practices and guidelines such as tables, columns and indexes.

Why Best Practices? – The standard and guidelines for database schema, tables, columns and indexes is for better manageability and simplify the future upgrades. In long term it would help to reduce the application day to day cost and successive upgrades.

Database Schema and Objects

The Business Analytics Warehouse (BAW) is the pre-built Data Warehouse as part of BI-Apps. The database schema will be decided by the installation team. A default is “BAW”. It contains both the staging tables and DW target tables. All customized tables, columns, indexes and other database objects MUST be added to BAW schema. (Best Practice DB 1)

Why to add in BAW Schema? – For better manageability and avoid any cross schema joins at OBIEE or Informatica Level as I understand cross schema joins within repository and Informatica would more complicated than keeping it simple. The mantra is keep it simple.

The database objects changes which is been done within the database schema and objects should use best practices for creating and managing versions and comments option of underline databases. The database objects customization should use versions and comments features of database for better traceability. (Best Practice DB 2)

Why Versions and Comments? Putting comments on tables and columns based on phase of projects, user created and date would be useful and provide enough traceability of customization done.

In addition make sure that what so ever customization done to database objects should be imported into the DAC container. The database objects customization should be imported into the DAC container. (Best Practice DB 3)

Why Import? DAC should be your central repository which would be used to migrate the database objects customization from development environment to test and then production. We should avoid using word/text file as a storage option for database objects.

Tables

Naming of Tables

The naming conventions used for tables in the Oracle BI Data Warehouse are as follows. The naming conventions for tables defined in BI Data Warehouse should be followed if custom tables are created in the warehouse. (Best Practice DB 4)

Why Standard Naming Conentions? It would help to identify meaning of each of the table and its significance and usability in overall solution.

The names of the Oracle Data Warehouse tables consist of three parts:

Prefix

Name

Suffix

The three components are capitalized and separated by an underscore (_), e.g. W_GL_OTHER_F. Each component has a meaning, as described in the table below:

Component Value Meaning

Prefix W_ Out-of-the-box warehouse table

Prefix WC_ Custom warehouse table

Suffix _F Fact table (base)

Suffix _FS Staging Fact table (base)

Suffix _A Aggregate fact table (base)

Suffix _D Dimension table (base)

Suffix _DS Staging Dimension table (base)

Suffix _DH Dimension hierarchy table (base)

Suffix _SCD Slowly changing dimension table (base)

Suffix _MD Mini-dimension table

Suffix _H Helper table

Suffix _HS Helper Staging Table

Suffix _TMP Pre Staging Temporary table

Suffix _G, _GS, _S Internal Table

Prefix _UD Unbounded dimension

Prefix _DHLS Staging for Dimension Helper

Prefix _DHL Dimension Helper Table

Prefix _M Map Dimension

Table Types

The types of tables used by the Oracle Business Analytics Warehouse are as follows.

The developer should use following types of tables; I believe the list of the tables used has covered almost all types of tables possible for DW solution. Any new addition would be a treated as a new entrant in the family and should be welcomed. (Best Practice DB 5)

Why different types? The type of the table and its description would be helpful to understand its significance in overall solution

Table Type Description

Aggregate tables (_A) Contain summed (aggregated) data

Dimension tables (_D) Star analysis dimensions

Staging tables for Dimension (_DS) Tables used to hold dimension information that have not been through the final ETL transformations.

Staging tables for Usage Accelerator

(WS_)

Tables containing the necessary columns for the ETL transformations

Dimension Hierarchy tables (_DH) Tables that store the dimension's hierarchical structure

Dimension Helper tables (_DHL) Tables that store M:M relationships between two joining dimension tables

Staging tables for Dimension Helper

(_DHLS)

Staging tables for storing M:M relationships between

two joining dimension tables

Fact tables (_F) Contain the metrics being analyzed by

dimensions.

Fact Staging tables (_FS) Staging tables used to hold the metrics being analyzed by dimensions that have not been through the final ETL transformations

Internal tables (_G, _GS, _S) Internal tables are used primarily by ETL mappings for data transformation and controlling ETL runs

Helper tables (_H) Helper tables are inserted between the fact and dimension tables to support a many-to-many (M:M) relationship between fact and dimension records

Map dimension tables (_M) Tables that store cross-referencing mappings between the external data ID and the internal ID

Mini dimension tables (_MD) Include combinations of the most queried attributes of their parent dimensions. The database joins these small tables to the fact tables

Primary extract tables (_PE) Tables that are used to support the soft delete feature. The table includes all the primary key columns (integration ID column) from the source system. When a delete event happens, the full extract from the source compares the data previously extracted in the primary extract table to determine if a physical deletion was done in the transactional database. The soft delete feature is disabled by default. Therefore, the primary extract tables are not populated until you enable the soft delete feature

Unbounded dimension tables(_UD) Tables containing information that is not bounded in transactional database data but should be treated as bounded data in the Oracle Business Analytics Warehouse

Columns

Naming of Columns

This section describes the naming conventions used for columns in the Oracle BI Data Warehouse. The naming conventions for columns defined in BI Data Warehouse should be followed if custom columns are created in the warehouse. (Best Practice DB 6)

Why Naming Convention for Columns? It would help to identify meaning of each of the column and its significance and usability in overall table usage.

The columns of the Oracle Data Warehouse tables consist of three parts:

Prefix

Name

Suffix

The three components are capitalized and separated by an underscore (_), e.g. X_BUDGET_AMT. Each component has a meaning, as described in the table below:

Component Value Meaning

Prefix X_ Custom warehouse column

Suffix _AMT This column contains a currency amount

Suffix _CD The column value is based on the contents of the List of Values (LOV)

Suffix _DT This column contains a date or date time value.

Suffix _FLG This column contains a Boolean value where Y

indicates Yes or True; N indicates No or False

Suffix _I Language Independent Code

Suffix _ID Columns are used in staging tables, they corresponds to the source identifier

Suffix _WID Identifier generated by Oracle BI linking dimension and fact tables, except for ROW_WID

Suffix _NAME Name corresponding to the code column (columns ending with _CODE)

Suffix _DESC Long Description corresponding to the code column (columns ending with _CODE)

Suffix _NUM This column contains a number or an identifying alphanumeric value

All new tables, whether Fact or Dimension MUST contain the following important fields :( Best Practice DB 7)

Why System Fields? It would provide metadata information about the data and DML operations. Metadata of data such as source identifier, source type, created by, creation date etc is useful in many ways such as setting up changed data capture, setting up DML operations such as insert, update and up-sert.

Column Name Column Type Required in Area

Description

INTEGRATION_ID VARCHAR2(80 CHAR)

Staging / DW

Stores the primary key or unique identifier of a record in the source table.

DATASOURCE_NUM NUMBER(10) Staging / DW

Stores the data source from which the data is extracted – this is setup in DAC. E-Business Suite = 4. New external sources should

have their own unique number.

ETL_PROC_WID NUMBER(10) DW Stores the ID of the ETL process information; details of ETL process are stored in the W_ETL_RUN in the warehouse

ROW_WID NUMBER DW Is the sequence number generated during the ETL process; unique identifier (sequence number) for the tables.

CREATED_ON_DT DATE Staging / DW

Stores the creation date of a record in the primary source table

CHANGED_ON_DT

AUX1_CHANGED_ON_DT

AUX2_CHANGED_ON_DT

AUX3_CHANGED_ON_DT

AUX4_CHANGED_ON_DT

DATE Staging / DW

Stores the last update date of a record in the primary source table

Additional column prefixed by AUX# can be added to store the last update date related to auxiliary source tables

CREATED_BY_ID VARCHAR2(80) Staging Stores the user id who created a record in the primary source table

CREATED_BY_WID NUMBER(10,0) DW Surrogate key of the user id who created a record in the primary source table

CHANGED_BY_ID VARCHAR2(80) Staging Stores the user id who last updated a record in the primary source table

CHANGED_BY_WID NUMBER(10,0) DW Surrogate key of the user id who last updated a record in the primary source table

TENANT_ID VARCHAR2(80) Staging / This permits to distinguish distinct source instances

DW having a common data source num identifier

Default value is ‘DEFAULT’

Miscellaneous Technical columns for new tables

All new tables, MAY contain the following important fields:( Best Practice DB 8)

Why Flg Columns ? It would provide metadata information about the data and DML operations. Metadata of data such as source identifier, source type, created by, creation date etc is useful in many ways such as setting up changed data capture, setting up DML operations such as insert, update and upsert.

Column Name Column type

Required in Area

Description

CURRENT_FLG CHAR(1) DW For Slowly changing dimension purpose

DELETE_FLG CHAR(1) Staging / DW

For Soft/Hard delete purposes

W_INSERT_DT DATE DW optional insertion date for ETL audit purpose

W_UPDATE_DT DATE DW optional update date for ETL audit purpose

SRC_EFF_FROM_DATE DATE Staging / DW

For Type 2,3 or 6 slowly changing dimension purpose

SRC_EFF_TO_DATE DATE Staging / DW

For Type 2,3 or 6 slowly changing dimension purpose

EFFECTIVE_FROM_DATE DATE Staging / DW

For Type 2,3 or 6 slowly changing dimension purpose

EFFECTIVE_TO_DATE DATE Staging / DW

For Type 2,3 or 6 slowly changing dimension purpose

Miscellaneous Technical columns for new tables

The currency codes for related system columns are as follows:( Best Practice DB 9)

Why Currency Codes? To support reporting in multiple currencies is one of the prime features of BI Apps implementations.

System Column Description

DOC_CURR_CODE Code for the currency in which the document was created in the source system.

LOC_CURR_CODE Usually the reporting currency code for the financial company in which the document was created.

GRP_CURR_CODE The primary group reporting currency code for the group of companies or organizations in which the document was created.

LOC_EXCHANGE_RATE

Currency conversion rate from the document currency code to the

local currency code.

GLOBAL1_EXCHANGE_RATE

Currency conversion rate from the document currency code to the primary group currency code.

GLOBAL2_EXCHANGE_RATE

Currency conversion rate from the document currency code to the GLOBAL1 currency code.

GLOBAL3_EXCHANGE_RATE

Currency conversion rate from document currency code to the GLOBAL2 currency code.

Indexes

Naming of Indexes

The naming conventions used for Indexes in the Oracle BI Data Warehouse are as follows. The naming conventions for indexes defined in BI Data Warehouse should be followed if custom indexes are created in the warehouse. (Best Practice DB 10)

Why Naming Conventions for Indexes ? It would help to identify custom indexes and its significance and usability in overall solution.

The index name of the Oracle Data Warehouse tables consists of two parts

Prefix

Name

Suffix

The three components are capitalized and separated by an underscore (_), e.g. W_GL_BALANCE_F_U1, each component has a meaning as described in the table below.

Component Value Description

Prefix W_ Standard Data Warehouse Table

Prefix WC_ Custom Table

Name XXX Table Name (e.g. W_GL_BALANCE_F)

Suffix _U1 Unique Index ( e.g. composite index on INTEGRATION_ID and DATASOURCE_NUM_ID

Suffix _F1 Non Unique Dimensional Reference Key Index ( e.g. bitmap index such as BALANCE_DT_WID )

Suffix _M1 Non Unique Index ( e.g. bitmap or normal index or composite normal index )

Best Practices for Indexes

The best practices for the indexes are as below.

The custom indexes in Source System ( e.g. Oracle EBS) for incremental Loads performance can be categorized into following groups and action need to taken based on facts

o Tables that do not have indexes on LAST_UPDATE_DATE in the latest EBS releases, but there are no performance implications reported with indexes on LAST_UPDATE_DATE column.

o Tables that have indexes on LAST_UPDATE_DATE columns, introduced in Oracle EBS Release 12.

o Tables that cannot have indexes on LAST_UPDATE_DATE because of serious performance degradations in the source EBS environments.

The indexes on source system should have prefix OBIEE_ and DO NOT follow to the standard source system index naming conventions (e.g. OBIEE_OE_ORDER_HEADERS_ALL)(Best Practice DB 11)

Why? This is to make sure that these custom indexes are managed separately from the rest of the standard indexes.

Staging tables typically do not require indices, not a strict rule

Don’t hesitate to include indexes on staging table if necessary(Best Practice DB12).

Why? Let common sense prevails, I hate strict rules; if you prove the index would help you in data load performance don’t worry go ahead and just do it.

Create indices on all columns that the ETL uses for dimensions and facts. For example:

o ROW_WIDs of dimensions and facts

o INTEGRATION_ID

o DATASOURCE_NUM_ID

o Flags

o Dimensional Reference Keys of fact tables

Create indices on all system columns such as ROW_WID, INTEGRATION_ID etc used within ETL processes.This is absolutely MUST for ETL performance. (Best Practice DB13).

Why? ETL Performance, without these indices I can tell you that your ETL won’t run efficiently.

Create Bitmap Indexes for Dimensional reference keys and flags, treat this a MUST requirement. (Best Practice DB14).

Why? With Bitmap Indexes for Dimensional reference keys and flags allows us to use STAR TRANSFORMATION.I believe it is absolutely essential to achieve best query performances.

Make sure that the database parameter for STAR TRANSFORMATION & Repository Database Parameters setting has been in place.

Carefully consider on which columns to put filter conditions (Best Practice DB15).

Why? Setting up index on column which we used to put condition in report would improve performance for obvious reason.

Define indices to improve query performance(Best Practice DB16)

o Inspect standard repository objects for guidance

Why? The index creation processes should be driven by business requirements rather than physical structures and conventional wisdom. I believe the reporting system is a mainly a business support system and should be driven by business requirements.

Register new indices in the DAC, treat this as a MUST requirement ( Best Practice DB17)

Options within DAC are

o Local or Partitioned Index

o Query Index or ETL Index

o The DAC server drops and re-creates indices during a full load and /or incremental load

Why? As I said earlier DAC should be your central repository for all database objects which includes indexes. This is helpful during migration of code from development instance to test and finally to production. DAC would be useful to maintain drop and re-recreate indices during full load and/or incremental load.

Review/Revisit Index Strategy on regular basis ( Best Practice DB18)

Why? Review and Revisit the Indices should be done on regular basis, particularly with every new release which might end up with different usage pattern and query performances.

The best practices for handling BI Applications indexes in Exadata Warehouse (Best Practice DB19):

Why? Exadata, Exalytics are specially designed engineering system and requires different approach. Based on my experience in this area the success mantra is don’t rush to any conclusion, make sure that proper comprehensive benchmarking is done before dropping or disabling any indices.

o Turn on Index usage monitoring to identify any unused indexes and drop / disable them in your environment.

o Consider pinning the critical target tables in smart flash cache

o Consider building custom aggregates to pre-aggregate more data and simplify queries performance.

o Drop selected query indexes and disable them in DAC to use Exadata Storage Indexes / Full Table Scans only after running comprehensive benchmarks and ensuring no impact on any other queries performance.

I hope you find the second part of this series informative and useful. My plan is to write next couple of blogs on guideline on Informatica, DAC and Repository customisation.

Oracle BI Apps 7.9.6.3 Tips and Best Practice Guidelines for Customization (Informatica) – Part C In previous blog I have tried to explain the OBIA customization types and its impacts on various components of the solution, I have also covered database objects database objects modification standard practices and guidelines such as tables, columns and indexes in my second blog of the series.

You can find out the previous blogs at: link to blogs

Oracle Business Intelligence Application 7.9.6.3 Customization Types, Tips and Best Practice Guidelines – Part A

http://bidirect.blogspot.co.uk/2012/05/oracle-business-intelligence.html

Oracle BI Apps 7.9.6.3 Tips and Best Practice Guidelines for Customization ( Database Objects)– Part B

http://bidirect.blogspot.co.uk/2013/01/oracle-bi-apps-7963-tips-and-best.html

In this blog my focus would be more on Informatica modification standard practices and guidelines such as mappings, map-lets and sessions. I would include ODI may be in future blog, the focus area of this blog is Informatica only.

Why Standards and Guidelines? –The standard and guidelines for Informatica folders, mappings, mapplets and sessions is for better manageability and

simplify the future upgrades. In long term it would help to reduce the application day to day cost and successive upgrades.

Informatica Folder Structures

The Informatica repository is designed into mapping folders. There is a different folder for each source adapter which is prefixed with ‘SDE’. These represent the Source Dependant Extracts from the ERP system of your source system to the Business Analytics Warehouse staging tables.

There are 2 folders, ‘SIL_Vert’ and ‘SILOS’, that represent the Source Independent Loads ETL mappings from the staging to the Data Warehouse schema.

There is another folder ‘PLP’, which represents the Post Load Process ETL mappings which do aggregations, snapshots, cross subject area relationship updates and update flags etc after completion of Source Independent Loads ETL mappings.

OOTB folders related to E-Business Suite source

The following folders are used for the typical E-Business implementation project:

SDE_ORAR11510_Adaptor - Source Dependant Extracts from the E-Business Source System 11.5.1.0

SILOS - Source Independent Loads from staging to Data Warehouse schema.

PLP - Post Load Process for aggregations, snapshots etc

No change should be allowed to be done in the OOTB Folders. Treat this as a MUST requirement, any change should be carefully recorded (Best Practice ETL1)

Why? To maintain OOTB intact is very important which would help with better ETL content manageability and future upgrades. Folder may be overwritten in future upgrades. Careful recording of customization is useful for re-application if required.

Custom folders related to E-Business Suite source

So the best option is to create custom folders. The following custom folders are to be created for the typical E-Business implementation project:

Custom_SDE_ORAR11510_Adaptor - Source Dependant Extracts from the E-Business Source System 11.5.1.0 for custom process (modified from the existing one or newly designed)

Custom_SILOS - Source Independent Loads from staging to Data Warehouse schema for custom processes (modified from the existing one or newly designed)

Custom_PLP - Post Load Process for custom aggregations, snapshots etc (modified from the existing one or newly designed)

The naming conventions for folders defined in BI Data Warehouse should be followed if custom folders are created in the warehouse. (Best Practice ETL 2)

Informatica Mapping, Maplet, Session and Workflow

Customization of OOTB Components

The setting up session parameters such as commit interval etc is not in the scope of the customization best practices for abivous reasons (Best Practice ETL 3)

Mapping

Prefix the mapping with a custom identifier (e.g. customer identifier is ‘XYZ’), then folder type (“SDE_” or “SIL_”), and include a descriptive name.

For example, an ETL mapping to extract Invoice distribution fact from E-Business ERP, a name may be XYZ_SDE_ORA_APInvoiceDistributionFact

While copy and paste mapping from OOTB folder to Custom folder, there are two options ‘Paste’ and ‘Paste Shortcut’, I would prefer to use paste shortcut only when a mapping reference is required for workflow and there is no need to change the mapping at all.

Don’t make any change in the mapping in OOTB folder, take a copy of the mapping and copy it to equivalent custom folder, and follow the naming convention as stated above (Best Practice ETL4)

Why? To protect mapping changes from future upgrade of BI Apps. Adding a custom identifier in the mapping name in the custom folder provides better manageability for migration of informatica metadata and upgrade.

Maplet

Prefix the maplet with a custom identifier (“XYZ”), the object type (“mplt_” = maplet), and include a descriptive name.

For example, a re-usable part of a mapping to load Party Organisation may be: XYZ_mplt_PartyOrgDimension

The maplet are copied when mapping is been copied to custom folder, no need to rename the existing maplet, but any change or customization in maplet should be done by following the naming convention suggested. (Best Practice ETL5)

Why? Any customization in maplet would be easily identified, as I believe most of the data load logic is within the maplet and there is a danger that any change in existing maplet would affect the functioning of other mapping as maplet is most reusable component.

Sessions

Prefix the session with a custom identifier (“XYZ”), the folder type (“SDE_” or “SIL_”) and include a descriptive name.

For example, running a session to extract Invoice distribution fact from E-Business ERP may be: XYZ_SDE_ORA_APInvoiceDistributionFact

Create session based on the modified mapping in Workflow Manager. I would prefer to create multiple sessions from the same mapping if required e.g. incremental load and full load (Best Practice ETL6)

Why? Session allows overriding the SQL Override if I would like to change the data load extraction logic from incremental load to full load or vice versa

Workflows

Prefix the Workflow with a custom identifier (“XYZ”), the folder type (“SDE_” or “SIL_”) and include a descriptive name.

For example, running a workflow to extract Invoice distribution fact from E-Business ERP may be: XYZ_SDE_ORA_APInvoiceDistributionFact

Create workflow which uses the session within based on the modified mapping in Workflow Manager. I would prefer to create multiple workflows from the same mapping if required e.g. incremental load and full load (Best Practice ETL7)

Why? Workflow is a fantastic option to manage dependency of data load, can access various operations such as sql, shell script and obvious Informatica sessions Design once and reuse often is the Mantra.

New Custom Components

Note: Based on my implementation experiences I believe the most of the customer really don’t follow the customization practices for OOTB mappings/sessions. Copying the OOTB mapping into the Custom Folder and modifying it within the custom folder is very essential and I would go easy if someone is not using the naming convention suggested for OOTB mapping customization.

The naming convention for new custom components is a must requirement. The best practices mentioned for OOTB custom components are also applicable for new custom components.

Mapping

Prefix the mapping with a custom identifier (e.g. customer identifier is ‘XYZ’), then folder type (“SDE_” or “SIL_”), and include a descriptive name.

For example, an ETL mapping to create a post load process to create invoice on hold snapshot be: XYZ_PLP_InvoiceOnHoldSnapshot

Maplet

Prefix the maplet with a custom identifier (e.g. customer identifier is ‘XYZ’), the object type (“mplt_” = maplet), and include a descriptive name.

For example, a re-usable part of a mapping to load Meeting List may be:

XYZ_mplt_MeetingListDimension

Sessions

Prefix the Workflow with a custom identifier (e.g. customer identifier is ‘XYZ’), the folder type (“SDE_” or “SIL_”) and include a descriptive name.

For example, an workflow to create a post load process to create invoice on hold snapshot be: XYZ_PLP_InvoiceOnHoldSnapshot

Workflows

Prefix the Workflow with a custom identifier (e.g. customer identifier is ‘XYZ’), the folder type (“SDE_” or “SIL_”) and include a descriptive name.

For example, a workflow to create a post load process to create invoice on hold snapshot be: XYZ_PLP_InvoiceOnHoldSnapshot

Transformation

Use Change Capture functionality for incremental updates it is different for different applications, the rule is to load delta rather than full load everytime.(Best Practice ETL8)

Why? Incremental data load make a perfect sense, I would recommend implementing incremental load for PLP/Aggregate data load also. I know almost 5/6 BI Apps implementations where full load is done every night, it is faster than incremental load. In reality the partial full load for certain areas e.g. Assignment/Positions etc is a reality at many places.

After creating a dimension in Informatica, try to create a reusable expression to handel surrogate key resolution (Best Practice ETL9)

Why? Reusability is the basic principal, treat Transformation, Maplet etc as a function in PL/SQL terms. Design once and reuse whenever possible.

Create a reusable lookup transformation LKP_W_dimension_D. It should be defined as the following:

Moreover, the mandatory conditions are:

o INTEGRATION_ID=IN_INTEGRATION_ID

o DATASOURCE_NUM_ID=IN_ DATASOURCE_NUM_ID

And the following conditions are only added if the dimension is specified as a SCD Type 2, 3 or 6:

o EFFECTIVE_FROM_DT<=IN_DATE

o EFFECTIVE_TO_DT>IN_DATE

Lookup SQL Override of Lookup should be commented to avoid order by clause in query and make sure it is cacheable if referenced multiple times within the different mappings (Best Practice ETL10)

Why? Removal of Order by Clause will improve the performance of lookup query and making it cacheable will improve performance of overall load process.

Setting up the cache for lookup queries is a double edge sword, make sure appropriate analysis of storage space allocated to ETL Server is must before setting up the caching in place. Also make sure the purging of caching is in place to avoid data refresh issues.

Use lookup transformation based on ETL_PROC_WID is essential for restar ability (Best Practice ETL11)

Why? To make sure that data will not be reloaded after session failure

Ensure that all INNER/OUTER joins performed in Source Qualifiers are correctly defined; resolve one-to-many relationship by using lookup instead of using SQL override (Best Practice ETL12)

Why? The inner joins always limit the dataset loaded into the DW database while outer join may end up with unnecessary data in DW, it is essential that the required and right data is loaded, use RIGHT join

E.g. For multiple assignment records use primary assignment condition within lookup rather than using it within SQL Override.

Note- I believe BI Apps Informatica data extraction logic is mostly defined within SQL Override followed by few transformation, lookup and update strategy, the OOTB is not really using any of the functionality of Informatica as Integration Platform can offer. I believe it is under utilization of ETL tool.

Non Indexes Join Paths should be replaced with Informatica lookups and convert non-index-able filters to index-able ones. Review all joins in SQL Override (Best Practice ETL13)

Why? A flag column filter is better than using name not like ‘%SP%’ where clause, data extraction (SDE) time should be minimum to avoid any adverse impacts on source system.

Miscellaneous Informatica for BI Apps Best Practices

As a best practice, you should comment custom code you have introduced. Comments should include at least the developer's name and the date the code was added.(Best Practice ETL14)

Why? It is useful for traceability and accountability. Developer name, code and reason (for example requirement/bug) would be useful for continuous support to reporting solution.

Any modification to a component must be checked-in and commented once the fix is validated (Best Practice ETL15).

Why? Be a good developer.

Add documentation to identify an integration_id column (MD070, Database comments…(Best Practice ETL16)

Why? The documentation to support development is useful.

The flag 'Fail parent if this task fails' should be checked for all the sessions within a workflow(Best Practice ETL17).

The flag 'Fail parent if this task does not run' should be checked for all the sessions within a workflow(Best Practice ETL18).

Why? It is a standard Informatica development practice.

The Stop on Errors parameter should be set to 1 for all sessions within a workflow. This parameter is located in the Error handling area of the Config Object tab in Informatica PowerCenter Designer(Best Practice ETL19).

Why? It is helpful to stop data processing on occurrence of first error.

For all entities, such as dimension and fact tables, create two workflows, one to be used for a full load and the other to be used for an incremental load. Both workflows are based on the same mapping. The same mapping gets executed during both full and incremental loads. This provides an opportunity to tune each of these load scenarios(Best Practice ETL20).

Why? A separate handler for tuning each of the load scenarios is a absolutely must.

Custom change capture img suffix : The image suffix should be three characters long. The recommended naming convention is to start with C. For example, use C01, C02 and so on (Best Practice ETL21).

Input fields of a mapplet are prefixed by ‘INP_’. Output fields of a mapplet are prefixed by ‘EXT_’ (Best Practice ETL22)

Why? This is for Siebel Application based BI Apps ( BI CRM).

The surrogate keys related to day date calendar (non-fiscal) are specified as NUMBER. The mask applied is YYYYMMDD where YYYY = Year, MM = Month #, DD = Day #. The following example for a meeting start date MEETING_START_DT_WID would be the result of the transformation TO_NUMBER(TO_CHAR(MEETING_START_DT,’YYYYMMDD’)) (Best Practice ETL23).

Why? A special treatment for date to date WID conversion as Date Dimension is referenced by WID from Fact tables.

I hope this blog Oracle BI Apps 7.9.6.3 Tips and Best Practice Guidelines for Customization (Informatica) is good enough to answer few of your questions/queries. I would appreciate your feeback/comments if any.