Transformation 20

download Transformation 20

of 27

  • date post

    10-Apr-2015
  • Category

    Documents

  • view

    1.417
  • download

    0

Embed Size (px)

Transcript of Transformation 20

INFORMATICA 7.1.1S. No 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Transformation Source Qualifier A/C Filter A/C Expression P/C Sequence Generator P/C Router A/C Union A/C Transactional control A/C Sorter A/C Normalizer A/C Stored Procedure P/C-UC Joiner A/C Aggregator A/C Rank RNK A/C Look-up A/C-UC XML Source Qualifier A/C Custom A-P/C External Procedure P/C-UC Update Strategy A/C Midstream XML Parser Midstream XML Generator Application Source Qualifier MQ Series Qualifier Application Multi group Qualifier SQ * * FIL * * EXP * * * SEQ * * RTR * * UN * * TC * * SRT * * NRM * * SP * * JNR * * AGG * * * * * * * LKP * * XML CT * * EP * * UPD * * *DD_INSERT, DD_UPDATE, DD_DELETE, DD_REJECT - UP as UP , UP as IN, UP else In , Tranc EXPRESSION - Reject file created

I O V R M L Distinct, Expression

Important

PA GE

8 4

Group by port, Sorted input, Expression

NEXTVAL,CURVAL Expression

10

Distinct Sequence, restart * * Bad file created Sorted Input Group by port, Sorted input Group by port, Expression *

7

*

M.SHANMUGAM

Transformation Details

Page 1 / 24

SHANMUGAM.M

M.SHANMUGAM

Transformation Details

Page 2 / 24

1. AGGREGATOR TRANSFORMATIONA/C - AGGDEFINITION The Aggregator transformation is Active and Connected. The Aggregator transformations allow you to perform to aggregate calculation. You can use the aggregator transformation to perform calculation on groups. The Row which meet the condition are passed to target. The doesnt meet the condition, rejected row store rejected file or Bad file directory. AGGREGATOR FUNCTIONS AVG COUNT FIRST LAST PORTSINPORTS

MAX MEDIAN MIN PERCENTILE

STDDEV SUM VARIANCE

OUTPORTS VRIABLE PORTS GROUP BY PORT

- (I) each input Receive data - (O) Pass the data to other transformation - (V) its stores the Intermediate result it can reference input ports Not to out ports -

PROPERTIES Cache Directory Tracing Level Sorted Input Aggregator Data Cache Aggregator Index Cache Transformation Scope COMPONENT Aggregate Expression Aggregate Cache Group by Port Sorted Input Non aggregate Expression / Conditional Class Which column you want group by Eg. Dept Reduce the amount of data cached - $PMCaheDir - Normal - 2000000 Bytes - 1000000 Bytes - All input ( Terse / Normal / Verbose initialization / verbose data )

(Transaction / All Input)

AGGREGATOR CACHE: The PCS stores data in the aggregate cache until it complete the aggregator calculation Index Cache : It stores the group value, As Configured in the group by port Data Cache : Stores calculation ( Row data Stores, output value) Based on group-by-ports OPTIMIZATION Group by simple columns like numbers instead of string or date Use sorted input Use incremental aggregation Minimize the aggregate function Before filter transformation best ( Reduce the Data) Lookup unconnected & stored procedure we can call

M.SHANMUGAM

Transformation Details

Page 3 / 24

2. EXPRESSION TRANSFORMATIONP/C - EXPDEFINITION Expression Transformation Passive and connected transformation This can be calculate values in a single row before writing to the target. Row by row calculation, Perform the any non aggregate function EXPRESSION FUNCTIONS The Expression Transformation is used for data cleansing and scrubbing There are over 80 functions within PowerCenter, such as salary, concatenate, instring, rpad, ltrim and we use many of them in the Expression Transformation. We can also create derived columns and variables in the Expression Transformation. COMPONENTExpression

-

we can call Unconnected Stored Procedure and Unconnected Lookup

PORTSINPORTS OUTPORTS VARIABLE PORTS

- (I) Each input port Receive data - (O) which provide the value to either target or next transformation in the mapping is called output ports - (V) Its stores the Intermediate result it can reference input ports ( -Which stores the variable information ) Normal ( Terse / Normal / Verbose initialization / verbose data )

PROPERTIESTracing Level

OPTIMIZATION

Factoring out common logic Minimizing aggregator function calls. For Eg.use SUM(A+B) instead of using SUM(A) + SUM(B) Replacing common sub expression with local variables Choosing Numeric Vs String operation Choose DECODE function Vs LOOK UP operation Choose CONCAT operation for Eg use |||| instead of CONCAT (Fname, Lastname) you can enter multiple expression in a Single Expression Transformation.

M.SHANMUGAM

Transformation Details

Page 4 / 24

3. FILTER TRANSFORMATIONA/C - FILDEFINITION This is a type of active and connected Transformation which is used to filter of the source rows based on a condition. Only the row which meet the condition are pass through to target. Any kind of source we can use filter Transformation Filter condition drops data that does not match the condition We can put one or more condition (more condition means we can use AND , OR operator) Discards rows dont appear in the session log or reject files PORTSINPORTS OUTPORTS

- (I) Receive data from source - (O) Pass the data to other Transformation

PROPERTIES Filter Condition Tracing Level OPTIMIZATION (TIPS)

: : Normal

(Normal / Terse / Verbose init / Verbose data )

Use the filter transformation early in the mapping (or) nearly in SQ The filter condition is case sensitive, and queries in some database do not take this into account.

TROUBLESHOOTING Case sensitivity Appended spaces FUNCTIONS You can use one or more condition in filter transformation AND , OR logical operator through : the filter condition is case sensitive : use the RTRIM function to remove additional space

M.SHANMUGAM

Transformation Details

Page 5 / 24

4. JOINER TRANSFORMATIONA/C - JNRDEFINITION - This is active and connected Transformation. - Can be used to join two sources coming form two different locations or same location. - We can use homo genius and hetero genius sources - Join a flat file and a relational sources or to join two flat files or to join a relational source and a XML source. CONDITION 1). Two sources there must be at least one matching ports or columns 2). Two sources there should have Primary key and Foreign key relationship PORTSINPORTS OUTPORTS MASTERPORTS

- (I) Receive data from source - (O) Pass the data to other Transformation - (M) If checked master(small) otherwise details (large) (to switch the Master Details relationship for the source )

PROPERTIES 1 Cache sensitive String Comparison (Character data only enable) 2 Cache Directory - $PMCacheDir 3 Join Condition 4 Join Type - NORMAL (Normal - M.outer -D.outer- Full Outer) 5 Null ordering in Master - Null is highest value (Null is lowest value) 6 Null ordering in Detail - Null is highest value (Null is lowest value) 7 Tracing Level - Normal (Normal / Terse / Verbose init / Verbose data ) 8 Joiner Data cache size - 2000000 9 Joiner Index cache size - 1000000 10 Sorted Input 11 Transformation Scope - All input (Transaction / All Input ) COMPONENT Case sensitive string comparison (Character data only enable) Cache directory Join condition Joiner type ( Normal, Master Outer, Detail Outer, Full outer) CACHE Joiner Data cache size : Out put value only Joiner Index cache size : The index cache holds rows from the master source that are in the join condition.

M.SHANMUGAM

Transformation Details

Page 6 / 24

Index cache Data Cache Stores index values for the master source table as Stores master source rows. configured in the join condition. FUNCTIONS Following types of source can be used in a joiner Two relational tables existing in separate databases Two flat files in potentially different file systems Two different ODBC sources Two instances of the same XML sources A relational table and a Flat file source A relational table and a XML source A joiner cannot contain the following types of source Both pipelines begin with the same original data sources. Both input pipeline originate from the same source qualifier transformation Both input pipeline originate from the same normalizer transformation. Both input pipeline originate from the same joiner transformation. Either input pipeline contains an update strategy transformations Either input pipeline contains a connected or unconnected sequence Generator transformation. PERFORMANCE Use sorted input (flat file ,relational data,) Minimizing the disk input and output Use in front of sorted transformation For an unsorted joiner transformation, designate as the master source the source with fewer rows For an sorted joiner transformation, designate as the master source the source with fewer duplicate key values Following Transformation we cant use before the joiner Transformation. - Sequence Generator Transformation directly - Update strategy Transformation TIPS - sorted input improve the session performance. - Dont use following transformation sort origin and joiner transformation - Custom , Unsorted aggregator, Normalizer, Rank. - Sort order from both table( master & Detail) - Normal or Master outer join perform than a full outer or detail outer join. Normal - Matched Rows form master and detail source Master - all rows data from the detail source and the matching rows from the master source Detail - all rows data from the master source and the matching rows from the detail source Full outer - all rows rows of data from both the master and detail sources

M.SHANMUGAM

Transformation Details

Page 7 / 24

5. RANK TRANSFORMATIONA/C - RNKDEFINITION This an Active and Connected Transformation Which is used to identify the Top or Bottom rank of data based on condition. Rank transformation to re