8.x Architecture

AgendaAgenda

ETL BasicsPowerCenter 7x ArchitecturePowerCenter 7x Components and ConnectivityPowerCenter 8x ArchitecturePowerCenter 8x Components and ConnectivityOverview of

o Service Oriented Architectureo Domains and Nodeso Services:

• Service Manager • Domain Operations• Application Services

Informatica 7x Server Grid Concept Vs Informatica 8x Server Grid Concept

ETL BasicsETL Basics

Extract, Transform, and LoadExtract, Transform, and Load

RDBMSRDBMS MainframeMainframe OtherOther

Operational Systems

DataDataWarehouseWarehouse

Decision Support

• Transaction level data• Optimized for Transaction

Response Time• Current• Normalized or De-

Normalized data

• Aggregated data

• Historical

ETLETLExtract Load

Transform

Cleanse DataApply Business Rules

Aggregate DataConsolidate Data

De-normalize

HR3-LMS DWHHR3-LMS DWH

OHRDB

PlateauFiles

ETL ProcessBy

InformaticaHR3 DWH HR3 LMS

Universe

ETL ToolsETL Tools

Some of the ETL Tools which we have currently in Market.

1.Informatica

2.Ab Initio

3.Data Transformation Services

4.IBM Web Sphere Datastage

5.Oracle Warehouse Builder

Etc…

ArchitectureArchitecture• PowerCenter 7x ArchitecturePowerCenter 7x Architecture• PowerCenter 8.5 ArchitecturePowerCenter 8.5 Architecture

PowerCenter 7x Architecture PowerCenter 7x Architecture

Informatica Architecture & Connectivity – 6.x and 7.xInformatica Architecture & Connectivity – 6.x and 7.x

TargetsRDBMS

native native

TCP/IP

HeterogeneousTargets

Repository

RepositoryServer

RepositoryAgent

TCP/IP

native

Server

HeterogeneousSources

Repository Designer Workflow Workflow Rep Server Manager Manager Monitor Administration Console

ODBC ODBC

Informatica Architecture & Connectivity – 6.x and 7.xInformatica Architecture & Connectivity – 6.x and 7.x

• The management of the repository is done through the ‘Repository Server’, since it is the only process which interacts with the Repository. The Repository Server can be managed and administered through the ‘Repository Server Administrator Console’ interface.

• One Repository Server can manage multiple repositories. For each repository there is one Repository Agent.

• Repository Agent is a multi-threaded process which inserts, updates and retrieves metadata from Informatica Repository

• All the client tools and Informatica Server communicate with the Repository through Repository Server

• Repository Server Administration Console is a client tool used to create/maintain repositories and configure Repository Servers. All the tasks like starting a repository, backup/restore and upgrade are performed using this tool

PowerCenter 7x ComponentsPowerCenter 7x Components

Informatica Power Center Components:

PowerCenter Repository PowerCenter Repository Server PowerCenter Client

» Repository Manager» Repository Server

Administration Console» Designer» Workflow Manager» Workflow Monitor

PowerCenter Server

PowerCenter 7x ArchitecturePowerCenter 7x Architecture

PowerCenter 7x Architecture (cont)PowerCenter 7x Architecture (cont)

•Sources •Targets •Repository •Repository Server •PowerCenter Client •PowerCenter Server

•Load Manager Process •Data Transformation Manager

(DTM) Process •Code Pages and Data

Movement Modes •PowerCenter Metadata

Reporter

Load Manager ProcessLoad Manager Process

•Load Manager Process

Manages workflow scheduling.

Locks and reads the workflow.

Reads the parameter file.

Creates the workflow log file.

Runs workflow tasks and evaluates the conditional links connecting tasks.

Starts the DTM, which runs the session.

Writes historical run information to the repository.

Sends post-session email in the event of DTM failure.

DTM ProcessDTM Process

•Data Transformation Manager (DTM) ProcessReads session information from

the repository. Expands the server, session,

and mapping variables and parameters.

Creates the session log file. Validates source and target

code pages. Verifies connection object

permissions.

Runs pre-session shell commands, stored procedures and SQL.

Creates and runs mapping, reader, writer, and transformation threads to extract, transform, and load data.

Runs post-session stored procedures, SQL, and shell commands.

Sends post-session email.

PowerCenter Server

Repository Serverdev001

PowerCenter Server

dev002

dev003

dev004

PowerCenter Server

Repository

Current EnvironmentCurrent Environment

PowerCenter 8.5 Architecture PowerCenter 8.5 Architecture

PowerCenter 8.x ComponentsPowerCenter 8.x Components

Standards, Messaging,

Web Services

Packaged Applications

Relational/Flat Files

Mainframe/ Midrange

Standards, Messaging, Web Services



Mainframe/ Midrange

Sources Targets

Integration Service(s)

PowerCenter Connects PowerExchange

Core Services

Application Services

Client Tools Designer WF Manager Monitor

Rep. Manager Administration Console

SAP BW Service

Web Services Hub

Repository Service(s)

Configuration Service

Authentication Service

Domain Service

Log Service

Gateway Service

Administration Service

Repository Database

Standards, Messaging,

Web Services Standards,

Messaging, Web Services


Relational/Flat Files Relational/Flat Files

Mainframe/ Midrange Mainframe/ Midrange

Standards, Messaging, Web Services



Mainframe/ Midrange

Sources Targets

Integration Service(s)

PowerCenter Connects PowerExchange

Core Services


Client Tools Designer Designer WF Manager WF Manager Monitor Monitor

Rep. Manager Rep. Manager Administration Console Administration Console

SAP BW Service

Web Services Hub

Repository Service(s)

Configuration Service

Authentication Service

Domain Service

Configuration Service Configuration Service

Authentication Service Authentication Service

Domain Service Domain Service

Log Service

Gateway Service

Administration Service

Log Service Log Service

Gateway Service Gateway Service

Administration Service Administration Service

Repository Database

Repository Database

OverviewOverview

• Service Oriented Architecture

• Domains, Nodes, and Services

• Services: Service Manager Domain Operations Application Services

Service Oriented Architecture (SOA)Service Oriented Architecture (SOA)

SOA: An application architecture in which all functions, or services, invoke software interfaces that perform business processes.

Service: A task performed by a service provider to achieve desired end results for a service consumer. Both provider and consumer are roles played by software agents on behalf of their owners.

Service Consumer

Service Provider

ResponseRequest

SOA (cont.)SOA (cont.)

•For example, in PowerCenter 8.5:PowerCenter Server has become a service, the

Integration ServiceNo more Repository Server, but PowerCenter includes a

Repository Service

•Client applications are the same, but work on top of the new services framework

Domains, Nodes and ServicesDomains, Nodes and Services

Domains, Nodes, and ServicesDomains, Nodes, and Services


Integration Service

Repository Service

SAP BW Service

Web Services Hub

Core Services

Service Manager

Node 1 (machine)


Node 2 (machine)

Domain


Node 3 (machine)

Integration Service

Repository Service

SAP BW Service

Web Services Hub

Core Services

Service Manager

Integration Service

Repository Service

SAP BW Service

Web Services Hub

Core Services

Service Manager

PowerCenter DomainPowerCenter Domain

•Domain can be a single PowerCenter installatione.g. single Integration Service, Repository Service

•Or it can be multiple PowerCenter installations

PowerCenter Domain (cont.)PowerCenter Domain (cont.)

MarketingHuman Resources

Sales

DomainsDomains

• A collection of nodes and services

• Primary Unit of administration

• Domains and components are managed in the Administration Console

• Centralizes configuration and administration

Configuring DomainsConfiguring Domains

• Domains and components are managed in the Administration Console

• Centralizes configuration and administration

NodesNodes

• The logical representation of a machine in a domain •Each node runs a Service Manager

Lightweight service

•Types of Nodes: Gateway Node Worker Node

Nodes (cont)Nodes (cont)

• Gateway Nodes Entry Point to the domain from the PowerCenter Client Routes service requests from PowerCenter Client to available

nodes Only one node in domain serves as a gateway for domain at a

given point of time Master Gateway Backup Gateway

If gateway node is unavailable, domain cannot accept service requests

ServicesServices

•All PowerCenter processes run as services on a node•All services installed by default.

Define which services run by configuring the node and services.

•Two types:ApplicationCore

Core ServicesCore Services

•Built in to the domain•Support the domain and application services•Service Manager manages services

Core ServicesCore Services

•Gateway Service•Log Service•Configuration Service•Domain Service•Service Process Controller •Licensing Service•Authentication Service•Admin Service

Licensing ServiceLicensing Service

•Manages PowerCenter licensesMaster Gateway Node

•Assigns licenses and validates application services when they start

•Licenses keys areEncrypted in a binary fileOriginal keysIncremental keys (upgrade)

License TasksLicense Tasks

•Create and remove•Assign and unassign•Update to add license keys•View license details•Configure user permissions on license

Log ServiceLog Service

•Collects all PowerCenter log events•Processes log events for core and application services,

workflows, and sessions•Runs on gateway node by default•When Log Service becomes unavailable:

Core and application services write log events to temporary files•When Log Service becomes available again:

Service Manager processes the temporary files and sends them to the Log Service

Application ServicesApplication Services

•Represent server-based functionality in PowerCenter•Following types:

Integration Service Repository Service Reporting Service Metadata Manager Service SAP BW Service Web Services Hub

•You designate a node on which to run the service process

Service ProcessesService Processes

•The runtime instance of a service running on a node•Service processes do all the work

36

Service Manager

• Controls PowerCenter services running on the node

• Primary point of control for PowerCenter services

• Service on Windows, daemon on UNIX

• If Service Manager is not running, node is unavailable

37

Services (cont.)


Integration Service

Repository Service

SAP BW Service

Web Services Hub

Core Services

Service Manager

Node 1


Integration Service

Repository Service

SAP BW Service

Web Services Hub

Core Services

Service Manager

Node 2

Service installed, but not configured to run

Service installed and configured to run

Domain

Domain Operations: Services (cont)Domain Operations: Services (cont)

Node 1 (Master Gateway)

Service ManagerService Manager

User Management

Logging

Licensing

Node Configuration

Domain Configuration

Authorization

Authentication

AlertsDomain Operations Node 2 (Backup Gateway)


Logging

Node Configuration

Domain Operations

Node 3 (Worker Node)


Logging

Node Configuration

Domain Operations

DomainDomain

Application Services: Services (cont)Application Services: Services (cont)

Node 1


Web Services Hub

SAP BW Service

Metadata Manager Service

Reporting Service

Repository Service

Integration ServiceApplication Services

DomainDomainNode 2


Web Services Hub

SAP BW Service

Metadata Manager Service

Reporting Service

Repository Service

Integration ServiceApplication Services

Same Clients, New FrameworkSame Clients, New Framework


Repository Repository Service 1Service 1

Integration Integration Service 1Service 1


Node 2 (Gateway Node)

RepositoryRepository

PowerCenter Client

TargetsSources

Administration Administration ConsoleConsole

Domain

Same Clients, New Framework (cont)Same Clients, New Framework (cont)







PowerCenter Client

Sources/Targets

Administration Administration ConsoleConsole

Domain

Repository ServiceRepository Service

• In 7x, the repository server can connect to multiple repositories starting a separate repository agent, one for each repository.

• In 8.5 one repository service can connect to only one repository. But one node can host more than one repository services.

Node 1



Repository Server

Repository 1Repository 1



Repository 2Repository 2Repository Repository

Agent 2Agent 2

Repository Repository Agent 1Agent 1

8.58.5

7x7x

Integration ServiceIntegration Service

•The Integration Service starts one or more Integration Service processes.

• Integration Service Process(8.5) = Load Manager(7x)

Source/Targets


Integration Service 1Integration Service 1

Service Service ManagerManager


RepositoryRepositoryPowerCenter

Client

Domain

Repository Service 1Repository Service 1

11

667755

443322

88

Server GridServer Grid• 7x Server Grid Concept7x Server Grid Concept• 8.5 Server Grid Concept8.5 Server Grid Concept

dev003

dev001

dev002

dev004

Master Server

Worker Server

Worker Server

Worker Server

7x Grid Architecture7x Grid Architecture

PowerCenter 8.5 Server GridPowerCenter 8.5 Server Grid

• Steps:Assign Resources to the nodesCreate the grid using the nodesConfigure the Integration

Service to run on the nodesRun the Workflow on the

Integration Service

• System Resources: CPU usage DTM buffer memory Cache memory Connection Information

Node 1Node 1

Node 2Node 2

Node 3Node 3

Node 4Node 4

ApplicatioApplication n

DatabaseDatabaseIntegration Integration

ServiceService

PowerCenter 8.5 Server Grid (cont)PowerCenter 8.5 Server Grid (cont)

• Shared Location:Session LogsWorkflow LogsCache FilesBad FilesSource Files (Flat Files, etc)Target Files (Flat Files, etc)

Node 1


IS Service IS Service process 1process 1

Node 3


Node 2



Application Application DatabaseDatabase

Shared Shared LocationLocation

ISIS

Server Grid ComparisonServer Grid Comparison

PowerCenter 7xPowerCenter 7x PowerCenter 8.5PowerCenter 8.5Essentially a server grid. A number of Informatica servers were added to form one Server grid.

Nodes are grouped to form a grid.

Only homogenous Informatica Servers can be a part of the grid. For forming a grid the requirements for the participating servers are :Same OSSame no. of CPU’sSame amount of memory spaceSame directory structureAny server not fulfilling the above prerequisites cannot be a part of the grid

Heterogeneous nodes can be grouped to form a grid. There is no restriction on the OS, no. of CPU’s or memory space for the participating nodes. The only restriction is that all the nodes have a common shared directory structure.

Server Grid Comparison (cont)Server Grid Comparison (cont)

PowerCenter 7xPowerCenter 7x PowerCenter 8.5PowerCenter 8.5One server in the grid acts as the Master Server to whom all the requests are be routed while other servers are be worker servers.The master server runs all the non-session tasks assigned to the grid.

Grid is a service just like the Integration Service. The Load Balancer is the component of the Integration Service that dispatches the different tasks to the nodes or the different threads to the DTM processes running on the nodes in the grid. The Load Balancer distributes tasks or threads based on node and resource availability

Essentially acts as workflow on grid. Individual sessions are assigned to an Informatica Server.

Acts as session on grid whereby the different threads of a session like the reader, writer and transformation threads can be assigned to different nodes.

SOA AdvantagesSOA Advantages

Replaced several servers that were used within the PowerCenter 7.x installer

Administration Tool is Web Based

SOA supports several integration service processes to run concurrently

Enables High Availability Option

Thank YouThank You

8.x Architecture

Documents

Transcript of 8.x Architecture