©1998, 1999, 2000, 2001 Rational Software - All rights reserved©1998, 1999, 2000, 2001 Rational Software - All rights reserved

Session CM11Session CM11

Release Managementfor the UBS Data Warehouse Project

Release Managementfor the UBS Data Warehouse Project

Friedrich LehnUBS AG, Switzerland

friedrich.lehn@ubs.com

Friedrich LehnUBS AG, Switzerland

friedrich.lehn@ubs.com

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

AgendaAgenda

Project Overview Project Infrastructure Change Management Release Process Summary

Project Overview Project Infrastructure Change Management Release Process Summary

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project OverviewProject OverviewUBS

Global, integrated investment services firmand leading bank in Switzerland

World’s largest private bank Total client assets over US$ 1.5 trillion Acquired in 2000

UBS

Global, integrated investment services firmand leading bank in Switzerland

World’s largest private bank Total client assets over US$ 1.5 trillion Acquired in 2000

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project OverviewProject OverviewData Warehouse Program (DWP)

Establish common infrastructure for analytical data processing

Provide a business oriented set of data warehouseand data mart services

Standardized business data model Align the bank’s data mart portfolio Improve flexibility, time-to-market and data quality

Data Warehouse Program (DWP)

Establish common infrastructure for analytical data processing

Provide a business oriented set of data warehouseand data mart services

Standardized business data model Align the bank’s data mart portfolio Improve flexibility, time-to-market and data quality

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project OverviewProject OverviewData FlowData Flow

System of RecordsSystem of Records

Common Data / Business WarehouseCommon Data / Business Warehouse

Data MartsData Marts

extract, condition & loadextract, condition & load

data mart sourcingdata mart sourcing

visualizationvisualization

business userbusiness user

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project OverviewProject OverviewDelivery Streams

Main organizational element:team working on a subject data area / data mart

Three letter acronym as base for naming standards

Standardized infrastructure:UNIX directories, access group, meta data area, ...

Each delivery stream has business responsible, data modeler, database administrator, delivery stream manager

Team size typically between 1 and 5

Delivery streams release independently from each other

Delivery Streams Main organizational element:

team working on a subject data area / data mart

Three letter acronym as base for naming standards

Standardized infrastructure:UNIX directories, access group, meta data area, ...

Each delivery stream has business responsible, data modeler, database administrator, delivery stream manager

Team size typically between 1 and 5

Delivery streams release independently from each other

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project OverviewProject OverviewApplication StructureApplication Structure

<MAR> <MAR>...

RCL MDR FDS

System of Records

DSFDSF

Sourcing Framework

<SDA> <SDA>...

RCL release control toolsMDR meta data repositoryDSF DWP sourcing frameworkFDS feed configuration files<SDA> subject data area<MAR> data mart

RCL release control toolsMDR meta data repositoryDSF DWP sourcing frameworkFDS feed configuration files<SDA> subject data area<MAR> data mart

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project InfrastructureProject InfrastructureSystem EnvironmentsSystem Environments

• IBM AIX SP2 cluster• DB2 UDB EEE• PowerCenter V1.7• DWP Sourcing Framework• Cognos / Business Objects• ClearCase V3.2

• IBM AIX SP2 cluster• DB2 UDB EEE• PowerCenter V1.7• DWP Sourcing Framework• Cognos / Business Objects• ClearCase V3.2

Development ProductionTest

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project InfrastructureProject InfrastructureLogical Environments and Release StructureLogical Environments and Release Structure

Development Test Production

D A P delivery stream development

T framework development

framework test / delivery stream migration to new framework releases

X F V

E emergency releases

mandatoryoptional

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project InfrastructureProject InfrastructureRelease Cycles

delivery stream development: D A P

framework development: T X F Vafter sign-off: X D A P

migration to new frameworkreleases: [DAP] X F

emergency releases: [DAP] E P

Release Cycles

delivery stream development: D A P

framework development: T X F Vafter sign-off: X D A P

migration to new frameworkreleases: [DAP] X F

emergency releases: [DAP] E P

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project InfrastructureProject Infrastructure

Two areas:/dwp_root release area, version controlled/dwp_data dynamic data, archival on demand

Additional directory level in order to support more than one logical environment on one system

/dwp_root is organized by delivery streams, e. g.:/dwp_root/d/streams/rcl/bin

~<user>/dwp_root user specific development area

Two areas:/dwp_root release area, version controlled/dwp_data dynamic data, archival on demand

Additional directory level in order to support more than one logical environment on one system

/dwp_root is organized by delivery streams, e. g.:/dwp_root/d/streams/rcl/bin

~<user>/dwp_root user specific development area

Directory StructureDirectory Structure

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Project InfrastructureProject InfrastructureDirectory Structure (continued)

/dwp_data is organized by logical processing steps, e. g.:/dwp_data/p/data/landing (landing area)/dwp_data/p/data/tgtfiles (target files)/dwp_data/p/logs/system (framework log area)

Tool support for generation of directories in source environments (delta processing)

Automatic creation of missing directories in target environments by release procedures

Directory Structure (continued)

/dwp_data is organized by logical processing steps, e. g.:/dwp_data/p/data/landing (landing area)/dwp_data/p/data/tgtfiles (target files)/dwp_data/p/logs/system (framework log area)

Tool support for generation of directories in source environments (delta processing)

Automatic creation of missing directories in target environments by release procedures

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Change ManagementChange ManagementDesign Principles

Support different, clearly separated environments with different responsibilities

All environments have identical structure (products, databases, server configurations)

All program changes are done on the development system

All changes on test and production systems go through the release process and are clearly tracked

Design Principles

Support different, clearly separated environments with different responsibilities

All environments have identical structure (products, databases, server configurations)

All program changes are done on the development system

All changes on test and production systems go through the release process and are clearly tracked

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Change ManagementChange ManagementClearCase Set-up

One single VOB /vobs/dwp

Fully automated access layer (freeze and deliver routines)

Same directory structure as below /dwp_root, directories are automatically created

In general: only linear version trees (important: synchronization with database change management)

Branch support planned for emergency releases and migration path only

ClearCase Set-up

One single VOB /vobs/dwp

Fully automated access layer (freeze and deliver routines)

Same directory structure as below /dwp_root, directories are automatically created

In general: only linear version trees (important: synchronization with database change management)

Branch support planned for emergency releases and migration path only

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Change ManagementChange ManagementRelease Naming

<delivery stream>_<major>.<minor>.<patch>e. g.: RCL_1.1.0

<major> major release number(high level “wave” planning)

<minor> minor release number(delivery stream development plan)

<patch> patch level (bug fixes)

Emergency releases:RCL_1.1.0_sos_<#>

Release Naming <delivery stream>_<major>.<minor>.<patch>

e. g.: RCL_1.1.0

<major> major release number(high level “wave” planning)

<minor> minor release number(delivery stream development plan)

<patch> patch level (bug fixes)

Emergency releases:RCL_1.1.0_sos_<#>

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Change ManagementChange ManagementVersioning ExampleVersioning Example

RCL_1.0.0

RCL_1.0.1

RCL_1.0.2

RCL_2.0.0

RLS_P

RLS_A

common.pm

/main/1

/main/2

/main/4

/main/5

/main/3

RLS_A current version inuser acceptance test

RLS_P current version in production

/main/sos/1RCL_1.0.0_sos_1

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessChange Control Board

Responsible for high level planning and impact analysis

Defines release scope and release numbers on baseof delivery streams

Assigns responsibilities (delivery stream manager, data modeler, database administrator, business responsible)

Result is documented in “wave plan” document

Change Control Board

Responsible for high level planning and impact analysis

Defines release scope and release numbers on baseof delivery streams

Assigns responsibilities (delivery stream manager, data modeler, database administrator, business responsible)

Result is documented in “wave plan” document

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessRoles & ResponsibilitiesRoles & Responsibilities

developer development, unit and integration testing

delivery stream manager manager, release planning

database administrator database change control

release manager deployment, tracking, configuration control,

administration

RoleRole ResponsibilityResponsibility

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessProcess OverviewProcess Overview

Development ProductionTest

ClearCase

Freeze

Deliver

Receive

Deployment Package

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessRelease Objects

PowerCenter mappings

scripts / SQLs

Job dependency data

Uniserv jobs (address conversions)

Database objects (see below)

not included: documentation (intranet database)

Release Objects

PowerCenter mappings

scripts / SQLs

Job dependency data

Uniserv jobs (address conversions)

Database objects (see below)

not included: documentation (intranet database)

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessDatabase Objects

tables (regular, summary)

views (business, security)

keys (unique, primary, foreign)

indexes

triggers

aliases

check constraints

not included: table spaces (system dependent)

Database Objects tables (regular, summary)

views (business, security)

keys (unique, primary, foreign)

indexes

triggers

aliases

check constraints

not included: table spaces (system dependent)

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessRelease ProcedureRelease Procedure ResponsibleResponsible

1. Submit database change request *) delivery stream manager

2. Implement database changes *) database administrator

3. Prepare release area delivery stream manager (UNIX, PowerCenter, job dependencies, Uniserv)

4. Submit release request delivery stream manager

5. Prepare release area (DDLs) *) database administrator

6. Create new release (Freeze) release manager

7. Create deployment package (Deliver) release manager

8. Apply database changes to target system *) database administrator

9. Install release in daily deployment window release manager(Receive) (IT integration / IT operation)

*) in case of database changes only

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessFreeze ProcessFreeze Process

freeze <delivery stream> [ -patch | -minor | -major | -sos <release> ]

1. Retrieve previous release

2. Compare with release area (check for new, changed, deleted files)

3. Display results and ask for confirmation

4. Apply changes in ClearCase

5. Create and attach release label

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessDeliver ProcessDeliver Process

deliver <delivery stream> -t <target env.> [ -r <release> ] [ -a ]

1. Retrieve specified / latest release in ClearCase

2. Retrieve target environment file versions and create delta

3. Use -a(ll) for initialization / synchronization

4. Create deployment package (tar file + control file)

5. Update target labels

6. Lock release label

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessReceive ProcessReceive Process

receive

1. Check hand-over area for pending releases

2. Remote copy deployment package to target system

3. Install it

4. Standardized post-installation steps: e. g. access permissions

5. Delivery stream defined post-installation steps (PostInstall.ksh file):e. g. for non-standard path names, generators, setuid bits

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessReleaseRequest (1)ReleaseRequest (1)

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessReleaseRequest (2)ReleaseRequest (2)

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessReleaseRequest (3)ReleaseRequest (3)

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessDatabase Change Management

PATROL DB-Change Manager by bmc software

Scope filter: assign database objects to delivery stream( view in ClearCase, name equal to delivery stream)

Apply database changes to development database

Create release baseline:freeze all database object versions for a delivery stream( label in ClearCase, name equal to release label)

Export DDL to release area(for documentation & change detection)

Database Change Management

PATROL DB-Change Manager by bmc software

Scope filter: assign database objects to delivery stream( view in ClearCase, name equal to delivery stream)

Apply database changes to development database

Create release baseline:freeze all database object versions for a delivery stream( label in ClearCase, name equal to release label)

Export DDL to release area(for documentation & change detection)

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessDatabase Change Management (target system)

Target baseline: target database version(delivery stream plus timestamp as name)

create delta DDL depending on release baselineand latest target baseline (PATROL)

apply delta DDL

create new target baseline

Database Change Management (target system)

Target baseline: target database version(delivery stream plus timestamp as name)

create delta DDL depending on release baselineand latest target baseline (PATROL)

apply delta DDL

create new target baseline

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessDatabase Retrofitting Process

Rationale: certain database changes have to be applied and tested directly on the Production system (load performance optimization: indexes, summary tables, ...)

In order to include target system changes into next regular release, changes are promoted back to Development using the Retrofitting Process

In principle: new, database administrator driven release in ClearCase and PATROL that is not delivered

Database Retrofitting Process

Rationale: certain database changes have to be applied and tested directly on the Production system (load performance optimization: indexes, summary tables, ...)

In order to include target system changes into next regular release, changes are promoted back to Development using the Retrofitting Process

In principle: new, database administrator driven release in ClearCase and PATROL that is not delivered

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessReleasing Meta Data

PowerCenter and JDM data are stored in the database

Uniserv jobs are stored in vendor specific directories

Unload utilities to export the corresponding dataand store it in the delivery stream´s release area (tar file)

Meta data is frozen and delivered together with all otherfile system objects

Load utility for loading objects in target environment

e. g.: meta_jdm <delivery stream> [ unload | load ]

Releasing Meta Data

PowerCenter and JDM data are stored in the database

Uniserv jobs are stored in vendor specific directories

Unload utilities to export the corresponding dataand store it in the delivery stream´s release area (tar file)

Meta data is frozen and delivered together with all otherfile system objects

Load utility for loading objects in target environment

e. g.: meta_jdm <delivery stream> [ unload | load ]

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessReleaseDatabaseReleaseDatabase

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Release ProcessRelease ProcessRelease Database(continued)Release Database(continued)

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

SummarySummaryExperiences

Over 1500 releases since May 2000

Effort for creation and installation of new release:5 - 60 minutes depending on amount of meta data(without database changes)

Main effort necessary for handling of meta data

No ClearCase problem encountered so far

Experiences

Over 1500 releases since May 2000

Effort for creation and installation of new release:5 - 60 minutes depending on amount of meta data(without database changes)

Main effort necessary for handling of meta data

No ClearCase problem encountered so far

Click on the desired colorClick on the desired color

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

Helpful tip: Double click the paintbrush tool to applycolor to more than oneobject at a time.

SummarySummaryWish List

Transaction concept for set of cleartool commands:What happens if the 199th check-in of 200 fails? Roll back?

Signal handling option for cleartool:Today it is not possible to ignore SIGINT in cleartool( manual clean up)

Wish List Transaction concept for set of cleartool commands:

What happens if the 199th check-in of 200 fails? Roll back?

Signal handling option for cleartool:Today it is not possible to ignore SIGINT in cleartool( manual clean up)

Questions?Questions?

Thank You!Thank You!

Friedrich H. Lehnfriedrich.lehn@fhlConsult.com

www.fhlConsult.com

Friedrich H. Lehnfriedrich.lehn@fhlConsult.com

www.fhlConsult.com

This presentation will be posted by tomorrow at: http://www.rational.com/ruc

This presentation will be posted by tomorrow at: http://www.rational.com/ruc