DEMOCRATIZING AND COMMODITIZING DATA IN BANKING USING DATA VIRTUALIZATION TO ELIMINATE SPREADSHEETS

Peter Memon Executive Director

New Product Development

Investment Banking - The Overview

Complex….very complex.

Different business the function independently.

High level of business and technology change.

Regulations have added a new level of complexity.

Huge demand for data.

We Need To Measure More

Don’t banks already measure everything???

New metrics requiring different data that measure something else!

Regulations have significantly increased our need to calculate new metrics.

Huge demand for data!

Data, Data Everywhere…

Generate an enormous amount of data daily (e.g. Market Data, Trades, Orders,

Research etc.)

A single business line easily equates to hundreds of different sources and

formats.

Technologists are REALLY good at creating data diversity!

How we do it today

Build a big data warehouse that tries to consume everything.

Spreadsheets everywhere.

People. Lots of people.

Try to model everything in a common schema.

Think of it and we have tried it.

These approaches….

They don’t scale.

Inflexible.

Costly.

Implementation time.

Lots and lots of transformations.

The Regulators

Asking for new consolidated cross asset metrics.

Provide additional behavioral insight.

The Problem

Over twenty-five very unique data sources.

Billions of rows of data.

Systems located across the globe.

Different architectures.

The Opportunity

Regulatory projects create excellent opportunities.

Management is very engaged.

Want the ‘problem’ to go away.

Rapidly gain access to many sources with little impact on application teams.

Not trying to solve….

Not trying to build a complex cross asset data model.

Be all things to all consumers.

Not spending time cleaning data but…

Solution

Need to build domain specific views of complex data.

Quickly.

Solution

Required data from the source transactional systems.

Not from a copy of a copy of a copy.

Not from a source where the data has gone through numerous transformations.

How did we get there?

Worked with application owners to acquire connectivity.

That was the hard part.

Single person was responsible for connectivity and view creation.

Domain knowledge was critical.

More of how we got there

Cisco engineer onsite.

Minimal impact on the application teams once we acquired database access.

Almost entirely self serviced.

Domain knowledge is critical.

Data Quality

Data quality wasn’t a stated goal but a positive side effect.

Data transparency leads to cleaner data.

When you get a spreadsheet of pre-aggregated data you cannot validate quality and lineage.

Process Issues

Another side effect is that we exposed problems with data in the existing

processes.

Also exposed process issues.

Implementation

Acquired access to required data sources.

Designed a canonical model that fit the requirements of the problem.

Normalized data in CIS.

Implementation

Formatting views were cached with the original source.

High volume, low latency sources not cached.

Incremental data from sources cached daily.

Business views cached and refreshed daily in an Oracle database.

Data transformations/normalizations in the logical tier.

Implementation

Data situated in Oracle, Sybase, files, NOSQL, proprietary database and DB2.

Data transformed into a canonical data model within CIS.

Tibco Spotfire used for data visualizations.

Web UI in Spotfire and persisted results into CIS.

Optimization

NOSQL, relational, non-relational, web services and proprietary data sources.

Fairly extensive data transformations done in CIS.

Query optimization is absolutely mandatory.

Strong database resource is critical.

Infrastructure

Environments - DEV, UAT, PROD and DR.

Large database class servers

Automatic failover, database replication and load balancing.

Push most changes programmatically.

Business Case

Manual data aggregation is both people and process intensive.

Reduction in data errors through transparency.

Significantly reduce the time it takes to identify issues.

Single virtual source of data in one location.

Data is now easily available to support business functions.

Where Next?

Hadoop.

APIs.

Many, many other opportunities.

Cross asset data analytics and visualization. (We have access to everything and we should use it)

Questions?

Your turn…

Disclaimer

The views expressed herein do not necessarily represent the views and opinions of JPMorgan Chase & Co. This material is provided for information only and is not intended as a recommendation or an offer or solicitation for the purchase or sale of any security or other financial instrument. In no event shall JPMorgan be liable for any use by any party of, for any decision made or action taken by any party in reliance upon, or for any inaccuracies or errors in, or omissions from, the information contained herein and such information may not be relied upon by you in evaluating the merits of participating in any transaction. JPMorgan and its affiliates may have positions (long or short), effect transactions or make markets in securities or financial instruments mentioned herein, or provide advice or loans to, or participate in the underwriting or restructuring of the obligations of, issuers mentioned herein. Nothing in these materials constitutes a commitment by JPMorgan or any of its affiliates to enter into any transaction. Clients should contact their salesperson at, and execute transactions through, a JPMorgan entity qualified in their home jurisdiction unless governing law permits otherwise. JPMorgan is the marketing name for the investment banking activities of JPMorgan Chase & Co. and its subsidiaries and affiliates worldwide. J.P. Morgan Securities LLC is a member of FINRA, NYSE and SIPC. Copyright 2015 JPMorgan Chase & Co. All rights reserved.