AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With...

28
` AI-Powered Data Cataloging Virtual Summit Navigating Data Lineage With Rabobank and Informatica

Transcript of AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With...

Page 1: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

`

AI-Powered Data Cataloging Virtual Summit

Navigating Data Lineage

With Rabobank and Informatica

Page 2: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Requirements:

Data Quality

Data Governance

Data Analytics

Data Privacy and Security

Regulatory Compliance

Data Lineage traces data from source to destination, covering the entire lifecycle of data. It includes

information about changes to data during its journey.

Data Lineage: A Business Imperative

Page 3: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

3 © Informatica. Proprietary and Confidential.

Data Lineage: The Foundational Use Case

• Dev Operations: Change Management & Impact Analysis - what-if analyses for changes

• Operational Efficiency: Eliminate proliferation, duplication, data silos, reduce costs

• DW/Apps Modernization: Complete understanding of the data landscape to enable app modernization & cloud migration

…and AI use cases

• Explainable AI & Data Science Governance: Track and assess data used to train models, govern AI projects. Support Explainable AI. Ensure training data variety.

Increasingly “IT” use cases are coming to the forefront…

Page 4: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

4 © Informatica. Proprietary and Confidential.

Data Lineage: Key Informatica CapabilitiesUser experience is key to understanding lineage, custom views at all levels and detail for every use case

• Business and logical views

• Dataset level lineage

• Detailed field level lineage

• Drill down details on data transformation logic

• Summarized, level-wise views

Page 5: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

5 © Informatica. Proprietary and Confidential.

Data Lineage: Key Informatica CapabilitiesComprehensive lineage to support all use cases requires wide and deep metadata connectivity as well as active metadata-driven intelligence

• Automatic data lineage stitching

• Discovered and curated

• Impact analysis with shareable exports

• Automatic lineage derivation from code: SQL Scripts, stored procedures, BI reports, ETL jobs & mappings

• Change notifications

• Data Similarity discovery

• Other relationships (joins, usage)

Page 6: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Gaurav Pathak Anil Bandarupali

Speakers

Senior Director,Product Management

Senior Software Engineer, Data Management

Page 7: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Rabobank experience with EDC

Anil Bandarupalli

Enterprise Data Catalog

Page 8: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

• Introduction to Rabobank

• Rabobank vision & strategy

• Data Management Office

• Data Governance team

• Scope of Data Lineage

• Data Lineage use case#1

• Data Lineage use case#2

• Lessons learned

• EDC next steps for Rabobank

Reading Guide

8

Page 9: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

2nd largerst bank in Netherlands101 local Rabobanks409 offices1.9 million members6.5 million private customers0.8 million commercial customers

Rabobank in the Netherlands

Mission Growing a better world together

Domestic Retail Banking

Almost 8.3 millioncustomers

7.3 million Dutch customers 1.0 million international customers

41 countriesInternational

Market Leader in Financing, Food and Agriculture

Private Sector Lending to Trade, Industry And Services

Rabobank at a GlanceSituation on December 31, 2018

9

Page 10: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Growing a Better World Together

10 StrategicTop Priorities

• 100% Digitalconvenience in everything

• Top customeradvice nearby

• Growth with innovation

• Top performance• Optimal balance sheet• Exceptionally good

execution

• Focus on social responsibility and sustainable contribution

• Involved members andcommunities

• Inspired employees• One-Rabobank culture

HOW?Through our values and behaviours

I dare to make a difference for the world

I make you betterI am doing the right thing

exceptionally wellI go the extra mile

for my clients

Banking for the Netherlands Banking for Food

Excellent Customer Focus

Rock-solidBank

EmpoweredEmployees

Meaningful Cooperative

We are client-driven and action-oriented

We are purposeful and courageous

We are professional and considerate

We bring out the best in each other and keep learning

10

Page 11: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Data Management Office

11

Data Management Office

Dat

a Li

neag

e

Dat

a A

rchi

tect

ure

Dat

a Q

ualit

y

Dat

a D

efin

itio

ns

Ente

rpri

se D

ata

Rec

ord

Keep

ing

Ref

eren

ce D

ata

Man

agem

ent

Dat

a Ex

chan

ge

Proc

ess

& C

ontr

ols

The Data Management Office supports the data governance structure within Rabobank.

The Rabobank Managing Board has mandated the Data Governance Board to increase the management of data with the following goals:• Define, approve and communicate data

strategies, data policies, data standards, data architecture, procedures and metrics

• Track and enforce compliancy of data policies, standards and architecture

• Initiate, track and oversee the delivery of data management projects and services

• Manage the resolution of data related issues

• Increase understanding and promote the value of data assets

Page 12: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Data Management & Data Logistics Department

12

Data Management

& Data Logistics

Data Quality

Page 13: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Why Data Lineage So Important?

13

Having Data Lineage for Enterprise will create business value in four areas:

Regulatory Requirements

Use end-to-end lineage BCBS 239 and other regulatory mandates

Data Quality Management

Data lineage improves the overall data quality by reducing the duration of root cause analyses process of Data Quality Issues

Change management

Provides the possibility to view the impact of proposed changes to the data

integration environment

Data Integration

Creates a better understanding what the data means, where it came from, where it is used

and how it has been transformed

Business Value of Data Lineage

Page 14: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Group Data layer

Group Reporting and Calculation

** ** ******

**** **** **

** **

GROUP

** ** **

****

**

**

**

**

Group Risk and Finance sources

**

****

****

**

****

** **

****

******

** **

**

Group

** **

LBB Risk LBB Finance

**

**

**

**

**

**

**

**

****

**

**

**

** ** **

**

**

**

**

BU Reporting and Calculation

BU Data Layer

BU Risk or finance sourcesWRR

** Data Exchange function

**

Data Warehouse level

Back / Mid office level

Front office level

FinancierenSparen Beatalingsverkeer Klant

** **

**

**

**

**

**

** **

**

**

** **

**

**

**

**

** **

**

**

**

**

****

* *

**

**

********

Dat

a Fl

ow

** System names are masked for security reason

PowerCenter

Scope of Systems f0r Data Lineage

14

Page 15: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

EDC — GUI interface

15

Page 16: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Using EDC for ComplianceUse case for BCBS#239 compliance

*BCBS#239 is Regulatory compliance on Data Governance

Page 17: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Manual vs Automated Lineage

17

Existing situation manual lineage in Excel

Automated lineage in EDC

Page 18: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Understanding Lineage Can Be Too Complex

18

Page 19: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Deriving Business Lineage Out of EDC

19

Page 20: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

EDC for Data GovernanceUse Case, Integrating EDC into Data Governance Center

Page 21: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Integrated View in Data Governance Center

21

Data Lineage starts in EDC

Reporting Repository

Business Data Element

Logical Data Attribute

Physical Attribute

FIND

YOU

R DATA

Page 22: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Horizontal Lineage in EDC— Technical Lineage

22

FOLLOW YOUR DATA

Page 23: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

• EDC catalog search is very powerful like Google for enterprise catalog

• EDC has a big list of connectors

• EDC has open API, makes it easy to integrate with other tools

• Technical data lineage can be too much information for business users

• When you are looking at a big lineage diagrams, it is not possible to restrict lineage for selected tables

Lessons Learned

23

Page 24: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

• Continue expanding data lineage for enterprise

• Adding business terms to data attributes

• Using data domain discovery for privacy regulations

• Deriving business lineage out of EDC

EDC Roadmap for Rabobank

24

Page 25: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

2525

Bedankt voor uw aandacht!

Page 26: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

Demo

Page 27: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

27 © Informatica. Proprietary and Confidential.27 © Informatica. Proprietary and Confidential.

Learn More

1. Don’t miss Keynotes and Deep-Dives at the AI-Powered Data Cataloging Virtual Summit:• Market and Analyst Perspectives featuring New York Life, Tableau, and Amalgam Insights

• Data Cataloging Solution Theaters featuring Maersk, Nissan, Rabobank and Biogen

2. Stop by an Informatica World Tour near you:• Chicago Sept-11 | Washington, DC Oct-15

• Frankfurt Oct-8 | London Oct-9 | Paris Oct-10

3. Watch a Product Webinar:• Advancing Analytics Maturity with an Intelligent Data Catalog: with Mattel and Aberdeen

• Meet the Expert PM Webinar: EDC 10.2.2 Release Deep-Dive & Demo

Page 28: AI-Powered Data Cataloging Virtual Summit Navigating Data ... · Navigating Data Lineage. With Rabobank and Informatica. Requirements: Data Quality. Data Governance. ... • Data

`

Thank You