EXPLORING THE CAVERN OF DATA GOVERNANCE AUGUST 2013

Planning and Information Office | SIBI

Darren Dadley | Business Intelligence, Program Director

Data Management Overview

2

SIBI Program Methodology

Definitions: Data Management & Data Governance

The exercise of authority and control (planning,

monitoring, and enforcement) over the management of

data assets. (*)

3

Data Management

The planning, execution and oversight of policies, practices

and projects that acquire, control, protect, deliver, and

enhance the value of data and information assets. (*)

Data Governance

(*) DAMA International 2009

4

Data Governance Challenges – Key reasons for Failure (*)

Data Governance Overview

Data Governance Challenges

Failure to Execute

Lack of knowledge and Understanding by Senior Management (i.e. skills requirements, strategic

outcomes, process improvement) leads to a

failure to execute.

Lack of Ownership

Ownership, responsibility and accountability not

assigned.

Lack of Awareness

Executives and key stakeholders of data

management capabilities have a lack

of knowledge and awareness of DG.

Lack of Accountability

Accountability not assigned to each

process

Task is overwhelming

DG is too big for any one person to accomplish.

Adequate resources are not assigned.

(*) Adapted from 2011 Baseline Consulting Group, Inc.

- Training

- Education

- Communications

- Workshops

- Assign sponsor

- DG Forums

- Personal

development

plans

- KPIs

- Education

- Best practices

- Bench marking

- Leverage other

successes

- RACI

- Data stewards

- Personal

development plans

- KPIs

- Pilot projects

- Series of

manageable

projects

- Identify key areas

of concern

- Split the tasks

- Identify and assign

resources

Data Governance Strategy

5

What is Data Governance

for the University

Develop processes

Identify a key initiative

as a Pilot

Define KPIs as measures for success

Educate and engage

stakeholders

Document improvements and processes

Communicate success

SUCCESSFUL DATA

GOVERNANCE

Managing Expectations

• Develop DG

vision

statement in

line with

University’s

strategic vision

• Define DG

• Scope DG with

context of

University

• Define Data

Governance

Framework

• Define DG

organisation

• Define roles and

responsibilities

(RACI)

• Select Data

Management

Pilot area

• Workshop to

identify and

develop KPIs

• Determine

accountability

for KPIs

• Identify KPI

benefits and

ROI

• Define Pilot

group

• Develop training

plan

• Develop

communications

and engagement

plan

• Educate

stakeholders

about DG

• Review and

define process

maps

• Establish SOPs

(standard

operating

procedures)

• Develop review

process

• Communicate

success to key

stakeholders and

broader audience

(email, bulletin,

newsletters)

Data Governance vs. Data Management

6

Data Governance (Organisation and Activities)

Strategy

Organisation and roles

Deliverables and standards

Projects and services

Issues management

Creating guiding principles

Data asset valuation

Data Management (Execution)

Data profiling

Data quality monitoring

Data cleansing

Semantic rules

Data enrichment

Business rules creation &

maintenance

Enterprise data modeling

Metadata definition

Business glossary definition

Data archival

Backup and Recovery

Authentication

• Provide Guidance

• Create & Implement

Deliverables

• Provide Feedback

• Track Progress

Data Management Overview

7

(DMBOK) Data Management Functions •Analysis •Measurement •Improvement

•Architecture •Integration •Control •Delivery

•Acquisition & Storage •Backup & Recovery •Content Management •Retrieval •Retention

•Architecture •Implementation •Training & Support •Monitoring and Tuning

•Acquisition •Recovery •Tuning •Retention •Purging

•External Codes & Internal Codes •Customer Data •Product Data •Dimension Management

•Enterprise Data Modelling •Value Chain Analysis

•Data Modelling •Database Design •SDLC •Implementation

•Standards •Classification •Administration •Authentication •Auditing

Data Architecture Management

Data Development

Data Security Management

Data Warehousing &

Business Intelligence

Management

Document & Content

Management

Data Quality Management

Reference & Master Data Management

Meta Data Management

Database Operations

Management

Data

Governance

Data Management Overview

› Data Governance

› Data Security Management – Data Visibility

› Data Quality and Data Profiling

› Master Data Management

› Metadata Management & Business Glossary

8

Current focus for SIBI

University of Sydney

Data Management

Data Governance

SIBI

Data Management Overview

9

DMBOK – 7 Environmental Elements

People

Process Technology

• Organisation & Culture

• Roles & Responsibilities

• Goals & Principles

• Activities

• Deliverables

• Practices & Techniques

• Technology

Provide a consistent way to describe and strategically plan each function Technology

Roles &

Responsibilities

Goals & Principles

Organisation &

Culture

Strategy

Deliv

era

ble

s

Acti

vit

ies

Pra

cti

ces &

Tech

niq

ues

Un

ive

rsit

y o

f S

yd

ney D

ata

Ma

nag

em

en

t F

ram

ew

ork

Data Management Overview

10

DMBOK – 7 Environmental Elements

› Goals & Principles – The directional business goals of each function and the fundamental principles that guide performance

of each function.

› Activities - Each function is further decomposed into lower level activities (tasks and steps)

› Deliverables - The information and physical databases and documents created as interim and final outputs of each function.

Some are considered essential, some are generally recommended, and others are optional depending on circumstances.

› Roles and Responsibilities - The business and IT roles involved in performing and supervising the function and the specific

responsibilities of each role in that function. Many roles will participate in multiple functions.

› Practices & Procedures - Common and popular methods and techniques used to perform the processes and produce the

deliverables. Risks and issues management.

› Technology - Categories of supporting technology (primarily software tools), standards and protocols, product selection

criteria and common learning curves..

› Organisation and Culture - These issues might include:

- Reporting Structures, Teamwork and Group Dynamics

- Budgeting and Related Resource Allocation Issues

- Authority & Empowerment

- Shared Values, Beliefs, Expectations & Attitudes

- Change Management Recommendations

- User engagement: communications / training / education

Data Governance Overview

11

Data Governance – University Organisation & Culture

• Support the DGC, by implementing and refining the data ownership, data stewardship and data custodian roles throughout the University. • Provide Subject Matter Expert (SME) knowledge and support to the data governance strategy

• Own the data governance strategy • Promote, endorse and approve the development and enhancement of the data governance management framework

Data Owners Management

Group (DOMG)

Data Modellers Database

Administrators

Data Stewards

Data Integration

Specialists

Data Quality

Specialists

Supported by:

Information / Data

Architect

Data Governance Committee

(DGC)

Organisation

• Operating model

• Arbiters & escalations points

• Data Governance organisation members

• Roles & Responsibilities

• Terms of Reference

• Data ownership and responsibility

Deans of Faculties and Directors of

Professional services Units, e.g.

Finance, Research, HR, ICT

Directors, Heads of department,

Managers of functional areas

12

University Principles and Goals (recommended)

Data Management

Principles

Trusted

Valued

Shared

Re-used

Managed

Governed

Data Management Overview

Trusted. We trust in our information. Access to and use of data

will promote trust and confidence through adherence to relevant

Data Governance Policies and procedures, privacy, confidentiality

and security requirements.

Valued. Data is valued as a strategic resource and an asset. As a

result, data and information will be of high quality, accurate,

relevant, timely and support confident business decisions.

Shared. Information and data is accessible, transparent and

available to be shared as part of the University’s sharing of

information obligations to; the community, staff, students,

researchers and alumni.

Re-Used. Data and information should be obtained from a single

authoritative source. Data and information is collected in a

consistent manner and is available to be used for different

purposes with confidence.

Managed. Data and information is managed throughout its

lifecycle and is compliant. Information Management Procedures

and practices are standardised and applied across the University

and apply to all involved in the data management lifecycle.

Governed. Data and information is governed in accordance with

the roles and responsibilities as defined in the University’s Data

Governance Framework, the University’s strategic goals and in

compliance with the requirements of Law.

13

Deliverables, Activities, Practices & Techniques

Data Management Overview

Data Management Overview

14

DMBOK Functions •Analysis •Measurement •Improvement

•Architecture •Integration •Control •Delivery

•Acquisition & Storage •Backup & Recovery •Content Management •Retrieval •Retention

•Architecture •Implementation •Training & Support •Monitoring and Tuning

•Acquisition •Recovery •Tuning •Retention •Purging

•External Codes & Internal Codes •Customer Data •Product Data •Dimension Management

•Enterprise Data Modelling •Value Chain Analysis

•Data Modelling •Database Design •SDLC •Implementation

Data Architecture Management

Data Development

Data Security Management

Data Warehousing &

Business Intelligence

Management

Document & Content

Management

Data Quality Management

Reference & Master Data Management

Meta Data Management

Database Operations

Management

•Standards •Classification •Administration •Authentication •Auditing

Data

Governance

Data Quality Management

15

Definition

Planning, implementation and control activities that apply quality

management techniques to measure, assess, improve and ensure the

fitness of data for use.*

*Source: DAMA-MBOK 2009

Communication

Pri

nci

ple

s

Organisation & Culture

Roles and Responsibilities

Data Quality Management Framework – HR Pilot

Accuracy

Completeness

Integrity

Timeliness

Validity

Consistency

Issues Log

Risk Matrix

Critical success factors

Authority & Empowerment

Information Compliance

Data Privacy

Govt. Legislation

Internal Audit

Roles

Forums

Data Custodian

Data Owner

Sponsor Data Steward

SIBI Program Board

BOG

Expectations & Attitudes

Pilot group structure

Change Management

Technology: Data Profiling (Informatica), Data cleansing (IDQ-Informatica)

University of Sydney Vision

Goals

*** Develop vision for Data

Quality Mgmt. and for Pilot

with HR data. (workshop)

User Engagement

Education Comms

Deliverables

Activities

Practices & Techniques

External Audit

Data Quality Management

17

Data Quality Dimensions

• Does the data accurately represent reality or a verifiable source?

Accuracy

• Is all necessary data present? Completeness

• Are all data elements consistently defined and understood?

Consistency

• Is the structure of data and relationships among entities and attributes maintained consistently?

Integrity

• Is data available when needed? Timeliness

• Do data values fall within acceptable ranges defined by the business?

Validity

Data Quality Methodology - Roadmap

18

2. Define DQ Requirements

Activities

Deliverables

Technology

3. Profile, Analyse & Assess DQ

IDE – Informatica Data Profiling tool

Baseline

Updated Issue Log

Scorecard Report

IDQ – Informatica Data Quality tool

Recommend Actions

Actions:

- Training / education / comms

- Business Processes

Improvement (SOPs)

- Data Validation (data entry

process)

Data Issue Log

Enables data profiling and analysis with the flexibility to filter and drill down on specific records for better detection of problems.

4/5.Define DQ metrics & Business rules

Enables architects and developers to discover and access all data sources, to improve the process of analyzing, profiling, validating, and cleansing data.

1. Promote DQ

Awareness 6. Test &

validate DQ Requirem.

7. Set & evaluate DQ service levels

10. Clean & correct DQ

defects

11. Design and implement DQM

procedures (SOPs)

Control

Activities

8. Continuously measure and monitor DQ

9. Manage DQ issues 12. Monitor operational DQM procedures and performance

Identify known data

issues

Extract & provide

data

Activities for DQ Pilot Activities for DQ methodology

Vision statement

Data files

DQM Framework

RACI

19

Next Steps

Data Management Overview