Chun-Ying ChuangChun-Ying Chuang

Deep PatelDeep Patel

Hsin-Ying LiuHsin-Ying Liu

Liang-Yu HuangLiang-Yu Huang

Yi-Chien LeeYi-Chien Lee

Yi-Feng HanYi-Feng Han

THE (OFTEN HIDDEN) COSTS OF POOR DATA

AND INFORMATION

Subprime mortgage meltdown

INTRODUCTION

Incorrect personal dataInaccurate credit scoresUnfamiliar with the

terms of their mortgage

Over relied on incorrect investment grade

Foreclosure proceeding

Subprime mortgage meltdownThe 2000 presidential electionMars spacecraftFort Monmouth closingTrader assistant’s errorMisgraded SATsJésica Santillán and Kaiser

PermananteDepartment of Veteran Affairs

laptopLaw enforcement and 9/11Financial reportingThe 2000 censusIntelligence

THE DIRTY DOZEN

SEVEN COMMON DATA QUALITY ISSUES

People can’t find the data they need

Incorrect dataPoor data definitionData privacy/data securityData inconsistency across sources [data “REDUNDANCY”

Too much dataOrganizational confusion

Data definitionsTechnical details of their physical storageWho may use them and permitted usagesWho may not use them and prohibited usagesTheir original sources Systems that store themDetails for gaining accessDomains that the values may takeDefinitions of what those values may meanConventions for naming the data and file

structures

METADATA

THE COST OF POOR DATA

OPERATIONS

Higher operating costLower employee moraleLower customer satisfaction

Item Unit Cost Item Total

88 “good” order

$1.00 $88.00

12 “bad” order

$10.00 $120.00

Grand Total $208.00

Harder to set & execute Fewer options to derive the required value

from data Distracts management attention Harder to align organization Strategy cannot be executed without full

support

STRATEGY

DECISION MAKING/ TACTICS

Lower trust between organizationsOrganizations tend to make poor

decisionsTend to lose salesTechnology risk increasesHarder to manage risk

DATA & IMPACTS

Current data status

Misinterpreted

30% not yield

10-25% errors

Inconsistent with sources

May be loss & theft.

Immediate consequences

Wrong or delayed decisions

30% of time on searching

Increase operation costs

Redundant or obsolete data

Political battles, failed system

Potential business lost 20%

Feared long-term consequences

Organization cannot:

Trust the data

Find new uses of data

Mine the data

Develop data-specific strategies

WHAT CAN WE DO

Pay attenteion to data in your job

What data are most important?

How good are they? Are theyccurent? Well-defined? How do we know?

How much easier if the data are 100% correct?

What to do about it?

Q & A