101 Lightbulb Moments in Data Management

7/25/2019 101 Lightbulb Moments in Data Management

1/16

!"#$"% '()*$+,$%-. //0

/1+ 23-1+4 536171

0"%#8$)(#"8+9

!"## %&'()

!"* +,--".

%,/ !0/1.

%,2"3 40.("/

!0&'1 50--".670/8,/,-"

9"'( 7:-/,/1

;("#


2/16

In God we trust; all others must bring data.

W. Edwards Deming (19001993)

101 Lightbulb Moments in Data Management

Sample


3/16

xi

!"! $%&'()*+) ,-./0(1 %0 23(3 ,303&/./0(

!"#$%"#&

Alightbulb moment occurs at the point where inspiration meetsaction. Sometimes the inspiration is immediate and unexpected.

Ofen, however, inspiration comes afer spending a signicant

amount of time researching a problem and searching for a solution. Tere

can be both failures and frustration before a breakthrough. But a lightbulb

moment marks the point where things start falling into place and results

start becoming visible.

Tink aboutTomas Edison. He enjoyed many lightbulb moments dur-

ing his life, demonstrated by the 1,093 U.S. patents he holds for inventions

that include a stock ticker, a mechanical vote recorder, a batery for an elec-

tric car, electrical power, recorded music, and motion pictures. For one of

his rst groundbreaking inventions, Edison took the knowledge from other

inventors who had already produced electric lightbulbs and combined it

with his own idea to use a carbon lament inside the bulb. Te result was

the rst commercially viable incandescent light. It was a perfect example of

inspiration combined with hard work and experimentation.

Tis book, 101 Lightbulb Moments in Data Management, contains

bright ideas that can help you overcome the short circuits in your data

management projects and build on your inspirations. It provides new

thoughts and approaches that you can apply to your organization as you

transform your corporate data into a valuable asset.

Te contributors to this book have truly been there and done that

in the data management world.Tey are noted contributors to the Data

Roundtable, a thought leadership site sponsored by DataFlux. I have had

the opportunity to work with them over the years, both as colleagues


4/16

xii | 101 Lightbulb Moments in Data Management

and partners.Tey are the data contributors that companies turn to with

the thorniest data management challenges. Tey understand the powerof data and its importance in driving a business forward. Today, perhaps

more than ever, companies need to harness that power. Properly managed

data will help you make beter decisions.

I hope you enjoy these lightbulb moments and that this book serves

as the lament of your own bright ideas.


5/16

!"#$ &'

("$" )"*"+,-,*$

Organizations continue to struggle managing their data. For all

sorts of reasons, effective data management seems to elude compa-

nies of every size. This is particularly difficult as organizations amass

moreand more types ofdata. The rise of unstructured data pres-

ents organizations with unique challenges, from both management

and data mining perspectives. What good is data if you cant use it,

access it, and ultimately make better decisions from it?

Fret not, as the contributors answer important questions like theseand provide compelling insights in this section.


6/16

!"#$ &' ()# *$+$ ,-$.$/0)# !12345/67 ,8# 93: ;#$"


7/16

2 | 101 Lightbulb Moments in Data Management

linearly, the actual impact on the workforce and service performs degradesin a nonlinear fashion. If we have a nonlinear rise in defects coupled with

a nonlinear rise in service impact, what will this mean for customer-facing

their poor customer experience.

Tis is not a problem that technology alone can resolve. For mostcompanies it requires a seismic cultural shif in how every person in theorganization perceives and man-ages data. Systems and informationtechnology (IT) infrastructure can

evolve endlessly, but you cant buya culture of data quality. It takestimesometimes many yearstocreate a vision, a sense of urgency,and an ability to adapt.

Consider that by 2013, some

analysts suggest that mobile commerce is set to double in the UK alone. In

other countries, the gures are much higher.

So when youre considering rejection of that business case for data

quality or data governance, consider the data landscape in ve years time.

!"#$ &' ()# *#+#,-.$/01#" *$-$23$/0-4 567#8#,-Joyce Norris-Montanari

Does it work well for a corporation if each department has its own data

-

partment knows it needs to pay atention to data quality, but the corpora-

tion hasnt decided how to manage the quality of the data across the organi-zation. Departments end up handling data quality on their own as a result.

Centralized data quality would come from the enterprise data manage-

ment group, but most of these groups are not thinking about master data or

corporate data. Employees dont have time to manage a platform for data

!" "$%&' "()&*'+)&"()&'

)$,- -&$.'*"+ /.&$"&$ 0('(+,1 $ '&,'& +2 3.4

5&,/-1 $,6 $, $7(8("- "+

$6$9":


8/16

Part I: Data Management | 3

quality and data integration for the

corporation. What that means is thatmost are not ready for a corporateinitiative and are really not sure howto go about implementing one.

Would it help if employees hadthe following?

Enterprise guidelines for de-

partmental data management:For example, how do employ-ees use enterprise metadata about data, and how do they contrib-ute information for consumption?

Enterprise sofware to manage the data quality and dataintegration:Tis would require a single platform for all themetadataa platform that could eliminate re-creation of thesame data analysis metadata.

Personnel who really understand the concepts of data man-

agement for the enterprise:Tis is the hard one. Personnelthink they understand how to do this, but theyve never reallyimplemented data management for a large organization.

A roadmap is needed for how to bite offsmall chunks for implementa-tion. Tis requires management buy-in and corporate participation. Fur-

thermore, corporate participation requires education and adherence.

!"#$ &' ($)*#+ ,$*$

-./).01"$*1./

David Loshin

I have long advised clients that adopting a master data approach with-out having a master data strategy may lead to uncontrolled data consolida-tion without the benet of the right kinds of controls to prevent confu-sion. Here is a good example: the offices for Knowledge Integrity are atone location, while we live at another location. Our company has a 401(k)plan with an investment company, and as president of the company, I am

!"#$ &'( )"$ '(&*+ ,"' &

-"'."'&$( /)/$/&$/0( &)*

&'( '(&11+ )"$ #2'( 3"4

$" 5" &6"2$ /7.1(7()$8

/)5 ")(9


9/16


a trustee for the 401(k) plan.Tere-

fore, mail with my name on it associ-ated with the plan is supposed to goto our business address. However, asan employee of my own company, Ialso have a 401(k) account.Te mailassociated with that account shouldgo to our home address.

However, I found that all the mailthat had my name on it was going tothe business address (both account and plan correspondence). I contacted

the company and explained that I expected my account records to be sentto my home account. No problem, the rep said, while changing the ad-dress for my correspondence to the home address. Te next thing I knew,the 401(k) plan correspondence also started coming to the home address.So I called the 401(k) company back and requested that the business cor-respondence go to the business address. No problem, the rep said whilechanging the address for my correspondence to the business address. Ofcourse, now the mail is back to the way it was beforeall leters come tothe business address.

Of course, I know what is going on here: the companys customer con-

solidation process does not allow for an individual to operate in more thana single role when it comes to location/contact mechanism. Changing onecontact mechanism associated with the unique entity changes it for all rolesthe entity plays.Tis is a failure in the companys modeling process since itis unwilling to accommodate multiple roles for unique entities.

!"#$ &' ()# !*+,-.$/0# ,120,+# 3/ 4$.$ 56$73.8 911,-.:

Jill Dych

When it comes to data quality, I fervently believe that it is destined for

widespread adoption. As a concept, data quality has been around for a while,

but until now its only truly been appreciated by a group of acionados. But

!"#$%&'( * +*,%-. "*%**$$.#*/0 1&%0#2% 0*3&'(

* +*,%-. "*%* ,%.*%-(4

+*4 5-*" %# 2'/#'%.#55-"

"*%* /#',#5&"*%' 1&%06

out the benet of the

.&(0% 7&'", #8 /#'%.#5, %#

$.-3-'% /#'82,'9


10/16


just like taco trucks, the HBO show In Treatment, video on demand, and

Adam Lambert, data qualitys best days are actually ahead of it.Part of the reason data quality hasnt yet hit its stride is because it re-

mains a difficult sell. Tose of us in the business intelligence and data in-tegration communities understand that accurate and meaningful data is a

business issue. And well intentioned though they may be, IT people havegone about making the pitch the wrong way.

Wevendors, consultants, and practitioners in the IT communityblather on about data quality being a business issue and requiring a busi-ness case and a repeatable set of processes, but at the end of the day auto-mation remains the center of most data quality discussions. As executives

try to explain the return on investment (ROI) of name and address correc-tion, deterministic matching, multisource data proling, and the pros andcons of the cloud, they are thinking two things:

would we begin?In fact, the topic of scope is a huge gaping hole in the data quality con-

versation. As we work with clients on seting up data governance, we ofenuse the bad reputation of corporate data as its pretext. We always, alwaystalk about the boundaries of the initial data quality effort. Unless you can

circumscribe the scope of data quality, you cant quantify its value.In our experience, there are ve levels of data quality delivery that can

quickly establish not only the scope of an initial data quality effort but theactual duties and resources involved in the initial project (see Figure 1 onpage 6).

Were also more likely to be solv-ing a real-life problem.Tus, we makethe initial win much more impactful,thus securing stakeholder participa-

data quality eff

ort is not necessarilywhere well nish, so we can ensurean incremental approach to seting up the program and its roles.

By specifying the initial scope of the data to be corrected, were establish-

ing the boundaries of the effort itself.

!" $%&'()"(*+ ,-& (*(,(./

$'0%& 0) ,-& 1.,. ,0 2&

'033&',&14 5&63& &$,.27

/($-(*+ ,-& 208*1.3(&$ 0)

,-& &))03, (,$&/)9


11/16


!"#$%& () !"*& +&*&,- ./ 0121 3$1,"24 0&,"*&%4

Business executives and users can consume a well-scoped problem,

especially if it makes their jobs easy or propels progress. And if we solve it

in a way that benets the businesseliminating risk, ensuring economies

of scale, and driving revenueswe might even get budget for a data qual-ity tool!

!"#$ &' ()# *#+#,-.#$/ 01234)

Jim Harris

In 2003, I purchased a townhouse in Ankeny, a northern suburb of

which, in the United States, meant that I didnt need to purchase privatemortgage insurance. I then secured a 7/23 adjustable-rate mortgage, which

meant Id have a xed interest rate for the rst seven years of a thirty-year

mortgage that would thereafer adjust on an annual basis. I also purchased

a homeowners insurance policy.


12/16

With the initial interest rate on my current loan due to expire at the end of

this year, my current mortgage company began sending me leters of notica-tion. One of the leters informed me that I had never secured the appropriate

amount of insurance for my townhouse and, as it was my responsibility to

protect the insurance companys nancial best interests, if I did not remedy

this situation, I was at risk of foreclosure. I ignored this leter because, as I

previously explained, I had secured the appropriate amount of insurance.

A follow-up leter informed me that, out of the glowing kindness of themortgage companys heart (and apparently with a passing consideration ofthe fact that I had never been late with a single mortgage payment in seven

years), it would not foreclose on me and would not be evicting me from my

townhouse. (How very nice of them, eh?)Instead, the company would secure the appropriate amount of insur-

ance for my townhouse through its insurance companyand send me the insurance cost only $200 USD a year.) Terefore, I decided to take all theleters from my current mortgage company and go see my insurance agent,

who was able to resolve the problem within a few hours.Apparently, despite the fact that I had secured and continued to pay for

the appropriate amount of insurance for my townhouse, my current mort-gage company had no record of my ever having insurance.

Even though you would have thought this would have caused the mort-gage company concern far sooner (afer all, it was my responsibility to pro-tect its nancial best interests), apparently Te Seven-Year Glitch nally

became an itch that had to be scratched because my mortgage was aggedfor a pending interest rate adjustment (and you know that the adjustmentisnt going to make my payment amount decrease).

Te root cause of the problem was that, in a practice common in theUnited States, my mortgage was sold (and without my having any say in themater) by my original lender within the rst few months afer I purchasedmy townhouse.

Terefore, my current mortgage company had to integrate my acquireddata into its systems. Since my billing data seemed to have been successfully

integrated (not only was I receiving my mortgage bills from the new lender,but my property taxes were being paid via my new lender as well), I just as-sumed that everything was ne.



13/16


Afer all, the recent (and threat-

ening) leters were the only nonbill-ing correspondence I had receivedfrom my current mortgage companyin seven years. But one critical pieceof data got lost in the acquisitionmy insurance information. As I said,my insurance agent was able to re-solve the problem within a few hours.Ten a few days later, I received another leter from my current mortgagecompany. Tis leter quite cheerily informed me that I should contact its

insurance company if I needed any further assistance from them.Te mortgage company would be happy to explain its many excellent

insurance products that Id be sure to nd more than competitive with mycurrent insurance provider. (Apparently, that $2,000 USD a month insur-ance policy wasnt the only product that it offered in its extensive portfolio.)

I cant help but wonder if it will take the mortgage company anotherseven years to realize that it will soon no longer be my mortgage company.

!"#$ &' ()# !*"+$* $*"

,)# -../0

Phil Simon

When asked recently about his new puter a few hours before teeing -sponded, Its usually the Indian, not the arrow.

One could write a book about that statement.In golf, its misplaced to blame your poor performance exclusively on

your clubs because they dont swing themselves. Of course, that doesntstop many hacks from doing it, especially afer a second ball ies into the

woods. Golf companies spend tens of millions of dollars annually convinc-

ing 25 handicaps that they can radically transform their games by buyingthe latest and most expensive clubs. In actuality, the clubs dont mater

top-of-the-line irons, and I certainly wont be breaking 70 anytime soon.Im just not that good.

! #$%&' ()*+ ,-' ./%0)1 23

2' .2** '$4) '() 5/1'6$6)

#/5+$%7 $%/'()1 8)9)%

7)$18 '/ 1)$*2:) '($' 2'

.2** 8//% %/ */%6)1 ,) 57

5/1'6$6) #/5+$%7;


14/16

Te same holds true in the en-

terprise sofware space. Sure, its en-tirely unreasonable to expect peopleto track their sales of their best cus-tomers on pen and paper. Ive neverseen anyone in a large organizationmanually writing journal entries inan old-school ledger. Five years ago,however, I worked at one organiza-tion that wrote manual checks with acorporate checkbook. (Hey, HR and

payroll are ofen the last to get withthe program, right?)

SIMON SAYS

Buying powerful enterprise resource planning (ERP) and customer re- -ets and savings, especially when the organization had been on the rightof the technology adoption life cycle. Te same holds true for data qualityapplications. Teres absolutely no doubt in my mind that many organiza-tions would benet from buying them. Im hard-pressed to think of a moreunderserved area of IT. But buying them is not the same as deploying them

effectively and ensuring that employees use them in ways that make sense.Teres no difference between enterprise sofware and sporting equip-

ment. Yes, the best of each can help, but it all comes back to the Indianand the arrow. How you use either is paramount to what youre using.

!"#$ &' (#)#*+,-# ($.'/$-"01"2+3 4,-2#-2 5#2$1)6#7,8"93 5$2$ :;$)12


15/16


into allowing their telecommunications services to be slammed, as they say.

But once I was already preoccupied with the thought process, I started todo a litle reading.

One thing the telemarketer said was that the company had an inter-connection agreement with Verizon, so I went to the Internet (aka thesource of all knowledge) and searched for interconnection agreement.One of the results, the Wikipedia page for Interconnect Agreement, hadan interesting set of comments:

fees based on call source and destination, connection times

and duration.

dene.

Of course, these two statements ping-ponged in my head to lead to amore basic question: what is a telephone call?

Actually, as with many commonly used phrases, the term itself is be-coming somewhat of an anachronism. Again, Wikipedia provides a litle

bit of information: A phone call is a connection over a telephone networkbetween the calling party and the called party. Today we might use a va-riety of devices and networks to establish a phone call. (Is a Skype calla phone call?) Calls are documented using a concept called a call detail

record (CDR), which contains the details associated with one of these con-nections, including the calling party, the receiving party, the start time, theduration, as well as a lot of other information that is to be used for billing.

Tis data is even more importantin the context of the interconnectionagreement, especially because themyriad calls made under the inter-connection agreements have to be in-

voiced back to the appropriate com-

simply, if I provide service throughyour network, you need to tell me what I owe you for using your network.Tis is where data quality becomes a real issue: validating invoice state-

ments against CDRs. In other words, when you charge me for a certainamount of bandwidth, how do I know that the amount you are billing me

!"#$ &'( )"*+,# -# .'+ *

)#+/*0$ *-'($/ '. 1*$23

402/"5 "'4 2' 6 7$'4

/"*/ /"# *-'($/ &'( *+#

10880$, -# .'+ 09 )'++#)/:


16/16

for is correct? To validate these numbers, you would need to verif y aggrega-

tion against the raw data. And then if there are discrepancies between mycalculations and yours, then what?

Basically, you need a third party to arbitrate by taking the raw data andthe invoices and then independently qualifying the data and resulting in-

voices and reporting back to both parties.Tis scenario is not unusual, andfundamentally its an example of a business model that is almost purely

based on data quality management.

!"#$ &' ()*#+,+-.# !)/0+1$*-0)2$)$3#1#)*453$-)Joyce Norris-Montanari

by the chief information officers (CIOs) of corporations. A few years ago governance issue. Guess what? You

interest in the IT area. It specializesin nding solutions for optimal useof information within organizations,such as to support corporate decisionsthat require the availability of data andinformation (tactical and strategic information). Te objective is to over-come traditional IT issues and manage information for the enterprise atthat level.

models and metadata

!"# %&'#()*+# *, )%

%+#-(%.# )-/0*)*%1/2 3!

*,,4#, /10 ./1/5# *16

7%-./)*%1 7%- )"# #1)#-6

8-*,# /) )"/) 2#+#29


101 Lightbulb Moments in Data Management

Documents

Transcript of 101 Lightbulb Moments in Data Management