Data-Ed: Data Systems Integration & Business Value Pt. 3: Warehousing
-
Upload
dataversity -
Category
Technology
-
view
642 -
download
1
description
Transcript of Data-Ed: Data Systems Integration & Business Value Pt. 3: Warehousing
Copyright 2013 by Data Blueprint
Data Systems Integration & Business Value Part 3: WarehousingCertain systems are more data focused than others. Usually their primary focus is on accomplishing integration of disparate data. In these cases, failure is most often attributable to the adoption of a single pillar (silver bullet). The three webinars in the Data Systems Integration and Business Value series are designed to illustrate that good systems development more often depends on at least three DM disciplines (pie wedges) in order to provide a solid foundation. Integrating data across systems has been a perpetual challenge. Unfortunately, the current technology-focused solutions have not helped IT to improve its dismal project success statistics. Data warehouses, BI implementations, and general analytical efforts achieve the same levels of success as other IT projects – approximately 1/3rd are considered successes when measured against price, schedule, or functionality objectives. The first step is determining the appropriate analysis approach to the data system integration challenge. The second step is understanding the strengths and weaknesses of various approaches. Turns out that proper analysis at this stage makes actual technology selection far more accurate. Only when these are accomplished can proper matching between problem and capabilities be achieved as the third step and true business value be delivered.
Date: September 10, 2013Time: 2:00 PM ET/11:00 AM PTPresenter: Peter Aiken, Ph.D.
1
Copyright 2013 by Data Blueprint
Commonly Asked Questions
1) Will I get copies of the slides after the event?
2) Is this being recorded so I can view it afterwards?
2
Copyright 2013 by Data Blueprint
Get Social With Us!
Live Twitter FeedJoin the conversation!
Follow us: @datablueprint
@paikenAsk questions and submit your
comments: #dataed
Like Us on Facebookwww.facebook.com/datablueprint
Post questions and commentsFind industry news, insightful
content and event updates.
Join the GroupData Management & Business
IntelligenceAsk questions, gain insights and
collaborate with fellow data management professionals
3
Copyright 2013 by Data Blueprint
4
Peter Aiken, PhD• 25+ years of experience in data
management• Multiple international awards &
recognition• Founder, Data Blueprint (datablueprint.com)
• Associate Professor of IS, VCU (vcu.edu)
• President, DAMA International (dama.org)
• 8 books and dozens of articles• Experienced w/ 500+ data
management practices in 20 countries• Multi-year immersions with
organizations as diverse as the US DoD, Nokia, Deutsche Bank, Wells Fargo, and the Commonwealth of Virginia
2
Data Systems Integration & Business Value Part 3: Warehousing
Presented by Peter Aiken, Ph.D.
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
6
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
7
Data Program Coordination
Feedback
DataDevelopment
Copyright 2013 by Data Blueprint
StandardData
Five Integrated DM Practice AreasOrganizational Strategies
Goals
BusinessData
Business Value
Application Models & Designs
Implementation
Direction
Guidance
8
OrganizationalData Integration
DataStewardship
Data SupportOperations
Data Asset Use
IntegratedModels
Leverage data in organizational activities
Data management processes andinfrastructure
Combining multipleassets to produceextra value
Organizational-entity subject area dataintegration
Provide reliable data access
Achieve sharing of data within a business area
Copyright 2013 by Data Blueprint
Five Integrated DM Practice AreasManage data coherently.
Share data across boundaries.
Assign responsibilities for data.Engineer data delivery systems.
Maintain data availability.
Data Program Coordination
Organizational Data Integration
Data Stewardship Data Development
Data Support Operations
9
Copyright 2013 by Data Blueprint
Hierarchy of Data Management Practices (after Maslow)
• 5 Data management practices areas / data management basics ...
• ... are necessary but insufficient prerequisites to organizational data leveraging applications that is self actualizing data or advanced data practices Basic Data Management Practices
– Data Program Management– Organizational Data Integration– Data Stewardship– Data Development– Data Support Operations
http://3.bp.blogspot.com/-ptl-9mAieuQ/T-idBt1YFmI/AAAAAAAABgw/Ib-nVkMmMEQ/s1600/maslows_hierarchy_of_needs.png
Advanced Data
Practices• MDM• Mining• Big Data• Analytics• Warehousing• SOA
Warehousing
• Data Management Body of Knowledge (DMBOK)– Published by DAMA International, the professional
association for Data Managers (40 chapters worldwide)
– Organized around primary data management functions focused around data delivery to the organization and several environmental elements
• Certified Data Management Professional (CDMP)– Series of 3 exams by DAMA International and
ICCP– Membership in a distinct group of
fellow professionals– Recognition for specialized knowledge in a
choice of 17 specialty areas– For more information, please visit:
• www.dama.org, www.iccp.org
Copyright 2013 by Data Blueprint
DAMA DM BoK & CDMP
11
Copyright 2013 by Data Blueprint
Series Context• Certain systems are more data
focused than others. Usually their primary focus is on accomplishing integration of disparate data. In these cases, failure is most often attributable to the adoption of a single technological pillar (silver bullet). The three webinars in the Data Systems Integration and Business Value series are designed to illustrate that good systems development more often depends on at least three DM disciplines (pie wedges) in order to provide a solid foundation.
• Data Systems Integration & Business Value – Pt. 1: Metadata Practices– Pt. 2: Cloud-based Integration– Pt. 3: Warehousing, et al.
12
Uses
Copyright 2013 by Data Blueprint
Part 1: Metadata Take Aways• Metadata unlocks the value of data, and therefore
requires management attention [Gartner 2011]
• Metadata is the language of data governance• Metadata defines the essence of integration challenges
SourcesMetadata Governance
Metadata Engineering
Metadata Delivery
Metadata Practices
MetadataStorage
13
Specialized Team Skills
Copyright 2013 by Data Blueprint
Part 2: Take Aways• Data governance, architecture,
quality, development maturity are necessary but insufficient prerequisites to successful data cloud implementation
• A variety of cloud options will influence cloud and data architectures in general– You must understand your architecture
and strategy in order to evaluate the options
• Data must be reengineered to be – Less– Better quality– More shareable – for the cloud
• Failure to do these will result in more business value for the cloud vendors/service providers and less for your organization
Copyright 2013 by Data Blueprint
Summary: Data Warehousing & Business Intelligence Management
15
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
16
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
17
Copyright 2013 by Data Blueprint
• Bank accounts are of varying value and risk
• Cube by – Social status– Geographical location– Net value, etc.
• Balance return on the loan with risk of default
18
• How to evaluate the portfolio as a whole?– Least risk loan may be to the very wealthy, but there are a very
limited number – Many poor customers, but greater risk
• Solution may combine types of analyses– When to lend, interest rate charged
Example: Portfolio Analysis
Copyright 2013 by Data Blueprint
Target Isn't Just Predicting Pregnancies
19
http://rmportal.performedia.com/node/1373
Copyright 2013 by Data Blueprint
15 years ago, CarMax started as a way to make the car buying experience simple, fair, and fun. Today CarMax is a FORTUNE 500 retailer and one of FORTUNE’s “100 Best Companies to Work For.” And we are hiring talented individuals who are interested in:--solving original, wide-ranging, and open-ended business problems--not only discovering new insights, but successfully implementing them--making a significant mark on a growing company--developing the fundamental skills for a rewarding business career
If that sounds like you, the Strategy Analyst position is the unique opportunity you’ve been looking for. The strategy team at CarMax currently consists of over 40 analysts, many of whom are recent college graduates from top schools with a variety of academic backgrounds (computer science, economics, English, engineering, journalism, math, political science). These analysts lead advances and decisions in several key business areas:-Inventory and pricing—what is the optimal selection of inventory, how do we acquire it, what should we pay for it, what should we price it for?-Expansion planning—which markets should we enter and how do we store those markets? Will each $10-30 million store investment generate a sufficient economic return?-Credit strategy—how can our bank (CarMax Auto Finance) approve more customers for loans and convert more approvals to sales?-Marketing and consumer insight—how do we reach our customers, increase traffic to our stores, and best use the internet to drive sales and build our brand-Industry and competitive research—what middle- and long-term risks are we exposed to, and how best do we prepare to respond? -Production—how do we increase vehicle reconditioning quality while reducing cost and production time?-Sales process and workforce—what is the best way to serve customers in our stores, and how do we manage, motivate and compensate our sales team?
Even early in your career at CarMax, you will have the responsibility to own an area of the business and will be expected to improve it. For example, one undergraduate recruit used data analysis to reformulate our retail pricing strategy, pitched and sold his idea to the senior executive team, and implemented a new system nationwide in his first 6 months with the company. That is the kind of impact you can make at CarMax. And as you do this, you will work closely with the senior executives and analytical managers to develop the fundamental and advanced skills that underpin a successful career in business. In fact, most of our managers in the strategy group started at CarMax as analysts, and our VP of Strategy and Analysis started his career here through our undergraduate recruiting program. While an MBA is not required to advance or contribute at CarMax, analysts who have chosen to pursue a business degree have enjoyed superior acceptance rates at their first choice schools, including Harvard, Chicago, UVa, Columbia, and Duke.
Your opportunities to develop, contribute, and lead as an analyst at CarMax are as great as the company’s opportunity to grow. While CarMax is already the largest used car retailer in the country (with over $8 billion in sales and over 90 superstores across the country), we have only 2% of the 1 to 6-year-old used car market, which, at $280 billion annually, is bigger than the home improvement or consumer electronics industries. CarMax is already growing at 15% a year, and over the next 10 years plans to have 250-300 stores and achieve $25+ billion in annual sales. As an analyst, you can be an integral part of that growth, all while enjoying a casual and friendly environment, a diverse group of talented associates, a healthy work-life balance, and excellent compensation and benefits.
An ideal candidate will have--Demonstrated top caliber analytic and problem solving skills --History of achievement demonstrated by top 15% GPA, with a quantitative major(s), and/or other recognition such as scholarships, awards, honor societies -- Passion for business and desire to develop into a strong business leader
We encourage you to apply. For more information, please visit us at the career fair, on our website (www.carmax.com/collegerecruiting), or email us at [email protected]://www.seas.virginia.edu/careerdevelopment/index.php?option=com_careerfairstudent&task=detailView&employerId=216&eventId=3
- datablueprint.com
CarMax Example Job Posting
24
own an area of the business and will be expected to improve it
--solving original, wide-ranging, and open-ended business problems--not only discovering new insights, but successfully implementing them--making a significant mark on a growing company--developing the fundamental skills for a rewarding business career
Copyright 2013 by Data Blueprint
DW, Analytics, BI, Meta-Integration TechnologiesDefinitions• Beyond the nuts and bolts of data
management• Analysis of information that had
not been integrated previouslyBusiness Intelligence• Dates at least to 1958• Support better business decision
making• Technologies, applications and
practices for the collection, integration, analysis, and presentation of business information
• Also described as decision support
21
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Data Warehousing• Operational extract, cleansing,
transformation, load, and associated control processes for integrating disparate data into a single conceptual database
Copyright 2013 by Data Blueprint
22
Definitions, cont’d• Study of data to discover and
understand historical patterns to improve future performance
• Use of mathematics in business
• Analytics closely resembles statistical analysis and data mining
– based on modeling involving extensive computation.
• Some fields within the area of analytics are
– enterprise decision management, marketing analytics, predictive science, strategy science, credit risk analysis and fraud analytics.
Copyright 2013 by Data Blueprint
23
from MicroStrategy, Better Business Decisions Every Day: Integrating Business Reporting & Analysis
Example: Set Analysis
Copyright 2013 by Data Blueprint
Polling Question #1
Do you have start data warehouse, data marts and/or other warehousing forms of integration?
a) Last year (2012)b) This year (2013)c) Next Year (2014)d) Nope
24
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
25
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
26
Copyright 2013 by Data Blueprint
• Inmon:–"A subject oriented, integrated, time variant, and non-
volatile collection of summary and detailed historical data used to support the strategic decision-making processes of the organization."
• Kimball:–"A copy of transaction data specifically structured for
query and analysis."• Key concepts focus on:
–Subjects–Transactions–Non-volatility–Restructuring
Warehousing Definitions
27
Copyright 2013 by Data Blueprint
Top 10 Data Warehouse Failure Causes1. The project is over budget2. Slipped schedule3. Functions and
capabilities not implemented
4. Unhappy users5. Unacceptable performance6. Poor availability7. Inability to expand 8. Poor quality data/reports9. Too complicated for users10.Project not cost justified
28
from The Data Administration Newsletter, www.tdan.com
Copyright 2013 by Data Blueprint
29
Basic Data Warehouse Analysis
• Emphasis on the cube
• Permits different users to "slice and dice" subsets of data
• Viewing from different perspectives
from MicroStrategy, Better Business Decisions Every Day: Integrating Business Reporting & Analysis
Copyright 2013 by Data Blueprint
30
Warehouse Analysis
• Users can "drill" anywhere
• Entire collection is accessible
• Summaries to transaction-level detail
from MicroStrategy, Better Business Decisions Every Day: Integrating Business Reporting & Analysis
Copyright 2013 by Data Blueprint
Oracle
31
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
R& D Applications(researcher supported, no documentation)
Finance Application(3rd GL, batch system, no source)
Payroll Application(3rd GL)
Payroll Data(database)
FinanceData
(indexed)
Personnel Data(database)
R & DData(raw)
Mfg. Data(home grown
database) Mfg. Applications(contractor supported)
Marketing Application(4rd GL, query facilities, no reporting, very large)
Marketing Data(external database)
Personnel App.(20 years old,
un-normalized data)
32
Multiple Sources of (for example) Customer Data
Copyright 2013 by Data Blueprint
Corporate Information Factory Architecture
33
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Corporate Information Factory Architecture
34
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
35
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Corporate Information Factory Architecture
Copyright 2013 by Data Blueprint
36
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Corporate Information Factory Architecture
Copyright 2013 by Data Blueprint
MetaMatrix Integration Example
37
• EII Enterprise Information Integration
– between ETL and EAI - delivers tailored views of information to users at the time that it is required
Copyright 2013 by Data Blueprint
Linked Data
38
Linked Data is about using the Web to connect related data that wasn't previously linked, or using the Web to lower the barriers to linking data currently linked using other methods. More specifically, Wikipedia defines Linked Data as "a term used to describe a recommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF."
linkeddata.org
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
39
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
40
Copyright 2013 by Data Blueprint
Kimball's DW Chess Pieces
41
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
3
Courtesy of: http://www.infosys.com/industries/healthcare/industryofferings/Pages/healthcare -data-warehousing.aspx
Data Warehousing
Copyright 2013 by Data Blueprint
3
Descriptive Ask: What happened? What is happening? Find: Structured data Show: Profiles, Bar/Pie charts, Narrative
Predictive Ask: What will happen? Why will it happen? Find: Structured/unstructured data Show: Risk Profiles, Pros/Cons, Care Recs
Prescriptive Ask: What should I do? Why should I do it? Find: Unstructured/structured data Show: Strategic Goals, Support Recs
u Organization-wide
u Volume and Noise
u Utility
u Meaningful scoring
u Actionable recs
u Realistic goals
u Support
u Manage & measure
Analytics in Health Care
Copyright 2013 by Data Blueprint
3
Descriptive Ask: What happened? What is happening? Find: Structured data Show: Profiles, Bar/pie charts, Narrative
Predictive Ask: What will happen? Why will it happen? Find: Structured/unstructured data Show: Risk Profiles, Pros/Cons, Care Recs
Prescriptive Ask: What should I do? Why should I do it? Find: Unstructured/structured data Show: Strategic Goals, Support Recs
BioMarin Licenses Factor VIII Gene Therapy Program for HemophiliaNovel Gene Therapy Approach to Hemophilia BSangamo BioSciences Receives $6.4 Million Strategic Partnership Award From California Institute for Regenerative Medicine to Develop ZFP Therapeutic®
Treating Hemophilia in the 2010s
Hemophilia Management
Copyright 2013 by Data Blueprint
45
Styles of Business Intelligence
from MicroStrategy, Better Business Decisions Every Day: Integrating Business Reporting & Analysis
Copyright 2013 by Data Blueprint
Health Care Provider Data Warehouse• 1.8 million members• 1.4 million providers• 800,000 providers no key• 2.2% prov_number = 9 digits (required)• 29% prov_ssn ≠ 9 digits• 1 User
46
"I can take a roomful of MBAs and accomplish this analysis faster!"
Copyright 2013 by Data Blueprint
Top Causes of Data Warehouse Failure• Poor Quality Data
–Many more values of gender code than (M/F)
• Incorrectly Structured Data
–Providing the correct answer to the wrong question
• Bad Warehouse Design
–Overly complex
47
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Indiana Jones: Raiders Of The Lost Ark
48
Copyright 2013 by Data Blueprint
49
Business Intelligence Features
Problematic Data Quality
Copyright 2013 by Data Blueprint
5 Key Business Intelligence Trends1. There's so much data, but too little
insight. More data translates to a greater need to manage it and make it actionable.
2. Market consolidation means fewer choices for business intelligence users.
3. Business Intelligence expands from the Board Room to the front lines. Increasingly, business intelligence tools will be available at all levels of the corporation
4. The convergence of structured and unstructured data Will create better business intelligence.
5. Applications will provide new views of business intelligence data. The next generation of business intelligence applications is moving beyond the pie charts and bar charts into more visual depictions of data and trends.
50
http://www.cio.com/article/150450/Five_Key_Business_Intelligence_Trends_You_Need_to_Know?page=2&taxonomyId=3002
Copyright 2013 by Data Blueprint
Polling Question #2
Do you have?
a) A single enterprise data warehouse
b) Coordinated data marts
c) Bothd) Uncoordinated
effortse) None
51
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
52
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
53
Copyright 2013 by Data Blueprint
Source:http://dmreview.com/article_sub.cfm?articleID=1000941 used with permission
Meta Data Models
54
Copyright 2013 by Data Blueprint
Metadata Data ModelSCREENELEMENTscreen element id #data item id #screen element descr.
INTERFACEELEMENTinterface element id #data item id #interface element descr.
INPUTELEMENTinput element id #data item id #input element descr.
OUTPUTELEMENToutput element id #data item id #output element descr.
MODELVIEWmodel view element id #data item id #model view element des.
DEPENDENCYdependency elem id #data item id #process id #dependency description
CODEcode id #data item id #stored data item #code location
INFORMATIONinformation id #data item id #information descr.information request
PROCESSprocess id #data item id #process description
USER TYPEuser type id #data item id #information id #user type description
LOCATIONlocation id #information id #printout element id #process id #stored data items id #user type id #location description
PRINTOUTELEMENTprintout element id #data item id #printout element descr.
STORED DATA ITEMstored data item id #data item id #location id #stored data description
DATA ITEMdata item id #data item description
55
Copyright 2013 by Data Blueprint
WarehouseProcess
WarehouseOpera-on
Transforma-on
XMLRecord-‐Oriented
Mul-DimensionalRela-onal
BusinessInforma-on
So?wareDeployment
ObjectModel(Core, Behavioral, Rela-onships, Instance)
WarehouseManagement
Resources
Analysis
Object-‐Oriented
(ObjectModel)
Foundation
OLAPData Mining
Informa-onVisualiza-on
BusinessNomenclature
DataTypes Expressions
KeysIndex
TypeMapping
Overview of CWM Metamodel
http://www.omg.org/technology/documents/modeling_spec_catalog.htm
56
Copyright 2013 by Data Blueprint
Marco & Jennings's Complete Meta Data Model
Source:http://dmreview.com/article_sub.cfm?articleID=1000941 used with permission
57
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
58
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
59
Copyright 2013 by Data Blueprint
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Goals and Principles1. To support and enable
effective business analysis and decision making by knowledgeable workers
2. To build and maintain the environment/infrastructure to support business intelligence activities, specifically leveraging all the other data management functions to cost effectively deliver consistent integrated data for all BI activities
60
Copyright 2013 by Data Blueprint
• Understand BI information needs
• Define and maintain the DW/BI architecture
• Process data for BI
• Implement data warehouse/data marts
• Implement BI tools and user interfaces
• Monitor and tune DW processes
• Monitor and tune BI activities and performance
61
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Activities
Copyright 2013 by Data Blueprint
Primary Deliverables • DW/BI Architecture
• Data warehouses, marts, cubes etc.
• Dashboards-scorecards
• Analytic applications
• Files extracts (for data mining, etc.)
• BI tools and user environments
• Data quality feedback mechanism/loop
62
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Roles and ResponsibilitiesSuppliers:• Executives/managers• Subject Matter Experts• Data governance council• Information consumers• Data producers• Data architects/analysts
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Participants:• Executives/managers• Data Stewards• Subject Matter Experts• Data Architects• Data Analysts• Application Architects• Data Governance Council• Data Providers• Other BI Professionals
63
Copyright 2013 by Data Blueprint
Technology • ETL• Change Management Tools • Data Modeling Tools• Data Profiling Tools• Data Cleansing Tools• Data Integration Tools• Reference Data Management Applications• Master Data Management Applications• Process Modeling Tools• Meta-data Repositories• Business Process and Rule Engines
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
64
Copyright 2013 by Data Blueprint
Guiding Principles1. Obtain executive commitment and
support. 2. Secure business SMEs. 3. Be business focused and driven. Let
the business drive the prioritization.4. Demonstrate data quality is essential.5. Provide incremental value.
65
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
6. Transparency and self service. 7. One size does not fit all: Find the right tools and products for each of
your segments.8. Think and architect globally, act and build locally.9. Collaborate with and integrate all other data initiatives, especially
those for data governance, data quality and metadata.10. Start with the end in mind. 11. Summarize and optimize last, not first.
Copyright 2013 by Data Blueprint
6 Best Practices for Data Warehousing
66
1.Do some initial architecture envisioning.
2.Model the details just in time (JIT).
3.Prove the architecture early.
4.Focus on usage.
5.Organize your work by requirements.
6.Active stakeholder participation.
http://www.agiledata.org/essays/dataWarehousingBestPractices.html
Copyright 2013 by Data Blueprint
Polling Question #3
Do you have a separate data warehouse department, sub-department, or group?
a) Yes b)No
67
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
68
Copyright 2013 by Data Blueprint
1. Data management overview
2. Motivation for warehousing integration technologies (reporting->BI->Analytics)
3. What are warehousing integration technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now: #dataed
Data Systems Integration & BV Part 3: Warehousing
69
Copyright 2013 by Data Blueprint
Summary: Data Warehousing & Business Intelligence Management
70
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Copyright 2013 by Data Blueprint
Series Take Aways
71
• Metadata– Metadata unlocks the value of data, and therefore requires management
attention [Gartner 2011]– Metadata is the language of data governance– Metadata defines the essence of integration challenges
• Cloud– Data governance, architecture, quality, development maturity are necessary but
insufficient prerequisites to successful data cloud implementation– A variety of cloud options will influence cloud and data architectures in general– You must understand your architecture and strategy in order to evaluate the
options– Data must be reengineered to be: less; better quality; more shareable – Failure to do these will result in more business value for the cloud vendors/
service providers and less for your organization
• Warehousing– Business value must precede technical design
Copyright 2013 by Data Blueprint
References
72
Copyright 2013 by Data Blueprint
References
73
Copyright 2013 by Data Blueprint
Additional References• http://www.information-management.com/infodirect/20050909/1036703-1.html • http://www.agiledata.org/essays/dataWarehousingBestPractices.html • http://www.cio.com/article/150450/
Five_Key_Business_Intelligence_Trends_You_Need_to_Know?page=2&taxonomyId=3002 • http://www.computerworld.com/s/article/9228736/
Business_Intelligence_and_analytics_Conquering_Big_Data?taxonomyId=9 • http://www.enterpriseirregulars.com/5706/the-top-10-trends-for-2010-in-analytics-business-
intelligence-and-performance-management/ • http://www.itbusinessedge.com/cm/blogs/vizard/taking-the-analytics-pressure-off-the-data-
warehouse/?cs=50698• http://www.informationweek.com/news/software/bi/240001922
74
Copyright 2013 by Data Blueprint
Questions?
It’s your turn! Use the chat feature or Twitter (#dataed) to submit
your questions to Peter now.
75
+ =
Copyright 2013 by Data Blueprint
Upcoming Events
76
October Webinar:SHOW ME THE MONEY: MONETIZING DATA MANAGEMENTOctober 8, 2013 @ 2:00 PM – 3:30 PM ET(11:00 AM-12:30 PM PT)
November Webinar:UNLOCK BUSINESS VALUE THROUGH REFERENCE & MDMNovember 12, 2013 @ 2:00 PM – 3:30 PM ET(11:00 AM-12:30 PM PT)
Sign up here:• www.datablueprint.com/webinar-schedule • www.Dataversity.net
Brought to you by: