Post on 23-Dec-2015
Final Exam Thursday Dec. 9, 19:30, BSB B154 Lecture notes 20 True/False and 20 multiple
Choice questions: 40 points 4 Short answer questions
(application and conceptual): 40 points
Project Presentation Email me Power Point presentation the day before Presentation by all members: 15 minutes Question and answer: 5 minutes Peer evaluation:1) Objective and the value of the project2) Data modeling3) Functions of the application (with demo)4) Quality of presentation5) Overall
Data Warehouse
Data Warehouse The idea of a data warehouse is to put a
wide range of operational data from internal and external sources into one place so it can be better utilized by executives, line of business managers and other business analysts.
Once the information is gathered, OLAP (on-line analytical processing ) software comes into play by providing the desktop analysis tools for querying, manipulating and reporting the data from the data warehouse.
Data Warehouse environment the source systems from which
data is extracted the tools used to extract data for
loading the data warehouse the data warehouse database itself
where the data is stored the desktop query and reporting
tools used for decision support
The Data Warehouse The Data Warehouse is an
integrated, subject-oriented, time-variant, non-volatile database that provides support for decision making.
Creating A Data Warehouse
Figure 13.3
Operational Vs. Multidimensional View Of Sales
The Data Warehouse Integrated
The Data Warehouse is a centralized, consolidated database that integrates data retrieved from the entire organization.
Subject-Oriented The Data Warehouse data is arranged
and optimized to provide answers to questions coming from diverse functional areas within a company.
The Data Warehouse Time Variant
The Warehouse data represent the flow of data through time. It can even contain projected data.
Non-Volatile Once data enter the Data Warehouse,
they are never removed. The Data Warehouse is always growing.
Operational Database vs. Data warehouse
Operational DB Similar data can have
different representations or meanings
Functional or process orientation
Current transaction Frequent updating
Data Warehouse Unified view of all
data elements Subject orientation
for decision support Historical
information with time dimension
Data are added without change
Data Mart A data mart is a small, single-
subject data warehouse subset that provides decision support to a small group of people.
Data Mart Data Marts can serve as a test vehicle
for companies exploring the potential benefits of Data Warehouses.
Data Marts address local or departmental problems, while a Data Warehouse involves a company-wide effort to support decision making at all levels in the organization.
Star Schema The star schema is a data modeling
technique used to map multidimensional decision support into a relational database.
Star schemas yield an easily implemented model for multidimensional data analysis while still preserving the relational structure of the operational database.
Star Schema Four Components:
Facts Dimensions Attributes Attribute hierarchies
Figure 13.14 A Three-Dimensional View of Sales
Figure 13.17 Attribute Hierarchies in Multidimensional Analysis
Figure 13.17 Star Schema For Sales
Star Schema Representation Facts and dimensions are normally
represented by physical tables in the data warehouse database.
The fact table is related to each dimension table in a many-to-one (M:1) relationship.
Fact and dimension tables are related by foreign keys and are subject to the primary/foreign key constraints.
Figure 13.18 Orders Star Schema
Star Schema Performance-Improving
Techniques Normalization of dimensional tables Multiple fact tables representing
different aggregation levels Denormalization of fact tables Table partitioning and replication
Figure 13.19 Normalized Dimension Tables
Multiple Fact Tables
Data Warehouse Implementation The Data Warehouse as an Active
Decision Support Network A Company-Wide Effort that
Requires User Involvement and Commitment at All Levels
Satisfy the Trilogy: Data, Analysis, and Users
Apply Database Design Procedures
Data Warehouse Implementation Road Map
On-Line Analytical Processing On-Line Analytical Processing (OLAP) is an
advanced data analysis environment that supports decision making, business modeling, and operations research activities.
Four Main Characteristics of OLAP Use multidimensional data analysis techniques. Provide advanced database support. Provide easy-to-use end user interfaces. Support client/server architecture.
Figure 13.7 OLAP Server Arrangement
http://www.dwinfocenter.org/
Data Mining The data warehouse that enterprises
are building until now have largely ignored
Factors make data mining feasible organizations are gathering more data
from on-line TPS with lower storage cost
high computation power allows using complex data mining algorithm
Data Mining With data mining, it is possible to
better manage product warranties, predict purchases of retail stock, unearth fraud, determine credit risk, and define new products and services.
Data-Mining Phases
Four Phases of Data Mining1.Data Preparation
Identify and cleanse data sets. Data Warehouse is usually used for data
mining operations.
2.Data Analysis and Classification Identify common data characteristics or
patterns using Data groupings, classifications, clusters, or
sequences. Data dependencies, links, or relationships. Data patterns, trends, and deviations.
Four Phases of Data Mining 3. Knowledge Acquisition
Select the appropriate modeling or knowledge acquisition algorithms.
Examples: neural networks, decision trees, rules induction, genetic algorithms, classification and regression tree, memory-based reasoning, or nearest neighbor and data visualization).
4. Prognosis Predict future behavior and forecast business
outcomes using the data mining findings.
Data Mining Data mining yields five basic type of
information: Association - occurrences are linked to a
single event. “beer purchasers also buy peanuts 70% of the time”
Sequences - events are linked over time. “a new carpet purchase linked to new curtains”
Classification - patterns are recognized that describe the characteristics of a group, such as customers who cancel credit cards
Data Mining Clustering - discovers
undiscovered groupings ``Buyers of expensive sport cars are typically young urban professionals whereas luxury sedans are bought by elderly wealthy persons.''
Forecasting - estimates future value such as inventory turnover
Database Marketing It seems a lot of companies are taking
a friendly interest in your life these days
Companies are collecting mountains of information about you, crunching it to predict how likely you are to buy a products, and using that knowledge to craft a marketing message precisely calibrated to get you to do so.
Database Marketing The trend:
Mass marketing Marketing segmentation Individual marketing
Nothing is more powerful than knowledge about customers’ individual practice and preferences.
Database Marketing Gathering massive quantity of data
about consumers from multiple sources Data are combined and analyzed using
powerful tools Has a primary goal of better
understanding current and potential customers in order to boost sales and build customer loyalty
American Express Reader’s Digest
Pioneers of New Marketing General Motors surveys 12 million
GM Card holders on their car preferences
Blockbuster has a database of 36 million households and 2 million daily transactions. It is testing a system that will recommend movies based on a customer’s past rentals
Pioneers of New Marketing Kraft amassed a list of 30 million users
from coupons and survey questions. It regularly send them tips on nutrition and recipes, as well as coupons for specific brands
56% of manufacturers and retailers are currently building a database for marketing
85% believe they will do database marketing in 2000.
Some concerns Private intelligence-gathering gives
some people the creeps Targeted marketing efforts are
intrusive and annoying The collection, manipulation, and
combination of lists of personal information amount to an ominous invasion of privacy
A Sample Of Current Data Warehousing And Data Mining Vendors
Table 13.10
http://www.irmac.ca/
http://www.almaden.ibm.com/cs/quest/TECH.html
Deploying Data Mining for Competitive Advantage The act of building data-mining
models does not, by itself, guarantee any business value
To be used as competitive weapon, data mining must be part of a larger process that ensures that the information learned by data mining is transformed into actionable results
A process of deploying data mining for competitive advantage
Problem definition Discovery Implementation Taking action Monitoring the results
Anderson 2001 survey: Data mining in retailer industry Using data mining: 52.5% in total
75% of very large retailers (>$500 million) 46.4% of large ($200-499 million) 34.8% of medium ($50-199 million) 20% of small (<$50 million)
Effect (contribution to the bottom line) 52.5% said "no contribution" 19.8% said "very little." 17.8% said "somewhat” 8.9% said "very much."
Hunt for terrorists Banks probe credit, debit card records in
hunt for terrorists: 'Data mining': Any suspicious transactions handed over to RCMP
Canada's major financial institutions are reviewing thousands of their customers' confidential transactions as part of a probe into terrorist funding that has gone beyond a list of 27 suspects provided by U.S. law enforcement agencies. Source: Financial Post (National Post) September 27, 2001