1

IS 605/606: Information Systems

Technology FocusEvolution of DSSIntroduction to Data WarehousingDr. Boris Jukić

2

PC/4GL Technology end-users able to directly control data and systems

(notion that data can be used for more than operational transactional purposes)

Management Information System MIS (later also known as Decision Support Systems-DSS) defined as

processing used to drive management decisions (as opposed to processing exclusively used to drive detailed operational decisions)

single-database-serving-all-purpose paradigm One DBMS supporting both regular transactional

processing and Decision Support

Evolution of Decision Support Systems1980’s

3

Extract Processing Extract Program

rummages through a file or database, uses some criteria for selection, and, upon finding qualified data, transports the data over onto another file or database

used to move data used for DDS out of the way of day-to-day transaction processing systems and to give the DSS user the control over that data

Evolution of Decision Support Systems1980’s: continued

4

Proliferation of Extracts Extracts everywhere (and extracts of extracts, and

extracts of extracts of extracts, and …) Productivity Problems

In order to write a corporate report Data must be located Lots of customized programs must be written

Queries for databases are relatively easy, but the programs that create reports must cross every technology that the company has, such as Older file systems Third party data

Evolution of Decision Support SystemsEarly 1990’s

5

Inability to go from Data to Information Due to the following:

Data Applications were built to service the needs of current transaction processing

Time horizon of data that is collected and kept accessible is typically 6 moths or so

Example; customer monthly stamens may be kept in direct access form for about 5-6 months, assuming that typical customer will only try to resolve billing disputes within that time period

The standard transactional database systems were never designed to hold the historical data needed for DSS analysis

Evolution of Decision Support SystemsEarly 1990’s: continue

6

Lack of Credibility of DataE.g. Account activity report done by 2 analysts

No set time basis for data (e.g. one analyst extracts data on Sunday evening, another extracts data on Wednesday afternoon)

Algorithmic differential (e.g. one analyst extracts data only on large accounts, another extracts data on all accounts)

The levels of extraction (each additional level of extraction increases the probability of discrepancy)

External Data (e.g. one analyst is bringing in Wall Street Journal data and another is bringing in Business Week data, and they both strip the data identity)

No common source of data to begin with (analysts are extracting data from different databases within the company)

E.g. Final ResultAnalyst A Analyst BActivity is up 10% Activity is down 15%

Evolution of Decision Support SystemsEarly 1990’s: continue

7

Approach Change Recognition that there are two fundamentally different kinds

of data OPERATIONAL DATA (PRIMITIVE DATA) ANALYTICAL DATA (DERIVED DATA, DSS DATA)

Data Warehouses emerge as the New DSS Architecture

Evolution of Decision Support SystemsLate 1990’s - Current

8

Need for Data Warehousing

Integrated, company-wide view of high-quality information.

Separation of operational and analytical systems and data.

9

Operational Data Analytical DataData Differences

Typical Time-Horizon: Days/Months Typical Time-Horizon: YearsDetailed Summarized (and/or Detailed)Current Values over time (Snapshots)

Technical DifferencesCan be Updated Read (and Append) OnlyControl of Update: Major Issue Control of Update: No IssueSmall Amounts used in a Process Large Amounts used in a ProcessNon-Redundant Redundancy not an IssueHigh frequency of Access Low/Modest frequency of Access

Purpose DifferencesFor “Clerical Community” For “Managerial Community”Supports Day-to-Day Operations Supports Managerial NeedsApplication Oriented Subject Oriented

OPERATIONAL vs. ANALYTICAL DATA

10

Application vs. Subject Oriented

11

Application vs. Subject Oriented

12

Application vs. Subject Oriented

13

OPERATIONAL vs. ANALYTICAL DATAHardware Utilization(Frequency of Access)

Operational Data Warehouse

14

Data Warehouse: Definition

Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data.(Bill Inmon, paraphrased by Oracle Data Warehouse Method)

15

Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data.(Oracle Data Warehouse Method)

enterprise-wide refers to the fact that a DW provides a company-wide view of the information it contains

Data Warehouse: Definition

16

Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data.(Oracle Data Warehouse Method)

structured repository refers to the fact that a DW is a structured data repository like any other data base

Data Warehouse: Definition

17

Data Warehouse: An enterprise-wide structured repository

of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data.(Oracle Data Warehouse Method)

subject-oriented refers to the fundamental difference in the purpose of a traditional Database System and a DW. Traditional Database System is developed in order

to support a specific business process (e.g. shipping company order-entry database, dental office appointment management database).

DW is developed to analyze a specific subject area (e.g. sales, profit).

Data Warehouse: Definition

18

Data Warehouse: An enterprise-wide structured

repository of subject-oriented, time-variant, historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data.(Oracle Data Warehouse Method)

time-variant refers to the fact that a DW contains slices of data across different periods of time. With these data slices, the user can view reports from now and in the past.

Data Warehouse: Definition

19

Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant,

historical data used for information retrieval and decision support. The data warehouse stores atomic and summary data.(Oracle Data Warehouse Method)

historical refers to the fact that a DW typically contains several years worth of data (as opposed to 60-90 days typical time horizon for data in traditional operational databases). In fact, DW often does not contain the latest transactional data.

Data Warehouse: Definition

20

Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data

used for information retrieval and decision support. The data warehouse stores atomic and summary data.(Oracle Data Warehouse Method)

information retrieval and decision support refers to the fact that a DW is a facility for getting information to answer questions. It is not meant for direct data entry; batch updates are the norm for refreshing data warehouses.

Data Warehouse: Definition

21

Data Warehouse: An enterprise-wide structured repository of subject-oriented, time-variant, historical data used for information retrieval and decision support.

The data warehouse stores atomic and summary data.(Oracle Data Warehouse Method)

atomic and summary data refers to the fact that a DW, depending on purpose, may contain atomic data, summary data, or both.

Data Warehouse: Definition

22

DW is Subject Oriented!

Data is organized around major subject areas of an enterprise, and is therefore useful for an enterprise-wide understanding of those subjects

E.g. a banking operational system keeps independent records of customer savings, loans, and other transactions. A warehouse pulls this independent data together to provide customer financial information (fees, profits, losses …)

Examples of subject areas Customer Financial Information Toll calls made in the telecommunications industry company Airline passenger booking information

Data from operational systems must be transformed so that is consistent and meaningful in the DW

Subject Orientation

Applications

auto

life

health

casualty

Insurance Co. Operational Databases Data Warehouse

claim

Subject

Subject Orientation

Applications

auto

life

health

casualty

premium

claim

Subjects

Insurance Co. Operational Databases Data Warehouse

25

DW is Integrated! In many organizations, data resides in diverse independent

systems, making it difficult to acquire meaningful information for analysis.

In DW data is completely integrated, even when the underlying sources store data differently

There is no magic wand: the transformation and integration process (which involves ETL – extraction, transformation, and transportation-load) can be time consuming and costly, and it requires commitment from every part of the organization, particularly top-level managers

26

Common Example of a Data Warehouse Purpose

Data warehouse is often designed and implemented to answer these TWO fundamental questions: Who is buying what? When and where are they doing so?

More specific Who [which customer] is buying [buying / using /

delivering / shipping / ordering / returning] what [products / services] from where [outlet / store / clinic / branch] on what occasion [when], how [credit card / cash / check / exchange / debit] and why [causation]?

27

Some Uses of a Data Warehouse

Airlines for aircraft deployment, analysis of route profitability, frequent flyer promotions, and maintenance

Banking for promotion of products and services, and customer service

Health care for cost reduction Investment and insurance companies for customer analysis,

risk assessment, and portfolio management Retail stores for buying pattern analysis, promotions, customer

profiling, and pricing Telecommunications for product and service promotions.

Source: Oracle Corp. 1999

Typical Use of Data Warehouse

0

5

10

15

20

25

30

35

40

45

Financial Manufacturing Telecom Retail Others (Health Care, Airline,Insurance, Clothing Distr.,

etc.)