Database Management System Prepared by Dr. Ahmed El-Ragal Reviewed & Presented By Mr. Mahmoud Rafeek...

Database Management System

Prepared by Dr. Ahmed El-Ragal

Reviewed & Presented By Mr. Mahmoud Rafeek Alfarra

College Of Science & Technology- Khan younis Information Technology & Computer Science Dep.

Part 8Data Warehousing

Info sources for Info sources for organizationsorganizations

April 20, 2023Data Warehouse2

Financial/ Accounting

CRM and

Internet

Procurement CallCenter

Inventory

Islands of Islands of informationinformation

Applications reporting Applications reporting systems?systems?

April 20, 2023 Data Warehouse 3

Ad-Ad-hochoc

Standard ReportsStandard Reports

ParameterizedParameterizedReportsReports

Data WarehousingData Warehousing

EnterpriseEnterpriseData Data

WarehouseWarehouse

Financial/ Accounting

CRM and

Internet

Procurement CallCenter

Inventory

A Simple Definition A data warehouse is a collection of data created to support decision- making applications.

Another Definition

Data warehousing is the entire process of data extraction, transformation, and loading of data to the warehouse and the access of the data by end users and applications.

Data Warehouse-- DefinedData Warehouse-- DefinedCollection of Data in Support of Management Reporting Needs and Decision-Making Processes.

Organized by subject areas (known as data marts) and structured for query. Integrated across subject areas. Associated with a point in time, such as pay period, fiscal year, semester. Query only, that is, the data does not change.

A Data Warehouse A Data Warehouse Designer’sDesigner’s

View of the BusinessView of the Business

MARKET

P R O D U C T

The threeThe threekey wordskey wordsbecome the become the dimensionsdimensionsof the cube.of the cube.

The points The points inside the inside the cube store the cube store the measurementmeasurementssof the of the business –business –a combination a combination of Product, of Product, Market,Market,and Time.and Time.These points These points arearethe business the business facts.facts.

Goals of the Data WarehouseGoals of the Data Warehouse

Provide Access to various scattered Data.

Include Business Descriptions of Data.

Promote Consistency Across Subject Areas.

Assure Data Quality.

Support User Directed Analysis.

Runseverynight

ExternalExternalDataData

OperationalOperationalDataData

ExtractExtractTransformTransform

LoadLoad

The Data Loading ProcessThe Data Loading Process

ExternalExternalDataData

ExtractExtractTransformTransform

LoadLoad

Ready for AccessReady for Accessand Queryand Query

OperationalOperationalDataData

DataDataWarehouseWarehouse

BusinessBusinessDescriptionsDescriptions

The Data Loading ProcessThe Data Loading Process

Characteristics of a Data Characteristics of a Data WarehouseWarehouse

Subject oriented – organized based on use (on subject not on application).

Integrated – inconsistencies removed.Nonvolatile – stored in read-only format

(historical ).Time variant – data are normally time series.Summarized – in decision-usable format.Large volume – data sets are quite large.Non normalized – often redundant.Metadata – data about data are stored.Data sources – comes from nonintegrated

sources.

Data Warehousing Data Warehousing ArchitectureArchitecture

Data MartsData Marts

ExtractExtractTransformTransformLoadLoad

External Sources

Operational db systems

ServeServe

OLAP serversOLAP servers

Analysis

Query/ Reporting

Data Mining

Data Warehouse vs. Data MartData Warehouse vs. Data Mart

Source: adapted from Strange (1997).

Data ExtractionData Extraction

Often performed by COBOL routines (not recommended because of high program maintenance and no automatically generated meta data).

Sometimes source data is copied to the target database using the replication capabilities of standard RDMS (not recommended because of “dirty data” in the source systems).

Increasing performed by specialized ETL software.

Reasons for “Dirty” DataReasons for “Dirty” Data

Dummy Values. Absence of Data. Multipurpose Fields. Cryptic Data. Contradicting Data. Inappropriate Use of Address Lines. Violation of Business Rules. Reused Primary Keys. Non-Unique Identifiers. Data Integration Problems.

Data CleansingData Cleansing

Source systems contain “dirty data” that must be cleansed.

ETL software contains rudimentary data cleansing capabilities.

Specialized data cleansing software is often used.

Leading data cleansing vendors include Vality (Integrity), Harte-Hanks (Trillium), and Firstlogic (i.d.Centric)

Steps in Data CleansingSteps in Data Cleansing

Parsing

Correcting

Standardizing

Matching

Consolidating

ParsingParsing

Parsing locates and identifies individual data elements in the source files and then isolates these data elements in the target files.

Examples include parsing the first, middle, and last name; street number and street name; and city and state.

CorrectingCorrecting

Corrects parsed individual data components using sophisticated data algorithms and secondary data sources.

Example include replacing an incomplete address and adding a zip code.

StandardizingStandardizing

Standardizing applies conversion routines to transform data into its preferred (and consistent) format using both standard and custom business rules.

Examples include adding a pre name, replacing a nickname, and using a preferred telephone.

MatchingMatching

Searching and matching records within and across the parsed, corrected and standardized data based on predefined business rules to eliminate duplications.

Examples include identifying similar names and addresses.

ConsolidatingConsolidating

Analyzing and identifying relationships between matched records and consolidating/merging them into ONE representation.

Data StagingData Staging

Often used as an interim step between data extraction and later steps.

Accumulates data from asynchronous sources using flat files, FTP sessions, or other processes.

At a predefined cutoff time, data in the staging file is transformed and loaded to the warehouse.

There is usually no end user access to the staging file.

An operational data store may be used for data staging.

Data TransformationData Transformation

Transforms the data in accordance with the business rules and standards that have been established.

Example include: format changes, deduplication, splitting up fields, replacement of codes, derived values, and aggregates.

Building The WarehouseBuilding The WarehouseTransforming Data

TransformTransformChangeChange

CombineCombine

CalculateCalculate

buyer_namebuyer_nameBarr, AdamChai, SeanO’Melia, Erin...

reg_idreg_id246...

total_salestotal_sales17.6052.808.82

reg_idreg_id246...

priceprice.55

1.10.99...

qtyqty32489...

reg_idreg_idIIIVVI...

priceprice.55

1.10.99...

qtyqty32489...

buyer_firstbuyer_firstAdamSeanErin

buyer_lastbuyer_lastBarrChai

O’Melia...

reg_idreg_id246...

Data LoadingData Loading

Data are physically moved to the data warehouse.

The loading takes place within a “load window”.

The trend is to near real time updates of the data warehouse as the warehouse is increasingly used for decisional activities that affect operations.

Meta DataMeta Data

Data about data.Needed by both information technology

personnel and users.IT personnel need to know data sources

and targets; database, table and column names; refresh schedules; data usage measures; etc.

Users need to know entity/attribute definitions; reports/query tools available; report distribution information; help desk contact information, etc.

Reviewed By Mr. Mahmoud Rafeek Alfarra

Database Management System Prepared by Dr. Ahmed El-Ragal Reviewed & Presented By Mr. Mahmoud Rafeek...

Documents

Transcript of Database Management System Prepared by Dr. Ahmed El-Ragal Reviewed & Presented By Mr. Mahmoud Rafeek...

Rafeek Bano Fareek Nahin

Computer Programming 2 Lecture 8: Object Oriented Programming Creating Classes & Objects Prepared & Presented by: Mahmoud Rafeek Alfarra MINISTRY OF EDUCATION.

Expert systems made with neural networkssysengr.engr.arizona.edu/publishedPapers/ExSysNN.pdf · Expert systems made with neural networks Rafeek M. Kottai and A. Terry Bahill Systems

Computer Programming 2 Lecture 9: Object Oriented Programming Array Of Objects Prepared & Presented by: Mahmoud Rafeek Alfarra MINISTRY OF EDUCATION &

A mass spectrometric study of secondary organic aerosols … · 2020. 8. 5. · 5280 M. R. Alfarra et al.: A mass spectrometric study of secondary organic aerosols to form aerosol

BIOSORPTION OF Cd (II), Cu (II), Fe (III) AND Pb (II) FROM ...umpir.ump.edu.my/id/eprint/12979/1/FKKSA - ABREEN RAMZI ALFARRA - CD 9665.pdf · berat yang lain seperti Fe (III), Pb

Mahmoud Rafeek Alfarra Computer Programming || Chapter 1: Introduction & OOP.

MR. RAFEEK A C

Prepared by: Mahmoud Rafeek Al-Farra

The Scalp BY: DR. Yahya h. Alfarra. The Scalp Definition layers Muscle of scalp nerve supply of scalp Arterial supply of scalp Veins of scalp.

Computer Programming 2 Lecture 5: String Processing Part b: Class StringTokenizer Prepared & Presented by: Mahmoud Rafeek Alfarra MINISTRY OF EDUCATION.

MTC - Manufacturing and Technology Centre · ragal to to of before using featm-e. Do not featm-e the prohibited. hospitals. Abide by re fur al gulatiom the of the the — the o outlet

WelcomeWelcomeWelcomeWelcome. University of Palestine Collage of Managment Concepts of Managment Prepared by : Abed Al Kareem Rafeek Alfarra Abed Al Kareem.

Nosso Paranápr.nossoparana.com.br/arquivos/DO.pdf · .allai IF ui amentos de Protecao Individual Ltda. Bom el Industria de Calcados Ltda ragal Comercio e Servicos Ltda Lote 2 Botas

IT Planning and Managment Lecture 3: Advanced Terminology Collage of Information Technology University of Palestine, Gaza Prepared by: Mahmoud Rafeek Alfarra.

Advances in Measurements of tropospheric aerosols in Measurements of tropospheric HhC aerosols Hugh Coe Rami Alfarra James Allan Keith Bower Mike Flynn GordonRami Alfarra, James Allan,

1 University of Palestine. 2 Prepared by : 1- Alaa Isam Hekal 120080038 & 2-Abd Alkareem Rafeek Alfarra 320070039 Supervised by : Dr. Ezz -Alarab Al Awoor.

A mass spectrometric study of secondary organic …aerosolsoftware.web.psi.ch/internet/publications/Alfarra...M. R. Alfarra et al.: A mass spectrometric study of secondary organic

Computer Programming 2 Lecture 10: File streaming (1) Prepared & Presented by: Mahmoud Rafeek Alfarra MINISTRY OF EDUCATION & HIGHER EDUCATION COLLEGE.

Skull BY: DR.Yahya Alfarra. Skull Defenition:it is the skeleton of the head * The skull is composed of several separate bones united at immobile joints.