September 4, 2015Data Mining: Concepts and Techniques1 Data Mining: Concepts and Techniques —...

April 21, 2023Data Mining: Concepts and

Techniques 1

Data Mining: Concepts and

Techniques

— Chapter 2 —


Techniques 2

General data characteristics

Basic data description and exploration

Measuring data similarity

What is about Data?


Techniques 3

What is Data?

Collection of data objects and their attributes

An attribute is a property or characteristic of an object

Examples: eye color of a person, temperature, etc.

Attribute is also known as variable, field, characteristic, or feature

A collection of attributes describe an object

Object is also known as record, point, case, sample, entity, or instance

Tid Refund Marital Status

Taxable Income Cheat

1 Yes Single 125K No

2 No Married 100K No

3 No Single 70K No

4 Yes Married 120K No

5 No Divorced 95K Yes

6 No Married 60K No

7 Yes Divorced 220K No

8 No Single 85K Yes

9 No Married 75K No

10 No Single 90K Yes 10

Attributes

Objects


Techniques 4

Important Characteristics of Structured Data

Dimensionality Curse of dimensionality

Sparsity Only presence counts

Resolution Patterns depend on the scale

Similarity Distance measure


Techniques 5

Attribute Values

Attribute values are numbers or symbols assigned to an attribute

Distinction between attributes and attribute values Same attribute can be mapped to different attribute

values Example: height can be measured in feet or

meters

Different attributes can be mapped to the same set of values

Example: Attribute values for ID and age are integers

But properties of attribute values can be different ID has no limit but age has a maximum and

minimum value


Techniques 6

Types of Attribute Values

Nominal E.g., profession, ID numbers, eye color, zip

codes Ordinal

E.g., rankings (e.g., army, professions), grades, height in {tall, medium, short}

Binary E.g., medical test (positive vs. negative)

Interval E.g., calendar dates, body temperatures

Ratio E.g., temperature in Kelvin, length, time, counts


Techniques 7

Properties of Attribute Values

The type of an attribute depends on which of the following properties it possesses: Distinctness: = Order: < > Addition: + - Multiplication: * /

Nominal attribute: distinctness Ordinal attribute: distinctness & order Interval attribute: distinctness, order & addition Ratio attribute: all 4 properties


Techniques 8

Attribute Type

Description Examples Operations

Nominal The values of a nominal attribute are just different names, i.e., nominal attributes provide only enough information to distinguish one object from another. (=, )

zip codes, employee ID numbers, eye color, sex: {male, female}

mode, entropy, contingency correlation, 2 test

Ordinal The values of an ordinal attribute provide enough information to order objects. (<, >)

hardness of minerals, {good, better, best}, grades, street numbers

median, percentiles, rank correlation, run tests, sign tests

Interval For interval attributes, the differences between values are meaningful, i.e., a unit of measurement exists. (+, - )

calendar dates, temperature in Celsius or Fahrenheit

mean, standard deviation, Pearson's correlation, t and F tests

Ratio For ratio variables, both differences and ratios are meaningful. (*, /)

temperature in Kelvin, monetary quantities, counts, age, mass, length, electrical current

geometric mean, harmonic mean, percent variation


Techniques 9

Discrete vs. Continuous Attributes

Discrete Attribute Has only a finite or countably infinite set of values E.g., zip codes, profession, or the set of words in a

collection of documents Sometimes, represented as integer variables Note: Binary attributes are a special case of

discrete attributes Continuous Attribute

Has real numbers as attribute values Examples: temperature, height, or weight Practically, real values can only be measured and

represented using a finite number of digits Continuous attributes are typically represented as

floating-point variables


Techniques 10

Types of data sets

Record Data Matrix Document Data Transaction Data

Graph World Wide Web Molecular Structures

Ordered Spatial Data Temporal Data Sequential Data Genetic Sequence Data


Techniques 11

Important Characteristics of Structured Data

Dimensionality Curse of Dimensionality

Sparsity Only presence counts

Resolution Patterns depend on the scale


Techniques 12

Record Data

Data that consists of a collection of records, each of which consists of a fixed set of attributes

Tid Refund Marital Status

Taxable Income Cheat

1 Yes Single 125K No

2 No Married 100K No

3 No Single 70K No

4 Yes Married 120K No

5 No Divorced 95K Yes

6 No Married 60K No

7 Yes Divorced 220K No

8 No Single 85K Yes

9 No Married 75K No

10 No Single 90K Yes 10


Techniques 13

Data Matrix

If data objects have the same fixed set of numeric attributes, then the data objects can be thought of as points in a multi-dimensional space, where each dimension represents a distinct attribute

Such data set can be represented by an m by n matrix, where there are m rows, one for each object, and n columns, one for each attribute

1.12.216.226.2512.65

1.22.715.225.2710.23

Thickness LoadDistanceProjection of y load

Projection of x Load

1.12.216.226.2512.65

1.22.715.225.2710.23

Thickness LoadDistanceProjection of y load

Projection of x Load


Techniques 14

Document Data

Each document becomes a `term' vector, each term is a component (attribute) of the

vector, the value of each component is the number of

times the corresponding term occurs in the document.

Document 1

season

timeout

lost

win

game

score

ball

play

coach

team

Document 2

Document 3

3 0 5 0 2 6 0 2 0 2

0

0

7 0 2 1 0 0 3 0 0

1 0 0 1 2 2 0 3 0


Techniques 15

Transaction Data

A special type of record data, where each record (transaction) involves a set of

items. For example, consider a grocery store. The set

of products purchased by a customer during one shopping trip constitute a transaction, while the individual products that were purchased are the items.

TID Items

1 Bread, Coke, Milk

2 Beer, Bread

3 Beer, Coke, Diaper, Milk

4 Beer, Bread, Diaper, Milk

5 Coke, Diaper, Milk


Techniques 16

Graph Data

Examples: Generic graph and HTML Links

5

2

1

2

5

<a href="papers/papers.html#bbbb">Data Mining </a><li><a href="papers/papers.html#aaaa">Graph Partitioning </a><li><a href="papers/papers.html#aaaa">Parallel Solution of Sparse Linear System of Equations </a><li><a href="papers/papers.html#ffff">N-Body Computation and Dense Linear System Solvers


Techniques 17

Chemical Data

Benzene Molecule: C6H6


Techniques 18

Ordered Data

Sequences of transactions

An element of the sequence

Items/Events


Techniques 19

Ordered Data

Genomic sequence data

GGTTCCGCCTTCAGCCCCGCGCCCGCAGGGCCCGCCCCGCGCCGTCGAGAAGGGCCCGCCTGGCGGGCGGGGGGAGGCGGGGCCGCCCGAGCCCAACCGAGTCCGACCAGGTGCCCCCTCTGCTCGGCCTAGACCTGAGCTCATTAGGCGGCAGCGGACAGGCCAAGTAGAACACGCGAAGCGCTGGGCTGCCTGCTGCGACCAGGG


Techniques 20

Ordered Data

Spatio-Temporal Data

Average Monthly Temperature of land and ocean


Techniques 21



Measuring data similarity


Techniques 22

Mining Data Descriptive Characteristics

Motivation

To better understand the data: central tendency, variation and spread

Data dispersion characteristics

median, max, min, quantiles, outliers, variance, etc.

Numerical dimensions correspond to sorted intervals

Data dispersion: analyzed with multiple granularities of precision

Boxplot or quantile analysis on sorted intervals

Dispersion analysis on computed measures

Folding measures into numerical dimensions

Boxplot or quantile analysis on the transformed cube


Techniques 23

Measuring the Central Tendency

Mean (algebraic measure) (sample vs. population):

Weighted arithmetic mean:

Trimmed mean: chopping extreme values

Median: A holistic measure

Middle value if odd number of values, or average of the

middle two values otherwise

Estimated by interpolation (for grouped data):

Mode

Value that occurs most frequently in the data

Unimodal, bimodal, trimodal

Empirical formula:

n

iix

nx

1

1

n

ii

n

iii

w

xwx

1

1

widthfreq

lfreqNLmedian

median

))(2/

(1

)(3 medianmeanmodemean

N

x


Techniques 24

Symmetric vs. Skewed Data

Median, mean and mode of symmetric, positively and negatively skewed data

positively skewed

negatively skewed

symmetric


Techniques 25

Measuring the Dispersion of Data

Quartiles, outliers and boxplots

Quartiles: Q1 (25th percentile), Q3 (75th percentile)

Inter-quartile range: IQR = Q3 – Q1

Five number summary: min, Q1, M, Q3, max

Boxplot: ends of the box are the quartiles, median is marked,

whiskers, and plot outlier individually

Outlier: usually, a value higher/lower than 1.5 x IQR

Variance and standard deviation (sample: s, population: σ)

Variance: (algebraic, scalable computation)

Standard deviation s (or σ) is the square root of variance s2 (or σ2)

n

i

n

iii

n

ii x

nx

nxx

ns

1 1

22

1

22 ])(1

[1

1)(

1

1

n

ii

n

ii x

Nx

N 1

22

1

22 1)(

1


Techniques 26

Boxplot Analysis

Five-number summary of a distribution:

Minimum, Q1, M, Q3, Maximum Boxplot

Data is represented with a box The ends of the box are at the first and third

quartiles, i.e., the height of the box is IQR The median is marked by a line within the box Whiskers: two lines outside the box extend to

Minimum and Maximum


Techniques 27

Histogram Analysis

Graph displays of basic statistical class descriptions Frequency histograms

A univariate graphical method Consists of a set of rectangles that reflect the counts

or frequencies of the classes present in the given data


Techniques 28

Histograms Often Tells More than Boxplots

The two histograms shown in the left may have the same boxplot representation The same values

for: min, Q1, median, Q3, max

But they have rather different data distributions


Techniques 29

Quantile Plot

Displays all of the data (allowing the user to assess both the overall behavior and unusual occurrences)

Plots quantile information For a data xi data sorted in increasing order, fi

indicates that approximately 100 fi% of the data are below or equal to the value xi


Techniques 30

Quantile-Quantile (Q-Q) Plot

Graphs the quantiles of one univariate distribution against the corresponding quantiles of another

Allows the user to view whether there is a shift in going from one distribution to another


Techniques 31

Scatter plot

Provides a first look at bivariate data to see clusters of points, outliers, etc

Each pair of values is treated as a pair of coordinates and plotted as points in the plane


Techniques 32

Loess Curve

Adds a smooth curve to a scatter plot in order to provide better perception of the pattern of dependence

Loess curve is fitted by setting two parameters: a smoothing parameter, and the degree of the polynomials that are fitted by the regression


Techniques 33

Positively and Negatively Correlated Data

The left half fragment is positively

correlated

The right half is negative correlated


Techniques 34

Not Correlated Data


Techniques 35

Data Visualization and Its Methods

Why data visualization? Gain insight into an information space by mapping data

onto graphical primitives Provide qualitative overview of large data sets Search for patterns, trends, structure, irregularities,

relationships among data Help find interesting regions and suitable parameters for

further quantitative analysis Provide a visual proof of computer representations derived

Typical visualization methods: Geometric techniques Icon-based techniques Hierarchical techniques


Techniques 36

Geometric Techniques

Visualization of geometric transformations and projections of the data

Methods Landscapes Projection pursuit technique

Finding meaningful projections of multidimensional data

Scatterplot matrices Prosection views Hyperslice Parallel coordinates


Techniques 37

Scatterplot Matrices

Matrix of scatterplots (x-y-diagrams) of the k-dim. data

Use

d by

erm

issi

on o

f M

. W

ard,

Wor

cest

er P

olyt

echn

ic In

stitu

te


Techniques 38

news articlesvisualized asa landscape

Use

d by

per

mis

sion

of B

. Wrig

ht, V

isib

le D

ecis

ions

Inc.

Landscapes

Visualization of the data as perspective landscape The data needs to be transformed into a (possibly artificial) 2D spatial

representation which preserves the characteristics of the data


Techniques 39

Attr. 1 Attr. 2 Attr. kAttr. 3

• • •

Parallel Coordinates

n equidistant axes which are parallel to one of the screen axes and correspond to the attributes

The axes are scaled to the [minimum, maximum]: range of the corresponding attribute

Every data item corresponds to a polygonal line which intersects each of the axes at the point which corresponds to the value for the attribute


Techniques 40

Parallel Coordinates of a Data Set


Techniques 41

Icon-based Techniques

Visualization of the data values as features of icons Methods:

Chernoff Faces Stick Figures Shape Coding: Color Icons: TileBars: The use of small icons representing the

relevance feature vectors in document retrieval


Techniques 42

Chernoff Faces

A way to display variables on a two-dimensional surface, e.g., let x be eyebrow slant, y be eye size, z be nose length, etc.

The figure shows faces produced using 10 characteristics--head eccentricity, eye size, eye spacing, eye eccentricity, pupil size, eyebrow slant, nose size, mouth shape, mouth size, and mouth opening): Each assigned one of 10 possible values, generated using Mathematica (S. Dickson)

REFERENCE: Gonick, L. and Smith, W. The Cartoon Guide to Statistics. New York: Harper Perennial, p. 212, 1993

Weisstein, Eric W. "Chernoff Face." From MathWorld--A Wolfram Web Resource. mathworld.wolfram.com/ChernoffFace.html


Techniques 43

Hierarchical Techniques

Visualization of the data using a hierarchical partitioning into subspaces.

Methods Dimensional Stacking Worlds-within-Worlds Treemap Cone Trees InfoCube


Techniques 44

Tree-Map

Screen-filling method which uses a hierarchical partitioning of the screen into regions depending on the attribute values

The x- and y-dimension of the screen are partitioned alternately according to the attribute values (classes)

MSR Netscan Image


Techniques 45

Tree-Map of a File System (Schneiderman)


Techniques 46



Measuring data similarity (Sec. 7.2)


Techniques 47

Similarity and Dissimilarity

Similarity Numerical measure of how alike two data objects

are Value is higher when objects are more alike Often falls in the range [0,1]

Dissimilarity (i.e., distance) Numerical measure of how different are two data

objects Lower when objects are more alike Minimum dissimilarity is often 0 Upper limit varies

Proximity refers to a similarity or dissimilarity


Techniques 48

Data Matrix and Dissimilarity Matrix

Data matrix n data points with

p dimensions Two modes

Dissimilarity matrix n data points, but

registers only the distance

A triangular matrix Single mode

npx...nfx...n1x

...............ipx...ifx...i1x

...............1px...1fx...11x

0...)2,()1,(

:::

)2,3()

...ndnd

0dd(3,1

0d(2,1)

0


Techniques 49

Example: Data Matrix and Distance Matrix

0

1

2

3

0 1 2 3 4 5 6

p1

p2

p3 p4

point x yp1 0 2p2 2 0p3 3 1p4 5 1

Distance Matrix (i.e., Dissimilarity Matrix) for Euclidean Distance

p1 p2 p3 p4p1 0 2.828 3.162 5.099p2 2.828 0 1.414 3.162p3 3.162 1.414 0 2p4 5.099 3.162 2 0

Data Matrix


Techniques 50

Minkowski Distance

Minkowski distance: A popular distance measure

where i = (xi1, xi2, …, xip) and j = (xj1, xj2, …, xjp) are

two p-dimensional data objects, and q is the order Properties

d(i, j) > 0 if i ≠ j, and d(i, i) = 0 (Positive definiteness)

d(i, j) = d(j, i) (Symmetry) d(i, j) d(i, k) + d(k, j) (Triangle Inequality)

A distance that satisfies these properties is a metric

qq

pp

qq

jx

ix

jx

ix

jx

ixjid )||...|||(|),(

2211


Techniques 51

Special Cases of Minkowski Distance

q = 1: Manhattan (city block, L1 norm) distance E.g., the Hamming distance: the number of bits that are

different between two binary vectors

q= 2: (L2 norm) Euclidean distance

q . “supremum” (Lmax norm, L norm) distance. This is the maximum difference between any component

of the vectors Do not confuse q with n, i.e., all these distances are defined

for all numbers of dimensions. Also, one can use weighted distance, parametric Pearson

product moment correlation, or other dissimilarity measures

)||...|||(|),( 22

22

2

11 pp jx

ix

jx

ix

jx

ixjid

||...||||),(2211 pp jxixjxixjxixjid


Techniques 52

Example: Minkowski Distance

Distance Matrix

point x yp1 0 2p2 2 0p3 3 1p4 5 1

L1 p1 p2 p3 p4p1 0 4 4 6p2 4 0 2 4p3 4 2 0 2p4 6 4 2 0

L2 p1 p2 p3 p4p1 0 2.828 3.162 5.099p2 2.828 0 1.414 3.162p3 3.162 1.414 0 2p4 5.099 3.162 2 0

L p1 p2 p3 p4

p1 0 2 3 5p2 2 0 1 3p3 3 1 0 2p4 5 3 2 0


Techniques 53

Binary Variables

A contingency table for binary data

Distance measure for symmetric binary

variables:

Distance measure for asymmetric binary

variables:

Jaccard coefficient (similarity measure

for asymmetric binary variables):

A binary variable is symmetric if both of

its states are equally valuable and carry

the same weight. cbaa jisim

Jaccard ),(

dcbacb jid

),(

cbacb jid

),(

pdbcasum

dcdc

baba

sum

0

1

01

Object i

Object j


Techniques 54

Dissimilarity between Binary Variables

Example

gender is a symmetric attribute the remaining attributes are asymmetric binary let the values Y and P be set to 1, and the value N be set

to 0

Name Gender Fever Cough Test-1 Test-2 Test-3 Test-4

Jack M Y N P N N NMary F Y N P N P NJim M Y P N N N N

75.0211

21),(

67.0111

11),(

33.0102

10),(

maryjimd

jimjackd

maryjackd


Techniques 55

Nominal Variables

A generalization of the binary variable in that it can take more than 2 states, e.g., red, yellow, blue, green

Method 1: Simple matching m: # of matches, p: total # of variables

Method 2: Use a large number of binary variables creating a new binary variable for each of the M

nominal states

pmpjid ),(


Techniques 56

Ordinal Variables

An ordinal variable can be discrete or continuous Order is important, e.g., rank Can be treated like interval-scaled

replace xif by their rank

map the range of each variable onto [0, 1] by replacing i-th object in the f-th variable by

compute the dissimilarity using methods for interval-scaled variables

11

f

ifif M

rz

},...,1{fif

Mr


Techniques 57

Ratio-Scaled Variables

Ratio-scaled variable: a positive measurement on a nonlinear scale, approximately at exponential scale, such as AeBt or Ae-Bt

Methods: treat them like interval-scaled variables—not a

good choice! (why?—the scale can be distorted) apply logarithmic transformation

yif = log(xif)

treat them as continuous ordinal data treat their rank as interval-scaled


Techniques 58

Variables of Mixed Types

A database may contain all the six types of variables symmetric binary, asymmetric binary, nominal,

ordinal, interval and ratio One may use a weighted formula to combine their

effects

f is binary or nominal:dij

(f) = 0 if xif = xjf , or dij(f) = 1 otherwise

f is interval-based: use the normalized distance f is ordinal or ratio-scaled

Compute ranks rif and Treat zif as interval-scaled

)(1

)()(1),(

fij

pf

fij

fij

pf

djid

1

1

f

if

Mrz

if


Techniques 59

Vector Objects: Cosine Similarity

Vector objects: keywords in documents, gene features in micro-arrays, …

Applications: information retrieval, biologic taxonomy, ... Cosine measure: If d1 and d2 are two vectors, then

cos(d1, d2) = (d1 d2) /||d1|| ||d2|| ,

where indicates vector dot product, ||d||: the length of vector d Example:

d1 = 3 2 0 5 0 0 0 2 0 0

d2 = 1 0 0 0 0 0 0 1 0 2

d1d2 = 3*1+2*0+0*0+5*0+0*0+0*0+0*0+2*1+0*0+0*2 = 5

||d1||= (3*3+2*2+0*0+5*5+0*0+0*0+0*0+2*2+0*0+0*0)0.5=(42)0.5 = 6.481

||d2|| = (1*1+0*0+0*0+0*0+0*0+0*0+0*0+1*1+0*0+2*2)0.5=(6) 0.5 = 2.245

cos( d1, d2 ) = .3150


Techniques 60

Correlation Analysis (Numerical Data)

Correlation coefficient (also called Pearson’s product moment coefficient)

where n is the number of tuples, and are the respective means of p and q, σp and σq are the respective standard

deviation of p and q, and Σ(pq) is the sum of the pq cross-product.

If rp,q > 0, p and q are positively correlated (p’s values

increase as q’s). The higher, the stronger correlation. rp,q = 0: independent; rpq < 0: negatively correlated

qpqpqp n

qpnpq

n

qqppr

)1(

)(

)1(

))((,

p q


Techniques 61

Correlation (viewed as linear relationship)

Correlation measures the linear relationship between objects

To compute correlation, we standardize data objects, p and q, and then take their dot product

)(/))(( pstdpmeanpp kk

)(/))(( qstdqmeanqq kk

qpqpncorrelatio ),(


Techniques 62

Visually Evaluating Correlation

Scatter plots showing the similarity from –1 to 1.

September 4, 2015Data Mining: Concepts and Techniques1 Data Mining: Concepts and Techniques —...

Documents

Transcript of September 4, 2015Data Mining: Concepts and Techniques1 Data Mining: Concepts and Techniques —...