Aim: Implementation of Apriori Algorithm for mining frequent itemsets for Boolean associations.

Procedure: A frequent itemset is an itemset whose support is greater than some user-specified

minimum support (denoted Lk, where k is the size of the itemset) A candidate itemset is a potentially frequent itemset (denoted Ck, where k is the size

of the itemset)

Apriori Algorithm

Apriori Algorithm:

Pass 1 1. Generate the candidate itemsets in C1

2. Save the frequent itemsets in L1

Pass k 1. Generate the candidate itemsets in Ck from the frequent

itemsets in Lk-1 1. Join Lk-1 p with Lk-1q, as follows:

insert into Ck select p.item1, p.item2, . . . , p.itemk-1, q.itemk-1 from Lk-1 p, Lk-1q where p.item1 = q.item1, . . . p.itemk-2 = q.itemk-2, p.itemk-1 < q.itemk-1

2. Generate all (k-1)-subsets from the candidate itemsets in Ck

3. Prune all candidate itemsets from Ck where some (k-1)-subset of the candidate itemset is not in the frequent itemset Lk-1

2. Scan the transaction database to determine the support for each candidate itemset in Ck

3. Save the frequent itemsets in Lk

Implementation: A working Apriori Itemset Generation program can be found on the Itemset Implementation page.

Example 1: Assume the user-specified minimum support is 40%, then generate all frequent itemsets.

Given: The transaction database shown below

TID A B C D ET1 1 1 1 0 0

T2 1 1 1 1 1T3 1 0 1 1 0T4 1 0 1 1 1T5 1 1 1 1 0

Pass 1

C1

Itemset X supp(X)A ?B ?C ?D ?E ?

L1

Itemset X

supp(X)

A 100%B 60%C 100%D 80%E 40%

Pass 2

C2

Nothing pruned since all subsets of these itemsets are infrequent

L2

Itemset X

supp(X)

A,B 60%A,C 100%A,D 80%A,E 40%B,C 60%B,D 40%B,E 20%C,D 80%C,E 40%D,E 40%

L2 after saving only the frequent itemsets

Itemset X

supp(X)

A,B 60%A,C 100%A,D 80%A,E 40%B,C 60%B,D 40%C,D 80%C,E 40%D,E 40%

Pass 3 …………..

Pass 4

First k - 2 = 2 items must match in pass k = 4

C4

Pruning: o For ABCD we check whether ABC, ABD, ACD, BCD are frequent. They

are in all cases, so we do not prune ABCD.

o For ACDE we check whether ACD, ACE, ADE, CDE are frequent. Yes, in all cases, so we do not prune ACDE

L4

Itemset X

supp(X)

A,B,C,D 40%A,C,D,E 40%

Both are frequent

Pass 5: For pass 5 we can't form any candidates because there aren't two frequent 4-itemsets beginning with the same 3 items.

INPUT

Apriori.csv file

Apriori.arff file

OUTPUT

1)Open apriori.arff file in weka software

2)Choose Associate

3)Set minimum support and minimum confidence values

4)Run the apriori.arff file

Result

Associator Output :

=== Run information ===

Scheme: weka.associations.Apriori -N 10 -T 0 -C 0.7 -D 0.05 -U 1.0 -M 0.4 -S -1.0 -c -1

Relation: apriori

Instances: 5

Attributes: 6 A B C D E K

=== Associator model (full training set) ===

Apriori

=======

Minimum support: 0.7 (3 instances)

Minimum metric <confidence>: 0.7

Number of cycles performed: 6

Generated sets of large itemsets:

Size of set of large itemsets L(1): 6

Size of set of large itemsets L(2): 8

Size of set of large itemsets L(3): 3

Best rules found:

1. B=TRUE 4 ==> A=TRUE 4 conf:(1) 2. D=TRUE 4 ==> A=TRUE 4 conf:(1) 3. C=TRUE 3 ==> A=TRUE 3 conf:(1)

4. E=FALSE 3 ==> A=TRUE 3 conf:(1) 5. K=FALSE 3 ==> A=TRUE 3 conf:(1) 6. K=FALSE 3 ==> B=TRUE 3 conf:(1)

7. E=FALSE 3 ==> D=TRUE 3 conf:(1) 8. B=TRUE D=TRUE 3 ==> A=TRUE 3 conf:(1)

9. B=TRUE K=FALSE 3 ==> A=TRUE 3 conf:(1) 10. A=TRUE K=FALSE 3 ==> B=TRUE 3 conf:(1)

Aim : Implementing K-means Algorithm from the following input parameters

Procedure:

The basic step of k-means clustering is simple. In the beginning we determine number of cluster K and we assume the centroid or center of these clusters. We can take any random objects as the initial centroids or the first K objects in sequence can also serve as the initial centroids.

Then the K means algorithm will do the three steps below until convergence

Iterate until stable (= no object move group):

1. Determine the centroid coordinate 2. Determine the distance of each object to the centroids

3. Group the object based on minimum distance

sic step of k-means clustering is simple. In the beginning we determine number of cluster K and we assume the centroid or center of these clusters. We can take any random objects as the initial centroids or the first K objects in sequence can also serve as the initial centroids.

Then the K means algorithm will do the three steps below until convergence

Iterate until stable (= no object move group):

1. Determine the

The numerical example below is given t object have (pH and weight index).

Object attribute 1 (X): weight index attribute 2 (Y): pH

Medicine A 1 1

Medicine B 2 1

Medicine C 4 3

Medicine D 5 4

E Each medicine represents one point with two attributes (X, Y) that we can represent it as coordinate in an attribute space as shown in the figure below. as shown in the figure below.

Suppose we use medicine A and medicine B as the first centroids. Let and denote the

coordinate of the centroids, then and

2. Objects-Centroids distance : we calculate the distance between cluster centroid to each object. Let us use Euclidean distance, then we have distance matrix at iteration 0 is

Each column in the distance matrix symbolizes the object. The first row of the distance matrix corresponds to the distance of each object to the first centroid and the second row is the distance of each object to the second centroid. For example, distance from medicine C =

(4, 3) to the first centroid is , and its distance to the second

centroid is , etc.

3. Objects clustering : We assign each object based on the minimum distance. Thus, medicine A is assigned to group 1, medicine B to group 2, medicine C to group 2 and medicine D to group 2. The element of Group matrix below is 1 if and only if the object is assigned to that group.

4. Iteration-1, determine centroids : Knowing the members of each group, now we compute the new centroid of each group based on these new memberships. Group 1 only has one

member thus the centroid remains in . Group 2 now has three members, thus the centroid is the average coordinate among the three members:

.

5. Iteration-1, Objects-Centroids distances : The next step is to compute the distance of all objects to the new centroids. Similar to step 2, we have distance matrix at iteration 1 is

6. Iteration-1, Objects clustering: Similar to step 3, we assign each object based on the minimum distance. Based on the new distance matrix, we move the medicine B to Group 1 while all the other objects remain. The Group matrix is shown below

7. Iteration 2, determine centroids: Now we repeat step 4 to calculate the new centroids coordinate based on the clustering of previous iteration. Group1 and group 2 both has two

members, thus the new centroids are and

8. Iteration-2, Objects-Centroids distances : Repeat step 2 again, we have new distance matrix at iteration 2 as

9. Iteration-2, Objects clustering: Again, we assign each object based on the minimum distance.

We obtain result that . Comparing the grouping of last iteration and this iteration reveals that the objects does not move group anymore. Thus, the computation of the k-mean clustering has reached its stability and no more iteration is needed. We get the final grouping as the results

Object Feature 1 (X): weight index

Feature 2 (Y): pH Group (result)

Medicine A 1 1 1

Medicine B 2 1 1

Medicine C 4 3 2

Medicine D 5 4 2

INPUT

Kmeans.csv file

kmeans.arff file

OUTPUT

1)Open the kmeans.arff file in weka software

2)Choose cluster

3) Choose SimpleKMeans

4)Set numClusters and choose Manhattan Distance

5)Run the kmeans.arff file

6)Choose Visualize Cluster Assignments

7)Clusterer Visualize

RESULT

Clusterer Output :

=== Run information ===

Scheme: weka.clusterers.SimpleKMeans -N 3 -A "weka.core.ManhattanDistance -R first-last" -I 500 -S 10

Relation: saikumar

Instances: 8

Attributes: 2

X

Y

Test mode: evaluate on training data

=== Model and evaluation on training set ===

kMeans

======

Number of iterations: 3

Sum of within cluster distances: 1.6071428571428572

Missing values globally replaced with mean/mode

Cluster centroids:

Cluster#

Attribute Full Data 0 1 2

(8) (3) (2) (3)

=======================================================

X 4.5 7 1.5 4

Y 5 4 3.5 9

Clustered Instances

0 3 ( 38%) 1 2 ( 25%) 2 3 ( 38%)

Aim : Implementing the DBSCAN algorithm from the input parameters

Procedure :

Dbscan (density based spatial clustering of applicatons with noise) is a density based approach to cluster data of arbitrary shape. An example of dbscan is illustrated in Figure 1.

Figure 1: The input dataset, dbscan clustering, and clarans clustering 1(a) The Input Dataset

1(b) DBScan Method

1(c) Clarans Method

Obviously, the dbscan finds all clusters properly, independent of the size, shape, and location of clusters to each other, and is superior to a widely used Clarans method

Dbscan is based on two main concepts: density reachability and density connectability. These both concepts depend on two input parameters of the dbscan clustering: the size of epsilon neighborhood e and the minimum points in a cluster m. Figure 2 shows the impact of the dbscan parameters to the clustering with m=5 and a giveh e). The number of points parameter impacts detection of outliers. Points are declared to be outliers if there are few othe points in the e-Eucledean neighborhood. e parameter controlls the size of the neighborhood, as well as the size of the clusters. If the e is big enough, the would be one big cluster and no outliers in the Figure. Now we will discuss both concepts of dbscan in detail.

Figure 2: The impact of dbscan parameters

Density reachibility is the first building block in dbscan. It defines whether two distance close points belong to the same cluster. Points p1 is density reachable from p2 if two conditions are satisfied: (i) the points are close enough to each other: distance(p1,p2)<e, (ii)there are enough of points in is neighborhood: |{ r : distance(r,p2)}|>m, where r is a database point. Figure 3(a) illustrates a density reachable point p2 from p1.

Figure 3: Reachability and connectivity concepts of dbscan 3(a) Reachability 3(b) Connectivity

Density connectivity is the last building step of dbscan. Points p0 and pn are density connected, if there is a sequence of density reachable points p1,i2,...,i(n-1) from p0 to pn such that p(i+1) is density reachable from pi.

A dbscan cluster is a set of all density connected points.

INPUT

Dbscan.csv file

dbscan.arff file

OUTPUT

1)Open dbscan.arff file in weka software

2)Choose Cluster

3)Choose DBSCAN

4)Run the file

6)Choose Visualize Cluster Assignments

7)Clusterer Visualize

RESULT

Clusterer Output:

=== Run information ===

Scheme: weka.clusterers.DBScan -E 0.4 -M 3 -I weka.clusterers.forOPTICSAndDBScan.Databases.SequentialDatabase -D weka.clusterers.forOPTICSAndDBScan.DataObjects.EuclidianDataObject

Relation: saikumar

Instances: 8

Attributes: 2

X

Y

Test mode: evaluate on training data

=== Model and evaluation on training set ===

DBScan clustering results

========================================================================================

Clustered DataObjects: 8

Number of attributes: 2

Epsilon: 0.4; minPoints: 3

Index: weka.clusterers.forOPTICSAndDBScan.Databases.SequentialDatabase

Distance-type: weka.clusterers.forOPTICSAndDBScan.DataObjects.EuclidianDataObject

Number of generated clusters: 2

Elapsed time: .01

(0.) 2,10 --> 1

(1.) 2,5 --> NOISE

(2.) 8,4 --> 0

(3.) 5,8 --> 1

(4.) 7,5 --> 0

(5.) 6,4 --> 0

(6.) 1,2 --> NOISE

(7.) 4,9 --> 1

Clustered Instances

0 3 ( 50%)

1 3 ( 50%)

Unclustered instances : 2