University of Eastern Finland School of Computing P.O. Box 111 FIN- 80101 Joensuu FINLAND Tel. +358...

University of Eastern FinlandSchool of ComputingP.O. Box 111FIN- 80101 JoensuuFINLANDTel. +358 13 251 7959fax +358 13 251 7955www.uef.fi/cs

K-means*: Clustering by Gradual Data Transformation

Mikko Malinen and Pasi Fränti

Speech and Image Processing Unit

School of Computing

University of Eastern Finland


K-means* clustering Gradual transformation of data

Model

Data

Fit the data to a model

Intermediate Final


K-means clustering

Iterate between two steps:

1. Assignment step Assign the points to the nearest centroids

2. Update step Update the location of centroids

)(

)(

)1( 1t

ij Sjt

i

ti

S x

x m

},...,1*:{ )(*

)()( kiS tij

tijj

ti m x m x x


K-means* clustering


Example of clustering (s2 dataset)


0% done


10% done


20% done


30% done


40% done


50% done


60% done


70% done


80% done


90% done


100% done


Empty clusters problem


Initialization

Data set transform

Empty clusters removal

K-means

Algorithm total

)(nOkfree kPhase )( nOk )1(Ok

)(nO

)(nO

)(nO

)(nO

)(nO

)(nO

)(nO

)(nO

)(nO)( 2nkO )( 3nO )( 2nO

)( 1kdknO )( 2)( dnOnO )( 2

3dn

nO )( 1kdnO

)( 1kdknO )( 2)( dnOnO )( 2

3dn

nO )( 1kdnO

Time Complexity


Time ComplexityFixed k-means

Initialization

Data set transform

Empty clusters removal

K-means

Algorithm total

)(nOkfree kPhase )( nOk )1(Ok

)(nO )(nO )(nO )(nO

)(nO)( 2nkO )( 3nO )( 2nO

)(knO )( 2nO )( 5.1nO

)(nO )(nO )(nO )(nO

)(nO)( 2nkO )( 3nO )( 2nO

)(nO


s1 d = 2n = 5000 k = 15

s2 d = 2n = 5000 k = 15

s3 d = 2n = 5000 k = 15

s4 d = 2n = 5000 k = 15

bridge d = 16n = 4096 k= 256

missa d = 16n = 6480 k= 256

house d = 3n=34000 k=256

thyroid d = 5n = 215 k = 2

iris d = 4n = 150 k = 2

wine d = 13n = 178 k = 3

Datasets


Mean square error

Dataset k-means proposed GKM optimal

s1 1.85 1.01 0.89 0.89

s2 1.94 1.52 1.33 1.33

s3 1.97 1.71 1.69 1.69

s4 1.69 1.63 1.57 1.57

bridge 168.2 164.7 164.1 160.7

missa 5.33 5.15 5.34 5.12

house 9.88 9.48 5.94 5.86

thyroid 6.97 6.92 1.52 1.52

iris 3.70 3.70 2.02 2.02

wine 1.92 1.90 0.88 0.88


Mean square error vs.number of steps


All correct:

Number of incorrect clusters

proposed: 36%k-means: 14%


1 incorrect:




2 incorrect:




3 incorrect:




Summary

• We have presented a clustering method based on gradual transformation of data and k-means. Instead of fitting the model to data, we fit the data to a model.

• The proposed method gives better mean square error than k-means.

University of Eastern Finland School of Computing P.O. Box 111 FIN- 80101 Joensuu FINLAND Tel. +358...

Documents

Transcript of University of Eastern Finland School of Computing P.O. Box 111 FIN- 80101 Joensuu FINLAND Tel. +358...