Spectral clustering
-
Upload
so-yeon-kim -
Category
Data & Analytics
-
view
98 -
download
3
Transcript of Spectral clustering
![Page 1: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/1.jpg)
Pattern Recognition and Machine Learning Summer School 2014
![Page 2: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/2.jpg)
Hierarchical methods
Agglomerative clustering
Divisive clustering
Iterative methods
k‐means clustering
EM algorithm
Mean‐shift algorithm
Spectral clustering
Normalized cut
Ratio cut
Graph‐cut
![Page 3: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/3.jpg)
Clustering based on the spectrum of the graph
the multiset of the eigenvalues of the Laplacian matrix
Treats clustering as a graph partitioning problem without making specific assumptions on the form of the clusters.
Clusters points using eigenvectors of matrices derived from the data.
Maps data to a low‐dimensional space that are separated and can be easily clustered.
L = D (degree matrix) – W (adjacency matrix)
![Page 4: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/4.jpg)
affinity or similarity of the two nodes
• Affinity matrix
• Laplacian matrix
• Similarity measures
– Cosine measure
– Bhattacharyya coefficient
• Distance measures
– Euclidean distance
– Manhattan distance
– Maximum distance …
![Page 5: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/5.jpg)
Find a label vector x !
Convert the discrete problem to continuous domain
But, NP-hard problem..
Average association
![Page 6: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/6.jpg)
Points in dominant cluster are non-zero
X(label) is divided
into 0 and 1
![Page 7: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/7.jpg)
But, favor for small and isolated clusters
Sum of the weights to cut edges
![Page 8: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/8.jpg)
Find the second minimum eigenvector
=
= assoc(G1,G) - assoc(G1,G1)
= cut(G1, G2)
(D-W) * 1 = 0 * 1
The smallest eigenvector is 1.
![Page 9: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/9.jpg)
y : binary vector representing the
cluster association
Favors partitioning with equal size segments
The second smallest eigenvalue
Based on the edge weights
‘NP-complete’
Find z in
![Page 10: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/10.jpg)
Pros Generic framework, can be used with many different
features
Cons High storage requirement and time complexity
Bias towards partitioning into equal segments
Need the number of clusters as parameter
![Page 11: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/11.jpg)
Incremental partitioning
Partition using only one eigenvector at a time
Use procedure recursively
Batch partitioning
Use k eigenvectors
Directly compute k‐way partitioning, for example, by k‐means clustering
Usually performs better
![Page 12: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/12.jpg)
Find a low‐dimensional
embedding by
eigen‐decomposition
separates data while projecting in the low dimensional space
allows clustering of non‐convex data effectively
![Page 13: Spectral clustering](https://reader033.fdocuments.net/reader033/viewer/2022042512/55a6b4841a28ab0b2c8b463e/html5/thumbnails/13.jpg)
Thank you !