1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

17
1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University
  • date post

    21-Dec-2015
  • Category

    Documents

  • view

    218
  • download

    2

Transcript of 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

Page 1: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

1

Context-Aware Clustering

Junsong Yuan and Ying WuEECS Dept., Northwestern University

Page 2: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

2

Contextual pattern and co-occurrences

?

Spatial contexts provide useful cues for clustering

Page 3: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

3

K-means revisit

Assumption: data samples are independent

Binary label indicator

Limitation: contextual information of spatial dependency is not considered in clustering data samples

EM Update

Page 4: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

4

Clustering higher-level patterns

• Regularized k-means

Distortion in original feature space

Distortion in hamming spacecharactering contextual patterns

Same as traditionalK-means clustering

Regularization term due tocontextual patterns

Not a smooth term!

Page 5: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

5

Chicken and Egg Problem

• Hamming distance in clustering contextual patterns

• Matrix form

• Cannot minimize J1 and J2 separately !

J1 is coupled with J2

Page 6: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

6

Decoupling

Fix Update

Fix Update

Page 7: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

7

Nested-EM solution

NestedE-step

M-stepUpdate and separately

the nested-EM algorithm can converge in finite steps.

Theorem of convergence

Page 8: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

8

Simulation results (feature space)

Page 9: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

9

Simulation results (spatial space)

Page 10: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

10

Page 11: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

11

K-m

eans

Initializatio

n

Page 12: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

12

1st rou

nd

Fin

al Ph

rases

Page 13: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

13

Page 14: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

14

Page 15: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

15

Page 16: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

16

Multiple-feature clustering

• Dataset: handwritten numerical (‘0’-‘9’) from UCI data set– Each digit has three different types of features– Contextual pattern corresponds to compositional feature

• Different types of features serve as contexts of each other– Clustering each type of features into 10 “words”– Clustering 10 “phrases” based on a word-lexicon of size 3x10

Page 17: 1 Context-Aware Clustering Junsong Yuan and Ying Wu EECS Dept., Northwestern University.

17

Conclusion

• A context-aware clustering formulation proposed– Targets on higher-level compositional patterns in

terms of co-occurrences– Discovered contextual patterns can feed back to

improve the primitive feature clustering

• An efficient nested-EM solution which is guaranteed to converge in finite steps

• Successful applications in image pattern discovery and multiple-feature clustering– Can be applied to other general clustering problems