Segmentation and Clustering. Déjà Vu? Q: Haven’t we already seen this with snakes?Q: Haven’t...
-
date post
21-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of Segmentation and Clustering. Déjà Vu? Q: Haven’t we already seen this with snakes?Q: Haven’t...
Segmentation and Segmentation and ClusteringClustering
Déjà Vu?Déjà Vu?
• Q: Haven’t we already seen this with Q: Haven’t we already seen this with snakes?snakes?
• A: no, that was about boundaries, A: no, that was about boundaries, this is about interiorsthis is about interiors– Want regions of similar contentsWant regions of similar contents
Segmentation via ClusteringSegmentation via Clustering
• Segmentation:Segmentation:Divide imageDivide imageinto regionsinto regionsof similar of similar contentscontents
• Clustering:Clustering:Aggregate pixelsAggregate pixelsinto regionsinto regionsof similar of similar contentscontents
But Wait!But Wait!
• We speak of segmenting foregroundWe speak of segmenting foregroundfrom backgroundfrom background
• Segmenting out skin colorsSegmenting out skin colors
• Segmenting out the moving personSegmenting out the moving person
• How do these relate to “similar How do these relate to “similar regions”?regions”?
Segmentation and ClusteringSegmentation and Clustering
• Defining regionsDefining regions– Should they be compact? Smooth Should they be compact? Smooth
boundary?boundary?
• Defining similarityDefining similarity– Color, texture, motion, …Color, texture, motion, …
• Defining similarity of regionsDefining similarity of regions– Minimum distance, mean, maximumMinimum distance, mean, maximum
Segmentation and Clustering Segmentation and Clustering ApplicationsApplications
SemanticsSemantics
““IntelligentIntelligentscissors”scissors”
FindingFindingskin-coloredskin-colored
regionsregions
Foreground /Foreground /backgroundbackground
segmentationsegmentation
Finding theFinding themoving objectsmoving objects
Finding theFinding thecars in acars in a
video sequencevideo sequence
ThemesThemes
• Energy minimizationEnergy minimization– Snakes: discretize, greedy minimizationSnakes: discretize, greedy minimization– Snakes, shape from shading: differential Snakes, shape from shading: differential
eqneqn– Stereo with smoothness: min-cost graph Stereo with smoothness: min-cost graph
cutscuts– Today: Today: kk-means clustering-means clustering
• StatisticsStatistics
• TemplatesTemplates
Hot topic in vision research: combining theseHot topic in vision research: combining these
Segmentation and Clustering Segmentation and Clustering ApplicationsApplications
““IntelligentIntelligentscissors”scissors”
FindingFindingskin-coloredskin-colored
regionsregions
Foreground /Foreground /backgroundbackground
segmentationsegmentation
Finding theFinding themoving objectsmoving objects
Finding theFinding thecars in acars in a
video sequencevideo sequence
StatisticsStatistics TemplatesTemplates
Clustering Based on ColorClustering Based on Color
• Let’s make a few concrete choices:Let’s make a few concrete choices:– Arbitrary regionsArbitrary regions– Similarity based on color onlySimilarity based on color only– Similarity of regions =Similarity of regions =
distance between mean colorsdistance between mean colors
Simple Agglomerative Simple Agglomerative ClusteringClustering
• Start with each pixel in its own Start with each pixel in its own clustercluster
• Iterate:Iterate:– Find pair of clusters with smallestFind pair of clusters with smallest
inter-cluster distanceinter-cluster distance– MergeMerge
• Stopping thresholdStopping threshold
Simple Divisive ClusteringSimple Divisive Clustering
• Start with whole image in one clusterStart with whole image in one cluster
• Iterate:Iterate:– Find cluster with largest intra-cluster Find cluster with largest intra-cluster
variationvariation– Split into two pieces that yield largest Split into two pieces that yield largest
inter-cluster distanceinter-cluster distance
• Stopping thresholdStopping threshold
Difficulties with Simple Difficulties with Simple ClusteringClustering
• Many possibilities at each iterationMany possibilities at each iteration
• Computing distance between clusters or Computing distance between clusters or optimal split expensiveoptimal split expensive
• Heuristics to speed this up:Heuristics to speed this up:– For agglomerative clustering, approximate For agglomerative clustering, approximate
each cluster by average for distance each cluster by average for distance computationscomputations
– For divisive clustering, use summary For divisive clustering, use summary (histogram) of a region to compute split(histogram) of a region to compute split
kk-means Clustering-means Clustering
1.1. Pick number of clusters Pick number of clusters kk
2.2. Randomly scatter Randomly scatter kk “cluster centers” in color “cluster centers” in color spacespace
3.3. Repeat:Repeat:a.a. Assign each data point to its closest cluster Assign each data point to its closest cluster
centercenterb.b. Move each cluster center to the mean of the Move each cluster center to the mean of the
points assigned to itpoints assigned to it
• Instead of merging or splitting, start Instead of merging or splitting, start out with the clusters and move them out with the clusters and move them aroundaround
kk-means Clustering-means Clustering
kk-means Clustering-means Clustering
kk-means Clustering-means Clustering
kk-means Clustering-means Clustering
kk-means Clustering-means Clustering
kk-means Clustering-means Clustering
kk-means Clustering-means Clustering
kk-means Clustering-means Clustering
k-means Clusteringk-means Clustering
• This process always converges (to This process always converges (to something)something)– Not necessarily globally-best assignmentNot necessarily globally-best assignment
• Informal proof: look at energy minimizationInformal proof: look at energy minimization
– Reclassifying points reduces (or maintains) energyReclassifying points reduces (or maintains) energy– Recomputing centers reduces (or maintains) energyRecomputing centers reduces (or maintains) energy– Can’t reduce energy foreverCan’t reduce energy forever
points clusters
2
i jijji assignedxxE
points clusters
2
i jijji assignedxxE
Results of ClusteringResults of Clustering
Original ImageOriginal Image kk-means, -means, kk=5=5 kk-means, -means, kk=11=11
Results of ClusteringResults of Clustering
Sample clusters withSample clusters with k k-means clustering-means clusteringbased on colorbased on color
Other Distance MeasuresOther Distance Measures
• Suppose we want to have compact Suppose we want to have compact regionsregions
• New feature space: 5DNew feature space: 5D(2 spatial coordinates, 3 color (2 spatial coordinates, 3 color components)components)
• Points close in this space are close Points close in this space are close both in color and in actual proximityboth in color and in actual proximity
Results of ClusteringResults of Clustering
Sample clusters with Sample clusters with kk-means clustering-means clusteringbased on color and distancebased on color and distance
Other Distance MeasuresOther Distance Measures
• Problem with simple Euclidean distance:Problem with simple Euclidean distance:what if coordinates range from 0-1000 but what if coordinates range from 0-1000 but colors only range from 0-255?colors only range from 0-255?– Depending on how things are scaled, gives Depending on how things are scaled, gives
different weight to different kinds of datadifferent weight to different kinds of data
• Weighted Euclidean distance: adjust Weighted Euclidean distance: adjust weights to emphasize different dimensionsweights to emphasize different dimensions
22)( iii yxcyx 22)( iii yxcyx
Mahalanobis DistanceMahalanobis Distance
• Automatically assign weights based Automatically assign weights based on actual variation in the dataon actual variation in the data
where C is covariance matrix of all where C is covariance matrix of all pointspoints
• Gives each dimension “equal” weightGives each dimension “equal” weight
• Also accounts for correlations Also accounts for correlations between different dimensionsbetween different dimensions
)(1T2yxyxyx
C )(1T2yxyxyx
C