Lecture’12:’Clustering’and’ Segmentaon’ - Stanford...
39
Lecture 12 - Fei-Fei Li Lecture 12: Clustering and Segmenta5on Professor FeiFei Li Stanford Vision Lab 16Oct13 1
Transcript of Lecture’12:’Clustering’and’ Segmentaon’ - Stanford...
Lecture 12 - !!!
Fei-Fei Li!
Lecture 12: Clustering and Segmenta5on
Professor Fei-‐Fei Li Stanford Vision Lab
16-‐Oct-‐13 1
Lecture 12 - !!!
Fei-Fei Li!
What we will learn today
• Introduc5on to segmenta5on and clustering • Gestalt theory for perceptual grouping • Agglomera5ve clustering
16-‐Oct-‐13 2
Reading: [FP] Chapters: 14.2, 14.4
Lecture 12 - !!!
Fei-Fei Li! 16-‐Oct-‐13 3
Lecture 12 - !!!
Fei-Fei Li!
Image Segmenta5on
• Goal: iden5fy groups of pixels that go together
Slide credit: Steve Seitz, Kristen Grauman
16-‐Oct-‐13 4
Lecture 12 - !!!
Fei-Fei Li!
The Goals of Segmenta5on
• Separate image into coherent “objects” Image Human segmenta5on
Slide credit: Svetlana Lazebnik
16-‐Oct-‐13 5
Lecture 12 - !!!
Fei-Fei Li!
The Goals of Segmenta5on
• Separate image into coherent “objects” • Group together similar-‐looking pixels for efficiency of further processing
X. Ren and J. Malik. Learning a classifica5on model for segmenta5on. ICCV 2003.
“superpixels”
Slid
e cr
edit
: Sv
etla
na L
azeb
nik
16-‐Oct-‐13 6
Lecture 12 - !!!
Fei-Fei Li!
Segmenta5on for feature support
50x50 Patch 50x50 Patch
Slide: Derek Hoiem
16-‐Oct-‐13 7
Lecture 12 - !!!
Fei-Fei Li!
Segmenta5on for efficiency
[Felzenszwalb and Huttenlocher 2004]
[Hoiem et al. 2005, Mori 2005] [Shi and Malik 2001] Slide: Derek Hoiem
16-‐Oct-‐13 8
Lecture 12 - !!!
Fei-Fei Li!
Segmenta5on as a result
Rother et al. 2004
16-‐Oct-‐13 9
Lecture 12 - !!!
Fei-Fei Li!
Types of segmenta5ons
Oversegmentation Undersegmentation
Multiple Segmentations
16-‐Oct-‐13 10
Lecture 12 - !!!
Fei-Fei Li!
One way to think about “segmenta5on” is Clustering
Clustering: group together similar points and represent them with a single token
Key Challenges: 1) What makes two points/images/patches similar? 2) How do we compute an overall grouping from pairwise similari5es?
Slide: Derek Hoiem
16-‐Oct-‐13 11
Lecture 12 - !!!
Fei-Fei Li!
Why do we cluster? • Summarizing data
– Look at large amounts of data – Patch-‐based compression or denoising – Represent a large con5nuous vector with the cluster number
• Coun2ng
– Histograms of texture, color, SIFT vectors
• Segmenta2on – Separate the image into different regions
• Predic2on – Images in the same cluster may have the same labels
Slide: Derek Hoiem
16-‐Oct-‐13 12
Lecture 12 - !!!
Fei-Fei Li!
How do we cluster?
• Agglomera5ve clustering – Start with each point as its own cluster and itera5vely merge the closest clusters
• K-‐means (next lecture) – Itera5vely re-‐assign points to the nearest cluster center
• Mean-‐shig clustering (CS231a, winter quarter) – Es5mate modes of pdf
• Spectral clustering (CS231a, winter quarter) – Split the nodes in a graph based on assigned links with similarity weights
16-‐Oct-‐13 13
Lecture 12 - !!!
Fei-Fei Li!
General ideas
• Tokens – whatever we need to group (pixels, points, surface elements, etc., etc.)
• Bokom up clustering
– tokens belong together because they are locally coherent
• Top down clustering – tokens belong together because they lie on the same visual en5ty (object, scene…)
> These two are not mutually exclusive
16-‐Oct-‐13 14
Lecture 12 - !!!
Fei-Fei Li!
Examples of Grouping in Vision
Determining image regions
Grouping video frames into shots
Object-‐level grouping
Figure-‐ground
Slide credit: Kristen Grauman
What things should be grouped?
What cues indicate groups?
16-‐Oct-‐13 15
Lecture 12 - !!!
Fei-Fei Li!
Similarity
Slide credit: Kristen Grauman
16-‐Oct-‐13 16
Lecture 12 - !!!
Fei-Fei Li!
Symmetry
Slide credit: Kristen Grauman
16-‐Oct-‐13 17
Lecture 12 - !!!
Fei-Fei Li!
Common Fate
Image credit: Arthus-‐Bertrand (via F. Durand)
Slide credit: Kristen Grauman
16-‐Oct-‐13 18
Lecture 12 - !!!
Fei-Fei Li!
Proximity
Slide credit: Kristen Grauman
16-‐Oct-‐13 19
Lecture 12 - !!!
Fei-Fei Li!
Muller-‐Lyer Illusion
• What makes the bokom line look longer than the top line?
16-‐Oct-‐13 20
Lecture 12 - !!!
Fei-Fei Li!
What we will learn today
• Introduc5on to segmenta5on and clustering • Gestalt theory for perceptual grouping • Agglomera5ve clustering
16-‐Oct-‐13 21
Lecture 12 - !!!
Fei-Fei Li!
The Gestalt School • Grouping is key to visual percep5on • Elements in a collec5on can have proper5es that result from rela5onships – “The whole is greater than the sum of its parts”
Illusory/subjec5ve contours
Occlusion
Familiar configura5on
hkp://en.wikipedia.org/wiki/Gestalt_psychology Slid
e cr
edit
: Sv
etla
na L
azeb
nik
16-‐Oct-‐13 22
Lecture 12 - !!!
Fei-Fei Li!
Gestalt Theory • Gestalt: whole or group
– Whole is greater than sum of its parts – Rela5onships among parts can yield new proper5es/features
• Psychologists iden5fied series of factors that predispose set of elements to be grouped (by human visual system)
Untersuchungen zur Lehre von der Gestalt, Psychologische Forschung, Vol. 4, pp. 301-350, 1923 http://psy.ed.asu.edu/~classics/Wertheimer/Forms/forms.htm
“I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, and trees.”
Max Wertheimer (1880-1943)
16-‐Oct-‐13 23
Lecture 12 - !!!
Fei-Fei Li!
Gestalt Factors
• These factors make intui5ve sense, but are very difficult to translate into algorithms.
Imag
e so
urce
: Fo
rsyt
h &
Pon
ce
16-‐Oct-‐13 24
Lecture 12 - !!!
Fei-Fei Li!
Con5nuity through Occlusion Cues
16-‐Oct-‐13 25
Lecture 12 - !!!
Fei-Fei Li!
Con5nuity through Occlusion Cues
Con5nuity, explana5on by occlusion
16-‐Oct-‐13 26
Lecture 12 - !!!
Fei-Fei Li!
Con5nuity through Occlusion Cues
Imag
e so
urce
: Fo
rsyt
h &
Pon
ce
16-‐Oct-‐13 27
Lecture 12 - !!!
Fei-Fei Li!
Con5nuity through Occlusion Cues
Imag
e so
urce
: Fo
rsyt
h &
Pon
ce
16-‐Oct-‐13 28
Lecture 12 - !!!
Fei-Fei Li!
Figure-‐Ground Discrimina5on
16-‐Oct-‐13 29
Lecture 12 - !!!
Fei-Fei Li!
The Ul5mate Gestalt?
16-‐Oct-‐13 30
Lecture 12 - !!!
Fei-Fei Li!
What we will learn today
• Introduc5on to segmenta5on and clustering • Gestalt theory for perceptual grouping • Agglomera5ve clustering
16-‐Oct-‐13 31
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
16-‐Oct-‐13 32
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
16-‐Oct-‐13 33
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
16-‐Oct-‐13 34
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
16-‐Oct-‐13 35
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering
16-‐Oct-‐13 36
Slide credit: Andrew Moore
Lecture 12 - !!!
Fei-Fei Li!
Agglomera5ve clustering How to define cluster similarity? -‐ Average distance between points,
maximum distance, minimum distance -‐ Distance between means or medoids
How many clusters? -‐ Clustering creates a dendrogram (a tree) -‐ Threshold based on max number of clusters
or based on distance between merges
dist
ance
16-‐Oct-‐13 37
Lecture 12 - !!!
Fei-Fei Li!
Conclusions: Agglomera5ve Clustering
Good • Simple to implement, widespread applica5on • Clusters have adap5ve shapes • Provides a hierarchy of clusters Bad • May have imbalanced clusters • S5ll have to choose number of clusters or threshold
• Need to use an “ultrametric” to get a meaningful hierarchy
16-‐Oct-‐13 38
Lecture 12 - !!!
Fei-Fei Li!
What we have learned today?
• Introduc5on to segmenta5on and clustering • Gestalt theory for perceptual grouping • Agglomera5ve clustering
16-‐Oct-‐13 39
