Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning...
Transcript of Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning...
![Page 1: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/1.jpg)
Image Analysis Lecture 9.1 - Segmentation
Idar Dyrdal
![Page 2: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/2.jpg)
Segmentation • Image segmentation is the process of
partitioning a digital image into multiple parts
• The goal is to divide the image into
meaningful and/or perceptually uniform regions
• Segmentation is typically used to locate
objects and boundaries of physical entities in the scene
• The segmentation process utilizes
available image information (graylevel, colour, texture, pixel position, …).
2
![Page 3: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/3.jpg)
Segmentation into three categories
3 Original image Segmented image
![Page 4: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/4.jpg)
Semantic Segmentation (meaningful regions)
Road
Building
Sky
Building
4
![Page 5: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/5.jpg)
Segmentation methods
• Active contours (Snakes, Scissors, Level Sets) • Split and merge (Watershed, Divisive & agglomerative clustering, Graph-based
segmentation) • K-means (parametric clustering) • Mean shift (non-parametric clustering) • Normalized cuts • Graph cuts • Graylevel thresholding • …
5
![Page 6: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/6.jpg)
Colour Segmentation - Example
6
Adaptive Weighted Distances
![Page 7: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/7.jpg)
Segmentation by thresholding
7
Number of pixels
Otsu’s method: • Automatic clustering-based thresholding • Minimization of intra-class variance • Analog to Fisher’s Discriminant Analysis
Graylevel
![Page 8: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/8.jpg)
Thresholding with Otsu’s method
8
3 thresholds
4 classes
![Page 9: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/9.jpg)
Binary segmentation – foreground vs. background
9
Threshold between two populations Threshold at given percentile
Number of pixels Number of pixels
Graylevel Graylevel
Foreground Background
Background
Foreground
![Page 10: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/10.jpg)
Binary thresholding – Object detection
10
Thermal image Thresholded image (Otsu’s method)
Global threshold selection threshold too low for detection of the object of interest
![Page 11: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/11.jpg)
Manual thresholding
11
Medium threshold High threshold
![Page 12: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/12.jpg)
Local thresholding
12
Threshold computed from graylevel statistics in selected window (Otsu’s method)
![Page 13: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/13.jpg)
Local thresholding using edge information
13
Edge image (Canny edge detector applied to selected window)
Thresholded window
Threshold = average graylevel along edges
![Page 14: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/14.jpg)
Object detection in video sequences (visible light)
14
• Change detection • Absolute difference image
(Current image - time averaged background image)
• Thresholding of difference image, i.e. Otsu’s method
• Requires fixed camera (or registration of images)
Daylight video frame Thresholded difference image
![Page 15: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/15.jpg)
Segmentation by clustering
15
Original image
Pixels represented as points in feature space
Segmented image
![Page 16: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/16.jpg)
K-means (parametric) clustering
1. Select K points (for example randomly) as initial cluster centers
2. Assign each sample to nearest cluster center
3. Compute new cluster centers (i.e. sample means)
4. Repeat steps 2 and 3 until no further re-assignments are possible.
16
Unlabeled dataset
![Page 17: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/17.jpg)
K-means clustering
Initial cluster centers (red, green and blue points) Samples assigned to nearest cluster center
17
![Page 18: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/18.jpg)
K-means clustering
Re-computed cluster centers Samples re-assigned to new cluster centers
18
![Page 19: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/19.jpg)
K-means clustering
Re-computed cluster centers Final clustering
19
![Page 20: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/20.jpg)
K-means clustering using colour
20
Original image Clustered image – 10 clusters
![Page 21: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/21.jpg)
Mean shift (non-parametric) segmentation
• Segmentation by clustering of the pixels in the image (colour and position)
• Non-parametric method (Parzen window technique) to find modes (i.e. peaks) in the density function
• All pixels climbing to the same peak are assigned to the same region.
(Szeliski: Computer Vision – Algorithms and Applications)
21
![Page 22: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/22.jpg)
Mean shift segmentation
22
(Szeliski: Computer Vision – Algorithms and Applications)
![Page 23: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/23.jpg)
Parzen method
23
Example:
Window (kernel) function:
Density estimate (smoothing of point cloud):
a
b
Original image
Plot of a vs. b for each pixel in Lab transformed image
![Page 24: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/24.jpg)
Mean shift segmentation
24
(Szeliski: Computer Vision – Algorithms and Applications)
![Page 25: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/25.jpg)
Mean Shift Segmentation - example
25 Original image Segmented in five categories
![Page 26: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/26.jpg)
Road segmentation for autonomous vehicles
26
Image data (graylevel, colour, local texture) from trapezoidal region is used to build a Gaussian model of the road surface.
Pixels with sufficiently high probability density with respect to the model are assigned to the road class (marked in green).
![Page 27: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/27.jpg)
Road segmentation – alternative approach
27
Original RGB image converted to an illumination invariant colour space (reduced variation due to sunlight and shadows). From this image a local entropy image is derived (Matlab: entropyfilt).
Segmentation by region growing of the local entropy image (Matlab: grayconnected) using the green dots (left image) as seed pixels.
![Page 28: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/28.jpg)
Morphological operations
• Non-linear filtering • Typically used to clean up binary images • Erosion: replace pixel value with minimum
in local neighborhood • Dilation: replace pixel value with maximum
in local neighborhood • Structuring element used to define the local
neighborhood:
A shape (in blue) and its morphological dilation (in green) and erosion (in yellow) by a diamond-shape structuring element.
28
![Page 29: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/29.jpg)
Morphological operations - Erosion
29
Structuring element (disk shaped)
![Page 30: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/30.jpg)
Morphological operations - Dilation
30
Structuring element (disk shaped)
![Page 31: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/31.jpg)
Opening = Erosion + Dilation
31
![Page 32: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/32.jpg)
Closing = Dilation + Erosion
32
![Page 33: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/33.jpg)
Opening - example
33
Thresholded image Result of opening
Disk shaped structuring element with radius = 2 pixels (5 x 5 filter mask)
![Page 34: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/34.jpg)
Closing - example
34
Thresholded image Result of opening
Disk shaped structuring element with radius = 2 pixels (5 x 5 filter mask)
![Page 35: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/35.jpg)
Active contours
(Szeliski: Computer Vision – Algorithms and Applications)
35
Fitting of curves to object boundaries: • Snakes (fitting of spline curves to strong
edges) • Intelligent scissors (interactive specification
of curves clinging to object boundaries) • Level set techniques (evolving boundaries
as the zero set of a characteristic function).
![Page 36: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/36.jpg)
Split and merge methods
Principles: • Recursive splitting of the image based on region
statistics • Hierarchical merging of pixels and regions • Combined splitting and merging
Methods: • Watershed segmentation • Region splitting (divisive clustering) • Region merging (agglomerative clustering) • Graph-based segmentation
(Szeliski: Computer Vision – Algorithms and Applications)
36
![Page 37: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/37.jpg)
Agglomerative clustering
37
Dendrogram
Dis
tanc
e m
easu
re
Distance measures
![Page 38: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/38.jpg)
Normalized cuts
Separation of groups with weak affinities (similarities) between nearby pixels
(Szeliski: Computer Vision – Algorithms and Applications)
38
![Page 39: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/39.jpg)
Graph cuts
(Szeliski: Computer Vision – Algorithms and Applications)
39
Energy-based methods for binary segmentation: • Grouping of pixels with similar statistics • Minimization of pixel-based energy function • Region-based and boundary-based energy
terms • Image represented as a graph • Cutting of weak edges, i.e. low similarity
between corresponding pixels.
![Page 40: Image Analysis Lecture 9.1 - Segmentation · • Image segmentation is the process of partitioning a digital image into multiple ... K-means clustering using colour . 20 ... image](https://reader035.fdocuments.net/reader035/viewer/2022070702/5e61fd6ed6d077089c7451d2/html5/thumbnails/40.jpg)
Summary
Image Segmentation: • Thresholding techniques • Clustering methods for segmentation • Morphological operations More information: Szeliski 5.1 - 5.5
40