VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline •...
Transcript of VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline •...
![Page 1: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/1.jpg)
Mestrado em Ciência de Computadores
Mestrado Integrado em Engenharia de Redes e
Sistemas Informáticos
VC 14/15 – TP8
Segmentation
Miguel Tavares Coimbra
![Page 2: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/2.jpg)
VC 14/15 - TP8 - Segmentation
Outline
• Thresholding
• Geometric structures
• Hough Transform
Acknowledgements: Most of this course is based on the excellent courses offered by Prof. Shree Nayar at
Columbia University, USA and by Prof. Srinivasa Narasimhan at CMU, USA. Please acknowledge the original
source when reusing these slides for academic purposes.
![Page 3: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/3.jpg)
VC 14/15 - TP8 - Segmentation
Topic: Thresholding
• Thresholding
• Geometric structures
• Hough Transform
![Page 4: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/4.jpg)
VC 14/15 - TP8 - Segmentation
Boundaries of Objects
Marked by many users
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/bench/html/images.html
![Page 5: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/5.jpg)
VC 14/15 - TP8 - Segmentation
Boundaries of Objects from
Edges
Brightness Gradient (Edge detection)
• Missing edge continuity, many spurious edges
![Page 6: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/6.jpg)
VC 14/15 - TP8 - Segmentation
Boundaries of Objects from
Edges
Multi-scale Brightness Gradient
• But, low strength edges may be very important
![Page 7: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/7.jpg)
VC 14/15 - TP8 - Segmentation
Image
Machine Edge Detection
Human Boundary Marking
![Page 8: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/8.jpg)
VC 14/15 - TP8 - Segmentation
Boundaries in Medical Imaging
Detection of cancerous regions.
[Foran, Comaniciu, Meer, Goodell, 00]
![Page 9: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/9.jpg)
VC 14/15 - TP8 - Segmentation
Boundaries in Ultrasound
Images
Hard to detect in the presence of large amount of speckle noise
![Page 10: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/10.jpg)
VC 14/15 - TP8 - Segmentation Sometimes hard even for humans!
![Page 11: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/11.jpg)
VC 14/15 - TP8 - Segmentation
What is ‘Segmentation’?
• Separation of the
image in different
areas.
– Objects.
– Areas with similar
visual or semantic
characteristics.
Not trivial! It is the holy grail of
most computer vision problems!
![Page 12: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/12.jpg)
VC 14/15 - TP8 - Segmentation
Subjectivity
• A ‘correct’
segmentation result is
only valid for a
specific context.
– Subjectivity!
– Hard to implement.
– Hard to evaluate.
Person Suitcase
Face
What is the
correct
segmentation?
![Page 13: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/13.jpg)
VC 14/15 - TP8 - Segmentation
Core Technique: Thresholding
• Divide the image into two areas:
– 1, if f(x,y)>K
– 0, if f(x,y)<=K
• Not easy to find the ideal k magic number.
• Core segmentation technique
– Simple
– Reasonably effective
Adequate
threshold
![Page 14: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/14.jpg)
VC 14/15 - TP8 - Segmentation
Finding the ‘magic number’
Wrong!
(k = 128) Correct
(k = 74)
![Page 15: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/15.jpg)
VC 14/15 - TP8 - Segmentation
http://www.cee.hw.ac.uk/hipr/html/adpthrsh.html
Global thresholds are not
always adequate...
![Page 16: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/16.jpg)
VC 14/15 - TP8 - Segmentation
Adaptive Thresholding
• Adapt the threshold
value for each pixel.
• Use characteristics of
nearby pixels.
• How?
– Mean
– Median
– Mean + K
– ...
Mean of 7x7 neighborhood
![Page 17: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/17.jpg)
VC 14/15 - TP8 - Segmentation 7x7 window; K = 7 75x75 window; K = 10
![Page 18: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/18.jpg)
VC 14/15 - TP8 - Segmentation
Topic: Geometric structures
• Thresholding
• Geometric structures
• Hough Transform
![Page 19: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/19.jpg)
VC 14/15 - TP8 - Segmentation
Points
• What is a point?
– Pixel with a significant
illumination difference
to its neighbors.
– Group of pixels?
• Spatial Mask!
• Need to define a
threshold K.
-1 -1 -1
-1 8 -1
-1 -1 -1
point!
9
1
KR
zwRi
ii
![Page 20: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/20.jpg)
VC 14/15 - TP8 - Segmentation
Lines
• Spatial filter
– One per line direction
– Sensitive to line width
-1 -1 -1
2 2 2
-1 -1 -1
-1 2 -1
-1 2 -1
-1 2 -1
Horizontal Vertical
Diagonal?
![Page 21: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/21.jpg)
VC 14/15 - TP8 - Segmentation
Edges
• Edge:
– Spatial discontinuity of pixel amplitude.
– High spatial gradient
• First derivative (peak)
• Second derivative (zero crossing)
![Page 22: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/22.jpg)
VC 14/15 - TP8 - Segmentation
Popular operators
• Edge detection
– Great utility for
several problems.
– Well studied
problem.
• A variety of
solutions exists.
![Page 23: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/23.jpg)
VC 14/15 - TP8 - Segmentation
![Page 24: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/24.jpg)
VC 14/15 - TP8 - Segmentation
Processing Edge Images
Image
Edge detection
and Thresholding Noisy edge image
Incomplete boundaries
Edge Tracking
Thinning
![Page 25: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/25.jpg)
VC 14/15 - TP8 - Segmentation
Edge Tracking Methods
• Adjusting a priori Boundaries
Given: Approximate Location of Boundary
Task: Find Accurate Location of Boundary
• Search for STRONG EDGES along normals to approximate boundary.
• Fit curve (eg., polynomials) to strong edges.
![Page 26: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/26.jpg)
VC 14/15 - TP8 - Segmentation
Edge Tracking Methods
• Divide and Conquer Given: Boundary lies between points A and B
Task: Find Boundary
• Connect A and B with Line
• Find strongest edge along line bisector
• Use edge point as break point
• Repeat
![Page 27: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/27.jpg)
VC 14/15 - TP8 - Segmentation
Fitting Lines to Edges (Least
Squares) cmxy
cmxy ii
),( ii yx
y
x
Given: Many pairs
Find: Parameters
Minimize: Average square distance:
Using:
Note:
),( ii yx),( cm
i
ii
N
cmxyE
2)(
0&0
c
E
m
Exmyc
i
i
i
ii
xx
yyxx
m2)(
))((
N
y
y i
i
N
x
x i
i
![Page 28: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/28.jpg)
VC 14/15 - TP8 - Segmentation
Topic: Hough Transform
• Thresholding
• Geometric structures
• Hough Transform
![Page 29: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/29.jpg)
VC 14/15 - TP8 - Segmentation
Hough Transform
• Elegant method for direct object
recognition
• Edges need not be connected
• Complete object need not be visible
• Key Idea: Edges VOTE for the possible
model
![Page 30: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/30.jpg)
VC 14/15 - TP8 - Segmentation
Image and Parameter Spaces
cmxy
),( ii yx
y
x
Equation of Line:
Find:
Consider point:
cmxy
),( cm
),( ii yx
iiii ymxcorcmxy m
c
),( cm
Image
Space
Parameter
Space
Parameter space also called Hough Space
![Page 31: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/31.jpg)
VC 14/15 - TP8 - Segmentation
Line Detection by Hough Transform
y
x
),( cm
Parameter Space
1 1
1 1
1 1
2
1 1
1 1
1 1
),( cmA
Algorithm:
• Quantize Parameter Space
• Create Accumulator Array
• Set
• For each image edge increment:
• If lies on the line:
• Find local maxima in
),( cm
),( cmA
cmcmA ,0),(
),( ii yx
1),(),( cmAcmA
),( cm
),( cmA
ii ymxc
![Page 32: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/32.jpg)
VC 14/15 - TP8 - Segmentation
Better Parameterization
NOTE:
Large Accumulator
More memory and computations
Improvement:
Line equation:
Here
Given points find
m
sincos yx
max0
20
(Finite Accumulator Array Size)
),( ii yx ),(
),( ii yx
y
x
Image
Space
Hough
Space
?
Hough Space Sinusoid
![Page 33: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/33.jpg)
VC 14/15 - TP8 - Segmentation
Image space Votes
Horizontal axis is θ,
vertical is rho.
![Page 34: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/34.jpg)
VC 14/15 - TP8 - Segmentation
![Page 35: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/35.jpg)
VC 14/15 - TP8 - Segmentation
![Page 36: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/36.jpg)
VC 14/15 - TP8 - Segmentation
Mechanics of the Hough Transform
• Difficulties – how big should the
cells be? (too big, and we merge quite different lines; too small, and noise causes lines to be missed)
• How many lines? – Count the peaks in the
Hough array
– Treat adjacent peaks as a single peak
• Which points belong to each line? – Search for points close
to the line
– Solve again for line and iterate
![Page 37: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/37.jpg)
VC 14/15 - TP8 - Segmentation
Fewer votes land in a
single bin when noise increases.
![Page 38: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/38.jpg)
VC 14/15 - TP8 - Segmentation
Adding more clutter increases
number of bins with false peaks.
![Page 39: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/39.jpg)
Real World Example
Original Edge Detection
Found Lines
Parameter Space
![Page 40: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/40.jpg)
VC 14/15 - TP8 - Segmentation
Other shapes
Original Edges when using circle
model
![Page 41: VC 14/15 TP8 Segmentationmcoimbra/lectures/VC...VC 14/15 - TP8 - Segmentation Outline • Thresholding • Geometric structures • Hough Transform Acknowledgements: Most of this course](https://reader033.fdocuments.net/reader033/viewer/2022060900/609df7bba655da753a0879e8/html5/thumbnails/41.jpg)
VC 14/15 - TP8 - Segmentation
Resources
• Gonzalez & Woods – Chapter 7
• N. Otsu, “A threshold selection method
from gray-level histograms,” IEEE Trans.
Sys., Man., Cyber., vol. 9, pp. 62–66,
1979.