3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
3-D Computer Vision3-D Computer Vision CSc 83020 CSc 83020
Clustering methods and boundary Clustering methods and boundary representationsrepresentations
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Image SegmentationImage Segmentation Generate clusters (regions) of pixels Generate clusters (regions) of pixels
that correspond to meaningful entities.that correspond to meaningful entities. Use metrics of “closeness” between Use metrics of “closeness” between
values.values. Use algorithms for combining “close” Use algorithms for combining “close”
values.values. Apply other constraints (connectivity).Apply other constraints (connectivity).
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Example (Forsyth & Ponce)Example (Forsyth & Ponce)
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Simple Clustering MethodsSimple Clustering Methods
Divisive clusteringDivisive clustering Everything is a big cluster at beginning Everything is a big cluster at beginning Split recursivelySplit recursively
Agglomerative clusteringAgglomerative clustering Each data (pixel) is a clusterEach data (pixel) is a cluster MergeMerge
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Simple clusteringSimple clustering
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Simple clusteringSimple clustering
What is a good inter-cluster What is a good inter-cluster distance?distance?
How many clusters are there?How many clusters are there?
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Simple clusteringSimple clustering
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Clustering by K-meansClustering by K-means
Assume that the number of clusters (k) is known.Assume that the number of clusters (k) is known. Each cluster has a center CEach cluster has a center Ci i (i=1..k)(i=1..k) Each data-point is a vector Each data-point is a vector xxj j (j=1..Number of pixels)(j=1..Number of pixels)
Examples:Examples: xj=[x-coord, y-coord, gray-value]xj=[x-coord, y-coord, gray-value]
or xj=[gray-value]or xj=[gray-value]or xj=[red-value, green-value, blue-value]or xj=[red-value, green-value, blue-value]
Assume that elements are close to center of clusters.Assume that elements are close to center of clusters. Minimize:Minimize:
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Clustering by K-meansClustering by K-means Iterative algorithm:Iterative algorithm:
Allocate each point to center of closest Allocate each point to center of closest cluster (assuming centers are known)cluster (assuming centers are known)
Calculate centers of clusters (assuming Calculate centers of clusters (assuming allocations are known)allocations are known)
How do we start?How do we start?
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Clustering by K-meansClustering by K-means
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Example (Forsyth & Ponce)Example (Forsyth & Ponce)
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Example (Forsyth & Ponce)Example (Forsyth & Ponce)
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Example (Forsyth & Ponce)Example (Forsyth & Ponce)
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
RANSACRANSAC
RANRANdom dom SASAmple mple CConsensusonsensus Model fitting methodModel fitting method Line-fitting exampleLine-fitting example
Fitting a line to a set of edges with 50% Fitting a line to a set of edges with 50% outliersoutliers
Least squares would failLeast squares would fail Solution: M-estimator or RANSACSolution: M-estimator or RANSAC
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
RANSAC (line-fitting RANSAC (line-fitting example)example)
Two edges (wout normal) define a line.Two edges (wout normal) define a line. General idea:General idea:
Pick two points.Pick two points. Write the equation of the line.Write the equation of the line. Check how many other points are “close” to line.Check how many other points are “close” to line. If number of “close” points is above threshold, doneIf number of “close” points is above threshold, done Otherwise, pick two new points.Otherwise, pick two new points.
Questions:Questions: Which points to pick?Which points to pick? Complexity in worst case?Complexity in worst case?
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
RANSAC (line-fitting RANSAC (line-fitting example)example)
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
RANSAC (line-fitting RANSAC (line-fitting example)example)
How large should k (max. number of iterations) How large should k (max. number of iterations) be?be?
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
RANSAC (line-fitting RANSAC (line-fitting example)example)
How large should k (max. number of iterations) be?How large should k (max. number of iterations) be? Assume that Assume that w w is the probability of picking a “correct” is the probability of picking a “correct”
point (i.e. a point on the line).point (i.e. a point on the line). Since we are picking Since we are picking n n (=2 for lines) points, 1-(=2 for lines) points, 1-wwn n is the is the
probability of picking probability of picking nn “wrong” points. “wrong” points. If we iterate k times we want the probability of failure If we iterate k times we want the probability of failure
to be small: i.e.to be small: i.e. (1-(1-wwnn))k k = z => k = log(z)/log(1-= z => k = log(z)/log(1-wwnn))
If z=0.1 and w=0.5 then k=8 (n=2)If z=0.1 and w=0.5 then k=8 (n=2) If z=0.01 and w=0.5 then k=16 (n=2)If z=0.01 and w=0.5 then k=16 (n=2) If z=0.001 and w=0.1 then k = 687 (n=2)If z=0.001 and w=0.1 then k = 687 (n=2)
How is the formula affected by n?How is the formula affected by n?
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
RANSAC (Conclusions)RANSAC (Conclusions) When can this method be successful?When can this method be successful? Can we detect circles?Can we detect circles?
In that case how many points do you need to fit a In that case how many points do you need to fit a circle?circle?
Can we detect other shapes?Can we detect other shapes?
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Boundary representation of Boundary representation of regionsregions
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Representation of 2-D Representation of 2-D Geometric StructuresGeometric Structures
To MATCH image boundary/region with MODEL
boundary/region, they must represented in the same
manner.
•Boundary Representation
•Snakes – Extraction of arbitrary contours from image.
•Region Representation
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Representation IssuesRepresentation Issues
Compact: Easy to Store & Match.Compact: Easy to Store & Match. Easy to manipulate & compute Easy to manipulate & compute
properties.properties. Captures Object/Model shape.Captures Object/Model shape. Computationally efficient.Computationally efficient.
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Boundary RepresentationBoundary Representation
Polylines: concatenation of line segments.
Breakpoint
Matching on the basis of:# of line segmentslengths of line segmentsangle between consecutive segments
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Running Least Squares Running Least Squares MethodMethod
ei
Move along the boundaryAt each point find line that fits previous points
(Least Squares)Compute the fit error E=Sum(ei) using previous pointsIf E exceeds threshold, declare breakpoint and start
a new running line fit.
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Approximating Curves with Approximating Curves with PolylinesPolylines
•Draw Straight line between end-points of curve•For every curve point find distance to line.•If distance is less than tolerance level for all points, Exit•Else, pick point that is farthest away and use as breakpoint.
Introduce new segments.•Recursively apply algorithm to new segments.
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Ψ-s CurveΨ-s Curve
s ψ
x
y
s
Ψ
1 2
3 4
5 6
x
y
12
3
4
5
6
π
2π
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Ψ-s CurveΨ-s Curve
s ψ
x
y
s
Ψ
1 2
3 4
5 6
x
y
12
3
4
5
6
Ψ-s is periodic (2π)Horizontal section in Ψ-s curve
=> straight line in the ImageNon horizontal section in Ψ-s
=> arc in Image
π
2π
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Slope-Density FunctionSlope-Density Function
s ψ
x
y H(Ψ)
Ψ
π 2π
HISTOGRAMLines
Arcs
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Slope-Density FunctionSlope-Density Function
s ψ
x
y H(Ψ)
Ψ
π 2π
HISTOGRAM
H(Ψ) shifts as objects rotates.H(Ψ) wraps around
Lines
Arcs
Fourier DescriptorsFourier Descriptors
s ψ
x
y
Find: Ψ(s)Define:Φ(s)= Ψ(s)-(2πs)/P
P: Perimeter.2π: Period of Φ(s)
Φ(s) is a Continuous, Periodic function.Fourier Series for Periodic Functions:
Fourier Coefficients:
Pes
k
sikk
2,)( 0
1
0
P
sikk dsesP 0
0)(1
Φk’s capture shape informationMatch shapes by matching Φk’sUse finite number of Φk’s
Fourier DescriptorsInput Shape Power Spectrum Reconstruction
# of coefficients
Φk
Note: Reconstructed shapes are often not closed sinceonly a finite # of Φk’s are used.
1
0
)()(N
iii sBvsX
B-SplinesB-Splines•Piecewise continuous polynomials used to INTERPOLATE between Data Points.•Smooth, Flexible, Accurate.
s=0
s=1s=2
x0
x1x2
xi
x
s
Spline X(s)
Data Point
We want to find X(s) from points xiCubic Polynomials are popular:
BASIS FUNCTIONS
COEFFICIENTS
N
B-SplinesB-SplinesBi(s) has limited support (4 spans)
i-2 i-1 i i+1 i+2
Each span (i->i+1) has only 4 non-zero Basis Functions:Bi-1(s), Bi(s), Bi+1(s), Bi+2(s)
i i+1
Bi(s)
Bi-1(s) Bi+2(s)
Bi+1(s)
t=0 t=1t
s:
s:
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
B-SplinesB-Splines
i i+1
Bi(s)
Bi-1(s) Bi+2(s)
Bi+1(s)
t=0 t=1t
CubicPolynomials
1/6
4/6
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
B-SplinesB-Splines
i i+1
Bi(s)
Bi-1(s) Bi+2(s)
Bi+1(s)
t=0 t=1t
6/)()(
6/)1333()()(
6/)463()()(
6/)133()()(
332
2321
231
2301
ttCsB
ttttCsB
tttCsB
ttttCsB
i
i
i
i
CubicPolynomials
1/6
4/6
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
B-SplinesB-Splines
0141
0303
0363
1331
6
1
1)(
)()()()()(
21123
2312110
C
vvvvCttttx
vtCvtCvtCvtCtxT
iiiii
iiiii
If we compute v0, … vN => continuous representation for the curve.
B-SplinesB-Splines
)0(
...
)0()0()0()0(...............................)0(
)0()0()0()0(................)0(
)0()0()0()0()0(
4332211022
3322110011
2312011000
NN xx
vCvCvCvCxx
vCvCvCvCxx
vCvCvCvCxx
NN v
v
v
v
x
x
x
x
...
410......0
0............0
0...1410
0...0141
0...0014
6
1
...2
1
0
2
1
0
We have our N+1 data points:
And in matrix form:
We can solve for the vi’s from the xi’sBoundary condition for closed curves: v0=vN, v1=vN+1.
B-SplinesB-Splines
Tiiiii vvvvCttttx 21123 1)(
So, for any i we can find:
xi xi+1
xi(t)
t
t=0 t=1
Note: Local support.Spline passes through all data points xi.
B-Spline demo: http://www.doc.ic.ac.uk/~dfg/AndysSplineTutorial/BSplines.html
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
SnakesSnakesElastic band of arbitrary shape.Located near the image contour.
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Snakes – ImplementationSnakes – Implementation
How can we minimize
i
imageicurviconti EEE
Discrete version: sum over the points.
Greedy minimization:For each point compute the best move over a small area.
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Snakes – ImplementationSnakes – Implementation
How can we minimize
i
imageicurviconti EEE
Discrete version: sum over the points.
Greedy minimization:For each point compute the best move over a small area.
Set βi=0 for points of high curvature (corners)
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Snakes – ImplementationSnakes – Implementation
How can we minimize
i
imageicurviconti EEE
Discrete version: sum over the points.
Greedy minimization:For each point compute the best move over a small area.
Set βi=0 for points of high curvature (corners)
Stop when a user-specified fraction of points does not move.
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Synthetic Results
Real Experiment
Snakes demo: http://www.markschulze.net/snakes/
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Region RepresentationRegion Representation
1 11 1 1 1 1
1 1 1 1 11 1 1 1
1
Spatial Occupancy Array
•Easy to implement.•Large Storage area.•Can apply set operations (unite/intersect).•Expensive for matching!
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Quad TreesEfficient encoding ofSpatial Occupancy Arrayusing Resolution Pyramids.
Black: Fully Occupied.White: Empty.Gray: Partially Occupied.
3-D Computer Vision CSc 83020 – Ioannis Stamos3-D Computer Vision CSc 83020 – Ioannis Stamos
Quad TreeLevel 0Level 1
Level 2
Level 3
Quad Tree Generation:Start with level 0. If Black or White, Terminate. Else
declare Gray node & expand with four sons.
For each son repeat above step.
NW NE
SW SE
Top Related