Computer and Robot Vision II
-
Upload
guinevere-pollard -
Category
Documents
-
view
115 -
download
5
description
Transcript of Computer and Robot Vision II
![Page 1: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/1.jpg)
Digital Camera and Computer Vision LaboratoryDepartment of Computer Science and Information Engineering
National Taiwan University, Taipei, Taiwan, R.O.C.
Computer and Robot Vision II
Chapter 18Object Models And Matching
Presented by: 傅楸善 & 徐子凡0989306249
[email protected]指導教授 : 傅楸善 博士
![Page 2: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/2.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.1 Introduction
object recognition: one of most important aspects of computer vision
![Page 3: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/3.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 4: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/4.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2 Two-Dimensional Object Representation
2D shape analysis useful in machine vision application: medical image analysis aerial image analysis manufacturing
![Page 5: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/5.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2 Two-Dimensional Object Representation
2D shape representation classes: 18.2.1 global features 18.2.2 local features 18.2.3 boundary description 18.2.4 skeleton 18.2.5 2D parts
![Page 6: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/6.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
2D object: can be thought of as binary image
value 1: pixels of objectvalue 0: pixels outside object
2D shape features: area, perimeter, moments, circularity, elongation
![Page 7: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/7.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
Shape Recognition by Moments : binary image function : 2D shape digital th moment of :
area of S number of pixels of S
1,|, yxfyxS
kj,
Syx
kjjk yxM
,
:00 SM
f
S
![Page 8: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/8.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
moment invariants are functions of digital moments invariant under certain
shape transformations. translation, rotation, scaling, skew
center of gravity of S: yx,
)(
)(
00
01
00
10
SM
SMy
SM
SMx
Syx
kjjk yxM
,
![Page 9: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/9.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
central th moment of S:
central moments: translation invariant normalized central moments of S:
),( kj
Syx
kjjk yyxx
),(
)()(
12
,00
kjjk
jk
)(
)(
00
01
00
10
SM
SMy
SM
SMx
![Page 10: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/10.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
seven functions that are rotation invariant
![Page 11: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/11.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Original
Half Size Mirrored Rotated 2° Rotated 45°
![Page 12: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/12.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
Fourier descriptors: another way for extracting features from 2D shapes defined to characterize boundary
The main idea is to represent the boundary as a function of one variable , expand in its Fourier series, and use the coefficients of the series as Fourier descriptors (FDs).
finite number of FDs: can be used to describe the shape
t t
![Page 13: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/13.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
Each coordinate pair can be treated as a complex number so that
Discrete Fourier transform
of is
( ) ( ) ( ),
for 0,1,2,..., 1
s k x k jy k
k K
( )s k21
0
1( ) ( ) ,
for 0,1,2,..., 1
j ukKK
k
a u s k eK
u K
![Page 14: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/14.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
The complex coefficients are called the Fourier descriptors of the boundary.
The inverse Fourier transform of these coefficients restores .
Suppose, only the first P coefficients are used.
( )a u
( )s k21
0
( ) ( ) ,
for 0,1,2,..., 1
j ukKK
u
s k a u e
k K
21
0
ˆ( ) ( )j ukP
K
u
s k a u e
![Page 15: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/15.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
![Page 16: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/16.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.1 Global Feature Representation
Some basic properties of Fourier descriptors.
Notation:
Impulse function :xy x j y
( ) 0, if 0
( ) 0, if 0
u u
u u
( )u
![Page 17: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/17.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 18: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/18.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.2 Local Feature Representation
2D object characterized by: local features, attributes, relationships
most commonly used local features: Holes
found by connected component procedure followed by boundary tracing
detected by binary mathematical morphology, if hole shapes known
properties: areas, shapes Corner
detection: can be performed on binary or gray tone image property: angle at which lines meet
![Page 19: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/19.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 20: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/20.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.3 Boundary Representation
boundary representation: most common representation for 2D objects.
3 main ways to represent object boundary: sequence of points chain code sequence of line segments
![Page 21: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/21.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.3 Boundary Representation The Boundary as a Sequence of Points boundary points from border-following or edge-
tracking algorithms interest points: boundary points with special property
useful in matching
![Page 22: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/22.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.3 Boundary Representation
The Chain Code Representation chain encoding:
can be used at any level of quantization saves space required for row and column coordinates
boundary encoded: first quantized by placing over square grid
square grid side length: determines resolution of encoding
marked points: grid intersections closest to curve and used in encoding
* : marks starting point of curve
![Page 23: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/23.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
chain encoding of boundary curve
![Page 24: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/24.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.3 Boundary Representation
line segments: links: to be used to approximate the curve
encoding scheme: eight possible directions assigned integer between 0, 7
chain: chain encoding: in the form
i
n
in aCAoraaaA
121 ...
![Page 25: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/25.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.3 Boundary Representation
length of chain code with n chains: can be simply estimated as n
no: number of odd chain codes
ne: number of even chain codes
nc: number of corners L: unbiased estimate of perimeter length Freeman suggested: oe nnL 2
![Page 26: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/26.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.3 Boundary Representation The Boundary as a Sequence of Line Segments line segment sequence: after boundary segmented
into near-linear portion line segment sequence: used in shape
recognition or other matching tasks : coordinate location where pair of lines
meet : angle magnitude where pair of lines meet sequence of junction points to
represent line segment sequence
:, ii YX
i:,...,, 21 nOOOO
iii YX ,,
![Page 27: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/27.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.3 Boundary Representation
sequence of junction points representing test object T
an association
goal: given O, T, to find F satisfying i < j F(i) < F(j) or F(i) = missing or F(j) = missing
:,...,, 21 mTTTT
:missing,...,2,1,...2,1: nmF
![Page 28: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/28.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 29: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/29.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.4 Skeleton Representation
strokes: long, sometimes thin parts forming shapes
![Page 30: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/30.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.4 Skeleton Representation
symmetric axis transform: set of maximal circular disks that fit inside object
symmetric axis: locus of centers of these maximal disks
![Page 31: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/31.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.4 Skeleton Representation
The symmetric axis is one example of a skeleton description of 2D object.
symmetric axis is not always completely representative of the strokes of an object. rectangle: consists of five line segments not
single line symmetric axis: extremely sensitive to noise
make it difficult to use in matching.
![Page 32: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/32.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.4 Skeleton Representation
local symmetry: midpoint P of line segment BA
α : angle between BA and outward normal Na at A
α : angle between BA and inward normal Nb at B
![Page 33: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/33.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.4 Skeleton Representation
The loci of local symmetries that are maximal w.r.t. forming a smooth curve are called axes or spines.
cover of axis: portion of shape subtended by axis axis cover properly contained in another cover:
second axis subsumes first
The short diagonal axes are subsumed by the horizontal and vertical axes and can be either deleted or relegated to a lower place in a hierarchical description of the shape (Chap. 19).
![Page 34: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/34.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.4 Skeleton Representation
Axes of smoothed local symmetries of several objects.
![Page 35: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/35.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 36: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/36.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.5 Two-Dimensional Part Representation
parts, attributes, interrelationships: form structural description of shape
nuclei: regions where primary convex subset overlap
nuclei
![Page 37: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/37.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.5 Two-Dimensional Part Representation
near-convexity: allows noisy distorted instances to have same decompositions
P1 , P2: two points on object boundary
LI relation: visibility relation if line completely interior to object boundary,
Apply the graph-theoretic clustering algorithm to determine clusters of visibility relation
1 2PP
ILPP 21,
![Page 38: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/38.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.2.5 Two-Dimensional Part Representation
decomposition of three similar shapes into near-convex pieces
![Page 39: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/39.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 40: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/40.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3 Three-Dimensional Object Representations
18.3.1 Local Features Representation. 18.3.2 Wire Frame Representation. 18.3.3 Surface-Edge-Vertex Representation. 18.3.4 Stick, Plates, and Blobs. 18.3.5 Generalized Cylinder Representation. 18.3.6 Super-quadric Representation. 18.3.7 Octree Representation. 18.3.8 The Extended Gaussian Image. 18.3.9 View-Class Representation.
![Page 41: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/41.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.1 Local Features Representation
Local Features Representation range data:
obtained from laser range finder, light striping, stereo, etc. from depth, try to infer surfaces, edges, corners, holes, other
features 3D matching more difficult than 2D because of occlusion
![Page 42: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/42.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 43: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/43.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.2 Wire Frame Representation
wire frame model: 3D object model with only edges of object
![Page 44: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/44.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.2 Wire Frame Representation
two-color hyperboloid and its line drawing
![Page 45: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/45.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.2 Wire Frame Representation
Necker cube: lower-vertical face or upper-vertical face closer to viewer
Schroder staircase: viewed either from above or from below
![Page 46: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/46.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.2 Wire Frame Representation
![Page 47: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/47.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.2 Wire Frame Representation
general-viewpoint assumption: none of the following situations 1. two vertices of scene objects represented at same picture
point 2. two scene edges seen as single line in picture 3. vertex seen exactly in line with unrelated edge
![Page 48: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/48.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.2 Wire Frame Representation
general-viewpoint assumption: heart of line-drawing interpretation
viewpoint in perspective projection: center of projection
viewpoint in orthographic projection: direction of projection
![Page 49: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/49.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
subjective contours of Kanizsa: white occluding triangle in space
18.3.2 Wire Frame Representation
![Page 50: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/50.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.2 Wire Frame Representation
line labels for visible projections of surface-normal discontinuities:
![Page 51: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/51.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 52: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/52.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.3 Surface-Edge-Vertex Representation
VISIONS system: Visual Integration by Semantic Interpretation of Natural Scenes
PREMIO system: Prediction in Matching Images to Objects
PREMIO 3D object model: hierarchical, relational model with five levels world, object, face/edge/vertex, surface/boundary, arc/2D,
1D piece
![Page 53: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/53.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.3 Surface-Edge-Vertex Representation
world level: arrangement of different objects in world object level: arrangement of different faces, edges,
vertices forming objects face level: describes face in terms of surfaces and
boundaries surface level: specifies elemental pieces forming
surfaces
![Page 54: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/54.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.3 Surface-Edge-Vertex Representation
2D piece level: describes pieces and specifies arcs forming boundaries
1D piece level: describes elemental pieces forming arcs
SDS: spatial data structure A/V: attribute-value table
![Page 55: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/55.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 56: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/56.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 57: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/57.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.4 Sticks, Plates, and Blobs
sticks, plates, blobs model: rough models of 3D objects used in rough-matching near-convex
sticks: long, thin parts with only one significant dimension cannot bend very much two logical endpoints set of interior points center of mass
![Page 58: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/58.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.4 Sticks, Plates, and Blobs
plates: flattish wide parts with two nearly flat surfaces two significant dimensions cannot fold very much set of edge points, set of surface points, center of mass
blobs: parts with three significant dimensions can be bumpy but cannot have concavities set of surface points and center of mass
![Page 59: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/59.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 60: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/60.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.4 Sticks, Plates, and Blobs
attribute-value table: contains global attributes simple-parts relation: lists the parts and their attributes connects-supports relation: gives connections between
pairs of parts triples relation: specifies connections between three
parts at a time parallel relation: lists pairs of parts that are parallel perpendicular relation: lists pairs of parts that are
perpendicular TYPE: 1 for stick, 2 for plate, 3 for blob
![Page 61: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/61.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 62: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/62.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 63: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/63.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.5 Generalized Cylinder Representation
generalized cylinder: volumetric primitive defined by axis and cross-section
cross section: swept along axis, creating a solid e.g. actual cylinder: generalized cylinder whose axis is straight-
line segment and whose cross section is circle of constant radius
e.g. cone: generalized cylinder whose axis is straight-line segment and cross section is circle with radius initially zero to maximum
![Page 64: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/64.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.5 Generalized Cylinder Representation
e.g. rectangular solid: generalized cylinder whose axis is straight line segment and cross section is constant rectangle
e.g. torus: generalized cylinder whose axis is circle and whose cross section is constant circle
generalized cylinder representation: uses generalized cylinders as primitives
torus
![Page 65: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/65.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.5 Generalized Cylinder Representation
surface-edge-vertex model: very precise
sticks-plates-and-blobs model: very rough
generalized cylinder model: somewhere in between
![Page 66: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/66.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.5 Generalized Cylinder Representation
person: modeled roughly as cylinders for head, torso, arms, legs
dotted lines: axes of cylinders
![Page 67: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/67.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 68: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/68.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.6 Super-quadric Representation
Super-quadrics: lumps of clay deformable and can be glued into object models
Super-quadric models: mainly used with range data
![Page 69: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/69.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.6 Super-quadric Representation
Super-quadrics are a flexible family of 3-dimensional parametric objects, useful for geometric modeling.
By adjusting a relatively few number of parameters, a large variety of shapes may be obtained.
![Page 70: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/70.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Figure 18.13 Range data image of (a) a doll, (b) its super-quadric fit (c), (d) wire frame
![Page 71: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/71.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 72: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/72.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.7 Octree Representation octree encoding:
geometric modeling technique used to represent 3D objects used in computer vision, robotics, computer graphics
octree hierarchical: 8-ary tree structure
each node in octree corresponds to cubic region of universe
![Page 73: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/73.jpg)
18.3.7 Octree Representation
DC & CV Lab.CSIE NTU
![Page 74: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/74.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.7 Octree Representation
full, empty, partial full:
if cube is completely enclosed by 3D object empty:
if cube contains no part of object partial:
if cube partly intersects object
partial: has eight children representing partition of cube into octants
labeled full or empty : no children
![Page 75: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/75.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 76: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/76.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 77: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/77.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.8 The Extended Gaussian Image
3D object: collection of surface normals, one at each point of object surface
planar surface: all points on surface map to same surface normal
convex with positive curvature everywhere: distinct surface normal everywhere
set of surface normals can be mapped to a unit sphere (Gaussian sphere) by placing tail at center head outward
Gaussian image of object: resultant set of points on Gaussian sphere
![Page 78: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/78.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 79: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/79.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.8 The Extended Gaussian Image
for planar objects: Gaussian image not invertible, not precise enough for use
δO: small surface patch of object δS: corresponding surface patch on Gaussian sphere Gaussian curvature K:
dS
dO
O
SK
O
0lim
![Page 80: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/80.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.8 The Extended Gaussian Image
(ξ,η): point on Gaussian sphere corresponding to point (u, v) on object surface
extended Gaussian image:
planar region: Gaussian curvature 0, point mass in extended Gaussian image
),(
1,
vuKG
![Page 81: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/81.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 82: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/82.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.9 View-Class Representation view classes: each representing set of viewpoints
sharing some property same object surfaces visible same line segments visible relational distances between relational structures are similar
characteristic views: sets producing topologically isomorphic line drawings
![Page 83: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/83.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.9 View-Class Representation three view classes of cube producing topologically
isomorphic line drawings
![Page 84: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/84.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 85: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/85.jpg)
DC & CV Lab.CSIE NTU
![Page 86: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/86.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.3.9 View-Class Representation aspect graph of object: graph structure where
1. each node represents topologically distinct view of object
2. a node for each such view of object
3. each arc represents a visual event at transition
4. there is an arc for each such transition
![Page 87: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/87.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 88: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/88.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 89: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/89.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4 General Frameworks for Matching
matching: finding correspondence between two entities
consistent labeling procedures: examples of matching algorithms
![Page 90: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/90.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4 General Frameworks for Matching
18.4.1 Relational-Distance Approach to Matching 18.4.2 Ordered Structural Matching 18.4.3 Hypothesizing and Testing with Viewpoint
Consistency Constraint 18.4.4 View-Class Matching 18.4.5 Affine-Invariant Matching
![Page 91: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/91.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.1 Relational-Distance Approach to Matching
relational distance: compares two structures and determines similarity
Relational-Distance Definition Dx : relational description
Dx = {R1 , … , RI} : sequence of relations X : set of parts of entity being described Ri : relation indicating various relationships among parts
DA : relational description with part set A
DB : relational description with part set B
![Page 92: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/92.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.1 Relational-Distance Approach to Matching
assumption: |A| = |B|, otherwise add dummy parts to smaller set f: any one-one, onto mapping from A to B N: positive integer
composition R 。 F of relation with function f:
NnbafwithRaaBbbfR nnNN
N ,...,1, ,...,|,..., 11
NAR
![Page 93: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/93.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.1 Relational-Distance Approach to Matching
f: maps parts from set A to parts from set B structural error of f for Ith pair of corresponding relations
in DA, DB :
total error of f with respect to DA, DB:
relational distance GD(DA, DB) between DA, DB:
||||)( 1iiii
iS RfSSfRfE
I
i
iS fEfE
1
fEDDGD
ontoBAf
lBA
:
1min,
1
1
,...,
,...,
A I
B I
D R R
D S S
![Page 94: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/94.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.1 Relational-Distance Approach to Matching
best mapping from DA to DB : mapping f that minimizes total error
![Page 95: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/95.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
best mapping from to is
for this mapping:
4,3,2,1A dcbaB ,,, dfcfbfaf 4,3,2,1
![Page 96: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/96.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 97: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/97.jpg)
DC & CV Lab.CSIE NTU
0
31213121
31213121
1 32
3121323121
3121323121
}332211{
321M 321M
11
21
)},)(,{()},)(,{(
)},)(,{(f')}',')(',{(RfS
)'',(
')}',')(',{(')}',')(',')(',{(
')}',')(',{(f)},)(,)(,{(SfR
')', f()', f()f(
'', ', , ,
1
![Page 98: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/98.jpg)
DC & CV Lab.CSIE NTU
1
56561516454
32413121
6532312161516454
32413121
6532312161516454
1 54
6532312161516454
32413121
6532312161516454
32413121
}665544332211{
654321M 4321M
1****************
1
**
****************
********
****************
43
}),)(,)(,)(,)(,{(
'')}'','')('','')('','')('',{(
)},)(,)(,)(,)(,)(,)(,)(,{(
'')}'','')('','')('','')('',{(
f)},)(,)(,)(,)(,)(,)(,)(,{(
RfS
),(
)},)(,)(,)(,)(,)(,)(,)(,{(
)},)(,)(,)(,{(
)},)(,)(,)(,)(,)(,)(,)(,{(
f'')}'','')('','')('','')('',{(SfR
''), f(''), f(''), f(''), f(''), f('')f(
, , , , , '''', '', '',
''''''''''''''''''''
''''''''''''''''''''''''''''''''
******
******
![Page 99: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/99.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.1 Relational-Distance Approach to Matching
Relational Distance as a Metric relational distance:
used to determine similarity of unknown object to an object model can also be used to compare object models to grouping models in
a large database
f relational isomorphism: if f one-one, onto from A to B and E(f) = 0
f: A → B relational isomorphism: DA, DB isomorphic
GD: relational-distance measure
![Page 100: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/100.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.1 Relational-Distance Approach to Matching
DA, DB , DC : metric property of GD:
BCCABA
ABBA
BABA
,DDGD,DDGD,DDGD
,DDGD,DDGD
,DD,DDGD
.3
.2
isomorphic 0 .1
![Page 101: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/101.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.1 Relational-Distance Approach to Matching
Attributed Relational Descriptions and Relational Distance
extend relational description and relational distance to include properties of parts properties of the whole properties of these relationships
![Page 102: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/102.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 103: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/103.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.2 Ordered Structural Matching
definition of ordering on primitives: greatly reduces complexity of search
![Page 104: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/104.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 105: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/105.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.3 Hypothesizing and Testing with Viewpoint Consistency Constraint
viewpoint consistency constraint: The locations of all projected model features in an image must
be consistent with projection from a single viewpoint.
![Page 106: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/106.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 107: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/107.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.4 View-Class Matching if 3D object represented by view-class model, matching
divided into 2 stages:
1. determining view class of object
2. determining precise viewpoint within that view class
![Page 108: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/108.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.4 View-Class Matching relational pyramid:
hierarchical relational structure to represent view class
Level-1 primitives: straight- and curved-line segments
Level-2 relations: junctions and loops
Level-3 relations: adjacency, collinearity, junction parallelness, loop-inside-loop
![Page 109: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/109.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.4 View-Class Matching Pose Determination within View Class
relational pyramid: hierarchical, relational structure to constrain matching
![Page 110: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/110.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 111: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/111.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.5 Affine-Invariant Matching
set of interest points lying in z = z0 plane rotation matrix relating model reference frame to camera
reference frame:
translation of object reference frame to camera reference frame:
M
mmm zyxM 10,,
333231
232221
131211
rrr
rrr
rrr
3
2
1
t
t
t
![Page 112: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/112.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.5 Affine-Invariant Matching f: distance between image plane and center of
perspectivity : observed image data points by perspective
projection:
when translation t3 in z-direction large compared with r31xm + r32ym :
2
1
2221
1211
b
b
y
x
rr
rr
v
u
m
m
m
m
N
nnn vuO 1,
30333231
10131211
tzryrxr
tzryrxrfu
mm
mmn
30333231
20232221
tzryrxr
tzryrxrfv
mm
mmn
![Page 113: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/113.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.5 Affine-Invariant Matching A : 2 x 2 (scaling, rotation, skewing) matrix b : 2D (translation) vector affine 2D correspondence: Aw + b
![Page 114: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/114.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.5 Affine-Invariant Matching necessary and sufficient to define plane uniquely:
3 noncollinear points
![Page 115: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/115.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.5 Affine-Invariant Matching The Hummel-Wolfson-Lamdan Matching Algorithm to match noncollinear triplets in model interest points
with scene: Step 1: preprocessing: convert model interest points into affine-
invariant model Step 2: recognition: match model against image using affine
representation
![Page 116: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/116.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.4.5 Affine-Invariant Matching Shortcomings of the Affine-Invariant Matching Technique affine-invariant matching technique:
mathematically sound in noiseless case
shortcomings of affine-invariant matching in practice: 1. if three noncollinear points not numerically stable, points not
reliable 2. coordinates of detected interest points: noisy in real image 3. partial object symmetries may cause wrong matching
![Page 117: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/117.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
Joke
![Page 118: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/118.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
18.5 Model Database Organization organize database of models: to allow rapid access to
most likely candidate group similar relational models into clusters and choose
representative arrows: indicate mapping from parts of object 2 to parts
of other objects
![Page 119: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/119.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
![Page 120: Computer and Robot Vision II](https://reader034.fdocuments.net/reader034/viewer/2022050704/5681317d550346895d97f6d1/html5/thumbnails/120.jpg)
DC & CV Lab.DC & CV Lab.CSIE NTU
END