Post on 09-Feb-2021
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
GEOMETRIC OPERATIONS
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Introduction
• Permit elimination of the geometric distortions.
• Geometric distortion may arise:
– Because of the lens
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Lens Geometric Distortions
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Image taken from http://nofilmschool.com/2011/11/lens-choice-affects-subjects-appearance
Introduction
• Permit elimination of the geometric distortions.
• Geometric distortion may arise:
– Because of the lens
– Because of perspective
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Perspective Distortions
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Image taken from www.techsupportalert.com
Introduction
• Permit elimination of the geometric distortions.
• Geometric distortion may arise:
– Because of the lens
– Because of perspective
– Because of human error
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Human Error Example: Tilt Horizon
Image taken from http://thehdrimage.com/5-quick-steps-to-better-hdrs-step-1/
Introduction
• Permit elimination of the geometric distortions.
• Geometric distortion may arise:
– Because of the lens
– Because of perspective
– Because of human error
– In situations where there are distortions inherent in the
imaging process such as remote sensing:
• For example is an attempt to match remotely sensed images of
the same area not taken from precisely the same position. In
this case it is necessary first to execute a geometric
transformation, and then compare the images.
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
Panorama Stitching
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Image taken from http://matthewalunbrown.com/autostitch/autostitch.html
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Definition ?
Class of operations modifying the neighboring structure of image
pixels (the structure of spatial organization of the image).
Geometric transform allow pixels to move within the image,
to new positions.
Intuitive model: image is printed onto a thin sheet of rubber, that
can be deformed at will.
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Geometric operations without constraints
If the movement of each pixel is independent with respect to the
movements of the rest of image pixels, the visual content could be
mixed or cut-off.
image quarters
are interchanged
“random” pixel
displacement
Constraint: the law describing pixel movement is NOT random and
is spatially invariant (all image pixels move according to the same
equations).
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Defining geometric transforms
Transform = equation describing the modification of pixel coordinates
),('
),('
yxYy
yxXx
x’ , y’ :
pixel coordinates
after transform
x, y :
pixel coordinates
before the transform
X, Y:
equations of the
coordinate modification
Particular forms of the functions of the coordinate transform,
X(x,y) and Y(x,y) particularize the geometric transforms.
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Image coordinates system
x
y(0,0) Origin is upper-left
corner, coordinates increase
from left to right and from top
to bottom.
First coordinate is the line coordinate.
Coordinates can be continuous (x, y)
or discrete (m, n)
Transforming image coordinates to plane Cartesian coordinates :
xplan
yplan
xy
yx
plan
plan
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Affine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
An affine transformation = function between affine spaces which
preserves points, straight lines and planes.
• from the Latin affinis: "connected with“
• parallel lines remain parallel;
• does not necessarily preserve angles between lines;
• does not necessarily preserve distances between points;
• preserve ratios of distances between points lying on a straight
line.
Examples: translation, scaling, reflection, rotation, shear mapping,
and compositions of them in any combination and sequence.
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Affine elementary geometric transforms
Translation
Displacement of the image content within the plane;
is equivalent to change of the origin of the image coordinate system.
0
0
),('
),('
yyyxYy
xxyxXx
Does it preserve inter-pixel distances?
Parameters : x0, y0 – vertical/ horizontal displacement.
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
Yes
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Translation
x
y(0,0)
x’
y’(0,0)
(x0, y0)
x
y(0,0)
x’
y’(0,0)
Affine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Scaling
0,
),('
),('
yyxYy
xyxXx Parameters : , - horizontal/ vertical scaling factors
Stretching/compressing image content according to one or
both coordinate axes
= : homogeneous scaling
Affine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
Does it preserve inter-pixel distances? No
, > 1:
, < 1:
stretching
compressing
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Scaling
x
y(0,0)
x’
y’(0,0)
> 1, < 1
< 1, > 1
Affine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
Scaling
• Compressing => image
reduction or subsampling
•Stretching => image
magnification or zooming
•Performed by replacement (of a
group of pixel values by one
arbitrarily chosen pixel value
from within this group) or by
interpolating between pixel
values in a local neighborhoods
Affine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Important issue when downscaling: aliasing
Aliasing: comes from the inappropriate sampling of a signal.
When reconstructed, an undersampled sine wave will create an
“alias”, i.e., a parasite low-frequency component, that might alter the
signal
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Image downscaling = projecting the same information onto a
coarser grid = downsampling.
Some of the high-frequency image components might create
visible aliases:
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Solution: low-pass filtering (blurring) the image prior to
downsampling it.
The high-frequency components susceptible of creating aliases
are removed from the image.
without filtering with filtering
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Skewing
Skewing is the displacement of image pixels according to a single
coordinate, dependent by the global location within the image.
The second coordinate remains unchanged.
0
),('
),('
t
ytxyxYy
xyxXx
0
),('
),('
s
yyxYy
syxyxXx
Horizontal skewing Vertical skewing
Parameters : t, s – skewing coefficients.
Affine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
Does it preserve inter-pixel distances? No
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Horizontal skewing
x
y(0,0)
x’
y’(0,0)
x=0 points are fixed points
0
),('
),('
t
ytxyxYy
xyxXxAffine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Vertical skewing
x
y(0,0)
x’
y’(0,0)
y=0 points are fixed points
0
),('
),('
s
yyxYy
syxyxXxAffine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Rotation
cossin),('
sincos),('
yxyxYy
yxyxXx
Circular displacement of pixels around the rotation center
(the origin of the coordinate system).
Does it preserve inter-pixel distances?
Parameter : - rotation angle
Affine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
Yes
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Rotation
Affine elementary geometric transforms
• Rotates points by an angle θ about origin
(θ >0: counterclockwise rotation)
• From OBP triangle:
• From OCP’ triangle:
=>
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
O C B
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Rotation
x
y(0,0)
x
y(0,0)
Affine elementary geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Reflection with respect to a reflection center (x0,y0)
The initial and final positions of any pixel form a
line segment; the reflection center is the middle of the segment.
yyyxYy
xxyxXx
0
0
2),('
2),('
x
y(0,0)
(x, y)
(x’, y’)
(x0, y0)
Composed affine geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Reflection with respect to a rotation axis
The initial and final positions of any pixel form a line segment;
the reflection axis is the median axis of the line segment.
x
y(0,0)
(x, y)
(x’, y’)
reflection
axis
y=mx+n
1
''
1
1
'
'
22
m
nmxy
m
nmxy
mxx
yy
22
2
2
222
2
1
2
1
1
1
2),('
1
2
1
2
1
1),('
m
ny
m
mx
m
myxYy
m
mny
m
mx
m
myxXx
Composed affine geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
horizontal axis: x = k = ct (1/m = 0, -n/m = k)
vertical axis: y = k = ct (m=0, n=k)
ykyxYy
xyxXx
2),('
),('
22
2
2
222
2
1
2
1
1
1
2),('
1
2
1
2
1
1),('
m
ny
m
mx
m
myxYy
m
mny
m
mx
m
myxXx
22
2
2
222
2
1
2
1
1
1
2),('
1
2
1
2
1
1),('
m
ny
m
mx
m
myxYy
m
mny
m
mx
m
myxXx
yyxYy
xkyxXx
),('
2),('
Composed affine geometric transforms
Reflection with respect to a rotation axis: particular cases
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
C. VERTAN
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Matrix form of affine geometric transforms
By
xA
y
x
'
'1. Translation
0
0
'
'
y
x
y
x
y
x2. Scaling
y
x
y
x
0
0
'
'
3a. Horizontal skewing
y
x
ty
x
1
01
'
'
3b. Vertical skewing
y
xs
y
x
10
1
'
'
4. Rotation
y
x
y
x
cossin
sincos
'
'
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
By
xA
y
x
'
'
1. Reflection with respect to a point
2. Reflection with respect to an axis
0
0
2
2
10
01
'
'
y
x
y
x
y
x
2
2
2
2
2
22
2
1
21
2
1
1
1
21
2
1
1
'
'
m
mm
mn
y
x
m
m
m
mm
m
m
m
y
x
transforms
composed by
translation and
rotation
Matrix form of affine geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Composed geometric transforms
Composing = iteration of elementary transforms
Composing geometric transforms is simple in matrix form ...
but translation has not the same matrix form and cannot be
expressed as a matrix product.
Solution : homogeneous coordinates
1
y
x
y
x(plane of z=1 from the 3D Cartesian space)
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Homogeneous matrix form of geometric transforms
11
'
'
y
x
Ay
x
1. Translation
1100
10
01
1
'
'
0
0
y
x
y
x
y
x
1100
0cossin
0sincos
1
'
'
y
x
y
x
4. Rotation
2. Scaling
1100
00
00
1
'
'
y
x
y
x
3a. Horizontal skewing
1100
01
001
1
'
'
y
x
ty
x
3b. Vertical skewing
1100
010
01
1
'
'
y
xs
y
x
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Composed geometric transforms
• Important: preserve the order of transformations!
translation + rotation rotation + translation
Images taken from: http://www.cse.unr.edu/~bebis/CS485/Lectures/GeometricTransformations.ppt
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Decomposing complex geometric operations
In the most general form, expressed in homogeneous coordinates:
11001
'
'
02221
01211
y
x
yaa
xaa
y
x
aij coefficients are due to rotation, skewing and scaling.
cossin
sincos
10
1
0
0
2221
1211 s
aa
aa
given a series of elementary
transforms, find an unique
equivalent transform.
given an affine transform,
find the composing
elementary geometric
transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
given a series of elementary
transforms, find an unique
equivalent transform.Trivial problem.
Ex. The equivalent transform for a rotation by 45º (=45º), vertical
skewing by coefficient 0.1 (s=0.1), horizontal scaling 2 (=2)
and vertical scaling 0.5 (=0.5).
cossin
sincos
10
1
0
0
2221
1211 s
aa
aa
2/22/2
2/22/2
10
1.01
20
05.0
2221
1211
aa
aa
2728.14142.1
3889.03535.0
2221
1211
aa
aa
Decomposing complex geometric operations
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
cossin
cossinsincos
2221
1211 s
aa
aa
2
22
2
21 aa 22
21arctana
a
221221
2
22
2
21aa
a
aa
221221
2
22
2
21
22
2
22
2
21
21
11 aaa
aaaaa
a
as
Decomposing complex geometric operationsgiven an affine transform,
find the composing
elementary geometric
transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Why using a single skewing ?
horizontal skewing
vertical skewing
rotation /2
reflection with respect to
vertical
Decomposing complex geometric operations
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Nonlinear geometric transforms
The “pillow” effect
displacement controlled by
sinusoidal law
yyxYy
W
yA
H
Hy
HyxXx x
),('
sin2
122
),('
Horizontal pillow effect
H
xA
W
Wy
WyxYy
xyxXx
y sin2
122
),('
),('
Vertical pillow effect
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
General affine transform
Nonlinear geometric transforms
W
xA
H
Hy
Hy
H
yA
W
Wx
Wx
y
x
sin2
122
'
sin2
122
'
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Lens distortion: pincushion and barrel effects
Nonlinear geometric transforms
Regular grid Distortion introduced by
typical lens
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Lens distortion: pincushion and barrel effects
Nonlinear geometric transforms
2
2
1'
1'
PAPP
PAPP
yyy
xxx
Distortion equation:
Px and Py: normalized coordinates.
Ax and Ay: constants between 0 (no action) and 0.1 (wide angle
lens).
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Lens distortion: pincushion and barrel effects
Nonlinear geometric transforms
22
22
1/1
'
1/1
'
PAPA
PP
PAPA
PP
yy
y
y
xx
xx
Images taken from: http://local.wasp.uwa.edu.au/~pbourke/miscellaneous/lenscorrection/
http://local.wasp.uwa.edu.au/~pbourke/miscellaneous/lenscorrection/
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
A geometric transform consists of two basis steps:
1. Pixel co-ordinate transformation, which maps the co-
ordinates of the input image pixel to the point in the output image.
The output point co-ordinates should be computed as continuous
values (real numbers) as the position does not necessarily match
the digital grid after the transform.
2. The second step is to find the point in the digital raster which
matches the transformed point and determine its brightness value.
Its brightness is usually computed as an interpolation of the
brightness of several points in the neighborhood .
Implementing geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Implementing geometric transforms
Two components:
1. where the pixel is moved
2. what value is placed in the new location ?
geometric transform
interpolation
Images are discrete: pixel coordinates are integer numbers.
In most cases the coordinates obtained after the geometric
transform are real numbers.
There is thus the need of producing integer coordintes.
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Solution 1: Pixel carry-over
Solution 2: Pixel filling
m
n
x’
y’T
(x’, y’) non-integer coordinates
after the transform(m, n) integer coordinates
before the transform
Implementing geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Pixel carry-over
m
nT
(x’, y’) non-integer coordinates
after the transform(m, n) integer coordinates
before the transform
the value must be distributed
to neighboring integer locations
x’
y’
Implementing geometric transforms
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Problems related to the direct implementation
2/),('
2),('
yyxYy
xyxXx
non-integer coordinates
non-appearing coordinates
0
1
2
3
4
5
0 1 2 3 4 5 6
initial coordinates
0
1
2
3
4
5
0 1 2 3 4 5 6
transformed coordinates
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Implementing geometric operations
Pixel filling
T-1
(m’, n’) integer coordinates
after the transform(x, y) non-integer coordinates
before the transform
m’
n’
x
y
the non-existing value from the initial image
must be obtained by the interpolation of the values
from the neighboring pixels.
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Interpolation methods
m
m+1
n n+1
integer coordinate positions;
image values are given
f(m, n), f(m+1, n), ...
x
y
non-integer coordinate position
where image value must be computed;
image function is supposed to be
continuous on [m, m+1] x [n, n+1]
Simplest: 0th-order interpolation
The value in (x,y) is the value of the closest point
having integer coordinates.
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Linear (order 1 interpolation) according to each dimension.
[x] [x]+1
f1=f ([x])
f2=f ([x]+1)
x
f (x)
])([][][)1]([
])([)1]([)( xfxx
xx
xfxfxf
112 ])[)(()( fxxffxf
Bilinear interpolation
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
(m+1,n) (m+1,n+1)
(m,n)
(m,n+1)
m = [x]
n = [y]
f1
f2
f3
f1 = f(m, n)
f2 = f(m+1, n)
f3 = f(m, n+1)
f4 = f(m+1, n+1)
f4x
g1
g2
f (x, y)
g1 is linearly interpolated from f1 and f2g2 is linearly interpolated from f3 and f4
f is linearly interpolated from g1 and g2
Bilinear interpolation
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
(m+1,n) (m+1,n+1)
(m,n)
(m,n+1)
f1
f2
f3
f4x
g1
g2
f (x, y)m = [x]
n = [y]
f1 = f(m, n)
f2 = f(m+1, n)
f3 = f(m, n+1)
f4 = f(m+1, n+1)
g1 is linearly interpolated from f1 and f2 1121 ))(( fmxffg
g2 is linearly interpolated from f3 and f4 3342 ))(( fmxffg
f is linearly interpolated from g1 and g2 112 ))(( gnyggf
Bilinear interpolation
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Bilinear interpolation
1121 ))(( fmxffg
3342 ))(( fmxffg
112 ))(( gnyggf
112
112334
))((
)())(())((
fmxff
nyfmxfffmxfff
113
121234
))((
))(()(
fnyff
mxffnymxfffff
)()()( nymxnymxf
LABORATORUL DE ANALIZA ŞI PRELUCRAREA IMAGINILOR - LAPI
Interpolation methods
Higher-order interpolation: 2, 3, ...
(spline functions – order 3 interpolation)