Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf ·...

38
Raffaella Lanzarotti

Transcript of Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf ·...

Page 1: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

Raffaella Lanzarotti

Page 2: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

2

Page 3: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

• Image Enhancement: – Color values modified– Pixel positions unchanged

• Geometric transformation– Pixel positions modifed– Colors unchanged

• Goal:– To create special effects– To register two images taken of the same scene at different times or different positions

– To morph one image to another

3

Page 4: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

• Let be an image• is a warping image of if

• and are called the warpingfunctions

4

B(k, l) = A(x(k), y(l))

A(i, j)A(i, j)B(k, l)

x(...) y(...)

Page 5: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

5

x(k) = k + 100

y(l) = l + 20

Page 6: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

6

x = k+ t

x =

x(k)y(l)

, k =

k

l

, t =

t

x

t

y

Page 7: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

7

Page 8: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

8

Page 9: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

• T = maketform('affine',[1 0 0;0 1 0; 100 20 1]);

• J = imtransform(img,T,'XData',[1,size(img,2)],'YData',[1 size(img,1)]);

9

The translationvector occupies the last row in the transformation matrixin MATLAB!!!

Page 10: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

10

x(k) = k · sky(l) = l · sl

Page 11: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

11

x = S · k

x =

x(k)y(l)

, k =

k

l

, S =

s

x

00 s

y

,

Page 12: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

12

Page 13: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

13

Page 14: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

T = affine2d([1.5 0 0; 0 .5 0; 0 0 1]);J = imwarp(img,T);

14

Page 15: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

15

Page 16: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

16

Page 17: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

17

Page 18: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

18

Page 19: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

19

x(k, l) = (k x0) cos() + (l y0) sin() + x0

y(k, l) = (k x0) sin() + (l y0) cos() + y0

(x0, y0) = the center of the image

Where:

= rotation angle

Page 20: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

20

x = Rk, where:

R =

cos sin sin cos

,

x =

x(k)y(l)

, k =

k x0

l y0

Page 21: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

Theta= pi/6;T = affine2d([cos(Theta) sin(Theta) 0;

-­‐sin(Theta) cos(Theta) 0;0 0 1]);

J = imwarp(img, T);

21

Page 22: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

22

=

!!!

"

#

$$$

%

&

!!!

"

#

$$$

%

&

!!!

"

#

$$$

%

& −

=

1yx

1000s000s

100tθcossinθtinθsθcos

y

x

y

x

!!!

"

#

$$$

%

&

!"

#$%

&=

!!!

"

#

$$$

%

&

!"

#$%

&!"

#$%

&=

110

1100

10yx

tSRyx

StR

Page 23: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

• A generalization of geometric transformations:

23

Q =

C d0T 1

C = any non singular 2 2 matrix

d = any vector in R2

Page 24: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

• Maps straight lines into strainght lines• Affine mapping coefficients uniquelydetermined from displacements of 3 vertices

24

Page 25: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

A =

a1 a2b1 b2

, d =

a0b0

25

x = a0 + a1k + a2l

y = b0 + b1k + b2l

Page 26: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

• Construct the transformation matrix T, givencouples of control points:

T = fitgeotrans(movingPts, fixedPts, ‘affine’); %takes the 3 pairs of control points, movingPts and fixedPts, and uses them to infer the affine transformation

• Example-­-­-­-­-­-­-­Create an affine transformation that maps the triangle with vertices(0,0), (6,3), (-­2,5) to the triangle with vertices (-­1,-­1), (0,-­10), (4,4):X= [ 0 0;;

6 3;;-­2 5];;

Y = [-­‐1 -­‐1;0 -­‐10;4 4];

T =fitgeotrans(X,Y, 'affine');26

Page 27: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

• Construct the struct corresponding to the affine transformation matrix T:

tform = affine2d(T) % builds a TFORM struct for an Ndimensional affine transformation. T defines a forward transformation such thatTFORMFWD(U,T), where U is a 1-­‐by-­‐N vector, returns a 1-­‐by-­‐N vector X such that X = U * T(1:N,1:N) + T(N+1,1:N).T hasboth forward and inverse transformations. N=2 for 2D image transformation

27

tform =

C

T0

d

T 1

T

Page 28: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

B = IMWARP(A,TFORM, INTERP) %transforms the image A according to the 2-­‐D spatialtransformation defined by TFORM; INTERP specifies the interpolation filter

• Example 1-­-­-­-­-­-­-­-­-­Apply a horizontal shear to an intensity image.

I = imread('cameraman.tif');tform = affine2d([1 0 0; .5 1 0; 0 0 1]);J = imwarp(I,tform);figure, imshow(I), figure, imshow(J)

28

Page 29: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

29

Page 30: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

30

Goal: map a square (or more generally a quadrangle) to a quadrangle

It requires 8 parameters:

x = a0 + a1k + a2l + a3kl

y = b0 + b1k + b2l + b3kl

Page 31: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

31

Page 32: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

• Includes all deformations that can be modeled by polynomial transformations(affine and bilinear are special cases)

32

y = b0 + b1k + b2l + b3kl + b4k2 + b5l

2 + ...x = a0 + a1k + a2l + a3kl + a4k

2 + a5l2 + ...

Page 33: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

• Polynolial:

tform = fitgeotrans(movingPts,fixedPts,'polynomial',degree)

• OR more in general:

tform = fitgeotrans(movingPts,fixedPts,transformationType)

takes the pairs of controlpoints, ”movingPts” and ”fixedPts”, and uses them to infer the geometric transformation specifiedby transformationType.

Where “transformationType”…

33

Page 34: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

34

Page 35: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

35

Wave1: x(k, l) = k + 20 sin

2

128l

; y(k, l) = l

Page 36: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

36

Wave2: x(k, l) = k + 20 sin

2

30k

; y(k, l) = l

Page 37: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

37

Warp: x(k, l) =sign(k x0)

x0 (k x0)

2 + x0; y(k, l) = l

(x0, y0) = the center of the image

Page 38: Lezione 03b Warping - homes.di.unimi.ithomes.di.unimi.it/lanzarotti/PDF/Lezione_03b_Warping.pdf · 19 x(k,l)=(k x 0)cos( )+(l y 0)sin( )+x 0 y(k,l)=(k x 0)sin( )+(l y 0)cos( )+y 0

38

(x0, y0) = the center of the image