Homographies Panoramas

8/12/2019 Homographies Panoramas

1/31

2D Homographies

Raul Queiroz Feitosa


2/31

3/19/2013 2D Homographies 2

Objective

To introduce the problem of estimating2D projective transformationsas well

as some basic tools for parameter

estimation.


3/31


Outline

Motivation

Direct Linear Transformation (DLT)

Normalization

Robust Estimation

Non Linear Method

Assignment


4/31


Motivation: Panoramas


5/31


Building a Panorama

Step 1:find pairs of corresponding points

p1p'2

p2p' p1

p'i pi

p'np

n

p'jpj


6/31


Building a Panorama

Step 2:Estimate the matrixH (the homography) such

that

pi=Hp'i

where piand p'i are homogeneousvectors representing

corresponding points, i.e.

k ( uivi1 )T=H( u'iv'i1 )T

for some k0 (homogeneous) and for all 1 i n.


7/31


Building a Panorama

Step 3:transform geometrically one image byusing the matrixH


8/31


Building a Panorama

Step 4:align and blend (not here) the images


9/31


Outline

Motivation

Direct Linear Transformation (DLT)

Normalization

Robust Estimation

Non Linear Method

Assignment


10/31


Direct Linear Transformation

Derivation:Let hjTbe thej-th row ofH, so we may write

Clearly . Thus

i

T

i

T

i

T

i

p

p

p

p

3

2

1

h

h

h

H

0

hh

hh

hh

i

T

ii

T

i

i

T

ii

T

i

T

i

T

i

ii

vu

u

v

pp

pp

pp

pp

12

31

23

H

0ii

pp H

T

T

T

hhh

hhh

hhh

3

2

1

987

654

321

h

h

h

H


11/31



Derivation (cont.):After some manipulation, you obtain

where

0

h

h

h

0

0

0

3

2

1

TT

ii

T

ii

i

T

ii

TT

i

i

T

ii

T

i

T

uv

u

v

pp

pp

pp

987

654

321

3

2

1

,

hhh

hhh

hhh

H

h

h

h

h

3

9


12/31



Derivation (cont.):Each pair of points generates two equations of the form

With npairs we get 2n equations on 9 unknowns!

4 pairs of points are enough!

0

h

h

h

0

0

3

2

1

T

ii

TT

i

T

ii

T

i

T

i

u

v

pp

pp

0

h

h

h

3

2

11

n


13/31


Outline

Motivation

The Direct Linear Transformation (DLT)

Normalization

Robust Estimation

Optimal Estimation

Assignment


14/31


Normalization

For improved estimation it is essential

for DLTto normalize the data sample, as follows

1. The points are translated so that their centroid is at the origin

2. The points are scaled so that the average distancefrom the

origin is equal to .3. This transformation is applied toeach of the two images

independently.

Multiple View Geometry in computer vision, 2nd Ed., R. Hartley and A. Zisserman, 2003, Cambridge, section 4.4.4, pp.107.

2

... ........... ......

u

v


15/31


Normalization

Normalization is carried out by multiplying the

data vector by proper matrices Tand T'

DLT is applied to to obtain ,

which is actually different from .

iiii pppp ~and~ TT

~Hii pp

~and

~

H

~~

1

iii pppp

TTTT HH

H


16/31


Normalization

The normalizing matrix T is given by

where are the mean values of ui,virespectively

and

The computation of T'is analogous.

/100

10

01

s

v

u

sT

vu ,

n

i

ii vvuuns

1

21

22

2


17/31


Normalization

Step-by-step Compute the normalization matrices Tand T'

Perform normalization by computing

Apply DLT to and compute

Denormalize the result by computing

~H

iiii pppp ~and~ TT

TT HH~

-1

ii pp

~and

~


18/31


Outline

Motivation


Normalization

Robust Estimation

Non Linear Method

Assignment


19/31


Robust Estimation

RANSAC robust estimation1. Repeat for kiterations

a) Select a random sample of 4 pairsof corresponding pointsand computeH.

b) Calculatefor each putative correspondence the distance,d (pi,Hp'i) =|| pi-Hp'i||

from the projected to the actual point position.

c) Determinethenumber of inliersconsistent withHi.e.,

d (pi,Hp'i) < tpixels.

2. Choose theH with the largest number of inliers.

3. RecomputeHwith the inliers.


20/31


Robust Estimation

The geometric distanceIt is the distance between the projected and the actual

position of the corresponding points

d (pi,Hp'i) =|| pi-Hp'i||

Hp'i

pi

Hp'i

image 1 image 2

d (pi,Hp'i)


21/31


Outline

Motivation


Normalization

Robust Estimation

Non Linear Method

Assignment


22/31


Non Linear Method

Reprojection error

Sum of the squared errorsof the forwardand the

backwardtransformation, i.e.,

21-2

iiiiiD p-pp-p HH

H

H-1 ii p-p-1

H

ii p-p H p'i

pi

Hp'i H-1pi

image 1 image 2


23/31


24/31


Panorama Example

Input Images

Panorama


25/31


Outline

Motivation


Normalization

Robust Estimation

Non Linear Method

Assignment


26/31


Assignment

General Formulation

i. Take a set of images of the same scene, rotating thecamera around its optical center.

ii.

For a pair of images collect manually a set ofcorresponding points (hint:use the functioncaptura_pontos).

iii. Compute the homographyHfrom these

correspondences.

iv. Create a panorama from both images and from H.

(hint:use the functionPanorama2).

v. Add new images by repeating steps ito iv.
http://www.ele.puc-rio.br/~visao/Topicos/Panorama/ImagesPanorama.rarhttp://www.ele.puc-rio.br/~visao/Topicos/captura_pontos.mhttp://www.ele.puc-rio.br/~visao/Topicos/Panorama2.mhttp://www.ele.puc-rio.br/~visao/Topicos/Panorama2.mhttp://www.ele.puc-rio.br/~visao/Topicos/captura_pontos.mhttp://www.ele.puc-rio.br/~visao/Topicos/Panorama/ImagesPanorama.rar


27/31


Assignment

Assignment 1

1. Write a MATLAB program that implements the

solution for the problem formulated in the previous

slidei. By using DLT without normalization

ii. By using DLT with normalization (hint: use the function

NormalizaPontos)

Compare the results obtained by each approach.
http://www.ele.puc-rio.br/~visao/Topicos/NormalizaPontos.mhttp://www.ele.puc-rio.br/~visao/Topicos/NormalizaPontos.m


28/31


Assignment

Assignment 1 (cont.)1. Your main task here is the development of a function

that implements DLT, which may have the followinghelp

% H=DLT(u2Trans,v2Trans,uBase,vBase,)

% Computes the homography H applying the Direct Linear Transformation

% The transformation is such that

% p = H p'

% (uBase vBase 1)'=H*(u2Trans v2Trans 1)'

%

% INPUTS:

% u2Trans, v2Trans - vectors with coordinates u and v of the image to be transformed (p')

% uBase, vBase - vectors with coordinates u and v of the base image p%

% OUTPUT

% H - 3x3 matrix with the Homography

%

% your name - date


29/31


Assignment

Assignment 2

2. Provide a new solution for the same general problem

where DLT is replaced by the non linear method.

Some hints: use the MATLAB function lsqnonlin.

the result ofHp'i is in homogeneous coordinates, i.e., it

must be scaled to make the third vector component equal

1, and to obtain the actual column-row coordinates.


30/31


Assignment

Assignment 3

3. Provide a new solution for the same general problem

where DLT/ the non linear method is replaced by

RANSAC.Some hints:

use DLT to obtain a initial solution

Use the reprojection error in RANSAC step 3 (you have it

from previous assignment)


31/31


Next Topic

Image Matching

Homographies Panoramas

Documents

Transcript of Homographies Panoramas