Single View Metrology - Stanford Computational Vision...

Lecture 4 -Silvio Savarese 21-Feb-14

Professor Silvio Savarese

Computational Vision and Geometry Lab

Lecture 4Single View Metrology


• Review calibration• Vanishing points and lines• Estimating geometry from a single image• Extensions

Reading:[HZ] Chapter 2 “Projective Geometry and Transformation in 3D”[HZ] Chapter 3 “Projective Geometry and Transformation in 3D”[HZ] Chapter 8 “More Single View Geometry”[Hoeim & Savarese] Chapter 2


Calibration Problem

jC

ii PMP

i

i

iv

up

In pixels

TRKM

100

0

cot

sino

o

v

u

K

World ref. system

Calibration Problem

jC

ii PMP

i

i

iv

up

Need at least 6 correspondences

11 unknown

TRKM

In pixelsWorld ref. system

Pinhole perspective projectionOnce the camera is calibrated...

TRKM

C

Ow

-Internal parameters K are known

-R, T are known – but these can only relate C to the calibration rig

Pp

Can I estimate P from the measurement p from a single image?

No - in general [P can be anywhere along the line defined by C and p]

Pinhole perspective projectionRecovering structure from a single view

C

Ow

Pp

unknown known Known/

Partially known/

unknown

http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/hut/hutme.wrl

Recovering structure from a single view

Transformation in 2D

-Isometries

-Similarities

-Affinity

-Projective


Isometries:

1

y

x

H

1

y

x

10

tR

1

'y

'x

e

- Preserve distance (areas)- 3 DOF- Regulate motion of rigid object

[Euclideans]


Similarities:

1

y

x

H

1

y

x

10

tRs

1

'y

'x

s

- Preserve- ratio of lengths- angles

-4 DOF


Affinities:

1

y

x

H

1

y

x

10

tA

1

'y

'x

a

2221

1211

aa

aaA )(RD)(R)(R

y

x

s0

0sD


Affinities:

1

y

x

H

1

y

x

10

tA

1

'y

'x

a

2221

1211

aa

aaA )(RD)(R)(R

y

x

s0

0sD

-Preserve:- Parallel lines- Ratio of areas- Ratio of lengths on collinear lines- others…

- 6 DOF


Projective:

1

y

x

H

1

y

x

bv

tA

1

'y

'x

p

- 8 DOF- Preserve:

- cross ratio of 4 collinear points- collinearity - and a few others…

The cross ratio

1423

2413

PPPP

PPPP

The cross-ratio of 4 collinear points

Can permute the point ordering

1

i

i

i

iZ

Y

X

P

3421

2431

PPPP

PPPP

P1

P2

P3

P4

Lines in a 2D plane

0cbyax

-c/b

-a/b

c

b

a

l

If x = [ x1, x2]T l 0

c

b

a

1

x

xT

2

1

l

x

y

Lines in a 2D plane

Intersecting lines

llx l

l

Proof

lll

lll

0l)ll(

0l)ll(

lx

x

lx

x is the intersecting point

x

y

x

2D Points at infinity (ideal points)

0x,

x

x

x

x 3

3

2

1

c

b

a

l

c

b

a

l '

'

0

a

b

llLet’s intersect two parallel lines:

• In Euclidian coordinates this point is at infinity

• Agree with the general idea of two lines intersecting at infinity

l

l

0

'

'

2

1

x

x

x

'/'/ baba

x

2D Points at infinity (ideal points)

0x,

x

x

x

x 3

3

2

1

c

b

a

l

c

b

a

l '

'

0

0

lT

a

b

cbax

Note: the line l = [a b c]T pass trough the ideal point

So does the line l’ since a b’ = a’ b

l

l

'/'/ baba

x

Lines infinity l

Set of ideal points lies on a line called the line at infinity

How does it look like?

l

1

0

0

l

T

2

1

0

x

x

0

1

0

0

Indeed:

A line at infinity can thought of the set of “directions” of lines in the plane

'

x

0

'

'

a

b

0

''

''

a

b''

x

Projective transformation of a point at infinity

pHp '

bv

tAH

?pH

z

y

x

p

p

p

bv

tA

'

'

'

0

1

1

is it a point at infinity?

…no!

?pH A

0

'

'

0

1

1

0y

x

p

p

b

tA

An affine

transformation

of a point

at infinity is

still a point at

infinity

Projective transformation of a line (in 2D)

lHl T

bv

tAH

?lH T

b

t

t

1

0

0

bv

tAy

xT

is it a line at infinity?

…no!

?lH T

A

1

0

0

1

0

0

1

0

1

0

0

10 TT

TT

At

AtA

Points and planes in 3D

1

x

x

x

x3

2

1

d

c

b

a

0dczbyax

x

y

z

0xxT

How about lines in 3D? • Lines have 4 degrees of freedom - hard to represent in 3D-space

• Can be defined as intersection of 2 planes

Vanishing points

Points where parallel lines intersect in 3D

In 3D, vanishing points are the equivalent of ideal points in 2D

Hp

0

3

2

1

x

x

x

x

4

3

2

1

p

p

p

p

p

Vanishing points

Points where parallel lines intersect in 3D

C

In 3D, vanishing points are the equivalent of ideal points in 2D

0

3

2

1

x

x

x

x

Hp

= direction of the line in 3D

The horizon line

horizon

lHlT

Phor

l

The horizon line

Planes at infinity & vanishing lines

x

y

z

• Parallel planes intersect the plane at infinity in a common

line – the vanishing line ( horizon)

• A set of vanishing lines defines the plane at infinity

• 2 planes are parallel iff their intersections

is a line that belongs to

1

0

0

0

plane at infinity

Vanishing points and their image

c

b

a

c

b

a

K0IKMXv

0

dv K

d = direction of the line

d

C

v

0

c

b

a

xM

d

Vanishing points - example

C2

v2

C1

v1

R,T

star

1

1

1

1

1K

K

v

vd

2

1

2

1

2K

K

v

vd

21R dd R

v1, v2: measurements

K = known and constant

Can I compute R?No rotation around z

1d2d

In 2D

d

R

R

1

1

1

1

1K

K

v

vd

2

1

2

1

2K

K

v

vd

R

v1

v2

Vanishing lines and their images

C

n

Parallel planes intersect the plane at infinity

in a common line – the vanishing line (horizon)

horiz

TK ln

horizl


• Review calibration• Vanishing points and line• Estimating geometry from a single image• Extensions



Estimating geometry & calibrating the camera from a single image

- Recognize the horizon line

- Measure if the 2 lines meet

at the horizon

- if yes, these 2 lines are // in 3D

•Recognition helps reconstruction!

•Humans have learnt this

Are these two lines parallel or not?

Angle between 2 vanishing points

2

T

21

T

1

2

T

1

vvvv

vvcos

C

d1

v1

v2d2

If 90 0vv 2

T

1

1)(

T

KK

Projective transformation of

1)(

T

KK

It is not function of R, T

654

532

421

symmetric

02 zero-skew31

square pixel02

1.

2.

3. 4.

TRKP 1

PP

Tω

Absolute conic

Angle between 2 scene lines

90

0vv 2

T

1

v1v2

1)(

T

KK

Constraint on K

Single view calibration - example

v2

v3

0vv 2

T

1

0vv 3

T

1

0vv 3

T

2

654

532

421

31

02 Compute :

known up to scale

1)(

T

KK K(Cholesky factorization; HZ pag 582)

v2

Once is calculated, we get K:

Single view reconstruction - example

lh

knownKhoriz

TK ln = Scene plane orientation in

the camera reference system

Select orientation discontinuities

Single view reconstruction - example

Recover the structure within the camera reference system

Notice: the actual scale of the scene is NOT recovered

C

- Recognize the horizon line

- Measure if the 2 lines meet

at the horizon

- if yes, these 2 lines are // in 3D

•Recognition helps reconstruction!

•Humans have learnt this

Are these two lines parallel or not?


• Review calibration• Vanishing points and lines• Estimating geometry from a single image• Extensions



Criminisi & Zisserman, 99

http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/merton/merton.wrl

La Trinita' (1426)

Firenze, Santa Maria

Novella; by Masaccio

(1401-1428)

http://www.robots.ox.ac.uk/~vgg/projects/SingleView/models/hut/hutme.wrl

Manually select:• Vanishing points and lines;

• Planar surfaces;

• Occluding boundaries;

• Etc..

Single view reconstruction - drawbacks

Automatic Photo Pop-upHoiem et al, 05

Automatic Photo Pop-upHoiem et al, 05…

Automatic Photo Pop-upHoiem et al, 05…

http://www.cs.uiuc.edu/homes/dhoiem/projects/software.html

Software:

Training

Image Depth

Prediction

Planar Surface

Segmentation

Plane Parameter MRF

Connectivity Co-Planarity

Make3D

youtube

Saxena, Sun, Ng, 05…

http://www.youtube.com/watch?v=33oHekQipTg

A software: Make3D

“Convert your image into 3d model”

Single Image Depth ReconstructionSaxena, Sun, Ng, 05…

http://make3d.stanford.edu/

http://make3d.stanford.edu/images/view3D/185

http://make3d.stanford.edu/images/view3D/931?noforward=true



http://make3d.stanford.edu/images/view3D/931?noforward=true

Coherent object detection and scene

layout estimation from a single image Y. Bao, M. Sun, S. Savarese, CVPR 2010,

BMVC 2010

Y. BaoM. Sun

Next lecture:

Multi-view geometry (epipolar geometry)

Single View Metrology - Stanford Computational Vision...

Documents

Transcript of Single View Metrology - Stanford Computational Vision...