1

Seamless Image Stitching in the

Gradient Domain

Anat Levin, Assaf Zomet, Shmuel Peleg and Yair WeissECCV 2004

Presenter: Pin Wu 1 Oct 2007

2

Outline

• Introduction

– Related Work

• GIST: Gradient-domain Image Stitching

– GIST1

– GIST2

• Implementation details

– Iterative Optimization

• Experiments & Discussion

– Panoramic View

– Object Parts

3

Image Stitching

• Combine individual images having some overlap into a composite image

• Commonly used for generating panoramic images

+ →

• Visual artificial edges at the seam due to the differences in

– Camera gain

– Scene illumination

– Geometrical misalignment

4

Related Approaches

• Optimal seam algorithm

– Optimal seam[3,2,4]

• Optimal seam curve that fits to the minimal difference

• Transition smoothing

– Feathering[6] or alpha blending

• Weighted combination with coefficient functions of distance

– Pyramid Blending[7]

• Different alpha masks in different frequency bands

• Optimization process

– Poisson Image Editing[10]

• Optimization over the gradient domain

– Gradient-domain Image Stitching(GIST)

[6]

[2]

2

5

Comparison of Stitching Methods

Photometric

inconsistencies

Horizontal

misalignments

Vertical

misalignments

6

Requirement

• Good stitching (seamless)

– The mosaic should be as similar as possible to the input images, both geometrically and photo-metrically.

– The seam between the stitched images should be invisible.

• Quality measurement is operated in gradient domain.

7

GIST: Gradient Image STitching

• GIST1: Optimizing a cost function over Image Derivatives

–1 2 1 1

2 2

1 2 1 2

ˆ ˆ( ; , , ) ( , , , )

ˆ ( , , , ),

where ( , , , ) ( ) ( ) ( )

p p

p

p

p pq

E I I I W d I I W

d I I U W

d J J W W q J q J qφ

τ ω

τ ω

φ∈

= ∇ ∇ ∪ +

∇ ∇ ∪ −

= −∑

8

GIST: Gradient Image STitching

• GIST2: Stitching Derivative Images

–

–

–

–

f

x

∂

∂

1 1 2 2

1 2 1 2

Compute deriatives: , , ,

Form a field ( , ), where

, are formed with stitching , and ,

x y

x y

I I I I

x y x y

F F F

I I I IF F

x x y y

∂ ∂ ∂ ∂

∂ ∂ ∂ ∂

=

∂ ∂ ∂ ∂

∂ ∂ ∂ ∂

(Feathering, Pyramid blending, or optimal seam)

minimize ( , , , ) ( ) ( ) ( )p

p pq

d I F U U q I q F qπ

π∈

∇ = ∇ −∑

3

9

GIST Properties

•1GIST1 under V.S optimal seam �

1 2GIST1 under V.S GIST1 under � �

2

1

: tending to mix the derivatives and hence blurring in the overlap region.

: tending to behave similarly to the optimal seam methods.

−

−

�

�

•

1GIST1 under is recommended �

• Which method to use?

2 2 GIST1 under Feathering of the gradient images (GIST2) under

(solution of Poisson Equation)

− ≡� �

Same results in geometric misalignments condition

GIST1 > optimal seam if no perfect seam, e.g. photometric inconsistencies

−

−

10

Implementation

• Iterative optimization

– Initial the solution image I

– Iterate until convergence:

• For all x,y in the image,

•

– Solving through FFT

2GIST1 under �

•

– Solving through Linear Programming

– Uniform intensity shift

(input image, mosaic image)

1GIST1 under �

2

2 1 2 1 2( , , , ) ( ) ( ) ( )q

d J J W W q J q J qφ

φ∈

= −∑

1 1 2 1 2( , , , ) ( ) ( ) ( )q

d J J W W q J q J qφ

φ∈

= −∑

: ( )

: ( ) ,

x 0, 0, 0

i ii

i i

i i

Min z z

Subject to Ax z z b

z z

+ −

+ −

+ −

+

+ + =

≥ ≥ ≥

∑

1�

( 1, ) ( , ), ( 1, ) ( 1, ),( , ) 2* ( { })

( , 1) ( , ), ( , 1) ( , 1)

j j

j

j j

I x y Dx x y I x y Dx x yI x y median

I x y Dy x y I x y Dy x y

+ − − + −← ∪

+ − − + −

11

Differences with Poisson Editing

• GIST use the gradients of both images in the overlap region.

• The optimization is done under different norms

Performance

+ Overcome global inconsistencies

+ Overcome Misalignments

- Speed

- Convergence

12

Double edge

Gradient-domain methodsImage-intensity methods V.S.

4

13

Stitching Panoramic View

1

(a) Optimal seam

(b) Feathering

(c) Pyramid blending

(d) Optimal seam (gradient)

(e) Feathering (gradient)

(f) Pyramid blending (gradient)

(g) Poisson editing

(h) GIST1 under �

14

Stitching Panoramic View

1

(b) Feathering

(e) Feathering (gradient)

(f) Pyr

(a) Optimal seam

(c) Pyramid blend

amid blending (gr

ing

(d) Optimal seam (gradient)

(g) Poisson editi

adient)

(h) GIST1 un

g

der

n

�

15

Stitching Panoramic View

1

(a) Optimal seam

(b) Feathering

(c) Pyramid blending

(d) Optimal seam (gradient)

(e) Feathering (gradient)

(f) Pyramid blending (gradient)

(g) Poisson editing

(h) GIST1 under �

16

Stitching Panoramic View

1

(b) Feathering

(c) Pyramid blending

(e) Feathering (gradient)

(f) Pyramid blending (gradi

(a) Optimal seam

(d) Optimal seam

ent)

(g) Poisson

(gradient)

(h) GIST1 und

editin

er

g

�

5

17

Stitching Object Parts

1GIST1 under � 2GIST1 under � Pyramid (gradient)Original composition18

Stitching Object Parts

1GIST1 under � 2GIST1 under � Pyramid (gradient) Pyramid (image)