IEEE Transactions on Consumer Electronics

Vol. 21, No.11, November 2012

Zheng Guo Li, Jing Hong Zheng, Susanto Rahardja

Presented by Ji-Heon Lee

School of Electrical Engineering and Computer Science

Kyungpook National Univ.

Detail-Enhanced Exposure Fusion

Abstract

Exposure fusion

– Obtaining pseudo-HDRI without generation of ture HDRI

Proposed method

– Detail-enhanced weight

• Novel quadratic optimization-based method

− Extracting fine details from LDRIs

– Fusion fine details

2/17

Introduction

Tone mapping

– E.Reinhard’s

• Obtaining pseudo-HDR image from true-HDR image

– Farbman’s method

• Correction of filter in Retinnex Theory

− Based on Global Filter

− Based on Edge-Preserving Filter

• General flow

Input image Three detail layers Output luminance

Compression

function Restore color

Filter

3/17

Exposure fusion

– Mertens’ method

• Considering contrast, saturation and well-exposure weight

• Fusion of LDR images using pyramid method

– Zhang’s method

• Using gradient direction for ghost-free

4/17

Overview of proposed method

– Using quadratic optimization-based method

• Obtaining fine details

– Using T.Mertens’ method for restriction of well-pixel

• Contrast, saturation, well-exposedness

• Fusion of LDR images with pyramid method

– Restricting fused image using fine-detail weight

5/17

Detail-enhanced fusion of differently exposed images

Vector field

– Gradient field

– Well-exposure restriction

– Weight factor of gradient vector

1; 127( )

256 ;

z if zz

z otherwise

(1)

( ( , 1) ( , ), ( 1, ) ( , ))k k kY i j Y i j Y i j Y i j

where is luma components.

( , )(1 )kY i j k N

where is weight function. ( )z

,1( , ) ( ( , )) ( ( , 1))k k kW i j Y i j Y i j

,2 ( , ) ( ( , )) ( ( 1, ))k k kW i j Y i j Y i j

(2)

(3)

,1 ,2( ( , ), ( , ))k kY i j Y i j

6/17

– Vector field

• Constructing by weighted average of gradients over all exposures

, ,1

,1

log( ), 1,2

N

k q k qkq N

k qk

q

W Yv

W(4)

where stand for desired vectors,

is vectors of . , ,, ,q k q k qv W Y , ,( , ) ' , ( , ) ' , ( , ) 'q k q k qv i j s W i j s Y i j s1 2( , ) ( ( , ), ( , ))Ti j v i j v i jv

7/17

Fine details weight

22

212

21 2

2 2

min( ) ( )d

dd

d

L

LL vvyxL

v v(5)

where norm, Euclidean distance,

is represents fine details to be extracted at position ,

is vector containing all ,

function selected as ,

is regularization factor which obtaining tradeoff between two terms.

22is l

( , )dL i j ( , )i j

dL ( , ) 'dL i j s

( ) | |z z ( )z

(6)

8/17

– Optimal solution of fine details using following equation

1 2 1 1 2 2( ) ( ) ( ) ( )T T T T

x x y y d x yI D A D D A D D A D A v v L v v v v (7)

where ,and are discrete differentiation operators,

and are and .

xDyD

1( )A v 2( )A v1

1

( ( , ))diag

v i j

2

1

( ( , ))diag

v i j

9/17

T.Mertens’ method for restriction of well-pixel

– Weight sum of LDR images using pyramid method

• Obtaining weight

− Contrast

− Saturation

− Well-exposedness

• Fusing image operation of pyramid level

( , )kC i j

( , )kS i j

( , )kE i j

( , ) ( , ) ( , ) ( , )k k k kW i j C i j S i j E i j

1

{ ( , )} [ { ( , )} { ( , )} ]N

l l l

k k

k

L Z i j L Z i j G W i j

where is weight map Gaussian pyramid,

is Laplacian pyramid of LDR images,

is fusion image Laplacian pyramid.

{ ( , )}l

kG W i j{ ( , )}l

kL Z i j

{ ( , )}lL Z i j

(8)

10/17

Final fusion

– Combining fine detail weight and T.Mertens’ well-pixel weight

int( , ) ( , ) exp( ( , ))f dZ i j Z i j L i j (9)

where is intermediate image generated by T.Mertens’ method. int ( , )Z i j

11/17

Experimental results

Comparison of different selection of

(a) (b) (c)

(d) (e) (f)

(g) (h) (f)

Fig. 1. Comparison of

different selections of λ.

The input images are

captured under the same

lighting conditions. Image

courtesy of Jacques Joffre.

(a) First input image. (b)

Second input image. (c)

Third input image. (d)

Details extracted by λ =

0.25. (e) Details extracted

by λ = 1. (f) Details

extracted by λ = 4. (g) Final

image obtained by λ = 0.25.

(h) Final image

obtained by λ = 1. (i) Final

image obtained by λ = 4. 12/17

Comparison with other methods

– Input image courtesy of “Laurance Meylan”

(a) (b) (c) (d) (e) (f) (g)

(h) (i) (j)

Fig. 2. Comparison of the proposed exposure fusion scheme with two multiple-scale exposure

fusion schemes in [4] and [6]. Image courtesy of Laurance Meylan. (a) First input image. (b)

Second input image. (c) Third input image. (d) Fourth input image. (e) Fifth input image. (f) Sixth

input image. (g) Seventh input image. (h) Final image obtained by the exposure fusion scheme in

[4]. (i) Final image obtained by the proposed fusion algorithm. (j) Final image obtained by the

exposure fusion scheme in [6]. 13/17

– Input image courtesy of “Jacques Joffre”

Fig. 3. Comparison of the proposed exposure fusion scheme with two multiple-scale exposure

fusion schemes in [4] and [6]. Image courtesy of Jacques Joffre. (a) First input image. (b) Second

input image. (c) Third input image. (d) Final image obtained by the exposure fusion scheme in [4].

(e) Final image obtained by the proposed exposure fusion scheme. (f) Final image obtained by

the exposure fusion scheme in [6].

(a)

(a) (b) (c) (d) (e) (f)

14/17

– Input image with flash image

(f) (e) (d)

(c) (b) (a)

Fig. 4. Comparison of the proposed exposure fusion scheme with the HDR

imaging scheme of Photoshop CS5 and the exposure fusion scheme in [4]. (a)

Input image without flash. (b) Input image with flash. (c) Details extracted by

the proposed exposure fusion scheme. (d) Final image obtained by the HDR

imaging scheme in Photoshop CS5. (e) Final image obtained by the proposed

exposure fusion scheme. (f) Final image obtained by the exposure fusion

scheme in [4].

15/17

– Comparison focus on number of lighting resource

(f) (e) (d)

(c) (b) (a)

Fig. 4. Comparison of the proposed exposure fusion scheme with the HDR

imaging schemes of Photoshop CS5 and the exposure fusion scheme in [4]. (a)

Input image with one lighting resource. (b) Input image with two lighting resources.

(c) Input image with three lighting resources. (d) Final image obtained by the HDR

imaging scheme in Photoshop CS5. (e) Final image obtained by the proposed

exposure fusion scheme. (f) Final image obtained by the exposure fusion scheme

in [4]. 16/17

Proposed method

– Obtaining displayable HDR image without tonemapping

– Novel quadratic optimization-based method

• Extracting fine details from LDRIs

– Fusion fine details

Experimental results

– Detail enhancement

• More pleasing resulting than previous methods

Conclusions

17/17