School of Electrical Engineering and Computer Science

Kyungpook National Univ.

An Analysis of Visual Adaptation and

Contrast Perception for Tone MappingIEEE Transactions on Pattern Analysis and Machine Intelligence

Vol. pp, No. 99, 2011

Sira Ferradans, Marcelo Bertalmio, Edoardo Provenzi and Vicent Caselles

Presented by Il-Su Park

Abstract

Tone mapping

– Problem of compressing the range of a High Dynamic

Range image

Proposed method

– Tone mapping operator

• Visual adaptation

– Based on experiments on human perception

» Point out the importance of cone saturation

• Local contrast enhancement

– Based on a variational model inspired by color vision

phenomenology

2 / 29

Introduction

Tone mapping operators

– Perceptually-based TMOs

• Global

– Very fast

– Not introduce halos or artifacts

– Poor contrast rendition

– Stevens‟ law

– Naka-Rushton formula

– Weber-Fechner‟s law

• Local

– Improving details

– Create halos and artifacts next to edges

3 / 29

– Gradient-based TMOs

• Shrinking large intensity gradients

• Preserving small fluctuations corresponding to fine details

– Proposed method

• Main purpose of TM

– Emulating as much as possible the perception of contrast and

color produced by the real-world scene

• Following the TM philosophy proposed by Ward et al.

• Replicating basic properties of the HVS

– Taking into account localized cone saturation

– Computing the semisaturation constant in the Naka-Rushton

equation

4 / 29

Visual adaptation

Human visual systems

– Operating over such a huge range

• From to

– Cannot operate over entire range simultaneously

– Adapting locally to the average of nearby retinal

illumination values

– Handling a smaller magnitude interval

6 210 cd/m 6 210 cd/m

5 / 29

– Neuroscience experiments

• Using very simple, non-natural images

– Superimposing brief pulses of light with intensity I on a uniform

background

• Measuring the electric response inside the retina

– Changing accordingly to the empirical law

» Michaelis-Menten equation or Naka-Rushton equation

max

n

n n

s

Vr

V

(1)

where is the light level at which the photoreceptor response is half maximal,

the change of electric potential is the photoreceptor‟s

physiological response to , and

is the highest difference of potential that can be generated, and

is a constant.

s

V

maxVn

6 / 29

– Foundational experiments of psychophysics

• Developed by the German physician E.H. Weber

• Measuring the noticeable difference between the background

light and the superimposed light

• Weber‟s law

– Constant ratio between the Just Noticeable Difference(JND)

and background intensity I

– Linear relationship between and in log-log units

– Not hold for low intensity values

k (2)

where is a perceptual constant called Weber fraction.0k

log log log k

7 / 29

• Weber-Fechner‟s law

– Introducing the concept of „dark light‟

– Matching the experimental data also at low intensity values

– Plotting JND vs. I data in log-log scale

» Known as threshold versus radiance(t.v.r.) or threshold

versus intensity(t.v.i.)

km

(3)

where is a quantity often interpreted as internal noise in the visual

mechanism.

0m

8 / 29

• Cone saturation

– Increasing very steeply after a certain point

Fig. 1. Cones saturate when the stimuli field is not steady

but pulsed: perceptual t.v.r. curves of cones for pulsed

and steady stimuli, figure adapted from Shevell [34].

9 / 29

• Rewriting Weber-Fechner‟s law

– Using JND as a sensation magnitude

– Integrating eq. (4)

k sm

(4)

where is the increment in the sensation-magnitude function ,

also called “perceived brightness”.

s s

0logs k m s (5)

where .1

kk

10 / 29

– Perceived brightness

• Weber-Fechner equations (WF)

– Describing detectable-difference threshold for small, steady-

state stimuli

• Naka-Rushton equations (NR)

– Modeling electrical responses to flashed stimuli

Fig. 2. Tone mapping curves for the logarithmic mapping (Weber-Fechner law),

the Naka-Rushton equation and the curve for the first stage of proposed method. 11 / 29

– First stage of proposed method

• For radiances below

– Weber-Fechnner‟s law

• For radiances above

– Naka-Rushton equation

0log ,

,

M

n

Mn n

s

k m s

c

(6)

where is chosen so as to ensure the continuity of at .0s cM

12 / 29

• TM results obtained using eqs. (1), (5) and (6)

– Left-hand image

» Overexposing the brightest regions

– Middle image

» Overall poor contrast

– Right-hand image

» Combining the best characteristics of the other two images

Fig. 3. Left: TM output using the Naka-Rushton equation. Middle: TM output using a logarithmic

mapping (Weber-Fechner‟s law.) Right: TM output of the first stage of our method, eq. (6). 13 / 29

Local contrast enhancement

Reproduction of the color and contrast perception

– Bertalmio et al.

• Proposing a perceptually-inspired color enhancement method

– Performing local contrast enhancement

– Approximating color constancy

• Defining an energy functional with three terms

– Local contrast, dispersion and attachment

0, , , ,wE I C I D I A I I (7)

E I

14 / 29

– Contrast term

» Measuring a weighted average contrast over the image

– Dispersion term

» Measuring the average departure of the values of the

pixels in from a given middle value

– Attachment to the original

» Measuring the average departure of the values of the

pixels in I from corresponding values in

I

0I

15 / 29

– Second step of proposed method

• Establish a close relationship between the functional in eq.

(7) and Retinex theory

– Final output of proposed method

• Image minimizing eq. (8)

2

2 2

0

1,

4E I w x y I x I y dxdy

I x dx I x I x dx

(8)

where we have chosen specific values for the constants

the Gaussian kernel gives locality to the contrast measure, and

is the average value of the original image .

, and w

0I

I

16 / 29

– First example

• Simple image with gray bands

• Preserving Mach band phenomena

Fig. 4. Top row: (a) original image, (b) after first stage of our TMO, (c) final output of

our TMO. Bottom row: corresponding profiles of scan lines.

17 / 29

– Second example

• Not changing the overall colors and contrast

• More detail on the stone arch and columns

Fig. 5. (a) Output of first stage (b) Output of second stage (c) Visualization of error of first stage (d)

Visualization of error of final output. Error computed with the metric of Aydin et al. [42].

(a) (b) (d)(c)

18 / 29

Implementation

Implementation details of proposed method

– Deal with each of the three color channel separately

– Background intensity

• Geometric average between the arithmetic mean and the

median of the radiance values

• Specifying the overall brightness of the final output

0.5 0.5median meanb

10b

19 / 29

– Semisaturation constant

• Center value in the 4-order operative range of the cones

– Inflexion point

• Switch from Weber-Fechner to Naka-Rushton

– Weber-fractions

• Taken from experimental data

• for red and green cones

• for blue cones

– Another parameter

10 10 10log log 0.37 4 log 1.9s b b

210M s

1001.85

k

1008.7

k

1.20.74, 10sn m

20 / 29

– Applying eq. (6) to the input HDR image

– Normalizing the result

– Looking for the image minimizing eq. (8)I

0 0

1

1

2

1 2

n

n

In

I x t I x R x

I xt

(9)

where ,

is the mean of and

with being a normalized 2D Gaussian kernel of effective radius .

0.2t

0 0I

, signn

n n

IR x w x y I x I y dy

(10)

w

21 / 29

– Effect of varying ρ and σ

Fig. 6. On the first row, effect of varying ρ. From left to right: ρ = −1, 0, 1, 2. Second row, effect

of varying σ. From left to right: σ = 50%, 20%, 4%, 2% of the number of rows or columns or

the image, whichever is greater.

22 / 29

Results and comparisons

First stage of proposed method with three gobal

TMOs

– Less contrast distortion

Fig. 7. Comparison with global TMOs. TM results (top row) and distortion maps (bottom row) for: (a)

Drago et al. [46] (b) Pattanaik et al. [8] (c) Reinhard and Devlin [9] (d) first stage of our TMO.

(a) (b) (d)(c)

23 / 29

Importance of having an objective quality metric

– Producing lots of amplification errors

– Ameliorating error

• Selecting an even higher value of σ

Fig. 8. (a) Output of the first stage. (b) Output of the second stage. (c) Visualization of the first

stage error. (d) Visualization of the final output error. Error computed with the metric of Aydin et

al. [42].

(a) (b) (d)(c)

24 / 29

Results for proposed method and for several

state of the art methods

Fig. 11. Results of our method for several images obtained from the MPI database, see

http://www.mpi-inf.mpg.de/resources/hdr/gallery.html for further details. 25 / 29

Fig. 12. TM results (rows 1 and 3) and distortion maps (rows 2 and 4) for these TMOs: (a) Drago et

al. [46] (b) Mantiuk et al. [21] (c) Pattanaik et al. [8] (d) Reinhard and Devlin [9] (e) the proposed

method. Original image from row 1 courtesy of ILM. Original image from row 3 courtesy of R.

Mantiuk.

(a) (b) (d)(c) (e)

26 / 29

Numerical results

Fig. 10. Error percentages computed with Aydin et al. [42], averaged over our image data set,

for several TMOs (see text.)

27 / 29

Computing statistics for the number of pixels

with error

– Slightly poorer performance with respect to the loss of

contrast

Fig. 9. Percentage of pixels with errors, for our image set and for the methods considered.

Horizontal axis: images in Figs. 11 and 12.

28 / 29

Conclusions and perspectives

Proposed method

– Tone mapping operator

• Inspired by basic perceptual properties of human vision

– Visual adaptation

– Local contrast enhancement

– Two parameters

• Control the overall brightness and the contrast of the final

image

29 / 29