An Automatic Image Enhancement Method Adaptive to the Surround Luminance Variation for Small Sized

Mobile Transmissive LCD

IEEE Transactions on Consumer Electronics vol. 56, no. 3, 2010

Youn Jin Kim

Presented by Dae-Chul Kim

School of Electronics Engineering Kyungpook National Univ.

Abstract Proposed method

– Improving the image contrast under various surround levels for small-sized mobile phone

• Determination of improving parts of the image − Using non-linear weighting function in spatial frequency domain

according to human visual system • Using gain control of a 2D contrast sensitivity function

2/21

Introduction Portable displays under a diverse range of viewing conditions

– Huge loss in contrast under bright outdoor viewing conditions • Surround effects, correlated color temperature and ambient lighting

– Survey of mobile imaging considering surround luminance and ambient illumination effects

• Color perception under various ambient illumination level − Surround and ambient lighting effects on color appearance modelling

3/21

Proposed method – Robust image enhancement filter based on gain control which

compensates for the effects of surround luminance • Human visual system in spatial frequency domain • Analyzing the contrast discrimination ability of HVS • Determination of adaptive enhancement gain using contrast

sensitivity function(CSF)

4/21

CSF representation – Amount of minimum contrast at each spatial frequency – Distinguish a sinusoidal grating – Gabor patterns over a range of spatial frequencies

Gabor features at 5 scales and 8 orientations Sinusoidal grating

5/21

– Luminance CSF • A peak in contrast sensitivity at moderate spatial frequencies

(~5.0 cycle per degree; cpd) • Falls off at both lower and higher frequencies

(band pass characteristics) − Fall-off in contrast sensitivity at higher spatial frequency

» Spatial limitation in the retinal mosaic of cone receptors − Reduction in contrast sensitivity at lower spatial frequency

» Center-surround receptive fields

6/21

Proposed surround luminance adaptive image enhancement

Contrast sensitivity reduction of the HVS – Surround luminance adaptive CSF

• Modification of existing CSF model suggested by Barten − CSF model suggested by Barten

0.51( ) exp( )[1 exp( )]( )t

CSF u au bu c bum u

= = − +

0.2

2

540(1 0.7 / )121

(2 / 3)o

La

X u

−+=

++

0.150.3(1 100 / )b L= + 0.06c =

CSF dependence on luminance CSF dependence on viewing angle 7/21

Contrast sensitivity reduction of the HVS – Surround luminance adaptive CSF

• Modification of existing CSF model suggested by Barten • Modelling of a function of spatial frequency and adapting luminance • A variable for compensating the surround luminance effect is

multiplied to adapting luminance ϕ

L

0.5( ) exp( )[1 exp( )]CSF u au bu c bu= − + (1)

0.2

2

540(1 0.7 / )121

(2 / 3)

La

w u

ϕ −+=

++

0.150.3(1 100 / )b Lϕ= + 0.06c =

where L is the mean luminance of white and black of display under given surround luminance LS and is the surround luminance effect function ϕ

8/21

• Surround illumination function − Non-linear function of luminance for approximating the perceived

brightness reduction effect caused by ambient illumination level increase − Used as a weighting function to determine parts of image having under

spatial frequency than a given ambient 410 /0.180.17 0.83 sLeϕ−− ⋅= + (2)

Fig. 1. Relation between the surround luminance factor and the normalized surround luminance(LS/104 )

9/21

– CSFs for three surrounds; dark(0lx), overcast(6100lx) and bright(32000lx)

• Spatial frequency with maximum contrast sensitivity − Moved toward a lower frequency from dark(4.4 cpd) to bright(3.8 cpd) − 7 and 15% loss in contrast sensitivity of human visual system for

overcast and bright, respectively

Fig. 2. Comparison of CSFs under dark and ambient illuminations

10/21

Gain control – Using adaptive weight filter

• Compensating the loss in image contrast • Correlated to the normalized contrast sensitivity difference between

reference(dark) and target surround luminance level

• Enhancement gain is multiplied to amplitude of image − Offset of these weight filters should be increased up to greater than 1 − A contrast value of 1 was added to

( , ) ( , ) ( , )R TD u v CSF u v CSF u v= − (3)

( , )D u v

( , ) ( , ) (1 )H u v D u v C= + + (4)

where max( ( , ))C D u v=

11/21

– Estimated adaptive weight filter for three surround level • Dark, overcast and bright • Field size of 5 degree and 89.17cd display’s adapting luminance • The enhancement threshold levels are determined as 0, 0.15 and 0.31

Fig. 3. The adaptive weighting filter estimates

12/21

Experimental setup

Test images – Blue sky, green grass, water, facial skin and fruit scenes – Using Samsung-SCH-S250(QVGA: 320X240 pixels)

Fig. 4. Test images

13/21

Characterization for estimation of displayed image – Using CIELAB color space

* 1/3

* 1/3

*

*

1/3

116( / ) 16 for / 0.008856

903.3( / ) for / 0.008856

500[ ( / ) ( / )]

200[ ( / ) ( / )]

( / ) ( / ) for / 0.008856( /

n n

n n

n n

n n

n n n

n

L Y Y Y Y

L Y Y Y Y

a f X X f Y Y

b f Y Y f Z Z

f F F F F F Ff F F

= − >

= ≤

= −

= −

= >

* * 2 * 2 * 2

) 7.787( / ) 16 /116 for / 0.008856

( ) ( ) ( )ab

n nF F F F

E L a b

= + ≤

∆ = ∆ + ∆ + ∆

(5)

(6)

14/21

Test images were manipulated in terms of three attributes – blurrness, brightness and noisiness – Adjusting resolution, luminance and noise level of image

• Resolution changing from 200 to 80 ppi with steps of 40 ppi • Luminance changing from 100 to 25% with steps of 25% • Adding Gaussian noise by changing variance of Gaussian

function from 0 to 0.006 with steps of 0.002

Fig. 5. Sample images 15/21

Test image selection – Using 35 randomly selected image among 110 images for 5

distinct test image

Table 1. The randomly selected test images

16/21

Results and discussion

Test image and enhanced image for two surround conditions – Histograms of luminance of composite channel(L) – Tonal variance in histogram yields quite spread and both mean and

standard deviation were increased • Mean was 113.65 for original, 129.76 for overcast and 137.37 for bright

17/21

Comparison of resulting images

Fig. 6. Example of enhanced images and their luminosity histogram

18/21

Comparison between enhanced and original images in terms of image quality scores – The enhanced image were judged as higher image quality

• Most of the data points are upper the 45-degree line

Fig. 7. Comparison between the original and enhanced image for each surround condition

19/21

Conclusion An adaptive image enhancement algorithm

– Quantification of contrast discrimination ability of human under ambient illumination

• Determination of part in image with spatial frequency under a certain surround luminance level

• Applying weighting function as image enhancement filter in spatial frequency domain

• Most of the enhanced images were rated as higher image quality scores

20/21