Journal of Electronic Imaging

Vol. 21, No. 2, 2012

Cheung-Wen Chang and Yung-Nien Sun

Presented by Euiwon Nam

School of Electrical Engineering and Computer Science

Kyungpook National Univ.

Color Correction with Hue-Determined

Compensation for Illumination Changing

Abstract

Aim to improve conventional transformation

– Linear hypothesis of color correction

• Using hue-determined matrix

Proposed method

– Hue-compensated diagonal model

• Correcting colors

− Chromatic-saturated illuminations

• Reinforcing illuminant complementary color

− Chromatic adaptation occurring in HVS

− Weighting von Kries coefficients

• Weighting terms

− Illuminant-hue specific based on relative absorption model

2/25

Introduction

Color constancy

– Correcting color perception

• Retinal adaptation mechanism in HVS

Color constancy algorithms

– Gray-world and White-Patch

• Adopting statistics of image colors to estimate illumination

• Adopting to transform casted colors to canonical colors

• Image-dependent and only applicable to certain types of image casting

– Finlayson’s correlation and general gamut-mapping methods

• Evaluating probability of casted colors in a gamut for specific illuminant

• Estimating optimal illumination by probability

3/25

Computational color constancy

– Neural network

– Support vector machine

– Compound approaches

– Achieving color corrections in image-processing aspect

– Data-dependent highly and expensive computation processes

Color constancy with color corrections

– Estimated illumination firstly

– Acquired corresponding canonical colors

• Using looking-up-table strategy or transformation methods

– Concerning transformation-based color corrections

• Adopted transformation matrix to transform casted colors

− Correcting cast colors

• Von Kries method or nonlinear matrices

4/25

Von Kries method

– Adopted LMS ratios of illuminants

– Simplicity and efficiency

– Difficult of demonstration with good performance for illuminants

• Possessing less-smooth spectrum

• Discussed degradation problem by Van Trigt

− Illuminant-dependent in von Kries quotient

− Synthetic and real-image data without feasible method

• Proposed Hue-compensated diagonal model(HCDM)

– Sharpening and non-diagonal PCA methods

• Synthetic data for improvement von Kries

• Computing more efficient diagonal scales

− Removal of variations of sensor sensitivity by changing bases of transformation

− Better transformation of color correction

» Selection of sensor sensitivity function

5/25

Funt’s study

– Adopted three eigenvectors from principle component analysis

– Overcoming Limitation of diagonal transformation

– Proving to outperform von Kries and sharpening methods

Proposed method

– Hue-compensated diagonal model

• Diagonal, matrix-based and illumination adaptive model of color correction

• Complementary illuminant color to compensate for color distortion

• Adjusting weights of von Kries coefficients

− According to complementary color estimated

» Based on relative absorption model

6/25

– Relative absorption model(RAM)

• Capable of building simple map

− Between hue and LMS cone signals

• Term of representing complementary illumination color hue

− Reinforcing by changing LMS cone signals

– Correction of object colors

• Using both proposed HCDM and comparative methods

Fig. 1. Flowchart of the hue-compensated diagonal model (HCDM). 7/25

Methods

Color correction by chromatic-adaptation transformation

– Diagonal transformations

• Tri-stimulus sensor response 𝑞𝑚

• Formulation of color correction

− Using chromaticity 𝑊𝑠𝑡𝑑 and 𝑊𝑖𝑙𝑙

( ) ( ) ( ) , ( ) , e( )m m mq s e h d h X X (1)

where is wavelength of human visible spectrum, is spectrum space,

( ) is surface reflectance, ( ) is spectra of illuminat and

( ) is ' th channel sensorm

X

s e

h m

( , ) std std ill illQ f W W Q

where is canonical colors, is casted colors, ( , ) is

function of illuminant chromaticity

std ill std illQ Q f W W

(2)

8/25

• von Kries transformation

− linear proportional terms of white points

– Non-diagonal transformations

• Principle component analysis transformation matrix

− Using barnard’s dataset in Funt’s study

(3)

0 0

0 0

0 0

L L

Lstd ill

M M

std M ill

S S

std illS

Cq q

q C q

q qC

where q is canonical colors(q , , , ), q is casted colors

(q , , , ) and is chromatic ratio

i i i

std std ill

i

ill i

i L M S

i L M S C

0 1 1 2 2 3 3

1 1

0 2 2

3 3

( )

std ill

T

ill ill

Q M cV c V c V Q

C V

M Q C V Q

C V

(4)

0where is three bases matrices, is mean matrix, C is cofficientsi iV M

9/25

Hue-compensated diagonal model(HCDM)

– Transformation matrix 𝑇𝐻𝐶𝐷𝑀

• Weighting von Kries transformation

0 0

0 0

0 0

L L L

Lstd ill ill

M M M

std HCDM ill M ill

S S S

std ill illS

Cq q q

q T q W C q

q q qC

(5)

10/25

• Weighting matrix of compensation coefficients

− Based on complementary operator 𝑅𝑗(ℎ𝑢𝑒𝑖𝑙𝑙)

− Based on hue-chromaticity characteristic function N(x, y)

(6) 3 3

3 3

0 0

0 0

0 0

( , ) 0 0

=I ( ) 0 ( , ) 0 ,

0 0 ( , )

, , ( )

L

j

L

j

L

j

L

j

L

x j ill j

L

j

j ill x

W

W W

W

N x y

R hue N x y

N x y

R

j r b R hue

1

2

3

( ) 0 0

0 ( ) 0

0 0 ( )

j ill

j ill

j ill

hue

R hue

R hue

where ( ) denotes -th component of complementray operator ( )j ill i j ill iR hue i R hue

11/25

Determination of complementary operation

– Complementary operator 𝑅𝑗 ℎ𝑢𝑒𝑖𝑙𝑙 of HCDM

• Adjusted toward in color compensation

– Relative absorption model

• Definition Relationships between illuminant hues and LMS cone signals

− Achieved compensation by adjusting cone signals

• 𝑅𝑟 ℎ𝑢𝑒𝑖𝑙𝑙 = −1 − 1 + 1 and 𝑅𝑏 ℎ𝑢𝑒𝑖𝑙𝑙 = −1 − 1 + 1

Synthetic datasets

– Theoretical illuminant spectra

• Plank’s dataset

− Low correlated color temperature from 2400K to 6500K

− High correlated color temperature from 6500K to 16000K

– Synthetic data surface

• IT8.72 industrial colors

− Wavelength from 380 to 780

12/25

– Natural illuminants

• Using Barnard’s dataset for 81 illuminants

− Including reddish (r) and bluish (b)

− CCT from 2700K to 5500K

− 80 interpolated illuminants

Fig. 2. The chromatic distribution of Barnard’s illuminant and those that are interpolated.

13/25

Determination of hue-chromatic characteristic function

– Hue chromatic relationship

• High-order exponential function

• Normalized hue(𝑚hue)

− hue Value r(LCT) illuminant

− (hueext-hue) value for b(HCT) illuminant

hue ( , , ) *exp[ ( ) / ] ,

, , i=1...s

i k k

j i i j j j j jm H k x y a dxy b c

r b

(7)

where is reference illumination as ( ) and ( )

for and classes, is range of order parameter,

and is data potints, is scale parameter

i

j i std i stddxy x x y y

r b k

x y

14/25

– Hue chromatic characteristic function

• Purpose of color compensation in LMS signals

− HCDM weighting matrix

Learning model parameters

– Model parameters of HCDM

• Illuminants in Barnard’s dataset

2 2( , ) ( div )*[1 * exp( / )]

, , ,

i

j i i j jN x y xy dxy

i L M S j r b

(8)

where is color difference function, is canonical ones,

( , , ) is corrected colors, is th casted color,

is number of test colors

p

std

p p p

HCDM ill ill

A q

T x y q q p

1

ˆ arg min ( , ( , , )p p p

std HCDM ill

p

A q T x y q

(9)

where and is scaling parameters and weighting parameters

for each channel, div is deviation for each LMS signal

i i

xy

15/25

Extended HCDM

– Learning HCDM parameters

• Using more widely-ranged illuminants

− Correction of casted colors in real images

– Optimization HCDM based on all given illuminants

• Barnard’s dataset with all tested colors

– Extended HCDM to include residual-information terms

• Function to extend HCDM 2( , ) ( div )*[1 * {exp[ ( / ) ] ( )})

( ) *( )

, , ,

i

j i i j

std std

N x y xy dxy f dxy

f dxy x x y y

i L M S j r b

(10)

where ( ) is regression function, is optimal value of quantitative parameterf dxy

21 1

ˆ arg min [ , ( , , ) ]m

p p p

std HCDS ill

ill p

A q T x y q

(11)

16/25

(a) (b)

(c)

Fig. 3. Regression function and the residuals.

(a) The regression (𝑚hue versus chromatic x)

for parameter determination in exponential

term, for the LCT data. (b) The regression

(𝑚hue versus chromatic y)for the HCT data.

(c) The linear regression of the residuals in

HCT data, with respect to chromatic term. 17/25

Table 1. The scale parameter determination information

18/25

Experiments and results

Synthetic-data experiments

– Performance of HCDM, PCA, and von Kries methods

• Outperformed both PCA and von Kries methods

− Sorted Barnard’s illuminants

• Comparison of Average errors

− HCDM and von Kries method

• Comparison of improvement ratios

− HCDM and PCA method

19/25

Fig. 4. Comparison of correction error for two datasets. Top: correction errors for the hue-

compensated diagonal model (HCDM), the von Kries method, and the principle component analysis

(PCA) method for Barnard’s illuminant dataset. Middle: the error improvement ratio by the HCDM

with respect to the PCA method. Bottom: correction errors for the HCDM, the von Kries method, and

the PCA method for interpolated illuminants. Note that the illuminants are sorted from red to blue

chromaticity. 20/25

– Average results

Table 2. Average correction errors for the

three methods

21/25

Real-image experiments

– Calibration of sensor responses

• Calibration functions

− Obtaining by minimizing color differences

» Between calibrated colors and synthetically computed colors

• Obtaining as calibrated casted colors

− Input colors of HCDM

Experimental results

– Comparison of extended HCDM and von Kries methods

• Real image experiments

22/25

– Results of five selected conditions

Table 3. Mean and standard deviation of recovery errors for

the HCDM and von Kries methods (for Macbeth colors) for the

selected five capture conditions.

23/25

– Correction errors for von Kries approach and extended HCDM

• Comparison in real images

Fig. 5. Comparison of correction errors in real-image experiments. The first two rows show images

with a red filter with −2 exposure [chromatic coordinates (0.34, 0.345)] and with a blue filter with −1

2∕3 exposure [chromatic coordinates (0.305, 0.309)]. The other two rows show results for the

website test images. Note that the illumination conditions of these images were denoted by

correlated color temperature (CCT) values. For example, the “solux 4100 + 4700” means the

illumination which is the mix of the 4100 K and 4700 K lights. 24/25

Conclusions

New color correction method

– Illuminant-adaptive weighting terms

• Synthetic illuminant data and real images

Proposed HCDM

– Complementary illuminant-hue information

– Weighting diagonal terms with illuminant hue

– Compensation of color distortion

25/25

Fig. 6. Simulation of the relative absorption model (RAM):

The numerical simulation for 1000 sequential values of

hues and their corresponding cone signals.