11/01/06Jean-François Lalonde Natural Color Statistics p. 1 Natural color statistics Jean-François...

11/01/06 Jean-François Lalonde

Natural Color Statistics

p. 1

Natural color statistics

Jean-François LalondeMisc-read, November 1st 2006



p. 2

Image formation & capture

k =F k k =∫ L R k d

L ,e,

e=∫−

∫0

/2f ,

i,

i,

e,

eE ,

i,

icos

isin

id

id

i

BRDF Irradiance

Sensor's response

Radiance

Radiance



p. 3

Color spaces RGB

Used in displays HSV

Separates luminance from chroma from “purity”

CIE L*a*b* Separates luminance

from chroma Close to perceptual

uniformity

R G

B



p. 4

Big Picture Is color really

important?



p. 5

Is color important? Colors contribute to recognition when

they are diagnostic of a scene category

[Oliva & Synchs, 2000] Diagnostic Colors Mediate Scene Recognition

Diagnostic Non-diagnostic



p. 6

Experiment 3 – phase 1



p. 7

Experiment 3 – phase 1



p. 8

Experiment #2 – phase 2



p. 9

Experiment #2 – phase 2



p. 10

Color is important! 3 Experiments

Colors contribute to recognition when they are diagnostic of a scene category

Faster verification of category membership of scenes when properly colored

Addition of colors to coarse luminance blobs enhance categorization

[Olivia & Torralba, 2006] Building the Gist of a Scene: The Role of Global Image Features in Recognition



p. 11

Big Picture – (bis)Is color really

important?

Can natural color

be compactly

represented?

[Oliva & Synchs, 2000]

[Oliva & Torralba, 2006]

YES!

important for scene

recognition



p. 12

3 channels? Human cones: L,M,S color receptors



p. 13

3 channels Low dimensionality of natural

reflectances PCA on spectral data over visible range 98% of energy can be represented using 3

components

[Chiao & Cronin, 2000]

Coral reef Forest



p. 14

Big Picture – revisited

Can natural color

be compactly

represented?


How??



YES!

important for scene

recognition

3 channels account

for 98% variability

Is color really

important?



p. 15

Mapping to know illuminant Color constancy: reduction of the effect of

the scene illumination Recover color under known illuminant

Not true object reflectance!

Estimate mapping from unknown known Linear model

Cknown

= A Cunknown



p. 16

Color constancy setupKnown (canonical) illuminant

Unknown scene

Unknown illuminant



p. 17

Classic paper: Gamut mapping Gamut: convex set

[Forsyth, 1990], [Barnard, 1998]

Known illuminant

Canonical gamut

Unknown illuminant

a

b

c

B

A

C

aA

aC

aB

Unknown gamutCanonical gamut



p. 18

Gamut mapping: transformations

cB

cA

cC

bA

bC

bB

aB

aC

aA

[Forsyth, 1990], [Barnard, 1998], [Finlayson, 1995]

Possible to do it in chromaticity space (2-D)



p. 19

G.D. Finlayson ~170 papers on color constancy

Somewhat incremental Color by Correlation

Colors in an image provide information about the illuminant

[Finlayson et al., 2001], [Schaefer et al., 2005]



p. 20

Color by Correlation

1. RGB chromaticity (no luminance)

2. Characterize illuminants Matrix M(c, l) = P(chromaticity c | light l)

3. Quantize input image

4. Compute correlation

5. Select best illuminant Max correlation

[Finlayson et al., 2001], [Schaefer et al., 2005]

Gamut mapping performs

similar to using

P(c|l) {0,1}



p. 21

Color constancy Major problem: require calibrated

illuminants! Alternative: color flows

How color commonly change together under natural illuminant variation

Allow non-linear transformations Data-driven

[Miller & Tieu, 2001]



p. 22

Color flows

Kernel densityestimator

Partially-observedcolor flow

Fullcolor flow

Slice of RGB cube



p. 23

Color eigenflows Subsample RGB cube Apply PCA, keep first k eigenflows First 3 eigenflows:

1

2

3

Original image [Miller & Tieu, 2001]



p. 24

Big Picture – yet again

Can natural color

be compactly

represented?


How??

[Barnard, 1998]

[Forsyth, 1990]

[Finlayson, 1995]

[Finlayson et al., 2001]


[Schaefer et al., 2005]



YES!

important for scene

recognition

3 channels account

for 98% variability

Mapping to illuminant

(color constancy)

Is color really

important?



p. 25

Color harmony Harmonic colors

Aesthetically pleasing in terms of human visual perception

Matsuda's regions

[Matsuda, 1995], [Tokumaru et al., 2002],[Cohen-Or et al., 2006]



p. 26

Color harmonization Find best-fitting template Squash all colors inside template (hue

only)

[Cohen-Or et al., 2006]



p. 27

Color harmonization


BeforeHarmonizing background

with foreground Does not change natural images!



p. 28

Color harmonization



p. 29

Big Picture – last time I promise

Can natural color

be compactly

represented?


How??

[Barnard, 1998]

[Forsyth, 1990]

[Finlayson, 1995]




[Matsuda, 1995]

[Tokumaru et al., 2002]




YES!

important for scene

recognition

3 channels account

for 98% variability


(color constancy)

Fixed hue intervals

(color harmony)

Is color really

important?



p. 30

Back to psychophysics Observation:

H,S,V variation retinal stimuli = highly non-linear!

Hypothesis: Retinal stimuli is more sensible to more likely

colors Probabilistic approach

[Long et al., 2006]



p. 31

Back to psychophysics

[Long et al., 2006]

1600 natural images



p. 32

Our neurons match the data!

[Long et al., 2006]



p. 33

I was just kidding, but now I’m really done

Can natural color

be compactly

represented?


How??

[Barnard, 1998]

[Forsyth, 1990]

[Finlayson, 1995]




[Matsuda, 1995]

[Tokumaru et al., 2002]


[Long et al., 2006]



YES!

important for scene

recognition

3 channels account

for 98% variability


(color constancy)

Fixed hue intervals

(color harmony)

Probabilistic?

Is color really

important?



p. 34

Conclusion What about spatial information?

See website... http://www.echalk.co.uk/amusements/

OpticalIllusions/colourPerception/colourPerception.html



p. 35

Thank you!

Your reward:

You’ve just read ~10 papers in 1 hour!



p. 36

References1. [Barnard, 1998] K. Barnard. Color constancy overview, 1998

2. [Chiao & Cronin, 2000] C.-C. Chiao and T. W. Cronin. Color signals in natural scenes: characteristics of reflectance spectra and effects of natural illumination. J. Opt. Soc. Am. A, 17(2), 2000.

3. [Cohen-Or et al., 2006] D. Cohen-Or, O. Sorkine, R. Gal, T. Leyvand, and Y.-Q. Xu. Color Harmonization. SIGGRAPH, 2006

4. [Finlayson, 1995] G.D. Finlayson. Coefficient color constancy. Ph.D. Thesis, Simon Fraser University, School of Computing, 1995

5. [Finlayson et al., 2001] G.D. Finlayson, S. Hordley and P. Hubel. Color by correlation: A simple, unifying framework for color constancy, PAMI 23(11) 2001

6. [Forsyth, 1990] D. A. Forsyth. A novel algorithm for color constancy. IJCV, 5(1):5-36, 1990

7. [Long et al., 2006] F. Long, Z. Yand, and D. Purves. Spectral statistics in natural scenes predict hue, saturation and brightness. Proc. Natl. Acad. Sci, 103(15), 2006

8. [Matsuda, 1995] Y. Matsuda. Color design, Asakura Shoten, 1995

9. [Miller & Tieu, 2001] E. Learned-Miller and K. Tieu. Color Eigenflows: Statistical Modeling of Joint Color Changes. ICCV, 2001

10. [Oliva & Synchs, 2000] A. Oliva and P.G. Schyns. Diagnostic colors mediate scene recognition. Cognitive Psychology, 41, 2000

11. [Oliva & Torralba, 2006] A. Oliva and A. Torralba. Building the Gist of a Scene: The Role of Global Image Features in Recognition. Progress in Brain Research: Visual perception, 155, 23-36, 2006

12. [Schaefer et al., 2005] G. Schaefer, S. Hordley, and G. Finlayson. A combined physical and statistical approach to colour constancy. CVPR, 2005

13. [Tokumaru et al., 2002] M. Tokumaru, N. Muranaka, and S. Imanishi. Color design support sustem considering color harmony. International Conference on Fuzzy Systems, 2002.

11/01/06Jean-François Lalonde Natural Color Statistics p. 1 Natural color statistics Jean-François...

Documents

Transcript of 11/01/06Jean-François Lalonde Natural Color Statistics p. 1 Natural color statistics Jean-François...