Challenges for Color Science inMultimedia Imaging

Roy S. BernsMunsell Color Science Laboratory, Chester F. Carlson Center for Imaging Science

Rochester Institute of Technology, 54 Lomb Memorial Drive,Rochester, NY 14623-5604 USA, berns@cis.rit.edu

Color science encompasses many technologies, one of which is multimedia. Looking at thebreadth of activities and industries that define color science can provide insight into currentand future problems for multimedia imaging and their solutions.

1. Introduction

Multimedia imaging requires the fundamental knowledge that constitutes such disciplines asmathematics, physics, chemistry, physiology, and psychology. We further require appliedknowledge that constitutes disciplines including computer science, information processing,imaging science, and color science.

This view of multimedia imaging places color science in a subordinate position. However, itcan be worthwhile to exchange the positions. That is, multimedia imaging is but one of manytechnologies that exemplify color science. This approach has been described previously bythe author regarding educational imperatives (1) and as a pedantic vehicle for describing therelationships between user controls and resulting color for various color technologies (2).This approach will be revisited within the context of multimedia imaging.

2. Color Matching

As defined by the CIE system (3), two stimuli match when their tristimulus values are equal,i.e., a metameric match. Metameric matches are ubiquitous with multimedia imaging. CRTdisplays, perhaps the most prevalent multimedia device, always result in metameric matches,shown in Figure 1.

7006005004000

20

40

60

80

100

Wavelength (nm)

Rel

ativ

e R

adia

nt P

ower

Color CRT

Caucasian Face in Daylight

Figure 1. Metameric match between CRT display and human skin illuminated by typicaldaylight.

If the stimuli are materials, the metameric match is usually depicted as a pair of spectralreflectance factor curves. The match is defined for a particular light source, observer, andviewing condition. Figure 2 illustrates an example of such a match (ÒStandardÓ andÒMetameric MatchÓ). Also shown in Figure 2 is a sample that is not a match; its tristimulusvalues are different from the standard (ÒSimple Color DifferenceÓ).

7006005004000.0

0.1

0.2

0.3

0.4

Standard

Metameric Match

Simple Color Difference

Wavelength (nm)

Re

fle

cta

nc

e

fac

tor

Figure 2. Stimuli designed to match x=0.27, y=0.33 and Y/Yn=0.25 for D65 and the 10°standard observer.

The metameric matches shown in Figures 1 and 2 are similar in their spectral dissimilarities.At the long wavelength region of the visible spectrum, each metameric pair divergessignificantly from one another. Accordingly, can we state that they have similar quality?Chances are that they have a similar degree of metamerism if one calculates general orspecial indices of metamerism (3,4).

Yet there is a large difference in quality. The CRTÕs primaries are fixed. CRT developers andusers have minimal if any flexibility in choosing the set of primaries. Metamerism cannot beavoided. The influence of metameric matches in multimedia imaging on color quality wasrecently explored by Alfvin and Fairchild (5). The variance in match equality due to observerdifferences was large, resulting in a dramatic reduction in quality.

In the case of materials, users often have tremendous flexibility in selecting colorants. Thespectral curve shown in Figure 2 labeled ÒSimple Color DifferenceÓ represents a samplewhere the same colorants were used as those chosen to produce the ÒStandardÓ. Because ofmanufacturing limitations, there is still a small residual color difference. Which sample willresult in higher quality, the ÒMetameric MatchÓ or the ÒSimple Color DifferenceÓ? Clearlythe latter is a better quality match. ItÕs small amount of color difference will be constantacross changes in illumination, viewing, and observers. The metameric match will result inlarge differences in color with similar changes. Thus the material-based metameric pair hasmuch lower quality than the CRT-based metameric pair. This is well known within the worldof manufacturing colored materials. Only spectral matches result in high quality. In fact,metameric matches have been defined as Òconditional matchesÓ to point out theselimitations (6).

3. Spectral Matching in Multimedia

As shown above, it is not possible to achieve a spectral match between objects and theirÒsoftcopyÓ CRT representations. However, in multimedia, there are tremendous possibilities

in achieving spectral matches between original objects and their printed reproductions. Threesubsystems are required: multispectral image estimation, ink selection minimizingmetamerism, and spectral-based printing models including separation algorithms. This isshown in Figure 3. In addition to these subsystems, trichromatic-based multimedia imagingdevices including CRT displays and desktop printers can be easily incorporated. Essentially,trichromatic-based systems are a subset of multichannel-based systems.

There is an obvious parallel to the world of color matching of materials. In the United States,many stores that sell house paint have color matching systems. These consist of aspectrophotometer, computer, and paint dispensing system. Most manufacturers includingtextiles, plastics, and coatings use similar systems. A standardÕs spectral reflectance factor ismeasured using the spectrophotometer. From a data base of colorants appropriate to theparticular coloration process, a subset is selected that when used to color the particularmaterial system (e.g. paint or fabric), a match will result that is minimally metameric. Thesoftware algorithms used in these systems can be divided into two steps. The first is colorantselection. From a large database of colorants, the three or four colorant combination (themost common number of colorants used in a formulation) is selected that results in theclosest spectral match (i.e. least metameric). Once the colorants are determined, theiramounts, the recipe, is defined based on a color mixing model such as Kubelka-Munk turbidmedia theory. (Reference 2 lists references on the coloration of materials.) If an exactspectral match will result, one can combine these steps. For example, forward-selectionmultiple linear regression (7) can be used to order the candidate colorants in terms of theirspectral matching potential. However, if an exact spectral match is not possible, there will beresidual colorimetric error; accordingly, the spectral matching algorithm might be used forcolorant selection followed by a colorimetric matching algorithm to insure a close match fora primary illuminant. Alternatively, iterative methods are used where every three or fourcolorant combination is evaluated.

The multispectral image capture is equivalent to the spectrophotometer. A set of candidateinks is equivalent to the database of appropriate colorants. The ink selection algorithm isequivalent to colorant selection minimizing metamerism. The spectral printing model andseparation algorithm is equivalent to determining the final color recipe.

Multispectral digital image capture

Multi-channel image storage

Colorimetric and appearance conversion

Gamut mapping

CRT device colorimetric

characterization (inverse)

Desktop printer device colorimetric characterization

(inverse)

Spectral-based printing separation

minimizing metamerism

Multi-ink direct digital printing

Copyright © 1993, The National Gallery, LondonCopyright © 1993, The National Gallery, London

Spectral reconstruction

Figure 3. Multispectral-based multimedia flowchart. (Van Gogh image copyright ofNational Gallery, London.)4. Image Capture Requirements for Multimedia

Traditional color science would argue that the requirements for building input devices arestraightforward. Trichromatic systems should have spectral responsivities that are linearlytransformable from color matching functions (sometimes called the Luther condition).Multispectral systems should sample the visible spectrum at constant bandpass andwavelength interval, preferably 10 nm. Interestingly, the CIE does not publish generalaccuracy requirements for instruments used for color measurement. A recommended metricfor photometers exists (8), but not for colorimeters although a CIE technical committee hasexisted for some time (2-16 Characterization of the Performance of TristimulusColorimeters). Spectroradiometric accuracy requirements have been defined when measuringlight sources (9) and CRT displays (10). Spectrophotometer accuracy requirements have notbeen defined. A CIE technical committee will soon be publishing techniques, but notrecommended specifications (2-28 Methods of Characterizing Spectrophotometers).According to CIE Publication 15.2, tristimulus errors will not be introduced when measuringmaterials if a 5 nm wavelength increment and bandpass are used. This would translate to atleast 60 channels. Obviously, it is necessary to reduce the number of channels.

Considerable research has been performed in determining the minimum number of channels,their spectral response, and how the multichannel information is used for spectral estimation(11-26). Issues include colorimetric accuracy, spectral accuracy, and noise propagation. Forexample, research at the Munsell Color Science Laboratory (11 - 14) has focused on usingseven readily available filters from Melles Griot in conjunction with a typical monochromedigital camera. The system spectral sensitivities are plotted in Figure 4. The particularmethod of spectral estimation results in large differences on spectral reconstruction accuracy

as shown in Figure 5. The number of available bits also has a large affect on accuracy, asshown in Figure 6. ItÕs clear that this is an active area of research.

0

0.25

0.5

0.75

1

Rel

ativ

e se

nsit

ivit

y

400 500 600 700

Wavelength, nm

Figure 4. The spectral sensitivity of each of the seven filter/sensor channels (14).

0

0.1

0.2

0.3

0.4

0.5

Spec

tral

refl

ecta

nce

400 500 600 700

Wavelength, nm

Camera signalsCubic splineMDST

PCAMeasurement

Figure 5. The spectral reconstruction using principal component analysis (PCA), modified-discrete-sine-transformation (MDST) and cubic spline interpolation, from camera signalvalues, for the Cyan sample from the Macbeth Color Checker chart (14).

20 40 60 80 100

L*

0

1

2

3

4

5

6

Average ÆE*

1210 8bitsAverage

∆E*

Figure 6. Average CIELAB color difference that results from the uniform quantization oftristimulus values at 8, 10, and 12 bit encoding for neutral colors (11).

5. Image Modeling Requirements for Multimedia

Reducing metamerism between objects and their printed reproductions implies spectralmatching. The color of the output device is defined by its spectral reflectance factor ratherthan colorimetric coordinates. In spectral-based research, it is more common to developmodels rather than build mxn dimensional look-up tables where m counts the number ofmeasured samples and n counts the number of wavelengths. Representative models aredescribed in references 27 - 36.

As an example Iino and Berns (31, 33) used the Yule-Nielsen modified Neugebauer equationsto model process printing, shown in Eq. (1) where ai are the effective dot areas of the 16Neugebauer primaries for four-color printing, described in the usual manner by the Demichelequations, Ri are the reflectance factors of each ith Neugebauer primary, c, m, y and k, are dotareas of each primary ink, and nλ is the Yule-Nielsen n value defined as a function ofwavelength. Typical spectral model fits are shown in Figure 7.

R a R a R a R a R

a R a R a R

a R a R a R

a R

c cn

m mn

y yn

k kn

r rn

g gn

b bn

ck ckn

mk mkn

yk ykn

rk

λ λ λ λ λ

λ λ λ

λ λ λ

λ λ λ λ

λ λ λ

λ λ λ

= + + +

+ + +

+ + +

+

( ,/

,/

,/

,/

,/

,/

,/

,/

,/

,/

1 1 1 1

1 1 1

1 1 1

λλ λ λ

λ λ λ

λ λ λ

λ λ λ λ

,/

,/

,/

,/

,/

,/ ) ,

rkn

gk gkn

bk bkn

cmy cmyn

cmyk cmykn

w wn n

a R a R

a R a R a

1 1 1

1 1 1

+ +

+ + +

(1)

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

380 420 460 500 540 580 620 660 700

Wavelength (nm)

Re

fle

cta

nc

e

Fa

cto

r

Figure. 7 Measured spectral reflectance factor (dashed line) and predicted spectralreflectance factor (solid line) of the magenta ramp based on Eq. (1).

One difficulty often encountered in using the Demichel equations to predict the entire colorgamut is a discrepancy between predicted and actual area coverage caused by opticalinteractions. Recognizing that the Yule-Nielsen modification is empirical, this can beremoved in favor of increasing the number of Neugebauer primaries. This approach is morein line with the true optics of ink paper interactions. The number of primaries can beincreased to i2 (34,35) where i counts the number of primaries or to infinity (35,36).Alternatively, the empirical equation is used with an augmentation where the opticalinteractions are accounted for in the conversion between theoretical and actual areacoverages (31,33,37). Iino and Berns described this using Eq. (2). Their colorimetricperformance was, on average, ∆E*ab of 2.2 with a maximum of 5 for independent colorssampling a printer color gamut represented by Matchprint III..

q f d f d f d

q f d f d f d

q f d f d f d

q f d f

c c m t m c y t y c k t k

m m c t c m y t y m k t k

y y c t c y m t m y k t k

k k c t c k m

=

=

=

=

_ , _ , _ ,

_ , _ , _ ,

_ , _ , _ ,

_ , _

( ) ( ) ( )

( ) ( ) ( )

( ) ( ) ( )

( ) (dd f dt m k y t y, _ ,) ( )

(2)

where qi is coefficient q for the overlapped ink i, the function fi_j(dt,j) is the decreasing

effective dot gain function of the secondary color (overlapped ink i by overlapping ink j),and dt,j is the theoretical dot area of each overlapping primary color.

6. Multi-Ink Requirements for Multimedia

In order to fully develop a multimedia system that is analogous to the color matching ofpaints or textiles, a large database of inks is required. The purpose of the database is t oprovide sufficient spectral variability, such that one can ÒtuneÓ a spectral reflectance factor,thereby matching the spectral properties of an object requiring color reproduction. Thisrequirement is different from typical multi-ink systems, largely concerned with increasingcolor gamut (37-46). Published research where minimizing metamerism is the goal ratherthan increasing color gamut has been limited to preliminary research by the author (47). Atthe Munsell Color Science Laboratory, this is an active area of research.

As an example, Figure 8 is a digital representation of an acrylic painting. Using a small-aperture spectrophotometer, the spectral reflectance factor of 100 positions was measured,plotted in Figure 9. Transforming the spectral data to absorption and scattering ratios usingKubelka-Munk turbid media theory, performing principal component analysis, and rotatingthe significant characteristic vectors to an all-positive representation, a statistical set ofpigments results, shown in Figure 10 (48). This statistical set of pigments will predict the100 measurements to an average ∆E*

94 of 0.8 and a maximum of 1.8. The degree ofmetamerism (49), expressed in CIE94 units, is 0.1 on average with a maximum of 0.3. Oncea printing system is defined and spectrally modeled, the optimal ink set can be determinedthat minimizes metamerism compared with the statistical pigment set.

Figure 8. Digital representation of an acrylic painting, by M. Gottsegen, 1989 (Kodak DC20digital camera).

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

39

0

41

0

43

0

45

0

47

0

49

0

51

0

53

0

55

0

57

0

59

0

61

0

63

0

65

0

67

0

69

0

71

0

73

0

Wavelength (nm)

Re

fle

cta

nc

e

fac

tor

Figure 9. Measured spectral reflectance factors of 10x10 grid sampling painting.

7006005004000.0

0.2

0.4

0.6

0.8

1.0

1.2

Wavelength (nm)

Re

lati

ve

A

bs

orp

tio

n/S

ca

tte

rin

g

Figure 10. Statistical colorants representing painting shown in Figure 8.

7. Defining Quality for Multimedia

Once the spectral-based multimedia system has been developed, quality metrics are required.These should include CIE94 (50), special indices of metamerism (1,4,49,51) indices of colorinconstancy (51), and indices that include the human visual systemÕs spatial properties(52,53).

8. Conclusions

Evaluating multimedia imaging as one of many applications of color science can result in anew paradigm for color imaging, multispectral-based imaging. However, developingmultichannel multimedia systems will likely cause a dramatic increase in cost (hardware,software, personnel training, data storage, image access, etc.). If only an increase in coloraccuracy compared with the current state of multimedia imaging was the benefit, a cost-benefit analysis would probably not result in a favorable outcome. Fortunately, there aremany additional benefits that result from spectral data bases and color printing thatminimizes metamerism. Perhaps most important is the potential to define and therebyarchive objects using the most fundamental definition, spectral reflectance factor. As ananalogy, the U.S.Õs National Institute of Standards and Technology only provides spectraldefinitions for their standard reference materials used for spectrophotometers. Matchingspectra minimize the need for controlling lighting, viewing conditions, and observers. Thebenefits we take for granted when having our automobile refinished after a collision or whenwe purchase clothing can be applied to multimedia.

9. References

1. R.S. Berns, Color science education in the 1990's, Color Res. Appl. 19, 74-76 (1994).2. R.S. Berns, A generic approach to color modeling, Color Res. Appl. 22, 318-325

(1997).3. Commission Internationale de l'E«clairage (CIE), Colorimetry, 2nd ed., Publ. CIE No.

15-2, Bureau Central de la CIE, Austria, 1986.

4. Commission Internationale de l'E«clairage (CIE), Special Metamerism Index: Changein Observer, Publ CIE No. 80, Bureau Central de la CIE, Austria, 1989.

5. R.L Alfvin and M.D. Fairchild, Observer variability in metameric color matches usingcolor reproduction media, Color Res. Appl. 22, 174-188 (1997).

6. F.W. Billmeyer, Jr. and M. Saltzman, Principles of Color Technology, 2nd ed., JohnWiley & Sons, New York, 1981.

7. N.R. Draper and H. Smith, Applied Regression Analysis, 2nd. ed., John Wiley & Sons,New York, 1981.

8. Commission Internationale de l'E«clairage (CIE), Methods of characterizingilluminance metters and luminance meters. Publication CIE #69, Bureau Central de laCIE, 1987.

9. Commission Internationale de l'E«clairage (CIE), The spectroradiometricmeasurement of light sources. Publication CIE #63, Bureau Central de la CIE, 1984.

10. Commission Internationale de l'E«clairage (CIE), The Relationship Between DigitalAnd Colorimetric Data For Computer-Controlled CRT Displays, CIE #122, BureauCentral de la CIE, Austria 1996.

11. P.D. Burns, Analysis of Image Noise in Multispectral Color Acquisition, Ph.D.Dissertation, Rochester Institute of Technology, 1997.

12. P.D. Burns and R.S. Berns, Error propagation in color signal transformations, ColorRes. Appl. 22, 280-289 (1997).

13. P.D. Burns and R.S. Berns, Modeling colorimetric error in electronic imageacquisition, Proceedings OSA annual meeting, in press (1997).

14. P.D. Burns and R.S. Berns, Analysis of multispectral image capture, Proc. 4thIS&T/SID Color Imaging Conference, 19-22 (1996).

15. M. L. Simpson and J. F. Jansen, Imaging Colorimetry: A New Approach, Appl. Opt.,30, 4666-4671 (1991).

16. K. Martinez, J. Cupitt and D. Saunders, High Resolution Colorimetric Imaging ofPaintings, Proc. SPIE 1901, 25-36 (1993).

17. T. Jaaskelainen, J. Parkkinen and S. Toyooka, Vector-subspace Model for ColorRepresentation, J. Opt. Soc. Am. A 7, 725-730 (1990).

18. L. T. Maloney, Evaluation Of Linear Models of Surface Spectral Reflectance WithSmall Numbers of Parameters, J. Opt. Soc. Am. A 10, 1673-1683 (1986).

19. M. J. Vrhel, R. Gershon and L. S. Iwan, Measurement and Analysis of ObjectReflectance Spectra, Color Res. and Appl. 19, 4-9 (1994).

20. K. Engelhardt and P. Seitz, Optimum Color Filters for CCD Digital Cameras, Appl.Opt. 32, 3015-3023 (1993).

21. M. J. Vrhel and H. J. Trussel, Color Correction using Principal Components, ColorRes. and Appl. 17, 328-338 (1992).

22. H. J. Trussell, A Review of Sampling Effects in the Processing of Color Signals, Proc.2nd IS&T/SID Color Imaging Conf., 26-29 (1994).

23. T. Keusen, Optimierta Auswertung Multispektraler Abtastsignale, DfwG-Tagung,Illmenau (1994).

24. T. Keusen and W. Praefcke, Multispectral Color System with an Encoding FormatCompatible to Conventional Tristimulus Model, Proc. IS&T/SID 3d Color ImagingConf., 112-114 (1995).

25. H. Maître, F. Schmitt, J Crettez, Y. Wu, and J.Y. Herdeberg, Spectrophotometricimage analysis of fine art paintings, Proc.s 4th IS&T/SID Color Imaging Conf., 50-53 (1996).

26. A. Abrardo, V. Cappelline, M. Cappelline, and A. Mecocci, Art-works colourcalibratrion using the VASARI scanner, Proc. 4th IS&T/SID Color Imaging Conf.,94-97 (1996).

27. H.E.J.Neugebauer, Die theoretischen grundlagen des mehrfarbendrucks, Zeitscrift furwissenschaftliche Photographie 36, 73-89 (1937) [Reprinted in Proc. SPIE 1184:Neugebauer Memorial Seminar on Color Reproduction, 194-202 (1989).]

28. J.A.S.Viggiano, The color of halftone tints, TAGA Proceedings 37, 647-661 (1985).29. H. Kang, Color Technology for Electronic Imaging Devices, SPIE Press, Bellingham,

Washington (1996).30. R.Rolleston and R.Balasubramanian, Accuracy of various types of Neugebauer model,

Proc. 3rd IS&T and SID Color Imaging Conf., 32-37 (1993).31. K. Iino and R.S. Berns, A spectral based model of color printing that compensates for

optical interactions of multiple inks, AIC Color 97, Proc. 8th Congress InternationalColour Association, 610-613 (1997).

32. K. Iino and R.S. Berns, Building color management modules using linearoptimization I. desktop color system, J. Imag. Sci. Tech., in press (1997).

33. K. Iino and R.S. Berns, Building color management modules using linearoptimization II. prepress system for offset printing, J. Imag. Sci. Tech., in press(1997).

34. J.S. Arney, P.G. Engeldrum, H. Zeng, An expanded Murray-Davies model of tonereproduction in halftone imaging, J. Im. Sci. Tech. 39, 502-508 (1995).

35. J.S. Arney, T. We, and C. Blehm, Modeling the Yule-Nielsen effect on color halftones,Proc. 5th IS&T and SID Color Imaging Conf., 62-65 (1997).

35. S. Gustavson, M.Wedin, B.Kruse, 3D modeling of light diffusion in paper, TAGAProceedings 47, 848-855 (1995).

36. S. Gustavson, Dot Gain in Color Halftones, PhD. dissertation, Link�ping University(1997).

37. J.P. Van De Capelle and B. Meireson, A new method for characterizing output devicesand its fit into ICC and HIFI color workflow, Proc. 5th IS&T and SID Color ImagingConf., 66-69 (1997).

38. D. Carli and M. Davis, High fidelity color rendering and reproduction, TAGA Proc., 8-19 (1991).

39. V. Ostromoukhov, Chromaticity gamut enhancement by heptatone multi-colorprinting, device independent color imaging and imaging system integration, Proc.SPIE, Vol. 1909, 139-151 (1993).

40. R. Herbert, Hexachrome color selection and separation - model for print media, IS&TThird Technical Symposium on Prepress Proofing and Printing, 28-30 (1993).

41. H. Boll, A color to colorant transformation for a seven ink process, IS&T ThirdTechnical Symposium on Prepress Proofing and Printing, 31-36 (1993).

42. L.W. MacDonald, J.M. Deane, and D.N. Rughani, Extending the colour gamut ofprinted images, J. Phot. Sci. 42, (1994).

43.. A. Di Bernardo and C. Matarazzo, Hi-Fi Color, Amer. Ink Maker, (April, 1995).

44. S. Herron, An exploration of the Pantone hexachrome six-color system reproductedby stochastic screens, Proc. 4th IS&T and SID Color Imaging Conf., 114-120 (1996).

45. E.M. Granger, Press controls for extra-trinary printing, Proc. SPIE, Vol. 2658, 147-150 (1996).

46. M. Mahy and D. DeBaer, HIFI color printing within a color management system,Proc. 5th IS&T and SID Color Imaging Conf., 277-283 (1997).

47. T. Kohler and R.S. Berns, Reducing metamerism and increasing gamut using five ormore colored inks, Proceedings of IS&T Third Technical Symposium on Prepress,Proofing and Printing, 24-28 (1993).

48. D.Y. Tzeng, personal communication, 1998.

49. H.S. Fairman, Metameric correction using parameric decomposition, Color Res. Appl.,12, 261-265 (1987).

50. Commission Internationale de l'E«clairage (CIE), Industrial color-differenceevaluation, CIE #1162, Bureau Central de la CIE, Austria 1995

51. R.S. Berns and F.W. Billmeyer, Jr., Proposed indices of metamerism with constantchromatic adaptation, Color Res. Appl. 8, 186-189 (1983).

52. X. Zhang and B.A. Wandell, A spatial extension to CIELAB for digital color imagereproduction, SID Symposium Tech. Digest 27, 731-734 (1996).

53. A. Poirson and B.A. Wandell, Appearance of colored patterns: pattern-colorseparability, J. Opt. Soc. Am. A 10, 2458-2470 (1993).