Color Harmonization - Results

Federal University of Rio de Janeiro - UFRJ - Campus Cidade Universitária - Rio de Janeiro - Ilha do Fundão - COPPE/PESC/LCG

Ten Minute Speech :: Overview of Activities Developed in Disciplines and Guided Studies :: Laboratory Seminars and Meetings

Ten Minute SpeechAn Overview of Activities Developed in Disciplines and Guided Studies

Michel Alves dos SantosGraduate Program in Systems Engineering and ComputingGraduate Program in Systems Engineering and ComputingFederal University of Rio de Janeiro - UFRJ - COPPEFederal University of Rio de Janeiro - UFRJ - COPPE

Advisors: D.Sc. Ricardo Marroquim & Ph.D. Cláudio Esperança

{michel.mas, michel.santos.al}@gmail.com

April, 2014April, 2014«Color Harmonization Results»

Michel Alves: Laboratory of Computer Graphics/LCG Graduate Program in Systems Engineering and Computing



Introductionground object to the background, so that together they form a har-monic color set (see Figure 1). In general, our algorithm is usefulfor enhancing colors in images that are comprised of a collection ofparts originating from different sources and whose colors requireharmonization.

2 Background and Related Work

The study of color harmony is historically intertwined with thestudy of the physical nature of light and color. Early discover-ies in the theory of color harmony were made by such mastersas Newton, Goethe, Young, and Maxwell. Modern color theory,which was developed at the beginning of the 20th century, dealsmainly with representations of colors, but it also discusses colorharmony [Munsell 1969; Ostwald and Birren 1969; Itten 1960].Moon and Spencer [1944] introduced a quantitative representa-tion of color harmony based on the Munsell color system [Munsell1969]. At the same time, Granville and Jacobson [1944] presented aquantitative representation of color harmony based on the Ostwaldcolor system [Ostwald and Birren 1969]. To a large degree, theseworks define harmony as order.

Itten [1960] introduced a new kind of color wheel in which he de-scribed color harmony, with an emphasis on hue. Itten’s color har-mony theory is based on the relative positions of the hues on thecolor wheel. For example, from the three primary colors of cyan,magenta, and yellow, Itten designed a hue wheel of twelve colors.He referred to complementary colors as a two-color harmony. Ittenalso recognized the three-color harmony of hues that form an equi-lateral triangle, the four-color harmony of hues forming a square,the six-color harmony of a hexagon, etc. His schemes have beenwidely adopted by artists and designers. Based on Itten’s schemesand extensive psychophysical research, Matsuda [1995] introduceda set of 80 color schemes, defined by combining several types ofhue and tone distributions. These schemes were used in [Tokumaruet al. 2002] for harmony evaluation and color design. Our colorharmonization method is also based on these schemes.

There are various interactive tools that provide designers with har-monic sets (e.g., [Color Schemer 2000; Color Wheel Expert 2000;Nack et al. 2003]). Such applications provide the user with a set ofharmonic colors that accommodates the user’s requirements spec-ified by a color seed and possibly a number of other parameters.Meier et al. [1988] presented a system for designing colors basedon several color rules, and applied them to a graphical user inter-face (GUI) building tool. The primary goal of their system was totest whether an automated mechanism would be a viable solution tothe problem of choosing effective and tasteful colors. None of theabove systems offers a means to harmonize a given arbitrary colorimage. The method we introduce in this paper automatically har-monizes a given color palette through an optimization process, andprovides a means to automatically recolor an arbitrary image.

Our work is also related to general recoloring methods [Reinhardet al. 2001; Welsh et al. 2002; Levin et al. 2004; Gooch et al.2005; Ironi et al. 2005; Rasche et al. 2005]. Automatic recoloringtechniques require the user to provide a reference image. The rela-tionship between the colors of the input and the reference imagesare learned and transferred to recolor the given image. One of thechallenges in these techniques is to recolor the image in a coherentway [Ironi et al. 2005]. In other words, contiguous spatial regionsin the input image should remain contiguous after the recoloring.Our color harmonization process uses a graph-cut optimization toenforce contiguous modification of colors in image space.

i type V type L type I type

T type Y type X type N type

Figure 2: Harmonic templates on the hue wheel. A collection ofcolors that fall into the gray areas is considered to be harmonic.The templates may be rotated by an arbitrary angle. The sizes ofthe sectors are specified in the Appendix.

3 Harmonic Schemes

The notion of color harmony in this work is based on the schemesdeveloped by Matsuda [Matsuda 1995; Tokumaru et al. 2002],which descend from Itten’s notions of harmony [Itten 1960], widelyaccepted in applicable fields involving colors. Figure 2 illustratesthe eight harmonic types defined over the hue channel of the HSVcolor wheel. Each type is a distribution of hue colors that definesa harmonic template: colors with hues that fall in the gray wedgesof the template are defined as harmonic according to this template.We refer to these distributions as templates, since they define theradial relationships on the color wheel, rather than specific colors(meaning that any template may be rotated by an arbitrary angle).The harmonic templates may consist of shades of the same col-ors (types i, V and T), possibly with complementary colors (seetemplates I, Y, X) or more complex combinations (template L andits mirror image). The sectors of these templates are the domainsover which simple membership functions are defined. Color har-mony is mainly affected by the hue channel; however, Tokumaru etal. [2002] also addressed tone distribution functions for the valuesof the S and V channels, and fuzzy rules for the correlation betweenthe hue templates and the tone distributions. For details, the readeris referred to [Tokumaru et al. 2002].

The type-N template corresponds to gray-scale images and thusis not dealt with in this work. Note that each of the remainingseven templates consists of one or two sectors. Each hue h onthe color wheel is then associated with one of these sectors. Thesimplest way is to associate h with the closest (in terms of arclength) sector. Thus, we define ETm(α)(p) as the sector borderhue of template Tm with orientation α that is closest to the hue ofpixel p (m ∈ {i, I,L,T,V,X ,Y}).

Given an image, we fit a harmonic template Tm to the hue his-togram of the image. We define a distance between the histogramand a template, and determine the template that best fits our im-age by solving an optimization problem. A template Tm togetherwith an associated orientation α defines a harmonic scheme, de-noted by (m,α). Given a harmonic scheme (m,α), we define afunction F(X ,(m,α)) which measures the harmony of an image Xwith respect to the scheme (m,α):

F(X ,(m,α)) = ∑p∈X

∥∥∥H(p)−ETm(α)(p)∥∥∥ ·S(p) , (1)

where H and S denote the hue and the saturation channels, respec-tively; the hue distance ‖ · ‖ refers to the arc-length distance on thehue wheel (measured in radians); hues that reside inside the sec-tors of Tm are considered to have zero distance from the template.

625

Saturation

HSV System

F(X, (m, α)) =∑p∈X‖H(p)− ETm(α)(p)‖ · S(p)

H ′(p) = C(p) +w2(1− Gσ(‖H(p)− C(p)‖))


[Color Harmonization, Sorkine et. al, 2006]



Simple Color Shift

The Birth of Venus: original clipping, hue shifted 30 ◦, and hue shifted 330 ◦.

Hnew(p) = Hold(p) + α


[Color Harmonization Project, Michel Alves, 2013]



Harmonization in Action

Original Template T, α = 35 ◦ Template V, α = 5 ◦

Original Template L, α = 40 ◦ Template Y, α = 320 ◦





Several Degrees of Freedom

Original Template i, 160 ◦ Template V, 5 ◦ Template L, 0 ◦

Template Y, 30 ◦ Template X, 90 ◦ Template T, 50 ◦ Template I, 145 ◦





Sophisticated Editing

Original Template V, 0 ◦ Template T, 0 ◦ Template T, 270 ◦





Preservation of Homogeneity

Original α = 80 ◦ α = 170 ◦ α = 270 ◦ α = 310 ◦

Template i





Excellent Tool for Designers

Original Template T, α = 250 ◦ Template V, α = 335 ◦

The exposed method can act as an aid tool foraesthetic and organization of environments, saving timeand effort of professionals in architecture and design!





ThanksThanks for your attention!

Michel Alves dos Santos - http://www.michelalves.com

Michel Alves dos Santos - (Alves, M.)Federal University of Rio de JaneiroE-mail: [email protected], [email protected]ésumé: http://lattes.cnpq.br/7295977425362370Personal Page: http://www.michelalves.com

http://www.facebook.com/michel.alves.santos

http://www.linkedin.com/profile/view?id=26542507


http://www.michelalves.com

http://www.ufrj.br

mailto:[email protected]

mailto:[email protected]

http://lattes.cnpq.br/7295977425362370

http://www.michelalves.com



BibliographyY. Baveye, F. Urban, C. Chamaret, V. Demoulin, and P. Hellier.Saliency-guided consistent color harmonization.In Computational Color Imaging Workshop - Chiba, Japan, pages 105–118, 2013.

D. Cohen-Or, O. Sorkine, R. Gal, T. Leyvand, and Y.-Q. Xu.Color harmonization.ACM Transactions on Graphics, 25(3):624–630, 2006.

W. C. Granville.Color harmony: What is it?Color Research & Application, 12(4):196–201, 1987.

Y. Matsuda.Color Design.Asakura Shoten (in Japanese), Tokio, Japan, 1995.

M. Tokumaru, N. Muranaka, and S. Imanishi.Color design support system considering color harmony.In Fuzzy Systems, 2002. FUZZ-IEEE’02. Proceedings of the 2002 IEEEInternational Conference on, volume 1, pages 378–383, 2002.


Color Harmonization - Results

Education

Transcript of Color Harmonization - Results