Color Harmonization For Videos

Color Harmonization for Videos

Nikhil Sawant and Niloy J. Mitra

Dept. of CSE, Indian Institute of Technology, Delhi

Color Harmonization ?

Color Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra

• Not all colors match• Colors can be ‘chaotic’ or ‘boring’• Overall impact can be disturbing• Notion of harmony depends on

place, time, context, etc.

Ultimate goal• Enhance visual harmony

Color Harmonization ?


• Videos are sets of images• Problematic due to:

– Rapidly changing scenes– Indoor and outdoor scenes– Changing illumination– Back to back un-harmonized scene

Ultimate goal• Enhance visual harmony of videos

Color Harmonization for Videos ?


Images

[Cohen-Or et al., Color Harmonization, SIGGRAPH, 2006]


• Templates for harmonization

• Need for optimization

Our approach


Template I Template V Template L Template I

Template T Template Y Template X Template N

[Johannes Itten. The Art of Color: the subjective experience and objective rationale of color. Van Nostrand Reinhold, 1973]

• Templates for harmonization

• Potential function:• P(θt, I) = [# image pixels in the shaded region]

• θot = argmax(P(θt, I)) θ є [0 – 360°]

• θH = argmax(P(θot, I)) T є {i, V, L, I, T, Y, X}

• Linear Color adjustment

Our approach


θt

θot


ResultTemplate X

ResultTemplate V


Segmentation problem• Splitting of color within same region


Template V

• Splitting of color within same region

• Solution - Find best position for splitting

Segmentation problem



• Less accurate but faster than traditional approaches

Nilkhil Sawant and Niloy J. MitraColor Harmonization for Videos

Videos


CH extended for videos• Videos are sets of images (frames)

• Naïve approach: each frame independently harmonized


CH extended for videos• Videos are sets of images (frames)

• Naïve approach: each frame independently harmonized

• Flickering artifacts observed

• Noticeable change in color due to large variation in θ

• Need to correlate the adjacent frames

• Grouping of frames


Grouping


Grouping

• Mean color for group of frames

• Calculation of θg considering mean color• Frames under a group harmonized by θg

• No flickering within a group

• Flickering at the group junction


Overlapping


Overlapping

• Adjacent groups contain common frames• Similar approach to grouping• Common frames harmonized according mean θg

of two groups (smoother transition)• Flickering artifacts between group are reduced• Change of scene still cause flickering artifacts


Change of scene

• Change of camera

• Change of camera view port

• Change in illumination

• Larger change in colors

• We categorize all such frames as I-frames


Change of scene detection(I-frames)

• Calculating MSE for each frame

• MSE is measure of color change

• Threshold is selectedColor Harmonization for Videos Nilkhil Sawant and Niloy J. Mitra

I-frames + overlapping


• An I-frame causes start of new group

• Between two I-frames overlapping method used

I-frames + overlappingNaïve approach

Results


Original Video Harmonized Video

Two parameter variation• Slight variation in the template is allowed

• Allow template to adjust according to image hue histogram

• Harmonization with minimal color change


Template X

Two parameter variation


Two parameter variation

• Large deviation in template may affect the harmonization


Conclusion

• Faster technique for color harmonization of images

• Less reliable but faster technique for segmentation problem

• Introduction of harmonization technique to Videos

• Reduction in flickering artifacts

• Introduction of Two parameter variation method


Thank you

Original Video Harmonized Video

Results


Thank you


Color Harmonization For Videos

Documents

Transcript of Color Harmonization For Videos