Probabilistic Color-by-Numbers: Suggesting Pattern Colorizations Using Factor Graphs
65
Probabilistic Color-by- Numbers: Suggesting Pattern Colorizations Using Factor Graphs Sharon Lin, Daniel Ritchie, Matthew Fisher, Pat Hanrahan
description
Probabilistic Color-by-Numbers: Suggesting Pattern Colorizations Using Factor Graphs. Sharon Lin, Daniel Ritchie, Matthew Fisher, Pat Hanrahan. Colored Patterns Are Everywhere. Flickr : Rowena of the Rants. Coloring Patterns Can Be Challenging. Hard to mentally visualize coloring. - PowerPoint PPT Presentation
Transcript of Probabilistic Color-by-Numbers: Suggesting Pattern Colorizations Using Factor Graphs
Probabilistic Color-by-Numbers: Suggesting Pattern Colorizations Using Factor GraphsSharon Lin, Daniel Ritchie, Matthew Fisher, Pat Hanrahan
Colored Patterns Are Everywhere
Flickr: Rowena of the Rants
Coloring Patterns Can Be ChallengingHard to mentally visualize coloring
Template by COLOURLover Any Palacios
Coloring Patterns Can Be ChallengingDifficult to explore other options
Such as:
Output Suggested Colorings
Suggest Pattern Colorizations to Facilitate the Process
?
User preferences
Input Template
Output Suggested Colorings
Suggest Pattern Colorizations to Facilitate the Process
?
User preferences
Input Template
Suggest diverse colorings
Allow refinement
Accommodate stylistic
preferences
Pattern Template AnatomyColor GroupsColored Template
COLOURLovers Nickity Split & ivy21
Related Work: Color CompatibilityWhat combinations of colors do people find appealing?
(Goethe 1810; Itten 1974; Matsuda 1995; Cohen-Or et Al 2006)
Related Work: Color CompatibilityWhat combinations of colors do people find appealing?
Low compatibility
High compatibility(O’Donovan et al.
2011)
Color Compatibility for PatternsNeed to take into account 2D arrangement
3.75 3.74 3.70 3.67
Template by COLOURLover jilbert
“loud” backgroundleaves blending into background
What about personal preferences?
Look at Examples for Guidance
COLOURLovers AlineDam, Any Palacios, wondercake, bhsav
Example-Based Color Suggestion
Model Suggester
(optional) user constraints
Input Template
Output Suggested Colorings
Examples
COLOURLovers AlineDam, Any Palacios, wondercake, bhsav
…
Can Change Style Based on Examples
Model Suggester
(optional) user constraints
Input Template
Output Suggested Colorings
Examples…
COLOURLovers AlineDam, Any Palacios, praxicalidocious, bhsav
Dataset: COLOURLoversMany patterns available:
Collected 8200 from 82 artists
For our tests: Trained on up to 913 patterns
MODEL
Scoring a Coloring
Unary Factors
Scoring a Coloring
Good
Scoring a Coloring
Poor
??
?
Scoring a Coloring
Scoring a Coloring
Scoring a Coloring
Scoring a Coloring
Pairwise
Factors
Scoring a Coloring
Scoring a Coloring
Color Theme:
Global Color Compatibilit
y[O’Donovan et al. 2011]
Scoring a Coloring
Color Theme:
Global Color Compatibilit
y[O’Donovan et al. 2011]
Scoring a Coloring
Color Theme:
Scoring a Coloring
Modeling Unary Color Factors
Property of
Region’s Color
Features of
Region’s Shape
…
LightnessSaturationName Saliency [Heer & Stone 2012]
Modeling Unary Color Factors
Property of
Region’s Color
Features of
Region’s Shape
…SizeElongationCentrality
Learning Factor Distributions
Predictor
0.20.70.4
=Size
ElongationCentrality …
0.8
0.92
LearnerSaturation
Distribution=
Learning Factor Distributions
0 1Lightness
Learning Factor Distributions
0 91 2 3 4 5 6 7 8
0.40.10.5
0.10.20.10.4
0.20.10.10.9
0.7
…
Lightness
Learning Factor Distributions
0 91 2 3 4 5 6 7 8
0.40.10.5
0.10.20.10.4
0.20.10.10.9
0.7
…
Lightness
Learning Factor Distributions
0 91 2 3 4 5 6 7 8
1
Lightness
0.40.10.5
0.20.10.4
0.20.10.10.9
0.7
…
…
Learning Factor Distributions
0 91 2 3 4 5 6 7 8
1
Lightness
0.40.10.5
0.20.10.4
0.20.10.10.9
0.7
Learning Factor Distributions
0 91 2 3 4 5 6 7 8
2
Lightness
…1
0.40.10.5
0.20.10.4
0.10.10.9
0.7
Learning Factor Distributions
0 91 2 3 4 5 6 7 8Lightness
…0.10.10.9
0.7210.40.10.5
0.20.10.4
Learning Factor Distributions
0 91 2 3 4 5 6 7 8
7
Lightness
0.10.10.9
210.40.10.5
0.20.10.4
…
Learning Factor Distributions
0 91 2 3 4 5 6 7 8
Classifier0.20.70.4
?
0
1
Lightness
7
…0.10.10.9
210.40.10.5
0.20.10.4
Learning Factor Distributions
0
1
0 1Lightness
Classifier0.20.70.4
?
70.10.10.9
210.40.10.5
0.20.10.4
…
Learning Factor Distributions
0
1
10 1Lightness
Classifier0.20.70.4
?
70.10.10.9
210.40.10.5
0.20.10.4
…
[Charpiat et al. 2008]
Example Learned Factors
Scoring a Coloring (Revisited)
Unary Factors
Pairwise
Factors
Global Color Compatibilit
y[O’Donovan et al. 2011]
Scoring a Coloring (Revisited)
Score = Product of Factors
(Factor Graph)
Generating Coloring SuggestionsMetropolis Hastings (MH)
Parallel Tempering
Maximum Marginal Relevance
REJECT
ACCEPT ACCEPT
RESULTS
Exploratory Suggestions
Refinement: Nearby Colorings
Refinement: Hard Constraints
Unconstrained
Flower Stem Color =
Style SimulationLight
Dark
Bold
Mellow
Application: Web Design
Application: Fashion Design
EVALUATION
4 x Uniform Random
4 x Color Compatibility
Only
4 x Full Model
4 x Hand-Colored
Better Than Other Automatic Methods(but not hand-colored patterns)
People Make ‘Bad’ Colorings Just as Often
FUTURE WORK
Limitation: Semantics
“sky”
Limitation: Known Color Groups
1 2 3 4 5
? ?
Integration into Interactive Tools
Looking Forward
THANKS!Support for this research provided by:
Intel (ISTC-VC)SAP (Stanford Graduate Fellowship)
