Nonlinear Laplacian Spectral Analysis of Rayleigh-Bénard ...
Pierre Bénard Ph.D. defense, 2011/07/07
-
Upload
pierre-benard -
Category
Education
-
view
639 -
download
1
Transcript of Pierre Bénard Ph.D. defense, 2011/07/07
Pierre Bénard Supervised by François Sillion & Joëlle Thollot
07/07/2011
• Expressive rendering
An alternative to photorealism
Tools for visual communication
• Interactive techniques
Direct user feedback
Dynamic visualization, video games…
[Herz98]
[GTDS04]
[CAS+97]
2
Okami Okami
Prince of Persia Prince of Persia
Team Forteress 2 Team Forteress 2
Jet Set Radio
3
Texture painting
McGuire et al., Stylized Rendering in Games, SIGGRAPH 2010 Course
4
McGuire et al., Stylized Rendering in Games, SIGGRAPH 2010 Course
Improved Sobel filter
5
McGuire et al., Stylized Rendering in Games, SIGGRAPH 2010 Course
Final compositing (textures + edges + fog)
6
McGuire et al., Stylized Rendering in Games, SIGGRAPH 2010 Course
Original Stylized
7
Temporal incoherencies
8
© Patrick Martin
Spatial Spatial
Attributes Attributes
Primitives Primitives Marks Marks 3D scene 2D image
perspective projection
2D lines and regions
color, shading, curvature…
pastel strokes
[Willats 97, Durand 02]
9
• Marks medium and pattern
Marks Marks
10
vs. Temporal continuity
« Il p
leut b
ergère », Jérém
y Dep
uyd
t (200
5)
11
Random changes Popping / Flickering
Static marks Shower-door effect
12
vs. Coherent Motion
Texture mapping 3D appearance
13
vs. Flatness
Motion coherence
Temporal continuity
Flatness
14
Conflicting goals compromise
Motion coherence
Temporal continuity
Flatness
Texture mapping
Static marks Random changes
15
Ideal solution
• REGIONS
Dynamic Solid Textures P. Bénard, A. Bousseau, J. Thollot, I3D 2009
NPR Gabor Noise
P. Bénard, A. Lagae, P. Vangorp, S. Lefebvre, G. Drettakis, J. Thollot, EGSR 2010
• LINES
Self-Similar Line Artmap P. Bénard, F. Cole, A. Golovinskiy, A. Finkelstein, NPAR 2010
Active Strokes work in progress
16
Temporally coherent bricks for stylized animations
Previous Work
Dynamic Solid Textures
NPR Gabor Noise
Perceptual Evaluation
Mapping Policies
Self Similar Line Artmap
Snakes-based Tracking and Parameterization
17
Previous Work
Dynamic Solid Textures
NPR Gabor Noise
Perceptual Evaluation
Mapping Policies
Self Similar Line Artmap
Active Strokes
19
• Distribution of marks
• Textures
[VBTS07] [VBTS07]
[CTP+03] [CTP+03]
20
• Distribution of marks
Few-marks
Many-marks
• Textures
[VBTS07] [VBTS07]
[CTP+03] [CTP+03]
21
Few-marks [Mei96,Dan99,HE04,
VBTS07,LSF10]
Motion coherence
Temporal continuity
Flatness
[Meie
r96]
3D distribution of anchor points
2D stroke texture
Painterly rendering
22
[VBTS07]
Good compromise: 2D strokes vs. 3D motion
Clutter / holes
Popping / flickering
[Meier96] [Meier96]
or
[LSF10] [LSF10]
[HE04] [HE04]
[Daniels99]
23
Many-marks [KC05,BKTS06]
24
Motion coherence
Temporal continuity
Flatness
Bridge the gap between texture and marks
Strong 2D appearance
Local motion and/or popping
Specific patterns (canvas fiber, watercolor pigments)
[KC05] [KC05]
[BKTS06]
25
• Distribution of marks
• Textures
In image space: local or global
In objet space
[VBTS07] [VBTS07]
[CTP+03] [CTP+03]
26
Local image space [BNTS07]
[BNTS07]
27
Motion coherence
Temporal continuity
Flatness
Strong 2D appearance
Regeneration artifacts
Post-production
[BNTS07]
28
Global image space [CTP+03,CDH06,
BSM+07]
+ +
[CTP+03]
29
Motion coherence
Temporal continuity
Flatness
Strong 2D appearance
Infinite zoom mechanism
3D motion approximated sliding
[CDH+06] [CDH+06]
[BSM+06] [BSM+06]
30
Object space [KLK+00,PHWF01,FMS01]
[PHWF01]
“Tonal Artmaps” 31
Motion coherence
Temporal continuity
Flatness
Exact motion
MIP-mapping
Perspective distortion
Limited styles [FMS01]
[KLK+00]
32
33
Local image space
Motion coherence
Temporal continuity
Flatness
Global image space
Motion coherence
Temporal continuity
Flatness
Object space
Motion coherence
Temporal continuity
Flatness
Few-marks
Motion coherence
Temporal continuity
Flatness
Many-marks
Motion coherence
Temporal continuity
Flatness
Previous Work
Dynamic Solid Textures
NPR Gabor Noise
Perceptual Evaluation
Mapping Policies
Self Similar Line Artmap
Active Strokes
Motion coherence
Temporal continuity
Flatness
Dynamic Solid Textures
34
• Real-time texture-based approach
• Accurate 3D motion object space approach
• Infinite zoom Dynamic Canvas [CTP+03]
• Easy parameterization solid textures
35
P. Bénard, A. Bousseau, J. Thollot, Dynamic Solid Textures for Real-Time Coherent Stylization. I3D 2009
36
• Texture “Fractalization”
1 solid texture : 4 octaves
octave 2 octave 1 octave 3 octave 4 weighted sum
freq. x2 freq. x2 freq. x2
37
• Zoom cycle
octave 2 octave 1 octave 3 octave 4 weighted sum
38
39
• One solid texture + 3D coordinates
• Procedural textures [Per85, Ola05]
or synthesized from 2D exemplars [KFCO+07]
• Ogre3D rendering engine
Additional cost: 10% compared to Gouraud shading
40
Homogeneity
Dynamic Solid Textures Standard 3D textures
41
42
Shot in Ogre3D
• Trade-off: Medium fidelity vs. Temporal continuity
• Linear blending new frequencies contrast loss
43
Statistical Analysis
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0
Grid Dots Hatching Paper Noise Paint Pigments Regular patterns
Irregular patterns
Cross- hatching
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0 1.5
Grid Dots Hatching Paper Noise Paint Pigments Near-regular
patterns Irregular patterns
Cross- hatching
S1
S2
P. Bénard, J. Thollot, F. Sillion, Quality Assessment of Fractalized NPR Textures. APGV 2009
44
Previous Work
Dynamic Solid Textures
NPR Gabor Noise
Perceptual Evaluation
Mapping Policies
Self Similar Line Artmap
Active Strokes
Motion coherence
Temporal continuity
Flatness
NPR Gabor Noise
45
P. Bénard, A. Lagae, P. Vangorp, S. Lefebvre, G. Drettakis, J. Thollot, A Dynamic Noise Primitive for Coherent Stylization. EGSR 2010
46
• Sparse convolution [Lewis 84,89]
• Spot Noise [van Wijk 91]
• Gabor Noise [LLDD09]
Offers significant spectral control
Support anisotropy
Our solution: NPR Gabor Noise
47
• Definition
Sum of randomly positioned and weighted kernels
Gabor kernel
noise
random positions and weights
48
anisotropic isotropic 𝐺 𝐾, 𝑎 × cos(𝐹0, 𝜔0)
• Flatness
Noise parameters in image space
Evaluation in image space
49
• Flatness
• Coherent motion
Point distribution on the surface of the 3D model
50
• Flatness
• Coherent motion
51
• Flatness
• Coherent motion
• Temporal continuity
Smooth LOD mechanism
52
• Sample 3D triangles (GPU)
2D Poisson distribution with constant screen space density
Far • small screen area • less points
Close • large screen area • more points
53
• Generate 2D point sprites
Point distribution Texture sprites
54
• Blending scheme using statistical properties
Reduce popping
Preserve noise appearance
Far Close
visibility = 0 visibility = 1
55
56
57
58
59
60
• Interactive scheme: remaining popping
“Procedural” approach
Slower, but should avoid popping
Useful for high quality offline rendering
• Gabor kernel
Temporally coherent spot noise
61
Previous Work
Dynamic Solid Textures
NPR Gabor Noise
Perceptual Evaluation
Mapping Policies
Self Similar Line Artmap
Active Strokes
62
• Evaluate success of various solutions according to:
Motion
coherence Temporal continuity
Flatness
P. Bénard, A. Lagae, P. Vangorp, S. Lefebvre, G. Drettakis, J. Thollot, A Dynamic Noise Primitive for Coherent Stylization. EGSR 2010
63
• Methodology
15 naïve subjects, ~ 20-30 minutes
• Ranking tasks “Rank the images/videos according to … ”
64
Extreme cases
Local image space Global image space Object space
Many-marks
Adv D2D DST
nprGN SD TM
65
Adv D2D DST
nprGN SD TM
66
• Image space methods more flat for simple scene
• Many 3D cues flatness hard to perceive in complex scene
Flatness
Coherent motion
Temporal continuity
Pleasantness
Shower Door
Texture Mapping
Object space
Local image space
Global image space
Many-marks /
67
• Object space methods more coherent
Flatness Coherent motion
Temporal continuity
Pleasantness
Shower Door
Texture Mapping
Object space
Local image space /
Global image space /
Many-marks / /
68
• High variance
• Advection and NPR Gabor Noise produce more changes: organic motion
Flatness Coherent motion
Temporal continuity
Pleasantness
Shower Door
Texture Mapping
Object space
Local image space /
Global image space / /
Many-marks / / /
69
• Object space approaches more pleasant
• Strong correlation with “motion coherence”
most important criteria to preserve
Flatness Coherent motion
Temporal continuity
Pleasantness
Shower Door
Texture Mapping
Object space
Local image space / /
Global image space / /
Many-marks / / / /
70
• First formal evaluation
• Intrinsic limitations
Hatching
3D scenes
Naïve users
71
• Two new solutions
Object space: Dynamic Solid Texture
Many-marks: NPR Gabor Noise
72
Motion coherence
Temporal continuity
Flatness
NPR Gabor Noise
Motion coherence
Temporal continuity
Flatness
Dynamic Solid Textures
• Simulation of brush strokes
Parameterization
0
l
Brush path
Line texture
74
Previous Work
Dynamic Solid Textures
NPR Gabor Noise
Evaluation
Mapping Policies
Self Similar Line Artmap
Active Strokes
75
• Stretching
• Tiling
76
77
• Stretching Temporally coherent
Stretch or compress the texture
• Tiling
Motion coherence
Temporal continuity
Flatness
Stretching
78
79
• Stretching Temporally coherent
Stretch or compress the texture
• Tiling Preserve the texture characteristics
Incoherent sliding
• Fading / Artmap
Motion coherence
Temporal continuity
Flatness
Tiling
80
81
• Stretching Temporally coherent
Stretch or compress the texture
• Tiling Preserve the texture characteristics
Incoherent sliding
• Fading / Artmaps No sliding
Blending artifacts Motion
coherence Temporal continuity
Flatness
Fading
82
Previous Work
Dynamic Solid Textures
NPR Gabor Noise
Evaluation
Mapping Policies
Self Similar Line Artmap
Active Strokes Motion coherence
Temporal continuity
Flatness
SLAM
83
P. Bénard, F. Cole, A. Golovinskiy, A. Finkelstein Self-Similar Texture for Coherent Line Stylization. NPAR 2010
84
Lmax / 2
Lmax
85
1) Density 2) Smooth variation
86
3) Self-Similarity
87
3) Self-Similarity
88
89
• Parametric texture synthesis [PS00]
• Property: (empirical observation, no formal proof)
small change in noise seed small change in output texture [PS00] Portilla and Simoncelli,
A Parametric Texture Model based on Joint Statistics of Complex Wavelet Coefficients, 2000
90
Exemplar
Synthesized Texture
White Noise Seed
[PS00]
=
Wsource Wf
Seed Pyramid
Wf
[WfWf]
0.5
1
α Wf + (1-α) [WfWf]
Lmax
Lmax
Lmax / 2
91
92
Synthesized textures Examples
93
94
95
96
• Input
Silhouettes
Suggestive contours
Apparent ridges
• Arc-length artifacts
Coherent parameterization
97
Previous Work
Dynamic Solid Textures
NPR Gabor Noise
Evaluation
Mapping Policies
Self Similar Line Artmap
Active Strokes
Work in progress in collaboration with C. Lu, F. Cole and A. Finkelstein 98
• Flatness
Remain linear in screen-space
• Motion coherence
Evolve according to the motion of the objet
• Temporal continuity
Adapt to the topological events
99
Coherent Stylized Silhouettes, Kalnins et al., SIGGRAPH 2003
Optimization between 2D and 3D
Multiple brush strokes per line
Dependent on the input connectivity
100
Kalnins et al., Coherent Stylized Silhouettes, SIGGRAPH 2003
101
Snakes
Snakes
3D scene
Lines extraction
Lines extraction
Feature samples
Line drawing
Brush Paths
Brush Paths
Parameterization Parameterization
Stylization Stylization Cleaning Vectorization
Cleaning Vectorization
Tracking Tracking
102
Feature samples
Line drawing
• Position • Local tangent • Velocity
Snakes
Snakes
3D scene
Lines extraction
Lines extraction
Feature samples
Line drawing
Brush Paths
Brush Paths
Parameterization Parameterization
Stylization Stylization Cleaning Vectorization
Cleaning Vectorization
Tracking Tracking
103
3D scene
Lines extraction
Lines extraction
Feature samples
Line drawing
Snakes
Snakes
Brush Paths
Brush Paths
Parameterization Parameterization
Stylization Stylization Cleaning Vectorization
Cleaning Vectorization
Tracking Tracking
104
• Polylines • In image space • Persistent
• Advection
• Relaxation
105 Frame f Frame f+1
• Advection
• Relaxation
106
107
108
3D scene
Lines extraction
Lines extraction
Feature samples
Line drawing
Snakes
Snakes
Brush Paths
Brush Paths
Parameterization Parameterization
Stylization Stylization Cleaning Vectorization
Cleaning Vectorization
Tracking Tracking
109
• Linear screen space parameterization
110
• Segments fitting
111
• Arcs fitting
112
113
114
115
116
Robustness toward noise / outliers
Temporal smoothing
117
Input samples Active strokes
• 4 new bricks for time coherent stylization
Dynamic Solid Textures
NPR Gabor Noise
Self-Similar Line Artmap
Active Strokes
• Formal evaluation
Formulation into 3 goals
2 perceptual studies
1 objective quality metric
119
• 2D Animation
Ongoing work with Antoine Boellinger (Master 1)
120
• 2D Animation
Ongoing work with Antoine Boellinger (Master 1)
• Evaluation & Objective metrics
Optical flow analysis
• Stylization by optimization
Versatile system to explore the space of trade-offs
121
122
123
124
125
• Simple
• Strong motion coherence and temporal continuity
• More suitable for unstructured patterns
• Real-time animation
126
• Input: pairs of 2D texture
• Definition: texture distortion = visual dissimilarity between original and transformed textures
• Goal: define a quantitative metric of this distortion
original transformed
P. Bénard, J. Thollot, F. Sillion, Quality Assessment of Fractalized NPR Textures. APGV 2009
127
• Procedure:
User study ranking of texture pairs according to their distortion
Statistical analysis scale of perceived quality
Correlation investigation objective metric
• Restrictions:
No texture mapping
Static images
128
• Ordinal scale
S1
S2
Statistical Analysis
129
• Thurstone’s law of comparative judgment [Tor58]
Statistical Analysis
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0
Grid Dots Hatching Paper Noise Paint Pigments Regular
patterns
Irregular
patterns
Cross-
hatching
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0 1.5
Grid Dots Hatching Paper Noise Paint Pigments Near-regular
patterns
Irregular
patterns
Cross-
hatching
S1
S2
130
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0 1.5
Grid Dots Hatching Paper Noise Paint Pigments Near-regular
patterns Irregular patterns
Cross- hatching
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0
Grid Dots Hatching Paper Noise Paint Pigments Regular patterns
Irregular patterns
Cross- hatching
S1
S2
131
Statistical Analysis
S1
S2
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0
Grid Dots Hatching Paper Noise Paint Pigments Regular patterns
Irregular patterns
Cross- hatching
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0 1.5
Grid Dots Hatching Paper Noise Paint Pigments Near-regular
patterns Irregular patterns
Cross- hatching
132
Overall contrast of patterns Feature shapes
Statistical Analysis
S1
S2
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0
Grid Dots Hatching Paper Noise Paint Pigments Regular patterns
Irregular patterns
Cross- hatching
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0 1.5
Grid Dots Hatching Paper Noise Paint Pigments Near-regular
patterns
Irregular
patterns
Cross-
hatching
Paper Noise Paint Pigments
133
Overall contrast of patterns
Feature shapes
Statistical Analysis
S1
S2
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0
Grid Dots Hatching Paper Noise Paint Pigments Regular
patterns
Irregular
patterns
Cross-
hatching
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0 1.5
Grid Dots Hatching Paper Noise Paint Pigments Near-regular
patterns
Irregular
patterns
Cross-
hatching
Irregular patterns Cross-hatching Dots Hatching
134
• Gray level co-occurrence matrix (GLCM) [HSD73]
Texture descriptor [TJ93]
Local image property
Match certain levels of human perception [JGSF76]
Linked to density and pattern coherence criteria
Parameters selection
• Average Co-occurrence Error [CRT01]
High correlation with the perceptual interval scale (Person’s correlation: 0.953 for S1 and 0.836 for S2 )
Correlation with Objective Metrics
135
ACE relevant estimator of the distortion
S1
S2
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0
Grid Dots Hatching Paper Noise Paint Pigments Regular patterns
Irregular patterns
Cross- hatching
-1.5
Z-Scores
-1.0 -0.5 0.0 0.5 1.0 1.5
Grid Dots Hatching Paper Noise Paint Pigments Near-regular
patterns Irregular patterns
Cross- hatching
AC
E
1.2
1.0
0.8
0.6
0.4
AC
E
1.4
1.2
1.0
0.8
0.6
r² = 0.6992
r² = 0.9075
136
• More suitable for low-contrast media
• Simple
• Strong motion coherence and temporal continuity
• Real-time animation
137
l=0.5
l=0
l=1
Lmax
138
Input: white noise seed
…
…
…
…
Parametric Texture Synthesis [PS00]
Seed Pyramid
Input: texture example
Output: SLAM
139
Self-Similar Texture for Coherent Line Stylization, Bénard et al. 2010
2D Infinite zoom: Self-Similar Line Artmap (SLAM)
Propagation using a 2D buffer aliasing ambiguities
One brush stroke per line
140
141
• Linear screen-space parameterization using Self-similar Line Artmaps
• Multiple brush strokes per feature line
• New propagation mechanism
142
• Feature samples – lines extracted in 2D or 3D
3D position
Local tangent
2D velocity
143
Snakes
Snakes
Tracking Advection Advection
3D scene
Lines extraction
Lines extraction
Feature samples
Relaxation Relaxation
Vectorization
Coverage Coverage Connectivity Connectivity
Line drawing
Brush Paths
Brush Paths
Geometry Geometry
Parameterization Parameterization
Stylization Stylization
144
• 2D polylines active contours (Kass et al. 1988)
• Goals
Coherence continuous evolution across frames
Accuracy faithfully represent shape
Coverage Level of Detail
Simplicity simple topology
Length stylization freedom
Trac
kin
g V
ect
ori
zati
on
145
• Directional coverage radius
• Local vectorization operators
Trim
Extend
Merge
Split
Coverage Coverage
Connectivity Connectivity
146
Coverage Coverage
Connectivity Connectivity
147
• Directional coverage radius
• Local vectorization operators
Trim
Extend
Merge
Split
• Applied sequentially in a greedy fashion
Coverage Coverage
Connectivity Connectivity
148
• Reprojection similar to Kalnins et al. 2003
• Computed for feature lines only connection between contour and feature sample needed
Advection Advection
149
• Minimize the energy:
• Re-sampling [DM00]
Relaxation Relaxation
External – Attraction by the features – Mass-spring forces prevents shrinking
Internal – Continuity – Smoothness
150
Snakes
Snakes
Tracking Advection Advection
3D scene
Lines extraction
Lines extraction
Feature samples
Relaxation Relaxation
Vectorization
Coverage Coverage Connectivity Connectivity
Line drawing
Brush Paths
Brush Paths
Geometry Geometry
Parameterization Parameterization
Stylization Stylization
151
• Consistently increasing subset of a snake
152
• Uniform in screen-space linear arc-length parameterization :
avec = slope = phase = arc-length
• Evolve according to the motion and topology of the contours parameterization at each vertex
Parameterization Parameterization
153
• Propagation at each vertex
Parameterization stored between two frames
Parameterization Parameterization
154
• Propagation at each vertex
• Linearization (in the least-square sense) p
aram
eter
izat
ion
arc-length
Parameterization Parameterization
155
• Propagation at each vertex
• Vectorization events
Split
Extend
Trim
Merge: mechanism to avoid parameterization discontinuities
• Linearization
propagated directly
Parameterization Parameterization
156
• Only if the slope and phase match
• Leveling mechanism pushes the 2 parameterizations to their mid-value:
with
par
amet
eriz
atio
n
arc-length
Parameterization Parameterization
157
• Feature lines dependency
Robustness toward noise / outliers
Temporal smoothing
158