WetBrush: GPU-based 3D Painting Simulation at the Bristle...

GTC 2016

WetBrush: GPU-based 3D Painting Simulation at the Bristle Level

Zhili Chen Byungmoon Kim

Adobe Research

GTC 2016 2

Oil painting with brush

Complex interactions

• Bristle-Bristle

• Bristle-Paint

• Paint-Paint

Motivation:

• Exhaustive simulation?

GTC 2016 3

Oil Painting Simulation

Brush Sim RendererUser Input

Brush head

movement

Two-way

Bristle-Paint Liquid

Interaction

Paint Liquid

GTC 2016 4

Brush Model

Previous works on brush model

2D Stamps 2D Surface wrapped

around skeleton

Individual bristles

Chu et al., 2002 Expresii DiVerdi et al., 2010 D-Brush in PhotoshopNatural brushes in Photoshop

• No variations from

brush dynamics

• No details from individual

bristles with large

deformation

• Brush shape not realistic without

inter-bristle interactions

GTC 2016 5

Brush Model

Our model

Bristle vertices

Bristle samples

50-100 bristles

~10 vertices per bristle

𝐩𝑖

𝐩𝑖+1

𝐩𝑖𝐩𝑖+1

𝐩𝑖−1

Bending

Stretching

GTC 2016 6

Brush Model

Inter-bristle interactions

• Collision

• Friction

GTC 2016 7

Paint Liquid Model

Height field Volumetric grid Particles

Chu, et al. 2010 Microsoft FreshPaint Overhang

Multi-layer color mixing

?• Track volume and mass better

• Less blurring from grid-sampling

• Model paint carried by brush

GTC 2016 8

Hybrid Fluid Representation

particles

Adaptive Hybrid Fluid Representation based on

• Distance to brush

• Velocity

GTC 2016 9

particles

Particles

• Velocity

Close to the brush

OR Fast Moving

Cover smaller region

GTC 2016 10

particles

Particles

Density

• Velocity

Close to the brush

OR Fast Moving

Cover smaller region

Further away from the brush

AND Slow moving

Cover larger region

GTC 2016 11

Fluid Conversions

Grid & Particles

Visualized

Only Particles

Visualized

FLIP• Particles sample from velocity grid

• Avoid expensive neighbor searching

Grid Fluid

Particle Fluid

• Sample new

particles

• Convert and delete

stationary particles

GTC 2016 12

Hybrid Fluid

Grid Only Grid+Particle

• Less volume loss

• Better surface tracking

• Sharper surface detail

• Sharper color mixing

GTC 2016 13

Brush-Paint Interaction

Brush Sim

Grid Fluid

Particle Fluid

• Brush emit new pigment particles

• Brush carries and pushes existing fluid particles

• Brush pick up color from paint on canvas

• Thick paint affect brush motion and shape

GTC 2016 14

Implementation

CUDA implementation

• Both brush and paint liquid in CUDA

• CUDA+OpenGL interop. for rendering

Rendering

• Ray casting on paint density grid and pigment grid

• Screen space particle rendering

GTC 2016 15

Implementation

k_ForwardPosition

SolveSelfCollision

k_ProjectConstraints

k_UpdateFinalSim

k_UpdateRendering

k_UpdateSorted

FindBrushBoundingBox

k_InterpolatePaintNodes

k_UpdatePaintNodesRendering

k_VoxelizeBrush

k_FilterBrushVoxelization

k_BrushDistanceMap

k_Advection_Particles

k_RasterizeParticleTrilinear

k_NormalizeRasterization

k_MixBrushVelocity

Diffusion

PostProcessVelocity

k_AdvectPaintDensityGrid

SpawnNewPaticles

k_BrushPickupPigment

k_BrushPaintNodesDistanceToPaintSurface

k_PaintNodeEmitNewParticlesBasedOnDistanceToPaint

k_FindConvertParticlesToPaintDensity

k_ExtendPigment

SaveActiveDomain

Forward

Inter-Bristle

Collision

Bristle

Stretching

and Bending

Update Brush

Attributes for

Rendering

Update Brush

Paint Nodes

that holds

pigment

Voxelize

Create Brush

Distance Field

Find Brush

Bounding Box

Forward FLIP

Particles

Rasterize

Particles

onto Velocity

Diffusion and

Dissapation

Mix with

Velocity Field

Grid Fluid

Simulation

Brush Pickup Pigment

Sample New Particles

from Density Grid

Brush Vertices emit

new Paint Particles

Resample FLIP Particle

Velocities

Convert inactive

particles to density

Save current

window back to full-

canvas texture

GTC 2016 16

Performance

GTX Titan X

Benchmark sequence: 46 fps, 92K Particles (~2M at maximum)

Grid-based liquid simulation(6.5ms)

Particle-based

liquid simulation (5.2ms)

Grid-particle transfer(5.2ms)

Brush simulation(1.1ms)

Rendering(2.0ms)

Bristle-particle transfer(1.6ms)

GTC 2016 17

Performance

Faster stroke requires higher framerate• Faster stroke => more stroke samples per unit time

• Maximum distance between stroke sample ~= 3-5 pixels

Stroke Sample

distance 2px

Stroke Sample

distance 30px

GTC 2016 18

Optimization

GTC 2016 19

Optimization

Adaptive simulation window (vs fixed simulation window)

Axis-aligned bounding

box for brush

Active simulation

window

thrust::minmax_element()

For grid fluid simulation46 fps-> 75 fps (63%)

GTC 2016 20

Optimization

Particle insertion

Num_Particles

Particle Array

AtomicInc(Num_Particles)

GTC 2016 21

Optimization

New Particle Array

• Pre-allocate maximum #

particle space for each cell• No more than 27 particles within one grid cell

• No more than one particle in one subcell

Marker Array

• 1 => a new particle is

added for the element

Cell 𝑖

Cell 𝑖 + 1

Optimized Particle insertion

Cell 𝑖 − 11

GTC 2016 22

Optimization

New Particle Array• Pre-allocate maximum #

particle space for each cell

Marker Array• 1 => a new particle is

added for the element

Cell 𝑖

Cell 𝑖 + 1

Optimized Particle insertion

Cell 𝑖 − 11

Compact New

Particle Array

thrust::remove_if()

Shrink List

Append

Particle Array

75 fps-> 92 fps (22%)

GTC 2016 23

Kernel Pipeline

k_ForwardPosition

SolveSelfCollision

k_UpdateFinalSim

k_UpdateRendering

k_UpdateSorted

k_VoxelizeBrush

k_BrushDistanceMap

k_MixBrushVelocity

Diffusion

PostProcessVelocity

SpawnNewPaticles

k_ExtendPigment

SaveActiveDomain

GTC 2016 24

Kernel Pipeline

k_ForwardPosition

SolveSelfCollision

k_UpdateFinalSim

k_UpdateRendering

k_UpdateSorted

k_VoxelizeBrush

k_BrushDistanceMap

k_MixBrushVelocity

Diffusion

PostProcessVelocity

SpawnNewPaticles

k_ExtendPigment

SaveActiveDomain

Low occupancy, GPU almost idle

GTC 2016 25

Dependencies

k_ForwardPosition

SolveSelfCollision

k_UpdateFinalSim

k_UpdateRendering

k_UpdateSortedFindBrushBoundingB

k_VoxelizeBrush k_FilterBrushVoxelization

k_MixBrushVelocity

Diffusion

PostProcessVelocity

k_ExtendPigment

SaveActiveDomain

k_BrushDistanceMap

SpawnNewPaticles

GTC 2016 26

Dependencies

k_ForwardPosition

SolveSelfCollision

k_UpdateFinalSim

k_UpdateRendering

k_BrushDistanceMap

k_MixBrushVelocity

Diffusion

PostProcessVelocity

k_ExtendPigment

SaveActiveDomain

• Brush # Vertices << Grid Dimension, Particle #

• Low occupancy

• First half of fluid simulation do not depend on brush

k_BrushDistanceMap

SpawnNewPaticles

GTC 2016 27

Dependencies

k_ForwardPosition

SolveSelfCollision

k_UpdateFinalSim

k_UpdateRendering

k_BrushDistanceMap

k_MixBrushVelocity

Diffusion

PostProcessVelocity

SpawnNewPaticles

k_ExtendPigment

SaveActiveDomain

• Only for rendering

• Can take place anytime

in a frame

GTC 2016 28

Dependencies

k_ForwardPosition

SolveSelfCollision

k_UpdateFinalSim

k_UpdateRendering

k_BrushDistanceMap

k_MixBrushVelocity

Diffusion

PostProcessVelocity

SpawnNewPaticles

k_ExtendPigment

SaveActiveDomain

• More parallelization

GTC 2016 29

Optimization

k_ForwardPosition

Hash SolveSelfCollisionk_ProjectConstraints

k_UpdateFinalSim

k_UpdateRendering

k_UpdateSorted

k_VoxelizeBrush k_FilterBrushVoxelization k_BrushDistanceMap

k_Advection_Particles k_RasterizeParticleTrilinear k_NormalizeRasterization k_MixBrushVelocity k_AdvectPaintDensityGrid

SpawnNewPaticles

k_PICk_FindConvertParticle

sToPaintDensityk_ExtendPigment SaveActiveDomain

0->1,3,4Sync

0->3Sync

cudaEventRecord()

cudaStreamWaitEvent()

PostProcessVelocityDiffusion

Using CUDA Streams

92 fps-> 117 fps (27%)

GTC 2016 30

Optimization

46 fps -> 117 fps (154%)

Shared memory, Loop unrolling, Adjusting block size

100~200fps on GTX Titan X

30~50 fps on Surface Book

GTC 2016 31

Results

GTC 2016 32

Results

Better 3D shape

Finer surface details

More pigment variations along strokes

Thick “Impasto” Style

A lot of overhang

Thinner

painting

Thank you!

WetBrush: GPU-based 3D Painting Simulation at the Bristle...

Documents

Transcript of WetBrush: GPU-based 3D Painting Simulation at the Bristle...

Eight Days of Art Style and Substance in Painting

RICETTA PASTA FRESCA IMPASTO PROFESSIONALE A … Pasta Fresca impasto... · Pasta Fresca Impasto ... 270 gr albume d'uovo varianti: X una sfoglia verde aggiungere 50gr di spinaci

Art 23 Texture / Impasto

Lime Painting - Branding Style Guide - 10.pdf

Impasto Painting Definition: Impasto is a way of applying paint, specifically a thick, textured application of paint where the marks made by the brush.

3 - Impasto Presentation - Branding

Lime Painting - Branding Style Guide - 7.pdf

Impasto Presentation - Creative

IMPaSTo: A Realistic, Interactive Model for Paint - …gamma.cs.unc.edu/IMPASTO/publications/Baxter-IMPaSTo_Web-NPAR… · IMPaSTo: A Realistic, Interactive Model for Paint William

The Classification of Style in Fine-Art Painting

Painting Style and the Master Study

Using Contemporary Painting Colored Lighting Style to ...

Y7: HT1 Theme: Natural Forms (insect and bugs) · Impressionism/ Post impressionism A style of art that shows different viewpoints at the same time Impasto A style of painting with

The Evolution of Painting Style under the Influence of ... … · The Evolution of Painting Style under the Influence of Regional Culture . Yuan Yuan . Northeast Normal University,

Lime Painting - Branding Style Guide - 2.pdf

Painting in layers painting with impasto-2015

Water Mixable Oil Painting Course Impasto Rooster ...courses.paintbasket.com/handouts/wamo/how-to-paint... · Water Mixable Oil Painting Course Impasto Rooster References Colours

Lime Painting - Branding Style Guide - 8.pdf

Impasto Presentation - Branding

Analysis on the Daofu Tibetan Color Painting Style for ...