Real-Time RenderingReal-Time Rendering

COMS 6998-3, Lecture 9

Real-Time Rendering DemoReal-Time Rendering Demo

Motivation: Interactive rendering with complex natural illumination and realistic, measured BRDFs

Complex Light TransportComplex Light Transport

ChallengesChallenges

• Illumination complexity

• Material (BRDF)/view complexity

• Transport complexity (shadows, interreflection)

Reflection MapsReflection Maps

Blinn and Newell, 1976

Environment MapsEnvironment Maps

Miller and Hoffman, 1984

Environment MapsEnvironment Maps

Interface, Chou and Williams (ca. 1985)

Environment MapsEnvironment Maps

Cubical Environment Map180 degree fisheyePhoto by R. Packo

Cylindrical Panoramas

Reflectance MapsReflectance Maps

• Reflectance Maps (Index by N)

• Horn, 1977

• Irradiance (N) and Phong (R) Reflection Maps

• Hoffman and Miller, 1984

Mirror Sphere Chrome SphereMatte Sphere

Irradiance Environment MapsIrradiance Environment Maps

Incident Radiance(Illumination Environment Map)

Irradiance Environment Map

R N

AssumptionsAssumptions

• Diffuse surfaces

• Distant illumination

• No shadowing, interreflection

Hence, Irradiance is a function of surface normal

Diffuse ReflectionDiffuse Reflection

B ERadiosity

(image intensity)Reflectance

(albedo/texture)Irradiance

(incoming light)

×=

quake light map

Analytic Irradiance FormulaAnalytic Irradiance Formula

Lambertian surface acts like low-pass filter

lm l lmE A LlA

2 / 3

/ 4

0

2 1

2

2

( 1) !2

( 2)( 1) 2 !

l

l l l

lA l even

l l

l0 1 2

9 Parameter Approximation9 Parameter Approximation

-1-2 0 1 2

0

1

2

( , )lmY

xy z

xy yz 23 1z zx 2 2x y

l

m

Exact imageOrder 01 term

RMS error = 25 %

9 Parameter Approximation9 Parameter Approximation

-1-2 0 1 2

0

1

2

( , )lmY

xy z

xy yz 23 1z zx 2 2x y

l

m

Exact imageOrder 14 terms

RMS Error = 8%

9 Parameter Approximation9 Parameter Approximation

-1-2 0 1 2

0

1

2

( , )lmY

xy z

xy yz 23 1z zx 2 2x y

l

m

Exact imageOrder 29 terms

RMS Error = 1%

For any illumination, average error < 3% [Basri Jacobs 01]

ComparisonComparison

Incident illumination

300x300

Irradiance mapTexture: 256x256

HemisphericalIntegration 2Hrs

Irradiance mapTexture: 256x256

Spherical HarmonicCoefficients 1sec

Time 300 300 256 256 Time 9 256 256

RenderingRendering

Irradiance approximated by quadratic polynomial

24 00 2 11 2 1 1 2 10 5 2

2 2

0

1 2 2 1 21 1 2 1 1 22

1 (3 1( ) 2 2 2

2 2 ( )2

)x y z z

x

E n c L c L c L c L c L

c L c L c Ly xz yz x yc L

( ) tE n n Mn

1

x

y

z

Surface Normal vectorcolumn 4-vector

4x4 matrix(depends linearly

on coefficients Llm)

Hardware ImplementationHardware Implementation

Simple procedural rendering method (no textures)• Requires only matrix-vector multiply and dot-product• In software or NVIDIA vertex programming hardware

( ) tE n n Mn

surface float1 irradmat (matrix4 M, float3 v) {

float4 n = {v , 1} ;

return dot(n , M*n) ;

}

Reflectance Space ShadingReflectance Space Shading

Cabral, Olano, Nemec1999

OLF ParameterizationOLF Parameterization

N LN

V

( , )B N V

OLF ParameterizationOLF Parameterization

N

V

( , )B N V

N

V

( , )B R V

RReparameterize

by reflection vector

OLF ParameterizationOLF Parameterization

• Captures structure of BRDF (and hence OLF) better

• Reflective BRDFs become low-dimensional

N

V

( , )B N V

N

V

( , )B R V

RReparameterize

by reflection vector

OLF Structure: LafortuneOLF Structure: Lafortune

( )VB R

Vie

wy

• Single 2D reflection map no longer sufficient

• But variation with viewing direction is slow

Viewx

OLF Structure: LafortuneOLF Structure: Lafortune

( )VB R

( )RB V

View maps vary slowly

Reflectionx

Ref

lect

ion

y

Vie

wy

Viewx

A Simple FactorizationA Simple Factorization

( )VB R

( )RB V

Viewx

Vie

wy

Ref

lect

ion

yReflectionx

( , ) ( ) * ( )B R V f R g V

*

Precomputed Radiance Transfer Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, for Real-Time Rendering in Dynamic,

Low-Frequency Lighting EnvironmentsLow-Frequency Lighting Environments

Peter-Pike Sloan, Microsoft Research

Jan Kautz, MPI Informatik

John Snyder, Microsoft Research

Previous Work – Where We FitPrevious Work – Where We Fit

TransportTransportComplexityComplexity

LightingLighting

simplesimple shadowsshadows

pointpointlightslights

full full env.env.mapmap

inter-inter-reflectionsreflections

singlesingleareaarealightlight

????[Moeller02][Moeller02]

[Blinn76][Blinn76]

[Heidrich00][Heidrich00]

[Crow77][Crow77]

[Ashikhmin02][Ashikhmin02]

our our techniquetechnique

Non-InteractiveNon-Interactive

[Matusik02][Matusik02]

MotivationMotivation• Better light integration and

transport • dynamic, area lights • self-shadowing • interreflections

• For diffuse and glossy surfaces

• At real-time rates

point lightpoint light area lightarea light

area lighting,area lighting,no shadowsno shadows

area lighting,area lighting,shadowsshadows

lightlight

2D example, piecewise constant basis, shadows only

2p2p

1p1p3p3p

Diffuse Self-TransferDiffuse Self-Transfer

PreprocessPreprocess

1p1p

2p2p

3p3p

Project LightProject Light

lightlight

RenderingRendering

•• ==1p1p

1p1p

2p2p

2p2p ==••

3p3p

3p3p ==••

PrecomputationPrecomputation

Basis 16Basis 16

Basis 17Basis 17

Basis 18Basis 18

illuminateilluminate resultresult

......

......

Diffuse Transfer ResultsDiffuse Transfer Results

No Shadows/Inter Shadows Shadows+InterNo Shadows/Inter Shadows Shadows+Inter

Arbitrary BRDF ResultsArbitrary BRDF Results

Other BRDFsOther BRDFs Spatially VaryingSpatially VaryingAnisotropic BRDFsAnisotropic BRDFs

Shadow Mappingwith Today’s OpenGL Hardware

CEDEC 2001Tokyo, Japan

Shadow Mappingwith Today’s OpenGL Hardware

CEDEC 2001Tokyo, Japan

Mark J. KilgardGraphics Software EngineerNVIDIA Corporation

Motivation Motivation

• Shadows increase scene realism• Real world has shadows• More control of the game’s feel

• dramatic effects• spooky effects

• Other art forms recognize the value of shadows• But yet most games lack realistic shadows

Common Real-time Shadow TechniquesCommon Real-time Shadow Techniques

ShadowShadowvolumesvolumes

Light mapsLight maps

ProjectedProjectedplanarplanarshadowsshadows

HybridHybridapproachesapproaches

ProblemsProblems

• Mostly tricks with lots of limitations• Projected planar shadows

• well works only on flat surfaces• Stenciled shadow volumes

• determining the shadow volume is hard work• Light maps

• totally unsuited for dynamic shadows• In general, hard to get everything shadowing

everything

Shadow MappingShadow Mapping

• Image-space shadow determination• Lance Williams published the basic idea in 1978

• By coincidence, same year Jim Blinn invented bump mapping (a great vintage year for graphics)

• Completely image-space algorithm• means no knowledge of scene’s geometry is required

• must deal with aliasing artifacts

• Well known software rendering technique• Pixar’s RenderMan uses the algorithm

• Basic shadowing technique for Toy Story, etc.

ReferencesReferences

• Important SIGGRAPH papers• Lance Williams, “Casting Curved Shadows on Curved

Surfaces,” SIGGRAPH 78• William Reeves, David Salesin, and Robert Cook

(Pixar), “Rendering antialiased shadows with depth maps,” SIGGRAPH 87

• Mark Segal, et. al. (SGI), “Fast Shadows and Lighting Effects Using Texture Mapping,” SIGGRAPH 92

Shadow Mapping Concept (1)Shadow Mapping Concept (1)

• Depth testing from the light’s point-of-view• Two pass algorithm• First, render depth buffer from the light’s point-of-view

• the result is a “depth map” or “shadow map”• essentially a 2D function indicating the depth of the

closest pixels to the light• This depth map is used in the second pass

Shadow Mapping Concept (2)Shadow Mapping Concept (2)

• Shadow determination with the depth map• Second, render scene from the eye’s point-of-view• For each rasterized fragment

• determine fragment’s XYZ position relative to the light• this light position should be setup to match the frustum

used to create the depth map• compare the depth value at light position XY in the depth

map to fragment’s light position Z

Shadow Mapping Concept (3)Shadow Mapping Concept (3)

• The Shadow Map Comparison• Two values

• A = Z value from depth map at fragment’s light XY position• B = Z value of fragment’s XYZ light position

• If B is greater than A, then there must be something closer to the light than the fragment

• then the fragment is shadowed• If A and B are approximately equal, the fragment is lit

Visualizing Shadow MappingVisualizing Shadow Mapping

• A fairly complex scene with shadows

the pointthe pointlight sourcelight source

Visualizing Shadow MappingVisualizing Shadow Mapping

• Compare with and without shadows

with shadowswith shadows without shadowswithout shadows

Visualizing Shadow Mapping Visualizing Shadow Mapping

• The scene from the light’s point-of-view

FYI: from theFYI: from theeye’s point-of-vieweye’s point-of-viewagainagain

Visualizing Shadow Mapping Visualizing Shadow Mapping

• The depth buffer from the light’s point-of-view

FYI: from theFYI: from thelight’s point-of-viewlight’s point-of-viewagainagain

Visualizing Shadow Mapping Visualizing Shadow Mapping

• Projecting the depth map onto the eye’s view

FYI: depth map forFYI: depth map forlight’s point-of-viewlight’s point-of-viewagainagain

Visualizing Shadow Mapping Visualizing Shadow Mapping

• Comparing light distance to light depth map

Green is Green is where the where the

light planar light planar distance and distance and

the light the light depth map depth map

are are approximatelapproximatel

y equaly equal

Non-green is Non-green is where where shadows shadows should beshould be

Visualizing Shadow Mapping Visualizing Shadow Mapping

• Scene with shadows

Notice how Notice how specular specular

highlights highlights never appear never appear

in shadowsin shadows

Notice how Notice how curved curved surfaces cast surfaces cast shadows on shadows on each othereach other

Hardware Shadow Map FilteringHardware Shadow Map Filtering

• “Percentage Closer” filtering• Normal texture filtering just averages color components• Averaging depth values does NOT work• Solution [Reeves, SIGGARPH 87]

• Hardware performs comparison for each sample• Then, averages results of comparisons

• Provides anti-aliasing at shadow map edges• Not soft shadows in the umbra/penumbra sense

Hardware Shadow Map FilteringHardware Shadow Map Filtering

GL_NEAREST: blockyGL_NEAREST: blocky GL_LINEAR: antialiased edgesGL_LINEAR: antialiased edges

Low shadow map resolutionLow shadow map resolutionused to heightens filtering artifactsused to heightens filtering artifacts

Soft Shadows?Soft Shadows?

ConclusionConclusion

• Variety of techniques• Shadows• Complex illumination• Complex materials

• Limited manipulability in most cases• Cannot change lighting, geometry, viewpoint etc.

• Is it possible to have everything simultaneously?