Efficient Streaming of 3D Scenes with Complex Geometry

and Complex Lighting

Romain Pacanowskiand

M. Raynaud X. Granier P. Reuter C. Schlick P. Poulin

INRIA Bordeaux University

Global illumination (indirect lighting)

• Increases realism of synthetic images

• Very long to compute unless using interactive/real-time techniques

MotivationGlobal illumination for remote visualization systems

All lighting computations done on client• Low data transfer requirements

• Rendering speed depends on scene geometric complexity

MotivationClient approach

• Pre/compute indirect illumination• Stream the indirect illumination to the client

BUT: How to avoid an overhead transfer time proportional to the size of the geometry ?

Need for an illumination representation not correlated to the geometry

MotivationServer approach

• Stochastic methods [Purcell03,Gautron05,…]• Fast but not real time • Depend on geometry

• Radiosity methods [Keller97, Segovia07]• [Dachsbacher07]: scene depth dependent • [Laine07]:

• Real time • Visual quality depends on geometric accuracy

Not suited for streaming context

Previous WorkInteractive/Real time global illumination

Concept: encode light transport effects in a structure

• [Sloan02,Wang04,Pan07]

• Real time even with dynamic scenes • Huge data size

• Direct-to-indirect transfer [Pellacini07]

• Data size is dependent on geometry complexity

Previous WorkPrecomputed radiance transfer approaches

• Most closely related to our work

• 3D regular grid [Mitchell06]

• Irradiance values at vertices• Geometric dependency of irradiance

Storage cost increases

Previous WorkIrradiance Volumes [Greger97]

• New structure for indirect illumination• Geometry independent • GPU friendly

• Streaming technique for our lighting structure• Client/Server visualization system

• Independent streaming of geometry and lighting

• Direct illumination on the client side

Our MethodOverview

Indirect Lighting RepresentationOverview

Regular 3D grid• 6 irradiance vectors at each vertex• Directional interpolation

To reconstruct irradiance for any normal

• Spatial interpolation • Easily compressed• GPU friendly

Indirect Lighting RepresentationIrradiance vector

IrradianceMaterialsReflected Radiance

Indirect Lighting RepresentationIrradiance vector directional interpolation

• Colored irradiance vector for direction : 3x3 matrix

• Compression: • Direction + Color

• If : no artefacts are introduced

Indirect Lighting RepresentationGPU : Irradiance vector compression

• Color 32 bits • R9_G9_B9_E5 GPU compatible format• RGBE [Ward91]

• Direction• XYZ: 24 bits (3x8 bits)• (θ,ϕ): 2x8 bits ([Jensen96])

Quantization used to reduce the transfer size

Indirect Lighting RepresentationGPU : Irradiance vector quantization

• Regular grid

12x3D Textures • 6 for direction • 6 for color • Format GL_RGB16F_ARB

• 6 texture fetches per pixel• Native trilinear interpolation

Indirect Lighting RepresentationGPU issues

Server• Precomputes and stores

• Illumination grids

• LOD for 3D objects

• Stores :

• Materials

• Planar Surfaces

Our Remote Visualization SystemOverview

Client

CPU processes:

• Geometry

• Lighting (Push-Pull)

Direct Transfer

Streaming

• Geometry, and then Lighting

• Lighting, and then Geometry

• Interleave Geometry and Lighting

Our Remote Visualization SystemStreaming strategies

• Initialization: 8 corners• Each client request:

N samples per slice• Not yet received data

vertices

Holes in data =

black spots

Our Remote Visualization SystemIrradiance vector grid streaming

Our Remote Visualization SystemPush-Pull : filling holes in the grid

2. 3D Hierarchical hole filling (PUSH)

1. 3D Hierarchy construction (PULL)

3. For each completed level

=> Pyramidal Filter

Our Remote Visualization SystemPush-Pull : Results

Without Push-Pull With Push-Pull and Filtering

Adaptation of [Melax98,Gueziec99] techniques• Vertex split to get a multiresolution mesh • Streaming :

• Vertices• Vertex Indices• Vertex lookup tables

• Mesh is globally updated

Our Remote Visualization SystemGeometry streaming

Our remote system:

• Server: Intel Q6600 with 4GB RAM• Client: Nvidia 8800GTX• Network: Wifi 802.11g

ResultsIndependence of geometry and lighting

ResultsStreaming geometry with constant illumination

ResultsStreaming illumination with constant geometry

ResultsInterleave streaming of geometry and illumination

ResultsTransfer time for indirect illumination

ResultsTransfer time for indirect illumination

ResultsTransfer time for geometry

New structure to represent indirect lighting:• 3D regular grid with irradiance vectors at

vertices• GPU friendly• Small memory footprint and short transfer

time overhead• Independent of geometric complexity• Easily integrated with geometry streaming

ConclusionSummary

• Server side

• Precomputation to fit cluster architectures• On-line precomputation

• Fast update mechanism for dynamic 3D scenes• Local recomputation in regions of important changes

• Client side: reducing the process timeNew push-pull process (GPU)

Future Work

Questions ?Thank you for your attention